Thursday, 2018-08-16

*** dklyle has quit IRC00:03
*** brinzhang has joined #openstack-nova00:17
*** _ix has joined #openstack-nova00:17
*** zzzeek has quit IRC00:31
*** zzzeek has joined #openstack-nova00:31
*** mriedem has joined #openstack-nova00:44
mriedem你好00:44
mriedemsmcginnis: if you're around, we need stable cores for https://review.openstack.org/#/c/591872/ RC200:47
*** alex_xu has joined #openstack-nova00:47
mriedemanyone know if jichen is going to clean this up? https://review.openstack.org/#/c/533125/00:48
mriedemefried: do we need to just update it ourselves?00:48
openstackgerritMerged openstack/nova master: Fix service list for disabled compute using MC driver  https://review.openstack.org/58393600:51
openstackgerritMerged openstack/nova master: Revisons on notifications doc  https://review.openstack.org/59162100:51
openstackgerritMerged openstack/nova master: use static pages for mitaka and newton release notes  https://review.openstack.org/59215000:51
mriedemtonyb: i guess you'd be a more appropriately time-zoned stable core for https://review.openstack.org/#/c/591872/ for RC200:52
mriedemKevin_Zheng: want to start backporting https://review.openstack.org/#/c/583936/ ? it needs to go to rocky, queens and pike00:54
mriedemwe'll hold it out until we've released rocky though00:54
tonybmriedem: on it00:55
*** dave-mccowan has quit IRC01:02
*** imacdonn has quit IRC01:04
Kevin_Zhengsure01:04
*** dave-mccowan has joined #openstack-nova01:04
tonybmriedem: I'll ping lyarwood01:05
mriedemtonyb: thanks but he's probably done for the day01:06
tonybmriedem: I can single core approve it01:07
tonybmriedem: given you're a stable core etc01:08
dansmithmriedem: tonyb I got it01:09
tonybdansmith to the rescue!01:10
*** r-daneel has joined #openstack-nova01:13
*** zzzeek has quit IRC01:14
*** zzzeek has joined #openstack-nova01:16
*** imacdonn has joined #openstack-nova01:16
*** mhen has quit IRC01:17
mriedemcool; i've emailed jichen about the zvm docs since those are the only things remaining on the RC2 todo list01:24
*** mhen has joined #openstack-nova01:29
*** r-daneel has quit IRC01:29
*** mrsoul has quit IRC01:31
*** sapd1 has joined #openstack-nova01:39
*** donghm has joined #openstack-nova01:39
*** hongbin_ has joined #openstack-nova01:41
mriedemefried: edmondsw: frankly i care very little if the powervm out of tree driver supports a feature, i don't plan on adding flags to nova to support out of tree features, re: https://review.openstack.org/#/c/591898/01:41
*** Dinesh_Bhor has joined #openstack-nova01:44
*** donghm has left #openstack-nova01:44
*** _ix has quit IRC01:53
openstackgerritfupingxie proposed openstack/nova master: Support list for alias in pci section in nova.conf  https://review.openstack.org/59224302:10
*** tinwood has quit IRC02:10
*** tinwood has joined #openstack-nova02:11
*** lei-zh has joined #openstack-nova02:39
*** markmc has quit IRC02:42
*** jpena|off has quit IRC02:45
openstackgerritTao Li proposed openstack/nova master: Rollback instance vm_state to original where instance claims failed  https://review.openstack.org/59225202:48
*** psachin has joined #openstack-nova02:48
*** rosey99_ has quit IRC03:06
*** hongbin_ has quit IRC03:29
pvc@edmondsw im usign ceph rbd03:32
*** rosey99 has joined #openstack-nova03:33
*** Dinesh_Bhor has quit IRC03:34
*** udesale has joined #openstack-nova03:36
*** pvc has quit IRC03:39
*** Bhujay has quit IRC03:43
*** lei-zh has quit IRC03:48
*** pvc has joined #openstack-nova03:52
pvchi03:52
pvcanyone using ceph03:52
*** gbarros has quit IRC03:52
*** erlon has quit IRC03:53
*** takashin has left #openstack-nova04:01
*** rosey99 has left #openstack-nova04:02
*** dave-mccowan has quit IRC04:04
*** abhishekk has joined #openstack-nova04:05
*** tbachman has quit IRC04:10
*** tbachman has joined #openstack-nova04:29
*** gcb_ has quit IRC04:31
*** janki has joined #openstack-nova04:34
*** Bhujay has joined #openstack-nova04:38
*** tbachman has quit IRC04:46
*** tetsuro has joined #openstack-nova04:47
*** gcb_ has joined #openstack-nova04:47
*** tbachman has joined #openstack-nova04:51
*** Bhujay has quit IRC05:07
*** Bhujay has joined #openstack-nova05:08
openstackgerritMerged openstack/nova stable/rocky: placement: ignore policy scope check failures if not enforcing scope  https://review.openstack.org/59187205:16
*** Dinesh_Bhor has joined #openstack-nova05:16
*** tbachman has quit IRC05:20
openstackgerritMatt Riedemann proposed openstack/nova master: Explicitly fail if trying to attach SR-IOV port  https://review.openstack.org/59189805:20
*** pcaruana has quit IRC05:26
*** ratailor has joined #openstack-nova05:27
*** lei-zh has joined #openstack-nova05:27
*** tetsuro has quit IRC05:39
openstackgerritMerged openstack/nova master: Imported Translations from Zanata  https://review.openstack.org/59112905:39
*** tetsuro has joined #openstack-nova05:42
*** pvc has quit IRC05:44
*** brinzh has joined #openstack-nova05:48
*** brinzhang has quit IRC05:52
*** dpawlik has joined #openstack-nova06:01
*** Bhujay has quit IRC06:02
*** Bhujay has joined #openstack-nova06:03
openstackgerritBrin Zhang proposed openstack/nova-specs master: Resource retrieving: add change-before filter  https://review.openstack.org/59197606:07
*** skatsaounis has joined #openstack-nova06:12
*** Dinesh_Bhor has quit IRC06:23
*** Dinesh_Bhor has joined #openstack-nova06:24
openstackgerritBrin Zhang proposed openstack/nova-specs master: Resource retrieving: add change-before filter  https://review.openstack.org/59197606:37
openstackgerritMatt Riedemann proposed openstack/nova master: Force refresh instance info_cache during heal  https://review.openstack.org/59160706:40
*** ratailor has quit IRC06:43
*** adrianc has joined #openstack-nova06:43
*** hoonetorg has quit IRC06:45
*** ratailor has joined #openstack-nova06:47
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Make monkey patch work in uWSGI mode  https://review.openstack.org/59228506:48
*** pcaruana has joined #openstack-nova06:48
openstackgerritTao Li proposed openstack/nova master: Rollback instance vm_state to original where instance claims failed  https://review.openstack.org/59225206:51
*** slaweq_ has joined #openstack-nova06:51
*** hoonetorg has joined #openstack-nova06:57
*** ispp has joined #openstack-nova06:59
*** slaweq_ has quit IRC07:00
openstackgerrithuanhongda proposed openstack/nova-specs master: Return "deleted" and "estimated_reclaim" time of instance for admin  https://review.openstack.org/58163807:01
*** adrianc has quit IRC07:02
openstackgerritBrin Zhang proposed openstack/nova-specs master: Support deleting data volume when destroy instance  https://review.openstack.org/58033607:03
openstackgerritBrin Zhang proposed openstack/nova-specs master: Add support specify volume type when boot instance  https://review.openstack.org/57952007:04
*** slaweq_ has joined #openstack-nova07:04
*** rha has joined #openstack-nova07:04
*** tetsuro has quit IRC07:14
*** tetsuro has joined #openstack-nova07:16
*** adrianc has joined #openstack-nova07:17
*** tetsuro has quit IRC07:17
*** tetsuro has joined #openstack-nova07:17
*** adrianc_ has joined #openstack-nova07:18
*** rcernin has quit IRC07:20
*** ispp has quit IRC07:22
*** adrianc has quit IRC07:23
*** lei-zh has quit IRC07:24
*** lei-zh has joined #openstack-nova07:24
*** tikitavi has joined #openstack-nova07:26
*** jpena has joined #openstack-nova07:31
*** tetsuro has quit IRC07:31
*** tetsuro has joined #openstack-nova07:33
*** tetsuro has quit IRC07:37
*** tetsuro has joined #openstack-nova07:38
*** adrianc__ has joined #openstack-nova07:40
*** adrianc_ has quit IRC07:43
*** adrianc__ has quit IRC07:43
*** sahid has joined #openstack-nova07:44
*** adrianc has joined #openstack-nova07:44
*** sayalilunkad has joined #openstack-nova07:51
openstackgerritZhenyu Zheng proposed openstack/nova stable/rocky: Fix service list for disabled compute using MC driver  https://review.openstack.org/59231008:01
openstackgerritZhenyu Zheng proposed openstack/nova stable/queens: Fix service list for disabled compute using MC driver  https://review.openstack.org/59231108:01
*** tssurya has joined #openstack-nova08:04
mriedemtssurya: o08:05
mriedemo/08:05
mriedemi've added a couple of things to the cells meeting agenda if you want to take a peek and start lending brain power08:05
tssuryamriedem: \o/08:05
mriedemcross cell migration,08:05
mriedemand down cell08:05
mriedemour operators want a config option to not return any VMs from nova list if the tenant has any in a down cell08:06
tssuryamriedem: oh okay08:06
mriedemto avoid confusing users that thought they had like 3 VMs but now only list 208:06
mriedem^ would be pre-microversion08:06
mriedemb/c with the microversion we can list the down cell VMs with the UNKNOWN status08:06
*** adrianc has quit IRC08:06
tssuryaright so we need to change this now nad backport also I guess to Queens ?08:07
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Make monkey patch work in uWSGI mode  https://review.openstack.org/59228508:07
tssuryabefore we start merging the UNKNOWN stuff.08:07
mriedemtssurya: not sure about backports, but maybe if it defaults to current behavior08:07
tssuryamriedem: the previous behaviour was 500 for down cells, do you suggest the config option to return an empty list if the user has stuff in the down cell then ?08:08
tssuryainstead of skipping08:08
tssuryathat cell08:09
mriedemoriginally the behavior was 500 right?08:09
tssuryayes08:09
mriedemyou fixed that to skip down cells08:09
tssuryayes08:09
mriedemso yes, they want an option to say don't skip, but just return an empty list08:09
mriedemwell,08:09
mriedemif the project has any instances in a down cell08:09
tssuryaokay,08:09
mriedemi think it would only apply to non-admin instance list; an admin doing nova-list --all-tenants wouldn't apply here08:10
openstackgerritfupingxie proposed openstack/nova master: Support list for alias in pci section in nova.conf  https://review.openstack.org/59224308:10
tssuryaack,08:10
*** sean-k-mooney[m] has quit IRC08:11
tssuryamriedem: in the agenda you say "show is OK", well show wouldn't work at all before the new microversion, would just be a 50008:12
tssuryathat's okay right ?08:13
mriedemoh yeah, right08:13
mriedemyeah08:13
mriedemnova CLI will do the right thing once you're using latest novaclient and server,08:13
mriedemin that it will request the latest microversion which will get users the UNKNOWN status behavior08:13
tssuryayes, speaking of which I am working on that right now: does the new microversion which skips certain keys need the api_samples functional tests ? or is it enough I clearly specify this in the api-ref docs ?08:15
mriedemwould you have a sample of what this looks like in the api-ref docs?08:17
mriedemi think we probably should08:17
mriedemand those are generated from the api-samples functional tests08:17
tssuryamriedem: yea, okay it woud surely be nice to have sample jsons of what a response from down cell versus up cell looks like08:18
mriedemyeah08:18
tssuryaokay then will add those tests too, but before that will do this pre-microversion thing config08:18
tssuryaand ping you bacl08:18
tssuryaback*08:18
mriedemok cool08:20
mriedemnot sure if this should be a bug or what08:20
mriedemthat's why i added it to the meeting agenda08:20
tssuryaright, we kind of skipped yesterday's meeting. so maybe we can discuss it with dansmith when he wkaes up ?08:21
*** tikitavi has quit IRC08:21
*** alexchadin has joined #openstack-nova08:22
*** tetsuro has quit IRC08:24
openstackgerritMatt Riedemann proposed openstack/nova master: Remove ChanceScheduler  https://review.openstack.org/57638408:25
*** davidsha has joined #openstack-nova08:25
*** Dinesh_Bhor has quit IRC08:26
*** Dinesh_Bhor has joined #openstack-nova08:28
sean-k-mooneymriedem: o/08:28
sean-k-mooneymriedem: i tested the live migration between ovs with iptables and ovs with connectracker hosts and you multi port binding changes work well for that case.08:29
mriedemho08:30
mriedem*hot08:30
sean-k-mooneymriedem: i need to still test kernel ovs to ovs-dpdk an ovs to ovn but atleast the usecase called out in the spec is good08:30
mriedemKevin_Zheng: yikun: your cells perf email mentions needing to monkey_patch eventlet in wsgi mode, but didn't mention you needed to monkey patch the time module, right?08:30
*** rpittau has joined #openstack-nova08:30
Kevin_ZhengAh, yes, I forget that part08:31
mriedemhttps://github.com/openstack/oslo.db/blob/9c6695969f6502f52bc86f22af0758715fdc57c0/oslo_db/sqlalchemy/engines.py#L5108:31
mriedemyeah08:31
Kevin_Zhengyikun could you amend that mail to show why the time module matters?08:31
mriedemi think you also only mention that all 10 cells are using a single DB at the end08:32
yikunyeah, sure08:32
mriedemprobably good to mention that early and why08:32
*** adrianc has joined #openstack-nova08:32
openstackgerritMatt Riedemann proposed openstack/nova master: Remove ChanceScheduler  https://review.openstack.org/57638408:34
Kevin_ZhengOK, I will be more careful in the future08:34
sean-k-mooneymriedem: what is the importance of time module in this case beyond causeing eventlet to yeild execution of the current green tread?08:35
mriedemsean-k-mooney: we iterate the cells with greenthreads,08:36
sapd1sean-k-mooney: melwitt hi. I have a solution for octavia to use SR-IOV port. this is a workaround with custom in octavia.08:36
mriedembut without monkey_patching the time module, we aren't actually iterating the cells in parallel08:36
mriedemsapd1: how so?08:37
sean-k-mooneymriedem: ah ok. well even with monkey patching its technicall concurent not parralel but i get the point08:37
mriedemsapd1: see https://review.openstack.org/#/c/591898/08:37
sean-k-mooneysapd1: do you have a review available or a paste of the patch we can review?08:38
sapd1mriedem: I launch amphora instance using SR-IOV network instead of plug SR-IOV port to amphora instance. So I can launch amphora sucess.08:38
sapd1s/sucess/success/08:38
mriedemi don't know what an amphora instance is08:38
sean-k-mooneymriedem: its a specifc load balancer impmentation08:38
sapd1mriedem: amphora instance is a instance running haproxy inside08:39
sapd1s/a/an08:39
*** vivsoni has joined #openstack-nova08:40
sean-k-mooneymriedem: there were a few specs in the past to actully make attach work for sriov. its non trival but doable in most cases08:40
sean-k-mooneymriedem: we are missing the resouce tracker device claim and in future the placemnet allocation update to make it work correctly08:41
sean-k-mooneymriedem: as part of the RT claim we also need to update the neutron port binding with pci adress info but thats a detail.08:41
sean-k-mooneymriedem: sapd1 would like to resubmit the spec for stien so octavia can support loadblancers with sriov interfaces08:42
tssuryaKeving_Zheng, yikun: I like this part of the cells perf email "so, now the queries are in parallel, but the whole thing still seems serial.", we have been having the same experience :) and thanks for the efforts08:42
*** derekh has joined #openstack-nova08:43
Kevin_Zhengtssurya, interesting, since you guys got 70 cells, how does it work?08:43
tssuryayea which is why we pushed for the patch of using inst_mappings to narrow it down to 2 or 3 cells per user08:44
tssuryaso that way it works fine for us on the user end08:44
tssuryamainly because of the way we have our cells designed08:45
*** mdbooth has joined #openstack-nova08:45
Kevin_ZhengHmm, that might not work for us, as I mentioned before, we are unable to limit our users to cells08:45
tssuryaKevin_Zheng: yea I remember your comment on that patch, but I guess we still need to find a way to make the data gathering part faster then08:46
mriedemsean-k-mooney: yeah i'm aware of the spec08:47
mriedemsean-k-mooney: i also added you to an sriov-bond spec that our product team is interested in08:48
mriedemtssurya: Kevin_Zheng: that reminds me,08:49
mriedemtssurya: the reason we can't restrict users to cells in our public cloud is because flavors are defined by cell (among other things)08:49
mriedemso if a user wants to use different flavors they can get instances in different cells08:49
mriedemtssurya: how is that handled at CERN?08:49
sapd1sean-k-mooney: I think nova should care about SR-IOV instead of octavia.08:50
tssuryamriedem: we actually map projects to cells08:50
tssuryathe default flavors are available everywhere and then the special flavors are available in certain cells which are then restricted for the tenants as well08:51
sean-k-mooneysapd1: well i think this is a nova issue rather then octavia. i cant unfortunetly make nova care about sriov but enough of our users do that we try to support it :)08:51
tssuryaso only certain tenats end up in certain cells08:51
sean-k-mooneymriedem: oh i had not seen it but i will take a look08:51
*** rha has quit IRC08:52
sean-k-mooneysapd1: i started looking into this again last night. i think i know a slightly hacky way to fix this in the libvirt virt diriver. i am look to see if we can adress the missing logic in the compute manager instead as most of the code should not depend on the driver08:53
mriedemtssurya: yeah i know cern mapped projects to cells,08:54
mriedemoh ok so basic flavors are global,08:54
mriedemthen per-cell flavors have tenant access for the tenants in those cells08:54
mriedemdo you have any idea how many projects exist in CERN?08:54
mdboothI'm trying to enable debug logs in a functional test. Thought it would be as simple as self.flags(debug=True), but apparently not.08:54
sean-k-mooneytssurya: are you using dansmith's prefilter to do the tenat to cell mapping or something else?08:55
*** panda|off is now known as panda08:56
mriedemmdbooth: OS_DEBUG=True08:56
lyarwoodOS_DEBUG=1 tox -e functional [..]08:56
lyarwoodyeah sorry08:56
mdboothmriedem lyarwood: Thanks! You saved me some rtfs :)08:56
mriedemsean-k-mooney: wouldn't most of the work for sriov attach be doing the RT claim for a pci_request and if that fails, we fail the attach request08:57
mriedemand then set the port host binding details as you mentioned08:58
mriedemmost of ^ is probably already done in the port binding code08:58
openstackgerritjichenjc proposed openstack/nova master: Add zvm admin intro and hypervisor information  https://review.openstack.org/53312508:59
openstackgerritjichenjc proposed openstack/nova master: Add zvm CI information  https://review.openstack.org/53351208:59
*** alexchadin has quit IRC09:00
sean-k-mooneymriedem: ya. which is why i think we can do all the work from the compute manager rather then the virt driver09:01
sapd1sean-k-mooney: Will you have plan to re-implement this feature? :D09:01
*** josecastroleon has quit IRC09:01
*** josecastroleon has joined #openstack-nova09:01
sean-k-mooneymriedem: sapd1 these are my notes form last night https://etherpad.openstack.org/p/sriov-attach09:03
openstackgerritMerged openstack/nova master: VMware: add missing os types in vSphere sdk 6.5  https://review.openstack.org/59177109:04
sean-k-mooneysapd1: https://bugs.launchpad.net/nova/+bug/1708433 was opened by my old team as we had a customer request to enable sriov attach generically09:05
openstackLaunchpad bug 1708433 in OpenStack Compute (nova) "Attaching sriov nic VM fail with keyError pci_slot" [Medium,In progress] - Assigned to Matt Riedemann (mriedem)09:05
sean-k-mooneysapd1: that was before my team was pulled off upstream work for internal projects. i have now move to redhat i might be able to work on this but not sure09:05
sean-k-mooneyi would have to ask09:05
*** cdent has joined #openstack-nova09:08
sean-k-mooneymriedem: there are a few more edge cases that we would have to handel. e.g. vm runs out of pci ports, updating instace request spec, numa affintiy and updating placement allocation would be the main ones. a spec is the best way to capture all the edgecases09:10
*** jaosorior has quit IRC09:10
*** alexchadin has joined #openstack-nova09:11
openstackgerritGhanshyam Mann proposed openstack/nova master: Remove the deprecated API extensions policies  https://review.openstack.org/58687209:11
sapd1sean-k-mooney: good point. Because I am not a developer, So I can't help you. Because nova souce code is too complicated09:12
openstackgerritGhanshyam Mann proposed openstack/nova master: Remove the deprecated API extensions policies  https://review.openstack.org/58687209:13
mriedemsean-k-mooney: that's why there was a spec :)09:13
mriedembut it's outdated by now probably09:13
mriedemhttps://review.openstack.org/#/c/139910/09:13
sean-k-mooneyyes i remember. i think we referenced that spec in the bug or it was pointed out at least09:14
sean-k-mooneysapd1: well if this would have been your first topic in nova, it is proably in one of the most complicated parts.09:16
cdentmriedem: I left a comment on https://review.openstack.org/#/c/592285/ trying to get more info about what is actually being fixed, because it's not clear what the actual problem is. Do you have more info?09:16
sean-k-mooneysapd1: mriedem am i can write a spec for this and submit it if you like. but i will have to check internally if i can do more then that and impelmnte it assumeing it was approved.09:17
mriedemcdent: yeah yikun was going to update the ML thread on cells perf testing,09:17
mriedembut he needed to monkey_patch the time module09:17
sean-k-mooneythere are some other topics related to this i was planning on writeing proposals for so its somwhat alinged with what i was going to suggest working on anyway.09:18
sapd1sean-k-mooney: I will vote for you. :D09:18
openstackgerritjichenjc proposed openstack/nova master: Add zvm admin intro and hypervisor information  https://review.openstack.org/53312509:19
openstackgerritjichenjc proposed openstack/nova master: Add zvm CI information  https://review.openstack.org/53351209:19
mriedemsean-k-mooney: i'm indifferent to sriov port attach capabilities in nova09:20
cdentmriedem: would be interesting to see/know what ways nova api (as a wsgi app) needs eventlet at all09:20
cdentor it just happens to be picking it up accidentally and getting into trouble09:20
*** Dinesh_Bhor has quit IRC09:21
sean-k-mooneymriedem: for me originally it was parity thing. e.g. it was yet one more thing you had to be aware of did not work with sriov.09:22
sean-k-mooneymriedem: it became more important with hardware offloaded ovs09:22
mriedemcdent: i left a comment09:22
cdentthanks09:22
mriedemsean-k-mooney: sure i get that09:22
mriedemmuch like volume-backed instances with 33% of the compute API09:22
mriedemor 80% of the compute API with non-libvirt drivers :)09:23
sean-k-mooneymriedem: :) the main thing i was thinking of working on in stien was  a generic device management framework for nova. to unify handeling of all passthough devices via a form of flavor decompostion. that is a lot of work though and im not sure its a high enough proirty to try and do in stien.09:27
*** dpawlik has quit IRC09:28
sean-k-mooneybasicaly the openstack server create --device stuff i mentioned in vancouver.09:28
*** dpawlik has joined #openstack-nova09:28
mdboothAh, ha! My test is non-deterministic because the fake image service returns images as a list from a dict, hence in non-deterministic order. If the first one returned defined an external kernel then the libvirt driver uses the Flat imagebackend explicitly, which I'm not mocking in the Rbd tests.09:30
mdboothThat was satisfying :)09:30
openstackgerritGhanshyam Mann proposed openstack/nova master: Merge extended availability zone response into server controller  https://review.openstack.org/50285909:31
sean-k-mooneymdbooth: if you ran it with python3 the dict has determistic ordering09:31
mdboothsean-k-mooney: Interestingly I was running py309:31
mdboothAlthough py35, which doesn't have deterministic dict order09:31
mdboothiirc that 36 or 37?09:31
sean-k-mooneypy36 i think09:32
sean-k-mooneymdbooth: that said the iteration order is fixed not sure about calls to values or keys but would assume they return ordered lists also09:33
mdboothYeah, I'd assume so09:33
openstackgerritGhanshyam Mann proposed openstack/nova master: Merge config drive extension response into server controller  https://review.openstack.org/58422309:33
openstackgerritGhanshyam Mann proposed openstack/nova master: Merge extended server attributes extension response  https://review.openstack.org/58459009:33
*** Dinesh_Bhor has joined #openstack-nova09:34
openstackgerritGhanshyam Mann proposed openstack/nova master: Merge extended server attributes extension response  https://review.openstack.org/58459009:35
mriedemit's not sorted https://github.com/openstack/nova/blob/master/nova/tests/unit/image/fake.py#L16409:35
openstackgerritGhanshyam Mann proposed openstack/nova master: Merge keypair extension response into server view builder  https://review.openstack.org/58474809:35
openstackgerritGhanshyam Mann proposed openstack/nova master: Merge server usage extension response into server view builder  https://review.openstack.org/58526209:35
openstackgerritGhanshyam Mann proposed openstack/nova master: Merge security groups extension response into server view builder  https://review.openstack.org/58547509:36
tssuryamriedem: correct basic flavors (larger flavors users open special request) are available everywhere, special flavors for special projects in special cells where those tenants have access, by default projects are not mapped (when they are normal tenants/personal projects) and they land in a set of default cells. we have around ~4500 projects roughly which depending on where the resources are allocated,09:36
tssuryaget mapped to that cell09:36
tssuryasean-k-mooney: we use the pre-filter from dansmith for scheduling based on the same criteria: projects mapped to cells/aggregates09:36
tssuryaso that placement returns only those nodes09:37
sean-k-mooneytssurya: cool. i was wondering if ye were still using the oldway with the tenant affinity/isolation filter but that does not work right post palcement09:38
tssuryaright09:38
*** josecastroleon has quit IRC09:38
*** adrianc has quit IRC09:39
sean-k-mooneytssurya: how have ye found the perfromce with the prefilter + placement vs cellsv1 and no placement. is it working well for ye?09:39
*** alexchadin has quit IRC09:41
tssuryasean-k-mooney: not really that good, we use the "limits" thing for limiting placement results for getting a decent performance09:41
tssuryawe are trying to improve the scheduling part performance wise, mriedem had some patches09:42
tssuryabut we have not seen much improvement in production after applying those09:42
sean-k-mooneythats unfortunet. there is still alot of work going into makeing schdulign faster in general. hopefully the performacne is adaquit for now.09:46
mriedemi imagine the big chunk of the per-request scheduling time is taken in the DB queries which iterate all selected compute nodes and then pull all instance UUIDs on each node09:46
mriedemacross all cells09:46
mriedemthat could be optimized into a single big join per cell probably09:46
mriedembecause CERN isn't using the track_instance_changes config option right?09:47
mriedemwhere the computes report instance per host changes to the scheduler09:47
mriedem?09:47
mriedemthis method https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L71009:48
mriedemspecifically https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L73509:48
mriedemthat queries the cell db per compute host to find the list of instance uuids on that host09:49
sean-k-mooneymriedem: for the cern case you really only need to fine out what cell the project is mapped too right? so if we had placement aggreates per cell we would just need to add a member of to the placement query for the cells assocated with the tenant and let placement do all the work09:50
mriedemsean-k-mooney: CERN already has that09:51
tssuryamriedem: yea the time consumption is in the pulling of all instances info09:51
mriedemthey have tenants mapped to host aggregates which are mapped to resource provider aggregates09:51
mriedemand that does the pre-filtering09:51
sean-k-mooneymriedem: oh ok cool09:51
*** josecastroleon has joined #openstack-nova09:51
mriedemthey use this stuff https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#aggregates-in-placement09:52
sean-k-mooneymriedem: so why do we ever pull the per host info back?09:52
openstackgerritjichenjc proposed openstack/nova master: Add zvm admin intro and hypervisor information  https://review.openstack.org/53312509:52
openstackgerritjichenjc proposed openstack/nova master: Add zvm CI information  https://review.openstack.org/53351209:52
mriedemsean-k-mooney: for the affinity filters09:52
sean-k-mooneymriedem: ah but that is after the placmenet query right?09:53
mriedemyes09:53
*** sambetts|afk is now known as sambetts09:53
mriedempre-filter -> GET /allocation_candidates -> get_host_states -> per-host-instance query -> filters/weighers09:53
sean-k-mooneyok i miss understood. i taught you were implying we were doing the db query in the prefilter09:54
mriedemno the only thing we query there is which host aggregates are restricted for the tenant09:54
sean-k-mooneyya that makes sense.09:54
mriedemhttps://github.com/openstack/nova/blob/master/nova/scheduler/request_filter.py#L2609:55
openstackgerritMatthew Booth proposed openstack/nova master: Py3 fix in fake image service  https://review.openstack.org/59205609:56
openstackgerritMatthew Booth proposed openstack/nova master: Add regression test for bug 1550919  https://review.openstack.org/59173309:57
openstackgerritMatthew Booth proposed openstack/nova master: Don't delete disks on shared storage during evacuate  https://review.openstack.org/57884609:57
openstackbug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9)09:57
*** ratailor has quit IRC09:57
*** ratailor has joined #openstack-nova09:57
sean-k-mooneymriedem: right we just get the aggreate uuids and stuff them in the destination object09:58
mdboothmriedem: ^^^ is my proposed for failed evacuate on shared storage. The functional tests exercise all 4 combinations of shared/nonshared instance directory and share/nonshared block.09:59
* mriedem goes to get dumplings09:59
*** mriedem has quit IRC10:00
mdboothmriedem: Good call ;)10:00
sean-k-mooneythere is no stien spec template ye correct? im going to make one reusing the rocky template.10:01
sean-k-mooneyis it stien or stein?10:02
mdboothsean-k-mooney: The latter10:02
sean-k-mooneydamb german words not doing the i before e thing10:03
sean-k-mooneythat said more english word dont follow that then do also10:03
mdboothOur crazy european neighbours10:03
mdboothDon't understand the weightiness of consistent spelling10:04
sean-k-mooneyyou have seen my  speeling correct ( yes i know i misspelled spelling, however that was kind of a typo). sometimes constency is a good thing but that does not mean i like us spelling more then uk.10:07
*** tikitavi has joined #openstack-nova10:07
*** rha has joined #openstack-nova10:12
openstackgerritVlad Gusev proposed openstack/nova stable/pike: Fix service list for disabled compute using MC driver  https://review.openstack.org/59233710:14
*** s10 has joined #openstack-nova10:15
*** claudiub has joined #openstack-nova10:18
*** abhishekk has quit IRC10:23
openstackgerritClaudiu Belu proposed openstack/nova master: tests: autospecs all the mock.patch usages  https://review.openstack.org/47077510:26
openstackgerritClaudiu Belu proposed openstack/nova master: hyper-v: autospec classes before they are instantiated  https://review.openstack.org/34221110:26
*** jaosorior has joined #openstack-nova10:28
*** lei-zh has quit IRC10:30
openstackgerritClaudiu Belu proposed openstack/nova master: WIP: replace spec with autospec  https://review.openstack.org/55729910:30
*** alexchadin has joined #openstack-nova10:31
*** dpawlik has quit IRC10:31
*** Sigyn has quit IRC10:31
*** dpawlik has joined #openstack-nova10:32
*** Sigyn has joined #openstack-nova10:32
*** maciejjozefczyk has joined #openstack-nova10:37
*** vivsoni_ has joined #openstack-nova10:37
openstackgerritChen proposed openstack/nova master: Fix soft deleting vm fails after "nova resize" vm  https://review.openstack.org/54692010:38
*** vivsoni has quit IRC10:39
openstackgerritClaudiu Belu proposed openstack/nova master: hyperv: Cleans up live migration Planned VM  https://review.openstack.org/47894310:41
*** Bhujay has quit IRC10:45
*** maciejjozefczyk has quit IRC10:49
*** maciejjozefczyk has joined #openstack-nova10:51
*** mvkr has quit IRC10:54
*** panda is now known as panda|lunch11:02
*** adrianc has joined #openstack-nova11:05
*** Dinesh_Bhor has quit IRC11:08
*** jpena is now known as jpena|lunch11:08
*** maciejjozefczyk has quit IRC11:20
*** maciejjozefczyk has joined #openstack-nova11:22
*** maciejjozefczyk has quit IRC11:23
*** mvkr has joined #openstack-nova11:25
*** donghm has joined #openstack-nova11:26
*** gcb_ has quit IRC11:27
*** josecastroleon has quit IRC11:28
*** josecastroleon has joined #openstack-nova11:28
*** udesale has quit IRC11:30
*** josecastroleon has quit IRC11:31
*** josecastroleon has joined #openstack-nova11:31
*** Bhujay has joined #openstack-nova11:35
*** erlon has joined #openstack-nova11:38
*** aloga has joined #openstack-nova11:42
*** Dinesh_Bhor has joined #openstack-nova11:45
*** Dinesh_Bhor has quit IRC11:46
*** adrianc has quit IRC11:47
*** aloga has quit IRC11:49
*** aloga has joined #openstack-nova11:50
*** dpawlik has quit IRC11:53
*** dpawlik has joined #openstack-nova11:55
*** dpawlik has quit IRC11:55
*** dpawlik has joined #openstack-nova11:55
*** maciejjozefczyk has joined #openstack-nova11:55
*** cdent has quit IRC11:57
openstackgerritMatthew Booth proposed openstack/nova master: Move static _get_power_off_values to compute_utils  https://review.openstack.org/57882211:59
openstackgerritMatthew Booth proposed openstack/nova master: Move static _get_power_off_values to compute_utils  https://review.openstack.org/57882211:59
*** adrianc has joined #openstack-nova12:01
mdboothAny py3-loving cores fancy knocking this on the head: https://review.openstack.org/#/c/592056/ Trivial: change to explicit string type required by py3.12:07
*** ratailor has quit IRC12:07
*** zul has joined #openstack-nova12:08
*** jpena|lunch is now known as jpena12:14
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Make monkey patch work in uWSGI mode  https://review.openstack.org/59228512:15
efriedmdbooth: I glanced at that yesterday, but I'm not a py2-to-3 expert and the commit message is... sparse. Can you help me understand what's wrong and why it needs to be fixed?12:15
mdboothefried: Because the argument has to be binary12:16
mdboothIf it's not, it gives an error that the argument has to be binary12:16
efriedaccording to...?12:16
efriedwhat gives an error? data.write or _imagedata.get?12:16
efriedpresumably the former12:17
mdboothYeah12:17
efriedand where does the error show up? It must not be in the py35 tox job...?12:17
mdboothYeah, it was happening in the functional job12:18
efriedoooohhhh12:18
mdboothAlthough it was non-deterministic until I twigged why12:18
efriedI was blinded by the fact that the fake driver lives in the /unit/ path.12:18
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Make monkey patch work in uWSGI mode  https://review.openstack.org/59228512:18
efriedbut it's used for the func tests too.12:18
mdboothThey're pretty incestuous :)12:18
efriedokay, now that I have a way to try to repro locally I can take a better look at it.12:18
efriedIf you wouldn't mind enhancing the commit message, future me will be grateful.12:19
mdboothMaybe if you open the file binary it has to have binary data written to it?12:19
efriedWell, it makes sense that you would *want* to write binary if you're opening with 'b'.12:19
mdbooth>>> f = open('/tmp/foo', 'wb+')12:20
mdbooth>>> f.write('foo')12:20
mdboothTraceback (most recent call last):12:20
mdbooth  File "<stdin>", line 1, in <module>12:20
mdboothTypeError: a bytes-like object is required, not 'str'12:20
mdboothThat's the error you get, anyway12:20
mdbooth>>> f.write(b'foo')12:20
mdbooth312:20
efriedyeah, so they must have tightened that down in py3, eh?12:21
mdboothpy2:12:21
mdbooth>>> f = open('/tmp/foo', 'wb+')12:21
mdbooth>>> f.write('foo')12:21
mdbooth>>> f.write(b'foo')12:21
mdboothBoth work12:21
efriedwhich func test would fail?12:21
efriedI try not to run full suites locally unless I need to reheat my coffee.12:22
mdboothSo... the following functional test would fail non-deterministically depending on whether or not fake image service gave it an image with a kernel and ramdisk or not12:22
*** donghm has left #openstack-nova12:23
efriedWe have a test that sometimes uses one kind of image and sometimes another, nondeterministically??12:23
mdboothefried: Yep. A few :)12:23
efriedthat's... I can't... but WHY?12:24
mdboothI felt really good about myself for find that this morning :)12:24
mdboothefried: non-deterministic dict ordering12:24
*** jaosorior has quit IRC12:25
* efried looks into crystal ball and sees patch earlier in same series splitting such tests to always hit both/all paths...12:25
*** jaosorior has joined #openstack-nova12:25
efried...because the reward for good work is...12:25
mdboothefried: It's not relevant to my bug, or I would have. I was just tripping over it.12:26
efriedI was about to say: assume the topic bug isn't really related, so this should be tagged with the py3-first tag (whatever that tag is)12:27
efriedbtw, I recently discovered with cdent's guidance that you can avoid gerrit re-topic'ing your whole series by using git-review -T (available only in recent versions of git-review)12:28
mdboothI mean the non-deterministic images, that is. My patch just needs a functioning image service in order to boot. kernel vs non-kernel is irrelevant, except that it caused a bug.12:29
mdboothI don't really use topics, tbh12:29
mdboothI've always found it more useful to use the fact that gerrit displays a series in dependency order in 'Related Changes'12:30
efriedum. For misc random patch that's fine, but when they're associated with a bug, blueprint, or initiative (like mox removal or py3 first), it's useful.12:30
efriedWell, yeah, on that note, in this case the fix really doesn't need to be in that series at all, does it?12:31
mdboothIt does, because the functional test which follows it was tripping up on it12:31
mdboothnon-deterministically12:31
efriedbut only in py3, which isn't set up in the gate yet?12:32
mdboothYeah, we run functional on py312:32
efried"we" do?12:32
efriedlooking again...12:32
efriedoic12:32
mdboothhttp://logs.openstack.org/33/591733/4/check/nova-tox-functional-py35/d9266d8/12:32
*** eharney has quit IRC12:33
efriedOkay; I was looking at that yesterday (for an unrelated reason) and totally missed it :(12:33
efriedhmph12:33
mdboothI didn't know either until I starting writing this test :)12:34
*** josecastroleon has quit IRC12:34
mdboothefried: Here's the failure in the gate, btw:12:34
efriedanyway, you never got around to telling me which func test fails. I'm getting now that it's one you're introducing in your series...12:34
mdboothhttp://logs.openstack.org/33/591733/2/check/nova-tox-functional-py35/c3d298a/job-output.txt.gz12:34
efried...not an existing one12:34
mdbooth2018-08-15 14:21:38.07721612:34
kosamaraHi efried. I'm revising the YAML format to include keys that any driver understands. Some keys like vendor_id make sense to many drivere; however, they might expect different values for them.12:35
mdboothAnything exercising that in py3 will hit it, because it's basically a syntax error. Seems nothing was previously.12:36
efriedRight, my point.12:36
kosamaraEg PowerVM -> \X{4}, libvirt -> \x{4}12:36
*** cdent has joined #openstack-nova12:36
*** josecastroleon has joined #openstack-nova12:37
kosamaraI would like to say that the value format is the driver's responsibility as well, but there needs to be a common schema.12:37
efriedkosamara: Those values come from the hardware and their form (32-bit integer) is ordained by a standard.12:37
mdboothOoh, timestamps are clicky: http://logs.openstack.org/33/591733/2/check/nova-tox-functional-py35/c3d298a/job-output.txt.gz#_2018-08-15_14_21_38_07721612:37
* mdbooth assumed it was just text12:38
kosamaraLet me double check12:38
*** tbachman has joined #openstack-nova12:38
efriedkosamara: we could ask for it to be represented as an integer via the schema, so you could do 0xC001, 0xc001, 0140001 (octal), 49153 (decimal), etc.12:39
kosamaraefried: libvirt reports \x{4}12:40
kosamaraefried: solves the problem, but sounds confusing for the operator12:41
efriedkosamara: I agree.12:42
*** rha has quit IRC12:42
kosamaraefried: I would prefer to dictate an arbitrary format \X{4} and then it will be the driver's responsibility to normalize it.12:42
efriedkosamara: I'm fine with that. Note that yaml tries to outsmart you on this, though. It interprets e.g. C001 (without quotes) as a string, but 4321 (without quotes) as an integer base 10.12:43
efriedkosamara: So we should document that these hex values be quoted.12:44
efriedmdbooth: Is your test case really valid if it's trying to fetch a nonexistent image? Maybe we're better off fixing the fake by taking off the default to .get() to force the test author to set it up.12:46
efriedkosamara: Hm, which makes me wonder whether I did my schema validation correctly in https://review.openstack.org/#/c/579289/12:47
efriedbecause I remember running across this problem.12:47
efriedbut now I can't remember if the test failed because 4321 got interpreted as an int or because it failed schema validation because not a string.12:47
efriedI would hope the latter. Need to add a test.12:48
*** owalsh has quit IRC12:53
*** janki has quit IRC12:59
*** janki has joined #openstack-nova13:00
*** jmlowe has joined #openstack-nova13:00
kosamaraefried: nice, thanks for the tip!13:04
*** abhishekk has joined #openstack-nova13:05
*** owalsh has joined #openstack-nova13:06
mdboothefried: My test case isn't concerned with images.13:07
mdboothefried: Except that it needs to boot an instance which has disks.13:07
*** jmlowe has quit IRC13:12
*** udesale has joined #openstack-nova13:12
*** jaosorior has quit IRC13:14
*** jaosorior has joined #openstack-nova13:15
*** alex_xu has quit IRC13:15
*** eharney has joined #openstack-nova13:17
efriedmdbooth: I can fix the commit message and +2.13:17
openstackgerritEric Fried proposed openstack/nova master: Py3 fix in fake image service  https://review.openstack.org/59205613:19
*** jaosorior has quit IRC13:20
efriedmdbooth: Done ^13:20
tobias-urdinhave a weird issue, i have some instances stuck in building (scheduling) state which show up in dashboard and with api, but server show throws NotFound exception so I cant remove them, anyubody? i've restarted nova-api, scheduler and conductors.13:21
*** eharney has quit IRC13:21
*** eharney has joined #openstack-nova13:22
mdboothefried: Thanks13:24
openstackgerritMatthew Booth proposed openstack/nova master: Add regression test for bug 1550919  https://review.openstack.org/59173313:24
openstackbug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9)13:24
openstackgerritMatthew Booth proposed openstack/nova master: Don't delete disks on shared storage during evacuate  https://review.openstack.org/57884613:25
*** imacdonn has quit IRC13:29
*** nicolasbock has joined #openstack-nova13:32
*** alexchadin has quit IRC13:33
*** alexchadin has joined #openstack-nova13:34
*** gbarros has joined #openstack-nova13:38
*** imacdonn has joined #openstack-nova13:42
openstackgerritSurya Seetharaman proposed openstack/nova master: Making instance listing skipping down cells configurable  https://review.openstack.org/59242813:53
tssuryamriedem: ^13:55
*** awaugama has joined #openstack-nova13:58
kosamaraefried: What does PowerVM want to do, if the YAML it parses includes identification keys it doesn't understand?13:59
*** brinzh has quit IRC14:00
efriedkosamara: Not sure.14:00
efriedkosamara: Options: ignore, log, error.14:00
kosamaraefried: I'm tending towards making it an error in libvirt, with the assumption that the operator has made an oversight and will be baffled to see his devices not recognized if it's just ignored.14:00
efriedRight14:00
efriedthat's a reasonable argument.14:01
*** janki has quit IRC14:01
kosamaraShould it be enforced by the spec for all drivers though?14:01
efriedBecause the conf is going to have to be host-specific, which means driver-specific.14:01
efriedkosamara: But my hesitation is this:14:01
*** dpawlik has quit IRC14:01
*** mvkr has quit IRC14:02
efriedAt some point in the future, we might like to support a global conf on the conductor, whose values percolate down to the computes, which can override (optionally) via local configs of their own.14:02
*** josecastroleon has quit IRC14:02
efriedIn that scenario, the global conf would presumably want to include the union of all the relevant keys.14:03
efriedand the computes would have to ignore the ones it doesn't understand.14:03
efriedBut maybe we can solve that when we get there.14:03
tssuryadansmith, melwitt: FYI, mriedem has some stuff on the cells meeting agenda14:04
dansmithtssurya: for next week?14:04
tssuryaheh not really, just up for open discussion/opinions I guess14:05
efriedkosamara: As for attempting to define the same rule for all drivers, I think that's a good idea for consistency.14:05
*** josecastroleon has joined #openstack-nova14:06
*** _pewp_ has joined #openstack-nova14:08
*** josecastroleon has quit IRC14:10
*** Bhujay has quit IRC14:12
*** cdent has quit IRC14:12
*** slaweq_ is now known as slaweq14:14
sean-k-mooneykosamara: i would generally try to minimise divergnce options in dirver14:16
sean-k-mooneyefried: host-specfic doen not inherintly mean driver-specific14:17
efriedsean-k-mooney: Can we have more than one nova-compute service running on a single host? Or more than one driver loaded in a single nova-compute process?14:17
sean-k-mooneyefried: yes we can have more then one nova compute on the same host14:20
sean-k-mooneyefried: normally only done with ironic14:20
sean-k-mooneywe can only have 1 driver loaded as far as i am aware14:21
efriedThat sounds really confusing. Are they reporting different host resources?14:21
efriedor the compute services each "own" a different subset of the ironic nodes?14:22
sean-k-mooneythe compute serveice each own a subset of the ironic nodes14:22
efriedokay, so still, is there an example where multiple different drivers could exist on a single host?14:22
sean-k-mooneyefried: not really. no.14:23
sean-k-mooneyyou could have ironic and libvirt on the same node14:23
sean-k-mooneyi have done that but only for dev14:23
efriedokay. So host <=> driver for all intents and purposes.14:23
sean-k-mooneyyou not really going to run libvirt and say nova-docker on the same host that said you could14:24
efriedsean-k-mooney: If you did have that kind of dual setup, would they be pointing to different configs?14:24
sean-k-mooneyefried: yes14:24
efriedokay, then the model holds.14:24
sean-k-mooneyefried: the point i was more trying to make is that the config them selve are not virt driver specic in the content they contain14:25
efriedwtf, of course they are14:25
sean-k-mooneywhat is different?14:25
efriedyou try to put a libvirt nova.conf onto a powervm compute node, it won't work at all.14:25
sean-k-mooneyefried: only the libvirt section would be different14:26
sean-k-mooneyanything outside the virt driver section should be identical14:26
efriedThat sounds like a theoretical statement, not a practical one.14:26
sean-k-mooneyefried: if it not we have serious bugs14:26
sean-k-mooneythe only section that should be virt driver sepecific is the virt driver section.14:27
efriedSame config *format* I'll grant you, but the idea of being able to compose one config file and then copy it verbatim around a heterogeneous cloud - I have a hard time believing that anybody actually does that.14:27
sean-k-mooneyefried: alot of the installers bacsically do that14:28
sean-k-mooneyyou are right here may be some hardware specific changes such as the vcpu_pin_set14:29
sean-k-mooneybut that dose not depend on the virt driver virt driver14:29
dansmithsean-k-mooney: to have more than one compute per host you have to tell them each that they are a different hostname, and thus a different config, which means they might as well be separate hosts14:29
efriedkosamara: Based on this discussion, how do you feel about recommending that virt drivers log a warning, but not raise an exception, when they encounter a field they don't recognize/support/use?14:31
kosamaraefried: That's what I was converging to, I think that's best14:31
efriedthis would give the admin the right clue as to why their device doesn't show up, if they were in fact expecting that field to be processed.14:32
kosamarawhich satisfies my scenario14:32
sean-k-mooneydansmith: that is true.14:32
*** jaosorior has joined #openstack-nova14:32
efried"WARNING: Device identification field XXX in inventory.yaml not recognized by the YYY driver; ignoring"14:32
*** mvkr has joined #openstack-nova14:33
sean-k-mooneyefried: do you need to raise an exception?14:33
efriedno, that's what kosamara and I are talking about.14:34
sean-k-mooneysorry you said not raise an exception14:34
efriedIf it's a common thing for configs to be copied around a cloud, then we should log and ignore.14:34
sean-k-mooneyefried: ya if we raise an exception from config parsing we normall hard crash the agent. but that seam wrong in this case14:35
efriedsean-k-mooney: This processing is happening in the compute service, in the virt driver's update_provider_tree method, on startup and/or SIGHUP, so I don't think it crashes the compute service; but it would make lots of things not work, depending where the exception was raised in update_provider_tree.14:37
sean-k-mooneyefried: sorry i was not being clear. it not where the parsing is done. we have two classes of config options. 1 where if its invalid we stop the agent form running and a second where we log a warning and continue on14:40
efriedsean-k-mooney: Note that this isn't oslo.config14:40
sean-k-mooneythis feels more like a warning to me then stop everything14:40
sean-k-mooneyefried: yes. the impentation is not what im concerned about its the semantics im thinking about14:41
efriedRight. And semantically we converged on the fact that it's (apparently) fairly common for operators to write a config file once and then copy it around the cloud.14:43
efriedFor this reason, as well as in anticipation of future support for a global inventory.yaml on the conductor, it makes most sense *not* to treat an unrecognized field as an error.14:43
efriedBut, to avoid baffling the operator, we should log a warning that she'll see when she goes to investigate why her device isn't showing up.14:43
efriedkosamara: Fair summary?  ^14:43
*** alexchadin has quit IRC14:44
sean-k-mooneyefried: its not that they write them once it more most deployment tools you a base config template with  minimal per host overrides so the confige are largly the same.14:44
efriedcool.14:45
sean-k-mooneythe downside to that is you typo somthing once and you break everything.14:45
sean-k-mooneyi may have hard locked 15 servers due to a config typo in the past..14:46
*** ratailor has joined #openstack-nova14:59
*** cdent has joined #openstack-nova15:01
*** erlon has quit IRC15:01
*** munimeha1_ has joined #openstack-nova15:03
*** abhishekk has quit IRC15:05
*** ratailor has quit IRC15:05
*** ratailor has joined #openstack-nova15:06
*** mvkr has quit IRC15:06
*** r-daneel has joined #openstack-nova15:07
melwitt.15:07
*** erlon has joined #openstack-nova15:13
jaypipesmelwitt: ,15:21
melwitt!15:21
jaypipes?15:21
jaypipes:P15:21
melwittwe're at an impasse...15:22
jaypipesmelwitt: was just a friendly comma saying hi to your full stop.15:22
melwittheh15:22
*** hemna_ has joined #openstack-nova15:25
*** psachin has quit IRC15:27
*** luksky has joined #openstack-nova15:32
kosamaraefried: yes15:36
*** efried is now known as efried_afk15:37
*** gbarros has quit IRC15:45
openstackgerritClaudiu Belu proposed openstack/nova master: tests: autospecs all the mock.patch usages  https://review.openstack.org/47077515:48
*** dave-mccowan has joined #openstack-nova15:49
*** gyee has joined #openstack-nova15:53
*** gbarros has joined #openstack-nova16:02
sean-k-mooneymelwitt: dansmith. i have found an issue in my livemigration. specifcally there is a case where livemigration fails, the instance is still running on the soruce but the vif_type is none beacuse the host_id is not set.16:05
*** udesale has quit IRC16:05
sean-k-mooneyi need to do more digging to localise the actull issue16:05
dansmithbecause we update the instance info in the db from the failure yeah?16:06
dansmithI think mriedem had a patch for that recently, IIRC16:06
dansmithsean-k-mooney: does the healer periodic fix it after it runs?16:07
sean-k-mooneyim not sure. i think we may have activated the port binding for the dest and not reactivated for source.16:07
*** davidsha has quit IRC16:08
sean-k-mooneydansmith: ill get the logs and file a propper bug16:08
dansmithokay there was a situation where we saved the failed binding info cache recently that matt was working on, so might be related16:08
dansmithbut yeah16:08
*** ratailor has quit IRC16:09
sean-k-mooneydansmith: i dont think its related as teh vif_type is none in neutron not binding_failed.16:09
dansmithah okay16:09
*** tikitavi has quit IRC16:12
*** jpena is now known as jpena|off16:19
*** luksky has quit IRC16:20
kosamaraefried: On the namespacing of driver-generated traits: if we demand that all driver-generated traits exist within a namespace, we don't allow the drivers to generate standard traits.16:23
*** panda|lunch is now known as panda|off16:28
kosamaraI propose to allow the driver to generate either namespaced or standard traits.16:28
*** derekh has quit IRC16:29
*** sahid has quit IRC16:30
*** cdent has quit IRC16:32
*** cdent_ has joined #openstack-nova16:32
*** s10 has quit IRC16:32
*** pcaruana has quit IRC16:34
*** dpawlik has joined #openstack-nova16:41
*** dpawlik has quit IRC16:46
*** adrianc has quit IRC16:48
*** openstackgerrit has quit IRC16:49
*** tssurya has quit IRC17:28
*** openstackgerrit has joined #openstack-nova17:34
openstackgerritmelanie witt proposed openstack/nova-specs master: Move rocky implemented specs  https://review.openstack.org/59262217:34
openstackgerritChris Dent proposed openstack/nova master: Add explanatory prefix to post_test_perf output  https://review.openstack.org/59185017:35
openstackgerritChris Dent proposed openstack/nova master: Add trait query to placement perf check  https://review.openstack.org/59262417:35
*** dave-mccowan has quit IRC17:39
openstackgerritmelanie witt proposed openstack/nova-specs master: Dynamically find releases for move-implemented-specs  https://review.openstack.org/59262817:42
*** slaweq has quit IRC17:44
*** Sundar has joined #openstack-nova17:53
SundarHi all, could somebody tell me the status of nested resource providers, including virt driver changes (to avoid stomping on custom resources)? If there is a bp to track it, please point me to that. Thanks!17:54
melwittI think the placement side is considered done, but none of the nova integration side has been started yet. I'm not aware of a bp tracking it yet17:55
*** owalsh has quit IRC17:58
*** tbachman has quit IRC17:58
Sundarmelwitt: Thanks. So, in particular, the work to prevent virt drivers from overwriting custom resources has not started yet, right? If so, what is the blocker -- is it still upgrade considerations?17:58
dansmithSundar: resources or traits?17:59
Sundardansmith: I think both. When last checked, the virt drivers just overwrite the inventory of the compute node.18:00
dansmithwell, nova-compute does that with the help of the virt driver,18:00
openstackgerritmelanie witt proposed openstack/nova-specs master: Add a script for counting blueprints  https://review.openstack.org/58191418:01
dansmithbut inventory of a provider should only be owned by a single entity,18:01
sean-k-mooneySundar: in dublin we said external service were not allowed to create inventoei in resouce providers they did not create18:01
dansmithand since there is only one provider for a compute node right now, overwriting inventory entirely is kinda expected18:01
dansmithright, what sean-k-mooney said18:01
dansmithso until the providers start reporting things as nested,18:01
*** efried_afk is now known as efried18:01
dansmiththere's no real opportunity for the coexistence of other nested providers with their own inventory they manage18:02
efriedkosamara: Yes, tricky question as to who "owns" a standard trait.18:02
dansmithand that is dependent on the upgrade bit getting worked out18:02
SundarYes. Even with nested RPs, we had the issue that the libvirt driver would overwrite the nested RP config. This was about a month or so ago.18:02
*** tbachman has joined #openstack-nova18:03
dansmithSundar: because it's not nested-aware, AFAIK18:03
efriedtrue ^18:03
efrieddansmith: The upgrade (reshape) series is ready for review btw.18:04
dansmithyeah I know18:04
efried:P18:04
sean-k-mooneySundar: cyborg will have to create a seperate resouce provider tree under the compute node. it should be possible to do that today but im not sure if the virt driver will remove them or not18:05
efriedkosamara: Do you understand the concern about trait "ownership" in general?18:05
efriedsean-k-mooney: The virt driver won't remove them; but we also won't schedule to them either.18:05
sean-k-mooneyefried: can you allocate from them18:06
sean-k-mooneythe intent would to still schduler to the compute node but consume the cyborg resouce from the cyborg subtrees18:06
efriedsean-k-mooney: basically, that whole code path has some bits that will work, some bits that definitely won't, and some that are still a mystery. This is work that needs to be done in Stein.18:06
efriedsean-k-mooney: I think the scheduler *should* consume the cyborg resources.18:07
efriedIt's just not allowed to muck with the cyborg providers' inventories or traits.18:07
Sundarsean-k-mooney: Yes, cyborg is trying to do nRPs. We hit the virt driver overwrite issue and developers decided to wait for that to be resolved.18:07
sean-k-mooneyefried: right. we may also have to revisit the idea of have a seperate root node just for the phyical server and then have per projec subtrees18:07
sean-k-mooneySundar: again if the overriete issue is you tried to create invtories on the comptue node resouce provider that will never be supported18:08
*** gbarros has quit IRC18:08
efriedrevisit? I don't remember visiting that in the first place, ever. Per-project subtrees?18:08
sean-k-mooneyefried: it came up in dublin on the friday when we were all tired18:09
Sundarsean-k-mooney: no, we created nested RPs within the compute node.18:09
efriedI think Sundar is talking about creating child providers of the compute node provider but having them ignored by the scheduler and/or deleted by the resource tracker. The latter may have happened in libvirt when it was still using get_inventory.18:09
SundarYes ^18:09
efriedI'm guessing you haven't tried it since we implemented libvirt's update_provider_tree method.18:09
Sundarefried: when was that done?18:10
efriedSundar: So if you have the resources and inclination, you could try that again. I guarantee it will still break, but it may get slightly further than it did before.18:10
efriedSundar: Let me find the patch...18:10
sean-k-mooneyefried: the idea being that nova would nolonger own the root node of the tree. the root would have no invetores and just represent teh server then each service on the serve would create there own nested resouce provider beneate that root node18:11
Sundarefried: "I guarantee it will still break" -- that won't help me persuade other Cyborg developers to start moving fwd :)18:11
efriedSundar: https://review.openstack.org/#/c/560444/ merged 5/2518:11
sean-k-mooneyefried: it was a fallback incase we did not figure out how to have multple service own different nodes under the current tree create by nova18:11
efriedsean-k-mooney: Nono, nova still owns the root provider. But nova does *not* own the device providers - cyborg owns those.18:11
efriedoh, sorry, you're talking about the project-based thing.18:12
sean-k-mooneyefried: ya sorry i was18:12
*** owalsh has joined #openstack-nova18:12
efriedsean-k-mooney: Without further placement API work, we would probably have to do that by lassoing the project subtree in an aggregate corresponding to the project ID.18:12
sean-k-mooneyefried: by project id you mean service id?18:13
*** mmedvede has quit IRC18:13
efriedBut regardless, that would still rely on scheduler and resource tracker implementation that we have not started yet. Same as the stuff Sundar is blocked behind.18:13
sean-k-mooneywhen i said per-procject i ment nova, neutron, cinder not tenants18:13
efriedum, oh, then forget what I'm saying.18:14
efriedBut yes, all of that needs to be thought through.18:14
efriedsean-k-mooney: I would think that nova would still own the root node.18:14
efriedSundar: Define "moving fwd"? You can't integrate with nova yet, for sure. But I assume there's work you can do while you're waiting for enablement.18:15
sean-k-mooneyefried: perhaps, the idea of spliting it out was in the event that nova is not the first service deployed on the node, we won know what uuid it will get so we cant create the provider tree18:15
efriedsomebody would have to reparent18:16
efriedwhich can be done as long as parent isn't already set.18:16
*** mmedvede has joined #openstack-nova18:17
sean-k-mooneyefried: once that reshaper stuff is reviewed :)18:17
sean-k-mooneyalthough i guess you could manually reparent now its just not as simple18:18
*** r-daneel has quit IRC18:20
efriedactually I don't think reshaper accounts for reparenting at all. I was talking about:18:20
efriedNeutron starts first. Creates the NIC RPs. They have no parent.18:20
efriedCyborg starts next. Creates the dev RPs. They have no parent (and aren't related to the NIC RPs).18:20
efriedNow Nova starts. It creates the compute node RP. It also somehow discovers that the NIC and device RPs are out there, and reparents them (PUT /resource_provider/{u} with parent_provider_uuid specified in the payload) to the compute node RP.18:20
*** tbachman has quit IRC18:22
Sundarefried: Why can't Cyborg and Neutron wait for Nova compute to create the top level first?18:24
*** awaugama has quit IRC18:24
*** claudiub has quit IRC18:26
SundarIf that is simpler, that is18:26
efriedI don't know. Maybe they can. Ask sean-k-mooney18:28
*** awaugama has joined #openstack-nova18:28
*** _pewp_ has quit IRC18:37
*** _pewp_ has joined #openstack-nova18:37
*** hemna_ has quit IRC18:38
openstackgerritMerged openstack/nova master: Py3 fix in fake image service  https://review.openstack.org/59205618:48
*** hemna_ has joined #openstack-nova18:51
*** harlowja has joined #openstack-nova18:51
openstackgerritEric Fried proposed openstack/nova-specs master: Repropose reshaper spec for Stein  https://review.openstack.org/59265018:53
*** awaugama has quit IRC18:55
*** hemna_ has quit IRC18:56
*** r-daneel has joined #openstack-nova18:56
*** awaugama has joined #openstack-nova18:56
*** slaweq has joined #openstack-nova19:03
openstackgerritJay Pipes proposed openstack/nova master: placement: use single-shot INSERT/DELETE agg  https://review.openstack.org/59265419:04
*** slaweq has quit IRC19:19
*** slaweq has joined #openstack-nova19:22
*** cdent_ has quit IRC19:23
*** _pewp_ has quit IRC19:25
openstackgerritEric Fried proposed openstack/nova-specs master: Repropose reshaper spec for Stein  https://review.openstack.org/59265019:26
*** _pewp_ has joined #openstack-nova19:27
*** tbachman has joined #openstack-nova19:28
*** tbachman has quit IRC19:32
*** slaweq has quit IRC19:38
*** tbachman has joined #openstack-nova19:40
*** awaugama has quit IRC19:50
*** eharney has quit IRC19:58
*** HW_Peter has joined #openstack-nova20:04
*** HW-Peter has quit IRC20:07
*** eharney has joined #openstack-nova20:13
*** pcaruana has joined #openstack-nova20:13
*** tbachman has quit IRC20:15
*** eharney has quit IRC20:24
*** slaweq has joined #openstack-nova20:35
*** r-daneel has quit IRC20:36
*** takashin has joined #openstack-nova20:44
melwittnova meeting in 10 minutes20:50
*** r-daneel has joined #openstack-nova20:53
*** pcaruana has quit IRC20:53
*** tbachman has joined #openstack-nova20:58
openstackgerritDan Smith proposed openstack/nova master: WIP: Batch results per cell when doing cross-cell listing  https://review.openstack.org/59269820:59
openstackgerritJay Pipes proposed openstack/nova master: placement: use single-shot INSERT/DELETE agg  https://review.openstack.org/59265421:02
*** Sundar has quit IRC21:05
*** mchlumsky has quit IRC21:11
openstackgerritDan Smith proposed openstack/nova master: WIP: Batch results per cell when doing cross-cell listing  https://review.openstack.org/59269821:24
*** erlon has quit IRC21:26
*** mvkr has joined #openstack-nova21:29
*** munimeha1_ has quit IRC21:35
*** rcernin has joined #openstack-nova21:49
*** slaweq has quit IRC21:49
openstackgerritJay Pipes proposed openstack/nova master: placement: use single-shot INSERT/DELETE agg  https://review.openstack.org/59265422:00
openstackgerritJay Pipes proposed openstack/nova master: placement: use single-shot INSERT/DELETE agg  https://review.openstack.org/59265422:04
*** tbachman has quit IRC22:13
*** harlowja has quit IRC22:14
openstackgerritEric Fried proposed openstack/nova-specs master: Delete approved template in move_implemented_specs  https://review.openstack.org/59275522:16
efriedmelwitt: In case you were busily working on this enhancement ^22:17
melwitthah, thanks22:19
*** edmondsw has quit IRC22:20
*** edleafe has quit IRC22:22
efriedmelwitt: Mind if I combine your patches for dynamically finding releases and counting blueprints into a series that commonizes the fs walk bit?22:24
melwittefried: nope, go ahead22:25
*** edmondsw has joined #openstack-nova22:25
*** chyka has joined #openstack-nova22:32
*** dklyle has joined #openstack-nova22:33
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (3)  https://review.openstack.org/57410422:37
openstackgerritTakashi NATSUME proposed openstack/nova master: Making consistent used of GiB and MiB in API ref  https://review.openstack.org/55998522:42
openstackgerritmelanie witt proposed openstack/nova-specs master: Move rocky implemented specs  https://review.openstack.org/59262222:49
openstackgerritEric Fried proposed openstack/nova-specs master: Add a script for counting blueprints  https://review.openstack.org/58191422:51
efriedmelwitt: ^ rebased on top of the dynamic-y one22:52
melwittcool, thanks22:54
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (4)  https://review.openstack.org/57410623:03
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (5)  https://review.openstack.org/57411023:06
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (6)  https://review.openstack.org/57411323:09
*** slaweq has joined #openstack-nova23:10
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (7)  https://review.openstack.org/57497423:11
*** chyka has quit IRC23:13
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (8)  https://review.openstack.org/57531123:14
*** slaweq has quit IRC23:15
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (9)  https://review.openstack.org/57558123:17
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (10)  https://review.openstack.org/57601723:20
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (11)  https://review.openstack.org/57601823:22
*** erlon has joined #openstack-nova23:23
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (12)  https://review.openstack.org/57601923:26
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13)  https://review.openstack.org/57602023:29
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14)  https://review.openstack.org/57602723:36
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15)  https://review.openstack.org/57603123:41
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16)  https://review.openstack.org/57629923:44
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17)  https://review.openstack.org/57634423:48
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18)  https://review.openstack.org/57667323:51
*** gyee has quit IRC23:53
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19)  https://review.openstack.org/57667623:54
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20)  https://review.openstack.org/57668923:57
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21)  https://review.openstack.org/57670923:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!