| *** igordc has quit IRC | 00:05 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Avoid logging traceback when detach device not found https://review.opendev.org/671640 | 00:05 |
|---|---|---|
| *** mlavalle has quit IRC | 00:24 | |
| *** igordc has joined #openstack-nova | 00:24 | |
| *** igordc has quit IRC | 00:30 | |
| *** brinzhang has joined #openstack-nova | 00:30 | |
| *** tetsuro has joined #openstack-nova | 00:30 | |
| *** brinzhang_ has joined #openstack-nova | 00:32 | |
| *** sapd1 has joined #openstack-nova | 00:32 | |
| *** dakshina-ilangov has quit IRC | 00:34 | |
| *** TxGirlGeek has joined #openstack-nova | 00:56 | |
| *** BjoernT has joined #openstack-nova | 00:57 | |
| *** BjoernT has quit IRC | 00:58 | |
| *** gyee has quit IRC | 01:00 | |
| *** ricolin_ is now known as ricolin | 01:04 | |
| *** brinzhang_ has quit IRC | 01:06 | |
| *** brinzhang has quit IRC | 01:06 | |
| *** brinzhang has joined #openstack-nova | 01:07 | |
| alex_xu | kashyap: I replied you on the maillist, looks like ceilometer still using the perf meters, include some of meter isn't just about cmt. | 01:07 |
| *** imacdonn has quit IRC | 01:16 | |
| *** BjoernT has joined #openstack-nova | 01:17 | |
| *** imacdonn has joined #openstack-nova | 01:17 | |
| *** BjoernT has quit IRC | 01:24 | |
| *** whoami-rajat has joined #openstack-nova | 01:34 | |
| *** JamesBenson has joined #openstack-nova | 01:55 | |
| *** JamesBenson has quit IRC | 01:59 | |
| *** bhagyashris has joined #openstack-nova | 01:59 | |
| *** TxGirlGeek has quit IRC | 02:05 | |
| *** tkajinam has quit IRC | 02:21 | |
| *** tkajinam has joined #openstack-nova | 02:21 | |
| *** tetsuro has quit IRC | 02:34 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix cleaning up console tokens https://review.opendev.org/637716 | 02:38 |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Replace "integrated-gate-py3" template with new "integrated-gate-compute" https://review.opendev.org/671551 | 03:09 |
| *** tbachman has joined #openstack-nova | 03:09 | |
| *** tbachman has quit IRC | 03:17 | |
| *** brinzhang_ has joined #openstack-nova | 03:18 | |
| *** brinzhang has quit IRC | 03:22 | |
| *** psachin has joined #openstack-nova | 03:37 | |
| *** ricolin_ has joined #openstack-nova | 03:40 | |
| *** ricolin has quit IRC | 03:42 | |
| *** rpittau|afk is now known as rpittau | 03:44 | |
| *** udesale has joined #openstack-nova | 04:04 | |
| *** _erlon_ has quit IRC | 04:12 | |
| *** pcaruana has joined #openstack-nova | 04:37 | |
| *** rpittau is now known as rpittau|Imonapla | 04:38 | |
| *** rpittau|Imonapla is now known as rpittau|flying | 04:38 | |
| *** ratailor has joined #openstack-nova | 04:39 | |
| *** dpawlik has joined #openstack-nova | 04:51 | |
| *** abhishekk has joined #openstack-nova | 04:56 | |
| *** Luzi has joined #openstack-nova | 04:57 | |
| *** tkajinam has quit IRC | 05:01 | |
| *** TxGirlGeek has joined #openstack-nova | 05:21 | |
| *** TxGirlGeek has quit IRC | 05:23 | |
| openstackgerrit | Alex Xu proposed openstack/nova-specs master: Use PCPU and VCPU in one instance https://review.opendev.org/668656 | 05:29 |
| openstackgerrit | Alex Xu proposed openstack/nova-specs master: Use PCPU and VCPU in one instance https://review.opendev.org/668656 | 05:37 |
| *** udesale has quit IRC | 05:42 | |
| *** udesale has joined #openstack-nova | 05:42 | |
| *** mrjk_ has joined #openstack-nova | 05:46 | |
| *** mmethot_ has joined #openstack-nova | 05:46 | |
| *** mchlumsky_ has joined #openstack-nova | 05:47 | |
| *** jistr has quit IRC | 05:47 | |
| *** kaisers has quit IRC | 05:47 | |
| *** mgoddard has quit IRC | 05:48 | |
| *** kaisers has joined #openstack-nova | 05:48 | |
| *** jistr has joined #openstack-nova | 05:48 | |
| *** mmethot has quit IRC | 05:49 | |
| *** mrjk has quit IRC | 05:49 | |
| *** artom has quit IRC | 05:49 | |
| *** mchlumsky has quit IRC | 05:49 | |
| *** mgoddard has joined #openstack-nova | 05:51 | |
| *** gibi has quit IRC | 05:55 | |
| *** tkajinam has joined #openstack-nova | 05:55 | |
| *** rcernin has quit IRC | 06:02 | |
| *** udesale has quit IRC | 06:12 | |
| *** udesale has joined #openstack-nova | 06:17 | |
| *** belmoreira has joined #openstack-nova | 06:21 | |
| *** gibi has joined #openstack-nova | 06:24 | |
| *** maciejjozefczyk has joined #openstack-nova | 06:39 | |
| *** hoonetorg has quit IRC | 06:39 | |
| *** hoonetorg has joined #openstack-nova | 06:56 | |
| *** damien_r has joined #openstack-nova | 06:58 | |
| *** belmoreira has quit IRC | 07:03 | |
| *** belmoreira has joined #openstack-nova | 07:04 | |
| *** belmoreira has quit IRC | 07:05 | |
| *** slaweq has joined #openstack-nova | 07:09 | |
| *** tssurya has joined #openstack-nova | 07:10 | |
| *** zbr has quit IRC | 07:15 | |
| *** zbr has joined #openstack-nova | 07:16 | |
| *** helenafm has joined #openstack-nova | 07:18 | |
| *** ttsiouts has joined #openstack-nova | 07:26 | |
| *** ttsiouts has quit IRC | 07:36 | |
| *** ivve has joined #openstack-nova | 07:44 | |
| *** luksky11 has joined #openstack-nova | 07:46 | |
| *** cdent has joined #openstack-nova | 07:50 | |
| kashyap | alex_xu: Will check; thank you | 07:58 |
| *** ralonsoh has joined #openstack-nova | 08:00 | |
| alex_xu | kashyap: not sure how ceilometer enable those meters https://github.com/openstack/ceilometer/blob/master/ceilometer/compute/virt/libvirt/inspector.py#L218, since nova doesn't pass those parameter to libvirt | 08:01 |
| kashyap | Yeah, was wondering the same; thanks for the code pointer | 08:01 |
| cdent | alex_xu, kashyap : I seem to recall that at some point ceilometer made their own compute node agent rather than listening to nova, because nova didn't want to give all the information that ceilo wanted to get | 08:04 |
| *** ccamacho has joined #openstack-nova | 08:04 | |
| cdent | I may be remembering wrong though. Also I have zero context here so I may be butting in with info that you've already got. | 08:04 |
| * cdent remembers somewhat fondly the simple days of being a ceilometer core, all those years ago | 08:04 | |
| kashyap | cdent: No, I didn't know what you said; that's useful to know | 08:04 |
| cdent | kashyap: you'll definitely want to check me on that. It's a vague memory | 08:05 |
| kashyap | cdent: The context is this patch: https://review.opendev.org/#/c/669129/ | 08:05 |
| kashyap | cdent: Linux kernel has removed the underlying infrastructure for Intel "CMT" | 08:05 |
| cdent | seems like it's a done deal then, yeah? | 08:06 |
| kashyap | Right | 08:06 |
| kashyap | We already today warn if any anyone specifies the "CMT" 'performance' events via Nova's config attributes | 08:06 |
| cdent | kill it! kill it with fire! | 08:06 |
| kashyap | Hehe, yeah. | 08:06 |
| kashyap | However there are *other* 'perf' events besides the CMT variants. So I asked on the mailing list if anyone is using them at all. | 08:07 |
| kashyap | [While bearing in mind -discuss list audience does not represent all users ever :D] | 08:07 |
| *** derekh has joined #openstack-nova | 08:08 | |
| *** ttsiouts has joined #openstack-nova | 08:08 | |
| * cdent has often wondered where the users hang out | 08:08 | |
| *** rpittau|flying is now known as rpittau | 08:13 | |
| alex_xu | kashyap: we can write other event than cmt https://github.com/openstack/nova/blob/master/nova/conf/libvirt.py#L785 | 08:13 |
| kashyap | alex_xu: Right, I wrote that documentation :-) | 08:14 |
| kashyap | I was merely wondering if there are real-world users who are using these other events at all | 08:14 |
| alex_xu | kashyap: I missed understand that conf only can can fill cmt, mbm..etc | 08:15 |
| kashyap | No, not really -- any of the allowable `perf` events by libvirt | 08:15 |
| alex_xu | kashyap: yes, that is always a question hard to answer | 08:15 |
| kashyap | Hence my question on the list about *non-CMT* events | 08:15 |
| kashyap | Right | 08:15 |
| alex_xu | oops | 08:17 |
| kashyap | alex_xu: Sorry if I didn't phrase it clearly. | 08:19 |
| *** cdent has quit IRC | 08:21 | |
| *** brinzhang has joined #openstack-nova | 08:25 | |
| *** brinzhang_ has quit IRC | 08:28 | |
| *** yaawang has quit IRC | 08:47 | |
| *** panda is now known as panda|drappt | 08:50 | |
| *** yaawang has joined #openstack-nova | 08:51 | |
| *** tetsuro has joined #openstack-nova | 08:56 | |
| *** ricolin_ is now known as ricolin | 08:59 | |
| *** ag-47 has joined #openstack-nova | 09:03 | |
| *** tkajinam has quit IRC | 09:05 | |
| *** derekh has quit IRC | 09:09 | |
| *** derekh has joined #openstack-nova | 09:19 | |
| *** xek has joined #openstack-nova | 09:20 | |
| openstackgerrit | zhangyangyang proposed openstack/nova master: Bump the openstackdocstheme extension to 1.20 https://review.opendev.org/671694 | 09:29 |
| *** cdent has joined #openstack-nova | 09:31 | |
| *** takashin has left #openstack-nova | 09:33 | |
| *** tetsuro has quit IRC | 09:43 | |
| *** ttsiouts has quit IRC | 09:45 | |
| *** ttsiouts has joined #openstack-nova | 09:46 | |
| *** artom has joined #openstack-nova | 09:47 | |
| *** artom has quit IRC | 09:47 | |
| *** artom has joined #openstack-nova | 09:47 | |
| *** ttsiouts has quit IRC | 09:50 | |
| *** ttsiouts has joined #openstack-nova | 09:55 | |
| *** bhagyashris has quit IRC | 10:01 | |
| *** udesale has quit IRC | 10:05 | |
| *** udesale has joined #openstack-nova | 10:06 | |
| *** panda|drappt is now known as panda | 10:14 | |
| *** ttsiouts has quit IRC | 10:23 | |
| *** ttsiouts has joined #openstack-nova | 10:24 | |
| *** brault has joined #openstack-nova | 10:26 | |
| *** ttsiouts has quit IRC | 10:28 | |
| *** luksky11 has quit IRC | 10:28 | |
| *** brinzhang has quit IRC | 10:29 | |
| openstackgerrit | zhangyangyang proposed openstack/nova master: Bump the openstackdocstheme extension to 1.20 https://review.opendev.org/671694 | 10:32 |
| *** brault has quit IRC | 10:32 | |
| *** cdent has quit IRC | 10:32 | |
| *** priteau has joined #openstack-nova | 10:32 | |
| *** priteau has quit IRC | 10:35 | |
| *** priteau has joined #openstack-nova | 10:37 | |
| *** udesale has quit IRC | 10:44 | |
| *** udesale has joined #openstack-nova | 10:45 | |
| *** priteau has quit IRC | 10:47 | |
| *** udesale has quit IRC | 10:54 | |
| *** udesale has joined #openstack-nova | 10:55 | |
| *** abhishekk has quit IRC | 10:56 | |
| *** luksky11 has joined #openstack-nova | 10:56 | |
| *** udesale has quit IRC | 11:02 | |
| *** ag-47 has quit IRC | 11:02 | |
| *** cdent has joined #openstack-nova | 11:08 | |
| *** ttsiouts has joined #openstack-nova | 11:09 | |
| *** tesseract has joined #openstack-nova | 11:33 | |
| artom | alex_xu, still around? I figure we can talk here rather than on gerrit, about vpmem and NUMA LM | 11:35 |
| *** panda is now known as panda|lunch | 11:42 | |
| *** xek has quit IRC | 11:43 | |
| *** xek has joined #openstack-nova | 11:43 | |
| artom | So the current flow is driver.cclm_dest -> rpc.cclm_source -> claim -> driver.numa_config | 11:44 |
| artom | The claim has to happen before driver.numa_config, and presumably before driver.vpmem_config or whatever you'd call it | 11:45 |
| artom | And rpc.cclm_source has to happen before the claim, because the source will tell the destination if it's trying to perform a NUMA live migration | 11:45 |
| artom | (We can't just check instance.numa_topology because the source might be an old compute) | 11:46 |
| artom | (But that could change in a later release) | 11:46 |
| artom | So really the question is, can rpc.cclm_source happen before driver.cclm_dest | 11:46 |
| * artom checks that cclm_source needs | 11:47 | |
| artom | *what | 11:47 |
| artom | Ah, it's passed the dest_check_data | 11:48 |
| artom | So it doesn't look like we can change the order | 11:51 |
| artom | I'll put all that ^^ in gerrit :) | 11:51 |
| *** spatel has joined #openstack-nova | 11:52 | |
| *** _erlon_ has joined #openstack-nova | 12:05 | |
| *** spatel has quit IRC | 12:08 | |
| *** belmoreira has joined #openstack-nova | 12:10 | |
| *** udesale has joined #openstack-nova | 12:21 | |
| *** ag-47 has joined #openstack-nova | 12:30 | |
| *** belmoreira has quit IRC | 12:32 | |
| *** belmoreira has joined #openstack-nova | 12:34 | |
| *** panda|lunch is now known as panda | 12:36 | |
| *** spatel has joined #openstack-nova | 12:36 | |
| *** spatel has quit IRC | 12:42 | |
| *** bbowen has joined #openstack-nova | 12:44 | |
| *** udesale has quit IRC | 12:49 | |
| *** belmoreira has quit IRC | 12:51 | |
| *** tssurya has quit IRC | 12:52 | |
| *** ratailor has quit IRC | 12:52 | |
| *** belmoreira has joined #openstack-nova | 12:52 | |
| *** udesale has joined #openstack-nova | 12:53 | |
| *** cdent has quit IRC | 12:53 | |
| *** belmoreira has quit IRC | 12:56 | |
| *** jdillaman has joined #openstack-nova | 13:00 | |
| *** bbowen has quit IRC | 13:02 | |
| *** bbowen has joined #openstack-nova | 13:02 | |
| *** xek has quit IRC | 13:17 | |
| *** eharney has joined #openstack-nova | 13:21 | |
| efried | stephenfin: there's this spec https://review.opendev.org/#/c/668656 about mixing VCPU/PCPU in one instance. | 13:22 |
| efried | I see you looked at it a few patch sets ago | 13:22 |
| stephenfin | Yup. I really don't want that to happen this cycle | 13:23 |
| efried | oh | 13:23 |
| efried | whyzat? | 13:23 |
| stephenfin | I've been working on cpu-resources, which that relies on, all this week and a good bit of last week | 13:23 |
| efried | yes, obviously that's a hard dep | 13:23 |
| stephenfin | The upgrade scenario is tricky af | 13:23 |
| stephenfin | but it's being made easier by some assumptions I can make through a VM being all one kind of CPU | 13:24 |
| efried | you mean reshaping allocations? | 13:24 |
| stephenfin | Among other things, yeah | 13:24 |
| efried | Hm | 13:24 |
| stephenfin | The reason we dragged that out in the first place was because cpu-resources was horribly complex | 13:25 |
| efried | yeah, I get that. | 13:25 |
| efried | can a VM today have pinned and shared? | 13:25 |
| stephenfin | Not today, no | 13:25 |
| efried | oh, I see. Yeah, that makes a difference. | 13:25 |
| stephenfin | Except for emulator threads | 13:25 |
| stephenfin | But that's not the same thing at all | 13:25 |
| stephenfin | Anyway, I'm not sure why dragging that complexity back in would be something we'd ever want to do | 13:26 |
| efried | well | 13:26 |
| stephenfin | This cycle, that is | 13:26 |
| efried | there's a strong customer use case | 13:26 |
| stephenfin | Next cycle, I'm all over that | 13:26 |
| efried | tbc, the ask on you would be review, not code | 13:26 |
| stephenfin | I figured there might be, aye :) | 13:26 |
| efried | just not sure why we would want to say "next cycle" rather than "once cpu-resources is done" | 13:27 |
| stephenfin | Tell you what, let's wait til I have code up | 13:27 |
| stephenfin | Ha, jinx | 13:27 |
| efried | which may wind up being the same thing, but ... yeah | 13:27 |
| efried | except that spec freeze is next week | 13:28 |
| stephenfin | Yeah, I'd like to wait to see how cpu-resources goes before committing to anything | 13:28 |
| stephenfin | I wanted to have that up this week but unit test coupling is killing me | 13:28 |
| stephenfin | and I haven't even started on the reshape | 13:28 |
| efried | as you well know, approving a spec for this release doesn't "commit" us to getting it in this release. | 13:28 |
| stephenfin | Technically, no, but it does send the wrong message to people that aren't as involved in nova as we are | 13:29 |
| stephenfin | (PMs) | 13:29 |
| *** belmoreira has joined #openstack-nova | 13:29 | |
| efried | It would be a better story for Alex et al if they could go back and tell their downstream "approved, but considerable risk due to deps" than "no chance in Train". | 13:29 |
| efried | yeah, Alex and I can manage PM expectations :) | 13:29 |
| stephenfin | Suggesting we think this can land when I, for example, have very little confidence that it will | 13:29 |
| stephenfin | Oh, I'm worried about my PM :P | 13:29 |
| efried | okay. Alex and I can't manage your PM. | 13:30 |
| stephenfin | Not sure anyone can, heh | 13:30 |
| stephenfin | Anyway, not really the point | 13:30 |
| stephenfin | As above, I'm not confident about landing that feature, but if the ask is to just review the spec, I can do that | 13:31 |
| stephenfin | Though not today. I want to keep chipping away at these unit tests and hopefully get the first few patches pushed | 13:31 |
| efried | I think it would be a good step if we can get you & sean-k-mooney agreeing to the design itself, before spec freeze. | 13:32 |
| stephenfin | Yeah, that makes sense | 13:33 |
| efried | Then we can armwrestle over whether it makes sense to merge it for Train with the caveat of high risk, or merge it in backlog/ | 13:33 |
| stephenfin | Okay, I can set aside time to do that early next week, maybe in person with sean-k-mooney | 13:33 |
| efried | thank you sir | 13:33 |
| efried | and with alex_xu | 13:33 |
| sean-k-mooney | sorry was away form my laptop what was the context? | 13:34 |
| stephenfin | In person might be tough there, but sure :) | 13:34 |
| efried | easier for you than for me. | 13:34 |
| sean-k-mooney | efried: i think he ment alex | 13:34 |
| efried | sean-k-mooney: we're talking about the mixed PCPU/VCPU spec | 13:34 |
| sean-k-mooney | oh ok | 13:34 |
| efried | TLDR, stephenfin and I both agree it would be quite high risk for Train, and need to decide whether we want to merge it with that caveat or take a hard line and explicitly punt it. But in either case, I would like to see the design sorted before spec freeze, so we merge it either to train/ or backlog/. | 13:35 |
| sean-k-mooney | for what its worth i think we could do that but im concerned with the scope. e.g. if we had completed teh rest fo the cpu stuff by m2 i would have been fine with it | 13:35 |
| sean-k-mooney | efried: ok | 13:36 |
| sean-k-mooney | ill look at it again | 13:36 |
| efried | thanks | 13:36 |
| sean-k-mooney | i think at most we only need 1 extraspec | 13:36 |
| sean-k-mooney | not the two that are proposed. and i would prefer it to be a new one rather then messing with hw:cpu_policy | 13:37 |
| sean-k-mooney | spec freeze is wednesday right? | 13:37 |
| sean-k-mooney | or thrusday? next week in either case | 13:37 |
| stephenfin | I'd rather we used 'resources:VCPU' and 'resources:PCPU' but I don't recall why that wasn't chosen again | 13:38 |
| stephenfin | I need to look at it again | 13:38 |
| stephenfin | next week though! | 13:38 |
| sean-k-mooney | stephenfin: because we never want people ot use resocues: driectly and that does not provide a way to say which cpus are the pinned ones | 13:38 |
| stephenfin | That's the one | 13:39 |
| stephenfin | jaypipes will be turning over in his OpenStack grave though | 13:39 |
| efried | FWIW at every turn I'm finding I prefer using non-Placement-isms in the flavor, that get translated internally to Placement-isms. | 13:41 |
| efried | particularly since we have the request_filter framework now. | 13:42 |
| *** ttsiouts has quit IRC | 13:42 | |
| sean-k-mooney | the main disadvandate to useing the placment stuff direcly is if the modeling in placment chagnes you have to chagne your flavors | 13:42 |
| efried | that too | 13:42 |
| efried | another big one is that you sometimes don't know enough at flavor time to construct all the proper placement-isms. | 13:43 |
| sean-k-mooney | right | 13:43 |
| efried | This will become more and more true as we start splitting out NUMA, devices, bandwidth, etc. | 13:43 |
| sean-k-mooney | the image can enable pinning or numa | 13:43 |
| efried | yup | 13:43 |
| efried | we have pieces coming together from several places and there will need to be placement syntax tying them together (e.g. affinity) | 13:44 |
| efried | but you can't know that until schedule time when all the pieces are assembled. | 13:44 |
| efried | gibi: you still around? | 13:44 |
| *** cdent has joined #openstack-nova | 13:47 | |
| *** mchlumsky_ has quit IRC | 13:47 | |
| *** TheJulia is now known as needssleep | 13:48 | |
| *** mchlumsky has joined #openstack-nova | 13:49 | |
| sean-k-mooney | needssleep: i like the name :) i woke up today thinking it was thursday so i can agree with that statement | 13:50 |
| gibi | efried: yes | 13:51 |
| efried | gibi: o/ | 13:51 |
| efried | What further is needed to secure your +2 on the providers.yaml spec https://review.opendev.org/#/c/612497/ ? | 13:51 |
| *** igordc has joined #openstack-nova | 13:52 | |
| *** mlavalle has joined #openstack-nova | 13:52 | |
| gibi | efried: the '$COMPUTE_NODE' templating seems too much. If we simply allow to identify an RP by name then we can spare the complexity of the templating | 13:53 |
| sean-k-mooney | gibi: personally i would be fine with suppport both uuid and name | 13:53 |
| gibi | efried: if the rest of the team feels ok with the templating I won't stop it | 13:53 |
| sean-k-mooney | you would usee uuid if you want to reference non root providers, or create them in the future and name to reference the root provider or any other named one | 13:54 |
| efried | gibi: I added `name` back to the `identification` | 13:54 |
| sean-k-mooney | im not oppsed to the templating by the way just my 2cents | 13:55 |
| * sean-k-mooney time for coffee before meeting | 13:55 | |
| efried | gibi: and `$COMPUTE_NODE` allows us to hit ironic nodes, which (as dansmith convinced me yesterday) you wouldn't want to try to identify by hostname because the nodes managed by this compute can change when the hashring reshuffles. | 13:55 |
| gibi | efried: sorry I stuck with PS12. reading PS 12 now | 13:55 |
| gibi | I mean 13 | 13:56 |
| efried | cool, thanks. | 13:56 |
| openstackgerrit | **** proposed openstack/nova master: Nova: node should be deleted when last service is deleted https://review.opendev.org/671731 | 13:56 |
| sean-k-mooney | am ^ | 13:57 |
| sean-k-mooney | no ...? | 13:57 |
| dansmith | lol | 13:57 |
| sean-k-mooney | apperently there is a bluepirnt but the link does not work | 13:58 |
| gibi | efried: so we have more than one node under a compute service. the admin wants to configure some node specific thing via the yaml file. It works for a while but then the node is moved under another compute service where the yaml file has different config. How this allows the admin to specify config for a single ironic node properly? | 13:58 |
| efried | gibi: If they want to have a config for "one specific ironic node no matter which compute service it's under" they could deploy the same config (a single file, that identifies that ironic node explicitly) to all compute hosts. | 13:59 |
| dansmith | gibi: yep, if the admin has different files on different nodes, then a rebalance will change inventory, regardless of if they use $COMPUTE_NODE or not.. that'll need to be a ..node: warning probably regardless | 13:59 |
| sean-k-mooney | dansmith: oh its just a bad commit title | 14:00 |
| dansmith | sean-k-mooney: apparently the submitter is a secret agent | 14:00 |
| dansmith | who cannot reveal their identity | 14:00 |
| gibi | so the templating only helps if the config in the yaml file is not node specific but compute service specific | 14:00 |
| sean-k-mooney | apparently so | 14:00 |
| sean-k-mooney | too bad they didnt update there commit email or username | 14:01 |
| dansmith | gibi: specific to all the nodes managed by that compute, but you have a good point: | 14:01 |
| gibi | but _all_ node changes so it is not well defined | 14:01 |
| dansmith | gibi: some people with different ironic node types would want to have a different selector but still apply one definition to multiple things | 14:01 |
| efried | gibi: Yes. Or if you know you want all of your ironic nodes in your deployment to have some common characteristic, like a trait. You could effect that by deploying one section with $COMPUTE_NODE to each host instead of all N to each compute host. | 14:01 |
| dansmith | the identification section presumably offers us the ability to identify nodes by means other than $COMPUTE_NODE or uuid, | 14:02 |
| efried | dansmith: as of PS13 it allows you to identify by name | 14:02 |
| efried | earlier revs had a whole panoply of identification mechanisms, but we trimmed it down to the bare bones. | 14:03 |
| dansmith | so that we can say things like "any node managed by this service with inventory of CUSTOM_IRONIC_GOLD would be matched | 14:03 |
| dansmith | efried: name isn't super helpful in this case, but ... yeah | 14:03 |
| dansmith | and that's good for early revs I think | 14:03 |
| gibi | OK, I think I see the use case of the templating now. | 14:03 |
| efried | but yes, you make a good case for future expansion of the identification section. | 14:03 |
| dansmith | I'm saying that's a good reason for having identification be a section and not a key | 14:03 |
| dansmith | right | 14:03 |
| *** belmoreira has quit IRC | 14:04 | |
| gibi | we might need a warning for ironic admins about using the yaml for node specific config | 14:04 |
| dansmith | like I said, a ..note: somewhere for sure | 14:04 |
| gibi | OK | 14:04 |
| gibi | let me read PS13 through, but I think I'm +2 on the spec | 14:05 |
| *** Luzi has quit IRC | 14:05 | |
| efried | Thanks gibi | 14:05 |
| *** luksky11 has quit IRC | 14:06 | |
| *** JamesBenson has joined #openstack-nova | 14:08 | |
| *** damien_r has quit IRC | 14:09 | |
| artom | dansmith, dunno if you saw efried's adding of yourself to https://review.opendev.org/#/c/671471/, could I pester you some more for it? | 14:10 |
| gibi | efried: does changing 'ensure' to 'additional' keeps the intended behavior idepmotent? I mean if I restart a compute serivce it won't try to re-create the inventory if already exists | 14:11 |
| dansmith | artom: I did not | 14:11 |
| dansmith | gibi: it's still idempotent language to me | 14:11 |
| efried | gibi: yes, it's just a bikeshed of the word being used. I think I explained the rationale in the comment on PS12 | 14:11 |
| efried | gibi: it's more than just "restart a compute service" -- this thing runs on every periodic | 14:12 |
| gibi | dansmith: I more after the behavior. if that is idepmontent then I'm happy | 14:12 |
| gibi | efried: OK. then I'm happy :) | 14:12 |
| dansmith | gibi: same behavior, more descriptive (IMHO) wording | 14:12 |
| *** JamesBenson has quit IRC | 14:12 | |
| efried | It makes me happy to make you happy, gibi | 14:12 |
| gibi | dansmith: OK. I accept the language change blindly as I'm not a native speakre | 14:13 |
| dansmith | heh | 14:13 |
| gibi | ... and try to learn in the process | 14:13 |
| efried | don't try to learn too much from s/ensure/additional/ | 14:13 |
| efried | that way lies madness | 14:13 |
| dansmith | efried: stop mocking me | 14:14 |
| efried | you can learn things about Dan, but not about English | 14:14 |
| efried | jinx | 14:14 |
| * dansmith scowls | 14:14 | |
| efried | kisses | 14:14 |
| gibi | :) | 14:14 |
| *** spatel has joined #openstack-nova | 14:15 | |
| gibi | I'm +2 on the provider config spec | 14:19 |
| *** igordc has quit IRC | 14:20 | |
| gibi | and I guess it is good time to inform you that I will disappeare for two weeks of vacation starting in an hour | 14:20 |
| efried | phew, then I'm glad I nabbed you for that +2. Have a great vacation gibi! | 14:22 |
| gibi | efried: thanks! | 14:23 |
| cdent | enjoy your time off gibi | 14:28 |
| gibi | cdent: thanks | 14:28 |
| dansmith | artom: you have to fix your many typos anyway, so I think you should move the code to a single check to LOG.warning on startup and then I'd be +2 if not for needing some diversity on it | 14:31 |
| artom | dansmith, in _init_host? | 14:32 |
| dansmith | that'd be startup yeah :) | 14:32 |
| artom | Yeah, that'd make sense | 14:32 |
| sean-k-mooney | if you put it in _init_host i would perfonally make it stop the agent booting | 14:32 |
| dansmith | no | 14:32 |
| artom | It wouldn't stop it | 14:33 |
| artom | But... it might take a long time | 14:33 |
| dansmith | this should be a LOG.warning | 14:33 |
| sean-k-mooney | im fine with it just being a warning to | 14:33 |
| artom | Not sure if that's acceptable | 14:33 |
| dansmith | artom: also a reno, just in case anyone is relying on this to do their homework for them | 14:33 |
| dansmith | artom: how long does this take? I'm not sure why it'd be slow really | 14:34 |
| artom | The bug's env has > 1000 nics, they timed it to over 2 minutes | 14:34 |
| *** dpawlik has quit IRC | 14:34 | |
| artom | ========== | 14:34 |
| artom | $ time python -c "import nova.compute.utils; nova.compute.utils.get_machine_ips()" | 14:34 |
| artom | real 2m32.642s | 14:34 |
| artom | user 0m4.358s | 14:34 |
| artom | sys 2m27.918s | 14:34 |
| artom | ========== | 14:34 |
| efried | wow | 14:34 |
| dansmith | which thing takes long? listing the nics themselves? | 14:35 |
| dansmith | because that really shouldn't take that long | 14:35 |
| efried | yeah, is there not a quicker way to find out if my IP is on the system? | 14:35 |
| efried | ifconfig -a | grep ?? | 14:35 |
| artom | efried, could be that "dumb", yeah. Would we be cool with this? | 14:35 |
| dansmith | if it's the list, then not much can be done, but if it's our processing, we could limit to 100 nics and bail after, or something | 14:36 |
| artom | ifconfig isn't on every system nowadays, but `ip` is | 14:36 |
| efried | considering what we're shooting for here, I would be totally fine with something that "dumb". | 14:36 |
| dansmith | no, don't do ip|grep | 14:36 |
| efried | if we were relying on this for real functionality that would be one thing | 14:36 |
| dansmith | this has been in place for four years and one person has complained, | 14:36 |
| efried | but this is a best-effort sanity check | 14:36 |
| dansmith | so I think once at startup which is way less impactful should be fine | 14:36 |
| artom | dansmith, well, thousands of nics aren't exactly common :) | 14:36 |
| dansmith | right | 14:36 |
| dansmith | which is why it's not a problem to do this for the 99.999% I think | 14:37 |
| efried | and as dansmith noted in the original, it's possible for it to fail and everything still be alright. | 14:37 |
| artom | I'm just worried there might be something that implicitly relies on compute service starting up in less than a few seconds | 14:37 |
| dansmith | artom: we have tons of things that are long-running in compute startup, | 14:37 |
| dansmith | so that's not a concern | 14:37 |
| dansmith | we block compute startup until we can talk to conductor in both directions, we clean up failed migrations, do disk IO, etc, etc | 14:37 |
| artom | dansmith, ack, sounds like we have a way forward with moving the warning to _init_host then | 14:38 |
| artom | (dansmith, btw, I suspect it's https://github.com/openstack/nova/blob/master/nova/compute/utils.py#L977 that's the time sink, not https://github.com/openstack/nova/blob/master/nova/compute/utils.py#L975, but that's moot, and unless we ask the reporter we'll never know) | 14:42 |
| dansmith | artom: yeah I just don't know _why_ it'd be slow, unless it's trying to reverse lookup all the addresses too | 14:42 |
| artom | Yeah, good question... | 14:43 |
| openstackgerrit | Eric Fried proposed openstack/nova-specs master: Followup for provider config file spec https://review.opendev.org/671749 | 14:43 |
| artom | If it was doing reverse lookups I think it'd be slower, because a single loopkup takes forever to timeout | 14:43 |
| efried | dansmith: Put the ironic NOTE in a fup so as not to lose gibi's +2 ^ | 14:43 |
| dansmith | artom: that call is implemented in C even | 14:44 |
| *** dpawlik has joined #openstack-nova | 14:46 | |
| *** luksky11 has joined #openstack-nova | 14:46 | |
| *** belmoreira has joined #openstack-nova | 14:46 | |
| alex_xu | stephenfin: sean-k-mooney thanks in person | 14:47 |
| alex_xu | sean-k-mooney: buyin your proposal about 1 extraspec and mask | 14:47 |
| *** dpawlik has quit IRC | 14:51 | |
| alex_xu | artom: ah...claim depends on migrate_data, just aware that, I need to dig into your patch more | 14:52 |
| artom | alex_xu, hey, if you find a way to make it work, I'm all for it :) | 14:52 |
| alex_xu | at least understand the dependence...and I pretty sure people will hate that I added a interface call get_vpmems :) | 14:53 |
| artom | alex_xu, yeah, that'd make 2 libvirt-specific interfaces that no other driver uses | 14:55 |
| alex_xu | yea | 14:55 |
| artom | Maybe consolidate them into something like "get_driver_specific_migrate_data"? | 14:56 |
| artom | Not super happy with that either | 14:56 |
| efried | fwiw, I think we need *something* like this ^ | 14:57 |
| efried | although... I thought we already *had* driver-specific migrate data | 14:58 |
| artom | Exactly | 14:58 |
| artom | It's more like... claims-dependant driver-specific | 14:58 |
| artom | Which is a mouthful | 14:58 |
| artom | Or... post-claim migrate_data | 14:59 |
| artom | Would that work? | 14:59 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Use a less chipper title for release notes https://review.opendev.org/671752 | 15:00 |
| *** JamesBenson has joined #openstack-nova | 15:02 | |
| artom | driver.migrate_data_from_claim(context, instance, claim) | 15:02 |
| artom | efried, alex_xu ^^ how does that sound? | 15:02 |
| *** ricolin_ has joined #openstack-nova | 15:03 | |
| *** belmoreira has quit IRC | 15:03 | |
| efried | artom: What are these LiveMigrateData subclasses for, if not for this? | 15:03 |
| dansmith | I'm not following this, | 15:04 |
| *** ricolin has quit IRC | 15:04 | |
| dansmith | but we do have per-driver migrate data for exactly this reason | 15:04 |
| artom | efried, it's a sequencing problem | 15:04 |
| dansmith | and that's what those subclasses are | 15:04 |
| artom | See https://review.opendev.org/#/c/634606/44/nova/compute/manager.py@6482 | 15:04 |
| artom | It's not a migrate_data class problem, it's "we need to call the driver twice, once for check_can_live_migrate_destination, and once after we've done claims" | 15:04 |
| artom | In my patch, that second call is very use-case specific "get_dst_numa_config" | 15:05 |
| sean-k-mooney | for what its woth we do something similar for sriov | 15:05 |
| artom | alex_xu will need to solve the same problem, so rather than adding "get_vpmem_config" or whatever to the driver interface, we're trying to come up with a more generic single method for that second call to driver | 15:05 |
| alex_xu | I'm trying to figure out why you need the migration_data for the claim | 15:06 |
| artom | Other way around, we need to claim to update the migrate data | 15:06 |
| artom | Because we get the NUMA topology on the dest host from the claim | 15:06 |
| *** gibi is now known as gibi_off | 15:07 | |
| alex_xu | artom: looks like you only want to know whether instance has numa topo https://review.opendev.org/#/c/634606/44/nova/compute/manager.py@6470 | 15:09 |
| sean-k-mooney | for sriov migration we added https://review.opendev.org/#/c/620115/35/nova/compute/resource_tracker.py which we callin in https://review.opendev.org/#/c/620115/35/nova/conductor/tasks/live_migrate.py | 15:10 |
| sean-k-mooney | * in check_can_live_migrate_destination | 15:10 |
| *** belmoreira has joined #openstack-nova | 15:11 | |
| artom | alex_xu, in my case, yeah | 15:11 |
| alex_xu | artom: I don't understand this https://review.opendev.org/#/c/634606/44/nova/virt/libvirt/driver.py@7463 | 15:12 |
| artom | alex_xu, it's to handle the N/N-1 problem | 15:13 |
| alex_xu | what different with we check instance.numa_topology at dest host compute manager | 15:13 |
| alex_xu | oh, upgrade issue | 15:13 |
| artom | Exactly | 15:13 |
| sean-k-mooney | alex_xu old host wont populate that field | 15:13 |
| artom | But you're right, once we get to U or V or whenever, that's no longer needed | 15:13 |
| openstackgerrit | **** proposed openstack/nova master: Nova: node should be deleted when last service is deleted https://review.opendev.org/671731 | 15:13 |
| sean-k-mooney | well actuly ignore that | 15:13 |
| sean-k-mooney | they wont populate it but we need it for other reasons | 15:14 |
| alex_xu | do we need that LM to be fixed in the middle of S to T upgrade | 15:14 |
| alex_xu | if not, then we can just enable the numa LM by the rpc version | 15:15 |
| artom | Are you asking whether we need to support S -> T NUMA LM? | 15:15 |
| alex_xu | we can only support that LM when all the nodes upgrade to T | 15:15 |
| sean-k-mooney | alex_xu: we need to assume that not all nodes will be upgrade at teh same time | 15:15 |
| artom | We do, it's apparently a big thing during upgrades, operators migrate workloads off compute nodes being upgraded | 15:15 |
| sean-k-mooney | alex_xu: that was discussed at lenght in the spec | 15:16 |
| artom | Or if you're asking "why not check min service version" to determine whether we can NUMA LM... | 15:16 |
| sean-k-mooney | alex_xu: https://specs.openstack.org/openstack/nova-specs/specs/train/approved/numa-aware-live-migration.html#upgrade-impact | 15:16 |
| artom | Yeah, we could do that, and at point it's what was done in the code, but I think dansmith came up with this way, which personally I find more elegant | 15:16 |
| alex_xu | the min service version is also require all the nodes upgrade to T, right? | 15:17 |
| artom | Yeah | 15:17 |
| artom | Although perhaps, in the interest of not adding driver interface methods, we could go back to checking min service version, and then we could do the claim before driver.check_can_live_migrate_at_destinaton | 15:18 |
| artom | (I think) | 15:18 |
| sean-k-mooney | we should not do the claim before that | 15:18 |
| sean-k-mooney | driver.check_can_live_migrate_at_destinaton is where the claim should be done | 15:19 |
| alex_xu | that check means, only the source node is new version, then we can do LM? | 15:19 |
| sean-k-mooney | alex_xu: we can live migrate but we wont update the guest xml | 15:19 |
| sean-k-mooney | so it will be jsut as broken as before | 15:20 |
| sean-k-mooney | but it will live migrate succesfully | 15:20 |
| alex_xu | sean-k-mooney: so in upgrade, we need to LM from the old to new, then LM from new back to old | 15:21 |
| sean-k-mooney | you dont need too | 15:21 |
| sean-k-mooney | we just wont break that | 15:21 |
| sean-k-mooney | the sriov migration feature is coded such that it falls back to the old behavior if the info is not available to use teh new flow | 15:22 |
| sean-k-mooney | the new flow is basically only used when both are new enough but its explained in the table in the spec | 15:22 |
| alex_xu | but I still can't migrate the instance from old to new...that break the first step of upgrade | 15:23 |
| sean-k-mooney | you can | 15:23 |
| alex_xu | you said the xml won't update | 15:23 |
| sean-k-mooney | correct | 15:24 |
| sean-k-mooney | it use the old live migration workflow | 15:24 |
| artom | It's impossible to update the XML unless both source and dest support it | 15:24 |
| artom | So we do best effort and revert to previous half-broken behaviour | 15:24 |
| alex_xu | the instance won't be break? | 15:24 |
| artom | It might :) | 15:24 |
| artom | That's why it's currently half-broken | 15:25 |
| sean-k-mooney | they havent been broken for the last 10 releases | 15:25 |
| alex_xu | hah | 15:25 |
| sean-k-mooney | what might happen is two vms get pinned to the same cores | 15:25 |
| sean-k-mooney | but that could happen before. | 15:25 |
| artom | It might actually break if it's pinned to CPUs that don't exist on the dest | 15:25 |
| sean-k-mooney | if the cores do not exist or there is not enough hugepages libvirt fails the migrtion due to the qemu error | 15:26 |
| alex_xu | yea, so...we should ask the user to do that | 15:26 |
| sean-k-mooney | and it auto reverts | 15:26 |
| alex_xu | s/should/shouldn't/ | 15:26 |
| sean-k-mooney | that is the same behavior as today | 15:26 |
| alex_xu | I remember we have a patch stop that LM api call by checking the instance has numa | 15:27 |
| sean-k-mooney | yep we merged that | 15:27 |
| artom | Yeah, it's now disabled by default | 15:27 |
| sean-k-mooney | so to live migrarte form S->T they need to re enabeld it | 15:28 |
| sean-k-mooney | although i think artom is removing that option in T? | 15:28 |
| sean-k-mooney | or is that puhsed to U | 15:28 |
| alex_xu | so...we will tell the user, now you can LM your instance again when you upgrade from S to T, but you instance may gone based on lucky :) | 15:28 |
| artom | sean-k-mooney, it's being deprecated but not removed, because of the upgrade impact | 15:28 |
| artom | We don't want to turn it on by default unless we're sure the whole cloud supports it | 15:28 |
| sean-k-mooney | alex_xu: the important thing is we do not regres or change any live migration behavior with artom code | 15:29 |
| alex_xu | sean-k-mooney: I agree with that...but doesn't mean we should encourge the user to do that | 15:30 |
| *** zbr is now known as zbr|out | 15:30 | |
| alex_xu | special for danger thing | 15:30 |
| sean-k-mooney | we are not encuragin people to do it | 15:30 |
| sean-k-mooney | if you have numa instance you should cold migrate them or do an inplace upgrade | 15:30 |
| sean-k-mooney | that is what we will be telling customers | 15:30 |
| *** luksky11 has quit IRC | 15:30 | |
| alex_xu | yes, so we needn't to support src to dest numa instance LM :) | 15:31 |
| alex_xu | for s to t | 15:31 |
| sean-k-mooney | alex_xu: orginally we didnt | 15:31 |
| sean-k-mooney | but both i belive dansmith and mriderman wanted the spec changed to the current form | 15:32 |
| artom | (Oh man, M. Riderman) | 15:32 |
| artom | I am lolling right now | 15:32 |
| * alex_xu not dare to question on dansmith and mriderman :) | 15:33 | |
| sean-k-mooney | actully it look like dansmith didnt review that spec but i know we spoke to him about it | 15:34 |
| sean-k-mooney | anyway for sriov live migration since it never worked before we have a hard check in the conductor | 15:35 |
| sean-k-mooney | and fail if both nodes are not knew enough | 15:35 |
| sean-k-mooney | since live migration wiht hugepages is a thing we support for ovs-dpdk we did not want to break it by blocking it | 15:36 |
| openstackgerrit | Merged openstack/nova master: Bump the openstackdocstheme extension to 1.20 https://review.opendev.org/671694 | 15:36 |
| alex_xu | I guess the only reason, the user know they have same type machine, and the instances won't break with pinning to same pcpu | 15:37 |
| sean-k-mooney | alex_xu: uncontidionally blocking old to new migration would be a regression. | 15:37 |
| alex_xu | ah.. | 15:38 |
| sean-k-mooney | well if the instance is not pinned but just has hugepages we dont want to break that | 15:38 |
| alex_xu | I see the reason now | 15:38 |
| artom | alex_xu, yeah, I think that's explicitly called out somewhere: NUMA LM is only "guaranteed" to work if you're migrating to an "empty" identical machine, identically configured, | 15:38 |
| alex_xu | got it | 15:39 |
| sean-k-mooney | in the hugepage case if there are not enoguht free spawning the qemu instace fail on the dest and we abort the migration before we even pause the vm on the source | 15:40 |
| sean-k-mooney | the same is true if the cores dont exist on the dest | 15:41 |
| sean-k-mooney | so the only issue that is not caght by that is two vms being pinned to the same host | 15:41 |
| sean-k-mooney | * core | 15:41 |
| alex_xu | can we check the src and dst service version in the conductor? | 15:41 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: libvirt: move checking CONF.my_ip to init_host() https://review.opendev.org/671471 | 15:42 |
| sean-k-mooney | we can but why would we | 15:42 |
| alex_xu | emm...I guess we need another param for the rpc | 15:42 |
| artom | alex_xu, that's what I had at some point, Dan talked me out of it | 15:42 |
| sean-k-mooney | we need the data that is pass back from can live migrate on dest | 15:42 |
| artom | dansmith, efried ^^ (the CONF.my_ip to init_host() thing) | 15:42 |
| sean-k-mooney | to be able to generate the updated xml | 15:43 |
| efried | sean-k-mooney: Is https://review.opendev.org/#/c/638680/ still waiting on https://review.opendev.org/#/c/670189/ ? | 15:43 |
| *** JamesBenson has quit IRC | 15:43 | |
| sean-k-mooney | kind of. im not changeing the public interface of the function it calls | 15:43 |
| sean-k-mooney | but i suspect that if i chekcout that patch locally it will still expode | 15:44 |
| sean-k-mooney | i can check that quickly. | 15:44 |
| efried | artom: would you mind adding a teeny unit test that asserts _check_my_ip is called during init_host? | 15:46 |
| *** JamesBenson has joined #openstack-nova | 15:48 | |
| artom | efried, ack, makes sense | 15:48 |
| efried | artom: I recognize that a unit test for init_host as a whole is missing, and would be an appropriate add, but unrelated... | 15:49 |
| *** pcaruana has quit IRC | 15:49 | |
| artom | efried, hrmm, yeah, went looking, didn't find anything | 15:49 |
| efried | yeah | 15:50 |
| artom | So, maybe as a separate fup patch on top? | 15:50 |
| *** rpittau is now known as rpittau|afk | 15:50 | |
| artom | The full unit test, I mean | 15:50 |
| efried | sure, that would be great, though I would also just not notice if it never happened | 15:50 |
| efried | There are lots of gaps in unit test in libvirt/test_driver | 15:50 |
| efried | lots | 15:50 |
| artom | My thinking it, given how much I'll have to mock to just assert this one thing being called, might as well do the full thing | 15:51 |
| artom | *is | 15:51 |
| sean-k-mooney | efried: its not failing because my kernel does not support sev. so https://review.opendev.org/#/c/638680/27/nova/virt/libvirt/host.py@1066 prevent the call to get_domain_capablities | 15:51 |
| efried | artom: There are other tests that call init_host but don't mock a zillion things. I didn't look too deeply into how they're doing that, but... | 15:52 |
| efried | probably because of FakeLibvirtDriver | 15:53 |
| artom | Ah, indeed | 15:53 |
| efried | naw, even the ones that use real LibvirtDriver with FakeVirtAPI | 15:53 |
| efried | so, it should be possible to do it without too much mocking. | 15:53 |
| sean-k-mooney | if i comment out the if i get this trace back | 15:54 |
| sean-k-mooney | http://paste.openstack.org/show/754594/ | 15:54 |
| efried | though that leads me to wonder how you're getting away with not mocking netifaces.interfaces() and .ifaddresses() | 15:55 |
| artom | get_machines_ips gets mocked out entirely | 15:55 |
| efried | artom: Is your test actually running netifaces.interfaces() on whatever system you're invoking it on?? | 15:55 |
| efried | yeah, I mean all the other tests that invoke init_host() :) | 15:55 |
| *** jangutter has quit IRC | 15:55 | |
| artom | Oh, hah | 15:56 |
| artom | I guess they do? | 15:56 |
| efried | that's... not great | 15:56 |
| efried | unpredictable results on linux, sure failure on other platforms, nah? | 15:56 |
| artom | It's supposed to be cross-platform | 15:56 |
| artom | But yeah | 15:56 |
| efried | Not that I would really expect test_driver to work on another platform anyway, but... | 15:56 |
| sean-k-mooney | artom: it should not be actully calling it however | 15:57 |
| efried | so perhaps you need to fixture out get_machine_ips (or netifaces.interfaces) at a higher level | 15:57 |
| *** helenafm has quit IRC | 15:57 | |
| sean-k-mooney | you shoudl be able to set up a mock in the setup function once | 15:57 |
| artom | Yep | 15:57 |
| artom | ;_: | 15:57 |
| artom | And here I was thinking I'd get to do a pure removal patch :( | 15:58 |
| efried | sorry artom. Your karma ran over your dogma. | 15:58 |
| *** gyee has joined #openstack-nova | 15:58 | |
| artom | Is that like my kar running over my dog, just with my ma involved? | 15:58 |
| efried | just so | 16:00 |
| *** ag-47 has quit IRC | 16:02 | |
| efried | sean-k-mooney: I may need some help, or you should bring in a SME, reviewing that get_dowain_capabilities patch. kashyap? | 16:03 |
| sean-k-mooney | sure. i could harden it more then i did but that at least fixes the orignial issue | 16:05 |
| sean-k-mooney | im still not catching the libvirt exection if we ligitetly misconfigur the config | 16:05 |
| sean-k-mooney | but we wont explode any more with the defaults | 16:07 |
| *** belmoreira has quit IRC | 16:07 | |
| sean-k-mooney | efried: do you know if aspires is still working on openstack? | 16:07 |
| sean-k-mooney | i have not seen them involded in the sev work lately | 16:08 |
| efried | I haven't seen him in quite a while, but I hope so. | 16:08 |
| efried | yeah | 16:08 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: libvirt: move checking CONF.my_ip to init_host() https://review.opendev.org/671471 | 16:08 |
| sean-k-mooney | maybe i should add Boris Bobrov to the review too since they seam to have taken over the sev stuff | 16:08 |
| *** bnemec is now known as beekneemech | 16:09 | |
| *** kmalloc is now known as needscoffee | 16:11 | |
| efried | artom: That rev wfm. Just one other thing: do functional tests have the possibility of hitting netifaces now? | 16:11 |
| artom | OK, this isn't a rabbit whole anywhere, it's the pit of hell | 16:12 |
| artom | *hole | 16:12 |
| artom | efried, but yeah, they do, good thought | 16:12 |
| artom | Maybe right up in nova/test.py? | 16:12 |
| efried | artom: do any func tests use the real libvirt driver? | 16:13 |
| efried | if they use fake, this is n/a | 16:13 |
| efried | no, I don't want a libvirt-specific fixture in nova/test.py | 16:13 |
| artom | They use the fake libvirt API, but real driver IIRC, no? | 16:13 |
| efried | (don't tell me if there are already libvirt-isms dirtying up the base class) | 16:13 |
| efried | artom: I don't know, I'm still pretty green on all this libvirt stuff. | 16:14 |
| sean-k-mooney | artom: i dont think so | 16:14 |
| sean-k-mooney | the unit test in generall dont use the drivers at all | 16:14 |
| sean-k-mooney | the functional test defualt to the fake dirver | 16:14 |
| sean-k-mooney | execpt the function test under the libvirt folder | 16:14 |
| sean-k-mooney | which do as you said | 16:15 |
| sean-k-mooney | e.g. fake libvirt api + real libvirt driver | 16:15 |
| sean-k-mooney | you could add a test in here somewhare https://github.com/openstack/nova/tree/master/nova/tests/functional/libvirt | 16:16 |
| artom | efried, well, wait a second, we had that before, no? | 16:16 |
| efried | had what before? | 16:16 |
| artom | Any migration test that didn't mock out get_host_ip_addr | 16:16 |
| artom | It ended up calling down to netifaces eventually | 16:16 |
| efried | seems likely, yeah | 16:16 |
| sean-k-mooney | we only have 2 migration tests i think | 16:16 |
| artom | Same for functional tests | 16:16 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/tests/functional/compute/test_live_migration.py | 16:17 |
| artom | I don't think live migration hit that | 16:18 |
| efried | let's settle this... | 16:18 |
| efried | I'm going to write a DNM with a poison fixture that will blow up any test that hits netifaces.interfaces() | 16:18 |
| efried | and let the gate tell me which tests are affected. | 16:18 |
| efried | which is almost, but not quite, unrelated to your patch. | 16:19 |
| *** ricolin_ has quit IRC | 16:19 | |
| efried | since you've expanded the number of code paths in which it could be getting called | 16:19 |
| artom | I'd be OK waiting for that to report before proceeding | 16:19 |
| artom | Even rebasing on top | 16:19 |
| efried | Well | 16:19 |
| efried | I'm not sure I want to *keep* the poison fixture | 16:19 |
| efried | although I guess that wouldn't be an awful thing. | 16:20 |
| artom | Well, we're either OK with listing the host interfaces, or we're not :) | 16:20 |
| artom | If we're not, which seems the case, don't even bother DNM'ing it | 16:20 |
| efried | yeah, that's a fair point. | 16:20 |
| efried | okay, stand by | 16:20 |
| * artom uses this pause to get lunch | 16:22 | |
| *** jmlowe has quit IRC | 16:24 | |
| *** _hemna has joined #openstack-nova | 16:27 | |
| *** beekneemech has quit IRC | 16:30 | |
| *** cdent has quit IRC | 16:30 | |
| *** bnemec has joined #openstack-nova | 16:35 | |
| *** mdbooth_ has joined #openstack-nova | 16:36 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Poison netifaces.interfaces() in tests https://review.opendev.org/671773 | 16:38 |
| efried | artom: ^ | 16:38 |
| *** mdbooth has quit IRC | 16:39 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: libvirt: move checking CONF.my_ip to init_host() https://review.opendev.org/671471 | 16:40 |
| efried | artom: rebased yours on top ^ | 16:40 |
| efried | we'll see if either/both explode :) | 16:40 |
| openstackgerrit | sean mooney proposed openstack/nova master: Libvirt: add support for vPMU configuration. https://review.opendev.org/671338 | 16:40 |
| sean-k-mooney | gibi_off: ill fix the failing tests in ^ on monday so you can ignore it for now | 16:41 |
| sean-k-mooney | GenericPoisonFixture i like the name :) | 16:43 |
| artom | ericyoung, ack, thanks. Don't we already have a DB poison fixture somewhere? If you're making a generic one, should it gobble that up? | 16:46 |
| artom | Whoops, efried ^^ | 16:46 |
| efried | artom: There are two other poison fixtures that I could conceivably roll into this one, but their messages are much more specific and they do more specific things, so I left them alone for now. | 16:46 |
| artom | *snerk* I understand you mean method | 16:49 |
| artom | But... maybe not have the name of a hard drug in our code? ;) | 16:49 |
| efried | artom: Heh, yeah, I just didn't want to shadow variable names. It makes my IDE mildly upset, and confuses me. | 16:52 |
| artom | func | 16:53 |
| artom | ? | 16:53 |
| efried | I should point out http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-07-15.log.html#t2019-07-15T20:49:08 | 16:53 |
| efried | (You thought nobody was listening) | 16:54 |
| artom | efried, what I looking at? 1836595? | 16:54 |
| artom | Oh, yeah, thank you zhufl ^_^ | 16:54 |
| *** belmoreira has joined #openstack-nova | 16:55 | |
| efried | dansmith: will you be able to get back to (and hopefully close) https://review.opendev.org/612497 (providers.yaml) today? | 16:56 |
| dansmith | hopefully | 16:56 |
| *** mdbooth_ has quit IRC | 16:58 | |
| *** belmoreira has quit IRC | 16:58 | |
| *** _hemna has quit IRC | 16:59 | |
| *** dustinc_OSCON is now known as dustinc | 17:06 | |
| *** derekh has quit IRC | 17:10 | |
| *** igordc has joined #openstack-nova | 17:10 | |
| *** tbachman has joined #openstack-nova | 17:22 | |
| *** tesseract has quit IRC | 17:24 | |
| *** jmlowe has joined #openstack-nova | 17:32 | |
| *** maciejjozefczyk has quit IRC | 17:33 | |
| *** ralonsoh has quit IRC | 17:34 | |
| *** whoami-rajat has quit IRC | 17:43 | |
| *** udesale has quit IRC | 17:48 | |
| dansmith | efried: you wanna fix at least my typo nit and I'll +2+W? No long test run required for these, so a quick bump for a typo seems fine | 17:51 |
| efried | dansmith: okay, you want me to smash in the fup while I'm at it? | 17:52 |
| dansmith | oh I didn't see the fup, yeah might as well and gibi is +2 there, so ... easy | 17:54 |
| efried | dansmith: Do you want me to say "exactly one" identification mechanism, or is "at least one" good? | 17:55 |
| dansmith | efried: the point of my comment is "exactly one" is important.. if you say "at least one" you need to define what happens if they put name and uuid and they don't match | 17:56 |
| efried | right, that would be an error. | 17:56 |
| efried | "exactly one" would restrict us in the future if we wanted to do some kind of pattern matching | 17:57 |
| efried | name_matches: /pattern/ | 17:57 |
| efried | has_trait: CUSTOM_FOO | 17:57 |
| efried | kind of thing | 17:57 |
| *** xXraphXx has joined #openstack-nova | 17:57 | |
| efried | but I'm fine either way. | 17:57 |
| dansmith | sure, but as it is today, I would think "exactly one" gets you a crisper definition.. then when you add a match you can say "zero or one of name/uuid and one or more filters" or something like that.. but I don't care whether you go loose or tight, just define the edges if you go loose | 17:58 |
| dansmith | if you want to put this in your fup for bikeshedding I can just +W the bottom one | 17:59 |
| efried | dansmith: so tbc you're okay if I leave it defined as is but add "error if you specify both and they don't match"? | 17:59 |
| dansmith | I think "at least one" is better than "either" because I'm not sure whether either means "one but not both", but yes, if you want "at least one" and define the edges, that's fine | 18:00 |
| dansmith | but, what happens if I do $COMPUTE_NODE and name and there's no match? Is that an error because I won't ever do anything, or is that a 'well no match but that's what you asked for' ? | 18:01 |
| dansmith | and, that might be right at startup, but not after a reshuffle, | 18:02 |
| dansmith | and in the zero hour case with ironic, the compute starts up with no nodes at first | 18:02 |
| dansmith | not trying to complicate this, which is why I thought "exactly one" makes this easier for the moment :) | 18:02 |
| efried | okay okay | 18:02 |
| *** bnemec is now known as beekneemech | 18:04 | |
| openstackgerrit | Eric Fried proposed openstack/nova-specs master: Spec: Provider config YAML file https://review.opendev.org/612497 | 18:04 |
| efried | dansmith: smashed and done ^ | 18:05 |
| * dansmith dusts hands | 18:06 | |
| efried | \o/ | 18:06 |
| efried | thanks dansmith | 18:06 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove legacy '_from_dict' functions https://review.opendev.org/537414 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Rename 'nova.objects.instance_numa_topology' https://review.opendev.org/671789 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Remove unnecessary try-catch around 'getCPUMap' https://review.opendev.org/671790 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: claims: Remove useless caching https://review.opendev.org/671791 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Add '[compute] cpu_dedicated_set' option https://review.opendev.org/671792 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting PCPU inventory to placement https://review.opendev.org/671793 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting 'pcpus' to the resource tracker https://review.opendev.org/671794 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Rename exception argument https://review.opendev.org/671795 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Remove unused function parameter https://review.opendev.org/671796 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hardware.get_host_numa_usage_from_instance' https://review.opendev.org/671797 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hardware.host_topology_and_format_from_host' https://review.opendev.org/671798 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hardware.instance_topology_from_instance' https://review.opendev.org/671799 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800 | 18:17 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support translating CPU policy extra specs, image meta https://review.opendev.org/671801 | 18:17 |
| stephenfin | efried: FYI ^ | 18:17 |
| efried | nice. alex_xu ^ | 18:18 |
| stephenfin | The second to last patch needs a lot more work to handle upgrades properly (tests are failing too) and I still need to do the reshape, but that's where my head's at | 18:18 |
| * stephenfin -> 🍻 | 18:19 | |
| *** bbowen has quit IRC | 18:24 | |
| efried | dansmith: Not sure if you're still following oslo.service SIGHUP, but you may be able to tell whether it's doing the right thing based on the logs I just posted https://review.opendev.org/#/c/641907/ | 18:24 |
| *** factor has joined #openstack-nova | 18:27 | |
| efried | artom: the results are in. So far your patch is going to wind up needing to undo the changes I'm making to the tests as a result. It may end up being more prudent to put the poison patch on top of yours, once yours finishes tests and we find out what *it* will have to change. | 18:30 |
| artom | efried, I don't mind handling any conflicts that come up | 18:33 |
| artom | efried, to put it bluntly, you handle yours, I'll handle mine :P | 18:34 |
| openstackgerrit | Merged openstack/nova-specs master: Spec: Provider config YAML file https://review.opendev.org/612497 | 18:34 |
| efried | There are only three failures: one func, two unit. All three will be n/a afaict under your patch. | 18:34 |
| efried | maybe I should just wait until yours comes back. | 18:34 |
| efried | dustinc: ----^ | 18:35 |
| artom | n/a afaict | 18:35 |
| artom | ? | 18:35 |
| efried | lmk when you want to start talking impl | 18:35 |
| dustinc | oh good! | 18:35 |
| dustinc | I am back in office today and want to get my freshly failing tests fixed on other, let's start Monday morning? | 18:36 |
| efried | artom: I mean all three of them come from invoking driver.get_host_ip_addr | 18:36 |
| efried | dustinc: sure | 18:36 |
| artom | efried, honestly, if it makes your life easier, drop my patch entirely | 18:36 |
| dustinc | efried: great, ttyt | 18:36 |
| efried | artom: I'm not looking for an easy life. I want danger. I want thrills. | 18:36 |
| artom | Have you tried insulting Putin? | 18:37 |
| artom | It's just... I won't know for sure what needs fixing until I have your full patch in front of me | 18:38 |
| efried | artom: Here's yours: | 18:38 |
| efried | http://logs.openstack.org/71/671471/5/check/openstack-tox-py27/0282bb0/testr_results.html.gz | 18:38 |
| efried | http://logs.openstack.org/71/671471/5/check/nova-tox-functional/318a334/testr_results.html.gz | 18:38 |
| artom | I dunno, your call | 18:39 |
| efried | the only overlap is the functional one, test_cold_migrate_with_physnet, which fails in yours because of init_host and in mine because of get_ip. The fix there might actually be the same. | 18:39 |
| efried | so | 18:39 |
| efried | let's do this | 18:39 |
| artom | Given that this is latent, we could indeed merge my fix first | 18:39 |
| efried | fix yours til it passes on top of mine | 18:39 |
| efried | then we'll swap them | 18:39 |
| efried | which should actually make mine pass. | 18:39 |
| efried | dig? | 18:40 |
| *** luksky11 has joined #openstack-nova | 18:41 | |
| artom | efried, right, I think I get you | 18:41 |
| efried | It's not exactly latent when your patch goes from 3 netifaces calls to 44 :) | 18:41 |
| artom | Yours breaks because of driver.get_host_ip_addr() | 18:41 |
| artom | Mine will fix that ^^ | 18:41 |
| efried | correct | 18:41 |
| artom | But it will break others things in the process | 18:41 |
| efried | correct | 18:41 |
| efried | until it doesn't. | 18:41 |
| artom | Yes, which is why I'm paid the big bucks, apparently | 18:42 |
| efried | gotta do something to afford your meth habit | 18:42 |
| artom | "Meth. Why?" is indeed a good anti-drugs slogan | 18:43 |
| *** jmlowe has quit IRC | 18:46 | |
| *** _hemna has joined #openstack-nova | 18:55 | |
| *** eharney has quit IRC | 18:57 | |
| *** psachin has quit IRC | 19:20 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: libvirt: move checking CONF.my_ip to init_host() https://review.opendev.org/671471 | 19:23 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: Poison netifaces.interfaces() in tests https://review.opendev.org/671773 | 19:23 |
| artom | efried, let's wait for Zuul, but I think I got them all | 19:24 |
| efried | artom: cool. did you reverse them as well? | 19:24 |
| artom | Aye | 19:24 |
| efried | okay. Thanks for sticking with this. | 19:24 |
| artom | Np :) | 19:24 |
| redkrieg | hello all, I'm trying to use rescue mode to install windows, but the official iso (which works in a computer when burned) says no bootable device found in rescue mode. an ubuntu iso works fine on the same host. has anyone else ever hit this issue? | 19:27 |
| *** _hemna has quit IRC | 19:29 | |
| *** efried1 has joined #openstack-nova | 19:33 | |
| *** efried has quit IRC | 19:33 | |
| *** efried1 is now known as efried | 19:33 | |
| *** jmlowe has joined #openstack-nova | 19:36 | |
| efried | redkrieg: That's... not exactly a nova issue, is it? | 19:38 |
| redkrieg | I don't know if it's something specific to rescue mode's way of mounting isos. I've been able to use the iso with libvirt in the past. it's fine, I'll find a more appropriate venue | 19:40 |
| efried | redkrieg: Sorry, yeah, #openstack is more appropriate for support. This channel is for dev. | 19:41 |
| *** eharney has joined #openstack-nova | 19:42 | |
| redkrieg | thanks, I'll ask there | 19:42 |
| artom | efried, hrmm, one thing I forgot to consider was the backport situation | 19:44 |
| artom | That poison patch isn't going back | 19:45 |
| efried | artom: that's fine, then, because you switched em. | 19:45 |
| efried | You don't rely on the poison patch | 19:45 |
| efried | and your swizzle of the mocks is appropriate to backport. | 19:45 |
| artom | Oh yeah | 19:45 |
| efried | and I'm not too worried about whether stable branches have other violations we'll be missing. | 19:45 |
| efried | though we can always backport the poison patch to find out, then abandon it. | 19:46 |
| efried | but it seems pretty unlikely. | 19:46 |
| artom | Yeah | 19:46 |
| artom | Thanks for your help in all this, btw :) | 19:46 |
| efried | for sure | 19:47 |
| *** betherly has joined #openstack-nova | 20:00 | |
| *** betherly has quit IRC | 20:05 | |
| *** bbowen has joined #openstack-nova | 20:11 | |
| *** ociuhandu has joined #openstack-nova | 20:28 | |
| *** brault has joined #openstack-nova | 20:33 | |
| *** ociuhandu has quit IRC | 20:50 | |
| *** ociuhandu has joined #openstack-nova | 20:51 | |
| *** ociuhandu has quit IRC | 20:53 | |
| *** ociuhandu_ has joined #openstack-nova | 20:53 | |
| *** ociuhandu_ has quit IRC | 20:54 | |
| *** ociuhandu has joined #openstack-nova | 20:54 | |
| *** ociuhandu has quit IRC | 20:55 | |
| *** ociuhandu has joined #openstack-nova | 20:55 | |
| *** ociuhandu has quit IRC | 20:56 | |
| *** ociuhandu_ has joined #openstack-nova | 20:56 | |
| *** betherly has joined #openstack-nova | 21:00 | |
| *** ociuhandu_ has quit IRC | 21:01 | |
| efried | artom: +2 | 21:02 |
| artom | efried, \o/ | 21:03 |
| efried | dansmith: we got this one all ready for you https://review.opendev.org/#/c/671471/ | 21:04 |
| *** betherly has quit IRC | 21:05 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Remove @safe_connect from _delete_provider https://review.opendev.org/671866 | 21:23 |
| *** luksky11 has quit IRC | 21:23 | |
| *** _hemna has joined #openstack-nova | 21:25 | |
| *** _erlon_ has quit IRC | 21:30 | |
| *** betherly has joined #openstack-nova | 21:31 | |
| *** betherly has quit IRC | 21:35 | |
| *** _hemna has quit IRC | 21:39 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Remove @safe_connect from _delete_provider https://review.opendev.org/671866 | 21:41 |
| *** brault has quit IRC | 21:44 | |
| *** JamesBenson has quit IRC | 22:01 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Disambiguate logs in delete_allocation_for_instance https://review.opendev.org/671869 | 22:03 |
| efried | o/ | 22:03 |
| *** spatel has quit IRC | 22:04 | |
| *** needssleep is now known as TheJulia | 22:11 | |
| *** eharney has quit IRC | 22:20 | |
| *** betherly has joined #openstack-nova | 22:22 | |
| *** betherly has quit IRC | 22:27 | |
| *** gyee has quit IRC | 22:34 | |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Use SDK for node.list https://review.opendev.org/656027 | 22:46 |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Use SDK for validating instance and node https://review.opendev.org/656028 | 22:46 |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Use SDK for setting instance id https://review.opendev.org/659690 | 22:46 |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Use SDK for add/remove instance info from node https://review.opendev.org/659691 | 22:46 |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Use SDK for getting network metadata from node https://review.opendev.org/670213 | 22:46 |
| *** betherly has joined #openstack-nova | 22:53 | |
| *** gyee has joined #openstack-nova | 22:57 | |
| *** betherly has quit IRC | 22:58 | |
| *** betherly has joined #openstack-nova | 23:14 | |
| *** betherly has quit IRC | 23:19 | |
| *** takamatsu has quit IRC | 23:31 | |
| *** betherly has joined #openstack-nova | 23:35 | |
| *** betherly has quit IRC | 23:39 | |
| *** mlavalle has quit IRC | 23:46 | |
| *** factor has quit IRC | 23:46 | |
| *** factor has joined #openstack-nova | 23:47 | |
| *** igordc has quit IRC | 23:48 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!