| *** tbachman has quit IRC | 00:05 | |
| *** hamzy has quit IRC | 00:18 | |
| *** hamzy has joined #openstack-nova | 00:19 | |
| *** r-daneel has quit IRC | 00:28 | |
| *** tbachman has joined #openstack-nova | 00:28 | |
| *** r-daneel has joined #openstack-nova | 00:28 | |
| *** moshele has joined #openstack-nova | 00:30 | |
| *** tbachman has quit IRC | 00:33 | |
| *** tbachman has joined #openstack-nova | 00:34 | |
| *** claudiub has quit IRC | 00:39 | |
| *** gyee has quit IRC | 00:44 | |
| *** imacdonn has quit IRC | 00:50 | |
| *** imacdonn has joined #openstack-nova | 00:50 | |
| *** ChanServ has quit IRC | 01:00 | |
| *** ChanServ has joined #openstack-nova | 01:03 | |
| *** barjavel.freenode.net sets mode: +o ChanServ | 01:03 | |
| *** mriedem_afk has quit IRC | 01:05 | |
| *** r-daneel has quit IRC | 01:05 | |
| *** itlinux has quit IRC | 01:07 | |
| *** itlinux has joined #openstack-nova | 01:07 | |
| *** moshele has quit IRC | 01:08 | |
| *** mrsoul has quit IRC | 01:23 | |
| *** Nel1x has joined #openstack-nova | 01:28 | |
| *** Dinesh_Bhor has joined #openstack-nova | 01:41 | |
| *** rmk has quit IRC | 01:46 | |
| *** raginbajin has quit IRC | 01:46 | |
| *** rmk has joined #openstack-nova | 01:47 | |
| *** mmedvede has quit IRC | 01:48 | |
| *** ericyoung has quit IRC | 01:48 | |
| *** raginbajin has joined #openstack-nova | 01:49 | |
| *** ericyoung has joined #openstack-nova | 01:49 | |
| *** masayukig has quit IRC | 01:49 | |
| *** dansmith has quit IRC | 01:49 | |
| *** Dinesh_Bhor has quit IRC | 01:50 | |
| *** dansmith has joined #openstack-nova | 01:51 | |
| *** mmedvede has joined #openstack-nova | 01:52 | |
| *** swamireddy has joined #openstack-nova | 01:56 | |
| *** Dinesh_Bhor has joined #openstack-nova | 01:58 | |
| *** lei-zh has joined #openstack-nova | 02:01 | |
| *** brinzhang has joined #openstack-nova | 02:03 | |
| *** threestrands has quit IRC | 02:08 | |
| *** hongbin has joined #openstack-nova | 02:21 | |
| *** gcb_ has quit IRC | 02:24 | |
| *** Nel1x has quit IRC | 02:28 | |
| *** imacdonn has quit IRC | 02:28 | |
| *** hamzy has quit IRC | 02:28 | |
| *** panda has quit IRC | 02:28 | |
| *** penick has quit IRC | 02:28 | |
| *** jdillaman has quit IRC | 02:28 | |
| *** nicolasbock has quit IRC | 02:28 | |
| *** jlvillal has quit IRC | 02:28 | |
| *** dims has quit IRC | 02:28 | |
| *** mtreinish has quit IRC | 02:28 | |
| *** skatsaounis has quit IRC | 02:28 | |
| *** ircuser-1 has quit IRC | 02:28 | |
| *** s1061123 has quit IRC | 02:28 | |
| *** mnaser has quit IRC | 02:28 | |
| *** TheJulia has quit IRC | 02:28 | |
| *** Kevin_Zheng has quit IRC | 02:28 | |
| *** burt has quit IRC | 02:28 | |
| *** mgariepy has quit IRC | 02:28 | |
| *** mdbooth has quit IRC | 02:28 | |
| *** egonzalez has quit IRC | 02:28 | |
| *** slaweq has quit IRC | 02:28 | |
| *** Miouge has quit IRC | 02:28 | |
| *** mordred has quit IRC | 02:28 | |
| *** rtjure has quit IRC | 02:28 | |
| *** Nel1x has joined #openstack-nova | 02:30 | |
| *** imacdonn has joined #openstack-nova | 02:30 | |
| *** hamzy has joined #openstack-nova | 02:30 | |
| *** panda has joined #openstack-nova | 02:30 | |
| *** penick has joined #openstack-nova | 02:30 | |
| *** jdillaman has joined #openstack-nova | 02:30 | |
| *** nicolasbock has joined #openstack-nova | 02:30 | |
| *** egonzalez has joined #openstack-nova | 02:30 | |
| *** mtreinish has joined #openstack-nova | 02:30 | |
| *** dims has joined #openstack-nova | 02:30 | |
| *** jlvillal has joined #openstack-nova | 02:30 | |
| *** skatsaounis has joined #openstack-nova | 02:30 | |
| *** ircuser-1 has joined #openstack-nova | 02:30 | |
| *** s1061123 has joined #openstack-nova | 02:30 | |
| *** Kevin_Zheng has joined #openstack-nova | 02:30 | |
| *** mnaser has joined #openstack-nova | 02:30 | |
| *** TheJulia has joined #openstack-nova | 02:30 | |
| *** burt has joined #openstack-nova | 02:30 | |
| *** mgariepy has joined #openstack-nova | 02:30 | |
| *** mdbooth has joined #openstack-nova | 02:30 | |
| *** slaweq has joined #openstack-nova | 02:30 | |
| *** Miouge has joined #openstack-nova | 02:30 | |
| *** mordred has joined #openstack-nova | 02:30 | |
| *** rtjure has joined #openstack-nova | 02:30 | |
| *** openstack has joined #openstack-nova | 13:21 | |
| *** ChanServ sets mode: +o openstack | 13:21 | |
| mnaser | we have plenty of capacity of both, but i have a proposed solution where it just defaults to requiring hardware virt but allowing user to control that value if need be | 13:21 |
|---|---|---|
| *** ttsiouts has joined #openstack-nova | 13:21 | |
| sean-k-mooney | mnaser: right but that is not something you can do via an image today | 13:21 |
| mnaser | sean-k-mooney: it is, using the `img_hv_type` property | 13:21 |
| sean-k-mooney | mnaser: that would be better handeled in a weigher | 13:21 |
| mnaser | if you set it to `kvm`, it'll filter only those that can do hardware acceleration | 13:22 |
| sean-k-mooney | mnaser: no that just allowas you to select the hypervior not the arch | 13:22 |
| mnaser | sean-k-mooney: combined with hw_architecture :) | 13:22 |
| mnaser | and https://review.openstack.org/#/c/566425/ | 13:22 |
| *** openstackstatus has joined #openstack-nova | 13:22 | |
| *** ChanServ sets mode: +v openstackstatus | 13:22 | |
| *** openstackstatus has quit IRC | 13:22 | |
| sean-k-mooney | hw_arcitecture is menat to defien the arch of the vm. | 13:22 |
| sean-k-mooney | mnaser: your relying on the fact that today kvm acclerate does not work when you are emulating a different architecture | 13:23 |
| openstackgerrit | Chris Dent proposed openstack/nova master: DNM: Make nova tests pass with externalized placement https://review.openstack.org/596291 | 13:23 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Add subtree filter for GET /resource_providers https://review.openstack.org/595236 | 13:24 |
| mnaser | sean-k-mooney: i don't think that's likely going to change anytime soon™ | 13:24 |
| *** openstackstatus has joined #openstack-nova | 13:24 | |
| *** ChanServ sets mode: +v openstackstatus | 13:24 | |
| sean-k-mooney | mnaser: well my first respocne to that patch is we should revert it with out a spec | 13:24 |
| mnaser | and that would mean we can't deliver multiarchitecture in our cloud | 13:25 |
| mnaser | so i'm not sure how that benefits anyone | 13:25 |
| *** gbarros has quit IRC | 13:25 | |
| mnaser | the patch doesn't hurt anyone except help users deliver a better openstack experience | 13:25 |
| sean-k-mooney | mnaser: you can you just need a weigher. | 13:25 |
| mnaser | doesnt the scheduler filter out N number of items to weight | 13:26 |
| sean-k-mooney | mnaser: my issue is you did this in a filter so its a hard requirement instead of a perference | 13:26 |
| mnaser | it is not | 13:26 |
| mnaser | if the hw_architecture field is empty | 13:26 |
| mnaser | or there is no default configured value | 13:26 |
| mnaser | there is no enforcement | 13:26 |
| sean-k-mooney | yes but i cant set hw_architecture and hv_type anymore | 13:27 |
| mnaser | yes you can | 13:27 |
| mnaser | this patch only uses the configured default IF none is specified in the image. | 13:27 |
| mnaser | you are more than welcome to set hw_arch to aarc64 and hv_type to qemu | 13:27 |
| mnaser | and you'll get software virtualized arm64 vm | 13:27 |
| mnaser | this is just the default values if *none* are specified, which the current openstack nova behavior = put it anywhere. | 13:27 |
| sean-k-mooney | so i can set hw_architecture=x86 and hv_type=kvm and land on a powerkvm node? | 13:28 |
| *** tbachman has quit IRC | 13:28 | |
| mnaser | sean-k-mooney: no, if no values are set, you can end up anywhere yes | 13:28 |
| mnaser | but the thing is, 99% of our customers *dont* send hw_arch and hv_type | 13:28 |
| mnaser | because they assume that the cloud is x86_64/kvm which is a reasonable assumption. | 13:29 |
| sean-k-mooney | mnaser: its not how rackspace used to run there public cloud | 13:29 |
| mnaser | multiarchitecture clouds are rare, so asking every single customer/user to tag their images with an architecture is not really smooth user experience, especially in a transition | 13:29 |
| mnaser | sure. so you're saying that for someone to upload an image into openstack, we need to start asking for 3 more parameters | 13:30 |
| sean-k-mooney | they had a large amount of powerpc nodes at one point with xen as the hypervior running with x86 guests | 13:30 |
| *** eharney has joined #openstack-nova | 13:30 | |
| mnaser | well, i'd say that's more of an outlier. i don't see how this helps solve things and make them easier for users overall in a multiarch environment | 13:31 |
| mnaser | because the rax env you're describing sounds still like a single arch one for the user facing stuff | 13:31 |
| sean-k-mooney | mnaser: well my point is if you implemented this as a weigher you would land on the same acrh in 99% of cases and fall back to hardware acllerated emulaton if not native nodes were available. | 13:32 |
| mnaser | sean-k-mooney: but don't weighers run on a subset of compute nodes? | 13:33 |
| sean-k-mooney | the chage is merged in anycase but i really dont think this should have been a bug. | 13:33 |
| sean-k-mooney | mnaser: am the run after all the filters | 13:33 |
| sean-k-mooney | so you go through placment then the filters then the weighers | 13:33 |
| mnaser | i was pretty sure that weighers ran on a subset of hosts | 13:33 |
| sean-k-mooney | they just reorder what the filters pass | 13:33 |
| mnaser | hence the values that would let you pick subsets of hosts when scheduling | 13:34 |
| sean-k-mooney | sorry i missed that last point? | 13:35 |
| mnaser | i thought weighers only ran on a subset of filtered hosts | 13:36 |
| sean-k-mooney | yes | 13:36 |
| mdbooth | mriedem: Briefly back to the topic of evacuation tests, am I right in thinking we probably won't be able to write a full job which does failure injection? If not, I don't think I don't think I'm blocked on that. If we *can* I'm happy to write it if you point me at the right repo and an example to crib from. | 13:37 |
| *** Luzi has quit IRC | 13:37 | |
| mnaser | sean-k-mooney: was rax doing x86 on ppc via software or hardware emulation? | 13:37 |
| mdbooth | 'example' wouldn't have to be that similar, just enough to orient myself. I haven't touched the top-level CI jobs before. | 13:38 |
| sean-k-mooney | mnaser: i belive xen allows cross arch hardware emulation | 13:38 |
| mdbooth | mriedem: The ability to assert success in the non-failure case might still be interesting, I guess. It would essentially be a regression test. | 13:40 |
| mnaser | sean-k-mooney: i am not too familiar with xen but i would assume that wouldn't be hardware assisted virt (i.e. using VT-x in intel, etc) | 13:40 |
| mnaser | but that's something i dunno much about so i could be entirely wrong | 13:40 |
| sean-k-mooney | mnaser: well xen does use vt-x normally if you are running x86 on x86 | 13:41 |
| sean-k-mooney | as does kvm | 13:41 |
| mnaser | right, so this would be the same as software virtualization which can be really slow, which i'm trying to avoid having anyone end up with | 13:42 |
| mnaser | and all of this is optional, if you set none of those settings, it won't affect you. but if you run a cloud where you want to make sure all VMs are hardware virt'd, then you can do that | 13:42 |
| *** antonym_ is now known as antonym | 13:43 | |
| sean-k-mooney | mnaser: its not so much that this change was done that i object to its that the semantics of the existing api were changed without a spec. | 13:47 |
| mnaser | sean-k-mooney: but if you don't configure this very optional value, there will zero behavior change | 13:47 |
| sean-k-mooney | mnaser: yes and if you had those configure before an you upgrade then your cloud may now be broken if they do not have the same semantics | 13:48 |
| mnaser | sean-k-mooney: you couldn't possible have them configured because they don't even exist before? | 13:48 |
| mnaser | s/possible/possibly | 13:49 |
| sean-k-mooney | the image properties | 13:49 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: api: Remove unnecessary default parameter https://review.openstack.org/564451 | 13:50 |
| *** mlavalle has joined #openstack-nova | 13:50 | |
| *** itlinux-away has quit IRC | 13:53 | |
| *** awaugama has joined #openstack-nova | 13:53 | |
| *** gbarros has joined #openstack-nova | 13:53 | |
| *** gbarros has quit IRC | 13:54 | |
| *** gbarros has joined #openstack-nova | 13:55 | |
| mriedem | mdbooth: what type of failure needs to be injected? like delete the disk before starting the evacuatoin? | 13:58 |
| mdbooth | mriedem: Any failure in spawn on the destination | 13:59 |
| mriedem | can't really do that from tempest, but could if run from a script | 13:59 |
| mdbooth | I think I chose plug_vifs in my functional test | 13:59 |
| dansmith | mriedem: so on your gate job, | 13:59 |
| dansmith | mriedem: just force-downing the service isn't going to be enough I think, since it will still be running, potentially mucking with allocations and state correction, etc right? | 14:00 |
| mdbooth | Just need to assert that if we fail in spawn in the dest, we only cleanup what we're supposed to and don't delete shared data. | 14:00 |
| mdbooth | Whether that's shared instance dir or shared block. | 14:00 |
| dansmith | you have to actually kill the n-cpu process and force-down it to avoid the wait | 14:00 |
| dansmith | (well, not mucking with allocations until it's deleted maybe anymore, but screaming at least) | 14:00 |
| * mdbooth -> meeting | 14:01 | |
| dansmith | but I think it's more legit if you stop it | 14:01 |
| mriedem | dansmith: force down is what we do in functional tests | 14:01 |
| mriedem | that's all the API checks for | 14:01 |
| dansmith | mriedem: I know it will make evac work, | 14:01 |
| dansmith | but the n-cpu for the "down" one will still be running and doing stuff, which isn't how we describe evac as working properly in real life | 14:02 |
| dansmith | we say you have to fence it | 14:02 |
| *** jaosorior has joined #openstack-nova | 14:02 | |
| mriedem | in a multinode job the post-test script runs on the control node which has an n-cpu process on it which we could kill | 14:02 |
| mriedem | we'd have to force the server to that host so it's the source | 14:03 |
| stephenfin | mriedem, melwitt, dansmith: btw, given that sahid immediately replied yesterday, I figured it wasn't necessary to investigate the queue size thing any further myself. Sorry if I picked that up wrong | 14:03 |
| mriedem | stephenfin: it's fine | 14:03 |
| mriedem | after looking at the existing release note and config option help for the rx/tx queue size stuff i think they were good enough to say that it was very limited to certain types of vifs | 14:04 |
| mriedem | and i think what moshe is trying to do is make it work for something else | 14:04 |
| *** r-daneel has joined #openstack-nova | 14:04 | |
| stephenfin | mriedem: aight, good to hear | 14:05 |
| melwitt | . | 14:06 |
| mriedem | mdbooth: btw yes i'm also interested in simply have regression testing for evac in a real env | 14:06 |
| mriedem | *having | 14:06 |
| mriedem | melwitt: that last rocky change is in the gate | 14:07 |
| melwitt | mriedem: I saw, thank you for getting that. the master change didn't land until late last night after I went to sleep | 14:09 |
| *** med_ has joined #openstack-nova | 14:13 | |
| mriedem | mdbooth: btw do you still care about this? https://review.openstack.org/#/c/572790/ | 14:13 |
| *** efried_afk is now known as efried | 14:16 | |
| *** efried is now known as senhor_granhular | 14:24 | |
| *** med_ has quit IRC | 14:28 | |
| *** jaosorior has quit IRC | 14:33 | |
| *** itlinux has joined #openstack-nova | 14:35 | |
| *** senhor_granhular is now known as fried_rice | 14:37 | |
| mriedem | dansmith: a couple of thoughts on batch size floor in https://review.openstack.org/#/c/593131/ | 14:38 |
| *** pooja_jadhav has joined #openstack-nova | 14:38 | |
| *** cfriesen has joined #openstack-nova | 14:38 | |
| *** dpawlik_ has quit IRC | 14:39 | |
| dansmith | mriedem: thanks will look shortly | 14:41 |
| *** moshele has joined #openstack-nova | 14:44 | |
| *** ociuhandu has joined #openstack-nova | 14:44 | |
| *** ttsiouts has quit IRC | 14:46 | |
| *** links has quit IRC | 14:52 | |
| *** jlvillal is now known as jlviva-viva | 14:52 | |
| *** ociuhandu has quit IRC | 14:53 | |
| *** itlinux has quit IRC | 14:55 | |
| *** ykarel is now known as ykarel|away | 14:55 | |
| mriedem | and i'm done with that stack \o/ | 14:57 |
| *** pcaruana has quit IRC | 14:59 | |
| mdbooth | mriedem: I do. Came up yesterday, in fact. | 15:02 |
| mdbooth | mriedem: I don't think I had any strong objections there? | 15:02 |
| mriedem | i'm pretty sure you wanted to +1 it | 15:03 |
| mdbooth | Just re-reading my last review. Sounds like I was prevaricating rather than objecting. | 15:04 |
| *** r-daneel has quit IRC | 15:04 | |
| mdbooth | Ah... the attach_mode thing. | 15:06 |
| *** r-daneel has joined #openstack-nova | 15:06 | |
| *** ttsiouts has joined #openstack-nova | 15:07 | |
| *** Bhujay has quit IRC | 15:08 | |
| *** itlinux has joined #openstack-nova | 15:09 | |
| mdbooth | mriedem: So there's attach_mode and access_mode | 15:13 |
| * mdbooth goes looking at cinder docs | 15:13 | |
| mdbooth | It seems like we only *use* access_mode, but that code is checking attach_mode | 15:14 |
| *** r-daneel_ has joined #openstack-nova | 15:18 | |
| *** r-daneel has quit IRC | 15:19 | |
| *** r-daneel_ is now known as r-daneel | 15:19 | |
| *** nicolasbock has joined #openstack-nova | 15:21 | |
| *** imacdonn has joined #openstack-nova | 15:22 | |
| openstackgerrit | Claudiu Belu proposed openstack/nova master: hyper-v: autospec classes before they are instantiated https://review.openstack.org/342211 | 15:23 |
| openstackgerrit | Claudiu Belu proposed openstack/nova master: WIP: replace spec with autospec https://review.openstack.org/557299 | 15:23 |
| mdbooth | mriedem: Who's a friendly cinder person? | 15:29 |
| *** macza has joined #openstack-nova | 15:29 | |
| *** ykarel|away has quit IRC | 15:30 | |
| mriedem | for attachment record stuff it would be jgriffith | 15:31 |
| *** whoami-rajat has quit IRC | 15:32 | |
| *** gbarros has quit IRC | 15:33 | |
| mdbooth | I've hopefully added the right one. | 15:33 |
| *** giblet is now known as giblet_off | 15:34 | |
| giblet_off | I will be back on Tuesday | 15:34 |
| *** gbarros has joined #openstack-nova | 15:34 | |
| mdbooth | dansmith: Is your no timeouts messaging thing in Queens, btw? Or backportable to Queens? | 15:38 |
| *** ShilpaSD has quit IRC | 15:45 | |
| *** ttsiouts has quit IRC | 15:46 | |
| mriedem | it is not in queens | 15:46 |
| mriedem | and not really backportable | 15:46 |
| mriedem | as that would require a min required version of oslo.messaging | 15:46 |
| mriedem | and we don't bump mins on libs in stable | 15:46 |
| mdbooth | mriedem: ack, thanks. | 15:47 |
| mdbooth | mriedem: Means I can't promise a fix to a customer in our next stable release :/ | 15:47 |
| sean-k-mooney | mdbooth: without it could you just set the time out really high to get a similar effect? | 15:48 |
| mdbooth | sean-k-mooney: Yeah. | 15:48 |
| mdbooth | sean-k-mooney: I still need to look way harder at the impact, anyway. | 15:49 |
| mriedem | mdbooth: you can fork whatever you want | 15:50 |
| *** pooja_jadhav has quit IRC | 15:50 | |
| mdbooth | mriedem: Well I would, but dansmith won't let me. | 15:50 |
| mdbooth | I was just going to rewrite in Go. | 15:51 |
| *** jpena is now known as jpena|off | 15:53 | |
| mdbooth | In all seriousness, though: pretty sure you were joking, but in case not we don't do that. At least not in any incompatible way. | 15:53 |
| mdbooth | If we can't do it, that's the way it is. | 15:54 |
| *** tssurya has quit IRC | 15:54 | |
| *** Sundar_ has joined #openstack-nova | 15:58 | |
| *** Bhujay has joined #openstack-nova | 16:01 | |
| *** dtantsur is now known as dtantsur|afk | 16:01 | |
| *** itlinux has quit IRC | 16:02 | |
| *** manjeets has joined #openstack-nova | 16:03 | |
| *** itlinux has joined #openstack-nova | 16:06 | |
| *** sahid has quit IRC | 16:08 | |
| *** munimeha1 has joined #openstack-nova | 16:10 | |
| *** gbarros has quit IRC | 16:13 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Note NUMA topology requirements for numa-aware-vswitches https://review.openstack.org/596393 | 16:14 |
| *** itlinux is now known as itlinux-away | 16:16 | |
| mdbooth | melwitt: Do you remember anything about either adding or removing refresh_conn_info from hard reboot? | 16:16 |
| *** itlinux-away is now known as itlinux | 16:16 | |
| mdbooth | It's vaguely ringing a bell, but I can't find any evidence of it. | 16:16 |
| melwitt | mdbooth: no, I never understood much around that. lyarwood would be the one to ask | 16:17 |
| melwitt | might be a WIP patch from him or something? not sure | 16:17 |
| openstackgerrit | Merged openstack/nova stable/rocky: Correct the release notes related to nova-consoleauth https://review.openstack.org/595890 | 16:18 |
| openstackgerrit | Merged openstack/nova master: tests: Move mocking to setUp https://review.openstack.org/595802 | 16:18 |
| *** r-daneel_ has joined #openstack-nova | 16:18 | |
| *** udesale has quit IRC | 16:19 | |
| *** gyee has joined #openstack-nova | 16:19 | |
| *** r-daneel has quit IRC | 16:20 | |
| *** r-daneel_ is now known as r-daneel | 16:20 | |
| mdbooth | melwitt: I think the change I'm thinking of is when we added the call to destroy() which calls cleanup() and therefore does other stuff like detaching volumes. We don't seem to refresh_conn_info, though | 16:20 |
| mdbooth | mriedem: Is refresh_conn_info still relevant in the new world order? | 16:21 |
| * mdbooth suspects it is | 16:21 | |
| melwitt | mdbooth: ah yeah, the "redo all the things" change | 16:21 |
| mdbooth | melwitt: Yeah. | 16:21 |
| melwitt | didn't have anything to do with refresh_conn_info tho (as you saw) | 16:22 |
| mdbooth | I'm trying to come up with a recovery recommendation for this pre_live_migration timeout. | 16:22 |
| mdbooth | The bug is: we called cleanup on the source when the dest timed out, even though it continued to run. The result is that the instance still has all volumes connected on the source where it continues to run, but some bdms have conn_info for source and others for dest. | 16:23 |
| mdbooth | So really we want some operation which would cause a refresh_conn_info. | 16:24 |
| mdbooth | Options seem to be cold and live migrations. | 16:24 |
| mdbooth | Wonder if reboot ought to be another. | 16:24 |
| mriedem | mdbooth: refresh_conn_info with the new attachment record flow isn't the same as how things worked with the old flow | 16:25 |
| smcginnis | melwitt: Looks like that last stable/rocky patch merged, and you were right that the hash needs to be updated. | 16:25 |
| mdbooth | mriedem: I'll take a look. | 16:26 |
| melwitt | smcginnis: doing it right now, thanks | 16:26 |
| mriedem | mdbooth: https://review.openstack.org/#/c/584648/20/nova/scheduler/client/report.py@2055 the difference is in the old flow, we'd always create a new connection to the storage backend which gives us a fresh connection_info from cinder, | 16:26 |
| mriedem | with the new flow, we are getting the attachment record which has a stashed connection_info, which might be potentially stale | 16:26 |
| mriedem | which is what i think you're worried about | 16:26 |
| mriedem | btw this is related https://review.openstack.org/#/c/579004/ | 16:27 |
| mdbooth | mriedem: Ah, good point. So in that case attachment_id also be out of date. | 16:27 |
| mdbooth | From that commit message: Note: Only hard reboots will currently update connection_info. | 16:28 |
| mdbooth | I wonder if I missed something | 16:28 |
| mriedem | that's just a result of the patch | 16:28 |
| mriedem | but i think the comment is wrong | 16:29 |
| mdbooth | mriedem: That's super-useful, thanks. | 16:30 |
| mriedem | i am both very useful and very handsome | 16:30 |
| mriedem | i think we can all agree | 16:30 |
| mdbooth | mriedem: Your modesty is also unparalleled in all of history ;) | 16:31 |
| mriedem | that's very kind of you | 16:32 |
| mriedem | so, | 16:32 |
| mriedem | without digging into the logs, | 16:32 |
| mriedem | i expect this is blowing up b/c attachment_update isn't meant to be used this way once it's already got a connector | 16:32 |
| mriedem | likely should be creating a new attachment record with new connector (and id) and deleting the old one | 16:32 |
| *** gbarros has joined #openstack-nova | 16:34 | |
| *** gbarros has quit IRC | 16:37 | |
| mriedem | commented and pinged jgriffith | 16:38 |
| mdbooth | Oh, the irony! | 16:38 |
| mdbooth | So, if connection_info is wrong that's obviously going to break any attempt to attach | 16:39 |
| mdbooth | But in this particular case because the volumes are already attached they're already working | 16:39 |
| *** fried_rice is now known as fried_rolls | 16:39 | |
| mdbooth | But since the robustification patch added detach/attach in hard reboot, we actually broke it | 16:40 |
| mdbooth | It needs to be combined with refresh_conn_info | 16:40 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Batch results per cell when doing cross-cell listing https://review.openstack.org/592698 | 16:40 |
| openstackgerrit | Dan Smith proposed openstack/nova master: List instances from all cells explicitly https://review.openstack.org/593717 | 16:41 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make instance_list perform per-cell batching https://review.openstack.org/593131 | 16:41 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Record cell success/failure/timeout in CrossCellLister https://review.openstack.org/594265 | 16:41 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Optimize global marker re-lookup in multi_cell_list https://review.openstack.org/594577 | 16:41 |
| mriedem | umm, | 16:41 |
| mriedem | well, | 16:41 |
| mriedem | we've talked about this hard refresh thing for a long time b/c of the ceph ip or auth changing and not wanting to live migrate your entire cloud to update the attachments | 16:42 |
| mriedem | i think coreycb is trying to fix this one case for reboot, and then see if we can extend to other operations | 16:44 |
| mriedem | "This only includes a fix for the 'openstack server reboot --hard' path as I didn't want to get too deep before knowing if this is a proper fix. A complete fix would also provide soft reboot, stop/start support, perhaps more?" | 16:44 |
| *** gbarros has joined #openstack-nova | 16:45 | |
| coreycb | mriedem: thanks for your comments. i dropped that patch for a bit but will get back to it once we get rocky done. | 16:46 |
| mdbooth | mriedem: I'm coming from a different angle. A weird unfixable bug means I've now got inconsistent state, and I want a thing to make everything right again. | 16:47 |
| mriedem | mdbooth: is that related to your other live migration patch? | 16:48 |
| mriedem | https://review.openstack.org/#/c/551302/ | 16:48 |
| mriedem | bw, you probably need some more spinning plates | 16:49 |
| mdbooth | mriedem: No. | 16:50 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Optimize global marker re-lookup in multi_cell_list https://review.openstack.org/594577 | 16:51 |
| mdbooth | mriedem: I thought about that one, but I don't think it would fix this. | 16:51 |
| mdbooth | I should resurrect that patch as being just a generally good idea, even if the original motivation for it has gone. It would have fixed a couple of other things since iirc. | 16:51 |
| stephenfin | mriedem: I'm not sure what I can do about this https://review.openstack.org/#/c/588570/ | 16:53 |
| stephenfin | mriedem: Have tried drafting a functional test but it looks like it's going to require a good deal of changes to the fake libvirt driver, assuming it's even possible to validate | 16:54 |
| mriedem | mdbooth: fwiw i think i was ok with that change but wanted to see the same rollback stuff done in a follow up first | 16:55 |
| mriedem | stephenfin: ok...i'm about out of here for lunch and errands, so probably need to hit me next week when i'm in more of a "f it" kind of mood | 16:56 |
| stephenfin | mriedem: Heh, fair | 16:57 |
| mriedem | because this definitely seems like new feature capability | 16:57 |
| mriedem | if it takes this much to fix a thing, it probably never worked in the first place right? | 16:57 |
| mriedem | the bug says a certain configuration is not "optimal" | 16:58 |
| stephenfin | I think it was more a case of encoding faulty assumptions | 16:58 |
| mriedem | this is optimizing things yes? | 16:58 |
| stephenfin | namely, that asymmetric host topologies weren't a thing | 16:58 |
| *** holser_ has quit IRC | 16:59 | |
| stephenfin | Yeah, optimization is probably fair | 16:59 |
| stephenfin | without the patch, you see issues when your hosts get a bit congested | 16:59 |
| stephenfin | ...and you're using emulator threads | 16:59 |
| mriedem | well, given (1) how large this is and multiple changes and (2) it sounds low priority bug-wise if it's not outright failing, and (3) i'm in no way an expert on this code, i'm hesitant to even get close to a backport of this | 17:00 |
| stephenfin | Also fair | 17:00 |
| mriedem | if dan puts on his nfv shoes maybe he'd care | 17:00 |
| mriedem | idk | 17:00 |
| *** med_ has joined #openstack-nova | 17:00 | |
| stephenfin | I'll see if I can rope leakypipes into it next week, given that he reviewed it first time round | 17:00 |
| stephenfin | If not...I'll think of something | 17:01 |
| stephenfin | Aight, home time o/ | 17:01 |
| melwitt | how do nova.conf get used with services running under uwsgi? how do you specify which conf | 17:02 |
| *** mriedem is now known as mriedem_afk | 17:02 | |
| dansmith | melwitt: depends on what wsgi container you're using I think | 17:05 |
| melwitt | dansmith: what does that mean? which nova service? or something external to that? | 17:07 |
| dansmith | no, like apache, ngnix, and then which wsgi module on each you use (uwsgi, etc) | 17:08 |
| melwitt | oh, ok | 17:08 |
| *** mdbooth has quit IRC | 17:09 | |
| dansmith | I don't know much about uwsgi, so not sure where to point you to for it | 17:09 |
| melwitt | context is, I was thinking this proposed devstack change is wrong https://review.openstack.org/#/c/564978/2/lib/nova but then, I don't get how the standalone metadata API service is currently working with querying instance metadata, for example, in the neutron-tempest-multinode-full job. it must be using the indirection api because it's standalone, but then how is it accessing the cell database? | 17:10 |
| dansmith | with mod_wsgi you have a little app definition python file which is where you'd do it (or, I'd do it) | 17:10 |
| melwitt | and I can't see how to see what nova.conf the uwsgi metadata service is running with | 17:10 |
| melwitt | I see | 17:10 |
| dansmith | do we run the standalone metadata api under wsgi? I don't think we do | 17:11 |
| *** macza has quit IRC | 17:11 | |
| dansmith | even still, | 17:11 |
| dansmith | the indirection_api thing isn't part of config so I don't think there's anything inconfig that is relevant | 17:12 |
| dansmith | https://github.com/openstack/nova/blob/master/nova/cmd/api_metadata.py#L45-L45 | 17:12 |
| dansmith | that's where it gets forced on for the standalone one | 17:12 |
| melwitt | okay, maybe I got confused by something. because there's a separate n-api-metadata file, but that doesn't mean it's running standalone necessarily right? | 17:12 |
| dansmith | file meaning.. log file? | 17:12 |
| melwitt | I was thinking it meant it's standalone | 17:12 |
| melwitt | yeah the log file | 17:12 |
| dansmith | yeah, but that doesn't mean it's running under wsgi... | 17:13 |
| melwitt | I can see it's running under wsgi from the ps.txt.gz here http://logs.openstack.org/90/595890/2/check/neutron-tempest-multinode-full/a4a0803/logs/ps.txt.gz | 17:13 |
| melwitt | but I'm not sure how to tell whether it's the standalone service or the all-in-one | 17:13 |
| *** macza has joined #openstack-nova | 17:13 | |
| dansmith | apparently we are | 17:14 |
| dansmith | http://logs.openstack.org/77/594577/6/check/neutron-grenade/f10f831/logs/etc/nova/nova-metadata-uwsgi.ini.txt.gz | 17:14 |
| dansmith | that's the uwsgi definition for it | 17:14 |
| melwitt | oh yeah, that too | 17:14 |
| dansmith | not sure if we capture that /usr/local/bin/nova-api-metadata-wsgi file though | 17:15 |
| dansmith | maybe that's auto-generated | 17:15 |
| melwitt | ah, so maybe it's set in devstack somewhere | 17:15 |
| dansmith | but again, what in the config are you looking for related to the indirection api? | 17:16 |
| dansmith | I'm not sure what you meant above about "it's standalone so much be using the indirection api" | 17:17 |
| *** gbarros has quit IRC | 17:17 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: Batch results per cell when doing cross-cell listing https://review.openstack.org/592698 | 17:18 |
| melwitt | no, sorry, I'm trying to determine whether that proposed change to devstack is right or not. it looks like it's trying to make the standalone service default to the cell database so it can find instance metadata. in devstack, it looks like if services_enabled includes n-api-meta, it is doing the standalone service which we said is not supposed to be used with multiple cells | 17:18 |
| openstackgerrit | Dan Smith proposed openstack/nova master: List instances from all cells explicitly https://review.openstack.org/593717 | 17:18 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make instance_list perform per-cell batching https://review.openstack.org/593131 | 17:18 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Record cell success/failure/timeout in CrossCellLister https://review.openstack.org/594265 | 17:18 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Optimize global marker re-lookup in multi_cell_list https://review.openstack.org/594577 | 17:18 |
| melwitt | then, I looked at an existing multinode job to see, how is the metadata API service currently working then without that change? | 17:19 |
| dansmith | well, I think that change isn't right, | 17:19 |
| dansmith | because it means we won't do multi-cell mode _unless_ the standalone metadata service is enabled right? | 17:20 |
| dansmith | which is clearly not what we want | 17:20 |
| melwitt | here, it looks like the neutron multinode job is running with the standalone meta service http://logs.openstack.org/90/595890/2/check/neutron-tempest-multinode-full/a4a0803/logs/devstacklog.txt.gz#_2018-08-23_21_19_27_916 so how is it finding metadata? just confused | 17:20 |
| *** dpawlik has joined #openstack-nova | 17:20 | |
| melwitt | oh, I was thinking that was an OR. my mistake I think | 17:21 |
| dansmith | Aug 23 21:29:00.426509 ubuntu-xenial-inap-mtl01-0001520794 devstack@n-api-meta.service[4018]: DEBUG nova.api.openstack.wsgi_app [None req-848ebec7-678d-4d46-923a-2b41bf306753 None None] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=4020) log_opt_values /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:3009}} | 17:21 |
| melwitt | oh, thank you | 17:21 |
| dansmith | that standalone service is running from nova.conf | 17:22 |
| melwitt | ok, and nova.conf is pointing at the cell0 database | 17:22 |
| dansmith | well, but, | 17:22 |
| dansmith | it's using indirection, which means it goes to superconductor | 17:23 |
| dansmith | which should be pointing at cell0 too | 17:23 |
| *** gbarros has joined #openstack-nova | 17:23 | |
| melwitt | yeah... so shouldn't getting metadata not work? that's what's confusing me. but I see requests going successfully in the meta API log | 17:23 |
| dansmith | sorry, I'm not being clear | 17:24 |
| mnaser | throwing this out here: how does barbican stable/rocky look right now? is it pretty much likely going to be the released version? (i.e. will/how much i get in trouble for deploying a new region on stable/rocky) | 17:24 |
| mnaser | :p | 17:24 |
| mnaser | er | 17:24 |
| mnaser | s/barbican/nova/ | 17:24 |
| dansmith | melwitt: the standalone runner would be using indirection_api, going to superconductor and hitting cell0 and failing | 17:24 |
| *** dpawlik has quit IRC | 17:25 | |
| dansmith | melwitt: but we're not running _our_ standalone metadata service, we're running metadata api as non-standalone, but in its own wsgi container | 17:25 |
| melwitt | dansmith: oh, geez, okay | 17:25 |
| dansmith | so it's behaving like the combined global api service would have previously when we smashed them all together, | 17:25 |
| dansmith | but with wsgi you yank them into life a little differently, without the nova-api service bootstrapping code | 17:26 |
| dansmith | which means it should be looking up the mapping for instances and talking to the right cell, like in global mode | 17:26 |
| *** NostawRm has joined #openstack-nova | 17:26 | |
| melwitt | cray cray | 17:26 |
| dansmith | calling it "non-standalone" is a bit of a misnomer, but compared to our old combined service bootstrapper thing, it's in "non-standalone" mode of operation | 17:27 |
| dansmith | but.. it's fairly standalone in that it's running all by itself and has its own log file | 17:27 |
| melwitt | dansmith: okay, so what is this then here? this looks like it means if service enabled n-api-meta, then remove the metadata from nova_enabled_apis and run it is _our_ standalone service. right? https://review.openstack.org/#/c/564978/2/lib/nova@470 | 17:27 |
| *** itlinux is now known as itlinux-away | 17:27 | |
| melwitt | *as | 17:27 |
| dansmith | that's probably now confused, | 17:28 |
| dansmith | as that used to be how devstack referred to *our* standalone service, and may still in some cases, idk, but now post-systemd and post-wsgi probably refers to the, uh, other one :) | 17:28 |
| melwitt | mnaser: we think so, but we also thought we'd only have one RC and now we have 3 | 17:29 |
| *** gbarros has quit IRC | 17:29 | |
| mnaser | maybe ican help you make it 4, hah :P | 17:29 |
| dansmith | melwitt: why are you even reviewing this with no details in the bug or commit message? | 17:29 |
| *** itlinux-away is now known as itlinux | 17:29 | |
| *** itlinux is now known as itlinux-away | 17:29 | |
| mnaser | thats reasonable. ill proceed being careful and give input :) | 17:29 |
| *** itlinux-away is now known as itlinux | 17:30 | |
| *** itlinux is now known as itlinux-away | 17:30 | |
| *** gbarros has joined #openstack-nova | 17:30 | |
| dansmith | melwitt: mriedem_afk: is this known? http://logs.openstack.org/71/594571/2/gate/nova-tox-functional-py35/fd2d9ac/testr_results.html.gz | 17:30 |
| melwitt | mnaser: haha :) always happy to get your testing of the code, and sorry we've been behind on the cells orphaned records patch reviews. things have been just really crazy | 17:31 |
| dansmith | failed on both jobs of one of my patches | 17:31 |
| mnaser | melwitt: nah its cool, my workaround for that was make sure our underlying infra doesnt break and thats ok enough :p | 17:31 |
| mnaser | i dont think things that break after unrelated nova issues would be priority number 1, imho | 17:31 |
| melwitt | thanks for the understanding | 17:32 |
| *** _hemna has joined #openstack-nova | 17:32 | |
| melwitt | dansmith: that's the first time I've seen that | 17:32 |
| melwitt | hm, weird FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test' | 17:33 |
| melwitt | it tries to write a file to test for directio support | 17:34 |
| *** itlinux-away has quit IRC | 17:35 | |
| melwitt | and that failed. does that mean the nova/instances/ dir missing from the test tree probably? hm | 17:35 |
| dansmith | just not sure what would have changed | 17:36 |
| melwitt | I can't remember if that was one of those wonky directories that get created during test runs, to put temp files in for tests | 17:36 |
| melwitt | like, what is that instances/ directory, I can't remember | 17:37 |
| melwitt | oh wait, it's that the actual code is trying to write that file in a real life scenario and in the test env there is no nova/instances dir. but I don't know how that used to be working if it was working before | 17:37 |
| melwitt | in func test land | 17:38 |
| *** med_ has quit IRC | 17:41 | |
| melwitt | CONF.instances_path | 17:46 |
| *** fried_rolls is now known as fried_rice | 17:47 | |
| *** eharney has quit IRC | 17:53 | |
| *** gbarros has quit IRC | 17:56 | |
| jrock_cfdg | could someone please help out? Trying to add a serial port with source mode='connect' to do low-level kernel debugging in windows instance; everytime i add device to instance it shuts down - is this even possible with nova? i can do this in libvirt w/out issue | 18:05 |
| Sundar_ | fried_rice, efried: Do you have a moment? | 18:08 |
| fried_rice | Sundar_: sure. | 18:08 |
| openstackgerrit | Merged openstack/nova master: Make CELL_TIMEOUT a constant https://review.openstack.org/594570 | 18:08 |
| fried_rice | Sundar_: I *am* trying to get to reviewing your spec really soon | 18:08 |
| fried_rice | Sundar_: I've just been inundated with other things. | 18:09 |
| Sundar_ | In the scheduling spec, we defined a way to extend the granular resource request syntax to support Cyborg: https://git.openstack.org/cgit/openstack/cyborg/tree/doc/specs/rocky/cyborg-nova-sched.rst?h=refs/changes/17/554717/18#n315 | 18:09 |
| Sundar_ | Just want to make sure you have reviewed it. | 18:09 |
| Sundar_ | fied_rice: NP :) I understand. We all have to roll on different tracks at the same time. | 18:10 |
| Sundar_ | And I may have added yet another track for you to roll on. | 18:11 |
| fried_rice | Sundar_: I remember seeing this and being okay with it, but I would have to do a major context switch to be able to discuss it intelligently at this point. | 18:12 |
| fried_rice | I noticed you said the extended keys aren't interpreted by nova. Presumably that means they're interpreted by cyborg. How does that information get to cyborg? And does it translate to some new/different placement syntax? I assume not, that it just tells cyborg how to do certain things. | 18:14 |
| *** cdent has quit IRC | 18:14 | |
| Sundar_ | fried_rice: Good to know that it is accepted. Yes, they will be interpreted by Cyborg (not sure if os-acc will also interpret it). The extra specs in the request are passed from Nova virt to os-acc to Cyborg. | 18:17 |
| Sundar_ | No new placement APIs needed. | 18:17 |
| melwitt | I have to run to an appointment, will be back later | 18:19 |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Merge security groups extension response into server view builder https://review.openstack.org/585475 | 18:34 |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Merge extended_status extension response into server view builder https://review.openstack.org/592092 | 18:35 |
| *** moshele has quit IRC | 18:39 | |
| openstackgerrit | Merged openstack/nova master: tests: Create functional libvirt test base class https://review.openstack.org/407055 | 18:41 |
| *** mriedem_afk is now known as mriedem | 18:45 | |
| *** eharney has joined #openstack-nova | 18:49 | |
| mriedem | dansmith: i haven't seen that functional test failure | 18:50 |
| *** gbarros has joined #openstack-nova | 18:58 | |
| mriedem | dansmith: umm, could https://review.openstack.org/#/c/407055/ that just merged be a regression? | 19:00 |
| mriedem | or the change below it | 19:02 |
| mriedem | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22FileNotFoundError%3A%20%5BErrno%202%5D%20No%20such%20file%20or%20directory%3A%20'%2Fhome%2Fzuul%2Fsrc%2Fgit.openstack.org%2Fopenstack%2Fnova%2Finstances%2F.directio.test'%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d | 19:02 |
| mriedem | just started | 19:03 |
| mriedem | so those are probably it | 19:03 |
| *** Bhujay has quit IRC | 19:04 | |
| mriedem | https://bugs.launchpad.net/nova/+bug/1788922 | 19:05 |
| openstack | Launchpad bug 1788922 in OpenStack Compute (nova) "SRIOVServersTest.test_create_server_with_VF intermittently fails due to "FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test'"" [Undecided,New] | 19:05 |
| mriedem | -1ed https://review.openstack.org/#/c/564978/ which it looks like you and melwitt were talking about - the change has no context | 19:07 |
| mriedem | "this fixes something" | 19:07 |
| *** Sundar_ has quit IRC | 19:10 | |
| *** mrjk has quit IRC | 19:13 | |
| mriedem | oh you know, this func test might only be failing on one node provider | 19:14 |
| mriedem | but that means we aren't mocking something out properly and accidentally passing on other hosts | 19:14 |
| mriedem | maybe | 19:14 |
| openstackgerrit | Eric Fried proposed openstack/nova master: [placement] split gigantor SQL query, add logging https://review.openstack.org/590041 | 19:23 |
| *** erlon has quit IRC | 19:25 | |
| *** dpawlik has joined #openstack-nova | 19:39 | |
| *** mriedem has quit IRC | 19:40 | |
| *** dpawlik has quit IRC | 19:44 | |
| *** mriedem has joined #openstack-nova | 19:48 | |
| mriedem | melwitt: question in https://review.openstack.org/#/c/540258/ | 19:49 |
| *** mrjk has joined #openstack-nova | 19:53 | |
| *** eandersson has joined #openstack-nova | 20:06 | |
| openstackgerrit | Merged openstack/nova master: Stash the cell uuid on the context when targeting https://review.openstack.org/594571 | 20:09 |
| mriedem | melwitt: have you thought about an agenda/schedule for the ptg? just thought of it since gorka was asking in the thread on cross-cell migration | 20:11 |
| *** fried_rice is now known as efried_afk | 20:13 | |
| *** itlinux has joined #openstack-nova | 20:14 | |
| *** efried_afk is now known as fried_rice | 20:14 | |
| mriedem | dansmith: i must not be understanding this comment https://review.openstack.org/#/c/592698/12/nova/tests/unit/compute/test_multi_cell_list.py@118 | 20:23 |
| dansmith | mriedem: copied from L89 above, and yes | 20:24 |
| dansmith | it means that since key0 is the same on both it should actually sort the two instances by key1 | 20:25 |
| dansmith | but since I copied it for this test and only have one instance it obviously doesn't need to be htere | 20:25 |
| mriedem | yeah... | 20:26 |
| mriedem | i was like, | 20:26 |
| mriedem | but it's only 1 record... | 20:26 |
| mriedem | cleanup patch on top of the series? | 20:26 |
| dansmith | whatever you want | 20:27 |
| dansmith | if we're blocked on that functional test, I might as well just fix it | 20:28 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Batch results per cell when doing cross-cell listing https://review.openstack.org/592698 | 20:29 |
| openstackgerrit | Dan Smith proposed openstack/nova master: List instances from all cells explicitly https://review.openstack.org/593717 | 20:29 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make instance_list perform per-cell batching https://review.openstack.org/593131 | 20:29 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Record cell success/failure/timeout in CrossCellLister https://review.openstack.org/594265 | 20:29 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Optimize global marker re-lookup in multi_cell_list https://review.openstack.org/594577 | 20:30 |
| *** tbachman has joined #openstack-nova | 20:35 | |
| mriedem | shouldn't be blocked http://status.openstack.org/elastic-recheck/#1788922 | 20:36 |
| mriedem | but logstash might be behind | 20:36 |
| dansmith | I had both functional jobs fail in one thing so it seemed like it was high frequency but okay | 20:36 |
| mriedem | yeah it might be higher fail rate than logstash/e-r is seeing right now | 20:40 |
| mriedem | or my query is missing something | 20:40 |
| mriedem | hmm, looking at this i think my query is only going to hit py27 jobs | 20:41 |
| mriedem | no i guess its hitting the opposite, only py35 | 20:42 |
| mriedem | +2s up the board | 20:57 |
| *** itlinux has quit IRC | 20:58 | |
| *** marvin_mhg has quit IRC | 21:04 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Report client: Real get_allocs_for_consumer https://review.openstack.org/584599 | 21:13 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Report client: get_allocations_for_provider_tree https://review.openstack.org/584648 | 21:13 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Report client: _reshape helper, placement min bump https://review.openstack.org/585034 | 21:13 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Report client: update_from_provider_tree w/reshape https://review.openstack.org/585049 | 21:13 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Compute: Handle reshaped provider trees https://review.openstack.org/576236 | 21:13 |
| *** munimeha1 has quit IRC | 21:15 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: fix volume attachment update policy note https://review.openstack.org/596489 | 21:29 |
| *** awaugama has quit IRC | 21:36 | |
| *** elod has quit IRC | 21:42 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: add a warning about calling swap volume directly https://review.openstack.org/596492 | 21:47 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Document no content on POST /reshaper 204 https://review.openstack.org/596494 | 21:49 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Compute: Handle reshaped provider trees https://review.openstack.org/576236 | 21:53 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional test for live migrate with anti-affinity group https://review.openstack.org/588935 | 21:53 |
| *** eharney has quit IRC | 22:07 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Fix race condition in reshaper handler https://review.openstack.org/596497 | 22:07 |
| melwitt | mriedem: thanks. I was going to make a similar comment on the devstack change but now I don't have to | 22:15 |
| mriedem | troll proxy | 22:16 |
| melwitt | haha | 22:16 |
| melwitt | mriedem: yeah, thinking monday I'll try and schedule some times slots for nova/cinder, nova/ironic, nova/neutron (assuming we need that to) | 22:17 |
| melwitt | *too | 22:17 |
| melwitt | originally had in mind to figure it out this week, but this week was too busy with RC3 and other things | 22:18 |
| mriedem | i didn't think we had any nova/ironic stuff except finalizing on resource classes and dropping the old flavor stuff, but that seems to have hit a fan so will probably be a bigger discussion i guess | 22:18 |
| mriedem | i thought we could just drop a lot of code | 22:18 |
| *** itlinux has joined #openstack-nova | 22:19 | |
| mriedem | jroll: btw do you have that tripleo bug handy? i want to throw it in the ptg etherpad | 22:19 |
| melwitt | I was thinking we'd do thursday as cross project meet day again but I need to find out if mlavalle and jungleboyj are ok with thursday. TheJulia said thursday afternoon is ok for ironic. she told me they have a thing where they want to discuss the possibility of being able to configure subsets of ironic nodes per nova-compute | 22:19 |
| melwitt | failure domain type stuff | 22:20 |
| mlavalle | melwitt: yes, Thursday works for us, as usual | 22:20 |
| melwitt | cool, thanks | 22:20 |
| melwitt | we'll tune what time of day. I haven't scheduled the nova team photo yet | 22:21 |
| mriedem | subsets of ironic nodes per compute... | 22:21 |
| mriedem | so like a mini az? | 22:21 |
| jroll | mriedem: https://bugs.launchpad.net/tripleo/+bug/1787910/ | 22:21 |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 22:21 |
| jroll | mriedem: I don't think it's a problem for master, as people should be dropping those filters anyway | 22:22 |
| * jroll hasn't heard about ironic node to nova-compute affinity in a while | 22:23 | |
| openstackgerrit | Merged openstack/nova-specs master: Repropose reshaper spec for Stein https://review.openstack.org/592650 | 22:23 |
| jungleboyj | melwitt: hey. Thanks for bringing this up. Then to follow up with you today in the day got away from me. | 22:23 |
| mriedem | jroll: ok, added the bug anyway - would be nice to know what the summary of that issue was | 22:23 |
| jungleboyj | melwitt: Thursday is fine for us. In the past we've done it in the morning but can be flexible. | 22:23 |
| jroll | mriedem: agree, I think the override is fine but the filter is doing something funky | 22:23 |
| mriedem | jroll: was tripleo still using the Exact* filters? | 22:23 |
| jroll | mriedem: I think just regular RamFilter | 22:24 |
| jroll | https://review.openstack.org/#/c/596093/5/environments/undercloud.yaml | 22:24 |
| mriedem | ok, did the flavor values have garbage in them? | 22:24 |
| jroll | they had flavor.memory_mb set, with flavor.properties[whatever:memory_mb]=0 | 22:24 |
| melwitt | mriedem: something like that, I think. just sounded like something good for a discussion to get on the same page and figure out what's the best way to approach it | 22:24 |
| melwitt | I don't know that much about it yet | 22:24 |
| mriedem | jroll: ah ok, so the filter say 0 ram on the HostState, | 22:25 |
| mriedem | and the filter rejected the host | 22:25 |
| mriedem | *saw | 22:25 |
| jroll | right | 22:25 |
| mriedem | i bet our docs on those filters don't say to stop using them... | 22:25 |
| mriedem | https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#ramfilter | 22:26 |
| melwitt | jungleboyj: cool, that's right. on monday I'll schedule our team photo and then guess a time slot for nova/cinder and adjust with you if needed | 22:26 |
| mriedem | nope | 22:26 |
| melwitt | don't you still need them if CachingScheduler? or no? | 22:27 |
| mriedem | yes | 22:28 |
| mriedem | but we should probably say they are deprecated | 22:28 |
| melwitt | so CachingScheduler people can't stop won't stop | 22:28 |
| mriedem | i can hork something out quick to start a discussion | 22:28 |
| mriedem | can't stop won't stop as in puff daddy? | 22:28 |
| melwitt | yeah. (I think?) | 22:29 |
| mriedem | i can't say i have the same answer for the aggregate ram/core/disk filters | 22:29 |
| mriedem | ... | 22:29 |
| melwitt | didn't leakypipes have a spec up about that? aggregate affinity | 22:30 |
| mriedem | yes | 22:31 |
| mriedem | https://review.openstack.org/#/c/544683/ | 22:32 |
| melwitt | ahoy | 22:33 |
| jungleboyj | melwitt: oh. I need to take a look at when we scheduled our picture. Think it's Wednesday. | 22:46 |
| *** gyee has quit IRC | 22:52 | |
| *** itlinux has quit IRC | 22:57 | |
| *** tbachman has quit IRC | 23:00 | |
| melwitt | mriedem: replied to instance group patch questions | 23:11 |
| melwitt | I'll be respinning to add the code comment but will hold off until I know whether to adjust _get_group_details. I thought I have what's there to avoid changing existing behavior as part of the bug fix but maybe I've misunderstood your question | 23:18 |
| *** slagle has joined #openstack-nova | 23:19 | |
| *** r-daneel has quit IRC | 23:24 | |
| melwitt | jungleboyj: ah, yeah. looks like team photos are only tues and wed this time | 23:25 |
| jungleboyj | Okay. So anytime Thursday is good for us then. | 23:27 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Deprecate Core/Ram/DiskFilter https://review.openstack.org/596502 | 23:28 |
| mriedem | melwitt: jroll: bauzas: leakypipes: dansmith: it's time ^ | 23:28 |
| melwitt | jungleboyj: k, let's plan for thursday morning per tradition | 23:29 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Deprecate Core/Ram/DiskFilter https://review.openstack.org/596502 | 23:32 |
| melwitt | mriedem: you wrapped lines on the commit message weirdly short. -1! | 23:32 |
| * mriedem puts it on the tc office hours agenda | 23:33 | |
| melwitt | haha | 23:33 |
| mriedem | i'll a heads up to the ML as well | 23:34 |
| jungleboyj | melwitt: sounds great. Thank you. | 23:38 |
| mriedem | melwitt: so i'm good to +W your setup_instance_group thing now, | 23:39 |
| mriedem | if you want to respin with that missing comment | 23:40 |
| mriedem | the _get_group_details stuff should be cleaned up in stein so we don't risk a regression in a backport | 23:40 |
| melwitt | yeah, let me add the comment since it's going to be backported and a follow on wouldn't get included | 23:40 |
| mriedem | yup just what i was thinking | 23:40 |
| openstackgerrit | melanie witt proposed openstack/nova master: Make scheduler.utils.setup_instance_group query all cells https://review.openstack.org/540258 | 23:45 |
| *** macza has quit IRC | 23:46 | |
| *** gbarros has quit IRC | 23:50 | |
| melwitt | mriedem ^ | 23:56 |
| *** tristanC has quit IRC | 23:56 | |
| mriedem | done | 23:57 |
| melwitt | yess | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!