| *** markvoelker has quit IRC | 00:06 | |
| *** y_sawai has joined #openstack-nova | 00:07 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova: libvirt: add more logging and result checking around attachDeviceFlags https://review.openstack.org/220357 | 00:08 |
|---|---|---|
| *** y_sawai has quit IRC | 00:08 | |
| mriedem | tonyb: added ^ to see if i can recreate with that logging | 00:14 |
| tonyb | mriedem: Cool. | 00:16 |
| *** promulo has joined #openstack-nova | 00:16 | |
| *** hemna is now known as hemnafk | 00:17 | |
| *** sachin_ has quit IRC | 00:20 | |
| mriedem | mtreinish: hmm, i wonder, | 00:20 |
| *** promulo__ has quit IRC | 00:20 | |
| *** shoutm has quit IRC | 00:20 | |
| mriedem | in test_stamp_pattern, when we detach the first volume after creating the timestamp, | 00:20 |
| *** sachin has joined #openstack-nova | 00:20 | |
| mriedem | should we check to see that the vdb partition isn't on the instance before we create a snapshot of that instance | 00:20 |
| mriedem | since i'm guessing the detach is aysnc with libvirt | 00:20 |
| mriedem | mtreinish: isn't there a boto test similar to that - does an attach and checks that partition list changes, then detaches and checks that partition list changes again? | 00:21 |
| *** browne has quit IRC | 00:21 | |
| oomichi | alex_xu: good morning! | 00:21 |
| oomichi | alex_xu: can we talk about https://review.openstack.org/#/c/217727? | 00:22 |
| mriedem | this http://git.openstack.org/cgit/openstack/tempest/tree/tempest/thirdparty/boto/test_ec2_instance_run.py#n228 | 00:22 |
| *** shoutm has joined #openstack-nova | 00:23 | |
| *** diogogmt_ has joined #openstack-nova | 00:26 | |
| *** thorst has quit IRC | 00:27 | |
| *** diogogmt has quit IRC | 00:27 | |
| *** diogogmt_ is now known as diogogmt | 00:27 | |
| mtreinish | mriedem: if it's async with libvirt shouldn't it be async in the nova api? | 00:28 |
| mtreinish | or cinder I guess? | 00:28 |
| mriedem | mtreinish: the nova virt driver does the attach/detach which is async in libvirt, | 00:28 |
| mriedem | then once it comes out of the virt driver it calls the cinder api to update the database | 00:29 |
| mriedem | cinder doesn't actually do anything with volume attach for iscsi/lvm | 00:29 |
| mriedem | just updates it's state | 00:29 |
| mriedem | which is what tempest is waiting for - for the volume status to change from in-use to available | 00:29 |
| mriedem | but that doesn't mean the guest is really ready | 00:29 |
| *** VW_ has quit IRC | 00:30 | |
| *** VW_ has joined #openstack-nova | 00:31 | |
| mtreinish | mriedem: fwiw, yes I think there is a boto test that does that, unless we started skipping it | 00:31 |
| mtreinish | mriedem: hmm, well I'm not sure logging into the guest to verify that things are really ready is a pattern you really want to say is needed | 00:31 |
| *** shoutm_ has joined #openstack-nova | 00:34 | |
| *** VW_ has quit IRC | 00:36 | |
| *** shoutm has quit IRC | 00:36 | |
| mriedem | heh, yeah... | 00:43 |
| mriedem | would be nice if we could make this synchronous on the virt driver side | 00:43 |
| mriedem | that boto test above does do this, | 00:44 |
| mriedem | but it doesn't check the specific device because you have to specify one with boto but you're not gauranteed to get that with libvirt | 00:44 |
| *** pixelbeat has quit IRC | 00:48 | |
| *** ijuwang has quit IRC | 00:51 | |
| mriedem | mtreinish: yeah so my theory is the partition isn't gone when we do the volume snapshot, | 00:53 |
| mriedem | and then when we attach that the 2nd time, libvirt detects it's already there or something and punts - or thinks it's there but it's not | 00:53 |
| mriedem | something like that | 00:53 |
| mtreinish | mriedem: I could see that happening, but we really need to be syncronizing these events somehow | 00:54 |
| mriedem | i would still expect some debug log message from libvirt in that case | 00:54 |
| mtreinish | because that's just crazy | 00:54 |
| mriedem | yeah, | 00:54 |
| mriedem | but so was test_attach_detach | 00:54 |
| mriedem | where it would immediately detach as soon as the volume was attached and we had a race updating the nova db | 00:54 |
| mriedem | like within 1 second | 00:54 |
| mtreinish | mriedem: heh, yeah that was a crappy test | 00:54 |
| mriedem | it's still there :) | 00:55 |
| mtreinish | did we fix the race? | 00:55 |
| mtreinish | it's all a blur to me | 00:55 |
| cburgess | Does anyone know if we have a way to block nova from launching instances that try and use nova storage? AKA only allow boot from volume? | 00:56 |
| mriedem | mtreinish: yeah it's fixed | 00:57 |
| *** SourabhP has quit IRC | 00:57 | |
| mriedem | cburgess: not sure, but we actually had that case for about a month in our internal dev cloud | 00:57 |
| mriedem | where we could only use volumes | 00:57 |
| cburgess | Yeah obviously you still need some local storage for things like console logs image cache etc. | 00:58 |
| cburgess | But I'm struggling to find a way that would actually prevent a non-bfv instance from spawning. | 00:58 |
| mriedem | in our case the spawn would just fail once it landed on a compute node | 00:58 |
| mtreinish | cburgess: iptables block glance? | 00:59 |
| cburgess | I thought there was a bfv work flow that had nova download the image and populate the volume? | 00:59 |
| mtreinish | cburgess: fwiw, I think there is an option for that | 00:59 |
| mriedem | there is | 01:00 |
| cburgess | Option for what? | 01:00 |
| mtreinish | to only boot from volume | 01:00 |
| mriedem | bfv source=image dest=volume | 01:00 |
| mtreinish | mriedem: no I meant like a nova conf setup that did it | 01:00 |
| mriedem | https://github.com/openstack/nova/blob/master/nova/virt/block_device.py#L349 | 01:01 |
| mriedem | yeah | 01:01 |
| mriedem | would have to look for a config option | 01:01 |
| mriedem | seems super easy to implement | 01:01 |
| mtreinish | I vaguely remember someone filing a tempest bug that they had that set and something didn't work | 01:01 |
| cburgess | Right yeah I know can get the nova compute node to setup the volume for me. | 01:01 |
| cburgess | Was hoping we had the config flag. | 01:02 |
| mriedem | cburgess: mtreinish: max_local_block_devices=0 | 01:02 |
| *** mtanino has quit IRC | 01:02 | |
| cburgess | OK REALLY | 01:02 |
| mriedem | (IntOpt) Maximum number of devices that will result in a local image being created on the hypervisor node. Setting this to 0 means nova will allow only boot from volume. A negative number means unlimited. | 01:02 |
| mtreinish | cburgess: https://bugs.launchpad.net/tempest/+bug/1436314 | 01:02 |
| openstack | Launchpad bug 1436314 in tempest "Option to boot VM only from volume is not available" [Medium,Confirmed] - Assigned to Soumit (soumit-mishra) | 01:02 |
| cburgess | HAWT! | 01:02 |
| mtreinish | max_local_block_device | 01:02 |
| cburgess | Where is that option defined? | 01:04 |
| *** armax has quit IRC | 01:04 | |
| *** vilobhmm_11 has quit IRC | 01:04 | |
| *** tpeoples has joined #openstack-nova | 01:04 | |
| mriedem | yar https://github.com/openstack/nova/blob/master/nova/compute/api.py#L113 | 01:04 |
| cburgess | Awesome. | 01:04 |
| cburgess | Thanks | 01:04 |
| mriedem | used here https://github.com/openstack/nova/blob/master/nova/compute/api.py#L1349 | 01:05 |
| *** armax has joined #openstack-nova | 01:05 | |
| mriedem | ALTHOUGH | 01:05 |
| mriedem | HOWEVER | 01:05 |
| mriedem | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L1460 | 01:05 |
| mriedem | that's only called from _validate_bdm | 01:05 |
| mriedem | so if you don't boot with block_device_mapping, it seems that wouldn't be checked | 01:05 |
| *** bauzas has quit IRC | 01:06 | |
| *** markvoelker has joined #openstack-nova | 01:07 | |
| mriedem | that bug has a good explanation from ndipanov | 01:07 |
| *** unicell has joined #openstack-nova | 01:07 | |
| mriedem | ndipanov suggested setting root_gb=0 in the flavor but when he tested that it didn't do as expected | 01:08 |
| *** markvoelker_ has joined #openstack-nova | 01:08 | |
| *** unicell1 has joined #openstack-nova | 01:09 | |
| mriedem | we should update the help text on the option since "Setting this to 0 means nova will allow only boot from volume." is clearly wrong | 01:10 |
| *** y_sawai has joined #openstack-nova | 01:11 | |
| *** derekh has quit IRC | 01:11 | |
| *** unicell has quit IRC | 01:11 | |
| *** markvoelker has quit IRC | 01:12 | |
| *** y_sawai has quit IRC | 01:13 | |
| *** husanu has quit IRC | 01:14 | |
| *** tri2sing has quit IRC | 01:14 | |
| *** markvoelker_ has quit IRC | 01:16 | |
| *** husanu has joined #openstack-nova | 01:16 | |
| *** unicell has joined #openstack-nova | 01:18 | |
| *** unicell1 has quit IRC | 01:20 | |
| cburgess | Oh wow.. thats lame. | 01:20 |
| cburgess | Yeah a 0 size flavor means something different. | 01:20 |
| cburgess | it makes no sense that we would only check that value when we boot with a volume. | 01:21 |
| cburgess | Thats a bug. | 01:21 |
| cburgess | Wow... why would we check it in that function... | 01:22 |
| *** Sukhdev_ has quit IRC | 01:22 | |
| *** husanu has quit IRC | 01:25 | |
| *** baoli has joined #openstack-nova | 01:26 | |
| *** husanu has joined #openstack-nova | 01:27 | |
| *** armax has quit IRC | 01:27 | |
| *** hhuang has joined #openstack-nova | 01:30 | |
| *** y_sawai has joined #openstack-nova | 01:32 | |
| *** hhuang has quit IRC | 01:32 | |
| *** y_sawai has quit IRC | 01:35 | |
| *** lyanchih has joined #openstack-nova | 01:35 | |
| *** yamahata has quit IRC | 01:37 | |
| *** sachin has quit IRC | 01:40 | |
| *** sachin has joined #openstack-nova | 01:42 | |
| *** unicell1 has joined #openstack-nova | 01:42 | |
| *** unicell has quit IRC | 01:43 | |
| *** sachin has quit IRC | 01:43 | |
| *** sachin has joined #openstack-nova | 01:44 | |
| *** sachin has quit IRC | 01:44 | |
| *** sachin has joined #openstack-nova | 01:44 | |
| *** shoutm_ has quit IRC | 01:45 | |
| mriedem | updated help text coming | 01:45 |
| openstackgerrit | Matt Riedemann proposed openstack/nova: Clarify max_local_block_devices config option usage https://review.openstack.org/220375 | 01:45 |
| *** xyang1 has quit IRC | 01:47 | |
| *** y_sawai has joined #openstack-nova | 01:48 | |
| *** y_sawai has quit IRC | 01:48 | |
| mriedem | tonyb: i'll be damned | 01:51 |
| mriedem | i think that kwargs thing might have been it | 01:51 |
| mriedem | but i have no idea why it would only fail on that one test i'm running | 01:52 |
| mriedem | this time the test passed | 01:52 |
| mriedem | on all master jobs, failed in kilo and juno | 01:52 |
| * tonyb is scared *very* scared. | 01:52 | |
| tonyb | mriedem: Do you have the logs? | 01:53 |
| tonyb | mriedem: or was it a private run? | 01:53 |
| mriedem | http://logs.openstack.org/55/218355/7/check/gate-tempest-dsvm-full/3a71d84/ | 01:53 |
| *** sachin has quit IRC | 01:53 | |
| mriedem | test_stamp_pattern is the test | 01:53 |
| mriedem | TestStampPattern shows up in n-cpu | 01:53 |
| mriedem | http://logs.openstack.org/55/218355/7/check/gate-tempest-dsvm-full/3a71d84/logs/screen-n-cpu.txt.gz#_2015-09-04_01_07_20_637 | 01:54 |
| *** sachin has joined #openstack-nova | 01:54 | |
| mriedem | i should update the nova change to do flags=flags again and see if it fails | 01:55 |
| *** shoutm has joined #openstack-nova | 01:57 | |
| *** jmccrory has quit IRC | 01:58 | |
| tonyb | mriedem: okay. I'm not super convinced by my analysis so that'd be good. | 01:58 |
| mriedem | there are actually several domain.method() calls in the nova.virt.libvirt.guest module that do flags as kwargs | 01:59 |
| mriedem | anyway, rechecking | 01:59 |
| mriedem | gonna call it a night | 02:00 |
| *** jmccrory has joined #openstack-nova | 02:00 | |
| *** dims has quit IRC | 02:01 | |
| *** annegentle has quit IRC | 02:01 | |
| *** bauzas has joined #openstack-nova | 02:01 | |
| *** sachin has quit IRC | 02:02 | |
| *** sachin has joined #openstack-nova | 02:02 | |
| *** penick has joined #openstack-nova | 02:07 | |
| openstackgerrit | Dan Smith proposed openstack/nova: Pull out the common bits of InstanceList into _BaseInstanceList https://review.openstack.org/219337 | 02:07 |
| openstackgerrit | Dan Smith proposed openstack/nova: Update objects test infrastructure for multiple versions https://review.openstack.org/217818 | 02:07 |
| openstackgerrit | Dan Smith proposed openstack/nova: Add Instance and InstanceList v2.0 objects https://review.openstack.org/217819 | 02:07 |
| openstackgerrit | Dan Smith proposed openstack/nova: Fix an issue with NovaObjectRegistry hook https://review.openstack.org/217816 | 02:07 |
| openstackgerrit | Dan Smith proposed openstack/nova: Make the conductor fixture use version manifests https://review.openstack.org/217851 | 02:07 |
| openstackgerrit | Dan Smith proposed openstack/nova: Refactor Instance tests to use objects.Instance https://review.openstack.org/217817 | 02:07 |
| openstackgerrit | Dan Smith proposed openstack/nova: Pull out the common bits of Instance into _BaseInstance https://review.openstack.org/217815 | 02:07 |
| openstackgerrit | Dan Smith proposed openstack/nova: Teach conductor to do manifest-based object_class_action() things https://review.openstack.org/219432 | 02:07 |
| dansmith | mikal: had to rebase ^ :( | 02:07 |
| *** sachin has quit IRC | 02:10 | |
| *** sachin has joined #openstack-nova | 02:11 | |
| *** armax has joined #openstack-nova | 02:11 | |
| *** sachin has quit IRC | 02:13 | |
| *** sachin has joined #openstack-nova | 02:14 | |
| *** yamahata has joined #openstack-nova | 02:14 | |
| *** sachin has quit IRC | 02:14 | |
| *** sachin has joined #openstack-nova | 02:15 | |
| *** sachin has quit IRC | 02:15 | |
| *** sachin has joined #openstack-nova | 02:15 | |
| *** sachin has quit IRC | 02:16 | |
| *** bauzas has quit IRC | 02:19 | |
| *** dims has joined #openstack-nova | 02:20 | |
| *** annegentle has joined #openstack-nova | 02:23 | |
| *** bauzas has joined #openstack-nova | 02:26 | |
| *** y_sawai has joined #openstack-nova | 02:27 | |
| gmann | alex_xu: oomichi : i found away to make schd hint work for v2 comp and v2.1. No change for v2.1 behavior and v2 comp will allow extra param | 02:27 |
| gmann | alex_xu: oomichi : if we just define some dummy item in schd hint schema and based on that we do not delete extra param in validator.py | 02:27 |
| alex_xu | oomichi: gmann , limits the scheduler-hints in the v2.1 api also looks like wrong | 02:27 |
| gmann | alex_xu: oomichi : that worked well | 02:28 |
| gmann | alex_xu: but for v2.1 we should not allow out of tree thing for schd hint right? | 02:28 |
| alex_xu | gmann: currently we support out-of-tree filter | 02:28 |
| *** y_sawai has quit IRC | 02:29 | |
| gmann | alex_xu: for v2.1 also? | 02:30 |
| alex_xu | gmann: yea | 02:30 |
| oomichi | alex_xu: gmann: just IMO: the change of the base part(validator.py) just for out-of-tree seems mess for maintenance in long-term | 02:30 |
| oomichi | alex_xu: gmann: I can accept to allow additionalProperties on specific schema for v2.1 also for avoiding such change | 02:31 |
| openstackgerrit | lyanchih proposed openstack/nova: libvirt - Flavor's quota instance resource not observed for non-ephemeral storage https://review.openstack.org/201019 | 02:32 |
| alex_xu | oomichi: the base part if for v2.1 compat mode | 02:32 |
| oomichi | alex_xu: gmann: Ideally, each out-of-tree owners should change schemas also on their own private patches, but it is difficult to do that, I can | 02:32 |
| alex_xu | oomichi: we use custom validation for additionalProperties in v2.1 compat mode | 02:32 |
| alex_xu | oomichi: the _soft_validate_additional_properties didn't check additionalProperties is True or False, that patch is going to fix that. | 02:33 |
| gmann | oomichi: alex_xu : for me allowing additional prop for v2.1 does not seems good it should be just for v2 comp. | 02:33 |
| oomichi | alex_xu: ? you are changing v2.1 schema also on the latest | 02:34 |
| alex_xu | oomichi: yes, that patch did that | 02:34 |
| gmann | because if we do so then it going to be always provide such out of tree things handling | 02:34 |
| *** markvoelker has joined #openstack-nova | 02:35 | |
| *** penick has quit IRC | 02:36 | |
| gmann | oomichi: alex_xu : becasue from johnthetubaguy comment on, i understood we can disallow that on v2.1 - https://review.openstack.org/#/c/217727/ | 02:36 |
| alex_xu | oomichi: gmann I remember there is discussion about scheduler-hints, I remember finally say we should allow out of tree hints | 02:37 |
| oomichi | gmann: yeah, I know. but I'd like to avoid trick/unreadable code only for his customers | 02:38 |
| oomichi | gmann: in-tree | 02:38 |
| oomichi | alex_xu: do you have any log about that? | 02:39 |
| alex_xu | oomichi: gmann http://lists.openstack.org/pipermail/openstack-dev/2015-June/067996.html | 02:39 |
| alex_xu | oomichi: gmann I talk with bauzas last night, he said finally they didn't make strict schema in Nova object | 02:39 |
| oomichi | alex_xu: thanks :) I can accept if the NOTE includes the link also | 02:41 |
| alex_xu | oomichi: ok cool... | 02:41 |
| alex_xu | gmann: what would you think? | 02:41 |
| gmann | alex_xu: reading... :) | 02:42 |
| oomichi | alex_xu: btw, are the other file changes also still necessary? | 02:42 |
| alex_xu | oomichi: yea, it is necessary | 02:42 |
| alex_xu | oomichi: otherwise _soft_validate_additional_properties always strip the extra parameters whatever addtionalProperties is True or False | 02:43 |
| alex_xu | gmann: let me know your opinion :) | 02:43 |
| oomichi | alex_xu: OK, I will see it carefully. thanks :) | 02:44 |
| alex_xu | oomichi: np, thanks for review and discussion :) | 02:44 |
| gmann | alex_xu: yea, m ok on now. i agree there we should not have schd hint thing on schema but thats later things we can decide. | 02:46 |
| alex_xu | gmann: thanks | 02:46 |
| alex_xu | oomichi: gmann thanks, I will update the commit message later | 02:46 |
| gmann | alex_xu: for now it works . Thanks for link :) it will be help if we add that in commit msg. | 02:46 |
| gmann | alex_xu: 1 question does _soft_validate_additional_properties will be called if ap is true ? | 02:47 |
| gmann | * gmann checking on json schema | 02:47 |
| alex_xu | gmann: yes, I will be called | 02:47 |
| alex_xu | gmann: I remember I tested it | 02:47 |
| *** bswartz has quit IRC | 02:49 | |
| *** markvoelker has quit IRC | 02:49 | |
| *** dims has quit IRC | 02:49 | |
| *** bswartz has joined #openstack-nova | 02:50 | |
| *** markvoelker has joined #openstack-nova | 02:50 | |
| gmann | alex_xu: yea they call it, but json schema should not call as it is said to be skip by defined schema but json schema lib thing :) | 02:51 |
| *** armax has quit IRC | 02:51 | |
| oomichi | alex_xu, gmann: I will send a mail for getting a consensus about this by replying http://lists.openstack.org/pipermail/openstack-dev/2015-June/067996.html | 02:52 |
| *** obondarev has quit IRC | 02:52 | |
| *** obondarev has joined #openstack-nova | 02:53 | |
| *** hakimo_ has joined #openstack-nova | 02:53 | |
| alex_xu | gmann: ok, cool | 02:53 |
| alex_xu | oomichi: thanks | 02:53 |
| gmann | oomichi: Thanks | 02:53 |
| *** hakimo has quit IRC | 02:55 | |
| *** deepthi has joined #openstack-nova | 02:57 | |
| openstackgerrit | Alex Xu proposed openstack/nova: Relax restrictions on server name https://review.openstack.org/220279 | 03:05 |
| *** markvoelker has quit IRC | 03:06 | |
| mriedem | sdague: johnthetubaguy: for when you're around, novaclient 2.28 is out https://pypi.python.org/pypi/python-novaclient/2.28.0 | 03:07 |
| mriedem | not sure if we need to make that the minimum required version in g-r for liberty now or not | 03:07 |
| mriedem | seems like we should... | 03:07 |
| *** darrenc has joined #openstack-nova | 03:08 | |
| alex_xu | mriedem: ^ just update the patch | 03:11 |
| *** openstackgerrit_ has joined #openstack-nova | 03:11 | |
| *** atuvenie has quit IRC | 03:11 | |
| *** venkat_p has joined #openstack-nova | 03:12 | |
| mikal | dansmith: mutter | 03:15 |
| mikal | dansmith: I will take another look through them in a little bit | 03:15 |
| dansmith | mikal: thanks | 03:15 |
| mikal | dansmith: NP | 03:15 |
| mikal | dansmith: need to have a post run stretch / shower first though | 03:16 |
| mikal | dansmith: no one wants a sweaty code review | 03:16 |
| *** yamahata has quit IRC | 03:16 | |
| *** Sree has joined #openstack-nova | 03:16 | |
| *** yamahata has joined #openstack-nova | 03:16 | |
| openstackgerrit | Alex Xu proposed openstack/nova: api: allow any scheduler hints https://review.openstack.org/217727 | 03:16 |
| alex_xu | oomichi: gmann & | 03:16 |
| alex_xu | s/&/^... | 03:16 |
| *** dims_ has joined #openstack-nova | 03:17 | |
| *** browne has joined #openstack-nova | 03:17 | |
| *** tojuvone has joined #openstack-nova | 03:18 | |
| mriedem | alex_xu: ok, i'll defer to sdague about what to do with the spaces at beginning and end of names | 03:18 |
| mriedem | i agree it seems pointless to have those, but the backwards compat thing is a concern | 03:19 |
| alex_xu | mriedem: ok, got it, thanks | 03:19 |
| alex_xu | mriedem: yeah | 03:19 |
| oomichi | alex_xu: thanks, just one comment | 03:20 |
| oomichi | alex_xu: and I sent a mail with one idea | 03:20 |
| mriedem | alex_xu: do we have any tests with a unicode server name? | 03:20 |
| oomichi | alex_xu: but that is long-term feature, and I can agree with current way | 03:20 |
| *** dave-mccowan has quit IRC | 03:20 | |
| alex_xu | oomichi: thanks, will update soon | 03:20 |
| alex_xu | mriedem: I guess not | 03:21 |
| oomichi | alex_xu: in short-term | 03:21 |
| alex_xu | mriedem: it is not. I didn't find out anything aobut unicode in test_servers | 03:21 |
| openstackgerrit | Alex Xu proposed openstack/nova: api: allow any scheduler hints https://review.openstack.org/217727 | 03:24 |
| *** comstud has quit IRC | 03:28 | |
| *** adam_g has quit IRC | 03:28 | |
| *** jroll has quit IRC | 03:28 | |
| *** d34dh0r53 has quit IRC | 03:28 | |
| *** bswartz has quit IRC | 03:28 | |
| *** flwang1 has quit IRC | 03:28 | |
| *** lbragstad has quit IRC | 03:28 | |
| *** gus has quit IRC | 03:28 | |
| *** Nakato has quit IRC | 03:28 | |
| *** rook has quit IRC | 03:29 | |
| *** apmelton has quit IRC | 03:29 | |
| *** Nakato has joined #openstack-nova | 03:30 | |
| *** gus has joined #openstack-nova | 03:31 | |
| *** adam_g has joined #openstack-nova | 03:32 | |
| *** adam_g has quit IRC | 03:32 | |
| *** adam_g has joined #openstack-nova | 03:32 | |
| *** apmelton has joined #openstack-nova | 03:33 | |
| *** d34dh0r53 has joined #openstack-nova | 03:33 | |
| *** lbragstad has joined #openstack-nova | 03:33 | |
| *** mriedem has quit IRC | 03:33 | |
| *** comstud has joined #openstack-nova | 03:34 | |
| *** bswartz has joined #openstack-nova | 03:34 | |
| *** jroll has joined #openstack-nova | 03:34 | |
| *** flwang1 has joined #openstack-nova | 03:34 | |
| *** erhudy1 has quit IRC | 03:42 | |
| *** armax has joined #openstack-nova | 03:44 | |
| *** VikasC has joined #openstack-nova | 03:45 | |
| openstackgerrit | Alex Xu proposed openstack/nova: Reject the cell name include '!' and '.' https://review.openstack.org/220386 | 03:49 |
| *** markvoelker has joined #openstack-nova | 03:53 | |
| *** jamespd has quit IRC | 03:56 | |
| * alex_xu back to vacation... | 03:56 | |
| *** jamespd has joined #openstack-nova | 03:56 | |
| *** markvoelker_ has joined #openstack-nova | 03:57 | |
| *** markvoelker has quit IRC | 03:58 | |
| *** dannywilson has joined #openstack-nova | 03:58 | |
| *** markvoelker_ has quit IRC | 03:58 | |
| *** rajesht has joined #openstack-nova | 04:03 | |
| *** tri2sing has joined #openstack-nova | 04:04 | |
| *** jamespd has quit IRC | 04:04 | |
| *** jamespd has joined #openstack-nova | 04:05 | |
| *** lyanchih has quit IRC | 04:06 | |
| *** dims_ has quit IRC | 04:09 | |
| rajesht | jaypipes: could you please review it. | 04:09 |
| rajesht | jaypipes: https://review.openstack.org/#/c/170118/ | 04:09 |
| rajesht | jaypipes: need one more +2 before it gets into merge conflict again :) | 04:10 |
| rajesht | jaypipes: thanks :) | 04:10 |
| *** dims has joined #openstack-nova | 04:10 | |
| *** links has joined #openstack-nova | 04:12 | |
| *** openstackgerrit_ has quit IRC | 04:14 | |
| jaypipes | rajesht: reviewed... | 04:16 |
| jaypipes | rajesht: I have a feeling it can be simpler than you've written it. | 04:16 |
| *** dims has quit IRC | 04:17 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova: Updated from global requirements https://review.openstack.org/216968 | 04:17 |
| rajesht | jaypipes: I have replied to your comment | 04:21 |
| rajesht | jaypipes: it won't work, bcz we don't want to update in_use and reserved quota all the time. | 04:22 |
| rajesht | jaypipes: we only want to update one or the other | 04:22 |
| rajesht | jaypipes: thanks for review | 04:22 |
| jaypipes | uhg this code is so friggin spaghetti. | 04:27 |
| jaypipes | rajesht: I really don't like changing the _create_reservations() function like this... | 04:31 |
| jaypipes | rajesht: the logic about checking if a migration is being done should be in the _init_instance() method. as this code stands, it's incredibly difficult to follow | 04:32 |
| gmann | alex_xu: looks good. Thanks | 04:46 |
| *** deepthi has quit IRC | 04:47 | |
| *** cfriesen has quit IRC | 04:48 | |
| *** deepakcs has joined #openstack-nova | 04:49 | |
| *** lyanchih has joined #openstack-nova | 04:50 | |
| *** annegentle has quit IRC | 04:56 | |
| *** hparekh has joined #openstack-nova | 04:57 | |
| *** shoutm has quit IRC | 04:57 | |
| *** haomaiwa_ has quit IRC | 04:57 | |
| *** shoutm has joined #openstack-nova | 04:58 | |
| *** chhavi has joined #openstack-nova | 05:00 | |
| *** VikasC has quit IRC | 05:12 | |
| *** tri2sing has quit IRC | 05:16 | |
| *** bkopilov has quit IRC | 05:19 | |
| *** vilobhmm_11 has joined #openstack-nova | 05:19 | |
| *** tri2sing has joined #openstack-nova | 05:20 | |
| *** oomichi has quit IRC | 05:21 | |
| *** dannywilson has quit IRC | 05:22 | |
| *** baoli has quit IRC | 05:23 | |
| *** vilobhmm_11 has quit IRC | 05:25 | |
| *** bkopilov has joined #openstack-nova | 05:30 | |
| *** derekh has joined #openstack-nova | 05:30 | |
| *** claudiub has quit IRC | 05:31 | |
| *** jamielennox is now known as jamielennox|away | 05:35 | |
| openstackgerrit | Tomi Juvonen proposed openstack/nova-specs: Get valid server state https://review.openstack.org/192246 | 05:35 |
| *** husanu has quit IRC | 05:36 | |
| *** husanu has joined #openstack-nova | 05:38 | |
| *** derekh has quit IRC | 05:41 | |
| *** sahid has joined #openstack-nova | 05:42 | |
| *** y_sawai has joined #openstack-nova | 05:46 | |
| *** e0ne has joined #openstack-nova | 05:49 | |
| openstackgerrit | Tomi Juvonen proposed openstack/nova-specs: Get valid server state https://review.openstack.org/192246 | 05:53 |
| *** e0ne has quit IRC | 05:54 | |
| *** jamielennox|away is now known as jamielennox | 05:58 | |
| *** nkrinner has joined #openstack-nova | 05:58 | |
| *** ishant has joined #openstack-nova | 05:59 | |
| *** ttrumm has joined #openstack-nova | 06:09 | |
| *** ttrumm_ has joined #openstack-nova | 06:11 | |
| *** dannywilson has joined #openstack-nova | 06:12 | |
| *** ttrumm has quit IRC | 06:13 | |
| *** VikasC has joined #openstack-nova | 06:15 | |
| *** dannywilson has quit IRC | 06:17 | |
| *** hparekh has quit IRC | 06:17 | |
| *** mpaolino has joined #openstack-nova | 06:18 | |
| *** vilobhmm_11 has joined #openstack-nova | 06:18 | |
| *** hparekh has joined #openstack-nova | 06:18 | |
| *** tri2sing has quit IRC | 06:19 | |
| *** deepthi has joined #openstack-nova | 06:22 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova: Imported Translations from Transifex https://review.openstack.org/212367 | 06:25 |
| *** y_sawai has quit IRC | 06:26 | |
| openstackgerrit | garyk proposed openstack/nova: VMware: ensure that instance is deleted https://review.openstack.org/220410 | 06:30 |
| *** ttrumm_ has quit IRC | 06:30 | |
| *** ankit_ag has joined #openstack-nova | 06:32 | |
| *** Murali_ has joined #openstack-nova | 06:34 | |
| *** ildikov has quit IRC | 06:35 | |
| *** tpeoples has quit IRC | 06:39 | |
| *** atuvenie has joined #openstack-nova | 06:42 | |
| *** husanu has quit IRC | 06:42 | |
| *** vilobhmm_11 has quit IRC | 06:44 | |
| *** husanu has joined #openstack-nova | 06:44 | |
| *** ociuhandu has quit IRC | 06:44 | |
| *** Sree has quit IRC | 06:46 | |
| *** gabriel-bezerra has quit IRC | 06:49 | |
| *** Sree has joined #openstack-nova | 06:51 | |
| *** Sree has quit IRC | 06:53 | |
| *** atuvenie has quit IRC | 06:55 | |
| *** abhishekk has joined #openstack-nova | 06:57 | |
| *** romainh has joined #openstack-nova | 06:58 | |
| openstackgerrit | Joe Cropper proposed openstack/nova: Ensure Nova metrics use coherent data values https://review.openstack.org/219153 | 07:02 |
| jwcroppe | bauzas: ^^^ | 07:03 |
| *** chhavi has quit IRC | 07:04 | |
| *** whenry has quit IRC | 07:06 | |
| *** Piet has quit IRC | 07:06 | |
| jwcroppe | jaypipes: ^^^ | 07:06 |
| *** jlanoux has joined #openstack-nova | 07:06 | |
| *** lpetrut has joined #openstack-nova | 07:07 | |
| *** jamielennox is now known as jamielennox|away | 07:08 | |
| *** alexschm has joined #openstack-nova | 07:11 | |
| *** chhavi has joined #openstack-nova | 07:12 | |
| *** matrohon has joined #openstack-nova | 07:17 | |
| *** shoutm_ has joined #openstack-nova | 07:19 | |
| *** browne has quit IRC | 07:19 | |
| alexschm | leseb: Hi, are you going to work on the blueprint "Improve Nova KVM IO support" for Mitaka? | 07:19 |
| *** whenry has joined #openstack-nova | 07:20 | |
| *** shoutm has quit IRC | 07:21 | |
| *** stefan_amann has joined #openstack-nova | 07:23 | |
| *** lyanchih has quit IRC | 07:24 | |
| *** ildikov has joined #openstack-nova | 07:26 | |
| *** oomichi has joined #openstack-nova | 07:26 | |
| *** oomichi has quit IRC | 07:26 | |
| openstackgerrit | Joe Cropper proposed openstack/nova: Ensure Nova metrics use coherent data values https://review.openstack.org/219153 | 07:28 |
| *** rdopiera has joined #openstack-nova | 07:31 | |
| *** jamielennox|away is now known as jamielennox | 07:32 | |
| *** rajesht has quit IRC | 07:34 | |
| *** rdopiera has quit IRC | 07:34 | |
| *** lyanchih has joined #openstack-nova | 07:34 | |
| *** rdopiera has joined #openstack-nova | 07:34 | |
| *** rdopiera has quit IRC | 07:34 | |
| *** rdopiera has joined #openstack-nova | 07:35 | |
| openstackgerrit | Joe Cropper proposed openstack/nova: Ensure Nova metrics use coherent data values https://review.openstack.org/219153 | 07:35 |
| openstackgerrit | Joe Cropper proposed openstack/nova: Ensure Nova metrics use coherent data values https://review.openstack.org/219153 | 07:41 |
| *** armax has quit IRC | 07:44 | |
| *** karimb has joined #openstack-nova | 07:47 | |
| *** Marga_ has joined #openstack-nova | 07:49 | |
| *** Marga_ has quit IRC | 07:49 | |
| *** Marga_ has joined #openstack-nova | 07:49 | |
| openstackgerrit | Joe Cropper proposed openstack/nova: Ensure Nova metrics use coherent data values https://review.openstack.org/219153 | 07:53 |
| *** afazekas__ has joined #openstack-nova | 07:53 | |
| *** e0ne has joined #openstack-nova | 07:55 | |
| *** yassine__ has joined #openstack-nova | 07:57 | |
| openstackgerrit | Joe Cropper proposed openstack/nova: Ensure Nova metrics use coherent data values https://review.openstack.org/219153 | 08:01 |
| *** atuvenie has joined #openstack-nova | 08:04 | |
| *** danpb has joined #openstack-nova | 08:05 | |
| *** Sree has joined #openstack-nova | 08:07 | |
| *** derekh has joined #openstack-nova | 08:07 | |
| *** derekh has quit IRC | 08:08 | |
| *** derekh has joined #openstack-nova | 08:08 | |
| *** lyanchih has quit IRC | 08:09 | |
| *** Marga_ has quit IRC | 08:11 | |
| *** Marga_ has joined #openstack-nova | 08:11 | |
| *** lucasagomes has joined #openstack-nova | 08:12 | |
| *** Sree has quit IRC | 08:12 | |
| *** shoutm_ has quit IRC | 08:15 | |
| *** sayali has joined #openstack-nova | 08:16 | |
| tobasco | after upgrade nova now spits out "IncompatibleObjectVersion: Version 1.2 of PciDeviceList is not supported" | 08:20 |
| tobasco | anybody seen it | 08:20 |
| tobasco | seems like it affect the conductor and compute | 08:20 |
| tobasco | stable kilo | 08:20 |
| tobasco | fixed by restarting the server running the conductor service.. however the same fix is not directly optimal for a compute server... | 08:24 |
| *** Marga_ has quit IRC | 08:25 | |
| leseb_ | alexschm: probably not | 08:26 |
| johnthetubaguy | tobasco: thats does ring a bell, not sure if the usage folks in #openstack will be able to to help more | 08:26 |
| leseb_ | alexschm: why? want to pick it up? | 08:26 |
| bauzas | tobasco: did you upgrade your conductor first ? | 08:26 |
| *** pixelbeat has joined #openstack-nova | 08:26 | |
| bauzas | tobasco: and restarting it before upgrading the compute nodes ? | 08:26 |
| johnthetubaguy | bauzas: good point, you need to upgrade all of the control plane (API and conductor) before doing the compute nodes | 08:27 |
| alexschm | leseb_ yes, I'd like to work on it to adopt qemu iothreads | 08:27 |
| *** paul-carlton has joined #openstack-nova | 08:27 | |
| tobasco | bauzas: actually i did the compute node first because i originally only wanted the cinder-volume upgrade but took all updates | 08:27 |
| *** shoutm has joined #openstack-nova | 08:27 | |
| *** wimdc has joined #openstack-nova | 08:27 | |
| bauzas | tobasco: okay, but it's not supported then | 08:27 |
| leseb_ | alexschm: cool :) | 08:28 |
| bauzas | tobasco: Nova supports upgrades like johnthetubaguy said | 08:28 |
| johnthetubaguy | tobasco: so thats why it failed, it should have been fine on stable, but I think kilo stable broke that | 08:28 |
| alexschm | leseb_ do you know anyone else who would work on this? or should I start a new blueprint? | 08:28 |
| jwcroppe | bauzas: hey...about to head to bed, but do you have a sec to chat about https://review.openstack.org/#/c/219153/ ? This is the metrics thing we discussed the other day | 08:28 |
| bauzas | jwcroppe: hey, night owl you too ? :) | 08:28 |
| tobasco | bauzas: i tried upgrading a second compute node now... after conductor has been upgraded.. and still the same issue | 08:28 |
| jwcroppe | bauzas: you know it | 08:28 |
| bauzas | jwcroppe: sorry I was afk until bow :) | 08:29 |
| bauzas | now evne | 08:29 |
| bauzas | even even | 08:29 |
| jwcroppe | bauzas: sleep is for the weak | 08:29 |
| * bauzas needs to caffeinate | 08:29 | |
| leseb_ | alexschm: AFAIK no one is working on this, I can assign it to you if you want | 08:29 |
| bauzas | jwcroppe: well, I appreciate working on my house in the early hours | 08:29 |
| bauzas | tobasco: stack trace? | 08:29 |
| alexschm | leseb_ I'd like to follow the way that was suggested in the reviews in your blueprint (letting nova enable iothreads transparently) | 08:30 |
| bauzas | tobasco: tbc, you said you're running stable kilo right ? | 08:30 |
| *** boris-42 has quit IRC | 08:30 | |
| alexschm | leseb_ yeah that would be great | 08:30 |
| jwcroppe | bauzas: I think this patch is better now. I also removed that crappy 1 second code | 08:30 |
| bauzas | tobasco: so you updated your computes to master, right? | 08:30 |
| bauzas | jwcroppe: yeah, quickly saw that | 08:30 |
| leseb_ | alexschm: oh wait not sure if we don't have a patch already hanging around let have a look | 08:31 |
| bauzas | jwcroppe: so, I need to review it | 08:31 |
| jwcroppe | bauzas: basically the idea is we now tell the metric collector whether we want it to refresh or not. | 08:31 |
| tobasco | bauzas: stack trace from conductor = http://pastebin.com/bGFsSAeh | 08:31 |
| tobasco | bauzas: stack trace from compute = http://pastebin.com/6rUz82xz | 08:31 |
| bauzas | tobasco: compute = master, right? | 08:31 |
| tobasco | bauzas: running on ubuntu stable kilo cloud-archive repo | 08:31 |
| bauzas | tobasco: ah | 08:31 |
| tobasco | deb http://ubuntu-cloud.archive.canonical.com/ubuntu trusty-updates/kilo main | 08:32 |
| bauzas | jwcroppe: okay, I got your idea, I just need time for thinking about that :) | 08:32 |
| jwcroppe | bauzas: no problem, will let you review. Just wanted to go over high level idea with you. | 08:32 |
| *** alex_klimov has joined #openstack-nova | 08:32 | |
| *** moshele has joined #openstack-nova | 08:32 | |
| bauzas | jwcroppe: honestly, I'm not yet enough caffeinated for that :D | 08:32 |
| tobasco | bauzas: aren't stable suppose to not break updates? ;) | 08:32 |
| jwcroppe | bauzas: it was with great pleasure that I hit the 'delete' key on the 1 second thing too :) | 08:32 |
| bauzas | tobasco: well, what I'm still missing is your deployment version | 08:33 |
| bauzas | tobasco: so, you were running ubuntu kilo UCA | 08:33 |
| bauzas | tobasco: then you moved to master for compute and conductor ? | 08:33 |
| leseb_ | alexschm: we have that: https://review.openstack.org/#/c/117442/ | 08:34 |
| tobasco | no, im running ubuntu kilo UCA right now, upgraded and now conductor and computes fail | 08:34 |
| bauzas | what I need to understand is that if it's either a bug on stable/kilo or a deployment issue | 08:34 |
| tobasco | bauzas: | 08:34 |
| bauzas | tobasco: so you upgraded from what? | 08:34 |
| jwcroppe | bauzas: I'm heading to bed now. Happy reviewing ... whenever you get time. :) This seems to be more approachable than before though. | 08:34 |
| bauzas | juno ? | 08:34 |
| tobasco | bauzas: no, i meant updated my kilo | 08:34 |
| bauzas | jwcroppe: sure thing, will try to look at it | 08:34 |
| jwcroppe | bauzas: no problem. I'd like jaypipes to look too. | 08:35 |
| bauzas | tobasco: you mean kilo-something to kilo-sthelse ? | 08:35 |
| jwcroppe | dansmith, jaypipes: FYI - https://review.openstack.org/#/c/219153/ | 08:35 |
| alexschm | leseb_ yeah, I think the result of that review was that the IO policy should not be configurable per compute node, but instead be decided within nova | 08:35 |
| tobasco | bauzas: simply apt-get upgrade on conductor and compute | 08:35 |
| tobasco | using that ubuntu kilo UCA repo | 08:35 |
| alexschm | leseb_ e.g. nova should pick the "ideal" io policy for file based vs block based devices | 08:35 |
| bauzas | tobasco: well, honestly, I don't exactly know which versions the ubuntu team is shipping | 08:36 |
| bauzas | tobasco: I'd defer that question to the #openstack channel and see if any people have the same problem | 08:36 |
| tobasco | nova-conductor 1:2015.1.0-0ubuntu1.1~cloud0 | 08:36 |
| tobasco | nova-compute 1:2015.1.1-0ubuntu1~cloud2 | 08:36 |
| alexschm | leseb_ so the io policy can be tuned per instance | 08:36 |
| bauzas | tobasco: if that's becoming confirmed, I'd appreciate if you could log a bug in Launchpad https://bugs.launchpad.net/nova/ or yell there | 08:37 |
| bauzas | s/there/here | 08:37 |
| leseb_ | alexschm: well I might need to read the comments again but image_type defines the type of disk that will be used for the instance so I'm not sure why this wouldn't be configurable | 08:37 |
| leseb_ | just like the cache policy | 08:37 |
| *** lyanchih has joined #openstack-nova | 08:37 | |
| leseb_ | so per compute is fine by me, you usually don't do multiple "image_type" per compute, so I don't see any problem having this on compute basis | 08:38 |
| *** atuvenie has quit IRC | 08:38 | |
| *** shoutm has quit IRC | 08:39 | |
| tobasco | bauzas: i will file a bug and try to reproduce it right away | 08:39 |
| bauzas | tobasco: okay, what we ideally would need in the bug report is | 08:39 |
| alexschm | leseb_ I think the comments indicated the IO policy should be derived from the image_type instead of having it defined in nova.conf for all instances on a compute node | 08:39 |
| bauzas | tobasco: 1. version on Nova (setup.cfg) | 08:39 |
| bauzas | tobasco: (for the controller node) | 08:39 |
| bauzas | tobasco: 2. version of Nova for the compute node | 08:40 |
| bauzas | tobasco: make sure that your restarted primarly your conductor on the controller node and then restart your compute | 08:40 |
| leseb_ | alexschm: this might be fine but in the end the effect is the same it's just saves you one extra nova option | 08:40 |
| leseb_ | no? | 08:40 |
| bauzas | tobasco: AFAICS, that's the compute which is unable to write its stats | 08:40 |
| *** Marga_ has joined #openstack-nova | 08:41 | |
| alexschm | leseb_ yes that's right, that's what I'd like to implement/suggest and see if it gets accepted | 08:41 |
| wimdc | Hey all, does anyone know something about pci_passthrough filter (SRIOV, nova scheduler) in combination with multi-segment networks and how that is supposed to work? | 08:41 |
| leseb_ | alexschm: I think it's a good start, but the thing I'd like to see the most is to have that for attached volume since this is where we will seek performance | 08:42 |
| leseb_ | alexschm: I'm just not sure if forcing a configuration is the way to do it, I mean ops might be upset for some reasons | 08:43 |
| tobasco | bauzas: https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1490361 | 08:43 |
| openstack | Launchpad bug 1490361 in nova (Ubuntu) "IncompatibleObjectVersion: Version 1.2 of PciDeviceList is not supported" [Undecided,Confirmed] | 08:43 |
| tobasco | bauzas: found it | 08:43 |
| *** jistr has joined #openstack-nova | 08:44 | |
| alexschm | leseb_ yes I agree, I would like to use iothreads as soon as a block volume is attached to an instance, and maybe in a second step increase the number of iothreads when more volumes are added | 08:44 |
| openstackgerrit | lyanchih proposed openstack/nova: libvirt: serial console ports count upper limit needs to be checked https://review.openstack.org/207373 | 08:45 |
| openstackgerrit | Alvaro Lopez Garcia proposed openstack/nova: libvirt: make live_migration_uri flag dependent on virt_type https://review.openstack.org/175780 | 08:46 |
| leseb_ | alexschm: exactly :) | 08:46 |
| leseb_ | alexschm: so it's actually enabling iothreads and some threads as well :) | 08:47 |
| leseb_ | alexschm: email address so I can assign you to the bp? | 08:47 |
| bauzas | tobasco: well, by seeing https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1490361/comments/2 I consider the bug as invalid | 08:47 |
| openstack | Launchpad bug 1490361 in nova (Ubuntu) "IncompatibleObjectVersion: Version 1.2 of PciDeviceList is not supported" [Undecided,Confirmed] | 08:47 |
| *** bradjones has joined #openstack-nova | 08:47 | |
| *** bradjones has quit IRC | 08:47 | |
| *** bradjones has joined #openstack-nova | 08:47 | |
| alexschm | leseb_ alexs@linux.vnet.ibm.com | 08:48 |
| bauzas | tobasco: like I said, we only provide compatibility for backwards versions, not forwards | 08:48 |
| tobasco | bauzas: i wouldn't just yet, my compute nodes are still not working after restarting all services and even restarting the servers.. everything upgraded | 08:48 |
| leseb_ | alexschm: done | 08:48 |
| bauzas | tobasco: so here the reporter said that controller 2015.1.0 can't deal with computes 2015.1.1 which is normal | 08:49 |
| alexschm | leseb_ okay thank you! | 08:49 |
| bauzas | tobasco: then that's another case | 08:49 |
| leseb_ | alexschm: np np | 08:49 |
| bauzas | tobasco: so, again, I need to understand your deployment => controller 2015.1.1 can't deal with computes 2015.1.1 ? | 08:50 |
| bauzas | tobasco: I guess you restarted all your services? | 08:50 |
| *** rotbeard has joined #openstack-nova | 08:50 | |
| *** hparekh has quit IRC | 08:51 | |
| tobasco | bauzas: found the issue, it didn't upgrade nova-conductor for some reason when doing apt-get upgrade, so i did apt-get -f upgrade nova-conductor and now it's upgraded | 08:51 |
| tobasco | bauzas: both using the same version and seems to work fine | 08:52 |
| tobasco | bauzas: ty for help | 08:55 |
| bauzas | np | 08:56 |
| bauzas | johnthetubaguy: FWIW https://blueprints.launchpad.net/nova/+spec/request-spec-object-mitaka | 08:56 |
| *** Marga_ has quit IRC | 08:58 | |
| *** Marga_ has joined #openstack-nova | 08:59 | |
| *** penick has joined #openstack-nova | 09:00 | |
| *** coolsvap_ has joined #openstack-nova | 09:01 | |
| *** pilgrimstack has left #openstack-nova | 09:01 | |
| *** penick has quit IRC | 09:04 | |
| *** rajesht has joined #openstack-nova | 09:05 | |
| *** yamahata has quit IRC | 09:09 | |
| *** hparekh has joined #openstack-nova | 09:11 | |
| *** ndipanov has joined #openstack-nova | 09:13 | |
| *** MarcF has quit IRC | 09:13 | |
| *** moshele has quit IRC | 09:13 | |
| *** Marga_ has quit IRC | 09:14 | |
| *** atuvenie has joined #openstack-nova | 09:16 | |
| *** oomichi has joined #openstack-nova | 09:16 | |
| *** alexschm has quit IRC | 09:27 | |
| *** alexschm has joined #openstack-nova | 09:29 | |
| *** alexpilotti_ has joined #openstack-nova | 09:31 | |
| openstackgerrit | Sergey Nikitin proposed openstack/nova: Changed backporting of embedded Network object when backporting a FixedIP https://review.openstack.org/173723 | 09:32 |
| *** marzif has joined #openstack-nova | 09:43 | |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/nova: WIP: Add auto-extending scheduler-hints API schema https://review.openstack.org/220440 | 09:48 |
| *** kse has quit IRC | 09:48 | |
| openstackgerrit | Vladyslav Drok proposed openstack/nova: DO NOT MERGE - Triggering experimental pipeline https://review.openstack.org/220229 | 09:49 |
| *** ociuhandu has joined #openstack-nova | 09:51 | |
| *** sayali has quit IRC | 09:53 | |
| *** sayali has joined #openstack-nova | 09:54 | |
| *** josecastroleon has joined #openstack-nova | 09:57 | |
| *** coolsvap_ has quit IRC | 09:58 | |
| *** oomichi has quit IRC | 10:00 | |
| *** claudiub has joined #openstack-nova | 10:04 | |
| *** lyanchih has quit IRC | 10:04 | |
| *** dave-mccowan has joined #openstack-nova | 10:06 | |
| *** wimdc has quit IRC | 10:06 | |
| *** marzif has quit IRC | 10:08 | |
| openstackgerrit | Vladyslav Drok proposed openstack/nova: DO NOT MERGE - Triggering experimental pipeline https://review.openstack.org/220229 | 10:12 |
| *** vponomaryov has left #openstack-nova | 10:14 | |
| openstackgerrit | Daniel Berrange proposed openstack/nova-specs: Virtual guest device role tagging https://review.openstack.org/195662 | 10:14 |
| *** paul-carlton has quit IRC | 10:17 | |
| *** y_sawai has joined #openstack-nova | 10:19 | |
| rajesht | jaypipes: I agree with your comment on https://review.openstack.org/#/c/170118/17/nova/compute/manager.py | 10:19 |
| alex_xu | sdague: dansmith johnthetubaguy one more fix for bug 1491511 https://review.openstack.org/#/c/220386/ | 10:19 |
| openstack | bug 1491511 in OpenStack Compute (nova) "Behavior change with latest nova paste config" [Critical,In progress] https://launchpad.net/bugs/1491511 - Assigned to Alex Xu (xuhj) | 10:19 |
| *** sayali has quit IRC | 10:20 | |
| rajesht | jaypipes: if nova-compute gets crashed then reserved_quota object will never commit. | 10:20 |
| claudiub | johnthetubaguy: hi. you previously had a +2 on this commit: https://review.openstack.org/#/c/148980/ very minor change, fixed a unit test that broke because of other changes that merged before it. | 10:20 |
| rajesht | jaypipes: but to commit reserved quota, first we need to sure that instance gets deleted. or we can move reserved quota commit call before instance deletion. | 10:21 |
| sdague | alex_xu: but unicode in cell names is fine? | 10:21 |
| sdague | I'd like to get that confirmed with johnthetubaguy and folks | 10:21 |
| *** IanGovett has joined #openstack-nova | 10:21 | |
| alex_xu | sdague: emm....yea, not sure the unicode | 10:21 |
| *** paul-carlton has joined #openstack-nova | 10:21 | |
| rajesht | sdague: could you please review https://review.openstack.org/#/c/170118/ | 10:21 |
| sdague | alex_xu: yet, lets wait for johnthetubaguy / alaski and get confirmation on it. I agree we should restrict it more | 10:22 |
| johnthetubaguy | sdague: yeah, I am unsure about unicode, I guess it works due to same way name works | 10:22 |
| alex_xu | sdague: ok | 10:22 |
| *** sayali has joined #openstack-nova | 10:23 | |
| sdague | johnthetubaguy: but cells explode if ! or . are in them? | 10:23 |
| claudiub | johnthetubaguy: ty! | 10:23 |
| johnthetubaguy | sdague: so ! is the separator for host and cell in some combined thingy | 10:23 |
| johnthetubaguy | I think the hostname gets the cells added, using ! | 10:23 |
| bauzas | sdague: alex_xu: you mean questions about instance names in cells ? | 10:23 |
| sdague | bauzas: no, about cell names | 10:24 |
| alex_xu | sdague: cell use '!' and '.' for some special route | 10:24 |
| bauzas | sdague: I don't think there is a special case for the cell names | 10:24 |
| *** paul-carlton has quit IRC | 10:24 | |
| bauzas | we only use the separators for routing the instances | 10:24 |
| sdague | ok, so given that we've probably never tested unicode actually working for cells, I'd kind of go more conservative | 10:24 |
| bauzas | lemme find you the module | 10:24 |
| alex_xu | sdague: bauzas something like https://github.com/openstack/nova/blob/master/nova/cells/manager.py#L300 | 10:25 |
| bauzas | sdague: https://github.com/openstack/nova/blob/master/nova/cells/utils.py#L29-L37 | 10:25 |
| *** paul-carlton has joined #openstack-nova | 10:25 | |
| johnthetubaguy | alex_xu: thats the example, hmm, I wonder if we need to exclude @ | 10:26 |
| bauzas | so for example parent_cell!child_cell@instance_a | 10:26 |
| sdague | yeh, seems like a thing. Also, I kind of hate this is all encoded in a name :P | 10:26 |
| openstackgerrit | Nikola Dipanov proposed openstack/nova: claims: Remove compat code with instance dicts https://review.openstack.org/220215 | 10:26 |
| alex_xu | johnthetubaguy: emm...yea, looks like v2 have bug also | 10:26 |
| sdague | alex_xu: so, honestly, I'd restrict this to the name definition - !, ., and @ | 10:27 |
| alex_xu | v2 only validate '!' and '.' https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/legacy_v2/contrib/cells.py#L189 | 10:27 |
| bauzas | sdague: +1 | 10:27 |
| johnthetubaguy | sdague: yeah, not sure about the "." I guess thats historic, but +1 the rest | 10:27 |
| alex_xu | sdague: yea, let me update the patch, looks like also need fix the v2 api | 10:27 |
| sdague | alex_xu: great | 10:28 |
| bauzas | johnthetubaguy: probably for FQDNs ? | 10:28 |
| *** lyanchih has joined #openstack-nova | 10:28 | |
| bauzas | I mean the dot | 10:28 |
| johnthetubaguy | bauzas: don't think that matters, unsure | 10:28 |
| johnthetubaguy | I suspect the original code used . and not @ | 10:29 |
| johnthetubaguy | looks like that is old code | 10:29 |
| johnthetubaguy | anyways, its a mute point, for consistency we need to exclude . | 10:30 |
| lxsli | moot >.> | 10:30 |
| *** ftersin has joined #openstack-nova | 10:30 | |
| johnthetubaguy | lol, yeah, that | 10:30 |
| sdague | something is way up with largeops | 10:33 |
| sdague | http://graphite.openstack.org/render/?from=-200hours&height=500&until=now&width=800&bgcolor=ffffff&fgcolor=000000&yMax=100&yMin=0&vtitle=Failure%20Rate%20in%20Percent&title=Test%20failure%20rates%20over%20last%20200%20hours&&target=lineWidth(color(alias(movingAverage(asPercent(stats.zuul.pipeline.check.job.gate-tempest-dsvm-large-ops.FAILURE,sum(stats.zuul.pipeline.check.job.gate-tempest-dsvm-large-ops.{SUCCESS,FAILURE})),%2712hours%27),%20%27g | 10:33 |
| sdague | ate-tempest-dsvm-large-ops%20%28check%29%27),%27ff0000%27),1)&target=lineWidth(color(alias(movingAverage(asPercent(stats.zuul.pipeline.gate.job.gate-tempest-dsvm-large-ops.FAILURE,sum(stats.zuul.pipeline.gate.job.gate-tempest-dsvm-large-ops.{SUCCESS,FAILURE})),%2712hours%27),%20%27gate-tempest-dsvm-large-ops%20%28gate%29%27),%27b00000%27),2) | 10:33 |
| sdague | hmmm... let me url shorten that | 10:34 |
| *** kiran-r has joined #openstack-nova | 10:35 | |
| openstackgerrit | Alex Xu proposed openstack/nova: Reject the cell name include '!', '.' and '@' for Nova API https://review.openstack.org/220386 | 10:37 |
| alex_xu | bauzas: sdague johnthetubaguy ^, it's done. or we should wait alaski take a look at it also. | 10:38 |
| bauzas | alex_xu: I would definitely wait for alaski | 10:38 |
| sdague | http://goo.gl/KVPPc0 | 10:38 |
| bauzas | oomichi has left, right? | 10:39 |
| alex_xu | bauzas: yea, agree | 10:39 |
| johnthetubaguy | alex_xu: yeah, I would like to see alaski take a look, just looking at this bit: test_cell_create_name_with_invalid_character_raises, why is that removed for v2? | 10:39 |
| bauzas | re: his hints convo | 10:39 |
| johnthetubaguy | sdague: oh thats interesting, there was that talk on the ML of a performance regression, could be related I supose | 10:39 |
| johnthetubaguy | although that was much earlier on | 10:40 |
| alex_xu | johnthetubaguy: the v2 api in that case, actually test the '!', there is another one for it test_cell_create_name_with_exclamation_point_raises | 10:41 |
| alex_xu | johnthetubaguy: and the v2.1 test_cell_create_name_with_invalid_character_raises at line 268 against '\x00 | 10:42 |
| alex_xu | which can't pass in v2 api. | 10:42 |
| *** karmatronic has joined #openstack-nova | 10:44 | |
| *** karmatronic has quit IRC | 10:45 | |
| * alex_xu rush to dinner | 10:45 | |
| sdague | johnthetubaguy: the fail I just looked at definitely had oslo message timeouts in it | 10:45 |
| *** lyanchih has quit IRC | 10:45 | |
| johnthetubaguy | alex_xu: ah, OK | 10:46 |
| *** ZZelle has quit IRC | 10:48 | |
| *** ZZelle has joined #openstack-nova | 10:48 | |
| *** jlanoux has quit IRC | 10:48 | |
| *** links has quit IRC | 10:49 | |
| *** lyanchih has joined #openstack-nova | 10:50 | |
| *** ishant has quit IRC | 10:52 | |
| *** y_sawai has quit IRC | 10:54 | |
| bauzas | johnthetubaguy: oomichi is based in Tokyo, right? | 10:54 |
| bauzas | johnthetubaguy: so I guess not there until tonight our time ? | 10:54 |
| bauzas | uh, even next week rather :) | 10:55 |
| johnthetubaguy | bauzas: I don't expect him to be around right now | 10:55 |
| bauzas | yeah, that reminds me we're on Friday | 10:55 |
| lxsli | \o/ | 10:55 |
| *** bauzas is now known as bauwser | 10:55 | |
| sdague | and what a week it was :) | 10:55 |
| bauwser | yeah, definitely a short one | 10:55 |
| *** venkat_p has quit IRC | 10:55 | |
| sdague | oh, for .eu folks, reminder that monday is federal holiday in the US | 10:56 |
| sdague | so don't expect to see any of us around | 10:56 |
| sdague | johnthetubaguy: so .... https://review.openstack.org/#/c/220279/ ... the name relaxation bit | 10:56 |
| *** chhavi has quit IRC | 10:57 | |
| sdague | dansmith, mriedem, and I were actually massively confused about the patch that said it implemented that in kilo, because it seemed to do exactly the oposite | 10:58 |
| sdague | so much so, that I'm trying to figure out if we were missing something | 10:58 |
| *** lyanchih has quit IRC | 11:00 | |
| *** ildikov has quit IRC | 11:02 | |
| rajesht | johnthetubaguy: could you please check https://review.openstack.org/#/c/170118 | 11:03 |
| sdague | rajesht: that's got a -1 from jaypipes | 11:08 |
| sdague | for, I think good reasons | 11:08 |
| sdague | so please stop asking for other reviews until you address those issues | 11:08 |
| rajesht | sdague: but there is no other way. We can't update both in_use and reserved quota at the same time. | 11:08 |
| rajesht | sdague: we need to update only on at a time. | 11:08 |
| rajesht | sdague: regarding his comment about nova-compute crash, I think that will be very rare, because this code is executed from init_instance on compute restart | 11:09 |
| rajesht | sdague: so it is very rare that compute gets crashed during starting/restarting. | 11:09 |
| *** kiran-r has quit IRC | 11:09 | |
| sdague | rajesht: I feel like you need to work with jaypipes on this one. And no nova core folks are likely to override him on it. | 11:10 |
| rajesht | sdague: ok, I think he is not on IRC, I have already replied him on chat. | 11:11 |
| rajesht | sdague: let me discuss with my team if I can do any changes as suggested by jay | 11:11 |
| *** chhavi has joined #openstack-nova | 11:13 | |
| johnthetubaguy | rajesht: its worth dropping him an email, he is often around later in the day | 11:14 |
| *** ildikov has joined #openstack-nova | 11:15 | |
| johnthetubaguy | rajesht: FWIW my preferred fix is to implement the simply quota spec, it would remove all of these types of bugs (at least in its current version) | 11:15 |
| rajesht | johnthetubaguy: thanks :) will try to catch him later. | 11:17 |
| johnthetubaguy | rajesht: this is the start of that conversation: https://review.openstack.org/#/c/182445/ | 11:17 |
| *** kiran-r has joined #openstack-nova | 11:18 | |
| rajesht | johnthetubaguy: thank you very much for reference | 11:18 |
| *** gabriel-bezerra has joined #openstack-nova | 11:24 | |
| openstackgerrit | Nikola Dipanov proposed openstack/nova: claims: Remove compat code with instance dicts https://review.openstack.org/220215 | 11:26 |
| *** lucasagomes is now known as lucas-hungry | 11:26 | |
| *** promulo has quit IRC | 11:27 | |
| *** sayali has quit IRC | 11:27 | |
| *** venkat_p has joined #openstack-nova | 11:27 | |
| *** sayali has joined #openstack-nova | 11:29 | |
| *** sayali has quit IRC | 11:31 | |
| sdague | johnthetubaguy: so, any opinions on the name relax? | 11:36 |
| openstackgerrit | Rajesh Tailor proposed openstack/nova: Fix order of arguments in assertEqual https://review.openstack.org/220466 | 11:36 |
| johnthetubaguy | sdague: oh, the allowing of a space? | 11:36 |
| sdague | well, it's broader than that | 11:37 |
| *** lyanchih has joined #openstack-nova | 11:38 | |
| sdague | johnthetubaguy: in investigating the issue we found that the patch that in theory implemented the name relaxation in kilo did the oposite - https://review.openstack.org/#/c/119741/8/nova/api/openstack/compute/schemas/v3/servers.py,cm | 11:39 |
| johnthetubaguy | oh dera | 11:40 |
| *** _kiran_ has joined #openstack-nova | 11:40 | |
| johnthetubaguy | non of the tests included a space I guess | 11:40 |
| johnthetubaguy | oh, they do | 11:40 |
| *** unicell has joined #openstack-nova | 11:41 | |
| *** unicell1 has quit IRC | 11:42 | |
| *** kiran-r has quit IRC | 11:43 | |
| *** thorst has joined #openstack-nova | 11:44 | |
| *** venkat_p has quit IRC | 11:46 | |
| johnthetubaguy | sdague: it does feel like we should get some functional tests for this kind of thing, so its easier to not screw up the tests, or something like that? | 11:46 |
| * johnthetubaguy needs to go get some food | 11:47 | |
| *** _kiran_ has quit IRC | 11:49 | |
| *** lyanchih has quit IRC | 11:50 | |
| *** husanu has quit IRC | 11:50 | |
| johnthetubaguy | dansmith: instance v2 is looking very close, just a few questions on the last two patches | 11:50 |
| *** husanu has joined #openstack-nova | 11:52 | |
| *** jlanoux has joined #openstack-nova | 11:52 | |
| *** takedakn has joined #openstack-nova | 11:52 | |
| andreykurilin | hi everyone! Does it known issue with unability to update quotas via novaclient + V2.1 ? | 11:53 |
| sdague | cells job has a new race where it can't find any networks - https://jenkins03.openstack.org/job/gate-tempest-dsvm-cells/1224/console | 11:53 |
| *** y_sawai has joined #openstack-nova | 11:53 | |
| sdague | andreykurilin: not that I know of | 11:53 |
| *** chhavi has quit IRC | 11:53 | |
| openstackgerrit | Nikola Dipanov proposed openstack/nova: RT: Migration resource tracking uses migration context https://review.openstack.org/218938 | 11:54 |
| openstackgerrit | Nikola Dipanov proposed openstack/nova: compute: migrate/resize paths properly handle stashed numa_topology https://review.openstack.org/218500 | 11:54 |
| openstackgerrit | Nikola Dipanov proposed openstack/nova: Claims: Make sure move claims create a migration context records https://review.openstack.org/218385 | 11:54 |
| andreykurilin | sdague: ok, I'll post a bug report. | 11:54 |
| *** husanu has quit IRC | 11:57 | |
| *** deepthi has quit IRC | 11:58 | |
| *** husanu has joined #openstack-nova | 11:59 | |
| andreykurilin | sdague: https://bugs.launchpad.net/nova/+bug/1492242 it relates to scheme validation in nova v2.1 | 11:59 |
| openstack | Launchpad bug 1492242 in python-novaclient "novaclient is unable to update quotas via Nova V2.1" [Undecided,New] | 11:59 |
| *** venkat_p has joined #openstack-nova | 12:00 | |
| *** rfolco has joined #openstack-nova | 12:00 | |
| *** husanu has quit IRC | 12:02 | |
| openstackgerrit | Claudiu Belu proposed openstack/nova: Hyper-V: Adds HyperVDriver unit tests https://review.openstack.org/148980 | 12:02 |
| *** sayali has joined #openstack-nova | 12:04 | |
| *** husanu has joined #openstack-nova | 12:04 | |
| *** smatzek has joined #openstack-nova | 12:04 | |
| sdague | andreykurilin: you have a novaclient update for that? | 12:05 |
| *** raildo-afk is now known as raildo | 12:05 | |
| *** ociuhandu has quit IRC | 12:06 | |
| bauwser | sdague: we had an identical problem with the networks for the cells last cycle IIRC | 12:07 |
| bauwser | sdague: it was basically because you need to also update the networks for the child cells within devstack | 12:07 |
| sdague | I wonder if this is related to the large ops jobs failing on networks | 12:07 |
| bauwser | sdague: it was basically because you need to also update the networks for the child cells within devstackhttps://jenkins03.openstack.org/job/gate-tempest-dsvm-cells/1224/console | 12:07 |
| sdague | bauwser: ok any idea why it would start popping up? | 12:08 |
| *** chhavi has joined #openstack-nova | 12:08 | |
| bauwser | sdague: last time, it was due to a Tempest change IIRC | 12:08 |
| bauwser | sdague: I need to see the logs | 12:08 |
| andreykurilin | sdague: the update for novaclient is small(just don't transmit tenant-id), but I don't does it work for nova V2 or not | 12:10 |
| sdague | andreykurilin: can you build a functional test for it in novaclient, that should make it clear | 12:11 |
| *** ociuhandu has joined #openstack-nova | 12:12 | |
| sdague | the tempest code indicates that it doesn't need that parameter | 12:12 |
| sdague | https://github.com/openstack/tempest/blob/1f60eaef0dd3000b55f32111dd9cb53c3d460893/tempest/services/compute/json/quotas_client.py#L44-L59 | 12:13 |
| andreykurilin | sdague: ok. will post a change in several minutes | 12:13 |
| sdague | yeh, something landed a couple of days ago that has impacted large-ops | 12:13 |
| sdague | message:"Failed to deallocate network for instance" AND build_name:*large-ops* | 12:13 |
| sdague | http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiRmFpbGVkIHRvIGRlYWxsb2NhdGUgbmV0d29yayBmb3IgaW5zdGFuY2VcIiBBTkQgYnVpbGRfbmFtZToqbGFyZ2Utb3BzKiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQ0MTM2ODU4MjYwOH0= | 12:14 |
| *** pkholkin has quit IRC | 12:14 | |
| sdague | bauwser: do you know if we have a bug for this regression already? | 12:14 |
| bauwser | sdague: that's what I'm currently looking | 12:14 |
| bauwser | :) | 12:14 |
| bauwser | sdague: also looking at logstash | 12:14 |
| bauwser | sdague: http://logstash.openstack.org/#eyJzZWFyY2giOiJidWlsZF9uYW1lOlwiZ2F0ZS10ZW1wZXN0LWRzdm0tY2VsbHNcIiBBTkQgbWVzc2FnZTpcIldvcmtlciBCYWxhbmNlXCIgQU5EIGJ1aWxkX3N0YXR1czpcIkZBSUxVUkVcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNDMyMDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDQxMzY4OTMwOTE0fQ== | 12:15 |
| bauwser | sdague: it seems it recently regressed | 12:15 |
| sdague | I wouldn't look at logstash for that | 12:16 |
| *** VikasC has quit IRC | 12:16 | |
| bauwser | do we have a stale logstash ? | 12:16 |
| *** Sree has joined #openstack-nova | 12:16 | |
| bauwser | sdague: tbc, I was looking at how many time the job was -1 | 12:17 |
| sdague | http://tinyurl.com/nr3c9ng | 12:17 |
| bauwser | sdague: and I just changed the message to "no networks found" | 12:17 |
| bauwser | oh yeah graphite, I really need to bookmark it | 12:17 |
| *** venkat_p has quit IRC | 12:17 | |
| sdague | bauwser: it's just that logstash hits don't nessessarily mean more fails, because it can mean just volume | 12:17 |
| bauwser | sdague: interesting.... | 12:17 |
| sdague | especially when the event rate is so low | 12:18 |
| bauwser | sdague: so yeah, we had a problem last week with the cells job, but it was fixed on Monday thanks to alaski | 12:18 |
| bauwser | now, that's another issue | 12:18 |
| sdague | that's 12 hr moving average, the # of events is low enough that you should be suspicious of fast jumps to 50% | 12:18 |
| bauwser | sdague: http://logstash.openstack.org/#eyJzZWFyY2giOiJidWlsZF9uYW1lOlwiZ2F0ZS10ZW1wZXN0LWRzdm0tY2VsbHNcIiBBTkQgbWVzc2FnZTpcIk5vIG5ldHdvcmtzIGZvdW5kXCIgQU5EIGJ1aWxkX3N0YXR1czpcIkZBSUxVUkVcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQ0MTM2ODk5OTMxNX0= | 12:18 |
| bauwser | sdague: pretty new | 12:19 |
| bauwser | sdague: ok, looking at the logs now | 12:19 |
| bauwser | sdague: gotcha for the # of events | 12:19 |
| sdague | well, except, if you filter on FAILURE in logstash, you start with the assumption that the rest of your search terms are always fatal | 12:20 |
| sdague | and you'd be surprised, they often are not | 12:20 |
| sdague | so it's much better to not do that, and see what the SUCCESS / FAILURE ratio is | 12:20 |
| bauwser | yup, you're right | 12:21 |
| bauwser | okay good to know | 12:21 |
| sdague | in this case, it looks like they all are fails anyway, but just a good strategy for future | 12:21 |
| *** links has joined #openstack-nova | 12:21 | |
| *** aysyd has joined #openstack-nova | 12:24 | |
| openstackgerrit | sulochan-acharya proposed openstack/nova: xapi: cleanup volume sr on live migration rollback https://review.openstack.org/217105 | 12:24 |
| bauwser | sdague: https://review.openstack.org/#/c/172583/ was the change I remembered when we saw some 404s for networks | 12:25 |
| *** baoli has joined #openstack-nova | 12:25 | |
| *** baoli has quit IRC | 12:25 | |
| sdague | https://bugs.launchpad.net/nova/+bug/1492249 - is the bug I filed for the large ops case | 12:25 |
| openstack | Launchpad bug 1492249 in OpenStack Compute (nova) "regression in network performance causing large-ops jobs failures" [High,Confirmed] | 12:25 |
| *** claudiub has quit IRC | 12:26 | |
| *** baoli has joined #openstack-nova | 12:27 | |
| *** baoli_ has joined #openstack-nova | 12:29 | |
| *** pkholkin has joined #openstack-nova | 12:33 | |
| *** baoli has quit IRC | 12:33 | |
| *** edmondsw has joined #openstack-nova | 12:33 | |
| *** husanu has quit IRC | 12:33 | |
| *** beagles has quit IRC | 12:34 | |
| *** venkat_p has joined #openstack-nova | 12:34 | |
| *** Sree has quit IRC | 12:35 | |
| *** beagles has joined #openstack-nova | 12:35 | |
| *** beagles is now known as Guest69944 | 12:35 | |
| *** husanu has joined #openstack-nova | 12:35 | |
| *** ankit_ag has quit IRC | 12:36 | |
| *** lucas-hungry is now known as lucasagomes | 12:37 | |
| *** Sree has joined #openstack-nova | 12:38 | |
| bauwser | sdague: I just wonder if https://github.com/openstack/tempest/commit/e4184ea28e21b86edc28c9f3c02ec28d0d2fb476 could be related | 12:39 |
| *** husanu has quit IRC | 12:39 | |
| *** takedakn has quit IRC | 12:40 | |
| *** dims has joined #openstack-nova | 12:42 | |
| *** vladikr has joined #openstack-nova | 12:42 | |
| *** Guest69944 is now known as b3nt_pin | 12:42 | |
| *** pkholkin has quit IRC | 12:44 | |
| *** annegentle has joined #openstack-nova | 12:46 | |
| *** b3nt_pin has quit IRC | 12:46 | |
| *** b3nt_pin has joined #openstack-nova | 12:47 | |
| *** vladikr has quit IRC | 12:47 | |
| bauwser | sdague: bingo https://github.com/openstack/nova/blob/master/devstack/tempest-dsvm-cells-rc#L63 | 12:49 |
| bauwser | sdague: that test is normally excluded from the job | 12:50 |
| bauwser | sdague: since mtreinish changed the name, it then ran | 12:50 |
| bauwser | sdague: providing a change to it | 12:50 |
| *** abhishekk has quit IRC | 12:53 | |
| *** pkholkin has joined #openstack-nova | 12:53 | |
| *** ildikov has quit IRC | 12:53 | |
| mtreinish | bauwser: oh, I broke the exclude regex | 12:54 |
| *** venkat_p has quit IRC | 12:54 | |
| *** Murali_ has quit IRC | 12:54 | |
| bauwser | mtreinish: np, changing the regex now | 12:54 |
| mtreinish | bauwser: fwiw, you could use the test's uuid to exclude it | 12:54 |
| *** Marga_ has joined #openstack-nova | 12:54 | |
| mtreinish | that way if there are other renames in the future it'll still work | 12:55 |
| bauwser | mtreinish: interesting | 12:55 |
| *** venkat_p has joined #openstack-nova | 12:55 | |
| bauwser | mtreinish: you mean the test.idempotent_id ? | 12:55 |
| mtreinish | bauwser: yep | 12:56 |
| bauwser | mtreinish: how can I tell to devstack to exlude it ? | 12:56 |
| bauwser | mtreinish: just giving the ID ? | 12:56 |
| mtreinish | bauwser: that gets set as an attr on the test as id-uuid | 12:56 |
| mtreinish | bauwser: so in the regex replace the test name with id-uuid and that should work | 12:56 |
| bauwser | mtreinish: okay, I can give a try | 12:56 |
| *** dustins has joined #openstack-nova | 12:57 | |
| johnthetubaguy | bauwser: ah good catch, using the uuids is a good idea | 12:59 |
| *** vladikr has joined #openstack-nova | 12:59 | |
| openstackgerrit | John Garbutt proposed openstack/nova: Reject the cell name include '!', '.' and '@' for Nova API https://review.openstack.org/220386 | 13:01 |
| *** ijuwang has joined #openstack-nova | 13:02 | |
| *** edleafe is now known as figleaf | 13:03 | |
| *** mriedem has joined #openstack-nova | 13:03 | |
| openstackgerrit | Andrey Kurilin proposed openstack/python-novaclient: [Bug-Fix] Update requests body for quota-update https://review.openstack.org/220488 | 13:04 |
| openstackgerrit | Andrey Kurilin proposed openstack/python-novaclient: [Bug-Fix] Update requests body for quota-update https://review.openstack.org/220488 | 13:05 |
| *** nkrinner has quit IRC | 13:06 | |
| openstackgerrit | Sylvain Bauza proposed openstack/nova: Fix Cells gate test by modifying the regressions regex https://review.openstack.org/220490 | 13:07 |
| *** diogogmt has quit IRC | 13:07 | |
| mriedem | sdague: johnthetubaguy: just got on - is there any thought on raising the minimum required novaclient to 2.28 in g-r? | 13:07 |
| bauwser | mtreinish: ^ if you could look at the regex, unsure I am :) | 13:07 |
| mriedem | while we can? | 13:07 |
| *** coolsvap_ has joined #openstack-nova | 13:08 | |
| *** nkrinner has joined #openstack-nova | 13:08 | |
| mtreinish | bauwser: sure, one sec | 13:08 |
| mtreinish | bauwser: fwiw, you can always run testr list-tests with the regex to test it | 13:08 |
| bauwser | mtreinish: oh yeah, you're right... | 13:08 |
| *** ildikov has joined #openstack-nova | 13:09 | |
| *** mdrabe has joined #openstack-nova | 13:10 | |
| *** takedakn has joined #openstack-nova | 13:10 | |
| bauwser | mtreinish: I'm actually testing that now :) | 13:10 |
| mtreinish | bauwser: that looks fine to me, I don't think you need the * at the end but I guess it doesn't hurt | 13:11 |
| *** jcoufal has joined #openstack-nova | 13:11 | |
| *** lbeliveau has quit IRC | 13:11 | |
| mriedem | bauwser: let me know when you're verifid | 13:11 |
| mriedem | danpb: sahid: could either of you take a look at this bug? https://launchpad.net/bugs/1492026 | 13:13 |
| openstack | Launchpad bug 1492026 in OpenStack Compute (nova) "domain.attachDeviceFlags randomly doesn't do anything in libvirt" [Medium,Confirmed] | 13:13 |
| mriedem | i have https://review.openstack.org/#/c/220357/ up as a test and with that change used in this tempest run that was failing before, it passes test_stamp_pattern now https://review.openstack.org/#/c/218355/ | 13:13 |
| mriedem | which seems a bit crazy if it's just the kwarg issue | 13:13 |
| danpb | mriedem: this is the same bug as the other day ? | 13:15 |
| *** drankis has joined #openstack-nova | 13:15 | |
| danpb | mriedem: did you ever succeed in getting guest dmesg / lspci logs ? | 13:15 |
| mriedem | danpb: i reported the bug last night to track what i've been seeing | 13:15 |
| mriedem | yeah | 13:15 |
| *** rlrossit has joined #openstack-nova | 13:15 | |
| mriedem | e.g. http://logs.openstack.org/55/218355/6/check/gate-tempest-dsvm-full/e0da291/console.html#_2015-09-02_17_48_53_884 | 13:15 |
| mriedem | times out waiting for vdb to show up but it never does | 13:15 |
| mriedem | the lspci output is not helpful | 13:15 |
| danpb | cool thanks | 13:16 |
| danpb | actually it is helpful | 13:16 |
| *** takedakn has quit IRC | 13:16 | |
| danpb | it shows there are only 3 virtio devices attach | 13:16 |
| mriedem | you must know the secret sauce | 13:16 |
| danpb | one disk, one nic and the balloon | 13:16 |
| danpb | so it definitely doesn't see the second pci device | 13:16 |
| *** raildo is now known as raildo-afk | 13:17 | |
| danpb | mriedem: does the tempest test wait for the guest OS to fully boot before hotpluging the second disk ? | 13:17 |
| *** Marga_ has quit IRC | 13:17 | |
| bauwser | mriedem: mtreinish: looks good to me "< tempest.api.compute.test_networks.ComputeNetworksTest.test_list_networks[id-3fe07175-312e-49a5-a623-5f52eeada4c2]" | 13:17 |
| danpb | i'm wondering if there's a race in plugging the disk while the OS is still booting ? | 13:17 |
| dims | ha! passed check failed in gate! - https://review.openstack.org/#/c/147514/ | 13:17 |
| mriedem | danpb: i assume so, the instance is active when we attach | 13:17 |
| johnthetubaguy | mriedem: raising the version makes sense to me | 13:18 |
| danpb | hmm, do we have a NIC device hotplug test working in tempest ? | 13:18 |
| danpb | that would at lesat confirm that PCI hotplug in general works | 13:18 |
| johnthetubaguy | mriedem: or at least add an exclude on the dodgy one, I guess | 13:18 |
| *** Marga_ has joined #openstack-nova | 13:18 | |
| mriedem | danpb: so this isn't pci attach, fwiw | 13:18 |
| mriedem | johnthetubaguy: well, i think they are all dodgy < 2.28 given the v2.0 on v2.1 change | 13:18 |
| mriedem | which is why i was thinking raising hte minimum | 13:18 |
| danpb | mriedem: oh really ?? it should be - we're not using scsi are we ? | 13:19 |
| mriedem | this is iscsi / lvm | 13:19 |
| danpb | the only things disk hotplug works for are pci virtio-blk guest, or virtio-scsi device | 13:19 |
| openstackgerrit | John Garbutt proposed openstack/nova: Add functional test for server group https://review.openstack.org/147514 | 13:19 |
| mriedem | http://logs.openstack.org/55/218355/6/check/gate-tempest-dsvm-full/e0da291/logs/screen-n-cpu.txt.gz#_2015-09-02_17_42_10_274 | 13:19 |
| danpb | mrodden: oh, that's just the backend - i meant from the guest device side | 13:19 |
| sdague | mriedem: yeh, there is another issue with quotas in novaclient because it was sending extra garbage | 13:20 |
| danpb | mriedem: from the guest hardware POV it ought to be PCI hotplug | 13:20 |
| mtreinish | danpb: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/scenario/test_network_basic_ops.py#n456 is the nic hotplug test | 13:20 |
| danpb | mtreinish: thanks, that at least shows hotplug works in general | 13:20 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova: Add functional test for server group https://review.openstack.org/147514 | 13:20 |
| sdague | mriedem: https://review.openstack.org/#/c/220488 | 13:21 |
| *** haomaiwang has joined #openstack-nova | 13:21 | |
| mriedem | sdague: https://review.openstack.org/#/c/220488/ | 13:21 |
| mriedem | yueah | 13:21 |
| mriedem | jinx | 13:21 |
| sdague | I just +2ed it | 13:21 |
| sdague | novaclient is just bonkers some times | 13:21 |
| *** mestery has joined #openstack-nova | 13:24 | |
| *** raildo-afk is now known as raildo | 13:24 | |
| *** Sree has quit IRC | 13:25 | |
| mriedem | sdague: looking, one concern in the functional test | 13:25 |
| sdague | yeh, turns out it's failing it's unit tests as well | 13:26 |
| mriedem | https://github.com/openstack/tempest/commit/e3e9da70c611d686a62c630fa9a2ddfb420ae8c4 | 13:26 |
| *** shoutm has joined #openstack-nova | 13:26 | |
| dansmith | johnthetubaguy: object_action and object_class action are currently different. I'm adding a new version of object_class_action which is object_class_action_versions, which behaves the same | 13:26 |
| * danpb wishes Nova logged in a structured data format like systemd journal to make it easier to extract relevant data | 13:26 | |
| dansmith | johnthetubaguy: pop out the code above the new method and you'll see it | 13:26 |
| clarkb | it can iirc oslo logging does json output now | 13:26 |
| *** mestery has quit IRC | 13:26 | |
| *** Sree has joined #openstack-nova | 13:27 | |
| *** mestery has joined #openstack-nova | 13:27 | |
| sdague | mriedem: Bug 1492249 - regression in network performance causing large-ops jobs failures - that's a bit concerning as well, especially the frequency | 13:27 |
| openstack | bug 1492249 in OpenStack Compute (nova) "regression in network performance causing large-ops jobs failures" [High,Confirmed] https://launchpad.net/bugs/1492249 | 13:27 |
| mriedem | sdague: that's a dupe | 13:27 |
| *** jcoufal_ has joined #openstack-nova | 13:27 | |
| *** Marga_ has quit IRC | 13:27 | |
| mriedem | let me find the dupe | 13:27 |
| dansmith | johnthetubaguy: added comments to the review to clarify | 13:27 |
| *** links has quit IRC | 13:27 | |
| mriedem | i was just about to write an e-r query for that | 13:28 |
| sdague | mriedem: ok, well there was no er sig | 13:28 |
| johnthetubaguy | dansmith: ah, thanks, will take another peak | 13:28 |
| *** artom has joined #openstack-nova | 13:28 | |
| mriedem | sdague: i know | 13:28 |
| mriedem | i'm only one man and just got online :) 0 coffee yet | 13:28 |
| dansmith | johnthetubaguy: thanks a bunch of hitting those :) | 13:28 |
| johnthetubaguy | dansmith: oh my, I totally see what I did there, oops | 13:28 |
| johnthetubaguy | dansmith: I was trying to work out how the tests could be passing, I figured I was missing something obvious! | 13:29 |
| mriedem | sdague: https://bugs.launchpad.net/nova/+bug/1491949 | 13:29 |
| openstack | Launchpad bug 1491949 in OpenStack Compute (nova) "gate-tempest-dsvm-large-ops fails to deallocate instance network due to rpc timeout" [High,Confirmed] | 13:29 |
| mriedem | i'll try to find a query | 13:29 |
| dansmith | johnthetubaguy: :) | 13:29 |
| mriedem | danpb: so the other odd thing, | 13:29 |
| *** ildikov has quit IRC | 13:30 | |
| mriedem | danpb: is that when i changed nova to call domain.attachDeviceFlags with flags as a positional arg rather than a kwarg, test_stamp_pattern started passing, which makes very little sense given there are other tests which attach volumes and those aren't failing | 13:30 |
| *** jcoufal has quit IRC | 13:31 | |
| mriedem | sdague: when https://review.openstack.org/#/c/220488/ lands i'm guessing 2.28.1 release and then make that the minimum required version? | 13:31 |
| johnthetubaguy | mriedem: hmm, does that confuse the stable branch handling? | 13:31 |
| *** rajesht has quit IRC | 13:31 | |
| johnthetubaguy | I guess it shouldn't | 13:31 |
| danpb | mriedem: bizarre - i'm just going to look at libvirtd.log to see if that makes a functional difference | 13:31 |
| mriedem | danpb: and when that attach fails, i don't even see a virDomainAttachDeviceFlags call for the domain in question in the libvirt logs, it's like it never even got there | 13:32 |
| openstackgerrit | George Peristerakis proposed openstack/python-novaclient: Search the list of servers with unicode text. https://review.openstack.org/200649 | 13:32 |
| mriedem | johnthetubaguy: i don't see why it would | 13:32 |
| *** tri2sing has joined #openstack-nova | 13:32 | |
| *** Sree_ has joined #openstack-nova | 13:32 | |
| mriedem | johnthetubaguy: we'll just cap stable/liberty on novaclient to <2.29 | 13:32 |
| *** Sree_ is now known as Guest4559 | 13:33 | |
| *** Sree has quit IRC | 13:33 | |
| johnthetubaguy | mriedem: yeah, makes sense, I was just thinking we normally do those point releases off a stable branch, but then realised I was over thinking it | 13:33 |
| mriedem | right, it's just semver | 13:33 |
| mriedem | not openstack policy | 13:33 |
| johnthetubaguy | yeah | 13:34 |
| johnthetubaguy | no new features, just bug fixes | 13:34 |
| *** cfriesen has joined #openstack-nova | 13:34 | |
| rlrossit | dansmith: I don't think I care enough to continue fighting my side of the debate from yesterday. Plus it's almost through gate anyways. | 13:34 |
| mriedem | or dependency updates | 13:34 |
| johnthetubaguy | true | 13:34 |
| dansmith | rlrossit: okay, then I'll put this nerf gun away I guess | 13:34 |
| mriedem | sounds like labor day weekend shenanigans | 13:35 |
| johnthetubaguy | dansmith: I was wondering about pushing this stuff into the base class but I guess its not really general enough: https://review.openstack.org/#/c/217819/10/nova/objects/instance.py,cm | 13:35 |
| dansmith | johnthetubaguy: it specifically only applies to v2, and it won't always work like this | 13:36 |
| dansmith | johnthetubaguy: this is code I expect to drop as soon as we open mitaka and drop v1 | 13:36 |
| dansmith | johnthetubaguy: if v2 was structurally more different from v1 then we'd have to do more monkeywork in there to make it happen | 13:36 |
| dansmith | johnthetubaguy: I actually hope maybe we can get api and conductor pre-backporting things based on the service version before we have to bump to instance v3, making this even easier | 13:37 |
| johnthetubaguy | dansmith: yeah, true, I keep forgetting it maps to the primitive on the wire, we could in theory change to XML if we wanted | 13:37 |
| johnthetubaguy | dansmith: yeah, +1 for pre backporting | 13:38 |
| danpb | mriedem: can you point me to the source for this tempest test for TestStamp | 13:38 |
| dansmith | johnthetubaguy: we could, but then I would ragequit and leave this mess to you :P | 13:38 |
| johnthetubaguy | dansmith: yeah, only mentioned XML to annoy you :P | 13:38 |
| dansmith | heh | 13:38 |
| mriedem | danpb: https://review.openstack.org/#/c/218355/ | 13:38 |
| *** tojuvone has quit IRC | 13:40 | |
| mriedem | sdague: that network regression in large ops definitely started in the last 48 hours http://goo.gl/SjdGuU | 13:40 |
| sdague | mriedem: yep | 13:41 |
| * mriedem does git blame | 13:41 | |
| sdague | I've got an ER signature up there | 13:41 |
| mriedem | oh, i just pushed one too | 13:41 |
| sdague | http://status.openstack.org//elastic-recheck/#1492249 | 13:41 |
| *** tri2sing has quit IRC | 13:41 | |
| sdague | mriedem: that's why I registered the bug :) | 13:41 |
| *** tri2sing has joined #openstack-nova | 13:41 | |
| mriedem | and now that bug is duped | 13:41 |
| *** edmondsw has quit IRC | 13:42 | |
| mriedem | i prefer the query here https://review.openstack.org/#/c/220508/ | 13:42 |
| mriedem | since it touches the screen logs | 13:42 |
| *** tri2sing has quit IRC | 13:42 | |
| *** angdraug has joined #openstack-nova | 13:42 | |
| mriedem | let me update my patch quick | 13:42 |
| sdague | my message is from the screen logs as well | 13:43 |
| sdague | I just didn't scope it | 13:43 |
| *** ildikov has joined #openstack-nova | 13:43 | |
| sdague | but yeh, feel free to rejigger | 13:43 |
| mriedem | "Failed to deallocate network for instance" is really generic | 13:43 |
| mriedem | imo | 13:43 |
| *** dane-fichter has joined #openstack-nova | 13:44 | |
| *** eharney has joined #openstack-nova | 13:44 | |
| *** thumpba has joined #openstack-nova | 13:44 | |
| sdague | in the large-ops jobs, it turnes out it's not | 13:44 |
| dane-fichter | does anyone know whether Nova checks the MD5 checksum for images that it recieves from Glance? | 13:44 |
| sdague | because we run one very specific kind of test | 13:44 |
| sdague | in the general case, I agree | 13:45 |
| gibi | johnthetubaguy, dims: I fixed the problem with https://review.openstack.org/#/c/147514/ functional test passes now, could you check it please and reapprove? | 13:45 |
| dims | thanks gibi, i can +1 :) | 13:45 |
| johnthetubaguy | danpb: seems like these need a rebase again :( https://review.openstack.org/#/c/189362/26 | 13:45 |
| sdague | but, I'm fine with your approach instead, you want to put a delete in for my sig in the process? | 13:45 |
| mriedem | yeah it's in there https://review.openstack.org/#/c/220508/ | 13:45 |
| sdague | ok, cool | 13:45 |
| mriedem | there are 3 changes that landed in nova.network in the last 2 days, 2 are neutron related | 13:46 |
| mriedem | so that leaves https://github.com/openstack/nova/commit/fd471b2fc1fee6e859ee3657b53548a61c0f3a1e | 13:46 |
| *** ccarmack has joined #openstack-nova | 13:46 | |
| danpb | mriedem: so in the tempest test which failed, I see the virDomainAttach call in libvirtd.log and it shows flags=3, so the value is making it through the python | 13:46 |
| mriedem | sdague: which could maybe explain why we're hitting this https://github.com/openstack/nova/blob/master/nova/network/manager.py#L1068 | 13:47 |
| mriedem | which only happens if there is a race to release and disassociate with dnsmasq | 13:47 |
| mriedem | so maybe that extra logging slowed things down | 13:47 |
| mriedem | danpb: that's the latest patch | 13:47 |
| mriedem | danpb: you have to check logs for patch set 6 | 13:47 |
| mriedem | before the change depended on that nova change for the flags kwarg | 13:48 |
| gibi | dims: thanks! :) | 13:48 |
| danpb | mriedem: yes, oi looked in http://logs.openstack.org/55/218355/6/check/gate-tempest-dsvm-full/e0da291/logs/libvirt/libvirtd.txt.gz | 13:48 |
| *** jecarey has joined #openstack-nova | 13:48 | |
| mriedem | danpb: but is it on the 2nd instance? | 13:48 |
| danpb | mriedem: timestmp 17:41:07.357+0000 | 13:48 |
| mriedem | there are 2 instances in that test | 13:48 |
| mriedem | the attach is fine on the first | 13:48 |
| sdague | mriedem: well, the logging will allow greenlet to switch out, which might just mean something's a terrible race in the first place | 13:49 |
| danpb | oh, hmm, i missed that | 13:49 |
| danpb | let me check more closely | 13:49 |
| mriedem | then it does a snapshot and tries the attach on the 2nd instance (booted from the snapshot) and that fails | 13:49 |
| danpb | mriedem: ok, second instance is just a bit later | 13:50 |
| danpb | 2015-09-02 17:42:10.274+0000: 32080: debug : virDomainAttachDeviceFlags:10280 : dom=0x7fafa800f8a0, (VM: name=instance-00000062, uuid=3ac46e96-ab71-4808-9fe0-4d97f16f9dca), xml=<disk type="block" device="disk"> | 13:50 |
| danpb | shows the expected flags passed in | 13:50 |
| openstackgerrit | sulochan-acharya proposed openstack/nova: xapi: cleanup volume sr on live migration rollback https://review.openstack.org/217105 | 13:50 |
| openstackgerrit | Andrey Kurilin proposed openstack/python-novaclient: [Bug-Fix] Update requests body for quota-update https://review.openstack.org/220488 | 13:50 |
| mriedem | danpb: i don't think that's the right instance, sec | 13:51 |
| danpb | that's the second instance from teh TestStampPattern test - the first has name instance-0000000060 | 13:51 |
| mriedem | danpb: so this is the instance uuid you want 3ac46e96-ab71-4808-9fe0-4d97f16f9dca | 13:51 |
| danpb | that's what i have there | 13:51 |
| mriedem | http://logs.openstack.org/55/218355/6/check/gate-tempest-dsvm-full/e0da291/console.html#_2015-09-02_17_48_54_015 | 13:51 |
| mriedem | hrm | 13:52 |
| mriedem | double checking | 13:52 |
| mriedem | there have seriously been like 10 things going on every day this week | 13:52 |
| mriedem | http://logs.openstack.org/55/218355/6/check/gate-tempest-dsvm-full/e0da291/logs/screen-n-cpu.txt.gz#_2015-09-02_17_42_01_490 | 13:53 |
| mriedem | yeah so instance-00000062 | 13:53 |
| danpb | mriedem: so if you take the screen-n-cpu.log file and grep for grep screen-n-cpu.log TestStam | grep Generated | grep XML | less | grep "'<disk" | 13:53 |
| mriedem | yup i found that before | 13:53 |
| danpb | you'll see 3 results, the first attach, a detach, and a second attach | 13:53 |
| mriedem | double checking libvirt logs | 13:53 |
| danpb | both attaches are visible in libvirtd.logs | 13:53 |
| mriedem | http://logs.openstack.org/55/218355/6/check/gate-tempest-dsvm-full/e0da291/logs/libvirt/libvirtd.txt.gz#_2015-09-02_17_42_10_274 | 13:54 |
| *** erhudy1 has joined #openstack-nova | 13:54 | |
| danpb | and i see the drive-add command sent to qemu & success reply | 13:54 |
| danpb | and also see the device-add command sent & success reply | 13:55 |
| *** alaski is now known as lascii | 13:55 | |
| mriedem | hrm yeah | 13:55 |
| mriedem | ok, well | 13:55 |
| *** dansmith is now known as superdan | 13:56 | |
| mriedem | that at least explains away the flags kwarg thing as being a problem | 13:56 |
| danpb | so according to QEMU everything is normal :-( | 13:56 |
| mriedem | i.e. the blackhole theory | 13:56 |
| *** ctrath has joined #openstack-nova | 13:56 | |
| *** j_king has left #openstack-nova | 13:56 | |
| * danpb wonders if snapshotting is at all involved here | 13:56 | |
| mriedem | that's what i was thinking also | 13:56 |
| * danpb looks at just what this does with qemu | 13:57 | |
| mriedem | mtreinish: is test_stamp_pattern the only scenario test that runs volume snapshots? | 13:57 |
| *** eharney has quit IRC | 13:57 | |
| mriedem | and test_volume_boot_pattern is just instance snapshots | 13:57 |
| danpb | itsnot snapshotting the live guest RAM is it | 13:57 |
| *** zzzeek has joined #openstack-nova | 13:57 | |
| danpb | its just snapshotting a disk and swapping the volume on the next boot ? | 13:57 |
| mriedem | live snapshot? | 13:57 |
| openstackgerrit | Vladyslav Drok proposed openstack/nova: DO NOT MERGE - Triggering experimental pipeline https://review.openstack.org/220229 | 13:58 |
| mriedem | it shouldn't be doing a live snapshot since we haven't set the config option to allow that | 13:58 |
| mriedem | isn't there some volume snapshot code in the libvirt driver? | 13:58 |
| *** eharney has joined #openstack-nova | 13:58 | |
| *** apuimedo|lunch has quit IRC | 13:58 | |
| *** thangp has joined #openstack-nova | 13:59 | |
| danpb | ok, its definitely not doing live RAM snapshot - just storage | 13:59 |
| danpb | so it shouldn't be tickling any bugs in QEMU save/restore code paths | 13:59 |
| johnthetubaguy | superdan: the conductor drop looks close, but I am thinking it will clash with your conductor add: https://review.openstack.org/#/c/168857/3 | 13:59 |
| *** garyk has quit IRC | 13:59 | |
| danpb | mriedem: how frequently does the test fail ? i see you had that one success at least | 13:59 |
| superdan | johnthetubaguy: yep, after they both merge I will work on the major bump | 14:00 |
| mriedem | danpb: so before i made the tempest change depend on the nova change in ps7, it was failing in every run | 14:00 |
| mriedem | at least in one job | 14:00 |
| mriedem | danpb: i could drop the dependency on the nova change and see if it happens again | 14:00 |
| danpb | i wonder if that success was just random lucky timing | 14:01 |
| *** haomaiwang has quit IRC | 14:01 | |
| mriedem | maybe | 14:01 |
| *** haomaiwang has joined #openstack-nova | 14:01 | |
| mriedem | test_stamp_pattern was skipped a long time ago for a reason | 14:01 |
| danpb | mriedem: have you tried putting in a sleep(10) the second attach_device call ? | 14:01 |
| mriedem | nope | 14:01 |
| danpb | /before/ the second | 14:01 |
| mriedem | to make sure the guest is up? | 14:01 |
| danpb | i'm still wondering if there's a race with volume attach and the guest OS booting | 14:02 |
| danpb | yeah | 14:02 |
| mriedem | if we can ssh into the guest, will that be good enough to know if the guest is up? | 14:02 |
| mriedem | rather than sleep | 14:02 |
| mtreinish | mriedem: I think there are likely dedicated volume snapshot tests, but they likely don't ssh into the guest to verify things worke | 14:03 |
| danpb | mriedem: possibly - it depends what is handling ACPI events and whether that's likely to be running before sshd | 14:03 |
| mtreinish | d | 14:03 |
| mtreinish | mriedem: so they'll pass as long as the api said it did the right thing | 14:03 |
| danpb | mtreinish: lol, that could explain things | 14:03 |
| danpb | anyway, the test case code looks sane, and the libvirt/qemu interaction looks sane | 14:04 |
| mriedem | ok, so i'll drop the dependency on the nova change | 14:04 |
| danpb | so this just has race condition written all over it | 14:04 |
| mriedem | danpb: the other thing i was thinking last night was before we take the volume snapshot, | 14:04 |
| *** garyk has joined #openstack-nova | 14:04 | |
| mriedem | danpb: we don't check to see that the partition is gone from the guest after detaching the volume | 14:04 |
| mriedem | we just wait for cinder to tell us the volume is 'available' | 14:04 |
| mriedem | but that is async with libvirt doing the detach | 14:05 |
| danpb | mriedem: oh, another good point | 14:05 |
| mriedem | so i thought about adding a check for that | 14:05 |
| mriedem | mtreinish: isn't crazy about it, neither am i | 14:06 |
| danpb | though, the second guest is completely fresh so shouldn't share any state with the first guest | 14:06 |
| *** deepakcs has quit IRC | 14:06 | |
| *** alexpilotti_ has quit IRC | 14:06 | |
| mriedem | i was wondering if it wasn't the guest that was the issue but what's in the volume it's attaching | 14:06 |
| openstackgerrit | Alexander Schmidt proposed openstack/nova: libvirt: Acquire TCP ports for console during live migration https://review.openstack.org/215102 | 14:06 |
| openstackgerrit | Alexander Schmidt proposed openstack/nova: libvirt: use guest as parameter for get serial ports https://review.openstack.org/220519 | 14:06 |
| danpb | well what's in the volume shouldn't matter | 14:06 |
| danpb | eg we'd at lesat see the PCI device appear even if the filesystem was fubar | 14:06 |
| mriedem | oh, well, we do snapshot the first instance | 14:07 |
| mriedem | and boot the 2nd instance from that snapshot image | 14:07 |
| *** j_king has joined #openstack-nova | 14:07 | |
| mriedem | so mabye it's not as clean as we think | 14:07 |
| j_king | mriedem: sorry I've been silent on the nova/neutron vif-unplugged event thing... I've been spending the last couple of afternoons coming up with a way to reproduce the race condition in question | 14:07 |
| danpb | mriedem: hmm, yeah actually we're snappshotting the instance too | 14:07 |
| mriedem | danpb: we're snapshotting everything and it's dog in that test :) | 14:08 |
| mriedem | j_king: np | 14:08 |
| mriedem | j_king: i noticed no one replied to your operators ML query :( | 14:08 |
| danpb | mriedem: if the test is all about checking volumes, how about the kill the snapshotting of the guest | 14:08 |
| mriedem | j_king: i think the other operators must just all think you're crazy | 14:08 |
| j_king | mriedem: maybe. :S | 14:08 |
| danpb | mriedem: ie just kill the first instance, and then boot the second instance fresh from CONF.comput.image_ref | 14:08 |
| jroll | mtreinish: wanted to make sure you saw https://review.openstack.org/#/c/220223/ . vdrok is his irc name | 14:09 |
| mriedem | danpb: we could | 14:09 |
| danpb | that way we take guest snapshot out of the equation as a troublespot | 14:09 |
| *** burgerk has joined #openstack-nova | 14:09 | |
| j_king | mriedem: pretty sure I'm not. :p | 14:09 |
| danpb | and just focus on volume snapshot functionality | 14:09 |
| mriedem | danpb: however, that makes the test less interesting | 14:09 |
| mriedem | mtreinish: what do you think about that idea? ^ | 14:09 |
| danpb | mriedem: depends what the goal of the test is i guess - either you can instance snapshotting, or test volume snapshotting - this seems to be trying to test both at once | 14:09 |
| danpb | which certainly makes diagnosing failure much harder | 14:10 |
| mriedem | yeah | 14:10 |
| *** karimb has quit IRC | 14:10 | |
| mriedem | but does test the system in a more thorough way | 14:10 |
| mriedem | granted, if we have to put a sleep or partition removal check after the first detach, it's a bit hokey | 14:10 |
| mriedem | danpb: is the domain detachDevice async in libvirt/qemu? | 14:11 |
| danpb | yes | 14:11 |
| mriedem | blarg | 14:11 |
| mriedem | i was wondering if there was something we could check on the nova side to make sure the thing is attached/detached so tempest didn't have to check that | 14:11 |
| mriedem | can libvirt provide that info? | 14:12 |
| *** xyang1 has joined #openstack-nova | 14:12 | |
| mtreinish | mriedem: I guess we could change the test, but I think it's specifically trying to simulate a workflow where your restore a guest from snapshot and reattach it's volume | 14:12 |
| mtreinish | like a fault recovery kinda thing | 14:12 |
| mtreinish | but I don't really know | 14:12 |
| mriedem | mtreinish: ok | 14:12 |
| mtreinish | jroll: ok, i'll take a look | 14:12 |
| mtreinish | mriedem: I'm definitely fine with adding more targetted tests that do a single thing and ssh into the guest to verify it | 14:13 |
| mriedem | sdague: i think i knwo what the network race is | 14:13 |
| mriedem | omg omg omg | 14:13 |
| jroll | mtreinish: I'm not sure it's the right direction, I haven't talked to him about the approach yet | 14:13 |
| *** edmondsw has joined #openstack-nova | 14:13 | |
| *** VW_ has joined #openstack-nova | 14:14 | |
| *** VW_ has quit IRC | 14:14 | |
| mtreinish | jroll: it's not really, we shouldn't couple the backend configuration to the tests | 14:14 |
| jroll | mtreinish: yeah, that was my thought | 14:14 |
| *** nkrinner has quit IRC | 14:15 | |
| *** VW_ has joined #openstack-nova | 14:15 | |
| *** aix has quit IRC | 14:15 | |
| mriedem | sdague: mtreinish: bingo https://review.openstack.org/#/c/220525/ | 14:15 |
| *** matrohon has quit IRC | 14:15 | |
| *** eharney has quit IRC | 14:15 | |
| mriedem | so we obviously have a bug in nova-network when multihost is true | 14:17 |
| mriedem | given https://review.openstack.org/#/c/218860/ | 14:17 |
| mriedem | but with the failure rates we're seeing in the large ops job in the gate, unless we have a quick fix i'm going with the revert for now | 14:17 |
| danpb | a bug in an area of nova that's unmaintained. how shockingly surprising ;-P | 14:17 |
| mriedem | i assume someone running nova-network at scale is running with multihost=true | 14:18 |
| mriedem | besides nebula | 14:18 |
| mriedem | RPI | 14:18 |
| mriedem | *RIP | 14:18 |
| *** electrocucaracha has joined #openstack-nova | 14:18 | |
| mriedem | and no vishy around to help us fix it :( | 14:18 |
| bauwser | mtreinish: FWIW, the ids weren't captured as exclusions :( | 14:18 |
| mriedem | cburgess: does metacloud run nova-net? | 14:18 |
| *** electrocucaracha has quit IRC | 14:19 | |
| *** Guest4559 has quit IRC | 14:19 | |
| bauwser | mriedem: mtreinish: https://jenkins02.openstack.org/job/gate-tempest-dsvm-cells/1180/console | 14:19 |
| mriedem | superdan: or do you know big clouds that are running nova-net? | 14:19 |
| *** electrocucaracha has joined #openstack-nova | 14:19 | |
| mtreinish | bauwser: I dislike regex... :( | 14:19 |
| bauwser | mriedem: mtreinish: given the criticity, I'm providing a PS2 just changing the names | 14:19 |
| mriedem | bauwser: works for me | 14:20 |
| sdague | mriedem: oh... interesting | 14:20 |
| mriedem | bauwser: i'd prefer to not have a smoldering rubble of shit before a major US holiday | 14:20 |
| bauwser | heh | 14:20 |
| *** browne has joined #openstack-nova | 14:20 | |
| sdague | mriedem: well thanks kind of a bummer, but ok then | 14:20 |
| mriedem | sdague: yeah, so it'd be a good data point to know if operators running multihost=true at scale are tweaking rpc timeouts | 14:20 |
| mriedem | i can email the ops ML | 14:20 |
| sdague | mriedem: so, everyone is running that way in a large site | 14:21 |
| sdague | so, honestly, the thing is that we're driving an unrealistic throughput on one node there | 14:21 |
| sdague | because we are using fakevirt | 14:21 |
| sdague | I suspect starting 175 vms on one node at once is probably not a thing with an actual hypervisor :) | 14:22 |
| sdague | anyway, +A on the revert | 14:22 |
| mriedem | heh | 14:22 |
| *** mtanino has joined #openstack-nova | 14:22 | |
| mriedem | i was going to suggest setting rpc_timeout higher in devstack if multihost=true and running with the fake virt driver | 14:23 |
| mriedem | as an alternative to the revert | 14:23 |
| *** aix has joined #openstack-nova | 14:23 | |
| sdague | so, if you could propose that patch after the revert, that's fine, we'll try to recheck it a bit | 14:23 |
| mriedem | ok | 14:23 |
| sdague | I think a straight revert to get back to working is the right call | 14:23 |
| mriedem | yeah | 14:23 |
| mriedem | again, smoldering pile of shit | 14:23 |
| mriedem | not good before a 3 day weekend | 14:24 |
| sdague | mriedem: thanks for tracking that down | 14:24 |
| mriedem | np | 14:24 |
| mriedem | fixing one thing out of 10 this week, not bad | 14:24 |
| *** eharney has joined #openstack-nova | 14:26 | |
| openstackgerrit | Sylvain Bauza proposed openstack/nova: Fix Cells gate test by modifying the regressions regex https://review.openstack.org/220490 | 14:26 |
| *** karimb has joined #openstack-nova | 14:26 | |
| superdan | mriedem: not sure anymore.. I usually poke cburgess | 14:27 |
| mriedem | sdague: superdan: lascii: want to +W https://review.openstack.org/#/c/220490/ | 14:28 |
| *** nic has joined #openstack-nova | 14:29 | |
| *** tonytan4ever has joined #openstack-nova | 14:29 | |
| lascii | yeah, I'll get it | 14:29 |
| * superdan is on a call | 14:29 | |
| *** vishwanathj has joined #openstack-nova | 14:32 | |
| mriedem | danpb: tonyb had some separate investigation: https://bugs.launchpad.net/nova/+bug/1492026/comments/3 | 14:33 |
| openstack | Launchpad bug 1492026 in OpenStack Compute (nova) "domain.attachDeviceFlags randomly doesn't do anything in libvirt" [Medium,Confirmed] | 14:33 |
| mriedem | I suspect qemu/ guest kernel as we can see at: | 14:33 |
| mriedem | http://logs.openstack.org/55/218355/6/check/gate-tempest-dsvm-full/e0da291/console.html#_2015-09-02_17_48_53_953that the new PCI device was never visible to the guest it should have been 00:05.0. | 14:33 |
| *** y_sawai has quit IRC | 14:35 | |
| *** nic has quit IRC | 14:36 | |
| danpb | mriedem: yeah, why i'd like to take guest snapshot out of the equation and just focus on volume snapshot + hotplug | 14:37 |
| danpb | its possible we shtudown the guest while it was in a bad state wrt to detaching the original volume | 14:37 |
| danpb | and now we restart it all things are fubar | 14:38 |
| openstackgerrit | Joe Cropper proposed openstack/nova: Ensure Nova metrics use coherent data values https://review.openstack.org/219153 | 14:38 |
| mriedem | danpb: i'm going to add a loop after the first volume detach that waits for the partition to be gone from the guest before we snapshot the volume and instance | 14:38 |
| mriedem | rather than remove the snapshots | 14:38 |
| mriedem | if that is still wonky, then i agree we can remove the instance snapshot part from the test | 14:38 |
| danpb | mriedem: ok, hopefully that should be enough | 14:38 |
| mriedem | danpb: still, there isn't a way for libvirt to tell us when things are actually gone from the guest? | 14:39 |
| mriedem | so we could check that in nova? | 14:39 |
| *** mdrabe has quit IRC | 14:39 | |
| danpb | mriedem: with new enough libvirt + QEMU nova should receive an event from libvirt | 14:39 |
| *** apuimedo has joined #openstack-nova | 14:39 | |
| mriedem | like how new? | 14:40 |
| mriedem | fedora23 new? | 14:40 |
| danpb | VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED | 14:40 |
| jwcroppe | superdan, jaypipes: the 219153 patch above fixes that odd 1 second race condition in the metrics stuff we talked about earlier in the week | 14:40 |
| *** vilobhmm_11 has joined #openstack-nova | 14:40 | |
| danpb | mriedem: v1.1.1 or later | 14:40 |
| mriedem | oo | 14:40 |
| mriedem | we have 1.2.2 | 14:40 |
| *** tri2sing has joined #openstack-nova | 14:41 | |
| *** coolsvap_ has quit IRC | 14:41 | |
| danpb | mriedem: we'd have to wire up the libvirt event thread to handle vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED, myDomainEventDeviceRemovedCallback, None) | 14:42 |
| danpb | currently we only monitor for libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE, | 14:42 |
| *** dave-mccowan has quit IRC | 14:43 | |
| mriedem | ok, i'm taking notes in the bug but we can keep that for a later change in mitaka | 14:43 |
| mriedem | and i'll leave a note in the tempest test code that says we are doing a thing as a workaround for that bug | 14:43 |
| danpb | ok | 14:44 |
| *** y_sawai has joined #openstack-nova | 14:45 | |
| *** venkat_p has quit IRC | 14:45 | |
| *** annegentle has quit IRC | 14:45 | |
| *** annegentle has joined #openstack-nova | 14:46 | |
| *** rotbeard has quit IRC | 14:46 | |
| *** j_king has quit IRC | 14:47 | |
| *** tonytan4ever has quit IRC | 14:47 | |
| mriedem | sdague: i'm +2 on https://review.openstack.org/#/c/220488/ now | 14:48 |
| mriedem | after that merges we'll cut 2.28.1 | 14:48 |
| mriedem | and bump g-r | 14:48 |
| *** yamahata has joined #openstack-nova | 14:48 | |
| openstackgerrit | Andrey Kurilin proposed openstack/python-novaclient: [Bug-Fix] Update requests body for quota-update https://review.openstack.org/220488 | 14:49 |
| sdague | mriedem: I'm going to +A it now, I'm going to be in an out for rest of day, hopefully it gets past tests | 14:50 |
| mriedem | i'll babysit | 14:50 |
| mriedem | $7/hour plus access to your fridge | 14:50 |
| sdague | I have a fridge full of beer, but that doesn't help you much | 14:52 |
| openstackgerrit | Nicolas Simonds proposed openstack/nova: libvirt: make snapshot use RBD snapshot/clone when available https://review.openstack.org/205282 | 14:52 |
| *** ildikov has quit IRC | 14:53 | |
| *** tonytan4ever has joined #openstack-nova | 14:53 | |
| mriedem | cold cuts? | 14:55 |
| mriedem | flav-r-ice? | 14:55 |
| *** j_king has joined #openstack-nova | 14:56 | |
| openstackgerrit | Nicolas Simonds proposed openstack/nova: libvirt: Fix/implement revert-resize for RBD-backed images https://review.openstack.org/187395 | 14:56 |
| openstackgerrit | Nicolas Simonds proposed openstack/nova: libvirt: Split out resize_image logic from create_image https://review.openstack.org/187857 | 14:56 |
| *** drankis has quit IRC | 14:57 | |
| *** dane-fichter has quit IRC | 14:57 | |
| *** mdrabe has joined #openstack-nova | 14:58 | |
| openstackgerrit | Alexis Lee proposed openstack/nova: Unify on _schedule_instances https://review.openstack.org/219761 | 14:58 |
| openstackgerrit | Alexis Lee proposed openstack/nova: Unify on _schedule_instances https://review.openstack.org/219761 | 14:59 |
| *** dave-mccowan has joined #openstack-nova | 15:00 | |
| *** haomaiwang has quit IRC | 15:01 | |
| *** haomaiwang has joined #openstack-nova | 15:01 | |
| openstackgerrit | John Garbutt proposed openstack/nova: New sensible network bandwidth quota values in Nova tests https://review.openstack.org/173467 | 15:03 |
| *** jistr is now known as jistr|call | 15:05 | |
| *** stefanb has quit IRC | 15:06 | |
| *** nic has joined #openstack-nova | 15:06 | |
| *** alexschm has quit IRC | 15:07 | |
| *** mtanino has quit IRC | 15:07 | |
| *** angdraug has quit IRC | 15:08 | |
| *** unicell1 has joined #openstack-nova | 15:09 | |
| *** karki_ has joined #openstack-nova | 15:10 | |
| *** emagana has joined #openstack-nova | 15:10 | |
| *** dannywilson has joined #openstack-nova | 15:10 | |
| karki_ | I am trying to get programatic control over my ESXi host using python. One way this could be done is by using the PyVmomi API. But by the looks of it, it does not seem as though it is actively maintained. So I was looking for alternative ways to control the ESXi instance. Is the oslo.vmware a suitable replacement for pyVmomi? or otherwise can someone educate me about how openstack handles communication with ESX/ vSphere | 15:10 |
| *** dannywilson has quit IRC | 15:10 | |
| *** karimb has quit IRC | 15:11 | |
| *** dannywilson has joined #openstack-nova | 15:11 | |
| *** unicell has quit IRC | 15:11 | |
| mriedem | garyk: ^ | 15:11 |
| *** VW_ has quit IRC | 15:12 | |
| mriedem | karki_: there is also an #openstack-vmware channel | 15:12 |
| *** VW_ has joined #openstack-nova | 15:12 | |
| *** VW_ has quit IRC | 15:14 | |
| *** burgerk has quit IRC | 15:14 | |
| *** venkat_p has joined #openstack-nova | 15:14 | |
| *** drankis has joined #openstack-nova | 15:14 | |
| *** y_sawai has quit IRC | 15:15 | |
| *** y_sawai has joined #openstack-nova | 15:15 | |
| *** VW_ has joined #openstack-nova | 15:16 | |
| *** y_sawai has quit IRC | 15:16 | |
| *** vilobhmm_11 has quit IRC | 15:16 | |
| openstackgerrit | Daniel Berrange proposed openstack/nova: ironic: convert driver to use nova.objects.ImageMeta https://review.openstack.org/189362 | 15:17 |
| openstackgerrit | Daniel Berrange proposed openstack/nova: objects: convert config drive to use ImageMeta object https://review.openstack.org/189323 | 15:17 |
| openstackgerrit | Daniel Berrange proposed openstack/nova: objects: add virtual 'image_meta' property to Instance object https://review.openstack.org/190145 | 15:17 |
| openstackgerrit | Daniel Berrange proposed openstack/nova: libvirt: convert to use instance.image_meta property https://review.openstack.org/190146 | 15:17 |
| openstackgerrit | Daniel Berrange proposed openstack/nova: compute: convert manager to use nova.objects.ImageMeta https://review.openstack.org/189685 | 15:17 |
| superdan | johnthetubaguy: you're okay with things like this ^ still going in, right? | 15:18 |
| superdan | ah, I guess so, you +2d yesterday | 15:18 |
| openstackgerrit | Brian Elliott proposed openstack/nova: Don't "lock" the DB on expand dry run https://review.openstack.org/215288 | 15:19 |
| johnthetubaguy | superdan: let me take a quick look again | 15:20 |
| *** karki_ has quit IRC | 15:21 | |
| superdan | that first one is going to conflict with my stuff in the gate | 15:21 |
| danpb | superdan: we're only putting the first 2 in | 15:21 |
| superdan | so I'd like to hold off a bit | 15:21 |
| superdan | danpb: okay | 15:21 |
| danpb | superdan: the third patch has a -2 to block it till mitaka | 15:21 |
| superdan | okay cool | 15:21 |
| danpb | so we'll have the ironic conversion, ad the config drive property one | 15:21 |
| danpb | the other three will wait | 15:21 |
| johnthetubaguy | superdan: yeah, it was +Wed when we cut the branch, so gave it a "technical" FFE, with the trade off about blocking that middle patch | 15:22 |
| johnthetubaguy | so +1 all the previous conversation basically | 15:22 |
| superdan | okay | 15:22 |
| danpb | the idea is we'll have all the virt drivers converted to use ImageMeta objects | 15:22 |
| superdan | the first one does touch RPC, which is why I brought it up | 15:23 |
| danpb | superdan: hmm, which one touchs RPC ?? | 15:23 |
| superdan | gah, my stuff just failed the cells job | 15:23 |
| johnthetubaguy | danpb: can you -2 those other too patch to avoid and confusion? | 15:23 |
| danpb | johnthetubaguy: ok | 15:23 |
| superdan | danpb: any one that touches an object version is an rpc change | 15:23 |
| danpb | oh i see what you mean | 15:24 |
| *** karimb has joined #openstack-nova | 15:24 | |
| superdan | oh, no that was the vmware job | 15:24 |
| superdan | it just reset the whole thing from position 3 | 15:24 |
| *** armax has joined #openstack-nova | 15:25 | |
| *** tpeoples has joined #openstack-nova | 15:26 | |
| *** karki_ has joined #openstack-nova | 15:27 | |
| karki_ | mriedem : Thanks a lot :) I'll ping there. | 15:27 |
| *** jistr|call is now known as jistr | 15:28 | |
| *** VW__ has joined #openstack-nova | 15:28 | |
| karki_ | but out of curiosity - how is the sex layer handled in nova - which API do you use to create/destroy/manage VMs on ESXi? | 15:28 |
| karki_ | *ESX layer | 15:28 |
| *** VW_ has quit IRC | 15:29 | |
| *** y_sawai has joined #openstack-nova | 15:29 | |
| *** afazekas__ has quit IRC | 15:30 | |
| mriedem | karki_: the nova vcenter driver uses oslo.vmware which i thought used pyvmomi | 15:30 |
| mriedem | or whatever that library is called | 15:30 |
| dims | mriedem: suds | 15:30 |
| mriedem | dims: oslo.vmware isn't using that other library yet? | 15:30 |
| mriedem | i thought that's what hartsocks went off to work on | 15:30 |
| dims | mriedem: specifically suds-jurko library on pypi | 15:30 |
| dims | mriedem: y, we never moved to pyvmomi | 15:31 |
| mriedem | oh | 15:31 |
| mriedem | yeah looks kind of dead https://github.com/vmware/pyvmomi/commits/master | 15:31 |
| *** baoli_ has quit IRC | 15:31 | |
| *** baoli has joined #openstack-nova | 15:32 | |
| mriedem | oh yeah, so we should also discuss https://review.openstack.org/#/c/220279/ | 15:34 |
| mriedem | superdan: johnthetubaguy: lascii: ^ | 15:34 |
| *** y_sawai has quit IRC | 15:34 | |
| mriedem | alex_xu: brought up the point that v2 api allows spaces at the beginning and end of a name | 15:34 |
| mriedem | with that change things are better in v2.1 but not fully backwards compatible with v2.0 | 15:34 |
| superdan | I'm not sure I think that's a problem | 15:34 |
| mriedem | i think it's kind of pointless too, but | 15:35 |
| *** VW__ has quit IRC | 15:36 | |
| superdan | there are certainly lots of things that self-sanitize leading a trailing whitespace | 15:36 |
| superdan | so I dunno | 15:36 |
| superdan | no strong opinion I guess | 15:36 |
| mriedem | me neither | 15:36 |
| mriedem | that's why i brought it up, i wanted someone else to decide :) | 15:36 |
| *** VW_ has joined #openstack-nova | 15:36 | |
| superdan | heh | 15:36 |
| mriedem | like johnthetubaguy so we can blame him when it turns out to be the wrong decision | 15:36 |
| * superdan nods | 15:37 | |
| karki_ | So can I use oslo.vmware in a standalone project - to control ESXi? (From what I saw, it gives me similar functionality to that of PyVmomi). | 15:37 |
| mriedem | karki_: you should really ask in that #openstack-vmware channel | 15:37 |
| karki_ | ok, Thanks :) | 15:37 |
| lascii | I have a preference for not allowing spaces, but not a strong opinion. I think it would be silly for a user to consider " foo" different than " foo" | 15:37 |
| mriedem | a no answer there is about the same as a no answer here right now | 15:37 |
| * karki_ leaves to vmware land | 15:37 | |
| superdan | lascii: leading/trailing you mean right? | 15:38 |
| lascii | superdan: right, just those | 15:38 |
| superdan | lascii: and self-sanitizing or failing? | 15:38 |
| mriedem | superdan: looks like i won :) https://review.openstack.org/#/c/220157/ | 15:38 |
| mriedem | eat it proposal bot | 15:39 |
| lascii | I would probably go with self-sanitizing | 15:39 |
| superdan | mriedem: nice | 15:39 |
| *** zzzeek has quit IRC | 15:39 | |
| superdan | lascii: bzzzzt, sorry we were looking for "failing" | 15:39 |
| lascii | superdan: heh | 15:39 |
| lascii | I didn't say "final answer" | 15:39 |
| superdan | heh | 15:39 |
| *** VW_ has quit IRC | 15:41 | |
| *** marzif has joined #openstack-nova | 15:42 | |
| *** mtanino has joined #openstack-nova | 15:44 | |
| mriedem | i just got the who wants to be a millionaire reference | 15:45 |
| johnthetubaguy | mriedem: :) | 15:47 |
| *** browne has quit IRC | 15:48 | |
| johnthetubaguy | lascii: I quite like your idea here, accept them, but strip them | 15:49 |
| lascii | it was superdans idea, I just applied a slight preference to it | 15:49 |
| johnthetubaguy | ah, I miss read the comments | 15:49 |
| johnthetubaguy | seems like a good trade off to me | 15:49 |
| mriedem | the only thing i don't like about that is doing it across the board where parameter_types.name is used in the schema validation | 15:49 |
| *** e0ne has quit IRC | 15:50 | |
| johnthetubaguy | mriedem: oh, so we could do it in a custom validator? | 15:50 |
| mriedem | well, i'd think we'd need to change the regex for parameter_types.name to allow leading/trailing whitespace, | 15:50 |
| *** unicell1 has quit IRC | 15:50 | |
| mriedem | and then add more code after that in the API plugins which actually does the sanitize | 15:51 |
| mriedem | in at least server API, but possibly any that use that parameter_types.name regex | 15:51 |
| mriedem | which to me kind of sucks | 15:51 |
| mriedem | unless there is some kind of magical callback you can hook into the json schema validator that calls a method with the value when it's hit | 15:51 |
| mriedem | alex_xu: ^ any idea of that's possible?> | 15:52 |
| *** lyanchih has joined #openstack-nova | 15:52 | |
| lascii | I'm not strongly against failing there, as I commented on the ticket. but sanitizing would be nicer to users | 15:52 |
| *** romainh has quit IRC | 15:53 | |
| johnthetubaguy | mriedem: trying to find what they did for v2.1 stuff already | 15:53 |
| mriedem | https://www.youtube.com/watch?v=aWEnqC1uPu0 | 15:53 |
| johnthetubaguy | https://github.com/openstack/nova/blob/master/nova/api/validation/validators.py#L135 | 15:54 |
| mriedem | yeah maybe we can hook in like _validate_datetime_format | 15:55 |
| *** dave-mccowan has quit IRC | 15:56 | |
| mriedem | but that's just passed in the value, we don't know if it's a name or what | 15:57 |
| *** kylek3h_ has joined #openstack-nova | 15:57 | |
| *** stefan_amann has quit IRC | 15:58 | |
| johnthetubaguy | mriedem: yeah, its missing info, bummer | 15:58 |
| *** eglynn has joined #openstack-nova | 15:58 | |
| mriedem | but i guess you follow the additionalProperties model for name | 15:58 |
| *** y_sawai has joined #openstack-nova | 15:59 | |
| mriedem | _soft_validate_name or something | 15:59 |
| *** ildikov has joined #openstack-nova | 15:59 | |
| *** rdopiera has quit IRC | 15:59 | |
| johnthetubaguy | you could match on the regex, but thats was too evil | 16:00 |
| johnthetubaguy | s/was/way/ | 16:00 |
| mriedem | actually, i think you could just do it like minimum and maximum are validated | 16:00 |
| mriedem | let me pull that change down and test this out | 16:00 |
| *** browne has joined #openstack-nova | 16:00 | |
| *** haomaiwang has quit IRC | 16:01 | |
| *** kylek3h has quit IRC | 16:01 | |
| *** haomaiwang has joined #openstack-nova | 16:01 | |
| *** kylek3h_ has quit IRC | 16:02 | |
| johnthetubaguy | mriedem: oh, like a custom validator, that should work | 16:03 |
| *** ijw_ has joined #openstack-nova | 16:04 | |
| *** jwcroppe_ has joined #openstack-nova | 16:06 | |
| *** jlanoux has quit IRC | 16:08 | |
| *** VW_ has joined #openstack-nova | 16:08 | |
| *** dave-mccowan has joined #openstack-nova | 16:08 | |
| *** jwcroppe has quit IRC | 16:09 | |
| superdan | lascii: so do we have a handle on the cells job failure isuse? | 16:10 |
| mriedem | yes https://review.openstack.org/#/c/220490/ | 16:10 |
| superdan | okay | 16:11 |
| mriedem | large-ops should be ok now too https://review.openstack.org/#/c/220525/ | 16:11 |
| superdan | cool | 16:11 |
| superdan | my stack just failed the cells job, as did a few before it | 16:11 |
| *** nic has quit IRC | 16:12 | |
| *** nic has joined #openstack-nova | 16:13 | |
| *** nic has quit IRC | 16:14 | |
| *** nic has joined #openstack-nova | 16:14 | |
| *** sahid has quit IRC | 16:17 | |
| *** jistr has quit IRC | 16:18 | |
| mriedem | johnthetubaguy: oh but i don't know if this actually coerces the value for us.... | 16:18 |
| mriedem | at least what i'm trying | 16:18 |
| mriedem | i guess i'll find out in my unit test | 16:18 |
| johnthetubaguy | mriedem: we were able to drop extra params, I think, at least I thought they had tests for that | 16:18 |
| *** claudiub has joined #openstack-nova | 16:19 | |
| mriedem | i'm failing and it's lunchtime | 16:21 |
| mriedem | bbiab | 16:21 |
| *** mriedem is now known as mriedem_lunch | 16:21 | |
| cburgess | mriedem superdan We run a mix of nova-net and neutron now. | 16:21 |
| *** artom has quit IRC | 16:23 | |
| *** ccarmack has left #openstack-nova | 16:24 | |
| *** artom has joined #openstack-nova | 16:25 | |
| *** tjones has joined #openstack-nova | 16:28 | |
| *** unicell has joined #openstack-nova | 16:31 | |
| *** tjones has quit IRC | 16:33 | |
| *** alex_klimov has quit IRC | 16:34 | |
| andrearosa | I am looking at the nova-manage code and it seems to me that we do not have any kind of authentication or checks, anyone can run the command without using any credentials, am I reading it correctly? | 16:35 |
| *** atuvenie has quit IRC | 16:35 | |
| *** kiran-r has joined #openstack-nova | 16:35 | |
| *** kiran-r has quit IRC | 16:35 | |
| *** kylek3h has joined #openstack-nova | 16:36 | |
| *** bswartz has quit IRC | 16:37 | |
| *** edmondsw has quit IRC | 16:38 | |
| *** tjones has joined #openstack-nova | 16:39 | |
| *** gerhardq1x has joined #openstack-nova | 16:40 | |
| *** davideagnello has quit IRC | 16:40 | |
| *** vilobhmm_11 has joined #openstack-nova | 16:41 | |
| *** drankis has quit IRC | 16:41 | |
| *** davideagnello has joined #openstack-nova | 16:41 | |
| *** kylek3h has quit IRC | 16:42 | |
| lascii | andrearosa: the credentials would need to be in the nova.conf file | 16:46 |
| lascii | without those it would not work | 16:46 |
| *** garyk has quit IRC | 16:47 | |
| *** eharney has quit IRC | 16:47 | |
| lascii | superdan: was out to lunch, but as mriedem pointed out Tempest renamed some tests to slip past the blacklist | 16:47 |
| andrearosa | lascii: thanks | 16:48 |
| *** eharney has joined #openstack-nova | 16:48 | |
| superdan | cool | 16:50 |
| openstackgerrit | John Garbutt proposed openstack/nova: Add a finish log https://review.openstack.org/183977 | 16:50 |
| *** marzif has quit IRC | 16:51 | |
| *** karki_ has left #openstack-nova | 16:52 | |
| *** vilobhmm_111 has joined #openstack-nova | 16:53 | |
| *** venkat_p has quit IRC | 16:54 | |
| *** vilobhmm_11 has quit IRC | 16:54 | |
| openstackgerrit | John Garbutt proposed openstack/nova: Provide working SQLA_VERSION attribute https://review.openstack.org/165993 | 16:55 |
| *** karimb has quit IRC | 16:55 | |
| *** derekh has quit IRC | 16:59 | |
| *** ijw_ has quit IRC | 16:59 | |
| *** haomaiwang has quit IRC | 17:01 | |
| *** haomaiwang has joined #openstack-nova | 17:01 | |
| *** tjones1 has joined #openstack-nova | 17:01 | |
| *** openstackgerrit_ has joined #openstack-nova | 17:02 | |
| *** yamahata has quit IRC | 17:03 | |
| *** dims has quit IRC | 17:03 | |
| *** edmondsw has joined #openstack-nova | 17:03 | |
| *** tjones has quit IRC | 17:04 | |
| *** VikasC has joined #openstack-nova | 17:05 | |
| openstackgerrit | sulochan-acharya proposed openstack/nova: xapi-tools: fixes cache cleaner script https://review.openstack.org/209526 | 17:07 |
| *** marzif has joined #openstack-nova | 17:07 | |
| *** artom has quit IRC | 17:08 | |
| *** exploreshaifali has joined #openstack-nova | 17:09 | |
| *** jwcroppe_ is now known as jwcroppe | 17:09 | |
| *** annegentle has quit IRC | 17:10 | |
| *** ijw_ has joined #openstack-nova | 17:11 | |
| openstackgerrit | Joe Cropper proposed openstack/nova: Ensure Nova metrics use coherent data values https://review.openstack.org/219153 | 17:12 |
| *** ijw__ has joined #openstack-nova | 17:12 | |
| *** danpb has quit IRC | 17:14 | |
| *** yassine__ has quit IRC | 17:15 | |
| *** tonytan4ever has quit IRC | 17:15 | |
| *** eharney has quit IRC | 17:15 | |
| *** ijw_ has quit IRC | 17:15 | |
| *** tjones1 has quit IRC | 17:18 | |
| *** VW_ has quit IRC | 17:18 | |
| *** VW_ has joined #openstack-nova | 17:19 | |
| *** mpaolino has quit IRC | 17:19 | |
| *** VikasC has quit IRC | 17:19 | |
| *** tonytan4ever has joined #openstack-nova | 17:20 | |
| *** dims has joined #openstack-nova | 17:21 | |
| *** dims_ has joined #openstack-nova | 17:21 | |
| *** dims_ is now known as dimsum__ | 17:23 | |
| *** VW_ has quit IRC | 17:23 | |
| *** gmmaha has joined #openstack-nova | 17:23 | |
| *** gmmaha has quit IRC | 17:24 | |
| *** dims has quit IRC | 17:25 | |
| johnthetubaguy | ndipanov: a funky looking BDM bug fix could do with your expert eye please, if you get chance: https://review.openstack.org/#/c/171984/12 | 17:30 |
| openstackgerrit | Joe Cropper proposed openstack/nova: Ensure Nova metrics use coherent data values https://review.openstack.org/219153 | 17:30 |
| *** eharney has joined #openstack-nova | 17:31 | |
| *** sachin_ has joined #openstack-nova | 17:32 | |
| *** jaypipes has quit IRC | 17:33 | |
| *** mriedem_lunch is now known as mriedem | 17:33 | |
| *** lbeliveau has joined #openstack-nova | 17:34 | |
| *** ZZelle_ has joined #openstack-nova | 17:34 | |
| *** Piet has joined #openstack-nova | 17:35 | |
| *** shoutm has quit IRC | 17:35 | |
| mriedem | was this ok given we're past string freeze? https://review.openstack.org/#/c/219768/ | 17:37 |
| mriedem | i'm not sure how that works for novaclient | 17:37 |
| superdan | well, | 17:38 |
| *** lyanchih has quit IRC | 17:38 | |
| superdan | they were untranslated before, if the translators miss this because of string freeze, then it's still untranslated | 17:39 |
| superdan | so seems like no worse to me | 17:39 |
| mriedem | honestly i don't even know how i18n works for novaclient | 17:39 |
| mriedem | there is no locales dir or bable config | 17:39 |
| mriedem | *babel | 17:39 |
| *** raildo is now known as raildo-afk | 17:40 | |
| mriedem | melwitt: do you know? ^ | 17:40 |
| *** claudiub has quit IRC | 17:42 | |
| *** yamahata has joined #openstack-nova | 17:44 | |
| *** SourabhP has joined #openstack-nova | 17:44 | |
| dimsum__ | mriedem: python-novaclient is not even listed on transifex - https://www.transifex.com/openstack/public/ | 17:44 |
| mriedem | yeah.... | 17:44 |
| mriedem | i'm pretty sure that the i18n stuff in novaclient has 0 effect | 17:45 |
| *** garyk has joined #openstack-nova | 17:45 | |
| mriedem | python-openstackclient has a babel config and some locale files https://github.com/openstack/python-openstackclient/tree/master/python-openstackclient/locale | 17:45 |
| *** VW_ has joined #openstack-nova | 17:45 | |
| *** tjones has joined #openstack-nova | 17:47 | |
| *** raildo-afk is now known as raildo | 17:48 | |
| *** SourabhP has quit IRC | 17:49 | |
| *** annegentle has joined #openstack-nova | 17:49 | |
| mriedem | i've bombed the ML | 17:49 |
| *** vladikr has quit IRC | 17:50 | |
| openstackgerrit | Geza Gemes proposed openstack/nova: Do not block instance boot on net without subnet https://review.openstack.org/200046 | 17:50 |
| *** nic has quit IRC | 17:52 | |
| *** y_sawai has quit IRC | 17:53 | |
| *** dannywilson has quit IRC | 17:53 | |
| *** dannywilson has joined #openstack-nova | 17:54 | |
| dimsum__ | mriedem: takes one review and one email to the i18n ML to get things going...so we should start doing it. | 17:55 |
| *** devlaps has joined #openstack-nova | 17:56 | |
| mriedem | doffm: ctrath: rlrossit: ^ i guess something to do if anyone wants to | 17:57 |
| mriedem | i already sent the email | 17:57 |
| mriedem | i don't know how the migration from transifex to zanata is going though | 17:58 |
| ctrath | mriedem: so, remove all i18n reference from novaclient? | 17:58 |
| mriedem | no | 17:58 |
| *** tjones has quit IRC | 17:58 | |
| mriedem | add the babel config stuff so it can be translated and messages extracted/compiled | 17:58 |
| mriedem | like this in keystoneclient https://github.com/openstack/python-keystoneclient/commit/a8f7879fcfec1f2b64eba96b75fca7d75ad8e711 | 17:59 |
| ctrath | ok.. doffm: are you ready to take this on? | 17:59 |
| mriedem | doffm: can't actually push code yet | 17:59 |
| mriedem | :) | 17:59 |
| ctrath | ah, sh*t | 17:59 |
| mriedem | i'd wait to hear back from the i18n people in the ML though | 17:59 |
| *** jcoufal_ has quit IRC | 17:59 | |
| ctrath | ok | 18:00 |
| *** eglynn has quit IRC | 18:00 | |
| *** haomaiwang has quit IRC | 18:01 | |
| *** tjones has joined #openstack-nova | 18:01 | |
| *** haomaiwang has joined #openstack-nova | 18:01 | |
| *** ociuhandu has quit IRC | 18:02 | |
| *** SourabhP has joined #openstack-nova | 18:03 | |
| *** vladikr has joined #openstack-nova | 18:03 | |
| *** lucasagomes is now known as lucas-beer | 18:05 | |
| dimsum__ | ctrath: mriedem: see my link to the translations wiki, it's very simple set of steps, no need to wait. once we get the review in, we can drop an email to openstack-i18n mailing list to enable translations the project. zanata vs transifex does not matter to what we need to do in the repo itself | 18:05 |
| dimsum__ | also we need to add translation-jobs in project-config | 18:05 |
| mriedem | ok, i don't think this is goign to happen today | 18:06 |
| mriedem | the guy that plans on doing it, doffm, doesn't have CLA approval yet, | 18:06 |
| mriedem | maybe next week | 18:06 |
| mriedem | johnthetubaguy: if you're around, i pushed a change to do a novaclient 2.28.1 release https://review.openstack.org/#/c/220604/ | 18:07 |
| ctrath | mriedem, dimsum__: how soon does this need to be pushed? | 18:07 |
| mriedem | very low priority | 18:07 |
| dimsum__ | yep | 18:08 |
| *** ftersin has left #openstack-nova | 18:09 | |
| cburgess | mriedem I'm not sure that tempest bug from last night is completely valid. max_local_block_devices=0 seems to block non BFV just fine for me. Granted I'm not running a modern vintage of nova. | 18:12 |
| mriedem | at '76? | 18:12 |
| mriedem | *a | 18:12 |
| mriedem | cburgess: so just a regular old: nova boot --flavor 1 --image foo bar? | 18:12 |
| mriedem | that falis? | 18:12 |
| cburgess | mriedem: retesting it now | 18:13 |
| mriedem | mtreinish: well this is fun, test_stamp_pattern is golden on master but fails kilo and juno | 18:15 |
| cburgess | mriedem Yeah I'm getting back the block device mapping error. | 18:15 |
| mriedem | cburgess: huh, i'm surprised | 18:15 |
| cburgess | Let me re-test this in a devstack on master. | 18:16 |
| *** unicell has quit IRC | 18:16 | |
| mriedem | i was just about to do that | 18:16 |
| *** unicell has joined #openstack-nova | 18:16 | |
| mtreinish | mriedem: feature flag use volumes :) | 18:16 |
| *** thangp has quit IRC | 18:16 | |
| *** VW_ has quit IRC | 18:16 | |
| mriedem | mtreinish: ? | 18:16 |
| cburgess | mriedem Oh.. I think I know why.. this AZ is configured to use CEPH for nova and cinder. | 18:16 |
| mtreinish | mriedem: to not run it on stable | 18:17 |
| cburgess | So even nova roots are block devices. I bet thats why. | 18:17 |
| *** VW_ has joined #openstack-nova | 18:17 | |
| *** IanGovett has quit IRC | 18:18 | |
| *** penick has joined #openstack-nova | 18:19 | |
| *** kiran-r has joined #openstack-nova | 18:21 | |
| *** VW_ has quit IRC | 18:21 | |
| mriedem | cburgess: yeah, works fine for me with master devstack and max_local_block_devices=0 | 18:23 |
| mriedem | when not bfv | 18:23 |
| mriedem | mtreinish: that would be an easy out, but this should also work on stable | 18:23 |
| mtreinish | mriedem: heh, I was just joking | 18:23 |
| mriedem | i swear to god this damn test is going to kill me | 18:24 |
| mtreinish | mriedem: I tried to warn you :) | 18:25 |
| mtreinish | there is a reason its been skipped for like 2 years | 18:25 |
| *** lpetrut has quit IRC | 18:25 | |
| *** ndipanov has quit IRC | 18:26 | |
| *** e0ne has joined #openstack-nova | 18:29 | |
| cburgess | mriedem Yeah so its working for me because CEPH makes all the requests block devices. | 18:29 |
| *** chhavi has quit IRC | 18:29 | |
| *** vladikr has quit IRC | 18:31 | |
| *** dannywilson has quit IRC | 18:31 | |
| *** erhudy1 has quit IRC | 18:31 | |
| *** tonytan4ever has quit IRC | 18:31 | |
| *** kiran-r has quit IRC | 18:32 | |
| *** dannywilson has joined #openstack-nova | 18:32 | |
| mriedem | mtreinish: well, i guess i can go back to trying to figure out why the lxc devstack + tempest falls over :) | 18:32 |
| mriedem | in this case on stable it actually fails on the first attach to the first instance, which is wrong | 18:33 |
| mriedem | i wonder if specifying a device makes it work - if so, that'd be some bug in nova | 18:33 |
| *** VW_ has joined #openstack-nova | 18:35 | |
| mtreinish | mriedem: the stamp pattern is way more useful than that :) | 18:35 |
| mtreinish | hmm, yeah I'd expect volume attach to work on stable too | 18:35 |
| mriedem | yeah so it fails to attach to even the first instance | 18:35 |
| mriedem | wtf | 18:35 |
| mriedem | well, volume attach with a specified device id must work on stable b/c there are other tests that do that | 18:36 |
| mriedem | however, they don't check that the partition shows up | 18:36 |
| mriedem | except that boto test | 18:36 |
| mriedem | tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_compute_with_volumes | 18:36 |
| mriedem | checks that the partition list changes after the attach and changes again after the detach | 18:36 |
| mtreinish | heh, no one expects boto | 18:37 |
| mriedem | ha, yeah | 18:37 |
| superdan | mriedem: was the cells job failing all the time? meaning, should I wait until it merges to recheck my set? | 18:37 |
| mriedem | superdan: yeah, i think so | 18:37 |
| superdan | ugh | 18:37 |
| *** e0ne has quit IRC | 18:37 | |
| mriedem | i need to get an e-r query up for that one to see how bad it is | 18:38 |
| mriedem | the fix has been sitting in the gate for awhile | 18:38 |
| mtreinish | mriedem: ask infra to put it in front | 18:40 |
| mriedem | mtreinish: i think it only blocks nova | 18:40 |
| *** burgerk has joined #openstack-nova | 18:40 | |
| mtreinish | mriedem: but that ends up hurting everything that has shared jobs | 18:41 |
| mtreinish | because if someone +A a nova change that causes a reset | 18:41 |
| superdan | last time I asked about this, | 18:41 |
| superdan | they said no because even though nova hurts everyone, it's just resets and not actual fails for the other people | 18:41 |
| superdan | which I don't understand, but... | 18:41 |
| mriedem | we should have just reverted the tempest change | 18:42 |
| mriedem | but i'm not sure how you'd work the dependencies to redo the change | 18:42 |
| *** mgarza_ has joined #openstack-nova | 18:42 | |
| mriedem | i guess with an OR in the regex | 18:43 |
| *** vladikr has joined #openstack-nova | 18:43 | |
| mtreinish | mriedem: yeah you could add an or to a regex | 18:44 |
| mtreinish | mriedem: but really we need to figure out how to use the uuids for that | 18:44 |
| mtreinish | because that's why we added them | 18:44 |
| mriedem | mtreinish: remember how i said something about adding a cells job to tempest....? | 18:44 |
| mtreinish | I think I was opposed to that | 18:44 |
| mriedem | you were | 18:44 |
| mtreinish | but I'm kinda tired, so I can't remember why | 18:44 |
| *** annegentle has quit IRC | 18:44 | |
| mriedem | you complained that 50 voting tests on tempest was too many | 18:45 |
| mtreinish | ah, yeah it is | 18:45 |
| mriedem | bitch bitch bitch | 18:45 |
| *** annegentle has joined #openstack-nova | 18:45 | |
| mtreinish | mriedem: also if we had things properly using feature flags so tempest knows what works with cells and what doesnt | 18:45 |
| mtreinish | this wouldn't have been an issue either | 18:45 |
| mriedem | can we say, networks don't work with cells? | 18:45 |
| mriedem | we could flip the service-available bit | 18:46 |
| mtreinish | hmm, that's a good question | 18:46 |
| mriedem | but that's only for neutron tests i thought | 18:46 |
| mtreinish | yeah the service available is only for neutron | 18:46 |
| mtreinish | we can't do a general network one because n-net and neutron makes it unclear | 18:46 |
| mriedem | so what is it about the tests that fails cells? creating a network? | 18:47 |
| mriedem | b/c that could be your feature toggle | 18:47 |
| mriedem | oh nope, test_list_networks just lists networks | 18:48 |
| mtreinish | mriedem: I dunno, I didn't look really look at it. lascii said it couldnt work with cells | 18:48 |
| openstackgerrit | Paul Carlton proposed openstack/nova: Mask the token used to allow access to vnc console https://review.openstack.org/220622 | 18:48 |
| mtreinish | my general attitude is cells doesn't really work with anything | 18:48 |
| mriedem | hey, cells does what cells wants | 18:48 |
| mtreinish | heh, exactly | 18:48 |
| mriedem | don't put cells in a corner | 18:49 |
| mtreinish | mriedem: what about a container? | 18:49 |
| mriedem | ooo | 18:49 |
| mriedem | i smell summit session | 18:49 |
| mriedem | lascii: ^ for austin | 18:49 |
| mtreinish | haha | 18:49 |
| mriedem | standing room only | 18:50 |
| mriedem | 'i've heard a lot about cells and a lot about containers, so mixing the two has to be awesome' | 18:50 |
| mriedem | superdan: the cells fix just failed in the gate, in the ceph job | 18:50 |
| mtreinish | mriedem: sprinkle some nfv in there and you've got a stew going | 18:51 |
| *** dave-mccowan has quit IRC | 18:51 | |
| *** paul-carlton has quit IRC | 18:52 | |
| superdan | mriedem: argh | 18:52 |
| mriedem | superdan: so if you had plans for the big weekend, you might as well get started on those now | 18:52 |
| superdan | mriedem: yeah, I was just thinking about how I was likely going to be watching this all weekend | 18:53 |
| mriedem | http://www.rottentomatoes.com/m/every_which_way_but_loose/ ? | 18:53 |
| mriedem | ^ not going to watch itself | 18:53 |
| *** claudiub has joined #openstack-nova | 18:53 | |
| superdan | heh | 18:53 |
| mtreinish | man I really caused a lot of damage with a rename to make things debuggable | 18:53 |
| mriedem | can't believe that only got a 31% | 18:54 |
| mriedem | that's a classic | 18:54 |
| *** y_sawai has joined #openstack-nova | 18:54 | |
| *** pixelbeat has quit IRC | 18:55 | |
| *** y_sawai_ has joined #openstack-nova | 18:56 | |
| *** y_sawai has quit IRC | 18:58 | |
| *** y_sawai_ has quit IRC | 19:01 | |
| *** haomaiwang has quit IRC | 19:01 | |
| *** 5EXAB2JHN has joined #openstack-nova | 19:01 | |
| *** tonytan4ever has joined #openstack-nova | 19:03 | |
| *** bradjones has quit IRC | 19:03 | |
| *** e0ne has joined #openstack-nova | 19:03 | |
| mriedem | mtreinish: look familiar? https://review.openstack.org/#/c/175067/ | 19:06 |
| sdague | so who is it that's supposed to be monitoring the ceph job? | 19:07 |
| mriedem | jbernard: | 19:07 |
| *** tjones has quit IRC | 19:08 | |
| raildo | mriedem: sdague quickly question, if I'm going to implement something new that will impact the nova API, may I need implement to nova v2 or just to nova 2.1? | 19:09 |
| sdague | because it's 100% fail now | 19:09 |
| sdague | http://logs.openstack.org/90/220490/2/gate/gate-tempest-dsvm-full-ceph/5c943bc//logs/screen-g-api.txt.gz#_2015-09-04_18_44_25_719 | 19:09 |
| sdague | raildo: v2 is frozen | 19:09 |
| mriedem | sdague: ok, glance_store regression in the rbd store then | 19:09 |
| raildo | sdague: that was i thought... thanks | 19:09 |
| mriedem | https://pypi.python.org/pypi/glance_store/0.9.0 released yesterday | 19:10 |
| *** bradjones has joined #openstack-nova | 19:10 | |
| *** bradjones has quit IRC | 19:10 | |
| *** bradjones has joined #openstack-nova | 19:10 | |
| superdan | sdague: jobernar is on it already | 19:10 |
| *** mvensky has joined #openstack-nova | 19:11 | |
| sdague | superdan: ok, would be good if they heads upped folks that approving cinder and nova changes is pointless :P | 19:11 |
| jbernard | mriedem: will the new version be pulled in automatically? | 19:11 |
| mriedem | i don't actually see what's new in 0.9.0 that would cause this | 19:12 |
| mriedem | this is the only new thing https://github.com/openstack/glance_store/commit/3f5a37cb6e27946da3a5943289d5996fa96a552a | 19:12 |
| *** ociuhandu has joined #openstack-nova | 19:12 | |
| superdan | jbernard: yeah, unless there is a cap in place | 19:12 |
| sdague | there is | 19:12 |
| sdague | constraints processing | 19:13 |
| sdague | https://review.openstack.org/#/c/219568/3/upper-constraints.txt,cm is in the gate now | 19:13 |
| mriedem | the ceph job also passed on https://review.openstack.org/#/c/200554/ | 19:13 |
| sdague | so we're protected, until that, then everything is toast | 19:13 |
| superdan | sdague: meaning there wasn't a cap, but there is a cap in the gate? | 19:13 |
| mriedem | sdague: is there a bug for this yet? | 19:13 |
| sdague | mriedem: not that I know of | 19:13 |
| *** Sukhdev_ has joined #openstack-nova | 19:13 | |
| mriedem | ok, opening and doing e-r | 19:13 |
| sdague | superdan: this is the whole requirements redo that lifeless did, now we're pinned for devstack runs in the gate until a patch merge that explicitly ups things | 19:14 |
| superdan | sdague: okay, sorry for not knowing, but you mean we're now always pinned but there is something that automatically moves the pins forward? | 19:14 |
| sdague | yes | 19:14 |
| superdan | okay, but that thing hasn't merged yet, or has? | 19:14 |
| jbernard | i don't complete understand how a glance-store regression would land with the ceph job failing | 19:15 |
| sdague | it has not merged yet | 19:15 |
| superdan | sdague: so that can't have caused all the fails, right? | 19:15 |
| sdague | jbernard: because glance is backed on ceph | 19:15 |
| mriedem | jbernard: i don't think it's glance-store actually | 19:15 |
| mriedem | this might be upstream package getting pulled in | 19:15 |
| jbernard | mriedem: ok, ill take a closer look | 19:16 |
| *** lpetrut has joined #openstack-nova | 19:17 | |
| mriedem | there is no pypi package for rados right? | 19:17 |
| mriedem | it comes fro mthe distro | 19:17 |
| jbernard | mriedem: correct | 19:17 |
| *** dave-mccowan has joined #openstack-nova | 19:19 | |
| sdague | mriedem: so it's clearly related to glance-store, because there are ceph jobs in front of that change in the gate that work, and everything else behind is failing | 19:20 |
| jbernard | sdague: it /looks/ like the distro rbd package may have been updated, and a regression introduced | 19:20 |
| superdan | which change? the constraints bump? | 19:20 |
| jbernard | sdague: im trying to verify now | 19:21 |
| sdague | superdan: yes | 19:21 |
| superdan | sdague: so any reason not to kick that out and cap it to avoid breaking things for the long weekend? :D | 19:21 |
| mriedem | jbernard: | 19:21 |
| mriedem | ii librados2 0.80.10-0ubuntu0.14.04.1 amd64 RADOS distributed object store client library ii librbd1 0.80.10-0ubuntu0.14.04.1 amd64 RADOS block device client library | 19:21 |
| jbernard | mriedem: thanks | 19:22 |
| sdague | distro package is the wrong culprit here, because if that's the case stuff in the check queue would be failing | 19:22 |
| mriedem | ii python-ceph 0.80.10-0ubuntu0.14.04.1 amd64 Python libraries for the Ceph distributed filesystem | 19:22 |
| sdague | or stuff in front of this | 19:22 |
| mriedem | 0.80.10 came out over a year ago | 19:22 |
| sdague | and this package came out in july | 19:23 |
| sdague | http://changelogs.ubuntu.com/changelogs/pool/main/c/ceph/ceph_0.80.10-0ubuntu0.14.04.1/changelog | 19:23 |
| jbernard | yeah | 19:23 |
| sdague | this is glance-store | 19:23 |
| mriedem | https://github.com/openstack/glance_store/commit/3f5a37cb6e27946da3a5943289d5996fa96a552a | 19:23 |
| mriedem | -from __future__ import absolute_import | 19:23 |
| mriedem | that change monkeyed with the imports | 19:23 |
| jbernard | mriedem: rbd.ImageExists was added (along with the 'add' method) in Aug of last year… | 19:24 |
| jbernard | mriedem: ahh, maybe the imports have changed | 19:24 |
| mriedem | i don't know why that change would have passed the glance_store src ceph job | 19:25 |
| jbernard | that confuses me | 19:25 |
| mriedem | the package was updated | 19:25 |
| mriedem | http://logs.openstack.org/54/200554/6/check/gate-tempest-dsvm-full-ceph-src-glance_store/d76d300/logs/dpkg-l.txt.gz | 19:25 |
| mriedem | ii librados2 0.80.9-0ubuntu0.14.04.2 amd64 RADOS distributed object store client library | 19:25 |
| mriedem | sdague: ^ | 19:25 |
| mriedem | 0.80.9 to 0.80.10 | 19:25 |
| sdague | mriedem: and it wasn't on tests that pass? | 19:26 |
| mriedem | right | 19:26 |
| sdague | you have to confirm the negative case to issolate this | 19:26 |
| mriedem | http://logs.openstack.org/54/200554/6/check/gate-tempest-dsvm-full-ceph-src-glance_store/d76d300/logs/dpkg-l.txt.gz is the ceph job for the only ceph change in latest glance-store | 19:26 |
| mriedem | http://changelogs.ubuntu.com/changelogs/pool/main/c/ceph/ceph_0.80.10-0ubuntu0.14.04.1/changelog | 19:27 |
| mriedem | can we find james page? | 19:27 |
| mriedem | https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1477174 | 19:27 |
| openstack | Launchpad bug 1477174 in ceph (Ubuntu Trusty) "[SRU] 0.80.10 stable release" [High,Fix released] - Assigned to James Page (james-page) | 19:27 |
| openstackgerrit | Taylor Peoples proposed openstack/nova: Return empty PciDevicePoolList obj instead of None https://review.openstack.org/220634 | 19:28 |
| sdague | mriedem: so, it's a workable theory that glance_store change was tested on the old version, ceph upgraded after that, and we didn't see it anywhere else until the new version | 19:29 |
| mriedem | let me push a glance_store change to test | 19:29 |
| *** Marga_ has joined #openstack-nova | 19:29 | |
| sdague | mriedem: yeh | 19:29 |
| sdague | I pulled the requirements bump | 19:29 |
| *** Marga_ has quit IRC | 19:29 | |
| mriedem | always amazing o na friday | 19:30 |
| mriedem | when this stuff happens | 19:30 |
| *** Marga_ has joined #openstack-nova | 19:30 | |
| *** rlrossit has left #openstack-nova | 19:31 | |
| *** atuvenie has joined #openstack-nova | 19:31 | |
| *** figleaf is now known as edleafe | 19:31 | |
| jbernard | mriedem: in 0.80.10, the rbd class does have ImageExists | 19:32 |
| jbernard | mriedem: maybe your imports idea could be true | 19:32 |
| sdague | mriedem: actually, no, it's 100% glance store issue | 19:32 |
| sdague | it appears that upper-constraints processing is trumping our libs-from-git code - http://logs.openstack.org/20/220320/1/check/gate-tempest-dsvm-full-ceph-src-glance_store/83b8b59/logs/devstacklog.txt.gz#_2015-09-04_13_51_33_905 | 19:32 |
| sdague | I'm done for the week, however that needs a conversation, because a whole ton of stuff we're doing is undone by that | 19:33 |
| sdague | so the glance_store code isn't actually getting tested | 19:33 |
| superdan | oof | 19:34 |
| *** mgarza_ has quit IRC | 19:34 | |
| *** vladikr has quit IRC | 19:36 | |
| mriedem | so i did https://review.openstack.org/#/c/220646/ but it sounds like it might not matter? | 19:36 |
| mriedem | i'm a bit lost on the upper-constraints stuff | 19:36 |
| mriedem | is it upper-constraints that's busted? or libs-from-gate? or glance_store? or all 3? | 19:37 |
| *** atuvenie has quit IRC | 19:37 | |
| sdague | glance_store is busted for sure | 19:38 |
| sdague | the libs_from_git testing for glance_store is busted | 19:38 |
| sdague | possibly related to upper-constraints and the fact that it has a _ instead of a - | 19:39 |
| sdague | http://logs.openstack.org/07/216707/3/check/gate-tempest-dsvm-neutron-src-python-glanceclient/c01da5f/logs/pip-freeze.txt.gz | 19:39 |
| mriedem | that would be in project-config? | 19:39 |
| sdague | other libraries seem fine | 19:39 |
| sdague | mriedem: no, it's probably the active code in requirements that does this kind of merge | 19:39 |
| sdague | however, this actually looks entirely issolated to glance_store | 19:39 |
| sdague | so that's a sigh of relief | 19:39 |
| mriedem | https://pypi.python.org/pypi/glance_store | 19:40 |
| mriedem | it's glance_store on pypi and in g-r | 19:40 |
| mriedem | but not pip-freeze | 19:40 |
| *** vladikr has joined #openstack-nova | 19:40 | |
| mriedem | oh, well https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L125 | 19:40 |
| mriedem | ok, i think i'm getting what you're putting down now | 19:40 |
| mriedem | https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L125 trumps the glance-store src job | 19:40 |
| mriedem | which should be running trunk glance-store on the change + devstack + tempest | 19:41 |
| mriedem | but it's only testing glance -store 0.8.0 | 19:41 |
| mriedem | ha | 19:41 |
| *** burgerk has quit IRC | 19:41 | |
| sdague | yeh, dhellmann thinks there is a simple _ - bit that can be changed | 19:41 |
| sdague | he's doing it now (was poking him in #openstack-relmgr-office | 19:42 |
| sdague | anyway, I kicked out the offending change | 19:42 |
| superdan | yay for the weekend | 19:42 |
| sdague | https://review.openstack.org/#/c/220648 | 19:43 |
| superdan | mriedem: there has to be some "the weekend is saved!" 90s surfer movie clip you can post the youtube link for, right? | 19:45 |
| mriedem | point break wasn't that uplifting | 19:46 |
| mriedem | i think you're thinking of the 80s | 19:46 |
| superdan | as the distance from both decades increases, their differences become less clear | 19:47 |
| superdan | I see stuff and I'm like "wow, the 80s were crazy" and then I realize said thing was from 1998 or something | 19:47 |
| mriedem | ha | 19:48 |
| mriedem | it's not that bad | 19:48 |
| mriedem | late 80s to early 90s sure | 19:48 |
| superdan | heh\ | 19:48 |
| mriedem | i propose that we bring the image service back into nova | 19:48 |
| mriedem | and may jaypipes mini-PTL on it | 19:48 |
| mriedem | *make | 19:48 |
| superdan | as punishment for his sins? | 19:49 |
| mriedem | among other reasons | 19:49 |
| *** thumpba has quit IRC | 19:50 | |
| *** tonytan4ever has quit IRC | 19:51 | |
| *** sumanth has joined #openstack-nova | 19:51 | |
| *** lpetrut has quit IRC | 19:53 | |
| doffm | mriedem: Since there are a few steps we need to take to get i18n working for novalclient. (See: https://wiki.openstack.org/wiki/Translations/Infrastructure) | 19:53 |
| doffm | Want me to open up a bug on novaclient to track progress? | 19:53 |
| *** sumanth has quit IRC | 19:54 | |
| mriedem | doffm: yeah that'd be good | 19:55 |
| *** y_sawai has joined #openstack-nova | 19:56 | |
| *** lucas-beer has quit IRC | 19:58 | |
| *** 5EXAB2JHN has quit IRC | 20:01 | |
| *** tjones has joined #openstack-nova | 20:01 | |
| *** haomaiwang has joined #openstack-nova | 20:01 | |
| *** otter768 has quit IRC | 20:02 | |
| doffm | mriedem: https://bugs.launchpad.net/python-novaclient/+bug/1492444 | 20:02 |
| openstack | Launchpad bug 1492444 in python-novaclient "There is no working i18n for novaclient." [Undecided,New] - Assigned to Mark Doffman (mjdoffma) | 20:02 |
| *** y_sawai has quit IRC | 20:03 | |
| mriedem | cool | 20:03 |
| *** otter768 has joined #openstack-nova | 20:03 | |
| *** openstackgerrit_ has quit IRC | 20:03 | |
| *** yassine has joined #openstack-nova | 20:04 | |
| *** boris-42 has joined #openstack-nova | 20:04 | |
| *** yassine has quit IRC | 20:04 | |
| *** openstackgerrit_ has joined #openstack-nova | 20:04 | |
| lascii | mriedem: we do have cells and containers working in glorious synergy | 20:05 |
| lascii | that's apmeltons job | 20:05 |
| *** Marga_ has quit IRC | 20:05 | |
| * lascii catches up to a conversation late | 20:05 | |
| mriedem | lascii: heh, we just can't run devstack + tempest on lxc | 20:05 |
| lascii | one day we'll have neutron, cells, ceph, lxc, and glance_store all CI tested in one job | 20:07 |
| bauwser | man, my cells devstack said bye bye :( | 20:07 |
| lascii | or not, but we can dream | 20:07 |
| lascii | I don't have mine anymore either | 20:09 |
| bauwser | mriedem: do we have an e-r query for https://launchpad.net/bugs/1492255 ? | 20:10 |
| openstack | Launchpad bug 1492255 in OpenStack Compute (nova) "Cells gate job fails because of 2 network tests" [Critical,In progress] - Assigned to Sylvain Bauza (sylvain-bauza) | 20:10 |
| bauwser | mriedem: I can see lots of rechecks because my change is still in the gate | 20:11 |
| bauwser | mriedem: so I wonder if e-r was provided | 20:11 |
| bauwser | (nevermind, will check by myself) | 20:11 |
| bauwser | mriedem: ok, I can see it, thanks | 20:12 |
| mriedem | bauwser: yes | 20:13 |
| *** jdurgin has quit IRC | 20:13 | |
| mriedem | i did it a bit ago | 20:13 |
| mriedem | bauwser: https://review.openstack.org/#/c/220619/ | 20:13 |
| bauwser | mriedem: yup, saw your commit | 20:14 |
| mriedem | top gate bug now http://status.openstack.org/elastic-recheck/gate.html | 20:14 |
| *** xyang1_ has joined #openstack-nova | 20:14 | |
| * bauwser should avoid to hassle people when he can just do git log | 20:14 | |
| *** asselin_ has joined #openstack-nova | 20:14 | |
| mriedem | it's fine | 20:14 |
| mriedem | <3 | 20:14 |
| bauwser | mriedem: yeah I know, nova is blocked until the change is merged :( | 20:14 |
| bauwser | hence me checking | 20:15 |
| bauwser | my dashboard just looks like a mature tomato | 20:15 |
| mriedem | i just posted the fubar summary to the ML on the gate status http://lists.openstack.org/pipermail/openstack-dev/2015-September/073711.html | 20:15 |
| mriedem | jogo: ^ that could have been you | 20:16 |
| *** xyang1 has quit IRC | 20:16 | |
| *** keekz has quit IRC | 20:16 | |
| *** asselin has quit IRC | 20:16 | |
| *** EmilienM has quit IRC | 20:16 | |
| *** slagle has quit IRC | 20:16 | |
| *** devananda has quit IRC | 20:16 | |
| *** wolsen has quit IRC | 20:16 | |
| *** b3nt_pin has quit IRC | 20:16 | |
| *** mordred has quit IRC | 20:16 | |
| *** lascii has quit IRC | 20:16 | |
| *** beagles has joined #openstack-nova | 20:16 | |
| *** mordred has joined #openstack-nova | 20:16 | |
| *** xyang1_ is now known as xyang1 | 20:16 | |
| *** mordred has quit IRC | 20:16 | |
| *** mordred has joined #openstack-nova | 20:16 | |
| *** EmilienM_ has joined #openstack-nova | 20:16 | |
| *** keekz_ has joined #openstack-nova | 20:16 | |
| *** wolsen has joined #openstack-nova | 20:16 | |
| *** devananda has joined #openstack-nova | 20:16 | |
| *** slagle has joined #openstack-nova | 20:16 | |
| *** beagles is now known as Guest22755 | 20:16 | |
| *** EmilienM_ is now known as EmilienM | 20:16 | |
| *** alaski has joined #openstack-nova | 20:17 | |
| *** kaushikc has joined #openstack-nova | 20:17 | |
| kaushikc | Any nova api developers around? | 20:17 |
| clarkb | mriedem: you missed the broken keystone unittests | 20:17 |
| *** eharney has quit IRC | 20:17 | |
| mriedem | clarkb: that's not integrated gate | 20:18 |
| clarkb | mriedem: :P maybe morgan can respond with details | 20:18 |
| mriedem | so f'em | 20:18 |
| clarkb | mriedem: yes it is... | 20:18 |
| mriedem | clarkb: well, that doesn't impact nova does it? | 20:18 |
| clarkb | mriedem: it does when it makes keystone fail and restart the entire gate | 20:18 |
| mriedem | superdan: sdague: jbernard: btw, this fixes the glance store issue on ceph https://review.openstack.org/#/c/220646/ | 20:18 |
| mriedem | dhellmann: ^ | 20:18 |
| mriedem | https://jenkins01.openstack.org/job/gate-tempest-dsvm-full-ceph-src-glance_store/21/console | 20:18 |
| mriedem | clarkb: oh psh | 20:18 |
| mriedem | clarkb: you know what would be amazing? | 20:18 |
| mriedem | if at least one glance core were around to approve my gate fix | 20:19 |
| mriedem | i guess we'll just block 0.9.0 in g-r | 20:19 |
| *** otter768 has quit IRC | 20:19 | |
| *** jdurgin has joined #openstack-nova | 20:19 | |
| kaushikc | lascii: around? | 20:20 |
| *** smatzek has quit IRC | 20:20 | |
| mtreinish | mriedem: oh, yeah that seems quite on point | 20:21 |
| *** otter768 has joined #openstack-nova | 20:21 | |
| *** otter768 has quit IRC | 20:21 | |
| *** dustins has quit IRC | 20:21 | |
| *** otter768 has joined #openstack-nova | 20:22 | |
| *** dustins has joined #openstack-nova | 20:22 | |
| alaski | kaushikc: yes | 20:23 |
| *** matrohon has joined #openstack-nova | 20:23 | |
| *** alaski is now known as lascii | 20:24 | |
| kaushikc | alaski: hi, i had a question on nova api | 20:24 |
| jogo | mriedem: having fun :) | 20:25 |
| mriedem | jogo: not possible | 20:26 |
| *** vladikr has quit IRC | 20:26 | |
| jogo | mriedem: sounds like your not having a pinstatic day | 20:26 |
| kaushikc | lascii: We have an openstack installation where the ip adresses for Vms are showing up 15-20 minutes after the VMs are active | 20:26 |
| mriedem | pinterrible | 20:26 |
| kaushikc | lascii: horizon is showing the IPs instantaneously but nova api is taking very long to report ips | 20:27 |
| kaushikc | lascii: this was an installation by Rackspace and they are saying it is expected behavior | 20:27 |
| *** baoli has quit IRC | 20:28 | |
| mriedem | kaushikc: this seems like the wrong channel for this... ? | 20:28 |
| * bauwser not_lascii | 20:28 | |
| *** baoli has joined #openstack-nova | 20:28 | |
| jogo | hehe | 20:28 |
| kaushikc | lascii: when i was working on nova, i never saw this behavior. | 20:28 |
| lascii | definitely not expected behavior. But I have no idea what installation you're talking about. I only work on our own installation. | 20:28 |
| lascii | kaushikc: but mriedem is right, this would be better in #openstack | 20:28 |
| lascii | which I am conviently not in. perhaps I should join | 20:29 |
| *** exploreshaifali has quit IRC | 20:29 | |
| kaushikc | lascii: okay sorry about the wrong channel. But i thought nova api had change's in the past year or so since i looked at it | 20:29 |
| lascii | it has had a lot of changes. but none that should affect how long it takes for IPs to appear | 20:30 |
| *** baoli has quit IRC | 20:30 | |
| kaushikc | lascii: I'll send you an email with more details. Your irc handle came to the top of my head. Thank you | 20:32 |
| lascii | kaushikc: np. we can discuss further in #openstack, though I'm not sure what would cause that other than message queues being processed slowly | 20:33 |
| *** moshele has joined #openstack-nova | 20:34 | |
| *** erhudy1 has joined #openstack-nova | 20:37 | |
| david-lyle | lascii the ip values are cache and take time to propagate, horizon makes a call directly to neutron for the values (assuming using neutron) | 20:38 |
| *** Piet has quit IRC | 20:38 | |
| david-lyle | *cached in nova | 20:39 |
| david-lyle | we changed horizon due to the latency to call neutron | 20:39 |
| lascii | david-lyle: yes, but time should be a matter of seconds at most | 20:39 |
| david-lyle | in my past experience in a large installation it took much longer, but that experience is out of date | 20:40 |
| david-lyle | things may have changed, but I think shade took a similar route to horizon IIRC | 20:40 |
| *** lpetrut has joined #openstack-nova | 20:41 | |
| * david-lyle goes back to horizon land now | 20:41 | |
| lascii | okay. it could happen. but in the installation I deal with we have IP addresses on the instances when they go active | 20:42 |
| lascii | the delay isn't expected | 20:42 |
| david-lyle | lascii: fair enough, just explaining why horizon and nova may be showing different data | 20:43 |
| david-lyle | source == different | 20:43 |
| lascii | gotcha, that makes sense | 20:43 |
| lascii | sdague: mordred I just got the configs merged that will open up /, /v2, and /v2/ on the Rackspace API. Hopefully that will be out within the month, but should be out whenever our next deploy is | 20:45 |
| lascii | and eventually /v2.1 and /v2.1/ when we turn that on | 20:46 |
| *** raildo is now known as raildo-afk | 20:46 | |
| *** openstackgerrit_ has quit IRC | 20:47 | |
| mtreinish | lascii: what about/v2.0 ? | 20:47 |
| lascii | is that a thing? | 20:47 |
| lascii | but yes, that too | 20:47 |
| *** openstackgerrit_ has joined #openstack-nova | 20:47 | |
| mtreinish | lascii: I think it is, oomichi said it was at least | 20:48 |
| mtreinish | I wrote a test that used /v2 and he said I should have used /v2.0 | 20:48 |
| *** openstackgerrit_ has quit IRC | 20:48 | |
| lascii | good to know | 20:48 |
| *** openstackgerrit_ has joined #openstack-nova | 20:49 | |
| jbernard | mriedem: nice work | 20:49 |
| superdan | mriedem: cells change is about to merge | 20:50 |
| mriedem | superdan: saw that | 20:50 |
| superdan | and then I'm rechecking the m-f-s out of my patch | 20:50 |
| mriedem | i'm sort of surprised | 20:50 |
| *** eharney has joined #openstack-nova | 20:50 | |
| *** moshele has quit IRC | 20:50 | |
| mriedem | jbernard: don't pat my back yet, https://review.openstack.org/#/c/220646/ can't be trusted until we recheck it after https://review.openstack.org/#/c/220648/ is merged | 20:50 |
| *** atuvenie has joined #openstack-nova | 20:50 | |
| mtreinish | mriedem: don't jinx it | 20:51 |
| *** thorst has quit IRC | 20:51 | |
| mriedem | jbernard: mtreinish: well, https://review.openstack.org/#/c/220657/ is the backdoor anyway | 20:52 |
| mriedem | so, i guess, feel free to pat my back after all :) | 20:52 |
| superdan | I think zuul is taking a dramatic pause | 20:53 |
| * jbernard pats mriedem's back | 20:53 | |
| *** openstackgerrit_ has quit IRC | 20:53 | |
| cburgess | mriedem I confirmed that the bug doesn't impact me because of using RBD for nova root storage. Those use the same BDM code so that setting gettins applied. | 20:54 |
| superdan | bewn | 20:55 |
| superdan | er, bewm even | 20:55 |
| mriedem | nice | 20:55 |
| *** nic has joined #openstack-nova | 20:59 | |
| *** lpetrut has quit IRC | 20:59 | |
| *** y_sawai has joined #openstack-nova | 20:59 | |
| *** annegent_ has joined #openstack-nova | 21:00 | |
| *** marzif has quit IRC | 21:00 | |
| *** tpeoples has quit IRC | 21:01 | |
| *** haomaiwang has quit IRC | 21:01 | |
| *** haomaiwang has joined #openstack-nova | 21:01 | |
| *** rfolco has quit IRC | 21:02 | |
| *** baoli has joined #openstack-nova | 21:03 | |
| openstackgerrit | Sylvain Bauza proposed openstack/nova: DNM: Revert "Implement compare-and-swap for instance update" https://review.openstack.org/220666 | 21:03 |
| *** annegentle has quit IRC | 21:03 | |
| *** y_sawai has quit IRC | 21:04 | |
| *** dannywilson has quit IRC | 21:05 | |
| *** mdrabe has quit IRC | 21:05 | |
| *** mdrabe has joined #openstack-nova | 21:05 | |
| *** dannywilson has joined #openstack-nova | 21:05 | |
| *** diegows has joined #openstack-nova | 21:05 | |
| *** davideag_ has joined #openstack-nova | 21:06 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova: Updated from global requirements https://review.openstack.org/220669 | 21:06 |
| *** tjones has quit IRC | 21:06 | |
| *** SourabhP_ has joined #openstack-nova | 21:07 | |
| *** baoli has quit IRC | 21:07 | |
| *** HenryG has quit IRC | 21:07 | |
| *** davideagnello has quit IRC | 21:07 | |
| *** SourabhP has quit IRC | 21:08 | |
| *** j_king has quit IRC | 21:08 | |
| *** SourabhP_ is now known as SourabhP | 21:08 | |
| *** FL1SK has quit IRC | 21:08 | |
| *** sachin__ has joined #openstack-nova | 21:08 | |
| *** lbeliveau has quit IRC | 21:08 | |
| *** timello has quit IRC | 21:08 | |
| *** jogo has quit IRC | 21:08 | |
| *** ircuser-1 has quit IRC | 21:09 | |
| *** devananda has quit IRC | 21:09 | |
| *** sgordon has quit IRC | 21:09 | |
| *** sachin_ has quit IRC | 21:09 | |
| *** j_king has joined #openstack-nova | 21:09 | |
| *** y_sawai has joined #openstack-nova | 21:10 | |
| *** changbl has quit IRC | 21:10 | |
| *** HenryG has joined #openstack-nova | 21:11 | |
| *** maurosr has quit IRC | 21:11 | |
| *** whydidyoustealmy has quit IRC | 21:11 | |
| *** dosaboy has quit IRC | 21:11 | |
| *** dosaboy has joined #openstack-nova | 21:12 | |
| *** changbl has joined #openstack-nova | 21:12 | |
| *** sgordon has joined #openstack-nova | 21:13 | |
| *** Sukhdev_ has quit IRC | 21:13 | |
| *** timello has joined #openstack-nova | 21:13 | |
| *** y_sawai has quit IRC | 21:15 | |
| *** garyk has quit IRC | 21:15 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/nova: Expose keystoneclient's session and auth plugin loading parameters https://review.openstack.org/214423 | 21:16 |
| *** dustins has quit IRC | 21:16 | |
| *** devananda has joined #openstack-nova | 21:16 | |
| *** bswartz has joined #openstack-nova | 21:17 | |
| *** whydidyoustealmy has joined #openstack-nova | 21:18 | |
| *** diegows has quit IRC | 21:18 | |
| *** garyk has joined #openstack-nova | 21:18 | |
| *** maurosr has joined #openstack-nova | 21:19 | |
| *** claudiub has quit IRC | 21:20 | |
| *** electrocucaracha has quit IRC | 21:21 | |
| *** karimb has joined #openstack-nova | 21:22 | |
| *** ircuser-1 has joined #openstack-nova | 21:24 | |
| *** VW_ has quit IRC | 21:24 | |
| *** electrocucaracha has joined #openstack-nova | 21:25 | |
| cburgess | Clearly I need to read my mailing list email more offten but what happened to gerrit? It no long works with launchpad logins? | 21:25 |
| *** lascii is now known as alaski | 21:25 | |
| *** tjones has joined #openstack-nova | 21:25 | |
| *** VW_ has joined #openstack-nova | 21:25 | |
| mriedem | cburgess: works for me | 21:25 |
| bauwser | cburgess: it does | 21:25 |
| *** erhudy1 has quit IRC | 21:26 | |
| bauwser | cburgess: eat/check your cookie | 21:26 |
| cburgess | Oh never mind... its an ongoing decision in infra. Apparently something broke. | 21:26 |
| bauwser | alaski: already in WE ? | 21:26 |
| mtreinish | cburgess: I think the ubuntu openid provider is just down | 21:27 |
| cburgess | mriedem: still think we should have an option to disable non BFV | 21:27 |
| cburgess | Yeah | 21:27 |
| mriedem | cburgess: i don't disagree | 21:27 |
| bauwser | alaski: I guess you're leaving soon ? | 21:27 |
| mriedem | cburgess: you could write up a backlog specy thingy | 21:27 |
| cburgess | mriedem: We have a use case for it that just came up. Deploying some clients now that want to use solidfire as only storage. | 21:27 |
| mriedem | 'i am chet and i want to restrict to boot from volume, make it so' | 21:27 |
| cburgess | LOL | 21:27 |
| *** matrohon has quit IRC | 21:28 | |
| cburgess | That one is easy enough that I might even include the code. I might end up writing it shortly for us anyways. | 21:28 |
| mriedem | cool | 21:28 |
| cburgess | In theory this should be easy. | 21:28 |
| cburgess | If flag and not BDM fail | 21:28 |
| *** karimb has quit IRC | 21:28 | |
| *** VW_ has quit IRC | 21:30 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/nova: Expose keystoneclient's session and auth plugin loading parameters https://review.openstack.org/214423 | 21:30 |
| *** diegows has joined #openstack-nova | 21:31 | |
| *** marzif has joined #openstack-nova | 21:31 | |
| ctrath | dimsum__ : thanks for the fix | 21:34 |
| *** diegows has quit IRC | 21:34 | |
| *** xyang1 has quit IRC | 21:35 | |
| mriedem | doffm: https://bugs.launchpad.net/nova/+bug/1491414 | 21:37 |
| openstack | Launchpad bug 1491414 in OpenStack Compute (nova) "LibvirtConfigObject.to_xml() should not log the xml str - leave that to the caller" [Low,Triaged] - Assigned to Matt Riedemann (mriedem) | 21:37 |
| mriedem | doffm: https://bugs.launchpad.net/nova/+bug/1491563 | 21:37 |
| openstack | Launchpad bug 1491563 in OpenStack Compute (nova) "Failed to spawn lxc instance due to nbd mount in use or rootfs busy" [Undecided,New] | 21:37 |
| doffm | mriedem: Thanks. | 21:37 |
| *** vilobhmm_111 has quit IRC | 21:39 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova: nova-net: fix missing log variable in deallocate_fixed_ip https://review.openstack.org/220253 | 21:39 |
| *** vilobhmm_11 has joined #openstack-nova | 21:40 | |
| *** marzif has quit IRC | 21:41 | |
| *** ctrath has quit IRC | 21:43 | |
| *** dims_ has joined #openstack-nova | 21:43 | |
| *** jwcroppe_ has joined #openstack-nova | 21:45 | |
| *** dimsum__ has quit IRC | 21:47 | |
| *** jwcroppe has quit IRC | 21:47 | |
| *** dims_ has quit IRC | 21:48 | |
| mriedem | calling it a day, later | 21:53 |
| *** mriedem has quit IRC | 21:53 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova: Updated from global requirements https://review.openstack.org/220669 | 21:54 |
| *** erhudy1 has joined #openstack-nova | 21:57 | |
| *** jogo has joined #openstack-nova | 21:57 | |
| *** jwcroppe has joined #openstack-nova | 21:58 | |
| *** jwcroppe_ has quit IRC | 21:59 | |
| *** haomaiwang has quit IRC | 22:01 | |
| *** 17SADHVEF has joined #openstack-nova | 22:01 | |
| *** baoli has joined #openstack-nova | 22:03 | |
| *** jwcroppe_ has joined #openstack-nova | 22:04 | |
| *** jwcroppe has quit IRC | 22:06 | |
| openstackgerrit | John L. Villalovos proposed openstack/nova: Add constraint target to tox.ini https://review.openstack.org/205931 | 22:06 |
| *** baoli has quit IRC | 22:08 | |
| openstackgerrit | Dan Smith proposed openstack/nova: Freeze v1 Instance and InstanceList schema hashes https://review.openstack.org/220689 | 22:10 |
| openstackgerrit | Dan Smith proposed openstack/nova: DNM: Drop Instance v1.x support https://review.openstack.org/220690 | 22:10 |
| *** y_sawai has joined #openstack-nova | 22:11 | |
| *** aysyd has quit IRC | 22:14 | |
| *** atuvenie has quit IRC | 22:15 | |
| *** y_sawai has quit IRC | 22:15 | |
| *** Piet has joined #openstack-nova | 22:15 | |
| *** karimb has joined #openstack-nova | 22:18 | |
| *** unicell has quit IRC | 22:18 | |
| *** unicell1 has joined #openstack-nova | 22:18 | |
| *** tjones has quit IRC | 22:22 | |
| *** y_sawai has joined #openstack-nova | 22:22 | |
| cp16net | hey folks - i'm seeing an issue with booting a nova vm from trove. we are getting a 400 on boot. http://paste.openstack.org/show/445851/ | 22:22 |
| cp16net | could this be related to this nova bug 1491325 ? | 22:22 |
| openstack | bug 1491325 in OpenStack Compute (nova) "nova api v2.1 does not allow to use autodetection of volume device path" [Critical,Fix committed] https://launchpad.net/bugs/1491325 - Assigned to Davanum Srinivas (DIMS) (dims-v) | 22:22 |
| *** y_sawai has quit IRC | 22:22 | |
| *** e0ne has quit IRC | 22:27 | |
| *** VW_ has joined #openstack-nova | 22:29 | |
| *** e0ne has joined #openstack-nova | 22:31 | |
| *** mdrabe has quit IRC | 22:31 | |
| *** VW__ has joined #openstack-nova | 22:32 | |
| *** marcusvrn_ has quit IRC | 22:35 | |
| *** ildikov has quit IRC | 22:35 | |
| *** VW_ has quit IRC | 22:36 | |
| *** VW__ has quit IRC | 22:37 | |
| *** emagana has quit IRC | 22:38 | |
| *** electrocucaracha has quit IRC | 22:38 | |
| *** tpeoples has joined #openstack-nova | 22:43 | |
| *** annegent_ has quit IRC | 22:47 | |
| *** tjones has joined #openstack-nova | 22:48 | |
| *** e0ne has quit IRC | 22:49 | |
| *** pixelbeat has joined #openstack-nova | 22:51 | |
| *** baoli has joined #openstack-nova | 22:52 | |
| *** baoli has quit IRC | 22:52 | |
| *** alex_xu has quit IRC | 22:53 | |
| *** alex_xu has joined #openstack-nova | 22:56 | |
| openstackgerrit | Dan Smith proposed openstack/nova: DNM: Drop Instance v1.x support https://review.openstack.org/220690 | 22:57 |
| *** Sukhdev_ has joined #openstack-nova | 22:57 | |
| *** tpeoples has quit IRC | 23:00 | |
| *** 17SADHVEF has quit IRC | 23:01 | |
| *** haomaiwang has joined #openstack-nova | 23:01 | |
| *** nic has quit IRC | 23:05 | |
| *** browne has quit IRC | 23:06 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova: Updated from global requirements https://review.openstack.org/220669 | 23:08 |
| *** FL1SK has joined #openstack-nova | 23:08 | |
| *** eharney has quit IRC | 23:11 | |
| *** tjones has quit IRC | 23:13 | |
| *** jamielennox is now known as jamielennox|away | 23:13 | |
| *** edmondsw has quit IRC | 23:19 | |
| *** tjones has joined #openstack-nova | 23:20 | |
| *** tjones has left #openstack-nova | 23:21 | |
| *** y_sawai has joined #openstack-nova | 23:23 | |
| vilobhmm_11 | superdan : ping | 23:27 |
| superdan | vilobhmm_11: ack, but it's late on the day before a US holiday, so if it's anything complicated, then I'm not here | 23:28 |
| vilobhmm_11 | is there a way to construct object from data coming from other than database | 23:28 |
| *** y_sawai has quit IRC | 23:28 | |
| vilobhmm_11 | this is about the object layer | 23:29 |
| superdan | vilobhmm_11: yes: | 23:29 |
| superdan | service = objects.Service(binary='foo', disabled=False) | 23:29 |
| vilobhmm_11 | if you have a link or so it would be helpful or some code reference that would be useful | 23:30 |
| superdan | that ^ | 23:30 |
| vilobhmm_11 | ok cool thanks | 23:30 |
| *** baoli has joined #openstack-nova | 23:33 | |
| *** otter768 has quit IRC | 23:37 | |
| *** baoli has quit IRC | 23:38 | |
| *** IanGovett has joined #openstack-nova | 23:38 | |
| *** otter768 has joined #openstack-nova | 23:41 | |
| *** erhudy1 has quit IRC | 23:47 | |
| *** ijuwang has quit IRC | 23:48 | |
| *** otter768 has quit IRC | 23:49 | |
| *** karimb has quit IRC | 23:50 | |
| *** dimsum__ has joined #openstack-nova | 23:52 | |
| *** dimsum__ has quit IRC | 23:52 | |
| *** tri2sing has quit IRC | 23:53 | |
| *** tri2sing has joined #openstack-nova | 23:53 | |
| *** Sukhdev_ has quit IRC | 23:57 | |
| *** tri2sing has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!