Tuesday, 2015-04-14

*** dboik has quit IRC00:00
*** achanda has quit IRC00:02
*** promulo_ has joined #openstack-nova00:02
*** otter768 has joined #openstack-nova00:03
*** asettle has quit IRC00:03
*** promulo__ has quit IRC00:05
*** otter768 has quit IRC00:07
*** Nic1 has joined #openstack-nova00:14
*** deepthi has joined #openstack-nova00:16
*** Nic has quit IRC00:16
*** deepthi has quit IRC00:16
*** heyongli_1 has quit IRC00:17
*** dave-mccowan has joined #openstack-nova00:23
*** mtanino_ has joined #openstack-nova00:24
*** mtanino has quit IRC00:25
*** dave-mccowan has quit IRC00:29
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] Store context in local store after spawn_n  https://review.openstack.org/17311200:29
*** emagana has quit IRC00:30
*** asettle has joined #openstack-nova00:32
*** kmartin has quit IRC00:32
*** patrickeast has quit IRC00:33
*** ijw has quit IRC00:35
*** zhenguo has joined #openstack-nova00:39
*** ozialien has joined #openstack-nova00:44
*** oro has quit IRC00:47
*** yuntongjin has joined #openstack-nova00:50
*** browne has quit IRC00:56
*** heyongli_ has joined #openstack-nova00:58
*** liusheng has joined #openstack-nova01:01
*** ociuhandu has joined #openstack-nova01:08
*** eliqiao has joined #openstack-nova01:09
*** marun has joined #openstack-nova01:11
*** Nic1 has quit IRC01:15
*** ozialien has quit IRC01:19
*** baoli has joined #openstack-nova01:20
*** erkules_ has joined #openstack-nova01:21
*** erkules has quit IRC01:23
*** jecarey has joined #openstack-nova01:24
*** tjones1 has joined #openstack-nova01:26
*** shaohe_feng has joined #openstack-nova01:27
*** aysyd has quit IRC01:31
*** neelashah has joined #openstack-nova01:33
*** popw has joined #openstack-nova01:34
*** eliqiao1 has joined #openstack-nova01:36
*** eliqiao has quit IRC01:36
*** dsanders has quit IRC01:41
*** igordcard_ has quit IRC01:44
*** browne has joined #openstack-nova01:45
*** salv-orlando has joined #openstack-nova01:46
openstackgerritFei Long Wang proposed openstack/nova: Fix nova backup for volume-backed instance  https://review.openstack.org/16449401:50
*** wuhg has joined #openstack-nova01:53
*** salv-orlando has quit IRC01:53
*** diegows has quit IRC01:54
*** baoli has quit IRC01:55
*** unicell has quit IRC01:56
*** unicell has joined #openstack-nova01:56
*** tjones1 has quit IRC01:57
*** kaisers1 has joined #openstack-nova01:59
*** erlon has quit IRC02:01
*** kaisers has quit IRC02:01
*** tjones1 has joined #openstack-nova02:01
*** baoli has joined #openstack-nova02:03
*** otter768 has joined #openstack-nova02:04
*** mtanino_ has quit IRC02:04
*** otter768 has quit IRC02:08
*** sbfox has joined #openstack-nova02:09
openstackgerritEli Qiao proposed openstack/nova: Set power state to None when evaucate failed do to recreate not support  https://review.openstack.org/17315002:14
*** zhangtralon has joined #openstack-nova02:15
*** baoli has quit IRC02:17
*** VW_ has joined #openstack-nova02:20
openstackgerritwangxiyuan proposed openstack/nova: word spell mistake in nova/compute/api.py  https://review.openstack.org/17315102:21
*** achanda has joined #openstack-nova02:21
*** Raildo__ has quit IRC02:21
*** bkopilov has quit IRC02:22
*** panbalag has quit IRC02:22
*** yamahata has quit IRC02:23
*** VW_ has quit IRC02:23
*** VW_ has joined #openstack-nova02:24
*** dims_ has quit IRC02:28
*** baoli has joined #openstack-nova02:28
*** asettle has quit IRC02:32
openstackgerritLianhao Lu proposed openstack/nova: Fixed some misspellings  https://review.openstack.org/17315302:33
*** isd has quit IRC02:37
*** claudiub has quit IRC02:41
openstackgerritshihanzhang proposed openstack/nova: Add config option to override url for versions  https://review.openstack.org/16026602:42
*** oro has joined #openstack-nova02:43
*** sbfox has quit IRC02:44
openstackgerritzhangtralon proposed openstack/nova: DB:support for migration objects multi-statues  https://review.openstack.org/16960402:45
*** annegentle has joined #openstack-nova02:46
*** DinaBelova has quit IRC02:48
*** DinaBelova has joined #openstack-nova02:48
*** unicell has quit IRC02:50
*** tsekiyam_ has joined #openstack-nova02:52
openstackgerritzhangtralon proposed openstack/nova: add support for muti_statuses_query from migration  https://review.openstack.org/16963102:53
*** tsekiyama has quit IRC02:55
*** annegentle has quit IRC02:55
*** tsekiyam_ has quit IRC02:56
*** wolsen_ has joined #openstack-nova03:01
*** tjones1 has quit IRC03:02
*** tjones1 has joined #openstack-nova03:04
*** tojuvone has joined #openstack-nova03:05
*** dsanders has joined #openstack-nova03:06
*** wuhg has quit IRC03:06
*** mmedvede has quit IRC03:06
*** timello has quit IRC03:06
*** ndipanov has quit IRC03:06
*** tdurakov has quit IRC03:06
*** anthonyper has quit IRC03:06
*** L33 has quit IRC03:06
*** wolsen has quit IRC03:06
*** tristanC has quit IRC03:06
*** tjones1 has quit IRC03:12
*** tristanC has joined #openstack-nova03:13
*** wuhg has joined #openstack-nova03:13
*** mmedvede has joined #openstack-nova03:13
*** timello has joined #openstack-nova03:13
*** ndipanov has joined #openstack-nova03:13
*** tdurakov has joined #openstack-nova03:13
*** anthonyper has joined #openstack-nova03:13
*** L33 has joined #openstack-nova03:13
*** achanda has quit IRC03:14
*** dave-mccowan has joined #openstack-nova03:14
*** xyang1 has quit IRC03:15
*** marun has quit IRC03:16
*** asettle has joined #openstack-nova03:18
*** asettle has quit IRC03:19
*** asettle has joined #openstack-nova03:19
*** sbfox has joined #openstack-nova03:20
*** tjones1 has joined #openstack-nova03:22
openstackgerritLianhao Lu proposed openstack/nova: Fixed some misspellings  https://review.openstack.org/17315303:25
*** yamahata has joined #openstack-nova03:27
*** dims has joined #openstack-nova03:28
*** otter768 has joined #openstack-nova03:32
*** dims has quit IRC03:33
*** oro has quit IRC03:37
*** VW_ has quit IRC03:37
openstackgerritwangxiyuan proposed openstack/nova: word spell mistake in nova/compute/api.py  https://review.openstack.org/17315103:39
openstackgerritwangxiyuan proposed openstack/nova: word spell mistake in nova/compute/api.py  https://review.openstack.org/17315103:39
*** jwcroppe has joined #openstack-nova03:41
*** dsanders has quit IRC03:43
*** ajayaa has joined #openstack-nova03:44
openstackgerritwangxiyuan proposed openstack/nova: word spell mistake in nova/compute/api.py  https://review.openstack.org/17315103:54
*** jecarey has quit IRC03:54
openstackgerritwanghao proposed openstack/nova: fix 'Ignoring EndpointNotFound' error in init host  https://review.openstack.org/14704203:56
*** sdake has joined #openstack-nova03:56
*** armax has quit IRC03:57
*** salv-orlando has joined #openstack-nova03:57
*** harlowja is now known as harlowja_away03:58
*** sdake_ has quit IRC03:58
*** tjones1 has quit IRC03:59
*** tjones1 has joined #openstack-nova04:01
*** bkopilov has joined #openstack-nova04:02
*** sdake_ has joined #openstack-nova04:02
*** otter768 has quit IRC04:03
*** salv-orlando has quit IRC04:04
*** sdake has quit IRC04:04
*** sbfox has quit IRC04:05
*** rajesht has joined #openstack-nova04:11
*** rajesht has quit IRC04:11
*** sbfox has joined #openstack-nova04:13
*** neelashah has quit IRC04:17
*** baoli has quit IRC04:19
*** ajayaa has quit IRC04:21
*** zhangtralon has quit IRC04:21
*** unicell has joined #openstack-nova04:23
*** krtaylor has quit IRC04:35
*** garyk has quit IRC04:37
*** dsanders has joined #openstack-nova04:38
*** krtaylor has joined #openstack-nova04:38
*** popw1 has joined #openstack-nova04:43
*** popw has quit IRC04:45
*** zhangtralon has joined #openstack-nova04:47
*** asettle has quit IRC04:50
*** asettle has joined #openstack-nova04:51
*** achanda has joined #openstack-nova04:52
*** jcoufal has joined #openstack-nova04:53
*** aswadr has joined #openstack-nova04:53
*** asettle has quit IRC04:53
*** asettle has joined #openstack-nova04:53
*** deepthi has joined #openstack-nova04:57
openstackgerritEric Brown proposed openstack/nova: Don't raise exception on resize of 0 disk flavors  https://review.openstack.org/17317004:59
*** _gryf|afk is now known as _gryf05:00
openstackgerritEric Brown proposed openstack/nova: VMware: Don't raise exception on resize of 0 disk  https://review.openstack.org/17317005:01
*** tjones1 has quit IRC05:06
*** aswadr has quit IRC05:06
*** erkules_ is now known as erkules05:07
*** erkules has joined #openstack-nova05:07
*** aswadr has joined #openstack-nova05:07
*** flwang1 has quit IRC05:08
*** ajayaa has joined #openstack-nova05:08
*** zhangtralon has quit IRC05:12
*** oomichi has joined #openstack-nova05:13
*** ishant has joined #openstack-nova05:14
openstackgerritshihanzhang proposed openstack/nova: Add config option to override url for versions  https://review.openstack.org/16026605:16
*** gmann has quit IRC05:17
*** gmann has joined #openstack-nova05:18
*** tjones1 has joined #openstack-nova05:19
*** welldannit has quit IRC05:19
*** tjones1 has quit IRC05:20
*** oro has joined #openstack-nova05:22
*** salv-orlando has joined #openstack-nova05:23
*** tjones1 has joined #openstack-nova05:23
openstackgerritKen'ichi Ohmichi proposed openstack/nova: Add min/max of API microversions to version API  https://review.openstack.org/17285605:24
oomichialex_xu: hi05:25
alex_xuoomichi: hi05:25
oomichialex_xu: thanks for reviewing https://review.openstack.org/172856 :)05:25
oomichialex_xu: that seems important for microversion05:26
alex_xuoomichi: yea, thanks for the explain, just know already wrote in spec05:26
oomichialex_xu: clients cannot know what microversion specify.05:26
oomichialex_xu: is it ok for you?05:26
alex_xuoomichi: yes, I'm thinking of that case also05:26
oomichialex_xu: thanks, but we need to wait for jenkins job05:27
alex_xuoomichi: just think about we version always return two items, one for v2, another one for v2.105:27
oomichialex_xu: yeah, right.05:27
alex_xuoomichi: and there 'v2.1' is special one05:27
oomichialex_xu: nice point.05:28
oomichialex_xu: so I felt it was enough to add "version" and "min_version" to "v2.1" only.05:28
oomichialex_xu: but that seemed inconsistent for "version" API.05:29
oomichialex_xu: and current patch adds the parameters to both APIs.05:29
*** Mike_D_laptop has joined #openstack-nova05:29
oomichialex_xu: but I still have this concern now..05:29
alex_xuoomichi: emm.... looks like we need a note resolve user's confuse05:29
*** salv-orlando has quit IRC05:30
alex_xuoomichi: can we just list as {version: [{id: v2.1...}, {id:v2.2..}, ...]}05:30
alex_xuoomichi: and is there any plan for version info in your json-home?05:31
oomichialex_xu: but after v2.100, version API needs to return 100 microversions list05:31
oomichialex_xu: that seems not good, and current patch is based on the spec.05:32
*** ociuhandu has quit IRC05:32
alex_xuoomichi: emm..., that's problem also :(05:32
*** Mike_D_laptop has quit IRC05:32
alex_xuoomichi: another way as I know and also used by ironic is return http header about the version info when the requested version didn't support05:33
oomichialex_xu: json-home just covers API URIs. microvresions don't change API URIs, so json-home cannot cover what is available microversions.05:33
*** sahid has joined #openstack-nova05:33
alex_xuoomichi: and I register a bug for it https://bugs.launchpad.net/nova/+bug/144349405:33
openstackLaunchpad bug 1443494 in OpenStack Compute (nova) "Return supported micro-version by http headers when request with invalid micro-version" [Undecided,In progress] - Assigned to Alex Xu (xuhj)05:33
*** rushiagr_away is now known as rushiagr05:33
alex_xuoomichi: what means for 'covers API URIs'?05:34
oomichialex_xu: API URIs means API resources or something like "/v2/{tenant-id}/servers"05:35
alex_xuoomichi: ok, probably know what mean now05:35
alex_xuoomichi: I will try think about more if there is better way resolve this confuse05:36
oomichialex_xu: json-home just provides available API URIs, but we need to expose available microversions now.05:36
oomichialex_xu: so we cannot use json-home for this problem.05:36
alex_xuoomichi: ok05:36
*** tjones1 has quit IRC05:37
alex_xuoomichi: which line talk about verison api in http://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/v2-on-v3-api.html?05:37
alex_xuoomichi: oops, sorry I know now05:37
*** zhangtralon has joined #openstack-nova05:37
openstackgerritzhangtralon proposed openstack/nova: resize-revert can break anti-affinity policy  https://review.openstack.org/16011005:37
oomichialex_xu: I have another concern about this problem.05:38
oomichialex_xu: we have added an extension for API changes as API change manner.05:38
*** tjones1 has joined #openstack-nova05:39
oomichialex_xu: but current patch doesn't add an extension even if adding new parameters "version" and "min_version".05:39
*** salv-orlando has joined #openstack-nova05:39
alex_xuoomichi: you mean for v2?05:40
oomichialex_xu: version API is independent from v2 and v2.105:40
*** Nikolay_1t has quit IRC05:40
oomichialex_xu: because the API URI is just "/" without "/v2" and "/v2.1"05:41
alex_xuoomichi: so why we need add an extension for add new parameters?05:41
oomichialex_xu: ah, nice point. really thanks!05:41
oomichialex_xu: I will write the reason why the patch doesn't add a new extension on the commit message :)05:42
alex_xuoomichi: np, cool :)05:42
alex_xuoomichi: what would you think about return http header for microversion?05:42
*** unicell1 has joined #openstack-nova05:42
oomichialex_xu: from http://docs-draft.openstack.org/71/172771/3/check/gate-nova-specs-docs/8f6dc33//doc/build/html/specs/kilo/implemented/api-microversions.html05:43
oomichialex_xu: the response should include "X-OpenStack-Nova-API-Version: version_number, experimental Vary: X-OpenStack-Nova-API-Version"05:44
*** unicell has quit IRC05:44
alex_xuoomichi: but that didn't include max and min version05:45
oomichialex_xu: yeah, I feel it is enough to used microversion only in the header.05:45
oomichialex_xu: yeah, I feel it is enough to include the used microversion only in the header.05:46
alex_xuoomichi: ok, if we have version api, the header didn't very usefule05:46
*** nkrinner has joined #openstack-nova05:46
alex_xus/usefule/useful05:47
oomichialex_xu: and API change is very important and we need bp/spec for the changes.05:48
oomichialex_xu: so at this time, we need to change version API based on the approved nova-spec.05:48
alex_xuoomichi: ok05:49
*** salv-orlando has quit IRC05:50
alex_xuoomichi: should we back port this patch?05:52
oomichialex_xu: yeah, I think so. this patch is very important for microversion users.05:52
alex_xuoomichi: ok cool05:53
*** Longgeek has joined #openstack-nova05:53
*** salv-orlando has joined #openstack-nova05:53
*** tjones1 has quit IRC05:54
*** dsanders has quit IRC05:56
*** popw1 has quit IRC05:56
openstackgerritKen'ichi Ohmichi proposed openstack/nova: Add min/max of API microversions to version API  https://review.openstack.org/17285605:56
*** tjones1 has joined #openstack-nova05:58
*** yamahata has quit IRC05:58
*** yamahata has joined #openstack-nova05:59
*** salv-orlando has quit IRC06:00
alex_xuoomichi: another question, should we change api sample file when microversion bump eachtime?06:01
alex_xuoomichi: or we should just fake a version in the api sample test?06:01
alex_xuoomichi: I mean this sample file https://review.openstack.org/#/c/172856/4/nova/tests/functional/api_samples/versions/versions-get-resp.json.tpl06:02
*** asettle has quit IRC06:03
openstackgerrityuntongjin proposed openstack/nova: Add missing policy for limits extention  https://review.openstack.org/17317806:04
*** otter768 has joined #openstack-nova06:04
*** garyk has joined #openstack-nova06:05
*** doron_ has joined #openstack-nova06:07
*** doron_ is now known as doron_afk06:07
*** otter768 has quit IRC06:09
*** hdd has joined #openstack-nova06:10
*** tojuvone has quit IRC06:10
*** devananda has quit IRC06:10
*** sahid has quit IRC06:10
*** _gryf has quit IRC06:11
*** zhangtralon has quit IRC06:11
*** krtaylor has quit IRC06:11
*** _gryf has joined #openstack-nova06:11
oomichialex_xu: sorry, I am back.06:13
alex_xuoomichi: np06:13
oomichialex_xu: I think we need to change "version" api sample file when microversion bump eachtime.06:13
oomichialex_xu: because the sample file should show current microversion as the sample.06:14
oomichialex_xu: your concern is that the patch will be big when microversion bump, right?06:15
alex_xuoomichi: yea, just need update every time06:15
alex_xuoomichi: but I think what you said is make sense, update it each time won't be too much extra work06:16
oomichialex_xu: yeah, maybe the review will be easy for that because the same change pattern for every microversion :)06:16
alex_xuoomichi: ok, got it, thanks for the explain :)06:17
*** devananda has joined #openstack-nova06:17
*** doron_afk is now known as doron_06:17
openstackgerritRui Chen proposed openstack/nova: Fix empty instance nw_info cache  https://review.openstack.org/14603606:17
oomichialex_xu: np, and thank you also for nice points :-)06:17
alex_xuoomichi: np06:17
*** apuimedo has joined #openstack-nova06:18
openstackgerritOpenStack Proposal Bot proposed openstack/nova: Imported Translations from Transifex  https://review.openstack.org/17194306:18
openstackgerritwangxiyuan proposed openstack/nova: word spell mistake in nova/compute/api.py  https://review.openstack.org/17315106:20
*** ankit_ag has joined #openstack-nova06:20
*** krtaylor has joined #openstack-nova06:22
chaochingaryk: Hi Gary06:27
chaochingaryk: I am testing ephemeral disk, and notice that vmware driver does not create filesystem in the ephemeral disk06:29
openstackgerritJuliane Aulbach proposed openstack/nova: Change config drive attachement from virtio-scsi to virtio-blk.  https://review.openstack.org/17318406:29
chaochingaryk: after I input guest_format=ext3 into block_device_mapping_v2 and boot the VM, I can not mount that disk in guest OS06:30
chaochingaryk: Is that a limitition of VCenter? Or you plan to implement that in your next step?06:31
openstackgerritwanghao proposed openstack/nova: Nova can't detach volume in init host routine  https://review.openstack.org/14704206:31
*** TobiasE has joined #openstack-nova06:32
*** jwcroppe has quit IRC06:32
*** zhangtralon has joined #openstack-nova06:32
*** Marga_ has quit IRC06:33
garykchaochin: the guest should be responsible to do that06:33
openstackgerritYongli He proposed openstack/nova: resource tracker style pci resource management  https://review.openstack.org/13132106:33
*** jwcroppe_ has joined #openstack-nova06:33
chaochingaryk: but libvirt driver can read guest_format, and create the filesystem for the guest OS. Is that technically doable for VMware?06:35
*** zhangtralon has quit IRC06:35
garykchaochin: i will check - do you have vmware tools installed on your guest?06:36
*** jwcroppe_ has quit IRC06:38
chaochingaryk: my image does not have vmware tool installed.  What libvirt driver does is to create a file, and run mkfs against that file before attaching it to guest os.06:38
chaochingaryk: will we do it in that way for VCenter?06:39
*** doron_ is now known as doron_afk06:39
garykchaochin: if you look at the settings then you will see that a new disk has been created for the ephemeral disk06:39
garykfrom your gues you should be able to see the disk and configure it - similar to the way that you would have worked if it was a cinder volume06:40
chaochingaryk: yes. I can see the new disk, and can create FS by myself in the guest os.06:40
chaochingaryk: just hope to make sure if we can make guest_format parameter in api request to take effect for vmware driver.06:41
garykchaochin: i am not sure that is the right way to go. i will invetigate06:42
chaochingaryk: ok, thank you!06:43
openstackgerritzhangtralon proposed openstack/nova: Change the reboot rpc call to local during init_host  https://review.openstack.org/17011006:43
openstackgerritSudipta Biswas proposed openstack/nova: Allow discrete blocks of online pCPUs for pinning  https://review.openstack.org/17318706:48
garykttx: are you around?06:49
*** park_heijlong has joined #openstack-nova06:50
*** sudipto has joined #openstack-nova06:51
openstackgerritSudipta Biswas proposed openstack/nova: Allow discrete blocks of online pCPUs for pinning  https://review.openstack.org/17318706:52
openstackgerritMike Durnosvistov proposed openstack/nova: libvirt: Decomposition plug methods in vif  https://review.openstack.org/14871906:55
*** tojuvone has joined #openstack-nova06:58
*** dave-mccowan has quit IRC06:58
*** tjones1 has quit IRC06:59
*** andreykurilin has joined #openstack-nova07:00
ttxgaryk: yes07:01
*** jamielennox is now known as jamielennox|away07:02
openstackgerritLi Yingjun proposed openstack/nova: Validate maximum limit for quota  https://review.openstack.org/16506907:02
garykttx: sorted thanks!07:02
ttxcool :)07:02
*** sahid has joined #openstack-nova07:03
*** tjones1 has joined #openstack-nova07:06
*** pask81 has joined #openstack-nova07:06
*** hdd has quit IRC07:07
*** hdd has joined #openstack-nova07:08
*** lpetrut has joined #openstack-nova07:08
garykttx: i am sure that i will need to pick your brain when we go from the proposed to stable :)07:10
ttxheh, not that hard actually. Documented at https://wiki.openstack.org/wiki/Release_Team/How_To_Release07:12
*** andreykurilin has quit IRC07:13
openstackgerritJuliane Aulbach proposed openstack/nova: Change config drive attachement from virtio-scsi to virtio-blk.  https://review.openstack.org/17318407:13
*** irenab has joined #openstack-nova07:17
*** Maike has joined #openstack-nova07:18
*** sbfox has quit IRC07:20
openstackgerritGhanshyam Mann proposed openstack/nova: Merge server personality tests between v2 and v2.1  https://review.openstack.org/17319607:22
*** arnaud___ has joined #openstack-nova07:23
*** ildikov has joined #openstack-nova07:24
openstackgerritRajesh Tailor proposed openstack/nova: Fix quota-update when nova-compute startup finish  https://review.openstack.org/17011807:26
*** achanda has quit IRC07:26
*** jichen has joined #openstack-nova07:27
*** dsanders has joined #openstack-nova07:27
*** mpaolino has joined #openstack-nova07:27
*** rajesht has joined #openstack-nova07:27
*** jistr has joined #openstack-nova07:29
rajeshtjogo: you around ?07:29
rajeshtjogo: I have submitted patch for quota update issue on nova-compute restart07:29
rajeshtjogo: you can give your suggestions:07:29
rajeshtjogo: [1] https://review.openstack.org/#/c/12234707:30
rajeshtjogo: [2] https://review.openstack.org/#/c/170118/07:30
*** yamahata has quit IRC07:30
*** park_heijlong has quit IRC07:30
*** dulek has joined #openstack-nova07:36
*** ndipanov has quit IRC07:41
*** romainh has joined #openstack-nova07:42
*** ndipanov has joined #openstack-nova07:42
*** jlanoux has joined #openstack-nova07:43
*** markus_z has joined #openstack-nova07:44
*** park_heijlong has joined #openstack-nova07:44
*** eglynn has joined #openstack-nova07:49
*** oro has quit IRC07:49
*** browne has quit IRC07:50
*** park_heijlong has quit IRC07:55
*** oro has joined #openstack-nova07:57
*** wuhg has quit IRC07:59
*** otter768 has joined #openstack-nova08:05
*** jcoufal_ has joined #openstack-nova08:06
*** tdurakov has quit IRC08:06
*** dims has joined #openstack-nova08:07
*** park_heijlong has joined #openstack-nova08:07
*** dulek has quit IRC08:08
*** dsanders has quit IRC08:08
*** jcoufal has quit IRC08:09
*** otter768 has quit IRC08:10
openstackgerritFeodor Tersin proposed openstack/nova: Do not specify default security group to Neutron by default  https://review.openstack.org/17320408:10
*** derekh has joined #openstack-nova08:10
*** BobBall_AWOL is now known as BobBall08:11
BobBallmriedem_away: There was a new race condition introduced in test_snapshot_pattern which seems to be hitting it quite hard.  We've not understood it yet.08:11
*** zz_johnthetubagu is now known as johnthetubaguy08:11
*** dims has quit IRC08:12
BobBallmriedem_away: Am considering disabling that test in our CI to get it back up and running, but have resisted so far08:12
*** gszasz has joined #openstack-nova08:14
*** oro has quit IRC08:17
*** gszasz has quit IRC08:18
*** doron_afk is now known as doron_08:19
*** gszasz has joined #openstack-nova08:19
*** ujjain has joined #openstack-nova08:20
*** pixelb has joined #openstack-nova08:21
*** sdake has joined #openstack-nova08:22
*** gszasz has quit IRC08:23
*** gszasz has joined #openstack-nova08:23
*** lucasagomes has joined #openstack-nova08:26
*** arnaud___ has quit IRC08:26
*** sdake_ has quit IRC08:26
*** ildikov has quit IRC08:27
openstackgerritwanghao proposed openstack/nova: Allow bdm boot_index=1 when creating vm from image  https://review.openstack.org/17198408:29
openstackgerritOleg Bondarev proposed openstack/nova: Log a warning if sec-groups and port id are provided on boot  https://review.openstack.org/15406808:30
openstackgerritOleg Bondarev proposed openstack/nova: Log a warning if sec-groups and port id are provided on boot  https://review.openstack.org/15406808:30
*** sdake has quit IRC08:30
heyongli_ndipanov:  patch updated accorrding discuss :)08:32
*** Nikolay_St has joined #openstack-nova08:32
openstackgerrityuntongjin proposed openstack/nova: Add missing policy for extension_info in API  https://review.openstack.org/17321008:32
*** eliqiao1 has quit IRC08:32
*** dulek has joined #openstack-nova08:32
*** yuntongjin has quit IRC08:33
*** heyongli_ has quit IRC08:33
openstackgerritSudipta Biswas proposed openstack/nova: Allow discrete blocks of online pCPUs for pinning  https://review.openstack.org/17318708:34
*** heyongli_ has joined #openstack-nova08:34
*** shaohe_feng has quit IRC08:35
*** oro has joined #openstack-nova08:36
ndipanovheyongli_, awesome - will take a look08:36
*** gszasz has quit IRC08:36
*** gszasz has joined #openstack-nova08:36
openstackgerritLianhao Lu proposed openstack/nova: Convert timestamp string in bandwidth_usage object  https://review.openstack.org/17321308:36
*** heyongli_ has quit IRC08:37
*** heyongli_1 has joined #openstack-nova08:37
*** oro has quit IRC08:45
*** heyongli_1 has quit IRC08:46
*** heyongli_ has joined #openstack-nova08:46
*** marun has joined #openstack-nova08:46
*** lpetrut has quit IRC08:47
oomichimikal, johnthetubaguy: hi08:48
johnthetubaguyoomichi: hi, hows things?08:48
oomichijohnthetubaguy: I found a problem on api microversions, can you take time for it?08:49
*** ujjain has quit IRC08:49
johnthetubaguyoomichi: possibly, whats up?08:49
oomichijohnthetubaguy: thanks. now we don't have the way to notify available microversions to clients.08:50
*** ujjain has joined #openstack-nova08:50
openstackgerritheijlong proposed openstack/nova: Merge extended-volume functional tests of v2 and v2.1  https://review.openstack.org/17321608:50
*** lpetrut has joined #openstack-nova08:50
oomichijohnthetubaguy: the way is described on the nova-spec, but it is not done yet.08:50
johnthetubaguyoomichi: ah, did we not get that implemented… I know there was talk that08:50
oomichijohnthetubaguy: yeah, https://review.openstack.org/#/c/172856/ is for fixing it.08:50
johnthetubaguyoomichi: OK, so python-novaclient needs that I guess08:50
oomichijohnthetubaguy: yes, right. novaclient also needs it.08:51
*** marun has quit IRC08:51
johnthetubaguyoomichi: OK, I marked it critical, lets get this into RC208:51
*** pkoniszewski has joined #openstack-nova08:51
oomichijohnthetubaguy: that is basic/important part of microversions.08:51
johnthetubaguyoomichi: agreed08:51
oomichijohnthetubaguy: that is great, thanks :)08:51
gilliardoomichi: do you have any reviews?08:52
gilliardoomichi: I am sure I saw something about it on the ML08:52
oomichigilliard: https://review.openstack.org/#/c/172856/ is that08:52
*** aix has quit IRC08:53
johnthetubaguyoomichi: question about the min version08:54
johnthetubaguyoomichi: maybe it should be ""08:54
johnthetubaguyoomichi: given v2.1 currently accepts requests with no version08:54
*** heyongli_ has quit IRC08:54
gilliardoomichi: I was thinking of http://lists.openstack.org/pipermail/openstack-dev/2015-April/061183.html08:54
ndipanovjohnthetubaguy, so kilo backport potential gets cherry-picked to a branch or? how does that work08:55
oomichijohnthetubaguy: yes, right. client can use v2.1 without specifying a microversion, and nova works as v2 default behavior08:55
johnthetubaguyndipanov: after we ship, if it didn't make it into an RC we get those backported onto the stable branch08:55
johnthetubaguyoomichi: but how should the min_version report that?08:56
*** park_heijlong has quit IRC08:56
johnthetubaguyoomichi: one day we might not allow those requests, I guess08:56
ndipanovjohnthetubaguy, ah no I meant - how to get it into RC08:56
ndipanovjohnthetubaguy, when the time rolls around - we just decide on a meeting?08:56
johnthetubaguyndipanov: to get it into RC, you need the kilo-rc-potential tag08:56
johnthetubaguybaically thiery makes the final call on what could get in there08:57
*** ujjain has quit IRC08:57
johnthetubaguyndipanov: I think we just backport into this branch to get it into the RC: https://github.com/openstack/nova/tree/proposed/kilo08:57
oomichijohnthetubaguy: you mean when raising microversions(v2.10, etc), nova should deny requrest without microversion?08:57
johnthetubaguyoomichi: no08:57
ndipanovright ok let me make sure that the stuff I am interested in (2) is tagged then08:57
johnthetubaguyoomichi: it might want to do that in the future08:58
johnthetubaguyndipanov: be clear to explain the impact in the bug, to make sure its clear, probably good to get the changes up for the backport as well, to save time08:58
ndipanovjohnthetubaguy, right - makes sense08:59
ndipanovwill do08:59
bauzasjohnthetubaguy: morning09:00
oomichigilliard: thanks for the point. I missed it, will reply to it.09:00
bauzasjohnthetubaguy: I'm reviewing all the stuff needing to be targeted for Liberty re: sched09:01
bauzasjohnthetubaguy: a couple of proposals are in https://etherpad.openstack.org/p/liberty-nova-summit-ideas L50 and below09:01
bauzasjohnthetubaguy: wrt that, I'm planning to work on https://blueprints.launchpad.net/nova/+spec/allocation-ratio-to-resource-tracker09:01
bauzasjohnthetubaguy: the spec was approved for Juno, could we target the bp as a series goal for Liberty ?09:02
bauzasjohnthetubaguy: I'll repropose the specs for all the bps I mentioned, as I don't exactly know jaypipes's support for Liberty09:03
johnthetubaguyndipanov: as a heads up the current plan is to cut RC2 on friday, after talking about it in the nova meeting09:04
*** frickler_ is now known as frickler09:06
johnthetubaguybauzas: the main thing is you need a spec merged for liberty, but I can fast approve that, if you add previous-approved in the commit message, and ping me on here09:07
bauzasjohnthetubaguy: sure, that's my next steps09:07
johnthetubaguybauzas: cool, thanks09:07
bauzasjohnthetubaguy: do you have any idea on if jay will have some commitment time by L N09:08
bauzas?09:08
bauzasjohnthetubaguy: I totally understand he's just overbusy09:08
ndipanovjohnthetubaguy, ok that makes sense09:08
johnthetubaguybauzas: I have no idea, whats L N?09:10
johnthetubaguyndipanov: same as last time, is the summary, basically09:11
ndipanovI think it's L (french keyboard questionmark key) :D09:11
bauzasjohnthetubaguy: my bad, "L ?" ie. 'Liberty ?'09:11
bauzasndipanov: well it's more that I have fat fingers...09:11
bauzasndipanov: it's not chilling today, sunny but I'm still typing with gloves09:11
*** ildikov has joined #openstack-nova09:12
johnthetubaguybauzas: your guess is a good as mine, I suspect09:13
ndipanovbauzas, some backports to warm you up https://review.openstack.org/#/c/173225/09:13
bauzasndipanov: cool09:13
ndipanovI had to work around your service migration thing so would be good for you to ack it09:13
bauzasndipanov: woah, I love them both09:13
bauzasndipanov: okay, will look into them09:14
ndipanovPaulMurray, ^^09:14
ndipanovif you could take a look it would be awesome09:14
*** mpavone has joined #openstack-nova09:14
oomichialex_xu: hi, I'd like to know your point on https://review.openstack.org/#/c/172771/09:15
*** pkoniszewski has quit IRC09:16
*** TobiasE1 has joined #openstack-nova09:16
*** TobiasE has quit IRC09:17
*** asrangne has joined #openstack-nova09:18
*** aswadr has quit IRC09:18
johnthetubaguyoomichi: thats probably mostly my fault, we can't deploy v2.1 at rackspace because of the fear it will break too many clients, we need a way to answer that question, or reduce the impact09:20
*** aix has joined #openstack-nova09:20
johnthetubaguyoomichi: talking with alex_xu we are thinking about relaxing additional properties checks when API calls don't have a version in them, and go into v2.109:21
johnthetubaguyoomichi: that should get us the best of both worlds, although as alex_xu points out, there are still other things that could cause problems, but I am OK with that09:21
johnthetubaguyoomichi: so rather than error out, I would want our code to just drop the attributes that are not expected before passing the user request to the API code09:22
johnthetubaguybut if there is a version specified, we error out with bad request, as planned09:22
*** ujjain has joined #openstack-nova09:23
oomichijohnthetubaguy: interesting, is it fine to have an option for disabling additional properties checks ?09:23
johnthetubaguyoomichi: no09:24
PaulMurrayndipanov, will do09:24
johnthetubaguyoomichi: reason why, I want the API to be the same for everyone when you use v2.109:24
johnthetubaguyoomichi: except for policy09:25
oomichijohnthetubaguy: ah, I see. when a client pass microversion, additional properties checks will work.09:25
johnthetubaguyoomichi: right09:25
oomichijohnthetubaguy: maybe it is possible, but we need time for bp/spec but it would be helpful for real world09:26
johnthetubaguyoomichi: I want to deprecate v2.0 at the end of liberty, so we can rip out that code when M opens09:26
johnthetubaguyoomichi: if we can get agreement, I can try get hands on keyboards to help with it, as it should help our adoption of v2.109:27
oomichijohnthetubaguy: ok, will write bp/spec for Liberty.09:28
*** jlanoux_ has joined #openstack-nova09:28
*** TobiasE1 has quit IRC09:30
johnthetubaguyoomichi: I can do that if it helps free you up for other things?09:30
*** jlanoux has quit IRC09:30
*** TobiasE has joined #openstack-nova09:32
*** doron_ is now known as doron_afk09:33
*** jwcroppe has joined #openstack-nova09:34
gilliardoomichi: johnthetubaguy: Do you plan for a future change to make requests-without-a-version illegal?09:35
johnthetubaguygilliard: I think we need to have the option of doing that, honestly, I can't see it happening for years09:36
gilliardOK, so I'm +1 on oomichi's patch then.09:36
oomichigilliard: I feel requests-without-a-version is fine forever.09:36
gilliardIt would be a breaking change which could not be covered with a microversion increment...09:36
johnthetubaguyoomichi: OK, but does that mean it never makes sense to raise the min microversion, that seems very imilar09:37
oomichigilliard: nova just work on min microversion as the default behavior at the time.09:37
*** jlanoux has joined #openstack-nova09:37
PaulMurrayndipanov, I think there is a missing decorator in https://review.openstack.org/#/c/173225/1/nova/tests/pci/test_pci_manager.py09:37
johnthetubaguyoomichi: I am wondering if the current min_version = "" for v2.1, so its consistent with v2.009:37
johnthetubaguybut that links the two together in a bad way09:38
oomichijohnthetubaguy: yeah, that is a point when we need to raises min microversion. I guess it is very difficult..09:38
*** jwcroppe has quit IRC09:38
ndipanovPaulMurray, hmm you are right but tests did seem to pass for me which should not have happened...09:38
ndipanovgood catch!09:39
johnthetubaguyoomichi: honestly, its almost going to be never, due to the same arguments09:39
bauzasndipanov: the tests didn't fail ?09:40
oomichijohnthetubaguy: ah, your previous request(relax validation) would make the version info difficult..09:40
ndipanovbauzas, I think so...09:40
bauzasndipanov: it should, because the signature is different, nope ?09:40
ndipanovright09:40
ndipanovweird - well jenkins will tell us soon09:40
bauzasndipanov: I missed that, my bad09:40
*** jlanoux_ has quit IRC09:40
ndipanovanyway - go look at the next one too09:40
oomichijohnthetubaguy: on the request, the behaviors will be different between specifying "2.1" and non-specifying.09:40
oomichijohnthetubaguy: when client specifying "2.1", I think additional-properties check should work. but if not specifying any microversion, the check will not work.09:42
johnthetubaguyoomichi: right09:42
johnthetubaguyoomichi: shall I write up the spec for this, so its clearer?09:43
oomichijohnthetubaguy: so min_version meaning will be changed.09:43
bauzasndipanov: I left a comment on the next patch09:43
johnthetubaguyoomichi: in a way, yes I guess it is09:43
ndipanovbauzas, awesome - will look into it09:43
oomichijohnthetubaguy: yeah, that is nice :)09:43
rajeshtjohnthetubaguy: hi,09:43
bauzasndipanov: basically, except the minor miss you did, I'm OK but I'm concerned by the test coverage09:43
ndipanovbauzas, thanks a lot09:43
johnthetubaguyrajesht: hello09:43
rajeshtjohnthetubaguy: I have submitted quota update patch, please give your suggestion whenever you get time09:44
bauzasndipanov: so I was asking for lxsli's help09:44
*** ZZelle has quit IRC09:44
rajeshtjohnthetubaguy: thanks :)09:44
sudiptobauzas, Hello, whenever you have time - https://review.openstack.org/#/c/173187/ could you please provide your thoughts09:44
bauzasndipanov: as he was deeply involved in the test refactoring09:44
*** eglynn has quit IRC09:44
johnthetubaguyrajesht: whats the patch?09:44
rajeshtjohnthetubaguy: https://review.openstack.org/#/c/170118/09:44
*** ZZelle has joined #openstack-nova09:44
bauzassudipto: ack, CC'ing me09:44
*** doron_afk has quit IRC09:44
johnthetubaguyrajesht: OK, I am slammed with release stuff right now, so not totally sure when that will be, but thanks for the heads up09:44
*** moshele has joined #openstack-nova09:45
sudiptobauzas, thanks!09:45
rajeshtjohnthetubaguy: updating quota for instance which are stuck in deleting task_state on compute start-up09:45
rajeshtjohnthetubaguy: np, will wait for community suggestions09:46
openstackgerritAbhijeet Malawade proposed openstack/nova: log request-id mapping of nova and cinder  https://review.openstack.org/17323409:46
johnthetubaguyrajesht: not sure only doing things in the start up task are a good idea, but its better than never, I should read the patch09:50
sudiptoPaulMurray, I have a concern on object versions in Nova and hence wanted to get your thoughts. I saw that currently there isn't a great way to determine whether your  nova-compute process is of a different release than your controller instance, unless you implicitly know what those IncompatibleObjectVersion...Is there a reason for such versioning that doesn't a very explicit way of telling the user about what went wrong?09:51
*** tjones1 has quit IRC09:51
sudiptoPaulMurray, I mean - the general openstack versions and the object versions didn't seem to match...09:52
bauzassudipto: that's not exactly done that way09:53
rajeshtjohnthetubaguy: sure, if you get any other ideas, please feel free to comment on patch.09:53
sudiptobauzas, the object versioning?09:53
*** BobBall_ has joined #openstack-nova09:54
bauzassudipto: long story short, we consider that 2 objects should be able to talk between themselves, provided the backportability is done09:54
*** lifeless1 has joined #openstack-nova09:54
bauzassudipto: so, each time you're bumping a new version, you need to carry out the backwards compatibility09:54
*** Maike has quit IRC09:56
sudiptobauzas, Yeah i have absolutely no issues with the versions being compatible, I would take a recent case, I was using Openstack Juno controller, and then tried to add a debian node to it. The debian node by default got icehouse, even though it's again a user error to not check what version of nova-compute got installed.09:57
sudiptobauzas, With that said, the error was detected correctly and it was reported that the remote objects aren't compatible. But there still wasn't enough information to map - what versions of openstack releases did these versions belong to...09:58
*** tjones1 has joined #openstack-nova09:58
*** Maike has joined #openstack-nova09:58
sudiptobauzas, Just a thought, to see if that's the way it's meant to be handled :)09:58
bauzassudipto: well, it's a little different between RPC versions and objects versions09:59
bauzasRPC versions have indeed a release field for saying the corresponding version number but objects don't10:00
PaulMurraysudipto, if the conductor isn't more up to date than the compute-node there are likely to be problems. It won't be able to backport to a future version10:00
bauzassudipto: I honestly don't think it's necessary10:00
bauzasyeah, as PaulMurray said, you need to upgrade the conductor first10:00
sudiptobauzas, hmm yeah, I agree that the upgrade is a necessity, I was just wondering about the detectability aspect...10:01
PaulMurraysudipto, so I think question is how do I know that your controller is on icehouse and your compute-nodes are on juno10:01
sudiptoPaulMurray, bang on! That's my question!10:01
*** ondergetekende_ has joined #openstack-nova10:02
PaulMurraysudipto, there is a verison number for nova10:02
sudiptoPaulMurray, I did a dpkg -l | grep nova and then figured the nova version10:02
sudiptoPaulMurray, Then went to the openstack releases link and mapped it10:02
*** bkopilov has quit IRC10:02
*** READ10 has quit IRC10:02
*** yuntong has quit IRC10:02
*** lifeless has quit IRC10:02
*** harlowja_away has quit IRC10:02
*** ondergetekende has quit IRC10:02
*** BobBall has quit IRC10:02
*** jerdfelt has quit IRC10:02
*** TobiasE has quit IRC10:02
sudiptoPaulMurray, whilst that really worked - I was just wondering if that's the most comfortable way to determine (unless you have a different suggestion and jumped the gun)10:03
sudipto*I jumped the gun10:03
PaulMurraysudipto, I don't think there is anything in nova that is going to warn you about controller nodes being on an older version than compute-nodes other than the interactions failing to work10:04
PaulMurraysudipto, which is what they are supposed to do10:04
sudiptoPaulMurray, Ok.10:04
PaulMurrayyou might need something in your configuration management system/monitoring for that10:04
sudiptoPaulMurray, hmm, ok.10:05
sudiptoPaulMurray, I might be wrong, but for me, the controller was on Juno and the compute was on Icehouse, and it didn't work due to the object miss match.10:06
*** otter768 has joined #openstack-nova10:06
sudiptoPaulMurray, I was thinking, we should atleast include the supported object version in this class : IncompatibleObjectVersion in the exception.py - do you agree?10:07
sudiptoPaulMurray, As in the object version that is supported.10:07
*** tjones1 has quit IRC10:07
PaulMurrayyes - I do agree - that would be helpful10:07
sudiptoPaulMurray, Ok let me work on getting that in...10:10
*** otter768 has quit IRC10:11
openstackgerritJohn Garbutt proposed openstack/nova-specs: api: relax validation for unversioned requests  https://review.openstack.org/17324310:12
openstackgerritJohn Garbutt proposed openstack/nova-specs: api: relax validation for unversioned requests  https://review.openstack.org/17324310:13
*** IanGovett has joined #openstack-nova10:14
johnthetubaguyoomichi: alex_xu: uploaded a spec to describe what I was talking about, let me know if that makes any sense now: https://review.openstack.org/#/c/17324310:14
oomichijohnthetubaguy: thanks. that is good for production envs, WIP seems unnecessary ;)10:17
oomichijohnthetubaguy: for getting more feedback10:17
sudiptobauzas, do you like the patchset, sorry if i am asking too early.10:17
johnthetubaguyoomichi: I was expecting lots of typos, maybe there are not so many10:17
gilliardjohnthetubaguy :)10:19
johnthetubaguygilliard: thanks you spotted them for me :)10:21
openstackgerritSylvain Bauza proposed openstack/nova-specs: Proposes allocation ratios to the Resource Tracker  https://review.openstack.org/17325210:25
*** jlanoux_ has joined #openstack-nova10:28
*** eglynn has joined #openstack-nova10:29
*** jlanoux has quit IRC10:30
openstackgerritJohn Garbutt proposed openstack/nova-specs: api: relax validation for unversioned requests  https://review.openstack.org/17324310:31
openstackgerritJohn Garbutt proposed openstack/nova-specs: api: relax validation for unversioned requests  https://review.openstack.org/17324310:31
*** ildikov has quit IRC10:33
johnthetubaguyoomichi: alex_xu: I just realized, we can use the presence of the v2.0 entry to say if you can send a request that does not include a version10:39
*** ijw has joined #openstack-nova10:43
*** ijw has quit IRC10:43
*** doron_afk has joined #openstack-nova10:44
*** ijw has joined #openstack-nova10:44
*** jlanoux has joined #openstack-nova10:45
*** jlanoux_ has quit IRC10:48
*** ishant has quit IRC10:50
*** dims has joined #openstack-nova10:54
PaulMurrayVek, you hve a -1 on https://review.openstack.org/#/c/163328/ due to string freeze, would you lift that now please, thanks10:54
*** lsmola_ has joined #openstack-nova10:58
*** dims has quit IRC10:58
openstackgerritJohn Garbutt proposed openstack/nova-specs: api: relax validation for unversioned requests  https://review.openstack.org/17324310:59
*** ijw has quit IRC11:01
*** ijw has joined #openstack-nova11:01
openstackgerritMarkus Zoeller (markus_z) proposed openstack/nova: WIP: Update hypervisor support matrix with column for kvm on system z  https://review.openstack.org/17239111:02
*** xgsa has quit IRC11:09
*** irenab has quit IRC11:13
*** panbalag has joined #openstack-nova11:15
*** ijw_ has joined #openstack-nova11:15
*** ijw has quit IRC11:16
*** irenab has joined #openstack-nova11:18
*** mwagner_lap has quit IRC11:23
*** irenab has quit IRC11:23
*** irenab has joined #openstack-nova11:24
*** irenab has quit IRC11:25
*** ujjain has quit IRC11:27
*** asrangne__ has joined #openstack-nova11:28
*** asrangne has quit IRC11:32
*** aix has quit IRC11:32
*** aix has joined #openstack-nova11:33
*** asrangne__ has quit IRC11:37
*** Anticimex has quit IRC11:41
*** tojuvone has quit IRC11:41
*** jistr is now known as jistr|class11:47
ndipanovjohnthetubaguy, heyongli - see my comment #35 on https://bugs.launchpad.net/nova/+bug/138346511:49
openstackLaunchpad bug 1383465 in OpenStack Compute (nova) "[pci-passthrough] nova-compute fails to start" [High,In progress] - Assigned to Yongli He (yongli-he)11:49
ndipanovand tell me if you agree11:49
*** lucasagomes is now known as lucas-hungry11:49
*** READ10 has joined #openstack-nova11:54
*** yuntong has joined #openstack-nova11:56
*** Marga_ has joined #openstack-nova11:57
*** tiagogomes_ has joined #openstack-nova11:57
*** mpavone has quit IRC11:59
*** sudipto has quit IRC12:01
*** mwagner_lap has joined #openstack-nova12:02
*** deepthi has quit IRC12:03
*** baoli has joined #openstack-nova12:04
*** sahid has quit IRC12:04
*** salv-orlando has joined #openstack-nova12:04
*** ritesh has joined #openstack-nova12:06
*** dprince has joined #openstack-nova12:07
*** otter768 has joined #openstack-nova12:07
riteshHi12:07
riteshCould anyone help me for the question asked in this link12:08
riteshhttps://ask.openstack.org/en/question/65056/use-nova-context-to-get-image-details/12:08
*** mpavone has joined #openstack-nova12:11
markus_zritesh: Do you want to use the python API of the nova client?12:11
*** otter768 has quit IRC12:12
*** oomichi has quit IRC12:12
*** eharney has quit IRC12:12
openstackgerritGhanshyam Mann proposed openstack/nova: Merge preserve_ephemeral_rebuild tests between v2 & v2.1  https://review.openstack.org/17328412:12
ritesh@mpavone : we are trying to invoke glance api from nova api, to get the image reference12:14
ritesh@mpavone : we are writing our own service , which is perioidic task it will retrive the image reference when we give image uuid. but we are not able to perform this operation becuase our context object does not have auth token , service catalgo , etc12:17
markus_zritesh: I posted an answer, maybe it helps12:20
dulekHi! I'm trying to test performing volume-backed live migration. The problem is my instances have CDROM device attached which counts as local disk.12:20
dulekHow can I turn the CD-ROM devices off?12:20
markus_zdulek: In the file "/etc/nova/nova.conf" set the option "force_config_drive=False"12:22
dulekmarkus_z: Ah, this seems to be it.12:22
markus_zdulek: On each compute node where you don't want automatically a config drive for the guests.12:22
riteshHI markus_z12:23
riteshdef get_nova_client(ip, tenant="demo", user="admin", password="openstack"):12:23
riteshthis funtion has predefined values12:23
dulekmarkus_z: Thanks a lot!12:23
riteshfor keystone12:23
riteshwho do we do by not keeping hardcode values12:23
markus_zritesh: right, you have to change that accordingly.12:23
markus_zritesh: oh, sorry, maybe I got your question wrong, sorry12:24
*** flaper87 has quit IRC12:24
*** flaper87 has joined #openstack-nova12:24
*** marun has joined #openstack-nova12:24
riteshcan we get it from some configuration file or some request object  ? I don't have any request object at that moment..12:25
*** lpetrut has quit IRC12:28
*** marun has quit IRC12:29
*** shi has quit IRC12:29
*** tojuvone has joined #openstack-nova12:30
*** shi has joined #openstack-nova12:30
markus_zritesh: I'm not quite sure where you want to extend something. Your service is not triggered by a logged in user, is it?12:31
*** TobiasE has joined #openstack-nova12:31
*** doron_afk is now known as doron_12:31
riteshmarkus_z : Yes service will not trigger by a logged in user. Its a periodic task running in background.12:32
*** claudiub has joined #openstack-nova12:32
*** ijw_ has quit IRC12:33
bauzasndipanov: I think you should be interested in https://review.openstack.org/#/c/173187/312:33
riteshThe code in the periodic task will get the image details . we wiill provide only uuid of the image and want the reference otf that imagme12:33
bauzasndipanov: I can't really review it as the NUMA effort is like out to me12:33
*** aysyd has joined #openstack-nova12:34
markus_zritesh: I'm not an expert in this topic, but I assume you could use the python-glanceclient and set the environment variables (like OS_USERNAME and OS_PASSWORD). As your service has to create the request object with proper authentication itself, I'm not aware of other ways doing that.12:37
openstackgerritSylvain Bauza proposed openstack/nova-specs: Proposes allocation ratios to the Resource Tracker  https://review.openstack.org/17325212:38
riteshmarkus_z : Thank you... I will try to do as you said.12:39
*** tiagogomes_ has quit IRC12:41
*** MVenesio has joined #openstack-nova12:42
*** busterswt has joined #openstack-nova12:42
*** tiagogomes has joined #openstack-nova12:42
*** ajayaa has quit IRC12:45
*** lucas-hungry is now known as lucasagomes12:46
*** setuid has quit IRC12:46
ndipanovbauzas, thanks12:46
kashyapmdbooth: Just as an info fyi, this is the multi-node DevStack configs w/ Neutron I settled -- https://kashyapc.fedorapeople.org/virt/openstack/multi-node-configs/12:48
mdboothkashyap: Thanks12:49
*** irenab has joined #openstack-nova12:49
*** irenab has quit IRC12:49
*** setuid has joined #openstack-nova12:51
*** irenab has joined #openstack-nova12:52
*** dims has joined #openstack-nova12:52
johnthetubaguyndipanov: yeah, I think that makes sense, but I haven't tried it out12:53
openstackgerritAnkit Agrawal proposed openstack/nova: 'deleted' filter does not work properly  https://review.openstack.org/17329612:53
ndipanovjohnthetubaguy, the reporter is our QE so we should get some feedback soon12:53
*** ociuhandu has joined #openstack-nova12:53
*** tjones1 has joined #openstack-nova12:55
*** xgsa has joined #openstack-nova12:57
*** xyang1 has joined #openstack-nova12:58
*** smccully has quit IRC12:59
*** markvoelker has joined #openstack-nova13:00
*** mitz has quit IRC13:01
*** ozialien has joined #openstack-nova13:02
*** jistr|class is now known as jistr13:03
*** doron_mtg has joined #openstack-nova13:04
*** doron_ has quit IRC13:04
*** mitz has joined #openstack-nova13:04
*** jecarey has joined #openstack-nova13:05
*** markvoelker_ has joined #openstack-nova13:06
*** TobiasE1 has joined #openstack-nova13:06
*** wanghao_ is now known as wanghao13:06
*** jecarey_ has joined #openstack-nova13:06
*** TobiasE has quit IRC13:06
*** jecarey has quit IRC13:07
*** jecarey__ has joined #openstack-nova13:07
*** markvoelker has quit IRC13:09
*** jecarey__ has quit IRC13:09
*** TobiasE1 has quit IRC13:11
*** jecarey_ has quit IRC13:11
*** claudiub has quit IRC13:11
*** changbl has quit IRC13:12
openstackgerritAlvaro Lopez Garcia proposed openstack/nova: Add new image cache monitor in compute nodes  https://review.openstack.org/17330313:12
openstackgerritAlvaro Lopez Garcia proposed openstack/nova: scheduler: weight nodes based on their cached images  https://review.openstack.org/17330413:12
*** ozialien_ has joined #openstack-nova13:12
*** ozialien has quit IRC13:12
*** ozialien_ is now known as ozialien13:12
*** adalbas has joined #openstack-nova13:13
*** ferest has joined #openstack-nova13:13
*** dosaboy has quit IRC13:14
*** dosaboy has joined #openstack-nova13:14
*** mriedem has joined #openstack-nova13:15
*** mriedem_away has quit IRC13:15
*** neelashah has joined #openstack-nova13:18
*** ferest has quit IRC13:18
mriedemsdague: johnthetubaguy: ndipanov: would be nice to have live migration working for kilo https://review.openstack.org/#/c/173099/113:18
*** irenab has quit IRC13:18
mriedemdansmith: ^ that does depend on the logging patch13:18
mriedembut the backport doesn't need to13:19
sdaguemriedem: ... is there a regression test that could have caught the issue in https://review.openstack.org/#/c/173099/1 ?13:20
*** liusheng has quit IRC13:20
mriedemsdague: yes, aiopcpu full13:20
*** irenab has joined #openstack-nova13:20
sdaguemriedem: a narrower one?13:20
*** liusheng has joined #openstack-nova13:21
mriedemsdague: that rpc proxy code goes away when we drop compute rpc 3.0 i think13:21
mriedemdansmith: ^?13:21
*** haomaiwang has joined #openstack-nova13:21
mriedemsdague: so that proxy code is temporary and we remove in liberty13:21
mriedemand the aiopcpu job hits live migration13:21
sdagueok13:21
mriedemwe want to get that job voting so we don't regress13:22
mriedemwhich is what we did last week13:22
sdagueand then migrate_data becomes non optional?13:22
sdagueI guess that's my question, if things don't work if we don't pass it, why is it optional13:22
mriedemthat i do'nt know13:22
mriedembut seems it shouldn't be13:22
mriedemit's optional in the compute manager, i'd have to dig into the history to know why13:23
johnthetubaguysdague: I thought that was backwards compatibility at some point13:23
johnthetubaguyat one point, we didn't send it13:23
mriedemyeah idk, maybe some older rpc version?13:23
sdaguejohnthetubaguy: ok, but apparently it doesn't work at all if it's not there13:23
mriedemright13:23
*** ozialien has quit IRC13:23
mriedemlive block migration doesn't at least13:23
sdagueso backwards compat to broken seems.... sub optimal13:23
mriedemshared storage live migration might work13:23
mriedemthe libvirt driver assumes you have shared storage if migrate_data=None13:24
mriedemthat could probably be smarter13:24
*** jecarey has joined #openstack-nova13:25
*** irenab has quit IRC13:25
sdagueok, fine with the fix, I see johnthetubaguy already +Aed as well13:26
sdaguejust wanted to figure out if I was missing something. Seems like a good cleanup as well13:26
*** mpavone has quit IRC13:26
openstackgerritSylvain Bauza proposed openstack/nova-specs: Proposes Implements RequestSpec object  https://review.openstack.org/17331613:28
*** eharney has joined #openstack-nova13:29
openstackgerritSylvain Bauza proposed openstack/nova-specs: Proposes Implements RequestSpec object  https://review.openstack.org/17331613:30
*** VW_ has joined #openstack-nova13:31
*** dims has quit IRC13:31
*** irenab has joined #openstack-nova13:32
*** tjones1 has quit IRC13:32
*** dims has joined #openstack-nova13:32
*** irenab has quit IRC13:32
*** ajayaa has joined #openstack-nova13:33
*** coolsvap|afk is now known as coolsvap13:34
dansmithmriedem: sdague yeah, that code is all gone in a few days, and testing it specifically would be really synthetic13:34
sdaguedansmith: sure, that's fine if we're making it non optional soon13:35
*** sudipto has joined #openstack-nova13:35
*** sdake has joined #openstack-nova13:35
dansmithsdague: indeed13:36
sudiptobauzas, Thanks for the review. Regarding the UT aspect, I actually did a grep in the code to figure out where I could fit this in, Found that there were no existing UTs for existing method either. Can you suggest a placeholder for the UTs?13:36
bauzassudipto: you mean, there is no coverage yet ?13:37
bauzassudipto: lemme check13:37
sudiptobauzas, Yeah for the existing parent method. it just had a mocked return in the test_driver.py module.13:37
bauzassudipto: orly ? :O13:38
bauzassudipto: indeed, you're right13:39
bauzassudipto: well, I think then you should add some tests for this method, because it's not just a proxy method13:39
*** sdake has quit IRC13:40
*** ferest has joined #openstack-nova13:41
*** rushil has joined #openstack-nova13:41
*** emagana has joined #openstack-nova13:41
bauzassudipto: to answer your question, you should add them at the tail of test_update_status(self, mock_open):13:41
*** sudipto has quit IRC13:42
*** jwcroppe has joined #openstack-nova13:44
*** dboik has joined #openstack-nova13:44
*** rushiagr is now known as rushiagr_away13:44
mriedemdansmith: who is going to reply to the direct email spam for reviews to this ben grassart? :)13:44
dansmithmriedem: I figure someone will ;)13:45
*** ferest has quit IRC13:46
*** burt has joined #openstack-nova13:47
*** ajayaa has quit IRC13:49
*** oro has joined #openstack-nova13:52
*** r-daneel has joined #openstack-nova13:52
*** jwcroppe has quit IRC13:52
bauzasoh man, he's French13:52
*** armax has joined #openstack-nova13:53
mriedemsic'em13:53
*** sudipto has joined #openstack-nova13:55
*** sdake has joined #openstack-nova13:57
*** apuimedo has quit IRC13:58
*** mpavone has joined #openstack-nova13:58
*** armax has quit IRC13:58
*** armax_ has joined #openstack-nova13:58
*** apuimedo has joined #openstack-nova13:58
*** dims has quit IRC14:00
*** dims has joined #openstack-nova14:00
mriedemkashyap: btw, the fc21 job is on nova's experimental queue now and there were some results in https://review.openstack.org/#/c/147332/14:01
* kashyap clicks14:01
mriedemkashyap: test_volume_boot_pattern had failed, but the original test that was failing with live snapshot passed14:01
mriedemi just rechecked it14:01
mriedemhttp://logs.openstack.org/32/147332/3/experimental/check-tempest-dsvm-f21/ea2dfbb/14:01
*** rushiagr_away is now known as rushiagr14:01
kashyapmriedem: Oh, the "original test" being the infamous bug report you filed? :-)14:01
*** vladikr has joined #openstack-nova14:02
mriedemkashyap: yeah14:02
Mike_D_wk :)14:02
mriedemtest_images_one_server or whatever14:02
*** browne has joined #openstack-nova14:02
mriedemthat used to fail about 25% of the time in the gate with live snapshot,14:02
mriedemso we'd probably want to recheck it a few times to see if it shows up with newer libvirt/qemu14:02
*** ajayaa has joined #openstack-nova14:02
kashyapYeah, for the life of me I couldn't get it reproduced locally, I simply ran out of motivation on getting it reproduced.14:02
kashyapYep, looking at the logs. mriedem, I also noticed a Nova patch fly by where you've enabled a bit more debugging for live migration14:03
*** mtanino has joined #openstack-nova14:04
*** thangp has joined #openstack-nova14:04
mriedemkashyap: yeah, b/c we broke live migration on friday but pre_live_migration had no debug logging14:04
mriedemto see what the flags were in that code14:04
*** mleroy has quit IRC14:05
*** jaypipes has joined #openstack-nova14:06
kashyapmriedem: Maybe I'm being pedantic, you say "recheck", but it's actually 'check experimental' given your "Add check-tempest-dsvm-f21 to experimental queue for openstack/nova" commit.14:06
mriedemkashyap: yes, i did 'check experimental'14:06
mriedem'recheck' is fewer chars to type :)14:07
*** doron_mtg is now known as doron_afk14:07
kashyapAnd, thanks for chasing this down with unwavering patience!14:08
*** Nikolay_St has quit IRC14:08
*** otter768 has joined #openstack-nova14:08
mriedemnp14:08
mriedempatience is my middle name, just ask my family14:08
*** irenab has joined #openstack-nova14:08
kashyapSeemed like so, patience and speed - pretty deadly combination you seem to possess there.14:09
*** dboik_ has joined #openstack-nova14:09
dansmithmriedem: so, the last live fix is in the gate,14:10
dansmithand proposed against kilo14:10
dansmithso I think we're all good until rc2 happens, yes?14:10
*** dboik has quit IRC14:11
mriedemdansmith: yeah i think so14:12
mriedemi have another patch for a race seen in live migrate testing,14:12
mriedembut it hasn't seemed to be a huge issue14:12
*** irenab has quit IRC14:12
mriedemamazingly after i pushed my patch up for it :)14:12
*** otter768 has quit IRC14:13
dansmithokay14:13
*** Anticimex has joined #openstack-nova14:14
dansmithI feel like we need to do some release notes editing14:14
*** irenab has joined #openstack-nova14:14
dansmithalaski: there are probably multiple things we want folks to do relating to the new cells cb, right? are those things on the release notes yet?14:14
mriedemdansmith: release notes editing for live migration? such as?14:14
mriedem'tested with live block migration but not shared storage?'14:14
alaskidansmith: they are not.  but I'm not sure we want people doing anything yet14:15
alaskidansmith: but I'm happy to add some info in case people would like to try them14:15
mriedemwe should do a scrub of commits with UpgradeImpact that merged in kilo to make sure they have corresponding entries in the release notes14:15
*** cfriesen has quit IRC14:16
*** cfriesen has joined #openstack-nova14:16
*** irenab has quit IRC14:16
mriedemhttps://review.openstack.org/gitweb?p=openstack%2Fnova.git&a=search&h=refs%2Fheads%2Fmaster&st=commit&s=UpgradeImpact14:17
*** cfriesen has quit IRC14:17
*** cfriesen has joined #openstack-nova14:17
*** zhenguo has quit IRC14:17
bauzasmriedem: just to be clear, when do we need to add an UpgradeImpact tag ?14:17
mriedemthere aren't that many, i'll post something to the ML14:17
mriedembauzas: if there is an upgrade impact? :)14:18
bauzasmriedem: awesome, like changing a py file ? woah14:18
*** irenab has joined #openstack-nova14:18
mriedembauzas: https://wiki.openstack.org/wiki/GitCommitMessages#Including_external_references14:18
mriedemthere is a more formal description there14:18
bauzasdamned, beated by the manual14:18
*** nelsnelson has joined #openstack-nova14:18
*** markvoelker has joined #openstack-nova14:19
bauzasmriedem: I was knowing all of them but not this one, so my detach-service series was probably also related to an UpgradeImpact tag14:19
*** markvoelker_ has quit IRC14:19
dansmithmriedem: no, I just meant in general for the release, not necessarily for live migration things14:19
dansmithalaski: okay, I thought we needed them to bootstrap the new database and such14:19
dansmithalaski: aren't we writing to the instance index tables already?14:20
alaskidansmith: not yet14:20
mriedemdansmith: yeah, i'll post to the ML14:20
dansmithalaski: okay14:20
cfriesenmriedem: is there any documented ordering for upgrading nova-compute, nova-conductor, nova-api, nova-scheduler, DB?14:20
mriedemcfriesen: it's doc'ed, sec14:21
dansmithcfriesen: all the control plane services (not compute) have to go at the same time, then you can do computes one at a time14:21
mriedemcfriesen: http://docs.openstack.org/developer/nova/devref/upgrade.html14:21
mriedemif there is anything wrong there, we should update those docs14:21
cfriesenokay.  I'm looking at http://docs.openstack.org/openstack-ops/content/ops_upgrades-general-steps.html  and it just says "Upgrade OpenStack Compute (nova)"14:22
dansmithcfriesen: that is the high-level upgrade across the stack14:22
dansmithcfriesen: within nova you have more options about how you do step 314:22
*** irenab has quit IRC14:24
cfriesenmriedem: thanks.  What's step 9 talking about?  "Run the new nova-manage command that ensures all DB records are “upgraded” to new data version"14:25
*** gszasz has quit IRC14:25
cfriesenthat sounds like something other than the db sync command14:25
dansmithflavor-migrate is one14:25
*** irenab has joined #openstack-nova14:26
dansmithcfriesen: https://wiki.openstack.org/wiki/ReleaseNotes/Kilo#Upgrade_Notes_214:26
*** tonytan4ever has joined #openstack-nova14:27
*** sudipto has quit IRC14:28
*** zzzeek has joined #openstack-nova14:30
dansmithsdague: we probably need something on the release notes about ec2 api deprecation too14:30
*** xyang1 has quit IRC14:31
sdaguedansmith: agreed, I'll work on writing something up today14:31
*** diegows has joined #openstack-nova14:31
dansmithcool14:31
*** moshele has quit IRC14:31
*** zz_jgrimm is now known as jgrimm14:32
*** ankit_ag has quit IRC14:33
*** Marga_ has quit IRC14:34
*** BobBall_ is now known as BobBall14:35
mriedemdansmith: sdague: yeah that had UpgradeImpact on it14:37
mriedemstill drafting that email, got sidetracked14:37
kashyapmriedem: For completenss' sake, maybe you'd also want to update the bug 1334398 with your new finding (that the offending test passed) from the experimental queue, and reassign yourself the bug?14:37
openstackbug 1334398 in OpenStack Compute (nova) "libvirt live_snapshot periodically explodes on libvirt 1.2.2 in the gate" [High,In progress] https://launchpad.net/bugs/1334398 - Assigned to Chet Burgess (cfb-n)14:37
*** irenab has quit IRC14:37
*** irenab has joined #openstack-nova14:38
openstackgerritMatt Riedemann proposed openstack/nova: libvirt: cleanup unused lifecycle event handling variables from driver  https://review.openstack.org/17335414:39
mriedemkashyap: yeah i'm not there yet14:39
*** Maike has quit IRC14:39
*** sahid has joined #openstack-nova14:39
*** gszasz has joined #openstack-nova14:40
kashyapmriedem: Yeah, didn't mean to rush, I realize you're doing more investigation.14:41
*** n0ano has joined #openstack-nova14:42
*** bkopilov has joined #openstack-nova14:43
*** Hosam has joined #openstack-nova14:44
bauzasalaski: I'm just code reviewing the cells modules and I see the Cells scheduler having a build_instances() which creates new instances ?!14:44
mriedemgaryk: did you see my question in the ML about the broken vmware ci?14:44
openstackgerritzhangjl proposed openstack/nova: Check if destination_type in block_device_mapping_v2  https://review.openstack.org/17335714:45
mriedemgaryk: it said that https://review.openstack.org/#/c/136935/ broke vmware ci but vmware ci passed on that change14:45
*** oro has quit IRC14:45
mriedemwe have a shitload of DocImpact changes in master (kilo)14:47
*** xyang1 has joined #openstack-nova14:48
mriedemdims: can you update the release notes for 1153a46 ?14:49
garykmriedem: yes, i saw the mail. was in the process of answering. seems like there was a shange in tempest that caused the issue14:50
mriedemgaryk: passing network uuid on create?14:51
mriedemthat was new and shook out a few bugs14:51
garykshange => change14:51
*** dave-mccowan has joined #openstack-nova14:51
garyki need to speak with ryan who investigated it - then i can get back to you.14:51
mriedemjohnthetubaguy: is it worth adding http://docs.openstack.org/developer/nova/devref/upgrade.html to the kilo release notes upgrade section? or is that more of a 'this is where we want to get to for upgrades, but maybe not there yet'?14:51
garykissue i that there are many moving targets14:52
johnthetubaguymriedem: its forward looking at this point14:52
mriedemjohnthetubaguy: ok14:52
openstackgerritdave-mcnally proposed openstack/nova: rebuild_instance doesn't detach cinder volumes correctly  https://review.openstack.org/17295114:52
*** thedodd has joined #openstack-nova14:52
cfriesenthis is maybe a bit esoteric, hoping maybe someone knows the answer.  I'm trying to figure out where in nova we end up updating the port 'binding:host_id' for neutron when doing an evacuation.  Saw a case where we failed to bring up the instance on the new host because the 'binding:host_id' was still pointing to the previous host.14:53
*** hdd has quit IRC14:54
*** cbader has quit IRC14:55
*** marun has joined #openstack-nova14:55
*** cbader has joined #openstack-nova14:57
*** irenab has quit IRC14:58
*** irenab has joined #openstack-nova14:59
mriedemcfriesen: hmm, the neutronv2 api code would do that on deallocate (if that's called from the compute manager on evacuate/rebuild)14:59
*** mleroy has joined #openstack-nova14:59
*** dboik_ has quit IRC14:59
*** Nikolay_St has joined #openstack-nova14:59
openstackgerritSergey Nikitin proposed openstack/nova: Fixed incorrect dhcp_server value during nova-network creation  https://review.openstack.org/17336215:00
openstackgerritzhangtralon proposed openstack/nova: resize-revert can break anti-affinity policy  https://review.openstack.org/16011015:00
mriedemcfriesen: http://git.openstack.org/cgit/openstack/nova/tree/nova/network/neutronv2/api.py#n34415:00
*** coolsvap is now known as coolsvap|afk15:00
*** ozialien has joined #openstack-nova15:01
*** zhangtralon has joined #openstack-nova15:01
openstackgerritSergey Nikitin proposed openstack/nova: Fixed incorrect dhcp_server value during nova-network creation  https://review.openstack.org/17336215:01
*** dulek has quit IRC15:01
*** rushil_ has joined #openstack-nova15:01
*** rushil has quit IRC15:02
alaskibauzas: yes, the cells scheduler has to write the instance to the db before calling conductor because conductor assumes it exists15:02
*** kmartin has joined #openstack-nova15:02
openstackgerritSergey Nikitin proposed openstack/nova: Fixed incorrect dhcp_server value during nova-network creation  https://review.openstack.org/17336215:02
bauzasalaski: okay15:02
alaskibauzas: in meetings all morning so a bit slow atm15:02
bauzasalaski: no worries, just code reviewing for fixing the bug15:02
cfriesenmriedem: thanks, will try following that15:02
bauzasalaski: making sure I can do that correctly15:03
*** nkrinner has quit IRC15:03
*** mtanino has quit IRC15:03
bauzasalaski: was thinking about providing an indirection field for the instance object15:03
*** sbfox has joined #openstack-nova15:05
mriedemcfriesen: so i think in the evacuate case, recreate=True, so when you get here15:06
mriedemhttp://git.openstack.org/cgit/openstack/nova/tree/nova/compute/manager.py#n287115:06
*** Nikolay_St has quit IRC15:06
mriedemthat won't call driver.destroy which would have deallocated the port15:06
*** zhangtralon has quit IRC15:07
mriedemcfriesen: the only other method that's called in the rebuild operation is this, which is not implemented for neutron15:07
mriedemhttp://git.openstack.org/cgit/openstack/nova/tree/nova/network/neutronv2/api.py#n23215:07
mriedemcfriesen: so i think http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/manager.py#n2871 is probably the issue15:07
mriedemcfriesen: is there a bug reported?15:08
*** Nikolay_St has joined #openstack-nova15:08
*** jwcroppe has joined #openstack-nova15:08
cfriesenmriedem: not that I know of.  This was an internal bug that one of our testers caught.15:09
cfriesenmriedem: it only happens occasionally though, most of the time evac works15:09
mriedemhmm15:09
cfriesenmriedem: also, this is effectively on kilo-2.  we're in the process up moving to a newer version15:10
mriedemhmm, wonder if we test rebuild in tempest with neutron15:11
*** tojuvone has quit IRC15:11
mriedemwith the aiopcpu job we could start testing evacuate15:11
mriedemjogo: dansmith: ^15:11
dansmithmriedem: aye15:12
dansmithmriedem: especially since I'm going to effing fix it15:12
mriedemthere is no evacuate test in tempest today...15:12
dansmithreally? I thought there was15:12
clarkbdansmith: mriedem where is the live migrate fix at?15:12
mriedemdansmith: oh it's rebuild15:13
mriedemclarkb: sec15:13
mriedemclarkb: oh status wise? should be in the gate for master, proposed to proposed/kilo15:13
mriedemclarkb: https://review.openstack.org/#/c/173099/15:13
mriedemmerged to master15:13
clarkbwoot I can rerun tests for things, thanks15:14
*** mtanino has joined #openstack-nova15:14
*** ozialien has quit IRC15:14
*** jwcroppe has quit IRC15:14
*** Nikolay_St has quit IRC15:15
openstackgerritsahid proposed openstack/nova: virt: remove small pages request  https://review.openstack.org/17207915:15
*** marun has quit IRC15:15
*** dboik has joined #openstack-nova15:15
*** Nikolay_St has joined #openstack-nova15:16
openstackgerritDan Smith proposed openstack/nova-specs: Add config-filter spec  https://review.openstack.org/17337515:16
*** marun has joined #openstack-nova15:16
*** marun has quit IRC15:17
*** marun has joined #openstack-nova15:17
*** smccully has joined #openstack-nova15:18
*** dboik_ has joined #openstack-nova15:18
mriedemdansmith: so we have rebuild tests in tempest, a lot of them, but no evacuate tests15:18
dansmithwow15:18
mriedemso that's a todo for liberty with the aiopcpu job and we can validate it against your spec changes15:19
dansmithmriedem: maybe it should be a pre-work-item for this: https://review.openstack.org/#/c/161444/15:19
mriedemyeah15:19
mriedemi'll add that comment15:19
*** ekhugen has quit IRC15:20
dansmithokay, I have more to address in that one anyway, so that'll be good15:20
*** timello has quit IRC15:21
mriedemmtreinish: do i need a qa-spec to add os-evacuate testing to tempest? it's going to require a config option probably for multi-node support15:21
*** dboik has quit IRC15:21
*** maurosr has quit IRC15:21
*** jgrimm has quit IRC15:21
openstackgerritLucas Alvares Gomes proposed openstack/nova-specs: Ironic: Configdrive extra data  https://review.openstack.org/17337915:22
*** Nikolay_St has quit IRC15:22
*** smccully has quit IRC15:22
mtreinishmriedem: http://specs.openstack.org/openstack/qa-specs/README.html#tempest-specs-for-new-tests15:23
*** flaper87 has quit IRC15:23
mtreinishmriedem: short answer is for a single resource test the answer is no15:23
mtreinishmriedem: although you can check the api for multiple nodes like we do for the migration tests15:23
*** yamahata has joined #openstack-nova15:24
mtreinishor is this a flag to say evacuate actually works with multi-node15:24
mriedemmtreinish: is that just using the hosts API to see if there are multiple compute hosts?15:24
mtreinishyep15:24
mriedemif not, then we skip the evacuate test15:24
mriedemok15:24
mriedemthen i guess we wouldn't need a config option15:25
*** flaper87 has joined #openstack-nova15:25
*** flaper87 has quit IRC15:25
*** flaper87 has joined #openstack-nova15:25
clarkbI think we use a config option for the migration test15:25
mriedemwe do15:25
mtreinishclarkb: there are config options to say the nova supports migration, and the other features15:25
mriedembut that's for live block migration and block migration with a backing volume - two different tests15:25
openstackgerritSergey Nikitin proposed openstack/nova: Fixed incorrect dhcp_server value during nova-network creation  https://review.openstack.org/17336215:25
mtreinishbut it also checks to make sure there is >1 host available15:25
mriedemthe only case that evacuate might be special case is baremetal?15:25
*** Nikolay_St has joined #openstack-nova15:25
clarkbI see, so the criteria for evacuate is much simpler and tempest can just figure that out15:26
mtreinishmriedem: that's always the case with baremetal15:26
mriedemi think ironic was the only driver that had it's own custom rebuild method15:26
mriedemmtreinish: :)15:26
mriedemwell, this is very early and i probably won't get to it today, but it's on my list now15:26
mtreinishmriedem: btw, I think we should totally do an evacuate test on a single node. :)15:27
mriedemmtreinish: you mean negative test?15:28
*** amotoki has joined #openstack-nova15:28
mtreinishwould it just scheduler error? or is it caught before that15:28
*** doron_afk has quit IRC15:28
*** diegows has quit IRC15:28
mriedemit's caught in the api15:28
mtreinishmriedem: oh then, that's something you might want to test in nova's functional tests15:29
mriedemprobably15:29
mriedemit would only be unit tested today15:29
mriedemthere is some os-extended-evacuate-find-host extension which makes host optional i guess15:29
*** Longgeek has quit IRC15:29
mriedemso if you specify host on the request and it's the same host the the instance is already on, you get a 400 back right away15:30
mriedemif you don't specify host, the scheduler picks on15:30
mriedem*one15:30
*** Nikolay_St has quit IRC15:30
openstackgerritjichenjc proposed openstack/nova-specs: Allow adjust ephemeral disks during resize  https://review.openstack.org/14573615:30
mtreinishmriedem: wait you make an evacuate call and don't specify a host? That seems weird to me15:30
mriedemright15:31
mriedemthen the scheduler tries to find a host15:31
*** Nikolay_St has joined #openstack-nova15:32
*** oro has joined #openstack-nova15:32
*** dsanders has joined #openstack-nova15:32
*** dboik_ has quit IRC15:33
bauzasyeah, the logic is now the same for evacuate and migrate15:33
*** dboik has joined #openstack-nova15:33
bauzasno dest host => ask the scheduler15:33
mriedemwas going to say, i'm not familiar with the scheduler code that knows not to pick the host the instance is already on15:33
cfriesenmriedem: currently the evacuate code will only let you evacuate if the old compute service fails the is_up() test.15:33
bauzasmriedem: everything is done on the conductor side15:34
*** WannabeCloudHost has joined #openstack-nova15:34
WannabeCloudHostThere is a way to change or reset an instance root password througth API OR CLI please ? nobody has an answer in room 101 ...15:34
bauzasmriedem: that's only if you're not specifying a dest host that then it does call select_dest()15:34
mriedemcfriesen: do you have a link to the code where that check is made?15:34
mriedemthe is_up() check?15:34
bauzasmriedem: but as the request_spec object is not persisted, that's just... lottery :)15:35
mriedemcfriesen: nvm, found it15:35
cfriesenhttp://git.openstack.org/cgit/openstack/nova/tree/nova/compute/api.py#n325415:35
mriedemyeah15:35
mriedemdamn, well, ....15:36
*** changbl has joined #openstack-nova15:36
mriedemwe can't really test that w/o isolation in tempest...15:36
mriedemmtreinish: ^15:36
mriedemyou'd have to lock out all other tests from running, take down the source host service, evacuate to host B, then bring up the service on host A in tearDown15:36
cfriesenmriedem: bauzas:  actually, was wondering if that is_up() check is correct for the scenario where one compute service maps to multiple compute nodes. (ironic, vmware)15:37
bauzascfriesen: it doesn't15:37
bauzascfriesen: because is_up() is checking the service db entry15:37
bauzascfriesen: so saying that you have distributed hosts behind a nova-compute service, it would be possible to have a service up but a node down15:38
cfriesenbauzas: right, so what happens if one host goes down?  how do we get the stuff that was running on that host onto a different host?15:38
cfriesenor is that outside of nova's responsibility?15:38
bauzascfriesen: that's not Nova15:39
bauzascfriesen: but there are some efforts to fence that15:39
mriedemcan i cue jaypipes to rant on this? :P15:39
bauzascfriesen: http://blog.russellbryant.net/2015/04/08/implementation-of-pacemaker-managed-openstack-vm-recovery/15:39
* jaypipes flies in from side stage15:39
mtreinishmriedem: well there is definitely no mechanism for doing that in tempest15:39
bauzascfriesen: you mean auto-evacuate ?15:40
*** gszasz has quit IRC15:40
mtreinishmriedem: but it doesn't work at the same time other api requests are being made? Like what happens if a node is in the middle of an evacuate and it gets a delete request for a server it's moving?15:40
cfriesenbauzas: no, I'm just trying to figure out how evacuate is supposed to work if a host goes down but the compute service that controls it is still up.  The self.servicegroup_api.service_is_up() call will pass, so it will prevent us from doing an evacuate.15:41
bauzascfriesen: oh that15:42
bauzascfriesen: because you hitted a flaw with distributed computes15:42
bauzascfriesen: there is a spec trying to fix that, hold on15:42
*** unicell has joined #openstack-nova15:42
*** ajayaa has quit IRC15:42
cfriesenbauzas: I'm commenting on the spec about dealing with host faults. (https://review.openstack.org/#/c/169836/8/specs/liberty/approved/report-host-fault-to-update-server-state-immediately.rst)15:43
openstackgerritPrzemyslaw Czesnowicz proposed openstack/nova: Fix handling of pci_requests in consume_from_instance.  https://review.openstack.org/17339415:43
*** unicell1 has quit IRC15:43
*** ekhugen has joined #openstack-nova15:44
bauzascfriesen: yeah, I was thinking about that spec15:44
*** markvoelker has quit IRC15:44
bauzascfriesen: but IMHO, it's just insane to consider that service == compute15:44
*** jgrimm has joined #openstack-nova15:44
cfriesenbauzas: with the current code I think if the compute service goes down we'd have to mark all the hosts controlled by it as "down" too, and fence them all.  Really not polite.15:44
pczesnondipanov: are you there?15:44
cfriesenbauzas:  in the case of kvm it works okay since we have the service running on each host15:45
bauzascfriesen: yeah, that's exactly the point of my first comment on that spec15:45
cfriesenbauzas: but for multiple hosts controlled by one service it sort of falls apart15:45
*** maurosr_ has joined #openstack-nova15:45
*** ajayaa has joined #openstack-nova15:46
bauzascfriesen: the scheduler knows about nodes, and return nodes15:46
*** dprince has quit IRC15:46
*** timello has joined #openstack-nova15:46
bauzascfriesen: the scheduler doesn't care about services, except only for checking that the attached service is not marked as disabled15:46
cfriesenbauzas: yes, but I think that's a flaw in the current nova design, not in that spec.  Ideally we should be able to report a single host down and the evac code should be checking whether the host is up15:46
cfriesenbauzas: yes, but it should also be checking whether the host is up, no?15:46
cfriesenor is that another level of scheduling15:47
bauzascfriesen: that's what I'm trying to explain : the proposal is wrong because it makes wrong assumptions15:47
*** dsanders has quit IRC15:47
bauzascfriesen: well, that's a good question, but it already does that kind of things in the HostManager15:47
cfriesenbauas: so what's your suggested solution?  The problem statement is that if a kvm host goes down, it takes 60 seconds for nova to notice.  An external monitor notices in say 5 seconds, and wants to tell nova that it should allow evacuation of all instances running on that host.15:48
cfriesenmake that bauzas. :)15:48
bauzascfriesen: https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L559-L56515:48
bauzascfriesen: I would just propose to amend os-hypervisors, that's what I said in the spec :)15:49
*** smccully has joined #openstack-nova15:49
bauzascfriesen: but it would probably require to change a little more of logic15:49
*** arnaud___ has joined #openstack-nova15:50
bauzascfriesen: like making sure that ComputeNodeList.get_all() doesn't return these disabled hosts15:50
cfriesenbauzas: what about the case where a compute service goes down?  Technically at that point we know nothing about what is going on for the hosts it controls, so the "reliable" thing to do would be to fence them all and evacuate them all.   But that seems a bit drastic.15:51
*** dave-mccowan has quit IRC15:51
*** jistr has quit IRC15:51
mriedembauzas: cfriesen: so http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/api.py#n3254 is pretty harsh, right? shouldn't that be more conditional based on the type of deployment or capabilities of the driver?15:51
bauzascfriesen: if a service goes down, then ComputeFilter fences the attached nodes15:52
mriedemlike you said, if the service is talking to hosts A and B, and the instance is on A and it's down, couldn't we evacuate to B?15:52
mriedemah15:52
cfriesenmriedem: yes, I think that would make sense.  (But I've only ever used nova with kvm.)15:53
bauzascfriesen: mriedem: I don't think it's necessary15:53
openstackgerritdave-mcnally proposed openstack/nova: rebuild_instance doesn't detach cinder volumes correctly  https://review.openstack.org/17295115:53
bauzaseither the service is down and then all related computes are fenced15:53
bauzaswhich is the behaviour we want15:53
cfriesenbauzas: the issue is if the service is up, and one of the hosts that service controls goes down.  Currently that check would prevent us from evacuating to another host controlled by the same service.15:54
bauzasor the service is up but an host is down and then we need something to tell that we want to prevent the scheduler to serve that host15:54
cfriesenbauzas: actually, it's worse.  if the service is still up it won't let us evacuate anywhere15:54
bauzascfriesen: if the service is up and the host is down, it's still considered as valid15:55
bauzascfriesen: oh right, inversed logic15:55
*** irenab has quit IRC15:55
mriedembauzas: cfriesen: well the good news is the service is_up check is there from the original impl https://review.openstack.org/#/c/11086/15:55
bauzasso, yeah, it would require a few additions, I agree15:55
mriedemsince grizzly15:55
*** garyk has quit IRC15:56
mriedemso in the compute api could we just change that to not checking the service but checking the host?15:56
*** dave-mccowan has joined #openstack-nova15:57
mriedemthis doesn't help adding a test to tempest.... :(15:57
bauzasmriedem: mmm15:57
*** krtaylor has quit IRC15:57
cfriesenmriedem: https://review.openstack.org/#/c/169836/8/specs/liberty/approved/report-host-fault-to-update-server-state-immediately.rst is looking at telling nova that a host is down.  maybe you could use that to fake it out as long as you're on local storage.15:58
mriedemcfriesen: yeah, i wanted to read that spec, just hadn't done it yet15:58
mriedemwill check that out15:58
cfriesenmriedem: when using openstack with vmware, do they have vmware-specific controls for doing evacuation from host to host?15:59
cfriesenmriedem: that spec is interesting but incomplete.  I think we need to handle both cases--the compute service being reported as down as well as the host being reported as down.  (and in the case of kvm those reports could come in at the same time)16:00
bauzascfriesen: agreed, provided it uses the right endpoints :)16:00
*** Longgeek has joined #openstack-nova16:00
cfriesenmriedem: bauzas: internally we already have something that basically does exactly what's in that spec.  but since we only use kvm we could short-circuit it and just report the service as down16:01
*** Longgeek has quit IRC16:01
bauzascfriesen: that's what is commonly done16:01
bauzascfriesen: hence the flaw16:01
*** Longgeek has joined #openstack-nova16:01
bauzascfriesen: because the 1:N relationship just breaks that assumption16:01
*** irenab has joined #openstack-nova16:02
*** erkules_ has joined #openstack-nova16:02
cfriesenbauzas: it'd be interesting to find out from someone actually using 1:N how they handle evac on host failure16:02
*** erkules has quit IRC16:02
bauzascfriesen: do you really want an answer ? :)16:02
*** doron_afk has joined #openstack-nova16:03
bauzascfriesen: I honestly doubt that VMware clusters are doing that feature on the OpenStack side16:03
cfriesenbauzas: well, if they have a hypervisor-specific method of doing it already, then maybe nova doesn't need to16:03
*** Nikolay_St has quit IRC16:03
*** oro has quit IRC16:03
bauzascfriesen: for Ironic, there is a one-size-fits-all thing for instances16:04
mriedemcfriesen: i honestly don't know the details on the vmware stuff16:04
mriedemgaryk and tjones aren't around either16:04
mriedemdims might know16:04
*** mpavone has quit IRC16:04
cfriesenit would be wasteful to spend a bunch of time making a generic 1:N solution for host failure if nobody is actually going to use it16:05
*** sahid has quit IRC16:05
*** ijw has joined #openstack-nova16:05
bauzascfriesen: but we need to support it, for the worst or the good16:05
*** ociuhandu has quit IRC16:06
*** Longgeek has quit IRC16:06
bauzascfriesen: we can't just say "eh, nobody uses it *now* so we can make assumptions" because that's how we create tech deby16:06
bauzasdebt16:06
dimscfriesen: vmware driver does not expose esxi hosts in the vsphere cluster anywhere in the Nova API or CLI.16:07
cfriesendims: okay, so if a host goes down vmware will handle evacuation internally?16:08
dimscfriesen: yes16:08
mriedemdims: how is that synced back up to the instance.host entry in the nova db?16:08
*** unicell has quit IRC16:09
bauzasmriedem: dims: IIUC, the whole cluster is considered as a single compute16:09
*** otter768 has joined #openstack-nova16:09
bauzasmdbooth: correct ?16:09
mriedemoh16:09
mriedemso nova doesn't know how many hosts are behind that single cluster, so instance.host doesn't change16:09
bauzasmriedem: dims: and a second compute mapped to the same service just means that's a second cluster16:09
bauzasmriedem: yeah16:09
mriedemok16:10
bauzasmriedem: so having to evacuate inst1 from hostA to hostB means "move from cluster A to cluster B"16:10
bauzasAFAIK16:10
mriedemhrm16:10
dimsbauzas: no way to do that in nova vmware driver through nova cli / api16:10
dimssee https://wiki.openstack.org/wiki/HypervisorSupportMatrix :) evacuate is empty16:11
bauzasdims: ok, you mean that rebuild_instance is not supported ?16:11
*** jcoufal_ has quit IRC16:11
*** sdake_ has joined #openstack-nova16:11
*** sdake_ has quit IRC16:11
*** krtaylor has joined #openstack-nova16:11
mriedemwell....16:11
mriedemevacuate has a default impl in the compute manager16:12
*** Marga_ has joined #openstack-nova16:12
*** sdake_ has joined #openstack-nova16:12
mriedemthe only virt driver that does a unique thing with that is ironic16:12
cfriesenhmm..that matrix says that ironic doesn't do evacuate either16:12
bauzaslet me check the driver.rebuild() implementation16:13
cfriesenwait...http://docs.openstack.org/developer/nova/support-matrix.html  shows that libvirt doesn't do evacuate.  that seems odd16:13
mriedemdims: hmm, hypervisor support matrix says that evacuate doesn't work for kvm either :) http://docs.openstack.org/developer/nova/support-matrix.html16:13
mriedemcfriesen: you beat me to it :)16:13
cfriesen:)16:13
dimshaha, don't know about that :)16:13
*** otter768 has quit IRC16:14
bauzasokay, IIRC rebuild() is not implemented for VMware16:14
bauzasin the driver I mean16:14
dimsso, using Nova API / CLI you can't move a vm from one vsphere cluster to another cluster16:14
mriedemcfriesen: dims: bauzas: http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/manager.py#n302716:14
*** Marga_ has quit IRC16:14
mriedemthat's where the driver is called, but if it doesn't impelement rebuild we use the compute manager default16:14
mriedembauzas: it's only implemented in the ironic driver16:14
mriedemhistorically16:14
*** markvoelker has joined #openstack-nova16:14
*** rushil_ has quit IRC16:15
dansmithjaypipes: https://review.openstack.org/#/c/133663/20 <-- converts service and servicegroup to objects16:15
*** Marga_ has joined #openstack-nova16:15
bauzasmriedem: yeah that's what I'm saying, driver's rebuild is not implemented for vmware16:15
jaypipesdansmith: k, will look in a bit.16:15
*** Nikolay_St has joined #openstack-nova16:15
jaypipesdansmith: you mean just for the DB driver, right?16:15
mriedembauzas: it's not implemented for libvrit either :)16:15
mriedembauzas: it's an optional override16:15
*** sdake has quit IRC16:15
bauzasmriedem: I see16:16
dimsi am talking from a customer pov, i have not looked in the code :)16:16
dansmithjaypipes: yeah, although the mc driver uses the db to list service hostnames, so it is affected too16:16
*** emagana has quit IRC16:16
jaypipesdansmith: right. design fail there.16:16
bauzasmriedem: reading the default impl, can see it's a destroy/respawn16:16
mriedembauzas: yeah16:16
*** emagana has joined #openstack-nova16:16
mriedemit just re-uses stuff that the drivers implement16:17
bauzasmriedem: so that's technically possible for vmware too...16:17
*** mtanino has quit IRC16:17
*** Marga_ has quit IRC16:18
bauzasmriedem: not related, are you aware of the cells job progress ?16:18
mriedembauzas: i'm not16:19
*** Marga_ has joined #openstack-nova16:19
mriedemit was nearly clean last week16:19
mriedemi'm still trying to get the cells job logs indexed16:19
bauzasmriedem: okay, so we have a few failures left16:19
mriedemspeaking of, clarkb can you help here? https://review.openstack.org/#/c/172231/16:19
bauzasmriedem: there is one which will be blacklisted by me16:19
bauzasmriedem: and there are a few left which are due to a race cond. on instance.save()16:20
mriedembauzas: are there bugs/patches up for the remaining failures?16:20
bauzasmriedem: yeah16:20
*** ajayaa has quit IRC16:20
bauzashttps://bugs.launchpad.net/nova/+bug/144364016:20
openstackLaunchpad bug 1443640 in OpenStack Compute (nova) "Cells: race condition when saving an instance" [Undecided,New] - Assigned to Sylvain Bauza (sylvain-bauza)16:20
*** dboik_ has joined #openstack-nova16:20
bauzasmriedem: I'm working on fixing ^16:20
clarkbmriedem can you point to a job with those logs?16:21
*** emagana has quit IRC16:22
mriedemclarkb: http://logs.openstack.org/63/133663/20/check/check-tempest-dsvm-cells/597717a/logs/16:23
openstackgerritjichenjc proposed openstack/nova-specs: Allow adjust ephemeral disks during resize  https://review.openstack.org/14573616:23
*** Marga_ has quit IRC16:23
*** dboik has quit IRC16:23
*** panbalag has left #openstack-nova16:24
*** obedmr has joined #openstack-nova16:25
*** masayukig has quit IRC16:25
*** markvoelker has quit IRC16:26
*** Nikolay_St has quit IRC16:26
*** ijw has quit IRC16:26
*** jlanoux has quit IRC16:27
*** emagana has joined #openstack-nova16:27
*** jwcroppe has joined #openstack-nova16:28
mriedemjgrimm: krtaylor: can you find someone from powerkvm that can comment on the pkvm support for numa and hugepages in this review? https://review.openstack.org/#/c/170780/16:28
mriedemzkvm said it's not supported16:28
jgrimmmriedem, i just posted note to internal ml on it16:28
*** baoli has quit IRC16:29
jgrimmmriedem, i'm confident hugepages is supported.. i thought numa was supported, but maybe there are caveats/limitations with respect to your note about qemu 2.116:29
mriedemjgrimm: more interested in someone saying they've tested pkvm with those16:30
markus_zmriedem: Do you have a pointer for which openstack scenarios we (zkvm) cannot do without this support?16:30
*** baoli_ has joined #openstack-nova16:30
*** annashen has joined #openstack-nova16:30
jgrimmmriedem, understood. lacking a spare system atm to test quickly.. so hoping to short cut with experts on what is doable16:31
mriedemmarkus_z: http://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/input-output-based-numa-scheduling.html16:31
*** dboik has joined #openstack-nova16:31
*** masayukig has joined #openstack-nova16:31
mriedemmarkus_z: http://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/virt-driver-cpu-pinning.html16:31
*** irenab has quit IRC16:31
cfriesenjgrimm: the issue is that if we enabled numa and hugepages it tries to use the new "memory-backend-file" API introduced in qemu 2.1....but there's a note there saying that it's only for x8616:32
openstackgerritLucas Alvares Gomes proposed openstack/nova-specs: Ironic: Configdrive extra data  https://review.openstack.org/17337916:32
markus_zThanks!16:32
*** emagana has quit IRC16:32
*** irenab has joined #openstack-nova16:32
*** jwcroppe has quit IRC16:32
jgrimmcfriesen, and we've started exploiting that API in kilo?16:33
*** emagana has joined #openstack-nova16:33
*** Nic has joined #openstack-nova16:33
jgrimmmarkus_z, also useful: https://blueprints.launchpad.net/nova/+spec/virt-driver-numa-placement16:33
openstackgerritLucas Alvares Gomes proposed openstack/nova-specs: Ironic: Configdrive extra data  https://review.openstack.org/17337916:33
lucasagomes:-( typo16:33
*** artom has joined #openstack-nova16:34
cfriesenjgrimm: yes, with the new NUMA pinning stuff.  We now support multiple numa nodes in the guest, and each guest numa node will map to a host numa node.16:34
markus_zjgrimm: thanks, I will take a look16:34
WannabeCloudHostThere is a way to change or reset an instance root password througth API OR CLI please ? nobody has an answer in room 101 ...16:34
*** kmartin has quit IRC16:34
cfriesenSo if you enable cpu pinning, or multiple numa nodes in guest, or hugepages in guest, it now wants to use the new qemu API16:34
*** amotoki has quit IRC16:35
*** dboik_ has quit IRC16:35
jgrimmcfriesen, ok, will try to understand plans around that feature16:35
*** pask81 has quit IRC16:35
*** romainh has left #openstack-nova16:36
*** garyk has joined #openstack-nova16:37
jgrimmcfriesen, do you happen to have any handy reference to bp that droppedin the memory-backend-file API exploitation?16:37
cfriesenjgrimm: one sec16:37
ndipanovpczesno, hey16:37
jgrimmcfriesen, no worries, thanks16:37
pczesnondipanov: will https://review.openstack.org/169245 be backported to kilo?16:38
ndipanovpczesno, for sure16:39
cfriesenjgrimm: looks like some of it was introduced by  https://bugs.launchpad.net/nova/+bug/138530816:39
openstackLaunchpad bug 1385308 in OpenStack Compute (nova) "Nova libvirt driver does not bind guest memory to NUMA nodes" [Medium,Fix released] - Assigned to Daniel Berrange (berrange)16:39
pczesnondipanov: i found couple of issues with that patch16:40
jgrimmcfriesen, thanks sir16:40
ndipanovpczesno, go on16:40
pczesnondipanov:  https://bugs.launchpad.net/nova/+bug/144402116:40
openstackLaunchpad bug 1444021 in OpenStack Compute (nova) "HostState.consume_from_instance fails when instance has numa topology" [Undecided,In progress] - Assigned to Przemyslaw Czesnowicz (pczesno)16:40
*** arnaud___ has quit IRC16:40
openstackgerritDan Smith proposed openstack/nova: Handle config drives being stored on rbd  https://review.openstack.org/12307316:41
*** maurosr_ is now known as maurosr16:41
*** rajesht has quit IRC16:41
*** nelsnels_ has joined #openstack-nova16:42
*** unicell has joined #openstack-nova16:42
*** unicell has quit IRC16:43
*** nelsnelson has quit IRC16:43
*** unicell has joined #openstack-nova16:43
*** derekh has quit IRC16:43
*** salv-orlando has quit IRC16:44
*** irenab has quit IRC16:45
ndipanovpczesno, sad16:45
*** Marga_ has joined #openstack-nova16:46
*** Marga_ has quit IRC16:46
*** nelsnels_ has quit IRC16:46
clarkbmriedem: approved. I like seeing the logs before hand to make sure we got the names right and that they are in a format we can index16:46
*** ijw has joined #openstack-nova16:47
ndipanovpczesno, "and change the order in which apply_requests and numa_fit_instance_to_host16:47
ndipanovare called. Calling apply_requests first will remove devices from pools16:47
ndipanovand  may make numa_fit_instance_to_host fail."16:47
*** nelsnelson has joined #openstack-nova16:47
bauzasalaski: still around for a cell question ?16:47
ndipanovpczesno, this is pretty much a separate bug16:47
ndipanovand also races could contribute to this as well16:48
*** irenab has joined #openstack-nova16:48
ndipanovso changing this means it wont' fall for the one request but may fall in the same way numa was failing before that fix16:48
ndipanovthis is just horrible :(16:48
ndipanovthat fix = my fix that broke it16:49
ndipanovsigh16:49
bauzasalaski: nvm, I think I understood : grand-parent cell is said 'api' for the cell_type while children and grand-children cells are both said as 'cell' for the cell_type16:49
*** markus_z has quit IRC16:49
*** apuimedo has quit IRC16:49
*** unicell has quit IRC16:50
alex_xujohnthetubaguy: cool for spec, will catch up it asap. I'm in travel, sorry for will reply late in this few days, but will try to catch up asap.16:51
*** Marga_ has joined #openstack-nova16:51
dansmithjaypipes: also in the queue for you is this: https://review.openstack.org/#/c/155006/16:52
dansmithjaypipes: my -1 is just for a spelling nit, so don't let that deter you16:52
pczesnondipanov: ok, i can split the patch in two16:52
jaypipeskk16:52
ndipanovpczesno, meh16:52
ndipanovthe issue is not only that16:52
*** apuimedo has joined #openstack-nova16:52
ndipanovthe issue is that apply_requests is also racy16:52
ndipanovand can blow up in the same way nova did16:53
pczesnondipanov: hmm, i'm not sure16:53
ndipanovyeah I'm pretty sure16:53
ndipanovthree concurrent requests pass the filter for 1 device16:54
ndipanovall of them try to claim it16:54
ndipanovsecond one blows up in consume from instance16:54
ndipanovfails the request16:54
*** dsanders has joined #openstack-nova16:55
ndipanovsad sad code16:55
*** salv-orlando has joined #openstack-nova16:55
pczesnoyes, you are right16:56
cfriesenwhat's the proper notification driver to use with ceilometer currently?  A bunch of instructions say to use "notification_driver=nova.openstack.common.notifier.rpc_notifier" but I'm pretty sure that doesn't exist anymore.16:56
ndipanovpczesno, anyway I will coment on it tomorrow16:57
pczesnondipanov: ok, thanks16:57
*** Marga_ has quit IRC16:58
alaskibauzas: correct.  cells got into a weird state where some things support more than 2 tiers and other things assume 216:58
pczesnondipanov: the CI to test the numa stuff is alomost ready, should be commenting in a week or two16:59
*** apuimedo has quit IRC16:59
*** harlowja has joined #openstack-nova16:59
pczesnondipanov: that would catch at least some of those problems16:59
bauzasalaski: trying to understand the need of that instance_update_at_top() stuff16:59
bauzasalaski: because it just assumes that we're changing something on the compute side which was not done on the api side17:00
johnthetubaguyalex_xu: no problems, thanks for looking17:00
bauzasalaski: which is acceptable, but with problems17:00
bauzasI'm goint to get that cells job green !17:00
alaskibauzas: things can definitely change in a cell without coming in from the api17:01
alaskiso instance_update_at_top is necessary17:01
*** ijw has quit IRC17:01
*** rmstar has joined #openstack-nova17:02
*** ijw has joined #openstack-nova17:02
rmstarhi guys.   i just want to run something by you and see if this it shoule fixed or not.17:02
alaskibauzas: what we need is some way to call instance.save(sync_cell=False), but preferable without actually modifying the save signature17:02
*** aswadr has joined #openstack-nova17:03
bauzasalaski: yeah, I don't like amending signatured17:03
bauzass*17:03
rmstari am using icehouse, and noticed that we are able to rename a flavor that is being used by an instance.  This should not happen, right?17:03
bauzasalaski: but I'm also taking time for reviewing cells code17:03
bauzasalaski: messaging module is reponsible for propagating manager's messages to either a specific cell or more ?17:04
*** tonytan4ever has quit IRC17:04
alaskibauzas: the messaging module is for cell->cell communication, yes17:05
bauzasalaski: ok, I can see the manager passing the CellStateManager to the MessageRunner17:05
bauzasalaski: so the messaging.MessageRunner knows how to triage those messages using the statemanager, understood17:06
*** irenab has quit IRC17:06
*** sdake has joined #openstack-nova17:07
*** sudipto has joined #openstack-nova17:08
*** Marga_ has joined #openstack-nova17:08
*** annashen has quit IRC17:08
*** doron_afk has quit IRC17:09
*** jwcroppe has joined #openstack-nova17:09
*** armax_ has left #openstack-nova17:09
*** armax has joined #openstack-nova17:09
*** sdake_ has quit IRC17:11
*** tonytan4ever has joined #openstack-nova17:11
mriedemclarkb: nice, thanks17:12
*** sdake_ has joined #openstack-nova17:14
*** jwcroppe has quit IRC17:16
*** dprince has joined #openstack-nova17:18
*** sdake has quit IRC17:18
*** aix has quit IRC17:18
*** dkliban_afk is now known as dkliban17:19
*** raph__ has quit IRC17:20
*** ndipanov has quit IRC17:22
*** isd has joined #openstack-nova17:23
*** jichen has quit IRC17:23
*** kmartin has joined #openstack-nova17:24
* artom tries his luck again since it wasn't fully cleared up yesterday.17:25
artomAbout the SPICE html5 console.17:25
*** mpaolino has quit IRC17:25
artomThe server_proxyclient_address config option is "The address to which proxy clients (like nova-spicehtml5proxy) should connect"17:26
artomI understand that as "the SPICE proxy will use that address as the SPICE server (ie, the compute host on which the instance being console'd is running)17:26
*** patrickeast has joined #openstack-nova17:26
artomTesting does confirm that - ie, setting it to 'foobar' will see nova-spicehtml5proxy attempt to connect to foobar and fail.17:27
artomI'm clearly missing something here, how was this meant to work in deployments with more than one compute?17:27
dansmithartom: that is so that the compute host knows what the client-visible proxy address is going to be right?17:29
dansmithartom: like vladikr said yesterday, the intent is to run one proxy out front that proxies connections for all the compute hosts, so that you don't expose the compute hosts directly17:30
dansmithspice could be different or broken, but that's the goal of the console proxy in general17:30
*** browne has quit IRC17:30
dansmith(and to do the openstack auth bit, of course)17:30
*** sdake has joined #openstack-nova17:31
artomdansmith, yeah, I get the intent.17:31
artomIt "reassures" me to hear you say that SPICE might be broken...17:32
dansmithI expect it's not,17:32
dansmithbut fewer people here probably know the details of the spice implementation17:32
artomLike you said, one proxy in front of all/many compute hosts. That implies code to look up which compute host the instance is on. And I can't find that code anywhere.17:33
artomAll I managed to find is that config option which apparently hardcodes it.17:33
vladikrartom, no, it's not, I think you are looking at the part that creates the socket on the host and then provides that socket to the proxy, in order to connect it to the source connected socket)17:33
dansmithyeah17:33
dansmithartom: get_spice_console() calls the correct compute host based on the instance17:34
dansmithand gets the console connection information, right?17:34
dansmiththat's where the which-compute-host switch happens17:34
dansmith(rpcapi.py)17:34
dansmith(compute/rpcapi.py)17:34
*** sdake_ has quit IRC17:35
*** Marga_ has quit IRC17:35
artomhttps://github.com/openstack/nova/blob/master/nova/compute/rpcapi.py#L53517:35
*** VW_ has quit IRC17:35
artomdansmith ^^17:35
dansmithartom: right17:35
artomThe server=_compute_host(None, instance) bit?17:35
*** Marga_ has joined #openstack-nova17:35
artomGoddammit, of course.17:36
*** Marga_ has quit IRC17:36
artomThat's so unlike what I was expecting...17:36
artomAnd so "innocuous"17:36
*** Marga_ has joined #openstack-nova17:36
dansmithit's how almost everything we do host-per-instance happens17:36
artomSo the RPC API handles sending the call to the right compute node.17:36
*** boris-42 has quit IRC17:38
artomAnd so then server_proxyclient_address is for the compute node to know it's "external" IP address.17:38
*** rushil has joined #openstack-nova17:39
artomMore precisely, the IP address through which the proxy can reach it.17:39
*** ajayaa has joined #openstack-nova17:40
*** hdd has joined #openstack-nova17:41
jogojohnthetubaguy: https://review.openstack.org/#/c/139272/ dropped my -2 on that one, anything else?17:41
*** lucasagomes is now known as lucas-dinner17:42
johnthetubaguyjogo: I don't remember, I think there were a few at the back end of the list of spec reviews: https://review.openstack.org/#/q/project:openstack/nova-specs+status:open,n,0032ddd30001ff6717:42
*** tonytan4ever has quit IRC17:44
*** sdake_ has joined #openstack-nova17:44
*** johnthetubaguy is now known as zz_johnthetubagu17:45
*** sdake has quit IRC17:45
artomdansmith, much thanks :) I was missing that crucial rpcapi bit.17:46
*** eglynn has quit IRC17:46
*** dboik has quit IRC17:47
*** dboik has joined #openstack-nova17:48
*** sdake_ has quit IRC17:48
*** sdake has joined #openstack-nova17:48
*** diegows has joined #openstack-nova17:48
*** ganso_ has joined #openstack-nova17:49
jogozz_johnthetubagu: thanks17:49
ganso_mriedem: Hello17:49
ganso_mriedem: I have not been able to recreate the problem, I rebuilt the Red Hat OpenStack 6 environment from scratch, and now either the problem is gone, or is random and has not appeared since17:50
*** achanda has joined #openstack-nova17:51
ganso_mriedem: It could be a configuration problem present in the previous environment17:51
mriedemganso_: ok17:51
mriedemwe still have it in the community gate i think17:51
mriedemso maybe i'll just recheck until we hit it, but it's rare17:52
*** sdake has quit IRC17:52
*** sdake has joined #openstack-nova17:52
ganso_mriedem: ok, I will let you know if I find anything related to it. Thanks for creating the patch, sorry for not being able to test it for you :\17:52
mriedemganso_: np17:53
*** eharney_ has joined #openstack-nova17:53
*** pczesno is now known as zzz_pczesno17:54
*** eharney has quit IRC17:54
dansmithartom: no problem17:54
*** eharney_ is now known as eharney17:55
*** Nic1 has joined #openstack-nova17:58
*** heyongli has quit IRC17:58
*** aswadr has quit IRC17:59
*** Nic has quit IRC17:59
*** dboik_ has joined #openstack-nova18:00
*** baoli_ has quit IRC18:01
*** hdd has quit IRC18:01
*** tonytan4ever has joined #openstack-nova18:02
*** dboik has quit IRC18:03
*** ajayaa has quit IRC18:04
*** jamielennox|away is now known as jamielennox18:05
*** baoli has joined #openstack-nova18:06
*** Hosam has quit IRC18:07
*** Hosam has joined #openstack-nova18:08
*** VW_ has joined #openstack-nova18:08
*** otter768 has joined #openstack-nova18:10
*** krtaylor has quit IRC18:12
*** heyongli has joined #openstack-nova18:12
*** Hosam has quit IRC18:12
rwsuanyone know to which queue the compute.instance.create.{start,error,end} amqp messages are sent?18:13
rwsuI'm not seeing them in notifications.info18:14
*** otter768 has quit IRC18:14
*** jwcroppe has joined #openstack-nova18:15
*** Mike_D_laptop has joined #openstack-nova18:19
*** yamahata has quit IRC18:22
*** salv-orlando has quit IRC18:25
*** tellesnobrega has quit IRC18:25
*** markvoelker has joined #openstack-nova18:25
*** sdake_ has joined #openstack-nova18:25
*** tellesnobrega has joined #openstack-nova18:26
*** jwcroppe has quit IRC18:27
*** tellesnobrega_ has joined #openstack-nova18:27
*** tellesnobrega_ has quit IRC18:27
*** sdake has quit IRC18:29
*** Marga_ has quit IRC18:29
*** tellesnobrega has quit IRC18:29
*** tellesnobrega has joined #openstack-nova18:29
*** tpatil has joined #openstack-nova18:30
*** markvoelker has quit IRC18:31
*** nelsnels_ has joined #openstack-nova18:31
*** Anticimex has quit IRC18:33
*** Anticimex has joined #openstack-nova18:33
*** nelsnels_ has quit IRC18:34
*** nelsnelson has quit IRC18:34
*** rushil has quit IRC18:34
*** nelsnelson has joined #openstack-nova18:34
*** rajesht has joined #openstack-nova18:39
*** sdake has joined #openstack-nova18:39
*** mleroy has quit IRC18:41
*** rajesht has quit IRC18:43
*** sdake_ has quit IRC18:43
*** yamahata has joined #openstack-nova18:45
*** browne has joined #openstack-nova18:45
*** pixelb has quit IRC18:47
*** baoli has quit IRC18:48
cfriesenrwsu: for what it's worth, I'm having issues with notifications as well.  I'm on a wonky setup though, so could very well be something local.18:48
cfriesenrwsu: are you getting other notifications from nova?18:49
*** Marga_ has joined #openstack-nova18:50
*** annegentle has joined #openstack-nova18:50
rwsucfriesen: I am seeing the internal notifications. I think my probem is I don't have config files setup to send notifications, https://ask.openstack.org/en/question/56034/how-to-make-nova-send-notifications/18:52
*** VW_ has quit IRC18:53
*** oro has joined #openstack-nova18:53
*** mleroy has joined #openstack-nova18:54
openstackgerritRadoslaw Smigielski proposed openstack/nova: New sensible network bandwidth quota values in Nova tests  https://review.openstack.org/17346718:54
*** VW_ has joined #openstack-nova18:54
*** annashen has joined #openstack-nova18:56
openstackgerritDan Smith proposed openstack/nova: Handle config drives being stored on rbd  https://review.openstack.org/12307319:02
dansmithmriedem: wanna take a look at this one ^ ?19:03
*** baoli has joined #openstack-nova19:03
dansmithit's been hanging for a long time and I've been trying to keep it fresh19:03
*** krtaylor has joined #openstack-nova19:03
mriedemdansmith: bad time today for in depth reviews, locked in a CR all day today19:04
dansmithokay19:04
mriedemdansmith: callback on that another day?19:04
dansmithsure19:04
*** vilobhmm11 has joined #openstack-nova19:07
*** rushiagr is now known as rushiagr_away19:10
*** dave-mccowan has quit IRC19:10
*** sudipto has quit IRC19:12
*** diegows has quit IRC19:16
*** oro has quit IRC19:17
*** mtanino has joined #openstack-nova19:18
*** annashen has quit IRC19:22
*** aix has joined #openstack-nova19:23
*** aysyd has quit IRC19:25
*** aysyd has joined #openstack-nova19:25
*** kmartin has quit IRC19:26
*** salv-orlando has joined #openstack-nova19:26
openstackgerritVilobh Meshram proposed openstack/nova-specs: Replace service groups with the tooz groups  https://review.openstack.org/13860719:28
*** markvoelker has joined #openstack-nova19:29
*** dguitarbite has quit IRC19:29
*** welldannit has joined #openstack-nova19:32
*** andreykurilin has joined #openstack-nova19:32
*** moshele has joined #openstack-nova19:33
*** nelsnels_ has joined #openstack-nova19:33
*** dave-mccowan has joined #openstack-nova19:33
*** markvoelker has quit IRC19:34
openstackgerritDan Smith proposed openstack/nova: Start the conversion to oslo.versionedobjects  https://review.openstack.org/17347919:34
*** nelsnelson has quit IRC19:36
mikal.19:36
*** sdake_ has joined #openstack-nova19:37
*** dave-mccowan has quit IRC19:37
mtreinishmikal: .... . .-.. .-.. ---19:37
*** salv-orlando has quit IRC19:37
*** sdake has quit IRC19:41
*** dboik_ has quit IRC19:42
moshelehi can nova cores review https://review.openstack.org/#/c/168317/ and https://review.openstack.org/#/c/168680/ ?19:43
*** obondarev_ has joined #openstack-nova19:43
*** vilobhmm11 has quit IRC19:47
dimsmtreinish: .... .- .... .-19:47
*** vilobhmm1 has joined #openstack-nova19:47
mtreinishdims: heh, it looses something in the translation :)19:49
*** mtanino has quit IRC19:49
mtreinishs/looses/loses19:49
dims:)19:49
*** obondarev has quit IRC19:49
*** jwcroppe has joined #openstack-nova19:49
vilobhmm1dansmith : ping19:49
dansmithvilobhmm1: ack19:49
vilobhmm1:)19:50
vilobhmm1i could not understand your comment on https://review.openstack.org/#/c/138607/19:50
vilobhmm1please let me know if you have any feedback would be happy to discuss19:50
vilobhmm1incorporate19:50
dansmithvilobhmm1: which comment?19:50
*** jwcroppe has quit IRC19:50
vilobhmm1Dan Smith19:51
vilobhmm19:04 AM19:51
vilobhmm1Patch Set 11: -Code-Review19:51
*** sdake has joined #openstack-nova19:51
vilobhmm1not sure what that means so thought will check with you19:51
dansmithvilobhmm1: that was me dropping my -2 now that liberty is open19:51
dansmithit's a vote of 019:51
vilobhmm1oh ok19:51
vilobhmm1cool sounds good…19:51
*** ozialien has joined #openstack-nova19:52
vilobhmm1johnthetubaguy : ping19:53
dansmithvilobhmm1: probably a little late for him.. he'll get to dropping his -2 too19:53
vilobhmm1sounds good dansmith; thanks :)19:54
*** sdake_ has quit IRC19:54
*** kmartin has joined #openstack-nova19:57
mosheledansmith: when you have time please review https://review.openstack.org/#/c/168317/19:57
vilobhmm1dansmith : same for https://review.openstack.org/#/c/138607/ when you get time please have a look19:59
*** setuid has quit IRC19:59
*** setuid has joined #openstack-nova20:01
*** achanda has quit IRC20:01
*** jdurgin has joined #openstack-nova20:01
*** garyk has quit IRC20:02
*** sdake_ has joined #openstack-nova20:05
*** sdake has quit IRC20:07
*** otter768 has joined #openstack-nova20:11
*** jgrimm is now known as zz_jgrimm20:11
*** unicell has joined #openstack-nova20:11
*** nelsnels_ has quit IRC20:12
*** ozialien has quit IRC20:12
*** nelsnelson has joined #openstack-nova20:13
*** andreykurilin has quit IRC20:13
*** oro has joined #openstack-nova20:13
*** jwcroppe has joined #openstack-nova20:14
*** dave-mccowan has joined #openstack-nova20:14
*** otter768 has quit IRC20:16
*** lucas-dinner has quit IRC20:17
*** jwcroppe has quit IRC20:17
*** annashen has joined #openstack-nova20:18
*** erkules_ is now known as erkules20:19
*** signed8bit has joined #openstack-nova20:22
*** eharney has quit IRC20:23
*** pixelb has joined #openstack-nova20:23
*** sdake has joined #openstack-nova20:23
openstackgerritAndrew Laski proposed openstack/nova: Cells: Skip initial sync of block_device_mapping  https://review.openstack.org/17350220:24
*** ijw has quit IRC20:24
*** achanda has joined #openstack-nova20:25
*** ijw has joined #openstack-nova20:25
*** sdake_ has quit IRC20:25
*** ganso_ has quit IRC20:27
*** Marga_ has quit IRC20:28
*** Hosam has joined #openstack-nova20:28
*** Nikolay_St has joined #openstack-nova20:29
*** ijw has quit IRC20:29
*** claudiub has joined #openstack-nova20:30
*** baoli has quit IRC20:32
*** ijw has joined #openstack-nova20:33
*** salv-orlando has joined #openstack-nova20:33
*** thangp has quit IRC20:33
*** markvoelker has joined #openstack-nova20:33
*** vladikr has quit IRC20:33
*** vilobhmm1 has quit IRC20:35
*** tjones1 has joined #openstack-nova20:36
*** changbl has quit IRC20:37
*** lifeless1 is now known as lifeless20:37
*** markvoelker has quit IRC20:37
*** apuimedo has joined #openstack-nova20:40
cfriesenmriedem: you around?20:42
*** moshele has quit IRC20:43
mriedemcfriesen: sort of20:43
*** neelashah has quit IRC20:43
*** dboik has joined #openstack-nova20:43
jogohave we cut the last release of novaclient for Kilo?20:44
jogoif so we should get ready to create the stable/kilo branch of novaclient20:44
*** ivasev has joined #openstack-nova20:45
mriedemjogo: this came up last week https://review.openstack.org/#/c/165932/20:46
mriedemi'm not sure if that needs a new bug fix release though20:47
*** ozialien has joined #openstack-nova20:47
mriedemit depends on this nova change https://review.openstack.org/#/c/166401/20:47
mriedemso it's a bit wonky20:47
*** apuimedo has quit IRC20:47
*** dboik has quit IRC20:49
jogomriedem: we could release a bug fix for it but not bump the minimum20:49
*** dboik has joined #openstack-nova20:49
jogobut we should create the stable/kilo novaclient branch around the same time a stable branch shows up in g-r20:50
cfriesenmriedem: I've been digging into the evacuate issues with another guy here.   in libvirt/driver.py in _create_domain_and_network() we call self._get_neutron_events() to figure out what events to wait for.  In the successful evacuates the events list is empty, while in the unsuccessful ones the event list is not empty.20:50
jogodhellmann: ^^ does that sound about right?20:50
mriedemjogo: i haven't kept up to speed on the stable branches for the clients20:50
mriedemnot sure what those are for20:50
cfriesenmriedem: in the unsuccessful case nova sits waiting for the events, but because the new host doesn't match the binding neutron will never send it the event20:51
mriedemare we going to release bug fix releases off the stable branches for the clients?20:51
*** sbfox has quit IRC20:51
*** dboik has quit IRC20:51
mriedemcfriesen: hmmm, and then we timeout after 5 minutes and fail the build20:52
*** dboik has joined #openstack-nova20:52
jogomriedem: maybe, we want to keep the option open to do so20:52
mriedemcfriesen: i'd have to dig into what's populating the event list20:52
cfriesenmriedem: in the successful cases the binding is still wrong in neutron, but because we don't wait for it it doesn't cause problems20:52
mriedemcfriesen: right20:52
jogomriedem: without a massive reqs change20:52
mriedemjogo: yeah, i see the appearl20:52
mriedem*appeal20:52
cfriesenmriedem: so interestingly, most of the time with 2 compute nodes you can evac back and forth okay because in one direction we don't wait and in the other direction the binding is correct.20:53
*** hdd has joined #openstack-nova20:53
dansmithcfriesen: is neutron at fault or are we?20:53
jogomikal: ^ stable/kilo for novaclient20:53
mikaljogo: in the tc meeting, please hold20:53
cfriesendansmith: I think we are...we need to call  _update_port_binding_for_instance() during the evacuate I think, same as we do for live migration20:53
dansmithmeaning neutron needs to be told that the host is changing or something?20:54
cfriesendansmith: yes20:54
dansmithso it knows the host is different and fires the event?20:54
dansmithokay20:54
cfriesendansmith: the guy I was working with has done more neutron work...he said something about neutron ignoring events where the host doesn't match what it expects from the binding20:55
dansmiths/events/requests, I assume you mean20:56
mriedemcfriesen: dansmith: yeah, we definitely aren't updating the port binding in the neutronv2 api code in the evacuate flow20:56
dansmithso wait20:56
dansmithWAIT20:56
dansmithevacuate is broken?20:56
mriedemha20:56
cfriesen:)20:56
mriedemsurprise!20:56
dansmithbroken-er20:56
* dansmith gets out the torch20:56
*** signed8bit is now known as signed8bit_ZZZzz20:57
mriedemcfriesen: so can you play with calling network_api.setup_instance_network_on_host in evacuate?20:57
*** dkliban is now known as dkliban_afk20:57
cfriesendansmith: mriedem: so currently even on a "successful" evacuate, nova shows the ports in the BUILD status20:57
cfriesenwhoops, neutron20:58
*** raildo has quit IRC20:58
cfriesendansmith: mriedem: that's for the ones where nova isn't waiting but the host doesn't match the port binding20:58
cfriesenmriedem: do we want to call that or just _update_port_binding_for_instance()?20:59
*** signed8bit_ZZZzz is now known as signed8bit20:59
cfriesenmriedem: oh wait, it's basically the same thing.20:59
cfriesenmriedem: will give it a shot21:00
*** ozialien has left #openstack-nova21:01
*** jecarey has quit IRC21:02
*** vilobhmm1 has joined #openstack-nova21:02
*** vilobhmm1 has quit IRC21:03
*** vilobhmm1 has joined #openstack-nova21:03
*** hdd has quit IRC21:04
*** obedmr has left #openstack-nova21:06
*** vilobhmm1 has quit IRC21:07
*** vilobhmm1 has joined #openstack-nova21:08
*** aysyd has quit IRC21:08
mikaljogo: ok, so hello. What are we talking about?21:10
mriedemcfriesen: yeah, you have to call the public network api, which calls the internal method21:10
*** marun has quit IRC21:13
dansmithmikal: are we going to have a conversation about the rc2 bugs?21:14
*** annegentle has quit IRC21:14
dansmithmikal: the list is pretty long and probably some of them can be early backports, IMHO21:14
dansmithbut there are some that are really critical and I don't want those to get missed21:14
*** dprince has quit IRC21:14
*** apuimedo has joined #openstack-nova21:14
*** ivasev has quit IRC21:15
jogomikal: we should get  cut a stable/juno novaclient branch21:16
jogoso that if we need to backport a bug fix to novaclient version in stable/kilo we can21:17
mikaldansmith: I am not allowed to say that there will be a rc2, but I think its very very likely21:17
mikaldansmith: so, things which are ready in that list get into rc2 I imagine21:17
dansmithmikal: right, I meant, you know, hypothetically, if we had one21:17
dansmithmikal: the list seems too long for an rc2, is my point21:17
mikaldansmith: if it shouldn't be in rc2, we should untag it and just work it through the nromal stabel backport process21:17
mikalstable even21:18
mikaldansmith: yeah, I agree21:18
dansmithright, so what I'm saying is, are we going to run through that list and decide what should be dropped?21:18
mikaldansmith: I think we dal with that as tag hygiene21:18
mikaldansmith: yes, but we can't do that until we "decide" to do an rc221:18
*** oro has quit IRC21:18
mikalSo, next week ish?21:18
dansmithokay, just want to make sure we do that review, if and when we do an rc221:19
mikaldansmith: its a fair point. Want to reply to that effect to my email so we have a written reminder?21:20
mikaljogo: I am unsure how to do that thing? How do we make branches like that?21:20
mikaljogo: not opposed, just unsure of the process21:20
dansmithokay21:20
*** pixelb has quit IRC21:20
*** annashen has quit IRC21:21
*** annashen has joined #openstack-nova21:22
jogomikal: same process as cutting a branch on nova21:22
alaskilooking for a third perspective on https://review.openstack.org/#/c/171299/ to tell me if I'm overthinking it21:22
jogomikal: https://review.openstack.org/#/admin/projects/openstack/python-novaclient,branches21:23
mikaljogo: how are they made? Some config file in infra land somewhere?21:24
*** WannabeCloudHost has quit IRC21:24
*** jecarey has joined #openstack-nova21:26
*** cbader has quit IRC21:30
jogomikal: no, you just pass in a sha1 and a name21:31
mikaljogo: so, the gerrit UI is read only for me21:31
mikaljogo: I don't see a place to actually make the branch?21:31
* mikal has asked infra for a hint21:31
jogohttps://wiki.openstack.org/wiki/ReleaseTeam/How_To_Release21:31
jogomikal: its self service21:31
jogomikal: you may not have the power to make a branch21:32
mikalHmmmm, fungi recons the release managers might be helping do those21:33
mikalI will ping ttx and ask21:34
*** tonytan4ever has quit IRC21:34
*** busterswt has quit IRC21:35
*** boris-42 has joined #openstack-nova21:39
*** annashen has quit IRC21:40
*** jwcroppe has joined #openstack-nova21:42
mikaljogo: ttx says he and dhellmann are going to do all the stable branches tomorrow21:42
* mikal wanders off to get out of pjs21:43
jogomikal: cool that works21:44
jogomikal: thanks for sharing, 'the more you know' :)21:44
mikaljogo: well, I have been here for your entertainment since 5:30am my time21:44
mikal:P21:44
jogowow21:45
*** signed8bit has quit IRC21:46
*** tpatil has quit IRC21:47
bauzasalaski: eh you beated me by https://review.openstack.org/#/c/173513/1 :)21:47
alaskibauzas: heh.  I'm looking at the disk_config failure again so I haven't opened one for that and the idempotent failure21:48
cfriesenmreidem: dansmith: in compute.manager.ComputeManager.rebuild_instance I added a call to self.network_api.setup_instance_network_on_host() right below the existing call to self.network_api.setup_networks_on_host().  That seems to fix the issue with the neutron binding:host_id.  I wonder if it would make sense to just have neutron's setup_networks_on_host() (which is currently empty) call self._update_port_binding_for_instance()21:48
cfriesen instead?21:48
*** apuimedo has quit IRC21:49
*** mwagner_lap has quit IRC21:50
cfriesenhmm...setup_networks_on_host() gets called from a bunch of places21:50
bauzasalaski: I'm rebooting my cells devstack to see if adding a new private field for the Instance object could help the race issue21:50
bauzasalaski: because we're passing the same object down to the child cell21:51
*** annegentle has joined #openstack-nova21:51
cfriesenwhy do we have setup_networks_on_host() (which also does teardown), setup_instance_network_on_host(), and cleanup_instance_network_on_host()?21:51
alaskibauzas: great21:52
bauzasalaski: yeah, instance_update_from_api() is called in Instance.save()21:52
bauzasthen it goes to the messaging module, and then do an instance.save() too21:53
bauzasalaski: as it's now on the child cell, that's why it goes back up21:53
alaskibauzas: yeah.  we need something like 'with dont_sync_cells: instance.save()'21:53
alaskijust for when it's called in instance_update_from_api21:54
alaskiand later when we add it to instance_update_at_top21:54
*** mriedem is now known as mriedem_away21:56
*** asettle has joined #openstack-nova21:56
cfriesenAny chance of getting another core to look at https://review.openstack.org/#/c/162746/ ?  It's already got a bunch of +1s and a +2.21:58
*** mriedem_away has quit IRC22:00
bauzasalaski: you mean in the save method ?22:01
*** marun has joined #openstack-nova22:01
alaskibauzas: something needs to tell save that it shouldn't call instance_update_at|from_top|api22:03
bauzasalaski: yeah, that's why I think we need to leave an indirection field22:04
bauzasalaski: anyway, I will work on it by tomorrow22:05
*** isd has quit IRC22:05
*** thomasem has quit IRC22:05
jogodansmith: looking over https://bugs.launchpad.net/nova/+bugs?field.tag=kilo-rc-potential22:07
jogoand there are a e few we can remove I think22:08
bauzasjogo: yeah I have one I can remove22:08
jogoalaski: I was thinking we can remove https://bugs.launchpad.net/nova/+bug/140426822:08
openstackLaunchpad bug 1404268 in OpenStack Compute (nova) "Missing nova context during spawn" [High,In progress] - Assigned to Andrew Laski (alaski)22:08
*** burt has quit IRC22:08
jogobauzas: link?22:08
bauzasjogo: just did https://bugs.launchpad.net/nova/+bug/143129122:09
openstackLaunchpad bug 1431291 in OpenStack Compute (nova) "Scheduler Failures are no longer logged with enough detail for a site admin to do problem determination" [High,In progress] - Assigned to Sylvain Bauza (sylvain-bauza)22:09
alaskijogo: I think that one is worth having.  logs are problematic atm on computes22:09
jogoalaski: no one is working on it22:09
jogoalaski: we can backport right away22:09
jogooh are you working on it now?22:10
alaskijogo: I have a review open for it22:10
*** markvoelker has joined #openstack-nova22:10
jogoalaski: that changes things22:10
jogolink?22:10
alaskijogo: https://review.openstack.org/#/c/171299/22:10
*** armax has quit IRC22:10
*** markvoelker_ has joined #openstack-nova22:10
*** ociuhandu has joined #openstack-nova22:11
jogoalaski: cool, I'll let you and dims hash that one out22:11
*** otter768 has joined #openstack-nova22:12
jogoalaski: when your ready for a review poke me as getting that in would be great22:12
jogojust don't think we should block on it22:12
*** annegentle has quit IRC22:12
alaskijogo: cool.  if no one speaks up by tomorrow I'll go with dims suggestion22:12
alaskijogo: yeah, don't want to block.  but if we do have an rc2 I think it's a good one for it22:12
jogoalaski: agreed22:13
jogolooks like https://review.openstack.org/#/c/172856/ is the only critical for RC2 that is still pending22:14
*** markvoelker has quit IRC22:14
jogoalaski dansmith: https://review.openstack.org/#/c/172856/22:14
*** otter768 has quit IRC22:16
*** artom has quit IRC22:17
*** annashen has joined #openstack-nova22:18
*** nelsnelson has quit IRC22:19
*** Marga_ has joined #openstack-nova22:21
jogosdague: https://review.openstack.org/#/c/163700/ is ready to be un-2d22:23
*** VW_ has quit IRC22:25
bauzasgrrr, call me stupid but I can't find where the host field is set on the Instance object within build_and_run_instance(), anyone remembers ?22:26
dansmithit's in there22:27
*** jecarey has quit IRC22:27
bauzasI can find where it's set for all the other cases (shelve, migrate, etc.)22:27
bauzasdansmith: then I'm blind22:27
cfriesendansmith: I just opened a bug on the evacuate issue: https://bugs.launchpad.net/nova/+bug/144417122:27
openstackLaunchpad bug 1444171 in OpenStack Compute (nova) "evacuate code path is not updating binding:host_id in neutron" [Undecided,New]22:27
bauzasdansmith: that's maybe because I saw Netflix's Daredevil...22:27
*** irenab has joined #openstack-nova22:28
dansmithbauzas: resource_tracker.py, _set_instance_host_and_node()22:29
dansmithcfriesen: cool, thanks22:29
bauzasdansmith: oh, when claiming ? damn.22:29
cfriesendansmith: I found a hack that'll work for neutron, but I think it might break nova-network.22:29
bauzasdansmith: I was actually reviewing https://review.openstack.org/#/c/163842/10/nova/compute/resource_tracker.py,cm but I didn't like to see the AZ info provided in there as I was thinking it was just done elsewhere22:30
dansmithcfriesen: does this apply at all to nova-network?22:30
cfriesendansmith: and there's weird stuff around setup_networks_on_host(), setup_instance_network_on_host(), and cleanup_instance_network_on_host() being used on one or the other or neithr22:30
bauzasdansmith: but I'll need to change my mind then22:30
cfriesendansmith: I have no idea22:30
cfriesendansmith: but we are already calling setup_networks_on_host(), which actually does something for nova-network and is empty for neutron22:31
bauzaserm, for some reason, we do that when claiming() for the build_and_run_instance() but not when migrating like in _prep_resize() for example22:31
*** eglynn has joined #openstack-nova22:31
dansmithokay, I really don't have context on all that, so I can't comment without looking22:31
cfriesendansmith: setup_instance_network_on_host() appears to do something for neutron but goes all the way to network-manager and then does nothing for nova-network22:31
bauzasthat's just... ugly :/22:31
*** ijw has quit IRC22:32
*** Sukhdev has joined #openstack-nova22:32
cfriesenbauzas: there's an open bug about claiming resources when migrating22:33
bauzascfriesen: I know22:33
bauzascfriesen: the claiming system is just... erm, ugly ?22:34
bauzascfriesen: fixing the open bug you mentioned is just a can of worms22:34
cfriesenbauzas: indeed.  and then you combine it with having to allocate specific CPUs when you live-migrate an instance with dedicated CPUs...22:34
*** smccully has quit IRC22:35
cfriesenbauzas: also, we're bad about freeing up resources on live migration...it waits around till the audit.  So you could live-migrate and then try to live-migrate back and it'll tell you that there are insufficient resources to go back to where you came from.22:36
*** markvoelker has joined #openstack-nova22:36
*** achanda has quit IRC22:36
*** thedodd has quit IRC22:36
bauzascfriesen: I know that too, that's why I say it's a can of worms22:36
*** markvoelker_ has quit IRC22:37
bauzascfriesen: instead of bugfixing stuff with a huge tech debt, we have to refactor it22:37
bauzascfriesen: that's something we discussed 6 months ago and that's still open - but basically claims should be a scheduler thing22:37
bauzascfriesen: unless the operator wants to directly go to a specific compute by giving a dest, and then that's his responsibility22:38
openstackgerritMitsuhiro Tanino proposed openstack/nova: Prevent unintentional detaching volume from instance  https://review.openstack.org/16669522:38
*** markvoelker_ has joined #openstack-nova22:41
*** achanda has joined #openstack-nova22:41
*** ociuhandu has quit IRC22:42
*** markvoelker has quit IRC22:44
*** eglynn has quit IRC22:44
*** tjones1 has quit IRC22:47
*** armax has joined #openstack-nova22:49
*** tjones1 has joined #openstack-nova22:49
*** Marga_ has quit IRC22:49
*** dave-mccowan has quit IRC22:54
*** sbfox has joined #openstack-nova23:06
*** sbfox has quit IRC23:06
*** dboik has quit IRC23:06
*** mwagner_lap has joined #openstack-nova23:06
*** sbfox has joined #openstack-nova23:06
*** mriedem has joined #openstack-nova23:07
*** Marga_ has joined #openstack-nova23:07
*** eglynn has joined #openstack-nova23:10
*** zzzeek has quit IRC23:10
*** dave-mccowan has joined #openstack-nova23:10
*** ijw has joined #openstack-nova23:11
*** Sukhdev has quit IRC23:13
*** mriedem has quit IRC23:13
*** Sukhdev has joined #openstack-nova23:13
*** nelsnelson has joined #openstack-nova23:14
*** dave-mccowan has quit IRC23:15
*** ociuhandu has joined #openstack-nova23:16
*** mriedem has joined #openstack-nova23:17
mriedemcfriesen: just saw your question, i'm not sure why we have two similar network setup on host methods, would have to dig into the history and usage of each. i assume cleanup is post-migration.23:18
*** dave-mccowan has joined #openstack-nova23:19
*** Mike_D_laptop has quit IRC23:20
cfriesenmriedem: I dug  a bit more.  looks like all three functions actually do something for FlatDHCPManager/VlanManager due to the FloatingIP class mixin23:20
cfriesenmriedem: for neutron only setup_instance_network_on_host() actually does anything23:21
*** Hosam has quit IRC23:21
mriedemcfriesen: yeah i figured there was a bunch of nova-network-isms in here23:21
mriedemalex_xu: had some patches in kilo to normalize some of that, not sure if it all landed though23:21
cfriesenIn most cases where we call setup_networks_on_host() we also call migrate_instance_finish(), which on neutron will also end up calling _update_port_binding_for_instance23:22
cfriesenmriedem: but it looks like we missed the evacuate case.23:22
cfriesenmriedem: not sure how to make evacuate work sanely for both neutron and nova-network23:22
cfriesenmriedem: I opened https://bugs.launchpad.net/nova/+bug/1444171 in case you want to point the appropriate people at it.23:23
openstackLaunchpad bug 1444171 in OpenStack Compute (nova) "evacuate code path is not updating binding:host_id in neutron" [Undecided,New]23:23
mriedemcfriesen: i think you're the appropriate person :)23:24
*** irenab has quit IRC23:24
cfriesenmriedem: gee, thanks!23:24
mriedemcfriesen: so where is setup_instance_network_on_host() used?23:24
*** salv-orlando has quit IRC23:24
cfriesenmriedem: _unshelve_instance() and _build_networks_for_instance()23:25
cfriesenand _build_networks_for_instance() is used only from _build_resources()23:25
mriedemso it's the build path rather than migrate23:26
mriedemthe other is migrate only it sounds like, for nova-network23:26
cfriesenmigrate_instance_finish() takes migration as an argument...for neutron it ends up calling down into the same underlying _update_port_binding_for_instance()23:27
cfriesenmriedem: could do a really hacky thing and only call setup_instance_network_on_host() in rebuild_instance if we're using neutron.23:28
mriedemcfriesen: yeah, thought about that23:28
mriedemsuper hack23:28
*** jwcroppe has quit IRC23:28
mriedemor just have setup_network_on_host() in neutron call the _update_port_binding23:28
cfriesenmriedem: setup_network_on_host() gets called all over the place though23:30
cfriesenmriedem: wasn't sure about side effects in neutron23:31
mriedemwell, _update_port_binding only changes the host if the host changes23:32
mriedembut yeah i hear you23:32
mriedemcfriesen: you could do a WIP patch and just see if the neutron full job falls over23:33
*** Sukhdev has quit IRC23:35
cfriesenmriedem: sure...may be able to have setup_network_on_host() call _update_port_binding_for_instance().  Then if we're lucky when we call migrate_instance_finish() it'll get called again for neutron and decide it has nothing to do.23:35
mriedemyeah23:36
mriedemwe definitely dno't want the compute manager to have to know that the neutronv2 network api has to be called differently for rebuild23:38
cfriesenmriedem: will try that.  gotta run it by someone first unfortunately.  maybe late tonight or sometime tomorrow.23:40
*** READ10 has quit IRC23:43
*** tjones1 has quit IRC23:44
*** Sukhdev has joined #openstack-nova23:46
mriedemcfriesen: ok, i might be able to wip that up quick23:47
mriedemthat was a little pun23:47
*** annashen has quit IRC23:49
cfriesen:)23:49
cfriesenmriedem: sorry, corporate policy stuff23:49
*** zhenguo has joined #openstack-nova23:51
*** markvoelker_ has quit IRC23:54
*** tjones1 has joined #openstack-nova23:55
cfriesenmriedem: you send in that patch yet?  If not I got the okay to send it in.23:57
*** mriedem1 has joined #openstack-nova23:59
mriedem1cfriesen: that was fast - go nuts23:59
cfriesenmriedem1: got lucky, he was online from home23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!