Monday, 2017-06-05

*** thorst has joined #openstack-nova00:02
openstackgerritTakashi NATSUME proposed openstack/nova master: List/show all server migration types (1/2)  https://review.openstack.org/43060800:07
*** dimtruck is now known as zz_dimtruck00:14
*** zz_dimtruck is now known as dimtruck00:14
*** thorst has quit IRC00:18
*** gouthamr has quit IRC00:21
*** yamamoto has joined #openstack-nova00:23
*** dikonoor has joined #openstack-nova00:25
*** jamesdenton has joined #openstack-nova00:26
*** yamamoto has quit IRC00:30
*** markvoelker has quit IRC00:31
*** tovin07_ has joined #openstack-nova00:35
openstackgerritTakashi NATSUME proposed openstack/nova master: Enable cold migration with target host(2/2)  https://review.openstack.org/40896400:36
*** gomarivera has joined #openstack-nova00:37
*** Shunli has joined #openstack-nova00:38
*** trungnv has quit IRC00:39
*** mdnadeem has joined #openstack-nova00:39
openstackgerritTakashi NATSUME proposed openstack/nova master: Add functional tests for cold migration to same host  https://review.openstack.org/41492600:41
openstackgerritTakashi NATSUME proposed openstack/nova master: Add functional tests for cold migration to same host  https://review.openstack.org/41492600:41
*** yamamoto has joined #openstack-nova00:43
*** gomarivera has quit IRC00:44
openstackgerritTakashi NATSUME proposed openstack/nova master: api-ref: Add parameters in cold migrate action  https://review.openstack.org/41004200:45
openstackgerritTakashi NATSUME proposed openstack/nova master: api-ref: Add parameters in cold migrate action  https://review.openstack.org/41004200:45
*** hongbin has quit IRC00:45
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix missing marker functions  https://review.openstack.org/46992000:45
openstackgerritTakashi NATSUME proposed openstack/nova master: Refactor a test method including 7 test cases  https://review.openstack.org/39543600:46
*** dikonoor has quit IRC00:46
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove unused exceptions  https://review.openstack.org/46452500:46
*** trungnv has joined #openstack-nova00:46
openstackgerritTakashi NATSUME proposed openstack/nova master: Use PCIAddressField in oslo.versionedobjects  https://review.openstack.org/46620100:46
*** jdwidari has quit IRC00:48
*** phuongnh has joined #openstack-nova01:03
*** liusheng has joined #openstack-nova01:10
*** thorst has joined #openstack-nova01:14
*** gcb has joined #openstack-nova01:19
*** thorst has quit IRC01:19
openstackgerritTakashi NATSUME proposed openstack/nova master: List/show all server migration types (2/2)  https://review.openstack.org/45948301:20
*** jamesdenton has quit IRC01:29
*** gomarivera has joined #openstack-nova01:31
*** gomarivera has quit IRC01:32
*** zhurong has joined #openstack-nova01:33
*** Apoorva has joined #openstack-nova01:37
*** jichen has joined #openstack-nova01:40
*** edmondsw has joined #openstack-nova01:41
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Microversion 2.48 - Enable cold migration with target host  https://review.openstack.org/40670701:42
*** zhurong has quit IRC01:44
*** edmondsw has quit IRC01:45
*** baoli has joined #openstack-nova01:46
*** vladikr has joined #openstack-nova01:52
*** gongysh has joined #openstack-nova01:52
openstackgerritTakashi NATSUME proposed openstack/nova master: Specify keymap on server boot (1/2)  https://review.openstack.org/45875201:53
*** tovin07_ has quit IRC01:57
*** thorst has joined #openstack-nova01:59
*** liujiong has joined #openstack-nova01:59
*** Apoorva has quit IRC01:59
openstackgerritJianghua Wang proposed openstack/nova master: DNM: test RFP failure  https://review.openstack.org/47079802:04
*** armax has joined #openstack-nova02:04
*** tovin07_ has joined #openstack-nova02:05
*** jwnx has joined #openstack-nova02:06
*** yassine has joined #openstack-nova02:12
*** yassine is now known as Guest7765102:12
*** crushil has joined #openstack-nova02:15
*** baoli has quit IRC02:20
*** thorst has quit IRC02:26
*** litao__ has joined #openstack-nova02:27
*** markvoelker has joined #openstack-nova02:28
*** baoli has joined #openstack-nova02:28
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: Specify keymap on server boot  https://review.openstack.org/44883002:29
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: Abort Cold Migration  https://review.openstack.org/33473202:29
*** zzzeek has joined #openstack-nova02:30
*** zhurong has joined #openstack-nova02:32
*** hongbin has joined #openstack-nova02:34
*** baoli has quit IRC02:40
*** gomarivera has joined #openstack-nova02:44
*** gomarivera has quit IRC02:46
*** gomarivera has joined #openstack-nova02:46
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Microversion 2.48 - List/Show all server migration types  https://review.openstack.org/43083902:49
*** gomarivera has quit IRC02:51
*** tiendc has joined #openstack-nova02:52
*** klkumar has joined #openstack-nova02:54
*** markvoelker has quit IRC03:02
*** gomarivera has joined #openstack-nova03:10
openstackgerritJay Pipes proposed openstack/nova master: placement project_id, user_id in PUT /allocations  https://review.openstack.org/46963403:11
openstackgerritJay Pipes proposed openstack/nova master: placement: Add GET /usages to placement API  https://review.openstack.org/46963503:11
openstackgerritJay Pipes proposed openstack/nova master: placement: use separate tables for projects/users  https://review.openstack.org/47064503:11
*** gouthamr has joined #openstack-nova03:19
*** Yangyapeng has joined #openstack-nova03:24
*** thorst has joined #openstack-nova03:26
openstackgerritZhenyu Zheng proposed openstack/nova master: Send out notifications when instance tags changed  https://review.openstack.org/45992303:28
*** dimtruck is now known as zz_dimtruck03:31
*** thorst has quit IRC03:31
openstackgerritTakashi NATSUME proposed openstack/nova master: Specify keymap on server boot (2/2)  https://review.openstack.org/46098703:32
*** thorst has joined #openstack-nova03:33
*** mamandle has joined #openstack-nova03:33
*** mingyu has joined #openstack-nova03:34
*** thorst has quit IRC03:35
*** klkumar has quit IRC03:36
*** hongbin has quit IRC03:37
*** psachin has joined #openstack-nova03:42
*** armax has quit IRC03:42
*** armax has joined #openstack-nova03:43
*** armax has quit IRC03:43
*** armax has joined #openstack-nova03:44
*** armax has quit IRC03:44
*** armax has joined #openstack-nova03:44
*** armax has quit IRC03:45
*** gomarivera has quit IRC03:45
*** armax has joined #openstack-nova03:45
*** armax has quit IRC03:46
*** armax has joined #openstack-nova03:46
openstackgerritMerged openstack/python-novaclient master: 2.46: match nova API version  https://review.openstack.org/47041303:46
*** armax has quit IRC03:46
*** vks1 has joined #openstack-nova03:50
*** klkumar has joined #openstack-nova03:51
*** markvoelker has joined #openstack-nova03:58
*** mamandle has quit IRC04:02
*** liusheng has quit IRC04:02
*** Dinesh_Bhor has joined #openstack-nova04:03
openstackgerritJianghua Wang proposed openstack/nova master: DNM: test RFP failure  https://review.openstack.org/47079804:04
*** liusheng has joined #openstack-nova04:04
*** thorst has joined #openstack-nova04:05
*** mingyu has quit IRC04:10
*** mingyu has joined #openstack-nova04:11
*** gouthamr has quit IRC04:12
*** dikonoor has joined #openstack-nova04:13
*** jwnx has quit IRC04:15
*** mingyu has quit IRC04:15
*** thorst has quit IRC04:22
*** bhagyashris has joined #openstack-nova04:24
*** markvoelker has quit IRC04:32
*** sridharg has joined #openstack-nova04:39
*** tuanluong has joined #openstack-nova04:39
*** dikonoor has quit IRC04:47
*** trinaths has joined #openstack-nova04:49
*** mingyu has joined #openstack-nova05:02
*** sree has joined #openstack-nova05:02
*** faizy has joined #openstack-nova05:07
*** salv-orlando has joined #openstack-nova05:08
*** claudiub|2 has joined #openstack-nova05:10
openstackgerritClaudiu Belu proposed openstack/nova master: POC: tests: autospecs all the mock.patch usages  https://review.openstack.org/47077505:10
*** beekneemech has quit IRC05:11
*** Fdaisuke has quit IRC05:11
openstackgerritTakashi NATSUME proposed openstack/nova master: Specify keymap on server boot (2/2)  https://review.openstack.org/46098705:12
*** hshiina has joined #openstack-nova05:12
*** huanxie has joined #openstack-nova05:12
*** udesale has joined #openstack-nova05:12
*** huanxie has quit IRC05:16
*** edmondsw has joined #openstack-nova05:17
*** mingyu has quit IRC05:19
*** dikonoor has joined #openstack-nova05:19
*** edmondsw has quit IRC05:21
*** prateek has joined #openstack-nova05:23
*** huanxie has joined #openstack-nova05:27
*** yamamoto has quit IRC05:28
*** huanxie has quit IRC05:28
*** klkumar has quit IRC05:31
*** huanxie has joined #openstack-nova05:35
*** gomarivera has joined #openstack-nova05:36
*** huanxie has quit IRC05:37
*** dikonoor has quit IRC05:38
*** huanxie has joined #openstack-nova05:39
*** huanxie has quit IRC05:41
*** liusheng has quit IRC05:41
*** liusheng has joined #openstack-nova05:43
*** kwathore has joined #openstack-nova05:49
*** klkumar has joined #openstack-nova05:52
*** dikonoor has joined #openstack-nova05:59
*** mamandle has joined #openstack-nova06:03
*** brault has joined #openstack-nova06:04
*** salv-orlando has quit IRC06:05
*** brault has quit IRC06:08
*** mingyu has joined #openstack-nova06:08
*** Oku_OS-away is now known as Oku_OS06:10
*** yamamoto has joined #openstack-nova06:13
*** mingyu has quit IRC06:14
*** mingyu has joined #openstack-nova06:17
*** ekuris has joined #openstack-nova06:17
*** thorst has joined #openstack-nova06:18
*** sgordon has quit IRC06:20
*** thorst has quit IRC06:23
*** markvoelker has joined #openstack-nova06:28
*** rcernin has joined #openstack-nova06:29
*** klkumar has quit IRC06:33
*** ltomasbo|away is now known as ltomasbo06:36
*** brault has joined #openstack-nova06:37
openstackgerritfalseuser proposed openstack/nova master: Clear instance root_device_name in rebuild operation  https://review.openstack.org/46499106:47
openstackgerritJianghua Wang proposed openstack/nova master: DNM: test RFP failure  https://review.openstack.org/47079806:48
*** claudiub has joined #openstack-nova06:49
*** dikonoo has joined #openstack-nova06:49
*** dikonoor has quit IRC06:50
*** claudiub|2 has quit IRC06:50
*** tuanluong has quit IRC06:50
*** bhagyashris has quit IRC06:50
*** zzzeek has quit IRC06:50
*** litao__ has quit IRC06:50
*** crushil has quit IRC06:50
*** vladikr has quit IRC06:50
*** liujiong has quit IRC06:50
*** trungnv has quit IRC06:50
*** hemna has quit IRC06:50
*** McNinja has quit IRC06:50
*** hoonetorg has quit IRC06:50
*** carl_baldwin has quit IRC06:50
*** vdrok has quit IRC06:50
*** samueldmq has quit IRC06:50
*** izaakk has quit IRC06:50
*** patrickeast has quit IRC06:50
*** TheJulia has quit IRC06:50
*** sballe_ has quit IRC06:50
*** ameade has quit IRC06:50
*** johnsom has quit IRC06:50
*** fyxim has quit IRC06:50
*** xgerman has quit IRC06:50
*** bhagyashri_s has joined #openstack-nova06:50
*** xgerman_ has joined #openstack-nova06:50
*** vladikr_ has joined #openstack-nova06:50
*** trungnv has joined #openstack-nova06:50
*** carl_baldwin has joined #openstack-nova06:50
*** izaakk has joined #openstack-nova06:50
*** fyxim has joined #openstack-nova06:50
*** tuanluong has joined #openstack-nova06:50
*** samueldmq has joined #openstack-nova06:50
*** mingyu has quit IRC06:50
*** hoonetorg has joined #openstack-nova06:50
*** crushil has joined #openstack-nova06:50
*** litao__ has joined #openstack-nova06:50
*** vdrok has joined #openstack-nova06:51
*** TheJulia has joined #openstack-nova06:51
*** sballe_ has joined #openstack-nova06:51
*** patrickeast has joined #openstack-nova06:51
*** johnsom has joined #openstack-nova06:51
*** ameade has joined #openstack-nova06:51
*** liujiong has joined #openstack-nova06:51
*** takashin has quit IRC06:52
*** siva_krishnan has quit IRC06:52
*** takashin has joined #openstack-nova06:53
*** McNinja has joined #openstack-nova06:53
*** hemna has joined #openstack-nova06:53
*** zzzeek has joined #openstack-nova06:54
*** siva_krishnan has joined #openstack-nova06:55
*** takashin has left #openstack-nova07:00
*** mingyu has joined #openstack-nova07:02
*** markvoelker has quit IRC07:02
*** salv-orlando has joined #openstack-nova07:04
*** pcaruana has joined #openstack-nova07:04
*** edmondsw has joined #openstack-nova07:05
*** klkumar has joined #openstack-nova07:07
*** edmondsw has quit IRC07:09
*** brault has quit IRC07:11
*** thorst has joined #openstack-nova07:20
*** thorst has quit IRC07:24
*** salv-orl_ has joined #openstack-nova07:28
*** danpawlik has joined #openstack-nova07:30
*** brault has joined #openstack-nova07:30
gmannsneti_:07:31
*** brault has quit IRC07:31
gmannsneti_: ignore please.07:31
*** salv-orlando has quit IRC07:31
*** brault has joined #openstack-nova07:31
*** zhurong has quit IRC07:31
*** brault_ has joined #openstack-nova07:33
*** brault has quit IRC07:33
*** Hosam has joined #openstack-nova07:33
*** jpena|off is now known as jpena07:36
*** mingyu has quit IRC07:38
*** oanson has joined #openstack-nova07:39
*** zhurong has joined #openstack-nova07:39
openstackgerritSergey Nikitin proposed openstack/nova master: Standardization of VM diagnostics info API.  https://review.openstack.org/35554007:46
*** oanson has quit IRC07:48
*** gomarivera has quit IRC07:48
*** oanson has joined #openstack-nova07:48
*** gomarivera has joined #openstack-nova07:51
*** mingyu has joined #openstack-nova07:52
*** oanson has quit IRC07:52
*** bkopilov has quit IRC07:53
*** gomarivera has quit IRC07:53
*** gomarivera has joined #openstack-nova07:54
*** aloga has quit IRC07:55
*** aloga has joined #openstack-nova07:56
*** markvoelker has joined #openstack-nova07:59
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-nova08:00
*** bkopilov has joined #openstack-nova08:02
Kevin_Zhengping Matt08:04
Kevin_Zhengping mriedem08:04
*** salv-orl_ has quit IRC08:07
*** salv-orlando has joined #openstack-nova08:10
*** mingyu has quit IRC08:18
*** jaosorior has joined #openstack-nova08:20
*** zhurong has quit IRC08:20
*** thorst has joined #openstack-nova08:20
*** mingyu has joined #openstack-nova08:23
*** gszasz has joined #openstack-nova08:24
openstackgerritJianghua Wang proposed openstack/nova master: Fix the race condition with novnc  https://review.openstack.org/47079808:24
*** dikonoo has quit IRC08:29
*** dikonoo has joined #openstack-nova08:30
*** karthiks has joined #openstack-nova08:31
*** markvoelker has quit IRC08:32
*** karimb has joined #openstack-nova08:35
openstackgerritDirk Mueller proposed openstack/nova-specs master: Fix citation references (Sphinx 1.6.x compatibility)  https://review.openstack.org/47025208:39
*** thorst has quit IRC08:40
*** Hosam_ has joined #openstack-nova08:41
*** gszasz has quit IRC08:43
*** gszasz has joined #openstack-nova08:43
*** mingyu has quit IRC08:44
*** Hosam has quit IRC08:45
*** mingyu has joined #openstack-nova08:45
*** mingyu has quit IRC08:47
*** Yangyapeng has quit IRC08:47
*** zhurong has joined #openstack-nova08:50
*** salv-orlando has quit IRC08:50
*** salv-orlando has joined #openstack-nova08:51
*** nicolasbock has joined #openstack-nova08:52
*** edmondsw has joined #openstack-nova08:53
*** claudiub has quit IRC08:54
*** winston-d_ has joined #openstack-nova08:55
*** kornicameister has quit IRC08:56
openstackgerritMerged openstack/nova master: Remove unused exceptions  https://review.openstack.org/46452508:57
*** edmondsw has quit IRC08:57
*** kornicameister has joined #openstack-nova09:03
openstackgerritMerged openstack/nova master: Fix missing marker functions  https://review.openstack.org/46992009:03
*** faizy_ has joined #openstack-nova09:08
*** faizy has quit IRC09:09
*** sambetts|afk is now known as sambetts09:13
*** salv-orlando has quit IRC09:13
*** salv-orlando has joined #openstack-nova09:14
*** markvoelker has joined #openstack-nova09:29
*** Shunli has quit IRC09:33
*** jichen has quit IRC09:35
*** mdnadeem has quit IRC09:35
*** thorst has joined #openstack-nova09:37
*** thorst has quit IRC09:41
*** jaosorior has quit IRC09:49
*** mingyu has joined #openstack-nova09:49
*** Hosam_ has quit IRC09:53
*** jaosorior has joined #openstack-nova09:53
*** Hosam has joined #openstack-nova09:54
*** faizy_ has quit IRC09:57
*** zhurong has quit IRC09:59
*** liujiong has quit IRC10:00
*** tovin07_ has quit IRC10:00
*** markvoelker has quit IRC10:02
*** nicolasbock has quit IRC10:07
*** mingyu has quit IRC10:08
*** zhurong has joined #openstack-nova10:12
*** pjm6 has quit IRC10:12
*** yamamoto has quit IRC10:17
*** phuongnh has quit IRC10:18
*** pjm6 has joined #openstack-nova10:19
*** dikonoo has quit IRC10:24
*** fragatina has joined #openstack-nova10:24
*** fragatina has quit IRC10:25
*** fragatina has joined #openstack-nova10:25
openstackgerritMikhail Feoktistov proposed openstack/nova master: Add is_vz_container function  https://review.openstack.org/44594710:33
*** sulo_ is now known as sulo10:38
*** edmondsw has joined #openstack-nova10:41
*** edmondsw has quit IRC10:45
*** tbachman has quit IRC10:46
*** sree has quit IRC10:48
*** tiendc has quit IRC10:54
*** gongysh has quit IRC10:58
*** nicolasbock has joined #openstack-nova10:59
*** markvoelker has joined #openstack-nova11:00
*** gomarivera has quit IRC11:00
*** sdague has joined #openstack-nova11:05
*** alexchadin has joined #openstack-nova11:09
openstackgerritSean Dague proposed openstack/nova master: Send request_id on glance calls  https://review.openstack.org/46724211:10
*** egonzalez has joined #openstack-nova11:11
*** eliqiao has quit IRC11:12
*** dikonoor has joined #openstack-nova11:13
*** eliqiao has joined #openstack-nova11:15
*** pewp has quit IRC11:19
*** pewp has joined #openstack-nova11:20
*** yamamoto has joined #openstack-nova11:27
*** rocky_ has joined #openstack-nova11:28
*** mtanino has joined #openstack-nova11:28
*** liverpooler has quit IRC11:33
*** markvoelker has quit IRC11:33
*** salv-orlando has quit IRC11:35
*** rocky_ has quit IRC11:35
*** salv-orlando has joined #openstack-nova11:35
*** wy has joined #openstack-nova11:36
*** mamandle_ has joined #openstack-nova11:36
wyhello,everybody?11:36
*** mamandle__ has joined #openstack-nova11:36
wyI have a question.Now,I do the live-migration successfully.However,when the VM(centos7) is playing video,the migration is failed.But,if I shut down the video,the migration is successful.11:38
wyDoes the video affect the live-migration?11:38
*** mamandle has quit IRC11:38
*** mamandle__ is now known as mamandle11:39
wyPlease help me....11:39
*** salv-orlando has quit IRC11:39
*** mamandle_ has quit IRC11:40
*** dikonoor has quit IRC11:42
*** ericyoung has joined #openstack-nova11:43
*** vks1 has quit IRC11:46
*** litao__ has quit IRC11:46
*** thorst has joined #openstack-nova11:51
*** karimb has quit IRC11:52
*** tbachman has joined #openstack-nova11:52
*** klkumar has quit IRC11:54
wyI do the live-migration successfully.However,when the VM(centos7) is playing video,the migration is failed.But,if I shut down the video,the migration is successful.11:54
wy<wy> Does the video affect the live-migration?11:54
*** tuanluong has quit IRC11:56
*** trinaths has quit IRC12:00
*** dikonoor has joined #openstack-nova12:01
openstackgerritOpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements  https://review.openstack.org/47093712:02
*** trinaths has joined #openstack-nova12:03
*** links has joined #openstack-nova12:04
*** zhurong has quit IRC12:04
*** dillaman has quit IRC12:05
openstackgerritOpenStack Proposal Bot proposed openstack/python-novaclient master: Updated from global requirements  https://review.openstack.org/46958112:07
*** trinaths has quit IRC12:07
*** efried has quit IRC12:09
*** mtanino has quit IRC12:11
*** cdent has joined #openstack-nova12:11
*** markvoelker has joined #openstack-nova12:16
*** dillaman has joined #openstack-nova12:19
*** efried has joined #openstack-nova12:21
*** winston-d_ has quit IRC12:23
*** Hosam has quit IRC12:26
kwathore@wy - no, video should not affect live-migration12:26
wybut why not success?12:26
wyThanks first12:26
kwathorewas video running on source compute node or destination compute node?12:27
kwathoreshare your nova compute logs12:27
*** mdrabe has joined #openstack-nova12:27
*** ekuris has quit IRC12:27
wyvideo playing on the source12:27
wythe logs has no error12:27
wywhat do you want to see?nova-api?nova-scheduler?12:28
*** udesale has quit IRC12:28
kwathorelogs must have some trace about live-migration successfull or not12:28
kwathorenova-compute logs from source compute node12:29
wyopenstack live-migration,when video is playing,the migration is too long.Then,the  vm does not migrate12:29
wyok12:29
wyplease wait12:29
*** trinaths has joined #openstack-nova12:32
*** karimb has joined #openstack-nova12:35
wyINFO nova.virt.libvirt.driver [req-b6f2476e-c9bc-42bd-9c34-9db9e5db0543 bb012ac6b0534fdba5bf70a16785fd8b 3bf434c0309549a291a79b38b34ed7e5 - - -] [instance: dc891233-dac6-4b9a-a1eb-fb51de84fddd] Data remaining 63950848 bytes, low watermark 729088 bytes 143 seconds ago12:35
wy2017-06-05 20:34:23.664 32069 WARNING nova.virt.libvirt.driver [req-b6f2476e-c9bc-42bd-9c34-9db9e5db0543 bb012ac6b0534fdba5bf70a16785fd8b 3bf434c0309549a291a79b38b34ed7e5 - - -] [instance: dc891233-dac6-4b9a-a1eb-fb51de84fddd] Live migration stuck for 150 sec12:35
wy2017-06-05 20:34:23.970 32069 ERROR nova.virt.libvirt.driver [req-b6f2476e-c9bc-42bd-9c34-9db9e5db0543 bb012ac6b0534fdba5bf70a16785fd8b 3bf434c0309549a291a79b38b34ed7e5 - - -] [instance: dc891233-dac6-4b9a-a1eb-fb51de84fddd] Live Migration failure: operation aborted: migration job: canceled by client12:35
wy2017-06-05 20:34:24.170 32069 ERROR nova.virt.libvirt.driver [req-b6f2476e-c9bc-42bd-9c34-9db9e5db0543 bb012ac6b0534fdba5bf70a16785fd8b 3bf434c0309549a291a79b38b34ed7e5 - - -] [instance: dc891233-dac6-4b9a-a1eb-fb51de84fddd] Migration operation has aborted12:35
wy2017-06-05 20:34:40.872 32069 INFO nova.compute.resource_tracker [req-36ecb5cc-4060-4006-8ed4-5c196020a583 - - - - -] Auditing locally available compute resources for node compute112:35
wycould this help??12:35
*** catintheroof has joined #openstack-nova12:35
wythis has error.But I do not do any operation12:35
wy@<kwathore>12:36
wy@kwathore12:36
wyThanks12:36
*** lyan has joined #openstack-nova12:37
*** sbezverk has joined #openstack-nova12:38
*** jpena is now known as jpena|lunch12:38
*** mamandle has quit IRC12:38
*** MVenesio has joined #openstack-nova12:38
*** Hosam has joined #openstack-nova12:39
*** salv-orlando has joined #openstack-nova12:40
*** ekuris has joined #openstack-nova12:43
wyplease help me!12:43
wyThanks ALL12:43
*** cleong has joined #openstack-nova12:44
*** oanson has joined #openstack-nova12:44
*** Guest58049 is now known as beagles12:45
*** jwnx has joined #openstack-nova12:45
openstackgerritEric Fried proposed openstack/nova master: Add service_token for nova-glance interaction  https://review.openstack.org/41966212:45
*** egonzalez has quit IRC12:45
wysorry...i do not understand12:47
asettlewy: that was a automated bot12:48
*** dave-mccowan has joined #openstack-nova12:48
wybut wy?12:49
wybut why?12:49
wyhow can i solve the problem?12:49
asettlewy: I don't know, have you tried googling your issue? Have you asked in ask.openstack.org? Have you asked in #openstack or #openstack-101 ?12:50
*** sbezverk has quit IRC12:50
*** cdent has quit IRC12:51
wygoogle has no the problem12:51
asettleMaybe you should try one of the other avenues12:52
*** jaypipes has joined #openstack-nova12:52
wyI am trying12:53
wybut have no result12:53
wyasette12:53
wyasettle thanks a lot12:53
*** Hosam has quit IRC12:54
asettlewy: just wait, everyone's working across the world and are not all awake at the same time. I would recommend waiting at least 48 hours for responses. If you are seriously struggling, email the openstack list with your query.12:54
*** jmlowe has quit IRC12:55
*** edmondsw has joined #openstack-nova12:55
wyasettle Ok,thanks very much.I am trying other ways12:56
*** sgordon has joined #openstack-nova12:57
openstackgerritEric Fried proposed openstack/nova master: remove ugly local import  https://review.openstack.org/45078712:57
*** gszasz has quit IRC12:58
*** gszasz has joined #openstack-nova13:01
*** mriedem has joined #openstack-nova13:02
*** tbachman has quit IRC13:02
*** jwcroppe has quit IRC13:05
*** ekuris has quit IRC13:05
*** alexchadin has quit IRC13:06
*** alexchadin has joined #openstack-nova13:06
*** alexchadin has quit IRC13:07
*** ekuris has joined #openstack-nova13:07
*** alexchadin has joined #openstack-nova13:07
*** alexchadin has quit IRC13:07
*** alexchadin has joined #openstack-nova13:08
*** alexchadin has quit IRC13:08
*** esberglu has joined #openstack-nova13:08
*** alexchadin has joined #openstack-nova13:09
*** gomarivera has joined #openstack-nova13:11
*** tbachman has joined #openstack-nova13:11
openstackgerritMerged openstack/nova-specs master: Fix citation references (Sphinx 1.6.x compatibility)  https://review.openstack.org/47025213:13
*** baoli has joined #openstack-nova13:13
*** READ10 has joined #openstack-nova13:13
*** alexchadin has quit IRC13:13
*** jamesdenton has joined #openstack-nova13:15
*** cdent has joined #openstack-nova13:16
*** brault_ has quit IRC13:16
*** gomarivera has quit IRC13:16
*** lucasxu has joined #openstack-nova13:17
*** crushil has quit IRC13:20
*** pewp has quit IRC13:23
*** salv-orlando has quit IRC13:23
*** peter-hamilton has joined #openstack-nova13:23
mordredmriedem: morning sir! over the weekend I did a bunch of looking in to new gate failures we're seeing creating servers in shade's functional tests, and I wrote it down in https://bugs.launchpad.net/nova/+bug/169563413:25
openstackLaunchpad bug 1695634 in OpenStack Compute (nova) "Something weird just broke in creating servers" [Undecided,New]13:25
*** pewp has joined #openstack-nova13:26
*** gomarivera has joined #openstack-nova13:26
mordredmriedem: tl;dr - seeing new racey tests that seem to be related to compute nodes not yet being registered with their cell13:26
*** pchavva has joined #openstack-nova13:27
*** jamesdenton has quit IRC13:27
*** mtanino has joined #openstack-nova13:27
*** tbachman has quit IRC13:28
*** jamesdenton has joined #openstack-nova13:29
mriedemmordred: comment 1 and 2 have the same link13:29
*** rpodolyaka has quit IRC13:29
mriedemmordred: also you might have been seeing this red herring in the scheduler logs on startup https://review.openstack.org/#/c/454426/13:30
mriedemnow i see in later comments you reverted that one - but i don't think that's your issue13:31
mordredmriedem: whoops: http://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/screen-n-sch.txt.gz#_Jun_02_12_45_15_755877 should be the second link13:31
mordredmriedem: and yes - I agree, I'm not sure that's the issue either13:31
*** jwcroppe has joined #openstack-nova13:32
mriedembecause all that should do is avoid a nasty stacktrace13:32
mriedemwhen we didn't really have anything to do yet anyway13:32
*** gouthamr has joined #openstack-nova13:32
mordrednod. avoiding stack-trace == good13:33
*** eharney has joined #openstack-nova13:34
mordredmriedem: if you look at the first link - it looks like maybe that attempt to create a server is happening before "Received a sync request from an unknown host 'ubuntu-xenial-osic-cloud1-s3500-9104911'. Re-created its InstanceList"13:35
mordredwhich is, I'm guessing, when the compute host gets registered with the cell, yeah?13:35
*** ekuris has quit IRC13:35
mriedemthat's a periodic task in the compute13:36
cdentmriedem: do you know when johnthetubaguy is expected to be back in circulation?13:36
mriedemcdent: this week i thought13:36
*** tbachman has joined #openstack-nova13:36
mriedemmordred: the compute service has a periodic task to send a list of it's instances to the scheduler13:36
*** brault has joined #openstack-nova13:36
*** jpena|lunch is now known as jpena13:37
mriedemmaybe there is something i messed up with "registering" the host on the initial startup13:37
mriedemgiven this could all be disabled based on the config options you're using, i'd hope that's not the case13:37
mriedemb/c it'd be dumb13:37
mordredyah - because it seems like the first time it happens at http://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/screen-n-sch.txt.gz#_Jun_02_12_47_14_428265 ...13:37
*** gszasz has quit IRC13:37
*** baoli has quit IRC13:38
mordredit indicates the host is unknonw - but then that issuedoesn't happen again, so it seems that the host is registered at that point?13:38
mriedemit registers it when it hits that sync13:38
mriedemand just defaults to an empty instance list for that host13:38
mordredyah13:38
*** baoli has joined #openstack-nova13:38
mriedemin an internal mapping13:38
mriedemso shouldn't be a problem13:38
mordredbut I manage to try to create a server before registration13:38
mriedemmordred: this is a single node job right?13:39
mordredit is13:39
mordredand it's just using devstack defaults for nova13:39
wymorning every one!13:41
wy~13:41
mriedemi thought we used to call discover_hosts or simple_cell_setup in here https://github.com/openstack-dev/devstack/blob/master/lib/nova13:41
mriedembut i'm not seeing that now13:41
wyI do the live-migration successfully.However,when the VM(centos7) is playing video,the migration is failed.But,if I shut down the video,the migration is successful.13:41
*** crushil has joined #openstack-nova13:41
wydoes the video affect the migration?13:41
wyINFO nova.virt.libvirt.driver [req-b6f2476e-c9bc-42bd-9c34-9db9e5db0543 bb012ac6b0534fdba5bf70a16785fd8b 3bf434c0309549a291a79b38b34ed7e5 - - -] [instance: dc891233-dac6-4b9a-a1eb-fb51de84fddd] Data remaining 63950848 bytes, low watermark 729088 bytes 143 seconds ago13:42
wy2017-06-05 20:34:23.664 32069 WARNING nova.virt.libvirt.driver [req-b6f2476e-c9bc-42bd-9c34-9db9e5db0543 bb012ac6b0534fdba5bf70a16785fd8b 3bf434c0309549a291a79b38b34ed7e5 - - -] [instance: dc891233-dac6-4b9a-a1eb-fb51de84fddd] Live migration stuck for 150 sec13:42
wy2017-06-05 20:34:23.970 32069 ERROR nova.virt.libvirt.driver [req-b6f2476e-c9bc-42bd-9c34-9db9e5db0543 bb012ac6b0534fdba5bf70a16785fd8b 3bf434c0309549a291a79b38b34ed7e5 - - -] [instance: dc891233-dac6-4b9a-a1eb-fb51de84fddd] Live Migration failure: operation aborted: migration job: canceled by client13:42
wy2017-06-05 20:34:24.170 32069 ERROR nova.virt.libvirt.driver [req-b6f2476e-c9bc-42bd-9c34-9db9e5db0543 bb012ac6b0534fdba5bf70a16785fd8b 3bf434c0309549a291a79b38b34ed7e5 - - -] [instance: dc891233-dac6-4b9a-a1eb-fb51de84fddd] Migration operation has aborted13:42
wy2017-06-05 20:34:40.872 32069 INFO nova.compute.resource_tracker [req-36ecb5cc-4060-4006-8ed4-5c196020a583 - - - - -] Auditing locally available compute resources for node compute113:42
mordredwy: hi! when you have things like that to share, please use http://paste.openstack.org/ and past a link instead13:43
*** brault has quit IRC13:43
mordredmriedem: discover_hosts seems to be top level now13:44
edleafeScheduler subteam meeting in 15 minutes in #openstack-meeting-alt13:45
*** klkumar has joined #openstack-nova13:45
mordredmriedem: http://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/devstacklog.txt.gz#_2017-06-02_12_46_16_56813:45
*** zhurong has joined #openstack-nova13:45
mriedem2017-06-02 12:46:19.540 | Creating host mapping for compute host 'ubuntu-xenial-osic-cloud1-s3500-9104911': bc5499e9-2e12-4ae6-9e5b-58b8104de16313:46
mriedemmordred: that's what we're waiting on - does your failed test run before that?13:46
mriedemlooks like the test runs after13:47
mriedemJun 02 12:46:58.103925 ubuntu-xenial-osic-cloud1-s3500-9104911 nova-api[16069]: INFO nova.osapi_compute.wsgi.server [None req-a92036cd-3f52-41d6-b104-cb446cfa0ca1 demo demo] 10.39.244.211 "POST /v2.1/servers HTTP/1.1" status: 202 len: 870 time: 0.616972013:47
*** ekuris has joined #openstack-nova13:47
wyok..thankd13:47
mriedemand we blow up here http://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/screen-n-cond.txt.gz#_Jun_02_12_46_59_07148513:49
wyhttp://paste.openstack.org/13:49
efriedmriedem mordred Haven't looked at the bug itself, but it sounds like it could be related to something we hit in the PowerVM CI, where discover_hosts was running before the compute service was ready to handle it.  esberglu added a --strict option to discover_hosts that makes it exit nonzero unless it actually registered your host, so you can run it in a loop until it succeeds.13:49
*** gszasz has joined #openstack-nova13:50
wyhttp://paste.openstack.org/show/X5rwWcSIkukpRU4ahJXm/13:50
*** felipemonteiro has joined #openstack-nova13:50
mriedemhmm13:51
mriedemJun 02 12:46:59.080235 ubuntu-xenial-osic-cloud1-s3500-9104911 nova-conductor[20263]: DEBUG oslo_concurrency.lockutils [None req-a92036cd-3f52-41d6-b104-cb446cfa0ca1 demo demo] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.get_or_set_cached_cell_and_set_connections" :: waited 0.000s: NoValidHost_Remote: No valid host was found. There are not enough hosts available.13:51
mriedemthat looks like the scheduler is locking on cell013:51
wySorry sir.If anyone sees my problem,plese help me. I come from China,later,I will go to sleep,It is my sorry. please help me !http://paste.openstack.org/show/X5rwWcSIkukpRU4ahJXm/13:52
*** diga has joined #openstack-nova13:52
*** felipemonteiro_ has joined #openstack-nova13:53
mriedemwy: it's probably better to ask that question on ask.openstack.org,13:53
mriedemyes video playing on the vm is probably causing issues during live migration,13:54
*** jamesdenton has quit IRC13:54
mriedemit would also be helpful to know which version of nova you're running13:54
wymy openstack version is Mikita13:54
wymriedem nova --version is 3.3.213:55
wyis any help?13:55
mriedemwy: fyi https://github.com/openstack/nova/commit/64a482c24d4dfc2aae42672de160ea38e948304c13:55
mriedem^ is about a known issue in newton, so it's probably also in mitaka13:55
*** karimb has quit IRC13:55
wythanks..i see13:56
mriedemhttps://review.openstack.org/#/c/429798/ was the change on master13:56
mriedemtry disabling that config option13:56
wybut is any ways to solve?13:56
*** felipemonteiro has quit IRC13:56
wyok..i will have a try13:56
mriedemplease see the change and related bug report13:56
*** burt has joined #openstack-nova13:57
*** tidwellr has joined #openstack-nova13:57
mriedemmordred: i've got to get my kid to school and then will continue looking at this - i can't see any other changes around friday that would have caused this except the one yo'uve already identified13:57
wyok...you are so kind13:57
mriedemthere is likely some tightly coupled gorp in the scheduler that i'm not aware of13:57
mordredmriedem: awesome - and thanks - I'll keep poking too13:58
*** zz_dimtruck is now known as dimtruck13:58
*** burt has quit IRC13:59
*** burt has joined #openstack-nova13:59
*** karimb has joined #openstack-nova14:00
*** bnemec has joined #openstack-nova14:00
*** coreywright has quit IRC14:00
edleafeScheduler subteam meeting starting now in #openstack-meeting-alt14:00
wymriedem hi,I have seen the url you send.However,My migration is live-migration and is based on the NFS.Importantly,I can not find the configuration--timeout...what should I do?14:04
wymriedem sorry..I have found.I will try,if have any results,I will tell you14:06
*** mingyu has joined #openstack-nova14:06
*** jamesdenton has joined #openstack-nova14:07
*** pewp has quit IRC14:07
*** pewp has joined #openstack-nova14:08
*** egonzalez has joined #openstack-nova14:09
*** alexchadin has joined #openstack-nova14:10
*** roeyc has joined #openstack-nova14:11
*** coreywright has joined #openstack-nova14:14
*** alexchadin has quit IRC14:14
*** pewp has quit IRC14:15
*** pewp has joined #openstack-nova14:16
openstackgerritSean Dague proposed openstack/nova master: Send request_id on glance calls  https://review.openstack.org/46724214:16
*** alexchadin has joined #openstack-nova14:18
*** prateek has quit IRC14:20
*** trinaths has left #openstack-nova14:20
*** pewp has quit IRC14:21
*** hongbin has joined #openstack-nova14:21
wyI have a question. live-migration speed has any relation to the band and others?14:21
artomwy, you mean bandwidth? Yes14:22
artomIt's essentially copying the VM's RAM from one host to the other14:22
*** prometheanfire has joined #openstack-nova14:22
wyartom just bandwidth?14:23
prometheanfireocata python-novaclient still caps pbr, can we get a release to start uncapping again?14:23
artomwy, not only. The rate at which the contents of RAM changes also affects it14:24
*** pewp has joined #openstack-nova14:24
mriedemprometheanfire: sure, will do14:24
wyNow the problem has none.But,I just open a video on a VM but not play it,now it has migrated 5minutes but not over14:24
artomThe faster the source memory is changing, the longer it'll take, since the destination has to "catch up" by receiving more memory14:24
wybut the VM is just opening a video not playing...But why it takes so long?14:25
mriedemprometheanfire: need to get https://review.openstack.org/#/c/464867/ in first14:26
*** pchavva has quit IRC14:26
wymriedem thanks for you.I have solved problem as you said..but the time it takes so long14:26
*** vks1 has joined #openstack-nova14:26
artomwy, sometimes the best things in life take time ;)14:26
wyartom ok..14:26
wyartom I do not understand14:27
artomwy, just a joke, ignore it :)14:27
wyartom the vm ram is not changing now..but it takes so long14:27
artomwy, I'm not sure what else you need. You said mriedem's suggestion solved your problem.14:27
prometheanfiremriedem: k, thanks14:27
mriedemartom: always the romantic14:28
wyartom yes...I changed the configuration.just disable the  live_migration_progress_timeout14:28
*** udesale has joined #openstack-nova14:29
mriedemwy: the question is, does it eventually complete14:29
mriedemnow that you've disabled that timeout14:29
*** awaugama has joined #openstack-nova14:29
wymriedem now,it is still migrating for about 8minutes ,it does not have error14:29
artommriedem, if you have a moment, mind taking a look at https://review.openstack.org/#/c/465205/?14:30
wymriedem could I think it effects?14:30
mriedemartom: would like lyarwood_ and mdbooth to look at that first,14:30
artomFairly straightforward fix to the XML that swap_volume generates14:30
mriedemthey've been involved in swap volume stuff in the libvirt driver for awhile14:30
artommriedem, they did, on a previous revision14:30
mriedemi see14:31
artomNothing's really changed14:31
artomComments, and formatting, and...14:31
mriedemwell i've got to dig into mordred's problem first14:31
artommriedem, no, my stuff is obviously way more important14:31
mriedemi can star it14:31
wymriedem but i think it would not migrate successfully forever14:31
mordredartom: :)14:31
*** psachin has quit IRC14:31
mriedemwy: i may be mistaken but i though there was a way to check progress with virsh or qemu command line14:32
mordredmriedem: I just pushed up a silly patch in shade to look for "Successfully synced instances from host" in the scheduler logs before starting to run shade's tests14:32
wymriedem i have found none.openstack has not provide command to see the process14:32
wymriedem i am seesing the compute log.but the remaining data decends for a while then asend..14:33
*** psachin has joined #openstack-nova14:34
*** mamandle has joined #openstack-nova14:34
wymriedem could I think it would not migrate successfully forever?14:34
*** baoli has quit IRC14:34
mriedemwy: i mean like "migrate -d URI" https://www.linux-kvm.org/page/Migration#Management14:34
*** baoli has joined #openstack-nova14:35
*** baoli has quit IRC14:35
mriedemwy: https://www.youtube.com/watch?v=Vy8rS_5vea4 about 9 minutes in talks about some of this14:36
mriedemvirsh domjobinfo <domain>14:36
mriedemfor example14:36
wymriedem thanks,i will have a look at14:36
wymriedem you are so helper14:36
*** baoli has joined #openstack-nova14:37
*** alexchadin has quit IRC14:37
*** alexchadin has joined #openstack-nova14:37
*** ekuris has quit IRC14:37
mordredmriedem: because looking througha few logs of successful runs, that seems to happen approximately around when we launch functional tests anyway - so it may just be a legit race-condition between the cloud settling in and the tests getting started (although also maybe there is a real bug obvs)14:37
wymriedem sorry14:37
wymriedem i do not understand14:37
wymriedem but what is the uri?14:38
mordredmriedem: also, every time someone types "virsh" I think they're mistyping vishy ...14:38
wymriedem what i should input in the command14:38
mriedemwy: virsh domjobinfo --help ?14:39
mriedemi assume <domain> is the domain id14:40
wymriedem It is still migrating about many minutes..I just shut down the video.it successfully migrate14:40
mriedeme.g. 00000000ef or something14:40
wymriedem ok14:40
wymriedem is the video affects?14:40
mriedemwy: i assume so yes14:41
mriedemeverything the source vm is doing has to be transferred to the destination vm14:41
wymriedem but i just open the video but not playing it ..it means the video is suspend14:42
*** vks1 has quit IRC14:42
*** pchavva has joined #openstack-nova14:42
wymriedem open the video but not playing it,this will contribute to the change of the ram?14:43
wymriedem or we can say,my bandwidth is not enough,so the changes of ram is bigger than the transmit??14:44
*** psachin has quit IRC14:47
wymriedem thanks very much.it is 11,i have to go to dormitory to sleep.It is my pleasure to make friends with you.You are so kind14:47
*** jmlowe has joined #openstack-nova14:47
wymriedem but ,If you have seen,please help me solve my questions...ahah14:48
*** cfriesen_ has joined #openstack-nova14:48
openstackgerritAndrey Volkov proposed openstack/nova master: [WIP] [placement] Add api-ref for allocations  https://review.openstack.org/47093314:50
openstackgerritAndrey Volkov proposed openstack/nova master: [placement] Add api-ref for aggregates  https://review.openstack.org/47093214:50
mriedemwy: you're welcome. i don't have a good answer to your question about a paused video during a live migration14:50
mriedemand this isn't really the place to debug that level of detail probably14:50
mriedemmordred: ok so back to your thing, this stacktrace is when the compute starts up and calls update_instance_info before the host is mapped http://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/screen-n-sch.txt.gz#_Jun_02_12_45_15_75587714:51
mriedemthen we map the host here http://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/devstacklog.txt.gz#_2017-06-02_12_46_19_54014:51
*** lbragstad has quit IRC14:51
mriedemthe server create request hits the scheduler here http://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/screen-n-sch.txt.gz#_Jun_02_12_46_59_05592114:52
mriedem40 seconds later14:52
*** pchavva has quit IRC14:54
*** hieulq_ has joined #openstack-nova14:54
mriedemmordred: methinks the problem isn't with the compute host not being mapped,14:55
*** armax has joined #openstack-nova14:55
mriedembut rather the compute host failing to register with placement14:55
mriedemb/c the scheduler is asking placement for resource providers (comptue nodes) to satisfy the request14:55
openstackgerritSean Dague proposed openstack/nova master: Be more tolerant of keystone catalog configuration  https://review.openstack.org/43804914:56
*** yamahata has joined #openstack-nova14:56
*** lbragstad has joined #openstack-nova14:57
openstackgerritEric Berglund proposed openstack/nova master: Add PowerVM to nova support matrix  https://review.openstack.org/47099914:58
*** lei-zh has joined #openstack-nova14:59
mordredmriedem: ah -ok that makes sense to me14:59
mriedemmordred: still tracing the n-cpu logs14:59
mriedemsdague: fyi while you were out, a few of the clients you needed for the global request id stuff got blacklisted14:59
mriedemthe versions that is, neutronclient 6.3.0 for sure15:00
cdentdansmith: that was just an example of a pattern, not a plan15:00
openstackgerritEric Berglund proposed openstack/nova master: Add PowerVM to nova support matrix  https://review.openstack.org/47099915:00
sdaguemriedem: yeh, saw that, but the glance one looks fine now15:00
edleafedansmith: what cdent said15:01
sdaguemriedem: so I flipped that around15:01
dansmithcdent: okay it seemed like it would be specifically targeted stuff for nova's scheduler though15:01
jaypipescdent, alex_xu, edleafe, dansmith: join #novaschedulercontd15:01
edleafedansmith: it doesn't have to be15:01
dansmithjaypipes: ugh, can't we hangout?15:02
*** liverpooler has joined #openstack-nova15:02
jaypipesdansmith: I'm happy to hangout.15:02
dansmithjaypipes: new irc channels ftl, even if short-lived15:02
*** jamesdenton has quit IRC15:02
cdenti can hangout15:02
jaypipesok, let's do a hangout.15:02
jaypipesone sec, I will set up15:02
dansmithcan we have five mins for food/coffee?15:02
dansmithI have had neither yet15:02
edleafedansmith: +115:02
*** wy has quit IRC15:03
*** felipemonteiro has joined #openstack-nova15:03
edleafeHow about at <yourtime>:15 ?15:03
*** gszasz has quit IRC15:03
cdentwfm15:03
mriedemsfinucan: looks like we deprecated the use_neutron config option, which is going to be a bit confusing if you're using nova-network with cells v115:03
openstackgerritChris Dent proposed openstack/nova master: Sync os-traits to Traits database  https://review.openstack.org/46957815:03
*** gyee has joined #openstack-nova15:04
dansmithyeah15:04
dansmithI can fill out three or four employment applications in 15 mins15:04
jaypipesdansmith, cdent, edleafe, alex_xu: https://hangouts.google.com/call/fxu4kc4ufrakhiwjyq6idxrmcqu15:04
jaypipesI'll just sit in the hangout. join as you will.15:04
cfriesen_did anyone sort out the issue with jenkins getting stricter about releasenotes?15:05
cfriesen_I'm looking at this in particular, hitting it on my local machine as well: https://review.openstack.org/#/c/43514115:06
*** hshiina has quit IRC15:06
*** crushil has quit IRC15:06
lei-zhjaypipes, for requesting traits part, are we going to let users to specify traits associated with top resource providers and nested ones?15:07
*** felipemonteiro_ has quit IRC15:07
*** karimb has quit IRC15:07
*** Oku_OS is now known as Oku_OS-away15:08
jaypipeslei-zh: my initial thoughts on that matter were that we would continue to use the flavor as the thing that users would request, and flavors would have a "required" attribute that was a list of string traits that were required by that flavor.15:09
*** pchavva has joined #openstack-nova15:09
*** karimb has joined #openstack-nova15:10
jaypipeslei-zh: long-term, I would love to have a compute API that allowed the user (or certain users I suppose) to request specific traits directly and not have to request a particular flavor.15:10
jaypipeslei-zh: but we're a ways from that. :)15:10
*** ckopper has joined #openstack-nova15:10
alex_xujaypipes: lei-zh , I'm thinking the meaning of required_traits is that 'give me a resource provider or it's nested provider includes those required traits'15:11
jaypipesalex_xu: correct. (also shared provider)15:11
*** felipemonteiro has quit IRC15:12
*** dikonoor has quit IRC15:12
mordredmriedem: in general, please let me know if there is anything I can do to help - I'm pretty sure we've passed my level of understanding of being helpful with debugging further, but I'll dig in to whatever if it's useful15:13
mriedemmordred: sure, if it's placement it should be fairly easy for me to see what's up15:14
*** liverpooler has quit IRC15:14
mordredmriedem: also, is there any way as a user to get "no hosts could schedule this request" is the cause of the error?15:14
*** liverpooler has joined #openstack-nova15:14
*** zhurong has quit IRC15:15
*** mvk has quit IRC15:19
*** amit345 has joined #openstack-nova15:19
sdaguemriedem: did you ever get the mysql specific test case merged from a couple of weeks ago?15:19
*** bnemec has quit IRC15:19
*** felipemonteiro has joined #openstack-nova15:20
mriedemsdague: nope15:20
*** amit345 has quit IRC15:20
mriedemsdague: haven't found time to get back to it yet15:20
*** tbachman has quit IRC15:20
sdaguemriedem: was it working?15:21
sdagueor not?15:21
*** amit345 has joined #openstack-nova15:21
mriedemsort of15:21
mriedemhttps://review.openstack.org/#/c/467772/15:21
mriedemhttps://review.openstack.org/#/c/467772/2/nova/tests/functional/test_servers.py@102915:21
mriedemi think we have to do some things manually for setting up the cell mapping pointed at the mysql backend15:22
mriedemand then create the HostMapping and map that to the CellMapping manually also15:22
*** amit345 has quit IRC15:22
mriedemfor our single nova-compute service15:22
*** amit345 has joined #openstack-nova15:22
mriedemmordred: the user gets a 202 so they don't see the cause15:22
sdagueok, maybe I'll play with it15:22
mriedemmordred: the best you can get is an instance fault recorded on the instance, but those are visible only to admins by default15:23
*** amit345 has quit IRC15:23
*** bnemec has joined #openstack-nova15:24
*** lei-zh has quit IRC15:25
*** lei-zh has joined #openstack-nova15:26
openstackgerritMatt Riedemann proposed openstack/nova master: DNM: Test removing live_migration_uri from devstack  https://review.openstack.org/47101915:28
openstackgerritMerged openstack/nova master: Updated from global requirements  https://review.openstack.org/47093715:29
*** MVenesio has quit IRC15:31
mordredmriedem: yah. that was what I was seeing. do you think it would be worth trying to write up a spec on exposing just a little more data in the error case?15:32
mordredmriedem: I know we've hit issues before where nova couldn't boot an instance but the error was ultimately "this image is in an invalid format"  - but is shows up just as "can't boot"15:33
mordredmriedem: I'd be happy to try to write something up - but if it's a bad or unworkable area there's no need to bother of course15:33
mriedemmordred: it's been tried before, usually ends up being a problem because of leaking internal details about the cloud so could be used for attacks15:34
*** marst has joined #openstack-nova15:34
*** lucasxu has quit IRC15:35
mriedemcfriesen_ had a spec and some code for improving the filtering in the scheduler logs to help operators, but that doesn't help the end user15:35
mriedemexcept get their ticket resolved faster maybe15:35
*** armax has quit IRC15:35
mordredmriedem: yah - that's what I'd be worried about from a "this spec cant' happen' pov15:35
*** lei-zh has quit IRC15:36
*** udesale has quit IRC15:36
*** armax has joined #openstack-nova15:37
mriedemmordred: so looking at your issue,15:37
*** mingyu has quit IRC15:37
mriedemnova-compute is failing to talk to placement http://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/screen-n-cpu.txt.gz#_Jun_02_12_46_14_47755415:38
mriedemgetting 503 back15:38
mriedemwhich means as far as placement and the scheduler are concerned, there are no compute nodes available15:38
*** mingyu has joined #openstack-nova15:39
mriedemmordred: the compute node finally gets registered here: http://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/screen-n-cpu.txt.gz#_Jun_02_12_47_14_57682915:39
mriedemwhich is 15 seconds after your test failed15:39
*** slaweq has joined #openstack-nova15:39
*** armax has quit IRC15:41
* artom needs to write a thing that auto-updates your microversion if someone else merges it before you do15:41
mordredmriedem: ah - so there was an early issue, but it eventually heals15:42
mordredmriedem: so in terms of the hacky thing I had for the tests, I could actually check for "Created resource provider record via placement API...$(hostname)" in the n-cpu logs15:43
mordredas a check for "is the cloud up and ready for action" before starting the tests15:43
*** mhenkel_ has joined #openstack-nova15:43
*** mingyu has quit IRC15:43
mriedemwell,15:44
mriedemthe better less hacky thing would be to GET /resource_providers against the placement endpoint15:44
*** prometheanfire has left #openstack-nova15:44
mriedemuntil you get something back that's not an empty body15:44
mordredmriedem: ah - yah. could do that for sure15:44
mriedemnote that placement is admin-only by default15:44
*** crushil has joined #openstack-nova15:45
mordredmriedem: that's ok - we've got admin accounts in the functional tets15:45
mordredtests15:45
mriedemi'm also curious why placement is failing in the first place15:45
mhenkel_mriedem: when you find some time, would you mind to take a look into https://review.openstack.org/#/c/441183/ ?15:45
mriedemthat should be started well before n-cpu and n-sch15:45
mhenkel_it passed ci and I addressed all the open points15:45
*** slaweq has quit IRC15:46
mordredI don't think this is actually a thing where shade sohuld be checking for the error in normal usage -it seems like a pretty specific first-install devstack race from an api consumer perspective15:46
mriedemmordred: also, i'm not sure why this would have regressed in the last few days, i don't see anything related to changing the startup order in devstack15:46
mordredmriedem: it might just have always been a race and something else caused something else to be slow so it just started showing up15:46
cfriesen_mriedem: mordred: in our environment the users are generally very tightly couple to the admins, so we expose all the sched failure logs directly to the user15:47
mordredmriedem: or, actually,it could have always been here, but now something is improved so we get all the way to tests before the self-healing15:47
mordredcfriesen_: yah - I have admin on almost none of my clouds :)15:47
mordredso I'm usually the annoying voice of "how do I learn this as a non-admin"15:47
mriedemlooking at the placement logs it takes quite a while for it to be ready it looks like15:48
mriedemhttp://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/screen-placement-api.txt.gz15:49
mriedemthe first request i see it process is this:15:50
mriedemJun 02 12:46:58.883989 ubuntu-xenial-osic-cloud1-s3500-9104911 devstack@placement-api.service[22675]: DEBUG nova.api.openstack.placement.requestlog [None req-6e9651be-51e5-40e6-a887-bd72120060a0 service placement] Starting request: 10.39.244.211 "GET /placement/resource_providers?resources=MEMORY_MB%3A256%2CVCPU%3A1" {{(pid=22678) __call__ /opt/stack/new/nova/nova/api/openstack/placement/requestlog.py:38}}15:50
mriedemwhich is the one during scheduling15:50
mriedemso....placement isn't up when the compute service is trying to check in15:50
mriedemwtf15:50
mriedemtakes >1 minute to startup placement-api15:51
mriedemi don't know if that's systemd / uwsgi shenanigans or what15:51
mriedemcdent: ^ any ideas?15:51
cdentmriedem: sorry on intense hangout15:51
cdentwill catch up soonish15:51
*** mhenkel__ has joined #openstack-nova15:52
*** rcernin has quit IRC15:52
* mordred can't wait for cdent to solve all the things15:52
mriedemcdent: just tell them we're going to start doing noop claims everywhere15:52
mriedemproblem solved15:52
*** Apoorva has joined #openstack-nova15:53
*** mhenkel_ has quit IRC15:54
mriedemsdague: maybe your spidey sense can detect something ^?15:54
*** tbachman has joined #openstack-nova15:55
mriedemlooks like we start placement here http://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/devstacklog.txt.gz#_2017-06-02_12_45_42_06115:55
mriedem2017-06-02 12:45:42.061 | Waiting for placement-api to start...15:55
mriedemwhich is *after* we started n-cpu15:56
mriedemderp15:56
jaypipesalex_xu: https://review.openstack.org/#/c/46957815:56
sdaguemriedem: it's only 4 seconds for start15:57
mriedemhttp://logs.openstack.org/95/468095/2/gate/gate-shade-functional/3f5d8dc/logs/devstacklog.txt.gz#_2017-06-02_12_45_10_00615:57
*** veyu has joined #openstack-nova15:57
mriedemso we start placement-api 32 seconds after we start nova-compute,15:58
*** kwathore_ has joined #openstack-nova15:58
mriedemnova-compute starts up and tries to register the compute node with placement but gets a 503 b/c placement isn't up yet,15:58
mriedemmeanwhile, the compute host is mapped to the cell and tests start,15:58
mriedemtest tries to create a server, scheduler asks placement for a resource provider (compute node) for the server, there are 0 because none are registered, and the server create fails,15:59
*** kwathore has quit IRC15:59
*** tbachman_ has joined #openstack-nova15:59
mriedem15 seconds later, n-cpu registers the compute node with placement during it's periodic task to update avialable resources (1 minute cron)15:59
openstackgerritJay Pipes proposed openstack/nova master: placement project_id, user_id in PUT /allocations  https://review.openstack.org/46963415:59
openstackgerritJay Pipes proposed openstack/nova master: placement: Add GET /usages to placement API  https://review.openstack.org/46963515:59
openstackgerritJay Pipes proposed openstack/nova master: placement: use separate tables for projects/users  https://review.openstack.org/47064515:59
mriedemmordred: ^15:59
*** tbachman has quit IRC16:00
*** tbachman_ is now known as tbachman16:00
mriedemcdent: edleafe: jaypipes: dansmith: can someone summarize whatever changes are coming out of the hangout you just had in the dev ML please?16:01
dansmithmriedem: jay is on it16:01
jaypipesmriedem: yes.16:01
jaypipesmriedem: I will. etherpad being worked.16:01
mriedemok thanks16:01
sdaguemriedem: yeh, our order is super bonkers16:01
sdaguewe should fix that16:01
cdentmriedem: the devstack problem, appears to be that fundamentally placement starts after nova in stack.sh16:01
mriedemcdent: yeah16:02
sdagueI actually have no idea why placement comes after nova, that's just silly16:02
sdaguesomeone propose the ordering patch, and I'll fast approve16:02
mriedemi think it's because nova.sh sets up the config or something?16:02
mriedemmight be circular dep16:02
cdentwe can start later though16:02
sdaguemriedem: the config stage is a lot earlier16:02
cdentwe can config after but start before16:02
sdagueand should be done16:02
cdentI'll push somehting16:02
cdentone sec16:02
sdagueyeh16:02
cdentis there a bug to related to in the commit?16:03
*** veyu has quit IRC16:03
mriedemcdent: https://bugs.launchpad.net/devstack/+bug/169563416:03
openstackLaunchpad bug 1695634 in devstack "Something weird just broke in creating servers" [Undecided,New]16:03
*** sbezverk has joined #openstack-nova16:04
*** mamandle has quit IRC16:05
*** mtanino has quit IRC16:06
*** mtanino has joined #openstack-nova16:06
cdentsdague, mriedem : https://review.openstack.org/47104216:07
cdenti started before neutron and cinder too16:07
*** salv-orlando has joined #openstack-nova16:07
*** hieulq_ has quit IRC16:08
*** hieulq_ has joined #openstack-nova16:09
*** hieulq_ has quit IRC16:09
mriedemcdent: question inline16:10
*** jamesdenton has joined #openstack-nova16:11
cdentmriedem: answer inline16:12
*** alexchadin has quit IRC16:17
mriedemok i'd just remove the comment16:18
mriedemthe note i mean16:18
mriedemthe comment is fine, the note is confusing16:18
cdentk16:18
*** jamesdenton has quit IRC16:18
cdentdone16:19
mriedemthanks16:19
*** roeyc has quit IRC16:21
*** roeyc has joined #openstack-nova16:21
*** pcaruana has quit IRC16:23
*** brault has joined #openstack-nova16:23
*** karimb has quit IRC16:23
*** READ10 has quit IRC16:24
*** yamamoto has quit IRC16:24
*** READ10 has joined #openstack-nova16:24
*** mriedem1 has joined #openstack-nova16:26
*** mriedem has quit IRC16:26
*** mriedem1 is now known as mriedem16:28
mordredcdent, mriedem: woot! thank you!16:28
*** lucasxu has joined #openstack-nova16:28
mordredI'm glad this turned out to be potentially an actual issue and not just a random testing weird16:29
cdentwas a good spot16:29
cdentthat the problem hasn't come up sooner is odd16:29
cdentyou must be doing something faster than other things are16:29
mordredyah. it's still interesting why it only started showing up friday16:30
cdentdid reverting that change you reverted make any difference?16:30
mriedemi don't see anything in devstack or nova around friday that would cause the start time issue16:30
mordredcdent: we have a very low overhead to start running tests and our tests are all pretty quick - so I think we're a really good race from end-of-devstack to "and now stuff is running"16:30
mordredmriedem: onlything I can think of is that something about the fix for the stacktrace maybe also caused something to not issue an extra registration sync16:31
mriedemmy patch stopped doing the pointless update_instance_info rpc cast from n-cpu to n-sch16:31
mordredso maybe there was an accidental retry happening that the stacktrace fix removed?16:31
cdentperiodic job timing?16:31
mriedemsending an empty list16:32
mriedemthe periodic kicks in ~1 minute after init_host on the compute16:32
mriedemif update_instance_info were an rpc call that would make sense why things were slower during n-cpu startup, but it's a cast16:33
cdentIf I remember right, init_host calls all the same stuff that the periodic does/16:33
jaypipesedleafe, dansmith, cdent, alex_xu: https://etherpad.openstack.org/p/placement-allocations-straw-man16:33
mriedemcdent: the pre_start_hook calls update_available_resource16:33
mriedemwhich calls the RT to create the ComputeNode record and do some things yeah16:34
mriedemthat's different from the update_instance_info stuff cast to the scheduler16:34
cdentthanks jaypipes, I'm going home, will look from there, where the lack of people ought to allow proper attention16:34
mriedem^ is all about server group affinity/anti-affinity filters16:34
jaypipeskk16:34
*** sridharg has quit IRC16:34
*** slaweq has joined #openstack-nova16:34
dansmithjaypipes: can I reset colors on that so we can see only deltas from this?16:35
*** links has quit IRC16:35
cdentdansmith++16:35
jaypipesdansmith: go for it.16:35
cfriesen_there are so many races around server group affinity/antiaffinity it's not even funny16:35
*** diga has quit IRC16:35
*** roeyc has quit IRC16:35
*** fragatin_ has joined #openstack-nova16:36
*** fragatina has quit IRC16:37
cfriesen_jaypipes: is it possible to have floating point values for resources?16:37
*** ltomasbo is now known as ltomasbo|away16:37
cfriesen_or is it integer only?16:37
*** pchavva has quit IRC16:37
jaypipescfriesen_: integer only.16:37
*** cdent has quit IRC16:38
cfriesen_jaypipes: we currently allow a mix of instances with shared/dedicated cpus on the same compute node (with cpusets to keep the affinity straight) and so report usage as floating point.   I guess we'll move to fixed point with placement.16:39
*** eharney has quit IRC16:39
*** klkumar has quit IRC16:40
edleafejaypipes: running out for a bit, so I'll go more in depth when I get back.16:43
sdaguemriedem: https://review.openstack.org/#/c/467242/ is the glance request_id change ready to go in16:44
*** harlowja has joined #openstack-nova16:45
*** pchavva has joined #openstack-nova16:45
jaypipescfriesen_: what do you mean "report usage as floating point"?16:46
*** klkumar has joined #openstack-nova16:49
jaypipesdansmith, mriedem: the three of us are having a very Reggae-colored conversation on this etherpad.16:51
mriedemwish i had some ganja16:52
jaypipesmriedem: if you lived in Sarasota...16:53
*** armax has joined #openstack-nova16:54
*** lyan has quit IRC16:54
*** ftersin has quit IRC16:55
cfriesen_jaypipes: we report the CPU usage of a compute node as sum(dedicated vcpus) + sum(shared vcpus)/cpu_allocation_ratio.   The total cannot be more than the available CPUs on the compute node.16:56
*** klkumar has quit IRC16:56
cfriesen_jaypipes: with integer usage we'll just have to multiply everything by cpu_allocation_ratio16:57
jaypipescfriesen_: yup.16:57
*** cdent has joined #openstack-nova16:58
*** roeyc has joined #openstack-nova16:58
*** mhenkel__ has quit IRC16:58
*** lyan has joined #openstack-nova16:58
*** cdent has left #openstack-nova16:59
*** cdent has quit IRC16:59
*** liverpooler has quit IRC17:01
*** xgerman_ has left #openstack-nova17:01
*** jaosorior is now known as jaosorior_away17:03
cfriesen_who's responsible for releasenotes issues?   My commit https://review.openstack.org/#/c/435141 keeps failing jenkins because it chokes on an older releasenotes file.  Something seems to have gotten more strict recently.17:04
mriedemsdague: +217:04
mriedemcfriesen_: open a novaclient bug17:05
*** liverpooler has joined #openstack-nova17:06
mriedemmy guess would be something with sphinx 1.6.1 or 1.6.217:06
mriedem1.6.1 is blacklisted but 1.6.2 isn't17:07
cfriesen_mriedem: okay17:07
mriedemjaypipes: we should have a separate bp and spec for this new placement api thing, the claims in the scheduler bp will depend on it17:08
mriedembut i assume you want to communicate it to the ML so it sinks in a bit first17:08
*** cdent has joined #openstack-nova17:09
sdaguecfriesen_: it might be worth checking with dhellman or the like17:12
mriedemi know the fix17:12
mriedemjust need the bug17:12
mriedemtesting locally17:12
*** tbachman has quit IRC17:12
mriedemnot sure why it passed before though17:13
*** lucasxu has quit IRC17:13
cfriesen_mriedem: https://bugs.launchpad.net/python-novaclient/+bug/169593917:13
openstackLaunchpad bug 1695939 in python-novaclient "jenkins failing on older releasenote file" [Undecided,New]17:13
*** trinaths has joined #openstack-nova17:14
*** lucasxu has joined #openstack-nova17:14
cfriesen_mriedem: I tried locally modifying test-requirements.txt but it still installed sphinx 1.6.2.  odd17:14
mriedemthe release note itself is bad17:14
cfriesen_mriedem: yeah, I tried fixing it but couldn't figure out how to get it to  pass17:15
mriedem1. there is no 'update' section17:15
mriedem2. it's not formatted for rst17:15
sdaguedansmith / melwitt: can either / both of you look at this devstack fix for cells that was proposed - https://review.openstack.org/#/c/47059817:15
jaypipesmriedem: dansmith and I are on a hangout talking about something related... we're paying attention. give us a bit :)17:16
cfriesen_mriedem: for 2, do all the single dashes become asterisks?17:16
mriedemcfriesen_: i'm fixing17:16
*** yamahata has quit IRC17:19
*** tbachman has joined #openstack-nova17:21
*** yamamoto has joined #openstack-nova17:25
openstackgerritChris Dent proposed openstack/nova master: [placement] Use util.extract_json in allocations handler  https://review.openstack.org/47106717:25
*** jpena is now known as jpena|off17:30
*** fragatin_ has quit IRC17:32
*** fragatina has joined #openstack-nova17:32
*** trinaths has quit IRC17:33
mriedemwtf i guess this is more complicated than i thought - i've made changes, committed them, rebuilt and it still picks up the old thing - probably because it's processing an old commit17:33
mriedemand i can't re-write history17:33
mriedembut reno should realize i'm changing something17:33
*** yamamoto has quit IRC17:33
*** MVenesio has joined #openstack-nova17:35
cfriesen_mriedem: bug in the CI?17:39
mriedemmight be a bug in reno, i'm talking to dhellmann about it17:40
jaypipesOMG I want to friggin shoot myself in the head right now.17:41
jaypipesI fucking hate SQL migrations.17:41
*** penick has joined #openstack-nova17:41
*** roeyc has quit IRC17:41
jaypipeszzzeek: you around? could really use your help :(17:41
dansmithsdague: done17:41
zzzeekjaypipes: hey17:43
jaypipeszzzeek: hi Mike. got a few minutes to help me with some very frustrating SQL migrations issues, pretty please?17:44
zzzeekyes17:44
sdaguedansmith: thanks, it looked weird to me. Good to get the real expert view in there17:44
jaypipeszzzeek: thanks :) it's about https://review.openstack.org/#/c/470645/17:44
jaypipeszzzeek: I've tried a bunch of permutations (see revisions on the 044_placement_add_projects_users.py file17:45
jaypipeszzzeek: and I'll fix PG and break SQLite/MySQL, and fix MySQL and fix PG.17:45
jaypipeszzzeek: going round and round in circles :(17:45
jaypipeszzzeek: http://logs.openstack.org/45/470645/4/check/gate-nova-tox-functional-ubuntu-xenial/0bf172a/testr_results.html.gz17:45
zzzeek*obligatory grr sqlalchemy-migrate is ancient crap* irc message17:46
zzzeekOK so onto the acvtual problem17:46
jaypipeszzzeek: essentially, I first tried just doing an ALTER TABLE MODIFY COLUMN by changing the project_id and user_id fields in the consumers table from VARCHAR to INT.17:46
cfriesen_dansmith: If I'm reading this right, "nova-manage db online_data_migrations" returns 0 if we're successfully done all the migrations or if there were errors and we bailed out.  This seems like a strange design.17:46
jaypipeszzzeek: using a col.alter(...) command.17:46
jaypipeszzzeek: PG hated that and requires me to drop and recreate the columns.17:47
zzzeekjaypipes: it does?  usually it just wants you to use a CAST so it can rewrite the data ...oh there'sa syntax for that hold on ... :)17:47
jaypipeszzzeek: so I did that, then MySQL complains about indexes either existing or not existing depending on whether I drop the columns before or after dropping indexes. and when I drop indexes, PG complains there's no such indexes.17:47
zzzeekjaypipes: what is the "PG hated that" error you are getting ?17:48
jaypipesone sec, grabbing.17:48
zzzeekjaypipes: also, your problem might be "migrate_engine.name == 'postgres'", sqlalchemy calls it "postgresql"17:49
*** sree has joined #openstack-nova17:49
*** awaugama has quit IRC17:50
jaypipeszzzeek: no, that definitely runs only on postgresql AFAICT... it's the name of the db driver in sqlalchemy-migrate :(17:50
zzzeekjaypipes: weird.17:50
zzzeekjaypipes: so....why not put the whole drop/create thing inside of "if name == 'postgres'" and just use normal ALTER for the others ?17:50
jaypipeszzzeek: I think I've tried that and it failed something about models being out of sync...17:51
zzzeekkeeping in mind that PG might just want you to say "ALTER COLUMN...USING", which is pg-specific anyway so you still need the conditional17:51
jaypipeszzzeek: unfortunately, CI doesn't seem to want to allow me to see past test run failures :(17:51
*** lucasxu has quit IRC17:51
*** fragatina has quit IRC17:52
jaypipeszzzeek: how would I do the ALTER COLUMN ... USING thing if using sqlalchemy-migrate's col.alter() thing?17:52
zzzeekjaypipes: that doesn't make any sense.  program goes into the migration, and comes out, then it verifies things.  if the migration works, then it worked, whether or not it did ALTER or drop/create17:52
zzzeekjaypipes: you'd have to do migrate_engine.execute("ALTER TABLE...")17:52
jaypipesk17:53
*** lucasxu has joined #openstack-nova17:53
jaypipessdague: is there a way to view old CI test runs?17:53
sdaguejaypipes: how old?17:53
zzzeekjaypipes: if that's the issue.  not sure if PG has done things w/ the indexes that are more of a big deal to change17:53
jaypipessdague: a day17:53
sdagueyeh, unhide the CI and look through the jenkins comments17:53
jaypipessdague: oh, duh, thx17:53
sdaguethe rollup of the last run is just a convenience function17:53
*** sree has quit IRC17:54
jaypipeszzzeek: ok, here's the failure from the original alter: http://logs.openstack.org/45/470645/1/check/gate-nova-tox-functional-ubuntu-xenial/8180560/testr_results.html.gz17:54
jaypipeszzzeek: that happened when I did the original here: https://review.openstack.org/#/c/470645/1/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/044_placement_add_projects_users.py17:55
zzzeekjaypipes: ha!  HINT:  You might need to specify "USING project_id::integer".    me wants another darth vader usb key17:55
jaypipeszzzeek: yeah... so instead of that, I did this: https://review.openstack.org/#/c/470645/2/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/044_placement_add_projects_users.py17:56
dansmithmriedem: so this passed the live migration job. Do you still want me to change it or what? https://review.openstack.org/#/c/458537/31/nova/tests/live_migration/hooks/ceph.sh17:56
jaypipeszzzeek: which failed with this: http://logs.openstack.org/45/470645/2/check/gate-nova-tox-functional-ubuntu-xenial/f94cf09/testr_results.html.gz17:56
*** pchavva has quit IRC17:57
*** sambetts is now known as sambetts|afk17:57
zzzeekjaypipes: OK, so theres the most ancient issue in SQLA to allow index.drop(checkfirst=True), however, I can get you through that anyway, if you want to go that route.  but still, "if engine != 'postgres'", then drop index17:57
zzzeekjaypipes: but still, "if engine == 'postgres': migrate_engine.execute("ALTER.... USING")"17:58
jaypipeszzzeek: not sure, frankly, why https://review.openstack.org/#/c/470645/2/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/044_placement_add_projects_users.py didn't work. I don't know why sqlalchemy found indexes in the consumers table to drop but when issuing the SQL statement, PG says no such index exists. :(17:58
zzzeekjaypipes: looks like it's because dropping the columns drops the index17:58
zzzeekjaypipes: then you go to emit DROP INDEX and PG says index does not exist17:59
*** pchavva has joined #openstack-nova17:59
jaypipeszzzeek: right, which I then tried to fix in the next revision: :)17:59
jaypipeshttps://review.openstack.org/#/c/470645/3/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/044_placement_add_projects_users.py17:59
zzzeekjaypipes: oh, wait, you are assuming consumers.indexes is correct. no.   this is not how that works :)17:59
zzzeekjaypipes: throw this into "sqlalchemy-migrate is misleading"18:00
jaypipeszzzeek: which PG then barfed on18:00
zzzeekjaypipes: yeah no, consumers.indexes is not changing itself when you drop those columns18:00
mordredcdent, mriedem: the shade test patch is all green!18:00
jaypipeszzzeek: hmm. ok.18:00
zzzeekjaypipes: so, if you want *that*, do this:  "from sqlalchemy import inspect;  insp = inspect(migrate_engine); indexes = insp.get_indexes("consumers")"18:00
cdentyayness. I think sdague said he's fast approve the devstack change18:00
mordredalthough linuxbridge had a sad18:00
* mordred hit the recheck magicwand18:01
jaypipeszzzeek: I think the simplest solution is to go back to revision 1 and do the execute(ALTER TABLE ... USING) thing for Postgresql?18:01
zzzeekjaypipes: that will just give you names of indexes.   if it were me I'd then just say "engine.execute("DROP INDEX <name>")", howver you can also say Index("myname").drop()18:01
zzzeekjaypipes: I think it is yes.  but all of these different scenarios can work, you're just having very bad luck stepping on every rake in the yard :)18:02
jaypipesindeed :)18:02
zzzeekjaypipes: try the USING with the conditional, i can make that or the other version work for you afte that18:02
zzzeekjaypipes: you don't run mysql/pg functional tests locally ?18:02
*** salv-orlando has quit IRC18:02
sdaguethat linuxbridge sad was a fail in test_device_tagging18:03
*** salv-orlando has joined #openstack-nova18:03
sdaguemriedem: that one being tracked yet, the error looks like it's going to be a race18:03
jaypipeszzzeek: only mysql and sqlite, not postgreSQL, no...18:03
*** peter-hamilton has quit IRC18:03
jaypipeszzzeek: suppose I should :)18:03
zzzeekjaypipes: it's just a yum install away... :)18:03
jaypipesyeah yeah :)18:04
zzzeekjaypipes: anyway I obv. have everythign running here so i can run through it later if you still cant get it working18:04
jaypipeszzzeek: I'll just install PG :)18:04
zzzeekjaypipes: OK.18:05
*** dillaman has quit IRC18:05
*** salv-orlando has quit IRC18:08
*** lucasxu has quit IRC18:08
*** lucasxu has joined #openstack-nova18:09
mriedemsdague: huh? link?18:10
* mriedem was lunching18:10
openstackgerritMerged openstack/nova master: Be more tolerant of keystone catalog configuration  https://review.openstack.org/43804918:12
mriedemsdague: oh you mean this http://logs.openstack.org/42/471042/2/check/gate-tempest-dsvm-neutron-linuxbridge-ubuntu-xenial/4cc66aa/console.html#_2017-06-05_17_23_22_27840918:12
mriedemi'm not aware of a bug for that18:12
mriedemdansmith: just leave it18:13
*** mvk has joined #openstack-nova18:17
openstackgerritDoug Hellmann proposed openstack/python-novaclient master: strip the remote prefixes from the release note branch specifiers  https://review.openstack.org/47108218:17
sdaguemriedem: yeh - http://logs.openstack.org/42/471042/2/check/gate-tempest-dsvm-neutron-linuxbridge-ubuntu-xenial/4cc66aa/console.html#_2017-06-05_17_23_22_33756018:18
sdagueI'll make a bug here in a minute for it18:18
sdaguethere are a few hits on it18:18
*** dillaman has joined #openstack-nova18:20
*** yamahata has joined #openstack-nova18:25
*** awaugama has joined #openstack-nova18:31
*** eharney has joined #openstack-nova18:37
*** penick has quit IRC18:39
*** gjayavelu has joined #openstack-nova18:40
*** slaweq has quit IRC18:45
*** cheran has joined #openstack-nova18:46
sdague\o/ for the diagnostics API standardization - https://review.openstack.org/#/c/355540/18:53
openstackgerritChris Dent proposed openstack/nova master: Fix ChanceScheduler returning a wrong node  https://review.openstack.org/46672519:03
mriedemcool. a few wording things in there that could be cleaned up in a follow up change19:03
mriedemnow onto the novaclient change19:04
*** brault has quit IRC19:04
mriedemcfriesen_: this is what you're waiting on btw https://review.openstack.org/#/c/471081/19:06
openstackgerritMatt Riedemann proposed openstack/nova master: Replace messaging.get_transport with get_rpc_transport  https://review.openstack.org/46965919:06
cfriesen_mriedem: mriedem: cool, thanks19:06
cfriesen_mriedem: the empty lines are needed for the nested lists to get formatted properly?  is that yaml or rst?19:08
mriedemrst19:09
cfriesen_is the hyphen a formatting element in rst?19:09
mriedemyeah19:09
mriedemhttp://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#bullet-lists19:10
cfriesen_got it19:10
efriedmikal Should https://review.openstack.org/#/c/4379/ have removed the ", by name (not ID!)" part of https://github.com/openstack/nova/blob/master/nova/virt/driver.py#L152 ?19:10
efriedor is there some reason we still need to enforce this lookup as being by name?19:12
mriedemi will enjoy whatever answer mikal gives you for a thing done 5 years ago19:12
mriedemoh,19:13
efriedYeah, anyone should feel free to answer this.19:13
mriedemand if/when he replies, make sure it's related to powervm :)19:14
efriedmriedem Since you asked, I was working on filling in a gap we missed in the in-tree impl: the only thing we populate in the InstanceInfo is the state.19:15
efriedWhile looking, I found that docstring and recognized that we've never looked that guy up by name (even in the out-of-tree driver).19:15
*** lucasxu has quit IRC19:15
efriedSo I went to figure out whether there was any basis for that edict.19:15
mriedemremove ", by name (not ID!)" from the docstring19:16
mriedemit's always passed an Instance object19:16
efriedDoing it.  Thanks.19:16
*** slaweq has joined #openstack-nova19:18
openstackgerritJay Pipes proposed openstack/nova master: placement project_id, user_id in PUT /allocations  https://review.openstack.org/46963419:19
openstackgerritJay Pipes proposed openstack/nova master: placement: Add GET /usages to placement API  https://review.openstack.org/46963519:19
openstackgerritJay Pipes proposed openstack/nova master: placement: use separate tables for projects/users  https://review.openstack.org/47064519:19
jaypipeszzzeek: \o/ ^19:20
openstackgerritChris Dent proposed openstack/nova master: [placement] Use util.extract_json in allocations handler  https://review.openstack.org/47106719:22
openstackgerritEric Fried proposed openstack/nova master: ComputeDriver.get_info not limited to inst name  https://review.openstack.org/47110619:24
efriedmriedem ^^  This is why I earn the big bucks.19:24
mriedemefried: it probably all ties back to the libvirt driver using this to get the guest info19:26
mriedemreturn conn.lookupByName(instance.name)19:26
mriedemmost things in the virt driver layer are libvirt-biased19:27
efriedmriedem Don't I know it.19:28
efriedmriedem The other thing I noticed is that the only two consumes of get_info (in manager.py) only ever use the state.  Makes me less panicky about hurrying to populate those other fields.19:31
efriedconsumers*19:31
*** esberglu has quit IRC19:31
*** esberglu_ has joined #openstack-nova19:33
*** lucasxu has joined #openstack-nova19:34
*** sbezverk has quit IRC19:39
*** eharney has quit IRC19:39
*** karimb has joined #openstack-nova19:43
*** cdent has quit IRC19:45
*** tbachman has quit IRC19:45
efriedjaypipes I presume the resource tracking stuff never plans to use get_info to figure out how much e.g. cpu/mem a VM is consuming - you're just going to record that at claim time and go with it?19:47
*** rcernin has joined #openstack-nova19:49
dansmithefried: we don't know how much memory a guest is consuming without introspection19:50
dansmithefried: and the amount of cpu it's using at some instantaneous point is not really an allocation thing19:50
efrieddansmith Okay, fair; but are there any plans for nova to actually use those InstanceInfo fields?19:51
dansmiththose...instanceinfo...fields...19:51
efriedPerhaps I'm missing some duck-typed dynamic usage of get_info19:51
dansmiththere are remnants of the extensible resource tracker and metric info in nova.. perhaps you mean that stuff?19:51
efrieddansmith https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L213-L217 <== those InstanceInfo fields19:52
efrieddansmith Perhaps I do mean that stuff.  What stuff do I perhaps mean?19:52
*** penick has joined #openstack-nova19:53
dansmithyeah, I _think_ that stuff is only used for choosing numa nodes19:53
*** eharney has joined #openstack-nova19:54
dansmithor maybe not even that, I don't even see that stuff considered in the scheduler19:54
efrieddansmith Does that happen in the nova tree?  I'm even grepping and not finding any consumption of those fields.19:54
*** fragatina has joined #openstack-nova19:55
dansmithefried: I think it's all within that file, and all for things like picking cpu topologies and stuff19:55
dansmithbut not a generic thing for reporting or choosing whole compute nodes, AFAIK19:56
dansmithdefinitely not the material of placement though19:56
efrieddansmith What I'm considering doing here is leaving my InstanceInfo so it just provides the state.  If anything ever gets written to consume any of the other fields, and assuming those code paths get hit in some tempest test, our CI will catch it (by raising AttributeError).19:56
efriedAnd at that time, we can figure out how best to implement those fields.  Cause there's a number of options, with different performance characteristics, one of which would be more appropriate than another depending on the actual usage.19:57
dansmithsounds reasonable19:58
mriedemhttps://review.openstack.org/#/c/133777/19:58
mriedemlooks like it was added for something...but very little context19:58
dansmithyeah19:58
cfriesen_weird python question....how is the "return" value from main() in cmd/manage.py expected to be consumed?19:59
cfriesen_it doesn't seem to have any effect on the process return value if run from the shell19:59
efriedRecently, too - but only relative to the aforementioned "by name (not ID!)" thing ;-)19:59
mriedemgoes back to https://github.com/openstack/nova/commit/19da125805eedbfcfd202abac4a90c57e6c538c420:02
mriedemhttps://github.com/openstack/nova/blob/c8e8b44ef27e49b3986659ee0cb6bd77b38430d8/nova/virt/libvirt_conn.py#L79720:02
*** pchavva has quit IRC20:02
mriedemso, it just always returned that other crap20:02
mriedembecause we could20:03
mriedembecause the libvirt domain api gave it to us20:03
*** salv-orlando has joined #openstack-nova20:03
mriedemhttps://github.com/openstack/nova/blob/c8e8b44ef27e49b3986659ee0cb6bd77b38430d8/nova/virt/xenapi/vm_utils.py#L68620:04
*** MVenesio has quit IRC20:05
efriedDig.  I'll go with the plan, then.  I'll just add a comment explaining it, so my posterity doesn't do a tim-the-tool-man like I did.20:05
mriedemefried: there is one place where something besides 'state' is used, and it's in the libvirt driver's _destroy() method20:06
mriedemchecking to see if the domain id changed20:06
mriedemunless there is another way to skin that id cat, we'd have to leave it,20:07
mriedembut otherwise, if it were me, i'd remove all of that other unused garbage20:07
mriedemfrom InstanceInfo20:07
*** liverpooler has quit IRC20:07
*** esberglu_ has quit IRC20:07
efriedmriedem Okay, I'll look into it.20:08
*** esberglu has joined #openstack-nova20:09
*** esberglu has quit IRC20:09
*** esberglu has joined #openstack-nova20:10
*** kfarr has joined #openstack-nova20:10
efriedesberglu Can you answer cfriesen_ above?  You figured out that the return values somehow mapped to exit codes?20:12
*** dtp has joined #openstack-nova20:14
cfriesen_efried: esberglu: it appears that the return code is being mapped to the process exit value.  it must be doing something clever.   In any case, I opened https://bugs.launchpad.net/nova/+bug/1695965 because the return code for "nova-manage db online_data_migrations" isn't very helpful.20:15
openstackLaunchpad bug 1695965 in OpenStack Compute (nova) ""nova-manage db online_data_migrations" exit code is strange" [Undecided,New]20:15
efriedcfriesen_ Cool, esberglu just made a change on a similar theme: https://review.openstack.org/#/c/467218/20:16
efriedcfriesen_ See bug https://bugs.launchpad.net/nova/+bug/169298220:16
openstackLaunchpad bug 1692982 in OpenStack Compute (nova) "Need --strict option for nova-manage cell_v2 discover_hosts" [Wishlist,Fix released] - Assigned to Eric Berglund (esberglu)20:16
*** jamesdenton has joined #openstack-nova20:17
*** tbachman has joined #openstack-nova20:18
cfriesen_efried: right.  In the case of online_data_migrations we really want to distinguish between three separate scenarios, which doesn't map nicely to binary exit values20:18
efriedcfriesen_ In your case, it's possible the answer might be, "code to the API, not the CLI" ;-)20:18
efriedBut I wouldn't be the one to advocate that.20:18
cfriesen_efried: heh...one nasty case is that the newton version of this command returns this "2 rows matched query pcidevice_online_data_migration, 0 migrated" even when there is no work left to do.20:19
cfriesen_outputs that, rather20:19
cfriesen_because the query is too generic.20:20
*** cheran has quit IRC20:20
*** slaweq has quit IRC20:21
*** tbachman has quit IRC20:21
efriedcfriesen_ FWIW, dansmith was the one in our corner on that --strict thing.  He had very useful feedback when the idea came up.20:21
*** lucasxu has quit IRC20:22
*** eharney has quit IRC20:23
cfriesen_dansmith: when you get a moment could you look at bug 1692982 ?20:24
openstackbug 1692982 in OpenStack Compute (nova) "Need --strict option for nova-manage cell_v2 discover_hosts" [Wishlist,Fix released] https://launchpad.net/bugs/1692982 - Assigned to Eric Berglund (esberglu)20:24
*** nicolasbock has quit IRC20:24
cfriesen_dansmith: whoops, I meant bug 169596520:25
openstackbug 1695965 in OpenStack Compute (nova) ""nova-manage db online_data_migrations" exit code is strange" [Undecided,New] https://launchpad.net/bugs/169596520:25
*** slaweq has joined #openstack-nova20:25
*** tbachman has joined #openstack-nova20:25
esbergluefried: cfriesen_: Sorry was off for a few there. I think I dropped for some of the convo. You still need anything from me?20:25
cfriesen_esberglu: just wondering if you knew offhand how the return value of "nova-manage" gets converted to process exit value20:26
*** dimtruck is now known as zz_dimtruck20:26
dansmithcfriesen_: we had migrations in the past that raised an exception when something wasn't new enough, but you needed to proceed when in order to get to the newer stuff, hence the zero return in that case20:26
dansmiththats why it was like that20:26
*** edmondsw has quit IRC20:26
dansmithbut I don't disagree with nailing it down as you describe20:26
cfriesen_dansmith: also, in Newton you can't even tell from the output logs whether or not you need to call it again, because of the PCI issue I showed above20:27
cfriesen_dansmith: the "count_all" is counting cases where there is no migration to be done20:27
*** kwathore has joined #openstack-nova20:27
esbergluefried: Not sure. I just took if granted based on other commands20:28
*** thorst has quit IRC20:28
*** kwathore_ has quit IRC20:28
*** thorst has joined #openstack-nova20:28
*** slaweq has quit IRC20:29
*** slaweq has joined #openstack-nova20:32
*** sbezverk has joined #openstack-nova20:33
*** thorst has quit IRC20:33
*** fragatina has quit IRC20:33
*** cleong has quit IRC20:35
*** slaweq has quit IRC20:36
*** fragatina has joined #openstack-nova20:37
*** clenimar has joined #openstack-nova20:39
cfriesen_dansmith: related issue: https://bugs.launchpad.net/nova/+bug/169599120:47
openstackLaunchpad bug 1695991 in OpenStack Compute (nova) ""nova-manage db online_data_migrations" doesn't report matched/migrated properly " [Undecided,New]20:47
*** rcernin has quit IRC20:47
*** lucasxu has joined #openstack-nova20:47
*** zz_dimtruck is now known as dimtruck20:48
*** thorst has joined #openstack-nova20:50
*** tbachman has quit IRC20:51
*** dave-mccowan has quit IRC20:54
mriedemdansmith: i think we've touched on this before, but since services are restricted to cell dbs, we don't have a global view of what a minimum compute version is in a deployment once we have multpile cells20:54
mriedemthinking of https://github.com/openstack/nova/blob/78c69f61ab186c3d3eb0fe5cc5114cc71f0fbdad/nova/objects/service.py#L42520:54
*** kfarr has quit IRC20:54
dansmithmriedem: I have a helper20:54
mriedemif we have an instance we can find the cell it's in and get the minimum compute for that cell, or get the service version for the compute the instance is on20:54
*** thorst has quit IRC20:54
*** liuqing has quit IRC20:55
cfriesen_how is Service._check_minimum_version() going to work with cells then?  check just the current cell?20:55
*** liuqing has joined #openstack-nova20:56
mriedemwell we could certainly iterate cells,20:56
mriedemfind the min for $service in all cells,20:56
mriedemand return the lowest20:56
dansmithmriedem: I'm sure I already wrote that helper.. looking20:56
mriedemi randomly think about this because we don't really test for it in our grenade tests20:57
mriedemsince grenade is so limited in scope to smoke tests20:57
mriedeme.g. https://github.com/openstack/nova/blob/78c69f61ab186c3d3eb0fe5cc5114cc71f0fbdad/nova/api/openstack/compute/servers.py#L496-L49920:57
mriedem^ won't be accurate with multiple cells20:58
mriedemi'll open a bug to track this20:58
dansmithand unless we had multiple real cells we wouldn't hit it anyway20:58
mriedemwe had == in ci?20:58
dansmithyeah20:59
mriedemi thought we were going to have the/a multinode job have 2 cells with 1 compute each?20:59
dansmithyeah but not one downlevel20:59
dansmithnot yet anyway20:59
mriedemsure, but we could with grenade20:59
mriedemeventually20:59
dansmitheventually20:59
mriedembut then we're stuck back with the grenade not running interesting tests issue21:00
dansmiththat helper doesn't appear to be in tree, but I know I wrote it.. might've been for a thing that we ended up not doing and so never committed it21:00
dansmithdon't we run smoke first and then full tempest post-upgrade?21:00
mriedemnope21:00
mriedemsmoke both ways21:00
sdagueno, it's smoke on both sides21:00
mriedemsmoke'em if you got'em21:01
dansmithokay21:01
sdaguehonestly, for the multi node, multi cell version testing, we should probably purpose build something for that. Because it feels like we want to be pushing very specific edges on test cases21:02
dansmithyou mean a specific list of tests?21:02
mriedemdansmith: https://bugs.launchpad.net/nova/+bug/169599321:02
openstackLaunchpad bug 1695993 in OpenStack Compute (nova) "supports_device_tagging check in /servers API won't work with multiple cells" [High,Triaged]21:02
mriedemdansmith: i think he means not grenade21:03
sdagueyeh, maybe not grenade either21:03
mriedemsdague: you're talking about the ansible idea from the PTG?21:03
*** fragatina has quit IRC21:03
dansmithhmm21:03
sdaguemriedem: I'm not honestly sure21:03
mriedemi remember you saying something like that once21:03
mriedemwhen i was monkeying with d-g and cells v2 setup21:03
sdagueit just feels like we might be better off building something that builds out the mixed cell version environment from scratch21:04
sdagueinstead of glomming too much onto the existing stuff21:04
mriedemmight have also been when talking about what rolling upgrades look like for cells v221:04
dansmithI'm not sure why21:04
mriedemwith doing api at the end21:04
mriedemsince devstack/grenade starts/upgrades API first?21:04
mriedemit had something to do with the upgrade order for api21:04
mriedemwe have notes from the BCN summit21:05
*** tbachman has joined #openstack-nova21:05
*** masuberu has joined #openstack-nova21:05
mriedemanyway, i'm not proposing we tackle testing mixed-level cells in ci right now21:05
*** masber has quit IRC21:06
dansmithmriedem: I will keep looking for my multicell service lookup patch in a couple dead end branches, but if I don't find it I'll just write it.. it's not hard21:06
mriedemsure - you want to take that bug?21:06
mriedemor just going to provide the helper?21:07
*** lucasxu has quit IRC21:07
dansmithmriedem: lemme write the helper and I'll survey the other uses... might be easy for me to just fix them uses too21:07
mriedemdansmith: here it is https://review.openstack.org/#/c/442787/1/nova/objects/service.py21:07
dansmithhaha21:08
dansmithno wonder I couldn't find it. that patch is dead to me21:08
mriedemi know21:08
dansmithlol21:08
mriedemi was looking at it on friday considering taking it over21:08
mriedemanyway, we should split that piece out into a separate change, and then the bug fix stuff on top probably21:09
dansmithI've been working on sql injection attacks at gerrit so I can reassign it to you21:09
dansmithyeah21:09
dansmithI shall21:09
*** jmlowe has quit IRC21:09
*** masuberu has quit IRC21:14
*** crushil has quit IRC21:14
*** slaweq has joined #openstack-nova21:21
*** burt has quit IRC21:23
*** thorst has joined #openstack-nova21:23
*** jwnx has quit IRC21:24
*** slaweq has quit IRC21:25
*** slaweq has joined #openstack-nova21:25
*** edmondsw has joined #openstack-nova21:28
edleafejaypipes: dansmith: cdent: Started a thread on the ML: http://lists.openstack.org/pipermail/openstack-dev/2017-June/117913.html21:28
jaypipesedleafe: thx ed21:29
*** marst has quit IRC21:29
*** crushil has joined #openstack-nova21:31
*** edmondsw has quit IRC21:32
*** penick has quit IRC21:34
*** baoli has quit IRC21:34
openstackgerritSteve Noyes proposed openstack/nova master: WIP: update live migration to use v3 cinder api  https://review.openstack.org/46398721:35
*** penick has joined #openstack-nova21:39
openstackgerritSteve Noyes proposed openstack/nova master: objects: Add old_attachment_id to BlockDeviceMapping  https://review.openstack.org/47114221:39
mriedemstvnoyes: that's not going to work21:40
*** crushil has quit IRC21:40
stvnoyesno?21:40
*** tidwellr has quit IRC21:40
mriedemprimarily because you haven't added a column to the block_device_mappings table21:40
mriedemhowever, we should talk to johnthetubaguy about this since i'd really hope we could avoid something like this21:41
stvnoyesI was following this pattern that did somethng about the same - https://review.openstack.org/#/c/437665/21:41
mriedemright but you missed https://review.openstack.org/#/c/437597/21:42
stvnoyesah ok. one of john's comments on the initial migrate review was for a place to keep the old attach id21:42
mriedemjohn garbutt?21:43
mriedemor griffith?21:43
stvnoyesah. ok (i am impressed that you remember all these things)21:43
stvnoyesgarbutt21:43
*** gouthamr has quit IRC21:43
mriedemhmm, i'd like to avoid an old/new attachment id column in the bdms table/object21:43
mriedeminitial thought if we have 2 bdms, one for the source host attachment and one for the dest host attachment21:44
mriedembut that might break other things21:44
stvnoyesdid you have another idea? I originally tried to do without i, by creating a new bdm. but had problems down in libvirt driver distinguishing between what was source and destination21:44
stvnoyeswithout 'it'21:44
mriedemright that's one of the issues21:44
mriedemthe other problem is we have a BlockDeviceMapping.get_by_volume_and_instance method21:45
stvnoyesso after going thru it a number of times, this came out to be less new code21:45
mriedemwhich expects to get 0 or 1 results21:45
jgriffithany way to do a "new/old" bdm?21:46
mriedemprobably not in a way that's any less bad than a single bdm with new/old attachmnets21:46
mriedem*attachments21:46
mriedemand like i said, a bunch of the code assumes volume id + instance id is uniqueu21:46
mriedem*unique even21:47
*** egonzalez has quit IRC21:47
* mriedem finds a bib for the vomit21:47
* jgriffith hands mriedem a bucket instead21:47
stvnoyesthe other stringe thing I found is that when you have a connector present in the bd, the attachment_id is in the bdm twice (bdm.attachment_id, and in the connection_info string)21:48
stvnoyesthat's not terrific21:48
mriedemstvnoyes: long-term i think we want to stop setting connection_info21:49
mriedemwhen we have attachment_id21:49
mriedembecause cinder will store the connection_info for us21:49
mriedemthat probably happens in the attach patch21:49
jgriffithstvnoyes that's because the attachment_id comes form that21:49
stvnoyesthat seems like a good idea (later)21:49
mriedemnova wants to not store the connection_info21:50
mriedemb/c it can be stale21:50
*** sree has joined #openstack-nova21:50
mriedemand, well, we just don't want to duplicate all of this state modeling in two services21:50
stvnoyesso how would you like to resolve whether we want to go with  old_attachment_id? talk with John Garbutt? I can update the rv in the meantime with that missing change.21:51
mriedemstvnoyes: yeah i'd say do what you're doing now, fix the db api thing, and then we talk to john to see if he had other idea21:53
mriedem*ideas21:53
*** catintheroof has quit IRC21:53
stvnoyesok cool21:53
mriedemi don't like it but it's probably the lesser of many evils21:53
mriedemunless of course dansmith has bright ideas he'd like to share about BDMs modeling and multi-attach21:53
*** gomarivera has quit IRC21:53
mriedem...21:53
*** slaweq has quit IRC21:53
dansmithbarf21:54
*** jamesdenton has quit IRC21:54
jgriffithYou guys running some kinda vomitorium in here?21:54
*** sree has quit IRC21:54
mriedemvomiting on oneself is a common reaction to things in this channel21:54
jgriffithsmall world21:55
jgriffithseems were all more alike than we thought21:55
*** takashin has joined #openstack-nova21:55
*** slaweq has joined #openstack-nova21:55
stvnoyesI found that libvirt driver needs to know what bdm is what and getting that info down in there was incredibly ugly. This way libvirt driver hardly needs to change at all.21:57
*** gomarivera has joined #openstack-nova21:57
mriedemold_attachment_id vs attachment_id also gets weird,21:58
mriedembecause we have to assume "old" means source host21:59
mriedemand ties it to migrations in the bdm object21:59
mriedembut, we have old/new flavors too for the same thing (resizes)21:59
stvnoyesusing john Gr's attach code, I did get this working ok. I could migrate back and forth successfully21:59
mriedemactually, flavors gives me an idea here22:00
mriedemnova.objects.Migration has an old_instance_type_id and new_instance_type_id field22:00
mriedemi'm wondering if we can store the old/new attachment id information in the migration object22:00
*** slaweq has quit IRC22:00
mriedemhowever, an instance has 1 flavor,22:01
mriedembut possibly many volume attachments22:01
mriedemso it'd have to be list22:01
stvnoyeshmmm, ok i can take a look at that. is that object persisted and available to both hosts?22:01
mriedemyes22:01
mriedemthat would be a cleaner interface if we can make it work22:02
*** gomarivera has quit IRC22:02
stvnoyesagree, i'm not a big fan of putting new fields in the bdm.22:02
stvnoyesalso, a migrate object seems more appropriate.22:02
stvnoyesi wasn't aware of that object. new things to learn.22:03
openstackgerritFelipe Monteiro proposed openstack/nova-specs master: Patrole (RBAC) Keystone Gating  https://review.openstack.org/47114522:04
*** thorst has quit IRC22:08
*** thorst has joined #openstack-nova22:09
openstackgerritEric Fried proposed openstack/nova master: Trim the fat from InstanceInfo  https://review.openstack.org/47114622:10
mriedemcfriesen_: your release notes are fixed22:10
efriedmriedem ^^22:11
mriedemefried: ok probably for tomorrow at this rate22:11
cfriesen_mriedem: sweet, thanks for the recheck too22:11
efriedmriedem Oh, for sure.22:11
*** felipemonteiro has quit IRC22:12
*** thorst has quit IRC22:13
*** jamielennox|away is now known as jamielennox22:13
*** mtanino has quit IRC22:14
*** gomarivera has joined #openstack-nova22:15
*** kwathore has quit IRC22:16
*** kwathore has joined #openstack-nova22:16
*** salv-orlando has quit IRC22:19
*** gouthamr has joined #openstack-nova22:23
openstackgerritEric Fried proposed openstack/nova master: Trim the fat from InstanceInfo  https://review.openstack.org/47114622:25
*** awaugama has quit IRC22:27
*** penick has quit IRC22:34
*** thorst has joined #openstack-nova22:42
*** thorst has quit IRC22:42
*** mriedem has quit IRC22:43
*** gouthamr has quit IRC22:46
*** lyan has quit IRC22:49
*** gouthamr has joined #openstack-nova22:49
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: Uncap sphinx version  https://review.openstack.org/47115322:49
*** jwcroppe has quit IRC22:53
*** gomarivera has quit IRC22:54
*** gomarivera has joined #openstack-nova22:55
*** thorst has joined #openstack-nova22:55
*** thorst has quit IRC22:55
*** slaweq has joined #openstack-nova22:56
*** gomarivera has quit IRC22:57
*** gomarivera has joined #openstack-nova22:57
*** gomarivera has quit IRC22:58
*** gomarivera has joined #openstack-nova22:59
*** slaweq has quit IRC23:01
*** roeyc has joined #openstack-nova23:02
*** gomarivera has quit IRC23:03
*** roeyc has quit IRC23:03
*** mriedem has joined #openstack-nova23:11
*** catintheroof has joined #openstack-nova23:13
*** tbachman has quit IRC23:18
*** karimb has quit IRC23:18
*** jwcroppe has joined #openstack-nova23:24
*** tbachman_ has joined #openstack-nova23:25
*** marst has joined #openstack-nova23:25
*** sdague has quit IRC23:26
*** thorst has joined #openstack-nova23:26
mriedemcfriesen_: why does anything in https://review.openstack.org/#/c/435141/10/novaclient/tests/unit/test_shell.py need to change?23:26
mriedemi.e. what's up with all of the list -> flavor-list changes?23:26
*** jwcroppe has quit IRC23:28
mriedemi see in the commit message now,23:28
mriedembut still, seems we can set a default api_version in whatever fixture is used for those tests so you don't need to change them all23:28
*** jamielennox is now known as jamielennox|away23:30
openstackgerritMerged openstack/nova master: Completed implementation of instance diagnostics for Xen  https://review.openstack.org/39961323:33
*** dtp has quit IRC23:36
*** Vek has quit IRC23:38
*** thorst has quit IRC23:42
*** gouthamr has quit IRC23:43
*** jamielennox|away is now known as jamielennox23:46
*** tbachman_ has quit IRC23:49
*** ckopper has quit IRC23:50
*** gouthamr has joined #openstack-nova23:52
mriedemartom: some questions inline before i approve https://review.openstack.org/#/c/465205/23:55
artommriedem, man of your word. Much thanks, let me take a look23:55
*** baoli has joined #openstack-nova23:55
*** hongbin has quit IRC23:56
*** slaweq has joined #openstack-nova23:57
mriedemi have to do something during girls gymnastics class (besides watch and support of course)23:57
artomWell crap, now I feel guilty for taking away father-daughter time23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!