Friday, 2017-01-13

*** gouthamr has quit IRC00:02
*** jamesdenton has joined #openstack-nova00:04
*** vladikr has quit IRC00:04
*** nic has quit IRC00:06
*** sdague has quit IRC00:12
*** jamesdenton has quit IRC00:15
*** Swami_ has quit IRC00:17
*** Swami_ has joined #openstack-nova00:17
*** tjones has joined #openstack-nova00:18
*** hongbin has quit IRC00:19
*** Swami__ has joined #openstack-nova00:23
*** gouthamr has joined #openstack-nova00:26
*** Swami has quit IRC00:26
*** kylek3h has joined #openstack-nova00:29
*** chyka has quit IRC00:30
*** mlavalle has quit IRC00:34
*** Guest29822 has quit IRC00:37
*** thorst_ has joined #openstack-nova00:38
*** jose-phillips has quit IRC00:40
*** thorst_ has quit IRC00:43
*** gouthamr has quit IRC00:47
*** mtani____ has quit IRC00:47
*** hoangcx has joined #openstack-nova00:48
*** hfu has joined #openstack-nova00:48
*** tjones has quit IRC00:48
*** tjones has joined #openstack-nova00:50
*** tjones has left #openstack-nova00:51
*** tjones has joined #openstack-nova00:51
*** dtp has quit IRC00:56
*** thorst_ has joined #openstack-nova00:59
*** tjones has quit IRC01:00
*** thorst_ has quit IRC01:03
*** thorst_ has joined #openstack-nova01:04
*** thorst_ has quit IRC01:04
*** tjones has joined #openstack-nova01:05
*** hamzy has joined #openstack-nova01:12
*** kevinz has joined #openstack-nova01:13
*** tjones has quit IRC01:13
*** rfolco has quit IRC01:18
*** zayro has joined #openstack-nova01:19
*** salv-orlando has joined #openstack-nova01:19
*** salv-orlando has quit IRC01:24
*** tjones has joined #openstack-nova01:26
*** baoli has joined #openstack-nova01:29
*** artom has quit IRC01:32
*** artom has joined #openstack-nova01:32
*** artom has quit IRC01:32
*** artom has joined #openstack-nova01:33
*** kaisers2 has joined #openstack-nova01:36
*** kaisers3 has joined #openstack-nova01:36
*** kaisers1 has quit IRC01:37
*** kaisers has quit IRC01:38
*** rfolco has joined #openstack-nova01:39
*** Sukhdev has quit IRC01:40
*** gongysh has joined #openstack-nova01:40
*** tjones has quit IRC01:43
*** weshay has quit IRC01:47
*** weshay has joined #openstack-nova01:49
*** yuntongjin has joined #openstack-nova01:50
*** baoli has quit IRC01:54
*** baoli has joined #openstack-nova01:54
*** dimtruck is now known as zz_dimtruck01:58
*** zz_dimtruck is now known as dimtruck01:59
*** Swami_ has quit IRC02:00
*** Swami__ has quit IRC02:00
*** tjones has joined #openstack-nova02:00
*** thorst_ has joined #openstack-nova02:05
*** rfolco has quit IRC02:06
*** thorst_ has quit IRC02:08
*** markvoelker has quit IRC02:09
*** browne has quit IRC02:09
openstackgerritZhenyu Zheng proposed openstack/nova: Don't delete neutron port when attach failed  https://review.openstack.org/40356802:12
*** kaisers has joined #openstack-nova02:17
*** dave-mccowan has joined #openstack-nova02:17
*** ducttape_ has joined #openstack-nova02:19
*** kaisers_ has quit IRC02:20
alex_xuKevin_Zheng: just commented on https://review.openstack.org/#/c/419256/7, we still can make it simpler02:24
*** hfu has quit IRC02:24
Kevin_ZhengThanks02:25
*** hfu has joined #openstack-nova02:25
*** hfu has quit IRC02:27
*** raunak has quit IRC02:28
*** hfu has joined #openstack-nova02:29
*** smatzek has joined #openstack-nova02:30
*** Jeffrey4l_ has joined #openstack-nova02:30
*** gouthamr has joined #openstack-nova02:33
*** Jeffrey4l has quit IRC02:34
*** tjones has quit IRC02:37
*** hfu has quit IRC02:41
*** hfu has joined #openstack-nova02:42
*** gouthamr has quit IRC02:43
*** gouthamr has joined #openstack-nova02:45
openstackgerritJeremy Liu proposed openstack/python-novaclient: Enable coverage report in console output  https://review.openstack.org/41972302:45
*** unicell has quit IRC02:46
*** tovin07 has joined #openstack-nova02:46
*** dimtruck is now known as zz_dimtruck02:48
*** mriedem has joined #openstack-nova02:49
*** smatzek has quit IRC02:49
*** tbachman has quit IRC02:50
*** ijw_ has joined #openstack-nova02:51
*** amotoki has quit IRC02:53
*** baoli has quit IRC02:53
openstackgerritAnusha Unnam proposed openstack/nova: Add keypairs field to InstancePayload  https://review.openstack.org/41973002:54
*** ijw has quit IRC02:54
*** hongbin has joined #openstack-nova02:59
*** ducttape_ has quit IRC02:59
*** ducttape_ has joined #openstack-nova03:00
*** hongbin_ has joined #openstack-nova03:03
*** hongbin has quit IRC03:03
openstackgerritAnusha Unnam proposed openstack/nova: Add keypairs field to InstancePayload  https://review.openstack.org/41973003:04
*** ducttape_ has quit IRC03:04
*** baoli has joined #openstack-nova03:05
*** diga has joined #openstack-nova03:15
*** liuyulong_ is now known as liuyulong03:16
*** ferest has joined #openstack-nova03:17
*** ducttape_ has joined #openstack-nova03:18
*** timello has quit IRC03:19
*** Jeffrey4l_ has quit IRC03:19
*** salv-orlando has joined #openstack-nova03:21
*** Jeffrey4l_ has joined #openstack-nova03:21
*** timello has joined #openstack-nova03:21
*** salv-orlando has quit IRC03:25
*** gcb has quit IRC03:26
*** raunak has joined #openstack-nova03:27
*** zz_dimtruck is now known as dimtruck03:27
*** ferest has left #openstack-nova03:28
mriedemedleafe: have you ever seen this failure in a functional test run for placement? http://logs.openstack.org/60/416760/1/check/gate-nova-tox-db-functional-ubuntu-xenial/f192837/console.html#_2017-01-12_15_22_00_60967803:28
openstackgerritzhaolihui proposed openstack/nova: Fix keypair quota error  https://review.openstack.org/40352903:28
*** gongysh has quit IRC03:29
mriedemseems to be random03:30
*** yamahata has quit IRC03:32
openstackgerritZhenyu Zheng proposed openstack/nova: [poc] add some code to reject known bad filters.  https://review.openstack.org/41925603:33
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP][py35] Trying to get cinder+rally working  https://review.openstack.org/41947603:33
*** thorst_ has joined #openstack-nova03:35
*** thorst_ has quit IRC03:37
mriedemnvm i think it's a problem in that change actually03:37
mriedemi'm probably missing some backport03:37
*** baoli has quit IRC03:39
*** baoli has joined #openstack-nova03:39
*** ducttape_ has quit IRC03:42
*** ducttape_ has joined #openstack-nova03:42
*** ijw_ has quit IRC03:42
*** ducttape_ has quit IRC03:44
*** ducttape_ has joined #openstack-nova03:44
*** baoli has quit IRC03:45
*** yuntongjin has quit IRC03:46
alex_xujohnthetubaguy: gmann, Kevin_Zheng puts two ideas into single patch https://review.openstack.org/419256, probably enough to take look at how those looks like in the code.03:46
*** mriedem has quit IRC03:49
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP][py35] Trying to get cinder+rally working  https://review.openstack.org/41947603:53
*** tjones has joined #openstack-nova03:53
*** links has joined #openstack-nova03:54
gmannalex_xu: johnthetubaguy  Kevin_Zheng  thanks, while giving try i got a way to reject known property from schema itself03:55
*** raunak has quit IRC03:55
gmannalex_xu: johnthetubaguy  Kevin_Zheng we can do with 'not' propoerty, i commented on  this about how - https://review.openstack.org/#/c/408571/3803:55
*** raunak has joined #openstack-nova03:56
gmannactually in draft3 formatter it was 'disallow' and in 4 it is 'not'03:56
*** mdnadeem has joined #openstack-nova03:56
gmannhttps://github.com/json-schema/json-schema/wiki/anyOf,-allOf,-oneOf,-not#not03:56
gmannhttps://github.com/json-schema/json-schema/wiki/anyOf,-allOf,-oneOf,-not#replacing-schemas-in-disallow03:56
gmannso basically with empty object with not like- 'not': {} we can reject all values of given element03:56
alex_xugmann: what kind of error message we can get?03:57
digagibi: Hi03:58
gmannalex_xu:  yea that is nice point03:58
gmannalex_xu: it will like Invalid input for field/attribute system_metadata. Value: . {} is not allowed for 'mymeta'03:58
gmannwhich we might need to modify seems03:58
Kevin_ZhengActually I like the format way better04:00
alex_xugmann: if we can get a correct message that will be great, not sure there is way to change that message04:00
Kevin_Zhengit will be harder than the new format thing I think04:01
alex_xuyea, i'm also worry about that04:01
gmannalex_xu:  yea error message is issue but let me check if we can easily modify that04:01
Kevin_Zhengthe mssage is below the one I changed for the current way04:02
Kevin_Zhengmaybe hard to identify the "not" thing because it might be a validation error?04:02
alex_xugmann: really appreciate that :)04:02
Kevin_ZhengVallidationError04:03
gmannKevin_Zheng:  yea it will be validation error from json itself04:03
*** gouthamr has quit IRC04:04
Kevin_Zhenggmann: yeah, but other also raise ValidationError and the msg was generated. So we have to identify the case of this04:04
Kevin_Zhengmaybe harder than the current way04:04
gmannKevin_Zheng:  yea, that is pain point.04:05
* gmann checking..04:05
*** ducttape_ has quit IRC04:08
*** ducttape_ has joined #openstack-nova04:09
*** kiran-r has joined #openstack-nova04:09
*** ducttape_ has quit IRC04:11
*** ducttape_ has joined #openstack-nova04:12
*** tjones has left #openstack-nova04:13
gmannalex_xu: Kevin_Zheng  yea we have to add hack for that which kindda new overridden code on top of json schema. That is what new formatter way.04:14
*** psachin has joined #openstack-nova04:14
*** nicolasbock has quit IRC04:16
*** armax has quit IRC04:17
*** ducttape_ has quit IRC04:18
*** liverpooler has quit IRC04:21
*** yuntongjin has joined #openstack-nova04:21
*** dikonoor has joined #openstack-nova04:24
*** pbandark has joined #openstack-nova04:24
*** manasm has joined #openstack-nova04:27
*** sree has joined #openstack-nova04:30
*** sree has quit IRC04:30
*** sree has joined #openstack-nova04:30
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP][py35] Trying to get cinder+rally working  https://review.openstack.org/41947604:31
gmannalex_xu: Kevin_Zheng i think formatter is ok which is much easy, how about showing few schema example with new 'format' also in https://review.openstack.org/#/c/419256/704:36
*** phuongnh has joined #openstack-nova04:39
*** baoli has joined #openstack-nova04:42
*** tojuvone has quit IRC04:44
*** karthiks has joined #openstack-nova04:44
*** amotoki has joined #openstack-nova04:46
*** baoli has quit IRC04:47
*** markvoelker has joined #openstack-nova04:52
*** kaisers has quit IRC04:54
*** larainema has quit IRC04:56
*** masber has joined #openstack-nova04:57
*** diga has quit IRC05:01
*** hongbin_ has quit IRC05:02
*** tbachman has joined #openstack-nova05:03
*** Sukhdev has joined #openstack-nova05:07
*** udesale has joined #openstack-nova05:08
*** ratailor has joined #openstack-nova05:08
*** tbachman has quit IRC05:09
*** tlian has quit IRC05:10
*** dimtruck is now known as zz_dimtruck05:11
*** unicell has joined #openstack-nova05:12
*** tbachman has joined #openstack-nova05:14
*** sudipto has joined #openstack-nova05:18
*** sudipto_ has joined #openstack-nova05:18
*** prateek has joined #openstack-nova05:19
openstackgerritZhenyu Zheng proposed openstack/nova: [poc] add some code to reject known bad filters.  https://review.openstack.org/41925605:19
Kevin_Zhenggmann: thanks, and I replied your comment, and I just came up with the idea to also assert the error message in one test, will add it later05:21
*** markvoelker_ has joined #openstack-nova05:22
gmannKevin_Zheng: i see, nice. thanks test case will be nice there05:24
*** markvoelker has quit IRC05:26
*** dave-mccowan has quit IRC05:27
*** gongysh has joined #openstack-nova05:30
*** larainema has joined #openstack-nova05:34
*** thorst_ has joined #openstack-nova05:37
*** adisky_ has joined #openstack-nova05:38
*** thorst_ has quit IRC05:42
*** baoli has joined #openstack-nova05:44
*** karthiks has quit IRC05:46
*** hoangcx has quit IRC05:47
*** tbachman has quit IRC05:47
*** baoli has quit IRC05:48
*** sacharya has quit IRC05:50
*** kaisers has joined #openstack-nova05:55
*** tbachman has joined #openstack-nova05:57
openstackgerritzhaolihui proposed openstack/nova: Fix keypair quota error  https://review.openstack.org/40352905:57
*** karthiks has joined #openstack-nova05:59
*** lpetrut has joined #openstack-nova05:59
*** salv-orlando has joined #openstack-nova06:07
*** hoangcx has joined #openstack-nova06:10
*** raunak has quit IRC06:13
*** tbachman has quit IRC06:14
*** tbachman has joined #openstack-nova06:14
*** swamireddy has quit IRC06:15
*** dikonoor has quit IRC06:16
openstackgerritJeremy Liu proposed openstack/python-novaclient: Enable coverage report in console output  https://review.openstack.org/41972306:16
*** hoangcx has quit IRC06:19
*** Kevin_Zheng has quit IRC06:20
*** Kevin_Zheng has joined #openstack-nova06:20
*** jwcroppe has quit IRC06:23
*** tojuvone has joined #openstack-nova06:25
*** oomichi has quit IRC06:25
*** oomichi has joined #openstack-nova06:26
*** tbachman has quit IRC06:26
*** phuongnh has quit IRC06:27
*** phuongnh has joined #openstack-nova06:27
*** oomichi has quit IRC06:30
*** oomichi has joined #openstack-nova06:31
openstackgerritIstvan Imre proposed openstack/python-novaclient: Pass relevant parameters to Token based authentication  https://review.openstack.org/41944106:35
*** kiran-r has quit IRC06:35
*** manasm has quit IRC06:36
*** oomichi has quit IRC06:38
*** kaisers has quit IRC06:40
*** oomichi has joined #openstack-nova06:40
*** mjura has joined #openstack-nova06:42
*** salv-orlando has quit IRC06:43
*** ijw_ has joined #openstack-nova06:45
*** lpetrut has quit IRC06:50
*** sacharya has joined #openstack-nova06:51
*** yamahata has joined #openstack-nova06:51
*** oomichi has quit IRC06:55
openstackgerritZhenyu Zheng proposed openstack/nova: [poc] add some code to reject known bad filters.  https://review.openstack.org/41925606:56
*** hfu has quit IRC06:56
*** sacharya has quit IRC06:56
Kevin_Zhenggmann, alex_xu tests added ^06:57
*** gongysh has quit IRC06:57
*** oomichi has joined #openstack-nova06:57
*** avolkov has joined #openstack-nova06:58
*** manasm has joined #openstack-nova07:00
*** hfu has joined #openstack-nova07:07
*** hfu has quit IRC07:11
*** tesseract has joined #openstack-nova07:11
*** hfu has joined #openstack-nova07:12
*** nkrinner_afk is now known as nkrinner07:15
*** andreas_s has joined #openstack-nova07:18
*** moshele has joined #openstack-nova07:18
*** fengzhr has quit IRC07:20
*** fengzhr has joined #openstack-nova07:23
*** voelzmo has joined #openstack-nova07:23
*** salv-orlando has joined #openstack-nova07:27
*** Kevin_Zheng has quit IRC07:28
*** Vek has quit IRC07:28
*** csatari has quit IRC07:28
*** fmccrthy has quit IRC07:28
*** auggy has quit IRC07:28
*** nikhil has quit IRC07:28
*** Oku_OSz has quit IRC07:28
*** Vek has joined #openstack-nova07:28
*** markmcclain has quit IRC07:29
*** moshele has quit IRC07:29
*** adisky_ has quit IRC07:29
*** tomhambleton has quit IRC07:29
*** TheJulia has quit IRC07:29
*** rpodolyaka has quit IRC07:30
*** tr3buchet has quit IRC07:30
*** yonglihe has quit IRC07:30
*** dhellmann has quit IRC07:30
*** rmcadams has quit IRC07:30
*** mwhahaha has quit IRC07:30
*** Hazelesque has quit IRC07:30
*** sridharg has joined #openstack-nova07:31
*** voelzmo has quit IRC07:31
*** Hazelesque has joined #openstack-nova07:31
*** mwhahaha has joined #openstack-nova07:31
*** nikhil has joined #openstack-nova07:31
*** rmcadams has joined #openstack-nova07:31
*** tomhambleton has joined #openstack-nova07:32
*** yonglihe has joined #openstack-nova07:32
*** rpodolyaka has joined #openstack-nova07:33
*** TheJulia has joined #openstack-nova07:34
*** markmcclain has joined #openstack-nova07:34
*** fmccrthy has joined #openstack-nova07:34
*** auggy has joined #openstack-nova07:34
*** Oku_OS-away has joined #openstack-nova07:35
*** dhellmann has joined #openstack-nova07:35
*** adisky_ has joined #openstack-nova07:36
*** tr3buchet has joined #openstack-nova07:36
*** kaisers has joined #openstack-nova07:36
*** Kevin_Zheng has joined #openstack-nova07:36
*** csatari has joined #openstack-nova07:37
*** thorst_ has joined #openstack-nova07:39
*** voelzmo has joined #openstack-nova07:41
*** thorst_ has quit IRC07:43
*** satyar has joined #openstack-nova07:46
*** sahid has joined #openstack-nova07:48
*** tiendc has joined #openstack-nova07:50
*** markus_z has joined #openstack-nova07:51
*** phuongnh has quit IRC07:51
*** fmccrthy has quit IRC07:57
*** fmccrthy has joined #openstack-nova07:57
*** nherciu has joined #openstack-nova07:58
*** manasm has quit IRC07:59
*** Sukhdev has quit IRC08:00
snikitinalex_xu: Hi Alex! good point about upgrade problem in diagnostics api. You described case when we have new api and old compute, right? I think we can handle this case in view class https://review.openstack.org/#/c/355540/18/nova/api/openstack/compute/views/server_diagnostics.py08:04
snikitinAs response of new micriversion is standartized we must return all diagnostics fields. But as we noted in spec and docs, some of this fields may be 'None' because some hypervisors can't provides some kinf in information.08:04
snikitinIn our upgrade case we can try to fill diagnostics fields by information we got from old hypervisor drivers (most of fields in old and new diagnostics objects are equal) and left other fields empty.  And of course we should log warning something like "Your compute is to old so data in diagnostics response may be incomplete" . What do you think?08:04
snikitin*some kind of information08:04
*** aloga has quit IRC08:05
*** aloga has joined #openstack-nova08:05
alex_xusnikitin: emm....i feel it is ok08:08
snikitinalex_xu, good! ty08:09
alex_xusnikitin: i'm thinking whether it makes the API behaviour different in the upgrade08:09
alex_xusnikitin: but yes, empty is valid value08:10
openstackgerritIstvan Imre proposed openstack/python-novaclient: Pass relevant parameters to Token based authentication  https://review.openstack.org/41944108:10
alex_xuwe didn't break any API contract08:10
snikitinokay08:10
*** manasm has joined #openstack-nova08:15
*** ralonsoh has joined #openstack-nova08:19
*** hfu has quit IRC08:22
*** hfu has joined #openstack-nova08:26
*** tuanluong has joined #openstack-nova08:28
*** manasm has quit IRC08:34
*** kaisers has quit IRC08:41
*** zayro has quit IRC08:41
*** salv-orlando has quit IRC08:43
*** salv-orlando has joined #openstack-nova08:44
*** manasm has joined #openstack-nova08:47
*** timello has quit IRC08:48
*** timello has joined #openstack-nova08:51
*** ri0 has joined #openstack-nova08:51
*** burt has quit IRC08:52
*** phuongnh has joined #openstack-nova08:55
*** burt has joined #openstack-nova08:56
*** zzzeek has quit IRC09:00
*** zzzeek has joined #openstack-nova09:00
*** tovin07_ has quit IRC09:02
*** ratailor has quit IRC09:04
*** mnestratov has joined #openstack-nova09:04
*** ociuhandu has joined #openstack-nova09:05
*** mnestratov|2 has quit IRC09:07
*** hfu has quit IRC09:07
*** ratailor has joined #openstack-nova09:08
*** hfu has joined #openstack-nova09:08
*** tiendc has quit IRC09:08
*** amotoki has quit IRC09:18
*** rmart04 has joined #openstack-nova09:18
*** ri0 has quit IRC09:19
*** yuntongjin has quit IRC09:19
*** karthiks has quit IRC09:19
*** ijw_ has quit IRC09:20
*** tiendc has joined #openstack-nova09:22
*** yassine has joined #openstack-nova09:23
*** yassine is now known as Guest6001509:23
*** rmart04_ has joined #openstack-nova09:25
*** mvk has quit IRC09:25
*** derekh has joined #openstack-nova09:25
*** dansmith has quit IRC09:27
*** dansmith has joined #openstack-nova09:27
*** rmart04 has quit IRC09:27
*** rmart04_ is now known as rmart0409:27
*** dansmith is now known as Guest4575909:27
*** satyar has quit IRC09:29
*** voelzmo has quit IRC09:29
*** sahid has quit IRC09:33
*** sahid has joined #openstack-nova09:35
*** tovin07 has quit IRC09:35
*** karthiks has joined #openstack-nova09:36
*** amotoki has joined #openstack-nova09:36
*** hfu has quit IRC09:38
*** sahid has quit IRC09:41
*** sahid has joined #openstack-nova09:41
*** voelzmo has joined #openstack-nova09:50
*** ayogi has joined #openstack-nova09:51
*** amoralej has joined #openstack-nova09:53
*** mvk has joined #openstack-nova09:56
*** sheel has joined #openstack-nova09:56
amoralejhi, i have a question about cellsv2 in ocata09:57
amoralejwhen initially deploying nova, we create cell0 and map the registered compute nodes using simple_cell_setup09:57
amoralejbut, if i add new compute nodes, are they automatically added or do i need to run nova-manage manually to map them?09:58
*** karthiks has quit IRC10:02
*** mnestratov|2 has joined #openstack-nova10:06
*** voelzmo has quit IRC10:07
*** lpetrut has joined #openstack-nova10:08
*** kevinz has quit IRC10:08
*** tuanluong has quit IRC10:09
*** mnestratov has quit IRC10:09
*** tuanluong has joined #openstack-nova10:10
*** tuanluong has quit IRC10:11
*** ratailor has quit IRC10:12
*** ratailor has joined #openstack-nova10:16
johnthetubaguyamoralej: I think this is the command you want: https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L136710:17
amoralejthanks johnthetubaguy10:17
*** JoseMello has joined #openstack-nova10:18
*** openstackgerrit has quit IRC10:18
*** salv-orl_ has joined #openstack-nova10:23
*** salv-orlando has quit IRC10:25
*** voelzmo has joined #openstack-nova10:26
*** cdent has joined #openstack-nova10:31
Kevin_Zhengjohnthetubaguy: hi, I update the code here: https://review.openstack.org/#/c/419256/10:31
Kevin_Zhengit contains two approach, could you have a look?10:31
Kevin_Zhengthe reject bad param thing :)10:31
*** BobBall_AWOL is now known as BobBall10:33
*** yamahata has quit IRC10:33
*** tiendc has quit IRC10:35
*** kaisers has joined #openstack-nova10:36
*** ayogi has quit IRC10:41
*** ayogi has joined #openstack-nova10:41
*** openstackgerrit has joined #openstack-nova10:44
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP][py35] Trying to get cinder+rally working  https://review.openstack.org/41947610:44
*** phuongnh has quit IRC10:46
*** ayogi has quit IRC10:48
*** ayogi has joined #openstack-nova10:48
*** udesale has quit IRC11:01
*** sudipto_ has quit IRC11:05
*** sudipto has quit IRC11:05
openstackgerritBalazs Gibizer proposed openstack/nova: handle uninited fields in notification payload  https://review.openstack.org/41585711:06
mdboothlyarwood: Any reason you split https://review.openstack.org/#/c/402163/ and its predecessor into separate patches?11:10
*** gszasz has joined #openstack-nova11:11
*** owalsh has joined #openstack-nova11:11
openstackgerritBalazs Gibizer proposed openstack/nova: handle uninited fields in notification payload  https://review.openstack.org/41585711:12
lyarwoodmdbooth: not that I can think of, happy to squash them togther tbh11:14
*** ijw has joined #openstack-nova11:14
mdboothlyarwood: Got a bit of a narrative going on there, btw. I asked you for an additional test, but then said you don't seem to need to parse filesystem at all.11:15
mdboothIn which case you wouldn't need a test, I guess.11:15
mdboothThe code which consumes this already assumes it's a disk, not a filesystem11:15
lyarwoodmdbooth: hmm do we not even call this when using lxc?11:18
*** ijw has quit IRC11:18
mdboothlyarwood: I don't know, but it's already busted in the case I think11:18
lyarwoodmdbooth: true11:19
mdboothhttps://review.openstack.org/#/c/402163/10/nova/virt/libvirt/guest.py11:19
mdboothnode = doc.find("./devices/disk/target[@dev='%s'].." % target_device)11:19
mdboothLine 31111:19
mdboothDo we support filesystems as volumes?11:20
*** manasm has quit IRC11:20
mdboothIf no, I think we don't worry about it for the moment11:20
mdboothMaybe leave a comment11:20
*** manasm has joined #openstack-nova11:21
lyarwoodmdbooth: I'm not sure tbh, the lookup hasn't changed here so if we do it's already broken11:21
mdboothMy thinking too11:21
mdboothI suspect we don't11:22
lyarwoodwe do as eph storage right?11:23
mdboothI'd have to check. Don't recall off the top of my head.11:25
mdbootheph is local, though, so completely different.11:25
*** smatzek has joined #openstack-nova11:26
mdboothlyarwood: Are you working on this series while I'm reviewing, btw?11:26
lyarwoodmdbooth: not yet, give me 10 and I'll squash the first two11:27
mdboothi.e. Do you want me to continue sending a stream of consciousness, or wait a bit between hitting the DENIED button?11:27
lyarwoodmdbooth: feel free to continue11:27
*** liuyulong has quit IRC11:30
*** ratailor has quit IRC11:30
mdboothlyarwood: Maybe hold off squashing...11:31
Kevin_Zhengjohnthetubaguy: so you are suggesting the decorator idea?11:31
mdboothThat's easy done later11:31
johnthetubaguyKevin_Zheng: that was my preference at the meeting, but honestly, I am tempted to merge what we have now, and refactor later11:32
johnthetubaguyKevin_Zheng: I don't think we are talking about behaviour changes (apart from maybe different error messages?)11:32
*** rfolco has joined #openstack-nova11:33
cdentKevin_Zheng: since you're around, are you still working on https://review.openstack.org/#/c/410128/ or would you prefer someone else pick that up?11:34
Kevin_Zhengcdent: one sec, I'm having some problem with my network now :(11:35
Kevin_Zhengcdent: Ah, you can pick it up11:37
dimscdent : good afternoon. have some time for me?11:37
amoralejmay i get some help on https://bugs.launchpad.net/puppet-nova/+bug/1656276 ?, i'd like to understand the best workflow to deploy nova environment with cellsv211:37
openstackLaunchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged]11:37
cdentKevin_Zheng: will do, I didn't want to mess with it if you still had is in progress11:37
cdentdims: I always have at least a little time for you11:37
mdboothlyarwood: I'm looking here, line 153011:37
mdboothlyarwood: And here: http://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockJobAbort11:38
Kevin_Zhengcdent: yeah thanks for ask11:38
dimscdent thanks :) please review a WIP for me https://review.openstack.org/#/c/419476/11:38
dimscdent : with those changes, i can get some nova scenarios working http://logs.openstack.org/43/418643/5/check/gate-rally-dsvm-py35-cinder-nv/0a95698/rally-plot/results.html.gz#/CinderVolumes.create_and_attach_volume/details11:38
mdboothlyarwood: It seems to me that if we already have an unambiguous full patch to the source disk, using target dev name is only a potential source of error?11:38
Kevin_Zhengjohnthetubaguy: you mean the current implement of https://review.openstack.org/#/c/408571/11:38
mdboothlyarwood: What was your thinking in making this change?11:39
* cdent looks11:39
johnthetubaguyKevin_Zheng: yeah11:39
mdboothI haven't looked at the next patch yet, so may be missing context11:39
*** baoli has joined #openstack-nova11:39
lyarwoodmdbooth: sorry you didn't link to the review, which file are we looking at?11:39
mdboothhttps://review.openstack.org/#/c/402163/10/nova/virt/libvirt/driver.py11:39
lyarwoodthanks11:40
Kevin_Zhengjohnthetubaguy: hmm fine by me, maybe check with alex?11:40
*** nicolasbock has joined #openstack-nova11:40
mdboothlyarwood: Trying to understand the motivation moving from disk_path -> target11:40
*** thorst_ has joined #openstack-nova11:40
mdboothThe purpose of this change and the previous one seems to be to move away from using disk_path, but I don't get why yet.11:40
*** manasm has quit IRC11:42
lyarwoodmdbooth: so off the top of my head I think this was to allow us to swap rbd volumes etc11:42
lyarwoodmdbooth: basically anytime we don't have an actual path11:42
lyarwoodmdbooth: let me go over this again quickly11:42
*** kaisers has quit IRC11:43
mdboothlyarwood: Ah....11:43
mdboothRBD has no path11:43
*** baoli has quit IRC11:44
mdboothAlthough, libvirt has its own representation of rbd 'paths' iirc11:44
openstackgerritjichenjc proposed openstack/nova: Allow -1 as limit value to pager  https://review.openstack.org/41989311:44
*** thorst_ has quit IRC11:45
lyarwoodmdbooth: ah and the main reason for doing this was that we would end up stashing either a disk_path or target_dev in BlockDevice._disk that was really confusing11:46
lyarwoodmdbooth: so I thnk i just went with target_dev as we should always have it and it's going to work when we don't have a file path11:47
cdentdims, left a comment, what's there is probably fine, but I also offered an alternative plan which I can do (but not today) if you think it is better.11:47
lyarwoodmdbooth: sorry I should really get all of this context back in my head before asking for a review ;)11:47
*** voelzmo has quit IRC11:47
cdentdims I gotta go to lunch now, but will check back with you and/or the review later in the day11:47
*** openstackgerrit has quit IRC11:47
*** cdent has quit IRC11:48
*** voelzmo has joined #openstack-nova11:50
*** sdague has joined #openstack-nova11:50
*** voelzmo has quit IRC11:50
*** voelzmo has joined #openstack-nova11:52
*** openstackgerrit has joined #openstack-nova11:52
*** hieulq_ has joined #openstack-nova11:54
*** salv-orl_ has quit IRC11:55
*** hieulq_ has quit IRC11:55
*** hieulq_ has joined #openstack-nova11:56
mdboothlyarwood: FYI, in terms of merging, I think I'd split your change to Guest.get_disk into a separate patch, and merge the rest of the patch into the previous patch.11:56
*** voelzmo has quit IRC11:57
mdboothI think that makes it a little more obvious what's going on.11:57
*** salv-orlando has joined #openstack-nova11:59
lyarwoodmdbooth: ack thanks12:01
*** dtp has joined #openstack-nova12:01
mdboothlyarwood: I'm still going, btw.12:01
mdboothI don't like the change to BlockDevice, just writing why.12:01
alex_xujohnthetubaguy: Kevin_Zheng i'm good with refactor the code later12:02
*** rtheis has joined #openstack-nova12:03
johnthetubaguyalex_xu: one thing, do we reject with 400 the bad ones in that patch?12:03
mdboothlyarwood: Hmm, potentially scratch that.12:03
alex_xujohnthetubaguy: you mean for joinedtable and internal attrs?12:04
johnthetubaguyalex_xu: the joinedtable ones12:05
alex_xujohnthetubaguy: yea, the current patch return 40012:05
openstackgerritMikhail Feoktistov proposed openstack/nova: libvirt: ephemeral disk support for virtuozzo containers  https://review.openstack.org/31248812:05
johnthetubaguyalex_xu: I think the behaviour is the key thing12:05
johnthetubaguyalex_xu: so we should be good12:05
alex_xujohnthetubaguy: ++, the behaviour is everyone expected12:05
openstackgerritsahid proposed openstack/nova: libvirt: update the active migrations DS to support migration result  https://review.openstack.org/40926012:07
openstackgerritsahid proposed openstack/nova: libvirt: ensure live-migration success  https://review.openstack.org/40926112:07
*** adisky_ has quit IRC12:09
*** manasm has joined #openstack-nova12:10
*** hieulq_ has quit IRC12:13
openstackgerritjichenjc proposed openstack/nova: make limit=0 means query 0 data  https://review.openstack.org/41991312:13
*** ijw has joined #openstack-nova12:15
*** hieulq_ has joined #openstack-nova12:15
mdboothlyarwood: Any reason for https://review.openstack.org/#/c/399735/ to depend on the previous 2 patches?12:17
mdboothSeems the middle 2 are about fixing something Rbd related?12:18
*** rtheis has quit IRC12:18
lyarwoodmdbooth: other than using self._guest.get_disk_by_target_device no I don't think so12:18
*** hieulq_ has quit IRC12:19
*** voelzmo has joined #openstack-nova12:19
*** ijw has quit IRC12:20
*** moshele has joined #openstack-nova12:22
*** catintheroof has joined #openstack-nova12:30
*** catintheroof has quit IRC12:31
*** catintheroof has joined #openstack-nova12:32
openstackgerritStephen Finucane proposed openstack/nova: scheduler: Don't modify RequestSpec in filtering  https://review.openstack.org/41947912:32
openstackgerritEmilien Macchi proposed openstack/nova: CI test - never merge  https://review.openstack.org/41992612:36
*** bvanhav_ has joined #openstack-nova12:38
*** kaisers has joined #openstack-nova12:39
*** lucasagomes is now known as lucas-afk12:39
openstackgerritEmilien Macchi proposed openstack/nova: Debug https://bugs.launchpad.net/tripleo/+bug/1656276  https://review.openstack.org/41992612:39
openstackLaunchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged]12:39
*** manasm has quit IRC12:41
EmilienMhey nova folks, we're having a new issue with a patch merged in Nova yesterday, https://review.openstack.org/#/c/409890/ - details are reported here: https://bugs.launchpad.net/tripleo/+bug/1656276 - any feedback is welcome12:41
openstackLaunchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged]12:41
*** thorst_ has joined #openstack-nova12:41
openstackgerritSergey Nikitin proposed openstack/nova: Standardization of VM diagnostics info API.  https://review.openstack.org/35554012:43
*** mfedosin has quit IRC12:44
*** manasm has joined #openstack-nova12:45
*** ayogi has quit IRC12:48
dtpgood morning.  are quotas enforced in the database?12:55
*** kevinz has joined #openstack-nova12:55
openstackgerritVasyl Saienko proposed openstack/nova: Support Ironic interface attach/detach in nova virt  https://review.openstack.org/36441312:56
*** liverpooler has joined #openstack-nova13:00
*** kevinz has quit IRC13:00
*** psachin has quit IRC13:02
*** voelzmo has quit IRC13:03
*** liverpooler has quit IRC13:03
*** liverpooler has joined #openstack-nova13:04
*** thorst_ has quit IRC13:05
*** thorst_ has joined #openstack-nova13:09
openstackgerritVasyl Saienko proposed openstack/nova: Support Ironic interface attach/detach in nova virt  https://review.openstack.org/36441313:13
*** moshele has quit IRC13:15
*** ijw has joined #openstack-nova13:17
*** sudipto_ has joined #openstack-nova13:19
*** sudipto has joined #openstack-nova13:19
*** amotoki has quit IRC13:19
*** ijw has quit IRC13:21
*** kaisers has quit IRC13:27
sahidJayF:13:27
*** Dinesh_Bhor has quit IRC13:29
*** sree has quit IRC13:30
*** sree has joined #openstack-nova13:30
*** ducttape_ has joined #openstack-nova13:31
*** sree has quit IRC13:31
*** sree has joined #openstack-nova13:31
*** sree has quit IRC13:33
*** sree has joined #openstack-nova13:33
*** manasm has quit IRC13:34
*** mdnadeem has quit IRC13:36
*** sree has quit IRC13:36
*** sree has joined #openstack-nova13:37
*** voelzmo has joined #openstack-nova13:37
*** yuntongjin has joined #openstack-nova13:40
*** sree has quit IRC13:40
*** cdent has joined #openstack-nova13:42
*** prateek has quit IRC13:43
*** edmondsw has joined #openstack-nova13:45
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] try py35 functional with a clean slate  https://review.openstack.org/41996813:45
openstackgerritVladyslav Drok proposed openstack/nova: Generate necessary network metadata for portgroups  https://review.openstack.org/38875613:45
cdenti'm back dims, in case that's of any use13:48
dimscdent : hehe, sorry i had gotten distracted with some shiny stuff. so yes, "EncodeUTF8 middleware" sounds awesome. we could do it as a 2 step tango too, let this change in and then do the middleware. right?13:50
dimscdent : i'am currently poking at what functional tests start working with this change13:50
cdentdims: yeah two step tango would work well13:53
dimscool thanks cdent. let me un-black list stuff some tests and get it ready later today.13:54
cdent\o/13:54
*** ducttape_ has quit IRC13:55
openstackgerritEd Leafe proposed openstack/nova: Fix missing RP generation update  https://review.openstack.org/41913713:57
*** pbandark has quit IRC13:58
openstackgerritVasyl Saienko proposed openstack/nova: WIP implement interface attach/detach in ironic virt driver  https://review.openstack.org/41997513:59
openstackgerritMikhail Feoktistov proposed openstack/nova: libvirt: ephemeral disk support for virtuozzo containers  https://review.openstack.org/31248813:59
*** bvanhav_ has quit IRC14:00
*** edleafe is now known as figleaf14:00
*** cdent_ has joined #openstack-nova14:01
*** cdent has quit IRC14:01
*** cdent_ is now known as cdent14:01
*** mdrabe has joined #openstack-nova14:01
*** vladikr has joined #openstack-nova14:02
*** dtp has quit IRC14:04
*** xyang1 has joined #openstack-nova14:06
openstackgerritsahid proposed openstack/nova: libvirt: update the active migrations DS to support migration result  https://review.openstack.org/40926014:06
openstackgerritsahid proposed openstack/nova: libvirt: ensure live-migration success  https://review.openstack.org/40926114:06
*** bvanhav_ has joined #openstack-nova14:08
openstackgerritJohn Garbutt proposed openstack/nova: WIP: Call preallocate_for_instance before scheduler  https://review.openstack.org/41312414:08
openstackgerritJohn Garbutt proposed openstack/nova: WIP: introduce preallocate_for_instance method.  https://review.openstack.org/41888114:08
openstackgerritJohn Garbutt proposed openstack/nova: Tidy up preallocate_for_instance  https://review.openstack.org/41997814:08
openstackgerritJohn Garbutt proposed openstack/nova: Fix no networks preallocate_for_instance return  https://review.openstack.org/41997914:08
lyarwoodmdbooth: just looking at the swap volume fix again, I think we do need the previous 2 patches still. _swap_volume only provides a file path so we can't lookup the disk in the domain using guest.get_disk. I'll squash these down now for review.14:10
EmilienMbauzas: have you looked https://bugs.launchpad.net/nova/+bug/1656276 ?14:11
openstackLaunchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged]14:11
*** eglynn has joined #openstack-nova14:13
*** yuntongjin has quit IRC14:13
*** amoralej is now known as amoralej|lunch14:14
*** gouthamr has joined #openstack-nova14:15
*** ijw has joined #openstack-nova14:18
*** kaisers has joined #openstack-nova14:22
*** ijw has quit IRC14:23
openstackgerritgaryk proposed openstack/nova: VMware: retry network lookup if network not ready  https://review.openstack.org/39177814:24
*** kevinz has joined #openstack-nova14:24
openstackgerritVasyl Saienko proposed openstack/nova: Support Ironic interface attach/detach in nova virt  https://review.openstack.org/36441314:25
*** baoli has joined #openstack-nova14:28
*** smatzek has quit IRC14:30
*** bvanhav_ has quit IRC14:30
bauzasEmilienM: hola, sorry was afk due to 10cm of snow and my frontyark needing cleanup :)14:30
bauzasfrontyard* even14:30
bauzasEmilienM: owalsh pointed me the issue14:30
EmilienMbauzas: 10 cm only? :-P14:31
bauzaswe're in France, and you know how 10cm of snow can be terrible for us :p14:31
bauzas(even living in the French alps)14:31
*** bvanhav_ has joined #openstack-nova14:31
bauzasanyway14:31
EmilienMbauzas: ok cool, let me know if there is something we need to change in the tripleo workflow or if you spot something in nova itself14:31
*** kevinz has quit IRC14:31
bauzasso, owalsh pointed me the sequence that puppet does14:31
bauzashttps://github.com/openstack/puppet-nova/blob/master/manifests/db/sync_cell_v2.pp#L41 means that it runs after API DB sync, right?14:32
bauzasEmilienM: ^14:32
lyarwoodmdbooth: correction, we need them still thanks to _live_snapshot, I've updated the review with a comment.14:32
EmilienMbauzas: right14:32
*** ababich has joined #openstack-nova14:32
*** bvanhav_ is now known as bvanhav14:32
bauzasEmilienM: okay, so you populated the DB, which is good14:32
bauzasEmilienM: but you haven't started the compute nodes, right?14:33
bauzasEmilienM: so they aren't registered in the Nova DB, correct?14:33
EmilienMbauzas: that is true14:33
bauzasokay14:33
openstackgerritVasyl Saienko proposed openstack/nova: WIP implement interface attach/detach in ironic virt driver  https://review.openstack.org/41997514:33
*** tommylikehu has quit IRC14:33
bauzasEmilienM: in theory, that shouldn't be a problem14:33
*** tommylikehu has joined #openstack-nova14:34
EmilienMbauzas: that was 2 weeks ago, wanna help? https://twitter.com/EmilienMacchi/status/81483816365497548814:34
bauzasheh14:34
bauzasEmilienM: the error occurs on a fresh new install, not an upgrade ?14:36
openstackgerritVladyslav Drok proposed openstack/nova: Generate necessary network metadata for portgroups  https://review.openstack.org/38875614:36
EmilienMbauzas: fresh14:36
bauzasok14:36
EmilienMbauzas: we haven't the CI job in place for upgrade yet (it's WIP)14:36
bauzasack14:37
EmilienMactually, we have but it fails because of this cellv2 thing14:37
EmilienMand mwhahaha is working on it14:37
EmilienMhttps://review.openstack.org/#/q/topic:bug/164934114:37
*** salv-orlando has quit IRC14:38
bauzasEmilienM: so, the goal of that simple_cells command is to populate your cellsv2 database with the necessary mappings for the compute nodes being in cells14:39
bauzasEmilienM: if you don't have nodes already, then you have no cell mapping AFAICT14:39
*** tlian has joined #openstack-nova14:39
bauzasbut when you add a node, then you have to run again the simple cells command in order to assign it a cell14:40
EmilienMwhy isn't it dynamic?14:40
EmilienMwhy conductor or something wouldn't do it for us?14:40
bauzasbecause it hasn't been done yet14:40
EmilienMah ok, it's in the roadmap14:40
bauzaswell14:40
bauzasthe fact is14:40
bauzasonly the operator can know what cell the node should be added to, nova can't guess for it except putting the node into a default cell14:41
EmilienMwe could have the cell name in the nova.conf on each compute node14:41
EmilienMand compute would send this name to conductor which write in db14:42
EmilienMso you can scale up/down cells dynamically without running nova-manage14:42
bauzasthat's a possibility but like I said, it's not written yet :)14:42
EmilienMcool np14:42
bauzasso14:43
bauzasI'm trying to understand where exactly puppet fails14:43
bauzaswhen running the simple cell command14:43
bauzasEmilienM: do you have some trace that I can look at ?14:43
bauzasI guess only https://bugs.launchpad.net/nova/+bug/1656276 ?14:43
openstackLaunchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged]14:43
EmilienMit's running "nova-manage cell_v2 simple_cell_setup" as part of the nova database initialization when no compute nodes have been created and it returns 114:43
openstackgerritVladyslav Drok proposed openstack/nova: Support Ironic interface attach/detach in nova virt  https://review.openstack.org/36441314:44
EmilienMyes there is no trace but I can easily give one to you, let me do something quickly14:44
openstackgerritEmilien Macchi proposed openstack/nova: Debug https://bugs.launchpad.net/tripleo/+bug/1656276  https://review.openstack.org/41992614:44
EmilienMbauzas: i'm preparing a CI job with more debug, let me 5 min14:45
bauzasEmilienM: anyway, I think the bug description has a first point to look14:45
bauzasEmilienM: so https://github.com/openstack/puppet-nova/blob/master/manifests/db/sync_cell_v2.pp#L36 means that we try 10 times every 5 secs until the return code is zero, right?N14:46
EmilienMbauzas: "No hosts found to map to cell, exiting"14:46
EmilienMbauzas: exactly14:46
*** amoralej|lunch is now known as amoralej14:46
*** eharney has quit IRC14:46
EmilienMbauzas: keep in mind we can change puppet to accept any return code for this Exec14:46
EmilienMbauzas: but I'm afraid accepting 0,1 is not the right thing to do14:47
bauzasEmilienM: okay, so why does the puppet command failed only once ?14:47
openstackgerritVladyslav Drok proposed openstack/nova: Generate necessary network metadata for portgroups  https://review.openstack.org/38875614:47
EmilienMbecause what is the exec fails for a good reason, we won't see it14:47
bauzasEmilienM: it should have failed 10 times ?14:47
EmilienMbauzas: it tried 10 times and show output of last failure14:47
EmilienMbut I guess it's 10 times the same failure14:47
bauzasah ok14:48
EmilienMit returned 1 ten times14:48
EmilienMthis "No hosts found to map to cell, exiting" means "no compute has been found, fail"?14:48
*** amotoki has joined #openstack-nova14:49
bauzashttps://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L1251 is the reason behind it14:49
bauzasEmilienM: which returns 1 indeed14:50
bauzashttps://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L111414:50
EmilienMindeed14:50
amoralejbauzas, EmilienM, i've been doing some additional tests and if no computes are found when running the command the cell is not added to cell_mappints table14:50
amoralejso, it may be right to fail if no compute nodes are found14:51
bauzasamoralej: that's an expected behaviour14:51
*** markvoelker has joined #openstack-nova14:51
*** markvoelker_ has quit IRC14:51
bauzasso, the point is, what is the meaning of that return code equal to 114:51
EmilienMso we need to run nova-compute *before* running this command? it might be very tricky14:51
openstackgerritJohn Garbutt proposed openstack/nova: WIP: Call preallocate_for_instance before scheduler  https://review.openstack.org/41312414:51
EmilienMin multinode deployments specially14:51
*** liverpooler has quit IRC14:51
*** hieulq_ has joined #openstack-nova14:51
amoralejbauzas, then in any case we need to get compute nodes before running it14:51
bauzasEmilienM: well, I think no computes nodes doesn't mean that the command failed14:52
EmilienMbauzas: https://github.com/openstack/nova/commit/a86d78f5f048e11ddb7e375f2d6cc707108b947814:52
bauzasEmilienM: so, there are 2 possibilites14:52
EmilienMalaski: ^14:52
bauzaseither we say it's okay to have no computes when running that command, and then we should soft-fail with return code 014:52
bauzasor, we ask users to deploy compute nodes before running that command14:53
amoralejbauzas, in option 1, we'd need to re-run it after adding the compute nodes?14:53
bauzasamoralej: indeed14:53
*** mjura has quit IRC14:54
amoralejthen, option 1 converts to option 214:54
*** prateek has joined #openstack-nova14:54
amoralejeven if command exit with rc 0, we need to orchestrate14:54
bauzasyou're right14:54
mdboothlyarwood: That's a separate issue, right?14:54
*** beagles is now known as beagles_afk14:54
mdboothI mean, isn't that broken either way?14:54
amoralejbauzas, i have a doubt about cell014:54
bauzasamoralej: EmilienM: I was a bit off the recent cellsv2 discussions, so I don't know if we're planning something like an automatic mapping when adding nodes14:55
amoralejwhat i find when i'm running the command after adding compute nodes is that a new cell is created in cell_mappings14:55
bauzasamoralej: cell0 is a total different story14:55
bauzasamoralej: cell0 is not a cell :p14:55
bauzaswell, it is a cell14:55
bauzasbut we could have named it "cemetery cell"14:56
amoralejok, then i have a new doubt, :)14:56
bauzasit's for failed instances not having a compute node14:56
bauzasie. when the scheduler returns NoValidHosts, then the instance is mapped to cell014:56
lyarwoodmdbooth: nope, my change introduces the lookup, previously all the calls made by BlockDevice to libvirt would work with either a path or target dev14:56
amoralejso, for single cell deployments, do i need only cell0 or cell0 and "reall cell" ?14:57
bauzasinstead of being mapped to the cell that the compute belongs to14:57
mdboothlyarwood: Of course. Got it.14:57
bauzasamoralej: for real deployments, you need cell0 *and* a single cell14:57
*** sheel has quit IRC14:57
amoralejok, understood14:57
*** prateek has quit IRC14:57
*** prateek has joined #openstack-nova14:58
bauzasamoralej: that's why we populate cell0 with a db sync14:58
bauzasso that we have everything like if it was a normal cell14:58
bauzasanyway14:58
bauzasEmilienM: amoralej: so, in order to tackle your problem14:58
bauzashttps://github.com/openstack/nova/commit/a18f601753f92ff4a2a42be0962a188f583bbfb9 just canary-tested an existing problem14:59
*** mtanino has joined #openstack-nova14:59
bauzasbecause previously, it was returning 0 as you were already having a cell015:00
bauzasnow, we're going past this point15:00
*** hongbin has joined #openstack-nova15:00
*** dave-mccowan has joined #openstack-nova15:00
amoralejyes bauzas, i'm going to summarize this findings in the LP15:02
*** mandy has joined #openstack-nova15:04
*** ducttape_ has joined #openstack-nova15:04
mandyhi15:05
*** liverpooler has joined #openstack-nova15:05
*** jaosorior has joined #openstack-nova15:05
mandycould you anyone help me ??15:06
*** jaosorior has quit IRC15:07
bauzasamoralej: so, just verified and FWIW, we create cells (and run the single cell command) *after* starting n-cpu in devstack15:08
bauzasEmilienM: ^15:08
bauzasthat's why the upstream nova jobs aren't trampled15:08
figleafmandy: just ask your question.15:08
*** eharney has joined #openstack-nova15:08
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] try py35 functional with a clean slate  https://review.openstack.org/41996815:08
*** mriedem has joined #openstack-nova15:08
amoralejbauzas, could you take a look to comment #3 in https://bugs.launchpad.net/nova/+bug/1656276 and confirm if it's right?15:09
openstackLaunchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged]15:09
*** claudiub|2 has joined #openstack-nova15:09
*** ducttape_ has quit IRC15:09
*** annegentle has joined #openstack-nova15:10
mandyactualy i am successfully create one vm in openstack using devtsack15:10
mandy* devstack but unable to ping that vm ....could you plz help me15:11
mandy??15:11
*** tblakes has joined #openstack-nova15:11
*** mnestratov|2 has quit IRC15:11
EmilienMbauzas: interesting15:11
*** liverpooler has quit IRC15:11
EmilienMbauzas: link?15:11
*** liverpooler has joined #openstack-nova15:11
mandyi used private network15:12
*** claudiub|3 has joined #openstack-nova15:12
*** claudiub has quit IRC15:12
mandyis anyone there15:12
mandy??15:12
mriedemceph job is now non-voting, fyi15:13
openstackgerritsahid proposed openstack/nova: compute: reset instance events  https://review.openstack.org/42002615:13
diana_clarkemandy: Those kinds of support/usage questions don't typically get answered here (see channel topic).15:13
mandyok15:14
mandythanks15:14
mandybye15:14
*** mandy has left #openstack-nova15:14
*** claudiub|2 has quit IRC15:15
*** jaosorior has joined #openstack-nova15:16
EmilienMmwhahaha: out of your mind, would it be doable to run this cell_v2 simple_cell_setup *after* starting nova-compute in tripleo?15:16
EmilienMmwhahaha: I'm wondering about the steps15:16
*** annegentle has quit IRC15:16
*** jwcroppe has joined #openstack-nova15:17
bauzasEmilienM: http://logs.openstack.org/90/409890/1/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/447ab55/logs/devstacklog.txt.gz#_2016-12-12_21_54_28_04215:17
mriedemmdbooth: how are you feeling about https://review.openstack.org/#/c/312488/ ?15:17
bauzas(sorry, in internal meeting now)15:17
mriedemthe vz ci is running test_verify_created_server_ephemeral_disk now15:17
*** satyar has joined #openstack-nova15:17
openstackgerritVasyl Saienko proposed openstack/nova: Implement interface attach/detach in ironic virt driver  https://review.openstack.org/41997515:17
openstackgerritVasyl Saienko proposed openstack/nova: Implement interface attach/detach in ironic virt driver  https://review.openstack.org/41997515:17
mwhahahawhy is this cell v2 setup stuff so complex15:18
mdboothmriedem: In a meeting. Get back in an hour or so.15:18
mwhahahait's so not 'simple' if you require cross node coordination prior to running it15:18
amoralejEmilienM, i can start by removing the command as part of nova::db::sync and adding it at the end in p-o-i15:19
amoralejthat would be the first step, am i right?15:19
*** jaosorior has quit IRC15:19
*** zz_dimtruck is now known as dimtruck15:19
amoralejmwhahaha ^15:19
*** xyang1 has quit IRC15:19
mwhahahano15:19
mwhahahait's lined up in the nova::deps15:19
EmilienMamoralej: change needs to happen in puppet-nova, not poi (if we have to change something)15:19
*** ijw has joined #openstack-nova15:19
EmilienMright15:19
mwhahahahttps://github.com/openstack/puppet-nova/blob/master/manifests/deps.pp#L23-L2615:20
*** smatzek has joined #openstack-nova15:20
mwhahahaalso we would need to remove https://github.com/openstack/puppet-nova/blob/master/manifests/db/sync_api.pp#L45-L4715:20
amoralejthis one is the one i have seen, not the deps stuff15:21
mwhahahawe would want to touch both because we want to run sync after compute service starts15:21
*** jaosorior has joined #openstack-nova15:22
mwhahahai still don't understand why these things keep getting added as hard requirements after the fact15:22
mwhahahafirst it was the db upgrade that broke everything and now it's you have to run it after computes, is any of this stuff documented?15:22
EmilienMI agree it's not easy for deployers :(15:22
bauzasEmilienM: amoralej: so, about a very short-term fix, here are my thoughts15:22
EmilienMmriedem: ^ fyi15:22
amoralejnot really mwhahaha, i've spent the whole morning trying to understand it15:23
mwhahahai've probably wasted over a month on this stuff and operators are expected to use it?15:23
bauzasEmilienM: amoralej: while I understand the point that it could be fancy for operators to have a way to automatically add the cell mapping when starting the compute, that's not written yet15:23
mwhahahawhy is cell v2 being a hard requirement if docs and the code aren't written yet?15:24
bauzasEmilienM: amoralej: here, instead, in order to punt your problem, I'd rather recommend you to modify https://github.com/openstack/puppet-nova/blob/master/manifests/db/sync_cell_v2.pp#L32 and accept return codes to be 0 and 115:24
amoralejbauzas, and requiring to run discover_hosts after every new compute nodes should be automated too, imho15:24
EmilienMbauzas: but would tables be created in DB?15:24
mwhahahabauzas: that still doesn't handle the upgrade case where we need to have it created before we run the api sync for n ->o15:24
mwhahahabauzas: that's not acceptable15:24
bauzasEmilienM: amoralej: it would put your CI back green, but would still require the cell command to be run *after* starting the compute15:25
*** ijw has quit IRC15:25
bauzasmwhahaha: upgrade is a very different case15:25
bauzasmwhahaha: because you already have computes15:25
mwhahahabauzas: please point me to the documentation for upgrades vs install of cell v215:26
*** armax has joined #openstack-nova15:26
bauzasmwhahaha: here, the discussion is about the critical bug that tramples both puppet users and the TripleO CI15:26
*** jpena has joined #openstack-nova15:26
bauzasI'm just trying to tackle the problem asp15:27
bauzasasap15:27
mwhahahathis problem has existed in one form or another for more than a month15:27
mwhahahaand every time we make progress, nova checks a change that completely destroys what we thought the process was previously15:27
mwhahahaso i'd like not to have to repeat this again in a week15:27
mwhahahaso we all understand how this thing is supposed to work, is there any documentation around how this is expected to function15:28
*** markvoelker has quit IRC15:28
bauzasmwhahaha: http://docs.openstack.org/releasenotes/nova/unreleased.html#id2 can help you15:28
EmilienMI agree this thing is complex to deploy outside devstack15:28
EmilienMspecially in production15:28
bauzas"Ocata requires that your deployment have created the cell and host mappings in Newton. If you have not done this, Ocata’s db sync command will fail. Small deployments will want to run nova-manage cell_v2 simple_cell_setup on Newton before upgrading. Operators must create a new database for cell0 before running cell_v2 simple_cell_setup. The simple cell setup command expects the name of the cell0 database to be <API database15:28
mwhahahabauzas: that's what we're doing15:29
mwhahahabauzas: but no mention of discover hosts or anything else15:29
mwhahahaalso release notes != documentation on the overall process15:29
*** tbachman has joined #openstack-nova15:29
EmilienMit doesn't mention ordering with nova-compute process, does it?15:29
* mwhahaha brb kids -> school15:29
*** andreas_s has quit IRC15:30
*** sacharya has joined #openstack-nova15:32
bauzasEmilienM: well, that's because we added the simple cell command in Newton :(15:33
bauzasEmilienM: to answer your question, http://docs.openstack.org/releasenotes/nova/unreleased.html#id1015:33
*** ducttape_ has joined #openstack-nova15:33
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [py35] Fixes to get rally scenarios working  https://review.openstack.org/41947615:35
mriedemso docs is a known gap, i.e. docs that aren't release notes15:35
mriedemi think dansmith was working on a blog post to start that,15:35
mriedembut it would eventually be in the devref15:35
mriedemlaski had started some docs at one point but it's probably diverged quite a bit from how things actualy work now15:35
mriedemhaving said all that, dan is out on vacation for a week15:36
mriedemi don't know how far along dan was on a blog post, but i might start a doc if i can get some time15:37
melwittyeah, there isn't yet detail about how compute nodes are needed before running simple_cell_setup15:37
*** nkrinner is now known as nkrinner_afk15:37
*** cleong has joined #openstack-nova15:37
mriedemmelwitt: you mean otherwise you have to run discover_hosts?15:37
mriedemi haven't read all of the scollback,15:38
melwittmriedem: well, discover_hosts actually won't work either because that requires you have at least one cell_mapping already15:38
melwittso really it would be you have to run map_cell_and_hosts I think15:38
melwittafter compute hosts show up later15:38
mriedembut if there are major things that people have run into in practice trying to get this setup, it would be helpful to post those in the mailing list for posterity rather than a dump in IRC which hits maybe 1% of people right now on a friday15:38
mriedemEmilienM: mwhahaha: amoralej: ^15:39
mriedembecause those are the things i want to make sure are addressed in a doc15:39
melwittwhat I'm wondering is whether the cells v2 upgrade check runs on a fresh install or not?15:39
mwhahahait doesn't15:39
*** kaisers has quit IRC15:39
mwhahahathe upgrade problem is something else15:39
mwhahahathe problem is that you're adding requirements that no one know sabout15:39
mriedemmelwitt: which cells v2 upgrade check? the db schema migration?15:39
bauzasmelwitt: https://bugs.launchpad.net/nova/+bug/1656276 refers to a fresh install15:39
openstackLaunchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged]15:39
mwhahahaoh yea,that's new as of today15:40
melwittmriedem: the thing that raises an error if there aren't two cell mappings15:40
bauzasmelwitt: and the proposed fix to amend https://github.com/openstack/puppet-nova/blob/master/manifests/db/sync_cell_v2.pp#L32 with return code 1 as valid would fix $15:40
bauzaswould fix ^ (I mean)15:40
mriedemmelwitt: mwhahaha: so the db schema migration runs on a fresh install by first checking to see if there are any flavors, if there aren't then it noops15:40
bauzaswhat mwhahaha is discussing is another issue, related to the upgrade path15:40
mwhahahahttps://bugs.launchpad.net/tripleo/+bug/1649341 was teh upgrade bug15:40
openstackLaunchpad bug 1649341 in tripleo "Undercloud upgrade fails with "Cell mappings are not created, but required for Ocata"" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz)15:40
bauzasso another bug15:40
melwittmriedem: the  (_('Cells v2'), _check_cellsv2), in nova/cmd/status.py15:40
mriedemmelwitt: so that's different15:41
melwittI see15:41
mriedemand we haven't integrated the nova-status upgrade check stuff into our CI pipeline yet15:41
mriedemanother TODO of mine15:41
mwhahahawhich the problem is to solve https://bugs.launchpad.net/tripleo/+bug/1649341 i need to know how to properly tackle https://bugs.launchpad.net/nova/+bug/165627615:41
openstackLaunchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged]15:41
mwhahahawhere i didn't until today15:41
mwhahahaso now we're just adding complexity on complexity15:41
mwhahahaand adding accept 1 doesn't solve the upgrade case15:42
melwittyeah, I don't think accepting 1 helps anything really15:42
mwhahahai had a solution for the upgrade until the requirement of compute nodes was added15:42
mriedemmwhahaha: EmilienM: so can we hold on a minute,15:43
mriedemi understand you're frustrated,15:43
bauzasmelwitt: it only helps their CI to be green15:43
* bauzas shrugs15:43
mriedemi need help with posting the list of issues you have in a form that doesn't require me to be chasing 3 conversations at once15:43
melwittI'm not sure it'll run correctly after that if there are no cell mappings created though (nova)15:43
EmilienMit's not about a green CI15:43
EmilienMit's about how people outside devstack deploy your software15:44
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [py35] Fixes to get rally scenarios working  https://review.openstack.org/41947615:44
mriedemmwhahaha: EmilienM: so can you guys help me out by starting a ML thread with the current known issues you have so we can work through it there?15:44
EmilienMmriedem: of course15:44
mriedemb/c right now i'm not keeping up with this fire hose of stuff15:44
mriedemand not everyone is around right now15:45
mriedemso we need to start a list of items in the ML and tackle those15:45
mwhahahayea we can move to the ML15:45
bauzasEmilienM: I don't disagree, I'm just trying to reduce the criticity so we can sit down and discuss without having the tense of urgency15:45
mriedemwe could start an etherpad too if that is helpful15:46
mriedemhell i'm pretty sure we already have an etherpad for cells v215:46
bauzasEmilienM: I'm really sorry that you folks are experiencing bugs against our recent improvements and I think your insights are definitely valid15:46
mriedemi can't find the etherpad, only the wiki page15:47
mriedemmelwitt: do you know if we had a generic cells v2 etherpad for work items?15:48
*** burt has quit IRC15:48
melwittmriedem: we've had a number of etherpads, let me see15:48
*** cdent has quit IRC15:48
mriedemmelwitt: yeah this is the older wiki https://wiki.openstack.org/wiki/Nova-Cells-v215:49
mriedembut not seeing anything generic there15:49
EmilienMbauzas: let's get this thing fixed15:49
melwitthttps://etherpad.openstack.org/p/nova-multiple-cells https://etherpad.openstack.org/p/cellsV2-remaining-work-items are old ones15:50
*** mdrabe has quit IRC15:50
*** prateek has quit IRC15:51
*** mvk has quit IRC15:51
mriedemmelwitt: thanks i added those to the wiki15:51
*** hieulq_ has quit IRC15:51
*** hieulq_ has joined #openstack-nova15:52
*** hieulq_ has quit IRC15:52
*** markvoelker has joined #openstack-nova15:54
mriedemtrying to update the TODOs in there also15:54
*** mlakat has quit IRC15:54
*** markvoelker_ has joined #openstack-nova15:55
*** markvoelker has quit IRC15:55
melwittthanks mriedem15:56
mriedemalso https://review.openstack.org/#/c/420051/15:57
mriedemEmilienM: melwitt: bauzas: mwhahaha: ok i've updated https://wiki.openstack.org/wiki/Nova-Cells-v2#Nova_Cells_v2 a bit - linked in those older etherpads, freshed up the TODOs, and added those bugs15:58
mriedemlet me know if i missed something15:58
EmilienMmriedem: thanks for helping15:59
EmilienMmriedem: are we sure to backport https://review.openstack.org/#/c/420051/ ?16:00
EmilienMit broke us on master :/16:00
*** mdrabe has joined #openstack-nova16:00
mriedemEmilienM: i thought that change was made on master specifically for you guys16:01
mwhahahano16:02
mwhahahait's not for us, we were working prior to that16:02
melwittI think it was sort of a coincidence that it broke you, because simple_cell_setup has always returned 1 in the case of not finding compute nodes. the reason you got past it is because you ran map_cell0 separately before running simple_cell_setup, and the old version of simple_cell_setup returned 0 if it found cell0 i.e. it assumed it had already run in hte past16:02
mwhahahawe don't run map_cell016:02
mwhahahacause we didn't know that was a thing16:02
*** marst has joined #openstack-nova16:03
mriedemok i don't have the context on how that change broke you, is that in a bug somewhere?16:04
melwittin https://bugs.launchpad.net/nova/+bug/1656276 amoralej said you were mapping cell0 separately?16:04
openstackLaunchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged]16:04
mriedemis that https://bugs.launchpad.net/tripleo/+bug/1649341 ?16:04
openstackLaunchpad bug 1649341 in tripleo "Undercloud upgrade fails with "Cell mappings are not created, but required for Ocata"" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz)16:04
*** sridharg has quit IRC16:04
melwittalso from the code it's the only way you could have gotten simple_cell_setup to return 0 without any compute nodes16:04
melwittunless I've missed something16:05
mriedemsorry i meant https://bugs.launchpad.net/nova/+bug/165627616:05
mwhahahaso we're relying on puppet, and all we knew is that we had to run the simple setup (thats what we were told last time we came here with problems)16:05
mriedemok i've -W'd the backport until we can sort it out16:05
mwhahahaso i went and implemented just the simple setup16:05
melwittmriedem: I *think* it's because they are running 'map_cell0' and then 'simple_cell_setup', before the idempotence patch it would all return 0, after the patch it returns 1 because it gets to the 'checking for hosts' part16:06
mwhahahacause i didn't know those other commands were required16:06
melwittthe other commands aren't required16:06
amoralejmelwitt, we get cell0 by simple_cell_setup command16:06
mwhahahamelwitt: evidently they are16:06
mwhahahamelwitt: they weren't until that patch16:06
alaskiEmilienM: bauzas I've been out of discussions for a bit, but I had a PoC up at one point for computes to self register with a cell. It put an entry in the db that they could pull which told them which cell they were in, and then I was adding an HTTP API call for them to hit for registration.16:07
mriedemmelwitt: so you mean they got here https://review.openstack.org/#/c/420051/1/nova/cmd/manage.py@1240 ?16:07
mriedemafter that change i mean16:07
melwittyes16:07
*** voelzmo has quit IRC16:08
amoralejmelwitt, we were getting return 0 in second execution16:08
mwhahahaare you sure we're just not failing from line 122616:08
amoralejbecause we retry it16:08
*** voelzmo has joined #openstack-nova16:08
mriedemmelwitt: and that's because they got to https://review.openstack.org/#/c/420051/1/nova/cmd/manage.py@1379 ?16:08
mriedemi.e. no computes yet so it would return 116:08
*** voelzmo has quit IRC16:08
mriedemwell,16:08
alaskibauzas: EmilienM https://review.openstack.org/#/c/369634/16:08
mriedemreturn None from that method, and simple_cell_setup would return 116:08
mriedemwhich then broke tripleo, right?16:08
*** raunak has joined #openstack-nova16:09
bauzasalaski: yeah I remember now our discussions16:09
alaskithat was just an idea I had though, there had been no real discussion or consensus on the direction16:09
bauzasyup, what I told to EmilienM and amoralej16:10
bauzasEmilienM: amoralej: mwhahaha: melwitt: mriedem: sorry, was in a meeting16:10
*** hieulq_ has joined #openstack-nova16:10
melwittalaski: that requires that a first cell was ever created though, right? they're in a situation where they have no compute hosts and no cell mapping other than cell016:11
amoralejbut, why can't create a cell without compute nodes?16:11
melwittI'm guessing to avoid creating orphaned cells16:11
bauzasmelwitt: we could simple setup cells with no computes, and populate the mappings when adding the nodes16:11
alaskimelwitt: right. that wouldn't solve their issue. I wasn't exactly clear what's being discussed now.16:11
mwhahahashouldn't the cell be created first then nodes added to it?16:12
mwhahahalike it seems weird to do the other way around16:12
melwittalaski: is that why we don't create cell mappings with no hosts? to avoid orphaned cells?16:12
amoralejfrom a functional, operational point of view, creating the cell first and registering compute nodes later make sense imo16:12
amoralejin fact compute nodes should be automatically added16:12
amoraleji'd say16:12
melwittthey could, but they'd have to know what cell to automatically map to16:13
mriedemthe compute nodes self registering with a cell mapping requires an upcall from the compute to the api which we were trying to avoid,16:13
mriedembut is what alaski's WIP patch was for i think16:13
mwhahahabut if you have the transport url, isn't it just a message?16:13
melwittif you have several, they'd have to automatically go to some 'staging cell' before you move them to where you actually want them. I think that's why it's not straightforward to be automatic. unless you're talking single cell16:13
mwhahahai mean whats the point of adding the transport url to the cell config16:13
alaskimriedem: yep, but I was going for an HTTP call which sidesteps it a bit. it's assuming there's an admin API for registering a node in a cell16:13
alaskimelwitt: I'm not sure there was a conscious decision to not allow mapping an empty cell, at least not that I'm aware of.16:14
melwittmwhahaha: that's for the api cell to call into a compute cell. when the cell mapping is created in the api, the compute cell's transport_url is part of the mapping so the api cell can talk to it16:14
alaskimelwitt: it is a bit strange to stand up a cell if there are no ready computes, but I can see why people would want that16:15
melwittalaski: okay. all the code is written that way so I think I assumed there was some consensus I missed :P16:15
melwittyeah. it only makes sense in the fresh install case16:15
alaskimelwitt: it's also possible I'm forgetting some discussion that happened16:15
mwhahahaalaski: i would respectfully disagree because as you deploy, people don't configure comptues first16:15
*** cdelatte has joined #openstack-nova16:15
mwhahahaalaski: controllers (and their config) generally are first which is why this is a problem16:16
mriedemhonestly i think we probably just always thought about cells v2 in an upgrade scenario, not fresh install16:16
mriedemfor the most part16:16
alaskiyeah16:16
alaskifor the initial deploy an empty cell makes more sense, but standing up a second cell wouldn't typically happen if there are no computes for it16:16
melwittokay. well, I saw andrey has a review up for creating an empty cell16:17
mwhahahano it still makes sense16:17
mwhahahabecause  you're growing your cloud, so you create the cell config and populate it with the new nodes16:17
alaskimwhahaha: I'm not saying it doesn't make sense, but it also makes sense to say don't stand up a new cell if there are no computes for it yet16:17
cfriesenmdbooth: are you there?16:18
melwitttypically I think we've thought of adding cells as something that you do while the cloud is running, not during code deploy. right?16:18
alaskiyes16:19
melwittit's just this initial state problem16:19
mdboothcfriesen: Yep16:20
*** ijw has joined #openstack-nova16:21
mwhahahaalaski: so i guess i'm not clear on what you mean 'to say don't stand up a new cell if there are no computes for it'16:21
cfriesenre: my question at https://review.openstack.org/#/c/337160/17/nova/virt/libvirt/driver.py.   I'm trying to figure out why we unconditionally call root_disk.create_snap().  Why is it needed for resize/migration to a different host?16:21
mwhahahaalaski: do you mean as a recommendation to operators that you shouldn't have an empty cell running?16:21
EmilienMI agree, compute nodes are usually deployed at the very end16:21
*** dtp has joined #openstack-nova16:22
mwhahahaalaski: what's the impact of the empty cell? would break something?16:22
EmilienMI think part of the reason is because in the past it used to requires some API to be running16:22
mdboothcfriesen: It's used for rollback of Rbd disks.16:22
alaskimwhahaha: I'm saying that as a deployer I would not add a new cell to the system unless I had computes ready to add to that cell. I'm not going to stand up a new cell that won't do anything yet16:22
mdboothcfriesen: IIRC it's a no-op for all other backends.16:22
mdboothHowever, cold migration/resize have a resize->confirm/rollback workflow16:23
mwhahahaalaski: well if i know i'm goign to have computes, i'd want to create the cell but wait to add the computes until i get to that phase of my install16:23
mwhahahaalaski: that's the thing, fuel, kolla, tripleo we all do controller configs first then add computes later because we need to have keystone and stuff running first16:23
mdboothFor disk-based local storage, we copy the instance dir, whether single host or migrating host16:23
mdboothFor rbd, they're the same disks, so if we didn't snapshot we wouldn't be able to rollback.16:24
alaskimwhahaha: an empty cell should not break anything afaik, but I've been out of cells for a bit now so I may be mistaken.16:24
cfriesenmdbooth: okay, that makes sense.  missed the fact it was a no-op for most cases.16:24
mwhahahaalaski: sure, just trying to understand this better16:24
mriedemalaski: is your perspective on that model biased to how it was done with cells v1?16:24
*** raunak has quit IRC16:24
mriedeman empty cell shouldn't break anything, no16:25
mdboothcfriesen: It's a reasonable hook. Possible other storage could be cleverer under some circumstances, but Rbd requires it.16:25
alaskimriedem: very likely, I'm irreparably damaged from v116:25
mriedemalaski: you're safe now, it's OK :)16:25
openstackgerritSven Anderson proposed openstack/nova: Fix suggested database migration command  https://review.openstack.org/42006016:25
cfriesenmdbooth: agreed, it's fine.  I just didn't understand what was going on.16:25
* mwhahaha hides cell v1 under alaski's bed16:25
mriedemok so i've updated https://wiki.openstack.org/wiki/Nova-Cells-v2#Nova_Cells_v2 with bugs, todos and open questoins16:26
mriedem*questions16:26
alaskinote that I'm not trying to argue for not allowing an empty cell, just explaining why that assumption was in my head when writing code which likely affected the direction16:26
*** ijw has quit IRC16:26
dtpdansmith melwitt i'm at your disposal today16:27
mriedemalaski: yup16:27
dimsmriedem : bauzas : https://review.openstack.org/#/c/419476/ gets the full cinder/rally test working. that test does a nova boot/attach/detach/delete etc. should be easy to review as well (almost all code protected with six.PY3 checks so wont regress existing stuff too)16:28
melwittdtp: dansmith is on PTO today FYI16:28
dtpok16:28
* dims waves to alaski 16:29
*** annegentle has joined #openstack-nova16:29
alaskio/16:29
melwittso things used to work prior to that patch because the first run of simple_cell_setup created cell0 and then failed when it found no compute nodes, then the retry found cell0 and returned 016:29
melwittso it continued on without creating any cell mapping16:30
bauzasmelwitt: yup16:30
*** markus_z has quit IRC16:31
dtpmelwitt i'll just be reviewing your and dan's changes under your blueprints.  lemme know if there's something else i should do16:33
*** kaisers has joined #openstack-nova16:35
dtpif i don't respond immdediately, i'm trapped in a meeting16:35
openstackgerritBalazs Gibizer proposed openstack/nova: handle uninited fields in notification payload  https://review.openstack.org/41585716:36
mriedemmelwitt: so the question right now is shoud this be a 1 or a 0 https://github.com/openstack/nova/blob/3ec43d81c324d8229bbb4b5db301175886a049b6/nova/cmd/manage.py#L111416:36
*** pcaruana has joined #openstack-nova16:36
*** rmart04 has quit IRC16:37
mriedemthat's also called from the map_cell_and_hosts CLI https://github.com/openstack/nova/blob/3ec43d81c324d8229bbb4b5db301175886a049b6/nova/cmd/manage.py#L131616:37
mriedemnote the comment in there about what the return codes mean (0 everything is done, 1 means you have more to do)16:37
melwittyeah. I think the 1 is supposed to be a signal that "hey, I didn't create any cell mappings other than cell0"16:38
mriedemi think i took issue with that a bit during a review because i consider >0 == error16:38
sdaguewhich is the standard unixism16:38
melwittyeah16:38
sdagueset -e on bash, and this will blow up any script running it16:38
bauzasmriedem: I think that if we wanna have simple_cell_setup idempotent, it's acceptable to have it return 0 if you have no hosts16:39
*** tjones has joined #openstack-nova16:39
mriedemright i tend to think that simple_cell_setup should not return >0 unless there is a real error16:39
melwittthat's what I'm not sure of. I thought the command means "create cell mappings" and if there are no hosts, it doesn't16:39
bauzasI agree16:39
bauzaswell, I agree with mriedem16:40
mriedemmap_cell_and_hosts *should* return 1 if there are no computes16:40
owalshmriedem: what's worse is that unhandled exceptions will return 116:40
mriedembecause you're specifically telling it to map hosts to cell mappings and if there are no computes, it's an error16:40
melwittmriedem: okay, I see what you're getting at16:40
mriedemowalsh: well it depends on what the exception is16:40
bauzasmriedem: I agree (again)16:40
bauzaspeople could put simple_cells_setup in a cron and leave it run16:41
*** yamahata has joined #openstack-nova16:41
mriedemwe also need to, at some point in the near future, probably do an audit of the various commands and figure out in which scenarios you'd use them16:41
mriedembecause we have a glut of CLIs for this right now16:41
owalshmriedem: anything uncaught that subclasses Exception it caugh in __main__16:41
mriedemowalsh: i don't see a problem with that16:41
bauzasmriedem: melwitt: fancy leaving me submit a patch for changing the return code to 0 ?16:42
mriedemowalsh: if we can't connect to the database and it raises an exception, then i think we should return a non-0 exit code16:42
dimsmriedem : "IBM PowerKVM" CI uses github to pull stuff? "GIT_BASE=git://github.com"  (http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/nova/76/419476/11/check/tempest-dsvm-full-xenial/2649398/console.html)16:42
bauzasmriedem: yup, if we can't connect, it's a very different situation16:42
mriedemdims: ask mmedvede16:42
bauzassomething is messed up16:42
owalshmriedem: yees, but having 1 == on computes, 2 == something else doesn't communicate any info16:42
owalshs/on/no/16:42
* bauzas firing a patch16:42
dimsmmedvede : looks like "IBM PowerKVM CI" got busted when github went down16:42
*** lamt has joined #openstack-nova16:42
*** mlavalle has joined #openstack-nova16:42
*** bnemec is now known as beekneemech16:43
mriedembauzas: i think that's something we should at least put up a patch for yeah for bug 1656276 - make simple_cell_setup return 0 if map_cell_and_hosts returns None, and change map_cell_and_hosts (the CLI) to return 1 if there is nothing mapped16:43
openstackbug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged] https://launchpad.net/bugs/165627616:43
mriedemowalsh: i'm saying i don't like a non-0 exit code communicating non-error conditions,16:43
bauzasmriedem: yup, and you reported a bug too ?16:43
mriedemi.e. 1 == no computes16:43
mriedembauzas: that's bug 165627616:43
mriedemit's what tripleo is hitting16:43
rfolcodims, looking16:44
mmedvededims: yeah, we had problems pulling from git.o.o at some point16:44
bauzasmriedem: I was thinking of https://bugs.launchpad.net/nova/+bug/165601716:44
openstackLaunchpad bug 1656017 in OpenStack Compute (nova) "nova-manage cell_v2 map_cell0 always returns a non-0 exit code" [Medium,Confirmed]16:44
mmedvedethat is the reason16:44
mriedembauzas: that's different16:44
dimsmmedvede : interesting, talked to our infra folks?16:44
bauzasmriedem: but yeah, the patch I'm writing on is against 165627616:44
bauzasmriedem: ie. changing what we return16:44
mriedemdtp: you could tackle bug 1656017 if you want16:44
mriedemdtp: that fix should be pretty straight-forward16:45
mmedvededims: it appeared to be a problem with our internal network, so I did not discuss with infra16:45
dtpok, i'll check it out16:45
dimsmmedvede : ack. thanks16:45
mriedembauzas: melwitt: the only reservation i have is that simpe_cell_setup will now return 0 for an incomplete setup16:45
mriedembauzas: melwitt: and we need to understand the implications of that16:45
mmedvededims: do you think there are inherent problems with using github vs git.o.o?16:45
melwittmriedem: yeah, that's what I was thinking about16:46
mriedemit might just mean, print a message saying you need to run map_cell_and_hosts or discover_hosts after you've added compute nodes16:46
bauzasmriedem: I know a bit of puppet world and idempotence, and I think it's reasonable16:46
bauzasI mean, it's a reasonable assertion that an idempotent script can only provide a portion of what it can modify on the first run16:46
mmedvededims: ok, I see your comment above now re our CI going down with github16:46
dimsmmedvede : just surprised. you may get a stronger reaction from infra folks if you tell them :)16:46
dimsmmedvede : right16:46
owalshmriedem: yea, I don't like it either, but if we do want to communicate a specific condition in this way then non-zero, but not 116:46
bauzasbut eventually, after X runs, it would do all required16:46
*** READ10 has joined #openstack-nova16:47
*** jose-phillips has joined #openstack-nova16:47
mriedembauzas: melwitt: note that with the nova-status upgrade check command,16:47
mriedemwe don't consider it a failure if you don't have any computes yet https://github.com/openstack/nova/blob/3ec43d81c324d8229bbb4b5db301175886a049b6/nova/cmd/status.py#L16816:47
mriedembut you have cell mappings and cell016:47
mriedemwhich is specifically for the fresh install case16:47
melwittmriedem: right, that's what I was trying to work backward from16:47
melwittmriedem: I like your idea about printing a message if simple_cell_setup finds no hosts, with a reminder what to do once you have compute hosts. the thing is the incomplete setup, things actually work fine for now because we have a bunch of fallbacks if no cell mappings, we run the old way16:49
rfolcodims, stronger reaction means they disagree ?16:50
*** Apoorva has joined #openstack-nova16:50
dimsrfolco : i think they would prefer direct git.openstack.org for sure16:50
*** liverpooler is now known as yrabl|lunch16:51
mriedemmelwitt: hmm, the db schema migration will fail if you don't have cell mappings,16:51
mriedemmelwitt: but if you run that before adding flavors, then it's a noop16:51
bauzasmriedem: melwitt: interesting, we return 0 in map_cell_and_hosts even if we have no hosts...16:51
rfolcodims, ah ok. thanks for the ping.16:52
mriedemso you could run the api_db sync, it passes b/c no flavors, then you run simple_cell_setup which only creates cell0, and then you register your computes - you won't have any cell mappings for those16:52
mriedembauzas: yeah i think that's part of the bug fix16:52
mriedemas noted above16:52
*** lamt has quit IRC16:52
mriedemmelwitt: i'm pretty sure we are dumping warnings in the logs if we don't find a cell mapping and fallback to the main db16:52
bauzasmriedem: yeah, I will change that16:52
melwittyeah, I can't remember how that got decided. probably the argument of "error condition vs not"16:53
mriedemmelwitt: the automatic compute node / cell mapping registration would fix this...but it's just not something we've been serious about adding16:53
mriedemthinking about gradually increasing levels of pain, which has been our MO,16:54
melwittmriedem: yeah. it's not straightforward how to do that because, how do you make assumptions about what cell to auto map new hosts to? usually that's deliberate decision on the part of the operator16:54
mriedemwe could do the fallback with a warning in ocata, then make that an error in pike16:54
dtpafk a bit16:54
mriedemmelwitt: right you'd need to probably flag some cell mapping as the 'default' to auto-register with16:54
mriedemso in ocata we have the fallback mechanism if we don't find a cell mapping,16:55
melwittyeah, that's what I had in mind. new stuff goes into some "staging cell" and then after that you have to re-map. though I don't even think we have a way to re-map16:55
mriedemwhich buys us some time to consider an auto-registration in pike, or remove the fallback and make it an error16:55
mriedemmelwitt: we'd probably have to write a REST API for remapping, or nova-manage CLI16:56
melwittyeah, probably nova-manage16:56
*** bvanhav_ has joined #openstack-nova16:56
*** bvanhav has quit IRC16:57
mriedemok i've added those questions to https://wiki.openstack.org/wiki/Nova-Cells-v2#Open_Questions16:58
melwittmriedem: I'm wondering if maybe map_cell_and_hosts should stay as returning 0 (since it prints what the deal is, if it found nothing to map)16:58
bauzasmriedem: melwitt: patch is wired, just a few mins for adapting tests16:59
*** ralonsoh has quit IRC16:59
melwittand later code warnings let you know that you need to do something, if no cell mappings are found while nova is running16:59
mriedemmelwitt: _map_cell_and_hosts treats that differently17:00
mriedemif things are already done, it returns a uuid https://github.com/openstack/nova/blob/3ec43d81c324d8229bbb4b5db301175886a049b6/nova/cmd/manage.py#L127517:00
mriedemif there are no computes, it returns None17:00
mriedemNone is the error, the uuid is the non-error we're already done case17:00
melwittokay, yeah17:01
bauzasmriedem: melwitt: is the proposed patch http://paste.openstack.org/show/594860/17:01
*** sudipto has quit IRC17:01
*** sudipto_ has quit IRC17:01
melwittI think we have had this debate before, is doing nothing really an error. I think we usually say it isn't17:02
melwittfrom the perspective of returning a non-zero17:02
mriedemit probably depends on the command17:04
*** ociuhandu has quit IRC17:07
*** sahid has quit IRC17:07
mriedemsdague: melwitt: if you're not otherwise distracted, can you take a look at this https://review.openstack.org/#/c/398472/ - would be good to get that in and avoid more rebasing17:08
*** burt has joined #openstack-nova17:08
*** derekh has quit IRC17:08
mriedembauzas: when you're done with this bug fix are you going to address my comments on the resource providers 'resources' query patch17:09
mriedem?17:09
*** cdent has joined #openstack-nova17:09
bauzasmriedem: I had an hard day17:09
bauzasmriedem: but yeah it's in my pipe17:09
mriedemheh, ok :)17:09
mriedemjay might be able to make those updates too but he's not around right now17:10
*** raunak has joined #openstack-nova17:11
bauzasmriedem: no worries, today is not ended for me17:12
dimsmriedem : sdague : the py35 change for nova boot/attach/detach/delete working is all green - https://review.openstack.org/#/c/419476/17:12
bauzasI'm not Jack Bauer but there 24 hours in a day17:12
mriedemsean-k-mooney: if you're willing it'd be great to rebase this series https://review.openstack.org/#/c/410737/17:13
mriedemthe merge conflict is i think b/c of the os-vif unit test fix i made yesterday17:13
mdboothbauzas: Jack Bauer would say that if he was trying to maintain cover...17:14
mdboothJust saying17:14
sean-k-mooneymriedem: sure if francesco is not around i have i can rebase them17:14
mriedemsean-k-mooney: i don't know his irc nick,17:14
mriedemor if he's even on irc but don't see him17:14
mriedemfrasantoro - nope17:15
sean-k-mooneyim just finishing up another commit but ill try and rebase this in the next hour or so if that works for you.17:15
mriedemsounds good, i'll get to it eventually if you can't, it's fine17:15
mriedemi'm going to head to lunch, bbiab17:15
*** rmart04 has joined #openstack-nova17:16
*** nic has joined #openstack-nova17:16
*** rmart04 has quit IRC17:17
openstackgerritSylvain Bauza proposed openstack/nova: Accept simple_cell_setup to partially complete its tasks  https://review.openstack.org/42007917:17
bauzasmelwitt: mriedem: ^17:17
*** lamt has joined #openstack-nova17:18
*** rwmjones|hols is now known as rwmjones17:19
*** mriedem1 has joined #openstack-nova17:19
*** mriedem has quit IRC17:20
*** swamireddy has joined #openstack-nova17:25
*** swebster has quit IRC17:27
*** rmart04 has joined #openstack-nova17:28
*** rmart04 has quit IRC17:28
*** nherciu_ has joined #openstack-nova17:28
*** eharney has quit IRC17:29
*** gszasz has quit IRC17:29
*** nherciu has quit IRC17:29
*** slaweq has quit IRC17:30
*** kaisers has quit IRC17:30
*** nherciu__ has joined #openstack-nova17:31
*** patriciadomin has joined #openstack-nova17:31
dimsthanks sdague !17:31
*** slaweq has joined #openstack-nova17:32
*** lpetrut has quit IRC17:33
*** nherciu_ has quit IRC17:34
*** mvk has joined #openstack-nova17:38
*** yamahata has quit IRC17:38
*** ri0 has joined #openstack-nova17:40
*** _ducttape_ has joined #openstack-nova17:41
*** tbachman has quit IRC17:41
*** rama_y has joined #openstack-nova17:41
openstackgerritJohn Garbutt proposed openstack/nova: WIP: Call preallocate_for_instance before scheduler  https://review.openstack.org/41312417:42
openstackgerritJohn Garbutt proposed openstack/nova: WIP: Add processed_requests to allocate_for_instance  https://review.openstack.org/42010317:42
*** ducttape_ has quit IRC17:44
*** cdent has quit IRC17:44
*** salv-orlando has joined #openstack-nova17:45
*** unicell has quit IRC17:47
*** ri0 has left #openstack-nova17:48
*** eharney has joined #openstack-nova17:51
openstackgerritSarafraj Singh proposed openstack/nova: Adopts keystoneauth with glance client.  https://review.openstack.org/41263417:52
cfriesenany libvirt experts around?   When nova calls self._domain.resume(), is that an async call or a sync call?  i.e. when it returns has qemu actually transitioned to a "running" state, or does it just send a message but it could be delayed due to scheduling?17:55
*** jwcroppe has quit IRC17:56
*** ociuhandu has joined #openstack-nova17:58
*** jwcroppe has joined #openstack-nova17:59
openstackgerritSarafraj Singh proposed openstack/nova: Adopts keystoneauth with glance client.  https://review.openstack.org/41263417:59
*** satyar has quit IRC18:01
openstackgerritSarafraj Singh proposed openstack/nova: Adopts keystoneauth with glance client.  https://review.openstack.org/41263418:01
openstackgerritJohn Garbutt proposed openstack/nova: WIP: Call preallocate_for_instance before scheduler  https://review.openstack.org/41312418:03
*** Guest45759 is now known as dansmith18:04
*** jpena is now known as jpena|brb18:04
*** ijw has joined #openstack-nova18:04
*** hieulq_ has quit IRC18:06
*** links has quit IRC18:12
*** swebster has joined #openstack-nova18:15
*** browne has joined #openstack-nova18:16
*** yamahata has joined #openstack-nova18:16
*** Swami has joined #openstack-nova18:22
jrollmriedem1: I'm +1 on the ironic attach/detach/portgroups patches, ruby is out today but I expect she'll be happy with them too18:24
jrollthe soft power client patch needs another re-spin18:24
jrollI think that's all I have to update you on...18:25
jrolloh, the hashring thing we did last cycle, that's released in tooz now, do y'all have time for a patch to move to use tooz this cycle yet? (should be mostly ripping code out)18:25
*** _ducttape_ has quit IRC18:25
*** pcaruana has quit IRC18:26
*** ducttape_ has joined #openstack-nova18:26
*** kaisers has joined #openstack-nova18:27
*** owalsh has quit IRC18:28
*** unicell has joined #openstack-nova18:28
*** owalsh has joined #openstack-nova18:29
*** tbachman has joined #openstack-nova18:30
*** mriedem1 is now known as mriedem18:33
mriedemjroll: umm, maybe18:33
mriedemjroll: there was also the inject nmi patch but i think that was waiting on some dependencies too18:34
mriedemcfriesen: i usually default to think that anything libvirt/qemu does is async18:34
jrollmriedem: yeah, that's not even done in ironic, I don't expect it this cycle18:34
mriedemi think most things are done via an async job18:34
mriedemcfriesen: kashyap or mdbooth or sahid are the people i'd ask18:35
mriedemjroll: ok18:35
jrollmriedem: I'll do the tooz thing now so you can see how simple it is. it passed tests with flying colors about a month ago with some weird depends-ons18:36
mriedemok, just got back from lunch and still sore from the cells v2 beating i took this morning18:37
openstackgerritJim Rollenhagen proposed openstack/nova: Move to tooz hash ring implementation  https://review.openstack.org/39902218:37
jrolloh, easiest merge conflict ever18:37
jrollthere you are if curious18:37
openstackgerritJim Rollenhagen proposed openstack/nova: Move to tooz hash ring implementation  https://review.openstack.org/39902218:38
jrollmuch better with the right version18:39
*** pcaruana has joined #openstack-nova18:39
*** sree has joined #openstack-nova18:41
*** tbachman has quit IRC18:41
*** dharinic is now known as dharinic|lunch18:42
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] Ignore me - ry py35 functional with a clean slate  https://review.openstack.org/41996818:43
*** lpetrut has joined #openstack-nova18:43
*** sree has quit IRC18:46
*** jwcroppe has quit IRC18:48
*** tbachman has joined #openstack-nova18:51
cfriesenmriedem: thanks...I did some experiments and it looks like it's actually sync...if I send a SIGSTOP to qemu or libvirtd the nova call to self._domain.resume() hangs until I send a SIGCONT18:51
*** jwcroppe has joined #openstack-nova18:53
openstackgerritDan Peschman proposed openstack/nova: nova-manage cell_v2 map_cell0 exit 0  https://review.openstack.org/42013218:53
*** tesseract has quit IRC18:53
dtpmriedem ^ going to lunch18:54
harlowjadtp thx for helping here :)18:54
dtpwhy you're welcome harlowja! :)18:55
harlowja:-P18:55
*** openstack has joined #openstack-nova18:59
sean-k-mooneymriedem: i have resolved most of the erros in francesco's patches locally but one of his changes need the fastpath os-vif plugin to work which is incorrect http://logs.openstack.org/61/385061/7/check/gate-nova-python27-db-ubuntu-xenial/336f68e/console.html#_2017-01-12_23_22_43_32559519:01
*** kaisers has quit IRC19:01
*** lpetrut has quit IRC19:01
sean-k-mooneymriedem: im going to invert the order of his patches so https://review.openstack.org/#/c/410737/ can merge19:01
sean-k-mooneyor at least be unblocked by the gate19:02
*** gouthamr has joined #openstack-nova19:02
*** pcaruana has quit IRC19:02
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] Ignore me - ry py35 functional with a clean slate  https://review.openstack.org/41996819:05
*** satyar has joined #openstack-nova19:06
*** lpetrut has joined #openstack-nova19:09
*** jpena|brb is now known as jpena19:09
mriedemsean-k-mooney: sounds good to me19:11
*** jamielennox is now known as jamielennox|away19:12
*** jpena is now known as jpena|away19:17
*** dharinic|lunch is now known as dharinic19:17
mriedemGET /jaypipes19:19
openstackgerritsean mooney proposed openstack/nova: os-vif: convert libvirt driver to use os-vif for fast path vhostuser  https://review.openstack.org/38506119:19
openstackgerritsean mooney proposed openstack/nova: os-vif-util: set vif_name for vhostuser ovs os-vif port  https://review.openstack.org/41073719:19
*** markvoelker_ has quit IRC19:19
openstackgerritLee Yarwood proposed openstack/nova: WIP - libvirt: workaround findmnt behaviour change  https://review.openstack.org/42014619:20
*** markvoelker has joined #openstack-nova19:20
openstackgerritLee Yarwood proposed openstack/nova: WIP - libvirt: workaround findmnt behaviour change  https://review.openstack.org/42014619:25
*** markvoelker has quit IRC19:25
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] Ignore me - ry py35 functional with a clean slate  https://review.openstack.org/41996819:35
mriedemjroll: is there any way for you to tell if this job failed for a known ironic timeout http://logs.openstack.org/73/398473/22/check/gate-tempest-dsvm-ironic-ipa-wholedisk-agent_ipmitool-tinyipa-ubuntu-xenial-nv/3dbaaef/ ?19:35
openstackgerritLee Yarwood proposed openstack/nova: libvirt: Use the mirror element to detect when to pivot  https://review.openstack.org/39973519:35
mriedemjroll: b/c it failed on https://review.openstack.org/#/c/398473/ which is moving a bunch of parts around in the compute resource tracker19:35
jrollmriedem: so, this is pretty typical http://logs.openstack.org/73/398473/22/check/gate-tempest-dsvm-ironic-ipa-wholedisk-agent_ipmitool-tinyipa-ubuntu-xenial-nv/3dbaaef/console.html#_2017-01-13_16_02_01_11148819:36
jrollbut could also be because pxe booting got broken somehow or whatever19:36
mriedemit passed on the patch before and after that one so i assume it's not due to the actual change being tested19:37
jrollwtf empty console logs19:37
jrollyeah, probably not then19:37
jrollmriedem: sometimes node* here is indicative (you'll see it cut off during boot or whatever) but something went wrong and they're empty http://logs.openstack.org/73/398473/22/check/gate-tempest-dsvm-ironic-ipa-wholedisk-agent_ipmitool-tinyipa-ubuntu-xenial-nv/3dbaaef/logs/ironic-bm-logs/19:38
mriedemcool, just rechecked it19:38
jrollcool19:38
lyarwoodmriedem: is the placement job voting in the stable/newton gate btw?19:38
jrollwe've been seeing timeouts creep back up lately, unclear why19:38
mriedemlyarwood: no19:38
lyarwoodmriedem: I just ack'd the two reviews from yesterday, they both failed the placement job a few days ago19:38
mriedemlyarwood: the placement job was failing on newton b/c of an infra issue with the job definition, it was trying to call a post test hook which didn't exist19:39
mriedemi fixed that on wednesday19:39
lyarwoodargh, they failed on Thursday19:40
mriedemlyarwood: maybe i fixed it thursday :)19:40
mriedemhttp://logs.openstack.org/17/419217/1/check/gate-tempest-dsvm-neutron-placement-full-ubuntu-xenial-nv/086c49e/logs/devstack-gate-post_test_hook.txt.gz19:40
mriedembut ^ was the issue19:40
mriedemhttps://review.openstack.org/#/c/419243/19:41
lyarwoodah cool19:41
mriedemyeah it was yesterday19:41
mriedembeen a long week19:41
lyarwoodyeah, tell me about it19:41
mriedemthe other series in newton for placement that i have here https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/newton+topic:bug/165167819:41
mriedemone of the changes towards the end is failing functional tests for some reason which i need to investigate19:42
mriedemso you can hold off on that series19:42
lyarwooddon't worry I was going to :)19:42
mriedemheh19:42
mriedemi mean at quarter to 8 on a friday, reviewing that series is about as much fun as you should have19:42
lyarwoodhaha aye, unfortunately my presence is required downstream to fight another fire, otherwise I'd be all over that at this time on Friday ;)19:44
*** Guest87783 is now known as med_19:47
*** med_ has joined #openstack-nova19:47
*** med_ is now known as medberry19:47
*** eglynn has quit IRC19:48
dimsmriedem : so got one +2 from sdague and a review from cdent in the morning. any chance for getting it merged today? https://review.openstack.org/#/c/419476/19:51
*** gouthamr has quit IRC19:51
dimsmriedem : thanks in advance!19:51
mriedemnot from me right now, i have other fires19:51
* mriedem puts out the nova core distress signal19:52
dims:) no worries. just did not want to keep asking19:52
*** dimtruck is now known as zz_dimtruck19:52
dimsmelwitt : how about you? :)19:52
melwittmaybe later, working on something related to cells v2 fire right now19:53
dimsack thanks melwitt19:54
*** amoralej is now known as amoralej|off19:54
*** gouthamr has joined #openstack-nova19:55
*** kaisers has joined #openstack-nova19:57
*** openstackstatus has quit IRC19:57
*** openstack has joined #openstack-nova19:59
*** david-lyle is now known as baling-wire20:00
*** nherciu__ has quit IRC20:01
sdaguetiming on that sucks, mostly with dansmith heading out to the wood with only a ham radio20:02
mriedemwell i think he's around a bit yet20:02
mriedembut yes20:02
melwitthe said he'll be around this afternoon, we'll have him on monday too. but starting tuesday he's out next week20:03
mriedemok i've replied to the ML thread20:05
mriedemwith my brain dump20:05
dimshmm, looks like cdent is done for the day as well. seeing some gabbit related issues20:06
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] Ignore me - ry py35 functional with a clean slate  https://review.openstack.org/41996820:06
mriedemsdague: we also need https://review.openstack.org/#/c/392569/ updaed20:08
mriedem*updated20:08
mriedemalthough i haven't gone through that whole patch yet20:09
mriedemand the things i pointed out already probably aren't the end of the world20:09
melwittmriedem: I'm fixing up the create_cell patch right now, just fyi20:10
*** psachin has joined #openstack-nova20:10
mriedemmelwitt: yup, i know (already) :)20:11
*** liusheng has quit IRC20:11
*** eharney has quit IRC20:11
*** liusheng has joined #openstack-nova20:11
EmilienMmriedem: thanks for your help, very appreciated20:12
mriedemyw20:13
*** tbachman has quit IRC20:17
*** READ10 has quit IRC20:19
*** rfolco has quit IRC20:19
*** moshele has joined #openstack-nova20:23
*** zz_dimtruck is now known as dimtruck20:24
*** eharney has joined #openstack-nova20:26
*** openstackstatus has quit IRC20:27
*** openstack has joined #openstack-nova20:29
*** moshele has quit IRC20:30
*** salv-orlando has quit IRC20:32
*** bvanhav_ has quit IRC20:36
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] Ignore me - ry py35 functional with a clean slate  https://review.openstack.org/41996820:41
*** cdelatte has quit IRC20:43
*** jaosorior has quit IRC20:45
*** jaosorior has joined #openstack-nova20:45
openstackgerritSlawek Kaplonski proposed openstack/nova: Set sysinfo_serial="none" in LibvirtDriverTestCase  https://review.openstack.org/42017620:46
openstackgerritMatt Riedemann proposed openstack/nova: Make metadata server know about cell mappings  https://review.openstack.org/41796220:48
openstackgerritMatt Riedemann proposed openstack/nova: Multicell support for instance listing  https://review.openstack.org/39677520:48
openstackgerritMatt Riedemann proposed openstack/nova: Make api_samples tests use simple cell environment  https://review.openstack.org/41047320:48
openstackgerritMatt Riedemann proposed openstack/nova: WIP Move instance creation to conductor  https://review.openstack.org/31937920:48
*** marst has quit IRC20:49
*** slaweq has quit IRC20:50
*** satyar has quit IRC20:51
*** annegentle has quit IRC20:55
*** annegentle has joined #openstack-nova20:59
*** claudiub has joined #openstack-nova21:00
mriedemsdague: i'm cool with this pretty simple patch https://review.openstack.org/#/c/417962/21:04
*** claudiub|3 has quit IRC21:04
*** lpetrut has quit IRC21:05
*** ijw has quit IRC21:05
*** browne has quit IRC21:07
openstackgerritmelanie witt proposed openstack/nova: Add nova-manage cell_v2 create_cell command  https://review.openstack.org/33271321:08
*** browne has joined #openstack-nova21:09
*** kaisers has quit IRC21:10
*** rha has quit IRC21:14
openstackgerritsean mooney proposed openstack/os-vif: [DO NOT MERGE] this a test patch for os-vif gate.  https://review.openstack.org/42017921:14
*** Apoorva has joined #openstack-nova21:16
openstackgerritDiana Clarke proposed openstack/nova: copy pasta error  https://review.openstack.org/42018021:17
diana_clarke^ easy review for Friday brains ;)21:17
*** rfolco has joined #openstack-nova21:18
*** rfolco has quit IRC21:23
*** JoseMello has quit IRC21:25
*** tbachman has joined #openstack-nova21:28
*** xyang1 has joined #openstack-nova21:28
*** gouthamr has quit IRC21:31
openstackgerritDiana Clarke proposed openstack/nova: DNM: check experimental queue  https://review.openstack.org/42018621:34
*** Jeffrey4l__ has joined #openstack-nova21:34
*** Jeffrey4l_ has quit IRC21:35
mriedemsean-k-mooney: i think https://review.openstack.org/#/c/410737/ is depending on the wrong patch21:38
mriedemthat's why gate-nova-requirements failed21:39
mriedemi'll update it quick21:39
openstackgerritMatt Riedemann proposed openstack/nova: os-vif: convert libvirt driver to use os-vif for fast path vhostuser  https://review.openstack.org/38506121:40
openstackgerritMatt Riedemann proposed openstack/nova: os-vif-util: set vif_name for vhostuser ovs os-vif port  https://review.openstack.org/41073721:40
*** weshay is now known as weshay_afk21:44
*** timello has quit IRC21:48
dtpanyone need anything re cells v2?21:51
mriedemdtp: for right now i don't think so21:51
dtpok21:51
*** timello has joined #openstack-nova21:53
*** cleong has quit IRC21:54
*** smatzek has quit IRC21:54
*** salv-orlando has joined #openstack-nova21:55
*** jamielennox|away is now known as jamielennox21:59
*** thorst_ has quit IRC22:02
*** ijw has joined #openstack-nova22:03
*** thorst has joined #openstack-nova22:03
*** eharney has quit IRC22:06
*** kaisers has joined #openstack-nova22:06
*** jdurgin1 has joined #openstack-nova22:07
*** thorst has quit IRC22:07
*** ijw has quit IRC22:07
*** ijw has joined #openstack-nova22:11
*** xyang1 has quit IRC22:21
melwittmriedem: trying to make sure I have this right in my head ... is the expectation during an upgrade to ocata that the appropriate cells v2 setup commands be run using the ocata code, before running the db sync? or is the expectation to run the setup using the newton code before starting the upgrade22:22
*** salv-orl_ has joined #openstack-nova22:22
mriedemmelwitt: in grenade we run simple_cell_setup in newton before going to ocata: https://review.openstack.org/#/c/393441/2/projects/60_nova/from-newton/upgrade-nova22:24
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] Ignore me - ry py35 functional with a clean slate  https://review.openstack.org/41996822:24
mriedembut simple_cell_setup is meant to be idempotent so you can run it as many times as needed22:24
mriedembut you're asking because we don't have create_cell in newton22:25
melwittokay. I was wondering if it's also fine to run them in ocata, just before the db sync22:25
melwittyes, pretty much22:25
*** salv-orlando has quit IRC22:25
mriedemwell, for the nova_api db sync to work, you have to have created the cell mappings and cell022:25
mriedemso (1) simple_cell_setup, (2) api-db sync22:25
melwittand also the simple_cell_setup has an enhancement in ocata to take the transport url from the conf, which is pretty important. I think it can be backported (I checked oslo.messaging stuff)22:25
mriedembecause (2) has the blocking schema migration22:25
melwittbut I was thinking, do we need to backport all this stuff or no22:25
melwittright22:26
mriedemmelwitt: i remember you asking about backporting that one, i think if it was ok w/ the oslo.messaging version in newton i think we can backport it22:26
mriedemif you haven't already done so22:26
*** baling-wire is now known as david-lyle22:26
openstackgerritDan Smith proposed openstack/nova: Add some cellsv2 setup docs  https://review.openstack.org/42019822:26
mriedemwell look who it is22:26
dansmithmriedem: melwitt ^22:26
melwittyeah, I was just wondering if it's necessary. maybe not necessary but it would be nice to have the option to run them in newton?22:26
*** ducttape_ has quit IRC22:27
dansmithafter writing that, I almost think maybe we should do away with the simple command, or at least not stress its usage22:27
dansmithonce we have the create command22:27
dansmithI think it's more straightforward to think about "create cell, discovery hosts"22:27
dansmithonce we merge the create command I will change devstack to do the same thing for cellsv1 and normal, removing the simple usage22:28
*** thorst has joined #openstack-nova22:28
melwittyeah, not surprisingly the highly niched batch command ends up being more confusing than it's worth22:29
dansmithwell, it was supposed to make it easy for people, but...22:29
mriedemdansmith: yeah i did mention something to that effect in the ML, i.e. doing different things for upgrade scripts (grenade) and fresh install scripts (devstack)22:29
mriedembut it depends on where we end up probably22:30
melwittdansmith: I dunno if you saw my earlier question but, I was asking is it equally okay to run cells v2 setup stuff after you've gotten ocata code, just make sure you do it before any db sync?22:30
openstackgerritScott DAngelo proposed openstack/nova: WIP Do Not Merge  https://review.openstack.org/42020122:30
melwittor do we need to backport all these command changes to newton22:30
mriedemif create_cell is going to be required for upgrade, then we're going to have to backport it to newton and change grenade22:30
dansmithyeah22:30
dansmithalthough the simple command will still work for people upgrading,22:30
melwittokay22:30
mriedemright22:30
mriedemi like simple_cell_setup in grenade, but not devstack22:31
dansmithso maybe we just backport changes to it to make it work, and bill it as the "get you to ocata command" and then do away with it later22:31
dansmithyeah22:31
mriedemsure22:31
scottdamriedem: That patch fixes the problem with not having cinderv3 in Nova's catalog ^^^^22:31
dansmithif one of you will skim that patch to see if there's anything majorly wrong with it, then I can send it to the ML on that thread for some wider view22:31
*** thorst has quit IRC22:31
mriedemomfg my eyes https://review.openstack.org/#/c/420201/1/nova/context.py22:32
mriedemscottda: you could have thrown that in the other WIP couldn't you?22:32
mriedemnot sure how many DNM patches we are going to have here :)22:32
melwittdansmith: the main thing I noticed is I wouldn't want to imply the db and mq urls are required by putting them in the examples. operators really don't want to put secret creds on the command line22:33
dansmithmelwitt: it just seems clearer to me to do that22:33
*** smatzek has joined #openstack-nova22:34
scottdamriedem: Yeah, I'm going to abandon the first...was too messy to keep and change all the logic.22:34
mriedemyou can make a note that it's for illustration but simple_cell_setup will read the transport url and database connection url from nova.conf22:34
melwittI've been noticing feedback about it and think it will make people think they have to put them on the command line22:34
dansmithmelwitt: because if you don't, then you don't see the magic that $db_cell0 is used, etc22:34
dansmithmelwitt: then it's just three commands you run, but you don't know what thing in the config file goes with what22:34
melwittokay. maybe we'll just note that it's not required in words in each section or something22:34
melwittbecause I think in real life, operators will not pass those things on the command line22:35
dansmithmelwitt: yeah, I can add something about that22:35
*** smatzek has quit IRC22:35
melwittit's more about what --config they pass to nova-manage22:35
melwittthat will determine where it gets the values from22:35
*** smatzek has joined #openstack-nova22:35
dansmithmelwitt: the thing is, I think the current situation we have with doing everything from the config file is just really obscure22:35
openstackgerritScott DAngelo proposed openstack/nova: WIP Do Not Merge  https://review.openstack.org/42020122:35
dansmithmelwitt: when writing that, I started without the args, and had to go look up how we figure out what the values would be22:36
melwittdansmith: it is. so I'm cool with just notes that they don't have to do that and probably shouldn't in a real scenario22:36
melwittthat it will pull the values from the nova.conf passed to nova-manage --config <nova.conf>22:37
*** catintheroof has quit IRC22:38
*** annegentle has quit IRC22:42
melwittdansmith: overall looks reasonable to me22:42
mriedemi don't consider those whitespaces as reasonable22:42
*** sree has joined #openstack-nova22:42
melwittthat's true. shame on you dansmith22:43
mriedemgdi where is jay when i need him22:44
melwittin the air, right? I thought he mentioned he was traveling today22:44
dansmithmelwitt: I feel like operators that are sensitive about their creds are also not hand-typing them into a history-preserving bash shell, but.. regardless, I think the changes I just made will appease the concern22:44
mriedemi think i figured it out22:44
melwittdansmith: what do you mean? i.e. what is the case you're talking about where they type them somewhere but don't put them in the nova-manage command?22:45
openstackgerritDan Smith proposed openstack/nova: Add some cellsv2 setup docs  https://review.openstack.org/42019822:46
*** sree has quit IRC22:47
dansmithmelwitt: not sure I grok that.. I'm just saying I think being afraid of creds on on the command line is being overly sensitive, given the lack of security we enforce on the very same systems you'd be running these on22:47
dansmithmelwitt: see what you think about that ^22:47
*** slaweq has joined #openstack-nova22:49
melwittdansmith: oh. I think at my last company they wouldn't put db/mq creds on the command line at all. but I have also gotten feedback from mwhahaha about the same issue recently. so I think there's enough reason to be really clear about it so people understand what their options are22:51
*** wasmum has joined #openstack-nova22:52
dansmithyeah, I totally get that people are worried about it, I just think it's silly :)22:52
melwittheh, okay22:52
mwhahahawell it's information leakage22:52
mwhahahait's always been a bad idea and continues to be so22:52
mwhahahaeven though many newer apps have all their stuff via command line22:53
* mwhahaha is looking at you kubernetes22:53
dansmithit didn't used to be reasonable to use secure throw-away VMs as consoles for this kind of stuff22:53
dansmithand since this should be stuff you run from an administrative console...22:53
dansmithbut regardless,22:53
dansmiththe visual correlation of which url goes with which thing is what I think is important for the examples22:54
mriedemfigleaf: can i not use env vars in a gabbit query string like this?22:54
mriedemPUT: /resource_providers/$ENVIRON['RP_UUID']/aggregates22:54
mwhahahawell it then ends up in scripts and other stuffs22:54
dansmithmuch easier to map things in your head when you're trying to grok it22:54
figleafmriedem: yes, that's pretty much how you have to do it22:54
mwhahahaanyway it's something we need to be conscious of especially if we want our apps to be consumed in the finance and government space22:55
mriedemok i was missing "content-type"22:55
mriedemoops22:55
melwittdansmith: cool, thanks. I see you kept the trailing whitespace though, mriedem wouldn't approve22:55
mriedemi was missing the content-type: application/json request header22:55
dansmithmelwitt: I fixed a few22:55
mwhahahawhether or not it's silly is moot in the eyes of regulations and audits22:55
figleafmriedem: I go through about 20 oops a day with gabbi22:55
dansmithmelwitt: thought I got them all.. my editor's rst mode is working against me22:55
melwittthat's all you're getting mriedem22:55
figleafthe only way to learn!22:55
mriedemfigleaf: yeah, copy/paste is about the only way i make this work22:56
mriedembut now i've got it22:56
mriedemok, RP ?resources= patch update coming22:56
mriedemwith a bunch more edge case testing22:56
openstackgerritMatt Riedemann proposed openstack/nova: Expose a REST API for a specific list of RPs  https://review.openstack.org/39256922:57
openstackgerritDan Smith proposed openstack/nova: Add some cellsv2 setup docs  https://review.openstack.org/42019822:57
mriedemOH F YEAH ^22:57
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] Ignore me - ry py35 functional with a clean slate  https://review.openstack.org/41996822:57
mriedembauzas: i addressed my concerns in https://review.openstack.org/39256922:57
mriedemand added a bunch more testing22:57
mriedemplus more docs about min/max unit and step size consideratoins22:58
*** jpena|away is now known as jpena22:58
mriedem*considerations22:58
*** Swami has quit IRC22:58
*** ducttape_ has joined #openstack-nova22:59
mwhahahadansmith: it should noted that these command in the case of puppet may get logged which is why I asked for using the config and not cmd line params. we have things to hide secrets when they get put into the config files but not so much for command line executions23:00
dansmithmwhahaha: I understand your plight23:00
*** pradk has quit IRC23:00
mriedemdoes it matter that we leak tokens in the logs all over the place?23:01
*** smatzek has quit IRC23:01
dansmithmwhahaha: hopefully you understand my point about illustrative examples with big fat caveats23:01
mriedemand people post the logs to bug reports and kibana23:01
mwhahahawe're not supposed to be leaking tokens anywhere23:01
mriedemif your token expires fast enough it's not an issue23:01
mriedembut...23:01
mwhahahabut yea, i get it but passwords don't change like tokens :D23:01
melwittmriedem: yeah, it matters. we had some patches to redact tokens in the logs23:01
mriedemi know, but it's whack a mole23:02
melwittyes. that's true23:02
mwhahahaalways is23:02
dansmiththese credentials are super easy for you to programmatically rotate too, if you're concerned about it23:02
mwhahahadatabase passwords are worse because many times the permissions people are setting with the service users are wide open23:02
mwhahahawell they should be, but i've done my fair share of db password rotations and it never goes well23:03
mwhahahacause someone ultimately uses it for something aren't supposed to and the world falls apart :D23:03
melwittthat depends on your infra whether it's easy to rotate them, but yeah23:04
dansmithwell, I see we're far outside the realm of realism here23:04
dansmithmelwitt: have you got anything brewing for that create_cell patch?23:04
mriedemsdague: alex_xu: i'm +2 on this now https://review.openstack.org/#/c/392569/23:04
melwittit was not a picnic when the passwords got compromised and had to be changed, in my recollection23:05
*** tblakes has quit IRC23:05
melwittdansmith: yeah, it's failing py35 test only and I'm trying to fix it23:05
dansmithoh,23:05
dansmithI see you already did23:05
mriedemalright i'm heading out23:06
mriedemwill be back on this week i'm sure23:06
mriedem*weekend23:06
melwitt*sad trombone*23:06
mriedembut getting that scheduler RPs query thing +2ed unblocks the scheduler using placement23:06
mriedemso is a high priority to get that patch in23:06
melwittis that the one you mentioned to me this morning?23:07
mriedemi don't think so?23:07
melwittokay. nvm then23:07
mriedemif you're looking, https://review.openstack.org/#/c/398472/ and https://review.openstack.org/#/c/417962/ are +2 and need to get in23:07
melwittokay yeah it was the first one you mentioned23:08
mriedemok now i'm out - i'm told there is casserole at home dontchaknow23:08
*** mriedem has quit IRC23:09
*** mdrabe has quit IRC23:10
openstackgerritAnusha Unnam proposed openstack/nova: Add keypairs field to InstancePayload  https://review.openstack.org/41973023:11
slaweqmriedem: hello, can You maybe have a look on https://review.openstack.org/#/c/420176/ ? thx in advance23:11
*** kaisers has quit IRC23:14
*** yrabl|lunch has quit IRC23:15
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: [WIP] Ignore me - Try py35 functional with a clean slate  https://review.openstack.org/41996823:24
*** marst has joined #openstack-nova23:26
*** salv-orl_ has quit IRC23:26
*** jpena is now known as jpena|off23:27
openstackgerritKen'ichi Ohmichi proposed openstack/nova: Add hacking rule to avoid unnecessary space in method  https://review.openstack.org/41758223:27
openstackgerritmelanie witt proposed openstack/nova: Add nova-manage cell_v2 create_cell command  https://review.openstack.org/33271323:28
*** edmondsw has quit IRC23:29
*** jaosorior has quit IRC23:31
*** thorst has joined #openstack-nova23:32
*** edmondsw has joined #openstack-nova23:36
*** thorst has quit IRC23:37
*** edmondsw_ has joined #openstack-nova23:38
*** edmonds__ has joined #openstack-nova23:40
*** edmondsw has quit IRC23:41
*** edmondsw_ has quit IRC23:43
*** figleaf is now known as edleafe23:43
*** baoli has quit IRC23:46
*** tommylikehu has quit IRC23:46
*** tommylikehu has joined #openstack-nova23:46
*** edmonds__ has quit IRC23:47
*** jwcroppe has quit IRC23:47
dimsdown to 1 test in py35 functional black list. yay!23:49
*** tbachman has quit IRC23:51
*** mriedem has joined #openstack-nova23:52
*** gouthamr has joined #openstack-nova23:53
*** harlowja has quit IRC23:56
*** tbachman has joined #openstack-nova23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!