Thursday, 2017-01-26

*** Swami has joined #openstack-nova00:00
*** gouthamr has joined #openstack-nova00:02
*** catintheroof has quit IRC00:03
*** karimb has quit IRC00:03
*** browne has quit IRC00:07
*** ducttape_ has quit IRC00:08
*** baoli has quit IRC00:08
*** yuntongjin has joined #openstack-nova00:11
*** nic has quit IRC00:17
*** yuntongjin has quit IRC00:17
*** annegentle has joined #openstack-nova00:18
*** burgerk_ has quit IRC00:18
*** browne has joined #openstack-nova00:19
*** dimtruck is now known as zz_dimtruck00:21
*** annegentle has quit IRC00:23
*** zz_dimtruck is now known as dimtruck00:23
*** annegentle has joined #openstack-nova00:23
*** ducttape_ has joined #openstack-nova00:24
*** esberglu has joined #openstack-nova00:24
*** dtp has quit IRC00:27
*** tbachman has quit IRC00:28
*** raunak has quit IRC00:28
*** esberglu has quit IRC00:29
*** Sukhdev has quit IRC00:31
*** chyka has quit IRC00:31
*** chyka has joined #openstack-nova00:31
*** hongbin has quit IRC00:33
*** Sukhdev has joined #openstack-nova00:35
*** chyka has quit IRC00:36
*** raunak has joined #openstack-nova00:42
*** raunak has quit IRC00:42
openstackgerritTakashi NATSUME proposed openstack/nova: Enable cold migration with target host(1/2)  https://review.openstack.org/40895500:44
openstackgerritTakashi NATSUME proposed openstack/nova: Enable cold migration with target host(1/2)  https://review.openstack.org/40895500:44
*** rfolco has quit IRC00:45
*** rfolco has joined #openstack-nova00:45
*** raunak has joined #openstack-nova00:45
openstackgerritTakashi NATSUME proposed openstack/nova: Enable cold migration with target host(2/2)  https://review.openstack.org/40896400:45
*** rfolco has quit IRC00:46
*** vsaienko has quit IRC00:46
openstackgerritTakashi NATSUME proposed openstack/nova: api-ref: Add parameters in cold migrate action  https://review.openstack.org/41004200:46
openstackgerritTakashi NATSUME proposed openstack/python-novaclient: Microversion 2.42 - Enable cold migration with target host  https://review.openstack.org/40670700:47
openstackgerritTakashi NATSUME proposed openstack/nova: Add functional tests for cold migration to same host  https://review.openstack.org/41492600:47
*** annegentle has quit IRC00:48
*** tovin07 has joined #openstack-nova00:49
*** tovin07_ has joined #openstack-nova00:49
*** armax has quit IRC00:50
*** dimtruck is now known as zz_dimtruck00:50
*** hoangcx has joined #openstack-nova00:50
*** zz_dimtruck is now known as dimtruck00:50
*** krtaylor has joined #openstack-nova00:51
*** dave-mcc_ has joined #openstack-nova00:51
*** browne has quit IRC00:51
*** rfolco has joined #openstack-nova00:52
*** rfolco has quit IRC00:52
*** dave-mccowan has quit IRC00:53
*** phuongnh has joined #openstack-nova00:54
*** hoangcx has quit IRC00:54
*** armax has joined #openstack-nova00:56
alex_xugmann: johnthetubaguy if we think 403 is behaviour change. Then this is a behaviour change https://review.openstack.org/#/c/415142/33/nova/api/openstack/compute/servers.py@1230 ?00:58
alex_xuKevin_Zheng: ^00:58
alex_xugmann: johnthetubaguy but expose host and node to the user is kind of bug fix. it isn't same case with all_tenants which is filter we support it public00:59
alex_xus/bug fix/bug/01:00
*** dimtruck is now known as zz_dimtruck01:03
dansmithmelwitt: this is good again: https://review.openstack.org/#/c/319379/01:05
dansmithmelwitt: I broke it in a very subtle way when fixing johnthetubaguy's comment, but it's good now01:05
*** ducttape_ has quit IRC01:06
*** bjolo_ has joined #openstack-nova01:06
*** Apoorva has quit IRC01:07
*** sacharya has joined #openstack-nova01:08
*** edmondsw has joined #openstack-nova01:12
mriedemdid bauzas get the grenade thing figured out?01:13
mriedemdansmith: simple change needed in https://review.openstack.org/#/c/424243/ and then i'm +201:13
dansmithmriedem: we're waiting for a run I think01:14
*** zz_dimtruck is now known as dimtruck01:15
gmannalex_xu:01:16
gmannalex_xu:  humm,01:16
*** shaner has quit IRC01:16
*** edmondsw has quit IRC01:17
*** annegentle has joined #openstack-nova01:17
dansmithmriedem:01:17
dansmithgah01:17
mriedem_LW _MFER01:17
openstackgerritDan Smith proposed openstack/nova: Only warn about hostmappings during ocata upgrade  https://review.openstack.org/42424301:18
dansmiththere01:18
*** dave-mccowan has joined #openstack-nova01:20
gmannalex_xu:  IMO, that as bug fix makes it valid01:21
*** dave-mcc_ has quit IRC01:22
oomichialex_xu: Does it mean the patch changes 403 to 400 ?01:24
oomichialex_xu: we are discussing error status code changes on https://review.openstack.org/#/c/421846 . I am ok to allow error status code changes anyways01:26
openstackgerritDan Smith proposed openstack/nova: Only warn about hostmappings during ocata upgrade  https://review.openstack.org/42424301:26
*** baoli has joined #openstack-nova01:28
gmannoomichi:  policy change for all_tenant will change 403->20001:29
gmannoomichi:  so its kinnda behavior change01:29
gmannoomichi:  and on sort key change it is bug fix (not expose host and node to non admin user) and 200->40001:30
gmannalex_xu: correct me if am wrong on sort key which used to me 200 now?01:31
*** baoli has quit IRC01:32
oomichigmann: humm, 200->400 seems against on changing policy: "The following types of changes are generally **not** considered acceptable:"01:32
oomichi"A change such that a request which was successful before now results in an error response"01:33
oomichiif that is not security bug01:33
oomichigmann: "not expose host and node to non admin user" seems a little bit security thing, which means abstraction doesn't work correctly for non-admin users01:34
mriedemmelwitt: this is an easy one https://review.openstack.org/#/c/424243/01:35
oomichibut I am not sure at this time01:35
gmannoomichi:  yea that seems security bug for me too. but johnthetubaguy  alex_xu  mriedem  can give better feedback on this - last paragraph in https://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/add-whitelist-for-server-list-filter-sort-parameters.html#proposed-change01:37
oomichigmann: yeah I see "For the non-admin user, the sort key ‘host’ and ‘node’ will be excluded. Those two columns are about the cloud internal. It can’t be leaked to the end user."01:39
*** mingyu has joined #openstack-nova01:40
*** hfu has joined #openstack-nova01:40
oomichigmann: if we consider it as security issue, the bug should be fixed on stable branches. But we need different patch for fixing it because it is difficult to backport the whole patches related to the bp01:41
gmannoomichi:  hummm01:43
*** catintheroof has joined #openstack-nova01:44
*** mingyu has quit IRC01:44
gmannalex_xu: johnthetubaguy oomichi  on policy all_tenant changes, should we first change the spec ? - https://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/add-whitelist-for-server-list-filter-sort-parameters.html#rest-api-impact01:46
*** Jeffrey4l_ has quit IRC01:49
*** gongysh has joined #openstack-nova01:49
*** catintheroof has quit IRC01:49
*** Jeffrey4l has joined #openstack-nova01:49
*** erlon has quit IRC01:50
*** ejat has quit IRC01:51
*** ejat has joined #openstack-nova01:51
*** ejat has joined #openstack-nova01:51
*** raunak has quit IRC01:57
*** dave-mcc_ has joined #openstack-nova01:57
*** dave-mccowan has quit IRC01:59
*** jamesdenton has joined #openstack-nova02:01
*** thorst_ has joined #openstack-nova02:01
*** raunak has joined #openstack-nova02:02
*** kaisers_ has joined #openstack-nova02:03
melwittdansmith: I left a comment on https://review.openstack.org/#/c/319379/ it looks like the IBM CI failure might be legit, OrphanedObjectError raised from L179002:04
melwittI don't yet understand how the instance could be orphaned at that point02:05
*** thorst_ has quit IRC02:06
*** jamesdenton has quit IRC02:06
*** jamesdenton has joined #openstack-nova02:06
*** kaisers has quit IRC02:07
*** sacharya_ has joined #openstack-nova02:07
*** duonghq has joined #openstack-nova02:07
*** sacharya has quit IRC02:08
*** unicell has quit IRC02:10
*** cdelatte has quit IRC02:11
*** armax has quit IRC02:12
*** Sukhdev has quit IRC02:14
*** yamahata has quit IRC02:16
alex_xuoomichi: yea, just as gmann said, this probably is a fix for key we didn't want to expose. I feel that is different with all_tenants02:19
*** ducttape_ has joined #openstack-nova02:19
alex_xugmann: i'm not sure about the spec. but give th time i'm not sure we can get all thing done. maybe worth a email also to get widely suggestion02:20
mriedemdansmith: bauzas: jaypipes: well it looks like the scheduler + placement stuff is failing grenade multinode because of this test:02:20
mriedemhttps://github.com/openstack/tempest/blob/8bdd42766a7f02180d07e6ab75f0914e5d876a91/tempest/scenario/test_server_multinode.py#L4902:20
gmannalex_xu: i did not fully understood your comment about capability discovery on this https://review.openstack.org/#/c/415330/2402:20
mriedemi believe because we have 2 computes, but one is newton and one is ocata, and the newton one doesn't have placement configured,02:20
*** eandersson__ has joined #openstack-nova02:20
mriedemso when the scheduler tries to place an instance on the newton compute, the scheduler pukes because there isn't a resource provider for that node02:20
*** dansmith has quit IRC02:20
*** eandersson_ has quit IRC02:20
*** dansmith has joined #openstack-nova02:21
gmannalex_xu: you mean 403->200 is ok to change and discover with capabilities of all_tenant ?02:21
gmannalex_xu: correct me if other way around02:21
alex_xugmann: whether the user is allowed to execute on a API, that should be discovered by capabilities discovery https://review.openstack.org/38655502:21
owalshdansmith, mriedem, melwitt: big thanks for https://review.openstack.org/42424302:21
dansmithmriedem: with my backported change the newton one should keep trying02:22
dansmithmriedem: this: https://review.openstack.org/#/c/419217/02:22
dansmithowalsh: np02:22
alex_xugmann: no, I agree with johnthetubaguy. We change a enforce from hard to soft, that is a change on the API contract. so yes, we need microversion02:23
mriedemdansmith: but do we ever configure the newton subnode in grenade with placement creds?02:23
mriedemdansmith: nope http://logs.openstack.org/61/417961/26/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/15545e4/logs/subnode-2/etc/nova/nova.conf.txt.gz02:23
*** mtanino has quit IRC02:23
dansmithmriedem: oh right, damn02:23
dansmithmriedem: so it fails because we don't have resources for multiple computes, right?02:24
* owalsh needs some sleep, have a good night/evening/etc...02:24
dansmithmriedem: that's farther than it was getting before, right? so that02:24
gmannalex_xu:  yea, so we all are on same page. thanks02:24
dansmithmriedem: is progress02:24
alex_xugmann: yea02:24
*** ducttape_ has quit IRC02:24
gmannalex_xu:  i ll push spec + patch updates and both can be reviewed in parallel02:25
alex_xugmann: thanks!02:25
mriedemdansmith: correct, this test uses AZs to specifically place an instance on each compute node02:25
mriedemso the forced_host thing or whatever02:25
mriedemwe could change the test to not be a smoke test and it wouldn't be run with grenade, but i could see people freaking out about that02:26
dansmithmriedem: well, it's either that or backport cred setup to newton devstack right?02:27
mriedemcred setup is in newton devstack,02:27
mriedemit's just grenade isn't configuring placement creds into nova.conf on the subnode02:27
dansmithoh02:27
mriedemthe other alternative, which you're not going to probably like, is for ocata we fallback to the compute node get_all and drop that in pike02:28
mriedembecause ocata computes will have placement setup for grenade from ocata->pike02:28
dansmithmriedem: I guess it's an important thing to check that the subnode is still live after the upgrade, assuming proper config02:28
dansmithmriedem: seems strange to push this hard and then not try to fix grenade at the last minute02:29
*** jamesdenton has quit IRC02:30
*** annegentle has quit IRC02:33
mriedemyeah i know, i'm not sure how hard this is to make grenade config the newton subnode for placement right now02:33
dansmithmelwitt: can you see my response to you on my patch just now?02:34
mriedemi have an emergency toenail situation to deal with that's much easier to fix, will check grenade after that02:34
melwittdansmith: yeah just opened it. replying02:34
*** shaner has joined #openstack-nova02:38
dansmithmelwitt: might be easier to chat here02:41
melwittdansmith: okay. that's setting the _context on the BuildRequest object though, not the BuildRequest.instance object02:41
melwittI don't see anywhere in _load_instance that it sets instance._context02:42
melwittself.instance._context02:42
*** owalsh is now known as owalsh-afk02:43
dansmithmelwitt: you're right, I remember arguing with alaski about this specifically02:43
dansmithmelwitt: so what do you want me to change it to?02:43
melwittyeah, I was not sure if it's intentional not to do it02:43
dansmithI wanted him to, and he wanted it to be orphaned02:43
melwitthrm.02:44
melwittassuming he had good reason for that (we should ask him next time we catch him in the channel), I would think to go back to what you had, sans the refresh() call02:44
*** jose-phillips has quit IRC02:44
dansmithbut your comment above the refresh explains why that's important02:45
*** Jeffrey4l has quit IRC02:45
melwittwhy what's important?02:46
dansmithwe also have to handle the case where we didn't actually look it up right?02:47
dansmithwhy the refresh is important02:47
dansmithor you want to keep the instance we got from _lookup?02:47
*** raunak has quit IRC02:47
melwittoh, I was saying that's another more complicated option if doing a proper refresh() is important02:47
dansmithokay, I'm totally confused02:48
melwittsorry. I was trying to say I think what we had with the _lookups makes sense considering that the Instance could have been a half-baked object generated from a BuildRequest02:49
dansmithto get the newer instance with .host set, right?02:49
melwittand only to do something more (like check whether it's orphaned or not to determine whether to _lookup vs refresh()) but I'm not sure that matters02:49
melwittyeah02:49
dansmithokay but PS79 wouldn't bail if we fail to do that lookup, so I also have to include the bit from PS81 that bails if we fail to lookup because we raced with another delete02:50
*** sacharya_ has quit IRC02:50
*** sacharya has joined #openstack-nova02:50
*** thorst_ has joined #openstack-nova02:50
*** thorst_ has quit IRC02:50
melwittif it's important to do a refresh if real Instance, then we'd have to check for that and do either or depending. and I was going to defer to you on whether we should do that here02:51
dansmithwe have to refresh if your comment above the refresh is correct02:52
dansmithotherwise we'll fail to send the delete request to the compute02:52
melwittI  thought the _lookup would serve as the refresh to send it. and then like you said also need the bail out part if it was a race with another delete02:53
*** markmcclain has quit IRC02:53
dansmiththat's what I'm saying02:54
dansmithhowever, it seemed like I was getting an orphaned instance from that lookup in the previous patchset because I was getting orphaned errors in jenkins, which I no longer am here02:55
*** baoli has joined #openstack-nova02:56
dansmithmelwitt: you happy with this? http://pastebin.com/QgsXKH1f02:57
melwittthat's unexpected02:57
dansmithmelwitt: what is unexpected? the orphaned warnings before? I can link you to the logs :)02:57
*** markmcclain has joined #openstack-nova02:58
melwittyeah. I believe you, it's just unexpected since I just looked through _lookup_instance and didn't think I saw a way it could get an orphaned instance loaded there02:58
dansmithactually, let me go back one more rev hang on02:59
dansmithbut, is the above what you're thinking?02:59
melwittdansmith: and yes that code is what I was thinking. but I'm concerned about the orphaned errors (although in the IBM CI it shows we still have them, maybe less of a window). I'm going to look through the logs now02:59
dansmithmelwitt: okay I got it03:00
dansmithmelwitt: PS79 we were getting failure trying to do instance.uuid from the return of _lookup, which can be None03:00
dansmithsince I wasn't bailing there, I was using that instance (which had _context) if it was good, and hitting None.uuid if not03:00
*** bkopilov has quit IRC03:00
dansmiththen in the next one,03:00
melwittdansmith: oh, because you lacked the bailout code03:00
dansmithI used _instance and was getting the orphaned errors03:00
dansmithbut maybe got lucky on the race03:01
dansmithso I think the above code, using the _lookup for the refresh and bailing if we get None is probably right03:01
melwittokay, so the code in the paste should catch all of those cases. I think03:01
dansmithyeah03:01
dansmiththe dep chain for this is like five deep so I just can't keep the sequencing of the errors in my head I guess03:01
openstackgerritDan Smith proposed openstack/nova: Move instance creation to conductor  https://review.openstack.org/31937903:02
dansmithand looks like the lowest dep is going to get kicked out of the gate.. yay03:03
dansmith#fml03:03
melwittguh03:03
*** markvoelker has joined #openstack-nova03:07
*** baoli has quit IRC03:10
*** yamahata has joined #openstack-nova03:10
dansmithI'm still kinda unsure.. this must be a fantastic race that we only hit occasionally03:11
dansmithbecause I think it ran a bunch of times in a row without hitting it when I was dicking with the listing patch03:12
dansmithbut whatever03:12
melwittback when I was working on the patch to pass the cells v1 job, it failed often enough with the ObjectActionError in that block that I knew to add something. guess I was "lucky"03:15
dansmithright but that was a race,03:16
dansmiththis is a race in the race handler :)03:16
*** takashin has joined #openstack-nova03:16
melwittyes. fwiw it got really stable after adding the catch block, it's just the instance.refresh() wasn't really doing anything03:16
dansmithI mean, I'm not saying this doesn't make sense, I'm just wondering why murphy hates me03:16
melwittyeah, I get you03:17
*** thorst_ has joined #openstack-nova03:17
*** thorst_ has quit IRC03:17
dansmithI think it was, but only in the cases where we get there from a buildrequest and when we didn't race with another delete right?03:17
melwitthm, yeah. I guess it would have had an orphaned error itself when it tried to reload itself if it came from a build request03:19
dansmithat least those deps got reset so I get another go at the gate without a recheck03:19
dansmitheven though it estimates 43h until completion :/03:19
melwittit's showing 1 hour estimate on my view for what's running right now03:20
dansmithit's now 15h for those03:20
dansmith 19: (421228,2) Add --verbose option to discover_hosts command (4h26m/++~~~~~~~/rem:15h43m)03:20
* dansmith for the gate03:21
dansmith1h for this conductor patch03:21
*** tbachman has joined #openstack-nova03:21
melwittoh, I see03:21
melwittthe gate was nuts today03:22
melwitt(busy)03:22
dansmiththe queues aren't very long right now, but it's still estimating long runtimes,03:22
dansmithso something might be failing a lot03:22
*** markvoelker has quit IRC03:23
dansmithit's dropping. maybe just a big recalculation and pessimism03:25
* dansmith likes pessimistic software03:26
melwitthaha03:26
melwittyeah, I was thinking, most of what's in the gate has only a 1 hr wait left, but I assume that doesn't account for resets03:26
melwittbbiab03:29
*** xiongjh1 has joined #openstack-nova03:30
*** pbandark has joined #openstack-nova03:31
*** bjolo_ has quit IRC03:31
*** markvoelker has joined #openstack-nova03:33
*** ijw has quit IRC03:35
*** ijw has joined #openstack-nova03:35
*** ijw has quit IRC03:35
*** Swami has quit IRC03:36
openstackgerritOpenStack Proposal Bot proposed openstack/nova: Updated from global requirements  https://review.openstack.org/42482803:36
openstackgerritTakashi NATSUME proposed openstack/nova: api-ref: Fix parameters and examples in aggregate API  https://review.openstack.org/42551203:38
*** baoli has joined #openstack-nova03:39
*** nicolasbock has quit IRC03:39
*** jaypipes_ has quit IRC03:40
*** ducttape_ has joined #openstack-nova03:40
*** mingyu has joined #openstack-nova03:41
*** xinliang has quit IRC03:42
*** mingyu has quit IRC03:46
*** Sukhdev has joined #openstack-nova03:48
*** gcb has joined #openstack-nova03:50
*** guchihiro has joined #openstack-nova03:50
*** esberglu has joined #openstack-nova03:51
*** esberglu has quit IRC03:55
*** jaypipes_ has joined #openstack-nova03:57
*** ducttape_ has quit IRC04:00
*** psachin has joined #openstack-nova04:03
openstackgerritChangBo Guo(gcb) proposed openstack/nova: Enable more virt.xenapi tests on Python 3  https://review.openstack.org/41442104:05
*** tbachman has quit IRC04:06
*** hongbin has joined #openstack-nova04:10
*** armax has joined #openstack-nova04:10
*** iceyao has joined #openstack-nova04:13
dansmithmelwitt: powerkvm passed04:16
dansmithbut jenkins is going to fail04:16
melwittah, I see that. checking what it is04:16
melwittit's that identity error again from keystone. I've seen that before but I dunno if anyone's documented it in a bug yet04:17
dansmithsee the sql failure?04:18
dansmithhttp://logs.openstack.org/79/319379/82/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/72d1a95/logs/screen-n-api.txt.gz?level=TRACE#_2017-01-26_04_09_09_70604:18
dansmithso fingers crossed it's not related04:18
dansmithbut it means I have to recheck uh-gain04:18
melwittyeah, looking at those now. weird04:18
melwittI've seen sql errors on other failed runs before lately04:20
*** baoli has quit IRC04:20
dansmithwell,04:21
dansmithI'm not going to be able to stay up for the end of the next run of this04:21
dansmithso I hope that we can get this happy tomorrow04:21
*** ducttape_ has joined #openstack-nova04:22
melwittI'll keep an eye on it04:23
dansmiththe powervm fail is a secgroup thing, FYI04:24
dansmithmelwitt: okay thanks04:25
melwittoh, why did it run it two on the same PS04:25
melwitts/two/twice/04:25
melwittoh, powervm vs powerkvm04:25
dansmithyeah04:25
melwitthaha, they look too similar to me04:25
dansmithand powervm is the out of tree driver04:25
dansmithand non-voting04:26
melwittokay, yeah04:26
*** bkopilov has joined #openstack-nova04:28
*** smatzek has quit IRC04:29
openstackgerritMatt Riedemann proposed openstack/nova: Scheduler calling the Placement API  https://review.openstack.org/41796104:31
*** dave-mcc_ has quit IRC04:32
*** armax has quit IRC04:34
*** ducttape_ has quit IRC04:34
*** ianw is now known as ianw_pto04:34
openstackgerritHironori Shiina proposed openstack/nova: ironic: Add trigger crash dump support to ironic driver  https://review.openstack.org/37654804:35
takashinalex_xu: Are you aroud?04:38
takashinoomichi: Are you aroud?04:41
*** cristicalin has joined #openstack-nova04:42
cristicalinhello, I have an issue with a nova which I upgraded from juno to mitaka04:42
cristicalinafter the upgrade it started flooding the neutron server with requests04:42
cristicalinis there any way to make it less aggressive ?04:43
*** mingyu has joined #openstack-nova04:43
cristicalinneutron is still running juno version at the moment04:43
cristicalinI see a lot of security group updates being sent to the neutron server which look redundant as they trigger no actual changes04:44
cristicalinand nova logs messaages like "Creating event network-changed:None"04:45
*** yamahata has quit IRC04:46
*** mingyu has quit IRC04:47
*** d-bark has joined #openstack-nova04:49
*** mdnadeem has joined #openstack-nova04:50
mriedemmnestratov: fyi http://lists.openstack.org/pipermail/openstack-dev/2017-January/111007.html re https://review.openstack.org/#/c/312488/04:50
mriedembauzas: for your morning pleasure http://lists.openstack.org/pipermail/openstack-dev/2017-January/111006.html04:51
*** raunak has joined #openstack-nova04:51
mikalI must say I find this PTG thing very depressing04:54
*** hongbin has quit IRC04:56
*** hongbin has joined #openstack-nova04:56
openstackgerritGhanshyam Mann proposed openstack/nova-specs: Few updates on server filter/sort spec  https://review.openstack.org/42553304:57
*** Sukhdev has quit IRC04:58
*** cburgess has quit IRC05:01
*** Guest58531 has quit IRC05:04
*** McNinja has quit IRC05:05
*** d-bark has quit IRC05:06
*** McNinja has joined #openstack-nova05:07
*** comstud has quit IRC05:07
*** cburgess has joined #openstack-nova05:07
*** mgagne has joined #openstack-nova05:07
*** mgagne is now known as Guest3353905:07
*** comstud has joined #openstack-nova05:08
*** cristicalin has quit IRC05:08
*** hongbin_ has joined #openstack-nova05:09
*** Sukhdev has joined #openstack-nova05:09
*** hongbin has quit IRC05:11
*** raunak has quit IRC05:13
*** sacharya has quit IRC05:14
*** moshele has joined #openstack-nova05:14
*** sacharya has joined #openstack-nova05:14
*** sacharya has quit IRC05:15
*** sacharya has joined #openstack-nova05:15
*** raunak has joined #openstack-nova05:17
*** thorst_ has joined #openstack-nova05:18
*** sacharya has quit IRC05:20
*** hongbin_ has quit IRC05:21
*** thorst_ has quit IRC05:23
*** browne has joined #openstack-nova05:23
*** claudiub has joined #openstack-nova05:25
*** unicell has joined #openstack-nova05:28
*** cristicalin has joined #openstack-nova05:30
*** browne has quit IRC05:34
*** nkrinner_afk is now known as nkrinner05:35
*** mdnadeem has quit IRC05:36
openstackgerritGhanshyam Mann proposed openstack/nova-specs: Few updates on server filter/sort spec  https://review.openstack.org/42553305:37
*** esberglu has joined #openstack-nova05:40
*** xiongjh1 has quit IRC05:40
*** cristicalin has quit IRC05:40
*** esberglu has quit IRC05:44
*** mdnadeem has joined #openstack-nova05:48
*** Sukhdev has quit IRC05:49
*** ducttape_ has joined #openstack-nova05:50
*** ekuris_ has joined #openstack-nova05:51
*** unicell has quit IRC05:53
*** unicell has joined #openstack-nova05:54
*** ducttape_ has quit IRC05:55
*** gouthamr has quit IRC06:00
*** sree has joined #openstack-nova06:02
*** kaisers_ has quit IRC06:02
*** mdnadeem has quit IRC06:02
*** mdnadeem has joined #openstack-nova06:02
*** baoli has joined #openstack-nova06:10
*** raunak has quit IRC06:14
*** baoli has quit IRC06:15
*** sacharya has joined #openstack-nova06:16
openstackgerritTakashi NATSUME proposed openstack/nova: Fix doc generation warnings  https://review.openstack.org/42554906:16
openstackgerritTakashi NATSUME proposed openstack/nova: Fix doc generation warnings  https://review.openstack.org/42554906:17
*** vsaienko has joined #openstack-nova06:19
*** raunak has joined #openstack-nova06:19
*** sacharya has quit IRC06:22
*** adisky_ has joined #openstack-nova06:34
*** mjura has joined #openstack-nova06:34
*** mingyu has joined #openstack-nova06:44
*** zerda2 has quit IRC06:49
*** mingyu has quit IRC06:49
openstackgerritmelanie witt proposed openstack/nova: Move instance creation to conductor  https://review.openstack.org/31937906:52
*** carthaca_ has joined #openstack-nova07:02
*** kaisers has joined #openstack-nova07:03
*** aojea has quit IRC07:07
*** tesseract has joined #openstack-nova07:11
*** edmondsw has joined #openstack-nova07:12
*** andreas_s has joined #openstack-nova07:15
*** edmondsw has quit IRC07:16
*** rha has quit IRC07:17
*** frickler has quit IRC07:18
*** thorst_ has joined #openstack-nova07:19
*** aojea has joined #openstack-nova07:19
*** ducttape_ has joined #openstack-nova07:21
*** moshele has quit IRC07:23
*** thorst_ has quit IRC07:23
*** mfeoktistov has joined #openstack-nova07:24
*** vsaienko has quit IRC07:26
*** ducttape_ has quit IRC07:26
*** zerda2 has joined #openstack-nova07:27
*** lennyb has quit IRC07:27
*** lennyb has joined #openstack-nova07:28
*** esberglu has joined #openstack-nova07:29
*** vsaienko has joined #openstack-nova07:29
*** esberglu has quit IRC07:33
*** kaisers has quit IRC07:33
*** thorst_ has joined #openstack-nova07:34
*** avolkov has joined #openstack-nova07:36
*** kaisers has joined #openstack-nova07:36
*** rha has joined #openstack-nova07:36
*** rha has quit IRC07:37
*** rha has joined #openstack-nova07:37
*** frickler has joined #openstack-nova07:37
*** vsaienko has quit IRC07:38
*** thorst_ has quit IRC07:39
*** kaisers has quit IRC07:40
*** raunak has quit IRC07:44
*** kaisers has joined #openstack-nova07:45
*** vsaienko has joined #openstack-nova07:46
*** tojuvone has joined #openstack-nova07:47
*** sahid has joined #openstack-nova07:48
*** kaisers has quit IRC07:49
*** duonghq has quit IRC07:51
*** kaisers has joined #openstack-nova07:52
*** vsaienko has quit IRC07:54
*** hfu has quit IRC07:55
*** hfu has joined #openstack-nova07:55
*** markus_z has joined #openstack-nova07:55
*** hfu has quit IRC07:55
*** hfu has joined #openstack-nova07:56
*** hfu has quit IRC07:56
*** hfu has joined #openstack-nova07:57
*** hfu has quit IRC07:57
*** hfu has joined #openstack-nova07:57
*** hfu has quit IRC07:58
openstackgerritGhanshyam Mann proposed openstack/nova-specs: Few updates on server filter/sort spec  https://review.openstack.org/42553307:58
*** hfu has joined #openstack-nova07:58
*** hfu has quit IRC07:59
*** slaweq has quit IRC08:00
*** tovin07 has quit IRC08:02
*** eglynn has quit IRC08:03
*** tovin07_ has quit IRC08:05
*** slaweq has joined #openstack-nova08:06
*** hieulq has quit IRC08:09
*** markvoelker has quit IRC08:12
*** markvoelker has joined #openstack-nova08:15
*** guchihiro has quit IRC08:16
*** ralonsoh has joined #openstack-nova08:16
*** vladikr has quit IRC08:21
*** frickler has quit IRC08:23
*** abalutoiu_ has joined #openstack-nova08:23
*** rha has quit IRC08:23
*** frickler has joined #openstack-nova08:24
*** rha has joined #openstack-nova08:25
*** rha has joined #openstack-nova08:25
*** abalutoiu__ has quit IRC08:27
*** phuongnh has quit IRC08:27
openstackgerritGhanshyam Mann proposed openstack/nova: Add new policy for server list/detail with all_tenants  https://review.openstack.org/41533008:35
*** mingyu has joined #openstack-nova08:45
*** ducttape_ has joined #openstack-nova08:48
*** mingyu has quit IRC08:50
*** unicell has quit IRC08:54
*** zzzeek has quit IRC09:00
*** zzzeek has joined #openstack-nova09:00
*** gongysh has quit IRC09:10
mnestratovmnes09:19
mnestratovmriedem: thanks09:19
*** lucas-afk is now known as lucasagomes09:25
*** sambetts|afk is now known as sambetts09:26
*** amotoki has quit IRC09:27
*** amotoki has joined #openstack-nova09:28
*** amotoki has quit IRC09:32
*** CristinaPauna has quit IRC09:32
*** efoley_ has joined #openstack-nova09:34
*** thorst_ has joined #openstack-nova09:35
*** efoley__ has joined #openstack-nova09:36
*** derekh has joined #openstack-nova09:37
*** gcb has quit IRC09:37
*** amotoki has joined #openstack-nova09:39
*** jpena|off is now known as jpena09:40
*** efoley_ has quit IRC09:40
*** thorst_ has quit IRC09:40
*** BobBall_AWOL is now known as BobBall09:42
*** karimb has joined #openstack-nova09:43
*** karimb has quit IRC09:43
*** CristinaPauna has joined #openstack-nova09:45
*** rmart04 has joined #openstack-nova09:46
johnthetubaguymikal: whats depressing?09:46
*** mvk has quit IRC09:47
johnthetubaguybauzas: let me know if I can lend a hand with some of the grenade stuff, I was helping with the zero downtime job where we added nova-api on the subnode09:48
*** gcb has joined #openstack-nova09:50
*** jose-phillips has joined #openstack-nova09:50
*** moshele has joined #openstack-nova09:50
*** thorst_ has joined #openstack-nova09:51
*** karimb has joined #openstack-nova09:52
*** abalutoiu_ has quit IRC09:53
*** jpena is now known as jpena|off09:53
*** jose-phillips has quit IRC09:54
*** markus_z has quit IRC09:55
*** aarefiev_afk is now known as aarefiev09:55
*** thorst_ has quit IRC09:56
BlackDexhello09:58
BlackDexi have some trouble with nova-consoleauth09:58
BlackDexi have an HA setup09:59
BlackDexand i added memcached for the console tokens09:59
openstackgerritStephen Finucane proposed openstack/nova: Use a service account to make vendordata requests.  https://review.openstack.org/41559709:59
openstackgerritStephen Finucane proposed openstack/nova: Cache boot time roles for vendordata.  https://review.openstack.org/41778009:59
openstackgerritStephen Finucane proposed openstack/nova: Optionally make dynamic vendordata failures fatal.  https://review.openstack.org/41639109:59
BlackDexthe servers are added to the config09:59
BlackDexbut they don't appear to be used09:59
*** jpena|off is now known as jpena10:00
*** efoley_ has joined #openstack-nova10:03
gmannjohnthetubaguy:  alex_xu  can you check spec updates for policy things - https://review.openstack.org/#/c/425533/10:04
gmannupdating patch with alex_xu  comment10:04
*** hshiina has joined #openstack-nova10:05
*** efoley__ has quit IRC10:06
openstackgerritMichael Still proposed openstack/nova: Use a service account to make vendordata requests.  https://review.openstack.org/41559710:07
openstackgerritMichael Still proposed openstack/nova: Cache boot time roles for vendordata.  https://review.openstack.org/41778010:07
openstackgerritMichael Still proposed openstack/nova: Optionally make dynamic vendordata failures fatal.  https://review.openstack.org/41639110:07
gmannjohnthetubaguy: alex_xu  we need to do warning inside if condition - https://review.openstack.org/#/c/415330/25/nova/policy.py10:08
gmannjohnthetubaguy: alex_xu  i we want to warn even people did not updated file right ?10:08
gmann* i think we10:09
johnthetubaguymaybe...10:09
gmanni mean if there is old rule overridden then warn them irrespective of updates in policy file or not10:09
johnthetubaguyOh, good point, it needs removing either way10:10
gmannyea10:10
johnthetubaguyI was thinking we should probably have a more general warning about folks who override a rule to it existing default value10:10
johnthetubaguybut thats something for pike, as part of the policy clean up10:11
*** esberglu has joined #openstack-nova10:12
gmannjohnthetubaguy:  but warn about what context ? 'be careful you are overriding default one' something like10:13
sfinucanmikal: Didn't know you were back - was addressing johnthetubaguy's comments on https://review.openstack.org/#/q/topic:bp/vendordata-reboot-ocata10:13
sfinucanAs you were :)10:13
johnthetubaguysfinucan: oh you did, sweet, let me jump on that10:14
sfinucanjohnthetubaguy: Well, I haven't pushed them yet10:14
sfinucanmikal pushed a rebase just now and I don't want to walk all over any changes he has planned10:15
* johnthetubaguy nods10:15
johnthetubaguysfinucan: let me know10:16
*** owalsh-afk is now known as owalsh10:17
sfinucanBlackDex: You might be better asking that on the general #openstack channel10:17
*** esberglu has quit IRC10:17
johnthetubaguysfinucan: raj_singh has some changes up on service tokens that are very similar10:17
sfinucanBlackDex: ...or waiting until tomorrow, when the feature freeze deadline has passed :)10:17
johnthetubaguyhad a chat to keystone folks about how the caching works, hence me spotting it10:17
*** amotoki has quit IRC10:18
BlackDexsfinucan: to busy with final stuff i guese10:18
sfinucanjohnthetubaguy: Have a link?10:19
mikalsfinucan: I've only fixed the merge conflict so far10:19
mikalsfinucan: if you've got fixes for the caching and reno stuff, I encourage you to push them up10:19
johnthetubaguysfinucan: https://review.openstack.org/#/c/412634/10:19
sfinucanmikal: Yeah, I've that done. Just this one I'm stuck on https://review.openstack.org/#/c/415597/4/nova/conf/vendordata.py10:19
sfinucanI'll push shortly10:19
*** carthaca_1 has joined #openstack-nova10:20
mikalOk10:20
mikalI'm actually not at work today, on prescription painkillers, and in a terrible mood10:20
mikalSo you're much more likely to get it up in time than me10:20
mikal;)10:20
*** carthaca_ has quit IRC10:21
openstackgerritOpenStack Proposal Bot proposed openstack/nova: Updated from global requirements  https://review.openstack.org/42482810:23
bauzasjohnthetubaguy: sorry, I was afk this morning10:25
*** wim_declercq has joined #openstack-nova10:25
bauzasjohnthetubaguy: you probably saw the problem with the scheduler upcall patch ?10:26
bauzasmriedem helped us by adding a new Depends-On10:26
bauzasbut I'm a bit afraid to see the patch not being merged by Ocata given all the problems we still have when upgrading10:26
bauzas:'10:26
bauzas:(10:26
*** sapcc-bot1 has joined #openstack-nova10:30
*** Cristina_ has joined #openstack-nova10:30
*** sapcc-bot has quit IRC10:31
*** karimb has quit IRC10:31
sahidjohnthetubaguy: did you notice my comments here ? https://review.openstack.org/#/c/385023/10:32
sahiddoes it make sense for you to +W it ?10:32
*** CristinaPauna has quit IRC10:33
johnthetubaguysahid: I haven't had chance to get back to that, stuck in priority features right now10:34
*** wim_declercq has left #openstack-nova10:34
openstackgerritSam Betts proposed openstack/nova: Change order of _cleanup_deploy and _unprovision in Ironic virt  https://review.openstack.org/42267810:38
*** mingyu has joined #openstack-nova10:47
*** edmondsw has joined #openstack-nova10:48
andreykurilinjohnthetubaguy: hi! can you put final mark to the last blocker for novaclient's release https://review.openstack.org/#/c/424256/  ?)10:49
*** karimb has joined #openstack-nova10:50
*** lpetrut has joined #openstack-nova10:51
*** mingyu has quit IRC10:51
*** edmondsw has quit IRC10:52
*** lpetrut has quit IRC10:55
*** Jack_V has joined #openstack-nova10:59
Jack_VHi folks!10:59
Jack_VI'm upgrading from mitaka to newton10:59
*** dimtruck is now known as zz_dimtruck10:59
Jack_Vand got this error during creation an instance from a volume10:59
Jack_VInvalidInput: Invalid input received: Invalid image identifier or unable to access requested image. (HTTP 400)10:59
*** kaisers has quit IRC11:04
*** kaisers has joined #openstack-nova11:05
*** tbachman has joined #openstack-nova11:05
*** kaisers_ has joined #openstack-nova11:08
openstackgerritmelanie witt proposed openstack/nova: Count resources to check quota for cells  https://review.openstack.org/41652111:08
*** kaisers has quit IRC11:09
mdboothmfeoktistov: Did you read the ML traffic on the privsep thing?11:10
mdboothSounds like the concensus is that you should just go ahead and use rootwrap. Oh, and do it quickly ;)11:10
*** kaisers_ has quit IRC11:12
*** karimb has quit IRC11:12
*** mfeoktistov has quit IRC11:14
*** mfeoktistov has joined #openstack-nova11:15
mfeoktistovmdbooth: Yes, thank you. I'll send patch in a minute11:16
*** openstackgerrit has quit IRC11:17
*** vsaienko has joined #openstack-nova11:19
*** adisky_ has quit IRC11:19
*** sree has quit IRC11:20
*** openstackgerrit has joined #openstack-nova11:25
openstackgerritMikhail Feoktistov proposed openstack/nova: libvirt: ephemeral disk support for virtuozzo containers  https://review.openstack.org/31248811:25
*** karimb has joined #openstack-nova11:27
*** karimb has quit IRC11:28
*** tbachman has quit IRC11:29
*** karimb has joined #openstack-nova11:30
johnthetubaguyandreykurilin: don't we support --os-auth-token and --os-token?11:34
andreykurilinjohnthetubaguy: I found note about --os-auth-token in the code, but there is no place if registration for that cli argument. PS: we use keystoneclient1 method for registrating identity cli arguments and it registrate just --os-token arg11:36
andreykurilin*no place of11:36
johnthetubaguyI guess we broke that at some point, oops11:36
johnthetubaguyor we deprecated it, and only partially removed it11:37
*** tbachman has joined #openstack-nova11:40
johnthetubaguyandreykurilin: are we missing OS_TOKEN environment variable reading, or does keystoneauth already do that for us?11:41
*** gcb has quit IRC11:43
*** nicolasbock has joined #openstack-nova11:43
andreykurilinjohnthetubaguy: ok, I found commits for you:)11:45
andreykurilinjohnthetubaguy: os-token argument was introduced with deprecation of os-auth-token: https://github.com/openstack/python-novaclient/commit/1f11840dd84f3570330d1fcd53d1e8eea5ff7922 . Actually that commit broke env variable and the whole token auth method via cli. https://github.com/openstack/python-novaclient/commit/08c4b746b2b894a5cb178c762850a1730e1c636e made a proper deprecation for os-auth-token with warning message. And it was finnaly removed h11:46
andreykurilinjohnthetubaguy: So it looks like we can avoid usage of --os-auth-token, since we have a release note that it was deprecated and removed11:47
johnthetubaguyyeah, that looks good, I vaguely remember deprecating those alias things, thank you!11:47
andreykurilinbut it sounds like you are right about env variable. keystoneauth doesn't do it for us11:48
andreykurilinjohnthetubaguy: should I update my patch (and lose bauzas vote) or send another one?:)11:48
*** ociuhandu has quit IRC11:48
johnthetubaguyandreykurilin: feels like a follow on patch11:49
andreykurilinok. give me a minute11:49
johnthetubaguyandreykurilin: bit worried about kevin's comment as well, feels like we should always access os_user_domain_id in a consistent way11:49
johnthetubaguybut happy for that to be a follow on11:49
*** efoley__ has joined #openstack-nova11:50
openstackgerritStephen Finucane proposed openstack/nova: Use a service account to make vendordata requests.  https://review.openstack.org/41559711:50
openstackgerritStephen Finucane proposed openstack/nova: Cache boot time roles for vendordata.  https://review.openstack.org/41778011:50
openstackgerritStephen Finucane proposed openstack/nova: Optionally make dynamic vendordata failures fatal.  https://review.openstack.org/41639111:50
johnthetubaguysfinucan: I see you11:51
sfinucanjohnthetubaguy: Cached the tokens and added renos for all patches11:51
sfinucanjinx :)11:51
andreykurilinjohnthetubaguy: if you look at previous patch-sets(for example this one - https://review.openstack.org/#/c/424256/8/novaclient/shell.py ), I tried several ways to make that lines a bit clear, but for some reasons I did not succed in making keystone v3 work11:51
johnthetubaguyandreykurilin: lets do a separate patch that adds a test for that maybe?11:51
andreykurilinok, I'll try to do it again in folowwing patch11:52
*** thorst_ has joined #openstack-nova11:52
*** efoley_ has quit IRC11:53
*** tbachman has quit IRC11:54
*** thorst_ has quit IRC11:57
*** lpetrut has joined #openstack-nova11:59
openstackgerritAndrey Kurilin proposed openstack/python-novaclient: [cli] Use environment variable for token  https://review.openstack.org/42564712:00
*** jpena is now known as jpena|lunch12:01
johnthetubaguyandreykurilin: I added some questions, got confused about the SecretHelper stuff12:01
*** liverpooler has quit IRC12:03
*** bkopilov has quit IRC12:05
andreykurilinjohnthetubaguy: answered:)12:13
*** karimb has quit IRC12:16
*** mvk has joined #openstack-nova12:16
*** sofie has quit IRC12:18
*** karimb has joined #openstack-nova12:19
*** smatzek has joined #openstack-nova12:20
openstackgerritAndrey Kurilin proposed openstack/python-novaclient: [cli] Use environment variable for token  https://review.openstack.org/42564712:23
*** vsaienko has quit IRC12:25
*** erlon has joined #openstack-nova12:27
*** catintheroof has joined #openstack-nova12:27
*** ociuhandu has joined #openstack-nova12:28
*** DineshBhor has joined #openstack-nova12:32
*** kaisers has joined #openstack-nova12:32
*** kaisers has quit IRC12:37
*** bvanhav has joined #openstack-nova12:37
*** mjura has quit IRC12:38
*** catintheroof has quit IRC12:38
*** catintheroof has joined #openstack-nova12:38
*** karimb has quit IRC12:39
*** mjura has joined #openstack-nova12:40
*** esberglu has joined #openstack-nova12:41
*** kaisers has joined #openstack-nova12:42
claudiubhi folks. any idea why nova list currently shows / returns an empty list (including with --all), even if I have spawned instances?12:43
*** DineshBhor has quit IRC12:43
*** vsaienko has joined #openstack-nova12:45
*** esberglu has quit IRC12:46
*** vsaienko has quit IRC12:47
*** thorst_ has joined #openstack-nova12:47
*** mingyu has joined #openstack-nova12:48
*** lucasagomes is now known as lucas-hungry12:52
*** mingyu has quit IRC12:53
openstackgerritOpenStack Proposal Bot proposed openstack/nova: Updated from global requirements  https://review.openstack.org/42482812:55
sfinucanalex_xu: Regarding https://review.openstack.org/#/c/394480 - does that require any changes on the RPC version?12:56
*** pradk has joined #openstack-nova12:56
sfinucanThat's object gets sent over the wire, as far as I understand, and changes to what we send there necessitate RPC bumps12:57
sfinucan...or so I thought12:57
*** catinthe_ has joined #openstack-nova12:57
*** catintheroof has quit IRC12:59
*** jpena|lunch is now known as jpena13:00
*** d-bark has joined #openstack-nova13:01
*** cdent has joined #openstack-nova13:06
mdboothExtreme code smell is when reviewing a 6 line change requires a code audit of 2 other modules complete with notes: https://review.openstack.org/#/c/389593/2/nova/virt/libvirt/driver.py13:09
*** tbachman has joined #openstack-nova13:09
*** kfarr has joined #openstack-nova13:10
*** edmondsw has joined #openstack-nova13:14
sfinucansnikitin: https://review.openstack.org/#/c/39961313:14
*** markvoelker has quit IRC13:14
*** vsaienko has joined #openstack-nova13:14
*** lucas-hungry is now known as lucasagomes13:16
cdentmdbooth: On some days I feel like that describes...everything13:17
mdboothcdent: Feeling all existential. I'm literally sitting here contemplating 'what does destroy_disks really *mean*'13:17
cdentbauzas, mriedem, dansmith: is there somewhere for me to leap in?13:18
* mdbooth has concluded that it is not an intent to destroy disks13:18
cdentmdbooth: I think that all depends on your position on social construction of meaning13:18
mdboothlol13:18
bauzascdent: honestly, I still need to correctly understand the problem13:19
cdentbauzas: Is there only one at this point?13:19
bauzascdent: yup, I fixed the functional issue13:20
bauzascdent: now, we still have the grenade problem13:20
bauzasthat's the blocking issue13:20
cdentdid mriedem option #4 not work (I haven't had a chance to check yet)?13:20
*** karimb has joined #openstack-nova13:20
bauzaswhich option, sorry?13:20
*** owalsh is now known as owalsh-brb13:21
bauzasoh, you mean the email ?13:21
cdentyeah13:22
bauzaswell, #4 is not working yet13:23
bauzasbut I agree with mriedem about trying to have #4 and then if not working, doing #313:24
*** amotoki has joined #openstack-nova13:24
bauzasbut we need to have a consensus before13:24
cdentI think 4 is the best choice13:25
cdent(assuming we can make it work)13:25
*** owalsh-brb has quit IRC13:26
*** esberglu has joined #openstack-nova13:26
bauzaswell, tbh, I hate the forced_host field13:26
bauzascdent: ^13:26
bauzasbecause it's silly13:26
*** baoli has joined #openstack-nova13:27
bauzasrather I prefer to use the requested_destination like we did for live-migrating and evacuate, ie. calling all the filters13:27
bauzasbut...13:27
johnthetubaguybauzas: so that option #3, I have a slightly different version that could help?13:27
*** baoli has quit IRC13:27
cdentYeah, I understand there's quite a lot of history on force_host13:27
cdent(social history)13:27
bauzasif we don't find a solution for #4 today, then the scheduler calling the placement couldn't be merged by Ocata13:27
johnthetubaguybauzas: if we check for the min_compute service version, and fallback to [] if everything has been upgraded, and the full list of old computenodes if everything hasn't yet been upgraded?13:28
bauzasjohnthetubaguy: the fact is that we verify the forced field *after* getting the list of nodes13:28
bauzaswhich is weirdo13:28
bauzasbecause if you want to force to a destination, you don't care about others13:28
johnthetubaguyoh, so the problem is different to what I am thinking it is, I thought it was just placement not being present13:29
bauzasjohnthetubaguy: nope, we fixed that by my grenade job13:29
johnthetubaguyOK, so this is a different error now?13:29
johnthetubaguyI missed that13:29
openstackgerritAndrey Kurilin proposed openstack/python-novaclient: [cli] Use environment variable for token  https://review.openstack.org/42564713:29
bauzasjohnthetubaguy: yup, see mriedem's email from this early morning13:29
*** owalsh-brb has joined #openstack-nova13:29
*** owalsh-brb is now known as owalsh13:29
bauzasjohnthetubaguy: we have a tempest test that's blocking us for a very specific behaviour13:29
bauzasI'll actually respond to mriedem13:30
johnthetubaguyyeah, I was reading that and it seemed to say different13:30
bauzasbecause I think the current forced_hosts thing is already weirdo13:30
cdentcould/should we reorder when force_host happens?13:30
johnthetubaguyso its a test thats confirm we can build on "both" nodes I guess?13:31
bauzascdent: or just accepting [] for forced_hosts that's it :)13:31
johnthetubaguyis thats whats failing here?13:31
bauzasjohnthetubaguy: it's failing because it forces to a destination using the --az hack that I *HATE*13:31
*** baoli has joined #openstack-nova13:31
*** baoli has quit IRC13:31
bauzasjohnthetubaguy: given the subnode is not yet providing its resources to placement, the placement call returns [] which fails later when looking at forced_hosts13:32
sfinucanjohnthetubaguy, raj_singh: could you enlighten me as to the purpose of these three lines https://github.com/openstack/nova/blob/master/nova/conf/neutron.py#L116-L11813:32
johnthetubaguyright, so thats a bigger issue though, we would want to the subnode to be providing its resources already, I assume?13:32
bauzasjohnthetubaguy: that's what we ask13:32
sfinucanI'm guessing they refer to the various version of the neutron API and we only need one, but that's a guess13:32
sfinucanhttps://github.com/openstack/watcher/blob/695ddf8ae7a2dccff4c62a28821888b7d49e2338/watcher/conf/_opts.py#L5813:33
bauzasjohnthetubaguy: in theory, you have to configure your computes to talk to the placement service even if they're Newton13:33
bauzasor you'd not get them when calling the placement13:33
*** claudiub|2 has joined #openstack-nova13:33
johnthetubaguysfinucan: its what is done for neutron, it describes all the setting you can have for the password keystoneauth settings (i.e register_auth_conf_options), AFAIK13:33
*** moshele has quit IRC13:34
*** Jeffrey4l has joined #openstack-nova13:35
cdentbauzas: I thought the point of option 4 was that was causing the subnode to have placement in newton13:36
bauzascdent: yup13:36
*** claudiub has quit IRC13:36
cdentso will that fix the force_host test?13:37
*** tbachman has quit IRC13:38
johnthetubaguycdent: thats what I was assuming too, feel like I am missing a bit13:38
*** baoli has joined #openstack-nova13:38
cdentjohnthetubaguy: yeah, exactly, I've not a clear hi-res picture of the situation13:38
johnthetubaguyunless the newton compute nodes are not providing placement with enough information, because of the new APIs we added this cycle?13:38
*** baoli has quit IRC13:39
johnthetubaguycertainly feel like the is a big piece I am missing here13:39
johnthetubaguythere13:39
*** baoli has joined #openstack-nova13:39
openstackgerritStephen Finucane proposed openstack/nova: Use a service account to make vendordata requests.  https://review.openstack.org/41559713:40
openstackgerritStephen Finucane proposed openstack/nova: Cache boot time roles for vendordata.  https://review.openstack.org/41778013:40
openstackgerritStephen Finucane proposed openstack/nova: Optionally make dynamic vendordata failures fatal.  https://review.openstack.org/41639113:40
*** tbachman has joined #openstack-nova13:40
sfinucanjohnthetubaguy: Appears those v2password, v3password lines refer to (keystone?) plugins https://github.com/openstack/keystoneauth/blob/master/keystoneauth1/loading/conf.py#L4613:41
johnthetubaguysfinucan: yeah, the auth config is pluggable13:42
sfinucanBlindly copied and pasted and ran genconfig without issue13:42
sfinucan...so I think we're good13:42
*** lpetrut has quit IRC13:42
johnthetubaguysfinucan: its consistent now, which I guess is good13:42
sfinucanSo long as it's consistently correct :)13:42
*** vsaienko has quit IRC13:43
johnthetubaguyI am OK with consistently wrong, over in-consistently wrong, but maybe I am in a funny mood13:43
*** gabor_antal has quit IRC13:45
cdentbauzas, johnthetubaguy so looking at that grenade run, it appears that placement was not configured on the old side. Was the change in option 4 supposed to cause that?13:45
*** bkopilov has joined #openstack-nova13:45
bauzascdent: I'm writing an email13:46
bauzascdent: you'll see my thoughts13:46
cdentk13:47
bauzasafter thinking more of that, I think we hit the bigger problem that johnthetubaguy said13:47
johnthetubaguyI think its time for option 313:47
bauzaswhich is that we need to ask operators to modify their compute configuration before upgrading13:47
johnthetubaguybut if we limit it only for during an upgrade, that feels better than raw option 3, like option 3.11 or something13:47
bauzasjohnthetubaguy: could you please reply to my email I'm sending in 2 mins ?13:48
* johnthetubaguy hangs head in shame at DOS joke13:48
johnthetubaguybauzas: sure13:48
cdentI wish I understood why it is hard to set the config on the subnode13:49
bauzasjohnthetubaguy: cdent: so I just sent my email13:50
bauzastl;dr: I think we have 2 problems, one being larger than other13:50
*** jheroux has joined #openstack-nova13:51
bauzasthe fact that if you force a destination it fails is to me a corner case that can easily be tackled if we make the scheduler more defensive13:51
bauzasbut13:51
*** efried has joined #openstack-nova13:52
bauzasit raises the main issue that it requires nova.conf to be updated for all your computes before you upgrade to Ocata, and that's a larger action item than the one we stated in the upgrade process http://docs.openstack.org/developer/nova/upgrade.html#rolling-upgrade-process13:52
bauzaswhere we only ask for modifying the upgrade_levels flag13:52
bauzasand even that13:52
*** ducttape_ has quit IRC13:53
bauzascdent: johnthetubaguywe all supposed that a rolling upgrade means that you can only focus on upgrading your controller without touching the computes (unless just upgrading them to the latest stable release)13:53
bauzasjohnthetubaguy: $13:53
bauzassnap, I'm wearing gloves13:53
bauzasthat's cold here, but not a reason13:53
cdentWe've known for a few months that there will be config change on the compute nodes, it's part of why we made the upgrade ready script, isn't it?13:54
* cdent reads the email13:55
*** eharney has quit IRC13:55
bauzascdent: tbh, I never identified that it would require a nova.conf change on old computes13:56
edleafebauzas: what change is required?13:57
cdentwe've said for a long time that newton computes will need to run placement before the ocata upgrade13:57
cdentin order ot set inventory13:57
cdentand in order to set inventory you have to have the [placement] section in nova.conf13:57
cdentedleafe: it contains the auth bits to do placement service discovery13:57
bauzasedleafe: what cdent just said13:57
bauzascdent: are we sure we are going to ask operators to modify all their computes nova.conf before upgrading ?13:58
edleafecdent: but isn't that what "have to run the placement service" would entail?13:58
cdentedleafe: yes, exactly and thus bauzas: yes!13:59
openstackgerritAndrey Kurilin proposed openstack/python-novaclient: [cli] Use environment variable for token  https://review.openstack.org/42564713:59
cdentbauzas: we act like this is a big deal because we pretend like someone, the day after feature freeze is going to upgrade to ocata, but that's rarely the case14:00
cdentpeople will be upgrading to newton while we are nearly finished fixing bugs in ocata14:00
edleafecdent: ok, that's what I thought. I felt I was missing bauzas concern14:00
*** bvanhav has quit IRC14:00
edleafeSo... nova meeting?14:01
bauzasI dunno14:01
bauzasin theory yes14:01
bauzaslemme see if there is an agenda14:01
johnthetubaguyI vote we skip the meeting, seems like we are having it here14:02
cdentjohnthetubaguy: makes good sense given the day it is14:02
*** kfarr has quit IRC14:02
edleafeno argument from me14:02
bauzasso the agenda is not up to date14:03
bauzasI could just open the meeting, do a vote for skipping it and leave us coming back here ?14:03
edleafebauzas: go for it14:04
sfinucanaye, I vote skip14:04
*** ekuris__ has joined #openstack-nova14:04
*** bvanhav has joined #openstack-nova14:05
*** esberglu has quit IRC14:05
*** esberglu has joined #openstack-nova14:06
*** ekuris_ has quit IRC14:06
*** esberglu has quit IRC14:07
*** mdrabe has joined #openstack-nova14:07
*** esberglu has joined #openstack-nova14:07
openstackgerritWang Qing wu proposed openstack/nova: WIP(6): Introduce base PowerVM driver (ssp drv)  https://review.openstack.org/42259414:10
*** vsaienko has joined #openstack-nova14:10
johnthetubaguybauzas: I replied to your email14:11
bauzasjohnthetubaguy: ack14:11
*** cleong has joined #openstack-nova14:12
*** ducttape_ has joined #openstack-nova14:13
*** esberglu has quit IRC14:14
*** haplo37_ has quit IRC14:15
*** esberglu has joined #openstack-nova14:16
*** hfu has joined #openstack-nova14:17
*** haplo37_ has joined #openstack-nova14:18
*** vsaienko has quit IRC14:20
claudiub|2sfinucan: ehm, it seems moshele isn't online atm, so can you take a look at the hyper-v pci passthrough patch and see if my answers seem reasonable to you? Thanks! https://review.openstack.org/#/c/420614/714:21
*** tblakes has joined #openstack-nova14:22
*** vsaienko has joined #openstack-nova14:26
*** eharney has joined #openstack-nova14:27
*** gouthamr has joined #openstack-nova14:29
*** vsaienko has quit IRC14:29
dansmithjohnthetubaguy: only /you/ can send this to its happy place: https://review.openstack.org/#/c/319379/14:29
johnthetubaguydansmith: looking14:30
*** vsaienko has joined #openstack-nova14:30
*** liverpooler has joined #openstack-nova14:30
sfinucanclaudiub|2: Sure thing14:30
*** _ducttape_ has joined #openstack-nova14:31
*** ducttape_ has quit IRC14:32
bauzasthe gate is 3 hours late, not a big deal14:32
bauzasfor a FF day :p14:32
*** vladikr has joined #openstack-nova14:33
*** smatzek has quit IRC14:33
bauzasdansmith: I'm maybe particularly exhausted and short-sighted but have we agreed to ask operators to modify their nova.conf to amend placement section in newton before upgrading ?14:33
dansmithbauzas: didn't we agree to that like weeks ago?14:34
*** hshiina has quit IRC14:34
bauzasI think I'm nervous14:34
dansmithif not, I'm not sure why we're doing all of this :)14:34
bauzasand I think my brain is litterally getting me mad14:34
dansmithI'm nervous that we might have to wait until pike to deliver any measurable scheduler improvement14:35
dansmithby the way,14:35
dansmithcdent: I know jaypipes did some scheduler benchmarking early on to test the sql-based approach, but do you know if anyone has measured the performance of the potentially large http queries the scheduler will be making? Like, under load with a large cloud...14:36
dansmithI was thinking about that last night14:36
cdentdansmith: have you seen the mail I sent last night on a similar topic14:37
claudiub|2dansmith: does this help? 2017-01-26 03:04:14.169 2780 WARNING nova.scheduler.client.report [req-b2db1cb5-b647-4e48-b939-b0dfd7509e17 - - - - -] No authentication information found for placement API. Placement is optional in Newton, but required in Ocata. Please enable the placement service before upgrading.14:37
dansmithclaudiub|2: help what?14:37
bauzasdansmith: nothing really yet AFAIU14:37
bauzaswe haven't tested yet the performance, only what yingxin and jay did14:37
claudiub|2a, sorry, bauzas asked about the placement section in nova.conf. my bad. :)14:38
cdentdansmith: my calculations are ~250 requests per second on a 1000 compute node cluster with 10 instances per node14:38
cdentthat vast majority of those GETs14:38
dansmithclaudiub|2: ah, yeah, that.. we added that warning to help exactly that, yeah :)14:38
johnthetubaguydansmith: are you very against doing scheduling the old way, but only until all compute nodes are upgraded? (a modified option 3)14:38
bauzasokay, we need to move on14:38
cdentand many of them redudant, with a few minor fixes to the resource_tracker (all in the email dansmith)14:38
dansmithcdent: ah, okay, that's a pretty high rate right?14:38
dansmithcdent: yeah, looking now14:38
cdentdansmith: yes14:38
bauzasI'll at least work on trying to have the force_hosts thing to accept an empty list14:39
johnthetubaguyis that without any builds happening?14:39
dansmithjohnthetubaguy: I have been yeah14:39
cdentjohnthetubaguy: yes, that's after the builds are done and change has frozen14:39
johnthetubaguydansmith: don't get me wrong, I would love to do a clean switch, but I would rather that than not land it14:39
cdentbut it can be cut _way_ back with a few changes. halfed with a bug fix that's already in progress14:39
johnthetubaguycdent: that sounds like a worthy aim14:40
cdentin order to make one fix, I/someone needs to understand why tracked_instances in the resource tracker is being cleared on every periodic job14:40
dansmithcdent: okay, but also, we can scale that horizontally14:40
cdentif that clear() doesn't happen, the number of requests can drop, alot14:41
cdentdansmith: yeah, which is nicely easy14:41
dansmithjohnthetubaguy: the problem with that is that I think people will rely on that crutch14:41
cdentyeah, that crutch is something we've been wanting to avoid14:41
*** esberglu has quit IRC14:41
dansmithjohnthetubaguy: but, if we check service_version and only do the fallback if there are old nodes, maybe that's a thing14:41
johnthetubaguydansmith: thats the one I was meaning14:41
dansmithjohnthetubaguy: that way once they've updated everything then they're stuck14:41
johnthetubaguydansmith: only during upgrade, allow the fallback14:41
johnthetubaguywe should totally not fallback once we are upgraded14:42
dansmithcdent: I know, I really want to avoid the crutch14:42
bauzaswe all want to avoid the crutch14:42
bauzas(whatever the word means for a non-English speaking person)14:42
dansmithbut, if we give them a crutch that is half cut through so that it breaks once they walk out the door14:42
* johnthetubaguy is now thinking about tiny tim from muppet christmas carol14:42
johnthetubaguydansmith: thats the one14:43
johnthetubaguydansmith: like trolley wheels that lock on if you steal the trolley14:43
bauzasjohnthetubaguy: dansmith: I can propose something about a conditional branch verifying the service version14:43
bauzasand if you upgrade all your computes, then good luck14:43
johnthetubaguybauzas: thats what I suggested on the ML anyways (at least thats what I tried to suggest)14:43
*** Jeffrey4l has quit IRC14:44
dansmithjohnthetubaguy: by trolley you mean "shopping cart" right?14:44
bauzasjohnthetubaguy: well, I mentioned that also in my email as a conclusion, but I said we turned down the idea14:44
*** Jeffrey4l has joined #openstack-nova14:44
bauzasjohnthetubaguy: if we revise that to something acceptable, then I can surely implement it14:44
bauzasbut I'd love to see some quorum here14:44
dansmithcdent: what do you think about that?14:44
* cdent catches up14:45
dansmithcdent: I'll summarize14:45
dansmithcdent: we can detect if all the computes have been upgraded or not,14:45
dansmithcdent: and so we could fall back on the crutch until that point, and then go scorched-earth after that and refuse to look at anything other than placement14:45
*** mlavalle has joined #openstack-nova14:46
dansmithcdent: which would solve our current problem and not let anyone (reasonably) keep from using placement in ocata14:46
dansmithmost people that don't do full-on live upgrades would have to deploy placement immediately (i.e. the tripleo types)14:46
cdentso each resource tracker would say "are all my peers up to date with me?" and if not, they'd use the old method of handling inventory?14:46
dansmithno,14:46
dansmithcomputes would all do placement and RT all the time like they do now,14:46
dansmithbut scheduler would refuse to look at the RT data once all the computes are upgraded14:47
dansmithso it would just be a change in bauzas' patch14:47
cdentbut the problem we're having right now is computes that _may_ be upgraded, but haven't changed their configuration?14:47
bauzaslemme fire a patch ip14:47
bauzasso cdent can see14:47
bauzasI mean a diff pastebin14:47
cdentI understand the change, I'm just not clear if it covers all the issues14:47
*** _ducttape_ has quit IRC14:47
dansmithI'm pretty sure it does14:48
*** pbandark has quit IRC14:48
johnthetubaguyso we might need one step worse...14:48
johnthetubaguythe placement data will be incomplete14:48
johnthetubaguyso we could use the crutch all the time, until the upgrade has completed14:48
dansmithjohnthetubaguy: right you'd have to favor the RT data until the time when you switch14:48
openstackgerritSergey Nikitin proposed openstack/nova: Change parent of diagnostics objects to NovaObject class  https://review.openstack.org/39448014:48
openstackgerritSergey Nikitin proposed openstack/nova: Standardization of VM diagnostics info API.  https://review.openstack.org/35554014:48
openstackgerritSergey Nikitin proposed openstack/nova: Completed implementation of instance diagnostics for Xen  https://review.openstack.org/39961314:48
dansmithjohnthetubaguy: the downside is that it makes the switch to different data non-obvious, so people may not know why things are suddenly being different,14:49
dansmithjust because a lab monkey upgraded the last one14:49
johnthetubaguyyeah14:49
* cdent aligns things in his brain14:49
bauzaseasy peasy question, can someone tell me the minimum service version for Ocata computes ?14:49
bauzaswe don't comment that in the SERVICE_VERSION_HISTORY dict14:49
dansmithjohnthetubaguy: we could log all-caps WARNING messages whilst using the crutch :)14:50
johnthetubaguybauzas: technically, you might want to bump that, so you know everyone has your patch14:50
dansmithbauzas: yes, it's easy14:50
bauzasjohnthetubaguy: we don't need that14:50
cdentokay, I think I'm caught up with you dansmith: the scheduler is the one that checks if all the compute nodes are up to date, and then makes a choice on what type of scheduling it does, yeah?14:50
bauzasjohnthetubaguy: all computes are already able to talk to placement since newton14:50
dansmithbauzas: https://github.com/openstack/nova/blob/stable/newton/nova/objects/service.py#L9214:50
bauzasjohnthetubaguy: I just want to detect a fresh code14:50
*** d-bark has quit IRC14:50
cdentbauzas: but only if they have the config...14:50
dansmithcdent: yes14:50
*** mtanino has joined #openstack-nova14:51
bauzascdent: but we say they need to upgrade to ocata, right?14:51
johnthetubaguyso does a compute node die if it doesn't have placement configured?14:51
bauzasdansmith: okay, so I need to set a new version14:51
*** vsaienko has quit IRC14:51
bauzasjohnthetubaguy: nope14:51
dansmithwill ocata computes fail to start if the placement config is not done?14:51
cdentbauzas: yes, but earlier you were complaining that making people change their configs to upgrade to ocata was in violation?14:51
*** hfu has quit IRC14:51
dansmithbauzas: I dunno what a new version would signal, TBH14:51
*** mtanino has quit IRC14:52
snikitinsfinucan: Hi! thank you for comments! I updated https://review.openstack.org/#/c/39961314:52
dansmithbauzas: if they don't die, then maybe we should make them die and make that a new service version, and then check for that version in the scheduler14:52
bauzasdansmith: good question about nova computes refusing to start14:52
bauzasdansmith: I agree with that plan14:52
johnthetubaguycdent: its not as bad, because the old compute could have had that same config, but its sure borderline14:52
dansmithI think it's what johnthetubaguy was saying too14:52
bauzasdansmith: that's the move defensive one14:52
johnthetubaguyyeah, make it obvious things are bad14:53
snikitinalex_xu: Hi Alex! I got two +2 on this patch https://review.openstack.org/#/c/394480/ can I get a +W, please?14:53
dansmithjohnthetubaguy: do you think it's too borderline?14:53
bauzasbut we said placement is mandatory for ocata, right?14:53
dansmithjohnthetubaguy: I feel like we've already agreed we're going to make this decision14:54
bauzasso that means you have to access it when restarting your services14:54
dansmithright14:54
dansmithwhat bauzas said14:54
dansmithand we're logging on newton14:54
johnthetubaguydansmith: I think I am OK because we get a massive gain, so we stick with it14:54
bauzashere, we're just trying to allow newton computes that are untouched to still be able to get instances on them14:54
dansmithjohnthetubaguy: but really, it was optional in newton, required in ocata, and this is just enforcing that no?14:54
bauzasdansmith: to summarize, if I'm bumping the service version now, that would mean I'm sure the service is upgraded to ocata14:55
johnthetubaguydansmith: I think the key bit that makes me OK, is they just need to update the config on the old system before they do the upgrade, if they haven't done that14:55
mriedemo/14:55
dansmithbauzas: you don't need to bump it now, but if you're going to make the computes refuse to start, I'd call that a service version.. otherwise you're adding a number that doesn't have any real meaning14:56
bauzasnow, the only question would be : should we silently soft-fail and do nothing with the risk for operators that they could loose capacity if they misconfigure ?14:56
johnthetubaguybig fire, always the big fire14:56
dansmithjohnthetubaguy: right, and they could have done that in newton months ago if they wanted to be proactive14:56
bauzasdansmith: the thing is that we haven't bumped the version in Ocata14:56
johnthetubaguydansmith: yeah, I think thats what makes me OK with it14:56
dansmithbauzas: not at all?14:56
bauzasdansmith: I'm still seeing version 15 in my local master branch14:56
dansmithmriedem: read back to my "let me summarize" comment to cdent14:56
mriedemwhat are we doing with a min service version check?14:57
dansmithbauzas: then yes you need to add one14:57
johnthetubaguyyeah, just add one14:57
dansmithmriedem: thinking we will add a fallback to RT data _if_ the computes are not all upgraded14:57
bauzason it14:57
dansmithmriedem: in the scheduler14:57
bauzasand we hard-fail, as johnthetubaguy said14:57
dansmithmriedem: which will let us get past grenade, and give the deployers a little longer grace period14:57
*** BobBall is now known as BobBall_AWOL14:57
*** markvoelker has joined #openstack-nova14:57
dansmithmriedem: once they upgrade all their computes, scheduler will stop looking at RT data14:58
*** owalsh is now known as owalsh-afk14:58
dansmithmriedem: which means if they only change config during the upgrade they will still be fine14:58
dansmithmriedem: whereas before, we had to make them configure their newton computes before starting the controller upgrade14:58
mriedemwe only fallback if we don't get any RPs *and* minimum compute isn't new enough right?14:59
sfinucanclaudiub|2: I looked at that and it seems fine, but I think one of Moshe's questions still stands14:59
dansmithmriedem: no we have to always fallback if computes are old14:59
mriedemso don't even check placement until then?14:59
johnthetubaguyotherwise you get incomplete data, and the test we want to pass will still fail14:59
sfinucan...and I don't think I can approve until one of ye proves it's not a problem :)14:59
dansmithmriedem: because otherwise after one compute reports RPs you'd hard switch and have a 1-compute cloud15:00
mriedemjohnthetubaguy: the test would pass if we would fallback to computenodelist.get_all if there were 0 RPs, but yes i understand we'd have a 1 compute cloud for awhile15:00
mriedemfwiw we talked about that weeks ago and people thought that was ok and a small window15:00
johnthetubaguymriedem: I was thinking the force host thing, but yeah15:01
dansmithmriedem: I think that was about waiting for the 60-second timer,15:01
dansmithmriedem: but it could be weeks if you upgrade one compute as a test15:01
dansmithmriedem: in this scenario15:01
mriedemdansmith: was refering to this http://lists.openstack.org/pipermail/openstack-dev/2016-December/109060.html15:03
sfinucansnikitin: Done and done15:04
mriedembut whatever, that was before we had a blocking issue with grenade15:04
dansmithmriedem: right but that was a thing that is triggered by doing the controller upgrade, right? this would be a thing triggered by the first compute upgrade15:04
sfinucanclaudiub|2: Call me a fool, but wasn't this kind of request what we were implementing qualitative resources for? https://review.openstack.org/#/c/42096015:05
snikitinsfinucan, thanks!15:06
dansmithmriedem: meaning, I think before we were assuming 1. Configure computes for placement, 2. upgrade controllers, 3. wait 60s for data to filter before scheduling works properly15:06
sfinucanjohnthetubaguy: would it be too cheeky of me to +2 this, given that I addressed comments on it? https://review.openstack.org/#/c/41559715:06
claudiub|2sfinucan: sure, but that is being used for the ImageMetaProps object. that still has to exist15:06
dansmithmriedem: now, it would be 1. upgrade controllers, 2. upgrade one compute, 3. stuck until you upgrade more computes15:06
sfinucanclaudiub|2: Ah, that's how the instance would request those properties while resource providers is for hosts publishing that information15:07
claudiub|2yep15:08
sfinucangotcha15:08
*** jaypipes_ has quit IRC15:08
mriedemdansmith: maybe, i know there have been several threads on this, and basically at this point those don't really matter :)15:08
dansmithaye15:08
mriedemi'm just reading back through the responses from the ML from earlier today15:08
sfinucansnikitin: If you could take a look at https://review.openstack.org/#/c/405954/ (and the follow up) today, I'd appreciate it. Think you should know enough about that code to comment15:09
*** smatzek has joined #openstack-nova15:09
*** esberglu has joined #openstack-nova15:09
johnthetubaguysfinucan: usually I would say no, depends how you feel I guess.15:09
sfinucanjohnthetubaguy: Sweet - I'll go for it so. It looks good and would be good to get it in15:10
snikitinsfinucan, sure15:10
*** lpetrut has joined #openstack-nova15:10
* sfinucan hopes mikal gives out prizes15:10
bauzasdansmith: question, what should I hard-fail ?15:11
bauzasdansmith: just that the placement section is empty, or that we aren't able to call the placement API by doing a query?15:11
*** moshele has joined #openstack-nova15:11
bauzasdansmith: I'd favor a real test call15:11
dansmithbauzas: just that the config isn't there I think15:12
dansmithbauzas: doing a real call in the middle of init_host to check that and failing would be terrible15:12
dansmithbauzas: because if it fails for some legit reason, computes wouldn't start15:12
bauzasdansmith: okay15:12
mriedemwhat do we need to touch init_host in the compute for?15:13
bauzasmriedem: for preventing ocata computes to start if they haven't filled in their placement section15:13
dansmithmriedem: to drive home that placement config is required15:14
dansmithmriedem: i.e. like anything else, there's not much point in starting if we don't have sufficient config to do anything15:14
claudiub|2sfinucan: so, VFs / PFs won't show up as assignable PCI devices, rather those are requested from Hyper-V, and Hyper-V will then assign them to the VMs.15:14
*** mtanino has joined #openstack-nova15:14
sfinucanclaudiub|2: as what? VIFs?15:14
johnthetubaguybauzas: +1 what dansmith said, that sounds right15:14
bauzasjohnthetubaguy: dansmith: and it would raise an exception if not CONF.placement?15:15
mriedemok so the grenade case should be ok b/c newton subnode will be running and ocata compute will have config so will start15:15
*** moshele has quit IRC15:15
dansmithmriedem: right15:15
mriedembut scheduler will use the old RT data because of the newton compute15:15
dansmithyes15:15
claudiub|2sfinucan: yeah. The vif has a property - if it SR-IOV is enabled for it or not. by default it is false.15:16
dansmithbauzas: yes, or whatever will arrest init_host cleanly15:16
bauzasdansmith: that's my question15:16
bauzasdansmith: should we log and stop ?15:16
bauzasor should we more like "sorry dude"15:16
dansmithI dunno what that means15:16
bauzasheh15:16
bauzasI mean15:16
bauzastwo choices15:16
*** hfu has joined #openstack-nova15:16
bauzas#1 log an ERROR and return15:16
bauzas#2 raise an exception15:17
bauzasI'd favor #115:17
dansmithbauzas: that won't stop compute from starting15:17
dansmiththat will just leave it further hamstrung15:17
bauzasokay, lemme see how init_host is working15:17
mriedeman unhandled exception from init_host will kill the nova-compute process15:18
mriedemwhich i think is the goal15:18
mriedemmeaning you upgraded the code to ocata but didn't configure it properly,15:18
mriedemso you need to get paged15:18
dansmithright, only an exception will stop it from what I see15:18
bauzasokay, that's my question15:19
bauzasbecause it's threaded ?15:19
*** sahid has quit IRC15:19
dansmithbauzas: so just raise an exception, which will be ugly, and we could maybe clean that up in service.py after the fact to make it a little less ugly on the eyes15:19
bauzasI mean gt15:19
dansmithnothing related to threading15:19
bauzasanyway15:19
bauzaslet's do the exception15:19
*** stvnoyes1 has quit IRC15:19
bauzasno time for introspecting the caller :)15:19
*** takashin has left #openstack-nova15:20
*** pkoniszewski_ has joined #openstack-nova15:20
mriedemi guess https://review.openstack.org/#/c/425524/ didn't work huh15:20
*** stvnoyes has joined #openstack-nova15:20
*** sahid has joined #openstack-nova15:20
dansmithI didn't look, this conversation was going on when I showed up15:21
dansmithhowever,15:21
openstackgerritGábor Antal proposed openstack/nova: Fixed indentation in virt/libvirt/driver.py  https://review.openstack.org/42574815:21
pkoniszewski_hey, can i have -W/+W there https://review.openstack.org/#/c/407518/ ? jenkins does not want to start gate job for some reason15:21
dansmiththe moreI explain it the more I think it's probably the better option anyway15:21
*** hongbin has joined #openstack-nova15:21
mriedembauzas: you can remove the dependency on https://review.openstack.org/#/c/425524/ when you respin your nova scheduler change,15:21
mriedemit didn't work anyway15:21
cdentmriedem: I checked into the grenade job and as far as I could tell, the subnode wasn't getting config'd, still15:22
mriedemcdent: yeah it wasn't15:22
mriedemthat features yaml is all magic to me15:22
* cdent nods15:22
bauzasmriedem: ack15:22
pkoniszewski_sfinucan: hi, are you around?15:23
sfinucanpkoniszewski_: aye15:23
pkoniszewski_sfinucan: can i have -W/+W there https://review.openstack.org/#/c/407518/ ? jenkins does not want to start gate job for some reason15:23
*** claudiub has joined #openstack-nova15:24
openstackgerritStephen Finucane proposed openstack/nova: Change exponential function to linear  https://review.openstack.org/40751815:24
*** claudiub|2 has quit IRC15:24
sfinucanpkoniszewski_: done15:24
pkoniszewski_sfinucan: thanks!15:24
openstackgerritArtom Lifshitz proposed openstack/nova: Fix tag attribute disappearing in 2.33 and 2.37  https://review.openstack.org/42475915:25
*** claudiub has quit IRC15:26
*** claudiub|2 has joined #openstack-nova15:26
dansmithI think we probably need to touch the oslo service stuff to make a really graceful exit from service startup15:27
*** ociuhandu has quit IRC15:27
dansmithactually, no15:27
mriedemi don't think so,15:28
mriedemwe had a bug where the pci whitelist parser was blowing up,15:28
openstackgerritClaudiu Belu proposed openstack/nova: Hyper-V PCI Passthrough  https://review.openstack.org/42061415:28
mriedemand that just killed the process15:28
mriedem^ came from the RT on init_host15:28
dansmithmriedem: no I mean to avoid a traceback15:28
mriedemoh15:28
openstackgerritDan Smith proposed openstack/nova: WIP: Allow graceful service exits from compute  https://review.openstack.org/42575115:29
*** psachin has quit IRC15:29
dansmithmriedem: something like this ^ to clean up what bauzas is adding now .. but later15:29
bauzasdansmith: I'm just writing UTs and then boooom15:30
sfinucanclaudiub|2: OK, I'd still like Moshe to jump in but I'm personally happy with that as is15:30
*** owalsh-afk is now known as owalsh15:30
claudiub|2sfinucan: cool, thanks! :D15:30
bauzasdansmith: you okay if I'm just shamely copying your exception ?15:30
claudiub|2sfinucan: will try to reach out to him as well, as soon as possible.15:30
bauzasdansmith: mine is too much explicit (PlacementNotConfigured)15:31
sfinucanSide note: we *really* need to find a way to standardized flavor extra specs like we do ImageMetaData15:31
bauzasdansmith: but I was basically doing the same, including the try/except clause in the cmd15:31
*** jaosorior has joined #openstack-nova15:31
*** sacharya has joined #openstack-nova15:31
*** iceyao has quit IRC15:32
mriedemi don't think create() is the correct method to handle,15:33
mriedemthat just creates the service object,15:33
dansmithbauzas: just use something else and we can merge my graceful bit later and I will clean it up15:33
mriedemstart() is the thing that blows up15:33
*** mdrabe has quit IRC15:33
bauzasdansmith: okay15:34
dansmithmriedem: yep, you're right15:34
dansmithmriedem: but I think I want to catch around both15:34
dansmithmriedem: in case we had to abort in low-level init or something15:34
mriedemi just don't know if serve() or wait() calls start()15:34
andreykurilinjohnthetubaguy: I answered all your comments and all both patches are vaiting for your review:)15:34
dansmithserve calls launch15:34
dansmithwhich calls start I think15:34
dansmiththis is why I made it fail so we can see :)15:35
openstackgerritDan Smith proposed openstack/nova: WIP: Allow graceful service exits from compute  https://review.openstack.org/42575115:35
*** burgerk has joined #openstack-nova15:35
*** READ10 has joined #openstack-nova15:35
*** stevemar has quit IRC15:36
*** armax has joined #openstack-nova15:37
*** stevemar has joined #openstack-nova15:37
*** catintheroof has joined #openstack-nova15:39
*** claudiub has joined #openstack-nova15:40
*** vsaienko has joined #openstack-nova15:41
cdentmriedem: if we stick to one full circle, I think we should consider that a win15:41
mriedemheh15:42
openstackgerritmelanie witt proposed openstack/nova: Count resources to check quota for cells  https://review.openstack.org/41652115:42
mriedemsfinucan: if you're looking for things, https://review.openstack.org/#/c/404990/ and the one after it have +2s15:42
mriedempretty simple15:42
*** claudiub|2 has quit IRC15:42
*** catinthe_ has quit IRC15:42
*** mdrabe has joined #openstack-nova15:43
*** vsaienko has quit IRC15:43
*** yamahata has joined #openstack-nova15:44
*** lpetrut has quit IRC15:44
sfinucanmriedem: I'll give it a shot15:44
*** karimb has quit IRC15:44
*** kfarr has joined #openstack-nova15:45
* sfinucan has veered away from anything that might involve SQL and Alchemy in the same sentence15:45
mriedemnone of that is sqla stuff15:45
raj_singhmriedem: If you have time, I got +2's on last two patches of service token bp.15:46
mriedemeven though it's a jaypipes patch15:46
mfeoktistovsfinucan: mriedem: mdbooth: sorry for disturbing you, I updated https://review.openstack.org/#/c/312488/  (rootwrap instead of privsep and small cleanup). Virtuozzo Storage CI fails because we have problem with cinder now, it's not releated to this review.15:47
melwittjohnthetubaguy, mriedem: I took the WIP off the counting quota patch last night after adding more test coverage15:47
*** raj_singh_ has joined #openstack-nova15:47
bauzasinteresting, how can I test that I don't have a conf section yet ?15:48
*** openstackgerrit has quit IRC15:48
andreykurilinmriedem bauzas johnthetubaguy: hi all! Do we want to put latest 7.1.0 (which is not released) to u-c ? If yes, there is only ~ 4 hours away now to make a release and propose c change to g-r repo15:48
bauzasI could mock the CONF object tho but I'd prefer to mimic15:48
dansmithbauzas: don't test for the section, test for the keys15:48
bauzasdansmith: same question, how could I unset that ?15:48
mriedemmfeoktistov: +215:48
mriedemmelwitt: how comfortable are you with the counting quotas stuff? is it too risky at this point? and if counting quotas doesn't get in, should we still block the patches below it to move those other quotas tables to the api db (except reservations and usages)?15:49
mriedemandreykurilin: are the patches all in for novaclient 7.1.0 yet?15:50
mriedembauzas: by default the tests won't have anything set in the CONF.placement values15:50
mdboothmfeoktistov: Did you change anything else?15:50
mriedemunless we have defaults15:50
*** openstackgerrit has joined #openstack-nova15:50
openstackgerritChris Dent proposed openstack/nova: Add more debug logging on RP inventory update failures  https://review.openstack.org/41423015:50
mriedemmfeoktistov: he moved some small things around it looks like15:50
mriedemmdbooth: ^15:51
mdboothIIRC I was +1 on this before, so if they only thing you've change is privsep I'm still +1.15:51
cdentmriedem: that's ^ the long promised test additions to that logging thing you did a while back. with some adjustments15:51
* mdbooth will look15:51
andreykurilinmriedem: only one(+one not critical in the chain) left - https://review.openstack.org/#/c/424256/ . It has one +2 and answered all comments15:51
*** lpetrut has joined #openstack-nova15:51
bauzasmriedem: dansmith: the only problem I see is that we expect keystone config options15:51
mriedemandreykurilin: ok, looks like johnthetubaguy has been reviewing https://review.openstack.org/#/c/424256/ so i'll leave the +W to him15:52
bauzasso I need to verify them15:52
melwittmriedem: I'm pretty confident in the counting stuff. I think it's no worse than what's already there. we would trade known issues and out-of-sync problems for something with a lot smaller footprint, but obviously has some element of the unknown compared to something we've had for years15:52
*** karimb has joined #openstack-nova15:52
mfeoktistovmdbooth: no, just rootwrap instead of privsep, move two lines of code from libvirt_driver to libvirt_utils and fix tests for it15:53
mriedembauzas: i think you just need to check for CONF.placement.password15:53
mriedemthat *has* to be set15:53
bauzasmriedem: okay, I can just do that15:53
melwittmriedem: as for the bottom patches, if we assume a single cell, I think it still works, it's just reservations and usages would be in the main db whereas the limits and classes would be in the api db.15:53
mriedemhttp://logs.openstack.org/72/404472/20/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/8aafdf2/logs/etc/nova/nova.conf.txt.gz15:53
mriedembauzas: ^ for what we set in devstack15:54
mriedembut there are some deprecated things in ksa so i'd just stick with checking for the password option15:54
bauzasmriedem: yup, I know15:54
mriedemlike user_id vs username, etc15:54
bauzasmriedem: I'm just hesitating about checking things related to keystone diretly15:54
mriedemand the 10 different ways you can specify a project in ksa15:54
mfeoktistovmdbooth: code which gets default filesystem type from nova conf or ext4 as default value15:54
bauzasmriedem: given we don't verify that by us but instead straight passing it to keystoneauth15:54
mdboothmfeoktistov: Yep, spotted15:55
mriedembauzas: i don't know what that means, or why we care15:55
bauzasanyway, let's not overcomplicate things15:55
cdentsfinucan: on https://review.openstack.org/#/c/404990/ there's this to fix that regex (next cycle): https://review.openstack.org/#/c/418393/15:55
mriedembauzas: init_host just checks, if CONF.placement.password is None: raise Exception15:55
bauzasmy patch is almost ready and I have to leave in 515:55
dansmithmriedem: on the quotas thing, I think it's risky, but I also think it's more likely that anyone else will be able to fix bugs in the counting stuff, so if you want to push, I'll dig into that last one15:55
*** mvk has quit IRC15:55
sfinucancdent: Aha - thank you. That did seem like a peculiar oversight15:56
dansmithmriedem: and I think there's probably truth to the argument that it's no worse than what we have now, but it is more obvious than what we have now15:56
mdboothpkoniszewski pkoniszewski_ (irc is the worst): Are you able to determine what I'm talking about through gerrit's formatting?15:56
dansmithmriedem: but also, we're merging other stuff with high risk, so...15:56
cdentsfinucan: it's because 409 is overloaded for at least two different types of conflict15:56
mriedemdansmith: heh, so while we're playing with house money...15:56
mriedemdansmith: feel free to push, i just won't be able to get to it15:56
pkoniszewski_mdbooth: but the point is that there is no such race15:56
dansmithmriedem: I meant "maybe we shouldn't take out another loan" but okay15:57
mdboothpkoniszewski_: Explain15:57
cdentit's called leverage15:57
mriedemdansmith: oh i thought you wanted to go balls out15:57
mdboothpkoniszewski_: I described the race. Can you point out the step I got wrong?15:57
*** annegentle has joined #openstack-nova15:57
dansmithmriedem: I meant the last statement was a hedge, not a "we've already got a second mortgage, why not go for a third"15:58
dansmithmriedem: let me look through it and see how I feel after that15:58
mdboothpkoniszewski_: I think cfriesen(?) said they'd hit it in practise, too.15:58
mriedemhey, the dow was over 20K so i think that's a safe bet right now15:58
mriedemwhat could possibly go wrong15:58
dansmithheh15:58
mfeoktistovmriedem: thanks for discussion in mailing list15:58
pkoniszewski_mdbooth: live migration will always be set to  'completed', there is no chance we will end up  with 'failed' or 'finished' state15:59
*** yamahata has quit IRC15:59
dansmithjohnthetubaguy: are you still around?15:59
pkoniszewski_mdbooth: the RPC has been changed to call15:59
pkoniszewski_mdbooth: https://review.openstack.org/#/c/389687/15:59
johnthetubaguydansmith: I am15:59
bauzasthere it is15:59
dansmithjohnthetubaguy: how are you feeling about the counting quotas stuff?16:00
openstackgerritSylvain Bauza proposed openstack/nova: Scheduler calling the Placement API  https://review.openstack.org/41796116:00
*** nkrinner is now known as nkrinner_afk16:00
mdboothpkoniszewski_: So it has, I must have been looking at an old version when looking at the old commit.16:00
mdboothWell... for the reasons stated I also think that's a bad idea :)16:00
mdboothBut yeah, that means there's no race.16:01
pkoniszewski_mdbooth: but there is still an issue with rollback, thats a  very good point16:01
melwittmriedem, dansmith, johnthetubaguy: one thing I realized about the counting quotas approach is that once we go multicell, if we lose contact with a cell db for whatever reason, we lose the ability to count resources in it, which would basically free up a person's resource usage for the time period the cell db is inaccessible16:02
sfinucanmfeoktistov: +W'd16:02
sfinucanalso, worst IRC nick ever :)16:02
sfinucan...and mine is bad16:02
dansmithmelwitt: yeah, that would be bad..16:03
*** yamahata has joined #openstack-nova16:05
dansmithmelwitt: that would be for what kind of resources?16:05
*** zz_dimtruck is now known as dimtruck16:05
dansmithmelwitt: we can count instances and server groups and keypairs all in the api db16:05
mdboothpkoniszewski_: Incidentally, you could add a comment to the 'completed' code that it can be safely deleted in Queens, because we can guarantee the destination will have already updated migration status.16:05
melwittdansmith: it seems like we'd need to have some sort of caching table for the usage that gets updated anytime we actually count16:05
melwittdansmith: currently, I think instances, cores, ram. but that's because I rolled the instance tally into the cores and ram count16:06
dansmithmelwitt: that kinda defeats the purpose,in that it becomes a thing that can get out of sync and that we can't refresh without being able to contact the other cell16:06
dansmithmelwitt: okay well we could count instances in the api db to mitigate that16:06
johnthetubaguymelwitt: maybe we hard fail on that16:07
dansmithmelwitt: so maybe we should get a read from operator(s) about how bad it would be be to not be if ram/cores leaks while a whole cell is offline16:07
johnthetubaguymelwitt: if we know something is there16:07
dansmithjohnthetubaguy: hard fail to create instances anywhere if one cell is offline? that'd be bad :)16:07
pkoniszewski_mdbooth: AFAIR nova does not assume that there will only be live migration from N-1 to N (N-2 to N must also be possible and safe)16:07
melwittdansmith: yeah. I haven't thought about it much but it seemed like it makes sense for it to be stale if the cell is gone. but if it went away permanently, then yeah that would be the refresh/sync case. so nevermind that idea16:07
mdboothpkoniszewski_: I understood the contract was that N and N-2 never exist in the same deployment, right?16:08
melwittdansmith, johnthetubaguy: yeah, we could fall back to InstanceMappings for the instance count but cores/ram would be unknown16:08
dansmithmelwitt: well, no I meant even if it's offline temporarily, you wouldn't be able to refresh that value from N-1 cells because you don't know how much of it is in the offline cell, unless you keep more data16:08
mdboothSo no migration between them is implicit.16:08
mdboothdansmith: ^^^ ?16:08
dansmithmdbooth: correct16:08
bauzasmriedem: dansmith: cdent: rev'd https://review.openstack.org/41796116:09
melwittdansmith: oh, right16:09
dansmithmdbooth: pkoniszewski you shouldn't break your RPC api between N-2 and N, but we also don't support that kind of deployment, so higher level assumptions might not be possible between those two16:09
cdentbauzas: cool, will look asap, currently double booked in two meetings ...16:09
*** krtaylor has quit IRC16:09
mriedembauzas: ok. i think we should move the compute/service version checking stuff into a separate patch for sanity16:10
mriedembauzas: but sounds like you're going to be afk for awhile16:10
dansmithbauzas: that service version and fail should be a separate patch16:10
dansmithwhat mriedem said16:10
bauzasmriedem: yup, I could potentially lightweight the change16:10
pkoniszewski_dansmith: got it16:10
melwittdansmith: actually I guess the long term answer was we're going to ask for cores/ram count from placement when that's doable16:11
bauzasmriedem: dansmith: gimme just 20 mins for my daughter's homework and I'm back16:11
dansmithmelwitt: ah, that's true16:11
dansmithmelwitt: I forgot about that part16:11
mdboothpkoniszewski_: Either way, it can be deleted as soon as we can assume the other end is doing the update. Worth adding a comment that we added the remote end in N.16:12
pkoniszewski_mdbooth: yes, i'll add comment there16:12
mfeoktistovsfinucan: i never thought that "ploop" sounds bad, until i heard the pronunciation of other developers not from virtuozzo. Yes, i agree it sounds strange, to say the least .16:14
mdboothpkoniszewski_: This code is hairy as hell, btw. Thanks for taking it on.16:14
*** karimb has quit IRC16:15
mfeoktistovsfinucan: we pronounce like pe + loop. "Parallels loop" . Parallels is previous name of our company16:15
mdboothmfeoktistov: diana_clarke particularly enjoys it, btw. IIRC it's her favourite disk format.16:16
sfinucanmfeoktistov: Aha - that nuance escaped me :)16:16
* sfinucan nearly choked the first time he heard his Donegal lecturer talk about "funny GUI things" 16:17
mdboothsfinucan: Had you previous used gee-you-eye?16:17
diana_clarkePloop! There it is! https://www.youtube.com/watch?v=Z-FPimCmbX816:17
mdboothdiana_clarke: lol16:18
*** mdnadeem has quit IRC16:19
sfinucanmdbooth: In my blessed ignorance, yes16:19
alaskidansmith: melwitt just caught part of scrollback from last night. I wanted BuildRequest.instance to be orphaned because I didn't want it treated like an instance, it's a holding place for what an instance should look like when it's created but isn't an instance itself. It's a subtle distinction made complicated by using an actual instance object there. I thought thinking of it that way would lead to16:19
alaskifewer bugs.16:19
dansmithalaski: right, I remember16:20
sfinucanWe got odd looks for wanting to play Call of Duty (COD) in past years too16:20
dansmithalaski: I even went looking for where we had that convo, but it must have been in IRC16:20
alaskidansmith: cool. I also said I would defer to those who were doing the work :)16:20
mfeoktistovdiana_clarke: ahaha! good track for presentation16:20
dansmithalaski: s'all good now16:20
*** krtaylor has joined #openstack-nova16:21
cfriesenmdbooth: thanks for taking a look at the live-migration claims patch.  It'd be good to finally get that stuff in.16:22
melwittalaski: yeah, once we realized what was happening, it was clear what to do16:22
alaskidansmith: melwitt excellent16:24
melwittand I could see allowing it to masquerade as a normal Instance object could lead to a lot of subtle problems, if anything were missing from its fields16:25
*** dtp has joined #openstack-nova16:25
dansmithon the other hand, in reality,16:25
dansmithwe used that to smooth over where we got the instance,16:25
dansmithso down the stack we've lost context (heh) about why it might not be a real one16:26
melwittyeah. I suspect if it had a real context, the problem would travel further down the stack and be really puzzling16:26
melwitt*even further16:27
johnthetubaguymelwitt: sorry, got distracted, which bits where you counting when you needed the cells again?16:27
dansmithI dunno, the refresh should work if we have uuid and context16:27
*** karimb has joined #openstack-nova16:27
*** tbachman has quit IRC16:27
dansmithjohnthetubaguy: the counting patches counts instances, ram and cores from the cell db16:28
*** liangy has joined #openstack-nova16:28
melwittjohnthetubaguy: the ram and cores, and I tally the instances while counting ram/cores. ram and cores are being counted currently by pulling the instances and going through their flavor extra specs16:28
dansmithjohnthetubaguy: if we lose connectivity with a cell for ten minutes, a user suddenly gets their quota back until we reconnect and could burn it in other cells16:28
* johnthetubaguy face palm16:28
johnthetubaguyyeah, that makes sense16:29
melwittjohnthetubaguy: the real plan is to call the placement api to get cores/ram count and use InstanceMappings for instances, but we can't get that from placement quite yet16:29
*** slaweq has quit IRC16:29
johnthetubaguymelwitt: ah, true, well that to make placement do quota too ;)16:30
johnthetubaguys/to/or16:30
*** jaypipes_ has joined #openstack-nova16:30
johnthetubaguymelwitt: FWIW, its tempting to fail hard if a cell is down, and we know there is something in there we should be counting16:30
dansmithjohnthetubaguy: how does that work?16:30
*** slaweq has joined #openstack-nova16:31
dansmithjohnthetubaguy: if one cell is down, we can't do anything? that kinda defeats the purpose no?16:31
dansmithwell, s/do/create/ but ..16:31
melwittyeah, that seems like it would draw much ire16:31
dansmithyeah16:31
*** pkoniszewski_ has quit IRC16:31
dansmithjohnthetubaguy: so, we can count instances in the api database,16:31
*** tbachman has joined #openstack-nova16:31
dansmithjohnthetubaguy: which would leave just cores and ram as leaky if a whole cell went down16:32
dansmithjohnthetubaguy: which should be rare, and maybe some quota leakage isn't the worst thing in that case, given we will someday have placement do better16:32
johnthetubaguyyeah, thats fair, if we did it that way, it feels less of a big deal16:32
dansmithbut I dunno16:32
dansmithI feel like maybe that needs to be a question to the ops, but we have no time for that16:32
johnthetubaguyyeah16:32
*** lpetrut has quit IRC16:33
johnthetubaguyhonestly, feels like fail hard and don't care will be both be requested, depending on how much RAM and CPU quota matters to you16:33
johnthetubaguyI like that the instance count one works just fine though16:33
*** raj_singh_ has quit IRC16:35
melwittusually maybe the cores/ram leak stays bounded by the instances quota, but if someone has set unlimited instances and limited cores/ram, that would be a lot bigger problem16:35
*** hfu has quit IRC16:36
dansmithyeah16:36
dansmiththat's why asking the operators would be good16:36
*** baoli has quit IRC16:36
*** hfu has joined #openstack-nova16:37
sfinucanmriedem: If /you're/ looking for something to do, could you take a look at https://review.openstack.org/#/c/405954 at some point?16:37
mriedemcurrently reviewing https://review.openstack.org/#/c/40447216:37
*** hfu has quit IRC16:37
mriedembut i already had https://review.openstack.org/#/c/405954 in a tab16:37
dansmithsfinucan: you know mriedem is on vacation right? I know, it's hard to tell16:38
*** hfu has joined #openstack-nova16:38
sfinucana tab will do nicely16:38
jaypipes_edleafe: are you handling the change to https://review.openstack.org/#/c/404472/17/nova/compute/resource_tracker.py that you said you were looking at yesterday?16:38
*** jamesdenton has joined #openstack-nova16:38
jaypipes_edleafe: if not, I will do so.16:38
jaypipes_and good morning fellas.16:38
*** hfu has quit IRC16:38
sfinucandansmith: I do. I've already given out to him for being around but he won't listen16:38
*** vsaienko has joined #openstack-nova16:38
dansmithsfinucan: he's not very good at vacation16:38
* dansmith goes to unendorse mriedem on linkedin for "vacation"16:39
*** chyka has joined #openstack-nova16:39
*** hfu has joined #openstack-nova16:39
sfinucanYou Americans and your pesky work ethic16:39
*** chyka has quit IRC16:39
*** hfu has quit IRC16:39
sfinucanWe clearly need a Greek PTL16:39
jaypipes_lol16:39
*** hfu has joined #openstack-nova16:39
* sfinucan apologies to any hard working Greek folks he may have offended :)16:40
jaypipes_my idea of vacation is doing code reviews by a pool.16:40
*** hfu has quit IRC16:40
dansmithmriedem: bauzas: that change is failing because it can't find placement.password16:40
kashyapsfinucan: Quit giving those non-apologies16:41
bauzasdansmith: yup, I'm working on a new PS16:41
dansmithbauzas: you have a fix?16:41
kashyapsfinucan: Reminds me about an article on the word 'humbled' in NY Times lately.  ;-)16:41
mriedemjaypipes: edleafe: just left some comments in https://review.openstack.org/#/c/404472/16:41
bauzasdansmith: by also excepting the NoConfOpt + modifying the ServiceFixture16:41
sfinucan:D16:41
mriedemjaypipes: i'm not all of the way through it yet16:41
mriedemjaypipes: but the 'latest' microversion in the scheduler report client is what the -1 is for16:41
dansmithbauzas: is that going to work?16:42
dansmithbauzas: because it's failing to find it on a run that should have it16:42
bauzasdansmith: if we ConfPatcher the ServiceFixture16:42
dansmithbauzas: you might have to check the region name, since that's our conf opt16:42
dansmithbauzas: it's failing in real devstack runs!16:42
dansmithhttp://logs.openstack.org/61/417961/28/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/ed475f7/logs/screen-n-cpu.txt.gz?level=TRACE16:42
dansmithbauzas: ^16:42
*** yamahata has quit IRC16:42
bauzasinteresting16:42
cfriesenjaypipes_: get a toughbook and you can do code reviews *in* the pool16:43
bauzasanyway, I'm just splitting the big change16:43
dansmithbauzas: I think it's because of lazy conf evaluation16:43
*** marst has joined #openstack-nova16:43
mriedemNoSuchOptError: no such option password in group [placement]16:44
mriedemthat seems wonky16:44
edleafejaypipes_: yes, I fixed that already16:44
*** chyka has joined #openstack-nova16:44
jaypipes_cfriesen: :)16:44
jaypipes_edleafe: ty sir16:44
jaypipes_edleafe: sorry was looking at an old rev comment16:45
*** lpetrut has joined #openstack-nova16:48
jaypipes_edleafe: I don't want to step on toes if you are already working on it, so let me know if you are addressing mriedem's comments?16:48
johnthetubaguybauzas: I would try auth_url anyways, as password bit is pluggable16:48
edleafeyeah, looking at them now16:48
johnthetubaguyfunky though, looks like it is there: http://logs.openstack.org/61/417961/28/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/ed475f7/logs/etc/nova/nova.conf.txt.gz16:49
edleafejaypipes_: aren't you suposed to be drinking pina coladas or something?16:49
jaypipes_edleafe: meh, coffee and email this morning :)16:49
cdentjaypipes_: what about getting caught in the rain?16:51
*** sahid has quit IRC16:51
dansmithit's not even morning in florida anymore16:52
edleafejaypipes_: any idea about mriedem's comment here: https://review.openstack.org/#/c/404472/20/nova/scheduler/client/report.py@17416:52
edleafeThat got changed in PS 816:53
*** vsaienko has quit IRC16:53
*** tesseract has quit IRC16:55
openstackgerritDan Smith proposed openstack/nova: WIP: Allow graceful service exits from compute  https://review.openstack.org/42575116:55
jaypipes_edleafe: I don't see that changed in PS8. but anyway, it's likely just an oversight. There were so many patches that changed that line that I've lost track at this point which introduced latest and which introduced 1.X specified explicitly.16:57
jaypipes_edleafe: in this particular case, it should be the microversion that added resource classes, so 1.2, yes?16:57
edleafejaypipes_: not sure, but that sounds about right16:58
bauzasjohnthetubaguy: dansmith: so I made the spit16:58
bauzassplit even16:58
bauzasjohnthetubaguy: dansmith: are you suggesting me other way to verify if the placement section is correct?16:59
bauzasI'm all ears16:59
edleafejaypipes_: https://review.openstack.org/#/c/404472/8..20/nova/scheduler/client/report.py@17416:59
edleafejaypipes_: that's where it got added16:59
dansmithbauzas: I said check os_region right?16:59
johnthetubaguybauzas: I was thinking of using auth_url CONF instead16:59
dansmithbauzas: that's our config value, should be there by the time you need it16:59
dansmithjohnthetubaguy: is that going to be different than password from an order of loading perspective? because his patch fails on password not even being a thing by the time we're loading17:00
dansmiths/loading/running/17:00
*** bvanhav has quit IRC17:00
bauzasdansmith: you mean os_region_name?17:00
*** andreas_s has quit IRC17:00
bauzasthat's something coming from us ?17:00
johnthetubaguydansmith: unsure, but I like your os_region suggestion better17:01
johnthetubaguyhttps://github.com/openstack/nova/blob/master/nova/conf/placement.py#L2217:01
*** bvanhav has joined #openstack-nova17:01
johnthetubaguydansmith: bauzas: password is part of the pluggable auth and not not actually required, so it shouldn't be that one17:01
bauzasokay, let's try os_region_name then17:02
bauzasdansmith is right, given it's the only one we have in tree17:02
*** raunak has joined #openstack-nova17:02
johnthetubaguy(assuming we don't later decide os_region_name should be defaulted to RegionOne, but lets just be sure not to do that this cycle!)17:02
bauzasand password means a password auth, which could be changing in the future17:02
bauzasjohnthetubaguy: I could write an unittest for preventing that17:03
*** liverpooler has quit IRC17:03
johnthetubaguybauzas: cool17:03
bauzasjohnthetubaguy: meaning that I'm expecting a None value17:03
johnthetubaguyack17:03
openstackgerritClaudiu Belu proposed openstack/nova: hyper-v: Adds support for VMs with nested virtualization  https://review.openstack.org/39850917:03
bauzasso someone defaulting the opt would have to cut that17:03
bauzasokay, let's try again17:03
*** liverpooler has joined #openstack-nova17:04
*** pbandark has joined #openstack-nova17:04
bauzasshit, I need a fixturre17:07
jaypipes_edleafe: I must be seeing something different than you... I don't see placement latest being added in that patch?17:07
bauzasdansmith: johnthetubaguy ^17:07
jaypipes_edleafe: i.e. that line isn't green in gerrit.17:07
jaypipes_edleafe: in any case, not sure it matter, right? solution is straight-forward. just s/latest/1.2/, yesh?17:08
sfinucanerm, vladikr - you're looking at this, right? https://review.openstack.org/#/c/424786/ :)17:09
mriedemjaypipes: i'd prefer the calling code is explicit about the microversion it needs, so GET and POST for resource classes,17:09
mriedemnot globally in the init17:09
* sfinucan imagines it's the test that's broken, but would like confirmation17:09
*** pbandark has quit IRC17:09
mriedemedleafe: ^17:09
vladikrsfinucan, yeah, but it's not really possible to test this with tempest17:09
sfinucanvladikr: Oh, wait - that was the thing that required hardware, right?17:10
edleafejaypipes_: this is what I see: http://imgur.com/a/ocO8X17:10
vladikrsfinucan, yeah, I should abandon that17:10
edleafejaypipes_: just so you don't think I crazy (well, crazier)17:10
sfinucanvladikr: Do please17:10
sfinucanthough I'd also suggest submitting a patch for https://github.com/wznoinsk/hw-tests17:10
dansmithbauzas: I'm confused -- is os_region_name what you're going to use instead of password?17:11
vladikryeah17:11
bauzasdansmith: yup17:11
sfinucanvladikr: wznoinsk should be able to provide more information on that project, if you ask nicely17:11
dansmithokay I thought you were saying you weren't17:11
dansmithso.. good17:11
bauzasdansmith: sorry then17:11
dansmith:)17:11
edleafemriedem: ok, I'll look for where those are needed.17:11
bauzasdansmith: so, it works with devstack (like the t-shirt)17:11
bauzasdansmith: but for UTs and functional tests, I need to make a fixture and add it to nova.test17:11
bauzasdansmith: in order to set the config opt17:12
mriedembauzas: just throw it in the global conf patcher17:12
bauzasunless I'm just setting the opt straight17:12
mriedemthat is run on all tests by default17:12
mriedemit's a 1 line change17:12
bauzasmriedem: oh right17:12
bauzasI forgot it, but yeah, much better17:12
*** ducttape_ has joined #openstack-nova17:12
jaypipes_edleafe: yes, that's what I see too. Notice that the line he is referring to is white, meaning the patch has not changed that line.17:13
jaypipes_mriedem: you mean passing in a specific microversion header for each GET/POST call?17:14
mriedemjaypipes_: yes, like we're already doing for 1.1 with aggregates17:14
jaypipes_mriedem: got it. edleafe, got it? :)17:15
edleafejaypipes_: yeah, but if you compare to PS7, it's green17:15
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver (cfg drv)  https://review.openstack.org/40940417:15
edleafemeaning it's been in the code since PS817:16
edleafeIAC, I'm working on the changes17:16
*** mjura has quit IRC17:16
mriedembauzas: yeah in here somewhere https://github.com/openstack/nova/blob/master/nova/tests/unit/conf_fixture.py#L3017:17
bauzasmriedem: already did17:17
mriedemdansmith: oh nice https://review.openstack.org/319379 is approved17:17
bauzasrunning tests and uploading in a sec17:17
mriedemwas just going to check on that17:17
dansmithmriedem: #10 in the gate MOFO17:17
dansmithmriedem: I'm watching it like my life depends on t17:18
johnthetubaguymelwitt: I sent a note on that quota patch, its not a blocker I am just curious what you think though.17:18
johnthetubaguymelwitt: looking at the latest one now.17:18
mriedemdansmith: was melwitt good with it?17:19
dansmithmriedem: she pushed the last change17:19
dansmithso I'm guessing17:19
mriedemoh excellent17:19
*** lucasagomes is now known as lucas-afk17:19
melwittmriedem, dansmith: yes, all good17:19
melwittjohnthetubaguy: I just saw, thanks. I do like the idea of making the limit check atomic but I'm not familiar with the generation id and how passing a deltas dict helps?17:20
johnthetubaguymelwitt: I was really thinking, we do the generation thing much later, but the new interface would allow us to do that without changing all the callers17:21
*** nic has joined #openstack-nova17:21
melwittjohnthetubaguy: ah, gotcha17:21
jaypipes_edleafe: danke17:21
*** ekuris__ has quit IRC17:22
*** derekh has quit IRC17:24
melwittjohnthetubaguy: okay, so you mean moving the counting call inside of Quotas.check_deltas for now. that makes sense17:24
johnthetubaguymelwitt: yeah, squash the two calls into a single call17:25
mriedemvladikr: question in https://review.openstack.org/#/c/405954/17:25
vladikrmriedem, looking17:26
*** unicell has joined #openstack-nova17:26
melwittjohnthetubaguy: it would clean things up a lot too17:26
johnthetubaguymelwitt: its a nit, I was wondering if get counts should be on the list object?17:26
*** _ducttape_ has joined #openstack-nova17:26
melwittjohnthetubaguy: it could. I wasn't opinionated about it. I feel like there might be an example of a count on an existing list object already17:28
*** purplerbot has quit IRC17:28
*** tbachman has quit IRC17:28
*** purplerbot has joined #openstack-nova17:28
johnthetubaguymelwitt: I could see arguments both ways I guess17:28
mriedemvladikr: similar in https://review.openstack.org/#/c/405951/17:29
mriedemvladikr: basically, we're doing the same stuff in both the metadata api code as in the libvirt driver,17:29
mriedemvladikr: i'm not sure why we don't just store the vlan on the NetworkInterfaceMetadata object in the libvirt driver, like the tag,17:30
mriedemand then use that in the metadata api17:30
*** ducttape_ has quit IRC17:30
melwittjohnthetubaguy: yeah. I just did a quick grep and didn't find an existing example of a count. so I think I just guessed :P17:30
melwittthere's the InstanceGroup.count_members_by_user but it counts members for a single group so that makes sense17:31
vladikrmriedem, yes, in the first patch we need to determine which vifs to include in the list of device that the metadata will receive and then in the second patch we will create the metadata only for those device that has a vlan or a tag17:31
vladikrmriedem, https://review.openstack.org/#/c/405951/13/nova/api/metadata/base.py@42817:31
*** mfeoktistov has quit IRC17:32
*** karimb has quit IRC17:33
*** karimb has joined #openstack-nova17:36
mriedemvladikr: yeah but we build that list twice17:36
mriedemwhen we already know it in the libvirt driver,17:36
mriedemand could store the vlan on the same object that has the tag17:37
*** cdent has quit IRC17:37
mriedemdansmith: bauzas: johnthetubaguy: i'm heading out for a few hours,17:37
mriedemjust a reminder on https://review.openstack.org/#/c/424256/ so we can release novaclient 7.1.017:37
*** kencjohnston has joined #openstack-nova17:38
bauzasmriedem: want me to tag the release btw. ?17:38
vladikrmriedem, yes, that's what the previous iteration of the code was doing17:38
openstackgerritSylvain Bauza proposed openstack/nova: Scheduler calling the Placement API  https://review.openstack.org/41796117:39
openstackgerritSylvain Bauza proposed openstack/nova: Fix server group functional test by using all filters  https://review.openstack.org/42518517:39
openstackgerritSylvain Bauza proposed openstack/nova: Block starting compute unless placement conf is provided  https://review.openstack.org/42580617:39
vladikrmriedem, I'll look at it again17:39
mriedembauzas: i'll be back around this afternoon to do that assuming we get the other things merged17:39
bauzasdansmith: mriedem: the branch is now up ^17:39
bauzasmriedem: okay, I'll have to bail out by around 11pm UTC because I have to wake up early tomorrow17:40
*** tbachman has joined #openstack-nova17:40
bauzasmriedem: ie. midnight my time17:40
bauzasnot sure which TZ you're in17:40
mriedembauzas: GMT17:40
bauzasmriedem: I need to go back and forth tomorrow to Patis17:40
bauzasParis even17:40
mriedembauzas: i'm asking about the o-3 timeline + FF in the release channel too17:40
bauzasso mostly on train during the day17:41
*** hfu has joined #openstack-nova17:41
*** baoli has joined #openstack-nova17:41
*** mriedem has quit IRC17:41
bauzasmriedem: oh f*** I tought I was following17:41
dansmithbauzas: is the change to _enabled_filters still needed there?17:42
johnthetubaguymelwitt: that destroy_and_raise_if_over_quota, is that worth the extra maintenance cost?17:43
bauzasyou mean the bottom change?17:43
*** jogo has quit IRC17:43
bauzasdansmith: ^17:43
*** liverpooler has quit IRC17:43
*** liverpoo1er has joined #openstack-nova17:43
melwittjohnthetubaguy: replying to that right now on the review17:43
dansmithbauzas: L48 of the bottom change17:43
bauzasdansmith: yeah it should still, because we have a different behaviour in case we don't have all the filters17:43
*** hfu has quit IRC17:43
bauzasdansmith:oh sec17:44
*** jogo has joined #openstack-nova17:44
bauzasdansmith: I misunderstood your question17:44
dansmithI don't think it should be different if we're ignoring those filters17:44
bauzasdansmith: in theory, the affinity filters are part of the default17:44
dansmithbauzas: also, the other tests didn't have a list for that parameter?17:44
bauzasdansmith: so we should not need L48 theorically17:44
bauzasdansmith: but17:44
dansmithbauzas: what about the service_down_time thing?17:45
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver  https://review.openstack.org/39128817:45
bauzasdansmith: I forgot to make a note here17:45
bauzasdansmith: I answered edleafe's on a previous PS17:45
bauzasand sorry pidgin crashed when I was replying to you about the _enabled_filters17:46
bauzasI preferred to explicitly ask for them in L48 given we could remove them from the default list later on17:46
dansmithyou use pidgin for irc like a n00b? :)17:46
dansmithalright17:46
*** karimb has quit IRC17:46
bauzasdansmith: indeed, I love GUIs17:47
melwittthat was a burn17:47
bauzasand you know what ? I loooove Atom17:48
artomdansmith, let me guess, you jiggle magnets around some wires to send your messages?17:48
bauzaswe're almost Friday17:48
bauzasso I think I have the wildcard for telling that17:48
dansmithbauzas: -1 on the middle patch17:48
dansmithI think the test is broken17:48
bauzasdansmith: ack, lookind17:49
dansmithbauzas: I love guis too17:49
bauzasoh "this" ans "that"17:49
bauzasmy English teacher should hit me17:49
*** jose-phillips has joined #openstack-nova17:49
dansmithbauzas: that's the bottom, see the middle one17:49
bauzasdansmith: I know :)17:50
dansmithokay17:50
bauzasdansmith: I just saw your comment :)17:50
bauzasdansmith: for the middle patch17:50
bauzasgood points17:50
bauzasI can respin quickly17:50
bauzasfor the 2nd test I'm adding17:50
dansmithI would recommend waiting until we get a run so we know if the top one is going to work, but if you have to go before it will finish, then just push17:51
bauzasthat is not running init_host, that's rather a defensive patch for making sure anyone can't just change nova.conf.placement opt and add a default without being kicked in the ass17:51
bauzasdansmith: well, I'll get an IndentationError anyway, right?17:51
bauzasdansmith: so I'm pretty sure it would fail anyway17:51
*** sambetts is now known as sambetts|afk17:51
bauzasdon't know how I left that FWIW17:51
bauzasprobably because I said I love Atom in public17:52
dansmithoh I see, well, might be good to assert the positive case in another test anyway17:52
bauzasdansmith: I can put that elsewhere17:53
bauzasdansmith: if you have ideas17:54
bauzasdansmith: I'm just about uploading a new Ps17:54
bauzasdansmith: unless you disagree17:54
dansmithdon't disagree17:54
*** _ducttape_ has quit IRC17:55
*** ducttape_ has joined #openstack-nova17:55
*** _ducttape_ has joined #openstack-nova17:56
bauzask17:57
bauzasshooting, then17:57
*** _ducttape_ has quit IRC17:58
*** kfarr has quit IRC17:59
*** david-lyle has quit IRC17:59
*** Apoorva has joined #openstack-nova17:59
*** ducttape_ has quit IRC18:00
johnthetubaguymelwitt: oh, resize is a bit of a bummer in that quota logic.18:02
openstackgerritSylvain Bauza proposed openstack/nova: Scheduler calling the Placement API  https://review.openstack.org/41796118:02
openstackgerritSylvain Bauza proposed openstack/nova: Block starting compute unless placement conf is provided  https://review.openstack.org/42580618:02
*** ralonsoh has quit IRC18:03
*** annegentle has quit IRC18:03
*** annegentle has joined #openstack-nova18:03
*** unicell has quit IRC18:04
melwittjohnthetubaguy: yeah, it's ugly. it would look a lot better with the check_deltas idea18:05
*** rmart04 has quit IRC18:05
johnthetubaguymelwitt: I was thinking, its not until resize confim that we update the DB with the new larger flavor so its counted correctly, thats quite a big window18:05
dansmithbauzas: did you not see my comment on the last patch about enabling service version caching?18:06
bauzasdansmith: it seems not18:06
dansmithbauzas: okay, I'll fix18:06
bauzasmy bad18:06
dansmithoh,18:06
dansmithI thought you were leaving soon, it's only 1900 your time right?18:06
bauzasdansmith: I'm still here for a couple of hours :)18:07
dansmitheither way, let us wait for a run of the top one anyway18:07
dansmithand if it's after you leave I'll fix it18:07
melwittjohnthetubaguy: ah, yeah. it's kind of like that now though, since it does a reserve right away, and quota is checked against reserved + in_use. the quota commit is in the confirm_resize but all that does it delete the reservation and increase the in_use18:07
bauzasdansmith: I just need to bail out for 60 mins18:07
dansmithokay18:07
bauzasdansmith: but yeah, it's 7:07pm18:07
bauzasdansmith: so I should be back around 9pm to midnight18:07
melwittjohnthetubaguy: oh, you mean with the new way, the in_use doesn't increase until the confirm. yeah18:08
dansmithokay18:08
bauzasie. 2000UTC to 2300UTC18:08
*** owalsh is now known as owalsh-afk18:08
johnthetubaguymelwitt: but the reservations count towards quota, so its a bit different18:08
bauzasdansmith: I can do the fix related to your -118:08
*** annegentle has quit IRC18:08
johnthetubaguymelwitt: yeah, I think the count instances just needs to use the new flavor if its a pending resize (I kinda hope there is a cleaner way though)18:08
bauzasdansmith: but if you have time for it, go18:08
dansmithbauzas: okay but let the top patch finish a run first18:08
bauzasdansmith: okay, so I leave you implement that18:09
melwittjohnthetubaguy: yeah, I get what you mean now. the old way reflects the increased resource usage immediately whereas the new way defers it until the confirm18:09
bauzasdansmith: because I'll be off for 1.5h-ish18:09
*** efoley__ has quit IRC18:09
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver  https://review.openstack.org/39128818:09
bauzasnow the patches are in the gate, I'm bailing out but I'll be back18:09
johnthetubaguymelwitt: yeah, sorry, thats what I mean18:09
bauzasandreykurilin: where are we with the client side ?18:09
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver (disk drv)  https://review.openstack.org/40940118:09
bauzasandreykurilin: oh, it's missing another +2/+W18:10
melwittjohnthetubaguy: I had been thinking it's good to not reflect the usage until it's really actually used. but, that opens the window for things to get past and go over quota when confirm happens later18:10
bauzasandreykurilin: when are you planning to deliver a new revision ?18:10
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver (console)  https://review.openstack.org/40940218:11
*** unicell has joined #openstack-nova18:11
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver (cfg drv)  https://review.openstack.org/40940418:11
bauzasandreykurilin: oh, you're saying all johnthetubaguy's comments can be addressed in a follow-up18:11
andreykurilinmriedem bauzas : let's release novaclient without merging fix for token+cli. It was broken for a year, so it looks like this feature is not in demand, but we need to release new version of novaclient now to release all warning messages which were introduced in 7.0.0 + we need full keystone v3 support18:11
bauzasandreykurilin: the token thing is fixed, that's just the CLI which is not yet fixed, right?18:12
bauzasandreykurilin: unless I'm wrong18:12
andreykurilinbauzas: yes. Token with via python novaclient's API is fixed18:13
andreykurilin*Token auth18:13
bauzasandreykurilin: could you then split your CLI change and just leave the reno file in a separate change ?18:13
melwittjohnthetubaguy: yeah, that would not be great to have to consider pending resize. I wonder if it could be as simple as checking count and quota in the confirm too. and if you are late with your confirm, you may not get to confirm it and have to revert18:13
bauzasandreykurilin: I'd like to include that note for the 7.1.0 release notes18:13
bauzasandreykurilin: and I feel brave enough for a single +2/+W18:14
melwittbut I guess that's not a good experience18:14
andreykurilinbauzas: sorry, I'm not near PC now:(18:14
bauzasandreykurilin: okay, lemme do that then18:14
*** owalsh-afk has quit IRC18:14
andreykurilinbauzas Ok, it looks like I'll able to do it in 10-15 mini18:15
andreykurilin*minutes18:15
openstackgerritDirk Mueller proposed openstack/nova: [WIP] Handle boot_index in block_device_info also for volumes  https://review.openstack.org/42581318:15
johnthetubaguymelwitt: I did wonder about that, it just feels like you could do loads of resizes to use too much quota, and just not bother doing and confirms so you get to keep the extra big instances18:15
bauzasandreykurilin: I'll need to bail out for 1.5 hours18:15
bauzasandreykurilin: so it's up to you, your call18:15
melwittjohnthetubaguy: oh, heh. yeah18:15
johnthetubaguymelwitt: we do have auto resize confirm to protect against that, but I duno, seems funky18:15
andreykurilinbauzas: oh.. we need to propose a release as soon as possible18:15
bauzasandreykurilin: if you can split that, I can +2 and ask another core to quickly +W the reno change only18:15
openstackgerritEd Leafe proposed openstack/nova: placement: RT now adds proper Ironic inventory  https://review.openstack.org/40447218:16
bauzasandreykurilin: I know, but I don't want to ship a release without the proper release note18:16
johnthetubaguymelwitt: I should probably worry that that attack is the first thing that pops into my head!18:16
bauzasand we can still tag the release in the next hours18:16
edleafejaypipes_: ^^ should fix mriedem's concerns18:16
*** ociuhandu has joined #openstack-nova18:16
bauzasthere is no rush, the deadline can be very late PST18:16
andreykurilinbauzas maybe johnthetubaguy can find a bit of time to review my answers to his comments?18:17
melwittjohnthetubaguy: haha :) well, it's good you're thinking of those cases18:17
bauzasask him :)18:17
andreykurilinbauzas g-r will be closed in an hour I think18:17
bauzaswhat I want is at least the reno file18:17
johnthetubaguyandreykurilin: I just don't understand that keychain stuff, and haven't had chance to read into it :(18:17
bauzasokay, too risky then18:17
bauzaslet's just merge the reno file18:17
johnthetubaguyandreykurilin: once you delete that code, I can't see that class being used anywhere18:18
johnthetubaguyso we can always do a stable branch backport for this one18:18
johnthetubaguyfor the CLI users18:18
*** owalsh-afk has joined #openstack-nova18:18
*** owalsh-afk is now known as owalsh18:18
melwittjohnthetubaguy: so on the race prevention thing, you were just thinking of checking the limit twice? once before object create and once after?18:18
johnthetubaguymelwitt: yeah, I mean thats racey, but in a quota goes low sense18:19
melwittI guess you'd have to redo the count too18:19
melwittyeah18:19
*** cdent has joined #openstack-nova18:19
*** lpetrut has quit IRC18:20
johnthetubaguymelwitt: yeah18:20
*** dulek has quit IRC18:20
johnthetubaguymelwitt: its just easier to reason about the correctness18:20
melwittyeah18:21
*** xek_ has joined #openstack-nova18:22
*** pkoniszewski has quit IRC18:22
melwittthat's actually a painfully easy fix for a bug we have open where someone was able to create 100+ more keypairs than the quota18:22
*** xek has quit IRC18:22
melwittwell, I guess there's still a window there so I dunno how much it would lessen it18:23
*** dulek has joined #openstack-nova18:23
*** pkoniszewski has joined #openstack-nova18:24
andreykurilinok, I'm near PC18:24
*** moshele has joined #openstack-nova18:24
johnthetubaguymelwitt: I think it would mean we would delete too many though, I could be wrong18:24
* johnthetubaguy has to run to rehersal18:25
melwittcool, thanks for all the comments on the review18:25
*** unicell has quit IRC18:25
andreykurilin johnthetubaguy SecretHelper is used at L851 - https://review.openstack.org/#/c/424256/10/novaclient/shell.py . It check args and if password is not there it create "prompt"18:26
*** yamahata has joined #openstack-nova18:26
andreykurilinlet's do it in Pike...18:26
bauzasandreykurilin: I need to bail out, but I leave you do the change and ping one of us for single-approving the reno change18:27
openstackgerritSergey Nikitin proposed openstack/nova: Change parent of diagnostics objects to NovaObject class  https://review.openstack.org/39448018:27
openstackgerritSergey Nikitin proposed openstack/nova: Standardization of VM diagnostics info API.  https://review.openstack.org/35554018:27
andreykurilinbauzas: I need a minute18:27
openstackgerritSergey Nikitin proposed openstack/nova: Completed implementation of instance diagnostics for Xen  https://review.openstack.org/39961318:27
bauzasandreykurilin: just tell the other core that I'm +218:27
bauzasandreykurilin: we need the patch to be merged because we can tag the release18:27
bauzasbefore*18:27
bauzasandreykurilin: as I need the SHA1 of the merge commit18:27
snikitinsfinucan, johnthetubaguy: Hi guys! I fixed your nits in diagnoistics. Could you please look? https://review.openstack.org/#/c/394480/18:27
cdentbauzas: i'm around for the next three hours, anything in partivular i should watching?18:28
bauzascdent: I think we addressed most of the concerns18:28
openstackgerritAndrey Kurilin proposed openstack/python-novaclient: Add release not for fixing token auth method  https://review.openstack.org/42582218:28
bauzascdent: the scheduler series is in good path, we're waiting for CI results18:28
andreykurilinbauzas: you can add Depends-On18:28
cdentawesome18:28
andreykurilinbauzas: https://review.openstack.org/#/c/425822/118:29
bauzascdent: dansmith had a good valid point about how we should cache the min version18:29
*** tblakes has quit IRC18:30
* cdent will catch up18:30
*** unicell has joined #openstack-nova18:30
bauzasandreykurilin: fast-approved18:31
andreykurilinbauzas: thanks18:31
bauzasandreykurilin: I can't add a Depends-On as I wouldn't know the SHA118:32
*** rcernin has joined #openstack-nova18:32
bauzasthat's not a change-id that I need to provide18:32
andreykurilinbauzas: why? you can use SHA1 from my commit and do not wait for merge-commit. I think it worked before18:32
bauzasandreykurilin: because I'm not sure we won't get a merge commit18:33
andreykurilinheh18:33
andreykurilinok, np. It looks like requirements team is ok to wait a bit:)18:33
*** amotoki has quit IRC18:33
bauzasif I'm tagging based on your patch SHA1, then it would cut the release based on your local branch, not gerrit's18:34
*** dharinic is now known as dharinic|lunch18:35
bauzasandreykurilin: we now support 2.41 ?18:38
bauzasjust writing release highlights18:38
andreykurilinbauzas: Human beings have dreams. Even dogs have dreams, but not gerrit, gerrit is just a program. An imitation of life. Can a program write a symphony? Can a program turn a... canvas into a beautiful masterpiece?... My local branch is better that gerrit's one. lol18:39
andreykurilinbauzas: yes, 2.41 is latest supported api version18:40
bauzasandreykurilin: okay, so it's confirmed, I can't use your SHA118:40
bauzasandreykurilin: http://git.openstack.org/cgit/openstack/python-novaclient/log/18:40
*** unicell has quit IRC18:40
bauzasandreykurilin: compared to https://review.openstack.org/#/c/425822/1//COMMIT_MSG@118:41
bauzasandreykurilin: your parent patch is not HEAD18:41
andreykurilinbauzas: It can be fixed easily by pushing "Rebase" button via gerrit18:41
bauzasdon't18:42
bauzasleave the patch be merged and I'll pick the merge commit sha118:42
andreykurilinnp :)18:42
*** mvk has joined #openstack-nova18:42
bauzasandreykurilin: FYI https://review.openstack.org/#/c/425827/18:44
bauzasandreykurilin: it's pointing on the current HEAD18:44
*** jpena is now known as jpena|off18:44
bauzasandreykurilin: in case I'm not there, just ping matt with that one and he'll update inline the change18:45
snikitinalex_xu, mriedem: If you are around could you too look at diagnostics patches, please? Today at one moment I had five +2 on my three patches, but had lost all of them because of rebases :( https://review.openstack.org/#/c/394480/18:45
bauzasor if we are too late, just remove the mention of DNM and leave that be cut18:45
andreykurilinbauzas: got it, thanks18:46
* bauzas afk for a bit 18:46
bauzasback in 1 hour-ish18:46
openstackgerritOpenStack Proposal Bot proposed openstack/nova: Updated from global requirements  https://review.openstack.org/42482818:48
*** moshele has quit IRC18:49
*** ijw has joined #openstack-nova18:51
*** bvanhav_ has joined #openstack-nova18:52
*** unicell has joined #openstack-nova18:52
*** tblakes has joined #openstack-nova18:52
dansmithbauzas: several things failing on both patches18:53
*** bvanhav has quit IRC18:53
*** mingyu has joined #openstack-nova18:53
openstackgerritSergey Nikitin proposed openstack/nova: Change parent of diagnostics objects to NovaObject class  https://review.openstack.org/39448018:56
openstackgerritSergey Nikitin proposed openstack/nova: Standardization of VM diagnostics info API.  https://review.openstack.org/35554018:56
openstackgerritSergey Nikitin proposed openstack/nova: Completed implementation of instance diagnostics for Xen  https://review.openstack.org/39961318:56
*** mingyu has quit IRC18:58
*** lpetrut has joined #openstack-nova19:01
*** harlowja has quit IRC19:02
*** jdurgin1 has quit IRC19:03
*** derekh has joined #openstack-nova19:05
openstackgerritSergey Nikitin proposed openstack/nova: Change parent of diagnostics objects to NovaObject class  https://review.openstack.org/39448019:07
openstackgerritSergey Nikitin proposed openstack/nova: Standardization of VM diagnostics info API.  https://review.openstack.org/35554019:07
openstackgerritSergey Nikitin proposed openstack/nova: Completed implementation of instance diagnostics for Xen  https://review.openstack.org/39961319:07
*** alexpilotti has joined #openstack-nova19:07
cburgessIs there an API call to get the contents of the block_device_mapping table for server or volume ID?19:08
cburgessOr is that purely internal?19:08
*** alexpilotti has quit IRC19:08
*** alexpilotti has joined #openstack-nova19:09
*** tblakes has quit IRC19:10
artomcburgess, the deprecated /os-volumes endpoint is probably the closest thing to what you want19:10
artomhttp://developer.openstack.org/api-ref/compute/?expanded=list-volumes-detail19:10
*** avolkov has quit IRC19:10
*** avolkov has joined #openstack-nova19:11
cburgessTried that.. tragically it doesn't seem to work.19:11
cburgessI'll take another look at it.19:11
artomcburgess, define not working...19:12
artomSince it's deprecated, you need to cap your client's compute api version to 2.3619:12
*** andymccr has joined #openstack-nova19:12
cburgessThis AZ happens to be liberty so its capped at 2.1219:12
*** mlavalle has quit IRC19:13
*** alexpilotti has quit IRC19:13
cburgessAs for not working... let me rerun the command.19:14
cburgessAhhh ok it works.19:16
*** tblakes has joined #openstack-nova19:16
cburgessBut it does't expose the connection_info field from the block_device_mapping table.19:16
cburgessAt least not that I can see19:17
cburgessI was previously passing a volume rather then an attachment id that was what was not working.19:17
dansmithbauzas: os_region_name is apparently unset on the grenade job so your computes are all not starting19:17
dansmithwhich makes sense I guess without the other grenade change because we're running with the default config from newton19:18
dansmithso your middle patch needs the dep from the last one19:18
*** unicell has quit IRC19:20
*** unicell has joined #openstack-nova19:20
*** dharinic|lunch is now known as dharinic19:21
*** browne has joined #openstack-nova19:24
openstackgerritDan Smith proposed openstack/nova: Scheduler calling the Placement API  https://review.openstack.org/41796119:25
openstackgerritDan Smith proposed openstack/nova: Block starting compute unless placement conf is provided  https://review.openstack.org/42580619:25
dansmithbauzas: made the caching change, changed the dep to the middle patch, and addressed cdent's comments ^19:25
artomcburgess, well, if you have a use case for it, you can always propose a spec for a new API microversion that does what you want :)19:25
cdentthanks dansmith19:25
* cdent re-reviews19:25
*** karimb has joined #openstack-nova19:26
cburgessartom I might later. RIght now just trying to fix some corruption that got into the DB (probably from some long ago fixed bug). Might make sense in the future but for now I'll just load up MySQLdb to find what I need.19:26
*** derekh has quit IRC19:27
artomcburgess, aha! Well, good luck in your quest ;)19:28
artomActually for that use case, straight up DB access is easier, no? You can pipe it to grep and sed and awk and stuff to find what you're looking for easier19:29
cburgessartom Well I'm writting a python script to do what I need but yeah I suppose it is. I just prefer to avoid having to muck about in the DB if I can avoid it.19:31
*** amotoki has joined #openstack-nova19:34
*** slaweq__ has joined #openstack-nova19:35
*** jheroux has quit IRC19:35
andymccrevenin' nova - on the placement api - is there a way to specify which interface (internal/public/admin) will be used to connect to the placement api? i can't see a conf option for that, and it seems to default to public.19:38
dansmithandymccr: it defaults to however you route to that host19:39
dansmithI mean,19:40
dansmithwe don't bind to an interface, we just connect to that host, so normal routing applies :)19:40
andymccrdansmith: i mean the "keystone endpoint interface" thats picked  - so it looks like its using the publicURL I'm wondering if I can get it to use the internalURL-19:41
dansmithandymccr: I don't know enough about the keystoneclient bits, but .. if you don't set a publicurl maybe?19:43
*** Swami has joined #openstack-nova19:43
*** Sukhdev_ has joined #openstack-nova19:44
dansmithwe load the standard keystone client options from [placement] so if you can do something with that, I'd think it works19:44
dansmithandymccr: i.e. https://github.com/openstack/nova/blob/master/nova/conf/placement.py#L45-L4619:44
*** david-lyle has joined #openstack-nova19:45
andymccrdansmith: ahh good tip - so if i set a keystone option within placement that will set the endpoint type that should work19:45
andymccrthanks a bunch :)19:45
andymccrwill look into that19:45
*** karimb has quit IRC19:45
dansmithandymccr: *should*19:45
andymccrhaha yeah - most things are like that :)19:45
artomDoes Nova officially support Python 3 now?19:46
*** jheroux has joined #openstack-nova19:53
*** mtanino has quit IRC19:55
*** weshay is now known as weshay_brb19:55
openstackgerritDan Smith proposed openstack/nova: Move instance creation to conductor  https://review.openstack.org/31937919:56
*** enki_anunnaki has quit IRC19:56
*** markvoelker has quit IRC20:00
openstackgerritArtom Lifshitz proposed openstack/nova: Fix tag attribute disappearing in 2.33 and 2.37  https://review.openstack.org/42475920:00
openstackgerritArtom Lifshitz proposed openstack/nova: Pass APIVersionRequest to extentions  https://review.openstack.org/42587620:00
*** READ10 has quit IRC20:01
*** Guest33539 is now known as mgagne20:01
*** mgagne has quit IRC20:01
*** mgagne has joined #openstack-nova20:01
openstackgerritArtom Lifshitz proposed openstack/nova: Fix tag attribute disappearing in 2.33 and 2.37  https://review.openstack.org/42475920:05
*** bvanhav_ is now known as bvanhav20:06
*** eharney has quit IRC20:08
openstackgerritArtom Lifshitz proposed openstack/nova: Pass APIVersionRequest to extentions  https://review.openstack.org/42587620:09
*** harlowja has joined #openstack-nova20:09
bauzasdansmith: cdent: back there20:10
bauzasscrolling back20:10
dansmithbauzas: back where?20:10
bauzasdansmith: back here20:10
bauzasje suis là20:10
dansmithbauzas: okay, well, just waiting for the latest run with my changes20:11
openstackgerritVladik Romanovsky proposed openstack/nova: Adding vlans field to Device tagging metadata  https://review.openstack.org/40595120:12
openstackgerritVladik Romanovsky proposed openstack/nova: objects: vlan field to NetworkInterfaceMetadata object  https://review.openstack.org/40595020:12
openstackgerritVladik Romanovsky proposed openstack/nova: libvirt: expose virtual interfaces with vlans to metadata  https://review.openstack.org/40595420:12
bauzasdansmith: oh good point, I missed the grenade Depends-On20:13
openstackgerritChris Dent proposed openstack/nova: DNM: Don't clear the tracked instances in resource tracker  https://review.openstack.org/42588520:13
bauzasdansmith: FWIW, I think matt asked for updating as well the devref20:15
bauzasdansmith: I just provided a reno note, but maybe we should do that in a follow-up ?20:15
dansmithyeah20:15
bauzasnot needing to be merged for o-3 but still be good for RC120:15
cdentdansmith: that review ^^ is checking one of investigations into the redundancies in the resourece_tracker20:16
bauzasdansmith: FWIW, release mgmt people accept for us a few FFE20:16
bauzasthe only problem is that we have RC1 next week20:16
bauzasso in case the gate is bad, we could possibly cut o-3 and then merge the needed changes right after20:17
bauzasoooooh20:19
bauzasthe requirements are frozen now :(20:20
bauzasandreykurilin: ^20:20
*** weshay_brb is now known as weshay20:20
bauzasWTF... http://logs.openstack.org/61/417961/31/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/cd57fce/console.html#_2017-01-26_20_10_22_24933520:22
bauzasdansmith: ^20:22
dansmithbauzas: that would be an expected error if you're not getting resources from the other node right?20:22
bauzashhhhmpppfff http://logs.openstack.org/61/417961/31/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/cd57fce/logs/new/screen-n-sch.txt.gz?level=INFO#_2017-01-26_20_06_58_56320:23
bauzaslemme see20:23
bauzashow it can be possible now we accept older computes20:24
*** eharney has joined #openstack-nova20:24
*** mriedem has joined #openstack-nova20:28
mriedemhow's it going?20:28
dansmithmriedem: fungi bumped the devstack fix in the gate, when my change was #1 and almost done, then mine failed on a known bug at #2, currently re-running20:30
dansmithmriedem: while I wait, I'm making a fungi-shaped voodoo doll20:30
dansmithmriedem: sounds like something is still up with bauzas' junk20:30
dansmithmriedem: and I'm eating a chicken pot pie as comfort food20:30
dansmithend of status20:30
dansmithoh great, that devstack fix is failing and going to reset the gate20:31
mriedemi thought bauzas saw the doctor about his junk issue20:31
dansmiththe cream isn't working apparently20:31
*** avolkov has quit IRC20:31
fungidansmith: https://www.seedsnow.com/blogs/news/7186766-how-to-make-a-mushroom-out-of-radish20:31
mriedemle creme20:31
dansmithfungi: hah20:31
* fungi starts humming super mario bros. theme music20:32
bauzasmriedem: unrelated, requirements is now frozen so https://review.openstack.org/#/c/425827/ is in the weeds for capping u-c20:32
bauzasmriedem: *la* crème20:32
fungidansmith: i think clarkb actually promoted it, but i'll see if i can dig it back up20:33
dansmithfungi: oh, well it's not too late for me to change this into a clarkb doll20:33
dansmithfungi: it needed doing, I'm just annoyed20:33
mriedembauzas: but we can still release novaclient 7.1.0 can't we?20:35
bauzasof course20:35
bauzasjust saying that for u-c20:35
bauzasmriedem: there is a patch in the gate for adding a reno note20:35
mriedemthe wifi in our room sucks so maybe sometime this afternoon i'll be able to load some of these links20:35
bauzasmriedem: that's why I'm putting  WIP20:36
mriedemreno for what?20:36
*** lpetrut has quit IRC20:36
bauzasmriedem: for the token fix20:36
*** lpetrut has joined #openstack-nova20:36
bauzasin the client20:36
bauzaswe agreed to not wait for the CLI20:36
bauzasbut given the CLI included the reno note mentioning the regression bugfix of 7.0.0, I preferred asking andreykurilin to provide a separate change for the reno file and I fast-approved it20:37
bauzasmriedem: https://review.openstack.org/#/c/425822/1 is the patch in question20:38
*** slaweq__ has quit IRC20:38
openstackgerritVladik Romanovsky proposed openstack/nova: Adding vlans field to Device tagging metadata  https://review.openstack.org/40595120:38
*** enki_anunnaki has joined #openstack-nova20:38
*** bvanhav has quit IRC20:39
*** mlavalle has joined #openstack-nova20:42
*** pbandark has joined #openstack-nova20:45
mriedembauzas: so the fix mentioned in https://review.openstack.org/#/c/425822/ is already fixed and merged?20:49
bauzasmriedem: yup, the python binding20:49
bauzasmriedem: not the CLI20:49
mriedemhttps://github.com/openstack/python-novaclient/commit/9940e3fe0e47ff5f2f6d05c9564d17fc19ca0f5c20:50
mriedemok20:50
bauzasmriedem: yup https://review.openstack.org/#/c/419441/20:51
fungidansmith: mriedem: i've reenqueued and promoted 425620,7 now20:53
dansmithfungi: and my change too, right? :D20:53
dansmithno, I know.20:53
* fungi can only get away with so much20:54
dansmithI know20:54
*** mingyu has joined #openstack-nova20:54
dansmithbauzas: looks like your bottom two might be merge-able soon20:55
bauzasdansmith: I'm still puzzled by why grenade-multi is unhappy after our changes20:56
bauzashttp://logs.openstack.org/61/417961/31/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/cd57fce/logs/new/screen-n-sch.txt.gz?#_2017-01-26_20_06_58_560 tells me that node-[...]154 is not present20:57
*** mingyu has quit IRC20:59
*** ijw has quit IRC20:59
*** esmiurium has quit IRC20:59
*** ijw has joined #openstack-nova20:59
cdentbauzas, dansmith: I need to checkout, anything I should chase tomorrow morning?21:00
dansmithcdent: as of now, I think we're just hoping to land the time-sensitive stuff before tomorrow21:00
dansmithcdent: but someone can send a mail to the list if there's anything21:01
cdentdansmith: I can move stuff around stick around now if there's something specific that I could help with?21:01
mriedemi'm looking through vladikr's vlan metadata stuff again21:01
mriedemafter that it's jaypipes' resource class ironic stuff21:01
mriedemcdent: i think we're mostly just watching the gate21:02
mriedemand rechecking21:02
dansmithcdent: unless you staring at the gate will make it go faster than me staring at the gate. otherwise, I'd go21:02
dansmithheh21:02
cdentmy experience is that when I look at the gate it fails, so I'm going to go. but yeah, if there's utc morning stuff to do, leave another of those public handoff emails21:02
cdentgood luck, break a leg or whatever it takes21:03
* cdent waves21:03
* dansmith eyes bauzas' leg21:03
* mriedem cues 1983 metallica21:03
bauzasmeh21:03
*** cdent has quit IRC21:04
bauzasthe old compute looks good21:04
_gryfdo we need a saw21:04
bauzasbut I still don't understand why we're not updating its hostState on the sheduler side21:04
_gryf?21:04
_gryfoh man. wrong channel21:04
bauzasdansmith: the only possibilty I see with the grenade-multi issue is that we don't have the right service version21:09
bauzaschecking whether we are calling placement21:10
mriedem+W on that bottom server groups test patch21:13
*** moshele has joined #openstack-nova21:14
bauzaserf, we don't follow-up passing the req-id to the placement API21:14
bauzaswe regenerate a new one21:14
bauzasso good luck for guessing21:14
mriedembauzas: we want a reno on this right? https://review.openstack.org/#/c/425806/21:15
mriedemheads up - your computes won't start if you upgrade to ocata but don't have placement configured21:15
mriedembauzas: yeah cdent is working on that21:15
bauzasmriedem: I did that in the top patch21:15
mriedembauzas: ok21:15
bauzasmriedem: but I should have splitted the note21:15
mriedemfor shame21:16
bauzasmriedem: that said, if we are unable to merge the top one in time, we can just add the note in a follow-up before RC121:16
bauzasunless you strongly disagree21:16
mriedemi think we're ok21:16
mriedemwe'll have a reno either way before rc121:16
bauzasok21:17
claudiubmriedem: hi. just an fyi, both moshele and sfinucan gave their blessings on the Hyper-V PCI passthrough patch. :D https://review.openstack.org/#/c/420614/21:18
*** annegentle has joined #openstack-nova21:18
mriedemyeah i saw21:18
mriedemit's in yon tab now21:18
*** edmondsw_ has joined #openstack-nova21:18
*** edmondsw_ has quit IRC21:18
mriedembauzas: fyi v21:18
mriedemhttps://review.openstack.org/#/c/414230/21:18
bauzasdansmith: mriedem: okay, I think we're still calling the placement API even if we have an old node21:19
bauzasproof here http://logs.openstack.org/61/417961/31/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/cd57fce/logs/apache/placement-api.txt.gz#_2017-01-26_20_06_58_53121:19
*** moshele has quit IRC21:19
bauzasthe timings are quite the same, vs. http://logs.openstack.org/61/417961/31/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/cd57fce/logs/new/screen-n-sch.txt.gz?#_2017-01-26_20_06_58_55621:19
bauzasdansmith: I think we need to instrument that and add more logs21:20
bauzasdansmith: you okay with me testing that?21:20
dansmithbauzas: do we have any other choice? so... sure :)21:20
*** liverpoo1er has quit IRC21:21
mriedemvladikr: are you sure we're getting an int back in the port details for vlan?21:21
mriedemor could it be a string?21:21
bauzaswhen I think I'm missing the semi-finals of the handball WC because of that...21:22
mriedemWC = bathroom?21:24
mriedemhandball in the men's room?21:24
mriedemkinky21:24
*** Jack_V has quit IRC21:24
mriedembauzas: https://review.openstack.org/#/c/425822/ is merged, i'll update the novaclient 7.1.0 patch while you instrument things21:24
*** unicell has quit IRC21:24
openstackgerritSylvain Bauza proposed openstack/nova: Scheduler calling the Placement API  https://review.openstack.org/41796121:27
*** tjones has joined #openstack-nova21:28
bauzasmriedem: cool21:28
bauzasmriedem: the logging is done, I'm just now waiting the results...21:28
*** Swami has quit IRC21:29
mriedemsi capitan21:29
*** slaweq has quit IRC21:29
bauzasFrance won and goes to final, one good thing at least21:30
*** slaweq has joined #openstack-nova21:30
*** smatzek has quit IRC21:31
dansmithmriedem: I made dash show starred changes in blue, which makes it handy for following stuff through the gate21:32
mriedemcool21:33
*** sacharya has quit IRC21:34
*** Jeffrey4l_ has joined #openstack-nova21:34
*** Jeffrey4l has quit IRC21:35
*** dimtruck is now known as zz_dimtruck21:37
dansmithI have been sitting in the same place since this morning, so I'm going to get up for a bit now that my change is in the gate again, and will be back in a bit21:38
dansmithlooks like bauzas' change is failing pep821:39
bauzasfk21:39
bauzasI edited inline21:39
*** cleong has quit IRC21:39
bauzashttp://logs.openstack.org/61/417961/32/check/gate-nova-pep8-ubuntu-xenial/b7bc39a/console.html#_2017-01-26_21_33_30_78145621:40
bauzaso_O21:40
*** vsaienko has joined #openstack-nova21:40
mriedembauzas: the wifi is so terrible here i can't even git review -d your novaclient release change21:40
mriedemso i guess it's on you to update that21:40
bauzasmriedem: roger21:41
bauzasmriedem: FWIW, will use inline gerrit21:42
*** rmart04 has joined #openstack-nova21:43
*** eandersson__ has quit IRC21:44
*** hfu has joined #openstack-nova21:44
bauzasmriedem: dansmith: do you guys think the pep8 issue would prevent us to see the logs correctly?21:45
dansmitheh?21:45
bauzasI don't give a shit to pep8 since I don't want to merge that like this, but I do care about whether the logs are good21:45
bauzasdansmith: http://logs.openstack.org/61/417961/32/check/gate-nova-pep8-ubuntu-xenial/b7bc39a/console.html#_2017-01-26_21_33_30_78145621:45
dansmithso you want to let it run? that's fine21:45
bauzasyeah21:45
dansmithbiab21:45
* dansmith &21:45
bauzasI'll have to bail out in 1h15 so I prefer leave it running21:46
*** slaweq has quit IRC21:46
*** catintheroof has quit IRC21:47
*** hfu has quit IRC21:47
*** slaweq has joined #openstack-nova21:48
*** tblakes has quit IRC21:51
*** ijw has quit IRC21:52
*** moshele has joined #openstack-nova21:52
*** tbachman has quit IRC21:54
*** Swami has joined #openstack-nova21:54
*** pbandark has quit IRC21:54
*** tblakes has joined #openstack-nova21:56
*** moshele has quit IRC21:57
*** gouthamr has quit IRC22:00
*** breitz has quit IRC22:01
*** breitz has joined #openstack-nova22:01
*** jheroux has quit IRC22:03
*** smatzek has joined #openstack-nova22:04
*** smatzek has quit IRC22:06
*** smatzek has joined #openstack-nova22:06
*** zz_dimtruck is now known as dimtruck22:07
*** Jack_V has joined #openstack-nova22:07
openstackgerritVladik Romanovsky proposed openstack/nova: Adding vlans field to Device tagging metadata  https://review.openstack.org/40595122:07
openstackgerritVladik Romanovsky proposed openstack/nova: libvirt: expose virtual interfaces with vlans to metadata  https://review.openstack.org/40595422:07
mriedemvladikr: -1 on https://review.openstack.org/#/c/405954 because that's not handling that we are getting strings back from neutron for the vlan id in the port detials22:07
mriedem*details22:07
vladikrmriedem, just fixed it22:08
vladikr^^22:08
mriedemah good timing :)22:08
vladikr:)22:09
*** mtanino has joined #openstack-nova22:09
mriedemjroll: do we have anything in CI that sets the node.resource_class field?22:10
edleaferecheck finally worked on https://review.openstack.org/#/c/404472/  \o/22:11
*** Jack_V has quit IRC22:11
*** ijw has joined #openstack-nova22:12
*** unicell has joined #openstack-nova22:13
mriedemedleafe: just wrapping up some comments in there22:13
mriedemedleafe: what sucks is we have nothing in CI to test this new flow http://logs.openstack.org/72/404472/21/check/gate-tempest-dsvm-ironic-ipa-wholedisk-agent_ipmitool-tinyipa-ubuntu-xenial-nv/d12f128/logs/screen-n-cpu.txt.gz#_2017-01-26_18_59_36_21522:13
edleafemriedem: okie dokie22:13
mriedemwe're still just doing the old thing22:14
edleafemriedem: yeah, the extra specs stuff for ironic flavors hasn't been done yet22:14
jrollmriedem: I have some stuff in progress to do that thing22:15
*** thorst_ has quit IRC22:15
bauzasdansmith: around ?22:15
* vladikr heads out for an hour22:16
*** ijw has quit IRC22:17
bauzasdansmith: confirmed that the min service version is 16 http://logs.openstack.org/61/417961/32/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/f07f0e1/logs/new/screen-n-sch.txt.gz#_2017-01-26_22_02_47_73522:18
bauzaswhile it should 1522:18
bauzas;(22:18
bauzasmriedem: ^22:18
*** ijw has joined #openstack-nova22:18
*** ijw has quit IRC22:18
*** ijw has joined #openstack-nova22:19
bauzaspotentially the cache isn't working well22:19
*** tbachman has joined #openstack-nova22:21
*** gouthamr has joined #openstack-nova22:23
*** unicell1 has joined #openstack-nova22:23
*** sacharya has joined #openstack-nova22:23
mriedemedleafe: comments posted22:24
mriedemmostly simple things22:25
mriedemand some missing tests22:25
*** unicell has quit IRC22:26
mriedemdansmith: you should be able to set a str to an IntegerField right?22:26
mriedemor does coerce() just make that work22:27
*** baoli has quit IRC22:28
*** edmondsw has quit IRC22:29
*** lpetrut has quit IRC22:30
openstackgerritEric Brown proposed openstack/nova: Add RPC version aliases for Ocata  https://review.openstack.org/42595822:32
dansmithbauzas: no, it shouldn't be the cache, unless we're down long enough to consider the other node down22:34
dansmithand then start before it checks in22:34
dansmithmriedem: I think it will coerce a string to an int yeah22:34
mriedemblarg22:34
mriedemok lesson learned22:34
bauzasdansmith: any thoughts on why then we have 16?22:35
mriedemvladikr: +2 on your middle change too, even though a couple of the unit tests need tweaks22:35
dansmithbauzas: how long is conductor stopped for?22:36
*** burgerk has quit IRC22:36
*** marst has quit IRC22:36
bauzasdansmith: verifying the tempest test22:37
dansmithservice_down_time defaults to 60s so if it is >60s between conductor going down and scheduler coming back up, then it could be excluding the "down" service22:37
*** vsaienko has quit IRC22:37
edleafemriedem: working on adding the missing tests22:39
bauzasdansmith: the test doesn't assume stopping anything22:39
dansmithbauzas: what test?22:39
bauzasthe tempest test22:39
bauzasoh, you mean, upgrade?22:39
dansmithduring the upgrade yeah22:40
bauzasso we try to boot on the old node22:40
bauzasI'm asking the service version for computes22:40
dansmithI'm not getting what you're saying22:43
bauzasdansmith: I'm trying to understand the inner logic22:43
*** esmiurium has joined #openstack-nova22:43
*** jamesdenton has quit IRC22:43
bauzasI'm not getting why the scheduler could be excluding the old node if we never upgraded the latter22:44
bauzaswrt the service down time you mentioned22:44
*** slaweq has quit IRC22:45
*** slaweq has joined #openstack-nova22:46
*** sacharya has quit IRC22:46
*** owalsh_ has joined #openstack-nova22:46
*** owalsh has quit IRC22:46
dansmithbecause conductor was down, so the subnode couldn't refresh its record22:46
*** thorst_ has joined #openstack-nova22:47
dansmithbauzas: see my change to your grenade patch just now22:49
*** slaweq has quit IRC22:50
*** pradk has quit IRC22:50
bauzasdansmith: ah, I see22:50
bauzasdansmith: we're restarting the scheduler too soon22:51
dansmithright, that's what I'm thinking22:51
bauzaswithout the compute having properly registered back22:51
dansmithright22:51
bauzasgotcha22:51
*** thorst_ has quit IRC22:51
bauzasokay, let's see22:51
dansmithif it's more than 60s then the scheduler's min version calculus will exclude it22:51
bauzasyou gonna recheck ?22:51
bauzasI mean the main change22:51
*** rmart04 has quit IRC22:52
dansmithjesus I have to do everything around here!22:52
bauzas:p22:52
bauzaswell, I can write a gerrit comment :p22:52
*** annegentle has quit IRC22:52
*** slaweq has joined #openstack-nova22:54
*** slaweq has quit IRC22:54
bauzasdansmith: it's pretty late now, and I have to pass my whole tomorrow on train22:54
dansmith...oh22:55
bauzasdansmith: I'll have to handover :(22:55
dansmithhmm, okay22:55
bauzasmriedem: thoughts on me officially asking for a FFE?22:55
*** Swami_ has joined #openstack-nova22:55
*** slaweq has joined #openstack-nova22:55
mriedemgranted22:56
*** mingyu has joined #openstack-nova22:56
*** Swami has quit IRC22:56
*** mdrabe has quit IRC22:57
bauzasdansmith: FWIW, do you think I should make a note in the reno file that operators have to make sure they leave a large enough window for having all the computes registered back or nope?22:57
dansmithbauzas: well, this is already the case for auto rpc mode, but sure22:57
bauzasright, my point22:58
bauzasit's not specific to the scheduler22:58
bauzasbut I just want to make sure it's written somewhere22:58
bauzaseither way, I'll have to bail out, but since I'm gonna pledge for a FFE, I will follow-up tomorrow in the train (with limited internet access, but mriedem proved it works)22:59
*** Swami_ has quit IRC23:00
mriedemheh it is possible23:00
mriedemdespite wanting to gouge your eyes out23:00
*** mingyu has quit IRC23:01
mriedemcrap do we have a meeting right now or was that htis morning?23:02
bauzasI did this morning23:02
bauzasand it should have been 1 hour before if odd week23:02
bauzasa very short one23:02
mriedemok you ran it, cool, thanks23:03
mriedemmy calendar is in my work email which i don't have up23:03
mriedemi'll work on vacation, just not for my employer23:03
bauzas:)23:03
bauzashope the weather is good at least23:03
openstackgerritMatt Riedemann proposed openstack/nova: Adding vlans field to Device tagging metadata  https://review.openstack.org/40595123:05
mriedemcloudy today23:05
mriedemvladikr: ^ updated the reno23:05
mriedemto mention it's libvirt-only right now23:05
mriedemthese are ready to go https://review.openstack.org/#/q/topic:bp/sriov-pf-passthrough-neutron-port-vlan+status:open23:06
mriedem+2s on all23:06
*** dtp has quit IRC23:07
bauzasmriedem: you have enough bandwidth for git rev'ing ocata-3 ?23:07
mriedembauzas: yeah23:07
mriedemthere is always the 'business center'23:07
bauzasor do you want me to do a WIP and you just update it with the gerrit UI?23:08
mriedemi've got it23:08
bauzasokay23:08
bauzasmriedem: dansmith: thanks for helping me on that busy day23:08
mriedemno problema23:09
mriedemer, de nada23:09
* bauzas disappears23:11
mriedem319379 is 38 in the gate?23:11
mriedemcrap23:11
mriedemdansmith: so i guess i'm going to just be tagging o-3 without the cells v2 change, but don't think that's a major issue23:12
*** slaweq has quit IRC23:12
dansmithmriedem: are you serious?23:12
*** slaweq has joined #openstack-nova23:12
mriedemumm i guess not?23:12
mriedemwhat breaks23:12
dansmithmy heart23:13
mriedemha23:13
*** tlian has quit IRC23:13
dansmithand the plan is for FFE?23:13
dansmithI really really wanted it in o323:13
dansmithit's in the gate, you know23:13
mriedemi dont think FFE == o323:13
mriedemat least that's not how i was thinking about it23:13
mriedemthey just land on the same day23:14
*** annegentle has joined #openstack-nova23:14
mriedemlet me ask23:14
dansmitho3 is FF yes? I'm confused23:14
mriedemmeet me in -release23:15
*** smatzek has quit IRC23:16
*** dimtruck is now known as zz_dimtruck23:16
mriedemsome comments in the grenade change https://review.openstack.org/#/c/424730/23:21
*** tlian has joined #openstack-nova23:21
*** tjones has left #openstack-nova23:21
*** tblakes has quit IRC23:21
*** esberglu has quit IRC23:23
sfinucanmriedem: Are we still merging code?23:25
* sfinucan keeps odd, European hours23:25
*** esberglu has joined #openstack-nova23:25
mriedemsfinucan: get these https://review.openstack.org/#/q/topic:bp/sriov-pf-passthrough-neutron-port-vlan+status:open23:26
sfinucanmriedem: That's exactly what I meant :) On it23:26
mriedemdansmith: https://review.openstack.org/#/c/424730/ barfed23:28
dansmithwow sure did23:29
mriedemhttp://logs.openstack.org/30/424730/9/check/gate-grenade-dsvm-neutron-ubuntu-xenial/bd02e4d/logs/grenade.sh.txt.gz#_2017-01-26_23_08_37_01423:29
*** slaweq has quit IRC23:29
mriedem/opt/stack/new/grenade/projects/60_nova/resources.sh: line 145: syntax error near unexpected token `10'23:29
mriedemsleep 1023:29
*** esberglu has quit IRC23:29
*** slaweq has joined #openstack-nova23:29
dansmithlol yeah23:30
dansmithpythash23:30
openstackgerritStephen Finucane proposed openstack/nova: libvirt: fix vCPU usage reporing for LXC/QEMU guests  https://review.openstack.org/39325423:31
*** tlian has quit IRC23:32
*** dtp has joined #openstack-nova23:35
*** slaweq has quit IRC23:36
openstackgerritStephen Finucane proposed openstack/nova: Adopts keystoneauth with glance client.  https://review.openstack.org/41263423:37
openstackgerritTakashi NATSUME proposed openstack/nova: Refactor a test method including 3 test cases  https://review.openstack.org/39473823:39
openstackgerritTakashi NATSUME proposed openstack/nova: Refactor a test method including 7 test cases  https://review.openstack.org/39543623:39
openstackgerritTakashi NATSUME proposed openstack/nova: api-ref: Fix parameters in os-hosts.inc  https://review.openstack.org/41644423:39
openstackgerritTakashi NATSUME proposed openstack/nova: api-ref: Fix deprecated proxy API parameters  https://review.openstack.org/41688223:39
openstackgerritTakashi NATSUME proposed openstack/nova: api-ref: Fix keypair API parameters  https://review.openstack.org/42062623:39
openstackgerritTakashi NATSUME proposed openstack/nova: Avoid forcing translation on logging calls  https://review.openstack.org/41387623:40
openstackgerritTakashi NATSUME proposed openstack/nova: api-ref: Fix path parameters in os-hypervisors.inc  https://review.openstack.org/41691123:40
openstackgerritTakashi NATSUME proposed openstack/python-novaclient: Microversion 2.42 - Enable cold migration with target host  https://review.openstack.org/40670723:40
openstackgerritTakashi NATSUME proposed openstack/nova: Enable cold migration with target host(1/2)  https://review.openstack.org/40895523:40
openstackgerritTakashi NATSUME proposed openstack/nova: Enable cold migration with target host(2/2)  https://review.openstack.org/40896423:40
openstackgerritTakashi NATSUME proposed openstack/nova: api-ref: Fix parameters and examples in aggregate API  https://review.openstack.org/42551223:41
openstackgerritTakashi NATSUME proposed openstack/nova: api-ref: Add parameters in cold migrate action  https://review.openstack.org/41004223:41
openstackgerritTakashi NATSUME proposed openstack/nova: Add functional tests for cold migration to same host  https://review.openstack.org/41492623:41
*** mingyu has joined #openstack-nova23:42
*** Swami has joined #openstack-nova23:43
*** takashin has joined #openstack-nova23:43
mriedemclaudiub: got the pci pass through one approved23:43
*** Swami has quit IRC23:46
*** Swami has joined #openstack-nova23:46
claudiubmriedem: \o/ Time to party! :D23:47
*** annegentle has quit IRC23:50
*** david-lyle has quit IRC23:51
*** jaosorior has quit IRC23:53
*** david-lyle has joined #openstack-nova23:54
*** zz_dimtruck is now known as dimtruck23:54
sfinucanmriedem: Anything else I need to look at before I log off for the night?23:54
mriedemsfinucan: nope23:55
mriedemsfinucan: everything else is in the gate or being watched by me and dan23:55
sfinucanmriedem: Sweet. Cheers :)23:55
mriedemthanks, later23:55
mlavallemriedem: you may want to take a look at this bug: https://bugs.launchpad.net/nova/+bug/165921523:55
openstackLaunchpad bug 1659215 in OpenStack Compute (nova) "Should not allow assign floating IPs that are already assigned to another port" [Undecided,In progress] - Assigned to Zhenyu Zheng (zhengzhenyu)23:55
mriedemmlavalle: bugs are the last thing i want to look at right now23:55
mlavallemriedem: cool, I'll mark it invalid and you can take a look later23:56
mriedemmlavalle: well, i'll forget,23:56
*** rcernin has quit IRC23:56
mriedembut if it's a big deal, or controversial, or whatever, just start a ML thread about it23:56
mlavallemriedem: it's not. i'll take care of it. I just wanted a Nova team memeber to take a quick look. But I understand you are too busy now23:57
*** rcernin has joined #openstack-nova23:57
mlavallemriedem: unless I yell for help, you can forget about it :-)23:58
*** rcernin has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!