| melwitt | dansmith: yeah, that was being maximally careful, not to pollute the db with any root_gb = 0 | 00:00 |
|---|---|---|
| dansmith | yeah, gotta do it, or change a bunch of RT code | 00:00 |
| mriedem | edleafe: what do you mean to get safe_connect? | 00:00 |
| mriedem | that's already in the report client | 00:00 |
| dansmith | it's just -> :/ | 00:00 |
| melwitt | yeah | 00:00 |
| mriedem | edleafe: if you're rebasing on https://review.openstack.org/#/c/404472/ then your fixes are effectively not making ocata | 00:01 |
| edleafe | mriedem: hmmm... for some reason I thought we added it in that patch | 00:01 |
| edleafe | mriedem: let me un-rebase it | 00:01 |
| gmann | mriedem: oh, that huge list is really difficult tasks always :) | 00:02 |
| mriedem | gmann: yeah i compared it one by one with the schema validation code | 00:02 |
| mriedem | and now my eyes are bleeding | 00:02 |
| gmann | mriedem: also we should mention internal attributes also along with joined table | 00:02 |
| gmann | i am going to do "find and search" each one | 00:03 |
| mriedem | gmann: these? https://github.com/openstack/nova/blob/c90acdbc55b743666baa9ef82c56a8e85475f739/nova/api/openstack/compute/schemas/servers.py#L280 | 00:03 |
| mriedem | the joined table ones are a 400 error, ^ are just ignored | 00:03 |
| gmann | no | 00:03 |
| gmann | https://github.com/openstack/nova/blob/c90acdbc55b743666baa9ef82c56a8e85475f739/nova/api/openstack/compute/schemas/servers.py#L365 | 00:03 |
| *** hongbin has quit IRC | 00:04 | |
| *** catintheroof has joined #openstack-nova | 00:04 | |
| gmann | ignore for sort only but we changed 500 -> 400 in filter and same foe _* | 00:04 |
| gmann | for | 00:04 |
| mriedem | gmann: so mention anything with ^_.* in with this list https://review.openstack.org/#/c/428285/2/releasenotes/notes/add-whitelist-for-server-list-filter-sort-params-2ae766d03ba895e5.yaml@103 ? | 00:05 |
| mriedem | gmann: i'm not going to actually go and hunt those down | 00:05 |
| gmann | https://github.com/openstack/nova/blob/c90acdbc55b743666baa9ef82c56a8e85475f739/nova/api/openstack/compute/servers.py#L234 | 00:05 |
| mriedem | i could just say, joined tables or internal data model attributes | 00:05 |
| gmann | yea that is enough | 00:06 |
| mriedem | ok | 00:06 |
| *** hfu has joined #openstack-nova | 00:12 | |
| mriedem | edleafe: i was wondering if that delete RP fix was a blocker but then i remembered again that ironic basically can't schedule if you're all upgraded and have one of those things lying around | 00:13 |
| mriedem | so yeah i guess it's a blocker | 00:13 |
| edleafe | mriedem:yeah, pretty much | 00:14 |
| edleafe | mriedem: running tests locally on the un-rebased change | 00:14 |
| *** nic has quit IRC | 00:16 | |
| *** esberglu has joined #openstack-nova | 00:16 | |
| *** dimtruck is now known as zz_dimtruck | 00:16 | |
| mriedem | ok | 00:16 |
| mriedem | i've gone through and marked everything with rc1, rc2 or defer https://etherpad.openstack.org/p/nova-ocata-rc1-todos | 00:16 |
| mriedem | dansmith: melwitt: jaypipes: ^ | 00:16 |
| jroll | can't schedule? or may schedule to a thing that doesn't exist | 00:16 |
| *** esberglu has quit IRC | 00:17 | |
| mriedem | schedule to a thing that blows up | 00:17 |
| jroll | right | 00:17 |
| jroll | a build failing here and there isn't the worst thing that could happen | 00:17 |
| *** esberglu has joined #openstack-nova | 00:17 | |
| mriedem | i don't know how to talk about scheduling to ironic compute nodes when the nodes are deleted | 00:17 |
| mriedem | or they are hosts, or wtf they are | 00:17 |
| mriedem | it's ironic | 00:17 |
| jroll | and how often to people actually delete machines from ironic | 00:18 |
| jroll | s/to/do/ | 00:18 |
| mriedem | jroll: does it retry in this case? | 00:18 |
| jroll | mriedem: dunno, I didn't reproduce, I'm guessing not | 00:18 |
| jroll | I guess I'm just thinking as long as this gets in rc2, it isn't a big deal if it's in rc1 | 00:18 |
| jroll | I could be wrong ofc | 00:18 |
| mriedem | jroll: i think it will reschedule https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2015 | 00:19 |
| jroll | mriedem: maaaybe, that's a lot of except blocks | 00:20 |
| mriedem | yeah we call spawn here https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L1927 | 00:20 |
| mriedem | ironicclient returns node not found | 00:20 |
| mriedem | which gets handled here https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2008 | 00:20 |
| jroll | huh, I swear it said something about not rescheduling in the bug | 00:21 |
| jroll | I could be drunk | 00:21 |
| mriedem | he said it didn't cleanup in a periodic run | 00:21 |
| mriedem | but i could also be drunk | 00:21 |
| *** ducttape_ has joined #openstack-nova | 00:21 | |
| jroll | yeah I guess not | 00:21 |
| *** esberglu has quit IRC | 00:22 | |
| jroll | I think it's fine | 00:22 |
| *** jose-phillips has quit IRC | 00:22 | |
| openstackgerrit | melanie witt proposed openstack/nova master: WIP Add functional test for boot-from-volume instances https://review.openstack.org/428505 | 00:22 |
| *** raunak has quit IRC | 00:22 | |
| melwitt | dansmith: ^ I put the RT change into there for testing. the instance create works but then fails the resize bc it's requesting disk | 00:23 |
| dansmith | yeah was just looking at it | 00:23 |
| dansmith | I've got a little bit more time and then I'm really out | 00:24 |
| dansmith | mriedem: when do you cut the tag? | 00:24 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Delete a compute node's resource provider when node is deleted https://review.openstack.org/428375 | 00:26 |
| edleafe | mriedem: ^^ un-rebased | 00:26 |
| *** mlavalle has quit IRC | 00:26 | |
| *** sdague has quit IRC | 00:27 | |
| melwitt | dansmith: it doesn't look like mriedem put it on the RC1 things etherpad. maybe we could hope for RC2? since other stuff is slated for it also anyway | 00:28 |
| dansmith | melwitt: I wasn't asking for this, I assume this isn't going to make it | 00:28 |
| edleafe | jroll: and while this was noticed by ironic, it aaplies to any compute node | 00:28 |
| melwitt | dansmith: oh, okay. yeah | 00:28 |
| jroll | edleafe: damnit I have draft comments | 00:28 |
| * jroll moves them | 00:28 | |
| dansmith | melwitt: until your claim thing, I was going to say it looked pretty backportable, but.. | 00:28 |
| *** raunak has joined #openstack-nova | 00:29 | |
| edleafe | jroll: gerrit needs a "move to latest PS" button | 00:29 |
| jroll | or a "don't let someone update this patch for a sec" button | 00:29 |
| jroll | :P | 00:29 |
| *** catintheroof has quit IRC | 00:30 | |
| melwitt | dansmith: yeah. it would still be technically, no db messings up. assuming that legacy request spec doesn't ever get saved anywhere but if it did, that would be a problem anyway | 00:30 |
| edleafe | Ah, I don't mind commenting on a stale patch. I really hate it when I write these deep comments, and then find that someone beat me to it, and mine sound like "me too" | 00:31 |
| dansmith | melwitt: but the scheduler would start sending instances to computes without knowing if they would agree to not count root_gb, if you updated scheduler first | 00:31 |
| dansmith | melwitt: so no architectural blocks, but still probably not free of bumbs | 00:31 |
| dansmith | er, bumps heh :) | 00:31 |
| melwitt | oh, hm | 00:31 |
| jroll | edleafe: yeah, I'm kidding :) | 00:32 |
| dansmith | I'm projecting a little of our OSP backport rules here, but still | 00:32 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add release note for filter/sort whitelist https://review.openstack.org/428285 | 00:32 |
| jroll | edleafe: I'm investigating that big comment to be sure, but have a few there for you | 00:32 |
| edleafe | jroll: heh, but I'm not :) | 00:32 |
| mriedem | laura kicked my ass into eating dinner with her so i'll be back in a bit | 00:32 |
| mriedem | and all questions will be answered | 00:32 |
| *** bkopilov_ has quit IRC | 00:33 | |
| *** Swami has quit IRC | 00:33 | |
| * jroll hopes mriedem doesn't get drugged and taken away for the night | 00:33 | |
| melwitt | dansmith: yeah. it takes me awhile to understand the backport implications to upgrades | 00:33 |
| dims_ | mriedem : better listen to her :) | 00:33 |
| dansmith | melwitt: wish you had brought this up earlier, but the actual patch for this request to the scheduler took like four weeks longer than expected, so :? | 00:34 |
| dansmith | er -> :/ | 00:34 |
| edleafe | jroll: doh! Yeah, the way that the merge conflicts were marked in git didn't help | 00:34 |
| *** sapcc-bot has quit IRC | 00:34 | |
| edleafe | cleaning that up now | 00:35 |
| *** sapcc-bot has joined #openstack-nova | 00:35 | |
| jroll | edleafe: yeah no worries | 00:35 |
| *** baoli has joined #openstack-nova | 00:35 | |
| jroll | the main problem is the get_by_host thing | 00:35 |
| jroll | I think it should likely be get_by_host_and_node | 00:35 |
| melwitt | dansmith: yeah, I should have been paying more attention to that. I didn't know whether the bfv thing was planned to be handled this cycle. and cells v2 madness occurred, yadayada | 00:36 |
| *** Swami has joined #openstack-nova | 00:36 | |
| *** chyka has quit IRC | 00:36 | |
| dansmith | melwitt: yeah | 00:39 |
| edleafe | jroll: the get_by_host_and_node makes sense. I was using what jaypipes suggested, but I think he typed that off the top of his head. | 00:40 |
| *** Swami has quit IRC | 00:40 | |
| jroll | edleafe: the host/node thing is weird when you aren't an ironic person :) | 00:41 |
| edleafe | jroll: and I thought it was always weird | 00:41 |
| dansmith | edleafe: and you would be right | 00:41 |
| jroll | edleafe: true that | 00:42 |
| jroll | I gotta bounce for the night, I'll look again tomorrow | 00:42 |
| edleafe | jroll: thanks | 00:43 |
| *** jaosorior has quit IRC | 00:46 | |
| *** salv-orlando has joined #openstack-nova | 00:49 | |
| *** salv-orlando has quit IRC | 00:54 | |
| *** sapcc-bot has quit IRC | 00:54 | |
| *** sapcc-bot has joined #openstack-nova | 00:54 | |
| *** dave-mccowan has joined #openstack-nova | 00:56 | |
| *** liusheng has joined #openstack-nova | 00:56 | |
| mriedem | dansmith: melwitt: let's put the scheduler BFV patch or whatever at the bottom of the etherpad in the nice to haves list | 00:57 |
| dansmith | I still have 21 failures without getting into unshelve | 00:58 |
| dansmith | and I'm about to drop off for the evening | 00:58 |
| dansmith | but, sure. | 00:58 |
| mriedem | ultimately it's going to depend on if we can backport it | 00:59 |
| *** salv-orlando has joined #openstack-nova | 01:00 | |
| mriedem | gmann: ok i think https://review.openstack.org/#/c/428285/ is good to go | 01:02 |
| mriedem | oomichi: ^ | 01:02 |
| mriedem | http://docs-draft.openstack.org/85/428285/3/check/gate-nova-releasenotes/aa1d391//releasenotes/build/html/unreleased.html#upgrade-notes | 01:03 |
| oomichi | mriedem: terminated_at is valid, not terminate_at :) | 01:04 |
| mriedem | ? | 01:04 |
| mriedem | terminate_at isn't in there | 01:05 |
| mriedem | i already fixed that bug yesterday | 01:05 |
| oomichi | mriedem: yeah, by you. LGTM for the reno | 01:05 |
| mriedem | just sign with your +W | 01:05 |
| *** salv-orlando has quit IRC | 01:06 | |
| oomichi | yep, done | 01:07 |
| *** jamesdenton has joined #openstack-nova | 01:07 | |
| mriedem | great, thanks | 01:07 |
| *** ducttape_ has quit IRC | 01:08 | |
| *** jamesdenton has quit IRC | 01:09 | |
| gmann | mriedem: +1, all grep and extra check good now | 01:13 |
| edleafe | jroll: thanks | 01:13 |
| edleafe | doh | 01:13 |
| mriedem | i'm going to tag the highest for ocata microversions in the docs too, for compute and placement | 01:13 |
| * gmann remember 47 filter params and 29 sort :) | 01:15 | |
| gmann | +1 | 01:15 |
| *** sam_nowitzki has joined #openstack-nova | 01:15 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Mark compute/placement REST API max microversions for Ocata https://review.openstack.org/428528 | 01:15 |
| mriedem | oomichi: ^ | 01:16 |
| oomichi | mriedem: so will you -2 patches which increase microversion in this cycle? | 01:17 |
| oomichi | if so, +2 on the patch | 01:17 |
| gmann | after that branch going to be made right | 01:18 |
| melwitt | dansmith, mriedem: what are the 21 failures? I don't see that on the jenkins run | 01:18 |
| mriedem | we won't have any other microversions in ocata | 01:18 |
| gmann | yea | 01:18 |
| oomichi | ok, +2 | 01:18 |
| dansmith | melwitt: 21 failures after I've started making the other functions do the same | 01:19 |
| openstackgerrit | Dan Smith proposed openstack/nova master: WIP Request zero root disk for boot-from-volume instances https://review.openstack.org/428481 | 01:19 |
| openstackgerrit | Dan Smith proposed openstack/nova master: WIP Add functional test for boot-from-volume instances https://review.openstack.org/428505 | 01:19 |
| dansmith | melwitt: that ^ | 01:19 |
| dansmith | and .. I'm out | 01:19 |
| melwitt | oh, gotcya | 01:20 |
| melwitt | that means ... gotcha and cya at the same time | 01:20 |
| *** thorst_ has quit IRC | 01:20 | |
| *** crushil has joined #openstack-nova | 01:20 | |
| *** zz_dimtruck is now known as dimtruck | 01:21 | |
| *** thorst_ has joined #openstack-nova | 01:21 | |
| mriedem | we need more depends-ons https://review.openstack.org/#/c/428531/ | 01:22 |
| *** jaypipes has quit IRC | 01:24 | |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: Resolve TODOs in 'database' https://review.openstack.org/393695 | 01:25 |
| *** thorst_ has quit IRC | 01:25 | |
| *** rdo has quit IRC | 01:26 | |
| *** rdo has joined #openstack-nova | 01:28 | |
| *** owalsh has joined #openstack-nova | 01:28 | |
| *** owalsh-afk has quit IRC | 01:28 | |
| *** raunak has quit IRC | 01:29 | |
| mriedem | melwitt: want to send this in https://review.openstack.org/#/c/428528/ ? | 01:29 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Delete a compute node's resource provider when node is deleted https://review.openstack.org/428375 | 01:30 |
| melwitt | mriedem: k | 01:30 |
| melwitt | +W. gotta run for now | 01:31 |
| edleafe | mriedem: jroll: ^^ fixed jroll's comments | 01:31 |
| *** gcb has joined #openstack-nova | 01:31 | |
| *** Kevin_Zheng has joined #openstack-nova | 01:32 | |
| *** baoli has quit IRC | 01:32 | |
| *** amotoki has joined #openstack-nova | 01:33 | |
| *** hongbin has joined #openstack-nova | 01:35 | |
| *** ducttape_ has joined #openstack-nova | 01:37 | |
| * edleafe heads out for the evening | 01:40 | |
| *** kberger has quit IRC | 01:44 | |
| *** edmondsw has joined #openstack-nova | 01:45 | |
| *** Apoorva has quit IRC | 01:47 | |
| *** baoli has joined #openstack-nova | 01:48 | |
| *** edmondsw has quit IRC | 01:51 | |
| *** zhurong has joined #openstack-nova | 01:52 | |
| *** tbachman has joined #openstack-nova | 01:53 | |
| *** unicell has quit IRC | 01:53 | |
| *** browne has quit IRC | 01:55 | |
| openstackgerrit | Zhenguo Niu proposed openstack/nova master: Fix breaking out of nested loops https://review.openstack.org/424931 | 01:55 |
| *** anteaya has quit IRC | 01:57 | |
| *** thorst_ has joined #openstack-nova | 02:00 | |
| *** thorst_ has quit IRC | 02:00 | |
| *** salv-orlando has joined #openstack-nova | 02:02 | |
| *** dave-mccowan has quit IRC | 02:03 | |
| *** anteaya has joined #openstack-nova | 02:04 | |
| *** ducttape_ has quit IRC | 02:06 | |
| *** salv-orlando has quit IRC | 02:06 | |
| *** gouthamr has joined #openstack-nova | 02:09 | |
| *** gongysh has joined #openstack-nova | 02:11 | |
| *** dimtruck is now known as zz_dimtruck | 02:13 | |
| *** baoli has quit IRC | 02:13 | |
| *** ayogi has joined #openstack-nova | 02:17 | |
| *** baoli has joined #openstack-nova | 02:17 | |
| *** baoli has quit IRC | 02:18 | |
| *** zz_dimtruck is now known as dimtruck | 02:19 | |
| *** hongbin has quit IRC | 02:19 | |
| *** mdnadeem has joined #openstack-nova | 02:20 | |
| *** hongbin has joined #openstack-nova | 02:21 | |
| *** ducttape_ has joined #openstack-nova | 02:24 | |
| *** mtanino has quit IRC | 02:28 | |
| *** hongbin has quit IRC | 02:28 | |
| *** thorst_ has joined #openstack-nova | 02:32 | |
| *** thorst_ has quit IRC | 02:32 | |
| *** unicell has joined #openstack-nova | 02:40 | |
| *** abalutoiu has quit IRC | 02:41 | |
| *** amotoki_ has joined #openstack-nova | 02:44 | |
| *** amotoki has quit IRC | 02:46 | |
| *** brault has quit IRC | 02:49 | |
| *** unicell has quit IRC | 02:53 | |
| *** dave-mccowan has joined #openstack-nova | 02:53 | |
| *** dimtruck is now known as zz_dimtruck | 02:54 | |
| *** larainema has quit IRC | 02:56 | |
| *** amotoki_ has quit IRC | 02:57 | |
| *** mlakat has quit IRC | 02:57 | |
| *** mlakat has joined #openstack-nova | 02:58 | |
| *** baoli has joined #openstack-nova | 02:58 | |
| *** brault has joined #openstack-nova | 03:00 | |
| *** yamahata has quit IRC | 03:02 | |
| *** salv-orlando has joined #openstack-nova | 03:03 | |
| *** timello has quit IRC | 03:04 | |
| *** crushil has quit IRC | 03:05 | |
| *** ducttape_ has quit IRC | 03:07 | |
| *** salv-orlando has quit IRC | 03:07 | |
| *** ijw has quit IRC | 03:14 | |
| *** dave-mccowan has quit IRC | 03:15 | |
| mriedem | edleafe: reviewed your patch, some issues, mostly minor, should be easy to deal with in the morning | 03:18 |
| *** baoli has quit IRC | 03:19 | |
| *** hongbin has joined #openstack-nova | 03:21 | |
| *** diana_clarke has quit IRC | 03:22 | |
| *** kfarr has quit IRC | 03:23 | |
| *** baoli has joined #openstack-nova | 03:23 | |
| *** timello has joined #openstack-nova | 03:24 | |
| *** crushil has joined #openstack-nova | 03:25 | |
| *** tbachman has quit IRC | 03:28 | |
| *** hongbin has quit IRC | 03:31 | |
| *** owalsh has quit IRC | 03:33 | |
| *** owalsh_ has joined #openstack-nova | 03:33 | |
| *** edmondsw has joined #openstack-nova | 03:33 | |
| *** zhurong_ has joined #openstack-nova | 03:36 | |
| *** Kevin_Zheng has quit IRC | 03:37 | |
| *** baoli has quit IRC | 03:37 | |
| *** amotoki has joined #openstack-nova | 03:38 | |
| *** edmondsw has quit IRC | 03:38 | |
| *** zhurong has quit IRC | 03:38 | |
| *** diana_clarke has joined #openstack-nova | 03:39 | |
| *** eharney has quit IRC | 03:40 | |
| *** crushil has quit IRC | 03:41 | |
| *** mdnadeem has quit IRC | 03:42 | |
| *** armax has quit IRC | 03:45 | |
| *** amotoki has quit IRC | 03:45 | |
| *** ducttape_ has joined #openstack-nova | 03:48 | |
| mriedem | EmilienM: in a tripleo experimental run on nova, which job should i be looking at? gate-tripleo-ci-centos-7-ovb-ha-tempest? | 03:48 |
| mriedem | i can't make heads or tails of the failures in gate-tripleo-ci-centos-7-ovb-nonha | 03:48 |
| *** ducttape_ has quit IRC | 03:48 | |
| *** ducttape_ has joined #openstack-nova | 03:48 | |
| mriedem | i guess the 2 hour 13 minute tempest one is the one i want http://logs.openstack.org/60/428360/2/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-ha-tempest/4eff29f/console.html#_2017-02-03_00_42_04_028094 | 03:49 |
| EmilienM | mriedem: no, gate-tripleo-ci-centos-7-nonha-multinode-nv | 03:49 |
| EmilienM | it's the fastest we have | 03:49 |
| mriedem | EmilienM: that doesn't run on nova's experimental queue | 03:49 |
| mriedem | looking at https://review.openstack.org/#/c/428360/ | 03:49 |
| EmilienM | it did, gate-tripleo-ci-centos-7-nonha-multinode | 03:50 |
| EmilienM | the -nv is when the job will move in check pipeline | 03:50 |
| EmilienM | mriedem: do you need anything else? otherwise I'm heading to sleep | 03:52 |
| mriedem | oh i was looking in the group of tripleo jobs | 03:52 |
| mriedem | no i'll poke in the morning i guess | 03:52 |
| mriedem | i'm gonna have a hell of a time figuring out why this job failed | 03:54 |
| EmilienM | mriedem: I can release my morning and debug with you on the tripleo job | 03:54 |
| EmilienM | I've patched our CI tonight to add more debug | 03:54 |
| EmilienM | if you run "check experimental" again, you should have debug = true on nova.conf | 03:55 |
| mriedem | ok | 03:55 |
| mriedem | i also see the heat stack failed on instance not found http://logs.openstack.org/60/428360/2/experimental/gate-tripleo-ci-centos-7-nonha-multinode/6ca334c/console.html#_2017-02-02_20_42_31_044252 | 03:55 |
| mriedem | so that killed it | 03:55 |
| mriedem | anyway, talk to you tomorrow | 03:55 |
| EmilienM | so this is a pingtest | 03:55 |
| EmilienM | which means we weren't able to spawn a server | 03:55 |
| EmilienM | http://logs.openstack.org/60/428360/2/experimental/gate-tripleo-ci-centos-7-nonha-multinode/6ca334c/logs/subnode-2/var/log/nova/nova-api.txt.gz#_2017-02-02_20_42_17_097 | 03:56 |
| EmilienM | recheck the job (and check experimental), so you'll have DEBUG | 03:56 |
| EmilienM | mriedem: good night | 03:56 |
| *** thorst_ has joined #openstack-nova | 03:58 | |
| *** thorst_ has quit IRC | 03:58 | |
| *** hfu has quit IRC | 04:00 | |
| *** owalsh_ is now known as owalsh-afk | 04:01 | |
| *** hfu has joined #openstack-nova | 04:02 | |
| *** armax has joined #openstack-nova | 04:04 | |
| *** ayogi has quit IRC | 04:07 | |
| *** crushil has joined #openstack-nova | 04:07 | |
| *** zhurong_ has quit IRC | 04:11 | |
| *** mdnadeem has joined #openstack-nova | 04:11 | |
| *** nmathew has joined #openstack-nova | 04:12 | |
| *** udesale has joined #openstack-nova | 04:13 | |
| *** esberglu has joined #openstack-nova | 04:17 | |
| *** gouthamr has quit IRC | 04:17 | |
| *** psachin has joined #openstack-nova | 04:20 | |
| *** thorst_ has joined #openstack-nova | 04:21 | |
| *** thorst_ has quit IRC | 04:21 | |
| *** esberglu has quit IRC | 04:21 | |
| *** sree has joined #openstack-nova | 04:24 | |
| mriedem | rc1 is basically ready to go i think https://review.openstack.org/#/c/428531/ | 04:25 |
| *** dikonoor has joined #openstack-nova | 04:32 | |
| *** crushil has quit IRC | 04:33 | |
| *** crushil has joined #openstack-nova | 04:33 | |
| *** tbachman has joined #openstack-nova | 04:35 | |
| *** armax has quit IRC | 04:37 | |
| *** crushil has quit IRC | 04:39 | |
| *** diga has joined #openstack-nova | 04:40 | |
| *** ducttape_ has quit IRC | 04:40 | |
| *** crushil has joined #openstack-nova | 04:41 | |
| *** ducttape_ has joined #openstack-nova | 04:42 | |
| *** yamahata has joined #openstack-nova | 04:43 | |
| *** ijw has joined #openstack-nova | 04:43 | |
| *** ducttape_ has quit IRC | 04:46 | |
| *** jvgrant has quit IRC | 04:48 | |
| *** ayogi has joined #openstack-nova | 04:48 | |
| *** pbandark has joined #openstack-nova | 04:49 | |
| *** nicolasbock has quit IRC | 04:52 | |
| *** jvgrant has joined #openstack-nova | 04:57 | |
| *** sudipto has joined #openstack-nova | 04:58 | |
| *** sudipto_ has joined #openstack-nova | 04:59 | |
| *** salv-orlando has joined #openstack-nova | 05:04 | |
| *** vladikr has quit IRC | 05:07 | |
| *** salv-orlando has quit IRC | 05:09 | |
| *** sudipto_ has quit IRC | 05:13 | |
| *** sudipto has quit IRC | 05:13 | |
| *** crushil has quit IRC | 05:16 | |
| *** crushil has joined #openstack-nova | 05:16 | |
| *** ijw has quit IRC | 05:17 | |
| *** edmondsw has joined #openstack-nova | 05:21 | |
| *** prateek has joined #openstack-nova | 05:25 | |
| *** edmondsw has quit IRC | 05:26 | |
| *** raunak has joined #openstack-nova | 05:26 | |
| *** lpetrut has joined #openstack-nova | 05:32 | |
| *** abalutoiu has joined #openstack-nova | 05:35 | |
| *** Sukhdev has joined #openstack-nova | 05:39 | |
| *** ducttape_ has joined #openstack-nova | 05:43 | |
| *** ratailor has joined #openstack-nova | 05:43 | |
| *** yamahata has quit IRC | 05:43 | |
| *** ducttape_ has quit IRC | 05:47 | |
| *** salv-orlando has joined #openstack-nova | 06:05 | |
| *** gabor_antal_ has quit IRC | 06:10 | |
| *** salv-orlando has quit IRC | 06:10 | |
| *** crushil has quit IRC | 06:13 | |
| *** vsaienko has joined #openstack-nova | 06:15 | |
| *** vsaienko has quit IRC | 06:17 | |
| *** thorst_ has joined #openstack-nova | 06:22 | |
| *** raunak has quit IRC | 06:22 | |
| *** lpetrut has quit IRC | 06:26 | |
| *** armax has joined #openstack-nova | 06:26 | |
| *** thorst_ has quit IRC | 06:27 | |
| *** mjura has joined #openstack-nova | 06:32 | |
| *** Kevin_Zheng has joined #openstack-nova | 06:33 | |
| *** mvk has quit IRC | 06:37 | |
| *** mvk has joined #openstack-nova | 06:39 | |
| *** mptacekx has joined #openstack-nova | 06:40 | |
| *** unicell has joined #openstack-nova | 06:41 | |
| *** Sukhdev has quit IRC | 06:43 | |
| *** zhurong has joined #openstack-nova | 06:45 | |
| *** satyar has joined #openstack-nova | 06:52 | |
| *** longlq_ has joined #openstack-nova | 06:56 | |
| *** salv-orlando has joined #openstack-nova | 06:57 | |
| *** unicell1 has joined #openstack-nova | 06:57 | |
| *** armax has quit IRC | 06:58 | |
| *** unicell has quit IRC | 06:59 | |
| *** sam_nowitzki has quit IRC | 06:59 | |
| *** hieulq has joined #openstack-nova | 07:02 | |
| *** claudiub has joined #openstack-nova | 07:03 | |
| *** claudiub|2 has joined #openstack-nova | 07:06 | |
| *** longlq_ has quit IRC | 07:06 | |
| *** andreas_s has joined #openstack-nova | 07:06 | |
| *** sam_nowitzki has joined #openstack-nova | 07:06 | |
| *** tbachman has quit IRC | 07:06 | |
| *** Oku_OS-away is now known as Oku_OS | 07:07 | |
| *** claudiub has quit IRC | 07:08 | |
| *** brault has quit IRC | 07:08 | |
| *** edmondsw has joined #openstack-nova | 07:10 | |
| *** edmondsw has quit IRC | 07:14 | |
| *** brault has joined #openstack-nova | 07:15 | |
| *** sridharg has joined #openstack-nova | 07:16 | |
| openstackgerrit | Jiri Suchomel proposed openstack/nova master: Retry few times when creating the initial connection to libvirtd. https://review.openstack.org/416928 | 07:17 |
| *** tesseract has joined #openstack-nova | 07:20 | |
| *** nkrinner_afk is now known as nkrinner | 07:23 | |
| *** rcernin has joined #openstack-nova | 07:26 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Should raise correct error instead of class exist in Placement API https://review.openstack.org/428612 | 07:28 |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Check destination_type when booting with bdm provided https://review.openstack.org/402372 | 07:31 |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Don't delete neutron port when attach failed https://review.openstack.org/403568 | 07:31 |
| *** kaisers2 has quit IRC | 07:34 | |
| *** kaisers1 has quit IRC | 07:34 | |
| openstackgerrit | gecong proposed openstack/os-vif master: Delete H803 from ignore list https://review.openstack.org/428614 | 07:36 |
| openstackgerrit | gecong proposed openstack/os-vif master: Delete H803 from ignore list https://review.openstack.org/428614 | 07:38 |
| *** ratailor is now known as ratailor|Lunch | 07:38 | |
| *** kaisers has joined #openstack-nova | 07:38 | |
| *** ducttape_ has joined #openstack-nova | 07:43 | |
| *** lpetrut has joined #openstack-nova | 07:45 | |
| *** ducttape_ has quit IRC | 07:48 | |
| *** salv-orl_ has joined #openstack-nova | 07:49 | |
| *** esberglu has joined #openstack-nova | 07:50 | |
| *** vsaienko has joined #openstack-nova | 07:51 | |
| *** salv-orlando has quit IRC | 07:51 | |
| *** hieulq has quit IRC | 07:54 | |
| *** esberglu has quit IRC | 07:54 | |
| *** panda|zZ is now known as panda | 07:57 | |
| *** unicell1 has quit IRC | 07:57 | |
| *** aloga has quit IRC | 07:59 | |
| *** aloga has joined #openstack-nova | 07:59 | |
| *** hieulq has joined #openstack-nova | 08:03 | |
| *** salv-orl_ has quit IRC | 08:04 | |
| *** salv-orlando has joined #openstack-nova | 08:04 | |
| *** avolkov has joined #openstack-nova | 08:04 | |
| *** salv-orlando has quit IRC | 08:08 | |
| *** sam_nowitzki has quit IRC | 08:11 | |
| *** hieulq has quit IRC | 08:13 | |
| *** ralonsoh has joined #openstack-nova | 08:13 | |
| tdurakov | johnthetubaguy: hi, are you around? | 08:16 |
| *** hieulq has joined #openstack-nova | 08:21 | |
| *** thorst_ has joined #openstack-nova | 08:23 | |
| *** amoralej|off is now known as amoralej | 08:26 | |
| *** thorst_ has quit IRC | 08:28 | |
| *** slaweq has joined #openstack-nova | 08:30 | |
| *** FL1SK has quit IRC | 08:32 | |
| *** wlhc has joined #openstack-nova | 08:36 | |
| *** wlhc has quit IRC | 08:37 | |
| openstackgerrit | Hieu LE proposed openstack/nova master: Remove mox from unit/virt/libvirt/test_imagebackend.py (end) https://review.openstack.org/335925 | 08:40 |
| *** hieulq has quit IRC | 08:45 | |
| *** sudipto has joined #openstack-nova | 08:48 | |
| *** sudipto_ has joined #openstack-nova | 08:48 | |
| *** ratailor|Lunch is now known as ratailor | 08:51 | |
| openstackgerrit | Jeremy Liu proposed openstack/nova master: [3/3]Replace six.iteritems() with .items() https://review.openstack.org/415080 | 08:51 |
| *** hieulq has joined #openstack-nova | 08:56 | |
| *** edmondsw has joined #openstack-nova | 08:58 | |
| openstackgerrit | Andrey Volkov proposed openstack/nova master: Update pci device fields from deleted object https://review.openstack.org/426243 | 08:58 |
| openstackgerrit | Silvan Kaiser proposed openstack/nova master: Removes getfattr from Quobyte Nova driver https://review.openstack.org/428646 | 08:59 |
| *** zzzeek has quit IRC | 09:00 | |
| *** zzzeek has joined #openstack-nova | 09:00 | |
| *** edmondsw has quit IRC | 09:02 | |
| *** liusheng has quit IRC | 09:07 | |
| *** liusheng has joined #openstack-nova | 09:07 | |
| *** derekh has joined #openstack-nova | 09:13 | |
| sudipto | bauzas, hi, just a small query - if i have a property like cpuset, how do we represent that in the form of a inventory table data (of the new placement API). My doubt is - cpu set is a list of cpus, however the inventory field says: Total as one of the field attributes... | 09:15 |
| sudipto | So how can i represent a cpuset inventory... | 09:15 |
| openstackgerrit | Sergey Nikitin proposed openstack/nova master: Change parent of diagnostics objects to NovaObject class https://review.openstack.org/394480 | 09:18 |
| openstackgerrit | Sergey Nikitin proposed openstack/nova master: Standardization of VM diagnostics info API. https://review.openstack.org/355540 | 09:18 |
| openstackgerrit | Sergey Nikitin proposed openstack/nova master: Completed implementation of instance diagnostics for Xen https://review.openstack.org/399613 | 09:18 |
| *** CristinaPauna has joined #openstack-nova | 09:21 | |
| *** karimb has joined #openstack-nova | 09:27 | |
| *** udesale__ has joined #openstack-nova | 09:28 | |
| *** udesale has quit IRC | 09:30 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Should raise correct error instead of class exist in Placement API https://review.openstack.org/428612 | 09:35 |
| *** lucas-afk is now known as lucasagomes | 09:38 | |
| *** haplo37 has quit IRC | 09:40 | |
| *** haplo37 has joined #openstack-nova | 09:42 | |
| *** ducttape_ has joined #openstack-nova | 09:45 | |
| *** udesale__ has quit IRC | 09:46 | |
| *** udesale has joined #openstack-nova | 09:46 | |
| *** udesale has quit IRC | 09:46 | |
| *** amotoki has joined #openstack-nova | 09:47 | |
| *** hfu has quit IRC | 09:47 | |
| *** ducttape_ has quit IRC | 09:49 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove redundant bdm serial mangling and saving during swap_volume https://review.openstack.org/427364 | 09:49 |
| *** sree has quit IRC | 09:52 | |
| *** sree has joined #openstack-nova | 09:52 | |
| *** udesale has joined #openstack-nova | 09:52 | |
| *** zhurong has quit IRC | 09:55 | |
| *** erlon has joined #openstack-nova | 09:57 | |
| *** haplo37 has quit IRC | 09:58 | |
| *** satyar has quit IRC | 10:01 | |
| *** amotoki has quit IRC | 10:05 | |
| *** salv-orlando has joined #openstack-nova | 10:06 | |
| *** haplo37 has joined #openstack-nova | 10:07 | |
| bauzas | sudipto: sorry, missed your question | 10:10 |
| sudipto | bauzas, np | 10:11 |
| bauzas | sudipto: for the moment, we only have quantitave resources represented in the Placement PAI | 10:11 |
| *** salv-orlando has quit IRC | 10:11 | |
| bauzas | sudipto: which are CPU usage, RAM and disk usage | 10:11 |
| sudipto | bauzas, hmm ok. | 10:11 |
| bauzas | sudipto: when saying cpu_set, which particular compute resource are you talking about ? | 10:12 |
| sudipto | bauzas, https://github.com/openstack/nova/blob/master/nova/objects/numa.py#L47 this one... | 10:13 |
| bauzas | sudipto: on the cpu pinning details | 10:13 |
| bauzas | sudipto: I mean the NUMA things | 10:13 |
| *** rfolco has joined #openstack-nova | 10:13 | |
| bauzas | sudipto: so, yeah, this one is targeted to be implemented thanks to a nested-resource-provided BP | 10:13 |
| sudipto | bauzas, yeah i read through this: https://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/nested-resource-providers.html | 10:14 |
| bauzas | basically, we'd say that that compute UUID == RP UUID | 10:14 |
| bauzas | and it would be parent of nested resource providers giving details on NICs or CPUs | 10:14 |
| bauzas | sudipto: exactly, so FWIW it will be a pike thing | 10:14 |
| sudipto | However, i could visualise how it would work for a numa_node's cpus or numa_node's memory (which would be qutitiative) - but if there's a set of cpus for a numa node, how do we represent it was my question... | 10:15 |
| sudipto | since we have defined a "total" field in the inventory table. | 10:15 |
| *** mkrai has joined #openstack-nova | 10:18 | |
| johnthetubaguy | tdurakov: sorry, late getting onto IRC this morning, hows things? | 10:18 |
| *** gongysh has quit IRC | 10:19 | |
| johnthetubaguy | bauzas: good point on updating the allocations, that change just fixes the bleeding | 10:20 |
| bauzas | sudipto: FWIW, I don't think cpu sets are quantitative amounts | 10:20 |
| johnthetubaguy | bauzas: I have a feeling if you restart nova-compute, we end up writing out updated allocations, but I am not 100% sure on that, the code scares me lots | 10:20 |
| bauzas | johnthetubaguy: so I asked yesterday | 10:20 |
| bauzas | johnthetubaguy: and I've been told that periodic updates rewrite the whole allocations | 10:20 |
| bauzas | so we should be good | 10:21 |
| *** carthaca_ has joined #openstack-nova | 10:21 | |
| *** sree has quit IRC | 10:21 | |
| johnthetubaguy | bauzas: hmm, honestly, that sounds terrible, but it fixes the bug | 10:21 |
| bauzas | but I can look at code | 10:21 |
| johnthetubaguy | bauzas: I want to do that for the ironic change anyways, I don't get how that works right now | 10:21 |
| sudipto | bauzas, but i am wondering that has to be accounted for in case of pinning cpusets | 10:22 |
| *** thorst_ has joined #openstack-nova | 10:24 | |
| bauzas | sudipto: I think a CPU set a list of differnet RPs with an inventory of a VCPU resource class with amount of 1 | 10:24 |
| bauzas | sudipto: and all those RPs are nested children of a parent RP which is compute node | 10:24 |
| sudipto | bauzas, ok so basically we will have to create RPs for each individual cpuset then? | 10:25 |
| *** vsaienko has quit IRC | 10:25 | |
| bauzas | sudipto: that's what I'd think but I'm not the mastermind :) | 10:25 |
| sudipto | and then map that to a parent RP of numa_node and root RP of the compute node | 10:25 |
| sudipto | ok, that sounds like a lot of queries to me. | 10:25 |
| johnthetubaguy | I think the resource providers end up modelling the NUMA node topology, as nested resource providers | 10:25 |
| bauzas | johnthetubaguy: that's what I said to sudipto | 10:25 |
| bauzas | johnthetubaguy: the related question was rather how we model that in a nested RP world | 10:26 |
| sudipto | yeah | 10:26 |
| johnthetubaguy | did that go in the spec, i remember it being nasty | 10:26 |
| bauzas | but like I said, I just think each cpu pin is a RP with an inventory of 1 | 10:26 |
| sudipto | johnthetubaguy, didn't read that in the spec. I may have missed if it was there. | 10:26 |
| bauzas | a NUMA cell would be a parent RP | 10:27 |
| bauzas | of that CPU pin | 10:27 |
| johnthetubaguy | The NUMA node child providers will have inventory records populated for the NUMA_CORE, NUMA_THREAD and NUMA_MEMORY_MB resource classes. | 10:27 |
| bauzas | and a compute RP would be the parent of all NUMA cell RPs | 10:27 |
| *** sambetts|afk is now known as sambetts | 10:27 | |
| johnthetubaguy | http://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/nested-resource-providers.html#proposed-change | 10:27 |
| bauzas | johnthetubaguy: right, but that doesn't mention the CPU pins | 10:27 |
| sudipto | yeah | 10:27 |
| johnthetubaguy | ah, gotcha | 10:27 |
| johnthetubaguy | thats a qualitative problem, which I guess is what bauzas said earlier | 10:28 |
| bauzas | johnthetubaguy: not really | 10:28 |
| bauzas | johnthetubaguy: you need to *consume* a pin | 10:28 |
| sudipto | yeah consume the pin quantitatively. | 10:28 |
| bauzas | johnthetubaguy: hence my thinking of having a RP being the CPU pin itself | 10:28 |
| bauzas | with an inventory of VCPU with amount of 1 | 10:29 |
| *** thorst_ has quit IRC | 10:29 | |
| johnthetubaguy | so there are a few sorts of pins right now, which one is this? | 10:29 |
| bauzas | and pinning the CPU would mean writing an allocation against that inventory | 10:29 |
| johnthetubaguy | really you care about getting a dedicated pCPU on the "correct" numa node, I thought? | 10:29 |
| johnthetubaguy | or is this something else | 10:29 |
| bauzas | johnthetubaguy: I'm considering the CPU-pinning feature we implemented in Nova | 10:30 |
| bauzas | there are many ways to schedule the pinning | 10:30 |
| bauzas | but AFAICT, we fully pin | 10:30 |
| *** udesale__ has joined #openstack-nova | 10:30 | |
| sudipto | we could choose to pin it to a NUMA node or choose to just pin a cpuset (older nova behaviour) | 10:30 |
| bauzas | whatever | 10:31 |
| sudipto | My problem is with the representation of the inventory data for cpusets itself with the current inventory field called 'total' | 10:31 |
| *** amotoki has joined #openstack-nova | 10:31 | |
| bauzas | and ? | 10:31 |
| johnthetubaguy | so the current inventroy doesn't work right, hence the need for nested, and the new resources of NUMA_CORE, etc | 10:31 |
| bauzas | if you consider a single pCPU being a RP | 10:32 |
| sudipto | so we start considering every single pCPU as a RP and then do what bauzas suggested. | 10:32 |
| bauzas | with an VCPU inventory attached to it | 10:32 |
| johnthetubaguy | well, no, I think a NUMA_CORE is a colleciton of pCPU, and you consume one of them | 10:32 |
| johnthetubaguy | but I may have this upside down | 10:32 |
| bauzas | johnthetubaguy: you mean a NUMA cell | 10:32 |
| johnthetubaguy | well, no | 10:33 |
| *** udesale__ has quit IRC | 10:33 | |
| bauzas | I'm confused then | 10:33 |
| johnthetubaguy | a NUMA cell, reports as a provider of NUMA_CORE | 10:33 |
| johnthetubaguy | contains lots of pCPUs you can claim | 10:33 |
| johnthetubaguy | I think | 10:33 |
| bauzas | righrt | 10:33 |
| bauzas | that's just another indirection | 10:33 |
| johnthetubaguy | the pCPUs are just the NUMA_CORE resources | 10:33 |
| johnthetubaguy | I think | 10:33 |
| bauzas | I just feel for CPU pinning that we need to have an inventory of 1 | 10:33 |
| *** udesale has quit IRC | 10:33 | |
| sudipto | ok | 10:34 |
| bauzas | so you "take" that pCPU | 10:34 |
| johnthetubaguy | you don't care about the particular pCPU though | 10:34 |
| johnthetubaguy | you only care about the NUMA node its on relative to the others you have | 10:34 |
| bauzas | that's a scheduling decision | 10:34 |
| bauzas | the fact is | 10:34 |
| johnthetubaguy | doesn't seem to need the ironic 1:1 treatment | 10:34 |
| bauzas | the current scheduler doesn't choose the ping | 10:34 |
| bauzas | pin | 10:34 |
| bauzas | it leaves the compute picking one pin | 10:35 |
| johnthetubaguy | sure, the new one can pick the NUMA node, doesn't have to pick the pCPU | 10:35 |
| bauzas | it just verifies that the the cell has at least one pin left | 10:35 |
| bauzas | but | 10:35 |
| bauzas | we have a shit ton of corner cases in the current model | 10:35 |
| bauzas | and libvirt is pretty laxist on that | 10:35 |
| *** owalsh-afk is now known as owalsh | 10:35 | |
| bauzas | I just discovered that libvirt is totally okay having two VMs asking the same full excluse access to a pCPU | 10:36 |
| bauzas | exclusive* | 10:36 |
| johnthetubaguy | yep, the key thing is what do you care about, you don't care its pin 7, you care its in the same NUMA_NODE as your other resources | 10:36 |
| bauzas | yeah, but there are situations where we fail | 10:36 |
| *** FL1SK has joined #openstack-nova | 10:36 | |
| bauzas | I'm not a NUMA specialist but I swear having seen problems with that | 10:36 |
| johnthetubaguy | we do today, because we don't lock on the NUMA cell, only on the compute node | 10:37 |
| bauzas | mostly because we don't claim that in the scheduler | 10:37 |
| johnthetubaguy | at least thats my understanding of the issue | 10:37 |
| bauzas | right | 10:37 |
| johnthetubaguy | hence the change above | 10:37 |
| lyarwood | johnthetubaguy / bauzas ; apologies if it's too late to ask but would https://review.openstack.org/#/c/420146/ be something we can squeeze in ahead of the RC1 tag? ./me lost track of it this week with everything else flying around. | 10:37 |
| bauzas | anyway, I think sorting that problem is not trival and could be discussed at PTH | 10:37 |
| bauzas | PTG even | 10:37 |
| johnthetubaguy | the buckets are NUMA_NODEs rather than the computes | 10:38 |
| bauzas | lyarwood: I need to pick my daughter | 10:38 |
| johnthetubaguy | lyarwood: lets take a look, there is always RC2 | 10:38 |
| bauzas | oh I thought RC1 sailed already | 10:39 |
| bauzas | but it's not AFAICT | 10:39 |
| bauzas | lemme verify the releases repo | 10:39 |
| *** hieulq has quit IRC | 10:39 | |
| lyarwood | I thought it was pushed to Monday, maybe I'm mistaken, but it can wait | 10:39 |
| johnthetubaguy | yeah, I don't see the tag right now | 10:39 |
| bauzas | yeah https://review.openstack.org/#/c/428531/ | 10:39 |
| *** amotoki has quit IRC | 10:40 | |
| johnthetubaguy | oh, this is based on the we don't do releases on fridays because it breaks weekends | 10:40 |
| johnthetubaguy | although this one shouldn't impact others | 10:40 |
| bauzas | all the Depends-On landed | 10:41 |
| bauzas | so nothing prevents https://review.openstack.org/#/c/428531/ to be merged except the -W of course | 10:41 |
| johnthetubaguy | lyarwood: I kinda like this use of os.path.ismount, seems sensible, shame out findmnt | 10:41 |
| bauzas | johnthetubaguy: the only problem with tagging late is that we delay the requirements unfreeze | 10:42 |
| bauzas | honestly, I don't think tagging RC1 requires to wait over monday | 10:42 |
| lyarwood | johnthetubaguy: yeah the regression in findmnt is being addressed but IMHO we just don't need to use it | 10:42 |
| bauzas | but I leave mriedem decide | 10:42 |
| bauzas | oh that reminds me, let me drop a courtesy highlight | 10:42 |
| johnthetubaguy | bauzas: we have tagged on friday before, using the timezone of thursday on some pacific island | 10:43 |
| lyarwood | lol | 10:43 |
| bauzas | \o/ | 10:43 |
| bauzas | problem solved. | 10:43 |
| johnthetubaguy | indeed :) | 10:43 |
| bauzas | anyway, I need to bail out, school time | 10:43 |
| bauzas | I could pretend the ISS is still on Thursday given they cross the world every 1.5h | 10:43 |
| *** cdent has joined #openstack-nova | 10:44 | |
| johnthetubaguy | that might give us a few more hours, sweet | 10:44 |
| bauzas | just ask Thomas Pesquet to pull the +W on the patch | 10:44 |
| *** edmondsw has joined #openstack-nova | 10:46 | |
| johnthetubaguy | lyarwood: are we sure our source is always the same format at what is listed in /proc/mounts | 10:46 |
| johnthetubaguy | lyarwood: just wondering about nfs vs nfs3, etc | 10:46 |
| lyarwood | johnthetubaguy: hmm I'm pretty sure they are the same but let me double check | 10:48 |
| johnthetubaguy | I guess if it works for findmnt it kinda has to be, but just wanted to double check | 10:49 |
| *** edmondsw has quit IRC | 10:50 | |
| cdent | anything special I should poke at, or let the etherpad be my guide? | 10:55 |
| johnthetubaguy | I think the etherpad is still king | 10:55 |
| johnthetubaguy | at this point, seems like its looking for things to backport to RC2 | 10:56 |
| cdent | i'll get the fixes for https://review.openstack.org/#/c/428375/ | 11:02 |
| *** tbachman has joined #openstack-nova | 11:05 | |
| tdurakov | johnthetubaguy: got question on that one and xen-api(https://review.openstack.org/#/c/424428/) live-migration process - what is the common process upgrading xen server in openstack( 6.5 > 7.0) | 11:06 |
| tdurakov | any recommendations on xenapi multinode deployment? | 11:06 |
| johnthetubaguy | tdurakov: the way we used to do it was live migrate everything off the box, re-image box. I am sure you could do in place upgrade, but all the VMs will be turned off for the duration | 11:07 |
| *** salv-orlando has joined #openstack-nova | 11:08 | |
| johnthetubaguy | tdurakov: also, never attempt to use the pools feature with Nova, just have several standalone boxes | 11:08 |
| johnthetubaguy | tdurakov: but the expert these days is BobBall, he can help you through the details | 11:08 |
| tdurakov | BobBall: are you around? | 11:09 |
| tdurakov | johnthetubaguy: thanks for details | 11:09 |
| johnthetubaguy | tdurakov: no worries | 11:10 |
| tdurakov | johnthetubaguy: no ci for xen multinode, so it's kind of tough thing to reveiw | 11:10 |
| johnthetubaguy | tdurakov: yeah, not sure if they have an internal one | 11:11 |
| *** salv-orlando has quit IRC | 11:12 | |
| openstackgerrit | Bob Ball proposed openstack/nova master: Fix live migrate with XenServer https://review.openstack.org/424428 | 11:12 |
| BobBall | hi tdurakov :) I am indeed around. | 11:13 |
| *** hfu has joined #openstack-nova | 11:13 | |
| tdurakov | BobBall: hi, want to make some testing of ^ | 11:13 |
| BobBall | And no, no multinode CI. Current testing is done manually. We recently got some more hardware installed in our CI env though with the purpose of enabling multinode for live migrate testing. Hopefully that'll be added in the next few weeks. | 11:13 |
| tdurakov | what's the best way doing multinode xen installation now? | 11:14 |
| BobBall | To test I normally set up two normal devstack environments then connect them together | 11:14 |
| tdurakov | ok | 11:14 |
| *** hfu has quit IRC | 11:16 | |
| lyarwood | johnthetubaguy: sorry took a while to ./stack.sh with the NFS plugin, both v3 and v4 leave the same source URI in /proc/mounts - http://paste.openstack.org/show/597531/ | 11:17 |
| johnthetubaguy | lyarwood: oh cool, thanks for digging | 11:17 |
| lyarwood | johnthetubaguy: np it was worth it tbh | 11:18 |
| *** panda is now known as panda|brb | 11:19 | |
| openstackgerrit | Chris Dent proposed openstack/nova master: Delete a compute node's resource provider when node is deleted https://review.openstack.org/428375 | 11:22 |
| *** gongysh has joined #openstack-nova | 11:24 | |
| *** nicolasbock has joined #openstack-nova | 11:24 | |
| cdent | johnthetubaguy: responded to you about the 404 thing: it's if not 404, log | 11:24 |
| cdent | so everything unusual is logged | 11:25 |
| *** salv-orlando has joined #openstack-nova | 11:25 | |
| johnthetubaguy | cdent: balls, I read that as == | 11:25 |
| cdent | on the bugger scale where do balls fit? | 11:25 |
| * cdent cringes | 11:25 | |
| johnthetubaguy | its hard to know, lol | 11:25 |
| *** tojuvone has quit IRC | 11:26 | |
| *** amotoki has joined #openstack-nova | 11:26 | |
| cdent | I made the exact same mistake where the same idiom was used elsewhere (I think on deleting allocations, but not sure). | 11:26 |
| jroll | morning y'all | 11:26 |
| cdent | johnthetubaguy: i put up a new version of that while you were commenting. I think getting all the previous comments | 11:26 |
| johnthetubaguy | cdent: yeah, just taking a peak | 11:27 |
| jroll | "It seems to be the webob pattern to make a non-2xx response False-y" webob whyyyy | 11:27 |
| cdent | morning jroll, you might want to look at https://review.openstack.org/#/c/428375/ too | 11:27 |
| cdent | that's request not webob doing that | 11:27 |
| jroll | yes, I'm already there :) | 11:27 |
| jroll | ok requests whyyyy | 11:27 |
| jroll | :) | 11:27 |
| cdent | and I would guess they do it because, like, man, requests is http for humans, man | 11:27 |
| cdent | it vaguely makes sense, but it does make for some confusion, and is to me biased | 11:28 |
| *** rmart04 has joined #openstack-nova | 11:28 | |
| cdent | I think response object would only be falsey if the web server couldn't be reached | 11:28 |
| jroll | yeah, it's non-obvious | 11:29 |
| jroll | oh well, learned something new today | 11:29 |
| johnthetubaguy | its better than success being falsey I guess | 11:29 |
| *** hfu has joined #openstack-nova | 11:30 | |
| *** ayogi has quit IRC | 11:30 | |
| * cdent feels a "it's better than a kick in the head" cascade coming on | 11:30 | |
| andymccr | if anybody is about to give https://review.openstack.org/#/c/426163/ a review, that'd be really appreciated :) | 11:32 |
| johnthetubaguy | cdent: should we worry about the compute nodes we already deleted, but have entries for in the placement API, I am thinking probably no? | 11:34 |
| *** ratailor has quit IRC | 11:37 | |
| *** amotoki has quit IRC | 11:37 | |
| cdent | johnthetubaguy: I'm not sure how we'd figure that out in that direction. | 11:38 |
| johnthetubaguy | cdent: good question, I guess we can't list the resource providers by host as such, only node? | 11:39 |
| *** sree has joined #openstack-nova | 11:40 | |
| *** panda|brb is now known as panda | 11:40 | |
| johnthetubaguy | cdent: I do wonder about a nova-manage tool to "re-sync" placement, and check for any odd things, its something that could live in such a tool. Although creating that tool feels a bit too much like giving up. | 11:40 |
| jroll | johnthetubaguy: left a comment, I'm thinking we throw people a release note about that, at least? | 11:44 |
| jroll | patch lgtm otherwise though | 11:44 |
| cdent | johnthetubaguy: yeah the host, node stuff is not something that the placement side knows. it just knows resource providers. it's up to the "nova" side to do that jiggery pokery | 11:46 |
| cdent | jroll: That release note seems potentially very confusing and scary for people and might lead to random deletions against the api? | 11:47 |
| jroll | cdent: yeah, we can add big scary warnings or something | 11:48 |
| *** tbachman has quit IRC | 11:48 | |
| jroll | warning: this only applies if you're doing CD by the day/week | 11:48 |
| jroll | idk maybe it's fine | 11:48 |
| jroll | feels like we should tell people somehow though | 11:49 |
| * cdent looks at johnthetubaguy | 11:49 | |
| *** vsaienko has joined #openstack-nova | 11:50 | |
| *** hfu has quit IRC | 11:53 | |
| *** tojuvone has joined #openstack-nova | 11:56 | |
| *** sdague has joined #openstack-nova | 11:56 | |
| *** Hosam has joined #openstack-nova | 11:56 | |
| *** crushil has joined #openstack-nova | 11:56 | |
| *** Hosam_ has joined #openstack-nova | 11:57 | |
| *** pabardina has quit IRC | 11:57 | |
| *** lucasagomes is now known as lucas-hungry | 11:57 | |
| johnthetubaguy | cdent: jroll: oh, so this is affecting ironic already, I kept thinking it wasn't | 11:57 |
| johnthetubaguy | wires crossed in my head | 11:58 |
| jroll | johnthetubaguy: yeah, since a few days ago, I believe | 11:58 |
| johnthetubaguy | surely since last cycle when we started reporting the inventory? | 11:59 |
| *** mnestratov|2 has quit IRC | 12:00 | |
| johnthetubaguy | or did we skip ironic till really recently? | 12:00 |
| jroll | johnthetubaguy: oh. ohhhhhh. | 12:00 |
| jroll | damn | 12:00 |
| jroll | yeah, we only noticed it when scheduling switched to inventory | 12:00 |
| *** Hosam has quit IRC | 12:01 | |
| *** ayogi has joined #openstack-nova | 12:01 | |
| johnthetubaguy | oh, right, its only impacting since we started caring about whats in there | 12:01 |
| jroll | but I guess the real problem might be much longer eh | 12:01 |
| jroll | I guess we could spin up a newton install and test this easily | 12:01 |
| cdent | johnthetubaguy: are you taking typing lessons from me "I was forgetting this could already an issue in production for folks, we should do that thing." | 12:02 |
| johnthetubaguy | heh | 12:03 |
| cdent | So how should this reno express itself? Do we explain just the issue or the issue and the hack for how to deal with it? Or something else? | 12:03 |
| *** kristian__ has joined #openstack-nova | 12:03 | |
| *** gongysh has quit IRC | 12:03 | |
| *** eharney has joined #openstack-nova | 12:04 | |
| *** ducttape_ has joined #openstack-nova | 12:04 | |
| johnthetubaguy | cdent: its probably worth saying the problem you will see with iroinc and placement if you don't clear out old deleted nodes | 12:04 |
| johnthetubaguy | cdent: its tempting to hack up some tool to help fix that, but we good to stop the bleeding with this patch anyways | 12:05 |
| * johnthetubaguy has a evil thought, is just checking... | 12:05 | |
| cdent | Just so I'm clear on the situation how would that hack operate? How do you know which things on the placement api side to delete? | 12:06 |
| johnthetubaguy | because they are not in the current compute node list? I am not 100% sure actually | 12:06 |
| jroll | doesn't resource_provider.uuid == ironic_node.uuid here? | 12:07 |
| *** sudipto has quit IRC | 12:07 | |
| *** sudipto_ has quit IRC | 12:07 | |
| *** vsaienko has quit IRC | 12:08 | |
| *** vsaienko has joined #openstack-nova | 12:08 | |
| johnthetubaguy | cdent: so compute node is soft_delete, so if they don't trim that table, we could double check from soft deleted compute nodes to double check they are all removed | 12:09 |
| *** dave-mccowan has joined #openstack-nova | 12:10 | |
| cdent | but if they do, then they truly are orphans, unless the rp.name is useful in some fashion? | 12:12 |
| cdent | so the hack is for each get_all_soft_delete_compute_nodes; try_to_delete_rp_with_same_uuid() | 12:14 |
| johnthetubaguy | name is hypervisor_hostname I think | 12:14 |
| johnthetubaguy | https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L630 | 12:14 |
| jroll | and hypervisor_hostname is the ironic node uuid | 12:16 |
| johnthetubaguy | can you get a list of the deleted uuids though? | 12:16 |
| *** bvanhav__ has joined #openstack-nova | 12:16 | |
| *** bvanhav__ is now known as bvanhav | 12:16 | |
| johnthetubaguy | I mean you can get all the current ones and spot the difference | 12:16 |
| johnthetubaguy | maybe thats the best way | 12:16 |
| jroll | yeah that'd be the only way | 12:16 |
| cdent | blechy | 12:17 |
| jroll | for cn in compute_nodes: if ironic.get_node(cn.hypervisor_hostname) gives a 404: deleted_nodes.append(cn.hypervisor_hostname) | 12:17 |
| johnthetubaguy | but we already deleted the compute node | 12:17 |
| jroll | delete from resource_providers where name in $deleted_nodes; | 12:17 |
| jroll | oh right, s/compute_nodes/inventory thing | 12:18 |
| jroll | you get the idea :) | 12:18 |
| johnthetubaguy | ah, gotcha | 12:18 |
| *** esberglu has joined #openstack-nova | 12:18 | |
| cdent | "inventory thing"? | 12:18 |
| cdent | (hasn't had sufficient coffee to get the idea) | 12:18 |
| jroll | heh | 12:18 |
| *** gcb has quit IRC | 12:19 | |
| jroll | for rp in resource_providers: if ironic.get_node(rp.name).status_code == 404: deleted_nodes.append(rp.name) | 12:19 |
| jroll | cdent: ^ | 12:19 |
| * johnthetubaguy is heading out to get some food, back in a bit | 12:19 | |
| johnthetubaguy | yeah, that | 12:19 |
| cdent | isn't that proposing deleting all resource providers that are not ironic nodes? | 12:20 |
| johnthetubaguy | feels like we should add a nova-manage to do that | 12:20 |
| *** catintheroof has joined #openstack-nova | 12:20 | |
| *** catintheroof has quit IRC | 12:20 | |
| jroll | cdent: yes, more sanity checks are needed for deployments that include not-ironic and such | 12:20 |
| johnthetubaguy | cdent: yeha, I guess we have to mix in compute nodes too | 12:20 |
| *** catintheroof has joined #openstack-nova | 12:21 | |
| jroll | if I was going for 100% ready to ship I wouldn't be writing this in irc :) | 12:21 |
| johnthetubaguy | listing all compute nodes, would do it, rather than iroinc I guess | 12:21 |
| cdent | and any other resource providers someone has chosen to create outside the context of the resource tracker | 12:21 |
| cdent | we cannot assume the list of resource providers is only things "we" have created | 12:21 |
| jroll | true | 12:22 |
| johnthetubaguy | well, feels liek we should add some user concepts in, when we can't assume that | 12:22 |
| *** esberglu has quit IRC | 12:22 | |
| jroll | maybe the reno is just "if you get build failures trying to schedule to a non-existent node, check *waves hands* stuff, you might need to delete the RP" | 12:23 |
| cdent | I gotta break away for several minutes, perhaps we can all percolate on this for a small window of time and we'll know something after that? | 12:23 |
| cdent | jroll++ :) | 12:23 |
| johnthetubaguy | yeah, a vague reno is better than no reno or fix getting in | 12:24 |
| cdent | bbs | 12:25 |
| *** panda is now known as panda|lch | 12:26 | |
| BobBall | johnthetubaguy / mriedem: Citrix XenServer CI is back healthy now - can username citrix_xenserver_ci be re-added to the voting group https://review.openstack.org/#/admin/groups/511,members? | 12:28 |
| *** satyar has joined #openstack-nova | 12:31 | |
| *** rcernin has quit IRC | 12:31 | |
| *** diga has quit IRC | 12:32 | |
| *** vladikr has joined #openstack-nova | 12:33 | |
| *** edmondsw has joined #openstack-nova | 12:34 | |
| *** ducttape_ has quit IRC | 12:35 | |
| *** thorst_ has joined #openstack-nova | 12:37 | |
| *** CristinaPauna has quit IRC | 12:38 | |
| *** edmondsw has quit IRC | 12:38 | |
| *** daemontool has joined #openstack-nova | 12:38 | |
| *** edmondsw has joined #openstack-nova | 12:43 | |
| *** gszasz has joined #openstack-nova | 12:43 | |
| *** ayogi has quit IRC | 12:45 | |
| *** CristinaPauna has joined #openstack-nova | 12:47 | |
| *** mkoderer has joined #openstack-nova | 12:49 | |
| *** ducttape_ has joined #openstack-nova | 12:53 | |
| *** jvgrant has quit IRC | 12:58 | |
| *** jvgrant_ has joined #openstack-nova | 12:58 | |
| *** lucas-hungry is now known as lucasagomes | 12:59 | |
| *** khappone_ has quit IRC | 13:00 | |
| *** tbachman has joined #openstack-nova | 13:04 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove redundant bdm serial mangling and saving during swap_volume https://review.openstack.org/427364 | 13:06 |
| *** zhurong has joined #openstack-nova | 13:09 | |
| *** timello has quit IRC | 13:10 | |
| johnthetubaguy | BobBall: OK | 13:18 |
| openstackgerrit | Silvan Kaiser proposed openstack/nova master: Removes getfattr from Quobyte Nova driver https://review.openstack.org/428646 | 13:18 |
| johnthetubaguy | BobBall: is a vote on this one coming really soon? https://review.openstack.org/#/c/424428/ | 13:18 |
| *** vsaienko has quit IRC | 13:19 | |
| BobBall | johnthetubaguy: If by soon you mean 25 minutes ago, then yes | 13:19 |
| BobBall | We commented but didn't having voting rights so no vote :) | 13:19 |
| *** sudipto has joined #openstack-nova | 13:20 | |
| *** sudipto_ has joined #openstack-nova | 13:20 | |
| *** timello has joined #openstack-nova | 13:22 | |
| *** salv-orlando has quit IRC | 13:24 | |
| tdurakov | BobBall: in case of re-roll please add change-log comment to xen migrate data object | 13:25 |
| *** jheroux has joined #openstack-nova | 13:26 | |
| johnthetubaguy | BobBall: heh, thats soon, cool. turns out you have to click refresh | 13:26 |
| *** pbandark has quit IRC | 13:26 | |
| BobBall | tdurakov: Sure will do - what do you mean by re-roll though? | 13:26 |
| tdurakov | BobBall: also, what if I'm migration instances, from 6.5 to 7.0 or vice versa is this field required for both of them? | 13:26 |
| tdurakov | BobBall: resubmit patch | 13:27 |
| tdurakov | in case of ^ | 13:27 |
| *** panda|lch is now known as panda | 13:27 | |
| *** sree has quit IRC | 13:28 | |
| johnthetubaguy | tdurakov: I believe this is Nova version related, not XenServer version related | 13:28 |
| BobBall | tdurakov: Migrating from 7.0 to 6.5 isn't supported by XenServer - XAPI will prevent that explicitly. 6.5 to 7.0 will work as the source doesn't look at or use the vif_map, it's prepared by and used by the target. | 13:28 |
| tdurakov | johnthetubaguy: from the but definiton I understood that xen version affect this as well | 13:28 |
| tdurakov | s/but/bug | 13:28 |
| johnthetubaguy | yeah, I see that detail now | 13:28 |
| johnthetubaguy | oops | 13:28 |
| mriedem | cdent: johnthetubaguy: jroll: vdrok: edleafe: on https://review.openstack.org/#/c/428375/ i think we actually should raise a ResourceProviderInUse or some kind of exception if we delete the compute node but can't delete the resource provider, | 13:29 |
| BobBall | johnthetubaguy: XS 7.0 introduced the requirement of the vif map | 13:29 |
| mriedem | because that means we won't reschedule | 13:29 |
| cdent | mriedem: but what would catch it? it's already getting logged as a warning? | 13:29 |
| *** markvoelker_ has joined #openstack-nova | 13:29 | |
| mriedem | way up in the compute manager | 13:30 |
| johnthetubaguy | mriedem: I thought this was just getting called from some periodic task? | 13:30 |
| mriedem | the thing that caught the NodeNotFound in the bug report | 13:30 |
| mriedem | no | 13:30 |
| mriedem | well, | 13:30 |
| johnthetubaguy | both I guess | 13:30 |
| mriedem | yeah it is | 13:30 |
| mriedem | the periodic task doesn't delete the RP, | 13:30 |
| mriedem | and then the scheduler uses it, | 13:30 |
| *** khappone has joined #openstack-nova | 13:30 | |
| mriedem | which then blows up with NodeNotFound in build_and_run_instances | 13:30 |
| mriedem | which triggers a reschedule to another node | 13:30 |
| mriedem | so, | 13:30 |
| mriedem | i guess raising the exception up doesn't help | 13:30 |
| mriedem | fooy | 13:30 |
| cdent | indeed | 13:31 |
| cdent | this is messy. mriedem did you see all the chatter above about how to word a reno note (I linked on the review if not) | 13:31 |
| mriedem | i didn't read that yet | 13:31 |
| *** bhagyashris has quit IRC | 13:32 | |
| *** crushil has quit IRC | 13:32 | |
| *** markvoelker has quit IRC | 13:32 | |
| mriedem | "btw this optional thing which you might be using might be borking your ironic scheduling but then again if you're not using it so nevermind but just in case you were and hit this thing....beware" | 13:32 |
| mriedem | something like that? ^ | 13:32 |
| *** jaypipes has joined #openstack-nova | 13:33 | |
| cdent | well yeah, exactly, that's pretty much the problem | 13:33 |
| *** hfu has joined #openstack-nova | 13:33 | |
| cdent | I'm drafting something now | 13:33 |
| *** hfu has quit IRC | 13:33 | |
| johnthetubaguy | could we use that NodeNotFound error to trigger a check of that resource provider, thinking about it | 13:34 |
| openstackgerrit | Sergey Nikitin proposed openstack/nova-specs master: PCI NUMA Policies https://review.openstack.org/361140 | 13:34 |
| vdrok | if the compute node was deleted but resource provider was not, the bit of deleting resource provider will not be retried by the periodic task? | 13:34 |
| *** hoonetorg has quit IRC | 13:35 | |
| jroll | vdrok: yeah, that could be a problem, maybe we do it the other way (RP first, CN second), and if RP blows up don't delete the compute node either | 13:36 |
| jroll | hrm | 13:36 |
| cdent | mriedem, johnthetubaguy, vdrok, jroll: I'll hold off on pushing up anything until this discussion resolves in some fashion, otherwise it feels like we're going to churn | 13:37 |
| jroll | johnthetubaguy: oh, we totally could | 13:37 |
| jroll | cdent: maybe etherpad the reno draft for now, then, so we can get that handled? | 13:37 |
| johnthetubaguy | mriedem: I am tempted to merge the current fix with a note, that stops *more* badness when stuff gets deleted, feels like we discuss the extra bits on top? | 13:37 |
| cdent | jroll: except that it sounds like we potentially have a more robust fix in the works, which would change the note quite a bit? | 13:38 |
| mriedem | johnthetubaguy: yeah i'm not looking to get a comprehensive fix in for rc1 | 13:38 |
| *** xyang1 has joined #openstack-nova | 13:38 | |
| cdent | or that, okay | 13:38 |
| jroll | heh | 13:38 |
| mriedem | we can backport something if needed | 13:38 |
| johnthetubaguy | +1 | 13:38 |
| mriedem | honestly i doubt very much that anyone is using placement yet, if even on newton yet | 13:38 |
| johnthetubaguy | I think thats fair, based on the massive bugs we found in the last few days | 13:39 |
| jroll | yeah, so to trigger this, you need newton running with placement, with ironic, and delete at least one ironic node | 13:39 |
| jroll | and it will be triggered upon install of current master | 13:40 |
| jroll | right? | 13:40 |
| mriedem | not just install master, | 13:41 |
| mriedem | but bring all of your computes up to ocata | 13:41 |
| mriedem | so full upgrade of all nova services | 13:41 |
| *** sree has joined #openstack-nova | 13:41 | |
| jroll | I don't think so | 13:42 |
| *** annegentle has joined #openstack-nova | 13:42 | |
| jroll | the bug is in newton if you're running placement | 13:42 |
| *** jpena is now known as jpena|lunch | 13:42 | |
| jroll | because you have inventories for things that don't exist | 13:42 |
| mriedem | jroll: but the scheduler in newton doesn't use placement | 13:42 |
| jroll | oh wait, the scheduler in ocata checks compute version, doesn't it | 13:42 |
| sean-k-mooney | mriedem: if you have got a moment would you have any objection to me creating an experimatal job for nova to run a subset of what is currently run in the intel NFV ci? we will be moving where the NFV ci is hosted in then next 2-4 months and now seems like a good opurtnity to upstream the tests that dont require special hardware. those that do will remain in the nfv ci. | 13:43 |
| jroll | the scheduler in ocata does, though, I just don't know if it waits for computes to be upgraded before using placement | 13:43 |
| *** prateek_ has joined #openstack-nova | 13:43 | |
| cdent | jroll: it does | 13:44 |
| cdent | recent change | 13:44 |
| jroll | ok cool, so builds don't break until you upgrade everything to ocata | 13:44 |
| jroll | thanks cdent | 13:44 |
| cdent | here's a stab at the reno: https://etherpad.openstack.org/p/bug-1661258-reno | 13:46 |
| *** sree has quit IRC | 13:46 | |
| cdent | rather hand wavey | 13:46 |
| *** ducttape_ has quit IRC | 13:46 | |
| cdent | or should it be an issue instead of fixes? | 13:47 |
| jroll | maybe both? | 13:47 |
| mriedem | sean-k-mooney: sure | 13:48 |
| *** hoonetorg has joined #openstack-nova | 13:48 | |
| *** amotoki has joined #openstack-nova | 13:48 | |
| mriedem | cdent: jroll: i'd just have 'issues' as in known issue | 13:48 |
| cdent | is leading to a reschedule the only outcome? I thought it could be more dire | 13:48 |
| *** Hosam_ has quit IRC | 13:49 | |
| mriedem | it triggers a reschedule | 13:50 |
| *** vsaienko has joined #openstack-nova | 13:50 | |
| jroll | hm, is there any way to trigger this for not ironic nodes? | 13:50 |
| mriedem | of which you get 3 and then NoValidHost | 13:50 |
| *** annegentle has quit IRC | 13:50 | |
| mriedem | jroll: you'd have to delete the compute node records manually for non-ironic | 13:50 |
| jroll | mriedem: right, because n-cpu wouldn't be running for a hyp that went away right | 13:51 |
| jroll | s/went away/was thrown in the dumpster/ | 13:51 |
| mriedem | n-cpu can be running and if it is, it will recreate the compute node record in the db | 13:51 |
| mriedem | but the service record might be gone | 13:51 |
| sean-k-mooney | mriedem: ok ill start working on the job. after a while i would hope we can move it to the check pipeline as non voting and then eventually voteing once it is shown to be stable. | 13:51 |
| jroll | mriedem: right-o | 13:52 |
| jroll | okay, I think I'm happy with that reno | 13:53 |
| cdent | jroll, mriedem I we're happy with that content I can smooth it | 13:53 |
| jroll | and I'm good with the fix as-is, for now, given the reno | 13:53 |
| *** crushil has joined #openstack-nova | 13:53 | |
| * cdent takes it | 13:53 | |
| jroll | cdent: ++ | 13:53 |
| sean-k-mooney | mriedem: im also going to work on some jobs for os-vif and a few other repos. i may have an update at the ptg on nfv testing in general. | 13:53 |
| mriedem | sean-k-mooney: splendid | 13:54 |
| mriedem | thanks for working on that | 13:54 |
| *** esberglu has joined #openstack-nova | 13:54 | |
| sean-k-mooney | mriedem: well its been kindof buging me that we dont have a linux bridge job for os-vif there are some other gaps in the unit testing in os-vif and nova that i want to take a look at too to tighten up the testing a little | 13:56 |
| *** kberger has joined #openstack-nova | 13:56 | |
| openstackgerrit | Chris Dent proposed openstack/nova master: Delete a compute node's resource provider when node is deleted https://review.openstack.org/428375 | 13:58 |
| cdent | ^^ adds the reno and changes one bug to partial-bug | 13:58 |
| bauzas | mriedem: FWIW, saw your discussion with dansmith and melwitt yesterday about BFV instances | 13:58 |
| *** esberglu has quit IRC | 13:59 | |
| jroll | cdent: trailing space on the reno, not sure if that breaks some check job | 13:59 |
| bauzas | mriedem: so I saw your comment on gerrit and replied to it by saying that we should still verify the disk resource in placement | 13:59 |
| cdent | dag nab it | 13:59 |
| bauzas | mriedem: unless the disk value is 0 | 13:59 |
| *** crushil has quit IRC | 14:00 | |
| openstackgerrit | Chris Dent proposed openstack/nova master: Delete a compute node's resource provider when node is deleted https://review.openstack.org/428375 | 14:00 |
| bauzas | mriedem: because once we will have Cinder RPs, then we will still take a allocation for the disk size | 14:00 |
| *** annegentle has joined #openstack-nova | 14:00 | |
| *** bvanhav has quit IRC | 14:00 | |
| *** daemontool has left #openstack-nova | 14:01 | |
| jroll | cdent: +1'd. now I'm wondering if we want to catch NodeNotFound and have a better error message, but it's probably fine | 14:01 |
| *** ociuhandu has quit IRC | 14:01 | |
| mriedem | jroll: you could still probably get the NodeNotFound for other reasons | 14:01 |
| mriedem | like some kind of race to delete the node and schedule to it at the same time | 14:01 |
| mriedem | i'd just leave it | 14:02 |
| jroll | mriedem: good point | 14:02 |
| mriedem | bauzas: i don't really have that all in my head, but dansmith has a WIP up for some of this | 14:02 |
| mriedem | https://review.openstack.org/#/c/428481/ | 14:02 |
| *** sree has joined #openstack-nova | 14:04 | |
| *** sridharg has quit IRC | 14:05 | |
| *** edleafe is now known as figleaf | 14:07 | |
| *** READ10 has joined #openstack-nova | 14:07 | |
| figleaf | cdent: thanks for picking up https://review.openstack.org/428375 | 14:07 |
| mriedem | +2 from me | 14:08 |
| cdent | figleaf: you're welcome, seemed like that one had some rush on it. If you're busy today I could also pick up the allocation schema stuff too (on which I left some comments about some wrongness) | 14:08 |
| figleaf | cdent: No, not busy. Just getting started, and the caffeine hasn't yet kicked in | 14:09 |
| figleaf | Gonna look at those next | 14:09 |
| cdent | roger that | 14:09 |
| jroll | figleaf: it's 8am already, why so late | 14:09 |
| figleaf | jroll: had to get the rest of the house out to work/school | 14:10 |
| cdent | I guess that means I should think about writing a "placement update" which I've been avoiding because I have no clue, probably because caffeine | 14:10 |
| figleaf | jroll: gotta be able to focus! | 14:10 |
| jroll | figleaf: that was a joke, to be clear :P | 14:10 |
| figleaf | jroll: so was my response :) | 14:10 |
| jroll | :D | 14:11 |
| * cdent has no sense of humor before of https://twitter.com/TechnicallyRon/status/826929148929179648 | 14:11 | |
| cdent | s/before/because/ | 14:11 |
| * cdent also has no sense of typing | 14:11 | |
| *** crushil has joined #openstack-nova | 14:12 | |
| *** sree has quit IRC | 14:14 | |
| *** gouthamr has joined #openstack-nova | 14:14 | |
| *** eharney has quit IRC | 14:15 | |
| *** sree has joined #openstack-nova | 14:15 | |
| *** mvk has quit IRC | 14:15 | |
| *** baoli has joined #openstack-nova | 14:16 | |
| cdent | mriedem: imma fix that nit | 14:16 |
| *** annegentle has quit IRC | 14:17 | |
| openstackgerrit | Chris Dent proposed openstack/nova master: Delete a compute node's resource provider when node is deleted https://review.openstack.org/428375 | 14:17 |
| *** bvanhav has joined #openstack-nova | 14:19 | |
| *** mdrabe has joined #openstack-nova | 14:19 | |
| *** ducttape_ has joined #openstack-nova | 14:19 | |
| bauzas | cdent: figleaf: open questions in https://review.openstack.org/#/c/428375/9 | 14:20 |
| *** Dinesh_Bhor has quit IRC | 14:20 | |
| figleaf | cdent: regarding https://review.openstack.org/#/c/416752/8/nova/api/openstack/placement/handlers/inventory.py, what should be the value for 'max_unit' in the INVENTORY_DEFAULTS (line 109) ? | 14:20 |
| cdent | figleaf: I would think since we want to set a schema minimum of 1 (since less than that is meaningless or wrong) then the default should be 1 as well? | 14:22 |
| cdent | bauzas: looking | 14:22 |
| *** tlian has joined #openstack-nova | 14:22 | |
| bauzas | mriedem: when are you removing -W for https://review.openstack.org/#/c/428531/ ? | 14:23 |
| *** sree has quit IRC | 14:23 | |
| *** cleong has joined #openstack-nova | 14:24 | |
| mriedem | bauzas: probably after https://review.openstack.org/#/c/428375/ is merged now | 14:24 |
| mriedem | johnthetubaguy: +2 on ^ now | 14:24 |
| johnthetubaguy | looking... | 14:25 |
| figleaf | cdent: didn | 14:25 |
| mriedem | cdent: figleaf: a functional report client test as noted by bauzas for deleting RPs would be good, but i'd do that in a follow up | 14:25 |
| figleaf | doh | 14:25 |
| *** owalsh is now known as owalsh-brb | 14:25 | |
| mriedem | bauzas: i already talked to tonyb about blocking requirements yesterday, it's not a big deal right now | 14:25 |
| figleaf | cdent: didn't seem right to set the default 'max' to 1 | 14:25 |
| mriedem | dims_: was telling everyone that yesterday was a soft date for rc1 and monday would be ok | 14:25 |
| *** mjura has quit IRC | 14:25 | |
| mriedem | bauzas: but i'll get rc1 out today if we get this last fix in | 14:26 |
| cdent | figleaf: it is weird yeah, but is better than 0 | 14:26 |
| * dims_ nods | 14:26 | |
| *** mjura has joined #openstack-nova | 14:26 | |
| figleaf | cdent: sure, but even db.MAX_INT felt better (even though that seems wrong, too) | 14:26 |
| bauzas | mriedem: okay, cool with me about RC1 | 14:26 |
| mriedem | i've got to get my kid on the bus and head into the office | 14:26 |
| bauzas | cdent: mriedem: about https://review.openstack.org/#/c/428375/8/nova/scheduler/client/report.py@772 my question is rather | 14:27 |
| *** mriedem has quit IRC | 14:27 | |
| bauzas | cdent: mriedem: if we're deleting a compute node, are we also deleting the nested instances elsewhere ? | 14:27 |
| cdent | figleaf: hmmm. yeah. a good point. that is probably better than 1 as it results in things being left in a working state if a max is not provided | 14:27 |
| cdent | figleaf: which is what DEFAULTS ought to be going for | 14:28 |
| cdent | figleaf: so yeah, step_size 1, min_unit 1, max_unit MAX_INT seems right for the defaults | 14:28 |
| figleaf | cdent: "Hi, I'd like 1 MEMORY_MB for my VM, please" | 14:29 |
| cdent | it's more that 640k so it's gotta be enough | 14:29 |
| *** owalsh-brb has quit IRC | 14:29 | |
| * figleaf goes to make more coffee | 14:30 | |
| cdent | metoo@aol.com | 14:31 |
| *** esberglu has joined #openstack-nova | 14:31 | |
| *** satyar has quit IRC | 14:31 | |
| *** esberglu has quit IRC | 14:31 | |
| *** esberglu has joined #openstack-nova | 14:32 | |
| *** liverpooler has joined #openstack-nova | 14:32 | |
| *** owalsh-brb has joined #openstack-nova | 14:32 | |
| *** eharney has joined #openstack-nova | 14:32 | |
| *** tbachman has quit IRC | 14:33 | |
| *** esberglu_ has joined #openstack-nova | 14:33 | |
| *** rmart04 has quit IRC | 14:35 | |
| *** tbachman has joined #openstack-nova | 14:35 | |
| bauzas | johnthetubaguy: I had a concern on https://review.openstack.org/#/c/428375/ | 14:35 |
| bauzas | I did +1 because I didn't wanted to pollute and say it was a blocker | 14:36 |
| bauzas | but I'm still wondering if all goes the right way | 14:36 |
| *** esbergl__ has joined #openstack-nova | 14:37 | |
| *** esberglu has quit IRC | 14:37 | |
| johnthetubaguy | bauzas: most of the time the list of instances will be empty, I think | 14:37 |
| johnthetubaguy | bauzas: its hitting the 409 from placement I guess, that would cause an issue | 14:37 |
| bauzas | johnthetubaguy: I need to look at code, because I wonder that | 14:38 |
| *** esberglu has joined #openstack-nova | 14:38 | |
| *** esberglu_ has quit IRC | 14:38 | |
| bauzas | johnthetubaguy: say I'm booting inst1, then deleting computeA, do I get a nova list empty? | 14:38 |
| bauzas | I suppose so | 14:38 |
| bauzas | so I'm wondering where the thing is done | 14:38 |
| *** paw has quit IRC | 14:39 | |
| *** amoralej is now known as amoralej|lunch | 14:40 | |
| *** eharney has quit IRC | 14:40 | |
| *** jpena|lunch is now known as jpena | 14:40 | |
| *** esbergl__ has quit IRC | 14:41 | |
| *** esberglu_ has joined #openstack-nova | 14:41 | |
| *** zz_dimtruck is now known as dimtruck | 14:42 | |
| *** paw has joined #openstack-nova | 14:42 | |
| *** andreas_s has quit IRC | 14:43 | |
| *** burt has joined #openstack-nova | 14:43 | |
| *** esberglu has quit IRC | 14:44 | |
| *** mvk has joined #openstack-nova | 14:44 | |
| openstackgerrit | Chris Dent proposed openstack/nova master: Add more debug logging on RP inventory update failures https://review.openstack.org/414230 | 14:44 |
| openstackgerrit | Chris Dent proposed openstack/nova master: Add placement request id to log when GET or POST rps https://review.openstack.org/427740 | 14:44 |
| openstackgerrit | Chris Dent proposed openstack/nova master: Add placement request id to log when GET aggregates https://review.openstack.org/426874 | 14:44 |
| openstackgerrit | Chris Dent proposed openstack/nova master: Add more debug logging on RP inventory delete failures https://review.openstack.org/426290 | 14:44 |
| *** kaisers has quit IRC | 14:45 | |
| cdent | johnthetubaguy: the bottom of that stack ^ was +W (+2 from you) but failing in the gate a few times, so I've rebased but now it is all bereft of its approval | 14:45 |
| *** tblakes has joined #openstack-nova | 14:46 | |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: Destroy the ReqSpec object when deleting the instance https://review.openstack.org/391060 | 14:46 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: Cover again the ReqSpec record creation testing https://review.openstack.org/391059 | 14:46 |
| *** amotoki has quit IRC | 14:48 | |
| *** amotoki has joined #openstack-nova | 14:48 | |
| *** amotoki has quit IRC | 14:49 | |
| *** ratailor has joined #openstack-nova | 14:51 | |
| *** mtanino has joined #openstack-nova | 14:52 | |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Add check for invalid allocation amounts https://review.openstack.org/416752 | 14:54 |
| figleaf | cdent: ^^ Addressed your concerns | 14:54 |
| *** dikonoor has quit IRC | 14:54 | |
| *** eharney has joined #openstack-nova | 14:54 | |
| *** AAR has joined #openstack-nova | 14:57 | |
| *** AAR has left #openstack-nova | 14:57 | |
| johnthetubaguy | cdent: ah, doh, will take a peak | 14:57 |
| *** mvk has quit IRC | 14:58 | |
| *** AAR has joined #openstack-nova | 14:58 | |
| *** mriedem has joined #openstack-nova | 14:59 | |
| *** AAR has left #openstack-nova | 14:59 | |
| *** sree has joined #openstack-nova | 15:01 | |
| *** mtanino_ has joined #openstack-nova | 15:01 | |
| *** mtanino has quit IRC | 15:02 | |
| *** eharney has quit IRC | 15:02 | |
| *** enki_anunnaki has quit IRC | 15:04 | |
| johnthetubaguy | cdent: I do wonder if we have too many new lines in some of the error messages in the logs, BTW | 15:06 |
| johnthetubaguy | cdent: its a micro nit, it might be more to do with the error message omitted by the placement API and how that looks in the logs | 15:07 |
| mriedem | that's the nature of the requests.response isn't it? | 15:07 |
| johnthetubaguy | it could be | 15:09 |
| *** smatzek has joined #openstack-nova | 15:09 | |
| *** mtanino_ has quit IRC | 15:09 | |
| *** haplo37_ has quit IRC | 15:09 | |
| *** dimtruck is now known as zz_dimtruck | 15:10 | |
| *** gabor_antal has joined #openstack-nova | 15:11 | |
| *** pcaruana has quit IRC | 15:11 | |
| *** mtanino has joined #openstack-nova | 15:11 | |
| *** zz_dimtruck is now known as dimtruck | 15:11 | |
| *** mptacekx has quit IRC | 15:11 | |
| cdent | johnthetubaguy, mriedem: that could be the way that webob is formatting error responses. the json_error_formatter stuff subclasses something which puts newlines in here and there for no apparently reason | 15:12 |
| johnthetubaguy | cdent: I was looking at that with pumaranikar the other day, I think it might be a bit new line happy | 15:12 |
| *** haplo37_ has joined #openstack-nova | 15:12 | |
| cdent | new line happy | 15:13 |
| cdent | yes | 15:13 |
| openstackgerrit | Sarafraj Singh proposed openstack/nova-specs master: Send service token along with user tokens https://review.openstack.org/426933 | 15:13 |
| cdent | i'm not sure I'm keen on the decorator idea, but the current middleware idea won't work | 15:13 |
| cdent | (but I've mostly been ignoring that) | 15:13 |
| *** sree has quit IRC | 15:15 | |
| cdent | johnthetubaguy: in any case, the logging on the rt side shouldn't care. we should just fix it on the api side, yeah? | 15:16 |
| *** sree has joined #openstack-nova | 15:16 | |
| johnthetubaguy | cdent: yeah, certainly if the extra new lines are from the API, which I think they are | 15:16 |
| johnthetubaguy | cdent: it was more your patches reminded my about the niggle I spotted debugging that bug the other day | 15:17 |
| *** pcaruana has joined #openstack-nova | 15:17 | |
| *** mlavalle has joined #openstack-nova | 15:17 | |
| cdent | let's make pumaranikar fix it :) | 15:17 |
| *** eharney has joined #openstack-nova | 15:18 | |
| johnthetubaguy | good idea | 15:18 |
| johnthetubaguy | pumaranikar: we found you another placement bug | 15:18 |
| EmilienM | mriedem: good morning, regarding the issue with Nova in WSGI in TripleO - does it make sense we stop doing that in Ocata (and revisit later)? | 15:19 |
| mriedem | so on the tripleo failures from yesterday, with a run on my debug patch, i see the instance that heat is trying to create fails | 15:19 |
| mriedem | http://logs.openstack.org/60/428360/2/experimental/gate-tripleo-ci-centos-7-nonha-multinode/7a6e5bc/logs/subnode-2/var/log/nova/nova-conductor.txt.gz#_2017-02-03_05_17_35_826 | 15:19 |
| mriedem | EmilienM: good timing | 15:19 |
| EmilienM | wow | 15:19 |
| mriedem | scheduler couldn't find a host | 15:19 |
| EmilienM | do you want me to stop deploing nova api in wsgi with apache and see if it helps? | 15:19 |
| *** amotoki has joined #openstack-nova | 15:20 | |
| mriedem | given we haven't really flushed out testing with that, i think it's prudent at this point in ocata | 15:20 |
| EmilienM | ok let me do it | 15:20 |
| *** sree has quit IRC | 15:20 | |
| mriedem | the 404 isn't because the instance was created and we just can't find it though, it's not found because it fails to build and the instance isn't created (it looks like), but the dead instance should go into cell0 and we should pull it from cell0 | 15:21 |
| mriedem | hrm, dansmith melwitt do we even look in cell0 for compute api get()? | 15:21 |
| *** marst has quit IRC | 15:21 | |
| cdent | johnthetubaguy, pumaranikar It can probably be built right in with the json_formatter stuff because it's all of a piece | 15:21 |
| melwitt | mriedem: we should, I think. do we not? checking | 15:22 |
| dansmith | melwitt: yes | 15:22 |
| dansmith | er, mriedem yes | 15:22 |
| dansmith | mriedem: the mapping tells us what cell to look in | 15:22 |
| melwitt | oh yeah | 15:22 |
| dansmith | which would be cell0 if it's there | 15:22 |
| *** nkrinner is now known as nkrinner_afk | 15:23 | |
| *** prateek has quit IRC | 15:23 | |
| *** prateek_ has quit IRC | 15:23 | |
| mriedem | what's odd is in this debug run i expect to see the 'found x cells' thing from https://review.openstack.org/#/c/428360/2/nova/compute/api.py@2526 | 15:24 |
| mriedem | but it's not in http://logs.openstack.org/60/428360/2/experimental/gate-tripleo-ci-centos-7-nonha-multinode/7a6e5bc/logs/subnode-2/var/log/nova/nova-api.txt.gz | 15:24 |
| kashyap | johnthetubaguy: "But asking for a friendlier looking supernova seems a bit... unscientific" :-) | 15:24 |
| johnthetubaguy | kashyap: :) | 15:24 |
| kashyap | Meanwhile, if there are any Virtualization maintainers of Ubuntu, you may want to triage this - https://bugs.launchpad.net/nova/+bug/1643911 | 15:25 |
| openstack | Launchpad bug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed] | 15:25 |
| kashyap | lease see my last comment there. | 15:25 |
| melwitt | mriedem: I thought we had found there weren't any 'GET .../servers' in those logs | 15:25 |
| kashyap | (s/lease/Please/) | 15:25 |
| mriedem | melwitt: my debug patch also did the cells thing in _get_instance() | 15:26 |
| melwitt | oh, you added logging for _get_instance | 15:26 |
| *** amotoki has quit IRC | 15:26 | |
| mriedem | melwitt: http://logs.openstack.org/60/428360/2/experimental/gate-tripleo-ci-centos-7-nonha-multinode/7a6e5bc/logs/subnode-2/var/log/nova/nova-api.txt.gz#_2017-02-03_05_17_30_873 is the GET | 15:26 |
| mriedem | but it's not hitting the logging i added here https://review.openstack.org/#/c/428360/2/nova/compute/api.py | 15:27 |
| *** owalsh-brb is now known as owalsh | 15:28 | |
| mriedem | so i'm thinking we're trying to pull it from a cell but it's not there, which would be nuts, but i'll add some more logging | 15:28 |
| mriedem | oh well i guess that would make sense if we pull the instance here https://review.openstack.org/#/c/428360/2/nova/compute/api.py@2269 | 15:28 |
| mriedem | because the service version is 0 | 15:29 |
| melwitt | yeah, I was about to say, we had found bc they're running through apache they're getting service version 0, or something | 15:29 |
| mriedem | yup, | 15:31 |
| mriedem | AND | 15:31 |
| mriedem | http://logs.openstack.org/15/428415/1/experimental/gate-tripleo-ci-centos-7-nonha-multinode/4a76477/ | 15:31 |
| mriedem | is dan's workaround patch | 15:32 |
| EmilienM | mriedem: https://review.openstack.org/428778 - once it pass our CI, I'll ask you you Depends-On this one, so you can continue the tests without WSGI in Apache | 15:33 |
| *** amoralej|lunch is now known as amoralej | 15:34 | |
| mdbooth | mriedem: Thanks for looking at that NFS patch, btw. You found 2 bugs in it :) | 15:36 |
| mriedem | mdbooth: but you told me it was rock solid! | 15:36 |
| *** burgerk has joined #openstack-nova | 15:37 | |
| mdbooth | mriedem: I was thinking a good limestone. | 15:37 |
| mdbooth | You know, pretty solid. | 15:37 |
| mdbooth | ;) | 15:37 |
| mriedem | heh | 15:37 |
| *** psachin has quit IRC | 15:37 | |
| mdbooth | Both would result in mounts not being unmounted, which is the opposite of the original problem of mounts being unmounted when they shouldn't be. | 15:38 |
| *** dave-mccowan has quit IRC | 15:40 | |
| *** jaosorior has joined #openstack-nova | 15:40 | |
| *** tbachman has quit IRC | 15:40 | |
| *** ratailor has quit IRC | 15:40 | |
| *** tbachman has joined #openstack-nova | 15:41 | |
| *** marst has joined #openstack-nova | 15:41 | |
| *** vsaienko has quit IRC | 15:42 | |
| *** raunak has joined #openstack-nova | 15:43 | |
| *** salv-orlando has joined #openstack-nova | 15:44 | |
| *** dimtruck is now known as zz_dimtruck | 15:48 | |
| *** dave-mccowan has joined #openstack-nova | 15:53 | |
| *** tbachman has quit IRC | 15:55 | |
| *** raunak has quit IRC | 15:56 | |
| *** annegentle has joined #openstack-nova | 15:59 | |
| *** mlakat has quit IRC | 16:02 | |
| *** crushil has quit IRC | 16:06 | |
| *** crushil has joined #openstack-nova | 16:06 | |
| *** stvnoyes has quit IRC | 16:06 | |
| *** hongbin has joined #openstack-nova | 16:06 | |
| EmilienM | mriedem: FYI https://review.openstack.org/#/q/topic:bug/1661360+owner:%22Emilien+Macchi+%253Cemilien%2540redhat.com%253E%22 | 16:07 |
| *** stvnoyes has joined #openstack-nova | 16:07 | |
| *** mvk has joined #openstack-nova | 16:07 | |
| *** tbachman has joined #openstack-nova | 16:08 | |
| *** BlackDex has quit IRC | 16:09 | |
| mdbooth | mriedem: On second thoughts, only 1 bug :) The other 1 just looks sufficiently weird that I misread it, too. | 16:09 |
| *** zz_dimtruck is now known as dimtruck | 16:09 | |
| *** yamahata has joined #openstack-nova | 16:10 | |
| mriedem | EmilienM: good lord that's a lot of change | 16:11 |
| mriedem | but i don't know puppet or tripleo | 16:11 |
| *** baoli has quit IRC | 16:13 | |
| pumaranikar | johnthetubaguy, cdent sry i missed initial conversation | 16:13 |
| johnthetubaguy | pumaranikar: no worries, was going to ping you | 16:14 |
| johnthetubaguy | pumaranikar: thinking about the errors produced by the placement API, they have lots of big new lines in them | 16:14 |
| johnthetubaguy | pumaranikar: it would be great if you could raise a bug for that and fix it | 16:14 |
| johnthetubaguy | pumaranikar: seems like the json formatter needs a tweak to fix that up | 16:15 |
| *** BlackDex has joined #openstack-nova | 16:16 | |
| *** BlackDex has quit IRC | 16:16 | |
| *** kristian__ has quit IRC | 16:17 | |
| EmilienM | mriedem: it was just an fyi | 16:17 |
| pumaranikar | johnthetubaguy, sure i will check it. | 16:17 |
| *** BlackDex has joined #openstack-nova | 16:17 | |
| mriedem | EmilienM: i made one very constructive comment on one patch | 16:17 |
| johnthetubaguy | pumaranikar: thanks | 16:17 |
| *** Oku_OS is now known as Oku_OS-away | 16:18 | |
| pumaranikar | johnthetubaguy, regarding json_formatter, should we try again with middleware or go with decorated function. what you suggest ? | 16:19 |
| *** baoli has joined #openstack-nova | 16:19 | |
| *** BlackDex has quit IRC | 16:20 | |
| *** armax has joined #openstack-nova | 16:20 | |
| *** dansmith is now known as superdan | 16:21 | |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Add check for invalid allocation amounts https://review.openstack.org/416752 | 16:21 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Add check for invalid inventory amounts https://review.openstack.org/428800 | 16:21 |
| figleaf | cdent: split the inventory and allocation changes ^^ | 16:21 |
| cdent | thanks figleaf | 16:22 |
| *** jaypipes is now known as leakypipes | 16:22 | |
| *** panda is now known as panda|bbl | 16:23 | |
| *** kristian__ has joined #openstack-nova | 16:23 | |
| *** kristian__ has quit IRC | 16:24 | |
| cdent | pumaranikar, johnthetubaguy I'd prefer middleware if we can make it work, but I haven't thought about it enough to know it can. We would need to make sure that it only engaged (processed the body) when it absolutely had to. The main issue is that anything happening "outside" of webob.dec.wsgify will not see an exception, only a response | 16:24 |
| *** mjura has quit IRC | 16:24 | |
| *** tbachman has quit IRC | 16:24 | |
| *** nmathew has quit IRC | 16:25 | |
| *** BlackDex has joined #openstack-nova | 16:25 | |
| *** kristian__ has joined #openstack-nova | 16:28 | |
| *** tbachman has joined #openstack-nova | 16:29 | |
| *** osallou has joined #openstack-nova | 16:31 | |
| osallou | hi, i have activated nova notifications and connected multiple subscribes to nova exchange, however a notifications.info queue is always created and is filled indefinitly is no subscribed to drain it, is it a configuration error? I need multiple subscribers on notifications, so I cannot simply listen to the queue... thanks | 16:33 |
| *** bnemec is now known as beekneemech | 16:33 | |
| *** mjura has joined #openstack-nova | 16:33 | |
| *** baoli has quit IRC | 16:34 | |
| *** zhurong has quit IRC | 16:34 | |
| *** tbachman has quit IRC | 16:35 | |
| *** mat128 is now known as mat128|mtg | 16:35 | |
| *** dtp has joined #openstack-nova | 16:36 | |
| *** osallou has left #openstack-nova | 16:37 | |
| *** baoli has joined #openstack-nova | 16:38 | |
| *** claudiub|2 has quit IRC | 16:38 | |
| pumaranikar | cdent, johnthetubaguy yes. I think for every resource based action we have added webob.dec.wsgify | 16:39 |
| EmilienM | mriedem: http://logs.openstack.org/86/428786/1/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/522b390/logs/nova/nova-api.txt.gz#_2017-02-03_16_15_23_470 | 16:41 |
| EmilienM | mriedem: should it be ::1:8774? | 16:41 |
| *** chyka has joined #openstack-nova | 16:42 | |
| *** tbachman has joined #openstack-nova | 16:44 | |
| mriedem | and remove the [] | 16:45 |
| mriedem | ? | 16:45 |
| mriedem | idk | 16:45 |
| mriedem | is that ipv6? | 16:45 |
| *** mdnadeem has quit IRC | 16:49 | |
| *** lpetrut has quit IRC | 16:50 | |
| *** kristia__ has joined #openstack-nova | 16:51 | |
| cdent | pumaranikar: sorry was on the phone. yes that's right, thus the difficulty | 16:51 |
| *** mtanino has quit IRC | 16:53 | |
| *** mriedem has quit IRC | 16:53 | |
| *** mtanino has joined #openstack-nova | 16:53 | |
| *** kristian__ has quit IRC | 16:53 | |
| *** mtanino has quit IRC | 16:53 | |
| *** mtanino has joined #openstack-nova | 16:53 | |
| *** mriedem has joined #openstack-nova | 16:54 | |
| openstackgerrit | Matthew Booth proposed openstack/nova master: libvirt: Fix races with nfs volume mount/umount https://review.openstack.org/383859 | 16:54 |
| *** satyar has joined #openstack-nova | 16:55 | |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: WIP conf: remove invalid TODOs https://review.openstack.org/428811 | 16:56 |
| pumaranikar | cdent, i will try with middleware and update you | 16:58 |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: WIP conf: remove invalid TODOs https://review.openstack.org/428811 | 16:58 |
| *** mdrabe has quit IRC | 17:03 | |
| johnthetubaguy | pumaranikar: we did that yesterday right, and it failed with middlewear? | 17:03 |
| johnthetubaguy | cdent: the exception just didn't seem to get as fair as that fault wrapper middleware | 17:04 |
| johnthetubaguy | it was already converted into a response with the correct error code, at least is seemed that way | 17:04 |
| pumaranikar | johnthetubaguy, yes. because we were getting valid response | 17:04 |
| cdent | yeah, if you use middleware, it has to be body processing instead of exception inspecting | 17:04 |
| johnthetubaguy | but the body doesn't have the data present that was in the exception I guess? | 17:05 |
| cdent | if the response ststus >= 400 and <500, then if it is json, then reformat it to the structure we want | 17:05 |
| johnthetubaguy | cdent: my suggestion to pumaranikar was to replace the wsgify with our own decorator, that ensure the exception gets the formatter added, its not ideal though | 17:05 |
| johnthetubaguy | it seems easier to remember that adding the formatter to the error message, but that might just be me | 17:06 |
| cdent | if there's a way to do that doesn't involve double decorating that might work | 17:06 |
| * cdent looks at webob code | 17:06 | |
| johnthetubaguy | yeah, its just the one decorator, I pushed that up yesterday into pumaranikar's patch | 17:06 |
| cdent | that's what I mean by double decorator | 17:07 |
| mriedem | artom: on https://blueprints.launchpad.net/nova/+spec/fix-tag-attribute-disappearing i think i'm going to just mark that as partially completed and then we'll finish the rest in pike | 17:07 |
| cdent | I think it would be better to subclass webob.dec.wsgify (which is a class) | 17:07 |
| johnthetubaguy | cdent: its not the nicest thing, but it seems to work: https://review.openstack.org/#/c/395194/5/nova/api/openstack/placement/handlers/allocation.py@274 | 17:07 |
| artom | mriedem, yep :) | 17:07 |
| johnthetubaguy | cdent: yeah, thats nicer | 17:08 |
| johnthetubaguy | pumaranikar: does that make sense? | 17:08 |
| cdent | except, unfortunately the part that one would want to override is smack in a tall stack of code :( | 17:08 |
| artom | mriedem, it's debatable whether the changes to tests are parts of the spec proper, in any case | 17:08 |
| johnthetubaguy | cdent: ah, doh, anyways my hack appeared to work, we could always make that nicer later | 17:09 |
| cdent | hmm, maybe override call_func | 17:09 |
| johnthetubaguy | cdent: did you see this way: https://review.openstack.org/#/c/395194/5/nova/api/openstack/placement/handlers/allocation.py@274 ? | 17:09 |
| cdent | pumaranikar: have a look at the wsgify class in webob/dec.py, may be some ideas | 17:09 |
| cdent | johnthetubaguy: yeah, saw that | 17:10 |
| cdent | I think that will work, but it feels icky for a reason I can't put my finger on | 17:10 |
| johnthetubaguy | cdent: totally with you on that | 17:11 |
| pumaranikar | cdent, looking at wsgify class. | 17:11 |
| *** raunak has joined #openstack-nova | 17:11 | |
| cdent | it might, however, be less messy than what I'm thinking. won't really know until we try it | 17:11 |
| johnthetubaguy | cdent: +1 | 17:11 |
| cdent | pumaranikar: I think overriding the call_fun method might work | 17:11 |
| *** mdrabe has joined #openstack-nova | 17:11 | |
| *** nic has joined #openstack-nova | 17:13 | |
| *** mjura has quit IRC | 17:14 | |
| *** raunak has quit IRC | 17:15 | |
| johnthetubaguy | cdent: I was curious on your gabbi thoughts too, this seemed the simplest way to ensure the formatter was present: https://review.openstack.org/#/c/395194/5/nova/tests/functional/api/openstack/placement/gabbits/allocations.yaml@158 | 17:15 |
| cdent | johnthetubaguy: yeah, pretty much that's it. look for a key and value that wouldn't be there if the formatter was not swinging | 17:16 |
| *** mvk has quit IRC | 17:17 | |
| *** kfarr has joined #openstack-nova | 17:17 | |
| EmilienM | mriedem: yes it's IPv6. Do you test nova in ipv6? | 17:18 |
| EmilienM | osapi_compute_listen=[::1] doesn't seem to work | 17:18 |
| mriedem | EmilienM: not sure off hand | 17:19 |
| mriedem | sdague or sc68cal might know | 17:19 |
| EmilienM | I'll try with ::1 | 17:19 |
| EmilienM | mriedem: triple jobs have ipv6 coverage, fyi | 17:20 |
| *** sambetts is now known as sambetts|afk | 17:20 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: Update the upgrades part of devref https://review.openstack.org/428824 | 17:21 |
| EmilienM | mriedem: btw, I saw your comment on https://review.openstack.org/#/c/428424/ - and I think we should have the job asap since things are broken for us, it would be helpful on both sides to have the job in place without having to run "check experimental". | 17:21 |
| superdan | mriedem: see my doc update for the upgrades.rst just now | 17:23 |
| superdan | mriedem: I was explaining that process to someone recently and wanted to RTFM them to that doc | 17:24 |
| *** xyang1 has quit IRC | 17:24 | |
| superdan | mriedem: but then realized it needed some serious freshening | 17:24 |
| *** annegentle has quit IRC | 17:25 | |
| *** xyang1 has joined #openstack-nova | 17:25 | |
| *** annegentle has joined #openstack-nova | 17:28 | |
| *** browne has joined #openstack-nova | 17:30 | |
| *** derekh has quit IRC | 17:30 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: utils.get_disk return the target dev of the root device https://review.openstack.org/402716 | 17:31 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Make guest.BlockDevice only use the target dev of a disk https://review.openstack.org/402163 | 17:31 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Wait for abort_job to complete when not resizing https://review.openstack.org/382449 | 17:31 |
| sc68cal | mriedem: EmilienM: [::1]:<port> | 17:32 |
| johnthetubaguy | superdan: I will take a peak at that upgrade doc, looks interesting | 17:32 |
| superdan | johnthetubaguy: thanks | 17:32 |
| sc68cal | mriedem: EmilienM: so I guess [::1]:8774 is what you're looking for | 17:32 |
| EmilienM | sc68cal: yeah? we had osapi_compute_listen=host until now | 17:33 |
| EmilienM | and osapi_compute_listen_port for the port | 17:33 |
| sc68cal | ok, then osapi_compute_listen=::1 should work | 17:35 |
| EmilienM | sc68cal: http://git.openstack.org/cgit/openstack/nova/tree/nova/conf/service.py#n96 | 17:35 |
| EmilienM | sc68cal: ok I'm trying it, thanks | 17:35 |
| EmilienM | sc68cal: FYI we add brackets because some other services need it | 17:35 |
| sc68cal | hang on let me poke it with netcat | 17:36 |
| EmilienM | and the best practise in IPv6 is to have brackets | 17:36 |
| EmilienM | tbh, I think having [::1] rather than ::1 is much safer | 17:36 |
| sc68cal | oh yes much safer | 17:36 |
| *** lucasagomes is now known as lucas-afk | 17:36 | |
| *** raunak has joined #openstack-nova | 17:37 | |
| *** dave-mccowan has quit IRC | 17:38 | |
| *** mkrai has left #openstack-nova | 17:41 | |
| *** kristian__ has joined #openstack-nova | 17:41 | |
| *** vsaienko has joined #openstack-nova | 17:41 | |
| sc68cal | some services want brackets, others don't | 17:42 |
| *** ducttape_ has quit IRC | 17:42 | |
| *** kristia__ has quit IRC | 17:44 | |
| sc68cal | are we passing the correct options to eventlet.listen? there's an optional kwarg for address family | 17:44 |
| sc68cal | it might be defaulting to v4 and trying to resolve ::1 as a dns name | 17:45 |
| johnthetubaguy | superdan: I thought SIG_HUP was enough after the initial restart? | 17:45 |
| johnthetubaguy | superdan: oh, missread, I see your point now | 17:46 |
| superdan | johnthetubaguy: not for static configs, yeah | 17:47 |
| superdan | that subtle detail could have been pretty annoying for someone :) | 17:47 |
| johnthetubaguy | yeah, totally | 17:48 |
| *** bvanhav_ has joined #openstack-nova | 17:50 | |
| superdan | johnthetubaguy: I was tempted to put "read the effing release notes" as a formal step in there :) | 17:51 |
| *** bvanhav has quit IRC | 17:52 | |
| johnthetubaguy | superdan: step 2, no really, read them. | 17:52 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Add check for invalid inventory amounts https://review.openstack.org/428800 | 17:53 |
| *** Swami has joined #openstack-nova | 17:53 | |
| superdan | johnthetubaguy: heh, right | 17:53 |
| figleaf | cdent: ^^ Fixed | 17:53 |
| *** yamahata has quit IRC | 17:53 | |
| *** kristian__ has quit IRC | 17:53 | |
| cdent | figleaf: ossum | 17:54 |
| *** lpetrut has joined #openstack-nova | 17:54 | |
| openstackgerrit | Anusha Unnam proposed openstack/nova master: Reno for additional-notification-fields-for-searchlight https://review.openstack.org/428835 | 17:54 |
| *** EmilienM has quit IRC | 17:56 | |
| *** scottynomad has quit IRC | 17:56 | |
| *** Kimmo_ has quit IRC | 17:56 | |
| *** mrodden has quit IRC | 17:56 | |
| *** tonyb has quit IRC | 17:56 | |
| *** jamespd has quit IRC | 17:56 | |
| *** mikal has quit IRC | 17:56 | |
| *** rodrigods has quit IRC | 17:56 | |
| *** Tahvok has quit IRC | 17:56 | |
| *** gmann has quit IRC | 17:56 | |
| *** gibi has quit IRC | 17:56 | |
| *** Nakato has quit IRC | 17:56 | |
| *** mkoderer___ has quit IRC | 17:56 | |
| *** d34dh0r53 has quit IRC | 17:56 | |
| *** Anticimex has quit IRC | 17:56 | |
| *** s1rp has quit IRC | 17:56 | |
| *** alex_xu has quit IRC | 17:56 | |
| *** tonyb has joined #openstack-nova | 17:56 | |
| *** mikal has joined #openstack-nova | 17:56 | |
| *** rodrigods has joined #openstack-nova | 17:56 | |
| *** gibi has joined #openstack-nova | 17:56 | |
| *** jamespd has joined #openstack-nova | 17:56 | |
| *** scottynomad has joined #openstack-nova | 17:56 | |
| *** Kimmo_ has joined #openstack-nova | 17:56 | |
| *** s1rp has joined #openstack-nova | 17:56 | |
| *** Tahvok has joined #openstack-nova | 17:56 | |
| *** andreaf has quit IRC | 17:56 | |
| *** aries has quit IRC | 17:56 | |
| *** agarner has quit IRC | 17:56 | |
| *** shaohe_feng has quit IRC | 17:56 | |
| *** devananda has quit IRC | 17:56 | |
| *** smcginnis has quit IRC | 17:56 | |
| *** kevinbenton has quit IRC | 17:56 | |
| *** jotr has quit IRC | 17:56 | |
| *** Madasi has quit IRC | 17:56 | |
| *** amrith has quit IRC | 17:56 | |
| *** d34dh0r53 has joined #openstack-nova | 17:56 | |
| *** Anticimex has joined #openstack-nova | 17:56 | |
| *** mrodden has joined #openstack-nova | 17:56 | |
| *** EmilienM has joined #openstack-nova | 17:56 | |
| *** jowisz_ has quit IRC | 17:56 | |
| *** lyarwood has quit IRC | 17:56 | |
| *** jgriffith has quit IRC | 17:56 | |
| *** zer0c00l has quit IRC | 17:56 | |
| *** scottda has quit IRC | 17:56 | |
| *** zer0c00l has joined #openstack-nova | 17:56 | |
| *** lyarwood has joined #openstack-nova | 17:56 | |
| *** jowisz has joined #openstack-nova | 17:56 | |
| *** smcginnis has joined #openstack-nova | 17:56 | |
| *** alex_xu has joined #openstack-nova | 17:56 | |
| *** devananda has joined #openstack-nova | 17:56 | |
| *** mkoderer__ has joined #openstack-nova | 17:56 | |
| *** amrith has joined #openstack-nova | 17:56 | |
| *** dave-mccowan has joined #openstack-nova | 17:57 | |
| *** aries has joined #openstack-nova | 17:57 | |
| *** Nakato has joined #openstack-nova | 17:57 | |
| *** gmann has joined #openstack-nova | 17:57 | |
| *** kevinbenton has joined #openstack-nova | 17:57 | |
| *** jgriffith has joined #openstack-nova | 17:57 | |
| *** andreaf has joined #openstack-nova | 17:58 | |
| *** Madasi has joined #openstack-nova | 17:58 | |
| *** jose-phillips has joined #openstack-nova | 17:59 | |
| *** aleph1 has joined #openstack-nova | 17:59 | |
| *** DuncanT has quit IRC | 17:59 | |
| *** test222 has quit IRC | 17:59 | |
| sc68cal | EmilienM: so looks like we had a netsplit | 18:00 |
| sc68cal | EmilienM: where did I lose you | 18:00 |
| *** jotr has joined #openstack-nova | 18:01 | |
| *** DuncanT has joined #openstack-nova | 18:01 | |
| EmilienM | sc68cal: better: http://logs.openstack.org/86/428786/3/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/d244aec/logs/nova/nova-api.txt.gz#_2017-02-03_17_44_34_097 | 18:01 |
| EmilienM | sc68cal: so using ::1 works :-/ | 18:01 |
| *** bvanhav_ is now known as bvanhav | 18:01 | |
| *** ralonsoh has quit IRC | 18:02 | |
| sc68cal | EmilienM: yep..... we have same issue with mysql vs alembic/sqlchemy | 18:02 |
| aunnam | gibi, if possible can you review this patch https://review.openstack.org/#/c/428835/ | 18:02 |
| *** shaohe_feng has joined #openstack-nova | 18:03 | |
| sc68cal | EmilienM: https://github.com/openstack-dev/devstack/blob/master/lib/database#L77 | 18:03 |
| *** test222 has joined #openstack-nova | 18:03 | |
| *** scottda has joined #openstack-nova | 18:04 | |
| *** Apoorva has joined #openstack-nova | 18:04 | |
| openstackgerrit | Bob Ball proposed openstack/nova master: Fix live migrate with XenServer https://review.openstack.org/424428 | 18:05 |
| BobBall | johnthetubaguy: ^^ Hopefully that'll fix the different network concern | 18:05 |
| *** alex_xu has quit IRC | 18:06 | |
| *** sudipto has quit IRC | 18:07 | |
| *** sudipto_ has quit IRC | 18:07 | |
| *** alex_xu has joined #openstack-nova | 18:08 | |
| *** ducttape_ has joined #openstack-nova | 18:09 | |
| cdent | mriedem, johnthetubaguy: the tagging has not happened has it? | 18:09 |
| cdent | writing up some notes and wanted to know if https://review.openstack.org/#/c/428375/ was part of the now-ish release | 18:10 |
| johnthetubaguy | https://review.openstack.org/#/c/428531/ says we haven't tagged yet, looks like we can include the ironic one if we wish | 18:11 |
| *** xinli has joined #openstack-nova | 18:16 | |
| mriedem | cdent: not yet | 18:26 |
| mriedem | but now that https://review.openstack.org/#/c/428375/ is merged and i'm back from lunch, i'm going to rev the rc1 tag patch | 18:26 |
| cdent | mriedem: how very exciting | 18:27 |
| *** annegentle has quit IRC | 18:30 | |
| mriedem | i know, i'm cutting glass over here | 18:31 |
| *** kristian__ has joined #openstack-nova | 18:31 | |
| superdan | lol | 18:33 |
| openstackgerrit | Sujitha proposed openstack/nova master: Reno for additional-notification-fields-for-searchlight https://review.openstack.org/428835 | 18:33 |
| *** vsaienko has quit IRC | 18:33 | |
| * johnthetubaguy makes smashing sounds | 18:33 | |
| *** yamahata has joined #openstack-nova | 18:34 | |
| johnthetubaguy | mriedem: I am guessing there is no way to backport this one, if it misses RC1 right? https://review.openstack.org/#/c/424428 | 18:34 |
| mriedem | johnthetubaguy: not with the object version change, | 18:35 |
| mriedem | but there is rc2 | 18:35 |
| johnthetubaguy | mriedem: what if we just get the object change in? | 18:35 |
| mriedem | heh, so i thought about that yesterday | 18:36 |
| johnthetubaguy | mriedem: but rc2 is a backport I guess | 18:36 |
| mriedem | yes rc2 is a backport to stable/ocata technically | 18:36 |
| mriedem | but it'd also be pre-release | 18:36 |
| superdan | johnthetubaguy: mriedem it's okay if the versions have no gap, and I think this is sufficiently defensive | 18:36 |
| superdan | I mean, it's technically okay if there's no gap | 18:36 |
| *** kristian__ has quit IRC | 18:36 | |
| johnthetubaguy | superdan: oh, true, the likely hood of moving from stable to a commit without that commit seems fairly tiny | 18:37 |
| superdan | johnthetubaguy: the biggestdeal is if you happen to get conductor on rc1 and compute on rc2 | 18:37 |
| superdan | however, | 18:37 |
| superdan | I think rcN is really to make sure that rc1 was what we wanted | 18:38 |
| superdan | mechanically it's a backport, | 18:38 |
| superdan | but I can't really think the semantics can be considered the same | 18:38 |
| johnthetubaguy | yeah, thats fair | 18:38 |
| *** tesseract has quit IRC | 18:38 | |
| mriedem | so..are we saying we can backport the whole thing for rc2, or do we hold up rc1 for just the object version change and then backport the rest? | 18:39 |
| johnthetubaguy | not hit an object change in an RC potential bug before, I guess it must have happened | 18:39 |
| johnthetubaguy | mriedem: the first one I think | 18:39 |
| mriedem | superdan: you agree? ^ | 18:39 |
| superdan | mriedem: yeah, all in rc2 | 18:39 |
| johnthetubaguy | its a driver specific object anyways, so the risk is lower I guess | 18:40 |
| superdan | this is the thing that is not a regression in ocata though right? | 18:40 |
| johnthetubaguy | it might have been newton, not sure actually | 18:40 |
| EmilienM | mriedem: can you revisit https://review.openstack.org/#/c/428424/ please? | 18:41 |
| mriedem | johnthetubaguy: superdan: BobBall said it was a regression in newton | 18:42 |
| superdan | yeah, thought so | 18:42 |
| *** prateek_ has joined #openstack-nova | 18:42 | |
| *** prateek has joined #openstack-nova | 18:42 | |
| *** Sukhdev has joined #openstack-nova | 18:45 | |
| EmilienM | mriedem: wow so https://review.openstack.org/#/c/428415/ passed for us? | 18:47 |
| EmilienM | mriedem: but fail on our baremetal jobs: http://logs.openstack.org/15/428415/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/92a342e/console.html#_2017-02-03_01_43_02_872309 | 18:48 |
| mriedem | and fails a ton of tests in nova | 18:48 |
| mriedem | and we don't really know what else would break with it | 18:48 |
| mriedem | since we have 0 burn in time | 18:48 |
| *** Hosam has joined #openstack-nova | 18:49 | |
| EmilienM | mriedem: this one looks strange: http://logs.openstack.org/15/428415/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/92a342e/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-02-03_01_06_02_020 | 18:49 |
| mriedem | yjs | 18:50 |
| mriedem | that's using ironic? | 18:50 |
| EmilienM | mriedem: yes | 18:50 |
| *** cleong has quit IRC | 18:50 | |
| *** slaweq has quit IRC | 18:50 | |
| *** burt has quit IRC | 18:50 | |
| mriedem | hmm, that's odd. something not working in the resource tracking, but that stuff should come in when nova-compute starts up, | 18:52 |
| mriedem | unless nova-compute is failing to get info from ironic | 18:52 |
| mriedem | Total disk: 40 GB, used: 40.00 GB | 18:52 |
| mriedem | right above that | 18:52 |
| EmilienM | could it be nova/wsgi again? | 18:52 |
| mriedem | and just before that Total disk: 40 GB, used: 0.00 GB | 18:53 |
| superdan | mriedem: unless nova-compute starts before any ironic nodes are provisioned | 18:53 |
| superdan | mriedem: also, they stop reporting resources for nodes that are in the middle of a change | 18:53 |
| superdan | mriedem: so they will go in and out and in and out over the course of a normal lifetime | 18:53 |
| superdan | it's weird | 18:53 |
| EmilienM | superdan: in the case opposite happens, nova-ocmpute is able to update nodes resources? | 18:53 |
| mriedem | there is an instance created right before that one that claims all the disk http://logs.openstack.org/15/428415/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/92a342e/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-02-03_01_06_01_052 | 18:53 |
| superdan | but jroll and I were looking at that the other day | 18:53 |
| EmilienM | let me look in logstash if tripleo ci had this error before | 18:54 |
| mriedem | EmilienM: logstash is super backed up | 18:54 |
| jroll | I didn't do it | 18:54 |
| mriedem | so probably no results there | 18:54 |
| mriedem | plus and http://logs.openstack.org/15/428415/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/92a342e/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-02-03_01_06_02_020 is debug level which we don't index | 18:55 |
| EmilienM | mriedem: over the last 30 days, one occurence, on 427536 changfe | 18:56 |
| mriedem | we only store 10 days of logs | 18:56 |
| mriedem | and it's been busted for several days | 18:56 |
| EmilienM | mriedem: I think it's pretty clear this patch is responsible | 18:56 |
| EmilienM | even 10 days | 18:56 |
| mriedem | well the patch is https://review.openstack.org/#/c/427536/ | 18:56 |
| EmilienM | mriedem: http://tripleo.org/cistatus.html we run so many jobs per day, we would have seen it before | 18:56 |
| mriedem | 2 deps | 18:56 |
| mriedem | i don't see how https://review.openstack.org/#/c/427782/ would cause this | 18:57 |
| mriedem | so what does https://review.openstack.org/#/c/427534/ do? | 18:57 |
| mriedem | my money is on ^ | 18:57 |
| mriedem | "Ensure that the ironic nodes have been picked up by the nova resource tracker before running nova-manage cell_v2 host discovery." | 18:57 |
| EmilienM | owalsh: you around? ^ | 18:57 |
| superdan | 427782 is only about getting instances via the api, really nothing to do with any of that | 18:57 |
| mriedem | right | 18:57 |
| mriedem | the latter is specifically about when to discover ironic hosts for resource tracking | 18:58 |
| superdan | mriedem: right, and it is important because they weren't enrolling nodes by the time they ran discover | 18:58 |
| superdan | without it they're 100% fail because they never have host mappings | 18:59 |
| mriedem | even though, regardless of all those changes, | 18:59 |
| mriedem | http://logs.openstack.org/15/428415/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/92a342e/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-02-03_01_06_01_049 | 18:59 |
| mriedem | Attempting claim: memory 4096 MB, disk 40 GB, vcpus 1 CPU | 18:59 |
| openstackgerrit | John Perkins proposed openstack/nova master: Integrate oslo.config validator https://review.openstack.org/428974 | 18:59 |
| mriedem | Total disk: 40 GB, used: 0.00 GB | 18:59 |
| mriedem | so that instance claims all of the disk | 18:59 |
| mriedem | which is why the next one fails | 18:59 |
| mriedem | unless, there should be more nodes reporting in | 18:59 |
| mriedem | i.e. we should have more than 40GB total | 19:00 |
| superdan | or the scheduler raced | 19:00 |
| cdent | whose 40GB is that? Is there a chance ironic is now trying to claim the wrong disk? (we are talking ironic here aren't we?) | 19:00 |
| mriedem | yes ironic | 19:01 |
| mriedem | 2017-02-03 01:05:33.326 28176 DEBUG nova.virt.ironic.driver [req-4a6f8f71-565c-4379-8288-347b8e9da028 - - - - -] Returning 3 available node(s) get_available_nodes /usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py:610 | 19:03 |
| mriedem | Hypervisor/Node resource view: name=7c2a1870-c2f8-46d5-b1a4-c72443bfc82f free_ram=8192MB free_disk=40GB free_vcpus=4 pci_devices=None | 19:03 |
| mriedem | Final resource view: name=7c2a1870-c2f8-46d5-b1a4-c72443bfc82f phys_ram=8192MB used_ram=0MB phys_disk=40GB used_disk=0GB total_vcpus=4 used_vcpus=0 pci_stats=[] | 19:03 |
| mriedem | Final resource view: name=aa97370e-02a2-4aa9-8833-616d30a4d589 phys_ram=8192MB used_ram=0MB phys_disk=40GB used_disk=0GB total_vcpus=4 used_vcpus=0 pci_stats=[] | 19:04 |
| mriedem | Final resource view: name=6a085a83-e051-480b-bf37-d092d7bbf13e phys_ram=8192MB used_ram=0MB phys_disk=40GB used_disk=0GB total_vcpus=4 used_vcpus=0 pci_stats=[] | 19:04 |
| mriedem | so those are the 3 nodes | 19:04 |
| mriedem | 40GB each | 19:04 |
| superdan | right and ironic doesn't sum the disks I think | 19:04 |
| superdan | to prevent you from trying to schedule a 120G instance to that compute right/ | 19:04 |
| mriedem | so 1 instance should be placed on 1 node right? | 19:04 |
| *** salv-orlando has quit IRC | 19:05 | |
| superdan | yep | 19:05 |
| superdan | I might be thinking of the summing in the api bit | 19:05 |
| superdan | anyway, three nodes, 40g each | 19:05 |
| mriedem | would be nice if the "Claim successful" message told us which node the resources were claimed on | 19:05 |
| mriedem | in this log it's building a few instances | 19:05 |
| mriedem | and it looks like one hits on a node that's used | 19:05 |
| *** annegentle has joined #openstack-nova | 19:06 | |
| openstackgerrit | John Perkins proposed openstack/nova master: Integrate oslo.config validator https://review.openstack.org/428974 | 19:06 |
| superdan | i.e. scheduler race | 19:06 |
| superdan | aka that thing we say is explicitly out of scope: scheduling to 100% capacity | 19:06 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: placement: RT now adds proper Ironic inventory https://review.openstack.org/404472 | 19:07 |
| *** salv-orlando has joined #openstack-nova | 19:08 | |
| openstackgerrit | OpenStack Release Bot proposed openstack/nova master: Update reno for stable/ocata https://review.openstack.org/428983 | 19:10 |
| *** unicell has joined #openstack-nova | 19:10 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add nodename to _claim_test log messages https://review.openstack.org/428985 | 19:12 |
| cdent | figleaf, johnthetubaguy: Is https://review.openstack.org/#/c/404472/ no longer targetted for some version of ocata, or is it still under consideration? | 19:13 |
| mriedem | cdent: i'm not considering it unless johnthetubaguy is super comfortable with it, | 19:14 |
| mriedem | i've been out of the loop on that one since last week | 19:14 |
| mriedem | as discussed a bit in here yesterday, i think we can do that in pike, | 19:14 |
| mriedem | and put a service version on it so that the scheduler could know when it can start making requests with custom resource classes | 19:15 |
| cdent | I think there are still some outstanding questions on it, maybe addressed by figleaf just now, so it does feel a bit late | 19:15 |
| *** crushil has quit IRC | 19:15 | |
| mriedem | in the ironic host manager scheduler thing | 19:15 |
| johnthetubaguy | yeah, the questions don't all seem answered yet | 19:15 |
| johnthetubaguy | I think we update the allocations more frequently than I first though, but I don't quite understand how the old allocations get refreshed | 19:16 |
| *** sdague has quit IRC | 19:16 | |
| johnthetubaguy | figleaf: ^ | 19:16 |
| cfriesen | I've got an idea for an enhancement, wondering if it's worth writing up a spec. Basically the idea is that currently when booting the instance we allow "--user-data" and "--file", but when you rebuild the instance we only support "--file". So I'd like to propose adding support for "--user-data" when doing a rebuild. | 19:17 |
| johnthetubaguy | it also seems odd to rebuild the inventory for the resource provider in two places | 19:17 |
| cfriesen | Is this a dumb idea that'd get shot down immediately, or is it worth writing up? | 19:17 |
| johnthetubaguy | cfriesen: seems worth adding a spec, make sense I think, config drive will be recreated either way, so why not I guess | 19:18 |
| figleaf | johnthetubaguy: I'm not very clear on what part of this is confusing to you. I guess the fact that a lot of it is confusing is an issue in and of itself. | 19:19 |
| *** eharney has quit IRC | 19:19 | |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: deprecate key_manager.fixed _key opt https://review.openstack.org/428992 | 19:19 |
| mriedem | cfriesen: didn't this come up recently-ish in the ML? | 19:20 |
| mriedem | cfriesen: something about injected files not persisting across a rebuild? | 19:20 |
| cdent | "I guess the fact that a lot of it is confusing is an issue in and of itself."++ | 19:20 |
| mriedem | or evacuate maybe | 19:20 |
| melwitt | yeah, I dunno what the reasons might be for not allowing it, but I have recalled mikal having the opinion that config drive is a once-at-init thing and not creating/changing it after instance boot | 19:21 |
| mriedem | cfriesen: which was tied to a thread i had about deprecating/removing file injection | 19:21 |
| melwitt | so I feel like he's the one to talk to | 19:21 |
| cfriesen | mriedem: yes, userdata currently gets lost over a rebuild | 19:21 |
| johnthetubaguy | figleaf: how do all the old instance allocations get refreshed, basically | 19:21 |
| melwitt | it doesn't get lost, it stays on the config drive. or it should | 19:21 |
| mriedem | i was thinking of the injected files / evacuate thing specifically i think | 19:21 |
| cfriesen | mreidem: are you talking about removing --userdata from boot as well? | 19:21 |
| johnthetubaguy | figleaf: I see how the new ones get the correct allocations | 19:21 |
| mriedem | cfriesen: no | 19:22 |
| johnthetubaguy | figleaf: I guess its possible when nova-compute first starts? | 19:22 |
| mriedem | just --files | 19:22 |
| mriedem | or personality or whatever | 19:22 |
| mriedem | let me see if i can find the thread | 19:22 |
| johnthetubaguy | mriedem: files will get lost, it only appears in the initial config drive, we don't store them | 19:22 |
| mriedem | johnthetubaguy: yup i know | 19:23 |
| cfriesen | mriedem: right. so currently we don't allow specifying files on an evacuate, but they are preserved | 19:23 |
| mriedem | it was one of the reasons for killing file injection | 19:23 |
| *** amotoki has joined #openstack-nova | 19:23 | |
| mriedem | cfriesen: http://lists.openstack.org/pipermail/openstack-dev/2016-November/107195.html | 19:23 |
| *** vsaienko has joined #openstack-nova | 19:23 | |
| cfriesen | mriedem: preserved in the config drive, right | 19:23 |
| johnthetubaguy | cfriesen: well, depends if you are using shared storage I guess | 19:24 |
| *** bvanhav has quit IRC | 19:24 | |
| cfriesen | mriedem: yes | 19:24 |
| cfriesen | mridem: so if we're still going to allow --user-data on boot, I think it'd make sense to allow it on rebuild | 19:24 |
| figleaf | johnthetubaguy: the _init_compute_node is one place, yes | 19:24 |
| *** bvanhav has joined #openstack-nova | 19:24 | |
| johnthetubaguy | figleaf: because tracked instances is empty to start with, so everything is treated as a new instance? | 19:24 |
| mriedem | cfriesen: yeah i think that's fine | 19:25 |
| figleaf | johnthetubaguy: then self._update_usage_from_instances keeps them in sync | 19:25 |
| melwitt | cfriesen: I'm still stuck on the part where you say userdata gets wiped out. it didn't use to on rebuild, it should just stay on the config drive? | 19:25 |
| melwitt | this came up before at my last company and we tested it and userdata stayed intact across a rebuild | 19:25 |
| figleaf | johnthetubaguy: yeah, that happens in _init_compute_node | 19:25 |
| mriedem | 14:20:38 <johnthetubaguy> #action deprecate localfs file injection in ocata, remove in pike | 19:25 |
| cfriesen | melwitt: currently injected files get lost on evacuate | 19:25 |
| mriedem | damn! | 19:25 |
| melwitt | hmm | 19:26 |
| *** big_macsz has joined #openstack-nova | 19:26 | |
| cfriesen | melwitt: currently userdata stays intact over a rebuild. my usecase is for people that want to rebuild to a different image with different userdata | 19:26 |
| *** vsaienko has quit IRC | 19:26 | |
| *** sdague has joined #openstack-nova | 19:26 | |
| melwitt | cfriesen: yeah. okay, that makes sense. I thought earlier you said something about userdata getting wiped out during a rebuild | 19:26 |
| cdent | figleaf, johnthetubaguy: tracked_instances is emptied every periodic job and _init_compute_node is called every periodic job | 19:26 |
| superdan | cfriesen: isn't that called "boot" ? | 19:26 |
| figleaf | cdent: oh geez - really? | 19:27 |
| cdent | figleaf: you read this yet: http://lists.openstack.org/pipermail/openstack-dev/2017-January/110953.html | 19:27 |
| cfriesen | superdan: you would think | 19:27 |
| *** dharinic is now known as dharinic|lunch | 19:27 | |
| EmilienM | superdan: do you think I need to file a bug on tripleo side for the ironic thing? (wrt the patch in tripleo-common owalsh did) | 19:27 |
| cfriesen | superdan: as I recall it ended up being related to heat updates | 19:27 |
| figleaf | cdent: yes, but I don't have it committed to memory :) | 19:27 |
| cdent | figleaf: oh, but you should | 19:28 |
| figleaf | cdent: Ok, I see the tracked_instances.clear() line | 19:28 |
| sdague | EmilienM: any behavior in the nova osapi_compute_listen field is because of the way oslo.service works - https://github.com/openstack/oslo.service/blob/7fa34bf8e7a37a6d9355aaf55aa85b1931567cd6/oslo_service/wsgi.py#L127-L151 | 19:28 |
| cdent | figleaf: it's all very bewildering and amazing | 19:28 |
| *** amotoki has quit IRC | 19:28 | |
| superdan | EmilienM: no | 19:28 |
| EmilienM | sdague: ok I see, thanks | 19:28 |
| johnthetubaguy | cdent: I guess thats where all those fetch inventory calls come from, from the attempted update | 19:30 |
| johnthetubaguy | cdent: hmm, I meant allocations | 19:30 |
| cdent | I'm gonna escape for the evening unless there's something needing me? | 19:31 |
| johnthetubaguy | cdent: I need to do the same really | 19:31 |
| johnthetubaguy | I have the oven on | 19:31 |
| figleaf | Have a great weekend, cdent and johnthetubaguy | 19:31 |
| *** vsaienko has joined #openstack-nova | 19:33 | |
| cdent | johnthetubaguy: yeah, dinner calls | 19:33 |
| * cdent bows | 19:33 | |
| *** prateek_ has quit IRC | 19:33 | |
| *** cdent has quit IRC | 19:33 | |
| *** prateek has quit IRC | 19:33 | |
| big_macsz | bon appetit | 19:35 |
| cfriesen | mriedem: did anyone actually start on a spec for deprecating file injection? | 19:35 |
| mriedem | cfriesen: no it was on my todo list at some point and then i forgot about it | 19:36 |
| mriedem | i blame the holidays | 19:36 |
| *** vsaienko has quit IRC | 19:36 | |
| cfriesen | heh...so are we now looking at deprecating in pike? | 19:36 |
| cfriesen | removing in q? | 19:36 |
| mriedem | yeah | 19:36 |
| * johnthetubaguy nods in shame | 19:36 | |
| mriedem | johnthetubaguy: it was on me | 19:36 |
| cfriesen | heh...okay, I can write that one up if you like. | 19:37 |
| openstackgerrit | Emilien Macchi proposed openstack/nova master: DNM - test tripleo without nova api in WSGI/apache https://review.openstack.org/429008 | 19:37 |
| johnthetubaguy | figleaf: that last question I had was about the inventory, it seems odd to need to update it when updating the instances, surely thats been done already in the RT? | 19:38 |
| EmilienM | mriedem: now rc1 is cut, can you look https://review.openstack.org/#/c/428424/ please? | 19:38 |
| *** tbachman has quit IRC | 19:39 | |
| *** vsaienko has joined #openstack-nova | 19:39 | |
| figleaf | johnthetubaguy: the only way inventory is updated there is if it doesn't match the number of allocations | 19:42 |
| figleaf | johnthetubaguy: so it removes everything and rebuilds everything to the "correct" way (old and new style) | 19:42 |
| johnthetubaguy | figleaf: which is for new instances too right? | 19:42 |
| figleaf | johnthetubaguy: next pass through, it should find that len(cur_allocs) == len(inv_data), so it won't do that again | 19:43 |
| johnthetubaguy | figleaf: but its not needed, if you look here: https://github.com/openstack/nova/blob/cab5dbfac7513d2cef1073787e49f6d063ec8fa7/nova/compute/resource_tracker.py#L575 | 19:43 |
| figleaf | johnthetubaguy: well, what we might think are "new instances" is different | 19:43 |
| johnthetubaguy | figleaf: we always update the inventory first | 19:43 |
| figleaf | johnthetubaguy: as cdent mentioned, the tracker is cleared on every pass, so they are always "new" | 19:43 |
| johnthetubaguy | yes, but thats a red herring here | 19:44 |
| johnthetubaguy | I mean if you boot an instance, every time we would now re-write the inventory | 19:44 |
| johnthetubaguy | (well attempt an update) | 19:44 |
| figleaf | johnthetubaguy: what we need is to map this all out visually. It's all too confusing otherwise | 19:46 |
| johnthetubaguy | figleaf: that probably would help | 19:46 |
| figleaf | johnthetubaguy: I still trip myself up following the ironic vs. non-ironic paths | 19:46 |
| johnthetubaguy | figleaf: so generally, the code updates the inventory, then updates the allocations | 19:47 |
| johnthetubaguy | figleaf: I think this new code should do the same thing, rather than trying to re-update the inventory | 19:47 |
| dtp | reading the release notes. do cells v1 deployments need to go through the cells v2 setup for Ocata? | 19:47 |
| superdan | dtp: yes | 19:48 |
| superdan | I thought it said that explicitly | 19:48 |
| dtp | it does, but from discussions we had previously, i'm under the impression folks can still run v1 cells w/ the CELLS_ENABLED flag or whatever it was | 19:49 |
| superdan | we've been through so many cells related renos I've lost track | 19:49 |
| superdan | they can | 19:49 |
| superdan | but they still have to do the cellsv2 stuff | 19:49 |
| *** claudiub|2 has joined #openstack-nova | 19:49 | |
| dtp | ok, including creating a cell 1? or just the cell 0? | 19:49 |
| *** salv-orl_ has joined #openstack-nova | 19:49 | |
| mriedem | both | 19:49 |
| mriedem | dtp: also https://review.openstack.org/#/c/427981/ | 19:49 |
| figleaf | johnthetubaguy: understood, but I'm not seeing where the problem is. I can only see that happening when the allocations don't match inventory for ironic | 19:51 |
| johnthetubaguy | figleaf: no, its a bit different | 19:51 |
| johnthetubaguy | ... trying to think of how to describe it | 19:52 |
| johnthetubaguy | we have the inventory update, so it includes the new resource calss | 19:52 |
| johnthetubaguy | class | 19:52 |
| figleaf | johnthetubaguy: hey! how about writing a functional test! :) | 19:52 |
| *** salv-orlando has quit IRC | 19:52 | |
| johnthetubaguy | we now call the update | 19:52 |
| figleaf | (which is what we should have done from the outset) | 19:52 |
| johnthetubaguy | figleaf: if it were not 7.50pm, sure | 19:52 |
| johnthetubaguy | basically, we start with updating the inventory | 19:53 |
| johnthetubaguy | then we have to check if the allocations match what should be in the inventory | 19:53 |
| johnthetubaguy | thats not the case if its new, or we are hitting the upgrade bump | 19:53 |
| johnthetubaguy | in that case we need to fix up the allocations, thats all good | 19:54 |
| johnthetubaguy | but there is no need to update the inventory, its already been done | 19:54 |
| *** vsaienko has quit IRC | 19:54 | |
| *** claudiub has joined #openstack-nova | 19:54 | |
| figleaf | johnthetubaguy: so you're saying that they inventory always has to be correct? IOW, it's always the allocations that are out of whack? | 19:55 |
| johnthetubaguy | I am saying the inventory is always updated before we update the allocations | 19:55 |
| *** dharinic|lunch is now known as dharinic | 19:55 | |
| *** vsaienko has joined #openstack-nova | 19:56 | |
| *** ducttape_ has quit IRC | 19:56 | |
| *** claudiub|2 has quit IRC | 19:57 | |
| * johnthetubaguy bravely runs away | 19:58 | |
| figleaf | johnthetubaguy: that could well be the case | 19:58 |
| figleaf | johnthetubaguy: I think leakypipes must have trying the belts and braces approach | 19:58 |
| figleaf | johnthetubaguy: the part I'm not clear on is where all the calls to RT._update_usage_from_instance() can be made, and whether those paths always update inventory first | 20:01 |
| *** tbachman has joined #openstack-nova | 20:01 | |
| cfriesen | is there a writeup of the as-implemented behaviour for resource providers, or are the specs/code all there is currently? | 20:01 |
| mriedem | didn't markus_z have a burndown chart somewhere for the config option cleanup work? | 20:03 |
| figleaf | cfriesen: There have been several excellent posts to the ML by cdent | 20:03 |
| cfriesen | figleaf: just saw his most recent one | 20:03 |
| figleaf | cfriesen: but the fact that those of us working on it are still a little fuzzy about the edge cases is a good sign that we need to do something more | 20:04 |
| mriedem | http://45.55.105.55:8082/config-options.html | 20:05 |
| *** salv-orl_ has quit IRC | 20:05 | |
| *** salv-orlando has joined #openstack-nova | 20:06 | |
| *** amoralej is now known as amoralej|off | 20:08 | |
| dtp | thanks mriedem superdan. left my comments on that review | 20:14 |
| openstackgerrit | melanie witt proposed openstack/nova master: Add step-by-step examples for Cells V2 setup https://review.openstack.org/427981 | 20:14 |
| *** panda|bbl is now known as panda | 20:15 | |
| melwitt | darn it, didn't see dtp's comments a minute ago | 20:15 |
| dtp | we were very close :) | 20:15 |
| *** sree has joined #openstack-nova | 20:16 | |
| *** ijw has joined #openstack-nova | 20:16 | |
| mriedem | macsz: sfinucan: if you have open config option cleanup changes, move them to https://blueprints.launchpad.net/nova/+spec/centralize-config-options-pike please | 20:17 |
| macsz | mriedem: i will move every single patch that is left | 20:17 |
| mriedem | thanks | 20:18 |
| macsz | mriedem can you re-assign me to it? | 20:18 |
| mriedem | sure | 20:19 |
| macsz | thanks | 20:19 |
| *** unicell has quit IRC | 20:20 | |
| *** Sukhdev has quit IRC | 20:20 | |
| *** raunak has quit IRC | 20:21 | |
| *** sree has quit IRC | 20:21 | |
| EmilienM | mriedem: do you have a CI job that test Nova API deployed with SSL? | 20:21 |
| *** ducttape_ has joined #openstack-nova | 20:21 | |
| EmilienM | I couldn't find one | 20:21 |
| EmilienM | gate-grenade-dsvm-neutron-multinode-ubuntu-xenial seems to deploy non-ssl | 20:22 |
| *** tbachman has quit IRC | 20:22 | |
| mriedem | EmilienM: dane-fichter and the john hopkins gang was working on creating a security-specific dsvm job that tests with that plus all of the other security features like signed images and using barbican as the key manager | 20:22 |
| mriedem | but that's not done, and honestly i don't know the status of it right now | 20:22 |
| *** tbachman has joined #openstack-nova | 20:22 | |
| EmilienM | ok | 20:22 |
| EmilienM | because I found a bug, I'll probably report it | 20:22 |
| mriedem | ok | 20:23 |
| mriedem | EmilienM: so on your multinode nonha job, is that running in the same nodepool as the rest of infra or a special subset? | 20:25 |
| EmilienM | mriedem: same | 20:26 |
| *** jheroux has quit IRC | 20:26 | |
| *** esmiurium has quit IRC | 20:26 | |
| mriedem | ok | 20:26 |
| EmilienM | mriedem: I really see a value of using TripleO jobs here. We are testing SSL, IPv6, (WSGI but will be removed today), and all of that for free | 20:26 |
| *** READ10 has quit IRC | 20:27 | |
| EmilienM | mriedem: is there something that you don't like in this idea? | 20:29 |
| EmilienM | I might be wrong but I see some resistance that we didn't have with other projects | 20:30 |
| *** dtp is now known as dtp-afk | 20:30 | |
| mriedem | EmilienM: well i'm a bit distracted with other things right now | 20:31 |
| mriedem | but also, | 20:31 |
| mriedem | and i know this isn't voting right now, | 20:31 |
| mriedem | but right now, devs have a reasonable shot at making changes in devstack to get things through nova that depend on devstack, but that learning curve gets higher if they have to touch devestack-gate or project-config or grenade, | 20:31 |
| mriedem | adding this means to get things in nova done, they now have to learn a whole new set of tools, | 20:32 |
| mriedem | i for one don't know puppet or ruby | 20:32 |
| mriedem | i've learned to monkey my way through devstack and it's bashtasticness over the last 4 years that i'm able to sort of know what i'm doing, | 20:32 |
| mriedem | until sdague hits me and tells me i'm wrong | 20:32 |
| EmilienM | I don't think we asked Nova team to learn about TripleO or did we? | 20:32 |
| EmilienM | what we ask is to tell us when that job is failing | 20:33 |
| EmilienM | because we think it can provide useful feedback | 20:33 |
| mriedem | no, but if we have changes that require deployment tooling changes to move forward, we can do that in devstack today | 20:33 |
| mriedem | yeah i mean there will be a lot of hand-holding at the least | 20:33 |
| mriedem | and it opens a door for the horde of other deployment projects (kolla, ansible, whatever else since fuel/chef/salt are dead i think) | 20:34 |
| superdan | mriedem: are you cool with getting some stuff like this into rc2? https://review.openstack.org/#/c/428824/ | 20:34 |
| superdan | or do you want to hold off | 20:34 |
| superdan | we could backport too, but I think it should be in the ocata packages | 20:34 |
| mriedem | superdan: i think docs are fine | 20:34 |
| superdan | okay, agree | 20:34 |
| mriedem | EmilienM: anyway you asked so those are the considerations, and which is why i'm not jumping on this right away | 20:35 |
| mriedem | plus it's been a hectic week and i'm getting burned out | 20:36 |
| EmilienM | mriedem: ok, let's talk about it next time you have free cycles. | 20:37 |
| *** ducttape_ has quit IRC | 20:40 | |
| *** gszasz has quit IRC | 20:41 | |
| *** karimb has quit IRC | 20:42 | |
| *** karimb has joined #openstack-nova | 20:43 | |
| *** dimtruck is now known as zz_dimtruck | 20:46 | |
| *** bvanhav has quit IRC | 20:47 | |
| *** Apoorva has quit IRC | 20:47 | |
| *** karimb has quit IRC | 20:48 | |
| *** kristian__ has joined #openstack-nova | 20:50 | |
| *** eharney has joined #openstack-nova | 20:51 | |
| *** burt has joined #openstack-nova | 20:53 | |
| *** kristian__ has quit IRC | 20:54 | |
| *** gouthamr has quit IRC | 20:59 | |
| *** cmartin has joined #openstack-nova | 21:04 | |
| *** gouthamr has joined #openstack-nova | 21:05 | |
| cmartin | Hey quick question about the upcoming PTG in Atlanta, roughly what time will the conference hours be? particularly on friday? I am trying to book my flight home after the conference ends on friday but am unsure what times I can select for the flight | 21:05 |
| *** xinli has quit IRC | 21:06 | |
| mriedem | i assume things start at 9am? | 21:06 |
| mriedem | dhellmann: ^? | 21:06 |
| mriedem | cmartin: by friday afternoon (after lunch) people are usually pretty spent on a week like this, | 21:06 |
| mriedem | so not a ton happens on friday afternoon | 21:06 |
| dhellmann | that's my assumption, too, but I don't know for sure. ttx would, but he's offline today due to travel. | 21:07 |
| *** tbachman has quit IRC | 21:07 | |
| cmartin | mriedem: I see, so a flight in the afternoon should be fine? | 21:07 |
| mriedem | cmartin: yup | 21:07 |
| superdan | cmartin: you'll be begging to leave by then | 21:07 |
| cmartin | mriedem: thanks that is what I needed to know! | 21:07 |
| cmartin | mriedem: lol we will see, but it will be nice to go back home and sleep in my own bed | 21:08 |
| *** Apoorva has joined #openstack-nova | 21:08 | |
| *** catintheroof has quit IRC | 21:11 | |
| *** kristian__ has joined #openstack-nova | 21:14 | |
| *** satyar has quit IRC | 21:14 | |
| *** tbachman has joined #openstack-nova | 21:16 | |
| EmilienM | mriedem: so when I disable wsgi, multinode job is passing :-) | 21:18 |
| EmilienM | mriedem: I'll check baremetal jobs how it goes | 21:18 |
| mriedem | EmilienM: yeah because of the service version thing | 21:18 |
| EmilienM | awesome | 21:18 |
| *** slaweq has joined #openstack-nova | 21:19 | |
| *** kristian__ has quit IRC | 21:19 | |
| cburgess | @superdan lol | 21:22 |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: resolved final todos in libvirt conf https://review.openstack.org/429042 | 21:22 |
| *** smatzek has quit IRC | 21:23 | |
| superdan | cburgess: what? | 21:23 |
| cburgess | superdan It was just funny your statement about begging to leave. I don't know why. Probably years of seeing the look on your face by Friday afternoons. | 21:24 |
| *** amotoki has joined #openstack-nova | 21:24 | |
| superdan | cburgess: oh, heh, yeah | 21:24 |
| cburgess | superdan I was trying to come up with something witty about Dan wanting to stab people by Friday afternoon but I failed. | 21:24 |
| cburgess | With regard to PTG, are we planning any time of team dinner like we have done in the past? | 21:25 |
| superdan | cburgess: heh | 21:25 |
| *** crushil has joined #openstack-nova | 21:26 | |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: deprecate key_manager.fixed _key opt https://review.openstack.org/428992 | 21:26 |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: WIP conf: remove invalid TODOs https://review.openstack.org/428811 | 21:27 |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: Deprecate two more nova-net options https://review.openstack.org/412390 | 21:28 |
| *** amotoki has quit IRC | 21:28 | |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: Resolve TODOs in 'database' https://review.openstack.org/393695 | 21:29 |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: add min parameter to scheduler opts https://review.openstack.org/428355 | 21:30 |
| *** burgerk_ has joined #openstack-nova | 21:30 | |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: resolved TODOs in compute configuration https://review.openstack.org/426793 | 21:30 |
| mriedem | cburgess: like team dinner from the midcycles? | 21:30 |
| cburgess | Yeah | 21:31 |
| mriedem | hmm, the midcycles are usually somewhere where the host person knows where to go... | 21:31 |
| cburgess | Granted those are semi planned anyways. | 21:31 |
| *** cmartin has quit IRC | 21:31 | |
| mriedem | right | 21:31 |
| melwitt | superdan: based on dtp-afk's questions, I'm wondering what the right thing to do for cells v1 is at this point. when they map hosts, how do they pick which cell database to pull them from? | 21:31 |
| mriedem | it's always like last minute | 21:31 |
| cburgess | Yeah | 21:31 |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: mark glance.api_servers opt as required https://review.openstack.org/428241 | 21:31 |
| mriedem | cburgess: i'd like to do something like that at least one night... | 21:31 |
| cburgess | There are lots of places options in and around ATL. Lack of transportation might limit us though. | 21:31 |
| melwitt | superdan: assuming we're not having them create one cell per cells v1 cell yet | 21:31 |
| cburgess | mriedem Yeah its been good the times I've been able to go. | 21:32 |
| mriedem | i wonder if there is anywhere downtown near the hotels people are staying at | 21:32 |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: config options: remove libvirt imagecache https://review.openstack.org/378497 | 21:32 |
| openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: remove deprecated image url options https://review.openstack.org/397800 | 21:32 |
| mriedem | i'll throw it on the etherpad so i don't forget about it | 21:32 |
| cburgess | ATL does have decent subway (MARTA) but it can still be a pain. I can ask around amongst some friends and see if they know of anything (grew up in ATL). | 21:32 |
| *** burgerk has quit IRC | 21:33 | |
| mriedem | isn't dhellmann and/or markmcclain in ATL? | 21:33 |
| mriedem | i have to assume they have a backyard... | 21:33 |
| cburgess | markmcclain Is yeah. I'm not sure about dhellmann, but sounds right. | 21:33 |
| *** r-daneel has joined #openstack-nova | 21:33 | |
| mriedem | and southern charm | 21:33 |
| mriedem | / hospitality | 21:34 |
| cburgess | LOL | 21:34 |
| *** Jeffrey4l has quit IRC | 21:35 | |
| superdan | melwitt: they use their nova-cell.conf file, per cell yeah | 21:35 |
| *** crushil has quit IRC | 21:35 | |
| *** Jeffrey4l has joined #openstack-nova | 21:35 | |
| superdan | melwitt: we bypass the lookups now, but that's what we were doing before and it worked other than that the timing of noticing the replicated statuses was then off | 21:36 |
| superdan | melwitt: is this on the step by step thing? | 21:36 |
| melwitt | superdan: okay, so at this stage we do want to tell them to go ahead and create a cell v2 per cell v1 cell | 21:36 |
| superdan | melwitt: yeah and that's what we do in devstack | 21:36 |
| *** liverpooler has quit IRC | 21:36 | |
| superdan | otherwise the work to convert later will be messy | 21:36 |
| melwitt | superdan: no but I'm asking to know if I should add it. sounds like I do need to. I had been thinking we were doing just a cheat of a single cell | 21:36 |
| melwitt | yeah. okay | 21:37 |
| superdan | melwitt: you mean from the api cell's database? | 21:37 |
| superdan | there's no reason to do that since they'll all be wrong when they want to go use them | 21:37 |
| melwitt | superdan: I guess so. but that wouldn't make sense because there's no compute hosts in there | 21:37 |
| melwitt | I hadn't thought it through | 21:38 |
| superdan | yeah | 21:38 |
| *** nic has quit IRC | 21:38 | |
| melwitt | okay, I'll add a new section for cells v1 people, I think that would be most clear | 21:39 |
| superdan | melwitt: okay | 21:39 |
| melwitt | thanks | 21:39 |
| superdan | melwitt: maybe in another patch so we have a better chance of getting this in while we noodle over the cellsv1 stuff? | 21:39 |
| melwitt | superdan: okay, if you think that's best | 21:40 |
| superdan | whatever you want | 21:40 |
| *** nic has joined #openstack-nova | 21:40 | |
| melwitt | I'm not opinionated about it. I hadn't thought of doing a follow up | 21:41 |
| superdan | I'd just like that bit to be in rc2 and I wouldn't want to miss that if we're messing around with cellsv1 stuff that's all | 21:41 |
| melwitt | yeah | 21:41 |
| superdan | the mainline bit I mean | 21:42 |
| melwitt | right | 21:42 |
| *** dtp-afk is now known as dtp | 21:43 | |
| *** burgerk_ has quit IRC | 21:45 | |
| *** salv-orlando has quit IRC | 21:45 | |
| melwitt | superdan: I'm trying to think of how the scheduling works with where we're at now. we're not yet pulling compute hosts from all cells or anything for choosing a host ... and cells v1 doesn't care about that since it chooses a cell first. so I guess that's how I had been thinking things could be gotten away with by setting up only one cell for cells v1 | 21:46 |
| melwitt | but doing that would be like, arbitrarily picking a cell to convert over and then later you'd have to do the rest, so not a great idea anyway | 21:46 |
| superdan | melwitt: yeah, but I think we're okay because we choose a cell first, then go to a nova-scheduler that only knows about the lower levels | 21:47 |
| superdan | no, I don't think we'll be able to do that unfortunatel | 21:47 |
| superdan | not from v1 to v2 anyway | 21:47 |
| mriedem | hmm, if you have a cell mapping per cell v1 cell, and the scheduler was checking all cell mappings, then it could hit something in another cell v1 cell couldn't it? | 21:47 |
| superdan | no | 21:47 |
| mriedem | if that were the case, i guess we'd just do if CONF.cells.enabled | 21:47 |
| superdan | because scheduler is at the cell level | 21:47 |
| superdan | and there's a cells scheduler that hasn't been touched in ages that knows nothing of this stuff | 21:47 |
| *** lpetrut has quit IRC | 21:47 | |
| mriedem | "scheduler is at the cell level" | 21:48 |
| mriedem | cells v1? - there are multiple levels, can you clarify? | 21:48 |
| superdan | if cells.enabled is false in the api cell, then it will start rpc'ing to the global schedule | 21:48 |
| melwitt | superdan: don't think we'll be able to what? didn't catch what part you were referring to | 21:48 |
| mriedem | i thought the cells scheduler was at the top level cell in cells v1? | 21:48 |
| superdan | mriedem: yeah, in cellsv1 the scheduler is at the lower levels | 21:48 |
| superdan | mriedem: no there's a special scheduler up there, regular scheduler down below | 21:48 |
| *** raunak has joined #openstack-nova | 21:48 | |
| mriedem | right | 21:48 |
| superdan | melwitt: won't be able to have a cell-by-cell conversion from v1 to v2 | 21:49 |
| superdan | melwitt: but it'll be more of a "turning stuff off" kind of procedure I think, if we get the mappings in place ahead of time | 21:49 |
| superdan | these are not friday afternoon questions | 21:49 |
| superdan | ask me shit like "is it beer o clock yet" or something | 21:50 |
| melwitt | superdan: heh, sorry. I blame dtp for asking me to think about stuff by asking about cells v1 step-by-step | 21:50 |
| mriedem | scheduler is global in cells v2 world right? | 21:50 |
| * superdan scowls at dtp | 21:50 | |
| superdan | mriedem: yeah | 21:50 |
| mriedem | so computes in multiple cells v2 cells are reporting into placement, | 21:50 |
| mriedem | scheduler asks placement for an RP, gets one, sends that to conductor, | 21:50 |
| mriedem | which is mapped to a cell via hostmappings, | 21:50 |
| mriedem | and then conductor creates the instance in that cell and builds it there | 21:50 |
| superdan | with pure v2 yes, but with v1, | 21:51 |
| superdan | that all happens inside a cellv1 cell | 21:51 |
| melwitt | superdan: I was thinking it doesn't matter if they set up all their cells now (technically) and that they could do them later. but I think that would make it more confusing | 21:51 |
| mriedem | what's preventing the low-level filter scheduler in cells v1 not picking an RP from another v1 cell? | 21:51 |
| superdan | mriedem: because they don't have visibility of each other? they'll all need their own placement, just like they need their own api | 21:52 |
| *** kristian__ has joined #openstack-nova | 21:52 | |
| mriedem | ok i thought placement was still global in cells v1 | 21:52 |
| superdan | mriedem: otherwise we have to teach placement about cellsv1, which would be ungood I think | 21:52 |
| mriedem | right | 21:52 |
| mriedem | we don't want cells v1 specific garbage in placement | 21:52 |
| mriedem | maybe we want a note of that somewhere so when i ask it 5 more times in 12 months, you can point me | 21:53 |
| superdan | placement is kinda part of the regular scheduler, which is replicated per v1 cell, which means placement is too | 21:53 |
| superdan | melwitt: are you doing a general "what to make of the v1 mess" section? you could put that ^ in there if so | 21:53 |
| melwitt | superdan: I'm doing a step-by-step for cells v1 peeps. what's this note about, just telling why not to worry about scheduling working right? | 21:54 |
| mriedem | melwitt: that placement is per cells v1 cell | 21:55 |
| mriedem | not global in cells v1 | 21:55 |
| superdan | that ^ | 21:55 |
| melwitt | oh ... hm | 21:55 |
| mriedem | in other words, placement only has knowledge of the computes in the cells v1 cell that it's running in | 21:55 |
| superdan | probably needs to go into some placement doc | 21:55 |
| mriedem | yeah, which we have | 21:55 |
| melwitt | yeah, I dunno anything about how to deploy placement for cells v1 | 21:56 |
| mriedem | just like nova-api | 21:56 |
| mriedem | except in the child cell | 21:56 |
| superdan | mriedem: I'll add it to placement.rst | 21:56 |
| mriedem | ok http://docs.openstack.org/developer/nova/placement.html | 21:56 |
| melwitt | so more like nova-scheduler then? | 21:56 |
| mriedem | not sure what the best place is | 21:56 |
| *** kristian__ has quit IRC | 21:56 | |
| mriedem | melwitt: yeah | 21:56 |
| superdan | melwitt: right, goes with scheduler | 21:57 |
| mriedem | don't we have nova-scheduler in the top cell too? | 21:57 |
| melwitt | k, that makes sense | 21:57 |
| mriedem | or is that the cells service? | 21:57 |
| mriedem | with the cell scheduler | 21:57 |
| *** gouthamr has quit IRC | 21:57 | |
| melwitt | it's the cells scheduler, I dunno if it's the same code as nova-scheduler or not | 21:57 |
| mriedem | cells-region or whatever | 21:57 |
| superdan | hmm, you know | 21:57 |
| superdan | because placement is all service-catalog-y, I wonder if people will be able to do that | 21:57 |
| *** tbachman has quit IRC | 21:58 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: Clarify the deployment of placement for cellsv1 users https://review.openstack.org/429056 | 21:59 |
| superdan | mriedem: no it's different I think | 21:59 |
| superdan | nova/cells/scheduler.py | 22:00 |
| melwitt | ah, cool | 22:01 |
| *** edmondsw has quit IRC | 22:04 | |
| *** edmondsw has joined #openstack-nova | 22:05 | |
| *** edmondsw has quit IRC | 22:05 | |
| *** edmondsw has joined #openstack-nova | 22:05 | |
| *** tbachman has joined #openstack-nova | 22:11 | |
| *** dtp is now known as dtp-afk | 22:11 | |
| *** zackf has joined #openstack-nova | 22:13 | |
| *** karimb has joined #openstack-nova | 22:14 | |
| superdan | mriedem: your rc1 todo etherpad says we have until monday to do rc1 but the announce already went out right? | 22:15 |
| mriedem | yeah, i'll fix | 22:15 |
| superdan | okay | 22:15 |
| *** slaweq has quit IRC | 22:16 | |
| *** Sukhdev has joined #openstack-nova | 22:17 | |
| *** karimb has quit IRC | 22:17 | |
| *** sree has joined #openstack-nova | 22:17 | |
| *** sree has quit IRC | 22:21 | |
| owalsh | superdan, mriedem: hola... TL;DR what is the suspected problem with https://review.openstack.org/#/c/427534/? | 22:23 |
| *** smatzek has joined #openstack-nova | 22:24 | |
| *** thorst_ has quit IRC | 22:25 | |
| superdan | owalsh: none that I know of | 22:25 |
| mriedem | owalsh: not totally sure, i dug into the compute logs a bit to see instances not building on ironic nodes because the disk on the nodes was fully consumed by another instance | 22:25 |
| mriedem | which i think those things just get rescheduled, | 22:25 |
| mriedem | but i didn't trace through any failures in real detail | 22:26 |
| superdan | but really, there's no reason for that patch to affect that kind of thing | 22:26 |
| openstackgerrit | John Perkins proposed openstack/nova master: Integrate oslo.config validator https://review.openstack.org/428974 | 22:26 |
| *** breitz has quit IRC | 22:26 | |
| superdan | and before it, no nodes were even registered | 22:26 |
| owalsh | ok, can't see how they could be related... that change just double checks nova has the ironic nodes in the RT before running discover_hosts | 22:26 |
| superdan | yep | 22:27 |
| *** breitz has joined #openstack-nova | 22:27 | |
| owalsh | peeling an onion me thinks... my fix just got us to the next layer | 22:27 |
| superdan | yep | 22:27 |
| superdan | I haven't even heard that it happened more than once | 22:27 |
| superdan | mriedem: did you see a bunch of these failures? | 22:27 |
| *** smatzek has quit IRC | 22:29 | |
| *** salv-orlando has joined #openstack-nova | 22:29 | |
| mriedem | i don't think i actually saw any error messages, just that resources couldn't be claimed on a given node because the disk was fully consumed on that node at the time of the instance build | 22:30 |
| mriedem | like i said, i didn't trace a job failure all the way through, | 22:30 |
| mriedem | was just pointed at a debug log message and was asked what it meant, and i poked around that abit | 22:30 |
| owalsh | ok https://review.openstack.org/428974 couldn't be related I think... | 22:30 |
| melwitt | superdan: sanity check me here but I don't think cells v1 peeps can use the discover_hosts periodic task because their nova-schedulers run inside cells, right? | 22:31 |
| ttx | mriedem: I don't have final hours yet but yes 9am start sounds like a safe bet. Up until 6pm in the afternoon I'd say. Probably wrap up slightly earlier on Friday (5pm or so) | 22:31 |
| mriedem | i also added this because of that debugging https://review.openstack.org/#/c/428985/ | 22:31 |
| mriedem | ttx: ok | 22:31 |
| superdan | melwitt: why not? | 22:31 |
| mriedem | melwitt: it would discover the computes in that cell | 22:31 |
| superdan | right | 22:31 |
| mriedem | which is what we want | 22:31 |
| melwitt | superdan: writes to the api database? okay | 22:31 |
| mriedem | writes the host mappings to the api db | 22:31 |
| melwitt | they just have to configure it per cell | 22:32 |
| mriedem | so do they have the api_database/connection in their nova-cells.conf for the low-level scheduler to do that? | 22:32 |
| mriedem | they're going to have to for any of the nova-manage commands | 22:32 |
| superdan | yeah, | 22:32 |
| mriedem | we should probably point that out somewhere too | 22:32 |
| melwitt | I think they do in that I think mgagne found out by experience that he had to configure it per cell (the api db connection) | 22:32 |
| mriedem | was just going to mention mgagne | 22:33 |
| melwitt | okay, cool. I shall continue my writings | 22:33 |
| superdan | melwitt: or did he just say it worked that way? | 22:33 |
| openstackgerrit | John Perkins proposed openstack/nova master: Integrate oslo.config validator https://review.openstack.org/428974 | 22:33 |
| superdan | melwitt: either way, doesn't matter that much | 22:33 |
| melwitt | superdan: it matters in that the discover_hosts needs for it to be there for to write the host mappings | 22:33 |
| melwitt | right? | 22:33 |
| superdan | if you run the periodic yeah | 22:34 |
| melwitt | okay. I'll mention that in there too. I dunno if it's already required for some other reason and thus moot to mention | 22:34 |
| *** tblakes has quit IRC | 22:36 | |
| *** baoli has quit IRC | 22:37 | |
| *** Sukhdev has quit IRC | 22:38 | |
| *** mvk has joined #openstack-nova | 22:38 | |
| *** owalsh is now known as owalsh-afk | 22:40 | |
| *** ducttape_ has joined #openstack-nova | 22:41 | |
| *** edmondsw has quit IRC | 22:41 | |
| mriedem | ok i'm calling it a day | 22:43 |
| mriedem | talk to you all later | 22:43 |
| mriedem | put some salve on the wounds from the last 2 days | 22:43 |
| melwitt | heh o/ | 22:43 |
| *** mriedem has quit IRC | 22:44 | |
| *** gouthamr has joined #openstack-nova | 22:45 | |
| *** ducttape_ has quit IRC | 22:46 | |
| *** karimb has joined #openstack-nova | 22:52 | |
| *** jaosorior has quit IRC | 22:52 | |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/nova master: Update hacking version https://review.openstack.org/389338 | 22:59 |
| *** vsaienko has quit IRC | 23:00 | |
| *** Swami has quit IRC | 23:01 | |
| *** mdrabe has quit IRC | 23:04 | |
| *** esberglu_ has quit IRC | 23:06 | |
| diana_clarke | superdan: Do you know off the top of your head what you need to do to run the functional MySQL tests locally? Like TestNovaAPIMigrationsWalkMySQL.test_walk_versions. | 23:09 |
| openstackgerrit | melanie witt proposed openstack/nova master: Add Cells V1 --> Cells V2 step-by-step example https://review.openstack.org/429075 | 23:12 |
| *** kfarr has quit IRC | 23:12 | |
| *** dtp-afk is now known as dtp | 23:14 | |
| diana_clarke | like setting OS_TEST_DBAPI_ADMIN_CONNECTION etc, perhaps? | 23:16 |
| superdan | diana_clarke: yeah, some envars to trigger it, IIRC | 23:16 |
| *** burt has quit IRC | 23:17 | |
| *** figleaf is now known as edleafe | 23:17 | |
| diana_clarke | superdan: Okie dokie, I'll try a few more things. Have a great weekend, folks! | 23:18 |
| *** armax has quit IRC | 23:18 | |
| *** panda is now known as panda|Zz | 23:18 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add Cells V1 --> Cells V2 step-by-step example https://review.openstack.org/429075 | 23:21 |
| *** unicell has joined #openstack-nova | 23:24 | |
| *** amotoki has joined #openstack-nova | 23:24 | |
| *** Sukhdev has joined #openstack-nova | 23:25 | |
| *** zz_dimtruck is now known as dimtruck | 23:25 | |
| *** amotoki has quit IRC | 23:29 | |
| *** xyang1 has quit IRC | 23:31 | |
| *** dtp has quit IRC | 23:32 | |
| *** annegentle has quit IRC | 23:35 | |
| *** kristian__ has joined #openstack-nova | 23:42 | |
| *** karimb has quit IRC | 23:45 | |
| *** kristian__ has quit IRC | 23:46 | |
| *** sdague has quit IRC | 23:48 | |
| *** salv-orlando has quit IRC | 23:48 | |
| *** salv-orlando has joined #openstack-nova | 23:49 | |
| *** paw has quit IRC | 23:51 | |
| *** Swami has joined #openstack-nova | 23:51 | |
| *** armax has joined #openstack-nova | 23:54 | |
| *** salv-orlando has quit IRC | 23:54 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add Cells V1 --> Cells V2 step-by-step example https://review.openstack.org/429075 | 23:56 |
| openstackgerrit | Diana Clarke proposed openstack/nova master: vomiting https://review.openstack.org/429114 | 23:56 |
| *** salv-orlando has joined #openstack-nova | 23:57 | |
| *** ducttape_ has joined #openstack-nova | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!