Thursday, 2018-08-23

openstackgerritSam Morrison proposed openstack/nova master: Allow ability for non admin users to use all filters on server list.  https://review.openstack.org/52655800:04
*** mriedem has quit IRC00:15
*** mriedem has joined #openstack-placement00:19
mriedemcdent: pep8 gonna getcha00:19
cdentargh00:19
cdentbecause I'm an idiot, I have managed to make the functional tests pass in the EdLeafe repo: edleafe, efried00:29
cdentwill annotate a pull request or something, but it is not for merging00:30
mriedemso i'd be +2 on the reshaper api00:35
mriedempep8 import order fix needed obviously00:35
cdentmriedem: if you're gung ho, please feel free00:35
mriedemi'll +1/+2 once that's done and then it can stick through rebases00:35
mriedemok00:35
cdentI'm up way past may bedtime00:36
openstackgerritMatt Riedemann proposed openstack/nova master: [placement] Add /reshaper handler for POST  https://review.openstack.org/57692700:36
mriedemyup00:36
mriedem+200:36
mriedemassuming efried will rebase in the morning00:37
*** tetsuro_ has quit IRC00:40
*** Nel1x has joined #openstack-placement00:45
cdentefried, edleafe: for whenever you wanna see a huge diff: https://github.com/EdLeafe/placement/pull/100:48
cdentgoodnight all00:51
*** cdent has quit IRC00:51
*** tetsuro has joined #openstack-placement00:54
openstackgerritTakashi NATSUME proposed openstack/nova master: Adds view builders for keypairs controller  https://review.openstack.org/34728901:12
*** mriedem has quit IRC01:48
*** deepak_mourya__ has joined #openstack-placement02:21
*** alex_xu has joined #openstack-placement02:27
*** dansmith has joined #openstack-placement02:28
*** deepak_mourya_ has quit IRC02:29
*** htimsnad has quit IRC02:29
*** deepak_mourya__ is now known as deepak_mourya_02:29
openstackgerritzhufl proposed openstack/nova master: Blacklist test_create_server_with_tags for nova-cells-v1  https://review.openstack.org/59539902:37
*** Nel1x has quit IRC02:53
openstackgerritChen proposed openstack/nova master: Normalize dashless 'resource provider create' uuid  https://review.openstack.org/56719103:24
*** nicolasbock has quit IRC03:44
openstackgerritChen proposed openstack/nova master: Fix create_resource_provider docstring  https://review.openstack.org/59545304:12
openstackgerritmelanie witt proposed openstack/nova master: Correct the release notes related to nova-consoleauth  https://review.openstack.org/59545504:57
*** lei-zh has joined #openstack-placement05:40
*** lei-zh1 has joined #openstack-placement05:55
*** lei-zh has quit IRC05:58
openstackgerritzhufl proposed openstack/nova master: Blacklist test_create_server_with_tags for nova-cells-v1  https://review.openstack.org/59539906:50
*** e0ne has joined #openstack-placement07:13
*** lei-zh1 has quit IRC07:22
*** lei-zh1 has joined #openstack-placement07:32
*** tssurya has joined #openstack-placement07:39
*** cdent has joined #openstack-placement08:20
openstackgerritZhenyu Zheng proposed openstack/nova master: Only lock when race condition happens in context.set_target_cell  https://review.openstack.org/59553008:25
*** sean-k-mooney has joined #openstack-placement08:46
*** tetsuro has quit IRC08:47
openstackgerritZhenyu Zheng proposed openstack/nova master: Only lock when race condition happens in context.set_target_cell  https://review.openstack.org/59553009:21
*** lei-zh1 has quit IRC09:31
openstackgerritChris Dent proposed openstack/nova master: [placement] Add functional test to verify presence of policy  https://review.openstack.org/59555909:39
cdentgibi, efried: you prolly like that ^09:40
*** tetsuro has joined #openstack-placement10:30
*** tetsuro has quit IRC10:40
*** tetsuro_ has joined #openstack-placement10:40
*** tetsuro has joined #openstack-placement10:50
*** tetsuro_ has quit IRC10:50
*** tetsuro has quit IRC10:51
*** tetsuro_ has joined #openstack-placement10:51
gibicdent: looking10:53
*** tetsuro has joined #openstack-placement10:54
*** tetsuro_ has quit IRC10:54
cdentdansmith: have you seen https://review.openstack.org/#/c/582899/ ? It's got a "mostly I'd like Dan Smith to review this patch" in it from jay10:59
cdentgibi, efried : this looks like an easy win: https://review.openstack.org/#/c/588470/ it already has a +1 from efried, who was looking for a bit of explanation, I guess, before doing a +211:01
gibicdent: the policy functional test looks good to me11:03
gibicdent: also +2 on https://review.openstack.org/#/c/588470/ , but I let efried to send it through11:06
cdentcool, thanks (on both)11:06
cdentgibi: another if you're in the mood: this fixes a race in aggregate management: https://review.openstack.org/#/c/592654/11:09
*** nicolasbock has joined #openstack-placement11:22
openstackgerritMoshe Levi proposed openstack/nova master: libvirt: set vfio driver in interface hostdev  https://review.openstack.org/59559211:33
*** mriedem has joined #openstack-placement11:33
*** jaypipes has joined #openstack-placement11:42
cdentmriedem: https://review.openstack.org/595559 is an auto-confirm policy is there tester11:46
cdentif you'd like to have a look11:46
mriedemwill do11:47
jaypipesgood morning y'all.11:48
* cdent waves11:48
cdentare you back back jaypipes or only a little back?11:49
jaypipescdent: fully back.11:49
gibijaypipes: welcome back11:50
jaypipesgibi: Kösz11:51
gibijaypipes: :)11:56
*** dims_ is now known as dims12:37
openstackgerritMerged openstack/nova master: [placement] Regex consts for placement schema  https://review.openstack.org/59186312:45
openstackgerritmelanie witt proposed openstack/nova master: Correct the release notes related to nova-consoleauth  https://review.openstack.org/59545512:54
openstackgerritMatt Riedemann proposed openstack/nova master: Merge keypair extension response into server view builder  https://review.openstack.org/58474813:06
openstackgerritMatt Riedemann proposed openstack/nova master: Merge server usage extension response into server view builder  https://review.openstack.org/58526213:06
openstackgerritMatt Riedemann proposed openstack/nova master: Merge security groups extension response into server view builder  https://review.openstack.org/58547513:06
openstackgerritMatt Riedemann proposed openstack/nova master: Merge extended_status extension response into server view builder  https://review.openstack.org/59209213:06
edleafecdent: Besides the sheer size, why is that PR a DNM?13:20
cdentedleafe: because it does things like cp -r nova/conf placement/conf without trimming13:20
edleafecdent: ok. Haven't had time to fully go over it13:21
edleafeExternal monitor died13:21
cdentit's essentially analysis to find out what matters so we can then do what matters, cleanly13:21
gibijaypipes, cdent: I think I found a possible race while I read https://review.openstack.org/#/c/592654  It is not super related to the patch itself but I wanted to check with you before I +W that13:21
cdentwhich seems more friendly to our descendents13:21
cdentedleafe: you have the worst luck with hardware13:22
cdentgibi: you have the best luck with finding bugs13:22
edleafegibi: wanna trade? :)13:22
gibiedleafe: I don't use any external monitor, so I wouldn't loose to much in the trade ;)13:23
efriedcdent, gibi: +A on the trivial refactor. +2 on the policy check, but want to let mriedem +A it.13:34
efriedjaypipes: Welcome back.13:34
efriedjaypipes: You have quite a bit of interesting reading waiting for you :)13:37
cdentedleafe, efried : made it go: curl http://ds1:8000/ \n {"versions": [{"id": "v1.0", "max_version": "1.29", "min_version": "1.0", "status": "CURRENT", "links": [{"rel": "self", "href": ""}]}]}13:40
efriedwoot!13:41
edleafeexcellent!13:41
efriedI'm gonna rebase the series now cdent, mriedem, jaypipes - cool?13:44
cdentefried: reshaper? yes please and thank you13:44
cdentedleafe, efried : all unit tests, all functional tests, all placecat tests are passing13:45
* edleafe bows down to cdent 13:45
openstackgerritEric Fried proposed openstack/nova master: reshaper: Look up provider if not in inventories  https://review.openstack.org/58503313:45
openstackgerritEric Fried proposed openstack/nova master: Make get_allocations_for_resource_provider raise  https://review.openstack.org/58459813:45
openstackgerritEric Fried proposed openstack/nova master: Report client: Real get_allocs_for_consumer  https://review.openstack.org/58459913:45
openstackgerritEric Fried proposed openstack/nova master: Report client: get_allocations_for_provider_tree  https://review.openstack.org/58464813:45
openstackgerritEric Fried proposed openstack/nova master: Report client: _reshape helper, placement min bump  https://review.openstack.org/58503413:45
openstackgerritEric Fried proposed openstack/nova master: Report client: update_from_provider_tree w/reshape  https://review.openstack.org/58504913:45
openstackgerritEric Fried proposed openstack/nova master: Compute: Handle reshaped provider trees  https://review.openstack.org/57623613:45
efriedthat's awesome cdent13:46
cdentsome major gaps to fill in terms of cleanliness and making things go properly, but the things that need to happen are pretty clear13:46
efriedcdent: I'll ask again: how clean/perfect do we need to be before seeding the official repo?13:46
cdentefried: I'm not too concerned about perfection. what I'm more worried about is inconsequential junk being in the way13:47
edleafeIf the tests are passing, I think we should do the rest of the work in the official repo13:47
efried(efried, why the hurry? I'm glad you asked. So I don't have to touch github. No other reason.)13:47
efriedI would be fine with ^13:47
mriedemnote the nova meeting is in 13 minutes so it would be nice to summarize the activity on the separate git repo for those of us not following along that closely13:47
mriedemi'm mostly interested in preserving git history13:47
cdentI'd rather that we never merge my current patches because they are ... chunky13:47
edleafemriedem: yeah, that was what involved some work. But it's there13:48
mriedemwhich it sounds like you've tried to do and then are having to push deltas on top?13:48
cdentif we can pick and choose (as humans) from the bits we like and make something cleaner I think that would be better13:48
edleafecdent: what do you suggest to deal with your chunkiness?13:48
mriedemnicely worded13:49
cdentreason 1 for not using what we've alread done is that some of the files in it had to be copied in raw from a nova repo, so do it not have history. we need to resolve that to make a better filter13:49
cdentand then redo the work that I've done (in a more clean fashion) on top of that13:49
cdentso with regard to my chunkiness it would be useful for people to gaze upon it and see which parts they like13:50
cdentand understand it fully13:50
cdentso that we can then parcel it out to all who are interested13:50
mriedemis it not possible to cherry-pick stuff if there are deltas?13:50
cdent(i'm not picky)13:50
cdentmriedem: that would be ideal if the base was good, but the base is not good13:51
cdentif we make a good base we can do some interactive cherry picking from my branches13:51
cdentbecasue quite a lot off stuff is just renaming imports and mocks13:51
mriedemis this because the dir structure needed to move up to /placement instead of /nova?13:51
cdentif people want, I can just do all that, but I was hoping for at least efried and edleafe to be involved so there's more breadth of understanding of the process involved (not just the changes themselves)13:52
cdentmriedem: yes, pretty much13:52
cdentbut there's also some decsion that edleafe made in the moving things around that we might want to discussion (edleafe see the etherpad about the 'api' package and whether we want that)13:53
edleafecdent: I can add the files you copied to the extraction script, and re-run it13:53
cdentedleafe: that woul be an excellent start, but if we can first figure out which conf bits we actually need (instead of just saying all of them) that might be tidier (it doesn't _matter_, but is nice)13:53
edleafeyeah, I saw that. I can redo the directory shuffling13:54
edleafeIt doesn'13:54
edleafeIt doesn't mater to me if that is done in the private repo or on the openstack repo13:54
edleafe(the conf bits)13:55
cdentanother open issue (that I'm hacking around right now with stuff from placedock) is db sync13:56
cdentbut we can also fix that later if we like13:56
edleafeI think that stuff like the db sync is fine to do in the openstack repo13:56
efriedgibi: Re https://review.openstack.org/#/c/592654/ are you holding +W until a test/bug has been written for the race condition? (Assume it would be a separate patch regardless)13:56
edleafeIt's part of the process13:56
*** takashin has joined #openstack-placement13:57
gibiefried: I wanted to check with Jay about that race before I +A but you are right I can file a bug instead13:58
gibiefried: +A-d13:59
efriedcool, thx14:00
jaypipessorry, was at breakfast. reading back.14:02
efriedgibi: Also, I think we're ready to land https://review.openstack.org/#/c/590041/14:02
gibiefried: added to my list14:03
efriedthx14:03
cdentedleafe, efried : https://github.com/EdLeafe/placement/pull/2 has the latest work.14:06
efriedack14:06
openstackgerritMerged openstack/nova master: Merge extended server attributes extension response  https://review.openstack.org/58459014:06
edleafecdent: ack. Will take a look soon14:06
efriedcdent: Attending nova mtg?14:06
cdentyeah, i'm there14:06
jaypipesgibi: addressed your concerns in the single-shot patch.14:07
jaypipesefried: I've read most of it already.14:07
jaypipesefried: just haven't had a chance to respond to gibi's consumer generation ML post.14:08
gibijaypipes: how _increment_provider_generation can ensure that the generation is correct my case if it doesn't get the expected consumer generation that was in the request?14:09
jaypipesgibi: only one thread would ever be able to update generation. the next thread that attempts to update the generation will get a failure.14:13
gibijaypipes: ohh I see. the RP is passed as a parameter and that RP is read from the db _before_ the generation check was doen in the handler14:13
jaypipesright14:13
jaypipesgibi: this is how all the generation increment code works (for provider and consumer)14:13
gibijaypipes: thanks. I think I got it now. The race I envisioned does not exist14:14
jaypipesgibi: np14:16
openstackgerritStephen Finucane proposed openstack/nova master: tests: Create functional libvirt test base class  https://review.openstack.org/40705514:25
openstackgerritStephen Finucane proposed openstack/nova master: tests: Move mocking to setUp  https://review.openstack.org/59580214:25
*** takashin has left #openstack-placement15:01
openstackgerritBalazs Gibizer proposed openstack/nova master: Remove noisy DEBUG log  https://review.openstack.org/59581015:02
openstackgerritStephen Finucane proposed openstack/nova master: tests: Move mocking to setUp  https://review.openstack.org/59580215:09
openstackgerritStephen Finucane proposed openstack/nova master: tests: Create functional libvirt test base class  https://review.openstack.org/40705515:09
openstackgerritMerged openstack/nova master: Fix nits in resource_provider.py  https://review.openstack.org/58847015:12
openstackgerritChen proposed openstack/nova master: Fix create_resource_provider docstring  https://review.openstack.org/59545315:31
*** e0ne has quit IRC15:51
*** mriedem is now known as mriedem_sudsy16:04
*** ttsiouts has joined #openstack-placement16:08
dims looks like the repo is fully pruned and has just commits for placement - nice start - https://github.com/EdLeafe/placement/graphs/contributors16:08
openstackgerritMerged openstack/nova master: placement: use single-shot INSERT/DELETE agg  https://review.openstack.org/59265416:08
*** mriedem_sudsy is now known as mriedem16:26
*** ttsiouts has quit IRC16:30
*** ttsiouts has joined #openstack-placement16:30
*** ttsiouts has quit IRC16:35
*** sean-k-mooney has quit IRC16:36
openstackgerritDan Smith proposed openstack/nova master: Make instance_list perform per-cell batching  https://review.openstack.org/59313117:04
openstackgerritDan Smith proposed openstack/nova master: Record cell success/failure/timeout in CrossCellLister  https://review.openstack.org/59426517:04
openstackgerritDan Smith proposed openstack/nova master: Optimize global marker re-lookup in multi_cell_list  https://review.openstack.org/59457717:04
*** mriedem is now known as mriedem_away17:36
openstackgerritKen'ichi Ohmichi proposed openstack/nova master: tests: Create functional libvirt test base class  https://review.openstack.org/40705517:37
*** tssurya has quit IRC17:52
efriedjaypipes: Can we talk about https://review.openstack.org/#/c/585049/12/nova/scheduler/client/report.py@1517 ?18:07
jaypipesefried: sure18:08
efriedjaypipes: Cause I'm still not understanding what you'd like to see changed.18:08
jaypipesefried: I haven't seen the latest revision on that patch so if you've changed stuff, lemme look at that first.18:09
efriedjaypipes: no, haven't changed anything, just rebases.18:09
efriedoh, yeah, I've changed things since PS12, but not that thing.18:10
jaypipesefried: ack, ok. so my point is that instead of invalidating cache entries proactively like you're doing there (and re-fetching the new provider generations) that we only re-fetch reactively when needed.18:10
efriedSo wrap each of the calls at https://review.openstack.org/#/c/585049/12/nova/scheduler/client/report.py@1651 through 1655 in a retry-if-409?18:11
jaypipesefried: in other words, have a single code path that, when a 409 is seen that mentions a provider generation mismatch, then we re-fetch the current state of the provider.18:12
efriedOkay, couple problems I have with that.18:12
efriedForemost being that it is not *always* appropriate to refetch the current state of the provider.18:13
efriedI would even contend that it is barely appropriate in the reshape flow, and we could only get away with it there because we're supposed to be doing this with the world shut down18:13
efriedBecause I'm replacing aggs/traits with what I have in my hand. And in a normal periodic flow, if the generation changed from under me because some other entity changed aggs/traits since the last periodic, I don't want to just refetch and overwrite.18:15
efried(I've said this before, but in case I've managed not to say it to you, this is why I was pulling for reshaper to return a payload with the new generations)18:16
efriedjaypipes: The other problem I have, less serious but still bugs me, is doubling the number of API calls. Especially when I *know* I need to do the refetch.18:17
*** mriedem_away is now known as mriedem18:17
jaypipesefried: even if the reshaper returned provider generations that doesn't mean another thread could not have changed things in between that time and the next time you went to update something on that provider, though, right?18:17
efriedCorrect, which would fail in the path below, which is what we would want.18:18
efriedAnd which would get resolved the next time around18:18
efriedPerhaps you're suggesting we do that - just remove the refetch entirely, let the thing below fail, and count on catching it the next time around (which would be on real compute startup, presumably).18:19
jaypipesefried: yes, that's exactly what I'm suggesting. sorry if I have not been clear.18:19
efriedWhich seems not good UX to me, because we're *guaranteed* that the operation will not complete properly for a large subset of uses.18:19
jaypipesefried: sorry, not on restart of the copmute service or startup.18:20
jaypipesefried: I'm just saying that instead of trying to proactively do cache invalidation that instead we just react whenever a 409 comes up.18:20
efried"just" makes it sound like that's easy to think about and do. It's not.18:21
efriedHowever18:21
jaypipesefried: I've just seen numerous bugs pop up when developers try to handle caches like this instead of just reacting to an expiry event (which is basically what the 409 on generation conflict is)18:22
efriedit's possible that the @retrying decorator on _update would do it for us18:22
melwittI was about to ask that, if the usual retry mechanism would kick in in this case as well18:22
melwittI don't know much about it but I had thought there was some auto-retry in the case of 409 elsewhere in general18:23
jaypipesmelwitt: we don't have a decorator that specifically catches the provider generation conflict, unless I'm mistaken.18:23
melwitt*elsewhere in the placement api18:23
jaypipesack18:23
efriedadded pretty recently with I3c5fbb18297db71e682fcddb5bf4536595d9238318:23
efriedjaypipes: ^18:23
melwittokay, yeah I thought I had seen it somewhere18:24
efriedhttps://review.openstack.org/#/c/556669/18:24
efriedso thinking this through, what *might* happen if we take that refresh out of there, is:18:24
efriedThe first time through, the reshaper succeeds, and then the agg/trait update will fail.18:25
jaypipesefried: yeah, sorry, I was saying we didn't have a decorator specifically for that (ala safe_connect is a specific decorator that traps specific exceptions)18:25
efriedthis will cause ufpt to invalidate the cache and raise the conflict exception that the aforementioned retry is set up to look for.18:25
efriedThe retry will run _update again, which will attempt to run update_provider_tree again.18:26
efriedThis time, the reshape already being done, update_provider_tree ought to *not* raise ReshapeNeeded, and instead go through the normal code path, posting back just the agg/trait changes18:26
efriedthe only thing I'm not completely sure about is how this plays with multiple providers.18:26
efriedufpt is set up to keep trying for other providers, but I'm not sure how completely18:27
jaypipesI have the feeling we're talking past each other.18:27
efriedE.g. if there are 4 tries total but 5 providers changing traits, could we run out of tries18:27
jaypipesefried: I'm certainly not suggesting wrapping the entire update_from_provider_tree() in a retrying decorator.18:28
efriedit already is.18:28
efriedaround _update18:28
efriedand that's where this is gonna go if I take out that refresh.18:28
*** e0ne has joined #openstack-placement18:28
efriedBecause I can't take out the refresh *and* add a retry further down in ufpt around just the agg/trait update calls.18:29
efriedbecause refreshing and redriving is not (necessarily) the right thing to do at that point.18:29
jaypipesefried: I don't understand why a retrying decorator cannot be put around smaller code chunks in the update_from_provider_tree() code path.18:30
efriedbecause we don't know why the generation changed.18:31
efriedIf, say, a cli user adds a trait to a provider in between upt and ufpt, that same bounce will happen at L165518:32
efriedif we were to retry there, we would overwrite and that trait would go away and the user wouldn't know why.18:32
jaypipesall we care about is that it *did* change, though. again, it's a signal that something changed, so the client code needs to re-get the state it's examining and re-determine if it needs to re-send the updates it wanted to originally or abandon its efforts.18:32
efriedyes, re-determine logic happens in upt, not in ufpt.18:32
jaypipeshmm18:32
efriedwhich is why the retry is around _update, not narrower.18:32
efrieddefinitely can't have things outside of upt attempting to guess whether to merge/replace/abort18:33
efriedjaypipes: Sorry, I thought this was going to go quicker; I have to run. Resume tomorrow? And/or via the review?18:38
jaypipesefried: sure18:39
*** efried is now known as efried_afk18:43
openstackgerritMatt Riedemann proposed openstack/nova stable/rocky: libvirt: Don't react to VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED events  https://review.openstack.org/59586418:44
*** e0ne has quit IRC18:46
cdent'night all19:02
*** cdent has quit IRC19:02
openstackgerritmelanie witt proposed openstack/nova master: Correct the release notes related to nova-consoleauth  https://review.openstack.org/59545519:21
openstackgerritmelanie witt proposed openstack/nova master: Correct the release notes related to nova-consoleauth  https://review.openstack.org/59545519:24
openstackgerritmelanie witt proposed openstack/nova master: Correct the release notes related to nova-consoleauth  https://review.openstack.org/59545519:37
*** jaypipes has quit IRC19:46
*** jaypipes has joined #openstack-placement19:47
openstackgerritMerged openstack/nova stable/rocky: block_device: Rollback volumes to in-use on DeviceDetachFailed  https://review.openstack.org/59080120:17
openstackgerritMerged openstack/nova stable/queens: block_device: Rollback volumes to in-use on DeviceDetachFailed  https://review.openstack.org/59080320:17
openstackgerritmelanie witt proposed openstack/nova stable/rocky: Correct the release notes related to nova-consoleauth  https://review.openstack.org/59589020:55
openstackgerritmelanie witt proposed openstack/nova stable/rocky: Correct the release notes related to nova-consoleauth  https://review.openstack.org/59589020:57
openstackgerritSurya Seetharaman proposed openstack/nova-specs master: Handling a down cell  https://review.openstack.org/59589221:03
openstackgerritMerged openstack/nova master: Normalize dashless 'resource provider create' uuid  https://review.openstack.org/56719121:16
openstackgerritMerged openstack/nova master: libvirt: Don't react to VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED events  https://review.openstack.org/59450821:30
openstackgerritMerged openstack/nova master: Update contributor guide for Stein  https://review.openstack.org/59125821:30
openstackgerritMerged openstack/nova master: [placement] Add functional test to verify presence of policy  https://review.openstack.org/59555921:30
*** mriedem is now known as mriedem_afk21:43
openstackgerritMerged openstack/nova master: Add functional test for affinity with multiple cells  https://review.openstack.org/58507322:50
*** jaypipes has quit IRC23:33

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!