Friday, 2018-06-01

*** cdent has quit IRC00:23
openstackgerritwanghongtao proposed openstack/nova master: Fix the metadata re to match the unicode  https://review.openstack.org/53623600:43
*** edmondsw has quit IRC01:54
*** edmondsw has joined #openstack-placement01:55
openstackgerrittianhui proposed openstack/nova master: Fix bug to api-ref  https://review.openstack.org/57137501:59
*** edmondsw has quit IRC01:59
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Fix invalid raise in test_compute_mgr  https://review.openstack.org/57161002:25
openstackgerritMerged openstack/nova stable/queens: libvirt: Skip fetching the virtual size of block devices  https://review.openstack.org/57142502:27
*** nicolasbock has quit IRC03:04
*** edmondsw has joined #openstack-placement03:40
*** edmondsw has quit IRC03:44
openstackgerritMerged openstack/nova master: libvirt: configure trust mode for vfs  https://review.openstack.org/45851404:19
*** e0ne has joined #openstack-placement04:21
*** e0ne has quit IRC04:30
*** e0ne has joined #openstack-placement05:02
*** e0ne has quit IRC05:16
*** edmondsw has joined #openstack-placement05:28
*** edmondsw has quit IRC05:33
*** ttsiouts has joined #openstack-placement06:37
*** ttsiouts has quit IRC06:55
*** ttsiouts has joined #openstack-placement06:59
*** tssurya has joined #openstack-placement07:07
*** ttsiouts has quit IRC07:19
*** edmondsw has joined #openstack-placement07:20
*** edmondsw has quit IRC07:24
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Adapt _validate_instance_group_policy to new policy model  https://review.openstack.org/57146507:34
*** ttsiouts has joined #openstack-placement08:14
*** ttsiouts has quit IRC08:17
*** ttsiouts has joined #openstack-placement08:22
openstackgerritElod Illes proposed openstack/nova stable/pike: placement: Fix HTTP error generation  https://review.openstack.org/57121808:23
openstackgerritjichenjc proposed openstack/nova master: z/VM Driver: Initial change set of z/VM driver  https://review.openstack.org/52338708:47
openstackgerritjichenjc proposed openstack/nova master: z/VM Driver: Spawn and destroy function of z/VM driver  https://review.openstack.org/52765808:47
openstackgerritjichenjc proposed openstack/nova master: z/VM Driver: add snapshot function  https://review.openstack.org/53424008:47
openstackgerritjichenjc proposed openstack/nova master: z/VM Driver: add power actions  https://review.openstack.org/54334008:48
openstackgerritjichenjc proposed openstack/nova master: z/VM Driver: add get console output  https://review.openstack.org/54334408:48
*** belmoreira has joined #openstack-placement09:01
*** edmondsw has joined #openstack-placement09:08
*** edmondsw has quit IRC09:13
openstackgerritjichenjc proposed openstack/nova master: not reraise DiskNotFound if instance is resized  https://review.openstack.org/57141009:19
*** ttsiouts has quit IRC09:33
openstackgerritStephen Finucane proposed openstack/nova-specs master: Add 'numa-aware-vswitches' spec  https://review.openstack.org/54129009:42
*** ttsiouts has joined #openstack-placement09:44
*** ttsiouts has quit IRC10:01
*** nicolasbock has joined #openstack-placement10:55
*** edmondsw has joined #openstack-placement10:56
*** edmondsw has quit IRC11:01
*** ttsiouts has joined #openstack-placement11:06
*** ttsiouts has quit IRC11:16
*** ttsiouts has joined #openstack-placement11:18
openstackgerritChen proposed openstack/nova master: WIP  https://review.openstack.org/57147211:25
*** ttsiouts has quit IRC12:05
*** edmondsw has joined #openstack-placement12:11
openstackgerritMerged openstack/nova master: libvirt:  add vf_trusted field for network metadata  https://review.openstack.org/56634312:12
*** edleafe is now known as figleaf12:14
openstackgerritMerged openstack/nova master: metadata: add vf_trusted field to device metadata  https://review.openstack.org/56634412:16
*** tssurya is now known as sususuryashines12:16
*** mriedem has joined #openstack-placement12:22
*** gibi is now known as giblet13:06
*** stephenfin is now known as finucannot13:06
*** jaypipes is now known as leakypipes13:18
*** bauzas is now known as PapaOurs13:25
*** mriedem is now known as hansmoleman13:25
*** efried is now known as fried_rice13:26
openstackgerritStephen Finucane proposed openstack/nova-specs master: Add 'numa-aware-vswitches' spec  https://review.openstack.org/54129013:31
*** dansmith is now known as superdan13:35
*** leakypipes is now known as MurderTheLeafBlo13:36
*** MurderTheLeafBlo is now known as leakypipes13:36
*** jroll is now known as jrollinhatin13:36
openstackgerritTsuyoshi Nagata proposed openstack/nova master: nova improvement of maximum attach volumes more than 26 vols  https://review.openstack.org/56747213:40
*** ttsiouts has joined #openstack-placement13:54
*** ttsiouts has quit IRC13:56
*** ttsiouts has joined #openstack-placement13:56
openstackgerritJan Gutter proposed openstack/nova master: Use vif.vif_name in _set_config_VIFGeneric  https://review.openstack.org/57146113:57
openstackgerritJan Gutter proposed openstack/nova master: Convert vrouter legacy plugging to os-vif  https://review.openstack.org/57132513:57
openstackgerritJan Gutter proposed openstack/nova master: Pass virtio multiqueue info to os-vif plugins  https://review.openstack.org/57146213:57
*** e0ne has joined #openstack-placement13:59
*** belmorei_ has joined #openstack-placement14:01
*** ttsiouts_ has joined #openstack-placement14:01
*** ttsiouts has quit IRC14:01
*** belmoreira has quit IRC14:02
openstackgerritAndrey Volkov proposed openstack/nova master: Test Compute API in multiple cells  https://review.openstack.org/54727314:03
*** melwitt is now known as jgwentworth14:09
*** e0ne has quit IRC14:21
openstackgerritMerged openstack/nova master: network: update pci request spec to handle trusted tags  https://review.openstack.org/45882014:32
openstackgerritTsuyoshi Nagata proposed openstack/nova master: nova improvement of maximum attach volumes more than 26 vols  https://review.openstack.org/56747214:56
*** e0ne has joined #openstack-placement14:56
*** e0ne has quit IRC15:09
*** ttsiouts_ has quit IRC15:21
openstackgerritBalazs Gibizer proposed openstack/nova master: Add bandwidth related standard resource classes  https://review.openstack.org/57084715:22
openstackgerritBalazs Gibizer proposed openstack/nova master: Transfer port.resource_request to the scheduler  https://review.openstack.org/56726815:22
openstackgerritBalazs Gibizer proposed openstack/nova master: Send resource allocations in the port binding  https://review.openstack.org/56945915:22
*** sususuryashines has quit IRC15:24
*** ttsiouts has joined #openstack-placement15:25
*** ttsiouts has quit IRC15:30
*** ttsiouts has joined #openstack-placement15:30
*** ttsiouts has quit IRC15:31
openstackgerritArtom Lifshitz proposed openstack/nova master: Refactor _build_device_metadata  https://review.openstack.org/53380415:32
openstackgerritArtom Lifshitz proposed openstack/nova master: Consider hostdev devices when building metadata  https://review.openstack.org/53380515:32
*** belmorei_ has quit IRC15:40
openstackgerritMatt Riedemann proposed openstack/nova master: Restrict CONF.quota.driver to DB and noop quota drivers  https://review.openstack.org/41099615:44
openstackgerritLee Yarwood proposed openstack/nova stable/ocata: libvirt: handle DiskNotFound during update_available_resource  https://review.openstack.org/57143216:02
openstackgerritLee Yarwood proposed openstack/nova stable/ocata: libvirt: Skip fetching the virtual size of block devices  https://review.openstack.org/57143316:02
openstackgerritMatt Riedemann proposed openstack/nova master: Trim the fat on HostState.instances  https://review.openstack.org/56924716:17
openstackgerritSylvain Bauza proposed openstack/nova-specs master: Proposes Multiple GPU types  https://review.openstack.org/55706516:26
*** PapaOurs is now known as bauzas16:27
*** fried_rice is now known as fried_rolls16:29
*** hansmoleman is now known as hans_lunch16:40
*** cdent has joined #openstack-placement16:50
*** nicolasbock has quit IRC16:56
openstackgerritDan Smith proposed openstack/nova master: Use oslo.messaging per-call monitoring  https://review.openstack.org/56669617:02
*** nicolasbock has joined #openstack-placement17:03
openstackgerritMerged openstack/nova master: libvirt: place emulator threads on CONF.compute.cpu_shared_set  https://review.openstack.org/51089717:25
*** fried_rolls is now known as fried_rice17:33
*** ttsiouts has joined #openstack-placement17:51
*** e0ne has joined #openstack-placement18:00
fried_riceleakypipes: I thought we had a big lock preventing claims from happening during rt update18:24
*** e0ne has quit IRC18:31
*** e0ne has joined #openstack-placement18:31
*** e0ne has quit IRC18:32
*** e0ne has joined #openstack-placement18:35
*** e0ne has quit IRC18:39
leakypipesfried_rice: rt update?18:53
fried_riceleakypipes: Yeah, the periodic task wherein update_from_provider_tree is being run.18:54
leakypipesfried_rice: we might lock the provider tree object, but that won't prevent the scheduler from allocating against any of the providers...18:54
leakypipesfried_rice: we need to do the inventory moves and allocation moves (for lots of consumers) in a single transaction.18:55
leakypipesfried_rice: that's the problem domain, AFAIU18:55
fried_riceleakypipes: If this was purely a placement problem, agreed; but it's not.  If we were synchronized as I thought we were on the nova side, we would be okay.18:55
fried_riceleakypipes: If that's not the case, then we already have this race condition in spades.18:56
fried_riceleakypipes: Because update_from_provider_tree operates via multiple separate placement calls to flush changes.18:56
fried_riceleakypipes: Help me understand: it's n-cpu that's invoking update_from_provider_tree, but n-sch that's doing GET /a_c and PUT allocations?18:58
*** hans_lunch is now known as mriedem18:59
leakypipesfried_rice: update_from_provider_tree never touches allocations.18:59
fried_riceI understand that's the case today, yes.18:59
leakypipesfried_rice: which is why there are no current race conditions with it.18:59
*** ttsiouts has quit IRC19:00
fried_riceStill a problem with allocations (from the scheduler) straddling changes to providers/inventories during update_from_provider_tree though.19:00
fried_riceOnce we have consumer generations, if both sides are trying to muck with allocations on the same instance, they'll bounce "naturally".  Having trouble seeing it as a different issue (vs. a tweak on the same issue).19:02
leakypipesfried_rice: is that you that is light blue in the etherpad?19:02
fried_riceI haven't looked at the etherpad yet, so no.19:02
leakypipescdent: are you the light blue?19:03
cdentyeah19:03
leakypipesgot it, thanks.19:03
leakypipescdent: strange that the line numbers are different between us.19:03
cdentyes not sure what's up with that19:03
cdenti've pasted an alternate allocations dict at the bottom19:04
*** openstackgerrit has quit IRC19:04
leakypipescdent: ack, thx19:05
fried_riceleakypipes: So your suggestion effectively dovetails with mine, right?  The virt driver still needs to provide the information; the difference is that update_from_provider_tree needs to do the allocation & inventory move in this single-transaction API call you've proposed, rather than stepwise as I described.19:07
leakypipesfried_rice: yep, that's the different in a nutshell.19:08
fried_riceack19:08
leakypipesI'm pretty positive superdan is going to throw up when he sees the proposal for another HTTP endpoint.19:10
cdentcan we put a "you must be this tall to ride this URI" on this thing?19:10
superdanI dunno what we're talking about, but barf19:12
leakypipeshehe19:13
* cdent passes superdan a tissue19:13
* superdan wipes on his sleeve like a barbarian19:13
* leakypipes passes superdan a superkerchief19:14
superdanthat's a sleeve yes?19:15
* superdan has to run out for a bit19:15
leakypipessuperdan: don't you mean you have to fly out for a bit? I mean, when you have the power...19:16
fried_ricecdent, leakypipes: If we're looking for "the smallest thing that could possibly work" - I think we can avoid the race conditions if we implement a flag on the allocations endpoint that allows me to exceed total - reserved.19:16
leakypipesfried_rice: not really a fan of that approach. just means mucking with reserved amounts to get around the fact that we need to perform multiple operations in a single transaction.19:17
fried_riceThe update_from_provider_tree algo starts off by effectively "locking" the inventories by setting reselved = total.  Then we don't have to worry about the scheduler interfering while we're mucking with stuff.  And we use that "overallocate" flag to move the allocations.  Then set reserved back to the proper value when done.19:18
fried_riceleakypipes: True story, but it gives us time to implement a more comprehensive solution, which is going to take time.19:18
leakypipesfried_rice: that sounds more complicated and tribal knowledge-y than just adding an HTTP endpoint that does what we're trying to do in the first place?19:18
cdentthat flag would have a less than great failure mode too (start the multi-step process, fails halfway through)19:19
fried_riceleakypipes: If we don't solve this upgrade problem in rocky, xen's vgpu nrp bp is dead in the water.19:19
leakypipescdent: excellent point.19:19
cdentfried_rice's point is good too19:19
leakypipesfried_rice: I'm less concerned with that than I am with doing this right.19:19
leakypipessame for the multi-vgpu type stuf.19:20
fried_riceIt will be very disappointing if nrp isn't usable in rocky.19:21
leakypipesfried_rice, cdent: if we can reach some rough agreement on the HTTP API, I can have the implementation of this hacked together this weekend.19:21
fried_riceI might have to be /nick efired19:22
cdenti've got to lunch, but will look back here and on the etherpad on when I return (I'm in PST today)19:22
leakypipescdent: ack, thx19:23
*** e0ne has joined #openstack-placement19:24
*** openstackgerrit has joined #openstack-placement19:44
openstackgerritKashyap Chamarthy proposed openstack/nova master: libvirt: Remove usage of migrateToURI{2} APIs  https://review.openstack.org/56725819:44
fried_riceleakypipes, cdent: Proposed slight tweak to the format at the bottom of the etherpad.  Glues together inv+allocs for one RC.20:06
fried_riceI think it handles splits and new inventory creation correctly.20:06
superdanleakypipes: my cape is in the shop20:14
*** e0ne has quit IRC20:19
cdentfried_rice, leakypipes I don't have strong opinion on the shape of things. fried_rice's has an advantage of being fairly distinct from any other structure we use, which in this case I think is useful20:28
openstackgerritMerged openstack/nova master: Restrict CONF.quota.driver to DB and noop quota drivers  https://review.openstack.org/41099620:56
openstackgerritMerged openstack/nova master: Fix invalid raise in test_compute_mgr  https://review.openstack.org/57161020:56
openstackgerritMerged openstack/nova master: Implement granular policy rules for placement  https://review.openstack.org/52442521:06
openstackgerritJay Pipes proposed openstack/nova master: Add a microversion for consumer generation support  https://review.openstack.org/56560421:12
*** edmondsw has quit IRC21:25
*** superdan is now known as dansmith21:25
openstackgerritJay Pipes proposed openstack/nova master: placement: always create consumer records  https://review.openstack.org/56767821:31
openstackgerritJay Pipes proposed openstack/nova master: add consumers generation field  https://review.openstack.org/55795821:31
openstackgerritJay Pipes proposed openstack/nova master: placement: Allocation.consumer field  https://review.openstack.org/56540521:31
openstackgerritJay Pipes proposed openstack/nova master: rework allocation handler _allocations_dict()  https://review.openstack.org/56540721:31
openstackgerritJay Pipes proposed openstack/nova master: Add a microversion for consumer generation support  https://review.openstack.org/56560421:31
*** mriedem has quit IRC21:42
openstackgerritMerged openstack/nova master: [placement] default to accept of application/json when */*  https://review.openstack.org/56863021:46
openstackgerritChris Dent proposed openstack/nova master: Extract part of PlacementFixture to placement  https://review.openstack.org/56835921:53
fried_riceleakypipes: Responded in etherpad (this is a weird way to have a conversation)22:11
openstackgerritkarim proposed openstack/nova master: Handle rebuild of instances with image traits  https://review.openstack.org/56949822:16
*** figleaf is now known as edleafe22:20
fried_ricecdent: Found out that gabbi query_parameters doesn't preserve param order.  Is that a bug?22:25
fried_ricecdent: I assume the http spec supports semantic significance to the ordering of qparams22:26
cdentfried_rice: I'm not certain, let me do some looking22:26
*** dabo has joined #openstack-placement22:26
*** edleafe has quit IRC22:28
*** dabo is now known as edleafe22:28
openstackgerritEric Fried proposed openstack/nova master: Test for multiple limit/group_policy qparams  https://review.openstack.org/56871322:29
fried_ricecdent: FYI ^22:30
cdentfried_rice: everything I'm reading is that there's no spec and that it is framework dependent and they don't offer guarantees22:35
fried_ricevay22:35
cdentsince gabbi's rep of query_parameters is a dict it is not ordered22:35
fried_ricewell, I figured that part was the case.22:35
fried_riceframework dependent, meaning the server?22:35
cdentyeah22:36
fried_ricecdent: Then I would say it is a bug, because it means gabbi isn't "supporting" servers where ordering is significant.22:36
cdentwell except that presumably ordering only matters when there are duplicates and a dict doesn't support duplicates22:37
cdentso if you want duplicates you need to put them in the url, and there you can do whatever you want22:37
fried_riceright, I get that there's a workaround, and I used it ^22:37
fried_riceI'm not a yaml expert; but the syntax for query_parameters can't be a yaml dict, because I'm allowed to duplicate keys in there.22:38
cdentalso if a service is relying on the ordering of parameters provided by a framework, I'd argue that's a bad server and I wouldn't want gabbi to support that because it would be supporting framework dependence22:38
fried_riceSo this could be "fixed" without requiring a change in the yaml syntax.22:38
cdentso I think the ordering tests you are making are probably a bad idea22:38
cdentand we should not be making and predictions based on such things in our code22:38
cdents/and/any/22:39
fried_riceWell, per the commit message, I'm doing it so we don't "accidentally" change the behavior that we've coded into place.22:39
fried_riceI agree the right thing would be to make the server behave better here - by disallowing multiples in this case.22:39
cdentthat we've coded that into place was an accident and we shouldn't legislate it22:40
cdentand if the argument is that we have to now because of microversions I'm pretty meh to that22:40
cdentbecause "we" are still the main clients22:40
fried_riceWell, would you support any level of fixup to this without a microversion?22:40
cdentI would say that fixing it to error on duplicates without a microversions is okay22:41
fried_riceLike the original patch, which only breaks if a consumer happens to be sending in multiple (and *different*) values for these qparams?22:41
cdentbut I tend to be in the minority on such things22:41
fried_riceor that, which will break if the consumer sends multiple, even if they're the same.22:41
fried_riceedleafe: thoughts on this, with your API hat?22:41
fried_rice(Guessing he doesn't give a "fig", since it's Friday and he's still edleafe :)22:42
cdenthave you tried that and turned on verbose to make sure you are sending what you think you are sending?22:43
cdentyaml allows you to duplicate keys in a dict and then silently doesn't complain22:44
fried_riceno, I didn't try that.22:44
fried_ricebut I'm convinced it's not worth "fixing" in gabbi22:44
cdenthttp://paste.openstack.org/show/722542/22:45
fried_riceack22:46
openstackgerritMatt Riedemann proposed openstack/nova master: Add nova-manage placement heal_allocations CLI  https://review.openstack.org/56588622:47
cdenti'm in meetings at the mo' but will look at that review soon22:47
fried_ricecdent: Apparently according to The Spec, we're doing it all kinds of wrong: https://esdiscuss.org/topic/json-duplicate-keys22:48
cdentfried_rice: what does json have to do with this topic?22:49
fried_ricehm, maybe nothing, except that we're using json schema to validate the querystring??22:49
*** leakypipes has quit IRC22:50
cdentyeah that's orthogonal22:50
fried_riceI guess the querystring isn't actually json??22:50
fried_ricesigh22:50
cdentcorect22:50
cdentthat we use jsonschema is effectively a hack (which happens to work)22:50
fried_ricewell, it's because I was trying to figure out how to update the schema to reject duplicates.22:50
cdentmax<something>22:50
cdent=122:51
fried_ricedoesn't look like it.22:51
* cdent looks more closely22:52
fried_riceit's not maxItems22:55
cdentbased on some reading I just did it probably doesn't exist because it's not valid javascript to have duplicated keys. however since the query params become a list we can check that the value is a list of one. but only if  the code in utils:validate_query_params is returning a dict of lists.22:58
cdentis reg.GET a dict of strings or lists?22:58
cdentfried_rice: looks like the behavior is a bit weird: https://docs.pylonsproject.org/projects/webob/en/stable/api/multidict.html23:00
cdentand will depend on how jsonscheme.validate is accessing values23:00
fried_riceI'm feeling it's not possible to validate this via schema.23:01
fried_riceif at all, would have to be in the handler.  Which is not the end of the world.23:01
cdentthis mixed() method (passed to validate) might do the right thing: https://docs.pylonsproject.org/projects/webob/en/stable/api/multidict.html#webob.multidict.MultiDict.mixed23:01
cdentthen if something is not a string it ought to kaboom23:02
fried_ricethat's not how I read it.23:02
cdentmaybe? (sorry for not being able to do more than throw links at you)23:02
cdent"the values are either single values, or a list of values when a key/value appears more than once in this dictionary"23:02
fried_riceI read that as, "if they specified one, you'll get back just the value; if they specified multiple, you'll get back a list."23:02
cdentright which is what you want for the validate (checking for string)23:03
cdentif 'resources' is 'type': 'string' then a list should 40023:03
fried_riceBut that method is on the GET object.23:03
fried_riceOh, but we're already past the schema validation by the time we would have the opportunity to call mixed(), nah?23:04
fried_riceoh, I see.23:04
cdentno I'm saying change the util method to pass req.GET.mixed() not req.GET23:04
cdent(into jsonschema.validate)23:04
fried_riceyeah, trying.23:05
fried_riceI suspect we're going to blow up in multiple places, though.23:05
cdentwill be interesting to know23:05
cdentbreaking stuff is good23:05
cdentbecause it leads to fixing stuff23:05
cdentI suspect that making this change will make the validation much more robust and is probably something we ought to do in nova's api too (if it isn't already there)23:06
fried_riceOkay, that actually does the trick.23:06
fried_riceI haven't gone all the way through, but I suspect our code to handle multi member_of will be broken if we do this.23:07
fried_riceAgree it will make things more robust.  Can we do it without a microversion?23:07
openstackgerritChris Dent proposed openstack/nova master: Optional separate database for placement API  https://review.openstack.org/36276623:07
fried_riceAnyway, I have to run.23:08
cdentyeah the schema for member_of will need to have it's schema adjusted to accept list23:08
fried_ricelist or string23:08
openstackgerritChris Dent proposed openstack/nova master: Isolate placement database config  https://review.openstack.org/54143523:08
cdentyeah, right23:08
openstackgerritChris Dent proposed openstack/nova master: Ensure that os-traits sync is attempted only at start of process  https://review.openstack.org/55385723:08
openstackgerritChris Dent proposed openstack/nova master: Add PLACEMENT_DB_ENABLED=True to the nova-next job  https://review.openstack.org/56406723:09
openstackgerritChris Dent proposed openstack/nova master: Use nova.db.api directly  https://review.openstack.org/54326223:10
fried_riceadded note to that review.  Now I'm really gone.23:12
*** fried_rice is now known as efried23:12

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!