| *** dklyle has joined #openstack-placement | 01:20 | |
| *** dklyle has quit IRC | 02:02 | |
| *** altlogbot_0 has quit IRC | 02:41 | |
| *** altlogbot_2 has joined #openstack-placement | 02:43 | |
| *** altlogbot_2 has quit IRC | 03:31 | |
| *** altlogbot_2 has joined #openstack-placement | 03:32 | |
| *** altlogbot_2 has quit IRC | 07:13 | |
| *** altlogbot_0 has joined #openstack-placement | 07:18 | |
| *** tssurya has joined #openstack-placement | 08:37 | |
| *** altlogbot_0 has quit IRC | 09:11 | |
| *** altlogbot_0 has joined #openstack-placement | 09:16 | |
| *** gibi_cape is now known as gibi | 09:24 | |
| *** ttsiouts has joined #openstack-placement | 09:42 | |
| *** ttsiouts_ has joined #openstack-placement | 09:56 | |
| *** ttsiouts has quit IRC | 09:59 | |
| *** ttsiouts_ has quit IRC | 10:14 | |
| *** altlogbot_0 has quit IRC | 11:19 | |
| *** altlogbot_2 has joined #openstack-placement | 11:21 | |
| *** ttsiouts has joined #openstack-placement | 11:26 | |
| *** ttsiouts has quit IRC | 11:43 | |
| *** ttsiouts has joined #openstack-placement | 11:48 | |
| *** ttsiouts_ has joined #openstack-placement | 11:53 | |
| *** ttsiouts has quit IRC | 11:56 | |
| *** ttsiouts_ has quit IRC | 12:10 | |
| *** altlogbot_2 has quit IRC | 12:14 | |
| *** altlogbot_1 has joined #openstack-placement | 12:21 | |
| *** ttsiouts has joined #openstack-placement | 12:28 | |
| *** ttsiouts_ has joined #openstack-placement | 12:36 | |
| *** ttsiouts has quit IRC | 12:40 | |
| *** ttsiouts_ has quit IRC | 12:46 | |
| *** ttsiouts has joined #openstack-placement | 12:46 | |
| *** ttsiouts has quit IRC | 12:53 | |
| edleafe | With travel recovery and meeting burnout, what say we skip this morning's meeting? We can continue to discuss outstanding issues in email. | 13:15 |
|---|---|---|
| *** altlogbot_1 has quit IRC | 13:15 | |
| *** altlogbot_1 has joined #openstack-placement | 13:17 | |
| *** mriedem has joined #openstack-placement | 13:19 | |
| *** altlogbot_1 has quit IRC | 13:21 | |
| *** ttsiouts has joined #openstack-placement | 13:22 | |
| *** altlogbot_2 has joined #openstack-placement | 13:25 | |
| *** altlogbot_2 has quit IRC | 13:25 | |
| *** altlogbot_0 has joined #openstack-placement | 13:27 | |
| *** efried has joined #openstack-placement | 13:41 | |
| efried | o/ | 13:42 |
| *** amodi has joined #openstack-placement | 14:00 | |
| *** ttsiouts has quit IRC | 14:00 | |
| gibi | edleafe: I agree | 14:04 |
| *** egonzalez has quit IRC | 14:22 | |
| *** egonzalez has joined #openstack-placement | 14:24 | |
| *** dklyle has joined #openstack-placement | 14:48 | |
| *** dklyle has quit IRC | 14:49 | |
| *** david-lyle has joined #openstack-placement | 14:49 | |
| *** belmoreira has joined #openstack-placement | 14:54 | |
| *** tssurya has quit IRC | 15:09 | |
| *** david-lyle is now known as dklyle | 15:13 | |
| *** belmoreira has quit IRC | 15:52 | |
| *** cdent has joined #openstack-placement | 16:24 | |
| efried | cdent: I never rendered an opinion on doing https://review.opendev.org/#/c/657074/ for other pyXXs. I'm in favor. | 16:34 |
| cdent | efried: noted. i'll take that action and get on it in the gaps. | 16:35 |
| efried | yeah, I don't think it's urgent | 16:36 |
| * cdent nods | 16:36 | |
| * cdent is knackered | 16:36 | |
| openstackgerrit | Chris Dent proposed openstack/placement master: WIP: Allow [A-Za-z0-9_-]{1,32} for request group suffix https://review.opendev.org/657419 | 17:58 |
| cdent | efried, mriedem, edleafe that ^ is stab to see what questions are raised by changing request group suffix. please have a look and add answers or more questions | 17:58 |
| edleafe | cdent: ack | 18:10 |
| openstackgerrit | Chris Dent proposed openstack/placement master: DNM: See what happens with 10000 resource providers https://review.opendev.org/657423 | 18:10 |
| openstackgerrit | Chris Dent proposed openstack/placement master: DNM: See what happens with 10000 resource providers https://review.opendev.org/657423 | 18:15 |
| openstackgerrit | Chris Dent proposed openstack/placement master: DNM: See what happens with 10000 resource providers https://review.opendev.org/657423 | 18:16 |
| efried | cdent: I pity the fool who has to go change numbered/unnumbered everywhere. | 18:25 |
| cdent | quite | 18:25 |
| openstackgerrit | Chris Dent proposed openstack/placement master: DNM: See what happens with 10000 resource providers https://review.opendev.org/657423 | 18:39 |
| cdent | my efforts on that ^ makes me think that we should choose to not be too overly concerned about all performance concerns in all situations because the tweaks required to fully stretch both uwsgi and mysql (and any other set up) is ... extensive | 18:40 |
| edleafe | Now *returning* 10K rps might be a different story... | 18:41 |
| efried | edleafe, cdent: I thought that ^ was the actual problem cern was having. | 18:45 |
| cdent | efried, edleafe : I'm not able to parse either of your statements | 18:46 |
| efried | cdent: Does perfload actually run a single query, either GET /a_c or GET /rps, that sends all the providers across the wire in a json payload? | 18:46 |
| edleafe | cdent: I meant that while placement can query/work with large data sets, those receiving large data set (CERN) might have other problems | 18:46 |
| cdent | efried: yes, it does a GET /a_c. The previous version returned 1000 | 18:47 |
| efried | oh, okay. So maybe edleafe is just saying that the client-side processing of that many records is the actual problem CERN doesn't want to deal with. | 18:47 |
| cdent | edleafe: yes, that's indeed the case, but it's a good data point from which to explore | 18:47 |
| efried | but I don't have a good handle on that. | 18:47 |
| cdent | yes, the client side processing is the the problem that CERN is having | 18:48 |
| cdent | so we need to make it easier for them to ask smaller | 18:48 |
| cdent | so in this case my patch is mostly an exploration, not a specific goal-oriented thing | 18:48 |
| edleafe | CERN's problem is that the nova scheduler has to repeatedly pass 10K results through filters and weighers | 18:49 |
| efried | Do either of you remember how we landed on CERN's anti-affinity issue? | 18:50 |
| efried | I've got these notes: | 18:50 |
| efried | """ | 18:50 |
| efried | Affinity/anti-affinity for CERN | 18:50 |
| efried | Affinity is easy: in_tree | 18:50 |
| efried | Anti-affinity, in_tree=!in:<small list>, but big lists are hard. | 18:50 |
| efried | tssurya has patch to remove the limit | 18:50 |
| efried | placement filter needs a spec - tssurya to own | 18:50 |
| efried | """ | 18:50 |
| efried | but I don't remember what the spec tssurya owns is supposed to do | 18:50 |
| efried | in_tree=!in<small_list> ? | 18:50 |
| efried | cdent: ^ | 18:51 |
| cdent | efried: that memory is currently swapped out but may come back later if some email is made | 18:52 |
| efried | mriedem, dansmith: do you remember? ---^ | 18:53 |
| dansmith | efried: her current bug fix is to reset the limit on anti-affinity | 18:53 |
| dansmith | well, either I guess | 18:54 |
| dansmith | efried: and I think we mostly said we'd take her patch, fix affinity because it's easy, | 18:54 |
| dansmith | and maybe punt on anti-, but when we do, try to be reasonable with !in_tree and some not crazy length limit | 18:54 |
| efried | dansmith: reasonable how? How big can a server group be? | 18:56 |
| dansmith | 100 by default | 18:56 |
| dansmith | but larger by confg | 18:56 |
| dansmith | a hundred uuids on top of the request of the query is too big, IMHO | 18:57 |
| dansmith | *rest of the query | 18:57 |
| efried | Well, it's not one uuid per server; it's one uuid per host-on-which-said-servers-have-previously-landed | 18:57 |
| dansmith | with anti-affinity, they are the same | 18:57 |
| dansmith | so, 20 uuids for a group of 20 instances | 18:57 |
| efried | Unless there's fewer than 20 hosts in your... what, cell? host agg? | 18:58 |
| dansmith | eh? | 18:58 |
| efried | i.e. what's the scope (number of hosts involved) in such a request? | 18:58 |
| efried | Basically "all of them"? | 18:58 |
| dansmith | yes, all of them | 18:58 |
| efried | okay. And in a small deployment, with like 10 hosts, your 11th anti-affinity-requesting spawn would get back zero allocation_candidates based on ^ design. So you would have to account for that and do a second query without the filter. | 18:59 |
| dansmith | unless we do something crazy, we'd build this part of the request purely from the list of hosts of other instances in your group, unrelated to any other scope-limiting factors, which we won't really be able to reduce on the nova side | 18:59 |
| dansmith | no, | 18:59 |
| dansmith | if you have 10 hosts and you boot an 11th in an anti-affinity group, there is no retry, you just fail | 19:00 |
| efried | o | 19:00 |
| efried | neat | 19:00 |
| dansmith | I mean, that's the point | 19:00 |
| efried | Okay, I thought it was for "spread" purposes | 19:00 |
| efried | as opposed to "never ever run two of these guys together" | 19:01 |
| dansmith | anti-affinity is for fault isolation only | 19:01 |
| efried | got it | 19:01 |
| efried | So we just enable in_tree=!<list> and either limit the size of <list> or let the implicit querystring length limit take care of it, and it's the deployment's problem to make sure they don't use enormous anti-affinity server groups? | 19:01 |
| dansmith | I would tend to say limit the list length, maybe with a tunable | 19:02 |
| dansmith | "the deployment" doesn't get to choose the group size, other than just the limit on member count they set | 19:03 |
| dansmith | it's a user-visible thing | 19:03 |
| efried | okay. These details to be worked out in the spec I suppose. | 19:03 |
| efried | thanks dansmith | 19:03 |
| dansmith | I dunno what the query length limit is in practice, but I guess I would expect that if it's too large, the sql query just gets nuts and starts to affect performance | 19:03 |
| efried | oh, I guess I would have expected wsgi to bounce before we got big enough to worry the db. But either way. | 19:04 |
| efried | expect to see your words regurgitated in next summary email :) | 19:04 |
| dansmith | well, some db-knowing person should opine on that detail | 19:05 |
| *** amodi has quit IRC | 19:11 | |
| edleafe | Ihave1newknee-0 | 19:16 |
| edleafe | Oh, well, now everyone can get into that VM ") | 19:16 |
| openstackgerrit | Chris Dent proposed openstack/placement master: WIP: Allow [A-Za-z0-9_-]{1,32} for request group suffix https://review.opendev.org/657419 | 19:17 |
| openstackgerrit | Chris Dent proposed openstack/placement master: WIP: Allow [A-Z0-9_-]{1,32} for request group suffix https://review.opendev.org/657419 | 19:18 |
| cdent | efried, edleafe : that ^ makes some tweaks but now lunch | 19:18 |
| *** cdent has quit IRC | 19:18 | |
| *** amodi has joined #openstack-placement | 19:42 | |
| *** cdent has joined #openstack-placement | 20:19 | |
| openstackgerrit | Chris Dent proposed openstack/placement master: WIP: Allow [A-Z0-9_-]{1,32} for request group suffix https://review.opendev.org/657419 | 20:42 |
| openstackgerrit | Chris Dent proposed openstack/placement master: DNM: See what happens with 10000 resource providers https://review.opendev.org/657423 | 20:45 |
| openstackgerrit | Chris Dent proposed openstack/placement master: DNM: See what happens with 10000 resource providers https://review.opendev.org/657423 | 20:48 |
| openstackgerrit | Chris Dent proposed openstack/placement master: Skip notification sample tests when running nova functional https://review.opendev.org/657455 | 21:06 |
| openstackgerrit | Chris Dent proposed openstack/placement master: DNM: See what happens with 5000 resource providers https://review.opendev.org/657423 | 21:11 |
| cdent | mriedem, efried : If we can get https://review.opendev.org/651939 merged we can save a few cycles in the gate and get a story off the radar. | 21:13 |
| cdent | mriedem: and this one too https://review.opendev.org/#/c/656717/ | 21:15 |
| efried | cdent: +A | 21:18 |
| efried | on the first one | 21:18 |
| cdent | rad | 21:18 |
| openstackgerrit | Merged openstack/osc-placement master: Use PlacementFixture in functional tests https://review.opendev.org/651939 | 21:27 |
| mriedem | sorry, been off in watcher land all day | 21:36 |
| mriedem | cdent: questions in https://review.opendev.org/#/c/656717/ | 21:51 |
| cdent | aye | 21:51 |
| mriedem | the answers look like they might all be in the commit message | 21:54 |
| mriedem | and i'm just slow | 21:54 |
| mriedem | gotta run, will check back later - else ping me in the morrow | 22:01 |
| openstackgerrit | Chris Dent proposed openstack/placement master: DNM: See what happens with 10000 resource providers https://review.opendev.org/657423 | 22:03 |
| openstackgerrit | Chris Dent proposed openstack/placement master: Package db migration scripts in placement pypi dist https://review.opendev.org/656717 | 22:10 |
| cdent | 10000 rps leads to ~46s GET /a_c: http://logs.openstack.org/23/657423/8/check/placement-perfload/07de88c/logs/placement-perf.txt | 22:21 |
| openstackgerrit | Eric Fried proposed openstack/placement master: Add NUMANetworkFixture for gabbits https://review.opendev.org/657463 | 22:29 |
| efried | cdent: I'm deliberately not putting any tests on this ^ because the series won't be linear | 22:31 |
| efried | it'll be, like, nested and shit. | 22:31 |
| efried | same reason I'm not tagging a task on it. | 22:31 |
| cdent | aye | 22:31 |
| openstackgerrit | Eric Fried proposed openstack/os-resource-classes master: Propose standard ACCELERATOR_FPGA resource class https://review.opendev.org/657464 | 22:32 |
| cdent | good idea | 22:32 |
| cdent | (the fixture) | 22:32 |
| cdent | efried: which, if any, spec was ACCELERATOR_FPGA defined in (it's not on that topic)? | 22:38 |
| cdent | doesn't really matter, but I wanted to confirm that there was some consensus building on that name | 22:39 |
| efried | cdent: Apparently it's not explicitly in any spec yet as a standard resource class. All the cyborg-ish specs currently use customs. Worth commenting on the spec linked from that commit... | 22:39 |
| cdent | " Worth commenting on the spec linked from that commit..." which one is that? | 22:40 |
| efried | nova-cyborg-interaction | 22:41 |
| efried | hold | 22:41 |
| cdent | the blueprint doesn't link to a spec was my starting investigation | 22:42 |
| efried | ahcrap | 22:42 |
| efried | https://review.opendev.org/#/c/603955/ | 22:42 |
| efried | yeah, the whiteboard should be updated. I'm updating the topic now... | 22:42 |
| cdent | ah, bad topic on that spec | 22:42 |
| efried | not sure if the bp gets updated automatically when that happens. I want to say it didn't used to, but that started happening lately. | 22:43 |
| efried | we'll see shortly.. | 22:43 |
| efried | I commented accordingly. | 22:43 |
| cdent | word | 22:43 |
| efried | I'm going to go now. | 22:43 |
| efried | I very well may take tomorrow off, since I didn't take today off... | 22:44 |
| cdent | aye aye | 22:45 |
| cdent | efried: if you're not gone yet, want to lay a seed in your brain about request group orphans | 23:05 |
| cdent | It looks like we only force required and member_of to have an associated resources, not in_tree | 23:08 |
| cdent | do we want resource-less member_of? | 23:08 |
| *** david-lyle has joined #openstack-placement | 23:55 | |
| *** dklyle has quit IRC | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!