| *** ttsiouts has joined #openstack-meeting | 00:00 | |
| *** brinzhang has joined #openstack-meeting | 00:03 | |
| *** jmasud has joined #openstack-meeting | 00:07 | |
| *** ttsiouts has quit IRC | 00:33 | |
| *** rfolco has quit IRC | 00:35 | |
| *** jamesmcarthur has joined #openstack-meeting | 00:37 | |
| *** larainema has joined #openstack-meeting | 00:42 | |
| *** _mlavalle_1 has quit IRC | 01:10 | |
| *** jamesmcarthur has quit IRC | 01:17 | |
| *** jmasud has quit IRC | 01:17 | |
| *** jmasud has joined #openstack-meeting | 01:39 | |
| *** gary_perkins has quit IRC | 01:40 | |
| *** hyunsikyang has joined #openstack-meeting | 01:52 | |
| *** hyunsikyang has quit IRC | 01:52 | |
| *** hyunsikyang__ has quit IRC | 01:52 | |
| *** hyunsikyang has joined #openstack-meeting | 01:52 | |
| *** Liang__ has joined #openstack-meeting | 01:54 | |
| *** ociuhandu has joined #openstack-meeting | 01:57 | |
| *** tetsuro has joined #openstack-meeting | 02:00 | |
| *** yasufum has joined #openstack-meeting | 02:01 | |
| *** hyunsikyang has quit IRC | 02:01 | |
| *** hyunsikyang has joined #openstack-meeting | 02:01 | |
| *** ociuhandu has quit IRC | 02:02 | |
| *** hyunsikyang has quit IRC | 02:02 | |
| *** hyunsikyang has joined #openstack-meeting | 02:02 | |
| *** jmasud has quit IRC | 02:11 | |
| *** yasufum has quit IRC | 02:18 | |
| *** hyunsikyang__ has joined #openstack-meeting | 02:20 | |
| *** hyunsikyang has quit IRC | 02:24 | |
| *** ttsiouts has joined #openstack-meeting | 02:30 | |
| *** yasufum has joined #openstack-meeting | 02:31 | |
| *** yasufum has quit IRC | 02:32 | |
| *** jamesmcarthur has joined #openstack-meeting | 02:33 | |
| *** tetsuro has quit IRC | 02:37 | |
| *** ttsiouts has quit IRC | 02:39 | |
| *** ttsiouts has joined #openstack-meeting | 02:39 | |
| *** luyao has joined #openstack-meeting | 02:49 | |
| *** jmasud has joined #openstack-meeting | 02:49 | |
| *** vishalmanchanda has joined #openstack-meeting | 03:03 | |
| *** baojg has joined #openstack-meeting | 03:03 | |
| *** ykatabam has joined #openstack-meeting | 03:03 | |
| *** ttsiouts has quit IRC | 03:04 | |
| *** jamesmcarthur has quit IRC | 03:06 | |
| *** jamesmcarthur has joined #openstack-meeting | 03:06 | |
| *** tetsuro has joined #openstack-meeting | 03:06 | |
| *** jamesmcarthur has quit IRC | 03:11 | |
| *** yasufum has joined #openstack-meeting | 03:14 | |
| *** tetsuro has quit IRC | 03:16 | |
| *** jmasud has quit IRC | 03:16 | |
| *** baojg has quit IRC | 03:18 | |
| *** tetsuro has joined #openstack-meeting | 03:22 | |
| *** yasufum has quit IRC | 03:26 | |
| *** psachin has joined #openstack-meeting | 03:36 | |
| *** jamesmcarthur has joined #openstack-meeting | 03:41 | |
| *** gary_perkins has joined #openstack-meeting | 03:45 | |
| *** tetsuro has quit IRC | 03:45 | |
| *** tetsuro has joined #openstack-meeting | 03:47 | |
| *** tetsuro has quit IRC | 03:54 | |
| *** rh-jelabarre has quit IRC | 04:02 | |
| *** openstack has joined #openstack-meeting | 04:21 | |
| *** ChanServ sets mode: +o openstack | 04:21 | |
| *** hyunsikyang has joined #openstack-meeting | 04:24 | |
| *** hyunsikyang__ has quit IRC | 04:28 | |
| *** hyunsikyang__ has joined #openstack-meeting | 04:29 | |
| *** hyunsikyang has quit IRC | 04:30 | |
| *** tetsuro has joined #openstack-meeting | 04:34 | |
| *** evrardjp has quit IRC | 04:35 | |
| *** evrardjp has joined #openstack-meeting | 04:36 | |
| *** gyee has quit IRC | 04:40 | |
| *** jmasud has joined #openstack-meeting | 05:00 | |
| *** ttsiouts has joined #openstack-meeting | 05:01 | |
| *** jamesmcarthur has quit IRC | 05:04 | |
| *** jamesmcarthur has joined #openstack-meeting | 05:05 | |
| *** jamesmcarthur has quit IRC | 05:11 | |
| *** jamesmcarthur has joined #openstack-meeting | 05:11 | |
| *** tetsuro has quit IRC | 05:25 | |
| *** tetsuro has joined #openstack-meeting | 05:27 | |
| *** ttsiouts has quit IRC | 05:33 | |
| *** cgoncalves has joined #openstack-meeting | 05:38 | |
| *** tetsuro has quit IRC | 05:43 | |
| *** tetsuro has joined #openstack-meeting | 05:44 | |
| *** ysandeep|away is now known as ysandeep | 05:51 | |
| *** diablo_rojo has quit IRC | 05:57 | |
| *** jamesmcarthur has quit IRC | 06:00 | |
| *** jamesmcarthur has joined #openstack-meeting | 06:01 | |
| *** ociuhandu has joined #openstack-meeting | 06:01 | |
| *** jamesmcarthur has quit IRC | 06:06 | |
| *** ociuhandu has quit IRC | 06:09 | |
| *** tetsuro has quit IRC | 06:09 | |
| *** tetsuro has joined #openstack-meeting | 06:10 | |
| *** ysandeep is now known as ysandeep|away | 06:12 | |
| *** tetsuro has quit IRC | 06:15 | |
| *** jmasud has quit IRC | 06:17 | |
| *** yamamoto has joined #openstack-meeting | 06:18 | |
| *** jmasud has joined #openstack-meeting | 06:19 | |
| *** yamamoto has quit IRC | 06:24 | |
| *** ysandeep|away has quit IRC | 06:29 | |
| *** ociuhandu has joined #openstack-meeting | 06:39 | |
| *** rubasov has quit IRC | 06:39 | |
| *** rubasov has joined #openstack-meeting | 06:39 | |
| *** jamesmcarthur has joined #openstack-meeting | 06:39 | |
| *** ralonsoh has joined #openstack-meeting | 06:45 | |
| *** tetsuro has joined #openstack-meeting | 06:49 | |
| *** tetsuro has quit IRC | 06:50 | |
| *** tetsuro has joined #openstack-meeting | 06:50 | |
| *** jmasud has quit IRC | 06:51 | |
| *** maciejjozefczyk has joined #openstack-meeting | 06:52 | |
| *** belmoreira has joined #openstack-meeting | 06:53 | |
| *** jmasud has joined #openstack-meeting | 06:55 | |
| *** slaweq has joined #openstack-meeting | 06:56 | |
| *** ociuhandu has quit IRC | 07:00 | |
| *** ttsiouts has joined #openstack-meeting | 07:04 | |
| *** jmasud has quit IRC | 07:14 | |
| *** ykatabam has quit IRC | 07:17 | |
| *** jmasud has joined #openstack-meeting | 07:19 | |
| *** lpetrut has joined #openstack-meeting | 07:19 | |
| *** witek has joined #openstack-meeting | 07:25 | |
| *** rpittau|afk is now known as rpittau | 07:31 | |
| *** cgoncalves has quit IRC | 07:34 | |
| *** cgoncalves has joined #openstack-meeting | 07:40 | |
| *** jmasud has quit IRC | 07:40 | |
| *** openstack has joined #openstack-meeting | 07:47 | |
| *** ChanServ sets mode: +o openstack | 07:47 | |
| *** yasufum has joined #openstack-meeting | 07:52 | |
| *** dmellado has quit IRC | 07:58 | |
| *** dmellado has joined #openstack-meeting | 08:04 | |
| *** ttsiouts has quit IRC | 08:07 | |
| *** ttsiouts has joined #openstack-meeting | 08:07 | |
| *** yasufum has quit IRC | 08:19 | |
| *** ociuhandu has quit IRC | 08:20 | |
| *** tetsuro has quit IRC | 08:26 | |
| *** tetsuro has joined #openstack-meeting | 08:26 | |
| *** larainema has quit IRC | 08:30 | |
| *** e0ne has joined #openstack-meeting | 08:30 | |
| *** ociuhandu has joined #openstack-meeting | 08:34 | |
| *** yasufum has joined #openstack-meeting | 08:37 | |
| *** baojg has joined #openstack-meeting | 08:40 | |
| *** jamesmcarthur has quit IRC | 08:41 | |
| *** tetsuro has quit IRC | 08:46 | |
| *** yasufum has quit IRC | 08:56 | |
| *** strigazi has joined #openstack-meeting | 09:00 | |
| *** jmasud has quit IRC | 09:02 | |
| *** tetsuro has joined #openstack-meeting | 09:15 | |
| *** jamesmcarthur has joined #openstack-meeting | 09:16 | |
| *** Liang__ has quit IRC | 09:24 | |
| *** jamesmcarthur has quit IRC | 09:26 | |
| *** rpittau is now known as rpittau|bbl | 09:44 | |
| *** dklyle has quit IRC | 09:45 | |
| *** tetsuro has quit IRC | 09:54 | |
| *** jamesmcarthur has joined #openstack-meeting | 09:57 | |
| *** ttsiouts has quit IRC | 09:58 | |
| *** ttsiouts has joined #openstack-meeting | 09:59 | |
| *** tetsuro has joined #openstack-meeting | 10:00 | |
| *** jamesmcarthur has quit IRC | 10:07 | |
| *** tetsuro has quit IRC | 10:12 | |
| *** tetsuro has joined #openstack-meeting | 10:15 | |
| *** tetsuro has quit IRC | 10:16 | |
| *** baojg has quit IRC | 10:18 | |
| *** tetsuro has joined #openstack-meeting | 10:18 | |
| *** ttsiouts has quit IRC | 10:27 | |
| *** ttsiouts has joined #openstack-meeting | 10:30 | |
| *** tetsuro has quit IRC | 10:36 | |
| *** jamesmcarthur has joined #openstack-meeting | 10:43 | |
| *** jamesmcarthur has quit IRC | 10:43 | |
| *** jamesmcarthur has joined #openstack-meeting | 10:43 | |
| *** maciejjozefczyk_ has joined #openstack-meeting | 10:43 | |
| *** jamesmcarthur has quit IRC | 10:44 | |
| *** maciejjozefczyk has quit IRC | 10:44 | |
| *** jamesmcarthur has joined #openstack-meeting | 10:44 | |
| *** maciejjozefczyk has joined #openstack-meeting | 10:44 | |
| *** tetsuro has joined #openstack-meeting | 10:48 | |
| *** ttsiouts has quit IRC | 10:48 | |
| *** maciejjozefczyk_ has quit IRC | 10:48 | |
| *** maciejjozefczyk_ has joined #openstack-meeting | 10:49 | |
| *** ykatabam has joined #openstack-meeting | 10:49 | |
| *** maciejjozefczyk has quit IRC | 10:50 | |
| *** maciejjozefczyk has joined #openstack-meeting | 10:50 | |
| *** jamesmcarthur has quit IRC | 10:51 | |
| *** janders has joined #openstack-meeting | 10:51 | |
| *** maciejjozefczyk_ has quit IRC | 10:53 | |
| *** maciejjozefczyk_ has joined #openstack-meeting | 10:55 | |
| *** maciejjozefczyk has quit IRC | 10:55 | |
| *** dh3 has joined #openstack-meeting | 10:57 | |
| *** jamesmcarthur has joined #openstack-meeting | 10:58 | |
| *** jamesmcarthur has quit IRC | 10:59 | |
| *** jamesmcarthur has joined #openstack-meeting | 10:59 | |
| *** tetsuro has quit IRC | 11:00 | |
| *** jamesmcarthur has quit IRC | 11:01 | |
| *** oneswig_ has joined #openstack-meeting | 11:01 | |
| oneswig_ | #startmeeting scientific-sig | 11:01 |
|---|---|---|
| openstack | Meeting started Wed May 6 11:01:55 2020 UTC and is due to finish in 60 minutes. The chair is oneswig_. Information about MeetBot at http://wiki.debian.org/MeetBot. | 11:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 11:01 |
| *** openstack changes topic to " (Meeting topic: scientific-sig)" | 11:01 | |
| openstack | The meeting name has been set to 'scientific_sig' | 11:01 |
| *** jamesmcarthur has joined #openstack-meeting | 11:02 | |
| janders | gday all | 11:02 |
| oneswig_ | Hi janders ahem sorry I'm late :-) | 11:02 |
| janders | :) | 11:03 |
| oneswig_ | what's new? | 11:03 |
| janders | not much | 11:03 |
| dh3 | openmpi/mpi4py (new to me at least) - not openstack related though | 11:03 |
| janders | some interesting attempts to use secgroups the other way round | 11:03 |
| *** priteau has joined #openstack-meeting | 11:04 | |
| oneswig_ | mpi4py always seemed like a curious combination | 11:04 |
| janders | (restricting egress with negative matching) | 11:04 |
| janders | ended up putting some extra private subnets and routers in instead | 11:04 |
| *** maciejjozefczyk_ has quit IRC | 11:04 | |
| janders | but apparently secgroups cant be used to allow a VM to ping everything except something (e.g. local network) | 11:05 |
| *** maciejjozefczyk_ has joined #openstack-meeting | 11:05 | |
| oneswig_ | janders: had an interesting issue with secgroups the other day. A VM with allowed_address_pairs to relax what IPs and MACs can be transmitted, also disables security groups. | 11:06 |
| *** jamesmcarthur has quit IRC | 11:07 | |
| janders | interesting feature | 11:07 |
| janders | a little scary though | 11:07 |
| janders | reminds me of a stuff up at one of AUS OpenStack operators back in nova-network days | 11:08 |
| janders | a puppet bug caused iptables flush cluster wide... while computes were on public IPs | 11:08 |
| janders | a bit of guessing and attacker can get console access to VMS | 11:08 |
| oneswig_ | yikes! | 11:09 |
| janders | quick rd.break and the floodgates are open | 11:09 |
| janders | yeah | 11:09 |
| janders | accidental disabling of iptables ain't fun | 11:09 |
| *** vkmc has quit IRC | 11:09 | |
| oneswig_ | could they remember all the rules to put back :-) | 11:10 |
| janders | i think the degree of compromise was large enough to sanction mass rebuild of stuff | 11:10 |
| oneswig_ | janders: on a different subject, have you contributed a case study to the OSF bare metal white paper? | 11:11 |
| janders | working on it | 11:11 |
| janders | getting close | 11:11 |
| oneswig_ | same here... | 11:11 |
| janders | issue is its a little too long | 11:11 |
| janders | cutting bits out before I can merge back | 11:11 |
| oneswig_ | there's an upper limit on length? | 11:11 |
| janders | good problem to have i suppose | 11:11 |
| janders | i dont think there's a specifc word count etc... mine had a fair bit of detail, code listings, etc | 11:12 |
| janders | that needs to go | 11:12 |
| oneswig_ | ah right. I wasn't sure on embedding config myself | 11:12 |
| *** ociuhandu has quit IRC | 11:13 | |
| *** ociuhandu has joined #openstack-meeting | 11:14 | |
| oneswig_ | On an administrative point | 11:14 |
| oneswig_ | #link Virtual PTG registration open https://www.eventbrite.com/e/virtual-project-teams-gathering-june-2020-tickets-103456996662 | 11:14 |
| priteau | OpenDev registration as well | 11:15 |
| oneswig_ | Hi priteau, thanks :-) | 11:15 |
| priteau | #link https://www.eventbrite.com/e/opendev-large-scale-usage-of-open-infrastructure-software-registration-102899719832 | 11:15 |
| janders | done! | 11:16 |
| janders | it's free - great! | 11:16 |
| dh3 | likewise, this morning :) | 11:17 |
| oneswig_ | Are they at the same time, how does that work again? | 11:17 |
| oneswig_ | ah, no, they are a few weeks apart. I should read up | 11:19 |
| *** ociuhandu has quit IRC | 11:19 | |
| oneswig_ | Now registered for both. | 11:20 |
| oneswig_ | Last week's discussion on Open Infra Labs was an interesting one. Seeing Adjutant in the list of PTG projects reminded me they use that for user onboarding. | 11:21 |
| oneswig_ | I'd not heard of it before. | 11:22 |
| janders | (googling) | 11:22 |
| janders | sounds a bit like my bunch of ansible used for onboarding | 11:22 |
| janders | or maybe im wrong | 11:24 |
| oneswig_ | Right - I wasn't sure what it looks like to use. | 11:24 |
| dh3 | looks like it exposes some user self-service panels to Horizon too. We are using Cloudforms/ManageIQ for that | 11:24 |
| *** tetsuro has joined #openstack-meeting | 11:24 | |
| *** ttsiouts has joined #openstack-meeting | 11:25 | |
| janders | whats your feel around the next summit - will it be physical or virtual> | 11:25 |
| janders | s/>/? | 11:25 |
| oneswig_ | Berlin in October? I wonder | 11:26 |
| witek | Octoberfest in Sep has been canceled, but that's not exactly similar size | 11:26 |
| oneswig_ | I expect there will be a split: people keen to travel and others anxious to stay at home. It sounds like Germany is slowly relaxing its lockdown. I'm doubtful the US will be in a good spot by then though. | 11:27 |
| oneswig_ | Hi witek :-) | 11:27 |
| witek | hi | 11:27 |
| oneswig_ | If there's a physical summit at all, perhaps it would be hundreds of attendees, not thousands. | 11:28 |
| janders | right | 11:28 |
| janders | at this stage it looks like we won't be allowed out of the country even if we wanted | 11:28 |
| janders | but at least I got full refunds for OpenDev/PTG and ISC no worries | 11:29 |
| janders | *flight refunds | 11:29 |
| *** ttsiouts has quit IRC | 11:30 | |
| oneswig_ | Flight refunds - interesting. I heard a European budget airline, all the refunds team are "currently not working due to coronavirus" - but the teams for rebooking are available to process your request! :-) | 11:30 |
| *** rfolco has joined #openstack-meeting | 11:30 | |
| janders | Qantas seemed to have a "system issue" which caused all cancellations to be turned into flight credits not refunds | 11:31 |
| janders | but that was easy to fix with a little nagging | 11:31 |
| dh3 | priorities...! we/Sanger went "no business travel" weeks before lockdown so I expect they will be cautious | 11:31 |
| *** ttsiouts has joined #openstack-meeting | 11:32 | |
| janders | much business travel turned into budget travel after 911 & GFC | 11:32 |
| janders | I wonder if this will go a step further and turn into videoconferencing instead | 11:33 |
| oneswig_ | It's an interesting point as opinion on travelling will have a large amount of subjective disposition to risk | 11:33 |
| janders | in many cases that wouldn't be a bad thing | 11:33 |
| oneswig_ | If I had a fitbit, my daily step count these days would be ~100... | 11:34 |
| dh3 | depends on the videoconf platform I think... I bailed from one virtual conference because they used AdobeConnect and it was too unreliable (audio dropping, screen shares failing) | 11:34 |
| oneswig_ | AdobeConnect - not heard of that. Everyone has a pet hate I think. Skype for Business never feels like fun. | 11:34 |
| janders | dont get me started on webex | 11:35 |
| janders | teams is probably the least bad microsoft product on the market though | 11:35 |
| oneswig_ | Can you tell which platforms are built on OpenStack? | 11:35 |
| oneswig_ | janders: teams, I have a name for my pain. | 11:36 |
| *** ociuhandu has joined #openstack-meeting | 11:36 | |
| janders | its the least bad tool we use here | 11:36 |
| janders | i deliberately dont use any reference to "good" or "better" though :P | 11:37 |
| *** raildo has joined #openstack-meeting | 11:37 | |
| *** witek has quit IRC | 11:39 | |
| *** yamamoto has joined #openstack-meeting | 11:40 | |
| *** witek has joined #openstack-meeting | 11:40 | |
| *** jamesmcarthur has joined #openstack-meeting | 11:40 | |
| *** ociuhandu has quit IRC | 11:41 | |
| oneswig_ | janders: got the VFLAG stuff working better. | 11:41 |
| janders | great! | 11:41 |
| janders | so - does it provide performance+redundancy? | 11:41 |
| oneswig_ | Upgrade to OVS 2.12 should have been redeploy - I think there were DB schema migration issues. | 11:41 |
| janders | what's the aggregate bandwidth? | 11:41 |
| *** yamamoto has quit IRC | 11:42 | |
| oneswig_ | Redundancy I haven't checked, performance is good but I only have it as an aggregate of clients on 5 hypervisors so far. | 11:42 |
| oneswig_ | I'm planning to get some meaningful data this afternoon, will put a graph on the SIG Slack channel if it's sufficiently pretty | 11:42 |
| janders | i have no updates on my cx6-ovs issues | 11:42 |
| oneswig_ | That's been a long-running issue for you | 11:43 |
| janders | mlnx are doing some testing in the lab to see if it is a generic bug or sth to do with my setup | 11:43 |
| janders | yeah | 11:43 |
| janders | we have some workarounds though | 11:43 |
| janders | but overall vpi-eth-cx6 wasnt smooth sailing | 11:43 |
| janders | cx6-ib - no worries | 11:43 |
| janders | but overall i dont really see a point... for eth cx5 is more than sufficient | 11:44 |
| janders | next stop pcie4/5 and 400GE I suppose | 11:44 |
| oneswig_ | I think there are some ipsec offloads in CX6 I was interested in testing. | 11:45 |
| janders | for ib it does make sense esp for storage | 11:45 |
| janders | its very very nice | 11:45 |
| oneswig_ | Also the NVMEoF support, I think there was something neat it did with presentation of VFs? | 11:45 |
| janders | that is possible | 11:45 |
| oneswig_ | not much help if it fails on the "basics" though | 11:46 |
| janders | pcie4 cx6-eth might be more smooth | 11:46 |
| janders | i sense that hacks around pcie3 limitations dont mix with ethernet all that well | 11:46 |
| janders | but - will know for sure when we have full explanation of the ovs issues | 11:46 |
| janders | and a fix | 11:46 |
| oneswig_ | janders: are you a Cumulus customer? | 11:48 |
| janders | not really | 11:48 |
| janders | its a little funny actually | 11:48 |
| janders | i ended up with some cumulus switches for.... ipmi | 11:49 |
| janders | but not using any SDN stuff there | 11:49 |
| janders | just dumb switches | 11:49 |
| janders | as in they are used as dumb switches | 11:49 |
| oneswig_ | I wonder what it's prognosis is as an open net os. I suspect vendors like Dell will take a dim view | 11:49 |
| oneswig_ | I don't have anything more to add - janders? | 11:52 |
| janders | i think im good | 11:52 |
| janders | thank you all - and stay safe | 11:52 |
| janders | till next time | 11:52 |
| oneswig_ | It must be late in canberra... | 11:52 |
| janders | 2152 | 11:52 |
| oneswig_ | Until next time! | 11:52 |
| oneswig_ | #endmeeting | 11:53 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 11:53 | |
| openstack | Meeting ended Wed May 6 11:53:03 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 11:53 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-05-06-11.01.html | 11:53 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-05-06-11.01.txt | 11:53 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-05-06-11.01.log.html | 11:53 |
| dh3 | bye! | 11:53 |
| *** dh3 has quit IRC | 11:53 | |
| *** janders has quit IRC | 11:53 | |
| oneswig_ | cheerio! | 11:53 |
| *** oneswig_ has quit IRC | 11:53 | |
| *** andrebeltrami has joined #openstack-meeting | 12:01 | |
| *** rh-jelabarre has joined #openstack-meeting | 12:02 | |
| *** rh-jelabarre has quit IRC | 12:02 | |
| *** rh-jelabarre has joined #openstack-meeting | 12:03 | |
| *** ykatabam has quit IRC | 12:09 | |
| *** rpittau|bbl is now known as rpittau | 12:18 | |
| *** ykatabam has joined #openstack-meeting | 12:19 | |
| *** ociuhandu has joined #openstack-meeting | 12:25 | |
| *** jamesmcarthur has quit IRC | 12:25 | |
| *** ykatabam has quit IRC | 12:37 | |
| *** ykatabam has joined #openstack-meeting | 12:37 | |
| *** tetsuro has quit IRC | 12:41 | |
| *** vkmc__ has joined #openstack-meeting | 12:50 | |
| *** vkmc__ is now known as vkmc | 12:52 | |
| *** hyunsikyang has joined #openstack-meeting | 12:58 | |
| *** jamesmcarthur has joined #openstack-meeting | 13:01 | |
| *** hyunsikyang__ has quit IRC | 13:02 | |
| *** ralonsoh has quit IRC | 13:08 | |
| *** ralonsoh has joined #openstack-meeting | 13:11 | |
| *** rfolco is now known as rfolco|rover | 13:13 | |
| *** eharney has joined #openstack-meeting | 13:17 | |
| *** moguimar has joined #openstack-meeting | 13:18 | |
| *** cgoncalves has quit IRC | 13:19 | |
| *** jamesmcarthur has quit IRC | 13:21 | |
| *** cgoncalves has joined #openstack-meeting | 13:25 | |
| *** ykatabam has quit IRC | 13:30 | |
| *** TrevorV has joined #openstack-meeting | 13:31 | |
| *** dmacpher has joined #openstack-meeting | 13:33 | |
| *** maciejjozefczyk_ is now known as maciejjozefczyk | 13:37 | |
| *** ttsiouts has quit IRC | 13:39 | |
| *** dmacpher_ has joined #openstack-meeting | 13:39 | |
| *** dmacpher has quit IRC | 13:41 | |
| *** ykatabam has joined #openstack-meeting | 13:53 | |
| *** LiangFang has joined #openstack-meeting | 13:59 | |
| *** dmacpher__ has joined #openstack-meeting | 14:00 | |
| *** jamesmcarthur has joined #openstack-meeting | 14:01 | |
| *** dmacpher_ has quit IRC | 14:03 | |
| *** ttsiouts has joined #openstack-meeting | 14:08 | |
| *** ttsiouts has quit IRC | 14:13 | |
| *** ociuhandu has quit IRC | 14:15 | |
| *** ykatabam has quit IRC | 14:16 | |
| *** ociuhandu has joined #openstack-meeting | 14:16 | |
| *** ociuhandu has quit IRC | 14:20 | |
| *** ttsiouts has joined #openstack-meeting | 14:22 | |
| *** jamesmcarthur has quit IRC | 14:22 | |
| *** jhesketh has quit IRC | 14:22 | |
| *** ociuhandu has joined #openstack-meeting | 14:24 | |
| *** belmoreira has quit IRC | 14:25 | |
| *** munimeha1 has joined #openstack-meeting | 14:28 | |
| *** lpetrut has quit IRC | 14:30 | |
| *** ttsiouts has quit IRC | 14:38 | |
| *** belmoreira has joined #openstack-meeting | 14:44 | |
| *** dmacpher__ has quit IRC | 14:45 | |
| *** dklyle has joined #openstack-meeting | 14:48 | |
| *** mlavalle has joined #openstack-meeting | 14:49 | |
| *** dmacpher has joined #openstack-meeting | 14:49 | |
| *** jhesketh has joined #openstack-meeting | 14:51 | |
| *** belmoreira has quit IRC | 14:54 | |
| *** belmoreira has joined #openstack-meeting | 14:56 | |
| *** smcginnis has quit IRC | 14:58 | |
| *** lpetrut has joined #openstack-meeting | 15:00 | |
| *** smcginnis has joined #openstack-meeting | 15:04 | |
| *** belmoreira has quit IRC | 15:11 | |
| *** ttsiouts has joined #openstack-meeting | 15:13 | |
| *** ociuhandu has quit IRC | 15:13 | |
| *** ociuhandu has joined #openstack-meeting | 15:14 | |
| *** ociuhandu has quit IRC | 15:16 | |
| *** ociuhandu has joined #openstack-meeting | 15:16 | |
| *** LiangFang has quit IRC | 15:18 | |
| *** ttsiouts has quit IRC | 15:18 | |
| *** belmoreira has joined #openstack-meeting | 15:23 | |
| *** yasufum has joined #openstack-meeting | 15:30 | |
| *** njohnston is now known as njohnston_ | 15:31 | |
| *** njohnsto_ has joined #openstack-meeting | 15:31 | |
| *** njohnsto_ is now known as njohnston | 15:35 | |
| *** belmoreira has quit IRC | 15:36 | |
| *** belmoreira has joined #openstack-meeting | 15:39 | |
| *** jmasud has joined #openstack-meeting | 15:41 | |
| *** dmacpher has quit IRC | 15:49 | |
| *** ociuhandu has quit IRC | 15:50 | |
| *** ociuhandu has joined #openstack-meeting | 15:51 | |
| *** dmacpher has joined #openstack-meeting | 15:51 | |
| *** gyee has joined #openstack-meeting | 15:53 | |
| *** jmasud has quit IRC | 15:55 | |
| *** ociuhandu has quit IRC | 15:56 | |
| *** maciejjozefczyk has quit IRC | 16:01 | |
| *** njohnston has quit IRC | 16:01 | |
| *** ociuhandu has joined #openstack-meeting | 16:01 | |
| *** jmasud has joined #openstack-meeting | 16:04 | |
| *** ttsiouts has joined #openstack-meeting | 16:05 | |
| *** maciejjozefczyk has joined #openstack-meeting | 16:06 | |
| *** maciejjozefczyk_ has joined #openstack-meeting | 16:07 | |
| *** witek has quit IRC | 16:08 | |
| *** maciejjozefczyk has quit IRC | 16:11 | |
| *** ociuhandu has quit IRC | 16:13 | |
| *** ociuhandu has joined #openstack-meeting | 16:14 | |
| *** ociuhandu has quit IRC | 16:19 | |
| *** jamesmcarthur has joined #openstack-meeting | 16:20 | |
| *** jmasud has quit IRC | 16:21 | |
| *** jmasud has joined #openstack-meeting | 16:23 | |
| *** ttsiouts has quit IRC | 16:23 | |
| *** jmasud has quit IRC | 16:23 | |
| *** rpittau is now known as rpittau|afk | 16:24 | |
| *** priteau has quit IRC | 16:25 | |
| *** ociuhandu has joined #openstack-meeting | 16:33 | |
| *** evrardjp has quit IRC | 16:36 | |
| *** evrardjp has joined #openstack-meeting | 16:36 | |
| *** ttsiouts has joined #openstack-meeting | 16:41 | |
| *** psachin has quit IRC | 16:41 | |
| *** ttsiouts has quit IRC | 16:51 | |
| *** jamesmcarthur has quit IRC | 16:55 | |
| *** witek has joined #openstack-meeting | 16:57 | |
| *** maciejjozefczyk has joined #openstack-meeting | 17:18 | |
| *** maciejjozefczyk_ has quit IRC | 17:20 | |
| *** irclogbot_0 has quit IRC | 17:20 | |
| *** maciejjozefczyk has quit IRC | 17:21 | |
| *** maciejjozefczyk has joined #openstack-meeting | 17:21 | |
| *** jamesmcarthur has joined #openstack-meeting | 17:21 | |
| *** maciejjozefczyk_ has joined #openstack-meeting | 17:22 | |
| *** irclogbot_0 has joined #openstack-meeting | 17:23 | |
| *** maciejjozefczyk_ has quit IRC | 17:23 | |
| *** ociuhandu has quit IRC | 17:26 | |
| *** maciejjozefczyk has quit IRC | 17:26 | |
| *** jamesmcarthur has quit IRC | 17:37 | |
| *** jamesmcarthur has joined #openstack-meeting | 17:38 | |
| *** ociuhandu has joined #openstack-meeting | 17:38 | |
| *** ociuhandu has quit IRC | 17:43 | |
| *** ttsiouts has joined #openstack-meeting | 17:45 | |
| *** ttsiouts has quit IRC | 17:49 | |
| *** ociuhandu has joined #openstack-meeting | 18:02 | |
| *** witek has quit IRC | 18:12 | |
| *** yasufum has quit IRC | 18:19 | |
| *** ralonsoh has quit IRC | 18:19 | |
| *** lpetrut has quit IRC | 18:22 | |
| *** e0ne has quit IRC | 18:34 | |
| *** bbowen has quit IRC | 18:41 | |
| *** bbowen has joined #openstack-meeting | 18:44 | |
| *** ociuhandu has quit IRC | 18:48 | |
| *** JangwonLee has quit IRC | 18:57 | |
| *** JangwonLee has joined #openstack-meeting | 18:58 | |
| *** jmasud has joined #openstack-meeting | 19:04 | |
| *** jmasud has quit IRC | 19:18 | |
| *** jamesmcarthur has quit IRC | 19:25 | |
| *** jamesmcarthur has joined #openstack-meeting | 19:30 | |
| *** jamesdenton has quit IRC | 19:48 | |
| *** ociuhandu has joined #openstack-meeting | 19:53 | |
| *** ayoung has joined #openstack-meeting | 19:54 | |
| *** belmoreira has quit IRC | 19:58 | |
| *** jamesdenton has joined #openstack-meeting | 20:04 | |
| *** jamesden_ has joined #openstack-meeting | 20:08 | |
| *** belmoreira has joined #openstack-meeting | 20:10 | |
| *** jamesdenton has quit IRC | 20:11 | |
| *** diablo_rojo has joined #openstack-meeting | 20:13 | |
| *** ociuhandu has quit IRC | 20:13 | |
| *** jamesdenton has joined #openstack-meeting | 20:21 | |
| *** jamesden_ has quit IRC | 20:22 | |
| *** belmoreira has quit IRC | 20:25 | |
| *** ociuhandu has joined #openstack-meeting | 20:46 | |
| *** jamesmcarthur has quit IRC | 20:50 | |
| *** ayoung has quit IRC | 20:51 | |
| *** jamesmcarthur has joined #openstack-meeting | 20:51 | |
| *** ociuhandu has quit IRC | 20:54 | |
| *** patchbot has joined #openstack-meeting | 20:54 | |
| *** ayoung has joined #openstack-meeting | 20:55 | |
| timburke | #startmeeting swift | 21:00 |
| openstack | Meeting started Wed May 6 21:00:05 2020 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:00 |
| *** openstack changes topic to " (Meeting topic: swift)" | 21:00 | |
| openstack | The meeting name has been set to 'swift' | 21:00 |
| timburke | who's here for the swift meeting? | 21:00 |
| seongsoocho | o/ | 21:00 |
| rledisez | hi o/ | 21:00 |
| kota_ | hi | 21:00 |
| mattoliverau | o/ | 21:00 |
| alecuyer | o/ | 21:01 |
| clayg | o/ | 21:01 |
| timburke | agenda's at https://wiki.openstack.org/wiki/Meetings/Swift | 21:02 |
| timburke | #topic PTG | 21:02 |
| *** openstack changes topic to "PTG (Meeting topic: swift)" | 21:02 | |
| timburke | mattoliverau very graciously offered to help with PTG planning/organization | 21:02 |
| clayg | ❤️ mattoliverau | 21:03 |
| mattoliverau | hey cool. | 21:03 |
| timburke | mattoliverau, what have you learned? | 21:03 |
| mattoliverau | So we need to get the registraion in and start putting in sessions to times | 21:03 |
| mattoliverau | sorry, irc is lagging | 21:04 |
| mattoliverau | So step 1, to get the registration in, I need to know if there are any other projects your interested in so we can try and avoid overlap | 21:04 |
| mattoliverau | I already have storelets and first contact sig | 21:04 |
| rledisez | I would be interested in Keystone (things like operator feedbacks etc…) | 21:05 |
| mattoliverau | Next we need to know what time suits people because of timezones. For this I'll come up with a doodle poll and post the link in our channel. | 21:05 |
| mattoliverau | rledisez: great! | 21:06 |
| mattoliverau | I'll hold the doodle pool, once I've created it, open until the end of Friday (my time, though can wait a bit longer) just so people have a change to pick times. | 21:07 |
| mattoliverau | I'm guessing meeting time isn't bad, but I'm happy to get up early or stay up late if need be. | 21:07 |
| timburke | i'm feeling the same way -- my current plan is to just show up in irc as much as possible that week :-D | 21:08 |
| kota_ | yup. in the day time, it might be hard because kids interrupt me always. | 21:08 |
| mattoliverau | But step 3, is to make sure all the topics you want to talk about is in the etherpad! So when we have times we can decide the number of blocks and get rooms booked. | 21:09 |
| kota_ | oic | 21:09 |
| mattoliverau | most of this I think happens by the 10th, though this could just be the registration side. | 21:09 |
| timburke | #link https://etherpad.opendev.org/p/swift-ptg-victoria | 21:09 |
| clayg | so there IS an in-person presence? | 21:10 |
| mattoliverau | but I'd like to get some virtual rooms booked before what ever our best times get taken by other projects | 21:10 |
| clayg | oh, virtual rooms | 21:10 |
| *** jamesmcarthur has quit IRC | 21:10 | |
| mattoliverau | *needs to happen by | 21:10 |
| mattoliverau | room booking spreadsheet | 21:11 |
| mattoliverau | #link https://ethercalc.openstack.org/126u8ek25noy | 21:11 |
| mattoliverau | if you want to see what it currently looks like. | 21:11 |
| timburke | kota_, otoh, we'll get to know each other's families so much better than we typically get just from pictures :-) | 21:11 |
| kota_ | timburke: :) | 21:12 |
| mattoliverau | lol | 21:12 |
| kota_ | good idea | 21:12 |
| mattoliverau | Anyway, sorry for the brain dump | 21:12 |
| timburke | don't apologize! that was just the sort of overview i was hoping for and never got around to putting together myself | 21:13 |
| timburke | again, thank you so much for taking that on, and sorry i didn't ask for help earlier | 21:13 |
| mattoliverau | basically, 1. If ther eis any project your interested in, let me know; 2. fill out doodle poll once I get a link up later today; 3. update the etherpad; | 21:13 |
| mattoliverau | timburke: nps | 21:14 |
| timburke | #topic object updater | 21:14 |
| *** openstack changes topic to "object updater (Meeting topic: swift)" | 21:14 | |
| clayg | updater 😡 | 21:14 |
| timburke | rledisez, thanks for putting this on the agenda! i think you may have noticed that we've been interested in this lately, too ;-) | 21:14 |
| rledisez | Yeah, I though I bring the point here cause some of us are having issue with it (at least we do, at ovh :)) | 21:15 |
| rledisez | there is mostly 2 issues in my mind: | 21:15 |
| rledisez | the first one is that async-pendings can quickly piles up on disks for many reasons (unsharded bug container, network issue, process hanged, …) | 21:16 |
| *** jamesmcarthur has joined #openstack-meeting | 21:16 | |
| rledisez | and while some of them will never be able to be handled by the updater (at least without an operator intervention), some of them can be handled because it was a really transiant situation (like a switch reboot…) | 21:16 |
| rledisez | the first review is about that: p 571917 | 21:17 |
| patchbot | https://review.opendev.org/#/c/571917/ - swift - Manage async_pendings priority per containers - 5 patch sets | 21:17 |
| rledisez | the blocking point seems to be that it changes the way async pendings are stored | 21:17 |
| rledisez | the second issue is that the way we communicate with container-server is by sending one request per async-pendings instead of batching them | 21:18 |
| rledisez | that what p 724943 is about | 21:18 |
| patchbot | https://review.opendev.org/#/c/724943/ - swift - WIP: Batched updates for object-updater - 2 patch sets | 21:18 |
| rledisez | I though I would bring this here to have some other oint of view | 21:18 |
| rledisez | (I'm done with the summary. any questions, remarks?) | 21:19 |
| timburke | so a bit of perspective from clayg tdasilva and i: we've got a cluster that's filling up, leading to quotas being implemented, leading to users wanting to delete a good bit of data, often in fairly large containers | 21:19 |
| clayg | it sounds like the issue we're having might be slightly different then - we accepted ~350M deletes into some sharded containers with billions of objects and the updaters keep dos-ing the container dbs 🤷♂️ | 21:20 |
| timburke | we're currently sitting at like 450+M async pendings across ~250 nodes, and that's still going up by ~2.5M/hr | 21:21 |
| alecuyer | ouch | 21:21 |
| clayg | there's just no flow control across the nodes to try and put db updates in at "the correct rate" | 21:21 |
| rledisez | clayg: yeah, I'm more looking into treating quickly what can be treated while still trying for the problematic containers | 21:21 |
| clayg | we're also learning there's still lots of OTHER updates going into these same containers so we're trying to break up the work and prioritize stuff | 21:22 |
| clayg | rledisez: yeah on a "per node" basis we need some way to have "bad containers" somehow get... I guess "error limited" or something like what you've done in the top-of-stack patch where it just "moves on" | 21:22 |
| clayg | I'd really like it AT LEAST per-node we could have a per-container rate limit | 21:23 |
| timburke | fwiw, "treating quickly what can be treated" is actually *exactly* what clayg did earlier this week -- run a filtered updater that ignores certain containers, then try running foreground updaters for the remaining ones | 21:23 |
| clayg | yeah I don't guess I have that gist up just now, one sec | 21:25 |
| rledisez | timburke: I saw that tool, there is a link in the review. my issue with it is that it has to open all async-pendings to filter them. I really want to avoid wasting I/O on that (are you running on SSD guys?) | 21:25 |
| clayg | oh, no ... i did, just lost it -> https://gist.github.com/clayg/c3d31a62eba590eebd5f5d257c24a297 | 21:26 |
| clayg | anyways - this is *useful* but not very scalable in terms of operator friendly | 21:26 |
| *** slaweq has quit IRC | 21:26 | |
| timburke | no, we just took the io hit -- worst case, it slows down object servers which might apply some backpressure to the clients :P | 21:27 |
| timburke | fwiw, another thought i'd had recently was to make the number of successes required to unlink configurable -- if we can get the update to 2/3 replicas, that's *probably* good enough, right? | 21:28 |
| clayg | we're not on SSDs but we do have SOME head room on iops | 21:28 |
| timburke | let container replication square it | 21:28 |
| *** jmasud has joined #openstack-meeting | 21:28 | |
| clayg | I hadn't really considered the overhead of opening an async to parse it only to find out that container is ratelimited 🤔 | 21:28 |
| rledisez | how does replication work on big-unsharded container? I'm not sure it would do a better job | 21:28 |
| clayg | @timburke wants to just put all the async updates in a database - I don't want to have to deal with another .pending file lock timeouts | 21:29 |
| timburke | i mean, they're big... but not *that* big. ~20M rows or so, i think? | 21:29 |
| clayg | timburke: I think rledisez was asking about *un* sharded | 21:30 |
| clayg | yeah, sharding was primarily in my mind to fix container replication | 21:30 |
| timburke | right, but i mean, we could shard that big shard -- we just haven't | 21:30 |
| clayg | replication works great on the shards! | 21:30 |
| clayg | timburke: yeah, we have more sharding to do | 21:30 |
| timburke | we've been pretty good about sharding the *biggest* guys, we've only got like 2 containers over 50M | 21:31 |
| timburke | one of them is actually itself a shard 🤔 | 21:31 |
| mattoliverau | then shard the shard :) | 21:31 |
| alecuyer | I'm wondering if this is "fixable" without throttling DELETEs? Unless you have excess IO capacity in your container servers, something is always going to be lagging in your situation, no? (it would be nice of course to prioritize some things but still) | 21:32 |
| clayg | we DO have IO headroom in the container dbs tho | 21:32 |
| alecuyer | ok so it's sqlite contention? | 21:33 |
| clayg | yeah it's some kind of locking - either we're doing or sqlite | 21:33 |
| clayg | yesterday it looked like the replication UPDATE request locked up the db for 25s - then FAILED | 21:34 |
| clayg | so we're leaving throughput on the floor and it's unclear we're making progress - we have more investigation to | 21:34 |
| clayg | I'll think more on the ordering asyncs by containers - I'm generally pretty happy with the filesystem layout | 21:34 |
| *** e0ne has joined #openstack-meeting | 21:35 | |
| rledisez | with a new layout, it would be pretty easy to batch updates to send to container as they are grouped all together | 21:36 |
| clayg | it's hadn't been obviously terrible to me that a cycle of the updater would read and open the 125K asyncs per-disk | 21:36 |
| rledisez | it was actually a followup I wanted on the first patch (and also to move "legacy" async-pendings) | 21:36 |
| mattoliverau | re container replication, if the 2 container are generally close in side, we use usync (ie sync a bunch of rows) so maybe only needing quruom successes and letting the container possibly usync (batch with other updates) is better, rather then waiting for all replicas. Of course large unsharded will always be an issue.. we just need to shard the buggers. | 21:36 |
| timburke | and i need to work on getting a good process going for manual shrinking -- part of why i'm hesitant to shard is that i think a lot of those shards are going to end up mostly-empty once everything settles, and we haven't really invested in shrinking yet :-/ | 21:37 |
| clayg | yeah, the batch updates may indeed be useful ... rledisez you're winning me over on the layout change | 21:37 |
| rledisez | yay! can I offer you a virtual-beer during the virtual-ptg? | 21:38 |
| mattoliverau | I like the idea of splitting the asyncs by container in the sense from a glance you can see how if containers are stuggling, but it that too much directory walking i/o, ie listdir? Maybe one per partition? which will map to container replicas | 21:38 |
| clayg | at this point my biggest complaint to change it probably just reservation about changing on-disk layouts and legacy migrations etc | 21:39 |
| mattoliverau | yeah | 21:39 |
| clayg | it's a bunch of work - but maybe it's worth it - thanks for bringing this up | 21:39 |
| clayg | I don't think I had a good picture of where your thinking was coming from - it's clearer now | 21:39 |
| rledisez | so, the current patch is compatible with current layout, so no break during an update. in case of downgrade, some move of files would be required | 21:39 |
| clayg | mattoliverau: yeah some workloads I'd seen had a BUNCH of containers in their cluster | 21:40 |
| clayg | I think of all the dirs we create if a node is offline for awhile | 21:40 |
| clayg | instead of "a handful" of problematic containers we get one TLD for each container on a node - which... might still be less 1M - but in the 100Ks | 21:40 |
| rledisez | the cost of listing a directory of 100K entries is not big, but the cost of inserting a new one is not negligeable | 21:41 |
| rledisez | (i made some measure during my tests) | 21:41 |
| mattoliverau | timburke: yeah, mark as SHRINKING And maybe get to tool to search for the donor etc. | 21:42 |
| timburke | hmm... i wonder if a db could still be a good idea -- have the object-server continue dropping files all over the fs, then have the updater walk that tree and load into db before fanning out workers to read from the db... | 21:42 |
| alecuyer | sounds good, I'm afraid of having too many files on disk, wonder why :) | 21:42 |
| rledisez | why dropping a file then? with the WAL, insertion should be quite fast, no? | 21:43 |
| mattoliverau | use the general task queue.. and just hope that container gets updates so we don't have to deal with async.. damn :P | 21:43 |
| clayg | mattoliverau: 😆 | 21:43 |
| rledisez | I would then even move to appending to a file, just to avoid too many fsync | 21:43 |
| *** e0ne has quit IRC | 21:44 | |
| timburke | ok, this has been a good discussion. are there any decisions or action items we can take away from it? | 21:45 |
| *** raildo_ has joined #openstack-meeting | 21:45 | |
| *** smcginnis has quit IRC | 21:46 | |
| rledisez | should we investigate the DB idea? | 21:46 |
| rledisez | I'm pretty sure it would be better, but it's also more work so won't be ready soon | 21:46 |
| alecuyer | (I have to say I have no updates on LOSF having had no time to work on it this week. rledisez left it on the agenda as I should get some time next week. So, we can have more time for object updater or other topics) | 21:47 |
| timburke | alecuyer, thanks for hte heads-up | 21:47 |
| clayg | i need to investigate the problem in our cluster - that's ahead of me making a decision on the suitability of rledisez 's purposed layout change - but I'd like to review that more seriously given new perspective and anything I learn trying to fix our mess | 21:47 |
| *** raildo has quit IRC | 21:48 | |
| timburke | ok, one more crazy idea, that might be a somewhat cheaper way to investigate the db idea: what about a db per container? we could put it in the disk's containers/ driectory... | 21:48 |
| clayg | I was seriously attacted to this idea because of leverage | 21:50 |
| rledisez | yeah, we can just run the container-replicator then | 21:50 |
| clayg | "can't stuff it in the primary container - just put it in a local handoff!" | 21:50 |
| timburke | i mean, we've already got this db schema for tracking exactly the info that's in these updates... | 21:50 |
| clayg | hehhe | 21:50 |
| clayg | i was concerned that for AC/O clusters there might be some distaste to adding a container-replicator to your object layer | 21:51 |
| timburke | *especially* for the shards -- then you run no risk of proxies getting getting bad acls from the handoff that got popped into existence when the primaries are overloaded | 21:51 |
| clayg | we could just import the container-replicator into the updater and ... well do something | 21:51 |
| timburke | true enough! we've already got the sharder doing something not so dissimilar | 21:52 |
| clayg | timburke: yeah vivifying containers in the read path is probably not ideal - i was thinking out of band | 21:52 |
| mattoliverau | as handoff conatiners? if so just make sure you get the rowids close to their parent. otherwise it might cuase a rsync_then_merge and that wouldn't go well on really large containers. | 21:53 |
| clayg | mattoliverau: good point | 21:54 |
| *** jamesmcarthur has quit IRC | 21:55 | |
| rledisez | we can avoid that by tuning the limits on the async updater/replicator I guess. but yes, something to take care of for sure | 21:55 |
| timburke | i should read the rsync-then-merge code again... | 21:56 |
| timburke | all right | 21:56 |
| timburke | #topic open discussion | 21:56 |
| *** openstack changes topic to "open discussion (Meeting topic: swift)" | 21:56 | |
| timburke | anything else we should talk about in these last few minutes of meeting time? | 21:56 |
| mattoliverau | nope, /me wants breakfst ;) | 21:58 |
| clayg | nom nom | 21:58 |
| clayg | i might cook eggs and bacon for dinner 🤔 | 21:58 |
| mattoliverau | you totally should! :) | 21:59 |
| kota_ | it seems kids woke up. | 21:59 |
| timburke | :-) then this is as good a time as any to | 21:59 |
| timburke | #endmeeting | 21:59 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 21:59 | |
| openstack | Meeting ended Wed May 6 21:59:41 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 21:59 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/swift/2020/swift.2020-05-06-21.00.html | 21:59 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/swift/2020/swift.2020-05-06-21.00.txt | 21:59 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/swift/2020/swift.2020-05-06-21.00.log.html | 21:59 |
| timburke | oh! and as always, thank you all for coming, and thank you for working on swift :-) | 22:00 |
| *** raildo_ has quit IRC | 22:01 | |
| *** smcginnis has joined #openstack-meeting | 22:05 | |
| *** andrebeltrami has quit IRC | 22:11 | |
| *** ayoung has quit IRC | 22:16 | |
| *** jmasud has quit IRC | 22:17 | |
| *** ayoung has joined #openstack-meeting | 22:17 | |
| *** jmasud has joined #openstack-meeting | 22:18 | |
| *** ykatabam has joined #openstack-meeting | 22:25 | |
| *** jmasud has quit IRC | 22:31 | |
| *** jmasud has joined #openstack-meeting | 22:32 | |
| *** jmasud has quit IRC | 22:46 | |
| *** TrevorV has quit IRC | 22:54 | |
| *** jmasud has joined #openstack-meeting | 23:03 | |
| *** rh-jelabarre has quit IRC | 23:07 | |
| *** ociuhandu has joined #openstack-meeting | 23:09 | |
| *** ociuhandu has quit IRC | 23:15 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!