| *** yamamoto has quit IRC | 00:03 | |
| *** rmcall has quit IRC | 00:06 | |
| *** tetsuro has joined #openstack-meeting | 00:09 | |
| *** tetsuro_ has joined #openstack-meeting | 00:10 | |
| *** jamesmcarthur has quit IRC | 00:11 | |
| *** tetsuro has quit IRC | 00:13 | |
| *** jamesmcarthur has joined #openstack-meeting | 00:22 | |
| *** yasufum has joined #openstack-meeting | 00:28 | |
| *** yasufum has quit IRC | 00:33 | |
| *** yasufum has joined #openstack-meeting | 00:33 | |
| *** gyee has quit IRC | 00:36 | |
| *** armax has joined #openstack-meeting | 00:57 | |
| *** ricolin_ has joined #openstack-meeting | 00:59 | |
| *** jamesmcarthur has quit IRC | 01:03 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:05 | |
| *** jmasud has quit IRC | 01:15 | |
| *** yamamoto has joined #openstack-meeting | 01:26 | |
| *** jamesmcarthur has quit IRC | 01:30 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:51 | |
| *** yamamoto has quit IRC | 02:00 | |
| *** apetrich has quit IRC | 02:14 | |
| *** rmcall has joined #openstack-meeting | 02:31 | |
| *** tetsuro_ has quit IRC | 02:40 | |
| *** jamesmcarthur has quit IRC | 02:43 | |
| *** jamesmcarthur has joined #openstack-meeting | 02:43 | |
| *** jamesmcarthur has quit IRC | 02:47 | |
| *** rcernin has quit IRC | 02:56 | |
| *** rcernin has joined #openstack-meeting | 02:58 | |
| *** ociuhandu has joined #openstack-meeting | 03:01 | |
| *** ociuhandu has quit IRC | 03:06 | |
| *** manpreet has joined #openstack-meeting | 03:13 | |
| *** rfolco has quit IRC | 03:14 | |
| *** redrobot has quit IRC | 03:36 | |
| *** armax has quit IRC | 03:38 | |
| *** psachin has joined #openstack-meeting | 03:39 | |
| *** jamesmcarthur has joined #openstack-meeting | 03:43 | |
| *** jamesmcarthur has quit IRC | 03:45 | |
| *** jamesmcarthur_ has joined #openstack-meeting | 03:45 | |
| *** jmasud has joined #openstack-meeting | 03:52 | |
| *** rfolco has joined #openstack-meeting | 03:52 | |
| *** rfolco has quit IRC | 03:56 | |
| *** yamamoto has joined #openstack-meeting | 03:59 | |
| *** yamamoto has quit IRC | 03:59 | |
| *** yamamoto has joined #openstack-meeting | 04:00 | |
| *** yaawang has quit IRC | 04:10 | |
| *** markvoelker has joined #openstack-meeting | 04:11 | |
| *** yaawang has joined #openstack-meeting | 04:11 | |
| *** markvoelker has quit IRC | 04:15 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-meeting | 04:33 | |
| *** masahito has joined #openstack-meeting | 04:34 | |
| *** vishalmanchanda has joined #openstack-meeting | 04:50 | |
| *** markvoelker has joined #openstack-meeting | 05:01 | |
| *** markvoelker has quit IRC | 05:05 | |
| *** links has joined #openstack-meeting | 05:11 | |
| *** diurnalist has quit IRC | 05:13 | |
| *** yaawang has quit IRC | 05:16 | |
| *** yaawang has joined #openstack-meeting | 05:16 | |
| *** bbowen has quit IRC | 05:32 | |
| *** bbowen has joined #openstack-meeting | 05:34 | |
| *** diurnalist has joined #openstack-meeting | 05:49 | |
| *** diurnalist has quit IRC | 05:53 | |
| *** psahoo has joined #openstack-meeting | 05:55 | |
| *** jamesmcarthur_ has quit IRC | 05:57 | |
| *** diurnalist has joined #openstack-meeting | 05:59 | |
| *** osmanlicilegi has quit IRC | 06:02 | |
| *** diurnalist has quit IRC | 06:04 | |
| *** osmanlicilegi has joined #openstack-meeting | 06:06 | |
| *** yasufum has quit IRC | 06:12 | |
| *** yasufum has joined #openstack-meeting | 06:12 | |
| *** bbowen has quit IRC | 06:17 | |
| *** psahoo has quit IRC | 06:22 | |
| *** yaawang has quit IRC | 06:26 | |
| *** yaawang has joined #openstack-meeting | 06:27 | |
| *** bbowen has joined #openstack-meeting | 06:29 | |
| *** dklyle has quit IRC | 06:35 | |
| *** psahoo has joined #openstack-meeting | 06:40 | |
| *** diurnalist has joined #openstack-meeting | 06:41 | |
| *** diurnalist has quit IRC | 06:45 | |
| *** yasufum has quit IRC | 06:47 | |
| *** ralonsoh has joined #openstack-meeting | 06:48 | |
| *** masahito has quit IRC | 06:50 | |
| *** takamatsu has quit IRC | 06:59 | |
| *** zbr|ruck has quit IRC | 07:02 | |
| *** zbr|ruck has joined #openstack-meeting | 07:03 | |
| *** zbr has quit IRC | 07:06 | |
| *** rcernin has quit IRC | 07:07 | |
| *** ociuhandu has joined #openstack-meeting | 07:17 | |
| *** yasufum has joined #openstack-meeting | 07:17 | |
| *** rcernin has joined #openstack-meeting | 07:18 | |
| *** ociuhandu_ has joined #openstack-meeting | 07:28 | |
| *** ociuhandu has quit IRC | 07:31 | |
| *** zbr has joined #openstack-meeting | 07:41 | |
| *** tosky has joined #openstack-meeting | 07:46 | |
| *** e0ne has joined #openstack-meeting | 08:06 | |
| *** markvoelker has joined #openstack-meeting | 08:12 | |
| *** markvoelker has quit IRC | 08:17 | |
| *** yaawang has quit IRC | 08:24 | |
| *** rcernin has quit IRC | 08:27 | |
| *** yaawang has joined #openstack-meeting | 08:31 | |
| *** tetsuro has joined #openstack-meeting | 08:31 | |
| *** ociuhandu_ has quit IRC | 08:33 | |
| *** ociuhandu has joined #openstack-meeting | 08:33 | |
| *** tetsuro has quit IRC | 08:35 | |
| *** Lucas_Gray has joined #openstack-meeting | 08:45 | |
| *** rcernin has joined #openstack-meeting | 08:49 | |
| *** yamamoto has quit IRC | 08:55 | |
| *** yamamoto has joined #openstack-meeting | 09:00 | |
| *** yamamoto has quit IRC | 09:02 | |
| *** yasufum has quit IRC | 09:07 | |
| *** yamamoto has joined #openstack-meeting | 09:08 | |
| *** apetrich has joined #openstack-meeting | 09:12 | |
| *** psahoo has quit IRC | 09:14 | |
| *** rcernin has quit IRC | 09:17 | |
| *** psahoo has joined #openstack-meeting | 09:30 | |
| *** ricolin_ has quit IRC | 09:49 | |
| *** yaawang has quit IRC | 10:24 | |
| *** yaawang has joined #openstack-meeting | 10:25 | |
| *** jraju__ has joined #openstack-meeting | 10:29 | |
| *** links has quit IRC | 10:30 | |
| *** yasufum has joined #openstack-meeting | 10:47 | |
| *** oneswig has joined #openstack-meeting | 10:52 | |
| oneswig | #startmeeting scientific-sig | 11:01 |
|---|---|---|
| openstack | Meeting started Wed Jul 15 11:01:15 2020 UTC and is due to finish in 60 minutes. The chair is oneswig. Information about MeetBot at http://wiki.debian.org/MeetBot. | 11:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 11:01 |
| *** openstack changes topic to " (Meeting topic: scientific-sig)" | 11:01 | |
| openstack | The meeting name has been set to 'scientific_sig' | 11:01 |
| oneswig | halloo | 11:01 |
| *** yamamoto has quit IRC | 11:01 | |
| oneswig | I am spinning plates today, apologies for quietness | 11:04 |
| *** tosky has quit IRC | 11:05 | |
| *** tosky has joined #openstack-meeting | 11:10 | |
| oneswig | #endmeeting | 11:26 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 11:26 | |
| openstack | Meeting ended Wed Jul 15 11:26:02 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 11:26 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-07-15-11.01.html | 11:26 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-07-15-11.01.txt | 11:26 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-07-15-11.01.log.html | 11:26 |
| *** Lucas_Gray has quit IRC | 11:32 | |
| *** rcernin has joined #openstack-meeting | 11:32 | |
| *** yamamoto has joined #openstack-meeting | 11:40 | |
| *** Lucas_Gray has joined #openstack-meeting | 11:46 | |
| *** rcernin has quit IRC | 11:47 | |
| *** sluna has joined #openstack-meeting | 11:53 | |
| *** rfolco has joined #openstack-meeting | 11:56 | |
| *** Lucas_Gray has quit IRC | 11:58 | |
| *** Lucas_Gray has joined #openstack-meeting | 11:59 | |
| *** Lucas_Gray has quit IRC | 12:04 | |
| *** yamamoto has quit IRC | 12:06 | |
| *** baojg has quit IRC | 12:17 | |
| *** baojg has joined #openstack-meeting | 12:17 | |
| *** yamamoto has joined #openstack-meeting | 12:23 | |
| *** Lucas_Gray has joined #openstack-meeting | 12:30 | |
| *** moguimar has joined #openstack-meeting | 12:40 | |
| *** yamamoto has quit IRC | 12:42 | |
| *** oneswig has quit IRC | 12:53 | |
| *** sluna has quit IRC | 13:13 | |
| *** sluna has joined #openstack-meeting | 13:14 | |
| *** yamamoto has joined #openstack-meeting | 13:24 | |
| *** liuyulong has joined #openstack-meeting | 13:26 | |
| *** baojg has quit IRC | 13:33 | |
| *** Guest14648 has joined #openstack-meeting | 13:43 | |
| *** Guest14648 is now known as redrobot | 13:45 | |
| *** yamamoto has quit IRC | 13:47 | |
| *** dmacpher has joined #openstack-meeting | 13:52 | |
| *** ZhuXiaoYu has joined #openstack-meeting | 13:58 | |
| liuyulong | tes | 13:58 |
| liuyulong | t | 13:58 |
| *** Liang__ has joined #openstack-meeting | 13:59 | |
| *** Liang__ is now known as LiangFang | 13:59 | |
| liuyulong | #startmeeting neutron_l3 | 14:00 |
| openstack | Meeting started Wed Jul 15 14:00:36 2020 UTC and is due to finish in 60 minutes. The chair is liuyulong. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: neutron_l3)" | 14:00 | |
| openstack | The meeting name has been set to 'neutron_l3' | 14:00 |
| liuyulong | hi there | 14:01 |
| liuyulong | #topic Announcements | 14:02 |
| *** openstack changes topic to "Announcements (Meeting topic: neutron_l3)" | 14:02 | |
| liuyulong | OK, maybe here are only me. One person meeting. : ) | 14:03 |
| liuyulong | Our team meeting is going to be changed: I should change the meeting like this: | 14:03 |
| liuyulong | https://review.opendev.org/#/c/739780/ | 14:03 |
| liuyulong | I should change the L3 meeting maybe. | 14:04 |
| liuyulong | I will upload a change to the opendev/irc-meetings repo. | 14:04 |
| ZhuXiaoYu | :) | 14:04 |
| ralonsoh | hi, sorry I'm late | 14:05 |
| liuyulong | hi ZhuXiaoYu | 14:05 |
| liuyulong | Hi ralonsoh | 14:05 |
| liuyulong | OK, I'm not alone now. : ) | 14:05 |
| liuyulong | Let's go through the bug list. | 14:06 |
| liuyulong | #topic Bugs | 14:06 |
| *** openstack changes topic to "Bugs (Meeting topic: neutron_l3)" | 14:06 | |
| liuyulong | #link http://lists.openstack.org/pipermail/openstack-discuss/2020-July/015784.html | 14:06 |
| liuyulong | #link http://lists.openstack.org/pipermail/openstack-discuss/2020-July/015897.html | 14:06 |
| liuyulong | Two weeks list for this meeting. | 14:07 |
| liuyulong | Most of them has been discussed before. | 14:08 |
| liuyulong | First one | 14:08 |
| liuyulong | #link https://bugs.launchpad.net/neutron/+bug/1886116 | 14:08 |
| openstack | Launchpad bug 1886116 in neutron "slaac no longer works on IPv6 tenant subnets" [Low,Fix released] - Assigned to Brian Haley (brian-haley) | 14:08 |
| haleyb | no router == no slaac | 14:09 |
| liuyulong | It is not high level now. | 14:09 |
| haleyb | that one is actually fixed | 14:09 |
| liuyulong | haleyb, yes, it is well documented. : ) | 14:10 |
| haleyb | fixed in documenting you need a router to have router advertisements | 14:10 |
| liuyulong | OK, next | 14:11 |
| liuyulong | #link https://bugs.launchpad.net/neutron/+bug/1886216 | 14:11 |
| openstack | Launchpad bug 1886216 in neutron "keepalived-state-change does not format correctly the logs" [Low,Confirmed] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 14:11 |
| liuyulong | ralonsoh, so the bug is only available for stable/stein? | 14:12 |
| liuyulong | It should be stein and earlier versions | 14:14 |
| liuyulong | So the fix is a workaround. | 14:15 |
| ralonsoh | yes, only stein | 14:15 |
| ralonsoh | not newer versions | 14:15 |
| liuyulong | OK, the result looks good to me. | 14:17 |
| liuyulong | Just confirm the bad log format in stable/queens. | 14:18 |
| ralonsoh | I should backport this patch up to queens | 14:18 |
| liuyulong | OK, next. | 14:19 |
| liuyulong | #link https://bugs.launchpad.net/neutron/+bug/1887148 | 14:19 |
| openstack | Launchpad bug 1887148 in neutron "Network loop between physical networks with DVR" [High,In progress] - Assigned to Darragh O'Reilly (darragh-oreilly) | 14:19 |
| ralonsoh | when is this happening? | 14:19 |
| liuyulong | So, it is now doing the revert. | 14:20 |
| ralonsoh | when the OVS has no other flows, correct? | 14:20 |
| ralonsoh | it is | 14:20 |
| liuyulong | Yes, I guess, restart vswitchd and ovs-agent can reproduce the bug when you have multiple physical bridges. | 14:21 |
| ralonsoh | can we find a way to, as I commented in the bug, to filter the traffic from phys-brs? | 14:21 |
| ralonsoh | and block in if this traffic is going back to other phys-br? | 14:22 |
| ralonsoh | I know this is tricky because the traffic is leaving br-int using normal action | 14:22 |
| ralonsoh | but the alternative is reverting the previous patch | 14:23 |
| liuyulong | Yes, before normal action we should know where the packet should go. | 14:25 |
| liuyulong | But the ovs-agent is just Initializing, no ports info, neutron knows nothing. | 14:25 |
| ralonsoh | and normal action will broadcast this packet | 14:26 |
| ralonsoh | to all bridges except the one this packet is coming | 14:26 |
| liuyulong | Yep, basically it works as traditional L2 switch | 14:26 |
| liuyulong | Learn and forward, or flood | 14:26 |
| liuyulong | Since host(vswitchd) is just started, there is no MAC learned. | 14:27 |
| liuyulong | So after the revert, I will have a try to change the dvr related flow priority. | 14:29 |
| liuyulong | I'm thinking during the DVR implementation these flows were unexpectedly overridded by dvr functions. | 14:30 |
| liuyulong | I will test that. | 14:31 |
| liuyulong | OK, next | 14:31 |
| *** mlavalle has joined #openstack-meeting | 14:32 | |
| liuyulong | #link https://bugs.launchpad.net/neutron/+bug/1886426 | 14:32 |
| openstack | Launchpad bug 1886426 in neutron "Neutron sending response to rabbitmq exchange with event type "floatingip.update.end" without updating the status of Floating IP" [Undecided,Incomplete] | 14:32 |
| liuyulong | This should a known issue. | 14:33 |
| liuyulong | The floating IP status is changing to ACTIVE only when the L3 agent did set it on the interface. | 14:33 |
| liuyulong | But before that, the L3 plugin has done DB update (binding fixed_ip_port from None -> one_port_id), so the notification just send with old status value -- DOWN. | 14:35 |
| *** dklyle has joined #openstack-meeting | 14:36 | |
| haleyb | i was going to run through a floating ip create and associate, just to see what state it shows in the DB. it's not as if floating IP isn't working | 14:36 |
| liuyulong | related code: | 14:37 |
| liuyulong | #link https://github.com/openstack/neutron/blob/master/neutron/agent/l3/router_info.py#L403 | 14:37 |
| liuyulong | #link https://github.com/openstack/neutron/blob/master/neutron/db/l3_db.py#L1355 | 14:37 |
| liuyulong | haleyb, yes, the status should not be considered for the link status of a floating IP. | 14:38 |
| liuyulong | actually there is no such way now. | 14:39 |
| haleyb | yes, status active, but is state DOWN? guess the API doc should describe that too | 14:40 |
| haleyb | no, ACTIVE, DOWN and ERROR, either way have to verify | 14:41 |
| liuyulong | router admin state can be DOWN, but floating IP status is not changed during this UP->DOWN action (need to confirm). So it looks a bit reliable again. | 14:42 |
| liuyulong | Not reliable... | 14:42 |
| liuyulong | OK, no more bugs from the deputy list. | 14:44 |
| liuyulong | But I have another one. | 14:44 |
| liuyulong | #link https://bugs.launchpad.net/neutron/+bug/1883321 | 14:44 |
| openstack | Launchpad bug 1883321 in neutron "Neutron OpenvSwitch DVR - connection problem" [High,New] - Assigned to LIU Yulong (dragon889) | 14:44 |
| liuyulong | This should be a really long story... | 14:45 |
| liuyulong | This should be firstly realted to the egress flooding issue... | 14:47 |
| liuyulong | And then, they enabled the "explicitly_egress_direct". | 14:47 |
| liuyulong | So looks like the "NORMAL" flow, aka learn and forward, just does not be happy again. | 14:48 |
| liuyulong | #link https://review.opendev.org/#/c/738551/ | 14:49 |
| liuyulong | I'm trying to address the issue in this patch. | 14:49 |
| liuyulong | But the bug reporter said it can be seen on openflow security driver too. | 14:49 |
| liuyulong | I'm try to reproduce that with openflow security driver, but no issue encountered. | 14:51 |
| liuyulong | The issue should be something like this. | 14:51 |
| liuyulong | When the VM with floating IP is communiting with out side world, the floating IP gateway MAC (or floating IP physical route IP mac) is learned on the br-int fg-dev. | 14:52 |
| liuyulong | But when another VM without floating IP is trying to reach outside world, that MAC will be learned to the patch port (int-br-tun, if the tunnel is enabled, or physical-br-patch-port). | 14:54 |
| liuyulong | I will continue the work of this bug. | 14:55 |
| liuyulong | But one more thing, from my personal experience, maybe we should reconsider to use the NORMAL flow. I'm not a fan of using it since neutron knows everything. | 14:59 |
| liuyulong | OK, time is up. | 15:00 |
| liuyulong | Bye guys, thank you | 15:00 |
| liuyulong | #endmeeting | 15:00 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:00 | |
| openstack | Meeting ended Wed Jul 15 15:00:37 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_l3/2020/neutron_l3.2020-07-15-14.00.html | 15:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_l3/2020/neutron_l3.2020-07-15-14.00.txt | 15:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_l3/2020/neutron_l3.2020-07-15-14.00.log.html | 15:00 |
| ralonsoh | bye | 15:00 |
| *** liuyulong has quit IRC | 15:01 | |
| *** diurnalist has joined #openstack-meeting | 15:02 | |
| *** LiangFang has quit IRC | 15:10 | |
| *** jraju__ has quit IRC | 15:12 | |
| *** TrevorV has joined #openstack-meeting | 15:14 | |
| *** bbowen has quit IRC | 15:15 | |
| *** bbowen has joined #openstack-meeting | 15:17 | |
| *** yamamoto has joined #openstack-meeting | 15:46 | |
| *** armax has joined #openstack-meeting | 16:05 | |
| *** TrevorV has quit IRC | 16:23 | |
| *** yamamoto has quit IRC | 16:24 | |
| *** moguimar has quit IRC | 16:26 | |
| *** moguimar has joined #openstack-meeting | 16:26 | |
| *** baojg has joined #openstack-meeting | 16:28 | |
| *** TrevorV has joined #openstack-meeting | 16:28 | |
| *** yasufum has joined #openstack-meeting | 16:30 | |
| *** yasufum has quit IRC | 16:34 | |
| *** ociuhandu_ has joined #openstack-meeting | 16:36 | |
| *** ociuhandu has quit IRC | 16:39 | |
| *** yasufum has joined #openstack-meeting | 16:39 | |
| *** ociuhandu_ has quit IRC | 16:40 | |
| *** e0ne has quit IRC | 16:46 | |
| *** e0ne has joined #openstack-meeting | 16:47 | |
| *** baojg has quit IRC | 16:49 | |
| *** baojg has joined #openstack-meeting | 16:50 | |
| *** psachin has quit IRC | 16:58 | |
| *** bbowen has quit IRC | 17:00 | |
| *** ZhuXiaoYu has quit IRC | 17:03 | |
| *** psahoo has quit IRC | 17:19 | |
| *** gyee has joined #openstack-meeting | 17:30 | |
| *** jamesmcarthur has joined #openstack-meeting | 17:36 | |
| *** Lucas_Gray has quit IRC | 17:41 | |
| *** haleyb has quit IRC | 17:45 | |
| *** yasufum has quit IRC | 17:53 | |
| *** yasufum has joined #openstack-meeting | 17:59 | |
| *** e0ne has quit IRC | 18:00 | |
| *** jamesmcarthur has quit IRC | 18:02 | |
| *** jamesmcarthur has joined #openstack-meeting | 18:03 | |
| *** jamesmcarthur has quit IRC | 18:07 | |
| *** bbowen has joined #openstack-meeting | 18:16 | |
| *** jamesmcarthur has joined #openstack-meeting | 18:17 | |
| *** vishalmanchanda has quit IRC | 18:20 | |
| *** slaweq has joined #openstack-meeting | 18:21 | |
| *** yamamoto has joined #openstack-meeting | 18:21 | |
| *** tosky has quit IRC | 18:25 | |
| *** tosky has joined #openstack-meeting | 18:25 | |
| *** baojg has quit IRC | 18:38 | |
| *** baojg has joined #openstack-meeting | 18:39 | |
| *** ociuhandu has joined #openstack-meeting | 18:46 | |
| *** ociuhandu has quit IRC | 18:50 | |
| *** yamamoto has quit IRC | 19:01 | |
| *** rmcall has quit IRC | 19:22 | |
| *** rmcall has joined #openstack-meeting | 19:23 | |
| *** rmcallis has joined #openstack-meeting | 19:25 | |
| *** rmcall has quit IRC | 19:28 | |
| *** rmcallis__ has joined #openstack-meeting | 19:28 | |
| *** haleyb has joined #openstack-meeting | 19:29 | |
| *** rmcallis has quit IRC | 19:31 | |
| *** moguimar has quit IRC | 19:33 | |
| *** zaitcev has joined #openstack-meeting | 19:44 | |
| *** ralonsoh has quit IRC | 19:49 | |
| *** raildo has quit IRC | 20:16 | |
| *** raildo has joined #openstack-meeting | 20:16 | |
| *** raildo has quit IRC | 20:22 | |
| *** raildo has joined #openstack-meeting | 20:24 | |
| *** slaweq has quit IRC | 20:29 | |
| *** raildo has quit IRC | 20:30 | |
| *** jamesmcarthur_ has joined #openstack-meeting | 20:45 | |
| *** jamesmcarthur has quit IRC | 20:48 | |
| *** yamamoto has joined #openstack-meeting | 21:00 | |
| timburke | #startmeeting swift | 21:01 |
| openstack | Meeting started Wed Jul 15 21:01:10 2020 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:01 |
| *** openstack changes topic to " (Meeting topic: swift)" | 21:01 | |
| openstack | The meeting name has been set to 'swift' | 21:01 |
| timburke | who's here for the swift meeting? | 21:01 |
| seongsoocho | o/ | 21:01 |
| rledisez | o/ | 21:01 |
| kota_ | morning | 21:02 |
| mattoliverau | o/ | 21:02 |
| timburke | agenda's at https://wiki.openstack.org/wiki/Meetings/Swift -- it's fairly light, just updates on ongoing work | 21:03 |
| timburke | #topic libec upgrade issue | 21:03 |
| *** openstack changes topic to "libec upgrade issue (Meeting topic: swift)" | 21:03 | |
| timburke | a bit ago, alecuyer noticed an issue running EC with a mix of focal and bionic nodes (iirc) | 21:04 |
| timburke | i wrote up https://bugs.launchpad.net/swift/+bug/1886088 as i started playing with a mix of libec versions | 21:04 |
| openstack | Launchpad bug 1886088 in OpenStack Object Storage (swift) "Mixed versions of liberasurecode leads to quarantined fragments" [Undecided,In progress] | 21:04 |
| timburke | then realized that i'd actually seen a report of this before: https://bugs.launchpad.net/swift/+bug/1867937 | 21:05 |
| openstack | Launchpad bug 1867937 in OpenStack Object Storage (swift) "Different erasure codes metadata checksums with pypy3 and python2" [Undecided,New] | 21:05 |
| timburke | wrote up a couple patches to deal with it: p 738959, p 739164 | 21:06 |
| *** patchbot has joined #openstack-meeting | 21:06 | |
| timburke | *ahem* p 738959, p 739164 | 21:06 |
| patchbot | https://review.opendev.org/#/c/738959/ - liberasurecode - Be willing to write fragments with legacy crc - 1 patch set | 21:06 |
| patchbot | https://review.opendev.org/#/c/739164/ - swift - ec: Add an option to write fragments with legacy crc - 1 patch set | 21:06 |
| timburke | the idea is, libec looks for an env var; if set, write the legacy CRC | 21:07 |
| timburke | then, since managing env vars isn't how we've typically tooled up our deployment tools, add a config option to swift that sets or clears the env var | 21:07 |
| timburke | does that seem like a reasonable way to go? so far in playing with it in dev, it's working out for me pretty well | 21:08 |
| timburke | i tried to put a decent write-up of what would be needed to do a smooth upgrade in the commit messages | 21:09 |
| kota_ | sounds reasonable. I didn't find better way. | 21:09 |
| rledisez | I guess the choice of env variable was to avoid changing the API, right? | 21:09 |
| *** gyee has quit IRC | 21:10 | |
| timburke | rledisez, yeah, that was my thinking. didn't really want to go breaking a bunch of signatures if i could help it (plus it would've required an additional patch for pyeclib) | 21:11 |
| *** gyee has joined #openstack-meeting | 21:11 | |
| zaitcev | Normally I hate it when people penetrate layers like that and just rummage down like it's monkey patching or something, but oh well. | 21:12 |
| timburke | i could be persuaded to do it anyway ;-) that's why i asked | 21:13 |
| rledisez | Wondering if we could "auto" detect that the data are in legacy mode and automatically switch to it? I mean, as an operator, how am I suppose to know if I should enable it or not… | 21:13 |
| timburke | i tried to put a decent write-up of how to tell whether you're affected in https://bugs.launchpad.net/swift/+bug/1886088/comments/2 | 21:15 |
| openstack | Launchpad bug 1886088 in OpenStack Object Storage (swift) "Mixed versions of liberasurecode leads to quarantined fragments" [Undecided,In progress] | 21:15 |
| kota_ | ah... exactly, to enable the env var, anyway we need to bump libec version. | 21:15 |
| rledisez | great! I missed that | 21:16 |
| timburke | i'm not so sure that we can do any auto-enabling, though, since the idea is to have this as a temporary measure during upgrade, then turn it back off after everything's upgraded | 21:16 |
| kota_ | newer libec is compatible with legacy. | 21:16 |
| timburke | yup -- i was at least *that* careful originally ;-) | 21:16 |
| *** TrevorV has quit IRC | 21:18 | |
| kota_ | oic. however, the env basically prevents to *write* a new fragment with zlib to be compatible with older legacy backend. | 21:18 |
| kota_ | it sounds hard to make it as auto-detect. | 21:19 |
| timburke | so, do people generally agree with zaitcev, that it'd be better to have an explicit option that we plumb through pyeclib as well? or is the env var "good enough"? | 21:20 |
| rledisez | I'm voting "good enough", maybe to remove in a future version (eg 1.7), and if somebody wants to upgrade, we say that 1.6 is a "checkpoint release" :) | 21:21 |
| timburke | yeah, i was about to ask about that part -- how long should we have this option available? at some point i feel like we'd be justified in removing support for it | 21:22 |
| timburke | (maybe that ends up informing the env var vs option debate) | 21:22 |
| kota_ | either is fine to me but if we choose option, we have to touch it carefully both libec and pyeclib that are maintained in separate repo :/ | 21:24 |
| kota_ | env may be just ignored if older libec was used. | 21:25 |
| timburke | all right, seems like we're fairly split at the moment -- let's think about it some more and we can revisit next week. will anyone have a chance to take a look at either or both patches? | 21:27 |
| *** rmcall has joined #openstack-meeting | 21:27 | |
| zaitcev | Not a consideration for me, fortunately, I'm going to add "Requires:" into packaging, so "dnf update" does the right thing. | 21:27 |
| *** rmcallis__ has quit IRC | 21:28 | |
| timburke | fortunately they're both pretty small, less than 100 lines between them | 21:29 |
| *** rmcall has quit IRC | 21:30 | |
| timburke | all right, moving on | 21:31 |
| *** rmcall has joined #openstack-meeting | 21:31 | |
| timburke | #topic waterfall EC | 21:31 |
| *** openstack changes topic to "waterfall EC (Meeting topic: swift)" | 21:31 | |
| timburke | so clay isn't around today, but i started trying it out a bit yesterday -- it's great! i'm loving it | 21:32 |
| zaitcev | Nice | 21:32 |
| timburke | i wrote up a bit of my findings on the main patch | 21:32 |
| timburke | #link https://review.opendev.org/#/c/711342/ | 21:32 |
| patchbot | patch 711342 - swift - Add concurrent_gets to EC GET requests - 12 patch sets | 21:32 |
| timburke | basically, i drove a known load via swift-bench against my saio -- then introduced some random delays and watched how the various timeouts available led to different behaviors | 21:34 |
| timburke | without clay's work, i could only really play with recoverable_node_timeout, which would help but not that much. and the lower i turned it down, the more failures i'd see as a client | 21:35 |
| *** rmcall has quit IRC | 21:35 | |
| *** yamamoto has quit IRC | 21:35 | |
| *** rmcallis has joined #openstack-meeting | 21:36 | |
| *** yamamoto has joined #openstack-meeting | 21:36 | |
| timburke | with the new behavior, throughput's up and failures are down. i'd still like to see what it does in a real cluster with real disks, but i like it so far | 21:36 |
| timburke | anyone else who's interested in EC and performance ought to take a look :-) | 21:36 |
| timburke | that's all i've got | 21:38 |
| timburke | #topic open discussion | 21:38 |
| *** openstack changes topic to "open discussion (Meeting topic: swift)" | 21:38 | |
| timburke | anyone else have topics we ought to bring up? | 21:38 |
| kota_ | definitely I'm interested but sorry I won't have enough time to play with it | 21:38 |
| kota_ | > waterfallEC | 21:38 |
| timburke | all right, let's let mattoliverau enjoy his birthday :-) | 21:40 |
| timburke | thank you all for coming, and thank you for working on swift! | 21:40 |
| timburke | #endmeeting | 21:40 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 21:40 | |
| openstack | Meeting ended Wed Jul 15 21:40:38 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 21:40 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/swift/2020/swift.2020-07-15-21.01.html | 21:40 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/swift/2020/swift.2020-07-15-21.01.txt | 21:40 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/swift/2020/swift.2020-07-15-21.01.log.html | 21:40 |
| *** patchbot has left #openstack-meeting | 21:40 | |
| *** zaitcev has left #openstack-meeting | 21:40 | |
| *** rmcallis has quit IRC | 21:42 | |
| *** rcernin has joined #openstack-meeting | 22:20 | |
| *** rcernin has quit IRC | 22:20 | |
| *** rcernin has joined #openstack-meeting | 22:21 | |
| *** manpreet has quit IRC | 22:33 | |
| *** armax has quit IRC | 22:34 | |
| *** armax has joined #openstack-meeting | 22:36 | |
| *** baojg has quit IRC | 22:43 | |
| *** makoto-h has quit IRC | 22:44 | |
| *** baojg has joined #openstack-meeting | 22:44 | |
| *** tosky has quit IRC | 22:48 | |
| *** yamamoto has quit IRC | 22:59 | |
| *** mlavalle has quit IRC | 22:59 | |
| *** jamesmcarthur has joined #openstack-meeting | 23:00 | |
| *** jamesmcarthur_ has quit IRC | 23:04 | |
| *** markvoelker has joined #openstack-meeting | 23:11 | |
| *** markvoelker has quit IRC | 23:15 | |
| *** yamamoto has joined #openstack-meeting | 23:30 | |
| *** yamamoto has quit IRC | 23:32 | |
| *** yamamoto has joined #openstack-meeting | 23:32 | |
| *** baojg has quit IRC | 23:58 | |
| *** jamesmcarthur has quit IRC | 23:58 | |
| *** baojg has joined #openstack-meeting | 23:59 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!