| *** sapd1 has quit IRC | 00:07 | |
| *** yangjianfeng has quit IRC | 00:41 | |
| *** armax has quit IRC | 01:08 | |
| *** sapd1 has joined #openstack-lbaas | 02:06 | |
| *** ramishra has joined #openstack-lbaas | 02:30 | |
| *** rcernin has quit IRC | 02:58 | |
| *** rcernin has joined #openstack-lbaas | 03:18 | |
| *** terdei has quit IRC | 03:58 | |
| *** terdei has joined #openstack-lbaas | 04:07 | |
| *** rcernin has quit IRC | 04:12 | |
| *** rcernin has joined #openstack-lbaas | 04:13 | |
| *** vishalmanchanda has joined #openstack-lbaas | 04:35 | |
| *** sapd1 has quit IRC | 04:55 | |
| *** sapd1 has joined #openstack-lbaas | 05:37 | |
| *** rcernin has quit IRC | 05:48 | |
| *** sapd1 has quit IRC | 05:54 | |
| *** rcernin has joined #openstack-lbaas | 06:03 | |
| *** sapd1 has joined #openstack-lbaas | 06:15 | |
| *** gcheresh has joined #openstack-lbaas | 06:39 | |
| *** ataraday_ has joined #openstack-lbaas | 06:48 | |
| *** gcheresh has quit IRC | 07:10 | |
| *** trident has quit IRC | 07:18 | |
| *** sapd1 has quit IRC | 07:24 | |
| openstackgerrit | Ann Taraday proposed openstack/octavia master: Update amphora v2 for the failover refactor https://review.opendev.org/738292 | 07:27 |
|---|---|---|
| openstackgerrit | Ann Taraday proposed openstack/octavia master: Add experimental amphorav2 jobs https://review.opendev.org/737993 | 07:28 |
| *** gcheresh has joined #openstack-lbaas | 07:44 | |
| *** zigo has joined #openstack-lbaas | 07:45 | |
| *** rcernin has quit IRC | 07:51 | |
| *** ramishra has quit IRC | 08:00 | |
| *** TMM has quit IRC | 08:00 | |
| *** irclogbot_1 has quit IRC | 08:02 | |
| *** ramishra has joined #openstack-lbaas | 08:05 | |
| *** TMM has joined #openstack-lbaas | 08:05 | |
| *** irclogbot_3 has joined #openstack-lbaas | 08:07 | |
| *** gcheresh has quit IRC | 08:41 | |
| *** gcheresh has joined #openstack-lbaas | 08:53 | |
| *** sapd1 has joined #openstack-lbaas | 08:59 | |
| *** trident has joined #openstack-lbaas | 09:08 | |
| *** sapd1 has quit IRC | 09:11 | |
| -openstackstatus- NOTICE: due to a new release of setuptools (50.0.0), a lot of jobs are currently broken, please do not recheck blindly. see http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016905.html | 09:16 | |
| *** ramishra has quit IRC | 10:09 | |
| *** ramishra has joined #openstack-lbaas | 10:51 | |
| *** servagem has joined #openstack-lbaas | 11:56 | |
| *** redrobot has joined #openstack-lbaas | 12:34 | |
| *** maciejjozefczyk has joined #openstack-lbaas | 12:55 | |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia-tempest-plugin master: WIP SCTP traffic scenario tests https://review.opendev.org/738643 | 12:57 |
| *** maciejjozefczyk has quit IRC | 13:05 | |
| *** TrevorV has joined #openstack-lbaas | 13:28 | |
| *** lemko has quit IRC | 13:33 | |
| *** lemko has joined #openstack-lbaas | 13:38 | |
| *** kevinz has quit IRC | 13:59 | |
| *** sapd1 has joined #openstack-lbaas | 14:24 | |
| *** openstackgerrit has quit IRC | 14:37 | |
| *** ataraday has quit IRC | 14:37 | |
| *** spatel has joined #openstack-lbaas | 14:53 | |
| *** servagem has quit IRC | 15:00 | |
| *** armax has joined #openstack-lbaas | 15:10 | |
| *** spatel has quit IRC | 15:29 | |
| johnsom | Hmm gerrit bot seems to be not working | 15:56 |
| johnsom | Cores, I would appreciate a second review on this API sort key fix: https://review.opendev.org/#/c/745239/ | 16:04 |
| johnsom | I'm going to need to backport it so that the new tempest API tests will pass on the stable branches. | 16:04 |
| xgerman | done. | 16:05 |
| johnsom | Thank you sir. It's the holdup here: https://review.opendev.org/#/c/744805 | 16:05 |
| *** sapd1 has quit IRC | 16:12 | |
| *** servagem has joined #openstack-lbaas | 16:19 | |
| JustAFlerkin | does amphora-agent try to rename eth interfaces when one is hotplugged on lb reconfigure? | 16:41 |
| JustAFlerkin | been trying to track the lost eth interfaces in the ubuntu 20.04 amphora. | 16:42 |
| johnsom | JustAFlerkin Yes, the amphora agent manages all of the interface names. They aren't really used by our code, we manage interfaces by the MAC address assigned from neutron | 16:42 |
| johnsom | I.e. we don't care what the OS calls them | 16:43 |
| JustAFlerkin | ok, that might be the source of the clash then. 5 seconds after an interface is found by kernel, ubuntu renames from ethX to ensX, something then changes the interface back to ethX and at that point ip, ifconfig dhclient etc can't open it and the config fils. | 16:43 |
| JustAFlerkin | fails. | 16:43 |
| JustAFlerkin | what's the thought process for amphora-agent changing the interface name? | 16:44 |
| johnsom | Well, they are inconsistent across the OS versions and we need to make sure there are not name conflicts when we move the interfaces into the network namespace. | 16:45 |
| JustAFlerkin | I suspect there's a race condition where the old interface is held open by something when amphora-agent makes the rename and that's causing the lost interface due to naming conflict. | 16:45 |
| JustAFlerkin | Would a replacement solution where amphora-agent finds the name instead of trying to force the name to it's ideal be acceptable? | 16:46 |
| johnsom | It would be interesting to see logs. I couldn't get 20.04 to even bring up the lb-mgmt-net interface which is managed by cloud-init and never touched but the amphora agent, so I didn't get far enough to see what was happening with the hotplug | 16:47 |
| JustAFlerkin | i'm also digging around at why ubuntu 20.04 ignores the interface don't-rename flags that usually work. | 16:47 |
| JustAFlerkin | if I can get one of the two to stop renaming ethX then I think the bug will be resolved. | 16:48 |
| johnsom | Really, we can't rely on names from the OS, they change too often IMO. That is why we always look them up by MAC. | 16:48 |
| JustAFlerkin | with the aug 12 20.04 build I can reliably get the instance eth interfaces to come up on hotplug. | 16:48 |
| JustAFlerkin | prior to that it was iffy like you said on hotplug interfaces coming up in an instance. | 16:49 |
| JustAFlerkin | which track would you prefer that I chase for a solution? stopping ubuntu renaming, or amphoa-agent renaming? | 16:51 |
| JustAFlerkin | the only logging i've connected so far is https://pastebin.com/PsTZVyQR kernel messages about renames. | 16:52 |
| JustAFlerkin | I managed to get brief glimpse of the ens8 and ens9 (eth1 and eth2) interfaces via ip link and ifconfig before the second renaming back to ethX. but I lost the screencap. | 16:53 |
| johnsom | So, ifconfig isn't going to find those because they have been moved inside the network namespace. You have to run "sudo ip netns exec amphora-haproxy ip a" to see them | 16:55 |
| JustAFlerkin | had a script on watch loop that blidly tried to set the interface IP then ping a known good ip. it worked for the 5 second interval between renames. | 16:55 |
| JustAFlerkin | ah. | 16:55 |
| JustAFlerkin | that explains that. | 16:55 |
| JustAFlerkin | thx. | 16:55 |
| JustAFlerkin | I didn't think to look for an netns | 16:55 |
| johnsom | Yeah, all of the interfaces that touch tenant traffic are isolated in a network namespace for security | 16:56 |
| JustAFlerkin | good idea. | 16:56 |
| *** jamesden_ has quit IRC | 17:47 | |
| *** jamesdenton has joined #openstack-lbaas | 17:49 | |
| *** zzzeek has quit IRC | 18:34 | |
| *** zzzeek has joined #openstack-lbaas | 18:36 | |
| *** zzzeek has quit IRC | 18:54 | |
| *** zzzeek has joined #openstack-lbaas | 18:56 | |
| *** njohnston has joined #openstack-lbaas | 19:29 | |
| *** TrevorV has quit IRC | 19:41 | |
| *** gcheresh has quit IRC | 21:00 | |
| *** ataraday_ has quit IRC | 21:10 | |
| *** vishalmanchanda has quit IRC | 21:15 | |
| *** terdei has quit IRC | 21:41 | |
| *** openstackgerrit has joined #openstack-lbaas | 22:36 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia-dashboard master: Add proxy protocol v2 support to dashboard https://review.opendev.org/747802 | 22:36 |
| openstackgerrit | Michael Johnson proposed openstack/octavia-dashboard master: Add proxy protocol v2 support to dashboard https://review.opendev.org/747802 | 22:37 |
| johnsom | Ugh, helps if I add the file | 22:37 |
| *** rcernin has joined #openstack-lbaas | 22:51 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!