opendevreview | Miguel Lavalle proposed openstack/neutron master: [PoC][DNM] Router flavors and service type for OVN https://review.opendev.org/c/openstack/neutron/+/883988 | 00:43 |
---|---|---|
opendevreview | Miguel Lavalle proposed openstack/neutron master: [PoC][DNM] Router flavors and service type for OVN https://review.opendev.org/c/openstack/neutron/+/883988 | 00:51 |
*** ramishra_ is now known as ramishra | 04:37 | |
opendevreview | Merged openstack/neutron stable/2023.2: Add dhcpagentscheduler API extension to the ML2/OVN extensions https://review.opendev.org/c/openstack/neutron/+/898292 | 05:04 |
opendevreview | Luis Tomas Bolivar proposed openstack/neutron master: Add support for FDB aging https://review.opendev.org/c/openstack/neutron/+/893333 | 06:51 |
opendevreview | Luis Tomas Bolivar proposed openstack/ovn-bgp-agent master: Add initial support for local OVN cluster instead of kernel-networking https://review.opendev.org/c/openstack/ovn-bgp-agent/+/881779 | 07:31 |
opendevreview | Luis Tomas Bolivar proposed openstack/ovn-bgp-agent master: Add initial support for local OVN cluster instead of kernel-networking https://review.opendev.org/c/openstack/ovn-bgp-agent/+/881779 | 08:01 |
ralonsoh | hi folks, please check https://review.opendev.org/c/openstack/neutron/+/897959. It could help on the fullstack tests, killing the pending processes when needed, avoiding the 90 seconds period tying to kill them | 08:14 |
opendevreview | Rodolfo Alonso proposed openstack/neutron master: Add policy enforcer for "tags" service plugin https://review.opendev.org/c/openstack/neutron/+/896509 | 08:16 |
ralonsoh | lajoskatona, ^ thanks for the reviews. I realize that SG rules have not tags, but Trunk and Subnetpools have | 08:17 |
ralonsoh | I realized* | 08:17 |
opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Change device_owner used for OVN LB HM port https://review.opendev.org/c/openstack/ovn-octavia-provider/+/897348 | 08:36 |
opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Add maintenance task to update entities on component start https://review.opendev.org/c/openstack/ovn-octavia-provider/+/898535 | 08:56 |
opendevreview | Merged openstack/neutron master: Add constant to identify OVN LB HM ports https://review.opendev.org/c/openstack/neutron/+/897345 | 09:59 |
opendevreview | Rodolfo Alonso proposed openstack/neutron master: Remove any IPAM allocation if port bulk creation fails https://review.opendev.org/c/openstack/neutron/+/898549 | 11:13 |
ralonsoh | slaweq, ^ if you have 2 mins, an easy one | 11:15 |
opendevreview | Rodolfo Alonso proposed openstack/neutron master: [OVN] Use the ``AgentCache`` deletion method https://review.opendev.org/c/openstack/neutron/+/898551 | 11:29 |
opendevreview | Rodolfo Alonso proposed openstack/neutron master: [OVN] Remove backwards compatibility with OVN < v20.09 https://review.opendev.org/c/openstack/neutron/+/884898 | 11:39 |
ralonsoh | folks, no rush at all. If you have 5 mins: https://review.opendev.org/c/openstack/neutron-lib/+/895940 | 11:40 |
ralonsoh | thanks! | 11:40 |
lajoskatona | ralonsoh: :-) | 11:43 |
spatel | ralonsoh hey! | 13:09 |
spatel | Did you check what is going on with my bug - https://bugs.launchpad.net/neutron/+bug/2039464 | 13:12 |
ralonsoh | spatel, no, I didn't have time yet. In any case, today's the bug triage | 13:13 |
ralonsoh | you need to confirm that you have or not the patch slaweq mentioned | 13:13 |
ralonsoh | this is a must | 13:13 |
spatel | I am checking code but how that is possible 2019 code just disappeared :) | 13:14 |
spatel | I am running very latest code (zed) | 13:14 |
ralonsoh | but maybe you don't have a version with this patch backported | 13:14 |
spatel | give me few min to confirm snippet | 13:14 |
opendevreview | Rodolfo Alonso proposed openstack/neutron master: [OVN] Populate the "router.distributed" flag in ML2/OVN https://review.opendev.org/c/openstack/neutron/+/886992 | 13:26 |
opendevreview | Rodolfo Alonso proposed openstack/neutron master: [OVN] Set the Neutron port status based on "lsp.up" and "lsp.enabled" https://review.opendev.org/c/openstack/neutron/+/896939 | 13:26 |
spatel | ralonsoh just verified and my code is already patched | 13:27 |
spatel | is shared: True correct or It should be False.. ? | 13:40 |
ioni | hi guys, in zed neutron-linuxbridge-agent is marked as experimetal. is there a procedure to migrate away from linuxbridge and use openvswitch? | 13:57 |
ioni | migrating in place | 13:57 |
haleyb | ping ykarel, mlavalle, mtomaska, slawek, obondarev, tobias-urdin, lajoskatona, amotoki | 14:00 |
mtomaska | o/ | 14:00 |
haleyb | #startmeeting networking | 14:00 |
opendevmeet | Meeting started Tue Oct 17 14:00:29 2023 UTC and is due to finish in 60 minutes. The chair is haleyb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
opendevmeet | The meeting name has been set to 'networking' | 14:00 |
obondarev | hi | 14:00 |
mlavalle | o/ | 14:00 |
ykarel | o/ | 14:00 |
frickler | \o | 14:00 |
isabek | Hi | 14:00 |
bcafarel | o/ | 14:01 |
haleyb | #topic announcements | 14:01 |
slaweq | o/ | 14:01 |
haleyb | vPTG is next week | 14:02 |
haleyb | #link https://etherpad.opendev.org/p/oct2023-ptg-neutron | 14:02 |
ralonsoh | hi | 14:02 |
haleyb | please add any topics, i will go through today and add some | 14:02 |
slaweq | speaking about PTG - do we have any slots booked already? | 14:03 |
haleyb | #link #link https://openinfra-ptg.eventbrite.com/ is the signup page, it's free | 14:03 |
slaweq | I didn't saw any in the ptg.opendev.org page today morning | 14:03 |
haleyb | slaweq: no, i will do that today, i have been out on personal leave most of the past week | 14:04 |
lajoskatona | o/ | 14:04 |
slaweq | haleyb++ thx a lot | 14:04 |
haleyb | slaweq: and thanks for the info you sent me on keystone | 14:05 |
haleyb | any other announcements? | 14:06 |
haleyb | #topic bugs | 14:07 |
slaweq | on Monday during the PTG there will be TC and community leaders session | 14:07 |
slaweq | 1600 to 1800 UTC | 14:08 |
slaweq | just a heads up if anyone would be interested | 14:08 |
rubasov | late o/ | 14:08 |
lajoskatona | thanks, sounds interesting | 14:08 |
haleyb | slaweq: ack, i will cross-reference that when picking time slots | 14:09 |
haleyb | ok, onto bugs then... | 14:11 |
haleyb | last week's deputy was amotoki | 14:11 |
haleyb | i am deputy next week | 14:11 |
haleyb | #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/4UAXT2CEYGHJ5H6VSI23JHBAVBVPVZXW/ | 14:11 |
haleyb | first unassigned on the list is | 14:12 |
haleyb | #link https://bugs.launchpad.net/neutron/+bug/2038978 | 14:12 |
haleyb | mnaser did propose a patch | 14:12 |
haleyb | #link https://review.opendev.org/c/openstack/neutron/+/897953 | 14:12 |
haleyb | mnaser: can i assign that to you? | 14:13 |
mnaser | Sure. | 14:13 |
opendevreview | yatin proposed openstack/neutron stable/victoria: Call the "tc qdisc" command for ingress qdisc without parent https://review.opendev.org/c/openstack/neutron/+/898567 | 14:14 |
haleyb | mnaser: thanks | 14:14 |
haleyb | next unassigned is | 14:15 |
haleyb | #link https://bugs.launchpad.net/neutron/+bug/2039464 | 14:15 |
ralonsoh | I've finished installing a zed env | 14:15 |
haleyb | i think this is something spatel was just discussing here | 14:15 |
ralonsoh | I'll try this today | 14:15 |
ralonsoh | I can't reproduce it in master or 2023.2 | 14:16 |
haleyb | ralonsoh: ack, do you want to assign it to yourself and set priority? | 14:17 |
ralonsoh | right now | 14:17 |
spatel | haleyb let me know if you need more info from me to fix this bug | 14:17 |
haleyb | ralonsoh: thanks for taking it - spatel please just help rodolfo if he has any further questions | 14:19 |
spatel | I will :) | 14:19 |
haleyb | next unassigned is | 14:20 |
haleyb | #link https://bugs.launchpad.net/neutron/+bug/2038931 | 14:20 |
haleyb | ovsfw: OVS br-int rule disappears from the table=60 after stop/start VM | 14:20 |
haleyb | submitter reproduced on devstack with master branch | 14:21 |
haleyb | liu yulong added a quick bit of info | 14:22 |
haleyb | i have marked as high and will ping liu to see if he has time to propose change | 14:24 |
haleyb | ok, next is an ongoing intermittne gate issue | 14:25 |
haleyb | #link https://bugs.launchpad.net/neutron/+bug/2039038 | 14:25 |
haleyb | [fullstack] The "dhclient" kill command can take more than 90 seconds | 14:25 |
haleyb | #link https://review.opendev.org/c/openstack/neutron/+/897959 | 14:25 |
haleyb | ralonsoh: thanks for debugging this, will look at change after meeting | 14:26 |
ralonsoh | I think the patch is ready for reviews | 14:26 |
haleyb | hopefully that helps make the fullstack job more stable, i've seen that failure off and on | 14:26 |
ralonsoh | please check the logs, the issue is not happening again | 14:27 |
ralonsoh | and the CI time is lower | 14:27 |
haleyb | :) | 14:27 |
haleyb | ralonsoh: and i see a few fwaas bugs, which mostly have changes merged, thanks! | 14:29 |
haleyb | is there any that still need attention? | 14:29 |
ralonsoh | except for the fullstack one | 14:29 |
haleyb | #link https://bugs.launchpad.net/neutron/+bug/2039285 | 14:29 |
ralonsoh | the fullstack job is failing always | 14:29 |
lajoskatona | +1, thanks for fwaas attention | 14:29 |
ralonsoh | (and I didn't have time to check it) | 14:30 |
lajoskatona | I am not sure if it ever worked | 14:30 |
ralonsoh | maybe the people who resurrected this project will spend some time on this | 14:30 |
haleyb | it's in the experimental queue too | 14:30 |
lajoskatona | to tell the truth I miss the historical records :-) | 14:30 |
haleyb | is there anyone working on fwaas here now? | 14:31 |
ralonsoh | inspur folks | 14:31 |
haleyb | thanks, i'll try and find someone to take ownership of the last one | 14:33 |
haleyb | any other bugs people want to talk about? | 14:33 |
haleyb | ok, moving on | 14:34 |
haleyb | #topic specs | 14:34 |
haleyb | we did not have a drivers meeting last week, and i have not seen any new RFE bugs | 14:35 |
haleyb | #link https://review.opendev.org/c/openstack/neutron-specs/+/891204 is the only ongoing one if anyone has review cycles | 14:35 |
haleyb | OVN Interconnect RFE | 14:35 |
mlavalle | I'll review it today and / or tomorrow | 14:36 |
haleyb | mlavalle: thanks! | 14:36 |
haleyb | #topic community_goals | 14:37 |
haleyb | #link https://review.opendev.org/c/openstack/neutron/+/896509 | 14:38 |
haleyb | RBAC for tags plugin | 14:38 |
haleyb | ralonsoh: is that ready for review? | 14:38 |
ralonsoh | thanks to lajoskatona for the last review, now I think the patch is ready | 14:38 |
mlavalle | added to my pile | 14:39 |
lajoskatona | +1 | 14:39 |
haleyb | great, thanks | 14:39 |
haleyb | the other open goal is neutronclient deprecation | 14:40 |
haleyb | #link https://etherpad.opendev.org/p/python-neutronclient_deprecation | 14:40 |
lajoskatona | I added this topic to the PTG page, to have a slot for it | 14:40 |
lajoskatona | that't it from me (again spent my time mostly with downstream things ....) | 14:41 |
haleyb | lajoskatona: ack, it will be good for me to refresh myself with where things are att | 14:41 |
lajoskatona | :-) | 14:41 |
haleyb | #topic on_demand | 14:42 |
slaweq | sorry for being late but regarding S-RBAC I will also add topic to the PTG etherpad | 14:42 |
haleyb | +1 | 14:42 |
slaweq | I just need to check everything what we already have and where we are exactly | 14:42 |
slaweq | I will do it this week for sure to be ready for PTG | 14:42 |
haleyb | and with the vPTG next week I will cancel this meeting and drivers next week, we can talk about any topics then | 14:43 |
haleyb | todays CI meeting is IRC i believe and in :17 | 14:43 |
ykarel | yes right it's IRC today | 14:44 |
mlavalle | ack | 14:44 |
opendevreview | Alexey Stupnikov proposed openstack/neutron master: Don't set port capabilities for OVS HW offloading https://review.opendev.org/c/openstack/neutron/+/898556 | 14:44 |
lajoskatona | ack | 14:44 |
haleyb | if nothing else i will close this meeting | 14:44 |
haleyb | #endmeeting | 14:45 |
opendevmeet | Meeting ended Tue Oct 17 14:45:08 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:45 |
opendevmeet | Minutes: https://meetings.opendev.org/meetings/networking/2023/networking.2023-10-17-14.00.html | 14:45 |
opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/networking/2023/networking.2023-10-17-14.00.txt | 14:45 |
opendevmeet | Log: https://meetings.opendev.org/meetings/networking/2023/networking.2023-10-17-14.00.log.html | 14:45 |
mlavalle | o/ | 14:45 |
obondarev | o/ | 14:45 |
haleyb | thanks everyone | 14:45 |
ykarel | o/ | 14:45 |
ralonsoh | bye | 14:45 |
lajoskatona | o/ | 14:45 |
mtomaska | o/ | 14:45 |
rubasov | o/ | 14:46 |
opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Add maintenance task to update entities on component start https://review.opendev.org/c/openstack/ovn-octavia-provider/+/898535 | 14:49 |
ykarel | #startmeeting neutron_ci | 15:00 |
opendevmeet | Meeting started Tue Oct 17 15:00:48 2023 UTC and is due to finish in 60 minutes. The chair is ykarel. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
opendevmeet | The meeting name has been set to 'neutron_ci' | 15:00 |
ykarel | ping bcafarel, lajoskatona, mlavalle, mtomaska, ralonsoh, ykarel, jlibosva, elvira | 15:00 |
ykarel | Grafana dashboard: https://grafana.opendev.org/d/f913631585/neutron-failure-rate?orgId=1 | 15:00 |
ralonsoh | hello | 15:00 |
slaweq | o/ | 15:00 |
haleyb | o/ | 15:01 |
mlavalle | o/ | 15:02 |
lajoskatona | o/ | 15:02 |
ykarel | Let's start, bernard notified he will not be joining today | 15:02 |
ykarel | #topic Actions from previous meetings | 15:02 |
ykarel | ralonsoh to backport https://review.opendev.org/c/openstack/neutron/+/878549 | 15:03 |
ykarel | Done https://review.opendev.org/q/I07fa45401788da6b963830e72a7b3a3cd54662e1 | 15:03 |
ykarel | it's merged | 15:03 |
ykarel | ralonsoh to check tempest failures related to metadata in openvswitch job | 15:03 |
ralonsoh | I'm still investigating this issue | 15:03 |
ralonsoh | no results so far | 15:04 |
ykarel | Ok thanks, you opened a bug already for it? | 15:04 |
ralonsoh | I don't remember now, I'll check it after this meeting | 15:04 |
ykarel | sure, thanks | 15:04 |
ykarel | ralonsoh to check failures with nftables job in stable periodics | 15:04 |
ykarel | https://bugs.launchpad.net/neutron/+bug/2039027 | 15:04 |
ralonsoh | yeah, this is more complicated | 15:04 |
ralonsoh | I realized that we were not testing that since 2 releases ago | 15:05 |
ralonsoh | we have merged this patch, fixing this issue | 15:05 |
ralonsoh | now the problem is to know why the linux bridge job is failing when using nftables | 15:05 |
ykarel | ralonsoh, it should be the same issue we seen few weeks back and worked around using ebtables legacy | 15:06 |
ralonsoh | if I'm not wrong, this is applied in these jobs | 15:06 |
ralonsoh | but I'll check it of course | 15:06 |
ykarel | i had checked some logs and show same ebtables errors | 15:07 |
lajoskatona | to tell the truth we sould have a session for this ebtables thing next week | 15:07 |
lajoskatona | I remember we discussed it but I can't tell where we are with it, and if we ahve to do any extra testing or migration | 15:07 |
ykarel | ralonsoh, yes that is applied but these jobs specifically switch to ebtables-nft as part of nftables setup | 15:07 |
ralonsoh | but I don't think this is related, let me find the newer job executions | 15:07 |
ralonsoh | https://zuul.opendev.org/t/openstack/builds?job_name=neutron-linuxbridge-tempest-plugin-nftables&skip=0 | 15:08 |
ykarel | https://github.com/openstack/neutron/blob/169deb24500caa43ce3f6c8cd837d2286d219d5f/roles/nftables/tasks/main.yaml#L16 | 15:08 |
ralonsoh | right | 15:08 |
ralonsoh | https://d233c9d65731f58c26a3-15368fe3522896553ebd232c1020ea9b.ssl.cf1.rackcdn.com/periodic/opendev.org/openstack/neutron/master/neutron-linuxbridge-tempest-plugin-nftables/6da8e44/testr_results.html | 15:08 |
ralonsoh | this is the last execution | 15:08 |
ykarel | and in one of those failures i saw:- DEBUG oslo.privsep.daemon [-] privsep: reply[731663a7-3c8c-4a4a-b8f6-ac6318cdb904]: (4, ('', 'ebtables v1.8.7 (nf_tables): RULE_DELETE failed (Invalid argument): rule in chain neutronMAC-tap17686dc1-fe\n', 4)) {{(pid=58939) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}} | 15:09 |
ralonsoh | I'll check after this meeting if there is any evident error | 15:09 |
ralonsoh | ^^^ like this oner | 15:09 |
ykarel | k Thanks | 15:09 |
ykarel | ykarel to open bug for functional failures TestMaintenance | 15:10 |
ykarel | i reported https://bugs.launchpad.net/neutron/+bug/2039417 | 15:10 |
ykarel | it's a random issue, any volunteer to check that? | 15:11 |
ralonsoh | not this week, sorry | 15:11 |
ykarel | k then i can check later this week | 15:11 |
mtomaska | i can take a look but later in the week | 15:11 |
mtomaska | Friday or Monday | 15:11 |
ykarel | #action ykarel to check https://bugs.launchpad.net/neutron/+bug/2039417 | 15:12 |
ykarel | thanks mtomaska, i will have a look | 15:12 |
ykarel | ykarel to check failures with openstacksdk and postgres job | 15:12 |
ykarel | https://bugs.launchpad.net/neutron/+bug/2039066 | 15:12 |
ykarel | required two patches merged in openstacksdk for this, so all good now | 15:13 |
ykarel | for train one, neutron is marked as eol https://review.opendev.org/c/openstack/releases/+/895196 | 15:13 |
ralonsoh | cool | 15:13 |
ykarel | so once branches are dropped these jobs will stop running | 15:13 |
ykarel | #topic Stable branches | 15:13 |
ykarel | all looks good in stable, Bernard communicated offline | 15:14 |
ykarel | #topic Stadium projects | 15:14 |
ykarel | periodic weekly all green | 15:14 |
ykarel | lajoskatona, anything to add here? | 15:15 |
lajoskatona | yes, | 15:15 |
lajoskatona | nothing more | 15:15 |
ykarel | Thanks | 15:15 |
lajoskatona | this week the fwaas patches were merged, so things are ok | 15:15 |
ykarel | nice | 15:15 |
ykarel | #topic Grafana | 15:15 |
ykarel | https://grafana.opendev.org/d/f913631585/neutron-failure-rate | 15:15 |
ykarel | let's see if all good here | 15:16 |
slaweq | IMO it looks good this week | 15:16 |
ykarel | yes, there are some failures but those are known and being worked upon | 15:17 |
ykarel | btw wdyt if we move this topic at last after discussing known issues/failures? | 15:17 |
mlavalle | pretty subdued | 15:17 |
haleyb | one thing i noticed in the dashboard is it doesn't have every job (i think) | 15:18 |
haleyb | for example py39 | 15:18 |
ykarel | yes it's possibly outdated as not updated from some time | 15:18 |
haleyb | or py311 (non-voting) | 15:18 |
haleyb | we might have renamed some other things recently | 15:18 |
lajoskatona | good catch, we also have some stable dashboards somewhere | 15:19 |
ykarel | Ok i will check the missings and add to the dashboard | 15:19 |
lajoskatona | https://opendev.org/openstack/project-config/src/branch/master/grafana/neutron-stable-minusone.yaml & https://opendev.org/openstack/project-config/src/branch/master/grafana/neutron-stable-minustwo.yaml | 15:19 |
ykarel | #action ykarel to update neutron grafana dashboard | 15:20 |
ykarel | for the question | 15:21 |
ykarel | wdyt if we move this topic at last after discussing known issues/failures? | 15:21 |
lajoskatona | ok from my side | 15:22 |
mlavalle | you mean to be the last topic in the agenda? | 15:22 |
ykarel | as we will have more clear picture once those are discussed on what's missing | 15:22 |
ykarel | yes before on demand one | 15:22 |
mlavalle | let's give it a try | 15:22 |
mlavalle | see what happens | 15:22 |
ykarel | k Thanks, i will move it | 15:23 |
ykarel | #topic Rechecks | 15:23 |
ykarel | we have more rechecks last week | 15:24 |
ykarel | and the intermittent failures in couple of jobs leading to it | 15:24 |
ykarel | and there were 3/14 bare rechecks | 15:24 |
ykarel | and good that many patches merged last week | 15:25 |
ykarel | let's keep doing less bare rechecks | 15:25 |
ykarel | Now let's talk about failures | 15:26 |
ykarel | #topic fullstack/functional | 15:26 |
ykarel | test_floatingip_mac_bindings(seen once) | 15:26 |
ykarel | https://4bb852441388777a25ca-c1d45ef7eb040e2d7e76303651507120.ssl.cf1.rackcdn.com/897045/2/gate/neutron-functional-with-uwsgi/8954870/testr_results.html | 15:26 |
ykarel | anyone recall seeing similar in past? | 15:26 |
ralonsoh | https://bugs.launchpad.net/neutron/+bug/1884986 | 15:28 |
ralonsoh | and I pushed https://review.opendev.org/c/openstack/neutron/+/769880 | 15:28 |
ykarel | ohkk that's long back | 15:29 |
ykarel | let's see if we see more occurrences for it and investigate. | 15:30 |
ykarel | for now seen just once | 15:31 |
ykarel | next is | 15:31 |
ykarel | test_restart_rpc_on_sighup_multiple_workers | 15:31 |
ykarel | it fails as RuntimeError: Expected buffer size: 10, current size: 25 | 15:31 |
ykarel | seen 4 occurrences | 15:31 |
ykarel | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_9cd/periodic/opendev.org/openstack/neutron/master/neutron-functional-with-uwsgi-fips/9cde7d1/testr_results.html | 15:31 |
ykarel | http://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_263/893659/12/check/neutron-functional-with-uwsgi/263727d/testr_results.html | 15:32 |
ykarel | https://20c2c42dd35909dbd48c-6b0cadbc155764a70d2c1954f64b7ab0.ssl.cf2.rackcdn.com/897472/3/check/neutron-functional-with-uwsgi/20b1e5e/testr_results.html | 15:32 |
ykarel | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_e50/897045/2/check/neutron-functional-with-uwsgi/e50c4f6/testr_results.html | 15:32 |
ykarel | this sounds a known bit? | 15:32 |
ralonsoh | this is maybe because the writing queue was not delted properly | 15:32 |
ralonsoh | and is reading the restart + the start messages | 15:32 |
mlavalle | dealt properly? | 15:33 |
ralonsoh | when the queue is retrieved, is deleted at the same time | 15:33 |
mlavalle | ahhh | 15:34 |
ralonsoh | but this is just pure speculation | 15:34 |
ralonsoh | I'll check it | 15:35 |
ykarel | ohkk any volunteer to look into this | 15:35 |
ykarel | k thanks | 15:35 |
ykarel | #action ralonsoh to check failures for test_restart_rpc_on_sighup_multiple_workers | 15:36 |
ykarel | #topic Tempest/Scenario | 15:36 |
ykarel | Failures downloading cirros image like in https://zuul.openstack.org/build/97f5b27439ca469a9cd110087a386c62 | 15:36 |
ykarel | https://review.opendev.org/c/openstack/project-config/+/873735 dropped older cirros images from cache | 15:36 |
ykarel | so images are downloaded during the jobs in stable branches where these images are used and can fail with network issues | 15:37 |
ykarel | We can switch to recent image like 0.5.2 on those branches where 0.5.1 is used to avoid such issues | 15:37 |
opendevreview | Takashi Kajinami proposed openstack/neutron master: Fix python shebang https://review.opendev.org/c/openstack/neutron/+/898591 | 15:37 |
ykarel | on 10/11th oct seen couple of failures due to it | 15:37 |
ralonsoh | qq: was it necessary to remove these images from the cache? | 15:37 |
ralonsoh | if these images are still being used in the CI | 15:37 |
opendevreview | Takashi Kajinami proposed openstack/neutron master: Fix python shebang https://review.opendev.org/c/openstack/neutron/+/898591 | 15:37 |
ykarel | as per comment those older images are not used much | 15:38 |
ralonsoh | but yes, we can bump to 0.5.2 | 15:38 |
ykarel | +1 | 15:38 |
ykarel | any volunteer to update it? | 15:38 |
ralonsoh | I'll push a patch today, it shouldn't be difficult | 15:39 |
ykarel | yeap | 15:39 |
ykarel | Ok thanks | 15:39 |
ykarel | #action ralonsoh to switch stable jobs to use cirros 0.5.2 | 15:39 |
ykarel | test_resize_volume_backed_server_confirm fails randomly in tempest-integrated-networking job, fails with kernel panic | 15:40 |
ykarel | /sbin/init: can't load library 'libtirpc.so.3' Kernel panic - not syncing: Attempted to kill init! exitcode=0x00001000 | 15:40 |
ykarel | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_bbd/884898/7/check/tempest-integrated-networking/bbdf62f/testr_results.html | 15:40 |
ykarel | i seen similar failure in other jobs too running across different projects | 15:40 |
ykarel | ping #openstack-qa to know if this is known bit but haven't got any response yet | 15:41 |
ykarel | Will check again tomorrow else will report a bug for it | 15:41 |
ykarel | #action ykarel to check and report bug for test_resize_volume_backed_server_confirm | 15:42 |
ykarel | #topic Periodic | 15:42 |
ykarel | https://zuul.openstack.org/builds?job_name=neutron-linuxbridge-tempest-plugin-nftables&project=openstack/neutron | 15:42 |
ykarel | this we already discussed, ralonsoh will be looking into it | 15:43 |
ykarel | These jobs are enabled in master periodic recently with https://review.opendev.org/c/openstack/neutron/+/897427 | 15:43 |
ykarel | Stable periodic | 15:43 |
ykarel | https://zuul.openstack.org/builds?job_name=neutron-functional-with-uwsgi-fips&project=openstack%252Fneutron&branch=stable%252Fyoga | 15:43 |
ykarel | this job is failing in stable/yoga since https://review.opendev.org/c/openstack/neutron/+/897044 merged | 15:43 |
ralonsoh | upsss | 15:44 |
ralonsoh | I'll check it | 15:44 |
ralonsoh | this is my fault | 15:44 |
slaweq | but it was green in that patch in check queue | 15:44 |
ykarel | slaweq, it's fips jobs failing,non fips passing | 15:45 |
slaweq | ahh, ok | 15:45 |
lajoskatona | that is still in experimental? | 15:45 |
ykarel | yes should be | 15:45 |
ykarel | k thanks ralonsoh | 15:46 |
slaweq | I wonder what's the different between fips and non-fips which is causing this failure | 15:46 |
slaweq | I wasn't expecting anything what would make there such difference really | 15:46 |
ralonsoh | yeah, the port binding | 15:46 |
ralonsoh | curious | 15:46 |
ykarel | yes looks strange | 15:46 |
ykarel | #action ralonsoh to check failures in neutron-functional-with-uwsgi-fips in stable/yoga | 15:46 |
ykarel | https://zuul.openstack.org/builds?job_name=neutron-functional&project=openstack%252Fneutron&branch=stable%252Fvictoria | 15:48 |
ykarel | it was missing backport https://review.opendev.org/q/I2f8130dc3cf3244be2a44a4ecbdbaa9c7f865731 | 15:48 |
ykarel | i proposed | 15:48 |
ralonsoh | FT job is passing, +2 | 15:49 |
ykarel | thx | 15:49 |
ykarel | https://zuul.openstack.org/builds?job_name=neutron-ovn-tempest-ovs-release-fips&job_name=neutron-ovs-tempest-fips&project=openstack%2Fneutron&branch=stable%2F2023.2&skip=0 | 15:49 |
ykarel | these are running on centos 9 stream and needs RDO stable/2023.2 release | 15:49 |
ykarel | I will check for the status of it | 15:49 |
ykarel | #action to check for failures in 9-stream jobs stable/2023.2 | 15:50 |
ykarel | https://zuul.openstack.org/builds?job_name=neutron-ovn-grenade-multinode-skip-level&branch=stable%252F2023.2 | 15:50 |
opendevreview | Takashi Kajinami proposed openstack/neutron-lib master: Fix python shebang https://review.opendev.org/c/openstack/neutron-lib/+/898604 | 15:50 |
ykarel | This failed in today's run | 15:50 |
ykarel | But This seems one time as https://review.opendev.org/q/I09a37ca451d44607b7dde344c93ace060c7bda01 stable/zed patch merged but not stable/2023.2 when job ran, next run will confirm | 15:50 |
opendevreview | Takashi Kajinami proposed openstack/os-ken master: Fix python shebang https://review.opendev.org/c/openstack/os-ken/+/898605 | 15:51 |
ykarel | So just need to monitor this | 15:51 |
ykarel | That's it for the jobs | 15:51 |
ykarel | #topic On Demand | 15:51 |
ykarel | CI meeting next two weeks | 15:52 |
haleyb | just a reminder the slurp job change is up for review, https://review.opendev.org/c/openstack/neutron/+/895515 | 15:52 |
haleyb | thanks ralonsoh and lajoskatona for reviews | 15:52 |
haleyb | ykarel: when that merges will also need dashboard update | 15:52 |
ralonsoh | we should merge it now | 15:53 |
ykarel | haleyb, sure will take care of it | 15:53 |
ykarel | Next week is vPTG and on Tuesday i will be out as holiday so i propose meeting cancellation next week | 15:53 |
mlavalle | ykarel: did you mean CI meeting wil not take place the next two weeks? | 15:53 |
ykarel | and next to next week i will be out full week | 15:54 |
mlavalle | ahhh ok | 15:54 |
ralonsoh | have fun next week! | 15:54 |
slaweq | ykarel and what about meeting in 2 weeks? We will be in Brno | 15:54 |
mlavalle | understood now | 15:54 |
slaweq | so maybe cancel it too? | 15:54 |
ykarel | slaweq, yes that's the next to next week | 15:54 |
slaweq | ok | 15:54 |
slaweq | ++ | 15:54 |
ykarel | ok to cancel for 2 weeks? | 15:54 |
ralonsoh | yes | 15:54 |
lajoskatona | +1 | 15:55 |
mlavalle | +1 | 15:55 |
ykarel | k Thanks i will send a mail | 15:55 |
ykarel | ANy thing else to discuss? | 15:55 |
slaweq | nothing from me | 15:56 |
ykarel | Ok thanks everyone, see you again after two weeks \o/ | 15:56 |
ykarel | #endmeeting | 15:56 |
opendevmeet | Meeting ended Tue Oct 17 15:56:59 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:56 |
opendevmeet | Minutes: https://meetings.opendev.org/meetings/neutron_ci/2023/neutron_ci.2023-10-17-15.00.html | 15:56 |
opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/neutron_ci/2023/neutron_ci.2023-10-17-15.00.txt | 15:56 |
opendevmeet | Log: https://meetings.opendev.org/meetings/neutron_ci/2023/neutron_ci.2023-10-17-15.00.log.html | 15:56 |
lajoskatona | Bye | 15:57 |
ralonsoh | bye | 15:57 |
slaweq | o/ | 15:57 |
opendevreview | Takashi Kajinami proposed openstack/neutron master: Fix python shebang https://review.opendev.org/c/openstack/neutron/+/898591 | 16:02 |
opendevreview | Takashi Kajinami proposed openstack/os-ken master: Fix python shebang https://review.opendev.org/c/openstack/os-ken/+/898605 | 16:03 |
opendevreview | Takashi Kajinami proposed openstack/neutron-lib master: Fix python shebang https://review.opendev.org/c/openstack/neutron-lib/+/898604 | 16:04 |
opendevreview | Rodolfo Alonso proposed openstack/neutron-tempest-plugin master: Replace cirros image versions not cached in the CI https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/898608 | 16:05 |
opendevreview | Rodolfo Alonso proposed openstack/neutron stable/zed: [stable-only] Replace cirros image versions not cached in the CI https://review.opendev.org/c/openstack/neutron/+/898609 | 16:09 |
opendevreview | Rodolfo Alonso proposed openstack/neutron stable/zed: [stable-only] Replace cirros image versions not cached in the CI https://review.opendev.org/c/openstack/neutron/+/898609 | 16:10 |
opendevreview | Rodolfo Alonso proposed openstack/neutron stable/yoga: [stable-only] Replace cirros image versions not cached in the CI https://review.opendev.org/c/openstack/neutron/+/898610 | 16:10 |
opendevreview | Rodolfo Alonso proposed openstack/neutron stable/xena: [stable-only] Replace cirros image versions not cached in the CI https://review.opendev.org/c/openstack/neutron/+/898611 | 16:13 |
opendevreview | Rodolfo Alonso proposed openstack/neutron stable/wallaby: [stable-only] Replace cirros image versions not cached in the CI https://review.opendev.org/c/openstack/neutron/+/898612 | 16:14 |
*** ralonsoh is now known as ralonsoh_ooo | 16:33 | |
opendevreview | Merged openstack/neutron master: Bump skip-level lower version to stable/2023.1 https://review.opendev.org/c/openstack/neutron/+/895515 | 17:35 |
opendevreview | Brian Haley proposed openstack/neutron master: Remove obsolete PID files before start https://review.opendev.org/c/openstack/neutron/+/895832 | 18:18 |
opendevreview | Alexey Stupnikov proposed openstack/neutron master: Don't set port capabilities for OVS HW offloading https://review.opendev.org/c/openstack/neutron/+/898556 | 19:34 |
opendevreview | Merged openstack/neutron-tempest-plugin master: Replace cirros image versions not cached in the CI https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/898608 | 20:50 |
haleyb | otherwiseguy: hey, if you're around had a question for you. Is there a theoretical limit for how many transactions ovsdbapp can handle in a single commit? | 21:43 |
haleyb | like is 6K too many :-o | 21:43 |
otherwiseguy | haleyb: nothing specific. It's really only limited by what you set the various timeouts to. | 21:43 |
haleyb | it seems to be the default of 180 seconds | 21:44 |
otherwiseguy | We've probably done many times more than that with the maintenance/ovn_db_sync stuff that batches. | 21:44 |
haleyb | yeah, this is ovn_db_sync related | 21:44 |
otherwiseguy | It's basically just kind of a trade off. Much faster than individual txns, but then timeout applies to whole txn. That and if any individual thing fails, whole thing fails. | 21:45 |
haleyb | otherwiseguy: i'm guessing maybe there's some bug in the sync util code (at least in the version i'm using), as i wouldn't have expected that many transactions for dhcp options, or maybe it's updating all the ports | 21:49 |
haleyb | SetLSwitchPortCommand, hmmm | 21:49 |
opendevreview | Brian Haley proposed openstack/neutron master: Update OVN db sync code log messages https://review.opendev.org/c/openstack/neutron/+/890832 | 22:26 |
*** haleyb is now known as haleyb|out | 22:30 | |
opendevreview | Terry Wilson proposed openstack/neutron master: Add support for OVN MAC_Binding aging https://review.opendev.org/c/openstack/neutron/+/893575 | 23:02 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!