| opendevreview | Merged openstack/neutron master: Dont raise RouterInterfaceNotFound on overlap check router ports https://review.opendev.org/c/openstack/neutron/+/866328 | 00:47 |
|---|---|---|
| opendevreview | liuyulong proposed openstack/neutron master: Add meter bandwidth bandwidth support https://review.opendev.org/c/openstack/neutron/+/860767 | 01:14 |
| opendevreview | liuyulong proposed openstack/neutron master: Add host metadata haproxy manager https://review.opendev.org/c/openstack/neutron/+/864649 | 01:59 |
| opendevreview | liuyulong proposed openstack/neutron master: Pass physical bridge informations to OVS agent extension API https://review.opendev.org/c/openstack/neutron/+/866635 | 01:59 |
| elodilles | hi neutron team! fyi, our script could not delete neutron repository's stein, rocky and queens branch because there were merged patches after the *-eol tag. we can still delete the branch (probably manually) if that is OK. this would mean that the last merged patches will be lost (from the history). is this acceptable for the team? | 10:27 |
| ykarel | elodilles, ^ only for neutron repo, right? i mean other network projects are fine | 10:38 |
| ykarel | for neutron i see it's just 1 commit merged post eol tags https://review.opendev.org/q/Ie8dd684a7b79b0a322b1f2d17fffb4d58cfe94fc | 10:38 |
| ykarel | i think it should be fine to loose that considering the commit and the branches state as -eol | 10:44 |
| opendevreview | Sebastian Lohff proposed openstack/neutron master: Gracefully restart dnsmasq to not break tcp DNS https://review.opendev.org/c/openstack/neutron/+/866489 | 10:45 |
| ykarel | just commenting on merged patches so we have history that that commit is lost as merged after creating eol tags | 10:45 |
| ykarel | lajoskatona, slaweq ^ wdyt? | 10:45 |
| ykarel | seems rodolfo is out today | 10:46 |
| amorin | hey team, I am following that page: https://docs.openstack.org/neutron/latest/contributor/testing/fullstack.html#fullstack-testing | 10:46 |
| amorin | but I am still facing error: AssertionError: backend 'mysql' unavailable | 10:46 |
| amorin | mysql is installed and working on my devstack | 10:46 |
| ykarel | amorin, you tried without devstack setup too? | 10:48 |
| amorin | hum, you mean on a fresh ubuntu, cloning devstack but not running stack.sh? | 10:48 |
| amorin | I can try | 10:49 |
| ykarel | yes just tools/configure_for_func_testing.sh /opt/stack/devstack -i | 10:49 |
| ykarel | and running test | 10:49 |
| amorin | ok, doing | 10:49 |
| opendevreview | Sebastian Lohff proposed openstack/neutron master: Gracefully restart dnsmasq to not break tcp DNS https://review.opendev.org/c/openstack/neutron/+/866489 | 10:53 |
| lajoskatona | elodilles, ykarel, Hi, reading.... | 11:00 |
| lajoskatona | ykarel: agree, those patches were accidnetally merged after EOL tag was put on the branch, so leaving comment on them is a good indication that Hey, you see something weird here.... | 11:03 |
| opendevreview | Merged openstack/os-vif master: remove unicode from code https://review.opendev.org/c/openstack/os-vif/+/854486 | 11:40 |
| ykarel | lajoskatona, ack | 12:05 |
| opendevreview | yatin proposed openstack/neutron stable/xena: Increase the timeout for arm64 jobs https://review.opendev.org/c/openstack/neutron/+/866695 | 12:06 |
| opendevreview | yatin proposed openstack/neutron stable/wallaby: Increase the timeout for arm64 jobs https://review.opendev.org/c/openstack/neutron/+/866696 | 12:08 |
| lajoskatona | ykarel: we have only this backport as you mentioned: https://review.opendev.org/q/Ie8dd684a7b79b0a322b1f2d17fffb4d58cfe94fc , perhaps we can bring this to weekly meeting, to have it known by the team | 12:10 |
| ykarel | lajoskatona, +1 | 12:10 |
| opendevreview | yatin proposed openstack/neutron stable/xena: Enable dstat and memory_tracker in functional/fullstack jobs https://review.opendev.org/c/openstack/neutron/+/866701 | 12:15 |
| opendevreview | yatin proposed openstack/neutron stable/wallaby: Enable dstat and memory_tracker in functional/fullstack jobs https://review.opendev.org/c/openstack/neutron/+/866702 | 12:17 |
| elodilles | ykarel_ lajoskatona : ack, thanks for looking after the issue | 13:01 |
| amorin | ykarel it works when not doing ./stack.sh | 13:04 |
| amorin | I also add to clone requirements repo | 13:04 |
| amorin | so we need devstack, requirements and neutron before executing the tools/configure_for_func_testing.sh | 13:05 |
| ykarel | amorin, yes i recall requirements repo is also required | 13:08 |
| amorin | what is confusing is this doc: https://docs.openstack.org/neutron/latest/contributor/testing/testing.html#id4 | 13:09 |
| amorin | it's said: "You can also run ‘./stack.sh’, and if successful, it will have also verified the package dependencies have been met" | 13:09 |
| amorin | so I was thinking that running stack.sh was going to prepare my env for testing, but it's not | 13:10 |
| amorin | it's breaking it | 13:10 |
| lajoskatona | amorin: I think it was me who added that line originally but that was few cycles back, and before focal, so seems the doc must be updated | 13:11 |
| ykarel | amorin, lajoskatona agree that part can be removed | 13:11 |
| amorin | ack, no worries | 13:11 |
| ykarel | additionally i see devstack/neutron aleady listed in docs, can add requirements too to the list | 13:11 |
| amorin | maybe we can directly link to https://docs.openstack.org/neutron/latest/contributor/testing/fullstack.html#fullstack-testing | 13:12 |
| amorin | because it seems duplicated doc | 13:12 |
| ykarel | or just https://docs.openstack.org/neutron/latest/contributor/testing/fullstack.html#how-to-run-fullstack-tests-locally | 13:16 |
| lajoskatona | ykarel, amorin: yeah it is really duplicated | 13:21 |
| slaweq | #startmeeting networking | 14:01 |
| opendevmeet | Meeting started Tue Dec 6 14:01:10 2022 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:01 |
| opendevmeet | The meeting name has been set to 'networking' | 14:01 |
| mlavalle | o/ | 14:01 |
| slaweq | o/ | 14:01 |
| frickler | \o | 14:01 |
| slaweq | ralonsoh is off today and he asked me to lead our team meeting today | 14:01 |
| lajoskatona | o/ | 14:01 |
| slaweq | lets wait few more minutes for others to join | 14:02 |
| rubasov | o/ | 14:04 |
| *** dasm|off is now known as dasm | 14:04 | |
| slaweq | ok, lets start | 14:04 |
| slaweq | #topic announcements | 14:04 |
| slaweq | #link https://releases.openstack.org/antelope/schedule.html | 14:04 |
| slaweq | we are in R-15 and getting closer and closer to Antelope-2 milestone | 14:05 |
| ykarel | o/ | 14:05 |
| slaweq | which will be Jan 2 Jan 6 (in 3 weeks) | 14:05 |
| slaweq | rechecks | 14:05 |
| slaweq | we are good with them | 14:05 |
| mlavalle | +1 | 14:06 |
| slaweq | please remember to check reasons of the failed jobs before rechecking Your patch and give some brief reason of rechecking when You are doing so | 14:06 |
| lajoskatona | do we have this week any serious issue in CI? | 14:06 |
| slaweq | any other announcements from anyone? | 14:06 |
| slaweq | lajoskatona this week I don't think so | 14:07 |
| lajoskatona | as I remember last week some were solved | 14:07 |
| slaweq | but we hade some issue with UT last week | 14:07 |
| lajoskatona | slaweq: ok cool, so I was in sync with reality :-) | 14:07 |
| slaweq | https://review.opendev.org/c/openstack/neutron/+/866572 | 14:07 |
| slaweq | this was fix from ralonsoh | 14:07 |
| slaweq | we do have some failures still, but nothng what would completely broke our gate | 14:08 |
| obondarev | hi | 14:08 |
| slaweq | ok, so I think we can move on | 14:09 |
| slaweq | #topic bugs | 14:09 |
| slaweq | I was bug deputy last week | 14:09 |
| slaweq | and it was pretty busy week TBH | 14:09 |
| slaweq | report is at https://lists.openstack.org/pipermail/openstack-discuss/2022-December/031387.html | 14:10 |
| slaweq | There is couple of bugs without assignee: | 14:10 |
| slaweq | https://bugs.launchpad.net/neutron/+bug/1998474 | 14:10 |
| slaweq | https://bugs.launchpad.net/neutron/+bug/1998749 | 14:10 |
| slaweq | https://bugs.launchpad.net/neutron/+bug/1998751 | 14:10 |
| slaweq | so if You have some time, please take a look at them | 14:11 |
| slaweq | ahh, sorry | 14:11 |
| slaweq | first one is fixed already | 14:11 |
| slaweq | so just 2 without assignee it seems | 14:11 |
| lajoskatona | the last 2 from Liu looks similar (at least both for DVR and flooding phys. bridges) | 14:13 |
| slaweq | yes | 14:13 |
| mlavalle | yeap | 14:13 |
| slaweq | there was also https://bugs.launchpad.net/neutron/+bug/1998235 which is duplicate of very old, and very complex to fix issue in DVR | 14:14 |
| slaweq | https://bugs.launchpad.net/neutron/+bug/1774459 | 14:14 |
| mlavalle | is that the huge patch? | 14:14 |
| slaweq | yes | 14:14 |
| lajoskatona | :-) with 105 iterations | 14:14 |
| slaweq | started by Swami long time ago | 14:15 |
| slaweq | later taken by haleyb and later by me | 14:15 |
| slaweq | but I don't have cycles to continue with it, at least for now | 14:15 |
| haleyb | mostly octavia having issues from what i remember. i don't have cycles at the moment either for such a large patch | 14:16 |
| slaweq | yeah, mostly octavia | 14:17 |
| haleyb | https://review.opendev.org/c/openstack/neutron/+/601336/ is the patch for anyone interested | 14:18 |
| slaweq | and there is workaround - if You will associate FIP to such VIP address then traffic will be centralized but will work | 14:18 |
| slaweq | thx haleyb | 14:18 |
| slaweq | and that's basically all bugs I wanted to mention here | 14:19 |
| slaweq | any other bugs You want to talk about today? | 14:19 |
| slaweq | if not, I guess we can move on | 14:20 |
| slaweq | this week haleyb is our bug deputy | 14:21 |
| slaweq | and next week amotoki will be | 14:21 |
| slaweq | if You have any problems with doing it now, please reach out to ralonsoh | 14:22 |
| slaweq | or me - I can change it too :) | 14:22 |
| slaweq | #topic community-goals | 14:22 |
| slaweq | Consistent and Secure Default RBAC | 14:23 |
| slaweq | we fixed few bugs found by tempest testing | 14:23 |
| lajoskatona | is that job run for every patches now? | 14:23 |
| slaweq | no | 14:24 |
| slaweq | patch https://review.opendev.org/c/openstack/tempest/+/614484 is not merged yet | 14:24 |
| slaweq | but job with new defaults is green https://zuul.opendev.org/t/openstack/build/9395b0d53c8940ecb51718ba933fd1a2 | 14:24 |
| lajoskatona | ok, thanks | 14:24 |
| slaweq | I now need to also try the same change in neutron-tempest-plugin jobs | 14:24 |
| slaweq | if that will be ok, we can switch to use new defaults by default in devstack | 14:25 |
| slaweq | or even change default values in neutron but I will be in contact with gmann about that | 14:26 |
| lajoskatona | ok, if you need help with review or similar, just ping me | 14:26 |
| slaweq | lajoskatona sure, thx a lot | 14:27 |
| slaweq | next goal | 14:27 |
| slaweq | Migration to Ubuntu 22.04 | 14:27 |
| slaweq | seems like our patches are merged and we are good now | 14:27 |
| slaweq | and last one | 14:28 |
| slaweq | Clean-up neutron-legacy code in devstack | 14:28 |
| slaweq | I'm still working on this | 14:28 |
| slaweq | patches https://review.opendev.org/q/topic:drop-lib-neutron | 14:28 |
| slaweq | I still have some problems with neutron-tempest-plugin jobs on top of that patch | 14:28 |
| slaweq | other than that all looks fine - tempest jobs are green, same for devstack jobs | 14:29 |
| slaweq | and that's all about community goals from me | 14:29 |
| slaweq | anything else You want to add/discuss here? | 14:29 |
| slaweq | if not, then lets move on | 14:30 |
| slaweq | #topic on-demand-agenda | 14:30 |
| slaweq | I see one topic there | 14:31 |
| slaweq | Reduce duplication of tests between neutron-tempest-plugin and tempest | 14:31 |
| slaweq | added by lajoskatona but I think it was discussed last week, right? | 14:31 |
| lajoskatona | hmmm, not sure about it | 14:31 |
| lajoskatona | yeah but the summary is this, we have redundatnt tests is tempest and neutron-tempest-plugin | 14:32 |
| lajoskatona | and the question what to do with them, have them only in tempest or only in neutron-tempest-plugin | 14:32 |
| lajoskatona | or let everything as it is now | 14:32 |
| slaweq | the best would be to have them in one place only | 14:33 |
| slaweq | but I think we should sync with kopecmartin or gmann to know where each of such tests should be | 14:34 |
| lajoskatona | I had one patch to move test from n-t-p to tempest (https://review.opendev.org/c/openstack/tempest/+/824440 ) | 14:34 |
| ykarel | lajoskatona, is it being tracked somewhere just to get history about it? | 14:34 |
| lajoskatona | yeah perhaps that is the best to check one by one | 14:34 |
| ykarel | because recently i noticed metering tests are also duplicated | 14:34 |
| lajoskatona | no but I can create an etherpad or such for it | 14:34 |
| ykarel | but there idea iiuc was to move from tempest to n-t-p | 14:34 |
| ykarel | many years back but that was not finished seems | 14:35 |
| lajoskatona | yeah my feeling was also that it is half done in most cases | 14:35 |
| lajoskatona | ok I can create an etherpad with what we have and we can discuss it with QA to have a common understanding and tracking | 14:36 |
| ykarel | +1 to etherpad to track these and cleanup duplicated | 14:36 |
| frickler | maybe you can raise that topic in the QA meeting, too | 14:36 |
| frickler | (right after this one in #-qa) | 14:36 |
| lajoskatona | yeah, I can, I shortly discusssed it with kopecmartin few weeks back, but I will go to him with etherpad also | 14:37 |
| lajoskatona | oh, that is too quick to have a finished etherpad :-) | 14:37 |
| lajoskatona | that's it for this topic from me, thanks for bringing this up | 14:38 |
| slaweq | thx lajoskatona for working on this | 14:38 |
| frickler | you likely have half an hour until we finish our usual agenda ;) | 14:38 |
| mlavalle | we have CI meeting also | 14:38 |
| lajoskatona | frickler: no comment | 14:38 |
| slaweq | anyone have any other topic for our today's meeting? | 14:38 |
| slaweq | if not, I will give You about 20 minutes back :) | 14:38 |
| mlavalle | slaweq: CI meeting on video? | 14:39 |
| lajoskatona | just one which elodilles brought to neutron channel today | 14:39 |
| slaweq | mlavalle today CI meeting is on irc only | 14:39 |
| slaweq | lajoskatona go on with it | 14:39 |
| mlavalle | ack | 14:39 |
| lajoskatona | the EOLing of q r and s branches failed for Neutron as we backported a small patch | 14:39 |
| slaweq | ouch | 14:40 |
| lajoskatona | and the question was to delete the branches and lose that patch, as the EOL tag is already added | 14:40 |
| slaweq | what's that patch | 14:40 |
| slaweq | ? | 14:40 |
| slaweq | do You know? | 14:40 |
| lajoskatona | https://review.opendev.org/q/Ie8dd684a7b79b0a322b1f2d17fffb4d58cfe94fc | 14:40 |
| bcafarel | documentation link on EOL release, definitely sounds like something we can drop there | 14:41 |
| slaweq | I would personally say - delete branches as it's only some docs update | 14:41 |
| lajoskatona | +1 | 14:41 |
| slaweq | but I think that is the question to ralonsoh and amotoki :) | 14:41 |
| lajoskatona | ykarel suggested to comment on the merged patches that this will not be under the EOL tag | 14:42 |
| lajoskatona | I think we can wait with it | 14:42 |
| slaweq | ralonsoh should be back tomorrow IIRC so we can talk about it then | 14:42 |
| lajoskatona | I don't think that release team is just before pushing the big red button :-) | 14:42 |
| slaweq | ok, with that I think we can finish this meeting for today | 14:44 |
| lajoskatona | that's it for this problem, if ralonsoh is back we can ask him to have his word | 14:44 |
| slaweq | thx for attending | 14:44 |
| lajoskatona | o/ | 14:44 |
| mlavalle | o/ | 14:44 |
| obondarev | o/ | 14:44 |
| ykarel | o/ | 14:44 |
| slaweq | and see You in 15 minutes on the CI meeting :) | 14:44 |
| slaweq | o/ | 14:44 |
| slaweq | #endmeeting | 14:44 |
| opendevmeet | Meeting ended Tue Dec 6 14:44:53 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:44 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/networking/2022/networking.2022-12-06-14.01.html | 14:44 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/networking/2022/networking.2022-12-06-14.01.txt | 14:44 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/networking/2022/networking.2022-12-06-14.01.log.html | 14:44 |
| rubasov | o/ | 14:45 |
| opendevreview | Arnaud Morin proposed openstack/neutron master: Add tests for br-tun of restoration https://review.opendev.org/c/openstack/neutron/+/866589 | 14:48 |
| frickler | does anybody know anything about the subnet_onboard extension? I'm not able to find it in the api-ref and was wondering whether that simply has been forgotten. it is also missing in the list of extensions supported by OVN | 14:49 |
| frickler | (and I also had in my mind that I already discussed this issue some time ago, but couldn't find a reference) | 14:49 |
| lajoskatona | frickler: it was never added to the api-ref as I see from the patch series: https://review.opendev.org/q/topic:bp%252Fsubnet-onboard | 14:59 |
| slaweq | #startmeeting neutron_ci | 15:03 |
| opendevmeet | Meeting started Tue Dec 6 15:03:48 2022 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:03 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:03 |
| opendevmeet | The meeting name has been set to 'neutron_ci' | 15:03 |
| mlavalle | o/ | 15:03 |
| slaweq | sorry for being late :) | 15:03 |
| slaweq | hi everyone | 15:03 |
| bcafarel | o/ | 15:04 |
| lajoskatona | o/ | 15:04 |
| slaweq | Grafana dashboard: https://grafana.opendev.org/d/f913631585/neutron-failure-rate?orgId=1 | 15:05 |
| slaweq | #topic Actions from previous meetings | 15:05 |
| mtomaska | o/ | 15:05 |
| slaweq | first one: | 15:05 |
| slaweq | ykarel to fix timeout of the ut jobs in stable/wallaby | 15:05 |
| ykarel | i pushed https://review.opendev.org/q/topic:fix-tox-job-override | 15:06 |
| ykarel | found some more missings so sent all within ^ | 15:06 |
| bcafarel | ykarel++ | 15:07 |
| slaweq | thx ykarel I just approved those patches | 15:08 |
| ykarel | thx | 15:09 |
| slaweq | next one | 15:09 |
| slaweq | ralonsoh to check issue with UT on py3.10 and neutron-lib master https://zuul.openstack.org/build/0820d9ef6a4448cea7f0937cac595ee2 | 15:09 |
| slaweq | ralonsoh is not here today but I'm pretty sure this is fixed now | 15:10 |
| slaweq | and the last one: | 15:10 |
| slaweq | mlavalle to check failing mariadb periodic job | 15:10 |
| mlavalle | I did check that | 15:10 |
| mlavalle | if you look at https://zuul.opendev.org/t/openstack/builds?job_name=neutron-ovn-tempest-mariadb-full&branch=master&skip=0, you will notice the failures started on Nov 19 | 15:10 |
| mlavalle | which is when this merged: https://review.opendev.org/c/openstack/devstack/+/860795 | 15:11 |
| mlavalle | so we started using Ubuntu 22.04 for that job | 15:11 |
| mlavalle | in 20.04 we were using maria db 10.3 | 15:11 |
| mlavalle | in 22.04 we use now 10.6 | 15:11 |
| mlavalle | and there was abig change in authentication in version 10.4: https://mariadb.org/authentication-in-mariadb-10-4/ | 15:12 |
| slaweq | so does it mean that devstack is not compatible with mariadb 10.6 which is in Ubuntu 22.04? | 15:12 |
| mlavalle | yes, the way we handle the creation of the root user password | 15:13 |
| mlavalle | but lat night I figured out how to do it | 15:13 |
| mlavalle | last night I was able to build a debstack in my develpoment envronment | 15:13 |
| mlavalle | so today I will propose a fix to devstack | 15:13 |
| lajoskatona | cool | 15:13 |
| slaweq | thx mlavalle++ | 15:14 |
| mlavalle | so you can keep this action item under my name one more week | 15:14 |
| slaweq | #action mlavalle to fix failing mariadb periodic job | 15:14 |
| ykarel | mlavalle, is https://github.com/openstack/devstack/blob/master/lib/databases/mysql#L117-L120 related ? | 15:14 |
| slaweq | I just changed "check/fix" :) | 15:14 |
| mlavalle | ykarel: yes, that's excatly where the problem is | 15:15 |
| mlavalle | I will tweak those lines | 15:15 |
| ykarel | mlavalle, ohkk | 15:15 |
| * mlavalle alresdy tweaked them in my development system | 15:15 | |
| ykarel | i meant if those task needs to be skipped like bullseye but seems no as per your comment | 15:16 |
| mlavalle | no, Ias I said, I fixed it in my dev system | 15:16 |
| ykarel | ack got it | 15:16 |
| mlavalle | and btw, if there is asimilar problem with bullseye, we might also fix it | 15:17 |
| mlavalle | just hadn't thought of it | 15:17 |
| slaweq | ok, thx mlavalle for working on this | 15:18 |
| mlavalle | :-) | 15:18 |
| slaweq | I think we can move on to the next topic now | 15:18 |
| slaweq | #topic Stable branches | 15:18 |
| slaweq | bcafarel any updates? | 15:18 |
| bcafarel | not a lot coming from me thanks Yatin and Ihar for fixes backported over the last week | 15:19 |
| bcafarel | apart from that no major issues :) | 15:19 |
| slaweq | that's good | 15:19 |
| slaweq | thx bcafarel :) | 15:20 |
| slaweq | #topic Stadium projects | 15:20 |
| slaweq | all jobs seems to be green, except networking-odl | 15:20 |
| lajoskatona | seems ok, netowrking-odl hasa timeout | 15:20 |
| slaweq | :) | 15:20 |
| lajoskatona | I don't know any issues or things to keep an eye on fro stadiums | 15:20 |
| lajoskatona | slightly related | 15:21 |
| slaweq | lajoskatona but that networking-odl job timed out after just 32 minutes | 15:21 |
| lajoskatona | perhaps you saw the mail from zigo regarding py311 failures in some projects | 15:21 |
| slaweq | shouldn't we maybe change it? | 15:21 |
| lajoskatona | but we are free except networking-l2gw, but that is not stadium :-) | 15:21 |
| slaweq | yeah, I saw that email | 15:22 |
| lajoskatona | yeah, we can increase that | 15:22 |
| lajoskatona | I will check it, why it is so low | 15:22 |
| slaweq | and I noticed that there's nothing related to neutron or neutron stadium | 15:22 |
| slaweq | thx lajoskatona | 15:22 |
| slaweq | I think we can move on | 15:23 |
| slaweq | #topic Grafana | 15:23 |
| slaweq | I think all is good in grafana | 15:24 |
| slaweq | anything You want do discuss about it today? | 15:25 |
| mlavalle | it looms good to me | 15:25 |
| mlavalle | looks | 15:25 |
| slaweq | next topic then | 15:26 |
| slaweq | #topic Rechecks | 15:26 |
| slaweq | recheck stats looks ok-ish | 15:26 |
| slaweq | we have 1 recheck in average to get patches merge last week | 15:26 |
| slaweq | but we had that issue with UT which ralonsoh fixed so I hope it will be better | 15:26 |
| slaweq | regarding bare rechecks - it looks very good: | 15:27 |
| slaweq | +---------+---------------+--------------+-------------------+... (full message at <https://matrix.org/_matrix/media/v3/download/matrix.org/lPgDdvPJwQWpngbBVSAzKYST>) | 15:27 |
| slaweq | all rechecks in last 7 days were made with a reason given | 15:27 |
| slaweq | thx a lot for that :) | 15:27 |
| lajoskatona | \o/ | 15:27 |
| mlavalle | +1 | 15:27 |
| ykarel | o/ | 15:28 |
| slaweq | ok, now lets talk about some failures in ci jobs | 15:28 |
| slaweq | #topic fullstack/functional | 15:28 |
| slaweq | neutron.tests.functional.agent.test_ovs_flows.ARPSpoofTestCase.test_arp_spoof_allowed_address_pairs_0cidr | 15:28 |
| slaweq | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a0f/866328/4/gate/neutron-functional-with-uwsgi/a0f0599/testr_results.html | 15:28 |
| slaweq | did You saw such failure before? | 15:29 |
| lajoskatona | no | 15:30 |
| ykarel | no | 15:30 |
| slaweq | of course there's nothing obviosly wrong in the log file :/ | 15:30 |
| slaweq | we can wait and see if similar issues will happen more often | 15:31 |
| slaweq | next one | 15:32 |
| slaweq | neutron.tests.functional.agent.l3.test_keepalived_state_change.TestMonitorDaemon | 15:32 |
| slaweq | https://ef5d43af22af7b1c1050-17fc8f83c20e6521d7d8a3ccd8bca531.ssl.cf2.rackcdn.com/861719/6/check/neutron-functional-with-uwsgi/a488990/testr_results.html | 15:32 |
| slaweq | two tests failed in very similar way | 15:32 |
| slaweq | and it seems that it couldn't connect to ovn: | 15:33 |
| slaweq | 2022-12-02 17:47:15.967 40517 ERROR neutron.agent.linux.utils [None req-c47c1008-664c-47f0-b279-520d6e8e5ac6 - tenid - - - -] Exit code: 1; Cmd: ['ovs-appctl', '-t', '/tmp/tmp3_reqb01/ovnsb_db.ctl', 'exit']; Stdin: ; Stdout: ; Stderr: 2022-12-02T17:47:15Z|00001|unixctl|WARN|failed to connect to /tmp/tmp3_reqb01/ovnsb_db.ctl | 15:33 |
| slaweq | ovs-appctl: cannot connect to "/tmp/tmp3_reqb01/ovnsb_db.ctl" (No such file or directory) | 15:33 |
| slaweq | so most likely some intermittent issue | 15:33 |
| slaweq | next one | 15:34 |
| slaweq | it's again dvr router lifecycle issue: | 15:35 |
| slaweq | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_5c1/866635/1/check/neutron-functional-with-uwsgi/5c12d80/testr_results.html | 15:35 |
| slaweq | lajoskatona I think You were checking something similar already | 15:35 |
| slaweq | am I right? | 15:35 |
| lajoskatona | yes the bug we discussed last week | 15:35 |
| lajoskatona | this one https://bugs.launchpad.net/neutron/+bug/1995031 and newer one: https://bugs.launchpad.net/neutron/+bug/1998337 | 15:36 |
| slaweq | and did You found anything? | 15:37 |
| lajoskatona | no to tell the truth I had no time to check since last week | 15:37 |
| slaweq | will You be able to look into it this week maybe? | 15:37 |
| lajoskatona | I try to allocate some time for it | 15:38 |
| slaweq | thx a lot | 15:38 |
| slaweq | #action lajoskatona to check dvr lifecycle functional tests failures | 15:38 |
| slaweq | and last one | 15:38 |
| slaweq | neutron.tests.functional.agent.l3.test_ha_router.L3HATestFailover.test_ha_router_lost_gw_connection - router object mismatch | 15:39 |
| slaweq | https://b262e0138780f2e869e5-bb966b1b7243c04cadcba4d7bfebb8b0.ssl.cf5.rackcdn.com/865575/1/gate/neutron-functional-with-uwsgi/5fa0dbc/testr_results.html | 15:39 |
| slaweq | found by ykarel | 15:39 |
| ykarel | yes noticed once | 15:39 |
| slaweq | it seems for me from quick look that failover of routers didn't happend there | 15:39 |
| ykarel | from etherpad seems it was seen 6 months back too | 15:39 |
| slaweq | but IIRC we did some improvements in those tests then | 15:40 |
| slaweq | and it was fine for long time | 15:40 |
| slaweq | so maybe this time it was some one time issue or we have something new broken there | 15:41 |
| slaweq | lets see if we will have it more often | 15:42 |
| slaweq | now fullstack tests | 15:42 |
| slaweq | test_multiple_agents_for_network(Open vSwitch agent) | 15:42 |
| slaweq | https://7c4060ce9e9515b5ad0d-5c947c8d22eb7769ff9d2de46bec4cc9.ssl.cf2.rackcdn.com/865994/1/check/neutron-fullstack-with-uwsgi/ebcfef6/testr_results.html | 15:42 |
| slaweq | it seems for me like connectivity to the dhcp namespace was not working fine: | 15:46 |
| slaweq | 2022-11-29 16:08:29.328 28264 DEBUG neutron.tests.fullstack.resources.machine [-] Stopping async dhclient [ip netns exec test-532a73a0-62e4-4220-a890-9a94566cc831 dhclient -4 -lf /tmp/tmph9mjx6t9/tmpjqjphk39/69e46a58-d5c0-495b-825c-78f428c64fd5.lease -sf /home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/bin/fullstack-dhclient-script --no-pid -d portf1ffe5]. stdout: [[]] - stderr: [['Internet Systems Consortium DHCP | 15:46 |
| slaweq | Client 4.4.1', 'Copyright 2004-2018 Internet Systems Consortium.', 'All rights reserved.', 'For info, please visit https://www.isc.org/software/dhcp/', '', 'Listening on LPF/portf1ffe5/fa:16:3e:bb:71:6a', 'Sending on LPF/portf1ffe5/fa:16:3e:bb:71:6a', 'Sending on Socket/fallback', 'DHCPDISCOVER on portf1ffe5 to 255.255.255.255 port 67 interval 3 (xid=0x38250956)', 'DHCPDISCOVER on portf1ffe5 to 255.255.255.255 port 67 interval 8 | 15:46 |
| slaweq | (xid=0x38250956)', 'DHCPDISCOVER on portf1ffe5 to 255.255.255.255 port 67 interval 15 (xid=0x38250956)', 'DHCPDISCOVER on portf1ffe5 to 255.255.255.255 port 67 interval 13 (xid=0x38250956)', 'DHCPDISCOVER on portf1ffe5 to 255.255.255.255 port 67 interval 8 (xid=0x38250956)', 'DHCPDISCOVER on portf1ffe5 to 255.255.255.255 port 67 interval 9 (xid=0x38250956)', 'DHCPDISCOVER on portf1ffe5 to 255.255.255.255 port 67 interval 10 | 15:46 |
| slaweq | (xid=0x38250956)']] _stop_async_dhclient /home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/machine.py:175 | 15:46 |
| slaweq | but it's hard to say why | 15:47 |
| slaweq | if we will see more often that ports aren't configured through DHCP properly in fullstack tests, we will need to have closer look into this issue | 15:47 |
| slaweq | next one | 15:48 |
| slaweq | neutron.tests.fullstack.test_qos.TestPacketRateLimitQoSOvs | 15:48 |
| slaweq | and this one happened at least twice last week: | 15:48 |
| slaweq | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_681/865061/3/check/neutron-fullstack-with-uwsgi/6812ab5/testr_results.html | 15:48 |
| slaweq | https://898e1c4e07fc90ea0741-aa41af48b127881681a990efb23ea8ce.ssl.cf1.rackcdn.com/865470/1/check/neutron-fullstack-with-uwsgi/f46a1f6/testr_results.html | 15:48 |
| lajoskatona | I think this is the fix for it: https://review.opendev.org/c/openstack/neutron/+/866210 | 15:49 |
| slaweq | yeah, thx lajoskatona | 15:49 |
| slaweq | so lets move on | 15:50 |
| slaweq | #topic Tempest/Scenario | 15:50 |
| slaweq | here I found one issue | 15:50 |
| slaweq | where ping from one vm to another failed | 15:50 |
| slaweq | https://cc4296039b95f28dde1a-22b43305544279849f41f0100b51a877.ssl.cf5.rackcdn.com/866328/4/check/neutron-tempest-plugin-linuxbridge/61e1c9a/testr_results.html | 15:50 |
| slaweq | it's linuxbridge job so if it will start happening more often we can simply disable it | 15:50 |
| slaweq | unless there is anybody who have some cycles and wants to check it | 15:51 |
| lajoskatona | +1 for skipping/disable :-( | 15:51 |
| slaweq | and that's pretty much all what I had for today | 15:51 |
| slaweq | in periodic jobs we should be good once mariadb job will be fixed | 15:52 |
| slaweq | anything else You want do discuss today? | 15:52 |
| mlavalle | nothing from me | 15:53 |
| ykarel | nothing from me too | 15:53 |
| slaweq | if not, lets finish earlier today | 15:53 |
| slaweq | thx for attending the meeting | 15:53 |
| slaweq | and have a great week | 15:53 |
| slaweq | #endmeeting | 15:53 |
| opendevmeet | Meeting ended Tue Dec 6 15:53:53 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:53 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/neutron_ci/2022/neutron_ci.2022-12-06-15.03.html | 15:53 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/neutron_ci/2022/neutron_ci.2022-12-06-15.03.txt | 15:53 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/neutron_ci/2022/neutron_ci.2022-12-06-15.03.log.html | 15:53 |
| slaweq | o/ | 15:53 |
| lajoskatona | o/ | 15:53 |
| mtomaska | o/ | 15:53 |
| mlavalle | o/ | 15:54 |
| ykarel | o/ | 15:54 |
| haleyb | lajoskatona: hopefully you saw https://review.opendev.org/c/x/networking-l2gw/+/866620 - i have an incentive to get debian things fixed these days | 16:13 |
| sahid_ | o/ a quick one that should fix a trace in CI https://review.opendev.org/c/openstack/neutron/+/865823 zuul is green, if you have a moment | 16:28 |
| * sahid_ feel bad has this came from one of its patches marged | 16:28 | |
| sahid_ | s/has/as, /marged/merged ... (with probably other grammars or typo issues...) | 16:31 |
| lajoskatona | haleyb: thanks, I will check it | 16:37 |
| lajoskatona | sahid_: thanks for taking care | 16:40 |
| sahid_ | sure | 16:42 |
| opendevreview | Merged openstack/neutron stable/xena: Increase the timeout for arm64 jobs https://review.opendev.org/c/openstack/neutron/+/866695 | 17:23 |
| opendevreview | Merged openstack/neutron stable/wallaby: Increase the timeout for arm64 jobs https://review.opendev.org/c/openstack/neutron/+/866696 | 17:31 |
| opendevreview | Brian Haley proposed openstack/neutron master: Fix firewall_driver examples in docs https://review.opendev.org/c/openstack/neutron/+/866793 | 18:49 |
| opendevreview | Merged openstack/neutron stable/zed: Fix duplicated routes exceptions https://review.opendev.org/c/openstack/neutron/+/865893 | 19:12 |
| opendevreview | Merged openstack/neutron stable/yoga: Fix duplicated routes exceptions https://review.opendev.org/c/openstack/neutron/+/865894 | 19:12 |
| opendevreview | Merged openstack/neutron stable/xena: Fix duplicated routes exceptions https://review.opendev.org/c/openstack/neutron/+/865895 | 19:13 |
| opendevreview | Merged openstack/neutron stable/wallaby: Fix duplicated routes exceptions https://review.opendev.org/c/openstack/neutron/+/865896 | 19:13 |
| opendevreview | Merged openstack/neutron master: neutron-dynamic-routing now works with OVN https://review.opendev.org/c/openstack/neutron/+/738641 | 20:15 |
| opendevreview | Merged openstack/neutron master: Bump pecan to fix #1829042 https://review.opendev.org/c/openstack/neutron/+/865575 | 20:22 |
| *** marlinc is now known as Guest1159 | 20:50 | |
| *** marlinc is now known as Guest1168 | 22:14 | |
| opendevreview | Miro Tomaska proposed openstack/neutron master: Improve agent provision performance for large networks https://review.opendev.org/c/openstack/neutron/+/861124 | 22:33 |
| *** dasm is now known as dasm|off | 22:51 | |
| *** marlinc is now known as Guest1173 | 23:32 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!