| *** tosky has quit IRC | 00:17 | |
| *** ralonsoh has quit IRC | 00:46 | |
| *** ralonsoh has joined #openstack-meeting-3 | 00:46 | |
| *** mlavalle has quit IRC | 01:26 | |
| *** freerunner has quit IRC | 02:12 | |
| *** DinaBelova has quit IRC | 02:12 | |
| *** DinaBelova has joined #openstack-meeting-3 | 02:13 | |
| *** freerunner has joined #openstack-meeting-3 | 02:13 | |
| *** psachin has joined #openstack-meeting-3 | 03:32 | |
| *** baojg has quit IRC | 05:48 | |
| *** baojg has joined #openstack-meeting-3 | 05:52 | |
| *** ralonsoh has quit IRC | 06:46 | |
| *** lpetrut has joined #openstack-meeting-3 | 07:53 | |
| *** slaweq has joined #openstack-meeting-3 | 08:02 | |
| *** ralonsoh has joined #openstack-meeting-3 | 08:15 | |
| *** tosky has joined #openstack-meeting-3 | 08:48 | |
| *** baojg has quit IRC | 08:58 | |
| *** baojg has joined #openstack-meeting-3 | 08:58 | |
| *** baojg has quit IRC | 09:06 | |
| *** baojg has joined #openstack-meeting-3 | 09:07 | |
| *** obondarev has joined #openstack-meeting-3 | 09:13 | |
| *** ricolin has joined #openstack-meeting-3 | 09:19 | |
| *** e0ne has joined #openstack-meeting-3 | 09:40 | |
| *** e0ne has quit IRC | 09:40 | |
| *** baojg has quit IRC | 09:42 | |
| *** baojg has joined #openstack-meeting-3 | 09:43 | |
| *** ricolin has quit IRC | 09:44 | |
| *** e0ne has joined #openstack-meeting-3 | 10:04 | |
| *** macz_ has joined #openstack-meeting-3 | 11:03 | |
| *** macz_ has quit IRC | 11:07 | |
| *** ralonsoh has quit IRC | 11:20 | |
| *** ralonsoh has joined #openstack-meeting-3 | 11:23 | |
| *** e0ne has quit IRC | 11:29 | |
| *** baojg has quit IRC | 11:46 | |
| *** baojg has joined #openstack-meeting-3 | 11:47 | |
| *** e0ne has joined #openstack-meeting-3 | 12:04 | |
| *** Luzi has joined #openstack-meeting-3 | 12:07 | |
| *** baojg has quit IRC | 12:15 | |
| *** baojg has joined #openstack-meeting-3 | 12:16 | |
| *** slaweq has quit IRC | 12:29 | |
| *** slaweq has joined #openstack-meeting-3 | 12:31 | |
| *** artom has joined #openstack-meeting-3 | 13:03 | |
| *** baojg has quit IRC | 13:05 | |
| *** baojg has joined #openstack-meeting-3 | 13:06 | |
| *** belmoreira has joined #openstack-meeting-3 | 13:07 | |
| *** obondarev has quit IRC | 13:19 | |
| *** lpetrut has quit IRC | 13:36 | |
| *** lpetrut has joined #openstack-meeting-3 | 13:38 | |
| *** obondarev has joined #openstack-meeting-3 | 13:48 | |
| *** Luzi has quit IRC | 13:55 | |
| *** lajoskatona has joined #openstack-meeting-3 | 13:59 | |
| slaweq | #startmeeting networking | 14:00 |
|---|---|---|
| openstack | Meeting started Tue Nov 24 14:00:15 2020 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: networking)" | 14:00 | |
| openstack | The meeting name has been set to 'networking' | 14:00 |
| * slaweq needs 5 minutes before we will start | 14:00 | |
| haleyb | hi | 14:00 |
| obondarev | hi | 14:00 |
| lajoskatona | o/ | 14:01 |
| rubasov | o/ | 14:01 |
| ralonsoh | hi | 14:01 |
| bcafarel | \o | 14:02 |
| slaweq | hi | 14:03 |
| slaweq | ok, lets start | 14:03 |
| *** hemanth_n has joined #openstack-meeting-3 | 14:04 | |
| slaweq | #topic Announcements | 14:04 |
| *** openstack changes topic to "Announcements (Meeting topic: networking)" | 14:04 | |
| slaweq | Next week is Wallaby-1 milestone | 14:05 |
| slaweq | Please focus on reviewing opened specs - https://review.opendev.org/q/project:openstack/neutron-specs+status:open | 14:05 |
| slaweq | there are some on the list which waits pretty long time to be reviewed :) | 14:05 |
| slaweq | next one | 14:05 |
| slaweq | as You probably all already noticed, this weekend was upgrade of gerrit | 14:06 |
| bcafarel | yep... | 14:06 |
| slaweq | some details about it can be found on http://lists.opendev.org/pipermail/service-announce/2020-November/000014.html | 14:06 |
| slaweq | and that's basically all announcements from me for today | 14:07 |
| slaweq | anythning else anyone wants to share now? | 14:07 |
| haleyb | And there is also https://etherpad.opendev.org/p/gerrit-3.2-post-upgrade-notes for the bugs | 14:08 |
| lajoskatona | yeah the most important is manual lp bug update as I remmeber | 14:08 |
| slaweq | correct | 14:09 |
| obondarev | does anyone know if it's possible to switch to old interface? :) | 14:09 |
| bcafarel | from that etherpad apparently no, the hooks are not there anymore :( | 14:09 |
| haleyb | i wish it was, as my very useful tampermonkey script is not so useful | 14:09 |
| bcafarel | (including the CI results view that could be updated with tampermonkey) | 14:10 |
| bcafarel | any JS motivated folks to fix that would be most welcome I think! | 14:10 |
| haleyb | bcafarel: yes, that is the worst thing for me, real time results | 14:10 |
| slaweq | obondarev: I wish it would be :) | 14:10 |
| haleyb | there's always gertty | 14:10 |
| obondarev | yeah, CI results are most annoying | 14:10 |
| slaweq | true, and it's even worst when I'm checking many patches before ci meeting :/ | 14:11 |
| slaweq | in old gerrit I saw failed jobs immediately, now I need to read through comments to find them :/ | 14:11 |
| obondarev | hopefully will be fixed soon! | 14:12 |
| slaweq | ++ | 14:12 |
| slaweq | ok, lets move on to the next topic | 14:12 |
| slaweq | #topic Blueprints | 14:12 |
| *** openstack changes topic to "Blueprints (Meeting topic: networking)" | 14:12 | |
| slaweq | Wallaby-1 https://bugs.launchpad.net/neutron/+milestone/wallaby-1 | 14:12 |
| slaweq | any updates here? | 14:13 |
| slaweq | lajoskatona: question about https://bugs.launchpad.net/neutron/+bug/1882804 | 14:14 |
| openstack | Launchpad bug 1882804 in neutron "RFE: allow replacing the QoS policy of bound port" [Wishlist,Confirmed] - Assigned to Lajos Katona (lajos-katona) | 14:14 |
| slaweq | is the only missing here the patch to tempest? | 14:15 |
| lajoskatona | slaweq: yes? | 14:15 |
| slaweq | https://review.opendev.org/q/topic:bug/1882804+(status:open+OR+status:merged) | 14:15 |
| lajoskatona | yes, I think so | 14:15 |
| slaweq | or there is somethig more not included on the list? | 14:15 |
| slaweq | ok | 14:16 |
| lajoskatona | no that's all, thanks fro reminder, I go for QA team to ask them for review | 14:16 |
| slaweq | I will try to look at it and also ping gmann to look at it :) | 14:16 |
| lajoskatona | as I remember last time they had no objection | 14:16 |
| slaweq | that's good :) | 14:17 |
| lajoskatona | thanks | 14:17 |
| slaweq | from the other things, please review https://review.opendev.org/c/openstack/neutron/+/715315 | 14:18 |
| slaweq | ralonsoh did great job with this and it seems that it is last piece to close this long standing BP about migration to the new engine facade | 14:18 |
| slaweq | (even if this engine facade isn't really new anymore :P) | 14:18 |
| ralonsoh | let's go for it!! | 14:18 |
| bcafarel | that will be wallaby cycle highlight for sure (even if it has been "new" for such long time) | 14:19 |
| slaweq | ralonsoh: it failed functional job https://e5392d8b5c67a9e24e1f-aa881d493e66ac85e397d3a18f294794.ssl.cf5.rackcdn.com/715315/24/check/neutron-functional-with-uwsgi/a153821/testr_results.html | 14:19 |
| slaweq | but it doesn't seems to be related to that patch at all | 14:19 |
| ralonsoh | not related, for sure | 14:19 |
| ralonsoh | but I'll keep an eye | 14:20 |
| slaweq | and that's all from my side regarding Blueprints | 14:20 |
| slaweq | any other updates about BPs? | 14:22 |
| slaweq | ok, I'll take it as "no" | 14:23 |
| slaweq | lets move on | 14:23 |
| slaweq | #topic Community Goals | 14:23 |
| *** openstack changes topic to "Community Goals (Meeting topic: networking)" | 14:23 | |
| *** lkoranda has joined #openstack-meeting-3 | 14:23 | |
| slaweq | Migrate from oslo.rootwrap to oslo.privsep | 14:23 |
| ralonsoh | we have several patches uploaded | 14:24 |
| slaweq | regarding this one I worked with ralonsoh a bit and prepared etherpad https://etherpad.opendev.org/p/neutron-migration-to-privsep | 14:24 |
| slaweq | and ralonsoh is pushing patches for that | 14:24 |
| slaweq | if anyone wants to help, please ask ralonsoh about what is todo (or check in etherpad but it may not be ready fully) | 14:24 |
| ralonsoh | (I need to put those links there) | 14:24 |
| slaweq | also - we need some volunteer to work on stadium projects regarding this goal | 14:25 |
| lajoskatona | I add bagpipe later | 14:26 |
| lajoskatona | there's some patches already uploaded for that | 14:26 |
| slaweq | lajoskatona: thx a lot | 14:26 |
| slaweq | regarding Migrate RBAC Policy Format from JSON to YAML - I don't know if amotoki did something already | 14:28 |
| amotoki | regarding the policy format to yaml, I am preparing a patch and will push it this week. it looks like simple enough and we can finish it only in the neutron repo change. | 14:28 |
| slaweq | amotoki: great, thx for info :) | 14:28 |
| slaweq | I think that we can move forward | 14:29 |
| slaweq | #topic Bugs | 14:30 |
| *** openstack changes topic to "Bugs (Meeting topic: networking)" | 14:30 | |
| slaweq | jlibosva was bug deputy last week | 14:30 |
| slaweq | but I didn't saw any email from him | 14:30 |
| slaweq | I just pinged him, maybe he will be able to join | 14:31 |
| slaweq | in the meantime, do You have any bugs You want to discuss today? | 14:33 |
| slaweq | ok, I think that this means "no" :) | 14:35 |
| bcafarel | I don't see any in my launchpad mail folder (some fixes already merged, rest looks "normal" bug week) | 14:35 |
| slaweq | "normal bug week" sounds good ;) | 14:35 |
| haleyb | yeah, there's only a lot when i'm deputy :-/ | 14:35 |
| bcafarel | :) | 14:35 |
| slaweq | haleyb: LOL | 14:35 |
| slaweq | haleyb: speaking about that, there is new round of bug deputy prepared by me | 14:36 |
| slaweq | it starts next week with bcafarel | 14:36 |
| slaweq | and haleyb's week is December 21st | 14:36 |
| slaweq | so I think it should be quiet week :) | 14:36 |
| slaweq | haleyb: are You ok with that? | 14:36 |
| *** jlibosva has joined #openstack-meeting-3 | 14:37 | |
| haleyb | slaweq: hmm, think i'll be on pto that week and the next | 14:37 |
| slaweq | hi jlibosva :) | 14:37 |
| jlibosva | hello :) | 14:37 |
| slaweq | jlibosva: any bugs You want to raise here from last week? | 14:37 |
| jlibosva | yes, I sent the report to openstack-discuss | 14:38 |
| jlibosva | there was one critical bug that needs an assignee | 14:38 |
| jlibosva | https://bugs.launchpad.net/neutron/+bug/1904897 | 14:38 |
| openstack | Launchpad bug 1904897 in neutron "Failure in test_l2_agent_restart(OVS,Flat network)" [Critical,New] | 14:38 |
| jlibosva | I can't find the email in the mailman, so here is the pastebin: http://paste.openstack.org/show/800367/ | 14:39 |
| jlibosva | other than that, there were 3 that I left undecided | 14:40 |
| jlibosva | https://bugs.launchpad.net/neutron/+bug/1904694 - I left a comment asking for clarification as it seemed like some tempest tests are not doing cleanup, not really a Neutron bug | 14:40 |
| openstack | Launchpad bug 1904694 in neutron "Enforce security group cleanup (Resources leaks)" [Medium,Incomplete] - Assigned to Abhishek Jaiswal (jkabhishek) | 14:40 |
| jlibosva | then there was this https://bugs.launchpad.net/neutron/+bug/1904751 that I closed because I couldn't reproduce | 14:41 |
| openstack | Launchpad bug 1904751 in neutron "Neutron – it’s possible to delete router’s port related to external gateway" [Undecided,New] | 14:41 |
| jlibosva | there was a patch in the report pointing to neutron-tempest-plugin to reproduce the issue but the test is failing somewhere else and I don't think it reproduces the issue | 14:41 |
| slaweq | I will work with reporter of https://bugs.launchpad.net/neutron/+bug/1904751 to be sure if that is an issue in Neutron or not | 14:42 |
| openstack | Launchpad bug 1904751 in neutron "Neutron – it’s possible to delete router’s port related to external gateway" [Undecided,New] | 14:42 |
| jlibosva | slaweq: I quickly also checked the last patch he sent and it fails now on "list_ports" based on router_id, api complaining router_id is not a valid keyboard for filtering | 14:43 |
| jlibosva | last undecided is: https://bugs.launchpad.net/neutron/+bug/1904869 - the bug is about BGP agent reporting as dead | 14:43 |
| openstack | Launchpad bug 1904869 in neutron "neutron-dynamic-routing reported as not alive after adding BGP Peer to BGP Speaker" [Undecided,Incomplete] | 14:43 |
| jlibosva | I asked if the RPC layer was configured well because I didn't see it in the conf, now I see the reporter replied | 14:44 |
| jlibosva | it was just my guess that BGP agent uses RPC too as other agents as I'm not well familiar with it | 14:44 |
| slaweq | jlibosva: thx | 14:46 |
| jlibosva | there were other bugs that I set priority and milestone accordingly, more in the report. not sure why it's not on the ML, I got an email that moderator has to approve it, so maybe he hasn't approved the mail yet | 14:46 |
| slaweq | thx jlibosva for the report | 14:47 |
| slaweq | if there will be no other volunteers I will try to investigate this issue with fullstack test | 14:48 |
| haleyb | jlibosva: maybe you used a different email to signup for the list? my redhat one is always rejected | 14:48 |
| jlibosva | haleyb: I actually don't remember :) I also have a RH alias so maybe that's causing the issue. I'll try to look at that, thanks for the tip | 14:49 |
| slaweq | I guess that with that we are done with our today's meeting | 14:51 |
| *** baojg has quit IRC | 14:51 | |
| slaweq | so I'm going to give You back few minutes | 14:51 |
| slaweq | see You on the CI meeting in 9 miutes :) | 14:51 |
| slaweq | o/ | 14:51 |
| jlibosva | o/ | 14:51 |
| ralonsoh | bye | 14:51 |
| slaweq | #endmeeting | 14:51 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 14:51 | |
| openstack | Meeting ended Tue Nov 24 14:51:36 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:51 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/networking/2020/networking.2020-11-24-14.00.html | 14:51 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/networking/2020/networking.2020-11-24-14.00.txt | 14:51 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/networking/2020/networking.2020-11-24-14.00.log.html | 14:51 |
| rubasov | o/ | 14:51 |
| lajoskatona | o/ | 14:52 |
| amotoki | o/ | 14:52 |
| *** baojg has joined #openstack-meeting-3 | 14:52 | |
| *** hemanth_n has quit IRC | 14:53 | |
| *** e0ne has quit IRC | 14:57 | |
| slaweq | #startmeeting neutron_ci | 15:00 |
| openstack | Meeting started Tue Nov 24 15:00:16 2020 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| *** openstack changes topic to " (Meeting topic: neutron_ci)" | 15:00 | |
| openstack | The meeting name has been set to 'neutron_ci' | 15:00 |
| * bcafarel finishes his coffee just in time | 15:00 | |
| gmann | slaweq: lajoskatona how can I help you? | 15:00 |
| slaweq | welcome (again) | 15:00 |
| slaweq | gmann: hi | 15:00 |
| gmann | please ping me link I will review the tempest one. | 15:01 |
| slaweq | gmann: we were just talking about patch | 15:01 |
| slaweq | https://review.opendev.org/c/openstack/tempest/+/743695 | 15:01 |
| slaweq | if You will have some time to review, that would be great :) | 15:01 |
| gmann | slaweq: ack, will check today | 15:01 |
| lajoskatona | Hi | 15:01 |
| slaweq | thx a lot gmann | 15:01 |
| gmann | np! | 15:01 |
| lajoskatona | gmann: Hi, I send it | 15:02 |
| lajoskatona | slaweq was quivker | 15:02 |
| gmann | lajoskatona: sure, i will review it today | 15:02 |
| lajoskatona | gmann: thanks | 15:02 |
| slaweq | ok, lets go with our ci meeting now :) | 15:03 |
| slaweq | Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:03 |
| slaweq | and agenda is on etherpad https://etherpad.opendev.org/p/neutron-ci-meetings | 15:03 |
| slaweq | #topic Actions from previous meetings | 15:03 |
| *** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)" | 15:03 | |
| slaweq | first one was: | 15:03 |
| slaweq | slaweq to report bug regarding errors 500 in ovn functional tests | 15:03 |
| *** macz_ has joined #openstack-meeting-3 | 15:03 | |
| slaweq | it already was reported: https://bugs.launchpad.net/neutron/+bug/1903008 | 15:03 |
| openstack | Launchpad bug 1903008 in neutron "Create network failed during functional test" [High,Confirmed] | 15:03 |
| *** macz_ has quit IRC | 15:03 | |
| slaweq | and we are actually waiting for ralonsoh's patch with engine facade migration first | 15:04 |
| slaweq | so this is "on hold" for now | 15:04 |
| *** macz_ has joined #openstack-meeting-3 | 15:04 | |
| slaweq | and next one was: | 15:04 |
| slaweq | ralonsoh will decrease number of test workers in scenario jobs | 15:04 |
| ralonsoh | merged | 15:04 |
| slaweq | fast :) | 15:04 |
| ralonsoh | https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/763051 | 15:05 |
| ralonsoh | sorry, I didn't find it | 15:05 |
| slaweq | I hope it will make those jobs more stable | 15:06 |
| bcafarel | crossing fingers | 15:06 |
| slaweq | thx ralonsoh :) | 15:07 |
| slaweq | ok, lets move on | 15:07 |
| slaweq | #topic Stadium projects | 15:07 |
| *** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)" | 15:07 | |
| slaweq | anything regarding stadium to discuss today? | 15:07 |
| bcafarel | small stable/stadium update, https://bugs.launchpad.net/neutron/+bug/1903689/comments/5 | 15:08 |
| openstack | Launchpad bug 1903689 in neutron "[stable/ussuri] Functional job fails - AttributeError: module 'neutron_lib.constants' has no attribute 'DEVICE_OWNER_DISTRIBUTED'" [Medium,In progress] - Assigned to Bernard Cafarelli (bcafarel) | 15:08 |
| lajoskatona | nothing special, perhasp this one: https://review.opendev.org/c/openstack/networking-odl/+/763210 | 15:08 |
| bcafarel | basically, adding neutron to upper-constraints needs to be done manually when creating new stable branch (maybe to add to a list of steps for that?) | 15:09 |
| bcafarel | I will send patches for train to victoria (forgot to do it yesterday) to catch up | 15:09 |
| slaweq | bcafarel: can You also check https://docs.openstack.org/neutron/latest/contributor/policies/release-checklist.html if it's is up to date? | 15:10 |
| slaweq | and maybe update with this info if needed | 15:10 |
| bcafarel | ooh nice, I wondered if we had something like that | 15:10 |
| bcafarel | slaweq: will do, and check other stuff I think of (adding branch tempest template, remove *master* jobs, etc) | 15:11 |
| slaweq | bcafarel++ thx a lot | 15:11 |
| slaweq | #action bcafarel to fix stable branches upper-constraints in stadium projects | 15:12 |
| slaweq | #action bcafarel to check and update doc https://docs.openstack.org/neutron/latest/contributor/policies/release-checklist.html | 15:12 |
| slaweq | ^^ just to not forget (that bcafarel voluneer for that :P) | 15:12 |
| bcafarel | :) | 15:12 |
| slaweq | lajoskatona: and regardig Your patch, I already +2 it | 15:13 |
| slaweq | so You need e.g. ralonsoh to check that | 15:13 |
| lajoskatona | slaweq: thanks, just som eadvertisement for more attention :-) | 15:13 |
| ralonsoh | np | 15:14 |
| slaweq | ok, next topic | 15:15 |
| slaweq | #topic Stable branches | 15:15 |
| *** openstack changes topic to "Stable branches (Meeting topic: neutron_ci)" | 15:15 | |
| slaweq | Victoria dashboard: http://grafana.openstack.org/d/pM54U-Kiz/neutron-failure-rate-previous-stable-release?orgId=1 | 15:15 |
| slaweq | Ussuri dashboard: http://grafana.openstack.org/d/dCFVU-Kik/neutron-failure-rate-older-stable-release?orgId=1 | 15:15 |
| bcafarel | my unread backlog for stable is not too bad, so I'd say stble branches are good | 15:16 |
| bcafarel | (well except still pending https://bugs.launchpad.net/neutron/+bug/1903531 ) | 15:16 |
| openstack | Launchpad bug 1903531 in neutron "Update of neutron-server breaks compatibility to previous neutron-agent version" [Critical,Confirmed] - Assigned to Slawek Kaplonski (slaweq) | 15:16 |
| slaweq | sorry, correct links: | 15:16 |
| slaweq | Victoria dashboard: https://grafana.opendev.org/d/HUCHup2Gz/neutron-failure-rate-previous-stable-release?orgId=1 | 15:16 |
| slaweq | Ussuri dashboard: https://grafana.opendev.org/d/smqHXphMk/neutron-failure-rate-older-stable-release?orgId=1 | 15:17 |
| slaweq | thx bcafarel - that is good topic to discuss | 15:18 |
| slaweq | and I forgot about it on previous meeting | 15:18 |
| slaweq | I don't really know what to do with it now :/ | 15:18 |
| slaweq | the problem is that IIUC fix should go to the agent's side | 15:20 |
| slaweq | and if agent will be already updated, there will be no issue at all | 15:20 |
| slaweq | is my understanding correct? | 15:20 |
| bcafarel | checking that original commit again | 15:21 |
| bcafarel | slaweq: so having a fix in agent to handle both types, and note that agents should be updated first for this bug? | 15:22 |
| slaweq | problem is that officially supported update path is that first server should be updated always | 15:23 |
| slaweq | as it should handle compatibility with older agents | 15:23 |
| lajoskatona | what I can't see what happens if the revert will be merged | 15:23 |
| slaweq | not vice versa | 15:23 |
| lajoskatona | how that affects these deployments | 15:23 |
| slaweq | lajoskatona: when we will revert that change, someone who already updated to e.g. 15.3 will have the same issue again | 15:24 |
| slaweq | but in the opposite direction | 15:24 |
| ralonsoh | exactly, they will experience the same problem | 15:24 |
| slaweq | because his server will send (ip, mac) tuple | 15:24 |
| lajoskatona | ok, so the fix would be better | 15:24 |
| slaweq | but how to fix it? | 15:24 |
| ralonsoh | because they have already rebooted the agents | 15:24 |
| *** e0ne has joined #openstack-meeting-3 | 15:24 | |
| ralonsoh | send a patch handling both possible RPC responses | 15:25 |
| ralonsoh | (IP) or (IP, MAC) | 15:25 |
| slaweq | ralonsoh: but that patch needs to be on Agent's side, right | 15:25 |
| slaweq | ? | 15:25 |
| ralonsoh | in both, if I'm not wrong | 15:25 |
| ralonsoh | this is something sent by the server | 15:25 |
| slaweq | ralonsoh: yes, it is send by server | 15:28 |
| slaweq | but how You want to send 2 things by server? | 15:28 |
| ralonsoh | no, if the server is updated, it should send (IP,MAC) | 15:29 |
| slaweq | yes | 15:29 |
| ralonsoh | but, TBH, to those deployments no updated | 15:30 |
| slaweq | so agent should be changed that it would be able to handle both cases | 15:30 |
| ralonsoh | if they follow the update procedures | 15:30 |
| ralonsoh | first the server, then the agents | 15:30 |
| ralonsoh | if we don't revert the original patch, then when the server is updated | 15:30 |
| ralonsoh | the RPC will send (IP,MAC) | 15:30 |
| ralonsoh | and the agents won't understand this | 15:30 |
| slaweq | yes, that's the problem | 15:31 |
| ralonsoh | so maybe we should just revert the patch in stable releases | 15:31 |
| slaweq | but if we will revert it in stable branches, then for deployments which already updated to latest Train (or Stein) the issue will be the same | 15:32 |
| slaweq | updated server will again send just IP | 15:32 |
| slaweq | and agent will expect (IP, MAC) as it will not have reverted change yet | 15:32 |
| bcafarel | I guess it will be limited number of deployments - if we cannot have fix soon it may be the "not so bad" option | 15:33 |
| bcafarel | fix it for people that have not updated yet, with the cost of additional trouble for those (hopefully few) who did | 15:33 |
| slaweq | I think that will be better but I'm not 100% sure about that | 15:34 |
| slaweq | ok, I will try to play with it a bit more | 15:34 |
| slaweq | and lets discuss that on drivers meeting on Friday and decide there what to do with it | 15:34 |
| slaweq | are You ok with this plan? | 15:35 |
| ralonsoh | perfect | 15:35 |
| bcafarel | sounds good | 15:35 |
| *** lpetrut has quit IRC | 15:35 | |
| slaweq | ok | 15:35 |
| slaweq | we may also say that e.g. 15.3 is "broken" and maybe remove it from pypi if possible | 15:36 |
| slaweq | so no new people will update to that verion | 15:36 |
| ralonsoh | that's also an option | 15:36 |
| slaweq | I will ask release team for that | 15:36 |
| bcafarel | +1 that would be good in the meantime | 15:36 |
| slaweq | #action slaweq to explore options to fix https://bugs.launchpad.net/neutron/+bug/1903531 | 15:36 |
| openstack | Launchpad bug 1903531 in neutron "Update of neutron-server breaks compatibility to previous neutron-agent version" [Critical,Confirmed] - Assigned to Slawek Kaplonski (slaweq) | 15:36 |
| slaweq | ok, lets move on now | 15:37 |
| slaweq | #topic Grafana | 15:37 |
| *** openstack changes topic to "Grafana (Meeting topic: neutron_ci)" | 15:37 | |
| slaweq | http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:37 |
| slaweq | in overall I think that it looks not so bad this week | 15:37 |
| ralonsoh | it's getting better, yes | 15:38 |
| slaweq | looking e.g. at https://grafana.opendev.org/d/PfjNuthGz/neutron-failure-rate?viewPanel=20&orgId=1 | 15:39 |
| slaweq | all except ovn job looks pretty good this week | 15:39 |
| slaweq | and there is much less ssh authentication failures recently IMO | 15:39 |
| bcafarel | that's nice | 15:40 |
| slaweq | regarding specific jobs | 15:40 |
| slaweq | #topic Tempest/Scenario | 15:40 |
| *** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)" | 15:40 | |
| slaweq | I was looking at various failures from last week today | 15:41 |
| slaweq | and I didn't found many new issues | 15:41 |
| slaweq | I just found 2 examples of SSH failure in ovn jobs: | 15:41 |
| slaweq | https://7513009f5bff8f76e461-f83d06667d580e000031601b82c71a43.ssl.cf2.rackcdn.com/763246/1/gate/neutron-ovn-tempest-ovs-release/acf920f/testr_results.html | 15:41 |
| slaweq | https://8c281a85ffc729001c78-68bc071a5cbea1ed39a41226592204b6.ssl.cf1.rackcdn.com/763777/1/check/neutron-ovn-tempest-ovs-release-ipv6-only/3e09df5/testr_results.html | 15:41 |
| slaweq | I didn't report it yet on LP | 15:41 |
| slaweq | but I will | 15:41 |
| slaweq | but really https://7513009f5bff8f76e461-f83d06667d580e000031601b82c71a43.ssl.cf2.rackcdn.com/763246/1/gate/neutron-ovn-tempest-ovs-release/acf920f/testr_results.html is probably known issue with some race in paramiko | 15:43 |
| slaweq | but there wasn't console output there | 15:43 |
| ralonsoh | did you push the patch to tempest? | 15:43 |
| slaweq | tempest patch is merged | 15:44 |
| ralonsoh | that one waiting for the VM output to mitigate the paramiko problem | 15:44 |
| ralonsoh | ooook | 15:44 |
| slaweq | https://review.opendev.org/c/openstack/tempest/+/761964 | 15:44 |
| slaweq | but in that case it was waiting for more than 10 minutes, checking console output | 15:44 |
| slaweq | and it failed later :/ | 15:45 |
| slaweq | I don't have any other examples of the failures in tempest jobs for this week | 15:47 |
| slaweq | lets move on | 15:47 |
| slaweq | #topic Rally jobs | 15:47 |
| *** openstack changes topic to "Rally jobs (Meeting topic: neutron_ci)" | 15:47 | |
| slaweq | I found today few cases with failure like: https://zuul.opendev.org/t/openstack/build/be642647ac1e4f5993a65e5f3f91a7a5 in rally job | 15:47 |
| slaweq | do You know maybe if that is known issue? | 15:48 |
| ralonsoh | no | 15:48 |
| slaweq | :/ | 15:48 |
| slaweq | I will report that agains rally as it doesn't seems to be issue in neutron really | 15:49 |
| slaweq | #action slaweq to report bug against rally | 15:50 |
| slaweq | and that's all what I had for today | 15:51 |
| slaweq | do You want to talk about anything else regarding CI today? | 15:51 |
| haleyb | i had one question, kind-of related to CI | 15:51 |
| haleyb | I've been randomly working on fixing issues using IPv6 addresses for tunnel endpoints | 15:52 |
| haleyb | and i sent out a WIP at https://review.opendev.org/c/openstack/neutron/+/760668 | 15:52 |
| haleyb | i was wondering if something like that should just be in one of the existing CI jobs | 15:52 |
| haleyb | truly making things ipv6-only | 15:53 |
| slaweq | haleyb: isn't it like that in tempest-ipv6-only job? | 15:53 |
| slaweq | https://zuul.opendev.org/t/openstack/build/7d267a79a9ef4a6ab3413619a09bf0aa | 15:53 |
| haleyb | i don't think it does the tunnel does it? | 15:54 |
| haleyb | i just added that TUNNEL_IP_VERSION to devstack, it actually hasn't merged yet | 15:55 |
| slaweq | maybe You can then change that tempest-ipv6-only job | 15:55 |
| slaweq | as it is indended to be ipv6-only :) | 15:55 |
| haleyb | slaweq: yes, i thought about that too, just didn't want to break everyone that inherited that | 15:56 |
| haleyb | but maybe noone will notice with the new gerrit :) | 15:57 |
| slaweq | haleyb: if You don't want to break anything for other projects You can propose new job like neutron-tempest-ipv6-only | 15:57 |
| ralonsoh | but slaweq is right, according to the playbooks, "ipv6-only-deployments-verification" should "Verify the IPv6-only deployments" | 15:57 |
| slaweq | which will inherit from tempest-ipv6-only and will also set this one var | 15:57 |
| ralonsoh | and this is executed in tempest ipv6 | 15:58 |
| slaweq | then we can run only this new job in our queue | 15:58 |
| slaweq | haleyb: can You sync with tempest folks what would be better for them? | 15:58 |
| haleyb | slaweq: sure, i can propose something there and ask them for comments | 15:59 |
| slaweq | haleyb++ thx | 15:59 |
| slaweq | ok, we are running out of time now | 15:59 |
| slaweq | thx for attending the meeting | 15:59 |
| slaweq | and see You online | 15:59 |
| ralonsoh | bye | 15:59 |
| bcafarel | o/ | 15:59 |
| slaweq | #endmeeting | 15:59 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:59 | |
| slaweq | o/ | 15:59 |
| openstack | Meeting ended Tue Nov 24 15:59:52 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:59 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-11-24-15.00.html | 15:59 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-11-24-15.00.txt | 15:59 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-11-24-15.00.log.html | 15:59 |
| *** lajoskatona has left #openstack-meeting-3 | 16:01 | |
| *** mlavalle has joined #openstack-meeting-3 | 16:13 | |
| *** obondarev has quit IRC | 16:15 | |
| *** psachin has quit IRC | 16:16 | |
| *** lkoranda has quit IRC | 16:58 | |
| *** e0ne has quit IRC | 17:11 | |
| *** jlibosva has quit IRC | 18:30 | |
| *** jlibosva has joined #openstack-meeting-3 | 18:57 | |
| *** jlibosva has quit IRC | 19:01 | |
| *** jlibosva has joined #openstack-meeting-3 | 19:29 | |
| *** jlibosva has quit IRC | 19:37 | |
| *** belmoreira has quit IRC | 20:08 | |
| *** baojg has quit IRC | 20:52 | |
| *** baojg has joined #openstack-meeting-3 | 20:52 | |
| *** jlibosva has joined #openstack-meeting-3 | 20:59 | |
| *** artom has quit IRC | 21:00 | |
| *** artom has joined #openstack-meeting-3 | 21:01 | |
| *** artom has quit IRC | 21:02 | |
| *** artom has joined #openstack-meeting-3 | 21:03 | |
| *** jlibosva has quit IRC | 21:04 | |
| *** ralonsoh has quit IRC | 21:17 | |
| *** raildo has quit IRC | 21:17 | |
| *** jlibosva has joined #openstack-meeting-3 | 21:40 | |
| *** jlibosva has quit IRC | 21:45 | |
| *** baojg has quit IRC | 21:46 | |
| *** baojg has joined #openstack-meeting-3 | 21:47 | |
| *** baojg has quit IRC | 22:11 | |
| *** baojg has joined #openstack-meeting-3 | 22:12 | |
| *** slaweq has quit IRC | 23:09 | |
| *** jlibosva has joined #openstack-meeting-3 | 23:42 | |
| *** jlibosva has quit IRC | 23:50 | |
| *** tosky has quit IRC | 23:57 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!