Tuesday, 2023-01-10

opendevreviewMerged openstack/neutron-fwaas master: tox4: Add ignore_basepython_conflict = True to tox.ini  https://review.opendev.org/c/openstack/neutron-fwaas/+/86954700:04
sahido/09:17
opendevreviewLajos Katona proposed openstack/neutron master: Add debug logs to OVSInterfaceDriver.plug_new  https://review.opendev.org/c/openstack/neutron/+/86966609:59
opendevreviewMerged openstack/os-vif master: Update gate jobs as per the 2023.1 cycle testing runtime  https://review.opendev.org/c/openstack/os-vif/+/86146810:27
opendevreviewLajos Katona proposed openstack/neutron master: Avoid failure of get_device_details if net from cache is empty  https://review.opendev.org/c/openstack/neutron/+/86967811:03
opendevreviewLajos Katona proposed openstack/neutron master: Add debug logs to OVSInterfaceDriver.plug_new  https://review.opendev.org/c/openstack/neutron/+/86966611:20
ralonsoh#startmeeting networking14:00
opendevmeetMeeting started Tue Jan 10 14:00:36 2023 UTC and is due to finish in 60 minutes.  The chair is ralonsoh. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
opendevmeetThe meeting name has been set to 'networking'14:00
mlavalleo/14:00
ralonsohhello all14:00
*** dasm|off is now known as dasm14:00
isabekHi14:01
haleybo/14:01
rubasovo-14:01
rubasovo/14:01
ralonsohOk, I think we can start. Other folks can join later14:02
ralonsoh#topic announcement14:02
slaweqo/14:02
ralonsoh#link https://releases.openstack.org/antelope/schedule.html14:02
ralonsohlast week was the Antelope-2 milestone14:02
ralonsoh#link https://review.opendev.org/q/project:openstack%252Freleases+status:open14:03
ralonsohIf I'm not wrong, all related project patches are merged14:03
ralonsohn-lib, os-ken and neutronclient14:03
lajoskatonaHi, sorry for beeing late14:04
ralonsohnp14:04
ralonsohlet's move to the next topic14:05
ralonsoh#topic bugs14:05
ralonsohI sent the report 5 mins ago, sorry for that14:05
ralonsohand I still don't have a link for it14:05
ralonsohin any case, there is just one single bug opened last week14:05
ralonsoh#link https://bugs.launchpad.net/neutron/+bug/200231614:05
ralonsohit is unassigned yet14:06
ralonsohyou are welcome to take a look at it, if possible14:06
mlavallethat was easy14:07
ralonsohthis week the bug deputy is lucasgomes14:07
ralonsohnext week jlibosva is14:07
ralonsohok, I'll ping both later to have confirmation 14:08
ralonsohwe have nothing related to os-ken14:09
ralonsohso we can move to specs14:09
ralonsoh#topic specs14:09
*** jlibosva is now known as Guest96514:09
ralonsohwe still have two under review and we should close then in the following 2 weeks14:09
ralonsohfirst one14:09
ralonsoh#link https://review.opendev.org/c/openstack/neutron-specs/+/86797914:09
ralonsohthis one is, IMO, ready14:09
ralonsohI'll check it today and most probably +2 it14:10
ralonsohplease spend some time reviewing it14:10
lajoskatonaack, I feel myself guilty ....14:10
mlavalleI'll look at it today14:10
ralonsohnp, just if you have some time14:10
ralonsohthe other one is14:10
ralonsoh#link https://review.opendev.org/c/openstack/neutron-specs/+/86807614:10
ralonsohI need to address the comments14:10
ralonsohbut I think is ready too (but I need to reply the reviews)14:11
ralonsohBTW, thanks!14:11
mlavalleas soon as you address the comments, I'll llok at it14:11
ralonsohIMO, we have done very well this cycle reviewing specs (and then implementing them)14:11
mlavalle+114:12
slaweqspeaking about implementation14:12
slaweqI recently sent patch https://review.opendev.org/c/openstack/neutron/+/869554 with API definition for new security group rules templates API14:12
slaweqplease review it when You will have some time14:13
ralonsohfor sure, added to my pile for today14:13
slaweqthx14:13
ralonsohnext topic then14:14
ralonsoh#topic community_goals14:14
ralonsohwe have two14:14
ralonsoh1) Clean-up neutron-legacy code in devstack 14:14
ralonsohmost of the job is done14:14
ralonsohpending clean-up patches14:14
ralonsoh#link https://review.opendev.org/q/topic:drop-lib-neutron14:14
slaweqI think that all is merged now14:14
ralonsohwell, yes14:14
ralonsohI was looking at this14:14
ralonsohslaweq, thanks!! good job14:15
slaweqso we can consider this as done now14:15
ralonsohI'll remove this topic then (and add it to the achivements of this cycle)14:15
slaweq++14:15
ralonsohnext one14:15
ralonsoh2) Neutron client deprecation14:15
ralonsoh#link https://bugs.launchpad.net/neutron/+bug/199977414:15
ralonsohlajoskatona, you can tell us more14:15
lajoskatonathat will be long story :-)14:16
lajoskatonaone thing which is that more interesting than my strugling with code and tests14:16
lajoskatonais that the SDK team agreed to move resources to SDK for stadiums14:16
ralonsohcool, nice to know this14:17
ralonsohthat means we can have all resource classes in SDK?14:17
lajoskatonathe main argument was that sdk is used by ansible and other tools so better to have all things there14:17
lajoskatonaexactly14:17
ralonsohperfect14:17
lajoskatonaI started with bgp (alphabetical order :-)), but it is more WIP: https://review.opendev.org/c/openstack/openstacksdk/+/869485?forceReload=true14:18
lajoskatonathat's it for this topic now I think14:18
ralonsohthat's cool that we can start migrating everything to SDK, thanks for this lajoskatona 14:19
ralonsohfor sure, that will take some time, but we'll eventually have one single repo with all the information14:19
ralonsohbtw, we can also collaborate with this14:19
ralonsohof course14:19
ralonsohwe should ask first in the channel (here)14:20
ralonsohjust not to clash efforts14:20
lajoskatona+114:20
ralonsohand that's all! let's move on14:21
ralonsoh#topic on_demand14:21
ralonsohI have nothing in the agenda14:21
ralonsohso please, you are welcome to propose anything14:21
lajoskatonaJust a warning: this evening sometime is the deadline for presentation proposal for the summit14:21
ralonsoh^^^14:22
lajoskatonaI think this is also the deadline for forum proposals14:22
ralonsohright14:22
fricklerare the tox4 issues all resolved now?14:22
ralonsohso far there are no issues in the CI14:22
ralonsohof course, we'll find new issues in the close future14:23
haleybit's never ending14:23
ralonsohprobably in stable branches we didn't send any patch yet14:23
ralonsohand we still need to fix the tox4 issues in fullstack14:23
slaweqdid we pin tox to < 4 in stable branches?14:23
ykarelyes 14:23
ralonsohyes14:24
slaweq++14:24
slaweqgreat14:24
ralonsohbut before we install it14:24
ralonsohtox3, we are using tox414:24
ralonsohthis is why we sent patches for stable releases14:24
ralonsohand that's the summary so far14:24
ralonsohok, we can close the meeting now. Remember we have the CI meeting in 30 mins14:26
ralonsohsee you there14:26
slaweqit will be on IRC only this week14:26
slaweqo/14:26
ralonsoh#endmeeting14:26
opendevmeetMeeting ended Tue Jan 10 14:26:36 2023 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:26
opendevmeetMinutes:        https://meetings.opendev.org/meetings/networking/2023/networking.2023-01-10-14.00.html14:26
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/networking/2023/networking.2023-01-10-14.00.txt14:26
opendevmeetLog:            https://meetings.opendev.org/meetings/networking/2023/networking.2023-01-10-14.00.log.html14:26
ykarelo/14:26
isabeko/14:26
slaweqsee You there14:26
rubasovo/14:26
lajoskatonao/14:26
amorinhey neutron team, I have some failure on mistral-devstack: https://review.opendev.org/c/openstack/mistral-tempest-plugin/+/86957114:59
amorinit failed twice, on two different scenario, but the root cause seems the same14:59
amorinsomething related to ovn14:59
amorindoes it ring a bell to someone?15:00
slaweq#startmeeting neutron_ci15:00
opendevmeetMeeting started Tue Jan 10 15:00:20 2023 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
opendevmeetThe meeting name has been set to 'neutron_ci'15:00
slaweqo/15:00
mlavalleo/15:00
ralonsohamorin, I'll check it after this meeting15:00
ykarelo/15:00
ralonsohhi15:00
* amorin will be quiet during the meeting, thanks ralonsoh15:00
lajoskatonao/15:01
slaweqI think we can start as we have quorum15:01
slaweqGrafana dashboard: https://grafana.opendev.org/d/f913631585/neutron-failure-rate?orgId=115:01
slaweqPlease open now :)15:01
slaweq#topic Actions from previous meetings15:01
slaweq    lajoskatona to check dvr lifecycle functional tests failures15:01
slaweqI saw the same (probably) issue this week too15:02
lajoskatonayes, I still can't reproduce it locally so today I tried to add some extra logs, but seems I touched some shaky part as due to the extra logs 3 unit tests are failing :-)15:02
lajoskatonahttps://review.opendev.org/c/openstack/neutron/+/86966615:03
lajoskatonaso I have to check why my logs break those tests15:03
lajoskatonathat's all for these issue from me15:04
slaweqI just commented there15:04
lajoskatonathanks15:04
slaweqI think You missed "raise" after logging of error15:05
slaweqas You now silently catching all exceptions there15:05
lajoskatonaI check it, the first ps reraised it but the  result was the same, but I will check it again15:05
slaweqok15:05
slaweqnext one15:05
slaweqlajoskatona to check networking-odl periodic failures15:05
lajoskatonano time for it, but I saw the red results, so on my list15:06
slaweq#action lajoskatona to check networking-odl periodic failures15:07
slaweqok, lets keep it for next week15:07
ykarelatleast one error i show was related to tox4 in odl15:07
lajoskatonaykarel: thanks I will check, perhaps it is just to add the magic words to tox.ini as for other projects15:08
ykarelyeap15:08
slaweq++15:10
slaweqthx lajoskatona 15:10
slaweqand ykarel 15:10
slaweqok, next one15:11
slaweqslaweq to check logs in https://55905461b56e292f56bb-d32e9684574055628f247373c3e6dda1.ssl.cf1.rackcdn.com/868379/2/gate/neutron-functional-with-uwsgi/60b4ea3/testr_results.html15:11
slaweqI was checking it and I proposed patch https://review.opendev.org/c/openstack/neutron/+/86920515:11
slaweqI was trying to reproduce the issue in  https://review.opendev.org/c/openstack/neutron/+/869225/ but so far I couldn't15:12
slaweqI run 4 times 20 neutron-functional jobs and all of them passed15:12
slaweqplease review this patch when You will have some time15:13
slaweqand thx haleyb for review15:13
slaweqnext one15:13
slaweqslaweq to talk with hrw about cirros kernel panic15:13
slaweqI talked with hrw last week and he told me to try new cirros15:14
haleybslaweq: np, i'll take a look if there are any updates15:14
slaweqand if that will be still happening, increase memory in the flavor to 192 or 256 MB15:14
ykarelslaweq, so they have seen similar kernel panic with older cirros versions?15:14
ykareli mean it was a known issue and newer version fixing it?15:15
slaweqso I proposed https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/869152 and https://review.opendev.org/c/openstack/neutron/+/86915415:15
ykareljust curios on what was the fix15:15
slaweqykarel he didn't confirm it for sure but his advice was "try newer cirros first"15:15
ykareluhhk i think we have seen that issue very rarely so just trying new cirros we can't confirm until it reproduces15:16
ykarelpersonally /me seen only once15:16
ralonsohdidn't we have problems with nested virt and this cirros version?15:16
slaweqI think I saw something similar once this week too15:16
slaweqhttps://28f5318084af7eb69294-d7da90a475a01486cfcea9707ed18dfb.ssl.cf2.rackcdn.com/864000/5/check/tempest-integrated-networking/10b07aa/testr_results.html15:17
ykarelslaweq, ^ is different15:17
ykareland workaround for that was to use uec image15:17
slaweqok, this one happened in tempest-integrated-networking which isn't using uec image15:18
slaweqthat would explain why it happened this week :)15:18
lajoskatonathis cirros 0.6.1 is not the one on which frickler is working to replace ubuntu minimal am I right?15:18
slaweqlajoskatona I'm not sure on which he was working on15:18
lajoskatonaok15:18
slaweqok, last one15:19
slaweqyatin to check status of ubuntu minimal as advanced image in ovn tempest plugin job15:19
ykarelralonsoh, yes there were issues with cirros 0.6.1 and nested virt15:19
ykareli pushed upated to not use host-passthrough and it worked15:19
ykarelbut there were still some failures in stadium project jobs but /me not checked 15:19
ykarelfor ubuntu minimal pushed https://review.opendev.org/q/topic:ubuntu-minimal-as-adv-image15:20
ykarelnoticed taas too was using regular image so updated those too to use minimal and as per tests working there too15:20
slaweqI just approved taas one15:21
ykarelthx15:21
lajoskatonathanks for it, good catch15:21
slaweqthx ykarel 15:21
slaweqok, I think we can move on to the next topic now15:21
frickleryou were thinking about https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/854910 I guess15:22
slaweq#topic Stable branches15:22
slaweqbcafarel is not here but do You have anything related to stable branches ci?15:22
fricklersince that's auto-abandoned, I guess I'll leave it at that15:22
slaweqfrickler we can restore it if You want to continue15:23
lajoskatonafrickler: yes that was it, the abandon I guess was just the usual cleanup process, we can check if it worth to continue15:23
slaweqok, if there's nothing regarding stable, lets move on15:25
slaweq#topic Stadium projects15:25
slaweqlajoskatona any updates, except odl which we already discussed earlier15:25
lajoskatonaI hope all of them are now safe from the tox4 issue ecept odl15:25
lajoskatonathat's it from me15:26
slaweqlajoskatona I saw many other projects red this week in periodic weekly jobs15:26
slaweqI didn't check results so I'm not sure if that was still tox4 issues or something else15:27
lajoskatonayes those were tox4 issues15:27
slaweqahh, ok15:27
slaweqso next week it should be much more green I hope15:27
lajoskatonaI hope :-)15:28
mlavalleLOL15:28
slaweq:)15:28
slaweqnext topic then15:28
mlavallelet's hope for greener pastures15:28
slaweq#topic Grafana15:28
slaweqhttps://grafana.opendev.org/d/f913631585/neutron-failure-rate15:28
slaweqTBH there is not much data from last few days there and I don't know exactly why15:29
mlavallepeople returnign from holidays until this week maybe15:30
slaweqthere was some spike of failures during the weekend but now seems that things are getting back to normal15:30
slaweqI think we can move on15:31
slaweq#topic Rechecks15:31
slaweqregarding number of rechecks, we are back to the much better numbers15:32
slaweq| 2023-1  | 0.29     | 15:32
slaweq| 2023-2  | 0.0      |15:32
slaweqand I hope it will stay like that for longer time15:32
slaweqregarding bare rechecks, it also seems good:15:32
slaweq3 out of 17 were bare, which is about 18%15:33
slaweqthx for checking ci issues before rechecking15:33
slaweqany questions/comments regarding rechecks?15:33
lajoskatonanope15:34
mlavallethanks for keeping track of it15:34
slaweqok, so lets move on15:35
slaweqnext topic15:35
slaweq#topic fullstack/functional15:35
slaweqfor functional I have 2 issues this week15:35
slaweqneutron.tests.functional.agent.l3.test_keepalived_state_change.TestMonitorDaemon.test_read_queue_change_state15:35
slaweqhttps://c50fdb7f046159692f4d-3059cf1890ea1358c70d952067d56657.ssl.cf2.rackcdn.com/869388/1/check/neutron-functional-with-uwsgi/1e50279/testr_results.html15:35
slaweqanyone wants to check it?15:36
mlavalleo/15:36
slaweqthx mlavalle 15:37
slaweq#action mlavalle to check failed neutron.tests.functional.agent.l3.test_keepalived_state_change.TestMonitorDaemon.test_read_queue_change_state15:37
mlavalleis that the only instance so far?15:38
slaweqsecond one is (probably) the same issue as lajoskatona is already checking:15:38
slaweqhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b1f/869163/3/check/neutron-functional-with-uwsgi/b1f4063/testr_results.html15:38
slaweqyes mlavalle, I found it only once15:38
mlavalleack15:38
ralonsohmlavalle, slaweq it is a timing issue15:38
ralonsohthe text is in the file15:38
ralonsohbut populated just before the timewait15:38
lajoskatonayes at least it is from the DVR ones, but the tracebeck is not exactly the same15:38
ralonsoh(it is in the logs)15:38
slaweqralonsoh so maybe we should increase the timeout slightly then?15:39
ralonsohyes, a couple of secs15:39
mlavalleok, I'll try that15:39
slaweqmlavalle so it should be easy patch for You :)15:39
mlavallehope so :-)15:39
slaweq:)15:39
slaweqnow fullstack15:40
slaweqneutron.tests.fullstack.test_agent_bandwidth_report.TestPlacementBandwidthReport.test_configurations_are_synced_towards_placement(Open vSwitch agent)15:40
slaweqhttps://7a456b090239dc19e21e-66179e131883a8ab832a0afb9e9b5999.ssl.cf5.rackcdn.com/869388/1/check/neutron-fullstack-with-uwsgi/d4f2039/testr_results.html15:41
lajoskatonaI check this one15:41
lajoskatonaonly one occurance?15:41
slaweqanyone wants to check if this is maybe some timing issue or something worth to report and investigate?15:41
slaweqthx lajoskatona 15:41
slaweqyes, it also happened only once so far15:41
lajoskatonaok15:42
slaweq#action lajoskatona to check fullstack failure neutron.tests.fullstack.test_agent_bandwidth_report.TestPlacementBandwidthReport.test_configurations_are_synced_towards_placement(Open vSwitch agent)15:42
slaweqand second one15:42
slaweqIP or gateway not configured properly (again): https://de87c0b256f64d4fa9ad-627fb04945741dffd55f8af38c253b04.ssl.cf2.rackcdn.com/869613/1/check/neutron-fullstack-with-uwsgi/bc5e466/testr_results.html15:42
slaweqthis one I will check15:43
slaweqas it's something what I was hope to be fixed few weeks ago already15:43
slaweq#action slaweq to check IP or gateway not configured properly (again): https://de87c0b256f64d4fa9ad-627fb04945741dffd55f8af38c253b04.ssl.cf2.rackcdn.com/869613/1/check/neutron-fullstack-with-uwsgi/bc5e466/testr_results.html15:43
slaweqand with that I reached end of my list for today :)15:43
slaweq#topic On Demand15:43
slaweqanything else You want do discuss today?15:44
slaweqrelated to the CI of course ;)15:44
ralonsohall good15:44
mlavallenot from me15:44
slaweqok, so I think we can end the meeting now15:45
slaweqthx for attending15:45
mlavalleo/15:45
slaweqhave a great week and see You online15:45
ralonsohsee you15:45
slaweqo/15:45
lajoskatonao/15:45
ykarelo/15:45
slaweq#endmeeting15:45
opendevmeetMeeting ended Tue Jan 10 15:45:54 2023 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:45
opendevmeetMinutes:        https://meetings.opendev.org/meetings/neutron_ci/2023/neutron_ci.2023-01-10-15.00.html15:45
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/neutron_ci/2023/neutron_ci.2023-01-10-15.00.txt15:45
opendevmeetLog:            https://meetings.opendev.org/meetings/neutron_ci/2023/neutron_ci.2023-01-10-15.00.log.html15:45
opendevreviewArnaud Morin proposed openstack/neutron master: Set DVR qr-xyz interfaces DOWN on backup node  https://review.opendev.org/c/openstack/neutron/+/86974115:55
opendevreviewLajos Katona proposed openstack/neutron-lib master: api-ref: fix nits in BGP-Speaker api-ref  https://review.opendev.org/c/openstack/neutron-lib/+/86974516:45
opendevreviewMerged openstack/neutron-tempest-plugin master: [taas] Use ubuntu minimal as advanced image  https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/86921817:03
opendevreviewLajos Katona proposed openstack/neutron-lib master: api-ref: fix nits in BGP-Speaker api-ref  https://review.opendev.org/c/openstack/neutron-lib/+/86974517:11
*** mtomaska__ is now known as mtomaska19:27
*** EugenMayer43 is now known as EugenMayer419:31
opendevreviewBrian Haley proposed openstack/neutron master: Add text to WSGI config guide for debugging  https://review.opendev.org/c/openstack/neutron/+/86977021:42
-opendevstatus- NOTICE: One of our CI job log storage providers appears to be having trouble with log uploads and retrievals. We are in the process of removing that provider from the pool.22:44

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!