Wednesday, 2020-03-04

*** slaweq has joined #openstack-meeting-300:11
*** slaweq has quit IRC00:15
*** igordc has joined #openstack-meeting-300:21
*** yamamoto has joined #openstack-meeting-300:23
*** yamamoto has quit IRC00:28
*** macz_ has quit IRC00:33
*** igordc has quit IRC00:40
*** njohnston_ has joined #openstack-meeting-301:01
*** njohnston_ is now known as njohnston|phone01:01
*** slaweq has joined #openstack-meeting-301:11
*** slaweq has quit IRC01:16
*** igordc has joined #openstack-meeting-301:20
*** njohnston|phone has quit IRC01:26
*** jamesmcarthur has joined #openstack-meeting-301:30
*** igordc has quit IRC01:33
*** jamesmcarthur has quit IRC01:39
*** jamesmcarthur has joined #openstack-meeting-301:40
*** jamesmcarthur has quit IRC01:54
*** yamamoto has joined #openstack-meeting-301:55
*** jamesmcarthur has joined #openstack-meeting-301:57
*** jamesmcarthur has quit IRC02:08
*** yamamoto has quit IRC02:09
*** yamamoto has joined #openstack-meeting-302:09
*** slaweq has joined #openstack-meeting-302:11
*** slaweq has quit IRC02:15
*** jamesmcarthur has joined #openstack-meeting-302:16
*** tsmith_ has joined #openstack-meeting-302:17
*** tsmith2 has quit IRC02:20
*** tsmith_ is now known as tsmith202:20
*** diablo_rojo has quit IRC02:48
*** hongbin has joined #openstack-meeting-302:50
*** slaweq has joined #openstack-meeting-303:11
*** jamesmcarthur has quit IRC03:13
*** jamesmcarthur has joined #openstack-meeting-303:14
*** slaweq has quit IRC03:16
*** apetrich has quit IRC03:18
*** links has joined #openstack-meeting-303:19
*** jamesmcarthur has quit IRC03:31
*** psachin has joined #openstack-meeting-303:33
*** jamesmcarthur has joined #openstack-meeting-303:53
*** apetrich has joined #openstack-meeting-303:56
*** slaweq has joined #openstack-meeting-304:11
*** slaweq has quit IRC04:17
*** jamesmcarthur has quit IRC04:50
*** hongbin has quit IRC04:54
*** diablo_rojo has joined #openstack-meeting-305:04
*** haleyb is now known as haleyb|away05:05
*** slaweq has joined #openstack-meeting-305:11
*** slaweq has quit IRC05:15
*** igordc has joined #openstack-meeting-305:38
*** yamamoto has quit IRC05:56
*** slaweq has joined #openstack-meeting-306:11
*** yamamoto has joined #openstack-meeting-306:11
*** igordc has quit IRC06:11
*** slaweq has quit IRC06:16
*** diablo_rojo has quit IRC06:17
*** e0ne has quit IRC06:55
*** lpetrut has joined #openstack-meeting-307:05
*** lpetrut has quit IRC07:06
*** lpetrut has joined #openstack-meeting-307:06
*** slaweq has joined #openstack-meeting-307:11
*** slaweq has quit IRC07:15
*** yamamoto has quit IRC07:25
*** slaweq has joined #openstack-meeting-307:38
*** slaweq has quit IRC07:43
*** slaweq has joined #openstack-meeting-307:49
*** kozhukalov has quit IRC07:51
*** yamamoto has joined #openstack-meeting-308:01
*** yamamoto has quit IRC08:06
*** kozhukalov has joined #openstack-meeting-308:13
*** maciejjozefczyk has joined #openstack-meeting-308:14
*** yamamoto has joined #openstack-meeting-308:32
*** ralonsoh has joined #openstack-meeting-308:33
*** kozhukalov has quit IRC08:39
*** e0ne has joined #openstack-meeting-309:05
*** yamamoto has quit IRC09:44
*** apetrich has quit IRC09:52
*** yamamoto has joined #openstack-meeting-310:22
*** yamamoto has quit IRC10:27
*** rubasov has joined #openstack-meeting-310:34
*** psachin has quit IRC11:33
*** psachin has joined #openstack-meeting-311:37
*** psachin has quit IRC11:49
*** apetrich has joined #openstack-meeting-312:01
*** raildo has joined #openstack-meeting-312:19
*** yamamoto has joined #openstack-meeting-312:27
*** yamamoto has quit IRC12:28
*** liuyulong has joined #openstack-meeting-313:01
*** yamamoto has joined #openstack-meeting-313:01
*** yamamoto has quit IRC13:06
*** jamesmcarthur has joined #openstack-meeting-313:23
*** jamesmcarthur has quit IRC13:23
*** jamesmcarthur has joined #openstack-meeting-313:24
*** jamesmcarthur has quit IRC13:32
*** jamesmcarthur has joined #openstack-meeting-313:33
*** jamesmcarthur has quit IRC13:38
*** rubasov has quit IRC14:01
*** yamamoto has joined #openstack-meeting-314:06
*** jamesmcarthur has joined #openstack-meeting-314:11
*** psachin has joined #openstack-meeting-314:16
*** jamesmcarthur has quit IRC14:18
*** ricolin has quit IRC14:25
*** rubasov has joined #openstack-meeting-314:31
*** links has quit IRC14:45
*** jamesmcarthur has joined #openstack-meeting-314:53
*** jamesmcarthur has quit IRC14:58
*** psachin has quit IRC15:00
slaweq#startmeeting neutron_ci15:00
openstackMeeting started Wed Mar  4 15:00:26 2020 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
slaweqhi15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
ralonsohhi15:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"15:00
openstackThe meeting name has been set to 'neutron_ci'15:00
njohnstono/15:00
slaweqplease give me 3 minutes before we will start15:00
bcafarelthat gave me time to find the way to this channel15:02
slaweqok, I'm ready now15:02
slaweqfirst of all15:03
slaweqGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate15:03
slaweq#topic Actions from previous meetings15:03
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"15:03
slaweqfirst one:15:03
slaweqslaweq to drop neutron-tempest-dvr job and finally replace it with neutron-tempest-dvr-ha-multinode-full15:03
slaweq    Done: https://review.opendev.org/#/c/710071/15:03
slaweqand the second one15:03
slaweqralonsoh to check periodic neutron-ovn-tempest-ovs-master-fedora job's failures15:03
ralonsohyes, that problem was caused by a OVN test failure15:04
ralonsohactually by an OVN problem15:04
ralonsohthis is now solved and released15:04
slaweqthx ralonsoh15:04
slaweqI saw this test passed e.g. yesterday15:06
slaweqand that's all actions from last week15:06
slaweq#topic Stadium projects15:06
*** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)"15:06
slaweqstandardize on zuul v315:07
slaweqEtherpad: https://etherpad.openstack.org/p/neutron-train-zuulv3-py27drop15:07
slaweqI think that my 2 patches for neutron-vpnaas jobs are ready to review15:07
njohnstonexcellent, I will take a look15:07
slaweqnjohnston: thx15:08
slaweqfrom other projects which I see we are still missing patches for networking-odl and midonet15:08
slaweqand that would be all IMO15:08
bcafarelnext we wan start tackling W goals then? ;)15:09
slaweqbut community seems to slow for us :P15:10
slaweq*to be to slow15:10
slaweqanything else related to the stadium projects?15:11
slaweqor can we move on?15:11
bcafarelnothing from me15:12
slaweqok, so lets move on15:13
slaweq#topic Grafana15:13
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"15:13
slaweq#link http://grafana.openstack.org/dashboard/db/neutron-failure-rate15:13
slaweqgenerally this week it isn't bad15:14
slaweqfunctional jobs went down a bit15:15
slaweqnon-voting ovn jobs are going down nicely too15:15
slaweqneutron-tempest-plugin jobs are below 10% of failures (except this dvr multinode one which is still non-voting)15:16
slaweqbut I again collected some stats about number of rechecks from last few weeks15:16
slaweqweek 5 of 2020 - 3.915:16
slaweqweek 6 of 2020 - 5.815:16
slaweqweek 7 of 2020 - 1.7515:16
slaweqweek 8 of 2020 - 5.415:16
slaweqweek 9 of 2020 - 5.615:16
slaweqweek 10 of 2020 - 3.315:17
slaweqand tbh that sucks :/15:17
bcafarelmaster only I suppose?15:17
slaweqbcafarel: yes, master branch only15:18
slaweqbut I'm affraid that for stable branches it may be even worst15:18
bcafareloh yes15:18
*** lpetrut has quit IRC15:18
bcafarelso if master-only yes these are not nice numbers15:19
bcafarelstable branches were quite unstable so far in 2020, but well most for known reasons15:19
slaweqyes, I know about reasons15:19
slaweqbut still - that sucks :/15:19
slaweqI don't know how but we should do better15:19
slaweq:)15:19
slaweqok, anything else related to grafana?15:20
slaweqahh, one small thing - we need to remove neutron-tempest-dvr job from grafana now, I will send patch for that15:22
slaweq#action slaweq to remove neutron-tempest-dvr job from grafana15:22
slaweqok, so lets go to the next topics15:22
slaweq#topic fullstack/functional15:22
*** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)"15:22
slaweqas usually I went through failures from last few days15:22
slaweqand I didn't found too many (new) problems in fact15:22
slaweqfor functional job there is bug https://bugs.launchpad.net/neutron/+bug/186545315:23
openstackLaunchpad bug 1865453 in neutron "neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.test_mech_driver.TestVirtualPorts.test_virtual_port_created_before fails randomly" [High,Confirmed]15:23
slaweqfailures like:15:23
slaweq    https://1af8099e455ea1c4b83e-70dbf706fcc5aea923a17246b1623369.ssl.cf1.rackcdn.com/710558/4/check/neutron-functional/a4c7cdd/testr_results.html15:23
slaweq    https://87ace9534a53ef030fdb-5ceaba4a59a2578b108bb2f456b59bef.ssl.cf5.rackcdn.com/710782/3/check/neutron-functional/fcf5a14/testr_results.html (different test from the same module)15:23
ralonsohthis is already solved15:23
ralonsohthat was a problem in OVN15:23
maciejjozefczykralonsoh, no, this is different case15:23
ralonsohah ok sorry15:23
ralonsohah yes15:23
maciejjozefczykthe case about port type has been fixed, but slaweq found other logs from the same class ;(15:24
slaweqralonsoh: it's error on assertion of uuids15:24
maciejjozefczykyes15:24
maciejjozefczykI'm gonna take a look on this one15:24
slaweqmaciejjozefczyk: you welcome :P15:24
maciejjozefczykslaweq, ;)15:24
slaweqthx maciejjozefczyk15:24
slaweq#action maciejjozefczyk to take a look at https://bugs.launchpad.net/neutron/+bug/186545315:24
openstackLaunchpad bug 1865453 in neutron "neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.test_mech_driver.TestVirtualPorts.test_virtual_port_created_before fails randomly" [High,Confirmed]15:24
slaweqnow You are really volunteered for it for this week ;)15:25
ralonsohmaybe this problem is due to a race condition, creating several ports at the same time in different tests15:25
ralonsohbut we can discuss this in launchpad15:25
slaweqsure ralonsoh, that can works too for me15:26
maciejjozefczykslaweq, we'll figure out whats wrong, potentially race condition15:27
slaweqthx maciejjozefczyk and ralonsoh :)15:27
slaweqok, lets move on15:27
slaweq    Deletion of namespace issue (again)15:27
slaweq    https://97c37a5ba6a0beef442c-e131daf8cc02ece8b285c96d25e8c4eb.ssl.cf2.rackcdn.com/709110/2/check/neutron-functional/dad905e/testr_results.html15:27
slaweqralonsoh: You should be familiar with this :/15:28
ralonsohI'll take a look at this again15:28
slaweqit's from today morning15:28
*** jamesmcarthur has joined #openstack-meeting-315:28
slaweqand that's all about functional job from me for today15:29
slaweqdo You have anything else You want to discuss?15:29
ralonsohno15:29
njohnstonI saw one failure today in neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.test_mech_driver.TestVirtualPorts, cecking to see if it happens elsewhere https://87ace9534a53ef030fdb-5ceaba4a59a2578b108bb2f456b59bef.ssl.cf5.rackcdn.com/710782/3/check/neutron-functional/fcf5a14/testr_results.html15:30
njohnstonotherwise nothing15:30
slaweqnjohnston: I think it's the same issue like in https://bugs.launchpad.net/neutron/+bug/186545315:30
openstackLaunchpad bug 1865453 in neutron "neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.test_mech_driver.TestVirtualPorts.test_virtual_port_created_before fails randomly" [High,Confirmed] - Assigned to Maciej Jozefczyk (maciej.jozefczyk)15:30
ralonsohmost likely15:31
slaweqso if there is nothing else, lets move on15:31
slaweq#topic Tempest/Scenario15:31
*** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)"15:31
slaweq    neutron_tempest_plugin.scenario.test_multicast.MulticastTestIPv4 failing often on deletion of server15:31
slaweq    https://e485228b4bd099d97917-b4e19c98e06eb23bd3ecd1208eab0884.ssl.cf2.rackcdn.com/703376/14/check/neutron-tempest-plugin-scenario-openvswitch-iptables_hybrid/35cab23/testr_results.html15:31
slaweqI saw it already couple of times15:32
slaweqis there anyone who wants to take a look at this?15:32
ralonsohI can15:32
slaweqthx ralonsoh15:32
slaweqwill You also open LP for that?15:32
ralonsoh(I think lucas sent a patch for this one, I need to check)15:32
ralonsohslaweq, sure15:32
slaweqwould be great, thx15:33
slaweq#action ralonsoh to check "neutron_tempest_plugin.scenario.test_multicast.MulticastTestIPv4 failing often on deletion of server"15:33
slaweqand another issue which I found is15:33
slaweq    Problem with logging of console output:15:33
slaweq    https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_dc4/706152/11/check/neutron-tempest-plugin-scenario-openvswitch/dc49892/testr_results.html15:33
slaweqand that's probably caused by my patch which was merged recently15:33
*** jamesmcarthur has quit IRC15:33
slaweqso I will take a look at it15:33
ralonsohups, that's mine15:34
slaweqyours?15:34
ralonsohI think so, I'll check that15:34
slaweqok, thx a lot15:34
slaweqthat's probably some trivial issue to fix15:34
slaweqand it's not reason why test failed15:34
slaweqbut that error happend after failure of ssh or ping15:35
slaweqbut would be good to get it fixed to see this console output15:35
slaweqaccording to random ssh issues in dvr jobs I still didn't had time to check it locally15:36
ralonsoh(ok, not mine, I added the next method, not this one)15:36
slaweqralonsoh: ok, I will check that15:36
slaweq#action slaweq to check problem with console output in scenario test15:36
slaweqok, and that's all what I have for today actually15:37
slaweqdo You have anything else regarding scenario jobs, or ci in generall?15:37
ralonsohno15:37
bcafarelnope15:38
slaweqgreat15:38
slaweqso lets finish a bit earlier today15:38
slaweqthx for attending and see You next week15:38
slaweqo/15:38
slaweq#endmeeting15:38
ralonsohbye15:38
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"15:38
openstackMeeting ended Wed Mar  4 15:38:51 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:38
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-03-04-15.00.html15:38
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-03-04-15.00.txt15:38
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-03-04-15.00.log.html15:38
bcafarelo/15:39
*** psachin has joined #openstack-meeting-315:40
*** yamamoto has quit IRC15:46
*** yamamoto has joined #openstack-meeting-315:47
*** yamamoto has quit IRC15:50
*** jamesmcarthur has joined #openstack-meeting-315:55
*** psachin has quit IRC15:56
*** psachin has joined #openstack-meeting-316:04
*** jamesmcarthur has quit IRC16:05
*** yamamoto has joined #openstack-meeting-316:27
*** yamamoto has quit IRC16:33
*** ricolin has joined #openstack-meeting-316:37
*** maciejjozefczyk_ has joined #openstack-meeting-316:51
*** maciejjozefczyk has quit IRC16:52
*** negronjl has quit IRC16:53
*** psachin has quit IRC16:53
*** maciejjozefczyk_ has quit IRC16:56
*** negronjl has joined #openstack-meeting-316:57
*** rubasov has quit IRC16:58
*** jamesmcarthur has joined #openstack-meeting-317:11
*** igordc has joined #openstack-meeting-317:33
*** njohnston has quit IRC17:53
*** njohnston has joined #openstack-meeting-317:57
*** igordc has quit IRC18:04
*** igordc has joined #openstack-meeting-318:05
*** diablo_rojo has joined #openstack-meeting-318:11
*** e0ne has quit IRC18:12
*** igordc has quit IRC18:27
*** igordc has joined #openstack-meeting-318:33
*** jamesmcarthur has quit IRC18:56
*** jamesmcarthur has joined #openstack-meeting-319:22
*** jamesmcarthur has quit IRC19:33
*** ralonsoh has quit IRC19:38
*** jamesmcarthur has joined #openstack-meeting-319:47
*** e0ne has joined #openstack-meeting-320:17
*** jamesmcarthur has quit IRC20:49
*** raildo has quit IRC21:02
*** jamesmcarthur has joined #openstack-meeting-321:12
*** yamamoto has joined #openstack-meeting-321:43
*** jamesmcarthur has quit IRC21:44
*** jamesmcarthur has joined #openstack-meeting-321:44
*** yamamoto has quit IRC21:49
*** jamesmcarthur has quit IRC21:56
*** jamesmcarthur has joined #openstack-meeting-321:57
*** jamesmcarthur has quit IRC22:08
*** jamesmcarthur has joined #openstack-meeting-322:10
*** slaweq has quit IRC22:21
*** jamesmcarthur has quit IRC22:22
*** jamesmcarthur has joined #openstack-meeting-322:32
*** jamesmcarthur has quit IRC22:37
*** jamesmcarthur has joined #openstack-meeting-322:44
*** jamesmcarthur has quit IRC22:47
*** jamesmcarthur has joined #openstack-meeting-322:52
*** jamesmcarthur has quit IRC22:53
*** jamesmcarthur has joined #openstack-meeting-322:54
*** jamesmcarthur has quit IRC23:02
*** jamesmcarthur has joined #openstack-meeting-323:09
*** slaweq has joined #openstack-meeting-323:11
*** jamesmcarthur has quit IRC23:12
*** slaweq has quit IRC23:16

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!