| opendevreview | Manu B proposed openstack/neutron-dynamic-routing master: * Status update https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/811300 | 05:48 |
|---|---|---|
| *** ianw is now known as ianw_pto | 07:21 | |
| opendevreview | Lajos Katona proposed openstack/neutron master: Change to publish for security-group db tests https://review.opendev.org/c/openstack/neutron/+/811311 | 07:35 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron stable/xena: [ovn] metadata functional tests don't support Chassis_Private https://review.opendev.org/c/openstack/neutron/+/811330 | 07:48 |
| opendevreview | Zhai Mengdong proposed openstack/neutron master: [OVN] Implement router gateway IP QoS https://review.opendev.org/c/openstack/neutron/+/749012 | 07:57 |
| opendevreview | Manu B proposed openstack/python-neutronclient master: Add neutron and osc commands for bgp speaker router associationw https://review.opendev.org/c/openstack/python-neutronclient/+/803318 | 07:59 |
| opendevreview | liuyulong proposed openstack/neutron master: Make flow installation trunk size automatic adjustment https://review.opendev.org/c/openstack/neutron/+/765072 | 08:56 |
| opendevreview | Eduardo Olivares proposed openstack/neutron master: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/neutron/+/811317 | 09:03 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [DVR] Check if SNAT iptables manager is initialized https://review.opendev.org/c/openstack/neutron/+/811318 | 09:07 |
| opendevreview | Anton Vazhnetsov proposed openstack/ovsdbapp master: ic: add support for OVN_IC_Northbound schema https://review.opendev.org/c/openstack/ovsdbapp/+/809151 | 10:41 |
| opendevreview | Anton Vazhnetsov proposed openstack/ovsdbapp master: ic: add support for OVN_IC_Northbound schema https://review.opendev.org/c/openstack/ovsdbapp/+/809151 | 11:09 |
| opendevreview | Merged openstack/neutron master: Change to publish for security-group db tests https://review.opendev.org/c/openstack/neutron/+/811311 | 11:45 |
| opendevreview | Anton Vazhnetsov proposed openstack/ovsdbapp master: tools: fix OvsOvnVenvFixture init https://review.opendev.org/c/openstack/ovsdbapp/+/809150 | 12:11 |
| lajoskatona | slaweq: Hi, on CI meeting shall we check together why we have different "api_extensions=all" in most of the tempest.confs? | 12:20 |
| slaweq | lajoskatona: sure, we can check it | 12:20 |
| lajoskatona | slaweq: I tried yesterday but can't find the source of it, and it seems that some jobs are failing due to this | 12:20 |
| lajoskatona | slaweq: the starnge is that for OVN jobs we have a list of extensions (https://7ab77c8a0a641a7ff2df-ecc33ccaab89feb6b0f7a7737c56ac51.ssl.cf5.rackcdn.com/807707/4/check/neutron-ovn-tempest-slow/a706584/controller/logs/tempest_conf.txt ) but for ovs jobs we have all (https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d36/807707/4/check/neutron-ovs-tempest-slow/d36b75e/controller/log | 12:21 |
| lajoskatona | pest_conf.txt ) | 12:21 |
| slaweq | lajoskatona: let's check it together on the meeting | 12:22 |
| gibi | lajoskatona: please ping me when you are discussing this I want to at least listen in | 12:24 |
| gibi | as this is breaking nova-grenade-multinode | 12:25 |
| lajoskatona | gibi: ack | 12:30 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Add API extenstion for QoS minimum pps rule https://review.opendev.org/c/openstack/neutron/+/803045 | 12:40 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: ovs-agent: Report pkt processing info in heartbeat https://review.opendev.org/c/openstack/neutron/+/800444 | 12:41 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Report CUSTOM_VNIC_TYPE_ traits on Neutron agent RP https://review.opendev.org/c/openstack/neutron/+/800445 | 12:42 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Report pkt processing capacity on Neutron agent RP https://review.opendev.org/c/openstack/neutron/+/800446 | 12:42 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Add port-resource-request-groups extension https://review.opendev.org/c/openstack/neutron/+/805637 | 12:43 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Enable QoS minimum packet rate rule for OVS backend https://review.opendev.org/c/openstack/neutron/+/805391 | 12:43 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Sanitize profile column of ml2_port_bindings table in the DB https://review.opendev.org/c/openstack/neutron/+/811411 | 12:43 |
| *** jamesdenton_alt is now known as jamesdenton | 12:47 | |
| opendevreview | Merged openstack/neutron master: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/neutron/+/811317 | 13:42 |
| opendevreview | Anton Vazhnetsov proposed openstack/ovsdbapp master: ic: add support for OVN_IC_Northbound schema https://review.opendev.org/c/openstack/ovsdbapp/+/809151 | 13:51 |
| dalvarez | erlon: hey let me catch up with your question | 13:52 |
| dalvarez | erlon: ok so you cant see gARPs but ARP replies instead, right? | 13:53 |
| liuyulong__ | lajoskatona, hi, I got your email, but I did not update the wiki page. I will raise some topics during #On Demand Agenda section if we have enough time. | 13:57 |
| *** liuyulong__ is now known as liuyulong | 13:57 | |
| lajoskatona | liuyulong__: Hi, we can have l3 topic, and discuss them there | 13:58 |
| lajoskatona | liuyulong__: I planned to ave at least a question for l3 if there's anybody with news :-) | 13:59 |
| lajoskatona | #startmeeting networking | 14:00 |
| opendevmeet | Meeting started Tue Sep 28 14:00:14 2021 UTC and is due to finish in 60 minutes. The chair is lajoskatona. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'networking' | 14:00 |
| ralonsoh | hello | 14:00 |
| lajoskatona | Hi everybody! | 14:00 |
| erlon | dalvarez: yes, so, I dont see ARP reponses if I dont' ping the ip | 14:00 |
| rubasov | o/ | 14:00 |
| liuyulong | o/ | 14:00 |
| amotoki | o/ | 14:00 |
| haleyb | o/ | 14:00 |
| slaweq | hi | 14:01 |
| dalvarez | erlon: i think you may be missing this: https://github.com/ovn-org/ovn/commit/96959e56d634c8d888af9e3ee340602593c7e4fa | 14:01 |
| dalvarez | (sorry for the noise) | 14:01 |
| erlon | but the thing is, there are some switches that once a ip is dicover, they keep sending pings to refresh the cache | 14:01 |
| lajoskatona | dalvarez: side effect of no dedicated meeting channel :-) | 14:01 |
| lajoskatona | #topic Announcements | 14:01 |
| lajoskatona | Xena cycle calendar https://releases.openstack.org/xena/schedule.html | 14:02 |
| lajoskatona | This week is final RC week, see: http://lists.openstack.org/pipermail/openstack-discuss/2021-September/025081.html | 14:02 |
| erlon | dalvarez: hmmm, that makes sense, sice the version Im using is 20.03, but have that other gARP patch of yours backported | 14:02 |
| bcafarel | o/ | 14:02 |
| njohnston | o/ | 14:02 |
| lajoskatona | Please check the priority patches to review things quickly if we have to push fixes quickly | 14:03 |
| erlon | dalvarez: do you remembered if you backported this one too? | 14:03 |
| lajoskatona | Don't know if You follow the current gate issue with apache urls: http://lists.openstack.org/pipermail/openstack-discuss/2021-September/025101.html | 14:03 |
| slaweq | lajoskatona: there is only one now: https://review.opendev.org/c/openstack/neutron/+/810975 | 14:04 |
| slaweq | and it seems to be blocked by the apache issue :/ | 14:04 |
| lajoskatona | slaweq: thanks | 14:04 |
| lajoskatona | I think these are the fixes on different branches in devstack: | 14:04 |
| lajoskatona | https://review.opendev.org/c/openstack/devstack/+/811399 & https://review.opendev.org/c/openstack/devstack/+/811389 & https://review.opendev.org/c/openstack/devstack/+/811303 | 14:05 |
| dalvarez | erlon: meeting going on, i don't want to interfere but not, the backport goes down only from master to v21.06.0 branch | 14:05 |
| lajoskatona | slaweq: by the way, and these: https://review.opendev.org/q/Id73368576a948f78a043d7cf0be16661a65626a9 ? | 14:05 |
| lajoskatona | we can discuss these on the bug section, but wanted to have a feedback if we have to have that in xena | 14:06 |
| erlon | hmm, sorry guys, didn't know the meetings were hosted on the channel now | 14:06 |
| slaweq | lajoskatona: true, that one is also important | 14:07 |
| slaweq | lajoskatona: I just set review-priority +2 for the Xena patch | 14:07 |
| lajoskatona | slaweq: ok, thanks | 14:08 |
| slaweq | lajoskatona: thx for reminding about it :) | 14:08 |
| lajoskatona | slaweq: if I understand well the xean patch is almost a backport/cherry-pick , so if possible would be good to merge the master one first | 14:09 |
| lajoskatona | but we can discuss it later or in review | 14:09 |
| slaweq | lajoskatona: ralonsoh will have details for sure, but it is like that | 14:09 |
| ralonsoh | sure | 14:09 |
| slaweq | I'm just worried if we will be able to get merged both master and xena so fast | 14:09 |
| lajoskatona | slaweq: yeah, that's my concern too | 14:10 |
| lajoskatona | ok, let's discuss it separately | 14:10 |
| lajoskatona | Do you have anything to announce? | 14:11 |
| lajoskatona | Ok let's move then | 14:12 |
| lajoskatona | #topic Bugs | 14:12 |
| lajoskatona | bcafarel was bug deputy last week: http://lists.openstack.org/pipermail/openstack-discuss/2021-September/025082.html | 14:12 |
| bcafarel | thanks I was looking for the link :) | 14:12 |
| lajoskatona | bcfarel: :-) quick copy-paste | 14:13 |
| lajoskatona | From the critical section we are waiting for https://review.opendev.org/c/openstack/neutron/+/810975 to be merged to xena, so we are good from that perspective | 14:14 |
| lajoskatona | bcafarel: do you have any bug to discuss? | 14:14 |
| bcafarel | lajoskatona: no all good, as you said critical are being backported, the rest has assignees (or even fixes) | 14:15 |
| bcafarel | (though as mentioned we have a few pending xena backports and that placement issue blocking gates) | 14:16 |
| lajoskatona | thanks bcafarel for the report and the open eyes :-) | 14:17 |
| lajoskatona | This week is rubasov's week and next week is for ralonsoh. | 14:17 |
| ralonsoh | ok | 14:17 |
| lajoskatona | is that ok? | 14:17 |
| rubasov | ok | 14:17 |
| lajoskatona | ok, thanks | 14:18 |
| lajoskatona | Ok next topic then: | 14:18 |
| lajoskatona | #topic L3 | 14:19 |
| lajoskatona | liuyulong has some topics, butperhaps he has network issues | 14:19 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron stable/xena: Execute the quota reservation removal in an isolated DB txn https://review.opendev.org/c/openstack/neutron/+/811124 | 14:20 |
| liuyulong | Hi | 14:21 |
| liuyulong | Thanks | 14:21 |
| liuyulong | #link https://review.opendev.org/c/openstack/neutron-specs/+/770540 | 14:21 |
| lajoskatona | liuyulong: welcome back online :-) | 14:21 |
| liuyulong | I'd like to raise the topic "elastic snat". | 14:21 |
| liuyulong | We have implemented this locally, but the spec is still in in-progress state upstream. | 14:22 |
| lajoskatona | liuyulong: ack, please do that | 14:22 |
| liuyulong | #link https://review.opendev.org/c/openstack/neutron-specs/+/770540/14/specs/xena/elastic_snat.rst@110 | 14:23 |
| slaweq | liuyulong: I will review it again | 14:24 |
| liuyulong | About this input "internal_cidrs", slaweq suggested that we make it coupled with the address group. | 14:25 |
| slaweq | yes, because we have already that resource which can represent it so why not use it here? | 14:26 |
| liuyulong | When we make this feature a real production on our cloud. | 14:26 |
| liuyulong | We have received complaints from the upper platform about the complexity of API calls copled with address group. | 14:27 |
| liuyulong | Every the user needs to update the ElasticSnat, they need to show it. And list/find the address groups, update the related address groups. | 14:28 |
| amotoki | liuyulong: is it described or replied in the spec? (I haven't read thru the spec though :p) | 14:29 |
| liuyulong | Instead of a simple update API of ElasticSnat. | 14:29 |
| slaweq | but address groups may give You some possibilities to e.g. create group which will be used in some SG and also in that SNAT if needed | 14:29 |
| liuyulong | amotoki, yes, I replied those. | 14:30 |
| slaweq | IMHO there are many different use cases and we should use what we already have in Neutron | 14:30 |
| slaweq | if that is possible | 14:30 |
| slaweq | anyway, I'm not going to block this only because of that, but I would like others opinion about it too :) | 14:30 |
| spatel | Does neutron support providing two nic for bonding? (this is the case of SRIOV based design) | 14:30 |
| ralonsoh | there is no support for this because that is part of the underlying infraestructure | 14:31 |
| lajoskatona | spatel: Hi, we have team meeting now, we can discuss it during the on demand agenda | 14:32 |
| spatel | sorry about that. | 14:32 |
| liuyulong | Update is just one example. Creating API has same issue. And for the API input validation, users address group IDs needs elastic_snat plugin to read DB one time to get the CIDR list. | 14:33 |
| liuyulong | While the real CIDRs will be used to verify if it is valid for this router/subnets/ports. | 14:33 |
| lajoskatona | slaweq, liuyulong: I tend to be on the side to use already existing API-s, | 14:33 |
| lajoskatona | slaweq, liuyulong: I will check the discussion in the review, to have deeper understanding of the problem here | 14:34 |
| slaweq | lajoskatona++ thx a lot | 14:34 |
| amotoki | I will check the spec too. Complaints in user side may be able to be addressed in CLI or tool side too (unless it leads to performance issues in the server side) | 14:35 |
| slaweq | amotoki++ thx | 14:36 |
| liuyulong_ | sorry, lost the connection. | 14:36 |
| liuyulong_ | Next one is https://review.opendev.org/c/openstack/neutron-specs/+/802854 | 14:37 |
| liuyulong_ | "Spec for distributed datapath for metadata" | 14:37 |
| liuyulong_ | It has the metadata data path in detail. So call for reviewers. | 14:37 |
| liuyulong_ | It's time to say goodbye to metadata-agent. | 14:38 |
| liuyulong_ | So here I have a very good picture about Neutron in the future. | 14:39 |
| lajoskatona | liuyulong_: I will/ or you can raise the priority, I think this week will be a little crowded with ongoing release, but I will check this one as well | 14:39 |
| liuyulong_ | Neutron has only ovs-agent and L3-agent. Then the pressure of MQ and DB may narrow down to make Neutron to adopt to a real large scale deployment. | 14:40 |
| slaweq | I also added it to my todo list | 14:40 |
| liuyulong_ | And more about L3 is, if someday the openflow related L3 are accomplished. (https://review.opendev.org/q/topic:%22bug%252F1931953%22+(status:open%20OR%20status:merged)) | 14:40 |
| liuyulong_ | With ovs-dpdk or smartNIC offload, the dataplane performance will get better as well. | 14:41 |
| liuyulong_ | So, go go go, let's make Neutron more and more powerful and efficient. | 14:42 |
| liuyulong_ | : ) | 14:42 |
| lajoskatona | liuyulong_: thanks, hope that performance will better with these options | 14:43 |
| obondarev_ | liuyulong_: sounds great, thanks! :) | 14:43 |
| opendevreview | Eduardo Olivares proposed openstack/neutron stable/xena: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/neutron/+/811340 | 14:43 |
| opendevreview | Eduardo Olivares proposed openstack/neutron stable/wallaby: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/neutron/+/811341 | 14:44 |
| liuyulong_ | lajoskatona, no more L3 topics from me now. : ) | 14:44 |
| *** obondarev_ is now known as obondarev | 14:44 | |
| opendevreview | Eduardo Olivares proposed openstack/neutron stable/victoria: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/neutron/+/811342 | 14:44 |
| lajoskatona | liuyulong_: One question: Do these options like flow based dvr need some extra care for migrating from old L3 or metadata? | 14:44 |
| opendevreview | Eduardo Olivares proposed openstack/neutron stable/ussuri: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/neutron/+/811343 | 14:44 |
| lajoskatona | liuyulong_: but I will check and ask in the spec reviews | 14:45 |
| lajoskatona | liuyulong_: thanks for the specs higlight | 14:45 |
| liuyulong_ | Link down is inevitable. Because the troditional namespace and iptables rules should be removed. | 14:45 |
| liuyulong_ | And the flows should be added then. | 14:46 |
| *** liuyulong_ is now known as liuyulong | 14:46 | |
| lajoskatona | Ok, thanks, do anybody has perhaps anything more to discuss regarding L3? | 14:46 |
| lajoskatona | #topic On Demand Agenda | 14:47 |
| liuyulong | We can ask the migration issues in the spec patch of gerrit. Actually, I have no details about it either. | 14:47 |
| liuyulong | I have one topic. | 14:48 |
| liuyulong | #link https://bugs.launchpad.net/neutron/+bug/1753466 | 14:48 |
| liuyulong | Since some smartNIC vendor indicates that their card may not support some CT actions. | 14:49 |
| liuyulong | So stateful security group will be needed for it. | 14:49 |
| liuyulong | #link https://review.opendev.org/c/openstack/neutron/+/804807 | 14:50 |
| liuyulong | #link https://review.opendev.org/c/openstack/neutron/+/789974 | 14:50 |
| ralonsoh | but is this only related to stateless FW? | 14:50 |
| liuyulong | These two patches are for OVN to support stateless security gourp and NAT. | 14:50 |
| liuyulong | So, here I'd like to mention that for ovs-agent we will need to supoort it too. | 14:51 |
| lajoskatona | This bug isfor ovs: https://bugs.launchpad.net/neutron/+bug/1885261 | 14:51 |
| liuyulong | ralonsoh, yes | 14:52 |
| liuyulong | lajoskatona, yes, long time no response | 14:52 |
| ralonsoh | the implementation is not trivial, as with iptables | 14:52 |
| liuyulong | [4] https://opendev.org/x/networking-ovs-dpdk/src/branch/stable/rocky/networking_ovs_dpdk/agent/ovs_dpdk_firewall.py | 14:52 |
| ralonsoh | I know this FW very well | 14:52 |
| ralonsoh | I implemented it | 14:52 |
| slaweq | https://review.opendev.org/c/openstack/neutron/+/804807 is for stateless FIPs, not SG | 14:52 |
| liuyulong | ralonsoh, this is mentioned in the bug 1885261. And I noticed it was implemented by you. | 14:53 |
| liuyulong | So it should be fine to move to neutron? | 14:53 |
| slaweq | do You want to have something similar for ML2/OVS case? How exactly? | 14:53 |
| ralonsoh | I don't think this is working at all as is right now | 14:53 |
| liuyulong | ralonsoh, yes, I tested it. It's not working. | 14:54 |
| ralonsoh | I tried, when I implemented it, to merge it in Neutron | 14:54 |
| lajoskatona | networking_ovs_dpdk is mostly abandoned, am I right? | 14:54 |
| ralonsoh | but the answer was no because we had the CT one | 14:54 |
| ralonsoh | lajoskatona, yes | 14:54 |
| liuyulong | But, mainly code is fine, I'm going to fix it. | 14:54 |
| ralonsoh | it is not needed anymore | 14:54 |
| ralonsoh | liuyulong, perfect. Now the question is if we can handle two FWs in Neutron code | 14:55 |
| ralonsoh | (and maintenance and testing and etc.) | 14:55 |
| liuyulong | SmartNIC offloading is general trend. IMO, we should. | 14:55 |
| lajoskatona | For this we need dpdk? | 14:55 |
| opendevreview | Eduardo Olivares proposed openstack/networking-ovn stable/train: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/networking-ovn/+/811430 | 14:55 |
| ralonsoh | lajoskatona, not anymore | 14:56 |
| lajoskatona | ralonsoh: thanks | 14:56 |
| ralonsoh | CT is now supported in dpdk | 14:56 |
| liuyulong | stateless fw is not only for ovs-dpdk, but also for smartnic offload | 14:56 |
| liuyulong | ralonsoh, it is still utilize the kernel CT tables? | 14:57 |
| ralonsoh | no, CT is implemented locally | 14:57 |
| liuyulong | ralonsoh, if so, the performance should not be good engough. | 14:57 |
| ralonsoh | it is now | 14:57 |
| ralonsoh | ok, I'm not against it | 14:58 |
| ralonsoh | but we should mark it as a testing feature | 14:58 |
| lajoskatona | Do You think that we have to continue this discussion on drivers meeting? | 14:58 |
| ralonsoh | and code should be isolated, as the other FW | 14:58 |
| ralonsoh | perfect | 14:58 |
| ralonsoh | that's the place | 14:58 |
| liuyulong | I will take a deep look at the CT in dpdk. and the support status of ovs-dpdk. | 14:58 |
| lajoskatona | I will add then this to the Friday agenda, is that ok liuyulong ? | 14:59 |
| liuyulong | ralonsoh, it will be configurable, a new fw driver will be added. | 14:59 |
| amotoki | lajoskatona: I think it is a good idea. even if we have no RFEs to discuss we can use the meeting time to discuss such improvements. | 14:59 |
| lajoskatona | amotoki: + | 14:59 |
| lajoskatona | +1 | 14:59 |
| liuyulong | I think it is approved. : ) | 14:59 |
| lajoskatona | ok, time is up, thanks for the discussion | 15:00 |
| slaweq | thx | 15:00 |
| liuyulong | OK, thanks | 15:00 |
| lajoskatona | liuyulong: yeah but to have everybody interested in the "room" and discussu in details | 15:00 |
| lajoskatona | Have a great week, see You later | 15:00 |
| lajoskatona | #endmeeting | 15:00 |
| opendevmeet | Meeting ended Tue Sep 28 15:00:58 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:00 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/networking/2021/networking.2021-09-28-14.00.html | 15:00 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/networking/2021/networking.2021-09-28-14.00.txt | 15:00 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/networking/2021/networking.2021-09-28-14.00.log.html | 15:00 |
| slaweq | #startmeeting neutron_ci | 15:01 |
| opendevmeet | Meeting started Tue Sep 28 15:01:03 2021 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:01 |
| opendevmeet | The meeting name has been set to 'neutron_ci' | 15:01 |
| lajoskatona | Hi | 15:01 |
| ralonsoh | hello again | 15:01 |
| slaweq | welcome again :) | 15:01 |
| bcafarel | no break this time :) | 15:01 |
| slaweq | Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:01 |
| lajoskatona | sorry for the long meeting | 15:01 |
| slaweq | bcafarel: no mercy :P | 15:01 |
| slaweq | we don't have any action items from last meeting, so I think we can quickly move to the next topics today | 15:01 |
| slaweq | #topic Stadium projects | 15:02 |
| lajoskatona | nothing from me, as I checked las time all is quiet | 15:03 |
| slaweq | great | 15:03 |
| slaweq | I also didn't noticed anything really bad there recently | 15:04 |
| slaweq | #topic Stable branches | 15:04 |
| bcafarel | nothing special either (except that placement issue in xena of course) | 15:04 |
| slaweq | yeah, that one is also in master | 15:04 |
| slaweq | bcafarel: You told me earlier about https://review.opendev.org/c/openstack/neutron-vpnaas/+/805969 | 15:05 |
| bcafarel | ah yes thanks for reminder | 15:05 |
| slaweq | :) | 15:05 |
| slaweq | yw | 15:05 |
| bcafarel | as we are EOL'ing the earlier branches, I am stating to think we can just drop tempest in this train (EM branch) | 15:06 |
| bcafarel | I am a bit out of ideas on why it does not find the tempest plugin (ralonsoh tried too) and at least train will have some working CI | 15:07 |
| slaweq | bcafarel: because in train those tests were still in neutron-vpnaas repo | 15:07 |
| slaweq | not in tempest_plugin | 15:07 |
| slaweq | so the job used there is wrong probably | 15:08 |
| bcafarel | slaweq: ah sorry bad wording, config does point to the tempest dir in neutron-vpnaas repo (and devstack log shows it configured in tempest as far as I can see) | 15:08 |
| bcafarel | but opinions/ideas welcome (sorry I have to run, have a nice rest of meeting) | 15:09 |
| lajoskatona | in that case isn't the problem is that neutron-tempest-plugin is cloned as well and tempest finds the tests in 2 paths? | 15:09 |
| slaweq | but in https://review.opendev.org/c/openstack/neutron-vpnaas/+/805969/11/.zuul.yaml#31 I see You defined tempest regex neutron_tempest_plugin.vpnaas | 15:09 |
| slaweq | which is wrong in the train | 15:09 |
| slaweq | why this change is made there? | 15:10 |
| ralonsoh | last change is mine, just for testing | 15:10 |
| ralonsoh | focus on PS10 | 15:10 |
| slaweq | ah, ok | 15:10 |
| slaweq | I will check that this week | 15:11 |
| ralonsoh | thanks | 15:11 |
| slaweq | maybe I will figure out something :) | 15:11 |
| slaweq | #action slaweq to check vpnaas stable/train patch https://review.opendev.org/c/openstack/neutron-vpnaas/+/805969/10 | 15:11 |
| slaweq | is that all regarding stable branches' ci for today? | 15:12 |
| slaweq | if so, I think we can move on | 15:12 |
| slaweq | ok, let's move on :) | 15:14 |
| slaweq | #topic Grafana | 15:14 |
| slaweq | #link http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:14 |
| slaweq | we can see that all our tempest jobs are going to 100% of failures today | 15:14 |
| slaweq | and it's know issue with the apache and placement | 15:15 |
| slaweq | basically all devstack based jobs (not only tempest) are affected by this | 15:15 |
| * slaweq wonders why things like that happens always in the RC weeks :/ | 15:16 | |
| ralonsoh | karma | 15:16 |
| slaweq | :D | 15:16 |
| lajoskatona | :-) | 15:17 |
| slaweq | other than that I don't see anything critical in the Grafana | 15:17 |
| slaweq | lets move on to the specific jobs' issues | 15:18 |
| slaweq | #topic fullstack/functional | 15:18 |
| slaweq | I found two times the same issue in functional tests: | 15:19 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: Execute the quota reservation removal in an isolated DB txn https://review.opendev.org/c/openstack/neutron/+/809983 | 15:19 |
| slaweq | https://450622a53297d10bce29-d3fc18d05d3e89166397d52e51106961.ssl.cf2.rackcdn.com/805637/8/check/neutron-functional-with-uwsgi/517e0d3/testr_results.html | 15:19 |
| slaweq | https://195cae8c9d2de3edf3c4-c1cb7fb0c5617ff6ee09319b2539cf1a.ssl.cf5.rackcdn.com/803268/8/check/neutron-functional-with-uwsgi/8d2d4d9/testr_results.html | 15:19 |
| slaweq | it's in different tests but seems like may be the same problem | 15:19 |
| ralonsoh | the keepalived state change process, maybe | 15:19 |
| ralonsoh | I could take a look at those errors, but maybe at the end of the week | 15:20 |
| slaweq | thx ralonsoh | 15:20 |
| slaweq | it's not very urgent for sure as it don't happens very often | 15:21 |
| slaweq | #action ralonsoh to check functional tests issue with router's state transition | 15:21 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron stable/xena: Execute the quota reservation removal in an isolated DB txn https://review.opendev.org/c/openstack/neutron/+/811124 | 15:22 |
| slaweq | ok, next one | 15:22 |
| slaweq | #topic Tempest/Scenario | 15:22 |
| slaweq | I reported new bug today https://bugs.launchpad.net/neutron/+bug/1945283 | 15:22 |
| slaweq | I saw this test fails at least twice this week | 15:22 |
| slaweq | but I also think I saw it earlier as well | 15:22 |
| slaweq | I will try to look at it if I will have some time | 15:23 |
| slaweq | but I can't promise I will do it this week | 15:23 |
| slaweq | so it's not assigned to me for now | 15:23 |
| ralonsoh | we can ping Eran | 15:23 |
| slaweq | if anyone wants to check it, feel free to take | 15:23 |
| ralonsoh | (if I'm not wrong, he implemented those tests) | 15:24 |
| slaweq | ralonsoh: thx, I will check and talk with him about it | 15:24 |
| slaweq | lajoskatona: You also wanted to talk about something related to the tempest jobs, right? | 15:25 |
| lajoskatona | yes, thanks | 15:25 |
| lajoskatona | it's about the api_extensions list in tempest.conf | 15:25 |
| * gibi lurks | 15:26 | |
| lajoskatona | it seems that earlier it was filled with a list, and now on master, xena, wallaby (citation needed) we have "all" | 15:26 |
| slaweq | do You have job's example? | 15:26 |
| lajoskatona | interestingly for example on master I found that for ovn jobs we have list (https://7ab77c8a0a641a7ff2df-ecc33ccaab89feb6b0f7a7737c56ac51.ssl.cf5.rackcdn.com/807707/4/check/neutron-ovn-tempest-slow/a706584/controller/logs/tempest_conf.txt ) | 15:27 |
| lajoskatona | and for ovs we have all: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d36/807707/4/check/neutron-ovs-tempest-slow/d36b75e/controller/logs/tempest_conf.txt | 15:27 |
| lajoskatona | ok I copied the good links | 15:28 |
| lajoskatona | I thought that the list is added from zuul.yaml, like this one: https://opendev.org/openstack/neutron-tempest-plugin/src/branch/master/zuul.d/master_jobs.yaml#L8 | 15:29 |
| gibi | the list -> all change happend during the weekend somehow | 15:30 |
| slaweq | hmm | 15:30 |
| gibi | and it make nova-grenade-multinode running with 'all' now but that job was never configured to run trunk tests | 15:30 |
| gibi | but 'all' means tempest wants to run trunk as well | 15:30 |
| lajoskatona | Yeah, and for nova grenade job I suppose the list is not from Neutron zuul.yaml-s | 15:31 |
| lajoskatona | I tried to fetch some repos which I thought might have some change related to this but without success | 15:31 |
| gibi | the 'all' is coming from devstack. when $NETWORK_API_EXTENSIONS is not defined it gots defaulted to 'all' | 15:31 |
| gibi | but yeah I stuck there with lajoskatona | 15:32 |
| lajoskatona | yeah but that was not changed recently, so I suppose there was something in env previously | 15:32 |
| slaweq | I think I know where it's configured for ovn jobs | 15:33 |
| slaweq | give me a sec | 15:33 |
| opendevreview | Elvira García Ruiz proposed openstack/neutron master: Fix misleading FixedIpsSubnetsNotOnSameSegment error https://review.opendev.org/c/openstack/neutron/+/811435 | 15:33 |
| ralonsoh | well, for OVN, the supported extensions are defined in a variable | 15:34 |
| ralonsoh | ML2_SUPPORTED_API_EXTENSIONS | 15:34 |
| opendevreview | Elvira García Ruiz proposed openstack/neutron master: Fix misleading FixedIpsSubnetsNotOnSameSegment error https://review.opendev.org/c/openstack/neutron/+/811435 | 15:35 |
| slaweq | ralonsoh: yes, that was my though as well | 15:35 |
| slaweq | but I don't see it used in the devstack anywhere :/ | 15:36 |
| ralonsoh | btw, what CI job backend is failing? | 15:36 |
| ralonsoh | OVS or OVS? | 15:36 |
| slaweq | https://opendev.org/openstack/devstack/src/branch/master/lib/neutron_plugins/ovn_agent#L421 | 15:36 |
| slaweq | here it is | 15:36 |
| ralonsoh | right | 15:36 |
| ralonsoh | so this is statically defined in the Neutron code | 15:36 |
| slaweq | for ML2 yes | 15:37 |
| slaweq | *ML2/OVN | 15:37 |
| lajoskatona | gibi: grenade is running with ovs? | 15:37 |
| slaweq | maybe we should do something similar for other backends as well? | 15:37 |
| gibi | yes I think so | 15:37 |
| gibi | slaweq: but how does this got broken just now? | 15:38 |
| ralonsoh | we didn't release any new API for the last weeks | 15:38 |
| gibi | ralonsoh: yeah I know, so this is really strange | 15:39 |
| slaweq | gibi: idk | 15:39 |
| gibi | lajoskatona: yepp I confirm now grenade-base job has Q_AGENT: openvswitch defined | 15:40 |
| gibi | one thing nova can do is to try to configure the grenade job to be able to run the trunk test successfully that would unblock us without explaning why this happened | 15:41 |
| slaweq | gibi: lajoskatona so the easiest solution for now would be to define that list, even statically for the jobs where it is broken | 15:41 |
| opendevreview | Terry Wilson proposed openstack/neutron master: Support SB OVSDB connections to non-leader servers https://review.opendev.org/c/openstack/neutron/+/803268 | 15:42 |
| slaweq | gibi: yeah, that is an option too - I think that it should be enough to add neutron-trunk: true in the job definition | 15:43 |
| slaweq | like https://github.com/openstack/neutron-tempest-plugin/blob/master/zuul.d/master_jobs.yaml#L554 | 15:43 |
| gibi | slaweq: yep lyarwood trying that | 15:43 |
| slaweq | and that would enable this service plugin so that API will be available | 15:43 |
| gibi | adding neutron-trunk | 15:43 |
| slaweq | gibi: and I will try to understand how it was earlier in the ovs based jobs | 15:44 |
| gibi | slaweq, lajoskatona, ralonsoh: thanks I think we can move on. If you have ideas later let me know | 15:44 |
| slaweq | #action slaweq to check api extensions list in ovs based jobs, how it was generated | 15:44 |
| ralonsoh | checking the q-svc logs, I see there are differences during the extension load process | 15:44 |
| ralonsoh | I'll take a look after the meeting | 15:44 |
| slaweq | ralonsoh: ok, please let me know if You will find anything | 15:45 |
| lajoskatona | slaweq, ralonsoh: thanks | 15:45 |
| slaweq | ralonsoh: lajoskatona gibi could it be that grenade job started testing from Xena to master now and in Xena we have https://review.opendev.org/c/openstack/neutron/+/793141 | 15:46 |
| gibi | slaweq: that feels related | 15:47 |
| slaweq | which basically changes the way to calculate what extensions should be really loaded | 15:47 |
| lajoskatona | slaweq: I checked that yesterday but had no proof | 15:47 |
| slaweq | ok | 15:47 |
| slaweq | thx lajoskatona | 15:47 |
| gibi | slaweq: during the weekend grenade was switched from testing wallaby -> master to xena -> master | 15:47 |
| slaweq | gibi: ok, thx, that's something to check for sure | 15:48 |
| gibi | slaweq: yepp | 15:48 |
| gibi | good point | 15:48 |
| gibi | thanks | 15:48 |
| slaweq | I will try to investigate it | 15:48 |
| slaweq | any other topics related to the tempest jobs? | 15:49 |
| lajoskatona | not from me, thanks for the time | 15:49 |
| ralonsoh | no | 15:49 |
| slaweq | ok, let's move on | 15:49 |
| slaweq | #topic Periodic | 15:49 |
| slaweq | I noticed that openstack-tox-py36-with-neutron-lib-master is failing since couple of days at least | 15:50 |
| slaweq | so I reported bug https://bugs.launchpad.net/neutron/+bug/1945285 | 15:50 |
| slaweq | and it's already fixed | 15:50 |
| slaweq | thx lajoskatona for quick fix | 15:50 |
| slaweq | :) | 15:50 |
| slaweq | except that one issue with periodic jobs, others seems to be ok | 15:51 |
| slaweq | fedora based job is failing again but it seems it's related to that placement/apache issue | 15:51 |
| ralonsoh | well, there were two errors in this job | 15:52 |
| ralonsoh | I solved one | 15:52 |
| ralonsoh | one sec | 15:52 |
| slaweq | https://9706dd0df42819547e88-7c45ade9796b4a439c7ceea8af4ef1aa.ssl.cf1.rackcdn.com/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-ovs-master-fedora/107c6b8/job-output.txt | 15:52 |
| slaweq | now I see that it's failing as placement seems to eb not running | 15:52 |
| ralonsoh | yes but before that | 15:52 |
| ralonsoh | well, I don't find now the bug | 15:53 |
| slaweq | I remember You had fixed some issue few weeks ago | 15:53 |
| slaweq | :) | 15:53 |
| slaweq | let's now wait for fix for the current problem with placement and we will see how it will be then | 15:53 |
| ralonsoh | https://review.opendev.org/c/openstack/tempest/+/808081 | 15:54 |
| ralonsoh | this is 50% of the solution | 15:54 |
| ralonsoh | as commented in https://launchpad.net/bugs/1942913 | 15:54 |
| ralonsoh | tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops is still failing | 15:54 |
| ralonsoh | (regardless of the placement problems right now) | 15:55 |
| ralonsoh | and I didn't find the problem | 15:55 |
| slaweq | ralonsoh: ok, thx | 15:55 |
| slaweq | so when placement problem will be solved we can blacklist that one test which is broken | 15:55 |
| ralonsoh | sure | 15:55 |
| slaweq | to have job green and then investigate this issue | 15:55 |
| slaweq | thx for taking care of it so far | 15:56 |
| slaweq | lets get back to it when placement will be fine | 15:56 |
| slaweq | and that's all from me for today | 15:56 |
| slaweq | any last minute topics? | 15:56 |
| ralonsoh | yes | 15:56 |
| ralonsoh | https://review.opendev.org/c/openstack/neutron/+/809983 | 15:57 |
| ralonsoh | I've refactored this patch | 15:57 |
| ralonsoh | in master and I've pushed the xena patch | 15:57 |
| ralonsoh | based on this one | 15:57 |
| ralonsoh | "this is the way" | 15:57 |
| slaweq | thx | 15:57 |
| slaweq | I will review it tonight or tomorrow morning | 15:57 |
| ralonsoh | thanks! | 15:58 |
| lajoskatona | Will do the same, thanks ralonsoh | 15:58 |
| slaweq | ok, I have to finish now as I have another meeting in a minute | 15:58 |
| slaweq | thx a lot for attending guys | 15:58 |
| slaweq | and have a great week | 15:58 |
| ralonsoh | you too | 15:58 |
| slaweq | #endmeeting | 15:58 |
| opendevmeet | Meeting ended Tue Sep 28 15:58:40 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:58 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-09-28-15.01.html | 15:58 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-09-28-15.01.txt | 15:58 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-09-28-15.01.log.html | 15:58 |
| lajoskatona | bye | 15:59 |
| sdanni | Hi all! I'm using a system scoped admin that doesn't belong to any project to run 'openstack port create' command and it returns error 'BadRequestException: 400: ... Running without keystone AuthN requires that tenant_id is specified.' I wonder if this is an issue resulting from secure rbac? | 16:02 |
| ralonsoh | it is yes, you need to specify a project | 16:03 |
| ralonsoh | but please, check that wil slaweq (if he's around) | 16:03 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: Check quota limits https://review.opendev.org/c/openstack/neutron/+/801470 | 16:04 |
| sdanni | ralonsoh: thx. slaweq: so basically, if i want to create a port, i have to specify a project? | 16:08 |
| gibi | ralonsoh, slaweq_ , lajoskatona: gmann knows the solution for the 'all' issues in stable/wallaby there is https://github.com/openstack/devstack/blob/stable/wallaby/lib/tempest#L669-L685 but it is not in stable/xena | 17:03 |
| lajoskatona | gibi: great news | 17:05 |
| ralonsoh | gibi, that's perfect. Is he going to push the patch? | 17:06 |
| ralonsoh | for stable/xena | 17:06 |
| gibi | ralonsoh: yes | 17:06 |
| ralonsoh | perfect | 17:06 |
| gibi | ralonsoh: after the placement apache fix is landed | 17:06 |
| ralonsoh | right | 17:06 |
| gibi | it is part of the stable/xena and grenade switchover process | 17:07 |
| gmann | ralonsoh: lajoskatona gibi if I understand correctly for trunk things tested we need 1. neutron-trunk service enable and 2. 'trunk' neutron extension also enabled. | 17:13 |
| ralonsoh | yes | 17:13 |
| gmann | in that case it seems we are not testing it in stable as we do not have 'trunk' extension enabled there so neutron-grenade-multinode pass because it skip test :) | 17:16 |
| gmann | https://zuul.opendev.org/t/openstack/build/f568752e9059402a82623a64ecdcd175/log/job-output.txt#61223 | 17:16 |
| gmann | so we can go with fix grenade + enable extension on stable on all stable | 17:17 |
| gmann | I am not sure why we miss this particular extension to enable | 17:17 |
| gmann | ah, we added the test in Xena itself that explain I2d75fae81145b4bd1c0d38fabd785bc26835be15 | 17:19 |
| lajoskatona | gmann: thanks for checking | 17:21 |
| *** slaweq_ is now known as slaweq | 17:22 | |
| opendevreview | Pedro Henrique Pereira Martins proposed openstack/neutron-lib master: Add default values in port forwarding port ranges definitions https://review.opendev.org/c/openstack/neutron-lib/+/811456 | 17:28 |
| jpic | hi all, i'm having this issue https://bugs.launchpad.net/neutron/+bug/1935983 where neutron loops-log the exception where 'NoneType' object has no attribute 'chassis_exists' | 17:44 |
| jpic | where NoneType refers to self._sb_ovn so I tried manually adding this patch: https://github.com/openstack/neutron/commit/474f5d700b6652426f93186890702c1608fa0fdc but then it won't start, complaining that ml2_geneve table does'n exist | 17:45 |
| jpic | I don't understand how this change can cause this missing table | 17:45 |
| opendevreview | Hang Yang proposed openstack/neutron master: Add shared field to SG API response and filter https://review.opendev.org/c/openstack/neutron/+/811242 | 19:24 |
| opendevreview | Merged openstack/ovsdbapp master: tools: fix OvsOvnVenvFixture init https://review.opendev.org/c/openstack/ovsdbapp/+/809150 | 20:37 |
| *** elodilles is now known as elodilles_pto | 20:52 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!