| *** hoonetorg has quit IRC | 00:04 | |
| *** yamamoto has joined #openstack-neutron | 00:10 | |
| *** dsneddon has quit IRC | 00:12 | |
| *** dsneddon has joined #openstack-neutron | 00:14 | |
| *** hoonetorg has joined #openstack-neutron | 00:17 | |
| *** hoonetorg has quit IRC | 00:36 | |
| *** also_stingrayza has joined #openstack-neutron | 01:23 | |
| *** stingrayza has quit IRC | 01:25 | |
| *** spatel has joined #openstack-neutron | 01:55 | |
| *** spatel has quit IRC | 02:34 | |
| *** spatel has joined #openstack-neutron | 02:51 | |
| *** psahoo has joined #openstack-neutron | 02:53 | |
| *** dsneddon has quit IRC | 03:08 | |
| *** dsneddon has joined #openstack-neutron | 03:11 | |
| *** dsneddon has quit IRC | 03:17 | |
| *** dsneddon has joined #openstack-neutron | 03:22 | |
| *** dsneddon has quit IRC | 03:27 | |
| *** dsneddon has joined #openstack-neutron | 03:28 | |
| *** dsneddon has quit IRC | 03:34 | |
| *** dsneddon has joined #openstack-neutron | 03:44 | |
| *** sapd1 has quit IRC | 03:54 | |
| *** iurygregory has quit IRC | 03:54 | |
| *** TheJulia has quit IRC | 03:54 | |
| *** sapd1_x has joined #openstack-neutron | 03:54 | |
| *** mbuil has quit IRC | 03:54 | |
| *** mbuil has joined #openstack-neutron | 03:55 | |
| *** TheJulia has joined #openstack-neutron | 03:55 | |
| *** sapd__x has joined #openstack-neutron | 03:57 | |
| *** ratailor has joined #openstack-neutron | 04:20 | |
| *** dsneddon has quit IRC | 04:22 | |
| *** lajoskatona has joined #openstack-neutron | 04:40 | |
| openstackgerrit | Terry Wilson proposed openstack/neutron master: Clean up some of the OVN agent API methods https://review.opendev.org/737106 | 04:44 |
|---|---|---|
| *** dsneddon has joined #openstack-neutron | 04:52 | |
| *** abdysn has joined #openstack-neutron | 04:54 | |
| *** ociuhandu has joined #openstack-neutron | 05:01 | |
| *** ociuhandu has quit IRC | 05:05 | |
| *** spatel has quit IRC | 05:09 | |
| *** waleedm has joined #openstack-neutron | 05:10 | |
| *** gcheresh has joined #openstack-neutron | 05:42 | |
| *** sapd__x has quit IRC | 05:59 | |
| *** dsneddon has quit IRC | 06:07 | |
| *** dsneddon has joined #openstack-neutron | 06:08 | |
| *** elod is now known as elod_off | 06:15 | |
| *** dsneddon has quit IRC | 06:18 | |
| *** hoonetorg has joined #openstack-neutron | 06:19 | |
| *** dsneddon has joined #openstack-neutron | 06:25 | |
| *** halali_ has quit IRC | 06:30 | |
| *** dsneddon has quit IRC | 06:37 | |
| *** boyvinall has joined #openstack-neutron | 06:46 | |
| *** halali_ has joined #openstack-neutron | 06:51 | |
| *** dsneddon has joined #openstack-neutron | 06:57 | |
| *** boyvinall has quit IRC | 06:59 | |
| *** rcernin has quit IRC | 07:05 | |
| *** waleedm_ has joined #openstack-neutron | 07:10 | |
| *** maciejjozefczyk has joined #openstack-neutron | 07:10 | |
| *** iurygregory_ has joined #openstack-neutron | 07:10 | |
| *** bengates has joined #openstack-neutron | 07:23 | |
| openstackgerrit | Lajos Katona proposed openstack/networking-odl master: Fix pep8 job https://review.opendev.org/739997 | 07:26 |
| *** bengates has quit IRC | 07:27 | |
| *** bengates has joined #openstack-neutron | 07:28 | |
| *** iurygregory_ is now known as iurygregory | 07:31 | |
| *** ralonsoh has joined #openstack-neutron | 07:31 | |
| *** halali_ has quit IRC | 07:34 | |
| *** yamamoto has quit IRC | 07:38 | |
| *** yamamoto has joined #openstack-neutron | 07:40 | |
| openstackgerrit | Lajos Katona proposed openstack/networking-odl stable/ussuri: Fix pep8 job https://review.opendev.org/740284 | 07:41 |
| *** sapd__x has joined #openstack-neutron | 07:55 | |
| *** dtantsur|afk is now known as dtantsur | 07:56 | |
| openstackgerrit | Maciej Józefczyk proposed openstack/neutron master: [DNM] Test OVN multinode with release branch https://review.opendev.org/740491 | 07:56 |
| openstackgerrit | Maciej Józefczyk proposed openstack/neutron stable/ussuri: [OVN] Wait for WaitForDataPathBindingCreateEvent event in functional tests https://review.opendev.org/738865 | 07:58 |
| openstackgerrit | Maciej Józefczyk proposed openstack/neutron stable/ussuri: [OVN] Unify OVN/OVS compilation https://review.opendev.org/740255 | 07:58 |
| openstackgerrit | Maciej Józefczyk proposed openstack/neutron stable/ussuri: Stop installing Octavia in OVN jobs https://review.opendev.org/740279 | 07:58 |
| openstackgerrit | Maciej Józefczyk proposed openstack/neutron stable/ussuri: [OVN] Stop using neutron_tempest_plugin in OVN singlenode job https://review.opendev.org/740281 | 07:59 |
| *** Habib-A has joined #openstack-neutron | 08:01 | |
| Habib-A | Hello all please help :D https://ask.openstack.org/en/question/128468/windows-server-2019-with-ovs-and-nova-compute/ | 08:02 |
| *** jlibosva has joined #openstack-neutron | 08:04 | |
| *** lucasagomes has joined #openstack-neutron | 08:08 | |
| *** rcernin has joined #openstack-neutron | 08:10 | |
| *** rcernin has quit IRC | 08:14 | |
| openstackgerrit | Li YaJie proposed openstack/neutron master: Fix delete-accepted-egress-direct-flow issue on deleting VM https://review.opendev.org/731446 | 08:14 |
| *** rcernin has joined #openstack-neutron | 08:15 | |
| ralonsoh | bcafarel, hi! | 08:20 |
| ralonsoh | do you remember if the team meeting is canceled? | 08:20 |
| frickler | ralonsoh: according to slaweq's mail no, mlavalle is to take over | 08:20 |
| ralonsoh | ahhh ok, yes | 08:21 |
| ralonsoh | thanks! | 08:21 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/neutron master: [OVN] Bump OVN version to v20.06.1 https://review.opendev.org/735802 | 08:38 |
| *** rcernin has quit IRC | 08:42 | |
| *** boyvinall has joined #openstack-neutron | 08:44 | |
| *** ociuhandu has joined #openstack-neutron | 08:50 | |
| *** donnyd has quit IRC | 08:53 | |
| *** donnyd has joined #openstack-neutron | 08:53 | |
| *** boyvinall has quit IRC | 08:54 | |
| *** also_stingrayza is now known as stingrayza | 08:58 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/neutron master: String to byte conversion should provide the encoding type https://review.opendev.org/740693 | 09:01 |
| openstackgerrit | vinay harsha mitta proposed openstack/neutron-lib master: Neutron agent-list API added with sort_key keyword https://review.opendev.org/738471 | 09:11 |
| openstackgerrit | vinay harsha mitta proposed openstack/neutron-lib master: Neutron agent-list API added with sort_key keyword https://review.opendev.org/738471 | 09:15 |
| *** gcheresh has quit IRC | 09:23 | |
| *** gcheresh has joined #openstack-neutron | 09:24 | |
| *** jangutter_ has joined #openstack-neutron | 09:28 | |
| openstackgerrit | Darragh O'Reilly proposed openstack/neutron master: Ensure drop flows on br-int at agent startup for DVR too https://review.opendev.org/740724 | 09:30 |
| *** jangutter has quit IRC | 09:31 | |
| *** frickler is now known as frickler_pto | 09:44 | |
| *** frickler_pto is now known as frickler | 09:47 | |
| *** dtantsur is now known as dtantsur|bbl | 09:49 | |
| *** tkajinam has quit IRC | 10:12 | |
| *** Habib-A has quit IRC | 10:15 | |
| *** yamamoto has quit IRC | 10:16 | |
| *** dsneddon has quit IRC | 10:19 | |
| *** yamamoto has joined #openstack-neutron | 10:22 | |
| *** gouthamr has quit IRC | 10:22 | |
| *** gouthamr has joined #openstack-neutron | 10:23 | |
| *** dsneddon has joined #openstack-neutron | 10:23 | |
| *** Jeffrey4l has quit IRC | 10:23 | |
| *** Jeffrey4l has joined #openstack-neutron | 10:24 | |
| *** yamamoto has quit IRC | 10:26 | |
| *** jangutter has joined #openstack-neutron | 10:34 | |
| *** dsneddon has quit IRC | 10:35 | |
| *** dsneddon has joined #openstack-neutron | 10:36 | |
| *** jangutter_ has quit IRC | 10:37 | |
| *** Kaftantzis has joined #openstack-neutron | 10:46 | |
| openstackgerrit | Maciej Józefczyk proposed openstack/neutron master: [DNM] Test OVN multinode with release branch https://review.opendev.org/740491 | 10:51 |
| *** dsneddon has quit IRC | 10:53 | |
| *** dsneddon has joined #openstack-neutron | 10:56 | |
| *** Kaftantzis has quit IRC | 11:01 | |
| *** yamamoto has joined #openstack-neutron | 11:02 | |
| *** dsneddon has quit IRC | 11:06 | |
| *** dsneddon has joined #openstack-neutron | 11:07 | |
| *** yamamoto has quit IRC | 11:08 | |
| *** rcernin has joined #openstack-neutron | 11:13 | |
| *** dsneddon has quit IRC | 11:13 | |
| *** dsneddon has joined #openstack-neutron | 11:33 | |
| *** Kaftantzis has joined #openstack-neutron | 11:39 | |
| *** yamamoto has joined #openstack-neutron | 11:40 | |
| *** dsneddon has quit IRC | 11:45 | |
| *** iurygregory has quit IRC | 11:46 | |
| *** dsneddon has joined #openstack-neutron | 11:46 | |
| *** sapd__x has quit IRC | 11:50 | |
| *** dsneddon has quit IRC | 11:53 | |
| *** dsneddon has joined #openstack-neutron | 11:54 | |
| *** spatel has joined #openstack-neutron | 11:58 | |
| *** iurygregory has joined #openstack-neutron | 12:00 | |
| *** raildo has joined #openstack-neutron | 12:02 | |
| *** spatel has quit IRC | 12:03 | |
| openstackgerrit | Merged openstack/ovn-octavia-provider master: Fix pep8 and functional jobs https://review.opendev.org/726917 | 12:06 |
| *** yamamoto has quit IRC | 12:08 | |
| *** dsneddon has quit IRC | 12:11 | |
| *** dsneddon has joined #openstack-neutron | 12:11 | |
| *** yamamoto has joined #openstack-neutron | 12:13 | |
| *** yamamoto has quit IRC | 12:27 | |
| openstackgerrit | XiaoYu Zhu proposed openstack/neutron-specs master: L3 router support ecmp https://review.opendev.org/729532 | 12:28 |
| openstackgerrit | Maciej Józefczyk proposed openstack/ovn-octavia-provider master: Update ROUND_ROBIN to SOURCE_IP_PORT in docs https://review.opendev.org/740620 | 12:30 |
| openstackgerrit | Jakub Libosvar proposed openstack/neutron master: ovn migration: Support stack name https://review.opendev.org/740949 | 12:37 |
| *** yamamoto has joined #openstack-neutron | 12:41 | |
| *** dsneddon has quit IRC | 12:47 | |
| *** rcernin has quit IRC | 12:48 | |
| *** baojg has joined #openstack-neutron | 12:54 | |
| openstackgerrit | Flavio Fernandes proposed openstack/neutron master: [ovn]: port forwarding https://review.opendev.org/723863 | 12:54 |
| *** Luzi has joined #openstack-neutron | 12:55 | |
| mnaser | question -- is there any reason why neutron is currently using it's own retry code instead of the built in stuff inside oslo_db for retrying on deadlocks ? https://github.com/openstack/neutron-lib/blob/master/neutron_lib/db/api.py#L109-L123 | 12:57 |
| mnaser | i'm seeing some issues that were fixed inside oslo.db -- so i think it would be at least good to rely on oslo_db for deadlocks/retry request/etc for it to handle, and letting neutron retry its own cases.. | 12:58 |
| *** spatel has joined #openstack-neutron | 12:58 | |
| *** dsneddon has joined #openstack-neutron | 12:58 | |
| openstackgerrit | Bence Romsics proposed openstack/neutron master: metadata-ipv6: DHCP namespace https://review.opendev.org/715482 | 13:01 |
| openstackgerrit | Bence Romsics proposed openstack/neutron master: metadata-ipv6: Router namespace https://review.opendev.org/715483 | 13:01 |
| openstackgerrit | Bence Romsics proposed openstack/neutron master: metadata-ipv6: Accept link local address in X-Forwarded-For https://review.opendev.org/718729 | 13:02 |
| ftarasenko | lajoskatona: Hi! Can you check https://bugs.launchpad.net/networking-bagpipe/+bug/1883102 cause it fails after patch at another place. Logs are attached to the bug. Tnx | 13:02 |
| openstack | Launchpad bug 1883102 in BaGPipe "bagpipe: bagpipe-bgp does not start with EVPN and OVS driver " [Undecided,In progress] - Assigned to Lajos Katona (lajos-katona) | 13:02 |
| *** dsneddon has quit IRC | 13:08 | |
| *** dsneddon has joined #openstack-neutron | 13:10 | |
| *** yamamoto has quit IRC | 13:10 | |
| lajoskatona | ftarasenko: sure | 13:11 |
| lajoskatona | ftarasenko: I read again your feedback in launchpad, and try to allocate time for it | 13:12 |
| ftarasenko | lajoskatona: thanks! I have lab with multiple servers available for tests at any time. | 13:13 |
| *** baojg has quit IRC | 13:15 | |
| *** dsneddon has quit IRC | 13:16 | |
| *** baojg has joined #openstack-neutron | 13:16 | |
| lajoskatona | ftarasenko: ok | 13:18 |
| openstackgerrit | Merged openstack/ovn-octavia-provider master: Update ROUND_ROBIN to SOURCE_IP_PORT in docs https://review.opendev.org/740620 | 13:19 |
| openstackgerrit | Flavio Fernandes proposed openstack/neutron master: [ovn]: port forwarding -- documentation https://review.opendev.org/740955 | 13:20 |
| *** dsneddon has joined #openstack-neutron | 13:23 | |
| *** dsneddon has quit IRC | 13:28 | |
| *** Kaftantzis has quit IRC | 13:29 | |
| *** Luzi has quit IRC | 13:29 | |
| *** dsneddon has joined #openstack-neutron | 13:33 | |
| openstackgerrit | Lajos Katona proposed openstack/networking-odl stable/ussuri: Fix pep8 job https://review.opendev.org/740284 | 13:35 |
| *** dsneddon has quit IRC | 13:38 | |
| *** benj_ has quit IRC | 13:39 | |
| *** dsneddon has joined #openstack-neutron | 13:39 | |
| *** benj_ has joined #openstack-neutron | 13:41 | |
| *** Luzi has joined #openstack-neutron | 13:45 | |
| *** dtantsur|bbl is now known as dtantsur | 13:45 | |
| *** sean-k-mooney has joined #openstack-neutron | 13:46 | |
| *** waleedm has quit IRC | 13:47 | |
| *** waleedm_ has quit IRC | 13:47 | |
| frickler | mnaser: I tend to agree, likely you should create a bug report that asks for that code to be improved | 13:48 |
| mnaser | frickler: i did that, and i'm currently working on something right now to refactor to using oslo.db | 13:48 |
| mnaser | https://bugs.launchpad.net/neutron/+bug/1887523 | 13:48 |
| openstack | Launchpad bug 1887523 in neutron "Deadlock detection code can be stale" [Undecided,New] | 13:48 |
| openstackgerrit | Merged openstack/ovn-octavia-provider master: Add a hacking check for importing mock library https://review.opendev.org/734670 | 13:48 |
| *** ralonsoh has quit IRC | 13:49 | |
| mnaser | the `_tag_retriables_as_unretriable` function is really confusing me though | 13:49 |
| frickler | mnaser: great, thx, added that to my watchlist. sorry I can't help with the details, I've never done much db work, also I'm off now ;) | 13:50 |
| mnaser | frickler: same -- take care!! :) | 13:50 |
| * mnaser will slowly figure it out | 13:50 | |
| *** frickler is now known as frickler_pto | 13:50 | |
| mnaser | found this https://github.com/openstack/neutron/commit/09c87425fa028dbe669e8c215e334297ccbf1c2a | 13:52 |
| openstackgerrit | Maciej Józefczyk proposed openstack/neutron master: [OVN] Fix logic issue while deleting port with QoS https://review.opendev.org/740967 | 13:54 |
| *** mlavalle has joined #openstack-neutron | 13:58 | |
| *** dsneddon has quit IRC | 14:00 | |
| *** ralonsoh has joined #openstack-neutron | 14:00 | |
| *** sapd__x has joined #openstack-neutron | 14:00 | |
| *** dave-mccowan has joined #openstack-neutron | 14:01 | |
| *** maciejjozefczyk has quit IRC | 14:01 | |
| *** dsneddon has joined #openstack-neutron | 14:02 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/neutron master: Change some tempest tests to explicitly use ML2/OVS https://review.opendev.org/740968 | 14:02 |
| *** lucasagomes has quit IRC | 14:13 | |
| *** lucasagomes has joined #openstack-neutron | 14:14 | |
| *** armax has joined #openstack-neutron | 14:16 | |
| *** ratailor has quit IRC | 14:17 | |
| *** Luzi has quit IRC | 14:18 | |
| *** knikolla has joined #openstack-neutron | 14:21 | |
| *** yamamoto has joined #openstack-neutron | 14:28 | |
| *** jobewan has joined #openstack-neutron | 14:29 | |
| *** abdysn has quit IRC | 14:29 | |
| openstackgerrit | Mohammed Naser proposed openstack/neutron master: quota: use native oslo.db retry https://review.opendev.org/740977 | 14:32 |
| lajoskatona | mnaser, frickler: Hi, regarding https://bugs.launchpad.net/neutron/+bug/1887523 , I added a question if it can be an RFE, and discuss on next drivers meeting ( or when slaweq will be back to have full team there) | 14:33 |
| openstack | Launchpad bug 1887523 in neutron "Deadlock detection code can be stale" [Undecided,New] | 14:33 |
| *** dave-mccowan has quit IRC | 14:33 | |
| mnaser | lajoskatona: i just pushed up a patch of what this might look like -- https://review.opendev.org/740977 -- i'm going to try and test it in a pretty busy env | 14:34 |
| lajoskatona | mnaser: ok, but I am sure if you need some help or historycal background the team is ready to have discussion :-) | 14:37 |
| *** dave-mccowan has joined #openstack-neutron | 14:39 | |
| openstackgerrit | Merged openstack/neutron master: migration: Use ansible-inventory to parse tripleo inventory https://review.opendev.org/738212 | 14:40 |
| *** johnthetubaguy has quit IRC | 14:50 | |
| *** johnthetubaguy has joined #openstack-neutron | 14:52 | |
| zigo | Hi there! I'm trying to make the Ryan Tidwell's patch here: https://review.opendev.org/#/c/669395/ to work, though when creating the subnet with --service-type 'network:routed', Neutron's API reject me with a 400 error saying: | 14:54 |
| zigo | "All of the subnets on network '<NETWORK-ID>' must either all be associated with segments or all not associated with any segment." | 14:54 |
| zigo | Is it possible that, since a year ago, there was some new checks added to Neutron API so that it would reject the API call? | 14:54 |
| zigo | amotoki mlavalle do you remember, over the last year, reviewing such patch that could potentially break #669395 | 14:54 |
| zigo | ? | 14:55 |
| *** dsneddon has quit IRC | 14:56 | |
| *** dsneddon has joined #openstack-neutron | 14:57 | |
| openstackgerrit | Flavio Fernandes proposed openstack/neutron master: [ovn]: port forwarding -- documentation https://review.opendev.org/740955 | 14:59 |
| *** yamamoto has quit IRC | 15:03 | |
| ralonsoh | lajoskatona, do you need something? I saw you in the meeting channel | 15:08 |
| lajoskatona | ralonsoh: no I just went there to see what happens around qos :-) | 15:09 |
| ralonsoh | lajoskatona, ah ok | 15:09 |
| lajoskatona | ralonsoh: and say that I progress (really) slowly with https://bugs.launchpad.net/neutron/+bug/1882804 | 15:10 |
| openstack | Launchpad bug 1882804 in neutron "RFE: allow replacing the QoS policy of bound port" [Wishlist,Confirmed] - Assigned to Lajos Katona (lajos-katona) | 15:10 |
| ralonsoh | lajoskatona, do you need some help? | 15:11 |
| lajoskatona | ralonsoh: I think now, I see now that I can cover it, thanks | 15:12 |
| *** dsneddon has quit IRC | 15:14 | |
| *** dsneddon has joined #openstack-neutron | 15:15 | |
| *** armax has quit IRC | 15:18 | |
| *** hamalq has joined #openstack-neutron | 15:23 | |
| openstackgerrit | Matt Vinall proposed openstack/neutron stable/ussuri: fix bulk reload exception and race hazard https://review.opendev.org/740988 | 15:26 |
| openstackgerrit | Merged openstack/neutron master: Remove tempest-integrated-networking job from neutron queue https://review.opendev.org/736186 | 15:27 |
| *** hamalq has quit IRC | 15:29 | |
| *** dsneddon has quit IRC | 15:31 | |
| openstackgerrit | Flavio Fernandes proposed openstack/neutron master: [ovn]: port forwarding -- extend support for OVN usage (cont.) https://review.opendev.org/740955 | 15:32 |
| *** hamalq has joined #openstack-neutron | 15:35 | |
| *** baojg has quit IRC | 15:37 | |
| openstackgerrit | Flavio Fernandes proposed openstack/neutron master: [ovn]: port forwarding -- documentation https://review.opendev.org/740955 | 15:38 |
| *** baojg has joined #openstack-neutron | 15:38 | |
| openstackgerrit | Flavio Fernandes proposed openstack/neutron master: [ovn]: port forwarding -- extend support for OVN usage (cont.) https://review.opendev.org/740993 | 15:40 |
| *** clarkb has joined #openstack-neutron | 15:41 | |
| clarkb | Hi, we've noticed that logstash indexing keeps falling behind lately and digging into our logs for indexing it seems that q-svc logs in neutron jobs have grown quite large and are likely contribnuting to the problem | 15:41 |
| clarkb | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a60/733250/19/check/neutron-tempest-dvr-ha-multinode-full/a6025d9/controller/logs/screen-q-svc.txt | 15:41 |
| clarkb | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_464/735125/10/check/neutron-ovn-rally-task/4648c27/controller/logs/screen-q-svc.txt | 15:42 |
| clarkb | those are a couple of the examples I've found. Are ya'll able to take a look and see if that is expected or intentional? | 15:42 |
| clarkb | Its likely to be an operator concern eventually if we start filling disks in production too | 15:42 |
| clarkb | wc -l screen-q-svc.txt -> 792240 ; grep Time-cost screen-q-svc.txt| wc -l -> 614025 | 15:48 |
| clarkb | seems like instrumentation of function calls has been added to logging output and its exploding the size of that output | 15:48 |
| clarkb | maybe that should be off by default and only enabled when you're doing performance checking? and even then probably redirected to a separate file so that we don't try and index it all | 15:49 |
| *** dsneddon has joined #openstack-neutron | 15:50 | |
| *** bengates_ has joined #openstack-neutron | 15:53 | |
| *** spatel has quit IRC | 15:54 | |
| *** dsneddon has quit IRC | 15:55 | |
| *** bengates has quit IRC | 15:57 | |
| *** dsneddon has joined #openstack-neutron | 15:58 | |
| *** lucasagomes has quit IRC | 15:59 | |
| *** dsneddon has quit IRC | 16:05 | |
| *** bengates_ has quit IRC | 16:13 | |
| *** bengates has joined #openstack-neutron | 16:13 | |
| *** dsneddon has joined #openstack-neutron | 16:17 | |
| *** bengates has quit IRC | 16:17 | |
| *** aedc_ has joined #openstack-neutron | 16:24 | |
| *** aedc_ has quit IRC | 16:24 | |
| openstackgerrit | Andrew Bogott proposed openstack/neutron-tempest-plugin master: Add basic tests for quota access policy https://review.opendev.org/740999 | 16:27 |
| *** aedc has quit IRC | 16:27 | |
| openstackgerrit | Flavio Fernandes proposed openstack/neutron master: [ovn]: port forwarding -- IDL changes https://review.opendev.org/741000 | 16:29 |
| openstackgerrit | Flavio Fernandes proposed openstack/neutron master: [ovn]: port forwarding -- IDL changes https://review.opendev.org/741000 | 16:35 |
| *** Reepicheep has joined #openstack-neutron | 16:44 | |
| *** jlibosva has quit IRC | 16:50 | |
| *** Reepicheep has quit IRC | 16:51 | |
| *** Reepicheep has joined #openstack-neutron | 16:55 | |
| *** ociuhandu_ has joined #openstack-neutron | 16:58 | |
| *** njohnston is now known as njohnston|lunch | 16:58 | |
| *** yamamoto has joined #openstack-neutron | 17:01 | |
| *** ociuhandu has quit IRC | 17:01 | |
| *** ociuhandu_ has quit IRC | 17:02 | |
| *** lajoskatona has quit IRC | 17:27 | |
| *** dtantsur is now known as dtantsur|afk | 17:31 | |
| *** psahoo has quit IRC | 17:38 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/neutron master: Remove "six" library https://review.opendev.org/740510 | 17:38 |
| openstackgerrit | Andrew Bogott proposed openstack/neutron-tempest-plugin master: Add basic tests for quota access policy https://review.opendev.org/740999 | 17:40 |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/neutron stable/stein: [stable only] Configure logging in keepalived_state_change https://review.opendev.org/739267 | 17:40 |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/neutron master: Remove "six" library https://review.opendev.org/740510 | 17:44 |
| *** yamamoto has quit IRC | 17:53 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/neutron master: [DNM] Test oslo.privsep patch https://review.opendev.org/741017 | 17:54 |
| openstackgerrit | Mohammed Naser proposed openstack/neutron master: quota: use native oslo.db retry https://review.opendev.org/740977 | 17:55 |
| *** ralonsoh has quit IRC | 17:57 | |
| *** dsneddon has quit IRC | 17:58 | |
| openstackgerrit | Lukas Steiner proposed openstack/neutron master: Flush ebtables arp protect chains before deleting them https://review.opendev.org/740588 | 17:59 |
| clarkb | bnemec: the logging I've called out above is from oslo utils, any idea what may be causing that? | 18:06 |
| bnemec | clarkb: I don't think the log messages are coming from oslo. It looks like the source is timecost: https://opendev.org/openstack/neutron-lib/commit/492fd109ed92f15e421ebbdbd4fba5a9129eef17 | 18:11 |
| bnemec | (which is using oslo.utils for timing) | 18:12 |
| clarkb | got it | 18:12 |
| clarkb | njohnston|lunch: ^ when lunch is over can you maybe reevaluate that? | 18:12 |
| clarkb | I'm not sure that having it on all the time is useful, particularly since it is likely to fill up disks in production | 18:13 |
| *** sapd__x has quit IRC | 18:18 | |
| openstackgerrit | Mohammed Naser proposed openstack/neutron master: quota: use native oslo.db retry https://review.opendev.org/740977 | 18:27 |
| *** njohnston|lunch is now known as njohnston | 18:41 | |
| njohnston | clarkb: ack, but that is a debug level log and rarely do prod environments run with debug logs enabled | 18:43 |
| clarkb | njohnston: I've been told the opposite is true | 18:44 |
| clarkb | (everyone has to run at debug level because its the only way to undersatnd why things break) | 18:44 |
| njohnston | ok. I'll push a change to disable the logging in the one place I added it, which is here: https://opendev.org/openstack/neutron/src/branch/master/neutron/objects/ports.py#L409 | 18:47 |
| clarkb | one approach could be to make it toggelable so that you can enable it when you know you want the profiling data (this is what we do in zuul for example). Another option is to have it log to another location though python logging levels make that a little clunky (you'd have to namespace it maybe then set that namespace to log above debug level by default?) | 18:49 |
| njohnston | Yeah, I was hoping to avoid Yet Another Config Option | 18:50 |
| njohnston | but c’est la vie | 18:51 |
| clarkb | in zuul's case it uses signals so not a config option | 18:53 |
| clarkb | signal turns on the behavior then repeating the signal turns it off | 18:53 |
| clarkb | that way you can toggle it on a running system in a simple and reliable way | 18:53 |
| *** slaweq has joined #openstack-neutron | 18:54 | |
| *** slaweq has quit IRC | 18:56 | |
| *** ramishra has quit IRC | 18:58 | |
| *** ociuhandu has joined #openstack-neutron | 19:14 | |
| *** ociuhandu has quit IRC | 19:23 | |
| *** ramishra has joined #openstack-neutron | 19:25 | |
| *** yamamoto has joined #openstack-neutron | 19:51 | |
| *** also_stingrayza has joined #openstack-neutron | 20:03 | |
| *** stingrayza has quit IRC | 20:07 | |
| *** yamamoto has quit IRC | 20:51 | |
| *** gcheresh has quit IRC | 21:07 | |
| *** dsneddon has joined #openstack-neutron | 21:11 | |
| *** markvoelker has joined #openstack-neutron | 21:23 | |
| *** markvoelker has quit IRC | 21:27 | |
| *** yamamoto has joined #openstack-neutron | 21:35 | |
| openstackgerrit | Andrew Bogott proposed openstack/neutron-tempest-plugin master: Add basic tests for quota access policy https://review.opendev.org/740999 | 21:35 |
| *** spatel has joined #openstack-neutron | 21:45 | |
| *** spatel has quit IRC | 22:04 | |
| *** spatel has joined #openstack-neutron | 22:23 | |
| *** spatel has quit IRC | 22:23 | |
| *** rcernin has joined #openstack-neutron | 22:44 | |
| *** armax has joined #openstack-neutron | 22:47 | |
| *** rcernin has quit IRC | 22:51 | |
| *** rcernin has joined #openstack-neutron | 22:51 | |
| *** tkajinam has joined #openstack-neutron | 22:58 | |
| *** hamalq has quit IRC | 22:58 | |
| *** mlavalle has quit IRC | 23:01 | |
| *** yamamoto has quit IRC | 23:02 | |
| *** armax has quit IRC | 23:12 | |
| mnaser | would anyone at all know why we have an update here -- https://github.com/openstack/neutron/blob/0efff0534689b2b9256852aeff536f51dc7266e1/neutron/objects/quota.py#L145-L147 ? | 23:22 |
| mnaser | the comment "as lock mode was just for protecting dirty bits an update on dirty will prevent the race." doesn't explain much | 23:22 |
| mnaser | there's quite literally no reason for that update lock to be there | 23:25 |
| mnaser | https://review.opendev.org/#/c/442181/ | 23:26 |
| mnaser | this is the change | 23:27 |
| mnaser | ok, so pretty much, the only use is for counting usage, and this means that for the whole time we do the quota recalculation, we have a hold on the row | 23:29 |
| mnaser | i.e. https://github.com/openstack/neutron/blob/55be92e253a33c9a51ac9db3e444942920372904/neutron/quota/resource.py#L250-L294 | 23:29 |
| *** yamamoto has joined #openstack-neutron | 23:33 | |
| mnaser | https://github.com/openstack/neutron/blob/d28654a0c740844ab726d557480ebb3fd3ac66c7/neutron/db/quota/driver.py#L225-L234 | 23:33 |
| mnaser | and then we do a lot of counting here | 23:33 |
| mnaser | so if we're creating many ports at once, we have many locks on the port/project_id row waiting one after the other | 23:34 |
| clarkb | mnaser: this could be totally wrong, but I think the .first() is rlated to that. Because you may end up getting more results after first() it holds the lock open while you do things in the old code | 23:42 |
| clarkb | with the added update() it does similar but less explicitly as you might update more rows. However, it appears we only ever expect a single row so maybe you can drop the update and use .one() instead of .first()? | 23:42 |
| clarkb | unless of course that lock needs to be help for thints in count | 23:42 |
| mnaser | clarkb: well, after parsing the code some more, it seems that this update() actually is there to do a select for update to lock the row for resource counting | 23:43 |
| mnaser | but the problem here is when you're getting 250+ port creations all at once, you start to end up with a bunch of deadlocks, as some threads are waiting for the thread before them to 'finish counting | 23:43 |
| mnaser | following the stack further up, that function is called in here -- https://github.com/openstack/neutron/blob/55be92e253a33c9a51ac9db3e444942920372904/neutron/quota/resource.py#L258-L260 | 23:44 |
| mnaser | which is also then called here (indirectly via count): https://github.com/openstack/neutron/blob/d28654a0c740844ab726d557480ebb3fd3ac66c7/neutron/db/quota/driver.py#L225-L234 | 23:45 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!