| *** hamalq has joined #openstack-oslo | 00:11 | |
| *** hamalq has quit IRC | 00:15 | |
| *** hamalq has joined #openstack-oslo | 02:51 | |
| *** hamalq has quit IRC | 02:56 | |
| *** cognifloyd has quit IRC | 04:15 | |
| *** dave-mccowan has quit IRC | 04:20 | |
| *** hamalq has joined #openstack-oslo | 04:30 | |
| *** hamalq has quit IRC | 04:35 | |
| *** hamalq has joined #openstack-oslo | 04:46 | |
| *** hamalq has quit IRC | 04:51 | |
| *** yoctozepto has quit IRC | 06:15 | |
| *** yoctozepto has joined #openstack-oslo | 06:16 | |
| *** hamalq has joined #openstack-oslo | 06:27 | |
| *** brinzhang has joined #openstack-oslo | 06:29 | |
| *** brinzhang has quit IRC | 06:30 | |
| *** brinzhang has joined #openstack-oslo | 06:30 | |
| *** hamalq has quit IRC | 06:31 | |
| *** brinzhang_ has joined #openstack-oslo | 06:31 | |
| *** brinzhang_ has quit IRC | 06:33 | |
| *** brinzhang0 has joined #openstack-oslo | 06:33 | |
| *** brinzhang has quit IRC | 06:35 | |
| *** elod has joined #openstack-oslo | 07:33 | |
| *** tosky has joined #openstack-oslo | 07:47 | |
| *** hamalq has joined #openstack-oslo | 08:13 | |
| *** hamalq has quit IRC | 08:18 | |
| *** hamalq has joined #openstack-oslo | 09:03 | |
| *** hamalq has quit IRC | 09:08 | |
| *** rcernin has quit IRC | 09:24 | |
| openstackgerrit | Moisés Guimarães proposed openstack/oslo.cache master: Bump dogpile.cache's version for Memcached TLS support https://review.opendev.org/745509 | 09:39 |
|---|---|---|
| *** moguimar has joined #openstack-oslo | 09:59 | |
| *** hamalq has joined #openstack-oslo | 10:11 | |
| *** hamalq_ has joined #openstack-oslo | 10:12 | |
| *** hamalq has quit IRC | 10:15 | |
| *** hamalq_ has quit IRC | 10:17 | |
| *** kevko has joined #openstack-oslo | 11:21 | |
| kevko | hi guys , what about my review ? https://review.opendev.org/#/c/742193/ it was already discussed some time ago, what is decision ? | 11:21 |
| *** raildo has joined #openstack-oslo | 11:49 | |
| *** brinzhang_ has joined #openstack-oslo | 12:02 | |
| *** brinzhang0 has quit IRC | 12:05 | |
| moguimar | hi kevko, thanks for the reminder, I'll bring it to light today during the team meeting | 12:38 |
| moguimar | unfortunately hberaud is on PTO til next week IIRC | 12:39 |
| moguimar | oh boy, is bnemec-pto on PTO still? | 12:40 |
| *** cognifloyd has joined #openstack-oslo | 12:45 | |
| *** lbragstad has quit IRC | 12:46 | |
| *** dave-mccowan has joined #openstack-oslo | 12:47 | |
| *** kevko_ has joined #openstack-oslo | 12:48 | |
| *** kevko has quit IRC | 12:49 | |
| *** Luzi has joined #openstack-oslo | 12:55 | |
| *** dave-mccowan has quit IRC | 13:30 | |
| *** Luzi has quit IRC | 13:34 | |
| *** vishakha has joined #openstack-oslo | 13:35 | |
| *** lbragstad has joined #openstack-oslo | 13:45 | |
| *** bnemec-pto is now known as bnemec | 13:54 | |
| *** kgiusti has joined #openstack-oslo | 13:54 | |
| *** harlowja has joined #openstack-oslo | 14:52 | |
| moguimar | bnemec, meeting? | 15:02 |
| bnemec | Sorry, missed my notification. | 15:03 |
| bnemec | #startmeeting oslo | 15:03 |
| bnemec | Courtesy ping for bnemec, smcginnis, moguimar, johnsom, stephenfin, bcafarel, kgiusti, jungleboyj | 15:03 |
| bnemec | #link https://wiki.openstack.org/wiki/Meetings/Oslo#Agenda_for_Next_Meeting | 15:03 |
| openstack | Meeting started Mon Aug 10 15:03:53 2020 UTC and is due to finish in 60 minutes. The chair is bnemec. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:03 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:03 |
| *** openstack changes topic to " (Meeting topic: oslo)" | 15:03 | |
| openstack | The meeting name has been set to 'oslo' | 15:03 |
| johnsom | o/ | 15:04 |
| bnemec | #topic Red flags for/from liaisons | 15:07 |
| *** openstack changes topic to "Red flags for/from liaisons (Meeting topic: oslo)" | 15:07 | |
| johnsom | Nothing from Octavia | 15:07 |
| bnemec | I was out last week so I have no idea what's going on. Hopefully someone else can fill us in. :-) | 15:08 |
| moguimar | nothing from Barbican | 15:08 |
| moguimar | we didn't have a meeting last week | 15:09 |
| moguimar | now Hervé is also on PTO | 15:09 |
| bnemec | Might be a quick meeting then. | 15:10 |
| moguimar | and we need to come to a decision about kevko_ 's patch | 15:10 |
| bnemec | Which is okay since I have a ton of emails to get through. :-) | 15:10 |
| moguimar | this: https://review.opendev.org/#/c/742193/ | 15:10 |
| bnemec | I've added it to the agenda. | 15:11 |
| bnemec | #topic Releases | 15:12 |
| *** openstack changes topic to "Releases (Meeting topic: oslo)" | 15:12 | |
| bnemec | I'll try to take care of these this week since Herve is out. | 15:12 |
| bnemec | I guess that's all I have on this topic. | 15:13 |
| bnemec | #topic Action items from last meeting | 15:13 |
| *** openstack changes topic to "Action items from last meeting (Meeting topic: oslo)" | 15:13 | |
| bnemec | "kgiusti to retire devstack-plugin-zmq" | 15:14 |
| kgiusti | in progress | 15:14 |
| bnemec | Cool, thanks. | 15:14 |
| bnemec | "hberaud to sync oslo-cookiecutter contributing template with main cookiecutter one" | 15:14 |
| bnemec | Pretty sure I voted on this patch. | 15:15 |
| *** hamalq has joined #openstack-oslo | 15:15 | |
| bnemec | Yep. | 15:15 |
| bnemec | #link https://review.opendev.org/#/c/743939/ | 15:15 |
| bnemec | It's blocked on ci. | 15:15 |
| bnemec | Which is fixed by https://review.opendev.org/#/c/745304 | 15:16 |
| bnemec | So, all in progress, which is good. | 15:16 |
| bnemec | #topic zuulv3 migration | 15:17 |
| *** openstack changes topic to "zuulv3 migration (Meeting topic: oslo)" | 15:17 | |
| bnemec | The zmq retirement is related to this.. | 15:17 |
| bnemec | I thought I saw something about migrating grenade jobs too. | 15:17 |
| tosky | yep | 15:17 |
| tosky | line 213: https://etherpad.opendev.org/p/goal-victoria-native-zuulv3-migration | 15:18 |
| kgiusti | Yeah - that's one of the "retirement" tasks | 15:18 |
| kgiusti | https://review.opendev.org/#/q/status:open+project:openstack/project-config+branch:master+topic:retire-devstack-plugin-zmq | 15:18 |
| tosky | - the openstack/devstack-plugin-zmq jobs are covered by repository retirement | 15:18 |
| openstackgerrit | Sean McGinnis proposed openstack/oslo-cookiecutter master: sync oslo-cookiecutter contributing template https://review.opendev.org/743939 | 15:18 |
| tosky | - oslo.versionedobjects is fixed by https://review.opendev.org/745183 | 15:18 |
| tosky | - and clarkb provided patches to port the pbr jobs (https://review.opendev.org/745171, https://review.opendev.org/745189, https://review.opendev.org/745192 ) | 15:19 |
| bnemec | \o/ | 15:19 |
| bnemec | So basically we have changes in flight to address all of the remaining oslo jobs. | 15:19 |
| tosky | correct | 15:20 |
| bnemec | stephenfin: See the above about the pbr jobs. I know you had looked at that too. | 15:20 |
| bnemec | Okay, we're on track for this goal. | 15:21 |
| bnemec | Thanks for the updates! | 15:21 |
| bnemec | And all the patches! | 15:21 |
| bnemec | #topic oslo.cache flush patch | 15:21 |
| *** openstack changes topic to "oslo.cache flush patch (Meeting topic: oslo)" | 15:21 | |
| bnemec | #link https://review.opendev.org/#/c/742193/ | 15:22 |
| bnemec | moguimar: kevko_: You're up! | 15:22 |
| bnemec | cc lbragstad since he had thoughts on this too. | 15:22 |
| moguimar | I think the patch is pretty much solid | 15:22 |
| lbragstad | i need to look at it again | 15:23 |
| moguimar | but I'm concerned about Keystone expecting the default behavior to be True and we flipping it to False | 15:23 |
| bnemec | If we do go ahead with the patch we must have a way for keystone to default that back to true, IMHO. | 15:24 |
| lbragstad | imo - it seems like they need to scale up their memcached deployment | 15:24 |
| bnemec | And since Keystone is one of the main consumers of oslo.cache I'm unclear how much it will help to turn it off only other places. | 15:24 |
| lbragstad | because it appears to the root of the issue is that a network event causes memcached to spiral into an unrecoverable error | 15:25 |
| lbragstad | i need to stand up an environment with caching configured to debug the issue where you don't flush, because i'm suspicious that stale authorization data will be returned | 15:26 |
| lbragstad | (e.g., when memcached is unreachable, the user revokes their token or changes their password, but their tokens are still in memcached) | 15:26 |
| moguimar | what if the default value was True instead | 15:27 |
| bnemec | I think it's just one server going down in the pool, then the token getting revoked on a different one, then the original server coming back up that is the problem. | 15:27 |
| bnemec | IIUC it can result in a bad cached value for the server that disconnected. | 15:27 |
| openstackgerrit | Merged openstack/oslo-cookiecutter master: Add ensure-tox support. https://review.opendev.org/745304 | 15:27 |
| lbragstad | right - you could have inconsistent data across servers | 15:27 |
| lbragstad | and we don't really handle that in keystone code | 15:28 |
| * bnemec proposes that we just rm -rf memcache_pool | 15:28 | |
| lbragstad | well - that's essentially what we assume since we flush all memcached data (valid and invalid) when the client reconnects | 15:28 |
| lbragstad | (we're not sure what happened when you were gone, but rebuild the source of truth) | 15:29 |
| lbragstad | rebuild from keystone's database, which is the source of truth * | 15:29 |
| lbragstad | i need to dig into this more, but i haven't had the time | 15:32 |
| lbragstad | so i don't want to hold things up if it's in a reasonable place (where keystone can opt into the behavior we currently have today) | 15:32 |
| moguimar | the patch does two things, turns it into a config option and flips the default behavior | 15:34 |
| bnemec | I'm curious what happens in the affected cluster if they just restart all of their services. Doesn't it trigger the same overload? | 15:39 |
| bnemec | Maybe on a rolling restart it's spread out enough to not cause a problem? | 15:39 |
| openstackgerrit | Moisés Guimarães proposed openstack/oslo.cache master: Bump dogpile.cache's version for Memcached TLS support https://review.opendev.org/745509 | 15:41 |
| bnemec | Okay, I've left a review that reflects our discussion here. Let me know if I misrepresented anything. | 15:42 |
| openstackgerrit | Merged openstack/oslo-cookiecutter master: sync oslo-cookiecutter contributing template https://review.opendev.org/743939 | 15:43 |
| bnemec | #topic enable oslo.messaging heartbeat fix by default? | 15:43 |
| *** openstack changes topic to "enable oslo.messaging heartbeat fix by default? (Meeting topic: oslo)" | 15:43 | |
| bnemec | This came up the week before I left. | 15:43 |
| kgiusti | seems like a safe bet at this point | 15:43 |
| bnemec | Related to the oslo.messaging ping endpoint change. | 15:44 |
| kgiusti | yeah, that change I'm not so thrilled about. | 15:44 |
| kgiusti | I was thinking of -2'ing that change, but wanted to discuss it here first. | 15:45 |
| kgiusti | too bad herve is off having a life :) | 15:45 |
| bnemec | Yeah, related only insofaras it came up in the discussion as an issue with checking liveness of services. | 15:45 |
| kgiusti | I wanted his opinion | 15:45 |
| kgiusti | bnemec: +1 | 15:45 |
| bnemec | We can probably wait until next week. This option has been around for quite a while now so it's not critical that we do it immediately. | 15:46 |
| bnemec | I'm not aware of anyone reporting issues with it though. | 15:46 |
| kgiusti | neither do I | 15:46 |
| kgiusti | but I think we do need to make a final decision of that ping patch | 15:47 |
| bnemec | Okay, I'll just leave it on the agenda for next week. | 15:47 |
| kgiusti | https://review.opendev.org/#/c/735385/ | 15:47 |
| bnemec | Was there more discussion on that after I logged off? | 15:47 |
| * bnemec has not been through openstack-discuss yet | 15:47 | |
| kgiusti | Lemme check... | 15:48 |
| kgiusti | http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016229.html | 15:48 |
| kgiusti | and the start of the discussion: http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016097.html | 15:49 |
| bnemec | I think it's in the same place as when I left. :-/ | 15:49 |
| kgiusti | KK | 15:50 |
| bnemec | It feels like a bug in Nova if a compute node can stop responding to messaging traffic and still be seen as "up". | 15:50 |
| kgiusti | Agreed. Seems like that proposed feature is out of scope for the oslo.messaging project IMHO. | 15:50 |
| kgiusti | Other than internal state monitoring, o.m. isn't intended to be a healthcheck solution | 15:51 |
| bnemec | I feel weird arguing against this when I've been advocating for good-enough healthchecks in the api layer though. :-/ | 15:52 |
| kgiusti | heck I _wanted_ this, but for my own selfish "don't blame me" reasons :) | 15:52 |
| kgiusti | Having dan's opinion made me rethink that from a more user-driver perspective. | 15:53 |
| kgiusti | Anyhow, that's where we stand at the moment. | 15:54 |
| kgiusti | I was wondering if any folks in Oslo felt differently. | 15:55 |
| bnemec | Unfortunately we're a bit short on Oslo folks today. | 15:55 |
| *** hamalq has quit IRC | 15:56 | |
| bnemec | I'm going to reply to the thread and ask if fixing the service status on the Nova side would address the concern here. That seems like a better fix than adding a bunch of extra ping traffic on the rabbit bus (which is already a bottleneck in most deployments). | 15:56 |
| *** hamalq has joined #openstack-oslo | 15:56 | |
| kgiusti | +1 | 15:56 |
| bnemec | #action bnemec to reply to rpc ping thread with results of meeting discussion | 15:57 |
| kgiusti | thanks bnemec | 15:57 |
| *** hamalq_ has joined #openstack-oslo | 15:57 | |
| bnemec | Okay, we're basically at time now so I'm going to skip the wayward review and open discussion. | 15:58 |
| bnemec | I think we had some good discussions this week though, so it was a productive meeting. | 15:59 |
| bnemec | If there's anything else we need to discuss, feel free to add it to the agenda for next week or bring it up in regular IRC. | 15:59 |
| bnemec | Thanks for joining everyone! | 15:59 |
| moguimar | not on my end | 15:59 |
| moguimar | o/ | 15:59 |
| bnemec | #endmeeting | 16:00 |
| *** openstack changes topic to "OpenStack Common Libraries | https://wiki.openstack.org/wiki/Oslo" | 16:00 | |
| kgiusti | See ya | 16:00 |
| openstack | Meeting ended Mon Aug 10 16:00:00 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/oslo/2020/oslo.2020-08-10-15.03.html | 16:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/oslo/2020/oslo.2020-08-10-15.03.txt | 16:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/oslo/2020/oslo.2020-08-10-15.03.log.html | 16:00 |
| bnemec | Right on time. :-) | 16:00 |
| kgiusti | spooky accurate :D | 16:00 |
| *** hamalq has quit IRC | 16:01 | |
| *** kevko_ has quit IRC | 17:39 | |
| *** brinzhang0 has joined #openstack-oslo | 17:42 | |
| *** brinzhang_ has quit IRC | 17:45 | |
| *** vishakha has quit IRC | 18:11 | |
| *** kgiusti has left #openstack-oslo | 18:35 | |
| *** tosky has quit IRC | 19:21 | |
| openstackgerrit | Merged openstack/pbr master: Begin work to modernize pbr's integration testing https://review.opendev.org/745171 | 19:37 |
| *** kgiusti has joined #openstack-oslo | 19:45 | |
| *** brinzhang_ has joined #openstack-oslo | 20:47 | |
| *** kgiusti has left #openstack-oslo | 20:49 | |
| *** brinzhang0 has quit IRC | 20:51 | |
| *** yoctozepto has quit IRC | 21:01 | |
| *** yoctozepto has joined #openstack-oslo | 21:04 | |
| *** moguimar has quit IRC | 21:32 | |
| openstackgerrit | Merged openstack/oslo.policy stable/rocky: Reload files in policy_dirs on primary file change https://review.opendev.org/734156 | 21:36 |
| *** rcernin has joined #openstack-oslo | 21:49 | |
| *** rcernin has quit IRC | 21:50 | |
| *** rcernin has joined #openstack-oslo | 21:50 | |
| *** camelCaser has quit IRC | 22:02 | |
| openstackgerrit | Merged openstack/sphinx-feature-classification master: Reduce need for openstack specific requirements https://review.opendev.org/557695 | 22:12 |
| *** camelCaser has joined #openstack-oslo | 22:32 | |
| *** tkajinam has joined #openstack-oslo | 22:46 | |
| *** raildo has quit IRC | 23:02 | |
| *** raildo has joined #openstack-oslo | 23:02 | |
| *** rcernin has quit IRC | 23:14 | |
| *** rcernin has joined #openstack-oslo | 23:17 | |
| *** raildo has quit IRC | 23:58 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!