| *** amoralej|off is now known as amoralej | 08:03 | |
| opendevreview | Rodolfo Alonso proposed openstack/releases master: Release ovn-octavia-provider RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875451 | 08:34 |
|---|---|---|
| opendevreview | Rodolfo Alonso proposed openstack/releases master: Release neutron RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875446 | 08:42 |
| opendevreview | Merged openstack/releases master: Release neutron RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875446 | 11:20 |
| opendevreview | Merged openstack/releases master: Release ovn-octavia-provider RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875451 | 11:25 |
| opendevreview | Merged openstack/releases master: Release tacker-horizon RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875470 | 11:25 |
| opendevreview | Merged openstack/releases master: Release heat-dashboard RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875412 | 11:25 |
| opendevreview | Merged openstack/releases master: Release heat-agents RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875411 | 11:25 |
| opendevreview | Merged openstack/releases master: Release kuryr-kubernetes RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875415 | 11:25 |
| opendevreview | Merged openstack/releases master: Release kuryr-libnetwork RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875416 | 11:26 |
| opendevreview | Merged openstack/releases master: Release ec2-api RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875406 | 11:26 |
| *** amoralej is now known as amoralej|lunch | 12:01 | |
| *** amoralej|lunch is now known as amoralej | 13:13 | |
| opendevreview | Merged openstack/releases master: [neutron] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/874461 | 13:21 |
| opendevreview | Merged openstack/releases master: Release openstackdocstheme to fix api-refs https://review.opendev.org/c/openstack/releases/+/875789 | 13:21 |
| opendevreview | Gregory Thiemonge proposed openstack/releases master: Release octavia RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875450 | 13:28 |
| elodilles | hberaud_ ttx : i think we have waited enough with the remaining stable branch cuts (for client libraries), so I've +2'd the patches. if you also think we can continue then please review them: https://review.opendev.org/q/topic:antelope-stable-branches+is:open | 13:29 |
| hberaud_ | sure | 13:30 |
| hberaud_ | OMW | 13:30 |
| elodilles | thanks \o/ | 13:31 |
| hberaud_ | do you know when will be proposed the automated patches to move from antelope's jobs to bobcat's jobs? | 13:33 |
| hberaud_ | just for my curiosity | 13:33 |
| elodilles | the patch is on review, but we should wait till the final release in my opinion | 13:34 |
| hberaud_ | ok | 13:34 |
| elodilles | hberaud_: this is the job template update patch: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/873051 | 13:37 |
| elodilles | hberaud_ ttx : i think we should also continue with this patch: https://review.opendev.org/c/openstack/releases/+/874414 | 13:38 |
| opendevreview | Merged openstack/releases master: [keystone] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/875233 | 13:44 |
| opendevreview | Merged openstack/releases master: [solum] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/875231 | 13:44 |
| opendevreview | Merged openstack/releases master: [sahara] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/875234 | 13:44 |
| opendevreview | Merged openstack/releases master: [monasca] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/875236 | 13:44 |
| opendevreview | Merged openstack/releases master: [murano] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/875235 | 14:02 |
| opendevreview | Merged openstack/releases master: [freezer] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/875232 | 14:02 |
| opendevreview | Merged openstack/releases master: [Telemetry] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/874464 | 14:02 |
| opendevreview | Merged openstack/releases master: [heat] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/874463 | 14:02 |
| opendevreview | Merged openstack/releases master: [designate] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/874470 | 14:02 |
| opendevreview | Merged openstack/releases master: [OpenStackSDK] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/874469 | 14:03 |
| opendevreview | Merged openstack/releases master: [ironic] Create 2023.1 branch for client and non-client libs https://review.opendev.org/c/openstack/releases/+/874457 | 14:03 |
| opendevreview | Elod Illes proposed openstack/releases master: [monasca] Release monasca-common and monasca-statsd for Antelope https://review.opendev.org/c/openstack/releases/+/874414 | 14:53 |
| opendevreview | Takashi Kajinami proposed openstack/releases master: Release storlets RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875469 | 14:56 |
| opendevreview | Takashi Kajinami proposed openstack/releases master: Release storlets RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875469 | 14:58 |
| opendevreview | Merged openstack/releases master: Release octavia-dashboard RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875449 | 14:59 |
| opendevreview | Takashi Kajinami proposed openstack/releases master: Release storlets RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875469 | 14:59 |
| opendevreview | Takashi Kajinami proposed openstack/releases master: Release storlets RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875469 | 14:59 |
| bauzas | I post here for better visibility but since the gate is busted, nova won't be able to deliver in time the required RC1 release today :( | 16:40 |
| clarkb | kind of related. The opendev team would like to restart Gerrit at some point to pick up replication plugin configuration updates that should reduce the number of gerrit restarts we need to replace our gitea servers. Is there a particularly good/bad time to do that? | 16:56 |
| hberaud_ | clarkb: I'd argue that in 1 or 2 hours we (the release team) will reduce our online activity, so, we could stop approve patch right now and let you apply your changes. ttx, elodilles that's ok for you? | 17:04 |
| hberaud_ | s/patch/patches/ | 17:04 |
| hberaud_ | bauzas: ok, thanks for the heads up | 17:05 |
| clarkb | hberaud_: I think we can wait longer than that too if it helps you | 17:06 |
| clarkb | I mostly just want to make sure that we minimize impact to the RC stuff | 17:06 |
| hberaud_ | clarkb: personally I'll be offline soon, so that's mostly depends on my teammates | 17:07 |
| clarkb | ok I can wait to see if anyone else asks to hold off for longer than a couple of hours | 17:08 |
| hberaud_ | sure | 17:08 |
| hberaud_ | thanks | 17:08 |
| bauzas | I have a couple of patches in flight but they already got rechecked a couple of times so once more isn't really a problem to me | 17:08 |
| *** hberaud_ is now known as hberaud | 17:08 | |
| bauzas | oh, gerrit | 17:09 |
| bauzas | nevermind | 17:09 |
| clarkb | if you were worried about zuul we do weekly rolling upgrades of zuul and 99% of the time no one even notices | 17:10 |
| bauzas | clarkb: this occured to me | 17:15 |
| bauzas | clarkb: for antelope-3 (ff) we got a couple of features impossible to merge due to gate failures | 17:15 |
| bauzas | did an incredible number of rechecks | 17:15 |
| bauzas | the saturday after it, someone did a single recheck on all the features in flight, and all of them got merged by once | 17:16 |
| clarkb | yes, openstack is its own noisy neighbor. I've shouted this until I was hoarse and no one listens | 17:16 |
| bauzas | that's why I'm confident to get Nova RC1 on Monday, not on Thurs :) | 17:16 |
| clarkb | Particularly with swapping I think | 17:16 |
| bauzas | well, I have an etherpad of doom | 17:16 |
| clarkb | basically every tempest job has a high chance to swap. When you run 8 tempest jobs for one patch and have 10 patches there is a good chance those have overlap within a cloud and on hypervisors | 17:17 |
| bauzas | trust me, this is not only swapping problems but a handful of distinct problems | 17:17 |
| clarkb | when you swap you kill your iops which kills the iops for everyone else which makes their swapping even worse but even if they aren't swapping yet generally they get slow | 17:17 |
| bauzas | clarkb: yeah, I remember our talk about it and we're trying to reduce our footprint | 17:17 |
| clarkb | I'm sure it isn't just swapping but the last time I did detailed analysis this was the result and everyone ignored the feedback so ya... | 17:17 |
| bauzas | but that's a hard thing to do | 17:17 |
| clarkb | But there is a pretty clear noisy neighbor feedback problem and the only way to address that is more resources (and no one is stepping up for that) or being smarter about applying load to existing resources | 17:18 |
| ttx | hberaud: yes ok for me | 17:34 |
| JayF | clarkb: is there a metric we publish about if jobs swap? I'm curious if Ironic jobs are part of the problem you describe | 17:46 |
| clarkb | JayF: not that I'm aware of. In fact some of the tools that collect the data (dstat) are deprecated and the copilot tool that picks up the functionality has broken packaging so isn't enabled by default iirc | 17:47 |
| clarkb | but the info is in devstack jobs (other tools grab it just without the fancy graphing) | 17:47 |
| JayF | ack; I'd be stunned if we are swapping on our non-integrated, bifrost-based jobs | 17:48 |
| clarkb | I think a lot of factors contribute to it too unfrotunately. I've talked about the death by a thousand cuts. libvirt/qemu increased the amount of memory needed for tiny VMs. privsep mysql/mariadb it all adds up | 17:49 |
| JayF | yeah; we are constantly fighting bloated cloud images in some of our tests | 17:51 |
| clarkb | zuul test runtime halved in the python3.8 to python3.10 switch. It doesnt look like openstack saw similar improvements which makes me wonder if zuul was doing something particularly bad under 3.8/3.9 or if zuul is written in a way that takes better advantage of the improvements that came with 3.10/3.11 | 17:53 |
| clarkb | but haven't seen similar from openstack projects. That isn't going to be memory related (more runtime related but less runtime is less time for contention) and may be another angle worth looking at to see if openstack needs changes to take advantage of the interpreter improvements. Unfortunately it wouldn't surprise me if eventlet negates those | 17:53 |
| elodilles | hberaud clarkb : sorry, i was commuting (today was an office day to me), but anyway, it is OK, as actually already around EOB today for me | 18:39 |
| clarkb | elodilles: thanks for confirming. I'll probably try to do it early afternoon my time | 18:40 |
| elodilles | clarkb: ack, sounds good! | 18:40 |
| *** amoralej is now known as amoralej|off | 18:45 | |
| opendevreview | Goutham Pacha Ravi proposed openstack/releases master: Release manila RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875420 | 20:34 |
| opendevreview | Goutham Pacha Ravi proposed openstack/releases master: Release manila RC1 for Antelope https://review.opendev.org/c/openstack/releases/+/875420 | 20:37 |
| clarkb | we're going to do that gerrit restart shortly | 21:02 |
| clarkb | the restart is done. We're just double checking things | 21:14 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!