| opendevreview | Takashi Kajinami proposed openstack/releases master: Puppet OpenStack 2024.2 release https://review.opendev.org/c/openstack/releases/+/931114 | 02:43 |
|---|---|---|
| opendevreview | Takashi Kajinami proposed openstack/releases master: Puppet OpenStack 2024.2 release https://review.opendev.org/c/openstack/releases/+/931114 | 02:45 |
| opendevreview | Takashi Kajinami proposed openstack/releases master: Puppet OpenStack 2024.2 release https://review.opendev.org/c/openstack/releases/+/931114 | 03:45 |
| opendevreview | Takashi Kajinami proposed openstack/releases master: Puppet OpenStack 2024.2 release https://review.opendev.org/c/openstack/releases/+/931114 | 04:38 |
| opendevreview | Takashi Kajinami proposed openstack/releases master: Puppet OpenStack 2024.2 release https://review.opendev.org/c/openstack/releases/+/931114 | 05:38 |
| ttx | It's release day! | 07:38 |
| elodilles | ~o~ | 07:46 |
| opendevreview | Joel Capitao proposed openstack/releases master: oslo.reports 3.4.1 https://review.opendev.org/c/openstack/releases/+/931140 | 07:48 |
| elodilles | no new reno links, so we don't need to rebase the release patch \o/ | 07:59 |
| ttx | yay | 08:20 |
| elodilles | the semaphore patch has landed yesterday ( https://review.opendev.org/c/openstack/openstack-doc-tools/+/931029 ) so we are good with that too \o/ | 09:31 |
| elodilles | btw, i've proposed this patch, with freshly generated static sitemap, but i guess this is OK to be merged slightly after the release: https://review.opendev.org/c/openstack/openstack-doc-tools/+/931029 | 09:33 |
| ttx | yeah, let's wait | 09:34 |
| ttx | Ready to push the big button in 20min | 09:34 |
| opendevreview | Elod Illes proposed openstack/releases master: Mark 2024.2 Dalmatian as released https://review.opendev.org/c/openstack/releases/+/931153 | 09:45 |
| elodilles | i almost forgot that one ^^^ | 09:45 |
| elodilles | ttx: please review it, so that we can merge this later | 09:46 |
| elodilles | so far ' All Systems Operational ' @ https://status.python.org/ | 09:47 |
| elodilles | T-5 | 09:55 |
| fungi | i'm awake (for some definition thereof) | 09:57 |
| elodilles | good morning fungi o/ | 09:57 |
| elodilles | status.python.org still reports things as operational | 09:58 |
| elodilles | as i see zuul is also healthy | 09:58 |
| elodilles | mostly busy with kolla patches :) | 09:59 |
| ttx | ok | 09:59 |
| elodilles | ttx: are you around to push the button? | 09:59 |
| elodilles | https://review.opendev.org/c/openstack/releases/+/930752 | 10:00 |
| ttx | button pushed | 10:00 |
| elodilles | \o/ | 10:00 |
| zigo | Does this releases EVERYTHING at once ?!? | 10:01 |
| elodilles | 'status.python.org' | 10:01 |
| elodilles | zigo: yes and no: these are the cycle-with-rc release model deliverables, and the post-release jobs will do the processing somewhat parallel but it will take some time | 10:02 |
| elodilles | s/''/ | 10:03 |
| elodilles | Starting gate jobs. | 10:03 |
| zigo | elodilles: Right, but then I can expect everything in a few minutes then... :P | 10:03 |
| elodilles | a bit more than a few minutes, but yes :] | 10:04 |
| fungi | zigo: "all at once" over the course of about 4-5 hours | 10:17 |
| zigo | Oh ok. So I can go have lunch ... :P | 10:18 |
| fungi | it's mostly automated aside from a few manual checks that the automation didn't miss anything, but it takes a while to complete yes | 10:18 |
| fungi | zigo: yes, lunch and more. the goal is to have it wrapped up and final documentation updates merged by 15:00 utc | 10:20 |
| fungi | tags will probably be present in all repositories by 13z, with things like tarballs and wheels generated/uploaded by 14z | 10:22 |
| fungi | possibly sooner, but that's a conservative estimate | 10:23 |
| ttx | Releasing Everything everywhere all at once | 10:24 |
| elodilles | :) | 10:24 |
| ttx | OK, pausing for lunch | 10:38 |
| opendevreview | Merged openstack/releases master: 2024.2 Dalmatian final releases for cycle-with-rc projects https://review.opendev.org/c/openstack/releases/+/930752 | 10:39 |
| elodilles | ttx: Bon Appétit! (will do the same soon) | 10:40 |
| elodilles | and the patch has merged! \o/ | 10:40 |
| elodilles | let's see how the release jobs starts to work! | 10:40 |
| fungi | tags are appearing now | 10:49 |
| elodilles | yepp. release queue is also increasing | 10:50 |
| fungi | tagging seems to be done, release pipeline has 51 items now | 10:54 |
| fungi | release pipeline topped out at 65 entries i think | 10:56 |
| elodilles | :-o | 10:57 |
| fungi | unfortunately all those kolla jobs had already sapped the available zuul capacity starting around 08:45 utc, so the release and tag jobs are having to wait for quota to free up in order to get new nodes assigned: https://grafana.opendev.org/d/21a6e53ea4/zuul-status | 10:59 |
| elodilles | i see some kolla patches in the check queue that have 15-20 mins left, so maybe we will have more nodes then o:) | 11:04 |
| ttx | back | 11:05 |
| fungi | some are running now | 11:13 |
| fungi | adjutant should have been uploaded | 11:13 |
| fungi | yeah https://pypi.org/project/python-adjutant/ looks good | 11:14 |
| elodilles | yepp, even nova and neutron are there on pypi | 11:15 |
| fungi | the tag pipeline is nearly empty already (releasenotes jobs) | 11:19 |
| fungi | release pipeline count is starting to fall now that release announcement and constraint update proposal jobs are completing | 11:20 |
| elodilles | ~o~ | 11:22 |
| fungi | the node request backlog is nearly cleared so most jobs should be running by now | 11:27 |
| fungi | all release and tag jobs have completed, i see no reported failures | 11:43 |
| ttx | that was... fast | 11:44 |
| * ttx checks status | 11:44 | |
| ttx | running missing-releases | 11:45 |
| elodilles | ++ | 11:46 |
| ttx | running into some issue with the script, AttributeError: module 'wheel.bdist_wheel' has no attribute 'safer_name' | 11:48 |
| ttx | investigating | 11:48 |
| fungi | could be due to distutils changes in recent setuptools? | 11:50 |
| fungi | wheel has apparently deprecated wheel.bdist_wheel in the most recent release: https://wheel.readthedocs.io/en/stable/news.html | 11:52 |
| ttx | yeah, bdist_wheel should no longer be consumed from wheel | 11:52 |
| ttx | I can workaround it quickly using _bdist_wheel instead | 11:52 |
| ttx | but needs a more thorough fix | 11:52 |
| ttx | adding a note about that in postmortem | 11:54 |
| elodilles | ACK | 11:55 |
| elodilles | (by using older setuptools the script works, so yepp...) | 11:56 |
| ttx | run in progress, no issue so far | 11:58 |
| fungi | not bad at all for being only 2 hours in | 12:00 |
| ttx | wondering if https://review.opendev.org/c/openstack/releases/+/929126 did not result in significantly faster turnaround | 12:02 |
| ttx | alright, no issue reporter in missing-releases | 12:05 |
| fungi | that would have sped up release announcement jobs, but any others? | 12:05 |
| ttx | elodilles: I think you can remove your W-1 from https://review.opendev.org/c/openstack/openstack-manuals/+/930332 | 12:05 |
| elodilles | ttx: done | 12:06 |
| ttx | any reason to hold on approving that one? | 12:06 |
| fungi | yeah, keep in mind that we've seen it take more than an hour to get the openstack-manuals change in and published, so getting started on that with plenty of time to spare would be good | 12:06 |
| elodilles | (so bdist_wheel is deprecated in 'wheel' package: https://github.com/pypa/wheel/commit/0add7d6c5fe6c03d7bec22d7d18f51d8a519862c ) | 12:06 |
| ttx | I'm fine approving it now | 12:07 |
| elodilles | i'm OK with that, too | 12:07 |
| ttx | (I have a call in 53min) | 12:07 |
| ttx | Alright, approving | 12:07 |
| elodilles | ttx: i don't have +2+W rights on openstack-manuals anyway o:) | 12:08 |
| elodilles | thanks o/ | 12:08 |
| fungi | zuul estimates about another 17 minutes for the promote job to complete, and then afs releases the read-only replica approximately every 5 minutes, after which time the docs site should reflect it | 12:37 |
| elodilles | ++ | 12:39 |
| elodilles | (meanwhile i've prepared the 2 announce mails content: https://etherpad.opendev.org/p/relmgmt-weekly-emails will send it around 15:00 UTC if everything goes fine) | 12:40 |
| fungi | awesome | 12:43 |
| fungi | i'm on hand to approve openstack-announce posts when the time comes | 12:43 |
| elodilles | +1 thanks in advance | 12:45 |
| fungi | promote upload just finished, 13:00 vos release should publish it | 12:58 |
| fungi | 2024-10-02 13:00:01,420 release DEBUG Running: ssh -T -i /root/.ssh/id_vos_release vos_release@afs01.dfw.openstack.org -- vos release docs | 13:00 |
| fungi | 2024-10-02 13:00:40,558 release INFO Release of docs successful | 13:01 |
| elodilles | \o/ | 13:01 |
| fungi | https://docs.openstack.org/ is now redirecting to https://docs.openstack.org/2024.2/ | 13:02 |
| fungi | which seems to have the expected content, e.g. "Currently viewing 2024.2 which is the current supported release." and "Documentation for 2024.2 (October 2024)" | 13:03 |
| elodilles | yepp, things look good there | 13:03 |
| fungi | the release notes link also correctly goes to https://releases.openstack.org/dalmatian/index.html | 13:04 |
| fungi | is https://review.opendev.org/c/openstack/releases/+/931153 the last thing before sending the announcement e-mails? | 13:07 |
| elodilles | that is the next step, yes, and then we have ' Remind Foundation staff to switch openstack.org 2024.2 Dalmatian messaging on (ttx)' task :) | 13:08 |
| elodilles | and then the announcement e-mails | 13:09 |
| ttx | yeah, let's wait maybe 50min to flip the switch on the openstack.org website | 13:23 |
| ttx | like ask them to do it around 14utc | 13:23 |
| elodilles | +1 | 13:24 |
| ttx | OK well it's done now | 13:28 |
| ttx | Let's prepare to approve https://review.opendev.org/c/openstack/releases/+/931153 | 13:29 |
| ttx | and send announcements emails maybe 30minutes ahead of deadline | 13:29 |
| ttx | Ping me if you want me to proofread stuff | 13:30 |
| elodilles | ACK. ttx feel free to +2+W the patch (if looks fine) as it was prepared by me o:) | 13:30 |
| elodilles | ttx: these will be the mails https://etherpad.opendev.org/p/relmgmt-weekly-emails | 13:30 |
| ttx | elodilles: ok will +a if you post another +2 on https://review.opendev.org/c/openstack/releases/+/931153 | 13:31 |
| elodilles | ttx: +2'd | 13:33 |
| ttx | +a'd | 13:35 |
| elodilles | ++ | 13:37 |
| ttx | emails look good, let's hold a bit before sending them.. maybe send around 14utc? | 13:38 |
| ttx | or 14:30utc | 13:38 |
| elodilles | both works for me :) | 13:38 |
| elodilles | first i have to send the mail to openstack-announce then fungi approves it, then i can send the 2nd to openstack-discuss | 13:39 |
| fungi | yep | 13:40 |
| fungi | sounds good | 13:40 |
| fungi | i like that we've got a bit of breathing room this time, for a change | 13:40 |
| fungi | it's been a while since we didn't have some sort of last-moment scramble | 13:41 |
| fungi | so i wouldn't take it as an indication that we should start later next time | 13:41 |
| elodilles | yeah, it depends many things, how many patches are on the gate, pypi and other systems health, etc | 13:43 |
| elodilles | so we can stick to starting @ 10:00 UTC | 13:44 |
| fungi | yeah, i think that's worked out pretty well | 13:44 |
| elodilles | yes | 13:45 |
| fungi | i'm going to take the opportunity for a quick shower, brb | 13:53 |
| elodilles | ++ | 13:54 |
| opendevreview | Merged openstack/releases master: Mark 2024.2 Dalmatian as released https://review.opendev.org/c/openstack/releases/+/931153 | 13:58 |
| elodilles | merged! | 13:58 |
| elodilles | let's see when it will take on effect on releases.o.o | 13:59 |
| Clark[m] | The jobs are queued/waiting in zuul to publish that then it's on the 5 minute afs release timer I think | 14:09 |
| elodilles | ++ | 14:10 |
| elodilles | now the publish-tox-docs-releases job has started | 14:18 |
| fungi | yep | 14:19 |
| fungi | also i'm back | 14:19 |
| ttx | back too | 14:21 |
| elodilles | publish job is still running | 14:23 |
| elodilles | finished | 14:24 |
| fungi | 14:25 vos release should have made it available on the site | 14:25 |
| ttx | It's on | 14:25 |
| ttx | Clear for takeoff | 14:25 |
| fungi | yep, dalmatian now shows as maintained | 14:25 |
| elodilles | yepp, 2024.2 Dalmatian | 14:25 |
| elodilles | Maintained | 14:25 |
| elodilles | ~o~ | 14:26 |
| ttx | Time to send some emails! | 14:26 |
| elodilles | ACK | 14:26 |
| fungi | i'm ready to approve the one to openstack-announce when it shows up in the moderation queue | 14:26 |
| elodilles | fungi: e-mail is sent | 14:28 |
| fungi | and it's accepted for delivery to the list | 14:28 |
| Clark[m] | I received it | 14:29 |
| elodilles | \o/ | 14:29 |
| ttx | I tooted it | 14:30 |
| fungi | toot toot! | 14:31 |
| elodilles | and there goes the other e-mail! | 14:31 |
| ttx | Both received | 14:34 |
| ttx | Congrats everyoen on a smooth release day | 14:35 |
| zigo | Indeed! :P | 14:36 |
| elodilles | yes-yes, congrats everyone! \o/ | 14:36 |
| fungi | great work everyone! | 14:36 |
| clarkb | plenty of time to review the SLURP upgrade path diagram I pushed up :) | 14:42 |
| clarkb | I still don't think it is urgent but from discussions yesterday do think it would be helpful to have something liek that | 14:43 |
| elodilles | and there it is: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_91c/931066/1/check/openstack-tox-docs/91c43bc/docs/#release-series | 14:44 |
| ttx | It's "Dalmatian" not "Dalmation" but otherwise I like it | 14:47 |
| clarkb | ttx: I'll get that fixed | 14:48 |
| opendevreview | Clark Boylan proposed openstack/releases master: Render a graph showing the SLURP and not SLURP upgrade paths https://review.opendev.org/c/openstack/releases/+/931066 | 14:49 |
| zigo | clarkb: You wrote A, C, B, with caracal before bobcat ... | 14:57 |
| zigo | Or am I missing something? | 14:58 |
| clarkb | zigo: correct. That affects the rendering if I listed them in alpha order then bobcat and dalmatian get rendered above the slurp path | 15:00 |
| clarkb | the graph order is defined by the edges ( -> lines ) and it should rendering in the correct order | 15:00 |
| fungi | which would still be a valid diagram of it, just not as appealingly arranged | 15:00 |
| clarkb | those names are also arbitrary I could've called them 1,2,3,4,5 and then labeled them appropriately I suppose | 15:01 |
| fungi | in dealing with the blockdiag/seqdiag issues, you've become our expert on sphinx's graphviz extension. now everyone will be sending diagramming requests to you directly ;) | 15:01 |
| clarkb | my only protip is to use https://dreampuf.github.io/GraphvizOnline/ to test and figure things uot quickly | 15:02 |
| clarkb | its much quicker than rerunning tox over and over or figuring ou the right command line | 15:02 |
| clarkb | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_4c8/931066/2/check/openstack-tox-docs/4c8b284/docs/#releases-with-skip-level-upgrade-release-process-slurp is the updated preview | 15:15 |
| zigo | clarkb: *very* nice, thanks for writing this. | 15:16 |
| zigo | It shows very clearly that this is fucked-up for Debian... :) | 15:16 |
| clarkb | unfortunately I don't think I can fix that with graphviz | 15:17 |
| zigo | Yeah... | 15:17 |
| zigo | Never mind, my upgrade script works very well. | 15:17 |
| zigo | :) | 15:17 |
| * zigo finished uploaded everything | 15:19 | |
| zigo | I need to wait tomorrow to see the result on my Q/A page to see if I forgot a project or not. | 15:19 |
| opendevreview | Merged openstack/releases master: Render a graph showing the SLURP and not SLURP upgrade paths https://review.opendev.org/c/openstack/releases/+/931066 | 15:56 |
| fungi | 15:47 <opendevreview> Merged openstack/project-config master: Revert "Temporarily remove release docs semaphores" https://review.opendev.org/c/openstack/project-config/+/930710 | 16:04 |
| elodilles | thx o/ | 16:10 |
| clarkb | https://releases.openstack.org/#releases-with-skip-level-upgrade-release-process-slurp hrm I haven't seen that before. Its like the doc build for publishing didn't try to render it at all | 16:22 |
| clarkb | trying to dig up that job now to see if the artifact there does the same thing | 16:22 |
| clarkb | I see the bug | 16:25 |
| JayF | https://releases.openstack.org/#releases-with-skip-level-upgrade-release-process-slurp is also wrong w/r/t next stage for Dalmation; as a non-SLURP release it gets directly to EOL, not to UM | 16:27 |
| JayF | similar to how bobcat is rendered | 16:28 |
| clarkb | JayF: I'm not sure I understand | 16:29 |
| JayF | talking about https://usercontent.irccloud-cdn.com/file/dt8YW5IN/image.png | 16:29 |
| JayF | (I should've dropped the anchor from the link) | 16:29 |
| clarkb | got it that is unrelated to my change | 16:30 |
| JayF | I don't know enough about how that page is generated to have known that, but it makes sense :) | 16:31 |
| * JayF sees if that's an obvious fix | 16:31 | |
| JayF | it is, I am testing but will PR shortly | 16:33 |
| clarkb | if we land that fix after the fix for the graphviz thing we should be able to correct both with the same publication run | 16:33 |
| clarkb | I'll get a fix up for graphviz shortly | 16:33 |
| JayF | It's literally a one line change, might be easier for you to pull it in | 16:33 |
| JayF | than to dance with gerrit to combine 'em | 16:33 |
| clarkb | remote: https://review.opendev.org/c/openstack/project-config/+/931204 Set bindep profiles for openstack release doc publication | 16:35 |
| clarkb | JayF: the issue is the fix for my issue is in project-config not releases | 16:35 |
| clarkb | also a one line fix | 16:35 |
| JayF | aha | 16:35 |
| clarkb | alternatively I could drop the doc profile from the bindep file in releases but that is a less correct fix. We only need to install graphviz when building docs | 16:36 |
| clarkb | fungi: ^ fyi on 931204 | 16:36 |
| opendevreview | Jay Faulkner proposed openstack/releases master: Next stage for Dalmation is EOL; not UM https://review.opendev.org/c/openstack/releases/+/931206 | 16:36 |
| clarkb | I'm also happy if we prefer to revert or just drop the profile in releases so that grpahviz is always installed. Let me know if we want a different appraoch | 16:40 |
| JayF | only installing graphviz when/where it's needed is wise; it's a package that has had vulns in the past iirc | 16:43 |
| clarkb | fungi: repsonded to your comment but also I'm wondering if anyone else will review 931204 or if we should proceed? | 16:58 |
| elodilles | JayF: my bad, sorry :S thanks for the patch! +2 +PTL-Approved+1'd it | 17:00 |
| JayF | not a problem, glad to be able to help even just a crumb :D | 17:02 |
| clarkb | I need ot step away for a bit but I am in this room on matrix so I'll see if anything needs updating | 17:02 |
| elodilles | JayF: at least you noticed it quite fast, so we can fix it ASAP o:) | 17:04 |
| elodilles | ttx: if you are around, please +2+W JayF's patch: https://review.opendev.org/c/openstack/releases/+/931206 | 17:04 |
| fungi | clarkb: approved, thanks! | 17:09 |
| elodilles | yep, thanks for that, too o/ | 17:13 |
| clarkb | https://review.opendev.org/c/openstack/project-config/+/931204 just merged so landing 931206 shoudl fix both issues | 17:25 |
| clarkb | I am in need of food so will pop out again. I'll check back to make sure the graph is resolved if 931206 or something like ti lands | 17:36 |
| ttx | Approved | 20:29 |
| opendevreview | Merged openstack/releases master: Next stage for Dalmation is EOL; not UM https://review.opendev.org/c/openstack/releases/+/931206 | 20:42 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
