| ttx | Happy release day! | 07:45 |
|---|---|---|
| elodilles | \o/ | 08:25 |
| elodilles | about ~2.5 hrs and we can start the machinery | 08:25 |
| elodilles | hmmm, btw, we have a task 'Few hours before approving the release patch ensure that all the available release note landing pages are present', and I have proposed this patch on Monday: https://review.opendev.org/c/openstack/releases/+/878004 | 08:38 |
| elodilles | though now we agreed to not merge patches to release repo | 08:38 |
| elodilles | so the question is whether 'semaphore removal' is really affecting this or should we merge the patch? | 08:39 |
| *** amoralej|off is now known as amoralej | 08:42 | |
| opendevreview | Elod Illes proposed openstack/releases master: Mark 2023.1 Antelope as released https://review.opendev.org/c/openstack/releases/+/878200 | 08:51 |
| hberaud | I think we can merge the patch | 08:52 |
| hberaud | IMO | 08:52 |
| elodilles | i also feel that this reno-link patch shouldn't be problematic with the temporary 'semaphore removal', but I'm not 100% sure | 08:56 |
| opendevreview | Rodolfo Alonso proposed openstack/releases master: [neutron] Transition Train release to EOL https://review.opendev.org/c/openstack/releases/+/878202 | 09:19 |
| opendevreview | Rodolfo Alonso proposed openstack/releases master: [neutron] Transition Train release to EOL https://review.opendev.org/c/openstack/releases/+/878202 | 09:20 |
| ttx | i think we can merge it now | 09:31 |
| ttx | but also ok to do it just after | 09:32 |
| elodilles | ack, I've removed the W-1 | 09:32 |
| elodilles | :) | 09:32 |
| ttx | Traditionally I think we fixed up release notes all release week :) | 09:32 |
| ttx | If there was a good reason to wait, we can wait a few hours | 09:32 |
| ttx | haven;t followed the semaphore discussion | 09:33 |
| elodilles | ttx: this is the patch https://review.opendev.org/c/openstack/project-config/+/877552 | 09:33 |
| elodilles | 'These semaphores are intended to avoid builds for | 09:34 |
| elodilles | different branches of the same project from racing one another and | 09:34 |
| elodilles | publishing out of sequence' | 09:34 |
| elodilles | so *probably* not a problem | 09:34 |
| elodilles | for our reno-link patch | 09:35 |
| ttx | Yeah so it only affects multiple release-notes build affecting the same project | 09:35 |
| ttx | as long as we only approve this one there should be no race | 09:35 |
| ttx | we should just wait until it completes befroe starting any other one | 09:36 |
| elodilles | yepp | 09:36 |
| ttx | I +2ed | 09:37 |
| * elodilles +2+W'd | 09:44 | |
| opendevreview | Merged openstack/releases master: Add release notes links for 2023.1 Antelope https://review.opendev.org/c/openstack/releases/+/878004 | 09:51 |
| hberaud | \o/ | 09:51 |
| hberaud | FYI, I'll have to grab my kids at school in ~20 minutes, so I'll be AFK during a while (20-30 min) | 10:37 |
| elodilles | hberaud: ack. ttx and I can approve the release patch :) | 10:39 |
| elodilles | according to our time plan it will happen exactly in 20 mins :) | 10:39 |
| fungi | yes, sorry if i wasn't clear earlier about the semaphore, as ttx indicated it was merely there to address problems with multiple builds for the same project merging at roughly the same time, resulting in job failures from trying to rsync into the same doctree simultaneously, and out-of-order publication from jobs for the same project taking variable amounts of time to complete | 10:43 |
| fungi | my concern was merging the temporary removal too early if there were going to be stable branch release requests approved, where the request included tags for more than one branch of a project in the same change, since that's where we typically saw issues those semaphores were mitigating | 10:44 |
| ttx | I'll be AFK from 12utc to 14utc, but will read messages around 12:30utc. After that I have meetings from 14utc to 16utc but can multiplex with release issues | 10:48 |
| elodilles | I've reviewed the release patch again ( https://review.opendev.org/c/openstack/releases/+/877942 ) and saw some ugly 'diff-start' versions, which shouldn't be a problem, but they are not nice :/ anyway +2'd the patch | 10:51 |
| ttx | I +2ed too, will let hberaud do the honors | 10:54 |
| zigo | PRESS THE BIG RED BUTTON, LET'S GOOOOOO ! :) | 10:54 |
| elodilles | ttx: as I understand he won't be around o:) | 10:55 |
| elodilles | so feel free to approve ;) | 10:55 |
| ttx | ah alright, misread | 10:55 |
| ttx | Oh well, fire in the hole! | 10:55 |
| elodilles | \o/ | 10:55 |
| * fungi ducks behind the blast barrier | 10:56 | |
| elodilles | gate jobs are queued | 10:57 |
| elodilles | pypi and zuul look healthy | 10:59 |
| ttx | validate job in progrses | 11:01 |
| elodilles | that will take long as far as I remember | 11:02 |
| zigo | Are we waiting for zull to process #877942 ? | 11:05 |
| elodilles | zigo: yepp | 11:10 |
| elodilles | :) | 11:10 |
| fungi | yes, particularly the openstack-tox-validate job for it | 11:10 |
| elodilles | after it got merged, the release tagging and other post jobs will happen | 11:11 |
| fungi | zuul's time estimate for it is likely incorrect since it bases that on other recent builds and the duration for that job depends a bit on how many new releases are in it | 11:11 |
| elodilles | yepp | 11:12 |
| fungi | i'm taking the opportunity to grab a quick break, brb | 11:12 |
| elodilles | fungi: ack | 11:13 |
| * hberaud back | 11:31 | |
| fungi | same here | 11:32 |
| elodilles | openstack-tox-validate is still running | 11:32 |
| fungi | on networking-bgpvpn | 11:32 |
| fungi | almost to oslo | 11:39 |
| fungi | nevermind, no oslo because those were already tagged | 11:40 |
| fungi | nearly there | 11:53 |
| fungi | zun! | 11:56 |
| fungi | 0 errors found | 11:57 |
| fungi | tox took approximately 53 minutes, so the job overall is running about an hour for a full coordinated release these days | 11:58 |
| opendevreview | Merged openstack/releases master: antelope final rc https://review.opendev.org/c/openstack/releases/+/877942 | 11:58 |
| elodilles | \o/ | 11:59 |
| fungi | and it's in release-post now | 11:59 |
| zigo | Still no tag in git ? | 11:59 |
| elodilles | impressive amount of time :-o | 11:59 |
| elodilles | zigo: this just validated the release patch :) | 11:59 |
| fungi | zigo: the tag-releases job in the release-post pipeline is what will start pushing tags momentarily | 11:59 |
| zigo | Ok ! :) | 11:59 |
| elodilles | the post jobs will do the work | 11:59 |
| elodilles | zigo: note that the official release (announcement) is about to happen around 15:00 UTC :) | 12:00 |
| fungi | once it gets its node assignment | 12:00 |
| elodilles | so this will take some time | 12:00 |
| fungi | keeping an eye on the node requests and test nodes graphs at https://grafana.opendev.org/d/21a6e53ea4/zuul-status?orgId=1 | 12:00 |
| fungi | though th release and tag pipeline graphs there will also be useful for watching the eventual burn-down | 12:01 |
| elodilles | so until 15:00 UTC hopefully everything get tagged, and uploaded to pypi / tarball storage, and we have to check whether there are any issue (missing releases) :S | 12:02 |
| elodilles | so... lots of 'fingers crossing' :D | 12:02 |
| fungi | also we may have all the release notes published by then this time, with those semaphores temporarily out of the way | 12:03 |
| *** amoralej is now known as amoralej|lunch | 12:03 | |
| fungi | tag-releases got its node request satisfied and is starting now | 12:03 |
| elodilles | ~o~ | 12:03 |
| fungi | tags are now getting pushed | 12:05 |
| fungi | https://pypi.org/project/python-adjutant/ shows 6.0.0 as latest now | 12:10 |
| * zigo is already building packages | 12:10 | |
| fungi | https://tarballs.opendev.org/openstack/adjutant/ has python-adjutant 6.0.0 tarballs/wheels and signatures for each | 12:11 |
| fungi | releasenotes jobs are running in parallel in the tag pipeline this time | 12:12 |
| elodilles | \o/ | 12:13 |
| armstrong | fungi: is this the first time that this parallel run is happening? | 12:14 |
| fungi | armstrong: no, they used to run in parallel but we added a semaphore to serialize them a while back because if someone requests tags for multiple branches in the same change and those end up running at about the same time they can cause job failures from the two rsync processes trying to write release notes builds to the same project tree, or can end up running out of sequence leading | 12:16 |
| fungi | to temporarily stale content | 12:16 |
| armstrong | Ok thanks | 12:16 |
| fungi | unfortunately, zuul doesn't have a per-project semaphore construct, it's for all builds in the pipeline, which means on release day running those jobs one at a time and waiting for one to finish before starting the next takes most of the day to finish | 12:17 |
| fungi | and since we're only pushing one tag for each project, the reason for using the semaphore isn't present | 12:17 |
| fungi | so we temporarily removed the semaphore yesterday, and will merge a change to revert that after the release jobs finish | 12:18 |
| armstrong | fungi: thanks! That’s a lot of work | 12:19 |
| fungi | and this time the tag pipeline is clearing out faster than the release pipeline, so we'll be done with release notes probably before we're done with publishing the release artifacts even | 12:21 |
| fungi | we're about halfway through the alphabet now in tag-releases | 12:22 |
| Clark[m] | There are docker job failures for monasca repos. I suspect this isn't a major problem for the overall release | 12:26 |
| fungi | yeah, i think that's expected based on https://zuul.opendev.org/t/openstack/builds?job_name=publish-monasca-notification-docker-image&project=openstack/monasca-notification | 12:28 |
| hberaud | IIRC the docker failure on monasca was already there during Zed's final | 12:28 |
| hberaud | monasca outreached the upload treshold on the docker hub or soemthing like that | 12:30 |
| hberaud | (during the previous series) | 12:30 |
| ttx | Yeah, not really our problem | 12:30 |
| hberaud | yeah | 12:31 |
| Clark[m] | The error is related to new docker and the unstated apparmor dependency. It's possible they would fail after fixing that though | 12:32 |
| fungi | well, i don't see any indication that job has worked since beginning of february | 12:32 |
| fungi | but yeah, it could be a new regression in running the docker client | 12:33 |
| Clark[m] | The apparmor requirement came with the docker 23.0.0 release whenever that was. May have been around then | 12:33 |
| fungi | seems highly likely to be correlated | 12:33 |
| Clark[m] | Their packages are broken and don't explicitly list the required apparmor dep | 12:34 |
| fungi | which will be up to the monasca folks to fix | 12:34 |
| Clark[m] | Yup | 12:35 |
| elodilles | actually it looks like monasca will be removed from 2023.2 Bobcat cycle :/ | 12:37 |
| elodilles | the team disappeared last year :/ | 12:37 |
| elodilles | so yes, we are aware of their docker job failure, but couldn't remove monasca yet from 2023.1 Antelope :/ | 12:38 |
| fungi | i've un-wip'ed https://review.opendev.org/877553 so we can approve it to put the semaphores back once the tag pipeline clears out | 12:39 |
| fungi | nearly at the end of the tags now | 12:39 |
| elodilles | ++ | 12:40 |
| fungi | and there's zun! | 12:41 |
| fungi | all tags have been pushed to gerrit now | 12:42 |
| fungi | tag-releases reports success, publish-tox-docs-releases should run shortly | 12:43 |
| fungi | we've got 35 items in the releases pippeline and 20 in the tag pipeline still going | 12:44 |
| zigo | Please slow down, I'm not packaging that fast ... :) | 12:47 |
| Clark[m] | Assuming release notes all look good do we need to update any docs about the semaphore removal hack? | 12:48 |
| fungi | Clark[m]: probably add a couple of notes to the release day checklist yeah | 12:48 |
| fungi | release site publication is probably going to miss the 12:50 afs vos release but should be included in the 12:55 | 12:50 |
| fungi | though i should start tailing the vos release logs, the burst of tarballs site updates may have clogged it a bit | 12:50 |
| fungi | nope, it's running smoothly | 12:51 |
| fungi | https://releases.openstack.org/antelope/index.html shows adjutant 6.0.0 | 12:52 |
| fungi | just barely caught its train | 12:52 |
| * hberaud dad-taxi be back in a couple of minutes | 12:53 | |
| fungi | sure, we're still waiting for the release and tag pipelines to burn down before we can start consistency checks | 12:53 |
| fungi | tag pipeline is down to the dregs, release pipeline has about 20 left to go | 12:54 |
| fungi | so roughly, the big releases change was approved at 11z and required about an hour in the gate, then around 12z we started getting tags, and they'll be about done processing by 13z, which gives us 2 hours for final checks and fixing things | 12:56 |
| fungi | looking at the test nodes graph, zuul didn't really even break a sweat. we peaked at around 80% of our aggregate quota | 12:57 |
| elodilles | ++ | 12:58 |
| fungi | and that's with normal development activity going on at the same time | 12:58 |
| fungi | tag pipeline's empty, 5 left in the release pipeline | 12:59 |
| Clark[m] | How close to job timeouts did the gate verification job and tag-releases job get? Non issue at this point but worth double checking now that we have hard data | 13:00 |
| fungi | great question, we can check start/end times later to find out | 13:01 |
| fungi | looks like trove may have been the last to finish its release-openstack-python job, so we'll want to make sure it's appearing on the tarballs site before we start checking | 13:03 |
| fungi | https://zuul.opendev.org/t/openstack/builds?job_name=release-openstack-python&skip=0 | 13:04 |
| fungi | yeah, 19.0.0 artifacts are already on https://tarballs.opendev.org/openstack/trove/ | 13:05 |
| fungi | release pipeline is empty now | 13:05 |
| fungi | https://lists.openstack.org/pipermail/release-job-failures/2023-March/ only has failures for monasca, already known | 13:09 |
| fungi | i guess we should be ready to proceed with the check for missing artifacts? | 13:10 |
| elodilles | I've started it earlier | 13:16 |
| elodilles | and finished with the 'usual' result: https://paste.opendev.org/show/bAagRq3rkx39T10YwG1v/ | 13:16 |
| elodilles | these were the ones missing in Zed release as well, | 13:17 |
| *** amoralej|lunch is now known as amoralej | 13:17 | |
| fungi | perfect | 13:17 |
| elodilles | and all of them exist actually, but instead of py3 only, with py2.py3 wheels | 13:17 |
| fungi | yeah, those projects are apparently still configured to build universal wheels rather than py3-only | 13:17 |
| elodilles | so we should be all good | 13:19 |
| fungi | is approving https://review.opendev.org/878200 the next step? | 13:19 |
| elodilles | nope | 13:20 |
| Clark[m] | Fwiw swift does still support py27 which makes its wheel accurate | 13:20 |
| elodilles | next step is about 14:00 UTC (in 40 mins) this patch: https://review.opendev.org/c/openstack/openstack-manuals/+/877304 | 13:20 |
| fungi | ah, right | 13:20 |
| elodilles | according to our plans :) | 13:21 |
| elodilles | but we can merge that sooner if you wish | 13:21 |
| fungi | up to you, but if there's nothing else we need to look into then having the docs change in a little early gives us time to troubleshoot if something goes wrong or more time to deal with unexpected publication delays | 13:22 |
| hberaud | WFM | 13:23 |
| Clark[m] | It also accurately reflects the state that the release is basically done if you are at that point. Waiting a bit doesn't change this :) | 13:23 |
| elodilles | yepp :) | 13:23 |
| elodilles | removin -W then | 13:23 |
| fungi | may require a recheck if zuul doesn't enqueue it straight into the gate | 13:24 |
| elodilles | yepp, let's see | 13:24 |
| fungi | or a core reviewer on the manuals repo to add a new workflow +1 | 13:24 |
| elodilles | fungi: do you have +W right for that repo? :) | 13:26 |
| fungi | i do not | 13:26 |
| elodilles | so that we can avoid rechecking | 13:26 |
| elodilles | oh i see | 13:26 |
| fungi | otherwise i would have offered to | 13:26 |
| elodilles | maybe ttx ? :-o | 13:26 |
| elodilles | yepp ttx you have +W right | 13:27 |
| elodilles | ttx: for this: https://review.opendev.org/c/openstack/openstack-manuals/+/877304 | 13:27 |
| elodilles | :) | 13:27 |
| fungi | he said he expected to be afk until 14z | 13:27 |
| elodilles | oh, right | 13:28 |
| fungi | the longest check job on that change took 11 minutes | 13:28 |
| Clark[m] | fungi you can direct enqueue with zuul | 13:28 |
| fungi | true, i can do that | 13:28 |
| Clark[m] | As one option since all the necessary votes are present | 13:28 |
| Clark[m] | But also 11 minutes isn't that long if rechecking | 13:29 |
| elodilles | either that or the recheck - both works for me | 13:29 |
| fungi | done | 13:29 |
| elodilles | thx \o/ | 13:29 |
| fungi | zuul's estimating 12 minutes to merge, which seems in line with the check job durations | 13:31 |
| Clark[m] | It hasn't started two of the jobs yet so the estimate isn't accurate | 13:31 |
| fungi | also i approved 877553 to revert the semaphore removal since the release notes jobs all finished, and it's merged now | 13:32 |
| Clark[m] | And that is effective for all jobs starting after the merge which should make landing other release changes safe now | 13:32 |
| fungi | yep | 13:33 |
| elodilles | ack, thx! | 13:33 |
| ttx | I'm back, approving | 13:40 |
| fungi | ttx: it's already taken care of | 13:40 |
| ttx | ah | 13:41 |
| fungi | since it was already approved previously, i used zuul admin access to prod it to enqueue into the gate so it wouldn't have to go back through check | 13:41 |
| fungi | should be merging at any moment now | 13:42 |
| fungi | and there it goes, now waiting for promote-openstack-manuals* jobs in the promote pipeline | 13:44 |
| fungi | zuul's estimating those will complete around 14:05z | 13:44 |
| Clark[m] | Probably need to wait for the AFS vos release after that too? | 13:55 |
| fungi | yep | 13:56 |
| fungi | up to 5 minutes depending on how lucky we get | 13:56 |
| elodilles | after that comes this patch: https://review.opendev.org/c/openstack/releases/+/860029 | 14:03 |
| elodilles | feel free to review in advance ^^^ | 14:03 |
| Clark[m] | I think it is done. 14:05 vos release should get it? | 14:03 |
| elodilles | and the right link: https://review.opendev.org/c/openstack/releases/+/878200 | 14:04 |
| elodilles | -.- | 14:04 |
| clarkb | https://docs.openstack.org/2023.1.antelope/index.html is present but there is a bug. The site says that zed is still current | 14:08 |
| clarkb | I suspect this is a bug in the docs themselves sorting and determining whichrelease is most recent | 14:08 |
| fungi | or we're waiting for the 14:10 vos release | 14:08 |
| clarkb | I don't think so. It says antelope is maintained but old and the change we just landed sets that release to maintained | 14:08 |
| clarkb | I think previously it would've said "in development" instead of maintained | 14:09 |
| fungi | i agree, "2023-03-22 14:05:01,935 release INFO Releasing docs" should have caught it | 14:09 |
| fungi | i think it's a sorting issue, similar to the reno problem | 14:09 |
| clarkb | the issue is likely in the docs determining which release is latest probably because it alphanumerically sorts or something | 14:09 |
| fungi | yes, exactly | 14:10 |
| fungi | right now it's putting 2023.1 before (earlier than) austin in the drop-down | 14:10 |
| clarkb | ya there is a ton of sorted() calls in there | 14:11 |
| clarkb | we may just need to drop sorting and rely on an explicit ordering (can do that by converting SERIES_INFO to an ordered dict) | 14:12 |
| clarkb | I can work on that if this is helpful | 14:12 |
| clarkb | ya'll can keep focusing on the release process proper | 14:13 |
| elodilles | :S | 14:13 |
| elodilles | thanks clarkb | 14:13 |
| fungi | glad we merged the docs change a little early and got a head start on finding that | 14:22 |
| elodilles | yepp | 14:22 |
| elodilles | (this is the problematic line as I see: https://opendev.org/openstack/openstack-manuals/src/commit/507ed95344131f7233f0c92a2cca116dc948ebd4/tools/www-generator.py#L93 ) | 14:22 |
| fungi | yeah, i think so | 14:23 |
| elodilles | maybe reno's sort key trick can be reused here as well: https://review.opendev.org/c/openstack/reno/+/876581/3/reno/scanner.py | 14:26 |
| elodilles | clarkb: ^^^ | 14:26 |
| clarkb | remote: https://review.opendev.org/c/openstack/openstack-manuals/+/878227 Fix 2023.1 Antelope sort order [NEW] | 14:27 |
| clarkb | I don't think we need something as complicated as reno. We have an explicit list already we just need to stop putting it into a hashtable that loses that order | 14:28 |
| fungi | should probably go ahead with the finalize release change while the docs fix is being tested | 14:28 |
| elodilles | clarkb: ack | 14:28 |
| clarkb | the list is already sorted in openstack-manuals. The problem is that dict is not sorted | 14:28 |
| elodilles | thanks! | 14:28 |
| clarkb | fungi: ++ | 14:28 |
| elodilles | yepp: who wants to push the button? https://review.opendev.org/c/openstack/releases/+/878200 | 14:29 |
| hberaud | elodilles: honor to you | 14:31 |
| * elodilles shamelessly pressed the button | 14:34 | |
| elodilles | ttx: would you prepare the release announcement mails? | 14:36 |
| fungi | with all jobs except openstack-tox-validate now building, zuul's estimating 7 minutes (and otv is probably going to go very quickly since there are no new tags?) | 14:37 |
| elodilles | yepp, no new tags. releases.o.o needs to be refreshed though | 14:37 |
| opendevreview | Merged openstack/releases master: Mark 2023.1 Antelope as released https://review.opendev.org/c/openstack/releases/+/878200 | 14:44 |
| elodilles | \o/ | 14:46 |
| elodilles | just needs releases.o.o to refresh | 14:47 |
| clarkb | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_9cf/878227/1/check/build-tox-manuals-publishdocs/9cfb424/docs/zed/index.html now shows zed as old but maintained and | 14:47 |
| clarkb | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_9cf/878227/1/check/build-tox-manuals-publishdocs/9cfb424/docs/2023.1.antelope/index.html as current and supported. However when you hit the root of docs.openstack.org you are redirected to the latest release anyone know how that works? I'm not sure if my change addresses that | 14:47 |
| clarkb | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_9cf/878227/1/check/build-tox-manuals-publishdocs/9cfb424/docs/.htaccess I think this answers that question | 14:48 |
| clarkb | specifically redirectmatch 301 ^/$ /2023.1.antelope/ | 14:48 |
| fungi | yeah, in www/.htaccess | 14:48 |
| fungi | uses macros | 14:49 |
| fungi | set series = 'latest' | 14:49 |
| fungi | set projects = PROJECT_DATA[series] | 14:49 |
| clarkb | so ya I think my change is sufficient. There are other approaches we could take like sorted on the date value or sorting specially like reno. But considering this data was already manually curated and sorted I think doing the simple thing here and taking that order is best | 14:49 |
| fungi | i think it's there? | 14:49 |
| clarkb | fungi: yes I think the .htaccess has the correct redirects | 14:50 |
| clarkb | we just can't verify that as easily via the preview site because the swift hosting of the logs isn't doing htaccess redirects so have to manually examine the file | 14:50 |
| elodilles | clarkb: then it works! thanks, cool \o/ | 14:54 |
| elodilles | I guess we need now ttx for the patch to merge, right? https://review.opendev.org/c/openstack/openstack-manuals/+/878227 | 14:55 |
| fungi | publish-tox-docs-releases is running in release-post finally, zuul estimates completion at 15:03z and then we need to wait for the next 5-minute afs vos release pulse | 14:55 |
| fungi | so releases.o.o should hopefully be updated by 15:05z but yes we need someone with docs core privs to approve clarkb's https://review.opendev.org/c/openstack/openstack-manuals/+/878227 and also to send a release-announce e-mail | 14:56 |
| fungi | i gave the foundation marketing folks a heads up that things are running a few minutes behind | 14:59 |
| elodilles | ++ | 15:00 |
| clarkb | for docs core is that basically ttx ? | 15:00 |
| elodilles | or maybe TC members as well | 15:01 |
| fungi | many of them have irc highlights configured for tc-members | 15:01 |
| fungi | so that might get someone's attention now | 15:02 |
| ttx | in a meeting? need me to push a button? | 15:02 |
| fungi | ttx: https://review.opendev.org/c/openstack/openstack-manuals/+/878227 | 15:02 |
| fungi | clarkb's fix for the docs site release sorting | 15:03 |
| ttx | OrderedDict goodness ok +2a | 15:03 |
| fungi | thanks! | 15:03 |
| fungi | we're about 1 minute out from the next afs vos release and then the releases site should be updated | 15:04 |
| fungi | it's running now | 15:05 |
| fungi | and done. https://releases.openstack.org/ looks correct! | 15:05 |
| fungi | who was sending the release announcement to the mailing lists? | 15:06 |
| fungi | i'm on hand to approve the openstack-announce copy through moderation | 15:06 |
| elodilles | i hoped ttx would send it | 15:06 |
| fungi | sounds like he's in a meeting | 15:06 |
| elodilles | OK, then I'm preparing the mail | 15:06 |
| fungi | thanks! | 15:07 |
| ttx | yeah, meeting hell today | 15:07 |
| elodilles | okay, I've sent it to openstack-announce | 15:10 |
| elodilles | fungi: please check it | 15:10 |
| fungi | looking | 15:11 |
| elodilles | hberaud: can you forward the announce mail once it arrives, to openstack-discuss? o:) | 15:11 |
| fungi | elodilles: i don't see it in the moderation queue. are you subscribed? | 15:11 |
| hberaud | elodilles: what do you mean by forward? | 15:11 |
| elodilles | fungi: one moment, I'll send again from the *right* mail acc :/ | 15:12 |
| hberaud | :) | 15:12 |
| ttx | classic :) | 15:12 |
| elodilles | fungi: now you should see it | 15:14 |
| fungi | yes, i see it this time | 15:14 |
| fungi | lgtm, approving it now | 15:14 |
| elodilles | hberaud: like this one: https://lists.openstack.org/pipermail/openstack-discuss/2022-October/030735.html | 15:15 |
| hberaud | elodilles: you want me to forward it to openstack-discuss, exact? | 15:15 |
| fungi | it may take a few minutes to arrive in your inbox since there are lots of subscribers to openstack-announce | 15:16 |
| elodilles | I can see it now: https://lists.openstack.org/pipermail/openstack-announce/2023-March/002067.html | 15:16 |
| fungi | yes, it shows up in the archives as soon as it's approved out of moderation, just takes time to get delivered to everyone | 15:17 |
| fungi | no more than a few minutes though | 15:17 |
| elodilles | cool | 15:17 |
| hberaud | I'm ready to forward it | 15:17 |
| hberaud | that's good for you? | 15:18 |
| elodilles | hberaud: then shoot :) | 15:18 |
| elodilles | thanks in advance! :) | 15:18 |
| elodilles | teamwork :) | 15:18 |
| hberaud | done | 15:18 |
| elodilles | there it is: https://lists.openstack.org/pipermail/openstack-discuss/2023-March/032872.html | 15:19 |
| elodilles | so the mails are out, releases.o.o is refreshed, though we still have to wait a bit until docs.o.o refreshes, right? | 15:21 |
| fungi | yeah, it's uploading now | 15:22 |
| clarkb | and no waiting all day for release notes | 15:23 |
| fungi | will hopefully make the 15:25z vos release, otherwise almost certainly updated by 15:30z | 15:23 |
| clarkb | fungi: zuul says we are 17 minutes away | 15:23 |
| clarkb | probably more like the 15:40z or 15:45z vos releases | 15:24 |
| fungi | (currently still on synchronizing afs roots) | 15:24 |
| fungi | yeah, it may take longer at this step than i realize | 15:24 |
| fungi | it's a lot of data to copy | 15:24 |
| fungi | zuul says a previous run took 18 mins 10 secs on upload-afs-roots: Synchronize files to AFS | 15:26 |
| fungi | and it's been running for only 7.5 minutes so far | 15:26 |
| fungi | so yes, probably the 15:40z vos release after all | 15:27 |
| fungi | no, it just finished | 15:27 |
| clarkb | I see it still running | 15:27 |
| fungi | nevermind, it didn't | 15:27 |
| fungi | the console stream bounced for some reason and faked me out | 15:27 |
| gmann | elodilles: hi, let me know which patch you need review from tc side ? | 15:31 |
| fungi | gmann: it's done | 15:31 |
| fungi | we're just waiting on it to finish publishing now | 15:31 |
| gmann | fungi: ohk, thanks | 15:32 |
| fungi | (fix for release ordering on docs.o.o) | 15:32 |
| fungi | at the moment the docs site thinks 2023.1 comes before austin, but should be reflecting it as the latest release in a few minutes | 15:33 |
| gmann | this one ? https://review.opendev.org/c/openstack/openstack-manuals/+/878227 | 15:33 |
| fungi | yes | 15:33 |
| fungi | it's uploading the new docs build to afs, just takes about 20 minutes for that to complete | 15:34 |
| gmann | ack | 15:34 |
| fungi | our docs are not... small ;) | 15:35 |
| fungi | good problem to have i suppose | 15:35 |
| clarkb | should be vos releasing nowish? | 15:40 |
| fungi | 2023-03-22 15:40:02,046 release DEBUG Running: ssh -T -i /root/.ssh/id_vos_release vos_release@afs01.dfw.openstack.org -- vos release docs | 15:40 |
| clarkb | I see the new content but I seem to have cached the old redirect | 15:41 |
| fungi | 2023-03-22 15:40:47,782 release INFO Release of docs successful | 15:41 |
| clarkb | I think its done but someone who hasn't cached the old redirect will be able to confirm /me opens a different browser to check | 15:41 |
| clarkb | yup second browser lgtm | 15:41 |
| frickler | the issue here still persists https://docs.openstack.org/releasenotes/neutron-dynamic-routing/2023.1.html 21.0.0 is zed and should not appear for 2023.1, my assumption is that this is caused by the final commit in 21.0.0 being the one that adds these renos. but no idea how to fix after digging reno code | 15:42 |
| fungi | clarkb: i concur. taking me to https://docs.openstack.org/2023.1.antelope/ now | 15:42 |
| elodilles | frickler: stange, as other places where I checked this issue, already show correct versions :-o | 15:44 |
| frickler | elodilles: which "this issue"? this is not the 2023.1 < zed issue | 15:45 |
| fungi | it's "no 2023.1/antelope release but showing 2023.1 release notes" | 15:46 |
| frickler | my issue probably is "commits adding renos that get tagged/used as branching point are treated wrong" | 15:48 |
| elodilles | frickler: i thought you mean the 'all versions listed under 2023.1', but yes, your issue is different | 15:48 |
| JayF | When is a good time if I wanted to propose a .1 release of Ironic's Antelope deliverable? Don't wanna step on any work you all are doing, but would prefer new installs get the handful of bugfixes we've backported... | 15:48 |
| elodilles | JayF: 2023.1 is released, so should be safe | 15:49 |
| frickler | iirc there should be a 14d cooldown period after a release | 15:49 |
| JayF | *blink* | 15:50 |
| fungi | we've done immediate point releases in the past when there's a critical or security bug fixed between the final rc deadline and release day | 15:51 |
| JayF | I would say on a scale of 1-5, the fixes are like a 3. Not super critical, not security, but absolutely nice to have stuff which will make operator experience better | 15:53 |
| frickler | well my memory may be wrong or outdated, haven't been able to find any written reference to this in current docs. so don't let this stop you proposing anything | 15:58 |
| fungi | "Ironic, the OpenStack bare metal provisioning service and not the Alanis Morissette song, ..." https://thenewstack.io/a-is-for-openstack-antelope/ | 15:59 |
| ttx | lol | 15:59 |
| elodilles | :D | 16:00 |
| JayF | oh wow, you mean there's a song named after Ironic? I gotta go check that out | 16:26 |
| * elodilles feeling old right now :P :D | 16:38 | |
| JayF | elodilles: that is 1000000000% a joke. I promise Ironic devs have been making Alanis Morrisette jokes since before I was on the project :) | 16:41 |
| elodilles | :D :D :D | 16:41 |
| JayF | elodilles: my wife has a particular groan of frustration anytime I get a trivia question about that song (which is more often than seems possible) | 16:41 |
| elodilles | but still, did you know the song BEFORE you joined Ironic? :D | 16:42 |
| JayF | Let me put it this way, I was born in 1984. This is the music of my youth for sure. | 17:01 |
| elodilles | explanation accepted :) | 17:01 |
| elodilles | sorry o:) | 17:02 |
| JayF | lol no need to apologize | 17:02 |
| JayF | I've got one hand in my pocket, and the other one's giving elodilles a high-5 | 17:02 |
| elodilles | :) | 17:08 |
| *** amoralej is now known as amoralej|off | 18:10 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!