| *** ykarel_ is now known as ykarel | 05:17 | |
| opendevreview | Merged openstack/releases master: Stable branch release for bobcat designate https://review.opendev.org/c/openstack/releases/+/947737 | 09:00 |
|---|---|---|
| opendevreview | Merged openstack/releases master: Release octavia-dashboard stable/2023.2 bugfix https://review.opendev.org/c/openstack/releases/+/947261 | 09:01 |
| opendevreview | Merged openstack/releases master: Release octavia-lib stable/2023.2 bugfix https://review.opendev.org/c/openstack/releases/+/947677 | 09:01 |
| opendevreview | Merged openstack/releases master: Release openstackdocstheme 3.5.0 https://review.opendev.org/c/openstack/releases/+/946535 | 10:54 |
| noonedeadpunk | hey folks! Any wild guess, why OSA roles are not having ussuri-eol tag on them? ie: https://opendev.org/openstack/openstack-ansible-galera_server/tags?q=ussuri-eol | 13:37 |
| noonedeadpunk | but: https://opendev.org/openstack/releases/src/commit/ded72cb946325f11656eee78a39be6adffb10c62/deliverables/ussuri/openstack-ansible-roles.yaml#L156-L157 | 13:37 |
| noonedeadpunk | (It could be I already asked this one ofc) | 13:38 |
| noonedeadpunk | but https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/851439 says it was included in ussuri-eol in gerrit... | 13:39 |
| noonedeadpunk | ok, yes, I asked that... It's some gitea issue... | 13:39 |
| noonedeadpunk | as tag in there for github... | 13:40 |
| noonedeadpunk | disregard | 13:40 |
| opendevreview | ribaudr proposed openstack/releases master: Add Nova deadlines to the official schedule https://review.opendev.org/c/openstack/releases/+/948077 | 13:41 |
| elodilles | noonedeadpunk: fwiw the above opendev.org link shows me the ussuri-eol tag. and yes, there were out of sync gitea instance issues recently as i remember but infra team solved that for us :/ | 14:20 |
| noonedeadpunk | It shows `No results found.` - doh | 14:22 |
| noonedeadpunk | (for me) | 14:22 |
| clarkb | noonedeadpunk: I don't understand https://opendev.org/openstack/openstack-ansible-galera_server/src/tag/ussuri-eol/ exists | 15:25 |
| clarkb | oh that is what elodilles said too. Maybe it is abcked specific let me check | 15:25 |
| clarkb | no all 6 backeds render that url just fine | 15:26 |
| clarkb | you can check yourself using gitea09.opendev.org:3081 through gitea14.opendev.org:3081 as the fqdn portion of the url | 15:26 |
| clarkb | I can't type backend apparently | 15:27 |
| clarkb | noonedeadpunk: elodilles there were two issues at one time giving us trouble with tasg like that. The first was database case insensitivity where collisions would break the web ui but the git repo would have the content if you fetched with git. The other was a memory leak that would lead to OOMs that could lead to things not replicating properly. Gitea fixed the memory leak | 15:28 |
| clarkb | but it is possible we may still have some content that needs to be force replicated to catch up (though I thought we did global force replication since then so shouldn't need that) | 15:28 |
| clarkb | if you do find a current example that would be helpful for us to debug and/or force replicate | 15:28 |
| noonedeadpunk | ok, it opens for me now as well | 15:31 |
| elodilles | clarkb: ACK, will report if i see such issue (to me gitea (gitea14.opendev.org; checked via certificate's common name) shows the ussuri-url for the above link) | 15:31 |
| noonedeadpunk | I actually check couple of repos - like openstack/openstack-ansible-haproxy_server was also missing the tag | 15:32 |
| * noonedeadpunk forgot how to check the backend - tried to see response headers, but there was no info | 15:32 | |
| noonedeadpunk | 09-14 working nicely now... | 15:33 |
| clarkb | noonedeadpunk: if you inspect the cert the altnames include the backed you are talking to. But then you can do it explicitly via those fqdn and ports I provided | 15:33 |
| noonedeadpunk | yeah, I recalled now... | 15:34 |
| noonedeadpunk | (once elodilles wrote that :D) | 15:34 |
| noonedeadpunk | ok, wait... | 15:35 |
| noonedeadpunk | so tag does exist. But it is not show in drop down or explicitl list of tags | 15:36 |
| noonedeadpunk | I can recall that same thing but we concluded it was some kind of weird bug | 15:36 |
| noonedeadpunk | so you can not navigate to gitea with it, unless you just use URL | 15:37 |
| clarkb | can you link to where you see that. For both of your repos I used teh dropdown on the main project page to select ussuri-eol on gitea09 iirc | 15:37 |
| clarkb | https://gitea09.opendev.org:3081/openstack/openstack-ansible-haproxy_server/ ok no it doesn't show up in the dropdown here | 15:38 |
| clarkb | https://gitea14.opendev.org:3081/openstack/openstack-ansible-haproxy_server/# it does show up here | 15:38 |
| noonedeadpunk | neither for https://gitea09.opendev.org:3081/openstack/openstack-ansible-galera_server/ | 15:38 |
| noonedeadpunk | yeah, right. Ok, I was looking through the wrong thing.. Let me double-check | 15:39 |
| clarkb | openstack/openstack-ansible-haproxy_server lacks it on 09-11 has it on 12-14 in the dropdown | 15:39 |
| clarkb | so I think this is a new issue | 15:39 |
| noonedeadpunk | It's there on 10 and 11? | 15:40 |
| clarkb | or at least a different one to the ones I noted above since we confirmed the tag is present and rendersif you navigate to it (so replication worked) | 15:40 |
| clarkb | noonedeadpunk: oh bah I was searching under branches not tags for some of them | 15:40 |
| noonedeadpunk | so only 09 seems to be missing | 15:40 |
| clarkb | ya | 15:40 |
| elodilles | yepp, here i cannot see the tag either: https://gitea09.opendev.org:3081/openstack/openstack-ansible-haproxy_server/tags?q=ussuri-eol | 15:40 |
| clarkb | my hunch is that populating that field is a db table and it requires gitea to notice it on push (eg it doesn't scan the repo later) and the oom problem may have caused that to not populate | 15:41 |
| noonedeadpunk | so the link to verify more reliably: https://gitea11.opendev.org:3081/openstack/openstack-ansible-galera_server/tags?q=ussuri-eol | 15:41 |
| clarkb | there may be a way to force gitea to rescan the repos and repopulate the db tables | 15:42 |
| clarkb | but I'm not sure | 15:42 |
| clarkb | https://docs.gitea.com/next/help/faq#missing-releases-after-migrating-repository-with-tags we might be able to try something like this | 15:45 |
| clarkb | we don't use the releases functionality in gitea, but maybe that side effects similar parts of the database | 15:45 |
| clarkb | nice that command is completely undocumented in https://docs.gitea.com/administration/command-line#admin | 15:46 |
| noonedeadpunk | undocumented commands are always nice to have :p | 15:49 |
| clarkb | internally that calls repo_module.SyncReleasesWithTags() which I guess I need to read to understand better | 15:53 |
| clarkb | ok I think that might work it does gitRepo.WalkReferences() passing in an anonymous function that calls PushUpdateAddTag(). PushUpdateAddTag is what gets called when tags are pushed so in theory it udpates all the necessary db bits | 15:56 |
| clarkb | probably the next step here is to hold a test node and populate that gitea with a non trivial repo and call that admin command against it just to double check there is no unexpected side effects | 15:57 |
| clarkb | in particular I think we disabled releases so this command might fail early or enable them when we don't want them | 15:57 |
| clarkb | but we just need to call PushUpdateAddTag() against all the tags | 15:57 |
| clarkb | noonedeadpunk: just to finish up ^ for now I've pushed a change that will allow me to hold a node and we can try to test this | 16:11 |
| clarkb | not sure when I'll actually get to testing it but havingthe auto hold in place is a good reminder for me | 16:11 |
| noonedeadpunk | yeah, I guess there is no real urgency, but good to be aware | 16:15 |
| clarkb | ya this is largely a UX issue. Not ideal but the data in the repo appears to be correct and present | 16:16 |
| clarkb | and if you know how to navigate to it you can bypass the ux issue (or use git etc) | 16:16 |
| clarkb | I did more digging in the gitea codebase and the admin dashboard has a sync repo tags function | 17:15 |
| clarkb | I think this is actually what we want | 17:15 |
| clarkb | it says "Sync tags from git data to database" | 17:16 |
| clarkb | so I'll try that out on the test node | 17:16 |
| clarkb | and I think that may be a function that looks at every repo and every tag so we don't need to iterate though the repos. That may be a good thing (less interactive work) or a bad thing (slows down gitea for a time) | 17:18 |
| clarkb | fungi: frickler: ^ if we are worried we can probably run that with the backend pulled out of haproxy and during a quieter time of day (like PST afternoon or something) | 17:18 |
| clarkb | but first let me test it on the held node | 17:18 |
| sean-k-mooney | o/ am i miss rememebring or does our release tooling not like if we do multiple releases in one go? | 18:13 |
| sean-k-mooney | i probably shoudl also just do them as seperate comits for review | 18:13 |
| sean-k-mooney | just confirming that i cant do multiple in one patch | 18:14 |
| sean-k-mooney | context: i was considering doing one patch for the eol tag for bobcat for all the watcher repos in one commit | 18:15 |
| sean-k-mooney | sepreatly im also looking at doign a release for other stable branches next week after i review who many comits are unrelease for each repo to see if it worth it or if we shoudl wait till m1 | 18:17 |
| clarkb | sean-k-mooney: https://review.opendev.org/c/openstack/releases/+/937723 seems you can do a bunch in one go | 18:17 |
| sean-k-mooney | oh ok | 18:18 |
| sean-k-mooney | for the other release it makes sense to have the summery but for the eol tagging i think comining them makes sense | 18:18 |
| fungi | clarkb: i'm not too worried about clicking that button whenever. we might want to do it on every backend just in case different repos are impacted on some of the others? | 18:19 |
| clarkb | fungi: yup I think if it makes things better on 09 we click it elsewhere | 18:23 |
| fungi | agreed | 18:29 |
| sean-k-mooney | general question what the advice for tempet plugins i.e. where you only have a master branch | 18:36 |
| sean-k-mooney | for the bobcat eol shoudl i just ignore the plugin | 18:36 |
| sean-k-mooney | or should i tag the current commit of master that is being used to test the bobcat version when it was eoled | 18:37 |
| sean-k-mooney | i feel like doing a release of it is kind of overkill but i dont know what is done normally | 18:37 |
| clarkb | I think tempest proper tags a commit when it is no longer able to function against the target branch | 18:41 |
| clarkb | looks like gman isn't here to confirm that | 18:41 |
| opendevreview | sean mooney proposed openstack/releases master: eol the bobcat release of watcher deliverables https://review.opendev.org/c/openstack/releases/+/948118 | 18:42 |
| sean-k-mooney | i belive they are back form pto on monday | 18:42 |
| sean-k-mooney | we are not in a rush but i think ^ woudl be correct | 18:42 |
| sean-k-mooney | ill add them to the revew and check in with them on monday | 18:43 |
| frickler | sean's gone but no need to create eol patches on their own, elod will do all of them in a bunch next week https://etherpad.opendev.org/p/flamingo-relmgt-tracking#L60 | 19:16 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!