| clarkb | the smaller backup server just hit 90% utilization again. A good reminder we have a change up to retire some backups so they can be purged | 00:00 |
|---|---|---|
| clarkb | fungi: maybe we land the retirement first thing tomorrow, then manually prune (to exercise that with retired repos) then purge? | 00:00 |
| clarkb | that should get the backup server into shape | 00:00 |
| clarkb | anything people want me to add/remove/edit from the meeting agenda? | 00:01 |
| clarkb | I may have to work off of last weeks email if the wiki is too sad to do it now | 00:01 |
| clarkb | I'm glad we did the gerrit update today in order to have plenty of time to work through this | 00:03 |
| fungi | sounds good | 00:03 |
| clarkb | would've been annoying to do 3.9.7 -> 3.10.3 and hvae to debug this with a much bigger delta | 00:03 |
| fungi | indeed | 00:03 |
| clarkb | if anyone asks about the weirdness you can have then refresh the page they are on before logging in or just accept they have to nvaigate to wherever they want after loggign in | 00:04 |
| clarkb | fungi: should I drop the mm3 vhost renaming or do you want to recap? | 00:06 |
| fungi | clarkb: at this point i expect we can drop it | 00:06 |
| clarkb | Done | 00:07 |
| clarkb | or will be when I finish the other edits and post them | 00:08 |
| clarkb | anything else to add? I'm removing the us holiay announcement and the mm3 vhost renaming. Adding a bit more info the gerrit upgrade entry | 00:09 |
| clarkb | we've already got gitea upgrade on there | 00:09 |
| clarkb | and backups thats the bulk of my inflight stuff right now | 00:09 |
| clarkb | fungi: did we restart mailman services to pick up the verp_probe config update? I can't remember if that is manual or not | 00:16 |
| mordred | clarkb: I joined the gerrit discord ... but sadly the #general channel is weird in the matrix bridge :( | 00:17 |
| clarkb | mordred: ya you have to use discord at this point | 00:18 |
| mordred | yeah. it's sad - because the channels in in the "Gerrit" space work fine - just the three channels in the "General" space are hidden | 00:18 |
| clarkb | meeting agenda should hit inboxes momentarily | 00:18 |
| clarkb | mordred: wait some of the matrix bridge does work? | 00:19 |
| mordred | oh yeah | 00:19 |
| clarkb | liek to the cicd room? | 00:19 |
| clarkb | oh! | 00:19 |
| mordred | I use it for teh signalK discord very well | 00:19 |
| mordred | and yes - I can see the cicd room just fine | 00:19 |
| clarkb | mordred: can you share with corvus and I how to do that? | 00:19 |
| fungi | clarkb: i think the containers should be set to restart automagically when the configs update, but i'll need to check the deploy log | 00:19 |
| clarkb | we semi regularly get pinged with zuul questions there then I Have to play telephone between amtrix and discord | 00:19 |
| clarkb | fungi: I seem to recall we may have manually restarted for the queueing thing but can't recall if we made it atuomatic as a result | 00:20 |
| mordred | clarkb: I am running mautrix-discord v0.7.1 (Nov 16 2024, 16:26:41) | 00:20 |
| clarkb | mordred: oh you have your own bridge? | 00:20 |
| clarkb | maybe that is the solution though | 00:21 |
| mordred | yeah - there is a bridge bot that I had to login with using the "login-qr" command | 00:21 |
| mordred | then - I can join "guilds" via the web interface. once I have done that, the "guilds status" command will show me what guilds I have joined. And I can then, using that output, issue a command like guilds bridge 775374026587373568 --entire | 00:23 |
| mordred | which makes all the things | 00:23 |
| clarkb | they had a bridge set up for the general channel I wonder if moving it into this general space broke it | 00:24 |
| clarkb | but that is what we relied on preivously | 00:24 |
| mordred | oh - yeah - I now see the general room - it's in a different location | 00:25 |
| mordred | I just tried creating a distinct portal for it, but the bot told me it was already portaled | 00:26 |
| clarkb | if you figure out how to make that work without running an extra bridge that would be great but it also isn't super urgent I've just accepted I have to discord to interact with them | 00:26 |
| mordred | clarkb: I'm using a hosted homeserver from etke.cc - so having them add a bridge is the path of least resistance for me :) | 00:29 |
| fungi | RUNNING HANDLER [mailman3 : down containers] | 01:18 |
| fungi | it did re | 01:18 |
| fungi | er, did down and up all the containers because of the config change | 01:18 |
| fungi | so we should be all set therre | 01:19 |
| opendevreview | Vladimir Kozhukalov proposed zuul/zuul-jobs master: Respect image registry in container roles https://review.opendev.org/c/zuul/zuul-jobs/+/936909 | 03:44 |
| opendevreview | Rodolfo Alonso proposed openstack/project-config master: Update Neutron channel projects https://review.opendev.org/c/openstack/project-config/+/936918 | 08:37 |
| opendevreview | Dr. Jens Harbott proposed opendev/system-config master: Update IRC channel docs https://review.opendev.org/c/opendev/system-config/+/936806 | 11:34 |
| opendevreview | Merged opendev/system-config master: Update IRC channel docs https://review.opendev.org/c/opendev/system-config/+/936806 | 14:26 |
| opendevreview | Merged openstack/project-config master: Update Neutron channel projects https://review.opendev.org/c/openstack/project-config/+/936918 | 14:31 |
| karolinku[m] | clarkb: during further work on running centos 10 image on nested-virt label (for support x86_64-v3 I need at least Haswell), I discovered that probably AVX flags for them are turned off. I deducted that by analysing syslog[2... (full message at <https://matrix.org/oftc/media/v1/media/download/Afba9Ni75fekik2axi4BPsbXYQ_gUxVyYv2dClAiAf8RqoGO-opOU4AcV0viOAnqzb6bwdwa_XvOOzyZxjUf_qZCeT1QizCwAG1hdHJpeC5vcmcvaHRTd0JsdmRuamdyWVNlVUF4WUlWTFFJ>) | 14:32 |
| opendevreview | Merged openstack/project-config master: Add repository for OSA's httpd role https://review.opendev.org/c/openstack/project-config/+/935693 | 14:36 |
| frickler | karolinku[m]: note that your last message was cut off by the matrix bridge. when using that bridge, please make sure to only send single-line messages that are not too long | 14:38 |
| karolinku[m] | ok, let me check for alternative client | 14:46 |
| fungi | yeah, this is what people on the irc end see for excessively long messages over the matrix-oftc bridge: https://meetings.opendev.org/irclogs/%23opendev/%23opendev.2024-12-03.log.html#t2024-12-03T14:32:17 | 14:49 |
| fungi | which isn't the end of the world, but they do need to then use a web browser to find out what the entire message content was (if they're even willing to) | 14:50 |
| karolinku | I see. so here is a full message | 14:57 |
| karolinku | during further work on running centos 10 image on nested-virt label (for support x86_64-v3 I need at least Haswell), I discovered that probably AVX flags for them are turned off. I deducted that by analysing syslog[2] from job[1] and by checking console output from VM based on C10 image[3] saying ``` Deprecated Hardware is detected: x86_64-v2:GenuineIntel:Intel Core Processor (Haswell) will not be maintained(...)```. Could you take a | 14:58 |
| karolinku | look or advice how to enable those flags? | 14:58 |
| karolinku | [1] https://review.opendev.org/c/openstack/diskimage-builder/+/936024 | 14:58 |
| karolinku | [2] https://156c043409960ed94eb1-a1d06e2d26608a8f46317a106e743309.ssl.cf1.rackcdn.com/936024/9/check/dib-nodepool-functional-openstack-centos-10-stream-src/abfd4db/syslog | 14:58 |
| karolinku | [3] https://156c043409960ed94eb1-a1d06e2d26608a8f46317a106e743309.ssl.cf1.rackcdn.com/936024/9/check/dib-nodepool-functional-openstack-centos-10-stream-src/abfd4db/instances/797394c5-d01f-48df-8fa6-2a5e902d4053/console.log | 14:58 |
| fungi | the test nodes are virtual machines booted in openstack-based public cloud providers (rackspace, ovh, vexxhost, et cetera) so we don't control anything other than maybe what can be switched on by setting specific glance image create properties | 15:00 |
| fungi | i know some cloud providers will allow you to, e.g., switch between different supported i/o drivers by setting image properties, but i don't know if the same is true for cpu flags | 15:01 |
| karolinku[m] | is there any other label which would support ISA v3? | 15:03 |
| fungi | since cpu flag mismatches break live migration, cloud providers tend to be conservative about which ones they enable in case they might need to migrate to other compute nodes with different processors | 15:03 |
| frickler | IMHO if centos 10 continues to be so strict on CPU flags, they'll have issues on a lot of clouds. so I don't think it makes much sense to try to work around that just in our special case | 15:06 |
| fungi | if none of the nodes with the nested-virt label are presenting avx support for their processors, then probably not no. that's essentially our label for cloud providers who have newer hardware and use kvm (no xen) | 15:06 |
| fungi | yeah, basically it sounds like centos 10 is not made for running in openstack public clouds, if it won't run on any of the ones we have nodes in (which are some of the largest and most popular openstack cloud providers worldwide) | 15:07 |
| fungi | we have an essentially "private" cloud in openmetal, but 1. it's just one region so no redundancy/resiliency, 2. it's only got a handful of nodes worth of capacity, and 3. i wouldn't know how to go about turning on avx nor whether the processors we have there support it | 15:10 |
| fungi | (it's built with kolla-ansible, so we could probably figure it out) | 15:10 |
| karolinku | Centos 10 is close to RHEL 10, so i'm not sure if they willing to do rebuilds for v2 - again (Centos ISA SIG already did v2 and v3 for C9). that would need to be consulted | 15:17 |
| clarkb | karolinku: note the nested virt label was a shot in the dark that may have restricted you to nodes that have the necessary cpu flags I wasn't sure if it would or not. Seems not. I just don't want there to be an impression I was trying to say it would work | 15:48 |
| clarkb | I have two thoughts on the bigger high level problem. The first is that I really like the approach opensuse has taken. Which is to detect cpu compatibility and mix in packages that take advantage of that where they will have the most impact (python is one such case iirc). Now I understand arguments against doing this as well in that the mechanisms for this mixing in could be a | 15:49 |
| clarkb | security vulnerability surface? | 15:49 |
| clarkb | the other thought I've got is that this is not really our fight. If red hat wants to push a certain instruction set then they should coordinate with cloud providers to ensure it is available | 15:50 |
| clarkb | and maybe they have but not broadly enough or there is still a lack of info on how to take advantage of that | 15:50 |
| clarkb | basically this is between red hat and the clouds imo and we should take advantage of whatever comes out of that rather than solving it ourselves | 15:52 |
| clarkb | on tumbleweed the special packages have a -x86-64-v3 suffix for example python312-x86-64-v3 | 15:54 |
| clarkb | and they are automatically selected if hardware detection notes the hardware is capable | 15:54 |
| clarkb | infra-root https://issues.gerritcodereview.com/issues/381996067 is the upstream gerrit login weirdness issue | 15:55 |
| karolinku[m] | yes, but that's a decision I'm very far from :) im just trying to add support for c10 DIB, I didn't expect to end up with such serious infra disussions, I hoped for some labels which would support v3. Now I have more insight into infra situation, I'll check what's on Centos side. Thank you for all suggestions! | 15:59 |
| clarkb | karolinku[m]: you can identify the clouds that have failed in the job logs and collet that for feedback to the clouds and/or red hat | 16:02 |
| clarkb | as a side note linking to logs in zuul rather than the raw files makes them a bit more easy to navigate as you can link to specific lines | 16:03 |
| clarkb | karolinku[m]: https://zuul.opendev.org/t/openstack/build/abfd4db97bae4610be3b7b99f737acb7/log/job-output.txt#48 for example this is the provider from the logs you linked above | 16:05 |
| karolinku[m] | sure. can I somehow browse though the zuul labels to check if there could be any kind of suitable hardware? (I know where are labels in a code, but specification behind them?) | 16:06 |
| clarkb | karolinku[m]: not on our side because that is determined by the clouds as fungi noted above | 16:07 |
| clarkb | https://docs.opendev.org/opendev/infra-manual/latest/testing.html this is what we have documented | 16:07 |
| fungi | Ramereth: i've heard through the grapevine that nvidia has funded a gpu-heavy supercomputer at osu, any idea if that's openstack-based or some other platform? | 16:58 |
| clarkb | infra-root fyi I haev rechecked https://review.opendev.org/c/opendev/system-config/+/893571 and will put a new hold in place for the 3.9 job there in order to try and test https://gerrit-review.googlesource.com/c/gerrit/+/444801 this is still just a workaround trying to get the initial login to a page that isn't "broken" but shouldn't redirect us ot the page we started on :/ | 17:00 |
| clarkb | I think I'll still need to reconfigure that server to do openid login or something but I'll test with the normal logins first as it may be reproduceable that way too | 17:01 |
| clarkb | infra-root I'd also like to land https://review.opendev.org/c/opendev/system-config/+/936203 if this change checks out for another reviewer to retire old backups on the vexxhost backup server. Once that has applied we can manually run the prune script there to trim down the size of things and confirm the retirement handling works | 17:02 |
| clarkb | and once that is done we can do a purge of those retired repos to further reduce the disk consumption on that server | 17:02 |
| fungi | i've approved 936203 just now | 17:04 |
| fungi | thought i had already reviewed that one, but i guess not | 17:04 |
| fungi | ah, i had only reviewed the initial ethercalc entry | 17:04 |
| fungi | and then tested it | 17:04 |
| clarkb | https://review.opendev.org/c/opendev/lodgeit/+/935712/ is another one that would be good to land. It does have the +2's needed but I've been half treating my +2 as a +1 since I've been helping fixup tests and adding a test via system-config to capture the captcha rendering | 17:05 |
| fungi | i'm happy to do the manual steps on the backup server once it deploys | 17:05 |
| clarkb | fungi: thanks! | 17:05 |
| clarkb | 935712 is the lodgeit captcha rendering fix | 17:05 |
| clarkb | I'm hoping that this afternoon I'll go through the 3.9 to 3.10 upgrade and downgrade process again with our newly built images too | 17:09 |
| clarkb | I'll update the upgrade etherpad with any changed details there though I don't expect there to be much if any | 17:09 |
| fungi | there are a couple of other outstanding changes i've been meaning to revisit... | 17:21 |
| fungi | https://review.opendev.org/c/opendev/git-review/+/934745 Support running on systems setting GIT_SSH_COMMAND environment variable | 17:21 |
| fungi | https://review.opendev.org/c/opendev/system-config/+/927462 Update openEuler mirror repo | 17:22 |
| fungi | i think the latter is going to need us to address the full quota issue with the volume though | 17:22 |
| opendevreview | Merged opendev/system-config master: Retire more backups on the vexxhost backup server https://review.opendev.org/c/opendev/system-config/+/936203 | 17:23 |
| clarkb | fungi: for 927462 it starts with a delete existing content step | 17:23 |
| clarkb | fungi: if we think the new content isn't any bigger than our quota that may work | 17:23 |
| clarkb | fungi: 936203 didn't trigger the backup jobs. I believe this is a file matcher problem. I will work on a fix | 17:34 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Trigger infra-prod-service-borg-backup when vars update https://review.opendev.org/c/opendev/system-config/+/936965 | 17:40 |
| clarkb | fungi: ^ that should get things updating before the periodic jobs run in ~8 hours | 17:40 |
| fungi | thanks@ | 17:42 |
| fungi | s/@/!/ | 17:42 |
| clarkb | fungi: I +2'd https://review.opendev.org/c/opendev/system-config/+/927462 since it seems worth a try to see if deleting old content first is sufficient to get new content to fit | 17:53 |
| fungi | thanks, i'll do the sync manually once it deploys | 17:55 |
| opendevreview | Merged opendev/lodgeit master: Captcha: Fix text layer location calculation https://review.opendev.org/c/opendev/lodgeit/+/935712 | 17:55 |
| fungi | i've got the openeuler-mirror held in a root screen session on mirror-update | 17:56 |
| fungi | er, the openeuler-mirror lock held | 17:56 |
| clarkb | fungi: I left a comment on https://review.opendev.org/c/opendev/git-review/+/934745 with a concern and a potential solution if that concern has merit | 18:00 |
| fungi | yeah, since i don't override ssh behavior with either GIT_SSH_COMMAND or GIT_SSH i'm unclear on the possible use cases for doing so | 18:07 |
| fungi | the release note for addition of GIT_SSH support seemed to indicate it was universal, but as you observed it's only getting used for query commands | 18:08 |
| clarkb | I suspect that 99% of the time the prurpose is to use a proxy or specific keys or something in which case overloading the env var is probably fine | 18:08 |
| clarkb | fungi: well I think when git-review invokes git commands git is honoring the env var for us | 18:09 |
| clarkb | so its only where git-review explicitly runs ssh that it also needs to do similar | 18:09 |
| clarkb | and git review is a subcommand of git technically so honoring the env var makes sense. I just wanted to make sure we don't create an inverse problem | 18:10 |
| fungi | that's a good point, that may be the only place git-review directly invokes ssh | 18:10 |
| opendevreview | Merged opendev/system-config master: Update openEuler mirror repo https://review.opendev.org/c/opendev/system-config/+/927462 | 18:33 |
| opendevreview | Merged opendev/system-config master: Trigger infra-prod-service-borg-backup when vars update https://review.opendev.org/c/opendev/system-config/+/936965 | 18:33 |
| clarkb | fungi: the mirror job completed | 18:37 |
| clarkb | and the backup job has queued up for 936965 as expected | 18:39 |
| fungi | thanks, i have the openeuler-mirror-update script running in a root screen session now | 18:40 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/diskimage-builder master: Add support for building Fedora 40 https://review.opendev.org/c/openstack/diskimage-builder/+/922109 | 18:45 |
| fungi | i notice the openeuler mirror script is pulling riscv64 packages, those could presumably be filtered | 19:23 |
| fungi | EPOL/main/riscv64/* specifically | 19:24 |
| clarkb | yes they should imo | 19:25 |
| fungi | backup change deploy succeeded, i'm starting a prune in a root screen session on backup02.ca-ymq-1.vexxhost | 19:40 |
| clarkb | fungi: did you check for retired stamp files in the backups? | 19:40 |
| clarkb | just to confirm the retirement action occurred as expected (I'm sure it did) | 19:40 |
| clarkb | the borg-ask01 entry got the stamp file so I think we're good | 19:41 |
| fungi | -rw-r--r-- 1 borg-ask01 borg-ask01 0 Dec 3 18:50 /opt/backups-202010/borg-ask01/.retired | 19:41 |
| fungi | yup | 19:41 |
| fungi | i can do the lodgeit pull/restart now too | 19:42 |
| clarkb | thanks! | 19:42 |
| fungi | opendevorg/lodgeit latest sha256:bc8154e427fe127f3e91347ded28faf014bc0bef4a2e16ded99ce6c1923c7137 2 hours ago 260MB | 19:44 |
| fungi | opendevorg/lodgeit <none> sha256:34b9cbb9cb52842a1f8c9de46adfe09470ec8e1274f50c1444bc826f85f7e13c 6 days ago 260MB | 19:44 |
| fungi | that's after a pull | 19:44 |
| clarkb | looks about right the 6 day old image is what we got for updating python testing for 3.11 onto jammy instead of default noble | 19:44 |
| clarkb | and then this most recent change to fix the captcha built on top of that | 19:45 |
| fungi | #status log Restarted the lodgeit container on paste.opendev.org to pick up a captcha rendering fix | 19:45 |
| fungi | captcha's are working for me when i try to paste a url | 19:46 |
| clarkb | excellent | 19:46 |
| opendevstatus | fungi: finished logging | 19:46 |
| clarkb | https://review.opendev.org/c/opendev/system-config/+/936297 was the related chagne used to test that | 19:46 |
| fungi | i'm going to pop out for a quick bite and then check back on the openeuler mirror progress as well as the results of the backup prune | 19:47 |
| clarkb | its not super important to have long term I expect btu also doesn't hurt and is a nice use of the test framework tooling | 19:47 |
| clarkb | ya I too need lunch | 19:47 |
| clarkb | looks like pruning got us down to 68% disk utilization | 20:28 |
| clarkb | so retiring and pruning does help over just retiring (thats all the delta content we don't need for the additional backups) | 20:29 |
| clarkb | fungi: if you confirm the pruning output looks correct then the next step is pruging those repos. I can push a change for that when we think we are ready | 20:29 |
| clarkb | does anyone know how to convert a gerrit install from the login type of development (in use on our held nodes) to openid? I updated the config and restarted gerrit but after authing with login.ubuntu.com I get a not found error. Maybe I need to rerun the init step? | 20:55 |
| * clarkb will try that next | 20:55 | |
| clarkb | oh wait maybe the problem is it tried ot redirect me back to review.opendev.org | 20:57 |
| clarkb | and then that login pass wasn't found | 20:58 |
| clarkb | I think that may be it and I need to edit /etc/hosts locally ugh | 20:58 |
| fungi | dropped us from 91% to 68% used on that volume | 21:01 |
| fungi | previous runs were only getting down to 77% used, so this was a fairly drastic improvement | 21:02 |
| clarkb | /etc/hosts was the ticket | 21:03 |
| clarkb | I get the same behavior with paladox's fix in place. I'm now going to try modifying the apache reverse proxy rules | 21:04 |
| clarkb | and see if we're appending an unwanted / there | 21:04 |
| clarkb | (this is all on my held test node so shouldn't affect anything) | 21:04 |
| clarkb | removing the trailing / from the proxypassreverse role does not chagne the behavior | 21:09 |
| fungi | good to know | 21:10 |
| clarkb | unfortunately I think this means that paladox's workaround isn't good enough for us | 21:10 |
| clarkb | we could potentially solve this how fungi suggested though by rewriting // to just / until we understand it better? | 21:10 |
| clarkb | but I think we can avoid that and keep poking at the code to figure this out properly | 21:10 |
| fungi | #status log Pruned backups on backup02.ca-ymq-1.vexxhost.opendev.org reducing volume utilization from 91% to 77% following addition of the server retirement change | 21:10 |
| clarkb | fungi: 68% ! | 21:11 |
| fungi | #status log CORRECTION! Pruned backups on backup02.ca-ymq-1.vexxhost.opendev.org reducing volume utilization from 91% to _68%_ following addition of the server retirement change | 21:11 |
| opendevstatus | fungi: finished logging | 21:11 |
| fungi | that's what i get for copying and pasting the wrong bits | 21:12 |
| clarkb | for the record for anyone looking at logs wanting to test this kind of thing in the future, gerrit supplies our canonical url to the openid provider which means you get redirected to the canonical url after successfully authenticating. This means you have to modify /etc/hosts if setting up a test gerrit with openid to point review.opendev.org at your test server | 21:12 |
| clarkb | but once you do that it seems to work just fine | 21:12 |
| opendevstatus | fungi: finished logging | 21:13 |
| fungi | openeuler update is still in progress | 21:16 |
| clarkb | "good" news is I don't think paladox's change makes anything worse etiehr | 21:17 |
| clarkb | so we can safely update to that cleanup (which it does seem to be a reasonable cleanup) and be fine | 21:17 |
| fungi | https://grafana.opendev.org/d/9871b26303/afs shows volume utilization is very low compared to quota, at this point, but i don't know how far or near completion it is | 21:18 |
| clarkb | I think I see why openid is different for upstream | 21:28 |
| clarkb | https://gerrit.googlesource.com/gerrit/+/refs/tags/v3.9.8/java/com/google/gerrit/httpd/auth/openid/OpenIdServiceImpl.java#478 this defaults us to a token = PageLinks.MINE (that constant is defined as '/') then at the end of the method we append the result of that token to rdr. I believe the append method there is doing path appends and separating things with /s | 21:30 |
| paladox | interesting problem you have. I couldn't reproduce the error you had locally but decided to strip the %2F. | 21:35 |
| clarkb | no that append method is for StringBuilder I think what happens when we do /login is we create an empty rdr then append and empty req path to that. Then at the end we append a single / from that default I noted and it works. If we do /login/ then we append empty string and / together then finally at the end append another / | 21:35 |
| clarkb | paladox: the problem is reproduceable in upstream gerrit through | 21:35 |
| clarkb | paladox: and also in our systems using local logins | 21:35 |
| clarkb | specifically the problem with the sign in links that lead to the behavior we observe. But I think if you don't use the openid auth system then you don't get the callback method above which appends extra stuff | 21:36 |
| clarkb | so you just redirect to your /dashboard/self instead | 21:36 |
| clarkb | which I think is still broken because if you refresh the page the sign in links update and then you redirect back to where you logged in from | 21:37 |
| paladox | I just logged in but didn't get the problem you had. Or is it expected it goes to /dashboard? | 21:37 |
| clarkb | paladox: if you go to gerrit.server/c/org/project/+/123456 and look at the sign in link it will be to gerrit.server/login/%2F or gerrit.server/login/ after your change. This is buggy it should go to gerrit.server/login//c/org/project/+/123456 | 21:38 |
| clarkb | paladox: ^ that issue is 100% observable in every gerrit I have checked newer than 3.9.8 | 21:38 |
| clarkb | if you refresh the chagne path then the sign in link updates and is correct | 21:39 |
| clarkb | so thats bug one the sign in links are non consistent and they don't point you back to where you start when you log in | 21:39 |
| clarkb | then separately I think there is another issue in the openid provider where it over appends an extra / when you start at / | 21:40 |
| paladox | I went to https://gerrit-review.googlesource.com/c/gerrit/+/444763 directly and logged in. Seemed to work? | 21:40 |
| clarkb | paladox: yes login works but it does so in a wrong way. After you login there it redirects you to /dashboard/self not /c/gerrit/+/444763 | 21:40 |
| paladox | https://imgur.com/a/WR6hqpX | 21:40 |
| clarkb | paladox: if you refresh the page at /c/gerrit/+/444763 before clicking sign in then it works as epxected | 21:40 |
| paladox | it redirected me successfully. | 21:40 |
| clarkb | paladox: did you naviatge to that change by typing in the url completely and hitting go? | 21:41 |
| paladox | yeh | 21:41 |
| clarkb | I think that also acts like a refresh. But if you navigate there from a dashboard or whatever then it is wrong | 21:41 |
| paladox | I could only reproduce when I do https://gerrit-review.googlesource.com/ and it going to /dashboard | 21:41 |
| paladox | I filed https://issues.gerritcodereview.com/issues/381996067 for that problem | 21:42 |
| clarkb | if you start at https://gerrit-review.googlesource.com/ you get redirected to https://gerrit-review.googlesource.com/q/status:open+-is:wip then if you click on one of the changes tehre you go to https://gerrit-review.googlesource.com/c/git-repo/+/444841 for example nd the sign in link is https://gerrit-review.googlesource.com/login/%2F | 21:42 |
| paladox | OH | 21:42 |
| clarkb | I just went through all of that in a new incognito tab in crhome to make sure I wasn't doing it wrong | 21:42 |
| paladox | ok I see the problem | 21:43 |
| clarkb | my hunch is that the openid extra / if going to / has always been there or has been there for a long time | 21:43 |
| paladox | guess it's the same problem as I reported in the issue | 21:43 |
| clarkb | the problem is that we notice it now because the sign in links are wrong | 21:43 |
| clarkb | if the sign in links are wrong then you redirect to /q/status:open+-is:wip or /c/git-repo/+/444841 and oepnid handles those properly | 21:43 |
| paladox | although that one has href= | 21:44 |
| clarkb | this sign in link problem also exists if you go to https://gerrit-review.googlesource.com/ and get redirected to https://gerrit-review.googlesource.com/q/status:open+-is:wip the sign in link is for /login/%2F not login/q/status:open+-is:wip | 21:44 |
| clarkb | upstream and in developer mode you don't really notice this unless you are surprised to have been redirected away from the page you originated on because it sends you to your /dashboard/self page | 21:45 |
| clarkb | because we use openid it sends us to // and there is nothing there and its weird | 21:45 |
| clarkb | and I feel fairly confident the sign in links are wrong because if you directly enter the full path to a url and hit enter and don't get redirected the sign in link is different. Or if you refresh one of those pages after being redirected to them | 21:46 |
| clarkb | basically at least one or the other is wrong and personally the old behavior and post refresh behavior makes more sense to me which is to return you to where you started from | 21:46 |
| clarkb | then separaetly there is the issue in the openid auth driver but I think we can ignore that if we fix the sign in links | 21:47 |
| clarkb | paladox: hopefully all that rambling makes sense? I'm going to try and pop out for some exercise while the sun is out and it isn't absolutely freezing outside but I'll be around and can help with more info/debugging if it is useful after (or tomorrow etc) | 21:50 |
| paladox | yeh it does, it's basically what I reported above | 21:51 |
| paladox | I just dunno how you fix it. | 21:51 |
| clarkb | paladox: ack I'll try to respodn to there pointing out that there is probably a second issue with openid that makes it worse for us and also that your workaround doesn't address it in the openid case as a result | 21:51 |
| clarkb | I can also add the addition of navigating to a change page from the change listing you get redirected to has the same problem since it isn't just the list pages. I think the code for all of that is shared in some header typescript thing though | 21:53 |
| clarkb | so fixing it for one will likely fix it for all | 21:53 |
| paladox | ok this is just weird | 21:55 |
| fungi | well, other than reverting the url handling refactor anyway | 21:55 |
| paladox | I can see the query in pathname | 21:55 |
| paladox | but then I use it, it just returns either the base url or / (for non base urls) | 21:55 |
| paladox | https://imgur.com/a/Xf0aPT3 | 21:56 |
| fungi | and i think we established that the fedora gerrit is old enough (for now) that it doesn't exhibit this behavior | 21:56 |
| fungi | if you need one for direct comparison | 21:56 |
| paladox | ahhhhhh | 21:58 |
| paladox | think I maybe have found the fix | 21:58 |
| paladox | oh | 21:59 |
| paladox | nope :/ | 21:59 |
| clarkb | ya its 3.7.something | 22:02 |
| clarkb | ok popping out now. Back in a bit | 22:02 |
| fungi | as for openeuler mirror progress, looks like the top-level WSL, edge_img and embedded_img trees could also be pruned | 22:31 |
| fungi | still only using about 18% of the volume quota though | 22:33 |
| paladox | issue is using page.js redirect(). Which also explains why clicking on a change even tho it has href set on <a> has the problem. | 22:51 |
| paladox | https://www.irccloud.com/pastebin/W8b8BZcW/ | 22:51 |
| clarkb | paladox: do you still want me to udpate the issue? | 23:33 |
| paladox | sure. | 23:33 |
| clarkb | ack I'm working on a hacky openid debugging patch that will also attempt to trim the trailing / if there are two at the end | 23:44 |
| clarkb | once that is pushed up in some sort of fashion I'll update the issue too | 23:45 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!