Thursday, 2022-01-20

-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 823744: just for debuggin https://review.opendev.org/c/zuul/zuul/+/82374402:17
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 823744: just for debuggin https://review.opendev.org/c/zuul/zuul/+/82374402:36
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 823744: just for debuggin https://review.opendev.org/c/zuul/zuul/+/82374402:55
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 823744: just for debuggin https://review.opendev.org/c/zuul/zuul/+/82374403:13
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 823744: just for debuggin https://review.opendev.org/c/zuul/zuul/+/82374403:47
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 823744: just for debuggin https://review.opendev.org/c/zuul/zuul/+/82374404:19
@felixedel:matrix.orgcorvus: Thanks for the backwards-compatibility fix (825349). I've left a comment on this change (although it's already merged), because I think the additional check for affected_projects is needed as well. Until yesterday I thought that this check you mentioned in your comment is enough, but I didn't think about re-serializing an old FrozenJob.06:23
-@gerrit:opendev.org- Benjamin Schanzel proposed: [zuul/nodepool] 825426: Add tests for exporting tenant limit stats https://review.opendev.org/c/zuul/nodepool/+/82542606:27
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 823744: just for debuggin https://review.opendev.org/c/zuul/zuul/+/82374407:20
-@gerrit:opendev.org- Dominik Jaeger proposed: [zuul/zuul] 823317: Make ZooKeeper wait for certs in test environment https://review.opendev.org/c/zuul/zuul/+/82331709:38
-@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 825480: Revert "Fix a bug in getting changed files" https://review.opendev.org/c/zuul/zuul/+/82548010:36
-@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 825480: Revert "Fix a bug in getting changed files" https://review.opendev.org/c/zuul/zuul/+/82548010:42
@tobias.henkel:matrix.orgcorvus, Clark this is a quite important one since currently the fileschanges call to the merger is broken ^10:45
-@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 825482: Initialize buildset files state correctly on reset https://review.opendev.org/c/zuul/zuul/+/82548210:49
@jim:acmegating.comthe zuul docs have all but disappeared from google results.  i'm starting to think that the versioning has something to do with it.15:01
@jim:acmegating.comlike, if i search for something, if i do get a zuul-ci.org result, it's to a random old version of the doc15:02
@jim:acmegating.comfungi: maybe we should add redirects?15:02
@jim:acmegating.comand do we need to annotate the links to old versions somehow?  or add a sitemap?15:03
@tobias.henkel:matrix.orgI guess we can either add redirects or wait to see how long google takes to reindex15:09
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [opendev/zone-zuul-ci.org] 825519: Add google site verification https://review.opendev.org/c/opendev/zone-zuul-ci.org/+/82551915:12
@jim:acmegating.comzuul-maint infra-root: ^15:12
@jim:acmegating.comlooking into sitemaps -- i wonder if having the versions be underneath the main docs is a problem15:21
@jim:acmegating.commaybe we need docs/zuul/latest so that we can specify that in the sitemap without inadvertently including all the old versions.15:22
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [opendev/zone-zuul-ci.org] 825519: Add google site verification https://review.opendev.org/c/opendev/zone-zuul-ci.org/+/82551915:24
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/project-config] 825529: Publish latest zuul docs to latest/ https://review.opendev.org/c/zuul/project-config/+/82552915:28
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 825532: Link to docs at latest/ https://review.opendev.org/c/zuul/zuul/+/82553215:32
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed:15:36
- [zuul/zuul] 825532: Don't run non-docs jobs on docs changes https://review.opendev.org/c/zuul/zuul/+/825532
- [zuul/zuul] 825533: Link to docs at latest/ https://review.opendev.org/c/zuul/zuul/+/825533
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed:15:40
- [zuul/zuul] 825532: Don't run non-docs jobs on docs changes https://review.opendev.org/c/zuul/zuul/+/825532
- [zuul/zuul] 825533: Link to docs at latest/ https://review.opendev.org/c/zuul/zuul/+/825533
@jim:acmegating.comand we should put link rel=canonical in all our non-latest versions15:45
@jim:acmegating.com * and we should put link rel=canonical in all our non-latest versions, pointing to the latest version15:46
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 825535: Add link rel=canonical tag to docs https://review.opendev.org/c/zuul/zuul/+/82553515:50
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-website] 825542: Add a sitemap https://review.opendev.org/c/zuul/zuul-website/+/82554215:59
@jim:acmegating.comzuul-maint: i can add any of your google accounts to the search console, just let me know16:03
@jim:acmegating.com(it's not useful just yet though; apparently it's updated via batch jobs or something)16:03
@jim:acmegating.com(says "please check again in a day or so")16:03
@clarkb:matrix.orgLooks like the two fixes from swest are already approved, but I reviewed them anyway and nothing jumped out at me as wrong so +2'd16:32
@clarkb:matrix.orgNow to review the docs stuff16:33
@jpew:matrix.orgThere have been a lot of Zuul updates recently that require deleting zookeeper state (which is fine), but it has highlighted an annoyance I've found: I'm running Zuul using ephemeral containers (e.g. docker-compose/kubernetes) and it's actually farily difficult to "stop all zuul components" and run the command to delete state... does anyone have suggestions for making that easier (or perhaps I'm missing something?)16:34
@clarkb:matrix.orgjpew: I believe we do it with a docker run command against our scheduler container, but replace the command with the zk cleanup command16:36
@clarkb:matrix.orgAlso there was discussion about doing testing of this better so that we can try and catch these problems and do backward compatibile changes as well. I need to catch up on that myself16:36
@jpew:matrix.orgClark: Ya, that's what I do... in truth I don't bother stopping everything (esp. using the Zuul Operator where this is pretty much impossible) and run the command multiple times until it succeeds.... it's a little stressful each time I do that though :)16:37
@clarkb:matrix.orgAnother option would be to replace the zk cluster and restore the stuff we want to be persistent from backup16:38
@clarkb:matrix.orgcorvus: looks like https://review.opendev.org/c/zuul/zuul/+/825532 is running the jobs we don't want, but that is because we have updated those job defs and haven't modified doc/ right?16:40
@jim:acmegating.comjpew: batch job16:47
@jim:acmegating.com * jpew: if using k8s: batch job16:47
@jim:acmegating.comClark: yeah, that's why i split it out; the .zuul.yaml change is going to run them regardless.16:48
@jim:acmegating.comthe following changes are working as expected (linters+docs)16:48
@clarkb:matrix.orgcorvus: for the /latest change do we need to update zuul-web to link to /latest ?16:49
@tobias.henkel:matrix.orgjpew: the easiest would be another deployment that uses the zuul-scheduler image but just runs a sleep or an endless loop. You can exec then into that and execute the command there16:50
@tobias.henkel:matrix.orge.g. we anyway have a running zuul-backup service that regularly backups the keys and we run it from there16:51
@jim:acmegating.comClark: i think we should just put a redirect in website; i'll do that16:51
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-website] 825655: Add a redirect for docs/zuul to docs/zuul/latest https://review.opendev.org/c/zuul/zuul-website/+/82565516:53
@jim:acmegating.comi'm going to try to figure out to rebuild all the old docs so we can get the canonical urls in them (and also the theme update, etc)17:04
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 825535: Add link rel=canonical tag to docs https://review.opendev.org/c/zuul/zuul/+/82553517:11
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 825535: Add link rel=canonical tag to docs https://review.opendev.org/c/zuul/zuul/+/82553517:37
@jim:acmegating.comzuul-maint: would you mind reviewing https://review.opendev.org/825532 so we can go ahead and get it recheck-bashed in? :)17:38
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 825535: Add link rel=canonical tag to docs https://review.opendev.org/c/zuul/zuul/+/82553517:40
@jim:acmegating.comif we rebuild the docs, we're likely to lose some of the command-line helptext from sphinxcontrib-programoutput on older versions, since there are some versions of zuul which are not installable anymore17:44
@jim:acmegating.comi tend to think that's acceptable17:45
@jim:acmegating.com(also, i wonder if we should phase that out; it seems to be a perennial source of problems)17:46
@jim:acmegating.comtobiash: tristanC fungi can you please review/approve https://review.opendev.org/825532 if you're around?  sorry to pester, but i'd like to try to improve this asap17:53
@jim:acmegating.comthx!  once that gets in, the other stuff should be a lot faster :)17:55
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/project-config] 825529: Publish latest zuul docs to latest/ https://review.opendev.org/c/zuul/project-config/+/82552918:13
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 825661: Add a script to rebuild old docs https://review.opendev.org/c/zuul/zuul/+/82566118:16
@jim:acmegating.comzuul-maint: ^ i ran that script locally and produced this: https://zuul-ci.org/docs/tmp/18:17
if we like that, i can replace the current afs contents with it.
@jim:acmegating.com(the main driver there is getting the canonical link tags on the old docs; a secondary benefit is harmonizing the docs themes)18:18
@tobias.henkel:matrix.orglgtm18:18
@jim:acmegating.comokay, i think the whole story is there.  if folks would please take a look at https://review.opendev.org/q/status:open+-is:wip+topic:zuul-seo that should address everything i can think of except adding redirects from the old urls.18:25
@clarkb:matrix.orgclicking on a random selection of the tmp built old logs they seem to have the theme and a front page. LGTM18:28
@jim:acmegating.comand i see `18:31
<link rel="canonical" href="https://zuul-ci.org/docs/zuul/latest/index.html" />` in the source
@jim:acmegating.comsome of those links aren't going to make sense, but i think that's okay.  my guess is that any old docs that point to correct canonical pages will help us, and if any of those links are dead it won't hurt us.18:32
@jim:acmegating.comif we do add redirects for the move, then that might improve that a bit more18:32
@jim:acmegating.com * if we do add redirects for the reorg, then that might improve that a bit more18:32
-@gerrit:opendev.org- Vasyl Vavrychuk proposed: [zuul/zuul] 825467: doc/source/operation.rst: Fix option to not demonize. https://review.opendev.org/c/zuul/zuul/+/82546718:47
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com:18:49
- [zuul/zuul-website] 825542: Add a sitemap https://review.opendev.org/c/zuul/zuul-website/+/825542
- [zuul/zuul-website] 825655: Add a redirect for docs/zuul to docs/zuul/latest https://review.opendev.org/c/zuul/zuul-website/+/825655
@jim:acmegating.comon the release front, i think we need to hold 4.12 for 825480 and 825482.  they're both approved; so once they land, i'll restart opendev again18:54
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 821711: Add IBM Cloud VPC driver https://review.opendev.org/c/zuul/nodepool/+/82171119:08
@fungicide:matrix.orgcorvus: sorry, been sidelined on other issues today but catching up, did you consider in-place editing the old files in afs rather than complete rebuilds, or was it just that the latter was a lot simpler to implement?19:10
@fungicide:matrix.orgalso, i guess a canonical link pointing to itself is effectively ignored?19:12
@fungicide:matrix.orgaha, so there's a sphinx feature specifically for this19:16
-@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [zuul/zuul] 825666: Increase our Sphinx minimum to 1.8 https://review.opendev.org/c/zuul/zuul/+/82566619:25
@fungicide:matrix.orgnoticed that when reading up on it19:26
@fungicide:matrix.orgalso, rfc 6596 ยง3 specifically answers my question... "The target (canonical) IRI MAY: [...] Be self-referential (context IRI identical to target IRI)"19:35
@jim:acmegating.com> <@fungicide:matrix.org> corvus: sorry, been sidelined on other issues today but catching up, did you consider in-place editing the old files in afs rather than complete rebuilds, or was it just that the latter was a lot simpler to implement?19:44
yeah, i think it's a lot easier to just rebuild; plus i've been wanting to do it anyway to get theme alignment. this just took it from low-priority to high.
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 825668: Skip flaky disconnect tests https://review.opendev.org/c/zuul/zuul/+/82566820:01
@jim:acmegating.comzuul-maint: ^ i've spent a lot of time looking into that and don't see the problem yet; i think it's worth the tradeoff.20:02
-@gerrit:opendev.org- Vasyl Vavrychuk proposed: [zuul/zuul] 825467: doc/source/operation.rst: Fix option to not demonize. https://review.opendev.org/c/zuul/zuul/+/82546720:03
@jim:acmegating.comtobiash: 825533 is currently running all jobs, but the previous buildset (before the most recent gate reset) was running the expected lesser set of jobs.  do you think that's a manifestation of the files bug?20:05
@jim:acmegating.comswest: ^20:06
@tobias.henkel:matrix.orgThe files bug is usually triggered by a gate reset20:07
@jim:acmegating.comif i understand the mechanism correctly: because files would be None after the gate reset, and so all the files matchers would match?20:07
@tobias.henkel:matrix.orgOr when fileschanges is used, but I thing that's seldom the case for gerrit changes20:07
-@gerrit:opendev.org- Vasyl Vavrychuk marked as active: [zuul/zuul] 825467: doc/source/operation.rst: Fix option to not demonize. https://review.opendev.org/c/zuul/zuul/+/82546720:07
@jim:acmegating.comyeah, i think it's not the fileschanges path in this case, but rather the buildset initializer case20:08
@tobias.henkel:matrix.orgyes20:08
@tobias.henkel:matrix.orgAlmost20:08
@tobias.henkel:matrix.orgThe files are fetched via fileschanges after a gate reset20:08
@fungicide:matrix.orgi'm looking into why 825532 has still not enqueued into the gate pipeline20:09
@tobias.henkel:matrix.orgSo there are two bugs working together20:09
@jim:acmegating.comfungi: it's at the head right now20:10
@jim:acmegating.comfungi: but it's not taking effect due to the bug tobiash and i are discussing (the bug is fixed in an approved-but-not-merged patch)20:10
@fungicide:matrix.orgoh! okay it apaprently got enqueued into check as well and i got confused by the lingering verified +120:10
@jim:acmegating.comfungi: yeah, i guess that doesn't get cleared when it supercedes20:11
@fungicide:matrix.orgno worries, catching up on the other thing now20:11
@tobias.henkel:matrix.orgso tldr, one bug triggers unneeded fileschanges after gate resets and the other is a broken fileschanges implementation20:12
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 825532: Don't run non-docs jobs on docs changes https://review.opendev.org/c/zuul/zuul/+/82553220:12
@fungicide:matrix.orgdid it take a manual action to merge that, or did it resolve on its own?20:13
@fungicide:matrix.orgoh, now i get what you meant by "not taking effect"20:14
@fungicide:matrix.orgthe bug wasn't preventing it from merging, it just wasn't being speculatively applied20:15
@tobias.henkel:matrix.orgthe bug makes zuul think there are more files in the change than there are making file matchers ineffective20:15
@fungicide:matrix.orgthanks, it's starting to come together for me now20:17
@jim:acmegating.comyep.  so when it and the following changes were first enqueued behind a change already in gate, everything was fine, but after the change ahead in gate failed, and gate reset, the bug took effect and all jobs ran.20:19
@jim:acmegating.comironically it was the bugfix which was ahead in the gate :)20:19
@tobias.henkel:matrix.orgcorvus: if there are now only doc changes in gate you might be able to accelerate that by dequeuing and reenqueuing them20:21
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com:20:22
- [zuul/zuul] 825533: Link to docs at latest/ https://review.opendev.org/c/zuul/zuul/+/825533
- [zuul/zuul] 825535: Add link rel=canonical tag to docs https://review.opendev.org/c/zuul/zuul/+/825535
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 825661: Add a script to rebuild old docs https://review.opendev.org/c/zuul/zuul/+/82566120:22
@tobias.henkel:matrix.orgnevermind, haven't looked at the gate queue20:23
@jim:acmegating.comsomehow we ran the gauntlet on those :)20:23
@jim:acmegating.comokay, docs are at latest/ now; i'll move the old rebuilds into position21:00
@jim:acmegating.comokay that's done21:20
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-website] 825678: Add individual directory/file redirects for the move to latest https://review.opendev.org/c/zuul/zuul-website/+/82567821:24
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-website] 825683: Add redirects from the most recent doc reorg https://review.opendev.org/c/zuul/zuul-website/+/82568321:50
@jim:acmegating.comokay there's a whole pile of redirects; i think that's ready to go now21:51
@jim:acmegating.comzuul-maint: i'd like to disable these two tests for a while; it should greatly reduce our false positive rate: https://review.opendev.org/82566821:52
-@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl:22:00
- [zuul/zuul] 825480: Revert "Fix a bug in getting changed files" https://review.opendev.org/c/zuul/zuul/+/825480
- [zuul/zuul] 825482: Initialize buildset files state correctly on reset https://review.opendev.org/c/zuul/zuul/+/825482
@jim:acmegating.comi'll go roll opendev now22:02
@jim:acmegating.comif anyone wants to approve the two website changes with redirects that'd be great :)22:06
-@gerrit:opendev.org- Zuul merged on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [zuul/zuul] 825666: Increase our Sphinx minimum to 1.8 https://review.opendev.org/c/zuul/zuul/+/82566622:06
@clarkb:matrix.orgcorvus: any concern with me approving the htaccess update changes while you rolling restart zuul?22:10
@jim:acmegating.comClark: nope should be fine22:11
@jim:acmegating.comhere's a fun situation: because we don't build images on docs changes and the last change to land was a docs change, we didn't get a promote for the bugfix (because of the way the promote pipeline works).  i'm manually enqueing it into promote now.22:12
@jim:acmegating.combut that's a really good reason for us to rip out the irrelevant-files matchers asap :)22:12
@clarkb:matrix.orgcorvus: wouldn't the prior change have promoted itself?22:13
@clarkb:matrix.orgthe sphinx minimum update shouldn't matter for this restart? Or are we just trying to make sure shas line up in prep for release stuff?22:14
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 825685: Revert "Don't run non-docs jobs on docs changes" https://review.opendev.org/c/zuul/zuul/+/82568522:14
@jim:acmegating.comClark: promote is supercedent, so the last change wins.  last change didn't run image promotion.22:14
@jim:acmegating.com(so we're actually going to rewind the docs publication one change because of my manual enqueue, however, the docs change in question was not a change to content so it doesn't matter)22:16
@clarkb:matrix.orgaha22:17
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com:22:23
- [zuul/zuul-website] 825678: Add individual directory/file redirects for the move to latest https://review.opendev.org/c/zuul/zuul-website/+/825678
- [zuul/zuul-website] 825683: Add redirects from the most recent doc reorg https://review.opendev.org/c/zuul/zuul-website/+/825683
@jim:acmegating.comopendev is running the bugfix now at 548eafe0b5729e78ab4024abea98b326678d83d8 -- so if we manage to get through the night without any more critical bugfixes (touch wood), it loos like that is lined up to be 4.12.023:00
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 821711: Add IBM Cloud VPC driver https://review.opendev.org/c/zuul/nodepool/+/82171123:24
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-website] 825690: Add XML namespace to sitemap https://review.opendev.org/c/zuul/zuul-website/+/82569023:33

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!