Tuesday, 2021-08-17

opendevreviewJames E. Blair proposed zuul/zuul master: Perform an implicit smart-reconfigure on startup  https://review.opendev.org/c/zuul/zuul/+/80480600:10
opendevreviewJames E. Blair proposed zuul/zuul-operator master: Fix wait for scheduler to settle in tests  https://review.opendev.org/c/zuul/zuul-operator/+/80446200:10
corvusswest, tobiash: can you take a look at https://review.opendev.org/804806 and let me know if that makes sense?00:30
opendevreviewJames E. Blair proposed zuul/zuul master: Add tenant name on NodeRequests for Nodepool  https://review.opendev.org/c/zuul/zuul/+/78868000:36
swestcorvus: lgtm06:10
opendevreviewIan Wienand proposed zuul/zuul master: web: JobVariant : convert to DescriptionList  https://review.opendev.org/c/zuul/zuul/+/80460106:19
opendevreviewIan Wienand proposed zuul/zuul master: web: JobVariant : use wide layout for DescriptionList  https://review.opendev.org/c/zuul/zuul/+/80460206:19
opendevreviewIan Wienand proposed zuul/zuul master: web: Job: don't display loading/refresh header  https://review.opendev.org/c/zuul/zuul/+/80482106:19
opendevreviewIan Wienand proposed zuul/zuul master: web: JobVariant : use wide layout for DescriptionList  https://review.opendev.org/c/zuul/zuul/+/80460206:56
opendevreviewBenjamin Schanzel proposed zuul/nodepool master: Add Tenant-Scoped Resource Quota  https://review.opendev.org/c/zuul/nodepool/+/80076507:09
opendevreviewBenjamin Schanzel proposed zuul/nodepool master: Add Tenant-Scoped Resource Quota  https://review.opendev.org/c/zuul/nodepool/+/80076507:11
opendevreviewBenjamin Schanzel proposed zuul/nodepool master: Add Tenant-Scoped Resource Quota  https://review.opendev.org/c/zuul/nodepool/+/80076507:21
lidaliu[m]Hello, I have a question related to the grafana plot shown here. How does zuul send out nodepool metrics to the grafana ?https://grafana.opendev.org/d/KRmop6EMz/nodepool?orgId=107:23
opendevreviewTobias Henkel proposed zuul/zuul master: Don't return nodeset on cancel  https://review.opendev.org/c/zuul/zuul/+/80483007:32
*** jpena|off is now known as jpena07:32
opendevreviewTobias Henkel proposed zuul/zuul master: Remove noop builds from builds list  https://review.opendev.org/c/zuul/zuul/+/80483107:49
*** rpittau|afk is now known as rpittau07:54
tobiash[m]zuul-maint: this fixes a serious memory leak since we leak all noop jobs and thus all layouts in the system ^08:10
tobiash[m](at least all layouts that involve noop jobs which are many in our system)08:11
opendevreviewTobias Henkel proposed zuul/zuul master: Remove noop builds from builds list  https://review.opendev.org/c/zuul/zuul/+/80483108:13
avass[m]tobiash: only got a nit that the code could check `if build.build_request_ref` instead of checking the inverse and returning early08:17
opendevreviewTobias Henkel proposed zuul/zuul master: Remove noop builds from builds list  https://review.opendev.org/c/zuul/zuul/+/80483108:26
tobiash[m]now with test08:26
opendevreviewIan Wienand proposed zuul/zuul master: web: Job: don't display loading/refresh header  https://review.opendev.org/c/zuul/zuul/+/80482109:04
opendevreviewIan Wienand proposed zuul/zuul master: web: JobVariant : convert to DescriptionList  https://review.opendev.org/c/zuul/zuul/+/80460109:04
opendevreviewIan Wienand proposed zuul/zuul master: web: JobVariant : use wide layout for DescriptionList  https://review.opendev.org/c/zuul/zuul/+/80460209:04
opendevreviewlikui proposed zuul/nodepool master: setup.cfg: Replace dashes with underscores  https://review.opendev.org/c/zuul/nodepool/+/80483709:20
opendevreviewMerged zuul/zuul master: Perform an implicit smart-reconfigure on startup  https://review.opendev.org/c/zuul/zuul/+/80480610:08
avass[m]corvus: does 800786 depend on 804462 ?11:10
*** dviroel|out is now known as dviroel|ruck11:26
*** jpena is now known as jpena|lunch11:39
opendevreviewMerged zuul/zuul-operator master: Make nodepool external_config mount more generic  https://review.opendev.org/c/zuul/zuul-operator/+/80078611:53
avass[m]I guess not :)12:19
alexhHi folks - hope this isn't out of place - we are running into an issue with Zuul (on the executor) if a git branch gets deleted -- even if that branch isn't the merge target or anything like that. It's during the "restore repo" logic that it tries to resolve, seemingly, all heads (even if not relevant?), and so if the commit for one of them is gone, because the branch was deleted, it trips up and keeps retrying forever. http://pastie.org/p/3AIJ5Vfzb0CY75V7PW12:31
avass[m]alexh: that link is giving me a 40412:32
alexhHm, odd - can you try https://pastebin.com/raw/GDRfC0Vj ?12:33
*** jpena|lunch is now known as jpena12:42
avass[m]alexh: that works12:45
corvusalexh: was the branch deleted between when the item was enqueued and the job started running?12:47
alexhcorvus: quite possibly. The jobs we have run for an hour or two, and there was most definitely stuff running at the time it got deleted.12:47
alexhOK, you specifically asked between enqueuing and starting to run -- not so sure about that.12:50
alexhSo far we've had a reasonably smooth experience using Zuul, until we deleted the odd branch last week, and again today.12:52
corvusalexh: you should be able to recover from this by deleting the repo on the executor12:52
alexhDeleting the cache repo? I tried that last Thursday, and it still failed -- but I think then it starts failing on commits that Zuul creates (sort of intermediate rebases for the gate pipeline, etc)12:53
alexhOr at least I couldn't find any other reference to those SHAs anywhere else, which is why I think they were intermediate rebases12:54
alexhIf the cache repo is the problem, sorta, is there a way to tell Zuul to only cache the branches it's actually asked to merge anything onto rather than all branches?12:55
alexhEven if deleting the cache repo worked, I don't think it's a sustainable option for us. At that point, I'd get someone to prevent branch deletions in the gerrit project, which also isn't fantastic.12:56
corvusalexh: yeah, it sounds like a bug12:56
opendevreviewMatthieu Huin proposed zuul/zuul master: web UI: user login with OpenID Connect  https://review.opendev.org/c/zuul/zuul/+/73408213:07
opendevreviewMatthieu Huin proposed zuul/zuul master: web UI: Alternative styling for autohold and re-enqueue buttons  https://review.opendev.org/c/zuul/zuul/+/80404313:08
opendevreviewMatthieu Huin proposed zuul/zuul master: Add authentication-realm attribute to tenants  https://review.opendev.org/c/zuul/zuul/+/73558613:09
opendevreviewMatthieu Huin proposed zuul/zuul master: web UI: allow a privileged user to dequeue a change  https://review.opendev.org/c/zuul/zuul/+/73485013:09
alexhAlright, thanks. I've created https://storyboard.openstack.org/#!/story/2009121 - hopefully that's the right thing to do.13:10
corvusavass: the operator change should be subject to the same issue; it must have raced in just the right way.13:10
opendevreviewMatthieu Huin proposed zuul/zuul master: web UI: allow a privileged user to re-enqueue a change  https://review.opendev.org/c/zuul/zuul/+/73677213:14
opendevreviewMatthieu Huin proposed zuul/zuul master: Web UI: allow a privileged user to request autohold  https://review.opendev.org/c/zuul/zuul/+/76811513:14
opendevreviewMatthieu Huin proposed zuul/zuul master: Web UI: add Autoholds, Autohold page  https://review.opendev.org/c/zuul/zuul/+/76819913:15
opendevreviewMatthieu Huin proposed zuul/zuul master: REST API, Web UI: add pipelines' manager, triggers data in status  https://review.opendev.org/c/zuul/zuul/+/73696813:15
opendevreviewMatthieu Huin proposed zuul/zuul master: web UI: allow a privileged user to promote a change  https://review.opendev.org/c/zuul/zuul/+/78185813:19
opendevreviewMatthieu Huin proposed zuul/zuul master: Web UI: Add "Create Autohold Request" form, improve API error messages  https://review.opendev.org/c/zuul/zuul/+/80255913:19
opendevreviewMatthieu Huin proposed zuul/zuul master: Example Docker compose: keycloak integration  https://review.opendev.org/c/zuul/zuul/+/76994313:19
fungiodd issue brought up today by one of our users... removed a project-template from within an untrusted repo, but it's still being selected to run jobs for subsequent changes *unless* those changes touch the zuul configuration: https://meetings.opendev.org/irclogs/%23openstack-infra/%23openstack-infra.2021-08-17.log.html13:32
corvusfungi: sounds like it could be a bug with the new config caching regime13:36
fungithat's what i was wondering, yes13:36
fungiany good way to debug this that you can think of? how do we inspect those caches?13:37
corvusfungi: i'd start by collecting logs related to it, especially to make sure we processed the change-merged event and correctly decided that needed a reconfiguration13:44
fungithanks, i'll dig those up after my next two conference calls13:54
fungisee if anything looks fishy13:55
clarkbcorvus: I think the matrix email didn't include the rooms address?13:55
clarkb(its early and I may ahve simply overlooked it when catching up on email)13:55
fungii thought that would get mentioned during the cut-over announcement13:56
corvusfungi, clarkb: yeah, was looking for an okay on timing first :)14:01
clarkbah ok. The email indicated people could join now/early. ALso it is early and my brain needs more fuel14:01
*** jpena is now known as jpena|off15:26
*** jpena|off is now known as jpena15:27
Clark[m]corvus: not sure those last messages ever made it to IRC.15:30
Clark[m]Not a big deal but interesting how things don't always get across the bridge15:31
corvusoh that's interesting.  fwiw, i suspect the different bridges may have different performance characteristics.  i think the libera.chat bridge is getting a lot of love and attention right now.15:32
corvusi think we've learned some thing that would be useful for folks who do want to maintain a bridged presence, even if we don't.15:34
corvusClark: also doubly ironic that a message about miscommunication was lost :)15:34
Clark[m]Indeed15:35
corvusClark: I also see that i didn't get your last msg in the other direction15:35
corvuser, disregard that.  i was mistaken15:36
corvuslooks like only mine were lost15:36
clarkbcorvus: ya I think I misparsed the email because it was too early for the brain15:46
*** jpena is now known as jpena|off15:48
*** marios is now known as marios|out16:08
*** rpittau is now known as rpittau|afk16:31
opendevreviewMatthieu Huin proposed zuul/zuul master: web UI: allow a privileged user to re-enqueue a change  https://review.opendev.org/c/zuul/zuul/+/73677216:47
opendevreviewMatthieu Huin proposed zuul/zuul master: Web UI: allow a privileged user to request autohold  https://review.opendev.org/c/zuul/zuul/+/76811517:04
opendevreviewMatthieu Huin proposed zuul/zuul master: Web UI: add Autoholds, Autohold page  https://review.opendev.org/c/zuul/zuul/+/76819917:06
opendevreviewMatthieu Huin proposed zuul/zuul master: REST API, Web UI: add pipelines' manager, triggers data in status  https://review.opendev.org/c/zuul/zuul/+/73696817:06
opendevreviewMatthieu Huin proposed zuul/zuul master: web UI: allow a privileged user to promote a change  https://review.opendev.org/c/zuul/zuul/+/78185817:06
opendevreviewMatthieu Huin proposed zuul/zuul master: Web UI: Add "Create Autohold Request" form, improve API error messages  https://review.opendev.org/c/zuul/zuul/+/80255917:06
opendevreviewMerged zuul/zuul master: Remove noop builds from builds list  https://review.opendev.org/c/zuul/zuul/+/80483117:42
clarkbpabelanger[m]: fwiw mitogen apparently also avoids the python forks. I didn't realize that so it would likely help us18:04
opendevreviewIan Wienand proposed zuul/zuul master: web: JobVariant : use wide layout for DescriptionList  https://review.opendev.org/c/zuul/zuul/+/80460219:13
pabelanger[m]clarkb: yah, I was going to push up a change to have you test.  It did make a big impact for us19:34
pabelanger[m]For console streaming, what do people think of adding some sort of job runtime field to the header on the console: https://dashboard.zuul.ansible.com/t/ansible/stream/bef5e502659144f8a9eff983d649d18e?logfile=console.log19:35
pabelanger[m]I often try to figure out how long a test has been running via console.log but need to do math :)19:36
clarkbpabelanger[m]: a chagne should be self testing and that would be much appreciated if you are willing20:02
*** dviroel|ruck is now known as dviroel|ruck|out21:15
corvusclarkb, tristanC: https://review.opendev.org/804462 looks like it's ready to go and should unblock the rest of the operator changes21:54
corvusi think the current master of zuul is a good place for a .1 release21:55
fungiseems so21:59
fungimy stale project config situation from earlier looks like it's not a cache bug, at least22:00
corvusopendev restart on current master is done; if this looks good, i'll plan on tagging a .1 tomorrow22:21
TylerPearce[m]Hello again! Where can I overwrite the base url Zuul uses when commenting on a PR? Right now the comment uses the base URL "localhost:9000" but I need to point that to my EC2 instance. I tried searching the wiki but couldn't find the rights docs 😅22:40
* TylerPearce[m] uploaded an image: (39KiB) < https://matrix.org/_matrix/media/r0/download/matrix.org/TwBdUCILjdrDfUywhUhZQkWR/image.png >22:41
clarkbTylerPearce[m]: in your zuul.conf there is a [web] section and that has root and status_url options I believe those control that22:43
TylerPearce[m]Awesome, thank you!22:44
pabelanger[m]https://zuul-ci.org/docs/zuul/discussion/components.html#attr-web.root22:44
pabelanger[m]I believe that is it22:44
clarkbpabelanger[m]: thanks I was trying to look that up22:45
TylerPearce[m]Ahh yes, thank you for the link!22:45
pabelanger[m]clarkb: I pushed up mitogen change, which server is opendev on now?22:45
pabelanger[m]for #opendev irc channel I mean22:46
clarkbpabelanger[m]: oftc #opendev22:46
pabelanger[m]tyty22:46
clarkbcorvus: have you had a chance to look at https://review.opendev.org/c/zuul/zuul/+/804456 for the sqlaclhemy 2.0 prep yet?23:02
clarkblanding that after the .1 may make sense then we can run it through its paces before the next release23:02
corvusclarkb: LGTM, also I want to merge the sigterm change after .123:12
opendevreviewClark Boylan proposed zuul/zuul master: Retry gerrit queryChange()  https://review.opendev.org/c/zuul/zuul/+/80492923:18
clarkbcorvus: fungi  ^ thats a quick first stab at retrying the change queries23:19
corvusclarkb: bare exception23:20
clarkbcorvus: ya We know what the http exception is but do you know what the ssh connection would raise in that case?23:20
clarkbcorvus: or do you think it would be ok to catch Exception there?23:23
corvusException23:25
corvusThere's a linter rule about "except"23:26
clarkbcorvus: ok I think paramiko.ssh_exception.NoValidConnectionsError might be what ssh raises. Would you prefer Exception or paramiko.ssh_exception.NoValidConnectionsError and requests.exceptions.ConnectionError?23:27
corvusException23:27
clarkbok23:27
corvussorry, wasn't trying to make this complicated23:27
corvusjust "except:" isn't going to pass linting; needs to be "except Exception:"23:28
corvusi'm strongly in favor of "except Exception:" for most error handling like this :)23:28
opendevreviewClark Boylan proposed zuul/zuul master: Retry gerrit queryChange()  https://review.opendev.org/c/zuul/zuul/+/80492923:28
corvustristanC: re sigterm, it looks like people like the zuul.conf followup at https://review.opendev.org/804464 -- if you don't object, i'll approve both after the release tomorrow.  just wanted to ping you since you +2d the parent.23:32
opendevreviewMerged zuul/zuul-operator master: Fix wait for scheduler to settle in tests  https://review.opendev.org/c/zuul/zuul-operator/+/80446223:36

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!