Friday, 2021-08-27

@jim:acmegating.commordred: i think there's an issue in your patch https://zuul.opendev.org/t/zuul/build/1c00416cec86457798fa8d1681b4964000:01
@jim:acmegating.commordred: i think we need to change the ffi6 line to not match bullseye?00:01
@jim:acmegating.combut why wasn't ffi7 on there..?00:01
@jim:acmegating.comoh00:01
@jim:acmegating.comyeah, re2-9 made it but ffi7 didn't00:03
@jim:acmegating.comthough i don't know why it didn't try to do both?00:04
@jim:acmegating.com * though i don't know why it didn't try to do both ffi6 and 7?00:04
@jim:acmegating.comah, because ffi7 is already installed, so it was trying both00:09
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed on behalf of Monty Taylor https://matrix.to/#/@mordred:inaugust.com: [zuul/zuul] Update bindep for debian bullseye https://review.opendev.org/c/zuul/zuul/+/80624900:10
@jim:acmegating.commordred: ianw: ^ i think that should do it00:10
@jim:acmegating.comi'm going to carry over ianw's +2 on that since it's now urgent00:24
-@gerrit:opendev.org- Zuul merged on behalf of Monty Taylor https://matrix.to/#/@mordred:inaugust.com: [zuul/zuul] Update bindep for debian bullseye https://review.opendev.org/c/zuul/zuul/+/80624901:50
-@gerrit:opendev.org- Ian Wienand proposed:05:49
- [zuul/zuul] web: Projects: use composable table https://review.opendev.org/c/zuul/zuul/+/805581
- [zuul/zuul] web: Project: cleanup query https://review.opendev.org/c/zuul/zuul/+/805742
-@gerrit:opendev.org- Ian Wienand proposed: [zuul/nodepool] Update bindep for bullseye update https://review.opendev.org/c/zuul/nodepool/+/80631205:57
@iwienand:matrix.orghrm, that's going to be more interesting than i thought06:08
@iwienand:matrix.orgyum isn't in bullseye, only dnf06:08
@iwienand:matrix.orgit doesn't seem to include a yum link; so ubuntu-minimal is going to have to learn to speak dnf i guess06:15
-@gerrit:opendev.org- Ian Wienand proposed: [zuul/nodepool] Update bindep for bullseye update https://review.opendev.org/c/zuul/nodepool/+/80631206:28
-@gerrit:opendev.org- Ian Wienand proposed: [zuul/nodepool] Update Docker and bindep for Bullseye base images https://review.opendev.org/c/zuul/nodepool/+/80631206:30
-@gerrit:opendev.org- Artem Goncharov proposed: [zuul/zuul] Fix failed merge detection in gitlab https://review.opendev.org/c/zuul/zuul/+/80623007:29
-@gerrit:opendev.org- Ian Wienand proposed: [zuul/nodepool] Update Docker and bindep for Bullseye base images https://review.opendev.org/c/zuul/nodepool/+/80631207:48
-@gerrit:opendev.org- Simon Westphahl proposed:08:03
- [zuul/zuul] Reference change dependencies by key https://review.opendev.org/c/zuul/zuul/+/805844
- [zuul/zuul] Implement ABC for caching changes in Zookeeper https://review.opendev.org/c/zuul/zuul/+/805835
- [zuul/zuul] Cache Gerrit changes in Zookeeper https://review.opendev.org/c/zuul/zuul/+/805837
- [zuul/zuul] Cache Github pull requests in Zookeeper https://review.opendev.org/c/zuul/zuul/+/805838
@iwienand:matrix.orgi've about hit my limit for today.  the tl;dr is that https://review.opendev.org/c/zuul/nodepool/+/806312 should update nodepool images to bullseye (with a very hacky dnf plugin install) and something like https://review.opendev.org/c/openstack/diskimage-builder/+/806318 should get dib working again.  however, i think we will have a circular dependency as dib uses the container images to run gate tests08:03
@iwienand:matrix.orgi have run a local centos-minimal using "dnf download" and "dnf" on the host and it seems to work08:04
@iwienand:matrix.orgthere may be other things in full builds lurking though; i just did absolute smallest 08:07
-@gerrit:opendev.org- Ian Wienand proposed: [zuul/nodepool] Update Docker and bindep for Bullseye base images https://review.opendev.org/c/zuul/nodepool/+/80631208:21
@avass:vassast.organyone know what the status of the admin ui is? https://review.opendev.org/q/topic:%22fffaff%22+(status:open%20OR%20status:merged)10:04
-@gerrit:opendev.org- Artem Goncharov proposed: [zuul/zuul] Fix failed merge detection in gitlab https://review.opendev.org/c/zuul/zuul/+/80623010:42
@pabelanger:matrix.orgmorning, we see a lot of the following in nodepool-builder logs and wanted to see how to best handle it on the nodepool side: https://paste.opendev.org/show/808378/13:25
@pabelanger:matrix.orggiven we cannot delete the image on the cloud provider, I am wondering if we should trap the error and still try to delete zk data13:25
@mordred:inaugust.comianw, corvus: maybe what we need to do is pin the nodepool image back to the sha of the buster images, land the dib change to support dnf download, then land an unpin?13:47
-@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [zuul/nodepool] Temporarily pin back to older python base image https://review.opendev.org/c/zuul/nodepool/+/80640613:59
-@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed on behalf of Ian Wienand: [zuul/nodepool] Update Docker and bindep for Bullseye base images https://review.opendev.org/c/zuul/nodepool/+/80631213:59
@mordred:inaugust.comianw, corvus : I pushed up a patch implementing that idea and updated the dib patch depends-on ... I *think* it should work14:03
@jpew:matrix.orgIf I already have cert-manager running on my K8s cluser, will zuul-operator install a new one?14:23
@jim:acmegating.commordred: that tag could be deleted at any time :(14:43
@mordred:inaugust.comShould we make a real one?14:44
@jim:acmegating.commordred: yeah -- maybe we should just manually tag that as :bullseye?14:47
@jim:acmegating.comer, not bullseye, the other one :)14:48
@jim:acmegating.combuster?14:48
@jim:acmegating.comunless you want to make the infrastructure to actually maintain it14:48
@fungicide:matrix.orgit's only needed until the unpin lands, right?14:49
@jim:acmegating.comthat's my thinking14:50
@jim:acmegating.commordred, fungi: shall i do that?14:55
@mordred:inaugust.comcorvus: yeah - I think that's likely a good idea. I don't know that I think making infrastructure to maintain a buster variant is worthwhile15:00
@jim:acmegating.commordred: -> #opendev15:03
@avass:vassast.orgjpew: I'm not actually sure but I think you can disable it? I hope15:04
@jim:acmegating.comavass, jpew: it detectes whether a certificaterequests.cert-manager.io CRD exists and installs certmanager only if it does not15:10
@jpew:matrix.orgcorvus: Cool15:10
@jim:acmegating.compabelanger: no we need to keep the zk data, it will be deleted when the last instance referencing it is deleted.  if you want to make the exception quieter, that would be fine.15:11
@jpew:matrix.orgFWIW, I tried manually installing PXC-operator (1.9.0) but it couldn't find it for some reason, so switched back to letting zuul-operator install it (but I didn't want it to install a cert-manager)15:11
@clarkb:matrix.orgIs there any reason to not approve zuul changes right now? Specifically planning on catching up on reviews for ianw's UI updates. Not sure if the docker image situation is stable enough for zuul?16:17
@jim:acmegating.comClark: they will not merge until the docker situation is resolved16:17
@clarkb:matrix.orgok, thanks. I'll review but won't approve in that case16:18
@clarkb:matrix.orgWhen docker images are set backto buster is https://review.opendev.org/c/zuul/zuul/+/789654 and https://review.opendev.org/c/zuul/nodepool/+/789655 something that we want to do next or is the focus more on getting to bullseye?16:55
@clarkb:matrix.orgI'm guessing the problems on bullseye were not python version related though so this may be orthogonal16:55
@clarkb:matrix.orgThe docker images have been updated. I'll approve https://review.opendev.org/c/zuul/zuul/+/805919 which should confirm this is happy now17:20
@jpew:matrix.orgI'm using a GitHub app with kubernetes and I can't figure out how to get the app private key into the scheduler config. The scheduler secret wants the path to a file, but how do I get the scheduler pod to mount the secret that holds the key?18:15
@avass:vassast.orgjpew: you should be able to create a kubernetes secret with the app private key and mount that on the pod and then refer to the file in the configuration18:48
@avass:vassast.orghttps://kubernetes.io/docs/concepts/configuration/secret/#using-secrets18:49
@jpew:matrix.orgavass: Ya, I just got it; I though I needed to put the app_key path to the key in the secret itself (with the key). But I realized I can move it to the Zuul yaml spec and reference where the key gets mounted18:50
@jpew:matrix.orgNow, I need to wait until GitHub isn't blocking my API requests :/18:50
-@gerrit:opendev.org- Zuul merged on behalf of Ian Wienand: [zuul/zuul] web: LogFile: fix severity toggle buttons https://review.opendev.org/c/zuul/zuul/+/80591918:51
@jpew:matrix.orgHmm, zuul doesn't appear to be picking up the changes to my config project for some reason18:57
@jpew:matrix.orgIt looks like zuul can't seem to see any of the files:19:05
```
2021-08-27 18:45:04,055 DEBUG cachecontrol.controller: Looking up "https://api.github.com/repos/wattissoftware/zuul-config/branches?per_page=100" in the cache
2021-08-27 18:45:04,055 DEBUG cachecontrol.controller: No cache entry available
2021-08-27 18:45:04,252 DEBUG cachecontrol.controller: Updating cache with response from "https://api.github.com/repos/wattissoftware/zuul-config/branches?per_page=100"
2021-08-27 18:45:04,252 DEBUG cachecontrol.controller: Caching due to etag
2021-08-27 18:45:04,253 DEBUG zuul.GithubRequest: GET https://api.github.com/repos/wattissoftware/zuul-config/branches?per_page=100 result: 200, size: 406, duration: 196
2021-08-27 18:45:04,253 DEBUG zuul.GithubRateLimitHandler: GitHub API rate limit (wattissoftware/zuul-config, 19112994) resource: core, remaining: 4998, reset: 1630093501
2021-08-27 18:45:04,253 INFO zuul.GithubConnection: Got branches for wattissoftware/zuul-config
2021-08-27 18:45:04,270 DEBUG zuul.TenantParser: Using files from cache for project github.com/wattissoftware/zuul-config @main: []
```
@jpew:matrix.orgHmm, I can't put the webhook_token in a K8s secret because it appears that the code badly formats it when it writes it to zuul.conf:20:28
```
webhook_token = b'REDACTED\n'
```
@clarkb:matrix.orgis the issue the b'' quoting?20:29
@jpew:matrix.orgYa20:29
@jpew:matrix.orgI figured out how o remove the newline20:30
@jpew:matrix.org * I figured out how to remove the newline, but it still failed20:30
@clarkb:matrix.orgThat might be a yaml type issue when it goes through python somewhere? b'' is a raw python byte string20:30
@jpew:matrix.orgThat was my guess as well20:31
@jpew:matrix.orgIf I put the webhook_token in the zuul k8s yaml file, it works, but that's obviously not ideal. The scheduler still subbornly refuses to acknowledge there is anything in my config repo though :(20:32
@jpew:matrix.orgIs anyone using github for their config repo? Maybe it's just broken?20:36
@clarkb:matrix.orgI suspect pabelanger is20:37
-@gerrit:opendev.org- Joshua Watt proposed: [zuul/zuul-operator] Decode operator secrets as strings https://review.opendev.org/c/zuul/zuul-operator/+/80645221:08
@jpew:matrix.orgUgh, finally got it21:45
@jpew:matrix.orgApparently, you can't use a branch other than "master" for the config repo21:45
@avass:vassast.orgjpew: i think you can and there's a tenant config option for that i believe21:47
@jpew:matrix.orgavass: It doesn't appear to work properly21:47
@avass:vassast.orgjpew: https://zuul-ci.org/docs/zuul/reference/tenants.html#attr-tenant.config-projects.%3Cproject%3E.load-branch21:48
But i think that has had issues in the past. mnaser may know more about that
@clarkb:matrix.orgI believe pabelanger is also using a non master config branch21:54
@clarkb:matrix.orgbut maybe it was only projects they changed and not the config repo21:54

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!