| *** josefwells has joined #zuul | 00:47 | |
| *** josefwells_ has quit IRC | 00:54 | |
| *** bhagyashris_ has joined #zuul | 01:01 | |
| *** bhagyashris has quit IRC | 01:09 | |
| opendevreview | Ian Wienand proposed zuul/zuul master: web: update to latest xterm; disabled selection overrides https://review.opendev.org/c/zuul/zuul/+/795195 | 01:11 |
|---|---|---|
| opendevreview | Ian Wienand proposed zuul/zuul master: web: update to latest xterm; disabled selection overrides https://review.opendev.org/c/zuul/zuul/+/795195 | 01:14 |
| *** josefwells has quit IRC | 01:43 | |
| *** zenkuro has quit IRC | 02:03 | |
| opendevreview | Ian Wienand proposed zuul/zuul master: web: update to latest xterm; disabled selection overrides https://review.opendev.org/c/zuul/zuul/+/795195 | 03:00 |
| opendevreview | Ian Wienand proposed zuul/zuul master: web: update to latest xterm; disabled selection overrides https://review.opendev.org/c/zuul/zuul/+/795195 | 03:20 |
| *** hashar has joined #zuul | 04:23 | |
| *** opendevreview has quit IRC | 04:26 | |
| *** marios has joined #zuul | 05:06 | |
| *** opendevreview has joined #zuul | 05:30 | |
| opendevreview | Merged zuul/zuul master: Fix layout cache and adding a project to a pipeline https://review.opendev.org/c/zuul/zuul/+/795189 | 05:30 |
| *** mhu1 has joined #zuul | 05:33 | |
| *** mhu has quit IRC | 05:37 | |
| *** hashar has quit IRC | 05:40 | |
| *** bhagyashris_ is now known as bhagyashris | 06:08 | |
| opendevreview | Merged zuul/zuul master: Fix reporting merge failure behind change adding ppc https://review.opendev.org/c/zuul/zuul/+/795190 | 06:19 |
| *** WanShan[m] has joined #zuul | 06:22 | |
| *** mhu1 has quit IRC | 06:52 | |
| opendevreview | Felix Edel proposed zuul/zuul master: Switch to ZooKeeper backed merge result events https://review.opendev.org/c/zuul/zuul/+/784195 | 07:05 |
| *** rpittau|afk is now known as rpittau | 07:30 | |
| avass[m] | Thanks for quick reviews on 795189 and 795190! | 07:30 |
| *** jpena|off is now known as jpena | 07:31 | |
| opendevreview | Merged zuul/zuul master: scheduler: call stop on SIGTERM https://review.opendev.org/c/zuul/zuul/+/794035 | 07:39 |
| *** mhu has joined #zuul | 07:42 | |
| *** hashar has joined #zuul | 07:49 | |
| ianw | tobiash[m]: could you try turning up the "vh" via inspecting the "terminal" div? | 07:53 |
| ianw | the height attribute I mean @ https://review.opendev.org/c/zuul/zuul/+/795195/4/web/src/pages/Stream.jsx | 07:54 |
| ianw | it's currently at 80, which i believe is 80% of the vertical height | 07:54 |
| ianw | maybe 90% is better? | 07:54 |
| tobiash[m] | 90% is too much, 85% looks better | 07:57 |
| opendevreview | Ian Wienand proposed zuul/zuul master: web: update to latest xterm; disabled selection overrides https://review.opendev.org/c/zuul/zuul/+/795195 | 08:00 |
| *** tosky has joined #zuul | 08:39 | |
| fbo | Hi, could nodepool-builder be used to build and a push AMI to AWS for EC2 ? | 08:49 |
| avass[m] | fbo: I think there's a patch for that | 08:49 |
| avass[m] | fbo: https://review.opendev.org/c/zuul/nodepool/+/735217 | 08:50 |
| avass[m] | fbo: there's also the option to build images in zuul: https://review.opendev.org/c/zuul/zuul-jobs/+/787677 though I may need to update that change | 08:52 |
| avass[m] | that ^ also tends to leak images unless coupled with a periodic cleanup | 08:53 |
| avass[m] | so far I like doing image builds in zuul since it | 08:54 |
| avass[m] | it's easy to reuse ensure-* roles, and as long as they're idempotent the jobs using the images can skip installation of whatever tool that is needed or install the correct it if it has the wrong version etc | 08:56 |
| fbo | avass[m]: Thanks a lot for the links. If I understand well you don't use nodepool-builder to build AMI but instead a Zuul job to configure and then promote an AMI ? | 08:59 |
| avass[m] | pretty much yeah, since we're also building windows images that makes it a bit easier | 08:59 |
| fbo | Ok may use case will be a good occasion to try 735217. | 09:00 |
| fbo | avass[m]: Also do you have information about the IAM profile that need to be setup for the Nodepool user https://lists.zuul-ci.org/pipermail/zuul-discuss/2021-June/001633.html ? | 09:01 |
| avass[m] | fbo: I took a look at it a long time ago, but I don't think we have it documented anywhere | 09:04 |
| fbo | Ok we'll see if someone can bring this informations. Thanks again avass[m] for the help | 09:10 |
| opendevreview | Ian Wienand proposed zuul/zuul master: web: update to latest xterm; disabled selection overrides https://review.opendev.org/c/zuul/zuul/+/795195 | 09:28 |
| opendevreview | Simon Westphahl proposed zuul/zuul master: Refactor config/tenant (re-)loading https://review.opendev.org/c/zuul/zuul/+/795263 | 09:34 |
| opendevreview | Simon Westphahl proposed zuul/zuul master: Refactor config/tenant (re-)loading https://review.opendev.org/c/zuul/zuul/+/795263 | 09:48 |
| opendevreview | Simon Westphahl proposed zuul/zuul master: Refactor config/tenant (re-)loading https://review.opendev.org/c/zuul/zuul/+/795263 | 10:20 |
| tobiash[m] | fbo: I think I can dig up some infos after lunch | 10:52 |
| *** zenkuro has joined #zuul | 11:08 | |
| *** jpena is now known as jpena|lunch | 11:31 | |
| tobiash[m] | fbo: this is what we came up with during our testing with aws: http://paste.openstack.org/show/806456/ | 12:02 |
| *** jpena|lunch is now known as jpena | 12:31 | |
| mnaser | corvus, avass[m]: newest version of cert manager supports generating jks files with passwords | 12:34 |
| mnaser | just used it for zk, worked nicely. pretty dope | 12:34 |
| avass[m] | mnaser: Cool, I've been meaning to take a look at cert manager for zk anyway | 12:50 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Test zuul-client console-stream https://review.opendev.org/c/zuul/zuul/+/794854 | 13:11 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: ci: fix helm chart path https://review.opendev.org/c/zuul/zuul-helm/+/795323 | 13:12 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Allow overriding image names, tags and pull policy https://review.opendev.org/c/zuul/zuul-helm/+/795112 | 13:12 |
| fbo | tobiash[m]: oh nice ! thanks you ! | 13:17 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Add extraVolumes and extraVolumeMounts https://review.opendev.org/c/zuul/zuul-helm/+/795326 | 13:19 |
| *** josefwells has joined #zuul | 13:33 | |
| *** josefwells_ has joined #zuul | 13:38 | |
| opendevreview | Mohammed Naser proposed zuul/zuul-jobs master: Bump default Helm version to 2.17.0 https://review.opendev.org/c/zuul/zuul-jobs/+/795329 | 13:40 |
| mnaser | going down the rabbit hole | 13:40 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: ci: fix helm chart path https://review.opendev.org/c/zuul/zuul-helm/+/795323 | 13:41 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Allow overriding image names, tags and pull policy https://review.opendev.org/c/zuul/zuul-helm/+/795112 | 13:41 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Add extraVolumes and extraVolumeMounts https://review.opendev.org/c/zuul/zuul-helm/+/795326 | 13:41 |
| *** josefwells has quit IRC | 13:44 | |
| *** josefwells__ has joined #zuul | 13:55 | |
| opendevreview | Benjamin Schanzel proposed zuul/zuul-jobs master: Add a meta log upload role with a failover mechanism https://review.opendev.org/c/zuul/zuul-jobs/+/795336 | 13:56 |
| opendevreview | Benjamin Schanzel proposed zuul/zuul-jobs master: Add a meta log upload role with a failover mechanism https://review.opendev.org/c/zuul/zuul-jobs/+/795336 | 13:59 |
| *** josefwells_ has quit IRC | 14:02 | |
| corvus | mnaser: is using a password advantageous in a k8s environment? i guess you could put a tighter rbac on a secret containing the password than the private key? | 14:21 |
| mnaser | corvus: it may just be me but it seems to be required by certmgr, i couldnt get it to gernate it without it | 14:22 |
| corvus | mnaser: (because if they had the same access levels, then it doesn't seem like much of a benefit -- they're both getting stored in etcd and accesible with the api) | 14:22 |
| mnaser | or maybe that was just me messing around with things too much and it didnt end up working out | 14:22 |
| mnaser | it may be possible to do it without it (and maybe i just couldnt get it to work properly and ENOTIME) | 14:23 |
| mnaser | seems liek zuul-helm needs a lot of work to get its func tests to pass :( | 14:23 |
| mnaser | mostly around tls/zk stuff | 14:24 |
| avass[m] | speaking of zuul-helm, I think we're gonna try out the zuul-operator soon. If anyone has time over I would be grateful if people took some time to review that stack :) | 14:24 |
| corvus | mnaser: oh, yeah i think i'm doing pkcs8 w/o password, so i think it's possible | 14:24 |
| corvus | mnaser: https://gerrit.googlesource.com/zuul/ops/+/refs/heads/master/k8s/zookeeper/certs.yaml | 14:24 |
| corvus | mnaser: hth; copy at will :) | 14:25 |
| opendevreview | Simon Westphahl proposed zuul/zuul master: Refactor config/tenant (re-)loading https://review.opendev.org/c/zuul/zuul/+/795263 | 14:26 |
| avass[m] | corvus, mnaser I'm not sure why I didn't take a look at cert-manager earlier. seems much more simple | 14:26 |
| avass[m] | I mean I'm using it for my own setup to manage ACME generated certs. just didn't occur to me that it was more generic | 14:27 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Allow disabling persistence https://review.opendev.org/c/zuul/zuul-helm/+/795350 | 14:27 |
| mnaser | avass[m]: yeah it's very neat! | 14:27 |
| mnaser | corvus: i will.. i'll just have to get around it, i'm a bit behind on a bunch of stuff, so i'm testing the fixes locally but would take a hot minute to do all that zuul-system stuff and get it tested/passing | 14:28 |
| avass[m] | Do we have a role in zuul-jobs to manage pip indexes by the way? extra-index-url and index-url in pip.conf? | 14:31 |
| avass[m] | oh we do for mirrors, but it doesn't look like it handles adding extra-index-url if it runs multiple times | 14:32 |
| avass[m] | and it doesn't look like it handles a list of mirrors either | 14:33 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Test zuul-client console-stream https://review.opendev.org/c/zuul/zuul/+/794854 | 14:40 |
| mnaser | avass[m]: ever seen something like centos-8 builds failing with dib ? | 14:46 |
| mnaser | http://paste.openstack.org/show/806459/ | 14:46 |
| mnaser | i wonder if i need to mount /sys/fs into the host or something | 14:47 |
| avass[m] | mnaser: no clue actually | 14:48 |
| mnaser | i think it fails to exec selinuxenabled bc the container is actually ubuntu | 14:50 |
| mnaser | or debain | 14:50 |
| mnaser | oh hold on | 14:51 |
| mnaser | http://paste.openstack.org/show/806460/ | 14:52 |
| mnaser | i think this is the actual failure | 14:52 |
| mnaser | man /dev/loop is always a pita inside containers | 14:52 |
| avass[m] | mnaser: apparently I've visited this before: https://serverfault.com/a/720496 | 14:55 |
| avass[m] | probably when setting up nodepool-builder without a privileged container | 14:56 |
| mnaser | yeah right now its running as privileged | 14:56 |
| mnaser | so i think it should be ok? | 14:56 |
| avass[m] | I'd guess so | 14:57 |
| mnaser | already passing /dev in the helm charts too sigh | 15:01 |
| mnaser | i bet this is a selinux thing | 15:02 |
| mnaser | ok, caught it | 15:05 |
| mnaser | dib isnt actually unmounting everything | 15:06 |
| mnaser | it runs: `sudo umount /tmp/dib_build.VoEuQcgV/mnt/` | 15:06 |
| mnaser | but in this case, this is till mounted `/dev/mapper/loop2p1 on /tmp/tmp.wBySdrDjUH` | 15:06 |
| mnaser | but i think its the selinux failure that is causing a bad cleanup | 15:07 |
| clarkb | dib exit handlers deal with cleanup including unmounting things. I think we have seen cases where dib being forcefully stopped can break that. Also possible that the cleanup cannot succeed due to perms enforced by selinux maybe? | 15:07 |
| mnaser | i think the state that it fails in actually just doesnt umount that part cleanly | 15:07 |
| mnaser | the real failure is the selinux, that other one is just in the trap_cleanuup | 15:08 |
| clarkb | ya also possible the exit handler isn't properly configured at that point | 15:08 |
| mnaser | i guess this might be because selinux = enabled on host, but container is running ubuntu, so /sys/fs/cgroup is there but `selinuxenabled` returns file not found | 15:09 |
| clarkb | if you are using the containers we publish they are not ubuntu, but are debian | 15:10 |
| mnaser | sorry, right, yes | 15:10 |
| fungi | avass[m]: for pip extra indices, i'm sure we could extend how that's managed, but also be aware that it may not work the way you expect (pip checks every index, and then decides which package it finds is the best fit based on distinctions like wheel vs sdist, platform, higher version numbers, et cetera) | 15:10 |
| mnaser | https://meetings.opendev.org/irclogs/%23openstack-infra/%23openstack-infra.2017-07-19.log.html#t2017-07-19T22:16:52 | 15:11 |
| mnaser | back from 2017, lol | 15:11 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Test zuul-client console-stream https://review.opendev.org/c/zuul/zuul/+/794854 | 15:21 |
| mnaser | welp | 15:25 |
| mnaser | disabling selinux on the host fixed it | 15:25 |
| fungi | all problems are solved by turning off security ;) | 15:28 |
| fungi | did it log which controls were being violated? | 15:28 |
| fungi | was it the same setfiles problem for the systemd modules list? | 15:30 |
| mnaser | fungi: im not sure, it was different files each time, plus this is fedora coreos | 15:32 |
| mnaser | running zuul on a magnum k8s cluster | 15:32 |
| mnaser | so there's all kinda of, weird. | 15:32 |
| fungi | exciting | 15:32 |
| mordred | mnaser: so - fedora coreos on the host, debian in the container, yeah? | 15:41 |
| mnaser | mordred: yeah | 15:41 |
| clarkb | https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/using_selinux/creating-selinux-policies-for-containers_using-selinux | 15:45 |
| clarkb | I suspect that tool wont work here but confirms the general need for container specific policies | 15:47 |
| *** eliadcohen has quit IRC | 15:50 | |
| mnaser | arr | 16:01 |
| mnaser | fedora-release-common-33-4.noarch has missing requires of fedora-release-identity = ('0', '33', '4') | 16:01 |
| *** marios is now known as marios|out | 16:02 | |
| avass[m] | fungi: yeah, we're in an interesting situation where our mirros are slower (and less reliable) than pypi and at the same time we need to hook up extra indices to be able to install some packages | 16:10 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Test zuul-client console-stream https://review.opendev.org/c/zuul/zuul/+/794854 | 16:16 |
| *** rpittau is now known as rpittau|afk | 16:31 | |
| *** spamaps[m] has joined #zuul | 16:32 | |
| spamaps[m] | o/ | 16:32 |
| *** jpena is now known as jpena|off | 16:33 | |
| corvus | spamaps: \o | 16:36 |
| mordred | yay it's a spamaps ! | 16:38 |
| *** marios|out has quit IRC | 16:39 | |
| mnaser | ianw: happen to by any chance know if/when/why/how f33+ can work with dib? | 17:00 |
| gtema | Same would be valid for f34 | 17:00 |
| mnaser | yeah, it seems like f33 and f34 are failing in the same way | 17:00 |
| clarkb | mnaser: as build targets? | 17:01 |
| gtema | Not absolutely, but very close (rpm installation) iirc | 17:01 |
| clarkb | f34 is being supported through the docker image base element, not the normal fedora element | 17:01 |
| fungi | i think the hope was the switch to basing builds on docker containers would address most of the problem there? | 17:01 |
| clarkb | fungi: yes that | 17:01 |
| fungi | yeah, what clarkb said | 17:01 |
| mnaser | oh i see | 17:01 |
| mnaser | so i need to play around with my build stuff and not use fedora-minimal | 17:02 |
| mnaser | (it seems like opendev hasn't gotte naround it yet so using project-config/nodepool as reference isnt as useful =P) | 17:02 |
| fungi | yeah basically the same "it's hard to find one platform which has packages of all other platforms' package managers at new enough versions" | 17:02 |
| *** eliadcohen has joined #zuul | 17:02 | |
| clarkb | but also fedora is notorious for changing everything every 6 months | 17:03 |
| clarkb | of all the distros dib supports it probably represents the most investment for fixing/updating as a result | 17:04 |
| gtema | For others you spend more time addressing issues and new packages through zuul-jobs | 17:04 |
| mnaser | https://opendev.org/openstack/diskimage-builder/commit/80ff74ca4d14990ffd2a50bc5fe11b4d548a3575 | 17:05 |
| clarkb | gtema: sure, but thats far more scalable | 17:05 |
| fungi | but we do also spend plenty of time catering to fedora behavior changes in the playbooks/roles in zuul-jobs | 17:06 |
| clarkb | as it doesn't require a new image be rebuilt and uploaded over and over and over until it works. | 17:06 |
| clarkb | mnaser: thats the one | 17:06 |
| * mnaser really hopes that's released or will have to build their own nodepool image | 17:06 | |
| clarkb | mnaser: ianw is pretty good about making releases on request if it hasn't been done yet | 17:07 |
| mnaser | yeah it's not in unfortunately | 17:07 |
| mnaser | bumma | 17:08 |
| mnaser | is there a clean way of rm -f'ing dib images from nodepool? | 17:19 |
| clarkb | nodepool image-delete ? | 17:20 |
| *** rav has joined #zuul | 17:20 | |
| rav | Hello | 17:20 |
| clarkb | hello | 17:20 |
| rav | I had a project which was configured to run zuul and it runs the scripts to test the commits. Initially it used to run py27 but now i have migrated my whole repo to python 3.6. But my zuul job still runs python27. How do i change this? Im using this for the first time hence not sure how to do it. My tox.ini runs 100% fine in my environment | 17:21 |
| rav | https://zuul.opendev.org/t/openstack/build/11645d4d4aeb4d33b9cec2b017a7eeaa i get this failure. but my tox has only py36 and pep8 requirement | 17:22 |
| mordred | rav: that's the openstack-tox-py27 job, which is going to run "tox -epy27" - you should update it to be openstack-tox-py36 in your .zuul.yaml file | 17:24 |
| clarkb | mordred: rav: except that project doesn't have a .zuul.yaml file it is all controleld in openstack/project-config | 17:24 |
| clarkb | https://opendev.org/openstack/project-config/src/branch/master/zuul.d/projects.yaml#L7234-L7238 | 17:24 |
| mordred | ah - well - step one - migrate the jobs to the project :) | 17:25 |
| clarkb | what you can do is cleanup the entry in project-config to remove openstack-python-jobs-neutron and replace it with the jobs you want in your in repo .zuul.yaml | 17:25 |
| clarkb | rav: the comment in https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/zuul.d/project-templates.yaml#L360-L392 tells you what you should use instead too | 17:26 |
| mnaser | clarkb: those are listed inside dib-image-list | 17:53 |
| mnaser | and when i issue a dib-image-delete, their age resets and then they go bakc in 'deleting' state | 17:53 |
| clarkb | mnaser: the dib records stick around until the cloud records go away. That would imply that you've got something preventing upstream image deletes I think | 17:54 |
| mnaser | clarkb: image-list shows 2 images only, which correspond to the two ready ones | 17:54 |
| mnaser | this did go some weird restarts-during-builds that may have gotten it in a weird state | 17:55 |
| clarkb | mnaser: only the builder that built the images can delete them iirc | 17:55 |
| clarkb | if the identity of the builder changed bceause you started new containers for example then that could explain it | 17:55 |
| mnaser | it hasn't :( but the files did disappear from beneath it during builds | 17:56 |
| mnaser | https://www.irccloud.com/pastebin/yAgZI1cp/ | 17:56 |
| clarkb | I would check the builder logs to see why it isn't deleting them then | 17:57 |
| fungi | mnaser: one reason we've seen in opendev is when providers use bfv, glance can't delete the images until all instances booted from them are also culled | 17:58 |
| fungi | and if there are server instances stuck in deleting state for unrelated reasons, that indefinitely blocks image removal | 17:59 |
| clarkb | ya but in that case I would've expected images in a deleting state reported by image-list | 17:59 |
| fungi | oh, yep | 18:00 |
| fungi | i agree, looking at that paste, i can't recall seeing a builder fail to delete the local image after it's successfully deleted the remote copies (these days it even deletes the local copies first without waiting for remote copies to disappear, i think?) | 18:03 |
| fungi | there are now corresponding records in zk for those, right? | 18:04 |
| clarkb | yes everything should be in zk, and while it deletes the on disk copies it won't remove the zk records until steady state in the cloud iirc | 18:04 |
| clarkb | I would definitely look for a traceback or error log message on the builder for why it isn't cleaning those up | 18:04 |
| fungi | yes, that should hopefully point to corrupted znodes vs filesystem errors vs whatever | 18:05 |
| opendevreview | James E. Blair proposed zuul/zuul master: Move build request cleanup from executor to scheduler https://review.opendev.org/c/zuul/zuul/+/794687 | 18:07 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Fix compatiblity with latest Zuul https://review.opendev.org/c/zuul/zuul-helm/+/795391 | 18:12 |
| *** Shrews has joined #zuul | 18:22 | |
| rav | I have made the changes: https://review.opendev.org/c/openstack/project-config/+/795396 can someone review it? | 18:33 |
| clarkb | rav: I just did, we should probaby move the discussion to #opendev now as this isn't zuul specific | 18:34 |
| rav | ok | 18:34 |
| ianw | mnaser: yeah as mentioned the fedora elements should work with containerfile; but i haven't got a release yet. as usual it's a yak shaving exercise :) | 19:02 |
| ianw | my idea is to reduce the support platform by running functional tests from the nodepool container. that means we only have to worry about maintaining one environment to build images | 19:03 |
| ianw | https://review.opendev.org/c/openstack/diskimage-builder/+/791888 | 19:04 |
| ianw | of course, i chose the 3.9 container, because that's the latest :) | 19:04 |
| clarkb | ianw: sounds like selinux may break that a bit, but ya its a good idea | 19:04 |
| fungi | only the freshest bugs for you | 19:05 |
| ianw | the problem is now that we don't have arm64 wheels, so building arm64 images is super slow | 19:05 |
| ianw | and times out the job | 19:06 |
| ianw | i started thinking about that, and then freenode decided to explode, so here we are :) | 19:06 |
| clarkb | why don't we have arm64 wheels? are there deps that dont' overlap with openstack and/or already publish arm64 wheels? | 19:08 |
| ianw | sorry we don't have python 3.9 arm wheels to be exact | 19:11 |
| clarkb | oh I see | 19:11 |
| clarkb | maybe we shoudl go ahead and build those since we expect it to be a thing in the future? | 19:11 |
| ianw | yes :) but the problem is that currently we build wheels in a platform manner -- bionic wheels, focal wheels, etc. | 19:12 |
| clarkb | ah and buster doesn't have 3.9 as an option | 19:12 |
| ianw | i think we need to switch to using something like a container to build python-version wheels | 19:12 |
| ianw | or i dunno, that's where i got to :) | 19:12 |
| fungi | right, debian/bullseye has 3.9, but it's not in buster (or even buster-backports apparently) | 19:15 |
| *** Shrews has quit IRC | 19:16 | |
| ianw | an option to expedite this, which i'm not totally opposed to, is to update nodepool to 3.8 instead of 3.9 | 19:21 |
| ianw | i was hoping to come up with something that made wheel building all around better, though | 19:21 |
| opendevreview | James E. Blair proposed zuul/zuul master: Add ExecutorApi https://review.opendev.org/c/zuul/zuul/+/770902 | 19:50 |
| opendevreview | James E. Blair proposed zuul/zuul master: Change zone handling in ExecutorApi https://review.opendev.org/c/zuul/zuul/+/787833 | 19:50 |
| opendevreview | James E. Blair proposed zuul/zuul master: Switch to string constants in BuildRequest https://review.opendev.org/c/zuul/zuul/+/791849 | 19:50 |
| opendevreview | James E. Blair proposed zuul/zuul master: Clean up Executor API build request locking and add tests https://review.opendev.org/c/zuul/zuul/+/788624 | 19:50 |
| opendevreview | James E. Blair proposed zuul/zuul master: Fix race with watches in ExecutorAPI https://review.opendev.org/c/zuul/zuul/+/792300 | 19:50 |
| opendevreview | James E. Blair proposed zuul/zuul master: Execute builds via ZooKeeper https://review.opendev.org/c/zuul/zuul/+/788988 | 19:50 |
| opendevreview | James E. Blair proposed zuul/zuul master: Move build request cleanup from executor to scheduler https://review.opendev.org/c/zuul/zuul/+/794687 | 19:50 |
| *** rav has quit IRC | 19:51 | |
| corvus | that's a rebase; there's some interaction with the config cache now that it's merged | 20:01 |
| opendevreview | Tristan Cacqueray proposed zuul/zuul-jobs master: fetch-translation-output: introduce zuul_use_fetch_output https://review.opendev.org/c/zuul/zuul-jobs/+/681887 | 20:12 |
| opendevreview | Tristan Cacqueray proposed zuul/zuul-jobs master: fetch-javascript-content-tarball: introduce zuul_use_fetch_output https://review.opendev.org/c/zuul/zuul-jobs/+/681903 | 20:12 |
| tristanC | turns out we missed a couple of patches, and zuul-jobs javascript roles does not work in container without them ^ | 20:13 |
| opendevreview | James E. Blair proposed zuul/zuul master: Move build request cleanup from executor to scheduler https://review.opendev.org/c/zuul/zuul/+/794687 | 20:17 |
| corvus | tristanC: are those just rebases? | 20:18 |
| tristanC | corvus: yes | 20:18 |
| corvus | cool. +2 reapplied :) | 20:19 |
| avass[m] | tristanC: looking | 20:19 |
| tristanC | corvus: avass[m]: thank you! | 20:19 |
| avass[m] | tristanC: 681887 fails linting | 20:20 |
| avass[m] | otherwise lgtm | 20:21 |
| avass[m] | both fail linting :) | 20:21 |
| opendevreview | Tristan Cacqueray proposed zuul/zuul-jobs master: fetch-translation-output: introduce zuul_use_fetch_output https://review.opendev.org/c/zuul/zuul-jobs/+/681887 | 20:26 |
| opendevreview | Tristan Cacqueray proposed zuul/zuul-jobs master: fetch-javascript-content-tarball: introduce zuul_use_fetch_output https://review.opendev.org/c/zuul/zuul-jobs/+/681903 | 20:26 |
| opendevreview | Tristan Cacqueray proposed zuul/zuul-jobs master: fetch-translation-output: introduce zuul_use_fetch_output https://review.opendev.org/c/zuul/zuul-jobs/+/681887 | 20:43 |
| avass[m] | tristanC: ping me when they pass | 20:44 |
| opendevreview | Tristan Cacqueray proposed zuul/zuul-jobs master: fetch-javascript-output: introduce zuul_use_fetch_output https://review.opendev.org/c/zuul/zuul-jobs/+/681748 | 20:48 |
| avass[m] | 795190, and 795189 fixed the reporting issues we were having :) | 20:54 |
| mnaser | ianw: do you think we can get a dib release alongside a zuul image rebuild? | 21:00 |
| *** hashar has quit IRC | 21:06 | |
| ianw | mnaser: for the containerfile element? | 21:10 |
| ianw | or was there something else you needed? | 21:10 |
| mnaser | ianw: containerfile so I can build f34 :) | 21:11 |
| ianw | ok sounds reasonable; we have that building in the dib gate | 21:12 |
| opendevreview | Ian Wienand proposed zuul/nodepool master: Update dib to 3.12.0 https://review.opendev.org/c/zuul/nodepool/+/795416 | 21:14 |
| ianw | mnaser: ^ :) | 21:15 |
| mnaser | ianw: yay, awesome | 21:19 |
| opendevreview | Tim Burke proposed zuul/zuul-jobs master: build-python-release: Add flag for whether to build a wheel or not https://review.opendev.org/c/zuul/zuul-jobs/+/795419 | 21:25 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Add keystore password https://review.opendev.org/c/zuul/zuul-helm/+/795421 | 21:28 |
| tristanC | avass[m]: https://review.opendev.org/c/zuul/zuul-jobs/+/681887 should be good, the other one needs more work | 21:31 |
| avass[m] | corvus: I'll carry your +2 on that and approve it since it's just linting | 21:33 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Test zuul-client console-stream https://review.opendev.org/c/zuul/zuul/+/794854 | 21:34 |
| opendevreview | Merged zuul/zuul-jobs master: fetch-translation-output: introduce zuul_use_fetch_output https://review.opendev.org/c/zuul/zuul-jobs/+/681887 | 21:44 |
| corvus | avass: ++ | 21:53 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Add extraVolumes and extraVolumeMounts https://review.opendev.org/c/zuul/zuul-helm/+/795326 | 22:03 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Allow disabling persistence https://review.opendev.org/c/zuul/zuul-helm/+/795350 | 22:03 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Fix compatiblity with latest Zuul https://review.opendev.org/c/zuul/zuul-helm/+/795391 | 22:03 |
| opendevreview | Mohammed Naser proposed zuul/zuul-helm master: Add keystore password https://review.opendev.org/c/zuul/zuul-helm/+/795421 | 22:03 |
| clarkb | ianw: fwiw I'm not sure there is a major reason to prefer 3.9 over 3.8 in code bases that aren't being updated to use switch statements :) | 22:19 |
| clarkb | at least not yet | 22:20 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Test zuul-client console-stream https://review.opendev.org/c/zuul/zuul/+/794854 | 22:30 |
| ianw | clarkb: yeah, we are pretty tied to distros with our wheel infrastructure anyway. i've gone down the path before of trying to make manylinux wheels but that really needs to be handled on a per-package basis | 22:40 |
| ianw | apropos features, i do like f-strings though, and will zuul >=3.6 i've been putting them in some recent reviews | 22:42 |
| clarkb | f-strings are in 3.8 though right? I'm basically saying .9 isn't a major improvement over .8 and we can live with that if we want for now | 22:44 |
| mordred | yeah - I think that's right | 22:45 |
| mordred | I feel like f-strings are the best syntactic sugar since list comprehensions | 22:45 |
| fungi | and breakfast burritos | 22:49 |
| ianw | oh yeah f-strings are >=3.6; i agree 3.9 isn't super important; it was just more the principle of trying to use the latest | 23:02 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Test zuul-client console-stream https://review.opendev.org/c/zuul/zuul/+/794854 | 23:02 |
| *** tosky has quit IRC | 23:03 | |
| ianw | btw 793704,22 on https://zuul.openstack.org/status has been stuck there for a day or so now, without any jobs under it | 23:18 |
| corvus | ianw: i'm having trouble finding 793704 on https://zuul.openstack.org/status | 23:21 |
| corvus | ctrl-f is failing me | 23:21 |
| ianw | ok, after a hard refresh it's gone for me | 23:22 |
| corvus | clarkb, ianw: what's using 3.9? | 23:22 |
| ianw | corvus: nothing; https://review.opendev.org/c/zuul/nodepool/+/789655 was proposing it but ... yeah | 23:23 |
| ianw | i have definitely been seeing "fresh" data on my zuul status page; but also that change was stuck on my page as well | 23:23 |
| corvus | re 39, ack, thx. | 23:24 |
| corvus | hrm; wonder if it's a js bug not clearing old stuff out or something | 23:24 |
| ianw | i think there must be something like that lurking; i have a pinned status tab so it's pretty much always there | 23:25 |
| ianw | i have definitely been jumping on that tab and looking at consoles for jobs i've uploaded, so it was getting fresh data | 23:25 |
| ianw | (highlighting ip addresses for hosts was what prompted me to update xterm https://review.opendev.org/c/zuul/zuul/+/795195) | 23:27 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Test zuul-client console-stream https://review.opendev.org/c/zuul/zuul/+/794854 | 23:28 |
| mhu | (the struggle is real - but I think I got it this time) | 23:29 |
| mnaser | hmm | 23:41 |
| mnaser | opendev doesn't use podman or does it | 23:41 |
| mnaser | i'm seeing a very strange error | 23:42 |
| mnaser | https://telecominfraproject.zuul.vexxhost.dev/build/749439769d8d489397cfea1c88011014 | 23:42 |
| mnaser | also yay finally zuul 4.x | 23:42 |
| clarkb | mnaser: we tried to use podman but it turns out its docker compat is poor | 23:42 |
| mnaser | that is what i remember | 23:42 |
| clarkb | specifically around bindmounts they have different behaviors | 23:42 |
| mnaser | chown giving 'invalid argument' seems very confusing | 23:42 |
| clarkb | also we specifically ran into problems with it and nodepool images iirc | 23:42 |
| clarkb | because you want to mount things that are bindmounted and podman is more restrictive about that than docker or something along those lines | 23:43 |
| mnaser | https://github.com/containers/podman/blob/master/troubleshooting.md#10-rootless-setup-user-invalid-argument | 23:44 |
| mnaser | i wonder if it is this | 23:44 |
| mnaser | well, nevermind, it looks like magnum is using docker | 23:47 |
| mnaser | time to look at opendev's roles! | 23:48 |
| mnaser | only difference i spot is that you're running network_mode=host | 23:49 |
| clarkb | mnaser: I think a uid/gid may be set on those images, possible you need to map those? | 23:49 |
| mnaser | clarkb: yeah but my podman theory was an incorrect assumption, they use docker | 23:50 |
| clarkb | we set user: zuul on our docker-compose services and then there is a zuul user in the image | 23:50 |
| mnaser | ooh interesting | 23:50 |
| clarkb | simiarly we set user: nodepool on the nodepool side | 23:51 |
| clarkb | this is curious. Using the USER directive in the dockerfile makes that the default user. But the zuul iamge seems to useradd in a shell and not use USER | 23:53 |
| clarkb | same thing with nodepool | 23:53 |
| clarkb | should we be setting USER in those images so that you don't need to explicitly set the user to zuul or nodepool? | 23:54 |
| clarkb | I don't know that this is the source of mnaser's problems but thought I'd call it out as an oddity | 23:54 |
| mnaser | interseting | 23:55 |
| mnaser | $ docker run -it --rm zuul/zuul-executor whoami => root | 23:55 |
| mnaser | yeah, im not sure why we'd not just "USER" in the image | 23:55 |
| clarkb | yup because we don't use the USER directive in the image | 23:55 |
| clarkb | we just run a shell command to useradd | 23:55 |
| clarkb | I wonder if the quickstart does this too /me looks | 23:56 |
| clarkb | looks like no. The quickstart does not set user: zuul | 23:57 |
| clarkb | maybe that means this isn't strictly required? | 23:57 |
| mnaser | well this is a weird scenario where the executor rsyncs back out from the system | 23:58 |
| mnaser | so maybe an edge case that generally might not be seen in quick start | 23:58 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!