| *** cdearborn has quit IRC | 00:47 | |
| *** rlandy|brb has quit IRC | 00:51 | |
| *** bhavikdbavishi has joined #zuul | 01:04 | |
| *** bhavikdbavishi1 has joined #zuul | 01:06 | |
| *** bhavikdbavishi has quit IRC | 01:08 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 01:08 | |
| *** swest has quit IRC | 01:09 | |
| *** swest has joined #zuul | 01:23 | |
| *** rfolco has quit IRC | 01:24 | |
| *** rfolco has joined #zuul | 01:25 | |
| *** Goneri has quit IRC | 01:43 | |
| *** rf0lc0 has joined #zuul | 02:03 | |
| *** bhavikdbavishi has quit IRC | 02:04 | |
| *** rfolco has quit IRC | 02:04 | |
| *** bhavikdbavishi has joined #zuul | 02:28 | |
| *** threestrands has joined #zuul | 02:33 | |
| *** mordred has quit IRC | 02:43 | |
| *** mordred has joined #zuul | 02:45 | |
| *** zxiiro has quit IRC | 03:30 | |
| *** wxy-xiyuan has quit IRC | 03:47 | |
| *** bhavikdbavishi has quit IRC | 03:47 | |
| *** wxy has joined #zuul | 03:49 | |
| *** zxiiro has joined #zuul | 04:08 | |
| *** bhavikdbavishi has joined #zuul | 04:23 | |
| *** evrardjp has quit IRC | 04:35 | |
| *** evrardjp has joined #zuul | 04:35 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul master: [wip] intermediate jobs https://review.opendev.org/722200 | 05:29 |
|---|---|---|
| *** reiterative has quit IRC | 05:36 | |
| *** reiterative has joined #zuul | 05:36 | |
| *** sgw has quit IRC | 05:55 | |
| *** dpawlik has joined #zuul | 06:01 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul master: [wip] intermediate jobs https://review.opendev.org/722200 | 06:07 |
| openstackgerrit | Ian Wienand proposed zuul/zuul master: [wip] intermediate jobs https://review.opendev.org/722200 | 06:08 |
| *** ysandeep is now known as ysandeep|afk | 06:28 | |
| openstackgerrit | Merged zuul/nodepool master: Actually install extras from nodepool_base https://review.opendev.org/722135 | 06:28 |
| AJaeger | mordred: the config-errors for the openstack tenant come from a retirement, we need to remove a couple of requireed-projects from old branches. noonedeadpunk has patches up for those | 06:30 |
| avass | mordred: oh, too bad, I thought it was just the tests that was failing, at least the build was green ;) | 06:39 |
| openstackgerrit | Ian Wienand proposed zuul/zuul master: [wip] intermediate jobs https://review.opendev.org/722200 | 06:42 |
| *** jcapitao has joined #zuul | 07:07 | |
| *** ysandeep|afk is now known as ysandeep | 07:10 | |
| *** bhavikdbavishi has quit IRC | 07:20 | |
| *** tosky has joined #zuul | 07:24 | |
| *** yolanda has joined #zuul | 07:24 | |
| *** rpittau|afk is now known as rpittau | 07:39 | |
| avass | what happens if you have reporters for two different gerrit instances in the same pipeline? would that produce an error or does zuul only report to the instance relevant to the change? | 07:50 |
| *** bhavikdbavishi has joined #zuul | 07:50 | |
| avass | wondering if we're going to need separate pipelines for each gerrit instance in that case, we have repos that are dependent on eachother between them | 07:51 |
| avass | or if we should just migrate that repo over to the other gerrit instance :) | 07:55 |
| *** jpena|off is now known as jpena | 07:57 | |
| *** igordc has quit IRC | 08:13 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul master: [wip] intermediate jobs https://review.opendev.org/722200 | 08:14 |
| *** threestrands has quit IRC | 08:33 | |
| *** zxiiro has quit IRC | 08:40 | |
| tobiash | avass: zuul will only report to the source gerrit of the change | 08:41 |
| *** ysandeep is now known as ysandeep|lunch | 08:44 | |
| avass | tobiash: nice, thanks! | 08:45 |
| tobiash | avass: it's handled by this: https://opendev.org/zuul/zuul/src/branch/master/zuul/driver/gerrit/gerritreporter.py#L52 | 08:47 |
| avass | tobiash: I guess the 'require:' is separate as well then, because it won't be able to be open and current patachset in both gerrits :) | 08:49 |
| tobiash | yes | 08:49 |
| tobiash | we also have pipelines wich work on both github and a gerrit simultaneously | 08:50 |
| avass | nice | 08:51 |
| avass | tobiash: we're probably going to need to do something like that as well soon, nice to know it works | 08:54 |
| *** sshnaidm|afk has quit IRC | 08:55 | |
| avass | tobiash: re 722058, so soft dependency works like, if the dependency is not in the buildset the job will run, but if it is but was skipped the job will not run and not produce an error? | 08:56 |
| avass | tobiash: so the current do should be 'will be ignored if the dependent job is not in the buildset/not configured to run in the pipeline' | 08:58 |
| avass | s/do/doc | 08:59 |
| tobiash | a soft dependency will still produce an error if the dependent job had an error | 08:59 |
| tobiash | but not entirely sure what happens if the dependent job has been skipped | 09:00 |
| tobiash | I think it will still run it in this case | 09:01 |
| avass | I actually meant failed and not skipped. But I should probably investigate this a bit to be sure how it works. The docs could probably be a bit clearer on this | 09:02 |
| tobiash | if the parent job fails it will be skipped regardless of the type of dependency | 09:03 |
| tobiash | a soft dependency just doesn't require that the parent job ran at all | 09:03 |
| avass | right, I'm probably just confused | 09:04 |
| avass | since it says 'run'. | 09:07 |
| avass | anyway | 09:15 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: doc fix: containes -> contains https://review.opendev.org/722237 | 09:17 |
| avass | :) | 09:17 |
| *** ysandeep|lunch is now known as ysandeep | 09:33 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Connect merger to Zookeeper https://review.opendev.org/716221 | 10:07 |
| *** sshnaidm has joined #zuul | 10:14 | |
| jkt | do I have a way of finding out the reason behind "Merge Failed." when I don't have access to mnaser's hosted Zuul logs? | 10:18 |
| jkt | I have a change that I uploaded: https://review.gerrithub.io/c/Telecominfraproject/oopt-gnpy/+/488748/4, its git parent is one behind current tip of the target branch | 10:18 |
| jkt | the tip of the target branch has modified repo's top level README.md, the change in question modified some unrelated Python code | 10:19 |
| jkt | if I upload another change with the same parent, this time adding an unrelated file, Zuul doesn't complain | 10:19 |
| *** rpittau is now known as rpittau|bbl | 10:21 | |
| jkt | hmm, and now that I've left a "recheck" comment, the builds have started. Perhaps it was a transient failure in the merger? | 10:22 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Connect executor to Zookeeper https://review.opendev.org/716262 | 10:24 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Tune automatic garbage collection of git repos https://review.opendev.org/722272 | 10:45 |
| tobiash | zuul-maint: this contains the stuff I learned about automatic garbage collection in git ^ | 10:46 |
| *** jcapitao is now known as jcapitao_lunch | 10:47 | |
| *** bhavikdbavishi has quit IRC | 10:58 | |
| openstackgerrit | Merged zuul/zuul master: doc fix: containes -> contains https://review.opendev.org/722237 | 10:59 |
| avass | tobiash: cool, this looks like it could be useful for us as well | 11:13 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Connect fingergw to Zookeeper https://review.opendev.org/716875 | 11:16 |
| *** jpena is now known as jpena|lunch | 11:34 | |
| tristanC | corvus: rootless podman needs userns and centos requires a sysctl to enable unprivileged userns with : echo 10000 > /proc/sys/user/max_user_namespaces | 11:47 |
| tristanC | corvus: otherwise running podman as root works out of the box | 11:47 |
| *** rlandy has joined #zuul | 12:00 | |
| *** bhavikdbavishi has joined #zuul | 12:02 | |
| *** rpittau|bbl is now known as rpittau | 12:02 | |
| *** bhavikdbavishi1 has joined #zuul | 12:09 | |
| *** bhavikdbavishi has quit IRC | 12:11 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 12:11 | |
| *** mhu has joined #zuul | 12:12 | |
| mordred | AJaeger: yah - actually though, the config errors are helpful right now, as they're showing an unexplained flaw in a zuul dashboard patch | 12:28 |
| *** jcapitao_lunch is now known as jcapitao | 12:29 | |
| *** jpena|lunch is now known as jpena | 12:39 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: merger: replace custom message logger https://review.opendev.org/720266 | 12:43 |
| openstackgerrit | Paul Albertella proposed zuul/zuul-jobs master: Add Bazel build and ensure roles https://review.opendev.org/693513 | 12:51 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Add testing of fetch-sphinx-tarball role https://review.opendev.org/715028 | 12:53 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Improve linters execution https://review.opendev.org/722307 | 12:59 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Improve linters execution https://review.opendev.org/722307 | 13:01 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: hlint: add haskell source code suggestions jobs https://review.opendev.org/722309 | 13:02 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Switch remaining tests to fedora-31 https://review.opendev.org/722310 | 13:06 |
| *** igordc has joined #zuul | 13:07 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: hlint: add haskell source code suggestions job https://review.opendev.org/722309 | 13:14 |
| *** Goneri has joined #zuul | 13:15 | |
| *** zxiiro has joined #zuul | 13:16 | |
| *** bhavikdbavishi has quit IRC | 13:17 | |
| AJaeger | zbr, looking at 722310, I would expect our jobs to fail - why do they pass? | 13:25 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: hlint: add haskell source code suggestions job https://review.opendev.org/722309 | 13:26 |
| zbr | AJaeger: why to fail? my only worry is that migration happened by missing these, not sure why. | 13:31 |
| *** ysandeep is now known as ysandeep|away | 13:38 | |
| *** panda is now known as panda|babysit | 13:43 | |
| AJaeger | zbr: asked differently why does https://review.opendev.org/722310 not update zuul-tests.d/container-roles-jobs.yaml ? | 13:44 |
| AJaeger | zbr: changes look fine, I'll review later again, just way puzzled by it and had no time to dig further | 13:45 |
| openstackgerrit | Merged zuul/zuul master: Tune automatic garbage collection of git repos https://review.opendev.org/722272 | 13:47 |
| AJaeger | zbr: ah, it does not handle node changes... | 13:47 |
| zbr | as long we merge it, we should be fine. | 13:47 |
| *** sgw has joined #zuul | 13:54 | |
| tobiash | corvus: I've responded on 720458 | 13:59 |
| AJaeger | zbr: yep | 13:59 |
| *** sshnaidm has quit IRC | 14:01 | |
| corvus | tobiash: ok. i was hoping we'd get a test case out of it, since it seems like something we'd like to avoid regressing in the future (it's really tempting for us to want to "simplify" the git processing code, and without regression tests, we'd reintroduce stuff like that) | 14:01 |
| AJaeger | zuul-jobs reviewer, please have a look at https://review.opendev.org/722307 https://review.opendev.org/722310 https://review.opendev.org/715028 https://review.opendev.org/721796 https://review.opendev.org/#/c/721248/ - all smaller fixes | 14:02 |
| corvus | tobiash: but if we just can't figure out how repos get in that state, not much we can do :) | 14:02 |
| tobiash | corvus: I know, but so far I had no luck with a direct reproducer :/ | 14:02 |
| AJaeger | zbr: want to abandon https://review.opendev.org/#/c/721571/ ? | 14:02 |
| *** sshnaidm has joined #zuul | 14:02 | |
| zbr | AJaeger: probably. but that change undelined several problem we still need to address. | 14:10 |
| *** sassyn has joined #zuul | 14:39 | |
| sassyn | Hi Amazing Group | 14:39 |
| sassyn | I wanted to let you know we started zuul in our production! | 14:40 |
| sassyn | the team~400 developers are happy! | 14:40 |
| sassyn | But I still have one issue to take care of | 14:40 |
| sassyn | My Question: User X commit a patch to the gerrit repo A. User Y and Z also commit patches to repo A. | 14:41 |
| sassyn | so we have PatchX, PatchY and PatchZ in that order | 14:41 |
| sassyn | all pacthes were triggered by zuul and got +1 verified! | 14:42 |
| sassyn | User BOSS go to gerrit and mark patchX and PatchZ as +2 while patchX as -2. | 14:42 |
| AJaeger | sassyn: glad to hear the team is happy! | 14:42 |
| corvus | sassyn: wait i don't understand 14:42 < sassyn> User BOSS go to gerrit and mark patchX and PatchZ as +2 while patchX as -2. | 14:43 |
| corvus | sassyn: was one of those patchX supposed to be a patchY ? | 14:43 |
| sassyn | AJaeger thank u | 14:43 |
| sassyn | corvus no. patch x is by user x and patch y is by user y | 14:43 |
| corvus | sassyn: did boss mark patchX both +2 and -2? | 14:44 |
| AJaeger | sassyn: I guess you wanted to say: "User BOSS go to gerrit and mark patchX and PatchY as +2 while patchZ as -2." instead of having X twice in there... | 14:44 |
| sassyn | no | 14:44 |
| sassyn | i'm saying user Boss mark patchX as +2 and patch Z +2 but didn't touch patch Y | 14:45 |
| sassyn | my mistake | 14:45 |
| sassyn | I hope now it is more clear | 14:45 |
| corvus | okay, got it, please continue :) | 14:45 |
| AJaeger | ok, now the question ;) | 14:45 |
| sassyn | Zuul firing a jpb for patchX, and that't it | 14:45 |
| sassyn | only if I do +2 on Y | 14:46 |
| sassyn | then zuul will put Y and Z | 14:46 |
| avass | AJaeger: reviewed the ones I could :) | 14:46 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: WIP Support multi-arch image builds with docker buildx https://review.opendev.org/722339 | 14:46 |
| openstackgerrit | Merged zuul/zuul-jobs master: Switch remaining tests to fedora-31 https://review.opendev.org/722310 | 14:46 |
| AJaeger | thanks, avass ! | 14:46 |
| tobiash | sassyn: are those patches stacked onto each other or independent from each other? | 14:47 |
| avass | just went for a run so I'm gonna take a shower, but I'll do a bunch of reviews when I'm back | 14:47 |
| sassyn | tobiash: let me check | 14:47 |
| *** cdearborn has joined #zuul | 14:49 | |
| *** cdearborn has quit IRC | 14:53 | |
| *** cdearborn has joined #zuul | 14:53 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Improve linters execution https://review.opendev.org/722307 | 14:55 |
| sassyn | tobiash: I don't know how to answer this | 14:57 |
| sassyn | the patches are for the same repo X | 14:57 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Connect merger to Zookeeper https://review.opendev.org/716221 | 14:57 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Connect executor to Zookeeper https://review.opendev.org/716262 | 14:57 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Connect fingergw to Zookeeper https://review.opendev.org/716875 | 14:57 |
| tobiash | sassyn: if Z is stacked on Y it has a hard dependency defined by git so Z can only enter the gate if Y is approved as well | 14:58 |
| tobiash | sassyn: you can see that when you checkout Z and look at the git history if Y is part of the history | 14:59 |
| sassyn | I run form my machine echo File1 >> File1 ; git add File1; git commit -m "file1"; git push origin HEAD:refs/for/master | 15:00 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add testing of fetch-sphinx-tarball role https://review.opendev.org/715028 | 15:00 |
| sassyn | and then I run form my machine echo File2 >> File2 ; git add File2; git commit -m "file2"; git push origin HEAD:refs/for/master | 15:00 |
| sassyn | and then I run form my machine echo File3 >> File3 ; git add File3; git commit -m "file3"; git push origin HEAD:refs/for/master | 15:00 |
| sassyn | consider file1, file2 and file3 as PatchA, patchB, patchC | 15:01 |
| AJaeger | sassyn: so, those are are on on the same branch stacked on top of each other | 15:01 |
| corvus | sassyn: in that case those commits are 'stacked' | 15:01 |
| AJaeger | sassyn: you can showcase using our sandbox repository ... | 15:01 |
| sassyn | what do u mean shocase? | 15:02 |
| AJaeger | sassyn: opendev/sandbox - in case you want to show us how it looks like so that we can explain a life example | 15:02 |
| sassyn | AJaeger, so what is the way to work? | 15:02 |
| corvus | sassyn: if they aren't related to each other, and you want to be able to merge them independently of each other, then you can make each commit on a new branch, and after making each commit, checkout the master branch again | 15:02 |
| sassyn | my repo is set as fast forward only | 15:02 |
| AJaeger | https://docs.opendev.org/opendev/infra-manual/latest/developers.html#starting-a-change | 15:03 |
| corvus | sassyn: oh, then that won't work | 15:03 |
| corvus | sassyn: the main reason to use ff-only is if you never want that to work. most people do want to be able to write independent changes, so ff-only is usually not the right choice for projects with multiple people working on them. | 15:05 |
| AJaeger | sassyn: you really serialize the work of your team ;( | 15:05 |
| avass | fungi: thanks for the tip yesterday, we found out that running an mqtt message broker could solve more problems so that's what we're going for now ;) | 15:06 |
| sassyn | Thank you all | 15:06 |
| *** panda|babysit is now known as panda|ruck | 15:08 | |
| avass | fungi: doesn't look like the mqtt driver includes the commit id though, but I guess that shouldn't be hard to add or workaround worst case | 15:11 |
| openstackgerrit | Andreas Jaeger proposed zuul/zuul-jobs master: Update ensure-javascript-packages README https://review.opendev.org/722354 | 15:19 |
| *** sshnaidm has quit IRC | 15:22 | |
| *** bhavikdbavishi has joined #zuul | 15:34 | |
| *** bhavikdbavishi1 has joined #zuul | 15:37 | |
| *** bhavikdbavishi has quit IRC | 15:39 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 15:39 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Add testing of fetch-sphinx-tarball role https://review.opendev.org/721584 | 15:42 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Add testing of fetch-sphinx-tarball role https://review.opendev.org/721584 | 15:43 |
| tristanC | https://review.opendev.org/#/q/topic:haskell-jobs adds two new zuul-jobs, could you please have a look when you have spare time? Thank you in advance! | 15:47 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 15:48 |
| *** sshnaidm has joined #zuul | 15:55 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: merger: replace custom message logger https://review.opendev.org/720266 | 15:59 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: bindep: Add missing virtualenv and fixed repo install https://review.opendev.org/693637 | 16:15 |
| fungi | sassyn: catching up, but this is precisely the problem i tried to explain last week or the week before that you would encounter if you continue to insist on configuring gerrit to require ff-only for your repositories. you basically give up a lot of zuul's efficiencies. you're also going to see that you need to rebase changes any time an unrelated change you've stacked your change on can't merge for some reason | 16:24 |
| fungi | (starts failing a job or whatever) | 16:24 |
| avass | tristanC: taking a quick look, but I'll go through it a bit later as well | 16:24 |
| fungi | sassyn: you'll need to switch to merge-if-needed or cherry-pick if you want to allow changes to land in the same repository independently of one another | 16:25 |
| fungi | zuul expects to manage the order in which changes merge, and can't do that if gerrit is controlling the order | 16:26 |
| *** avass has quit IRC | 16:29 | |
| *** evrardjp has quit IRC | 16:35 | |
| *** evrardjp has joined #zuul | 16:35 | |
| *** avass has joined #zuul | 16:36 | |
| tristanC | avass: thanks! I'm using forked version, but i think it would be good if zuul could test more languages | 16:38 |
| bolg | does someone know why ZK is not able to read certificate in https://zuul.opendev.org/t/zuul/build/d2601858cf2e4af385f6cf42273d30b2/log/container_logs/zk.log while here (https://zuul.opendev.org/t/zuul/build/2bf3734c628b4712919015d564000974/log/container_logs/zk.log) it does? | 16:38 |
| mordred | avass: yeah - honestly, making the mqtt driver better seems like a good general idea - I'm glad it turned out to be a good solution for you! | 16:40 |
| tristanC | bolg: perhaps a race condition on cert permission? | 16:41 |
| bolg | tristanC: so recheck till it works? | 16:42 |
| tristanC | bolg: the permission issue is odd, it seems like it happens when zookeeper access the file between L7 and L6 of https://opendev.org/zuul/zuul/src/branch/master/doc/source/examples/playbooks/setup.yaml . which seems rather unlikely | 16:45 |
| tristanC | bolg: there are successfull connection happening after the failure, are you sure the build failure is caused by this zk permission failure? | 16:47 |
| tristanC | bolg: e.g., there are `AttributeError: 'MergeJob' object has no attribute 'updated'` in https://zuul.opendev.org/t/zuul/build/d2601858cf2e4af385f6cf42273d30b2/log/container_logs/scheduler.log#125 | 16:48 |
| avass | mordred: yeah, and it seemed like a good way to let people subscribe to a lot of events from zuul or anything else we publish there | 16:48 |
| *** rpittau is now known as rpittau|afk | 16:49 | |
| bolg | tristanC: will check, thanks for the hint | 16:50 |
| bolg | tristanC: I think it is connected to the ZK, the change makes executor use zk. At 15:31:22 executor tries to connect to it (https://zuul.opendev.org/t/zuul/build/d2601858cf2e4af385f6cf42273d30b2/log/container_logs/executor.log). About the same time ZK cannot access the cert (https://zuul.opendev.org/t/zuul/build/d2601858cf2e4af385f6cf42273d30b2/log/container_logs/zk.log) and later the scheduler fails at 15:37:11. Is it possible that the excutor wants | 16:57 |
| bolg | to connect to ZK too early? | 16:57 |
| *** bhavikdbavishi has quit IRC | 16:59 | |
| tristanC | bolg: this seems to indicate that a single connection failure is not corrected by the following `zuul.zk.ZooKeeper: Retrying zookeeper connection` attempt. | 16:59 |
| tristanC | if that is the case, then perhaps there is something to be done in the zuul.zk module to ensure proper reconnection attempt | 17:00 |
| tristanC | otherwise we could delay zookeeper startup by waiting for the certificate to be provisioned by the setup playbook, and also wait that the certificates are readable by the zookeeper service user | 17:01 |
| *** cdearborn has quit IRC | 17:02 | |
| *** jpena is now known as jpena|off | 17:13 | |
| bolg | tristanC: I feel the problem may be connecting to ZK before the executor starts, maybe? https://review.opendev.org/#/c/716262/6/zuul/cmd/executor.py | 17:16 |
| tristanC | bolg: perhaps that early connection revealed another issue in zuul.zk not reconnecting properly in that situation (when service tls files are not readable) | 17:27 |
| bolg | tristanC: thanks for the hint I will dive into it | 17:28 |
| tristanC | bolg: you're welcome. Please let me know if I can help or review your changes. | 17:29 |
| bolg | I will | 17:31 |
| *** sshnaidm is now known as sshnaidm|afk | 17:50 | |
| *** yolanda has quit IRC | 17:55 | |
| *** rlandy has quit IRC | 18:03 | |
| *** rlandy has joined #zuul | 18:03 | |
| *** jcapitao has quit IRC | 18:16 | |
| *** sgw has quit IRC | 18:48 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Support multi-arch image builds with docker buildx https://review.opendev.org/722339 | 19:20 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Add commit id to Change https://review.opendev.org/722478 | 19:23 |
| avass | before I spend too much time on this, would anyone be against something like that? | 19:23 |
| tristanC | avass: that looks great to me | 19:32 |
| fungi | avass: that and any other event data zuul knows about which you think might be useful in the mqtt payload | 19:35 |
| * fungi gives thumbs-up | 19:35 | |
| avass | fungi, tristanC: sure I'll go crazy and let you review then :) | 19:36 |
| fungi | avass: i think the feature has just been minimally implemented awaiting an interested user who could point out what was missing to make it more functional | 19:39 |
| *** igordc has quit IRC | 19:39 | |
| corvus | avass: i think that's good, and we may want to continue to not expose it in the job (unless we have a good reason) -- that might confuse some folks (we definitely don't want anyone doing 'git checkout {{commit_id}}' | 19:40 |
| corvus | but i think it's probably fine in the reporters | 19:41 |
| avass | corvus: sure | 19:42 |
| avass | oh and this is actually working for real this time now: https://review.opendev.org/#/c/717260/ | 19:58 |
| avass | I'd appreciate if someone could take a look at it :) | 19:58 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Support multi-arch image builds with docker buildx https://review.opendev.org/722339 | 20:02 |
| openstackgerrit | Monty Taylor proposed zuul/nodepool master: Build multi-arch images for x86 and arm https://review.opendev.org/722483 | 20:03 |
| mordred | corvus: ^^ | 20:04 |
| mordred | clarkb, ianw: ^^ you might also find that interesting | 20:04 |
| *** adamw has quit IRC | 20:08 | |
| *** sgw has joined #zuul | 20:16 | |
| *** adamw has joined #zuul | 20:16 | |
| *** adamw has quit IRC | 20:17 | |
| noonedeadpunk | folks, what version of nodepool should I isntall so that it supported queens glance? as now I have kinda "openstack.exceptions.NotSupported: The image service for default:Default exists but does not have any supported versions." | 20:25 |
| fungi | noonedeadpunk: it's probably going to depend more on the version of openstacksdk | 20:25 |
| noonedeadpunk | yeah, just started to write that | 20:26 |
| noonedeadpunk | thanks fungi | 20:26 |
| fungi | noonedeadpunk: also that seems odd... i doubt it's that your sdk is "too new" | 20:26 |
| fungi | openstacksdk aims to provide reasonable support for versions of openstack found in the wild, going pretty far back in the history of those apis | 20:27 |
| fungi | so it seems more like it's complaining that it expected to get version information and didn't | 20:27 |
| noonedeadpunk | it's 0.45.0 so quite new... | 20:27 |
| fungi | i'm not super familiar with the glance version discover code in there, but maybe mordred has ideas | 20:28 |
| fungi | er, version discovery | 20:28 |
| *** igordc has joined #zuul | 20:32 | |
| noonedeadpunk | maybe I have configured region in some weird way... | 20:32 |
| noonedeadpunk | Ok, thanks anyway! | 20:32 |
| noonedeadpunk | in terms configured nodepool to the weird region:) | 20:33 |
| mordred | noonedeadpunk: openstacksdk should support all verisons of glance, and nodepool should support them too | 20:34 |
| clarkb | maybe clouds.yaml says use a version that isnt available? | 20:35 |
| clarkb | I thibk we had issues like that when clouds dropped keystone v2 api | 20:35 |
| noonedeadpunk | clouds.yaml is pretty plain - has just identity_api_version: 3 | 20:41 |
| noonedeadpunk | ok, from python shell I guess I know what's the issue is... | 20:44 |
| noonedeadpunk | thanks for help everyone! | 20:44 |
| fungi | noonedeadpunk: don't keep us in suspense! | 20:45 |
| noonedeadpunk | glance had weird public endpoint which couldn't bbe resolved from nodepool | 20:47 |
| noonedeadpunk | so like conn.image() resulted in `Failed to contact the endpoint at http://controller:9292 for discovery. Fallback to using that endpoint as the base url.` | 20:48 |
| fungi | aha, glad you found it, and thanks for the details in case we see a similar error report in the future | 20:48 |
| noonedeadpunk | sure, sorry for disturbing | 20:48 |
| fungi | maybe the error message coming out of openstacksdk could be improved | 20:48 |
| noonedeadpunk | yeah... but now don't have time fixing it unfortunatelly... | 20:50 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add tests for multiarch build https://review.opendev.org/722496 | 20:53 |
| corvus | mordred: ^ that's tests for your multiarch change | 20:53 |
| mordred | corvus: neat! | 21:05 |
| mordred | corvus: the nodepool-build-image job worked | 21:05 |
| mordred | nodepool-functional-k8s had a sad | 21:06 |
| corvus | mordred: via depends-on? | 21:06 |
| corvus | yep https://review.opendev.org/722483 | 21:06 |
| mordred | yeah | 21:07 |
| mordred | something about minikube log export | 21:07 |
| corvus | looks like we're still waiting for a report there | 21:07 |
| corvus | https://zuul.opendev.org/t/zuul/build/92966a60125f4e0b82e5019996436ac2 | 21:08 |
| *** dpawlik has quit IRC | 21:09 | |
| mordred | corvus: that seems unrelated - but I'd like to understand it more | 21:09 |
| corvus | it looks like everything succeeded except that log export | 21:11 |
| mordred | yeah | 21:12 |
| mordred | so - I mean - it seems like the image building parts worked | 21:12 |
| mordred | corvus: nope! | 21:13 |
| mordred | corvus: it did not run the new code | 21:13 |
| mordred | corvus: like - it looks like it did not get the new role | 21:14 |
| corvus | k8s seems to have been cycling through coredns pods at that point. i don't know why; maybe we just see if this happens again, and then try to figure out if there's something we need to fix with the k8s, or if we should just ignore errors in log collection | 21:15 |
| corvus | mordred: i think the 'not running new code' is probably more significant | 21:15 |
| mordred | yeah | 21:15 |
| mordred | yeah - the job in base-jobs isn't blocking it from being speculative is it? | 21:15 |
| corvus | mordred: yeah -- the build-docker-image role is in a trusted playbook | 21:17 |
| mordred | ah. well that'll do it | 21:17 |
| mordred | corvus: so your tests patch should at least tell us if landing it will break things | 21:18 |
| corvus | mordred: probably easiest thing is to iterate with my test change in zuul-jobs, then i think it should be safe to land. that should be a thorough test of the whole system. | 21:18 |
| mordred | yeah | 21:18 |
| corvus | (alternatively, we could start splitting that job up into trusted and non-trusted parts; and maybe we should, but later?) | 21:18 |
| mordred | yeah - let's not go too far down the rabbit hole | 21:19 |
| noonedeadpunk | sorry - another stupid question - "nodepool.exceptions.LaunchNetworkException: Unable to find public IP of server" | 21:23 |
| noonedeadpunk | Like I've provided private network for the pool, but nodepool has access to this network | 21:24 |
| noonedeadpunk | I obviously just don't see some missed option... | 21:24 |
| mordred | noonedeadpunk: if you're wanting nodepool to just use the private network to talk to nodes, you should put "private: true" in the clouds.yaml entry | 21:24 |
| mordred | that will tell openstacksdk to treat the private ip as "the interface for talking to the server" | 21:25 |
| noonedeadpunk | mordred: hm, will try, I just thought that openstacksdk is not the one who should talk to the server... | 21:26 |
| mordred | noonedeadpunk: it's not- but it's the one that figures out from openstack how to talk to the server | 21:28 |
| mordred | noonedeadpunk: it is sadly a very difficult topic | 21:28 |
| noonedeadpunk | yeah, I see :) | 21:28 |
| noonedeadpunk | Was just really expecting this to be configured in nodepool.yaml | 21:28 |
| mordred | there are a _bunch_ of network-related settings one can make for openstack - instead of duplicating clouds.yaml, we decided to just have nodepool refer to a named cloud from clouds.yaml and let all of those options get configured there | 21:29 |
| noonedeadpunk | now that makes sense to me:) | 21:30 |
| mordred | noonedeadpunk: it would be much better if we didn't have to have people configure crazy things about their clouds :) | 21:30 |
| noonedeadpunk | yeah lol | 21:30 |
| noonedeadpunk | btw, the last thing (I hope). Today I was looking through DIB elements, but I think I didn't find any that would isntall cloud-init? Like only https://docs.openstack.org/diskimage-builder/latest/elements/cloud-init/README.html but it's for gentoo only? | 21:32 |
| noonedeadpunk | others are more about cloud-init config | 21:32 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Support multi-arch image builds with docker buildx https://review.opendev.org/722339 | 21:33 |
| noonedeadpunk | maybe the thing is that I'm using minimal element for the image itself... | 21:33 |
| mordred | corvus: fixed your comment and then squashed with the tests | 21:33 |
| mordred | noonedeadpunk: we tend to use simple-init/glean instead of cloud-init | 21:34 |
| mordred | noonedeadpunk: largely because the only thing we need such a thing for is installing ssh key and setting up networking - and cloud-init pulls in python deps and does a bunch of extra stuff | 21:34 |
| mordred | noonedeadpunk: which is mostly just an explanation of why you might not find great cloud-init support | 21:34 |
| mordred | the folks who use cloud-init are more likely to use the non-minimal base images which are based on the upstream published cloud images | 21:35 |
| openstackgerrit | Oleksandr Kozachenko proposed zuul/zuul-jobs master: Fix the checking helm_values_file definition https://review.opendev.org/722516 | 21:35 |
| noonedeadpunk | mordred: oh, I see. | 21:35 |
| mordred | noonedeadpunk: (simple-init is the dib element name, glean is the program it runs that's not cloud-init but doe the minimla things) | 21:35 |
| mordred | noonedeadpunk: glean also only works with config-drive - so you want that enabled of course | 21:36 |
| noonedeadpunk | that's actually good notice | 21:37 |
| noonedeadpunk | ok, thank you so much for your time! | 21:38 |
| tristanC | There are 19 zuul-operator changes pending reviews, the tip is starting at https://review.opendev.org/720822 . This include useful refactor and other important updates like cert-manager integration and zuul-registry configuration. Please have a look when you have sometime. | 21:46 |
| *** igordc has quit IRC | 21:51 | |
| ianw | mordred: thanks, will look at the arm64 work. i don't think it will make much difference, but i have a change out to drop pip-and-virtualenv from the arm images .. i'd really like it if we started this work without depending on that so we don't have to unwind it | 21:51 |
| ianw | ... looks like frickler raised something on that ... let me look | 21:52 |
| openstackgerrit | Merged zuul/nodepool master: Parallelize initial static node synchronization https://review.opendev.org/721205 | 21:58 |
| mordred | ianw: ah - I see that you did see that | 22:06 |
| mordred | corvus: the multiarch test job failed - but it looks like it's unhappy with zuul_return processing which is weird | 22:10 |
| *** Goneri has quit IRC | 22:21 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Support multi-arch image builds with docker buildx https://review.opendev.org/722339 | 22:25 |
| *** igordc has joined #zuul | 22:26 | |
| corvus | mordred: that's a red herring; it has ignore_errors set to true | 22:41 |
| corvus | mordred: the real problem was the one you fixed -- fail_message | 22:41 |
| corvus | mordred: now it is intentionally failing because buildset_registry is not defined | 22:42 |
| *** zxiiro has quit IRC | 22:42 | |
| corvus | mordred: ah i see | 22:45 |
| corvus | mordred: the job starts by building an image "out of band" and pushing it to the fake intermediate registry in order to test the situation where we come into a real job with an image already in the intermediate registry | 22:45 |
| corvus | mordred: i think we can back out the "arch" setting from that one, and only add that to the normal image build part of the job | 22:47 |
| corvus | i will push an update | 22:47 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Support multi-arch image builds with docker buildx https://review.opendev.org/722339 | 22:48 |
| corvus | mordred: the job calls those the "upstream" and "downstream" images. we want to leave the upstream image alone, and just test the multi-arch stuff on the downstream image. | 22:49 |
| mordred | corvus: yes - I agree | 22:55 |
| mordred | corvus: is our "upstream" image based on an image that's already multiarch? | 22:55 |
| *** tosky has quit IRC | 22:58 | |
| *** igordc has quit IRC | 23:05 | |
| corvus | mordred: debian:testing | 23:08 |
| corvus | so yes i think so | 23:08 |
| openstackgerrit | Merged zuul/nodepool master: Siblings container build: set work dir to nodepool https://review.opendev.org/721514 | 23:29 |
| openstackgerrit | Ian Wienand proposed zuul/zuul master: [wip] intermediate jobs https://review.opendev.org/722200 | 23:39 |
| openstackgerrit | Merged zuul/nodepool master: func-tests: drop debuntu specific env vars https://review.opendev.org/721754 | 23:50 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!