| *** tumble has quit IRC | 00:11 | |
| *** rfolco|rover|off has quit IRC | 00:25 | |
| *** Goneri has quit IRC | 00:41 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: bindep: use virtualenv_command from ensure-pip https://review.opendev.org/727561 | 01:37 |
|---|---|---|
| *** swest has quit IRC | 01:46 | |
| *** rlandy has quit IRC | 01:58 | |
| *** swest has joined #zuul | 02:01 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Unset bindep_command to exercise install paths https://review.opendev.org/727593 | 02:31 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Unset bindep_command to exercise install paths https://review.opendev.org/727593 | 02:39 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Unset bindep_command to exercise install paths https://review.opendev.org/727593 | 02:44 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Unset bindep_command to exercise install paths https://review.opendev.org/727593 | 02:54 |
| *** bhavikdbavishi has joined #zuul | 03:00 | |
| *** bhavikdbavishi1 has joined #zuul | 03:03 | |
| *** bhavikdbavishi has quit IRC | 03:05 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:05 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Unset bindep_command to exercise install paths https://review.opendev.org/727593 | 03:08 |
| *** dustinc has quit IRC | 03:34 | |
| EmilienM | is there a way in a zuul job to pass a variable which is a list of the files that are touched by a patch? | 03:36 |
| EmilienM | e.g. in a job I want to know which files are touched | 03:36 |
| EmilienM | from a patch in gerrit | 03:37 |
| EmilienM | cloudnull: ^ asking for our container image thing | 03:37 |
| *** sanjayu_ has joined #zuul | 03:49 | |
| *** cdearborn has quit IRC | 03:53 | |
| *** bhavikdbavishi has quit IRC | 04:13 | |
| *** bhavikdbavishi has joined #zuul | 04:26 | |
| *** evrardjp has quit IRC | 04:36 | |
| *** evrardjp has joined #zuul | 04:36 | |
| *** dmellado has quit IRC | 04:45 | |
| clarkb | EmilienM: you can do a git diff | 04:59 |
| clarkb | EmilienM: diff $branch origin/$branch | 04:59 |
| *** dpawlik has quit IRC | 05:04 | |
| *** ysandeep|away is now known as ysandeep | 05:50 | |
| *** dpawlik has joined #zuul | 06:25 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Switch back to python 3.7 https://review.opendev.org/727367 | 06:39 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Deprecate ansible 2.7 https://review.opendev.org/727344 | 06:40 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Drop support for ansible 2.6 https://review.opendev.org/727157 | 06:40 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Drop support for ansible 2.7 https://review.opendev.org/727373 | 06:40 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Update images to use python 3.8 https://review.opendev.org/727374 | 06:42 |
| *** avass has joined #zuul | 06:51 | |
| *** fbo|off is now known as fbo|afk | 06:59 | |
| *** bhavikdbavishi has quit IRC | 07:07 | |
| *** jcapitao has joined #zuul | 07:08 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Policy rule for ownership between remote and executor https://review.opendev.org/724855 | 07:20 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 07:20 |
| *** yolanda has joined #zuul | 07:21 | |
| *** rpittau|afk is now known as rpittau | 07:29 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 07:29 |
| *** tosky has joined #zuul | 07:35 | |
| *** dmellado has joined #zuul | 07:37 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Policy rule for ownership between remote and executor https://review.opendev.org/724855 | 07:41 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 07:41 |
| avass | AJaeger: how about that? | 07:43 |
| *** bhavikdbavishi has joined #zuul | 07:45 | |
| *** jpena|off is now known as jpena | 07:56 | |
| AJaeger | avass: LGTM, thanks | 08:07 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: bindep: Add missing virtualenv and fixed repo install https://review.opendev.org/693637 | 08:08 |
| *** nils has joined #zuul | 08:16 | |
| yoctozepto | morning | 08:17 |
| yoctozepto | I could not catch you folks on some other day so made a story in sb https://storyboard.openstack.org/#!/story/2007663 | 08:17 |
| yoctozepto | (more persistent than channel logs :-) ) | 08:17 |
| *** guillaumec has joined #zuul | 08:18 | |
| *** jcapitao has quit IRC | 08:27 | |
| *** jcapitao has joined #zuul | 08:30 | |
| *** sugaar has joined #zuul | 08:37 | |
| *** jcapitao has quit IRC | 08:38 | |
| avass | yoctozepto: oh interesting | 08:38 |
| avass | yoctozepto: my guess is that zuul expects the file to be there so it errors and never applies the updated config | 08:39 |
| yoctozepto | avass: could be, I can't say as it proceeds gently with merged config | 08:40 |
| yoctozepto | and I'm just Zuul user | 08:40 |
| *** jcapitao has joined #zuul | 08:42 | |
| *** jcapitao has quit IRC | 08:48 | |
| *** jcapitao has joined #zuul | 08:50 | |
| *** jcapitao has quit IRC | 08:50 | |
| *** jcapitao has joined #zuul | 08:50 | |
| *** bhavikdbavishi has quit IRC | 08:52 | |
| avass | yoctozepto: yeah, I haven't looked into those parts of zuul a lot so I'm just guessing from experience :) | 08:53 |
| openstackgerrit | Andreas Jaeger proposed zuul/zuul-jobs master: Fix nodejs-npm-run-test https://review.opendev.org/727670 | 09:03 |
| AJaeger | avass: could you review ^ quickly, please? mordred's last change broke nodejs-npm-run-test and thus horizon and friends ;( | 09:03 |
| avass | sure | 09:05 |
| avass | AJaeger: lgtm, gave it a +3 | 09:07 |
| AJaeger | thanks, avass ! | 09:07 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Don't require tox_envlist https://review.opendev.org/726829 | 09:14 |
| openstackgerrit | Merged zuul/zuul-jobs master: Fix nodejs-npm-run-test https://review.opendev.org/727670 | 09:20 |
| *** bhavikdbavishi has joined #zuul | 09:37 | |
| *** bhavikdbavishi has quit IRC | 09:54 | |
| openstackgerrit | Merged zuul/zuul master: Provide some documentation for the checks API implementation https://review.opendev.org/711493 | 10:01 |
| *** ysandeep is now known as ysandeep|lunch | 10:09 | |
| *** dpawlik has quit IRC | 10:21 | |
| *** bhavikdbavishi has joined #zuul | 10:26 | |
| *** bhavikdbavishi1 has joined #zuul | 10:38 | |
| *** bhavikdbavishi has quit IRC | 10:39 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 10:39 | |
| *** ysandeep|lunch is now known as ysandeep | 10:55 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: intercept-job -- self-service SSH access https://review.opendev.org/679306 | 10:56 |
| avass | AJaeger: do you know if there was a reason we didn't merge that ^ ? | 10:57 |
| *** jcapitao is now known as jcapitao_lunch | 10:57 | |
| *** rpittau is now known as rpittau|bbl | 11:04 | |
| *** bhavikdbavishi1 has joined #zuul | 11:15 | |
| *** bhavikdbavishi has quit IRC | 11:17 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 11:17 | |
| tobiash | avass: I think we just forgot about that | 11:22 |
| AJaeger | avass: too many changes in zuul-jobs ;( | 11:25 |
| avass | tobiash, AJaeger: alright, I like the idea so I want it merged :) | 11:29 |
| *** guillaumec has quit IRC | 11:30 | |
| AJaeger | avass: I didn't understand the idea when first seeing it and therefore ignored it ;( | 11:31 |
| *** armstrongs has joined #zuul | 11:32 | |
| *** dpawlik has joined #zuul | 11:32 | |
| *** fbo|afk is now known as fbo | 11:34 | |
| *** jpena is now known as jpena|lunch | 11:34 | |
| EmilienM | clarkb: ok, thanks. | 11:41 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-coverage-output: do not synchronize owner https://review.opendev.org/727717 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-javascript-content-tarball: do not synchronize owner https://review.opendev.org/727718 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-javascript-output: do not synchronize owner https://review.opendev.org/727719 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-javascript-tarball: do not synchronize owner https://review.opendev.org/727720 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-markdownlint: do not synchronize owner https://review.opendev.org/727721 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-phoronix-results: do not synchronize owner https://review.opendev.org/727722 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-puppet-module-output: do not synchronize owner https://review.opendev.org/727723 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-python-sdist-output: do not synchronize owner https://review.opendev.org/727724 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-sphinx-output: do not synchronize owner https://review.opendev.org/727725 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-sphinx-tarball: do not synchronize owner https://review.opendev.org/727726 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-tox-output: do not synchronize owner https://review.opendev.org/727727 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-translation-output: do not synchronize owner https://review.opendev.org/727728 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: fetch-subunit-output: do not synchronize owner https://review.opendev.org/727729 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: prepare-workspace: do not synchronize owner https://review.opendev.org/727730 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: publish-artifacts-to-fileserver: do not synchronize owner https://review.opendev.org/727731 | 11:45 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: upload-logs: do not synchronize owner https://review.opendev.org/727732 | 11:45 |
| avass | just a small stack :) | 11:46 |
| *** guillaumec has joined #zuul | 11:48 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 11:49 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: tarball-post.yaml: do not synchronize owner https://review.opendev.org/727735 | 11:49 |
| AJaeger | avass: wow ;) | 11:50 |
| *** rfolco has joined #zuul | 12:04 | |
| *** rfolco is now known as rfolco|rover | 12:05 | |
| *** avass has quit IRC | 12:05 | |
| *** rlandy has joined #zuul | 12:09 | |
| *** avass has joined #zuul | 12:12 | |
| *** sshnaidm|afk is now known as sshnaidm | 12:16 | |
| *** ysandeep is now known as ysandeep|brb | 12:17 | |
| *** armstrongs has quit IRC | 12:18 | |
| *** rpittau|bbl is now known as rpittau | 12:19 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Drop support for ansible 2.7 https://review.opendev.org/727410 | 12:20 |
| *** jcapitao_lunch is now known as jcapitao | 12:22 | |
| *** avass has quit IRC | 12:31 | |
| *** avass has joined #zuul | 12:38 | |
| *** jpena|lunch is now known as jpena | 12:40 | |
| *** ianychoi has quit IRC | 12:55 | |
| *** sgw has quit IRC | 13:15 | |
| tristanC | avass: it seems like this will conflict with `topic:zuul-jobs-with-kubectl` , could we do those first please? | 13:25 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: fetch-sphinx-tarball: introduce zuul_use_fetch_output https://review.opendev.org/681870 | 13:28 |
| AJaeger | tristanC: I don't see a conflict directly but I'm fine getting those 3 or 4 changes without merge conflict in... | 13:33 |
| *** sgw has joined #zuul | 13:34 | |
| tristanC | AJaeger: https://review.opendev.org/#/c/681905 and https://review.opendev.org/#/c/681870 (which enable doc building on kubectl node) conflicts with the synchronize owner stack | 13:34 |
| *** harrymichal has joined #zuul | 13:37 | |
| AJaeger | tristanC: now I see it... | 13:38 |
| AJaeger | tristanC: let's get a second reviewer, I gave my +2s | 13:39 |
| tristanC | AJaeger: thanks, we have third-party-ci job waiting to be enabled once this is merged | 13:41 |
| harrymichal | Hey guys! Do you have somewhere on your roadmap adding support for https://shields.io?? I know you have a static shield but that one is not as customizable as standard shields. | 13:52 |
| mhu | harrymichal, you mean https://zuul-ci.org/docs/zuul/howtos/badges.html ? | 13:54 |
| *** bhavikdbavishi has quit IRC | 13:55 | |
| harrymichal | Yes, by that I mean the static shield. | 13:55 |
| harrymichal | A use case for dynamic shield is to show status of last periodic job. | 13:56 |
| mhu | what would you like to customize? Because as mentioned in that doc, by definition projects gated by Zuul are never broken | 13:56 |
| mhu | oh I see | 13:56 |
| harrymichal | In case of 'gate' jobs this is not a problem because the response is given in the PR but there's no obvious way to see status of periodic pipeline apart from looking at the dashboard. | 13:57 |
| mhu | so basically add this kind of endpoint to job results https://shields.io/endpoint | 13:58 |
| harrymichal | Possibly. I never used "advanced" shields myself but I see them very frequently. | 13:58 |
| mordred | https://review.opendev.org/#/c/702128/ | 13:59 |
| mordred | harrymichal, mhu ^^ | 13:59 |
| mhu | noice! | 14:00 |
| harrymichal | Ah, you already knew about this. Perfect! | 14:00 |
| * mordred takes credit for tobiash being smart | 14:01 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Combine javascript deployment and deployment-tarball jobs https://review.opendev.org/727370 | 14:08 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Set node_version in js-build base job https://review.opendev.org/727774 | 14:08 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Combine javascript deployment and deployment-tarball jobs https://review.opendev.org/727370 | 14:09 |
| *** Goneri has joined #zuul | 14:20 | |
| mordred | AJaeger: ^^ I split the "set node_version" into its own patch - it doens't work without that, I think something to do with defaults and include_role | 14:31 |
| mordred | AJaeger: but I think those may be good | 14:31 |
| AJaeger | cool, will review later | 14:34 |
| *** ysandeep|brb is now known as ysandeep | 14:38 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: How-to: using the REST API with cURL https://review.opendev.org/727785 | 14:40 |
| avass | mhu: ooh, nice | 14:57 |
| mhu | avass, thanks! with tristanC we just realized installing the CLI isn't very straightforward to install, so we might as well give some examples with cURL | 14:58 |
| mhu | I mean pip install zuul has a lot of requirements | 14:59 |
| avass | mordred: what's the problem with default variables? | 15:02 |
| Open10K8S | hi team | 15:03 |
| Open10K8S | The ensure-kubernetes job is failing now. | 15:03 |
| Open10K8S | of course the minikube is provisioning, but the coreDNS is failed with loop detection. | 15:03 |
| Open10K8S | I found the reason, but am not sure about the solution. | 15:03 |
| Open10K8S | I'd like to discuss the solution with you. | 15:03 |
| Open10K8S | The new version of minikube has been released in 2days. v1.10.0 and v1.10.1 | 15:03 |
| Open10K8S | As you know, the minikube detects the systemd-resolved automatically and set the kubelet.resolv-conf with /run/systemd/resolve/resolv.conf. | 15:03 |
| Open10K8S | But once we set the kubelet.resolv-conf explicitly, then the systemd resolv.conf is not appended automatically, and only the conf files which we set would be appended in the former versions of minikube. | 15:03 |
| Open10K8S | For now, the new versions are appending the systemd resolv.conf always, and the main problem is systemd resolv.conf is including localhost only. | 15:03 |
| Open10K8S | So the coreDNS is looping. | 15:03 |
| mordred | avass: NO CLUE | 15:04 |
| Open10K8S | I am not sure why systemd resolv.conf includes localhost | 15:04 |
| avass | mordred: I mean, what's not working as you think it should? :) | 15:04 |
| mordred | avass: having ensure-javascript-build-tools role with node_version: 14 in its defaults.yaml include_role: ensure_yarn which include_role: ensure-nodejs winds up with ensure-nodejs installing its default versio of 6 | 15:05 |
| avass | mordred: oh yeah, that's since ensure-nodejs has a default of v6 doesn't it? | 15:05 |
| mordred | avass: so somewhere in that stack something is deciding that the including roles' defaults are not appropriate and it should go with ensure-nodejs defaults | 15:05 |
| mordred | yeah | 15:05 |
| avass | mordred: so pass it explicitly? | 15:05 |
| avass | mordred: I don't think a default will override another roles default | 15:06 |
| mordred | ah - nod. that's sadmaking | 15:06 |
| mordred | we'd have to pass it expicitly through the whole stack | 15:06 |
| avass | mordred: since they technically have the same precedence | 15:06 |
| avass | mordred: oh, no just include_role: name=ensure-nodejs vars= {version={{version}}} | 15:07 |
| avass | I think that should work | 15:07 |
| mordred | lemme try that again (I tried once, but I may have missed something) | 15:09 |
| clarkb | Open10K8S: systemd resolv.conf includes localhost because it (systemd-named or whatever its called) wants to be the local dns server | 15:09 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Install correct node version in ensure-yarn https://review.opendev.org/727789 | 15:09 |
| clarkb | Open10K8S: to fix that we were setting the resolv.conf directly using the dns servers we want to configure on the test node | 15:09 |
| avass | mordred: so that ^ doesn't work? | 15:09 |
| clarkb | Open10K8S: are you saying that minikube is ignoring our override and using systemd/s values anyway? | 15:09 |
| Open10K8S | clarkb: I dont think so | 15:10 |
| Open10K8S | clarkb: it use both | 15:10 |
| Open10K8S | clarkb: it uses both | 15:10 |
| clarkb | Open10K8S: right that seems wrong, why have an override if its just gonna add in the default anyway :/ | 15:10 |
| avass | mordred: uuh, the js stack got a bit complicated didn't it | 15:10 |
| clarkb | Open10K8S: I think the correct fix there is for minikube/coredns to not mix in additional content when you override that config | 15:11 |
| clarkb | (but realize they may never fix that either) | 15:11 |
| Open10K8S | clarkb: sounds logical | 15:11 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Pass node_version explicitly https://review.opendev.org/727774 | 15:11 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Combine javascript deployment and deployment-tarball jobs https://review.opendev.org/727370 | 15:11 |
| Open10K8S | clarkb: need to escalate to the minikube dev team? | 15:11 |
| clarkb | Open10K8S: ya I think we should see if we can have them weigh in on the behavior. My expectation is that if I override the default behavior (because it won't work) that the default not be mixed in anyway | 15:12 |
| clarkb | Open10K8S: if they think that is a bug too then this might get fixed quickly. If they don't then we may have to find another workaround | 15:13 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Install correct node version in ensure-yarn https://review.opendev.org/727789 | 15:13 |
| mordred | avass: o - I think we just overlapped each other | 15:13 |
| Open10K8S | clarkb: ok, I agree | 15:13 |
| Open10K8S | clarkb: do you know the best way to let them know this and discuss? | 15:14 |
| clarkb | Open10K8S: I don't. mnaser or tristanC may have ideas? | 15:14 |
| clarkb | my guess is a bug on github but not completely sure | 15:14 |
| Open10K8S | clarkb: ok, I will discuss with mnaser then | 15:15 |
| Open10K8S | thank you | 15:15 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Install correct node version in ensure-yarn https://review.opendev.org/727789 | 15:16 |
| avass | mordred: do you have a change you can test that ^ with? | 15:16 |
| avass | mordred: actually I think I got that wrong | 15:17 |
| clarkb | Open10K8S: fwiw I think on our test nodes the localhost comes from systemd grabbing /etc/resolv.conf and using that config too. And since we configure unbound to be a caching server it grabs localhost that way. But if we didn't do that what systemd would do is read resolv.conf's configured resolvers. Set those up for it to resolve to then configure systemd to be the system wide resolver | 15:17 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Install correct node version in ensure-yarn https://review.opendev.org/727789 | 15:17 |
| clarkb | Open10K8S: I expect this is actually a fairly common situation in the real world as a result | 15:18 |
| mordred | avass: https://review.opendev.org/#/c/727774/ :) | 15:18 |
| mordred | avass: I thinkw e're both hacking on the same thing | 15:18 |
| mordred | avass: but - https://review.opendev.org/#/c/717371/ | 15:19 |
| avass | mordred: do we need to pass it in both ensure-js-tool and ensure-yarn maybe? | 15:20 |
| mordred | avass: no - I don;'t think so - ensure-yarn doesn't set its own default, and once we pass it as a var like that I'm pretty sure it'll carry through | 15:20 |
| mordred | avass: that said - we'll find out with the recheck of 717371 - it does NOT like node v6 | 15:21 |
| avass | mordred: oh, yes, I missed 727774 got lost in 717371 | 15:22 |
| avass | mordred: hopefully, unless things misbehave :) | 15:22 |
| mordred | avass: ) | 15:22 |
| mordred | :) | 15:22 |
| *** rlandy_ has joined #zuul | 15:23 | |
| clarkb | Open10K8S: it is also possible they will show us the proper way to accomplish that override. | 15:23 |
| tristanC | clarkb: Open10K8S: i would not know where to contact minikube maintainer, perhaps https://github.com/kubernetes/minikube/issues/ ? | 15:24 |
| Open10K8S | clarkb: tristanC I am not sure except of that | 15:25 |
| *** rlandy has quit IRC | 15:25 | |
| *** rlandy_ is now known as rlandy | 15:25 | |
| *** sshnaidm is now known as sshnaidm|afk | 15:28 | |
| *** zxiiro has joined #zuul | 15:32 | |
| avass | mordred: nice, recursion error | 15:35 |
| *** jcapitao has quit IRC | 15:41 | |
| avass | mordred: looks like the workaround is to use set_fact in advance to get around this: https://github.com/ansible/ansible/issues/36274 | 15:45 |
| *** guillaumec has quit IRC | 15:50 | |
| *** bhavikdbavishi has joined #zuul | 15:51 | |
| avass | mordred: yeah, it works if you run a set_fact before the include_role | 15:53 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: How-to: using the REST API with cURL https://review.opendev.org/727785 | 15:56 |
| avass | mordred: wooow, but that can lead to VERY interesting results: http://paste.openstack.org/ | 16:01 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Dequeue changes via github checks API https://review.opendev.org/709135 | 16:03 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Dequeue changes via github checks API https://review.opendev.org/709135 | 16:04 |
| *** armstrongs has joined #zuul | 16:08 | |
| mordred | avass: I'm really starting to think that just setting the var in teh job is better here | 16:09 |
| avass | mordred: yeah I think so too | 16:09 |
| *** bhavikdbavishi has quit IRC | 16:13 | |
| *** felixedel has joined #zuul | 16:13 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Set node_version in js-build base job https://review.opendev.org/727774 | 16:13 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Combine javascript deployment and deployment-tarball jobs https://review.opendev.org/727370 | 16:13 |
| *** bhavikdbavishi has joined #zuul | 16:13 | |
| mordred | avass: ^^ done | 16:14 |
| felixedel | corvus: I have a question about https://review.opendev.org/#/c/711023/5. I can rename the "cancel" back to "dequeue", but I will keep the logic that the dequeue reporter only applies to dequeued items which are not success or failure, right? Otherwise successful and failed items would be reported twice. | 16:18 |
| *** rf0lc0 has joined #zuul | 16:20 | |
| avass | mordred: re 727370 | 16:21 |
| *** rpittau is now known as rpittau|afk | 16:21 | |
| *** rfolco|rover has quit IRC | 16:23 | |
| *** fbo is now known as fbo|off | 16:23 | |
| AJaeger | mordred: -1 on that one ^ | 16:28 |
| AJaeger | avass: good spotting | 16:28 |
| avass | mordred: oh noticed I copied the wrong link: http://paste.openstack.org/show/793535/ :) | 16:33 |
| corvus | felixedel: yes, i think we'll just need to make that clear in documentation | 16:33 |
| felixedel | corvus: Ok. Would this be sufficient: "The dequeue reporters will only apply, if the item wasn't a success or failure."? | 16:36 |
| *** evrardjp has quit IRC | 16:36 | |
| *** evrardjp has joined #zuul | 16:36 | |
| armstrongs | hey i'm currently trying to test out the gitlab driver as i discussed on here a few days back. I have it connected to gitlab and recognising the projects initiatlly if i configure the scheduler. However, i am not seeing zuul pick up proceeding events after initial configuration. Just wondered where i should be looking as the executor and scheduler | 16:36 |
| armstrongs | logs don't have anything | 16:36 |
| corvus | felixedel: yes, or we could say 'if the item was dequeued without a result' | 16:36 |
| felixedel | Maybe there is a better phrase than "wasn't a success or failure" I just didn't come up with one | 16:36 |
| felixedel | Maybe so, yes | 16:36 |
| tobiash | armstrongs: gitlab works with webhooks right? | 16:37 |
| armstrongs | yup | 16:37 |
| tobiash | armstrongs: in this case I'd start with zuul-web logs | 16:37 |
| felixedel | corvus: Ok. I like your version better :) | 16:37 |
| corvus | armstrongs, tobiash: if the webhook receives something there should be a log line in zuul-web like "Gitlab Webhook Received event kind: ..." | 16:38 |
| corvus | oh wait | 16:38 |
| corvus | that's actually a scheduler log line | 16:38 |
| tobiash | for github zuul-web logs something containing 'payload' | 16:39 |
| corvus | armstrongs, tobiash: the zuul-web line should say "Event header:" and "Event body:" | 16:40 |
| openstackgerrit | Merged zuul/zuul-jobs master: Set node_version in js-build base job https://review.opendev.org/727774 | 16:40 |
| corvus | armstrongs: so basically -- zuul-web should report that when it receives the event from gitlab; then it should send the event over to the scheduler, which, when it receives it should report that first log line i described | 16:40 |
| corvus | hopefully that will help narrow down what's missing | 16:41 |
| felixedel | corvus: Thanks for the hint on the other change https://review.opendev.org/#/c/709135/9/zuul/driver/github/githubreporter.py . That really was a leftover from my tests. Thus, I simply removed it and we can go without the "queue" in this case. I've also adapted the release notes - assuming we merge the reporter change https://review.opendev.org/#/c/711023/5 before this one. | 16:41 |
| corvus | armstrongs: also, github has a page that shows you the results from webhooks that it sent; if gitlab has something similar, you might be able to see if it's getting a 404 or a connection error or something before even getting to zuul-web | 16:42 |
| armstrongs | corvus: so im testing this with public gitlab repo. What are the prerequisites on the zuul instance to communicate via the web-hook as im not seeing anything from gitlab. I currently have it on a load balancer | 16:42 |
| armstrongs | i will have a dig thanks for the info guys | 16:44 |
| corvus | armstrongs: zuul-web will need to be accessible, and you'll probably need to give gitlab the webhook url which should be constructed in the same way as the github driver: https://zuul-ci.org/docs/zuul/reference/drivers/github.html#web-hook | 16:45 |
| tobiash | armstrongs: and probably check tls settings if you'r using encryption | 16:45 |
| corvus | armstrongs: it looks like the webhook_token and api_token described there are also required for gitlab | 16:46 |
| armstrongs | http://paste.openstack.org/show/793536/ thats the connection settings im using | 16:48 |
| openstackgerrit | Andreas Jaeger proposed zuul/zuul-jobs master: Combine javascript deployment and deployment-tarball jobs https://review.opendev.org/727370 | 16:48 |
| tobiash | armstrongs: webhook_token seems to be missing | 16:48 |
| tobiash | that should be the same as configured on the webhook in gitlab | 16:49 |
| AJaeger | mordred: fixed it for you ^ | 16:49 |
| armstrongs | ah ok i got this from the test fixtures | 16:49 |
| tobiash | armstrongs: hrm, the gitlabconnection tells me that it just skips webhook validation if no webhook_token is configured | 16:50 |
| armstrongs | https://review.opendev.org/#/c/685682/8/tests/fixtures/zuul-gitlab-driver.conf thats what i was copying | 16:51 |
| tobiash | so next I'd check the url and ssl verification checkmark in the webhook config in gitlab | 16:51 |
| tobiash | maybe use curl to throw something to zuul-web to test it | 16:51 |
| tobiash | armstrongs: what's the url whcih is configured in gitlab? | 16:52 |
| tobiash | you can mask the hostname if that's secret ;) | 16:53 |
| fungi | the other thing i think we figured out in here a week or so ago is that the gitlab driver expects the webhook token embedded in the url, rather than providing a separate option for it (but that splitting it out like the github driver does would be a good idea) | 16:53 |
| fungi | but also possible i missed something looking through the source code | 16:53 |
| armstrongs | i think i see whats up now. Will report back shortly cheers for the help guys | 16:54 |
| tobiash | armstrongs: webhook url must be ``http(s)://<zuul-hostname>:<port>/api/connection/<connection-name>/payload | 16:54 |
| armstrongs | yup just seen that | 16:54 |
| tobiash | that should be the same for all drivers | 16:54 |
| armstrongs | :) | 16:54 |
| fungi | also apparently by default the webhook url gets used as a url embedded in results, which can then expose the credentials if they're included in the url | 16:54 |
| tobiash | hrm, did we land gitlab completely without docs? | 16:55 |
| *** rf0lc0 is now known as rfolco|rover | 16:56 | |
| armstrongs | yeah no docs yet so reverse engineering it :) | 16:57 |
| avass | mordred: so uh, I don't like that solution since now we essentially require node_version to be set if we use yarn, but don't require it if we don't use yarn | 16:57 |
| AJaeger | if you want to contribute docs after reverse engineering, it would be appreciated ;) | 16:57 |
| armstrongs | that was my plan | 16:57 |
| AJaeger | great! | 16:58 |
| avass | mordred: it would have made sense to require node_version to be set if the user wants yarn, but since we autodetect yarn it can lead to confusion | 16:59 |
| fungi | tobiash: yes, docs weren't included since the driver is still considered incomplete/experimental | 17:00 |
| tobiash | ah ok | 17:00 |
| fungi | (and documenting it was one of the to do items, but at least without documentation it's not discoverable and we've refrained from announcing it as available/supported) | 17:00 |
| avass | fungi: I'm pretty sure it was discovered ;) | 17:02 |
| fungi | yup, the idea is that it's discovered by people asking us so we can at least get a chance to explain it's incomplete ;) | 17:02 |
| clarkb | fwiw I think we could have a warning in the documentation if we were worried documenting what is there would lead to confusion | 17:03 |
| tobiash | mordred, corvus, tristanC: the switch back to py37 needed an update: https://review.opendev.org/727367 | 17:03 |
| clarkb | "This driver is new and currently not expected to be stable. The best way to help us make it stable is to try it out and providefeedback or bug reports with what you find" | 17:03 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Drop support for ansible 2.6 https://review.opendev.org/727157 | 17:06 |
| fungi | clarkb: yep, i agree | 17:07 |
| *** jpena is now known as jpena|off | 17:11 | |
| *** ysandeep is now known as ysandeep|away | 17:12 | |
| corvus | clarkb, tobiash: i think maybe the explicit 3.7 would be better there? in 727367 | 17:12 |
| clarkb | ya that would rpobably more clearly meet the goals here | 17:13 |
| clarkb | though for now I don't expect we'll flip the base image soon so its ok if we want to do that ina followup | 17:13 |
| corvus | okay, i +3d | 17:13 |
| *** bhavikdbavishi1 has joined #zuul | 17:26 | |
| avass | mordred: nvm, I got that wrong | 17:27 |
| *** felixedel has quit IRC | 17:27 | |
| AJaeger | zuul-jobs-maint, there're two changes to fix missing virtualenv for bindep: https://review.opendev.org/#/c/727561/ and https://review.opendev.org/693637 . IMHO, we should get 727561 merged, maybe with the change requested - but the other changes in 693637 need further discussion. Especially the bindep change to serialize is something I feel bad about. | 17:28 |
| *** bhavikdbavishi has quit IRC | 17:28 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 17:28 | |
| avass | corvus, clarkb, tristanC, mordred: want to merge the new zuul-jobs policy ? https://review.opendev.org/#/c/724855/ | 17:29 |
| AJaeger | corvus, mordred, want to review the fetch_output stack at https://review.opendev.org/#/c/681905/ , please? | 17:29 |
| *** nils has quit IRC | 17:30 | |
| *** dpawlik has quit IRC | 17:36 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Default to Ansible 2.9 https://review.opendev.org/727345 | 17:37 |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Specifically use python 3.7 base images https://review.opendev.org/727852 | 17:46 |
| clarkb | corvus: tobiash ^ A followup to make that explicit 3.7 dep | 17:46 |
| tobiash | ++ | 17:47 |
| mnaser | clarkb, Open10K8S, tristanC: i'm wondering what life would be like if we used kind instead of minikube with 'none' driver | 17:47 |
| clarkb | mnaser: what is kind? | 17:47 |
| mnaser | clarkb: https://kind.sigs.k8s.io | 17:47 |
| AJaeger | zuul-jobs maintainer, https://review.opendev.org/727475 looks like an important fix for tox siblings. mordred, could you review, please? | 17:47 |
| Open10K8S | k8s in docker | 17:48 |
| mnaser | you can also do things like simulated multi-node clusters | 17:48 |
| tobiash | k8s in 20s? | 17:48 |
| tobiash | awesome | 17:48 |
| mnaser | that's the claim :) i've seen it perform prett yfast | 17:49 |
| *** felixedel16 has joined #zuul | 17:49 | |
| mnaser | also, if we start using a docker cache in infra (unsure if we already do?) -- it will be even faster | 17:49 |
| mnaser | no need to download that big blob of binary in every build | 17:49 |
| mnaser | it also feels like it works better for our model, minikube --driver=none seems like it's an "afterthought" imho | 17:49 |
| fungi | in opendev we proxy-cache dockerhub on machines in the same location as our job nodes | 17:50 |
| mnaser | yeah so that means our ensure-kubernetes jobs should leverage that! | 17:50 |
| fungi | so hopefully subsequent fetches of the same images go over local data center networking and not the internet at large | 17:50 |
| clarkb | mnaser: I think it already does | 17:50 |
| clarkb | as our docker installation roles configure the mirror | 17:50 |
| mnaser | clarkb: right now minikube downloads this huge blob of data that it exports locally from i dunno what | 17:51 |
| mnaser | its like some 180MB of i dont know what really. | 17:51 |
| mnaser | haven't cared enough to dig into the details | 17:51 |
| clarkb | ya I don't really have an opinion on k8s installers | 17:51 |
| *** felixedel16 is now known as felixedel | 17:51 | |
| clarkb | minikube was chosen because it could run without talkign to cloud resources iirc | 17:51 |
| mnaser | https://kind.sigs.k8s.io/docs/user/private-registries/ | 17:51 |
| *** felixedel has quit IRC | 17:51 | |
| mnaser | it could also potentially neatly integrate with registry too | 17:51 |
| mnaser | i mean or we can just build something from kubeadm which is the defacto setup... | 17:52 |
| clarkb | looks like kind may not support other runtimes? | 17:52 |
| clarkb | that might be a regression, I'm not sure if anyone is using the minikube stuff with not docker | 17:53 |
| mnaser | possible. if we just did kubeadm, we'd eliminate the extra cruft and it'll work with docker and not docker too :) | 17:53 |
| clarkb | someones got to do that work though | 17:53 |
| mnaser | and it would enable multinode jobs | 17:53 |
| clarkb | "just do kubeadm" is underselling it | 17:53 |
| mnaser | of course. we can probably help pushing that work | 17:54 |
| *** felixedel has joined #zuul | 17:54 | |
| clarkb | thats like saying "just do multinode devstack" :) | 17:54 |
| mnaser | you'd be surprised at how k8s world has managed to simplify deployment | 17:54 |
| clarkb | mnaser: the issues dont' tend to be with the software | 17:54 |
| mnaser | its a matter of kubeadm init on the first host and kubeadm join on the other hosts. it's kinda ridiculous. heh. | 17:55 |
| clarkb | the issues tend to be with networking with you start talking multinode | 17:55 |
| jkt | can I somehow force tests for project B to run when I propose a change for project A? Right now B pulls in A as a submodule, and I'm only using the check pipeline, not gate. I can manually upload changes with Depends-On to the B project, and "the right thing" happens | 17:55 |
| clarkb | you'll have vxlan running on vxlan | 17:55 |
| clarkb | or whatever and then get to debug all the 1400MTU problems | 17:55 |
| mnaser | yeah, the CNIs are smart enough to read the host device and figure out mtus and what not. they're built to run on clouds ;) | 17:55 |
| jkt | (think of A as a provider and B as a consumer) | 17:55 |
| clarkb | mnaser: thats good | 17:55 |
| mnaser | vs openstack which is built to run on metal | 17:55 |
| clarkb | jkt: you can update the pipeline config for project A to run project B's tests. YOu might need to udpate the tests slightly to understand running within the context of project A | 17:56 |
| clarkb | jkt: this is a fairly common pattern for integration testing. You'll have some test suite/tool define jobs then all the projects it integrates run those jobs | 17:57 |
| clarkb | mnaser: my only real interaction with a "proper" k8s install has been via magnum and that basically forced me to do everything by hand | 17:57 |
| jkt | clarkb: I've been using Zuul for like five years, but I think I cannot figure this out :) | 17:57 |
| clarkb | jkt: let me try and pull together some examples | 17:58 |
| mnaser | clarkb: yeah it's quite a big difference imho | 17:58 |
| *** harrymichal has quit IRC | 17:58 | |
| *** harrymichal has joined #zuul | 17:58 | |
| clarkb | jkt https://opendev.org/openstack/tempest/src/branch/master/.zuul.yaml#L165-L195 that is an integration job defined by the test tool (tempest). https://opendev.org/openstack/keystone/src/branch/master/.zuul.yaml#L217-L218 is consumption of that job by project being integrated in the job | 18:03 |
| clarkb | jkt: from the jobs perspective you need to set require-projects list to list out all the expected repos then operate with the assumption they are there not just against the change's src dir | 18:03 |
| clarkb | mnaser: I guess with kubeadm we'd have the benefit of being able to directly configure coredns | 18:05 |
| mnaser | clarkb: yes actually that's a good thing too, so we can just inject that right away | 18:05 |
| clarkb | jkt: https://opendev.org/openstack/devstack/src/branch/master/.zuul.yaml#L376-L383 thats what setting the required project for that job above looks like roughly (its actually done in several layers of inheritance but you can see here its a list of things that will be tested together. That then allows you to have the job oeprate assuming all are present to the correct checkouts and not use variables relative to | 18:07 |
| clarkb | the change under test) | 18:07 |
| jkt | clarkb: is that as simple as defining a job in the tempest repo, adding keystone into tempest's required-projects, and then listing that tempest-specific job as a job within keystone? | 18:07 |
| clarkb | jkt: yup, then have the job playbooks understand that it is always working on keystone, nova, neutron etc | 18:08 |
| clarkb | rather than just `execute command in pwd` | 18:08 |
| jkt | now that's either about 666x much simpler than I thought :), or I might be missing something super-obvious. Let me try this, thanks! | 18:09 |
| *** rlandy is now known as rlandy|lunch | 18:10 | |
| fungi | another example might be how some of the oslo libraries run unit test jobs for other projects which declare dependencies on those libraries | 18:10 |
| fungi | so that they can see if (made-up example, i don't know if this is an exact one) changes to oslo.config break keystone's unit tests | 18:11 |
| openstackgerrit | Merged zuul/zuul master: Switch back to python 3.7 https://review.opendev.org/727367 | 18:11 |
| fungi | so like clarkb says, the real trick in that case is making sure that the job runs tox in the keystone source tree even though it's triggered by a change for oslo.config | 18:11 |
| AJaeger | clarkb, could you review https://review.opendev.org/#/c/726449/ as cleanup for zuul-jobs, please? | 18:12 |
| clarkb | AJaeger: ya let me pull up ansible lint docs to udnerstand that but I'll leave a vote soon | 18:12 |
| AJaeger | clarkb: no need for ansible lint docs, that'S a zuul-tests.d change - so a job change | 18:14 |
| clarkb | AJaeger: ya and .rules/ is a custom rule for ansible-lint in zuul-jobs | 18:15 |
| clarkb | or rather a dir with custom rules | 18:15 |
| AJaeger | yep | 18:15 |
| openstackgerrit | Merged zuul/zuul master: Deprecate ansible 2.7 https://review.opendev.org/727344 | 18:15 |
| corvus | mnaser, clarkb: i have previously used kind and was happy with it, but i haven't tried to configure it to use a buildset registry | 18:19 |
| corvus | (depending on how much docker gets passed through, it could be that no work is required for that) | 18:20 |
| corvus | i agree it's worth looking into as a minikube replacement | 18:21 |
| *** hashar has joined #zuul | 18:21 | |
| jkt | so in case of tempest, the key is really this one, https://github.com/openstack/tempest/blob/master/roles/run-tempest/tasks/main.yaml#L53-L62 | 18:24 |
| corvus | jkt: yeah, especially the the chdir | 18:24 |
| jkt | because that thing tells tox to run in tempest's own directory no matter what project triggers that | 18:24 |
| corvus | jkt: we have a convention in zuul_jobs of defining a variable "zuul_work_dir" which defaults to the current project under test, but can be overidden by a job to specify a specific project's dir | 18:25 |
| corvus | jkt: that lets you easily turn a simple job that runs the current project's tox tests into a job that run's a *specific* project's tox tests | 18:26 |
| jkt | with something like https://gerrit.cesnet.cz/plugins/gitiles/ci/zuul-jobs-cesnet/+/refs/heads/master/zuul.yaml#1 , I cannot really do that unless I introduce a specific role for my consumer, right? | 18:26 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Report dequeued changes via Github checks API https://review.opendev.org/711023 | 18:28 |
| corvus | jkt: heh, unfortunately run-test-command is not a job that uses zuul_work_dir. but we can make it do that, then you could easily do that. 1 sec. | 18:28 |
| jkt | corvus: yeah, I am specifically doing https://gerrit.cesnet.cz/plugins/gitiles/ci/zuul-jobs-cesnet/+/refs/heads/master/roles/run-test-command/tasks/main.yaml#4 | 18:29 |
| felixedel | corvus: I've renamed everything back to "dequeue" in https://review.opendev.org/#/c/711023/6. I've also adapted the https://review.opendev.org/#/c/709135/9 accordingly (the release notes and the "queue" leftover) | 18:30 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add zuul_work_dir to run-test-command https://review.opendev.org/727855 | 18:31 |
| jkt | also, it might be even more interesting because we actually have project A, which is company-internal, and therefore using "company-internal" Zuul tenant, then project B, which is opensource, so it's using a "public" Zuul tenant, and *both* are consumed in project C which is once again company-internal, and therefore once again via that "company-internal" tenant | 18:31 |
| corvus | jkt: ^ that should do the trick | 18:31 |
| jkt | and I would like to have A and B trigger a build of C | 18:32 |
| jkt | looks like I would have to have both Zuul tenants build project B :( | 18:32 |
| openstackgerrit | Merged zuul/zuul-jobs master: ansible-lint-rules: Fix bad path and filename https://review.opendev.org/726449 | 18:33 |
| corvus | jkt: crossing tenants gets really tricky; they are meant to be completely isolated. there are lots of possibilities; they just get complex. | 18:33 |
| jkt | yeah, well, I started using them because I wanted to avoid having the general public access our internal build logs | 18:33 |
| avass | jkt, corvus: can confirm, it gets complicated | 18:34 |
| jkt | everything lives in one Gerrit server, so I have to use fake FQDNs when doing Depends-On, etc | 18:34 |
| corvus | felixedel: thanks! | 18:34 |
| jkt | because of courtse we also have repo Z which provides some artifacts to both A and B above | 18:34 |
| jkt | pain, pain, pain. | 18:34 |
| corvus | sometimes zuul reflects the real world a bit more than would be ideal :) | 18:35 |
| jkt | btw, I have not found this cross-project integration (even without getting into the multi-tenant mess) in the docs. Was that my fault, or is that not documented? | 18:37 |
| corvus | jkt: i don't think it's documented; that would be a great howto | 18:37 |
| avass | corvus: oh regarding that, do you think it would be possible to limit node labels to specific projects? | 18:37 |
| avass | corvus: is there anything blocking that being implemented? | 18:38 |
| corvus | jkt: these get *really* close to talking about what's needed, but don't quite get there i think: https://zuul-ci.org/docs/zuul/howtos/user.html | 18:38 |
| *** rlandy|lunch is now known as rlandy | 18:39 | |
| jkt | BTW, in openstack, where is that zuul_work_dir actually set? It is in neither of the linked files AFAICS | 18:39 |
| corvus | jkt: those jobs are old, they don't use it | 18:39 |
| corvus | jkt: here's an example of a job that does though: https://opendev.org/zuul/nodepool/src/branch/master/.zuul.yaml#L14 | 18:40 |
| corvus | jkt: we run that job on changes to both zuul and nodepool, and it always runs the "tox -e nodepool" in the zuul repo. | 18:41 |
| jkt | corvus: let me rephrase, then -- where should I introduce this "project srcdir awareness"? So far I was using job names that were reusable among my A, B, Z projects, like fedora31-gcc, or fedora29-clang-asan | 18:42 |
| *** felixedel has quit IRC | 18:42 | |
| jkt | I actually have a unique name for my jobs in C (the "final consumer"), so it's possible to hardcode stuff in there | 18:42 |
| corvus | avass: i think we'd probably want a quick spec to talk about that; i don't think there's anything like that right now -- the only kind of 'restrictions' we have are by tenant, so that doesn't really have a precedent. | 18:42 |
| corvus | jkt: yeah probably C. you'll want a dedicated job name for this. doesn't need to do much than inherit from run-test-command and set the vars. | 18:44 |
| corvus | jkt: and set required-projects. | 18:44 |
| corvus | jkt: should look almost exactly like https://opendev.org/zuul/nodepool/src/branch/master/.zuul.yaml#L1-L15 but without the pre- and post-run | 18:45 |
| avass | corvus: since I think we might want to be able to keep projects since some of them are separeted just because they don't want to share nodes | 18:45 |
| avass | corvus: and I found that I wanted that today when I tried to implement a job to automatically deploy our nodepool config :) | 18:46 |
| corvus | avass: yeah. i think the biggest thing is whether that should be a project-config setting (config-projects only? in the future when we have tenant-config-projects, would it apply there?) or a tenant-config? or nodepool? (we usually say "no" to this, but always worth considering) | 18:47 |
| corvus | avass: so we'll want to collect some use cases :) | 18:47 |
| avass | corvus: there's no hurry :) | 18:48 |
| jkt | corvus: I think I get this now, thanks for your help! | 18:48 |
| corvus | jkt: np, good luck! | 18:48 |
| corvus | AJaeger, tristanC: first 2 fetch-output changes lgtm; #3 needs lunch :) | 18:49 |
| * jkt will needs that, these are C++ projects where build result caching makes a difference between two mintues and twenty minutes builds | 18:49 | |
| corvus | jkt: there is currently a feature to restrict node labels to tenants, just not projects | 18:49 |
| corvus | jkt: so if you trust internal projects equally, and external projects different but equally, then you can use that | 18:50 |
| avass | corvus: I'll start on a spec for it whenever I get time to do it | 18:50 |
| corvus | jkt: (with your current tenant separation) | 18:50 |
| jkt | corvus: I'm OK with running both "company-internal" and "opensource" projects on the same cloud; the only thing I want is separation of logs and artifacts and live consoles | 18:52 |
| jkt | ("one the same cloud" == "identical set of nodepool servers") | 18:53 |
| avass | corvus: and I think in the future we might have nodes connected to hardware that a team under NDA is using, so if we don't need to separate those into another tenant that would be good | 18:58 |
| openstackgerrit | Merged zuul/zuul-jobs master: fetch-sphinx-output: introduce zuul_use_fetch_output https://review.opendev.org/681905 | 18:59 |
| avass | corvus: but we might need to do that anyway to restrict logs etc | 18:59 |
| jkt | hear, hear :( | 18:59 |
| tristanC | corvus: mnaser: i heard good thing about kind, not sure it is a drop-in replacement for k8s though, perhaps we could add a new role? | 19:00 |
| tristanC | fwiw i also wrote a small k8s installer based on usernetes here: https://github.com/podenv/silverkube/ | 19:01 |
| jkt | perhaps I'm looking for bandaids and kludges here, but is it possible to have two tenants (or even separate zuul servers) working with one Gerrit repo, but executing different jobs? i.e., configure tenant A to use $PROJECT/.zuul.yaml, and tenant B to use $PROJECT/.zuul-B.yaml ? | 19:01 |
| tristanC | jkt: yes you can, though i think you need tenant A to use a .zuul-A.yaml | 19:02 |
| tristanC | jkt: using this configuration https://zuul-ci.org/docs/zuul/reference/tenants.html#attr-tenant.untrusted-projects.%3Cproject%3E.extra-config-paths | 19:03 |
| avass | jkt: yes, but I wouldn't recommend having multiple zuuls gating the project | 19:03 |
| corvus | jkt: you can also do that without specifying different files by being strategic about what config options you put in the repo or instruct the various tenants to load. for example, have all the tenants load jobs, but only have one tenant load "project" objects, and use a config-project to specify the project-pipeline in the other tenant. | 19:03 |
| openstackgerrit | Merged zuul/zuul-jobs master: fetch-sphinx-tarball: add missing zuul_success default https://review.opendev.org/727272 | 19:04 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add zuul_work_dir to run-test-command https://review.opendev.org/727855 | 19:04 |
| jkt | tristanC: awesome, thanks! | 19:04 |
| jkt | corvus: I'll have to think about this one | 19:04 |
| jkt | avass: right :(, I have not thought about that. Right now we're only "check"ing | 19:04 |
| jkt | corvus: ok, I think I understand. That makes sense (and I think I understand the cost, now that project's testing config is no longer self-contained...) | 19:08 |
| *** y2kenny has joined #zuul | 19:08 | |
| openstackgerrit | Merged zuul/nodepool master: container functional test : allow to specify elements-dir https://review.opendev.org/724452 | 19:09 |
| mnaser | tristanC, corvus: Open10K8S is going to hack at using kubeadm instead of minikube (minikube is really a wrapper around kubeadm mostly) so we can do easy multinode stuff and really have an equivilant of a real production deployment | 19:14 |
| openstackgerrit | Merged zuul/zuul master: Specifically use python 3.7 base images https://review.opendev.org/727852 | 19:19 |
| openstackgerrit | Merged zuul/zuul master: pagure: Make use of the new project webhook/token endpoint https://review.opendev.org/717732 | 19:37 |
| mnaser | Open10K8S, clarkb: https://github.com/kubernetes/minikube/blob/294a5c3e7b363a999bdb5f73ef6e1b2b8b275193/pkg/minikube/driver/driver.go#L172-L177 i think here is the issue | 19:39 |
| Open10K8S | mnaser: yeah | 19:40 |
| mnaser | Open10K8S: it seems like a big hack, but maybe we can disable systemd-resolved service and rm that file before running minikube? | 19:47 |
| mnaser | (this whole dns thing is a hack in the first place) | 19:47 |
| Open10K8S | mnaser: in fact, i tried to stop and disable the systemd-resolved already | 19:48 |
| Open10K8S | but the minikube tries to auto detect that file | 19:48 |
| Open10K8S | and if there is no such file, then fail, maybe | 19:48 |
| *** hashar has quit IRC | 19:48 | |
| openstackgerrit | Merged zuul/zuul master: Ensure we use recent enough virtualenv https://review.opendev.org/718425 | 19:50 |
| Open10K8S | mnaser: remove that file seems work | 19:52 |
| Open10K8S | let me double check | 19:52 |
| mnaser | Open10K8S: ok so maybe what we can do is remove that file, minikube start and add it again..? so we leave the server in "pristine" state | 19:52 |
| *** bhavikdbavishi has quit IRC | 19:56 | |
| Open10K8S | mnaser: i am afraid if the systemd resolv is using for the minikube bootstraping | 20:00 |
| mnaser | Open10K8S: true. damn :( | 20:01 |
| Open10K8S | mnaser: once minikube dev team accept the current problem as a bug, then we can wait for them | 20:02 |
| Open10K8S | meanwhile, we can patch the corefile in the ensure-k8s job | 20:02 |
| Open10K8S | That is my opinion | 20:02 |
| Open10K8S | mnaser: clarkb | 20:03 |
| Open10K8S | mnaser: clarkb: how about this? | 20:03 |
| clarkb | patching the coredns config? makes sense | 20:04 |
| Open10K8S | clarkb: yes | 20:05 |
| Open10K8S | clarkb: kubectl patch cm coredns -n kube-system --patch="Corefile: | | 20:05 |
| Open10K8S | .:53 { | 20:05 |
| Open10K8S | errors | 20:05 |
| Open10K8S | health { | 20:05 |
| Open10K8S | lameduck 5s | 20:05 |
| Open10K8S | } | 20:05 |
| Open10K8S | ready | 20:05 |
| Open10K8S | kubernetes cluster.local in-addr.arpa ip6.arpa { | 20:05 |
| Open10K8S | pods insecure | 20:05 |
| Open10K8S | fallthrough in-addr.arpa ip6.arpa | 20:05 |
| Open10K8S | ttl 30 | 20:05 |
| Open10K8S | } | 20:05 |
| Open10K8S | prometheus :9153 | 20:05 |
| Open10K8S | forward . 8.8.8.8 | 20:05 |
| mnaser | Open10K8S: please use a pastebin in the future | 20:06 |
| Open10K8S | aha, ok | 20:07 |
| mnaser | such as paste.openstack.org | 20:07 |
| Open10K8S | mnaser: I got your point. | 20:07 |
| mnaser | Open10K8S: i think thats an ok workaround. just make sure you use the dns servers that we supply in the role. let's do that for now and leave a note linking to the minikube bug | 20:07 |
| Open10K8S | mnaser: ok | 20:08 |
| openstackgerrit | Oleksandr Kozachenko proposed zuul/zuul-jobs master: Patch CoreDNS corefile The minikube v1.10.x is appending the systemd-resolved conf always. So to workaround this problem, do a patch after deployment. https://review.opendev.org/727868 | 20:30 |
| clarkb | mnaser: ya its a list I tried to clarify the structure in my comment | 20:33 |
| mnaser | ++ | 20:33 |
| openstackgerrit | Oleksandr Kozachenko proposed zuul/zuul-jobs master: Patch CoreDNS corefile The minikube v1.10.x is appending the systemd-resolved conf always. So to workaround this problem, do a patch after deployment. https://review.opendev.org/727868 | 20:39 |
| *** armstrongs has quit IRC | 20:41 | |
| *** y2kenny has quit IRC | 20:41 | |
| openstackgerrit | Oleksandr Kozachenko proposed zuul/zuul-jobs master: Patch CoreDNS corefile The minikube v1.10.x is appending the systemd-resolved conf always. So to workaround this problem, do a patch after deployment. https://review.opendev.org/727868 | 20:42 |
| corvus | Open10K8S: you probably want a blank line between the first 2 lines of that commit message (that's why the bot is outputting the whole thing here) | 20:47 |
| Open10K8S | corvus: gotcha! | 20:50 |
| openstackgerrit | Oleksandr Kozachenko proposed zuul/zuul-jobs master: Patch CoreDNS corefile https://review.opendev.org/727868 | 20:58 |
| openstackgerrit | Oleksandr Kozachenko proposed zuul/zuul-jobs master: Patch CoreDNS corefile https://review.opendev.org/727868 | 21:05 |
| Open10K8S | clarkb: POST_FAILUER https://review.opendev.org/#/c/727868/ | 21:13 |
| Open10K8S | clarkb: http://paste.openstack.org/show/793540/ | 21:14 |
| Open10K8S | clarkb: Could you check this? | 21:14 |
| clarkb | Open10K8S: yup looks like we failed to install docker and that meant minikube wasn't running. Later in post-run we try to collect minikube logs and that failed because there was no minikube install so we report post failure | 21:14 |
| clarkb | corvus: ^ should zuul only report post failrue if run was successful? | 21:15 |
| clarkb | Open10K8S: I think that might be an "Internet is flaky" type error | 21:15 |
| clarkb | it failed to do a tls handshake with docker's apt mirror | 21:15 |
| clarkb | Open10K8S: I think you can just recheck this case | 21:16 |
| *** rfolco|rover has quit IRC | 21:16 | |
| corvus | clarkb: no -- post_failure does not imply any particular "run" result | 21:17 |
| corvus | clarkb: iow, post_failure masks success/failure. the only thing it doesn't mask is a pre-run failure of some kind. | 21:17 |
| Open10K8S | clarkb: ok | 21:17 |
| clarkb | ya I guess thats true. But I think it points peopel to the wrong location in some cases | 21:17 |
| clarkb | we almost need to report the states for each stage (but that will get noisy and annoying too I bet) | 21:18 |
| corvus | arguable post-failures are more important to fix than run failures -- they mean that the results themselves (in terms of logs) are not reliable | 21:18 |
| corvus | (in other words, be careful diagnosing a run failure if there has been a post failure) | 21:19 |
| corvus | anyway, i'm hesitant to change it due to the workflow when constructing a new job | 21:20 |
| corvus | that's a slightly more advanced activity, and i think we can expect that extra investigation may be required. | 21:21 |
| clarkb | ya it still accurate reports something is wrong and you have work to do | 21:21 |
| corvus | oh, also this is in a zuul-test job; normally i'd expect ensure-k8s to be in a pre-playbook | 21:22 |
| corvus | where that failure would have triggered a retry anyway | 21:23 |
| corvus | so it's even more unusual in this case | 21:23 |
| clarkb | ah good point | 21:23 |
| corvus | (ie, under normal usage, it's entirely reasonable for the post-run log collection to assume k8s is present) | 21:23 |
| clarkb | we aren't testing minikube or docker, they are just tools we need to bootstrap | 21:23 |
| clarkb | except in the zuul-jobs role tests | 21:23 |
| *** Goneri has quit IRC | 22:07 | |
| *** Goneri has joined #zuul | 22:10 | |
| *** harrymichal has quit IRC | 22:10 | |
| openstackgerrit | Oleksandr Kozachenko proposed zuul/zuul-jobs master: Patch CoreDNS corefile https://review.opendev.org/727868 | 22:13 |
| ianw | does setting bindep_command in https://review.opendev.org/#/c/727593/5/test-playbooks/bindep.yaml not override the zuul site version? | 22:15 |
| ianw | ooohhh, i'm unsetting in pre_tasks then running bindep in tasks | 22:16 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Unset bindep_command to exercise install paths https://review.opendev.org/727593 | 22:17 |
| ianw | hrm, that still doesn't seem to override bindep_command ... https://zuul.opendev.org/t/zuul/build/e405e33f0ce0428289c784ca8b3b30e4/console | 22:42 |
| clarkb | ianw: "Per the standard Ansible variable precedence rules, many other types of variables have a higher priority, so this value may be overridden." I wonder if that is related | 22:43 |
| clarkb | found that quote on https://docs.ansible.com/ansible/latest/modules/set_fact_module.html | 22:43 |
| ianw | yeah ... i didn't realise a zuul site-variable would override a set_fact, maybe that makes sense | 22:53 |
| ianw | it leaves it a bit difficult to really test the bindep role though, because it is always finding the inbuilt bindep ... we could add a testing flag to the production roles i guess | 22:54 |
| clarkb | ianw: what if you set it explicitly whne calling the role? | 22:54 |
| clarkb | not as a set_fact but as a direct var to the role call | 22:54 |
| ianw | oh as in var: | 22:55 |
| clarkb | ya | 22:55 |
| ianw | worth a try | 22:55 |
| clarkb | I wonder if that has higher precendent due to locality | 22:55 |
| ianw | we shall find out | 22:57 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Unset bindep_command to exercise install paths https://review.opendev.org/727593 | 22:57 |
| *** tosky has quit IRC | 23:06 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Unset bindep_command to exercise install paths https://review.opendev.org/727593 | 23:06 |
| clarkb | https://blogs.gnome.org/alexl/2020/05/13/putting-container-updates-on-a-diet/ probably a bit early for us to start worrying about that, but the heavyeight cost of container updates is something I've been looking at in opendev because it fills disks and puts pressure on networks | 23:25 |
| clarkb | it would be great to see more work in that space to make our mirrors and caches and disks that much happier | 23:25 |
| clarkb | (also I find it really weird its ok to completely change the format of docker images in podman and skopeo but adding ipv6 support would be too much) | 23:27 |
| clarkb | I guess that PR isn't merged yet | 23:27 |
| *** threestrands has joined #zuul | 23:29 | |
| *** rlandy has quit IRC | 23:33 | |
| tristanC | clarkb: fwiw we have been looking at performing in-place update (instead of complete rebuild) to ship only new layer when relevant | 23:38 |
| clarkb | tristanC: doesn't that have problems too of needing all those intermediate steps? | 23:40 |
| clarkb | (and eventually the total of those will be greater than the total of a single rebuild) | 23:40 |
| tristanC | that is indeed an issue, and there is also a limit for the maximum number of layer an image can have (iirc 255) | 23:42 |
| clarkb | I guess for long lived servers pullin gsmaller intermediate updates is better for bw | 23:44 |
| clarkb | but for disk use it might not fix all the problems. | 23:44 |
| clarkb | and for single use throwaway instances used in eg testing it might be similar to the rebuilt updates? | 23:45 |
| tristanC | disk usage is also problematic, but perhaps that could be fixed using local squashing | 23:48 |
| tristanC | also we haven't figure out how to ensure in-place update works for multi-image setup like zuul use with a builder throwaway container | 23:50 |
| tristanC | for now we only looked at distro package, and the update job simply performs `dnf update -y` | 23:51 |
| tristanC | here is the zuul pipeline and jobs we have been testing: https://softwarefactory-project.io/cgit/zuul-images-jobs/tree/example/pipeline.yaml | 23:52 |
| ianw | https://zuul.opendev.org/t/zuul/build/c4fa4acaf42041de9e42971e3d061f0d/console .. it doesn't seem i can convince it to ignore the bindep_command set in site variables | 23:53 |
| ianw | ... although ... i wonder if blank has different semantics | 23:54 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] Unset bindep_command to exercise install paths https://review.opendev.org/727593 | 23:56 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!