| openstackgerrit | Merged zuul/zuul master: Add ensure-pip to quick-start job https://review.opendev.org/735905 | 00:19 |
|---|---|---|
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-virtualenv: call ensure-pip for Xenial https://review.opendev.org/736067 | 00:23 |
| *** wuchunyang has quit IRC | 00:38 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-virtualenv: call ensure-pip for Xenial https://review.opendev.org/736067 | 00:38 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-pip: remove Xenial venv workaround https://review.opendev.org/736068 | 00:38 |
| *** wuchunyang has joined #zuul | 00:39 | |
| *** wuchunyang has quit IRC | 00:51 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-virtualenv: call ensure-pip for Xenial https://review.opendev.org/736067 | 00:53 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-pip: remove Xenial venv workaround https://review.opendev.org/736068 | 00:53 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-pip: install from EPEL for Python 2 https://review.opendev.org/736070 | 00:53 |
| *** Goneri has joined #zuul | 00:54 | |
| *** olaph has quit IRC | 01:14 | |
| *** wuchunyang has joined #zuul | 01:20 | |
| *** wuchunyang has quit IRC | 01:31 | |
| *** wuchunyang has joined #zuul | 01:48 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Remove the -plain job variants https://review.opendev.org/735774 | 01:48 |
| *** Goneri has quit IRC | 01:48 | |
| *** swest has quit IRC | 01:52 | |
| *** swest has joined #zuul | 02:06 | |
| *** wxy has quit IRC | 02:21 | |
| *** wuchunyang has quit IRC | 02:21 | |
| *** wuchunyang has joined #zuul | 02:35 | |
| *** wuchunyang has quit IRC | 02:41 | |
| *** wuchunyang has joined #zuul | 02:41 | |
| *** bhavikdbavishi has joined #zuul | 02:55 | |
| *** wuchunyang has quit IRC | 03:00 | |
| *** bhavikdbavishi1 has joined #zuul | 03:06 | |
| *** bhavikdbavishi has quit IRC | 03:08 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:08 | |
| *** wuchunyang has joined #zuul | 03:16 | |
| *** wuchunyang has quit IRC | 03:19 | |
| *** ysandeep|away is now known as ysandeep | 03:55 | |
| *** bhavikdbavishi has quit IRC | 04:20 | |
| *** bhavikdbavishi has joined #zuul | 04:21 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #zuul | 04:33 | |
| *** wuchunyang has joined #zuul | 04:38 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Remove the -plain job variants https://review.opendev.org/735774 | 04:48 |
| *** wuchunyang has quit IRC | 04:52 | |
| openstackgerrit | Merged zuul/zuul-jobs master: ensure-pip: install from EPEL for Python 2 https://review.opendev.org/736070 | 04:53 |
| openstackgerrit | Merged zuul/zuul-jobs master: ensure-virtualenv: call ensure-pip for Xenial https://review.opendev.org/736067 | 04:53 |
| *** felixedel has joined #zuul | 05:22 | |
| openstackgerrit | Merged zuul/nodepool master: Add ensure-tox to k8s and openshift jobs https://review.opendev.org/735906 | 05:23 |
| *** threestrands has joined #zuul | 05:27 | |
| *** wuchunyang has joined #zuul | 05:31 | |
| *** threestrands has quit IRC | 05:33 | |
| *** wuchunyang has quit IRC | 05:35 | |
| *** bhavikdbavishi has quit IRC | 05:41 | |
| openstackgerrit | Merged zuul/nodepool master: Implement an Azure driver https://review.opendev.org/554432 | 05:46 |
| *** felixedel has quit IRC | 06:00 | |
| *** bhavikdbavishi has joined #zuul | 06:04 | |
| *** wuchunyang has joined #zuul | 06:12 | |
| *** felixedel has joined #zuul | 06:36 | |
| openstackgerrit | Merged zuul/zuul-jobs master: ensure-pip: remove Xenial venv workaround https://review.opendev.org/736068 | 06:37 |
| *** rpittau|afk is now known as rpittau | 06:56 | |
| *** jcapitao has joined #zuul | 07:01 | |
| *** bhavikdbavishi has quit IRC | 07:14 | |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Rework quick-start and prepare for other tutorials https://review.opendev.org/732066 | 07:17 |
| *** bhavikdbavishi has joined #zuul | 07:30 | |
| *** wuchunyang has quit IRC | 07:33 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: REST API, Web UI: implement nodes filtering https://review.opendev.org/736042 | 07:36 |
| *** tosky has joined #zuul | 07:36 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: zuul-web: support OPTIONS for protected endpoints https://review.opendev.org/734134 | 07:38 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: zuul-web: refactor auth token handling code https://review.opendev.org/734139 | 07:38 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: CLI: Fix errors with the REST client https://review.opendev.org/728061 | 07:44 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: REST API: fix discrepancies between RPC and REST outputs for autohold https://review.opendev.org/728073 | 07:44 |
| *** jpena|off is now known as jpena | 07:55 | |
| *** nils has joined #zuul | 08:23 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Add optional support for circular dependencies https://review.opendev.org/685354 | 08:23 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "gate your first patch" https://review.opendev.org/732067 | 08:26 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "Use zuul jobs" https://review.opendev.org/732068 | 08:26 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "gate pipeline" https://review.opendev.org/732069 | 08:27 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "job secrets" https://review.opendev.org/732070 | 08:27 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "job dependencies" https://review.opendev.org/732071 | 08:27 |
| *** felixedel has quit IRC | 08:35 | |
| *** sgw1 has quit IRC | 09:06 | |
| sshnaidm | folks, I'd appreciate your review on https://review.opendev.org/#/c/730360/ thanks | 09:18 |
| sshnaidm | can we merge this? https://review.opendev.org/#/c/728073 fix discrepancies between RPC and REST outputs for autohold | 09:22 |
| mhu | sshnaidm, it's part of a patch chain, a few need to be merged first | 09:24 |
| sshnaidm | mhu, ack | 09:25 |
| mhu | but all the patches prior to this one are ready to be merged IMO | 09:25 |
| mhu | at least ready for review | 09:25 |
| *** ysandeep is now known as ysandeep|afk | 09:26 | |
| *** wxy has joined #zuul | 09:31 | |
| *** bhavikdbavishi has quit IRC | 09:34 | |
| *** bhavikdbavishi has joined #zuul | 09:35 | |
| *** bhavikdbavishi1 has joined #zuul | 09:54 | |
| *** ysandeep|afk is now known as ysandeep | 09:54 | |
| *** bhavikdbavishi has quit IRC | 09:56 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 09:56 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: REST API, Web UI: implement nodes filtering https://review.opendev.org/736042 | 09:58 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Add simple testing for Zuul CLI & REST API https://review.opendev.org/728098 | 10:01 |
| *** jcapitao is now known as jcapitao_lunch | 10:15 | |
| *** rpittau is now known as rpittau|bbl | 10:18 | |
| *** bhavikdbavishi has quit IRC | 10:25 | |
| *** bhavikdbavishi has joined #zuul | 10:26 | |
| *** bhavikdbavishi has quit IRC | 10:42 | |
| jkt | what is Zuul going to do when I already have a .zuul.yaml in some feature-branch, and I upload a merge commit from this feature-branch to master, where master did not have any .zuul.yaml? | 10:56 |
| jkt | because right now what I seem to be getting with a managed Zuul from vexxhost, is that this build briefly shows up in the status page, and then it disappears without conducting any builds | 10:57 |
| jkt | OTOH, after adding a noop job to check in master's .zuul.yaml and submitting/merging that change, it seems to work | 10:58 |
| jkt | I'm surprised by this because I would have thought that whatever jobs are defined in the change in question should take precedence, and therefore also override the "no config here" in my old master | 10:59 |
| *** jpena is now known as jpena|lunch | 11:30 | |
| *** sshnaidm_ has joined #zuul | 11:32 | |
| *** sshnaidm has quit IRC | 11:33 | |
| *** sshnaidm_ is now known as sshnaidm | 11:35 | |
| *** rfolco|rover has joined #zuul | 11:36 | |
| *** tosky has quit IRC | 11:56 | |
| *** tosky has joined #zuul | 11:59 | |
| *** bhavikdbavishi has joined #zuul | 12:02 | |
| *** bhavikdbavishi1 has joined #zuul | 12:05 | |
| *** rpittau|bbl is now known as rpittau | 12:06 | |
| *** bhavikdbavishi has quit IRC | 12:06 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 12:06 | |
| *** jcapitao_lunch is now known as jcapitao | 12:10 | |
| *** felixedel has joined #zuul | 12:12 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Record artifact checksums and signatures to stdout https://review.opendev.org/735929 | 12:15 |
| *** hashar has joined #zuul | 12:18 | |
| zbr | recent regression on build-python-release role: no longer works w/o sudo | 12:19 |
| zbr | mainly it now requires sudo on "Install wheel" | 12:19 |
| AJaeger | zbr, yes - see my comment on https://review.opendev.org/#/c/736001 | 12:21 |
| zbr | tristanC: mordred: mainly https://review.opendev.org/#/c/736001/ broke the role, | 12:21 |
| AJaeger | fixes welcome | 12:22 |
| zbr | i would revert it until we fix it, another change made w/o tests around it | 12:22 |
| zbr | when can we introduce a rule not to allow touching or roles w/o tests covering them? | 12:22 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Revert "Make sure wheel is installed for python releases" https://review.opendev.org/736183 | 12:25 |
| zbr | AJaeger: tristanC mordred ^ | 12:25 |
| tristanC | zbr: could you please share a console log with the sudo issue? | 12:27 |
| tristanC | oh i see AJaeger comment | 12:28 |
| tristanC | AJaeger: iiuc without 736001 python-release was no longer working in opendev (cc mordred), would revert do any good? | 12:29 |
| fbo | @zuul-maint is an alternative to cherrypy has been considered. The cheroot pinning is problematic as upstream is not really keen on merging the supposed fix https://github.com/cherrypy/cheroot/pull/277 | 12:30 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Avoid become on build-python-release https://review.opendev.org/736185 | 12:32 |
| zbr | tristanC: clearly a revert will fix breakage made to zuul.ansible.com instance | 12:34 |
| zbr | writing it with tests will take few hours and I am not happy to prevent others from merging their changes due to this | 12:35 |
| *** jpena|lunch is now known as jpena | 12:37 | |
| AJaeger | sorry, need to look at other things and can't help here further... | 12:45 |
| swest | tobiash: any plans to merge https://review.opendev.org/#/q/status:open+project:zuul/zuul+branch:master+topic:change-queues soon? ;) | 12:48 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 12:52 |
| tobiash | swest: that needs a non-me review ;) | 12:54 |
| webknjaz | @fbo: I can merge that fix once it doesn't break tests... | 12:54 |
| tobiash | mordred, tristanC ^ | 12:54 |
| corvus | zbr: thanks for the fix! two things: can you rebase that on the revert, since i've approved it? and also i left 2 comments inline. | 13:02 |
| *** sgw1 has joined #zuul | 13:02 | |
| fbo | webknjaz: I have no clue how to fix it btw. Who have ? | 13:05 |
| webknjaz | only the-allanc, I guess. but he only appears on GH occasionally | 13:06 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:06 |
| webknjaz | I'll try to figure out what I can do meanwhile | 13:06 |
| fbo | webknjaz: thanks a lot that would be nice. | 13:08 |
| corvus | webknjaz: is the issue the travis build? https://github.com/cherrypy/cheroot/pull/277/checks?check_run_id=590442991 it says the jobs were 'canceled' ? | 13:09 |
| openstackgerrit | Merged zuul/zuul-jobs master: Revert "Make sure wheel is installed for python releases" https://review.opendev.org/736183 | 13:10 |
| AJaeger | zbr: thanks for digging into this! | 13:11 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:12 |
| webknjaz | @corvus: there's multiple stages, failure in linting stage fails the build and the next ones don't go further. But there's also GHA jobs that mostly duplicate envs and the failures are visible there | 13:15 |
| corvus | webknjaz: ah, i can't see those since i'm not logged in to github :( | 13:16 |
| webknjaz | oh | 13:16 |
| corvus | i can see it failed, but i can't see the error | 13:16 |
| corvus | just "X Test with tox" | 13:16 |
| webknjaz | it should be clickable and show the console output | 13:17 |
| corvus | yeah there's i thing that says "Sign in for the full log view" | 13:17 |
| webknjaz | mostly the problem seems to be that it (or the previous commit in PR) fails in "slow" envs like macos | 13:17 |
| corvus | webknjaz: but the gha aren't running on macos, right? so it's failing at least some on other platforms?\ | 13:18 |
| webknjaz | GHA has ubuntu/macos/windows jobs in the matrix | 13:18 |
| corvus | but "Test suite / tests (3.7, ubuntu-latest, python)" is one of the failures, right? | 13:19 |
| webknjaz | yeah, I think it wasn't failing everywhere w/o a few last commits in that branch | 13:19 |
| webknjaz | without the last experimental commit it's greener: https://github.com/cherrypy/cheroot/pull/277/checks?check_run_id=585066716 | 13:20 |
| webknjaz | only macos failures + one flaky windows that can probably be ignored in the context of that PR | 13:20 |
| webknjaz | yeah, that's a TLS test and testing SSL stuff is PITA | 13:21 |
| corvus | okay, i logged in, and i now see the test failing on ubuntu, and it's failing keepalive-related tests | 13:22 |
| zbr | corvus: good to know about the simple role importer tick. sadly, it will not help us, as we need to run at least ensure-pip before, | 13:25 |
| zbr | as pip is missing from some platforms | 13:25 |
| webknjaz | @corvus see the last link, ubuntu is green there | 13:26 |
| corvus | zbr: in that case, a new playbook is warranted. should we think about making an ensure-wheel role, so this could be put in a pre-run playbook? | 13:27 |
| corvus | webknjaz: oh i think i understand now -- you're saying to ignore the most recent results because they're just an experiment, the real problems we need to fix are shown in https://github.com/cherrypy/cheroot/pull/277/checks?check_run_id=585066716 | 13:28 |
| webknjaz | yeah | 13:28 |
| corvus | webknjaz: sorry i missed that, it's obvious now :) | 13:28 |
| zbr | how about an ensure-double-shot-latte? there is no end to the ensure-* part. | 13:28 |
| zbr | i am more inclined to have a more intelligent role where we can specify what we want, in the end wheel is just a wheel like any other. | 13:29 |
| fbo | corvus: webknjaz I've experimented in that PR (with some additional commit) but w/o success https://github.com/cherrypy/cheroot/pull/287 | 13:29 |
| corvus | zbr: sure, but you can't split the installation into a pre-playbook without a separate role | 13:30 |
| corvus | zbr: and the ensure roles are all there so that we can do that, so if there's a momentary network problem installing wheel, then it's resolved. | 13:30 |
| zbr | in fact the the correct approach is to include ensure-pip as part of this role | 13:30 |
| zbr | ansible is smart and does not include role twice. | 13:31 |
| webknjaz | @fbo @corvus I think the slow workers influence the test in a way that timeouts appear more often in that env | 13:31 |
| corvus | zbr: well, ensure-pip would normally be in a different playbook, so it'll actually get run twice in produciton. | 13:31 |
| mordred | corvus: can I jump in on helping with the wheel issue? | 13:34 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:35 |
| corvus | mordred: we reverted the change, and zbr has a proposed improvement ^ | 13:35 |
| zbr | few extra eyes would not hurt here, i keep uncovering extra issues | 13:36 |
| mordred | cool, yeah. I see the conversation about splitting out an ensure-wheel and a second playbook - zbr are you working on that, I can if you're not | 13:36 |
| zbr | for example pyenv suite of jobs is triggered when in fact it should not | 13:36 |
| * mordred almost did that yesterday, should have | 13:36 | |
| corvus | zbr: you can drop the files matchers from there | 13:37 |
| corvus | zbr: that may have predated the job-self-change detection | 13:37 |
| zbr | corvus: so, in general there is no need to mention the file where a job is defined as zuul will know to auto include a job if its definition is touched by the change | 13:38 |
| zbr | that is indeed a very useful thing | 13:39 |
| corvus | mordred: no, in fact, it didn't predate the self-change detection; i think you may have just been a little overzealous in adding a files matcher to that job? | 13:39 |
| *** rpittau is now known as rpittau|brb | 13:39 | |
| corvus | zbr: correct | 13:39 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:40 |
| corvus | mordred, zbr: oh! we want at least one file matcher though | 13:40 |
| corvus | otherwise it'll run all the time | 13:41 |
| corvus | zbr, mordred: we should change that file matcher to "roles/ensure-python" | 13:41 |
| *** rlandy is now known as rlandy|mtg | 13:41 | |
| zbr | okey | 13:42 |
| corvus | and test-playbooks/ensure-python-pyenv.yaml | 13:42 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:44 |
| zbr | probably it would have being easier to use .*foo.* patterns for these | 13:45 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:45 |
| *** rpittau|brb is now known as rpittau | 13:53 | |
| *** bhavikdbavishi has quit IRC | 13:54 | |
| corvus | regexes are fine | 14:01 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: WIP: Introduce Patternfly 4 https://review.opendev.org/736225 | 14:06 |
| mhu | felixedel, woah I was just thinking about that, but I'm glad someone else is tackling this :p | 14:07 |
| *** ysandeep is now known as ysandeep|brb | 14:07 | |
| felixedel | mhu: Nice to hear that :) | 14:08 |
| AJaeger | corvus, do you want https://review.opendev.org/736226 to fix the configuration errors for the Zuul tenant? | 14:09 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 14:09 |
| mhu | felixedel, how hard would the migration to PF4 be? | 14:09 |
| mhu | for example I think query filters are handled differently in PF4 | 14:10 |
| felixedel | mhu: To be honest, I have no idea. I was hoping that somebody in this community is more familiar with this CSS framework and the transition. | 14:10 |
| mhu | felixedel, ahah my hope exactly | 14:11 |
| mhu | we really need to catch a few frontend devs and entrap them | 14:11 |
| felixedel | :D | 14:12 |
| felixedel | I wanted to check in which way PF3 and PF4 could be used together, so one could migrate one component after the other. So far, it looks like this might work. The most noticeable change is the larger font size and the new font type. | 14:13 |
| felixedel | Apart from that everything seems to look like before. Which should be a good starting point for a migration :) | 14:15 |
| *** felixedel has quit IRC | 14:23 | |
| *** felixedel has joined #zuul | 14:24 | |
| *** felixedel has quit IRC | 14:38 | |
| *** rlandy|mtg is now known as rlandy | 14:39 | |
| clarkb | jkt: I'm not sure you got an answer but I would expect the .zuul.yaml to apply to master if it was merged there. The exception to that behavior is if the project is a trusted project then you need to merge the change before config will apply | 14:48 |
| avass | mordred, corvus, zbr: looks like I missed a lot, can I get a tl;dr? | 14:52 |
| corvus | avass: we thought the build-python-release role was tested, but it wasn't, and it broke when used with revoke-sudo. we reverted, and zbr is working on a second attempt in https://review.opendev.org/736185 | 14:53 |
| corvus | https://review.opendev.org/736001 was the breaking change | 14:54 |
| avass | ah, because it was always running with super-user privileges? | 14:56 |
| *** Goneri has joined #zuul | 14:57 | |
| corvus | avass: i think it almost never runs that way; the initial problem was that opendev removed 'wheel' from its images | 14:58 |
| corvus | though i guess in opendev, we ran it without revoke-sudo, because we were able to run the job after that change merged | 15:00 |
| corvus | so i guess opendev runs it without revoke-sudo, but other installs do use revoke-sudo | 15:00 |
| AJaeger | corvus: we were not able to run the job | 15:00 |
| mordred | yeah- I checked the job in opendev and saw it wasn't running with revoke-sudo | 15:00 |
| mordred | AJaeger: oh, we weren't? my bad then | 15:00 |
| AJaeger | https://zuul.opendev.org/t/openstack/build/af6e45cad84245e7bdad1e4034ce505d | 15:00 |
| corvus | mordred, AJaeger: i thought https://review.opendev.org/735905 was the change that needed mordreds fix | 15:00 |
| corvus | AJaeger: key part of my sentence from 15:00 "after that change merged" -- where "that change" refers to https://review.opendev.org/736001 | 15:01 |
| corvus | once more from the top: build-python-release was broken by opendev's image changes, as illustrated by https://review.opendev.org/735905 so mordred wrote https://review.opendev.org/736001 to fix that, and it worked in opendev and https://review.opendev.org/735905 merged, but that broke ansible's zuul because they use revoke-sudo in their version of that job. so we reverted mordred's fix in | 15:04 |
| corvus | https://review.opendev.org/736183 meaning opendev is broken again now, and are looking to fix it for everyone in https://review.opendev.org/736185 | 15:04 |
| *** ysandeep|brb is now known as ysandeep | 15:04 | |
| avass | I left some comments on that, I don't think it should need 'become: true' when it's installing wheel for the user | 15:07 |
| AJaeger | corvus: Oh, what a cascade... | 15:08 |
| clarkb | avass: zbr also does installing a package with --user make it available when we later run "{{ release_python }} setup.py sdist bdist_wheel {{ bdist_wheel_xargs }}" ? I would expect those to be two different execution contexts | 15:09 |
| clarkb | since --user is basically a virtualenv isn't it? | 15:09 |
| clarkb | fungi: ^ you probably know | 15:09 |
| corvus | i think that's why mordred was thinking of using a venv for the whole role | 15:09 |
| zbr | clarkb: it is not | 15:09 |
| fungi | --user isn't quite a virtualenv | 15:10 |
| fungi | it's more like an additional tree added to the import path | 15:11 |
| fungi | so it acts sort of like a virtualenv with --system-site-packages but one which you don't need to explicitly add/activate... but it's not that similar really | 15:12 |
| clarkb | huh TIL | 15:12 |
| clarkb | I guess the only major issue then is the one avass points out? | 15:12 |
| clarkb | then we could switch to doing it all in a virtualenv as a followon? | 15:12 |
| *** harrymichal has joined #zuul | 15:12 | |
| fungi | you would at least have to make sure the same python interpreter which was tied to the pip which installed wheel was also the one used to invoke something importing it | 15:14 |
| avass | clarkb: yeah, the package should be available as long as it's run with the same user and interpreter. | 15:15 |
| fungi | oh, right, same user too obviously ;) | 15:15 |
| avass | and become: true installs it for root, so that wouldn't work :) | 15:15 |
| fungi | so say you have a job which needs to create separate wheels for different python versions, you'd need to separately install wheel for each of those | 15:16 |
| *** sshnaidm is now known as sshnaidm|bbl | 15:17 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 15:19 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Update guideline docs for os specific tasks https://review.opendev.org/731606 | 15:24 |
| avass | we should merge the policy update ^ | 15:25 |
| corvus | avass: done (that was a trivial change so i think we can consider AJaeger's vote sticky) | 15:26 |
| avass | :) | 15:26 |
| avass | corvus: also, if you have time you might want to take a look at: https://review.opendev.org/#/c/735402/ it adds tests for upload-docker-image, I'm thinking about extending that for buildx as well | 15:38 |
| avass | mordred: you might also be interested ^ | 15:38 |
| *** hashar has quit IRC | 15:38 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Update guideline docs for os specific tasks https://review.opendev.org/731606 | 15:40 |
| *** dennis_effa has joined #zuul | 15:53 | |
| zbr | avass: corvus mordred AJaeger: please look again at https://review.opendev.org/#/c/736185/10 -- now is green, so i will wait for last comments before updating commit message. | 16:06 |
| zbr | i think it would be a good idea to add a new param "sudoless" to the generic role tester playbook, to tell it to remove sudo before including the role. Very useful for testing that some roles work without sudo (and we do not add regressions accidentally) | 16:08 |
| zbr | but that would be another change | 16:08 |
| corvus | zbr: do we call an ensure-* role like that in any other situation? | 16:09 |
| avass | corvus: I believe we do, but cannot recall where | 16:09 |
| zbr | a grep returned lots of include_roles, and I was expecting it | 16:10 |
| zbr | very few roles are effectively standalone | 16:11 |
| corvus | looks like ensure-tox is one similar example | 16:11 |
| corvus | zbr: should we be changing the default python in this change without an announcement? | 16:12 |
| zbr | well... the change does not have a title yet, we just found one. | 16:12 |
| corvus | zbr: i mean that everything else in the change should not change any behavior (at least, for anyone other than opendev); but changing the default python version may be a behavior change for folks, so we may want to make that a standalone change and send an announcement to zuul-announce | 16:14 |
| zbr | but because we didn't have any tests we didn't know that this role was broken on any modern platform (afaik all use python3 exec) | 16:14 |
| zbr | the symlink for "python" being optional | 16:14 |
| corvus | i don't think we can assume that we have no users where python=python2 | 16:15 |
| *** rpittau is now known as rpittau|afk | 16:17 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Make build-python-release use python3 https://review.opendev.org/736185 | 16:19 |
| zbr | in fact python -> python2 for >95% of the cases, as PSF guidelines where very strict on this aspect and most distros respected that. | 16:21 |
| zbr | if someone wants to switch to defaulting to ansible interpreter, i am ok with that too. | 16:21 |
| zbr | in fact ansible interpreter python is the only python interpreter that we are sure to exist :D | 16:22 |
| mordred | zbr: actually, defaulting to ansible interpreter is very incorrect for us adn we should definitely not do that | 16:23 |
| corvus | zbr: sorry, my question about python3 is a -1 level thing; i didn't manage to post a review before you updated | 16:27 |
| corvus | afaik, no one is broken by the python/python3 thing, so let's make that a separate change | 16:27 |
| zbr | corvus: false: nobody knows is broken because we have no tests. let me prove it. | 16:28 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: POC: test build-python-release using python https://review.opendev.org/736293 | 16:29 |
| corvus | zbr: i think you misunderstood me. i'm saying that as far as i know, no production systems are broken because that defaults to 'python' right now. that means that what you're proposing could be a behavior change that could break some people, and we should announce it in advance. | 16:30 |
| corvus | zbr: i don't doubt that there are theoretical systems where it may not work as written. | 16:31 |
| mordred | yeah - and we should definitely fix that - same as the sphinx role | 16:31 |
| mordred | in fact - we could send that in the same announcement | 16:31 |
| corvus | ++ | 16:31 |
| corvus | that's https://review.opendev.org/735923 | 16:31 |
| * mordred hasnt' sent the sphinx announcmeent yet | 16:31 | |
| corvus | i -1d that for the same reason | 16:31 |
| zbr | i am not against any announcement, or waiting bit more before merging. | 16:32 |
| mordred | well - I think we need to merge the rest of your patch immediately :) | 16:32 |
| corvus | zbr: cool, then if you can split that part out, we can merge the more urgent fix soon | 16:32 |
| mordred | yeah | 16:32 |
| zbr | better to put a -W and a +2 on it, to make it clear you support the change but not merging it. | 16:32 |
| mordred | zbr: I don't support the patch as written - it needs to remove the python change. for that it's a -1. Once the python change is split into a follow up patch, it'll be +2 with a -W on the followup patch | 16:33 |
| mordred | zbr: that said - I have a question ... | 16:33 |
| mordred | you said to smcginnis that we expect ensure-pip to be a no-op in production, is that right in opendev? I've honestly lost track | 16:34 |
| zbr | i have no pressure to fix this, the only reason i raised this patch was because the earlier merge we reverted affected my "daily productivity" ;) | 16:35 |
| mordred | zbr: ok. would you mind if I take over the patch then? because I do have a pressure to fix this, as the job is broken in opendev | 16:35 |
| zbr | mordred: not at all, feel free to do anything with it | 16:35 |
| zbr | is already late here | 16:36 |
| mordred | zbr: cool - thanks for the help!! | 16:36 |
| tristanC | i thought the `ensure-*` roles were not meant to be used in `run` phase | 16:36 |
| zbr | in general i do not mind if others are taking my chages | 16:36 |
| *** ysandeep is now known as ysandeep|away | 16:36 | |
| mordred | tristanC: correct. I expect them to run in pre | 16:36 |
| corvus | tristanC: this is my understanding | 16:36 |
| zbr | fyi, that expectation are something fully undocumented, not clear for average user. if we really want to avoid confusions we may have to add pre_ and run_ prefixed to these roles. | 16:37 |
| zbr | (not supporting this myself) | 16:38 |
| corvus | build-python-release is defined in zuul-jobs, so if we want something run in a pre-playbook, we should add a pre-playbook to that job | 16:38 |
| tristanC | ok good to know, then this is a good candidate for a custome ansible-lint rule | 16:38 |
| corvus | so perhaps we should put ensure-pip and ensure-wheel in a pre-run playbook for that job | 16:39 |
| mordred | corvus: yeah | 16:40 |
| avass | we don't need an ensure-wheel since it doesn't need sudo | 16:40 |
| corvus | but there's some ambiguity here, in that the tox role includes ensure-pip | 16:41 |
| corvus | we should probably figure out if we want to handle missing things gracefully like that (and install them in the run phase) or just have the roles error and document that they have pre-requisites | 16:41 |
| corvus | i think the latter is what we have traditionally done | 16:41 |
| zbr | one more note before i go offline for the day: i prefer for the roles to be self-aware of their requirements, mainly to call roles that ensure their requirements instead of failing in confusing ways. this would make them much easier to use by other users, users that may not know (or have time/curiosity to learn) about the entire dependency tree. | 16:42 |
| corvus | avass: i don't think the ensure- roles are just for things that need sudo; they're for getting pre-requisites in place for other roles; they're separate so they can be in a pre-run playbook so they are subject to retries | 16:42 |
| tristanC | avass: do you think such `ZuulJobsPreEnsure` rule to prevent ensure-* role usage in non-pre run is possible to implement using an AnsibleLintRule object? | 16:42 |
| avass | tristanC: absolutely | 16:43 |
| corvus | tristanC: we can't know kind what playbook a role will be used in | 16:43 |
| corvus | tristanC: we can't know what kind of playbook a role will be used in | 16:43 |
| avass | corvus: right, since it's not part of the test and it fails it should try again somewhere else | 16:43 |
| avass | corvus: we can check if the role is included from another ensure-* role, but yeah I guess that's it | 16:43 |
| tristanC | corvus: arg good point, what if we only check direct `pre-run` inclusion? | 16:44 |
| corvus | zbr: that's reasonable; but if we choose not to do that, then we would document the pre-requisites in the role. so for example, in this case we would configure the job to have all of the necessary ensure- roles in a pre-playbook. but if a user chose to use the build-release role on its own, they would need to read the documentation to know it needs pip and wheel. | 16:44 |
| corvus | there is a good argument to be made for that approach because it means that we're doing more setup work in a pre-run playbook, and then we're not spending extra time re-running some of the same tasks in the run playbook. | 16:45 |
| corvus | tristanC: i think before we make a linter rule, we probably need to agree on an approach. | 16:46 |
| corvus | it seems we're of two minds about whether execution roles should perform setup tasks if things are missing | 16:46 |
| tristanC | corvus: some container user might have images with all the tools pre-installed (and validated), for example we have a python-f32 image with twine, wheel, setuptools, tox, ..., we would like to run just the `build wheel` and `publish to pypi` tasks, and simply skip all the ensure-* role | 16:47 |
| corvus | oh you know what, i think i'm somewhat wrong about that | 16:48 |
| corvus | tristanC: i agree you should be able to do that | 16:48 |
| corvus | now that i look more closely, it looks like while we do have some 'ensure' roles calling other ensure roles, the only execution role we have that calls an ensure role is 'js-package-manager' | 16:48 |
| corvus | so we are actually pretty close to being consistent about not having execution roles perform setup tasks | 16:49 |
| corvus | which means the most consistent way to implement this would be not to change the build-python-release *role* at all, but instead, write an 'ensure-wheel' role, have 'ensure-wheel' call 'ensure-pip', and then add a pre-run playbook to the build-python-release *job* that calls ensure-wheel | 16:50 |
| corvus | what do folks think about that ^? | 16:50 |
| avass | I can agree to that | 16:51 |
| tristanC | corvus: that sounds ideal | 16:51 |
| corvus | i think that gets us to our goal of "the job just works for everyone" and if you want to use the roles standalone, you can (you can make a custom version of the job without the pre-run playbook if you know your image has the pre-reqs) | 16:51 |
| tristanC | corvus: not sure how easy or interesting it would be to make `custom version of the job without the pre-run playbook` | 16:52 |
| corvus | tristanC: it's definitely easy; it's a 4 line job and a 4 line playbook. it's probably not interesting. :) | 16:54 |
| tristanC | i'd rather see a skip tag vars or something that could be used to disable the play that does setup action, but that's maybe premature optimisation | 16:54 |
| corvus | so ideally, the pre-run playbook should noop quickly. | 16:54 |
| corvus | tristanC: i think the big hit is going to be running the playbook at all | 16:55 |
| tristanC | corvus: yeah, that's what we observe most of the time | 16:55 |
| avass | maybe we should start using gather_facts: false whenever a playbook doesn't use facts? that's usualy a big speedup | 16:55 |
| mordred | corvus: yes. I just did that locally fwiw | 16:56 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Ensure wheel exists for build-release-python https://review.opendev.org/736185 | 16:56 |
| mordred | corvus: there is ensure-wheel in a pre playbook | 16:56 |
| avass | I don't know how common that is though | 16:56 |
| mordred | avass: we should have facts cached | 16:57 |
| mordred | by nodepool | 16:57 |
| avass | ah | 16:57 |
| corvus | mordred: by nodepool? | 16:58 |
| corvus | zuul caches facts in its setup playbook i think | 16:59 |
| corvus | oh, heh, i had an old copy of zuul-jobs and didn't realize we already just added a pre-playbook to that job :) | 17:01 |
| corvus | mordred: cool, i think that matches what we just talked about :) | 17:02 |
| mordred | corvus: it's a little ugly due to the variable passing | 17:03 |
| mordred | corvus: I kind of think it might be nice to introduce a general variable that can be used by different things to select the python they want to use | 17:03 |
| mordred | corvus: but I don't know that my brain can handle that atm | 17:04 |
| *** jpena is now known as jpena|off | 17:06 | |
| avass | mordred: you mean like how we use zuul_work_dir ? | 17:07 |
| corvus | mordred: yeah, we've done similar -- however, it was really useful to have "sphinx_python" be different.... so.... maybe have "sphinx_python" default to "cool_name_for_general_python" which defaults to "python3". same for wheel_python. | 17:07 |
| mordred | corvus: yeah | 17:08 |
| mordred | avass: and yeah | 17:09 |
| *** jcapitao has quit IRC | 17:11 | |
| *** sugaar1 has quit IRC | 17:29 | |
| *** paulalbertella has joined #zuul | 17:32 | |
| *** reiterative has quit IRC | 17:32 | |
| corvus | omg omg omg omg we have tags! https://hub.docker.com/r/zuul/zuul-web/tags | 18:19 |
| corvus | mordred, tobiash: ^ 3.x dockerhub tags for zuul are up | 18:20 |
| corvus | i will restore the release jobs now, and start work on nodepool | 18:20 |
| tobiash | awesome :) | 18:20 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Restore release jobs https://review.opendev.org/736327 | 18:21 |
| corvus | mugsie, tristanC, tobiash: azure merged! awesome! thanks :) | 18:23 |
| avass | nice! :) | 18:23 |
| mordred | corvus: \o/ | 18:25 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 18:28 |
| *** hashar has joined #zuul | 18:29 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 18:33 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Run upload-docker-image on release https://review.opendev.org/736333 | 18:36 |
| corvus | mordred, tobiash: ^ that should be the next step | 18:36 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 18:37 |
| mordred | corvus, fungi: ^^ that should be good now | 18:37 |
| mordred | avass: we had an idea related to an issue opendev was having with some jobs ^^ | 18:37 |
| mordred | corvus: I feel like your release tag list thing could eventually be a base feature of the image jobs | 18:39 |
| corvus | mordred: yeah, i'm mulling over how universally applicable it is | 18:39 |
| mordred | I think it's a reasonably common pattern people do | 18:40 |
| avass | mordred: lgmt | 18:44 |
| avass | I guess the problem is if tox is for some reason already present at /usr/bin/tox but there's another tox detected somewhere else earlier | 18:45 |
| fungi | yeah, ensure-tox short-circuits if there's already a tox installed that it can find, right? | 18:45 |
| avass | I mean if for some reason there's a tox in bot /bin/tox and /usr/bin/tox and ensure-tox detects /bin/tox then it will remove the /usr/bin/tox and replace it with a symlink to /bin/tox | 18:46 |
| avass | or something like that :) | 18:47 |
| *** jamesmcarthur has joined #zuul | 18:47 | |
| fungi | avass: maybe it needs to add "when: tox_preinstalled.rc != 0" then? | 18:47 |
| fungi | or move into the "Install tox to local env" block? | 18:48 |
| mordred | well - I mean it'll just symlink it to /usr/local | 18:48 |
| mordred | so the result of having /usr/bin/tox is that it would make a symlink from /usr/local/bin/tox to /usr/bin/tox | 18:49 |
| mordred | but in that case the user probably didn't need to set the variable, yeah? | 18:49 |
| fungi | yeah, that's true | 18:49 |
| fungi | probably results in no actual difference in behavior for the user either way | 18:50 |
| avass | mordred: sorry, I meant /usr/local/bin/tox | 18:50 |
| avass | so if there's a tox in something like /usr/bin/tox and /usr/local/bin/tox then it will probably fail because /usr/local/bin/tox is probably not writeable by the user | 18:51 |
| mordred | avass: oh - thanks. sorry - this is missing a become | 18:52 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 18:52 |
| mordred | (this is really only useful in cases where the job _does_ have root access available so the work to avoid doing things as root is making things harder for their content) | 18:53 |
| fungi | mordred: i guess we should call that out in the readme too then? | 18:54 |
| fungi | like noting that option requires sudo access or something | 18:54 |
| corvus | ++ | 18:54 |
| *** jamesmcarthur has quit IRC | 18:54 | |
| fungi | otherwise it might be non-obvious and result in confusing errors | 18:55 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 18:55 |
| mordred | ++ | 18:55 |
| mordred | done | 18:55 |
| *** jamesmcarthur has joined #zuul | 18:55 | |
| avass | well, it will still remove a tox executable, but I don't know if those could point to different tox versions or if it would ever matter | 18:57 |
| mordred | avass: yeah - I think the user shouldn't set the flag then | 18:58 |
| fungi | avass: what will it remove? there's a check in there to not overwrite (when: ... tox_executable != '/usr/local/bin/tox') | 18:59 |
| *** jamesmcarthur_ has joined #zuul | 18:59 | |
| avass | fungi: the edge case is if it detects tox_executable == /usr/bin/tox (or somewhere else earlier in PATH) and there's also a /usr/local/bin/tox | 19:00 |
| *** jamesmcarthur has quit IRC | 19:01 | |
| fungi | avass: oh! right, got iy | 19:01 |
| *** rlandy is now known as rlandy|brb | 19:01 | |
| fungi | doesn't check whether /usr/local/bin/tox already exists, only that /usr/local/bin/tox isn't the first one it found | 19:02 |
| avass | yeah | 19:02 |
| fungi | though i suppose if its search algorithm looked for /usr/local/bin/tox first that would never happen | 19:02 |
| *** sshnaidm|bbl is now known as sshnaidm | 19:09 | |
| avass | I wonder if we could add something like /zuul/bin that's always added to PATH by the forked command and shell modules | 19:09 |
| avass | mordred: It's probably fine, but -1 for docs :) | 19:09 |
| mordred | wow. also - that failed some tests pretty spectacularly | 19:13 |
| mordred | fungi: I feel like you might have this paged in - can you suggest what's wrong here: https://zuul.opendev.org/t/zuul/build/9df0b90f523244e1842fb02245a30a8d ? | 19:14 |
| fungi | looking | 19:17 |
| fungi | wow, i bet this is the first time we've touched that role/triggered that job since the image changes | 19:17 |
| fungi | it assumes pip will be present so it can use it to uninstall things pip installed | 19:18 |
| avass | mordred: that looks unrelated | 19:18 |
| fungi | i wonder if we could skip all the pip uninstall calls if pip dne | 19:18 |
| fungi | like is it sane to assume there are no pip-installed packages in the system context if there's no pip in the execution path? | 19:19 |
| fungi | oh ,though some of those may be nonfatal | 19:19 |
| fungi | the failure to import setuptools/pkg_resources is likely also related to not preinstalling | 19:20 |
| fungi | ianw did a workaround yesterday for ubuntu xenial. i think debian stretch needs it too | 19:21 |
| fungi | they are of a similar vintage after all | 19:21 |
| * fungi finds | 19:22 | |
| mordred | fungi: also centos 7 is having a sed too | 19:22 |
| fungi | it whould have an awk instead | 19:22 |
| mordred | hah | 19:22 |
| mordred | s/sed/sad/ | 19:22 |
| fungi | neither of us can type today | 19:22 |
| mordred | fungi: centos-7 same thing - pkg_resources missing | 19:22 |
| clarkb | I think the fix for that is ensure-pio? | 19:23 |
| clarkb | at least for ubuntu it was and I think ianw was looking at centos7 yesterday | 19:23 |
| fungi | oh, maybe | 19:23 |
| fungi | the fix i was misremembering is https://review.opendev.org/724788 | 19:23 |
| fungi | which uses a backported pip on xenial from a ppa, so not generally extended to other distros without a lot of work | 19:24 |
| mordred | clarkb: this role uses ensire-pip | 19:24 |
| clarkb | mordred: ya I think there is a bug ianw was fixing | 19:24 |
| mordred | all of his patches seem to have landed for zuul-jobs | 19:25 |
| clarkb | hrm maybe he didnt get to it then | 19:25 |
| mordred | maybe if I say his name three times he'll appear ... ianw ianw ianw | 19:25 |
| clarkb | he definitely mentioned the centos7 python2 case had a problem | 19:25 |
| clarkb | installing virtualenv under python3 might fix it there | 19:26 |
| *** rlandy|brb is now known as rlandy | 19:26 | |
| fungi | oh, right, it's the related https://review.opendev.org/736067 i was thinking of | 19:26 |
| mordred | well - in this case I don't know that we're installing virtualenv at all | 19:28 |
| mordred | but maybe that's the issue | 19:28 |
| mordred | maybe we need to have ensure-tox call ensure-virtualenv | 19:28 |
| mordred | ensure-tox is calling ensure-pip already ... but I think then it's installing tox into a puython -m venv | 19:29 |
| mordred | oh - except it can't | 19:29 |
| clarkb | I would expect that to work as its the newer python3 toolchain | 19:29 |
| mordred | because ensure-virtualenv installs virtualenv for package | 19:29 |
| mordred | and ensure-tox expects to be able to work without root | 19:30 |
| clarkb | ah | 19:30 |
| mordred | but - let's ignore that ... I'm guessing something is wrong on stretch where something is missing ... oh wiait - isn't it that stretch needs python3-pkg-resources for pip to work in python3 -m venv? | 19:30 |
| clarkb | fungi: ^ | 19:31 |
| * mordred is checking in a container | 19:31 | |
| mordred | works for me in a stretch container :( | 19:32 |
| corvus | is it too late for me to chime in with an "awkward" joke about sed? | 19:32 |
| mordred | corvus: hah | 19:32 |
| mnaser | lets just create our own base testing os :) | 19:33 |
| mnaser | zuulos | 19:33 |
| fungi | corvus: it's never too late, that joke is a true perl | 19:33 |
| mordred | mnaser: don't tempt me | 19:33 |
| mnaser | because if people werent confused enough about zuul and openstack | 19:33 |
| mnaser | zuulos will really drive that confusion | 19:33 |
| mordred | mnaser: maybe oszuul instead | 19:34 |
| fungi | it's a ci system, no it's a linux distro... stop--you're both right! | 19:34 |
| mordred | but seriously ... WHY IS THIS TEST FAILING *headdesk* | 19:34 |
| clarkb | time to hold a node? | 19:34 |
| corvus | mordred: which change are you looking at? | 19:34 |
| corvus | you're looking at 330, right? | 19:35 |
| mordred | Failed to import the required Python library (setuptools) on debian-stretch-rax-ord-0017203635's Python /usr/bin/python. | 19:35 |
| mordred | corvus: yes | 19:36 |
| mordred | corvus: the stretch job | 19:36 |
| corvus | https://review.opendev.org/736185 failed more spectacularly :( | 19:36 |
| mordred | that failure message ^^ | 19:36 |
| corvus | that's the build-release-python change | 19:36 |
| mordred | indicates that it's not using python3 | 19:36 |
| corvus | i'll take a look at 185 | 19:37 |
| fungi | okay, so it's trying to run something like `/usr/bin/python3 -m venv /home/zuul/.local/tox` which is what's raising the error about failing to import setuptools with /usr/bin/python (not /usr/bin/python3) | 19:37 |
| fungi | WEIRD | 19:37 |
| fungi | oh! specifically it's /tmp/ansible_pip_payload_PoyajU/ansible_pip_payload.zip/ansible/modules/packaging/language/pip.py which is trying to import pkg_resources | 19:38 |
| clarkb | ah its the ansible pip module | 19:38 |
| fungi | so that's the pip module in ansible | 19:39 |
| fungi | yep | 19:39 |
| mordred | I have a really good solution to that | 19:39 |
| fungi | it's a "pip" action in the task, according to the console | 19:39 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 19:40 |
| mordred | let's just remove the use of the pip module | 19:40 |
| mordred | no added value | 19:40 |
| mordred | corvus: I'm now joining yo on 185 | 19:41 |
| mordred | corvus: the ubuntu-bionic failure seems fun with unicode | 19:43 |
| avass | ansible installs it's own pip module?? | 19:43 |
| corvus | mordred: is it the case that the setup/distutils stuff we have just doesn't work under python2? so maybe our test jobs should just run under python3? | 19:43 |
| mordred | corvus: it would not surprise me at all | 19:43 |
| mordred | avass: well, it has a pip module for use in playbooks that tries to do pip things in python | 19:44 |
| corvus | mordred: i'm working on a patch | 19:44 |
| avass | mordred: sorry, I meant it installs it's onw pip when you use the pip module... strange | 19:44 |
| corvus | mordred: (that seems to jive what what zbr says, but it's possible someone is using this under python2 with some kind of local workaround, so i think we should stick with the announcement plan for changing the default) | 19:45 |
| mordred | avass: no, I don't think so - it's just expecting a global pip to work ... but if we don't hav eone | 19:45 |
| *** jamesmcarthur_ has quit IRC | 19:45 | |
| mordred | corvus: I agree - as long as we can fix our jobs to work :) | 19:45 |
| clarkb | mordred: avass ya its using global pip even if operating in a virtualenv | 19:45 |
| clarkb | but since we have a virtuaelnv and possibly no global pip we should explicitly operate in the venv instead | 19:46 |
| mordred | yeah | 19:46 |
| avass | mordred: oh yeah, I think I got confused since the pip modules name is literally 'pip.py' | 19:46 |
| mordred | avass: you and me both | 19:46 |
| *** jamesmcarthur has joined #zuul | 19:46 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Ensure wheel exists for build-release-python https://review.opendev.org/736185 | 19:46 |
| corvus | mordred: take a look at that interpatchset diff and let me know if that looks good | 19:46 |
| mordred | clarkb: do we need the chmod? I think that's a thing that the pip install should handle, no? | 19:46 |
| clarkb | mordred: we may for the dirs above it but I don't thnk its a regression hence my +2 | 19:47 |
| mordred | corvus: ++ | 19:47 |
| clarkb | mordred: basically if /home/zuul is 700 or /home/zuul/.local is 700 we'd break | 19:47 |
| clarkb | mordred: the executable itself should be fine | 19:47 |
| clarkb | but I think thats a followon cleanup since we were already doing the in venv install | 19:47 |
| mordred | clarkb: ++ | 19:47 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 19:47 |
| * mordred fix whitespace issue that the linter gods will be displeased with | 19:48 | |
| mordred | clarkb: mind re:+2ing | 19:48 |
| clarkb | doine | 19:48 |
| *** nils has quit IRC | 19:58 | |
| corvus | mordred: can you +2 https://review.opendev.org/736327 and tobiash can you +2 https://review.opendev.org/736333 ? | 19:59 |
| mordred | corvus: yes - but it's red | 19:59 |
| corvus | i can get those approved once the release jobs are fixed | 19:59 |
| clarkb | corvus: if you're just looking for extra reviews I'm happy to review too | 20:00 |
| corvus | yeah, we're not landing anything until https://review.opendev.org/736185 merges | 20:00 |
| corvus | clarkb: that'd be great thx | 20:00 |
| corvus | i figured if we can get the +2s lined up there, i can continue pushing that through | 20:01 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: Add ensure pre-run policy to ansible-lint https://review.opendev.org/736367 | 20:01 |
| mordred | corvus: also - if you're bored, https://review.opendev.org/#/c/735910 | 20:01 |
| corvus | mordred: cool +2; i didn't bother +3ing; we should wait until the release job is fixed | 20:02 |
| corvus | i'm going to afk this afternoon; i'll check back in during the evening | 20:02 |
| clarkb | corvus: done | 20:03 |
| mordred | clarkb: ++ | 20:03 |
| mordred | corvus: ++ | 20:03 |
| *** dennis_effa has quit IRC | 20:09 | |
| *** rlandy_ has joined #zuul | 20:09 | |
| *** decimuscorvinus_ has joined #zuul | 20:10 | |
| *** webknjaz_ has joined #zuul | 20:11 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: phoronix-test-suite: extract ensure- role from run playbook https://review.opendev.org/736368 | 20:11 |
| *** tobberydberg_ has joined #zuul | 20:12 | |
| *** tosky_ has joined #zuul | 20:12 | |
| *** tosky has quit IRC | 20:13 | |
| *** tosky_ is now known as tosky | 20:13 | |
| *** irclogbot_0 has quit IRC | 20:14 | |
| jkt | clarkb: this is on mnaser's Zuul instance where I don't have access to the logs. Anyway, I suspect something is fishy somewhere, for example I'm also not getting any jobs for an annotated tag pushed to refs/tags/, etc | 20:17 |
| clarkb | mordred: corvus (and all other zuulians) the community update on the 25th that is running on a more apac friendly schedule (7pm pacific I think thtas like 0200 UTC) is actually being done in english and we are being asked if we hvae any volunteers to do that one. I've told the foundation I can do it if we don't have non staff volunteers | 20:17 |
| *** tobberydberg has quit IRC | 20:18 | |
| *** rlandy has quit IRC | 20:18 | |
| *** decimuscorvinus has quit IRC | 20:18 | |
| *** webknjaz has quit IRC | 20:18 | |
| *** mgoddard has quit IRC | 20:18 | |
| *** mrcirca has quit IRC | 20:18 | |
| *** rlandy__ has joined #zuul | 20:18 | |
| *** webknjaz_ is now known as webknjaz | 20:18 | |
| fungi | jkt: the usual triggers for tags only fire on signed tags | 20:19 |
| *** rlandy_ has quit IRC | 20:19 | |
| avass | tristanC: fiy, I added tests to the linting rules in test-playbooks/ansible-lint-rules | 20:20 |
| fungi | jkt: might be how it's set up there too | 20:20 |
| *** mgoddard has joined #zuul | 20:21 | |
| mnaser | oh | 20:21 |
| mnaser | interesting | 20:21 |
| mnaser | AttributeError: 'Tag' object has no attribute 'number' | 20:21 |
| jkt | mnaser: we're using the "v" prefix | 20:22 |
| *** irclogbot_3 has joined #zuul | 20:22 | |
| jkt | fungi: that's... undocumented as far as I can tell | 20:22 |
| mnaser | https://www.irccloud.com/pastebin/YYfyoZQr/ | 20:22 |
| clarkb | jkt: fungi: it depends entirely on how the pipeline is configured | 20:23 |
| clarkb | mnaser: ya tags don't have a change number they are just refs | 20:23 |
| clarkb | mnaser: jkt can you share the pipeline config? | 20:23 |
| mnaser | i mean, i guess it still shouldn't throw a traceback but | 20:23 |
| mnaser | let me pull it out | 20:23 |
| clarkb | mnaser: well its probably misocnfigured | 20:23 |
| clarkb | my hunch is you're using change attributes on the pipeline config | 20:24 |
| clarkb | and so its a mismatch between expected input and asserted config | 20:24 |
| clarkb | (I remember this happening a bit way back in the gozer days) | 20:24 |
| jkt | it would be nice if Zuul made the tenant config available over its API | 20:24 |
| tristanC | avass: nice, i'll have a look. in the meantime there are some non pre- playbooks that include ensure-* roles we need to fix if we want the rule to be valid. | 20:24 |
| jkt | use case: a managed service where I don't have access to that | 20:25 |
| mnaser | https://www.irccloud.com/pastebin/9JRXPcrD/ | 20:25 |
| clarkb | jkt: I think the intent is that the configs should be public (secrets are encrypted), but I agree that could be helpful | 20:25 |
| jkt | mnaser: it's using "dependent" pipeline manager, is there a reason for that? | 20:25 |
| avass | tristanC: and you might want to do ZUUL0003, I'll try to finish https://review.opendev.org/#/c/727642/10 | 20:25 |
| clarkb | mnaser: the manager is the problem I think | 20:26 |
| clarkb | dependent implies merging because it wants to sequence things in sepculative fashion | 20:26 |
| jkt | makes sense | 20:26 |
| mnaser | probalby an oversight here. is there a way we can user-friendl-ify this inside zuul? | 20:27 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 20:27 |
| clarkb | mnaser: we could probably have some heuristics to emit warnings, but I don't think we can make it foolproof without making the gerrit (and github etc) input types way more strict in their semantics | 20:28 |
| mnaser | yeah given its event based it's probably hard to detect | 20:28 |
| clarkb | mnaser: ya its an external input and zuul does its best to accomodate them as they change over time | 20:29 |
| mnaser | jkt: you're good, changed the manager | 20:29 |
| *** jamesmcarthur has quit IRC | 20:31 | |
| *** jamesmcarthur has joined #zuul | 20:32 | |
| jkt | clarkb: can you imagine a use case where "dependent" is used along ref-updated gerrit trigger? | 20:34 |
| *** dcastellani has quit IRC | 20:35 | |
| clarkb | jkt: not with gerrit at least. | 20:36 |
| clarkb | but refs don't have to be tags | 20:36 |
| clarkb | so it could be a thing in some system | 20:37 |
| *** jamesmcarthur has quit IRC | 20:37 | |
| jkt | I think this should be an easy diagnostics -- apparently "dependent" likes changes in gerrit, PRs in github, etc | 20:37 |
| jkt | let me sort out our project's releae piepeline first and then propose a diagnostics for this one | 20:38 |
| clarkb | ya dependent wants pre merge input however that is expressed by the driver | 20:39 |
| *** rlandy__ is now known as rlandy | 20:39 | |
| *** iamweswilson has quit IRC | 20:40 | |
| *** iamweswilson has joined #zuul | 20:40 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 20:42 |
| mordred | clarkb, avass: doh. had to update the test playbook too | 20:42 |
| mordred | clarkb: didn't we purposefully set up a dependent post-merge pipeline in opendev? | 20:42 |
| mordred | before the serial manager existed? | 20:43 |
| *** webknjaz has quit IRC | 20:43 | |
| clarkb | mordred: supercedent? | 20:43 |
| mordred | clarkb: I think I might be hallucinating | 20:44 |
| *** iamweswilson has quit IRC | 20:44 | |
| *** webknjaz has joined #zuul | 20:46 | |
| *** iamweswilson has joined #zuul | 20:46 | |
| *** PrinzElvis has quit IRC | 20:49 | |
| *** webknjaz has quit IRC | 20:51 | |
| *** PrinzElvis has joined #zuul | 20:53 | |
| *** webknjaz has joined #zuul | 20:57 | |
| *** PrinzElvis has quit IRC | 20:57 | |
| *** iamweswilson has quit IRC | 21:01 | |
| *** jamesmcarthur has joined #zuul | 21:04 | |
| *** webknjaz has quit IRC | 21:06 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 21:07 |
| *** rfolco|rover has quit IRC | 21:12 | |
| *** noonedeadpunk has quit IRC | 21:12 | |
| *** jamesmcarthur has quit IRC | 21:13 | |
| jkt | mnaser: can you please check if there was something wrong with my recent push of v2.2a3? I've fixed some config errors on my side, these were reported as CONFIG_ERROR, that's OK, but I don't have any result for this one, and it also wasn't visible in the status UI | 21:14 |
| openstackgerrit | Ahmad Mahmoudi proposed zuul/zuul-jobs master: (fix) Changed pip to pip3 https://review.opendev.org/736379 | 21:16 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: Add ensure pre-run policy to ansible-lint https://review.opendev.org/736367 | 21:19 |
| mnaser | jkt: probably best to take them to a support ticket for now as i have to run out in a sec | 21:19 |
| jkt | mnaser: ok, will do. Thanks for your quick fix! | 21:21 |
| *** harrymichal has quit IRC | 21:28 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 21:31 |
| mordred | clarkb, fungi: sorry - syntax error | 21:31 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Fix ansible-lint rules tests https://review.opendev.org/736387 | 21:34 |
| avass | mordred: todays not your day :) | 21:34 |
| *** PrinzElvis has joined #zuul | 21:35 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 21:35 |
| mordred | avass: it's really not | 21:36 |
| mordred | avass: stupid indentation | 21:36 |
| fungi | "this end should point toward the ground if you want to go to space. if it starts pointing toward space you are having a bad problem and will not go to space today" | 21:36 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Fix ansible-lint rules tests https://review.opendev.org/736387 | 21:37 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 21:37 |
| *** dcastellani has joined #zuul | 21:37 | |
| avass | also, I noticed that the tests for the linting rules that was supposed to fail were failing but not the way we want to https://review.opendev.org/736387 | 21:38 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "gate your first patch" https://review.opendev.org/732067 | 21:39 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "Use zuul jobs" https://review.opendev.org/732068 | 21:39 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "gate pipeline" https://review.opendev.org/732069 | 21:39 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "job secrets" https://review.opendev.org/732070 | 21:39 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "job dependencies" https://review.opendev.org/732071 | 21:39 |
| *** iamweswilson has joined #zuul | 21:40 | |
| *** webknjaz has joined #zuul | 21:40 | |
| mordred | fungi: ++ | 21:50 |
| *** jamesmcarthur has joined #zuul | 21:50 | |
| *** hashar has quit IRC | 21:52 | |
| mordred | fungi, clarkb, avass, corvus: https://review.opendev.org/#/c/736185/ is still failing. investigating | 21:52 |
| *** sanjayu_ has quit IRC | 21:52 | |
| mordred | fungi: I think perhaps when we install wheel, we should also install setuptools | 21:53 |
| mordred | fungi: https://zuul.opendev.org/t/zuul/build/20152d87d3b342cbb738f98acf7eea45/log/job-output.txt#296 | 21:53 |
| mordred | oh - well - I mean | 21:54 |
| mordred | sigh | 21:54 |
| *** jamesmcarthur has quit IRC | 21:54 | |
| mordred | we need to run the setup role in a pre playbook if the role test is supposed to work | 21:55 |
| avass | :) | 21:55 |
| avass | mordred: also, shouldn't ensure-wheel default to 'python' and not python3 if that's the way build-python-release works? | 21:55 |
| *** tosky has quit IRC | 21:56 | |
| *** tosky has joined #zuul | 21:56 | |
| mordred | avass: probably so, yes | 21:57 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Fix ansible-lint rules tests https://review.opendev.org/736387 | 21:58 |
| mordred | avass: oh - no, because in the playbook when we're using it for build-python-release we set the value to default to release_python | 21:58 |
| mordred | so that it'll behave like build-python-release ... but so that we can also add the new role correctly defaulting to python3 | 21:59 |
| *** rlandy is now known as rlandy|bbl | 21:59 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Ensure wheel exists for build-release-python https://review.opendev.org/736185 | 21:59 |
| mordred | I think this should fix it | 21:59 |
| avass | mordred: oh... yeah that's slightly confusing :) | 22:00 |
| mordred | avass: yeah ... it's not awesome. hopefully we'll get that default changed soon | 22:00 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 22:05 |
| mordred | woot | 22:11 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Fix ansible-lint rules tests https://review.opendev.org/736387 | 22:13 |
| *** armstrongs has joined #zuul | 22:21 | |
| mordred | avass, clarkb, fungi, corvus: https://review.opendev.org/#/c/736185/ is green! please to approving | 22:24 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 22:29 |
| *** armstrongs has quit IRC | 22:30 | |
| avass | mordred: lgtm, but it's getting late so I'll let someone else approve it in case I missed something obvious | 22:30 |
| mordred | avass: tyty | 22:31 |
| ianw | mordred: sorry, i got a bit lost in scrollback, is there something i need to look at wrt ensure-pip etc madness? | 22:31 |
| mordred | ianw: I think it's under control - but ... | 22:32 |
| mordred | ianw: if you get a minute - it would be good to have you look at a few just to make sure we didn't miss something obvious | 22:32 |
| ianw | ok i have to do school run and will pull everything up in about 15 mins | 22:32 |
| mordred | ianw: https://review.opendev.org/736185 https://review.opendev.org/736330 | 22:33 |
| mordred | ianw: those are the two biggies | 22:33 |
| mordred | ianw: https://review.opendev.org/#/c/736334 is the motivation for 736330 - there were release jobs that needed tox inside of scripts | 22:33 |
| mordred | anyway - I think that should all be in good shape | 22:34 |
| fungi | scripts which were all like, "hey i'll just run tox... why isn't there a tox in my path?" | 22:34 |
| fungi | surprising how entirely not simple "i just want to run tox" turns out to be sometimes ;) | 22:35 |
| *** _erlon_ has joined #zuul | 22:40 | |
| avass | can we merge https://review.opendev.org/#/c/736387/ ? the tests for the custom ansible-lint rules has been failing since we upgraded ansible-lint | 22:45 |
| clarkb | avass: looking | 22:46 |
| clarkb | lgtm | 22:46 |
| openstackgerrit | Merged zuul/zuul-jobs master: Ensure wheel exists for build-release-python https://review.opendev.org/736185 | 22:48 |
| *** tosky has quit IRC | 23:09 | |
| ianw | mordred: why did we choose not to install the packages for wheel? | 23:16 |
| ianw | tbh i would have considered it a pretty reasonable thing to just add to the package list for ensure-pip and skip separate roles | 23:17 |
| fungi | the ensure-wheel role seems like it wants to possibly install without root privs, not sure if that had anything to do with it | 23:23 |
| fungi | granted the original https://review.opendev.org/736001 also used pip, but the reason it got reverted is that it needed root access | 23:25 |
| fungi | maybe putting the distro package addition into a pre play would have sufficed though | 23:26 |
| ianw | i thought we decided that "ensure" roles could install stuff with root as required, but should not need sudo on the no-op path | 23:42 |
| ianw | we have a mix of things installing packages, installing into user areas, installing into user areas and creating a /usr/local/bin/ symlink as root | 23:44 |
| fungi | it's a good question. as things stabilize we can rework stuff for better consistency | 23:47 |
| fungi | we're also finding that some of our actual roles relied on ansible's pip module, which doesn't work if setuptools (or at least pkg_resources) isn't already installed | 23:49 |
| fungi | s/actual/existing/ | 23:49 |
| clarkb | *isn't already installed in the global pip | 23:49 |
| clarkb | it was there in the virtualenv we were trying to operate in | 23:49 |
| clarkb | but it uses global pip to operate in virtuaelnvs | 23:49 |
| ianw | right, that's why "ensure-pip" says that it installs everything required to run the ansible "pip:" module ... | 23:50 |
| clarkb | ianw: I think that didn't happen in this case | 23:50 |
| clarkb | but I could be wrong about that | 23:50 |
| ianw | clarkb: right, well i'd consider it a bug ... "This role is intended install the requirements for the pip module on hosts." is the first line of the README :) | 23:51 |
| clarkb | ianw: 19:29:16* mordred | ensure-tox is calling ensure-pip already ... but I think then it's installing tox into a puython -m venv | 23:51 |
| clarkb | ya could be the more proper fix was to fix ensure-pip if it indeed failed to make pip module function | 23:52 |
| ianw | ensure-tox was another one of these that was installing --user | 23:52 |
| ianw | that breaks in situations such as system-config running testinfra where we want tox as root | 23:52 |
| ianw | i can see an argument that there's much less likelyhood of needing to run "wheel" in a root/non-zuul user context | 23:53 |
| fungi | well, except insofar as pip installing an sdist these days uses the wheel module to first build a wheel from the sdist and then install the resulting wheel | 23:54 |
| ianw | ... why i think that that probably adding wheel to "ensure-pip" isn't overreach of the role, they're pretty closely linked | 23:55 |
| fungi | yeah | 23:56 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!