Thursday, 2020-05-14

*** rfolco|rover has joined #zuul00:20
ianwnup ... i dunno00:20
*** rfolco|rover has quit IRC00:25
*** ysandeep|away is now known as ysandeep00:28
jktyay, a first cross-project build has just passed00:41
jktnow let's get some caching into the mix so that it doesn't rebuild that full embedded linux image, including the kernel00:41
*** Goneri has quit IRC01:11
*** swest has quit IRC01:45
*** jpena|off has quit IRC01:52
*** swest has joined #zuul02:01
*** bhavikdbavishi has joined #zuul02:14
*** bhavikdbavishi1 has joined #zuul02:45
*** bhavikdbavishi has quit IRC02:46
*** bhavikdbavishi1 is now known as bhavikdbavishi02:46
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Patch CoreDNS corefile  https://review.opendev.org/72786803:23
*** zxiiro has quit IRC03:57
*** evrardjp has quit IRC04:36
*** evrardjp has joined #zuul04:36
*** felixedel has joined #zuul05:10
*** sgw has quit IRC05:18
*** bhavikdbavishi has quit IRC05:20
*** reiterative has quit IRC05:37
*** bhavikdbavishi has joined #zuul05:37
*** reiterative has joined #zuul05:37
openstackgerritFelix Edel proposed zuul/zuul master: Report dequeued changes via Github checks API  https://review.opendev.org/71102305:42
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add container and pod log in the test for ensure-kubernetes role  https://review.opendev.org/72792905:43
*** dpawlik has joined #zuul06:06
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Warn user when dynamic layout ignores zuul config  https://review.opendev.org/72024906:13
openstackgerritTobias Henkel proposed zuul/zuul master: Warn user when dynamic layout ignores zuul config  https://review.opendev.org/72024906:30
openstackgerritTobias Henkel proposed zuul/zuul master: Drop support for ansible 2.7  https://review.opendev.org/72737306:33
openstackgerritTobias Henkel proposed zuul/zuul master: Update images to use python 3.8  https://review.opendev.org/72737406:34
openstackgerritTobias Henkel proposed zuul/zuul master: Drop support for ansible 2.6  https://review.opendev.org/72715706:42
openstackgerritTobias Henkel proposed zuul/zuul master: Drop support for ansible 2.7  https://review.opendev.org/72737306:42
openstackgerritTobias Henkel proposed zuul/zuul master: Update images to use python 3.8  https://review.opendev.org/72737406:42
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Patch CoreDNS corefile  https://review.opendev.org/72786806:55
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add container and pod log in the test for ensure-kubernetes role  https://review.opendev.org/72792906:55
*** jcapitao has joined #zuul07:10
*** bhavikdbavishi has quit IRC07:14
*** guillaumec has joined #zuul07:31
*** tosky has joined #zuul07:31
*** rpittau|afk is now known as rpittau07:33
*** harrymichal has joined #zuul07:39
*** fbo|off is now known as fbo|afk07:41
*** bhavikdbavishi has joined #zuul07:44
openstackgerritTobias Henkel proposed zuul/nodepool master: WIP: Azure api versions  https://review.opendev.org/72795807:50
*** jpena has joined #zuul07:57
*** guillaumec has quit IRC08:04
*** ysandeep is now known as ysandeep|lunch08:14
*** nils has joined #zuul08:17
*** harrymichal has quit IRC08:40
*** dpawlik has quit IRC08:40
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: yamlint: EOF newlines and comments indent  https://review.opendev.org/72551608:42
*** dpawlik has joined #zuul08:45
*** ysandeep|lunch is now known as ysandeep09:03
*** sshnaidm|afk is now known as sshnaidm09:11
openstackgerritMerged zuul/zuul-jobs master: yamlint: EOF newlines and comments indent  https://review.opendev.org/72551609:27
*** bhavikdbavishi has quit IRC09:31
*** bhavikdbavishi has joined #zuul09:37
*** felixedel has quit IRC09:42
*** harrymichal has joined #zuul10:05
*** rpittau is now known as rpittau|bbl10:18
*** guillaumec has joined #zuul10:19
*** root____4 has joined #zuul10:32
zbravass: does anyone remember the debate about wrapping console vs not-wrapping, here is another example: https://review.rdoproject.org/zuul/build/5f77d48e438a443a8946355c65db89b4/console10:34
zbravass: oops, was not for you in particular.10:34
zbrlack of wrapping makes impossible to read the error line10:35
root____4Hello, I would like to ask the maintenance staff of zuul? I encountered a failure when trying to install nodepool-launcher. The error is reported as follows, I tried to manually modify this part of the code, let him automatically create the corresponding path folder. But it has not been resolved. http://paste.openstack.org/show/793574/10:39
root____4This is the document I checked for installation. https://zuul-ci.org/docs/zuul/howtos/nodepool_install.html?highlight=nodepool10:39
*** root____4 is now known as ruffian_sheep10:40
ruffian_sheepDo any zuul maintainers or others know how to solve this problem? My nodepool service cannot start normally now. Even if I modified the code to let it create the folder itself.10:41
*** fbo|afk is now known as fbo10:43
ruffian_sheepclarkb | SpamapS | ianw | corvus | frickler | fungi | jlk | jhesketh | mordred | pabelanger | rcarrillocruz | tobiash | tristanC : are u guys online?10:44
jheskethruffian_sheep: have you checked the permissions of the /var/run/nodepool folder?\10:47
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-tox-output: use tox_extra_args  https://review.opendev.org/72802310:50
avasszbr: I don't think i had a strong enough opinion in that topic, but I think the conclusion was that 'it depends' and I think I agree with that, so I'm not against be able to wrap the output but maybe making it configurable?10:51
avasszbr: though, that output looks more like a problem with ANSI characters than not wrapping the line doesn't it?10:52
avasszbr: and hmm, not line breaking for \n10:52
ruffian_sheepjhesketh :If I creat it, it will be d-wxr ---- t, if I do n’t make changes in code. This folder will not be generated automatically, and it will show as I pasted it, there is no corresponding folder10:52
avasszbr: I doubt that would be much easier to read even if it wrapped10:53
jheskethruffian_sheep: you shouldn't need to modify the code10:53
jheskethruffian_sheep: have you checked the ownership of the folder and made sure you are running nodepool with the right user who has write access to it?10:53
avassjhesketh, ruffian_sheep: I think that might be it10:54
ruffian_sheepAccording to the documentation, I don't need to switch users to nodepool, and the folder / var / run / nodepool does not exist at the beginning. And it won't work until I finish the operation. Is there anything I haven't finished yet? In addition to the operations shown in this link. Can't I start the service yet?10:57
avassruffian_sheep: are you running it with systemd?11:00
jhesketh(was about to ask the same thing)11:03
ruffian_sheepavass | jhesketh :After I failed to use systemctl start nodepool-launcher, I directly executed the corresponding command: / usr / local / bin / nodepool-launcher found that there is a situation where the folder will not be automatically created.11:03
fricklerruffian_sheep: the docs need to be updated, see this code for how we do it in our CI https://opendev.org/zuul/nodepool/src/branch/master/playbooks/nodepool-functional-openstack/run.yaml#L105-L11411:04
*** jcapitao is now known as jcapitao_lunch11:04
avassruffian_sheep, jhesketh: isn't systemd supposed to solve that for you?11:05
avassfrickler: ^11:06
avassfrickler: looking at our build scripts we don't create those11:06
avassfrickler: but we also run it with systemd :)11:06
avassfrickler: and you don't so that's probably why11:06
ruffian_sheepavass : I don't quite understand what you mean. Can you explain it?11:09
ruffian_sheepfrickler : I should change the code?11:09
avassruffian_sheep: I believe systemd creates /var/run/nodepool for you. So you shouldn't get that error if you run 'sudo systemctl start nodepool-launcher'11:10
avassruffian_sheep: if you installed the service files11:10
avassruffian_sheep: do you get the same error if you use systemd?11:11
avassruffian_sheep: do you get an error if you run 'nodepool-launcher -d'? that shouldn't require a pidfile so maybe you're getting a different error as well11:12
frickleravass: I was referring to the deployment example in the docs, which doesn't use systemd11:12
avassfrickler: which ones? :)11:13
ruffian_sheepI thinkhttp://paste.openstack.org/show/793577/11:13
avassfrickler: maybe this needs to be updated with an example how to do it without systemd: https://zuul-ci.org/docs/zuul/howtos/nodepool_install.html?highlight=nodepool#service-file11:14
ruffian_sheepavass : yes, Really different11:14
avassruffian_sheep: ah yeah, so systemctl start nodepool is probably crashing with that error11:14
tobiashruffian_sheep: your last paste shows that you have an issue with the nodepool config (the provider references the clodu devstack-admin which probably has no entry in the clouds.yaml)11:15
ruffian_sheepavass : My environment does not seem to have the systemd command. It seems that the service can only be started by sudo systemctl start nodepool-launcher11:15
avassruffian_sheep: systemctl is part of systemd :)11:16
ruffian_sheeptobiash :I think the last pasted link caused an error due to a problem with this part of my configuration. cloud: devstack-admin # This needs to match the name in clouds.yaml. Does this directly affect the service startup?11:19
frickleravass: oh, indeed, I read this wrongly, sorry. but an example to run manually for debugging purposes might be helpful, yes11:20
ruffian_sheepavass : may. . . I still don't quite understand the meaning of using systemd operation.11:20
avassruffian_sheep: yes11:21
zbrhow can I avoid getting a POST_FAILURE with zuul when normal run failed? post runs and is expected to fail in this case11:21
avassWe had a simlar problem this week with a mispelled profile name for AWS, and that caused nodepool-launcher to not start11:21
avasszbr: check 'zuul_success' if it succeeded :)11:21
zbrmainly I want to do something like "ignore_errors: if run is failed" kind of behavior.11:21
zbrlike "ignore_errros: zuul_success is failed" ?11:22
avassI've been thinking about pushing a fix to ignore the erroring pool but ahven't gotten around to it yet11:22
avasszbr: something like 'when: zuul_success|bool|default(True)'11:22
avasszbr: would be better wouldn't it?11:23
zbrnot really because I still want to run the post, as it may be able to collect some partial output.11:23
avasszbr: https://zuul-ci.org/docs/zuul/reference/jobs.html?highlight=zuul_success#var-zuul_success11:23
avasszbr: hmm, can you check if the output exists instead then?11:23
zbri only want to ignore errors from POST when build already failed.11:23
avasszbr: because ignore_errors can lead to other problems if it fails when it shouldn't11:24
avasszbr: but yes you can use ignore_errors11:24
zbrnot really, because i want to post to fail if run finished succesfully without producting artifacts I expect11:24
zbris a conditional ignore errors11:24
zbrthanks, going to try it now.11:25
avasszbr: but it should probably be 'ignore_errors: "not {{ zuul_succes|bool }}' :)11:26
*** ruffian_sheep has quit IRC11:27
*** bhavikdbavishi has quit IRC11:37
jktI wonder why I have to explicitly add other-project into my job's required-projects when that job is only defined in that other-project11:47
jkt(and of course it works without required-projects in case this change Depends-on a change from other-project, that one I understand)11:48
*** lseki has quit IRC11:51
avassjkt: because a job can be defined in any project. that's why we have zuul-jobs for example. :)11:51
*** rlandy has joined #zuul11:53
jktavass: so you're essentially saying that it doesn't matter in which project a job is defined. OK, this makes sense, even if I find it a bit surprising11:53
jktI somehow thought that these jobs "linked from config projects" were special11:54
jktnow that explains why I cannot define two jobs with the same name in two "leaf" projects :)11:54
AJaegerjkt: we have a global namespace for jobs etc. So, everything is shared. That why we have a naming policy to name normal jobs $project-X instead of X - to avoid name collisions11:55
openstackgerritMerged zuul/zuul-jobs master: Combine javascript deployment and deployment-tarball jobs  https://review.opendev.org/72737012:02
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: fetch-sphinx-tarball: introduce zuul_use_fetch_output  https://review.opendev.org/68187012:03
*** jpena is now known as jpena|lunch12:07
zbravass: not sure why but apparently ignore errors in post does not seem to work, see https://review.rdoproject.org/r/#/c/27547/12:15
zbri am going to try with true, just too see if condition is wrong12:15
*** rpittau|bbl is now known as rpittau12:15
*** jcapitao_lunch is now known as jcapitao12:17
tobiashzbr: I didn't know that ignore errors on play level is a thing12:18
tobiashso far I've only seen that on task level12:20
*** rfolco|rover has joined #zuul12:20
openstackgerritMatthieu Huin proposed zuul/zuul master: CLI: Fix errors with the REST client  https://review.opendev.org/72806112:21
*** saneax has joined #zuul12:21
*** saneax has quit IRC12:22
*** saneax has joined #zuul12:23
*** sanjayu_ has quit IRC12:23
*** Goneri has joined #zuul12:28
*** sanjayu_ has joined #zuul12:30
*** saneax has quit IRC12:31
*** saneax has joined #zuul12:32
*** bhavikdbavishi has joined #zuul12:32
*** sanjayu_ has quit IRC12:35
*** saneax has quit IRC12:35
*** saneax has joined #zuul12:35
zbrtobiash: yes it is, and is very useful in practice for special kind of playbooks.12:37
zbrbad, part, it seems not to work with zuul_return, maybe due to the way the variable is defined, maybe is host fact?12:38
avasszbr: maybe it's a ansible version thing?12:38
zbri doubt12:39
*** lseki has joined #zuul12:41
avasszbr: zuul_success is an extra_var12:41
*** saneax has quit IRC12:41
avasszbr: so that should work there shouldn't it ?12:41
zbri am doing some more testing, hopefully i will narrow it down.12:42
avasszbr: oh, I don't think ignore_errors acts like when, so maybe you're just giving it a string at the moment?12:43
avasszbr: so you probably need to template the entire expression12:43
avasszbr: Yeah I think that's why, tested it locally12:45
zbravass: right! found it few seconds ago.12:45
zbrand funny, now I remember this is not the first time hit this.12:45
zbrmainly ansible evaluates it as a string instead of having a behavior like when12:46
openstackgerritMatthieu Huin proposed zuul/zuul master: REST API: fix discrepancies between RPC and REST outputs for autohold  https://review.opendev.org/72807312:54
*** saneax has joined #zuul13:00
*** jpena|lunch is now known as jpena13:05
*** saneax has quit IRC13:07
*** bhavikdbavishi has quit IRC13:19
*** bhavikdbavishi has joined #zuul13:19
*** harrymichal has quit IRC13:20
*** tumble has joined #zuul13:26
*** bhavikdbavishi1 has joined #zuul13:29
openstackgerritMerged x/pbrx master: Retire repository  https://review.opendev.org/72646213:29
*** bhavikdbavishi has quit IRC13:30
*** bhavikdbavishi1 is now known as bhavikdbavishi13:30
*** zxiiro has joined #zuul13:31
AJaegerthanks, mordred. please abandon: https://review.opendev.org/#/q/project:x/pbrx+is:open13:32
mordredAJaeger: done!13:35
mordredAJaeger: also - I don't see what failed in https://review.opendev.org/#/c/717371/ - do you?13:35
AJaegerthanks, mordred13:39
openstackgerritMonty Taylor proposed zuul/zuul-website master: Add blog to website  https://review.opendev.org/72464813:40
AJaegerhttps://zuul.opendev.org/t/zuul/build/35a94ce1bd8447dea37c5431377e1425/log/job-output.txt#566 is the failure13:41
AJaegerSo, what's wrong in copy-media.yaml ?13:41
mordredoh! weird13:45
mordredlooking13:45
mordredAJaeger: tasks is a list13:47
mordrednot a dict :)13:47
mordredAJaeger: although - that said - I think there is a flaw in this13:47
mordredI'll push up a fix for this but then we can think about making it better for zuul-website (which is doing something a little weird)13:48
AJaegerah13:48
openstackgerritMonty Taylor proposed zuul/zuul-website master: Switch website to Gatsby  https://review.opendev.org/71737113:48
mordredAJaeger: oh - nevrmind, the thing I thought was broken is actually good13:49
AJaegerNow I see the problem...13:49
mordredthe "build javascript tarball" role is a post playbook int he base job, so it'll run _after_ we copy the media in place, so the tarball should be correct13:49
AJaegerah13:50
mordredso - I think we still may need to make a publish job for this right?13:51
mordredalthough it was working before13:51
mordredAJaeger: it runs opendev-promote-javascript-content - so I think that should work with this having uploaded a tarball right?13:52
mordredAJaeger: oh - I need to update the download_artifact_job var13:55
AJaegercorrect, you need to update that one13:55
mordredAJaeger: nod. and actually- that job isn't the right job for this - that's about the tarballl13:57
*** bhavikdbavishi has quit IRC13:57
mordredAJaeger: I think what I need is to update the job similar to what it was before, but with the "fetch tarball and untar it" step13:57
mordredwhich I think might be another base-jobs job13:58
* mordred works13:58
*** sgw has joined #zuul13:58
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-tox-output: use tox_extra_args  https://review.opendev.org/72802314:15
openstackgerritMonty Taylor proposed zuul/zuul-website master: Switch website to Gatsby  https://review.opendev.org/71737114:16
openstackgerritMatthieu Huin proposed zuul/zuul master: Add simple testing for Zuul CLI & REST API  https://review.opendev.org/72809814:17
fungittx pointed me to this new leboncoin blog post with significant zuul mention, it's a good read: https://medium.com/leboncoin-engineering-blog/leboncoin-commits-life-960a86cd35ff14:23
corvusthat's pretty slick! :)14:27
corvus"We are currently working on automated tests allowing automatic production deployment."  i like that :)14:29
avassfungi: oh, cool14:33
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-tox-output: use tox_extra_args  https://review.opendev.org/72802314:37
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-tox-output: use tox_extra_args  https://review.opendev.org/72802314:38
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-tox-output: use tox_extra_args  https://review.opendev.org/72802314:54
*** evrardjp has quit IRC14:55
*** evrardjp has joined #zuul14:56
openstackgerritMerged zuul/zuul-jobs master: tox siblings installed packages: Add PEP 440 direct reference format  https://review.opendev.org/72747515:00
openstackgerritMatthieu Huin proposed zuul/zuul master: Add simple testing for Zuul CLI & REST API  https://review.opendev.org/72809815:09
openstackgerritMerged zuul/nodepool master: Update dib to 2.36.0  https://review.opendev.org/72376115:11
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-tox-output: use tox_extra_args  https://review.opendev.org/72802315:23
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: tox: use tox_extra_args, add tox_config_file  https://review.opendev.org/72802315:30
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: tox: use tox_extra_args, add tox_config_file  https://review.opendev.org/72802315:31
*** ysandeep is now known as ysandeep|sleep15:31
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: tox: use tox_extra_args, add tox_config_file  https://review.opendev.org/72802315:39
openstackgerritMatthieu Huin proposed zuul/zuul master: REST API: remove useless tenant when doing autohold query by id  https://review.opendev.org/72811815:42
*** jcapitao is now known as jcapitao_afk15:44
AJaegercorvus: want to approve the ownership policy change at https://review.opendev.org/#/c/724855/ and tristanC's https://review.opendev.org/#/c/681870/ fetch-sphinx-tarball change? Both have two +2s, so I could also +A but wanted to give you a change to review15:45
corvusAJaeger: yep, i think we got good sign-offs on the policy on earlier revs15:48
AJaegercorvus: agreed15:49
avasscorvus: we wanted to wait with merging the rest of the stack until next week though, right?15:51
corvusavass: yep, i think leaving your WIP vote in place on 727717 until then is good15:52
avasshmm, we're using tox so much so I'm almost thinking about creating a tox module for ansible instead of updating and trying to keep the tox jinja consistent everywhere15:56
avassthat would probably make things much easier15:56
corvusAJaeger, tristanC, mordred: was 681870 the one where we were talking about the docs/ and artifacts/ directory under zuul-fetch-output?  what was the result of that conversation?15:57
avasscorvus: there's no way to supply an ansible module with zuul-jobs without a role is there?15:59
*** bhavikdbavishi has joined #zuul16:00
*** rpittau is now known as rpittau|afk16:01
tristanCAJaeger: corvus: mordred: iiuc the current proposal using zuul-output is idendical to what synchronize do. We could use zuul-output/docs but that requires using the merge-zuul-output role, thus i'd like to do that in a follow-up16:01
AJaegercorvus: that was the one - and I don't remeber. Happy to remove my +2...16:01
corvusavass: no.  but symlinks between roles might work.16:01
avasscorvus: yeah that might be enough16:01
avasscorvus: how about having a library in the root and symlink to the library for each role?16:02
avasscorvus: or, just put the tox module in the tox role :)16:03
corvustristanC, AJaeger, mordred: yeah, i just think that the reason we're not using those other directories yet is that it's really hard to change all the synchronize invocations.  this seems like the time to do it (or abandon it).  basically, if we make a "new system" (ie, fetch-output + merge-output) we should either use it or abandon it.16:04
openstackgerritMerged zuul/zuul-jobs master: Policy rule for ownership between remote and executor  https://review.opendev.org/72485516:04
corvusavass: i think either would be fine16:04
corvusavass: i'd probably tend toward putting it in tox.  seems like a logical place.16:04
*** bhavikdbavishi1 has joined #zuul16:04
avasscorvus: but it would have been nice to be able to supply a module library the same way we can re-use a role in zuul-jobs in a different repo16:05
corvustristanC, AJaeger, mordred: because if we duplicate synchronize to fetch-output, then i don't know how we will migrate from fetch-output-like-synchronize to fetch-output-as-designed16:05
*** bhavikdbavishi has quit IRC16:05
*** bhavikdbavishi1 is now known as bhavikdbavishi16:05
tristanCcorvus: the migration should be transparent at that point16:05
tristanCcorvus: e.g. once merge-output is mandatory16:06
corvustristanC: i don't know about that -- won't we end up with a patchwork of roles expecting docs in logs, and other ones expecting them in docs....16:06
corvustristanC: isn't it already, if you want your docs to be in logs?16:06
tristanCcorvus: right now, `fetch-sphinx-tarball` puts the docs in logs, my proposal with `zuul_use_fetch_output` is to do the same, in a backward compatible way16:07
corvustristanC: right, i'm saying that the fetch-output role is designed differently, and as we use it, we should use it as designed16:08
corvustristanC: or, if we don't want to use it that way, we should change its design to what we want16:08
tristanCcorvus: i wasn't aware of the merge-zuul-output requirement and we haven't integrate it (our post job doesn't run that job)16:09
tristanCrole*16:09
corvustristanC: is it problematic to add it?16:10
corvusthere was a conversation about this from a few days ago, what were the agreed upon outcomes from that?=16:11
tristanCcorvus: i don't get why we should abandon or enforce the ~/zuul-output/docs` usage. That can be done transparently in a follow-up16:11
corvustristanC: i'm not sure it can16:11
corvustristanC: does anything use zuul-output/docs?  have we ever seen that work?16:12
corvusis there a docs publication job anywhere that uses that directory?16:13
corvusif not, how would we start using it?16:13
corvushow could such a job know to use ~work/docs vs ~/work/logs?16:13
tristanCcorvus: i don't know16:14
corvusi don't either, which is why i don't know that we can make this change transparently later.  i just want to know what the plan is.16:15
*** y2kenny has joined #zuul16:16
tristanCcorvus: i meant, the current proposal should be transparent, it just make the fetch-sphinx role works with kubectl node, resulting in the same artifacts as the one produced with synchronize16:17
tristanCso i guess that if we change that, e.g. make fetch-sphinx produce artifacts in the log directory, then that is already a behavior change16:17
corvustristanC: right, but it uses fetch-output in a way that i don't think fetch-output is expecting.  if we're going to use fetch-output we should use it right, or make sure that we're not about to use it in a way that will prevent us from migrating to it correctly.16:17
corvusif we're going to put docs in the logs directory instead of the docs directory, then i think we need one of two things: (a) a plan for how we're going to move to using the docs directory in the future; or (b) to abandon use of the docs directory if no one understands how it's supposed to be used.16:18
corvusthe one thing i don't think we should do is use fetch-output in a way that will prevent us from using it the way we want to use it in the future, and without a plan for (a) i don't know we can say that.16:19
tristanCthen i vote for (b), it doesn't seems to be used and i don't understand what is the benefit16:20
tristanCin the meantime, the sphinx and reno job are simply not working with kubectl node16:20
*** sshnaidm is now known as sshnaidm|afk16:21
corvustristanC, mordred: this is your recent conversation: http://eavesdrop.openstack.org/irclogs/%23zuul/%23zuul.2020-05-12.log.html#t2020-05-12T16:27:1216:21
corvustristanC, mordred: can you discuss that further and come to an agreement on what fetch-output should do?16:22
corvusi'm fine with either (a) or (b).16:22
corvustristanC: and whichever of those we do, we can still proceed with http://lists.zuul-ci.org/pipermail/zuul-discuss/2020-April/001215.html16:23
tristanCmordred: what is the purpose of using ~/zuul-output/docs/ if the docs is going to be copied over to {{ zuul.executor.log_root }} ?16:23
corvustristanC: (if we move everything to fetch-output, we can just later have fetch-output use the appropriate synchronize module)16:23
tristanCunderstanding the purpose of ~/zuul-output/docs would help figure out a play for (a)16:25
tristanCplan*16:25
mordredtristanC: the purpose is so that jobs can write docs to be published into a dir on the remote node separately from the log output they publish. the implementation of publishing draft documentation by way of the log publication is an implememntaion detail16:25
mordredthat way a job can always say "these are the docs I produce"16:25
tristanCmordred: so it's just so that we can write ~/zuul-output/docs instead of ~/zuul-output/logs/docs ?16:26
mordredthen for jobs taht want to publish documentaiton can always just put them into the docs dir - and contextually the underlying system to eitehr put them into the logs dir for preview or into a final location that isn't the log publishing system for publication16:26
*** dpawlik has quit IRC16:27
mordredtristanC: yes. that said - I'm ok with changing the intent there - but that's the why it was done that way originally - so that the admin of a zuul could define behavior for logs and docs differently16:27
corvusit seems like most of our docs publication in opendev has moved to being promote-based, and that only requires a docs archive in the logs dir16:28
mordredyeah. I *think* it's ok to do what tristanC is suggesting here - I'm pretty sure our understanding of the world has moved on16:28
mordredI'm mostly just trying to make sure I'm sharing as much as I understand about the original whuy16:28
mordredwhy16:28
corvusmordred: thanks.  that's kind of where i'm ending up too: that artifact-based promote has superceded the idea of specific directories on build nodes16:29
tristanCi guess the other quirk is that for multinode jobs, then ~/zuul-output/logs gets published using a hostname prefix dir16:29
mordredtristanC: yah - but maybe just don't do docs builds in multinode jobs :)16:29
corvustristanC: oh that's a good point.16:30
mordredmaybe docs should always go to zuul-output/logs/docs without a hostname prefix in all cases?16:30
mordredjust in cae someone decides they need a multi-node docs job - or maybe it's a multi-node job that also makes docs or something (like a 5 node C++ build farm that also has a code-doc output)16:31
tristanCnot sure what would be the expected behavior if a multinode job publish a ~/zuul-output/logs/report.html file, i guess only one will get published so that's not much better anyway16:32
*** evrardjp has quit IRC16:33
corvustristanC: if a multinode job did that, it would be published in its hostname subdir16:33
*** evrardjp has joined #zuul16:33
tristanCi find ~/zuul-output/{docs,logs,artifacts} idea clever, but it is also subtle and it can be confusing when the behavior is different depending on the number of node, thus i would still vote for (b) to make things simpler16:34
*** guillaumec has quit IRC16:34
corvustristanC: (b) is to use only zuul-output/logs, but we still have the hostname subdirectory on multinode jobs there16:35
corvus(that's like the one part of the fetch-output system that is fully used)16:35
corvususing zuul-output/{docs,artifacts} might actually let us work around that problem16:36
tristanCcorvus: according to https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/fetch-output/tasks/main.yaml#L19 , if all the nodes of a multinode job produces a ~/zuul-output/docs/file , then only the last one will get published as the synchronize dest doesn't use hostname subdirectory prefix for docs16:36
corvustristanC: yes that's documented: https://zuul-ci.org/docs/zuul-jobs/log-roles.html#role-fetch-output16:37
corvusi think mordred's use cas of a multinode build job with one docs output is a useful way to think about it16:38
tristanCoh, well i find it a confusing behavior then16:38
corvuswe're not trying to publish multiple docs builds, we're trying to publish one doc build, that just happened to be an output from a multinode job16:38
tristanCand in such extrem case of multinode doc building job, wouldn't it be better to explicitely refer to the node hostname to avoid such overwriting issue?16:39
corvusthat's a reasonable approach16:39
mordred++16:39
*** rlandy is now known as rlandy|biab16:40
corvusso it sounds like we're still in favor of b: drop docs/artifacts and the merge-output-to-logs role?16:40
tristanCyes, but i can go with (a) if the majority prefer, i don't have a strong opinion16:43
tristanCmordred: has the job to publish all the files from the ~/zuul-output/docs ever be written yet?16:43
mordredtristanC: I don't see anything that does, no. like corvus said - I think the promote based workflow really superceded this, since doc publication jobs in opendev now fetch the doc tarball and untar it to the correct publication dir16:46
mordredI did a search though base-jobs, project-config, ozj and z-j and it doesn't look like anyhting is using this interface on our end16:46
*** jcapitao_afk is now known as jcapitao16:49
corvusmordred, tristanC, AJaeger: based on that, i +2d with comment on https://review.opendev.org/681870  --  that look right?16:49
tristanCcorvus: sure, then should i send an advance notification about the removal of ~/zuul-outputs/{docs,artifacts} directory support?16:50
tristanCthen in the future, we could drop the merge-output-to-logs role and remove the custom docs,artifacts creation and synchronize task16:50
tristanCand perhaps makes ~/zuul-output/logs a symlink to ~/zuul-output and synchronize ~/zuul-output directly16:51
corvustristanC: yep i think so -- i think i'd send the plan to zuul-discuss first just to give anyone else a chance to say "wait we're using that"16:51
corvustristanC: i dunno about the last one... it's hard to come back from that if we change our minds :)16:52
corvussaying explicitly that you want something to go into the logs seems reasonable16:52
corvusso maybe we keep zuul-output/logs and we reserve the ability to have zuul-output/foo in the future, we just don't have anything that uses it right now16:53
*** jcapitao has quit IRC16:53
tristanCalright, it's also less work to keep it like that16:53
AJaegercorvus: go ahead and +A ;)16:55
*** bhavikdbavishi has quit IRC16:58
*** bhavikdbavishi has joined #zuul16:59
*** bhavikdbavishi1 has joined #zuul17:02
*** bhavikdbavishi has quit IRC17:03
*** bhavikdbavishi1 is now known as bhavikdbavishi17:03
* AJaeger just did it17:06
AJaegeranybody else to review an upload-artifactory role, please? https://review.opendev.org/72567817:07
corvusAJaeger, avass: i have one nit on that, which is that we usually have the zuul-tests.d/ file match the docs file.17:10
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: Simplify zuul-output usage and remove merge-output-to-logs  https://review.opendev.org/72815117:11
*** jpena is now known as jpena|off17:13
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Move artifactory test job  https://review.opendev.org/72815217:13
avasscorvus: corvus: We do?17:13
corvusavass, AJaeger: but we don't need a 54th patchset for that17:13
avass:)17:13
tristanCcorvus: mordred: let me know if i understand the (a) plan correctly in https://review.opendev.org/72815117:13
avasscorvus: oh I see17:13
corvusavass: yeah, it should be 1:1.  at least, it was the last time i touched it :)17:14
avasscorvus: I don't think it is anymore :)17:14
corvussigh17:14
avasscorvus: But I could try to clean that up some day17:14
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Add a tox module  https://review.opendev.org/72815417:14
avassbefore I get too invested in this, how about something like that ^?17:15
avassoh oops17:15
*** panda is now known as panda|out17:15
avasswell you get the point but I need to fix that mess from git17:15
corvusavass: after my change, every "zuul-tests.d/foo-roles-jobs.yaml" file corresponds to a "doc/source/foo-roles.rst" file17:16
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Add a tox module  https://review.opendev.org/72815417:16
avasscorvus: Oh, I thought I had messed that up, looks like I didn't17:16
corvusavass: (there are some foo-roles.rst files that don't have foo-roles-jobs.yaml files, but that's probably mostly just roles that don't have tests yet)17:17
corvusavass: yeah, i think we're still pretty organized :)17:17
avasscorvus: anyway, I didn't want to interrupt the discussion earlier, but any thoughts on being able to distribute modules the same way we do with roles in zuul-jobs?17:18
corvusavass: what would use those modules?17:19
corvus(if not roles in zuul-jobs)17:19
AJaegercorvus, tristanC , do we need an announcement for https://review.opendev.org/72646317:19
openstackgerritMerged zuul/zuul-jobs master: fetch-sphinx-tarball: introduce zuul_use_fetch_output  https://review.opendev.org/68187017:19
corvusAJaeger: i don't think it's a zuul project?17:19
avasscorvus: well, say you want to run tox, but you don't want the entire tox role17:20
avassI guess I don't have a concrete example of it :)17:20
corvusavass: isn't the point of the tox role just to run tox?17:20
AJaegercorvus, tristanC : let me try again: https://review.opendev.org/#/c/728151/1 removes a role, do we need an announcement for it?17:21
corvusavass: (that's why we have other roles like ensure-tox which are separate)17:21
corvusavass: the idea with zuul-jobs is that roles are small reusable units of work, so it's easy to assemble jobs as lists of roles17:21
corvusavass: so if "tox" is doing too much, then that's a problem :)17:22
avasshmm, I guess that's true. I can't think of a good reason to do it17:23
*** bhavikdbavishi has quit IRC17:23
openstackgerritMerged zuul/zuul-jobs master: Add upload-artifactory role  https://review.opendev.org/72567817:26
*** yolanda has quit IRC17:28
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Add a tox module  https://review.opendev.org/72815417:28
avasscorvus: thoughts on that then ^ :)17:28
*** guillaumec has joined #zuul17:30
*** bhavikdbavishi has joined #zuul17:35
openstackgerritGuillaume Chauvel proposed zuul/zuul master: Fix quickstart gating, Add git name and email to executor  https://review.opendev.org/72409617:42
*** fbo is now known as fbo|off17:48
corvusavass: structurally it's fine i think, but the ansible/jinja doesn't look that much less complicated -- are you planning on doing stuff that would be more complex in ansible/jinja than it would be in the module?  maybe it'll be more clear when i see the new use.17:50
avasscorvus: to be honest, not really, I just abandoned it because I realized I can't see it solving the problem I wanted to solve originally17:54
avasscorvus: I'll probably just add more tests to the roles instead :)17:55
corvusavass: more tests are always a good investment :)17:57
avassI guess it only made the task easier to read compared to the jinja, but I don't think that's worth maintaining in this case17:57
AJaegerzuul-jobs-maint, https://review.opendev.org/726836 makes us not fail if testr is not installed - I tend to ignore the fly-by -1 on that one and will +2 now. Anybody else to review?18:03
AJaegerand here's a change to empty output dirs - not sure about that one: https://review.opendev.org/72713518:04
corvusi think avass left a good explanation there18:04
openstackgerritGuillaume Chauvel proposed zuul/zuul master: WIP: Import user tutorials from Software Factory project blog  https://review.opendev.org/72819318:05
openstackgerritGuillaume Chauvel proposed zuul/zuul master: WIP: Add tutorial tests  https://review.opendev.org/72819418:05
openstackgerritMerged zuul/zuul-jobs master: Move artifactory test job  https://review.opendev.org/72815218:05
avasscorvus: yeah, I noticed we can't use the tox job directly since we don't use testr18:05
avasscorvus: and we were leaving logs on static nodes :)18:05
openstackgerritGuillaume Chauvel proposed zuul/zuul master: WIP: Add tutorial tests  https://review.opendev.org/72819418:06
avasscorvus: So I added a cleanup to remove things in zuul-output, but making sure they're empty to start with is probably good too18:06
corvusclarkb, tobiash, mordred: i found the missing piece for the loading_errors bug thanks to the extra logging we added (we just hit the bug in a change i wrote).  it happens when a change that touches the config depends on another change that touches the config and the merge job for the second change completes before the first and these changes are in a tenant which has a standing config error in a different18:09
corvusproject in the same tenant because that project's config references a config object that is defined in a project that is not part of this tenant (but is part of another tenant) and the other tenant loads without error and is listed before this one.18:09
AJaegercorvus, avass, team, I think https://review.opendev.org/#/c/725030 is fine in general (pep8 linting change) but I would rename the regex variable. But I don't think we need https://review.opendev.org/#/c/725100/ - I'm not seeing a use case for that. What's your take?18:11
*** hashar has joined #zuul18:11
corvusclarkb, tobiash, mordred: to make it concrete: it happens in the zuul opendev tenant because it loads system-config, which is broken in the zuul tenant because the puppet jobs aren't defined.  but it's not broken in the 'opendev' tenant, and the opendev tenant is loaded first.  that means the config objects are cached.18:11
AJaegercorvus: wow! That's quite a condition!18:11
avasscorvus: wow18:11
avassI had to read that a couple of times18:12
clarkbavass: I'm still trying to read and parse it :)18:12
corvussorry about that18:12
AJaegermordred: so, want to continue to retire the puppet modules so that we can get rid of that config error ? https://review.opendev.org/#/c/720901/ and friends needs some more work...18:13
corvusi have a repro test case; i should be able to finish this up this afternoon18:14
corvusbut i'm going to break for exercise and lunch first18:15
avassAJaeger: well, the first change undoes what tobiash did when he separated the different matchers to avoid warnings, but the second change, which is not yet ready, was supposed to move the matchers out to the config18:15
avassAJaeger: because I think it will be impossible to separate a growing number of output formats that are almost the same and at the same time avoid something that might be errors, so I think it should be possible for the user to configure what they want to match or avoid themselves18:17
avasss/errors/warnings18:17
AJaegerso, you want the user to pass in a new regex to use instead of PEP8_RE ?18:23
avassAJaeger: yeah, or a list of them18:24
avassAJaeger: or a negative matcher if for some reason they need that18:24
avassAJaeger: the regexes feels a lot like configuration and not something that should be hard coded into the module to me :)18:26
AJaegeravass: Ok, getting it. what about the following: rename PEP8_RE to LINT_RE in 725030 and then base 725100 on top of it - and thus allow to replace LINT_RE, and then a third change (part of 725100) for the negative matcher?18:27
avassAJaeger: yeah sure18:28
AJaegerI think each is a good logical step and we can iterate one after the other - and see whether that is acceptable.18:28
*** rlandy|biab is now known as rlandy18:30
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: tox: update lint regex to not require column  https://review.opendev.org/72503018:31
avassAJaeger: I'll update 725100 later :)18:32
AJaegeravass: no urgency - thanks.18:33
* AJaeger reviews later ;)18:33
avass:)18:33
avassAJaeger: there's no hurry, it was just a rabbit hole I fell into when wondering why ansible-lint wasn't leaving comments in gerrit :)18:34
*** bhavikdbavishi has quit IRC18:36
AJaegerAh!18:36
tristanCwhat is the recommended way to use a template file located in the project src dir?  when using zuul.project.src_dir it is failing because the file doesn't exists on the Ansible Controller18:45
tristanCshould the user try `{{ zuul.executor.src_root }}/{{ zuul.project.canonical_name }}` ?18:46
avasstristanC: is the template file in the project but in a role or in a files directory next to the playbook?18:49
avassnot in a role*18:49
avasstristanC: if so I would guess that's the way to do it18:50
*** dpawlik has joined #zuul18:51
tristanCavass: alright thank you for confirming18:53
*** felixedel has joined #zuul18:53
avasstristanC: but I don't understand why it wouldn't be in a location easily accessible by ansible :)18:54
*** felixedel has quit IRC19:00
*** nils has quit IRC19:20
*** hashar has quit IRC19:22
*** hashar has joined #zuul19:23
openstackgerritGuillaume Chauvel proposed zuul/zuul master: WIP: Add tutorial tests  https://review.opendev.org/72819419:28
openstackgerritGuillaume Chauvel proposed zuul/nodepool master: Rename quick-start to tutorial  https://review.opendev.org/72826219:28
tobiashcorvus: awesome finding :)19:42
*** dpawlik has quit IRC19:46
guillaumecis it possible to change a job's name with multiple changes like I'm trying to do with https://review.opendev.org/#/c/728194/ and https://review.opendev.org/728194 ?19:49
corvusguillaumec: you need a 3-step process (copy, switch, delete)19:50
corvusi wonder if there's a name for that19:51
corvusguillaumec: not sure if you saw earlier, but we were admiring your blog post; very nice!19:51
guillaumeccorvus: I didn't,  which blog post ?19:53
corvusguillaumec: oh i'm sorry! i mistook you for a different guillaume C.  my apologies; this was the post https://medium.com/leboncoin-engineering-blog/leboncoin-commits-life-960a86cd35ff19:55
corvuson the plus side, it's great our community is growing enough we have name collisions :)19:56
avassthat's an interesting coincidence19:56
guillaumecthat's not me :)  not the same name !19:56
corvusguillaumec: i understand that now, i apologize for my mistake19:56
avasscorvus: hmm, just noticed that zuul doesn't error if you add a job to a pipeline that doesn't exist20:07
guillaumeccorvus: my (short) story. I am merely starting to use Zuul, the quick-start tutorial is nice, but having only one change for the check and gate pipelines doesn't show the true power of gating, it even felt redundant.   I tried to test gating myself, didn't work because of a strange merge error even if I had made sure it could not happen (patch pending: https://review.opendev.org/#/c/724096/) .  Then I found20:08
guillaumecsoftwarefactory-project tutorials which are nice,  tristanC pointed me that a change was initiated to port those to zuul documentation.20:08
corvusguillaumec: i agree that a great next step after the quick-start is to teach people how to set up gating.  thanks for working on that :)20:09
openstackgerritGuillaume Chauvel proposed zuul/zuul master: WIP: Add tutorial tests  https://review.opendev.org/72819420:12
avass\o/ we're now automatically deploying our zuul and nodepool configuration20:14
guillaumeccorvus: thanks for the blog post nonetheless, I had seen an article about leboncoin using zuul, this one is also nice. I really like Zuul and will try to have it adopt by my company.20:17
openstackgerritJames E. Blair proposed zuul/zuul master: Fix loading_errors bug  https://review.opendev.org/72828620:32
corvustobiash, mordred, clarkb: ^ hopefully that's more clear :)20:32
corvusas expected, it's a one-line fix with 71 lines of testing :)20:33
avass:)20:34
clarkbcorvus: added to my review queue. I think that is change ~620:35
openstackgerritJames E. Blair proposed zuul/zuul master: Fix loading_errors bug  https://review.opendev.org/72828620:37
corvuspep8 ^20:37
*** hashar has quit IRC20:49
clarkbcorvus: commented. I think therre is a small update that would be good to have to avoid future confusion (and then a second minor nit thing)20:54
openstackgerritJames E. Blair proposed zuul/zuul master: Fix loading_errors bug  https://review.opendev.org/72828620:58
corvusclarkb: thanks fixed20:59
clarkblgtm thanks20:59
*** Goneri has quit IRC21:04
openstackgerritJames E. Blair proposed zuul/zuul master: Fix loading_errors bug  https://review.opendev.org/72828621:05
corvusclarkb: oops, one more thing ^ i forgot to remove my extra debugging21:05
openstackgerritGuillaume Chauvel proposed zuul/zuul master: WIP: Add tutorial tests  https://review.opendev.org/72819421:42
*** mattw4 has joined #zuul22:35
*** mattw4 has quit IRC23:03
*** tosky has quit IRC23:05
*** tumble has quit IRC23:15
*** jamesmcarthur has joined #zuul23:29
*** jamesmcarthur has quit IRC23:39
*** jamesmcarthur has joined #zuul23:44
*** jamesmcarthur has quit IRC23:46
*** guillaumec has quit IRC23:56

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!