Tuesday, 2019-10-01

*** shanemcd has quit IRC00:03
*** shanemcd has joined #zuul00:04
*** jamesmcarthur has quit IRC00:08
*** jamesmcarthur has joined #zuul00:17
*** jamesmcarthur has quit IRC00:31
*** jamesmcarthur has joined #zuul00:32
*** jamesmcarthur has quit IRC01:14
*** jamesmcarthur has joined #zuul01:17
*** ianychoi has quit IRC01:17
*** ianychoi has joined #zuul01:18
*** jamesmcarthur has quit IRC01:22
*** igordc has quit IRC01:26
*** jamesmcarthur has joined #zuul01:40
*** jamesmcarthur has quit IRC02:06
*** jamesmcarthur has joined #zuul02:07
*** bhavikdbavishi has joined #zuul03:12
*** raukadah is now known as chandankumar03:23
*** spsurya has joined #zuul03:29
*** bhavikdbavishi1 has joined #zuul03:31
*** bhavikdbavishi has quit IRC03:32
*** bhavikdbavishi1 is now known as bhavikdbavishi03:32
*** igordc has joined #zuul03:58
*** ianychoi has quit IRC03:59
*** ianychoi has joined #zuul04:00
*** bolg has joined #zuul04:03
*** bolg has quit IRC04:13
*** rfolco has quit IRC04:17
*** bolg has joined #zuul04:20
*** jamesmcarthur has quit IRC04:31
*** jamesmcarthur has joined #zuul04:33
*** spsurya has quit IRC04:37
*** spsurya has joined #zuul04:38
*** pcaruana has joined #zuul05:01
*** ianychoi has quit IRC05:03
*** ianychoi has joined #zuul05:05
*** jamesmcarthur has quit IRC05:17
*** ianychoi has quit IRC05:27
*** ianychoi has joined #zuul05:29
*** bolg has quit IRC05:29
*** jamesmcarthur has joined #zuul05:47
*** igordc has quit IRC05:52
*** ianychoi has quit IRC06:00
*** ianychoi has joined #zuul06:01
*** bolg has joined #zuul06:11
*** bolg has quit IRC06:18
*** jamesmcarthur has quit IRC06:20
*** jamesmcarthur has joined #zuul06:28
*** ianychoi has quit IRC06:40
*** ianychoi has joined #zuul06:42
openstackgerritIan Wienand proposed zuul/nodepool master: dib-functional : allow extra elements to be passed  https://review.opendev.org/68588406:44
*** bhavikdbavishi has quit IRC06:48
*** hashar has joined #zuul06:58
*** themroc has joined #zuul07:01
*** jamesmcarthur has quit IRC07:04
*** zbr is now known as zbr|ruck07:08
*** brendangalloway has joined #zuul07:12
*** tosky has joined #zuul07:16
*** recheck has quit IRC07:27
*** recheck has joined #zuul07:28
*** badboy has joined #zuul07:29
badboyis it possible to set timezone offset in the build section of the web console?07:30
*** jpena|off is now known as jpena07:43
*** jangutter has quit IRC07:56
*** hashar has quit IRC08:05
*** jangutter has joined #zuul08:14
*** yolanda has joined #zuul08:23
*** hashar has joined #zuul08:51
*** jamesmcarthur has joined #zuul09:01
toskymordred: hi, question about fetch-subunit-output (related to this old review https://review.opendev.org/#/c/534431/ ) - do I understand the code correctly that tox_envlist is no more used?09:04
*** jamesmcarthur has quit IRC09:06
mordredtosky: yeah. I mean - tox_envlist is still used in the tox jobs, but the value of tox_envlist is no longer used by fetch-subunit-output - that now just looks for all of the tox output available with the find command09:15
*** bhavikdbavishi has joined #zuul09:27
mhumordred, hi, I have a question about tox_install_sibling_packages.py in the tox jobs09:28
mhuit does not seem to install siblings if they are not present in tox's venv after the first dry run09:29
mhuis that normal?09:29
*** rfolco has joined #zuul09:29
mhuor more generally speaking, how is the "required-projects" supposed to work with a tox job?09:30
mordredmhu: yes - it's trying to use the dry run to figure out which of the sibling git repos it should attempt to install09:30
mordredso - adding a project to required-projects should cause it to be installed into the tox env if it's a library tox woudl normally install09:31
mordredbut if it's not, like, say for instance, openstack/requirements - then it should get skipped09:31
mhumordred, ah gotcha!09:32
mhuthanks, I think I see what I was doing wrong09:32
mordred\o/09:32
*** bhavikdbavishi1 has joined #zuul09:35
*** bhavikdbavishi has quit IRC09:37
*** bhavikdbavishi1 is now known as bhavikdbavishi09:37
*** hashar has quit IRC09:43
*** hashar has joined #zuul09:45
mhumordred, so I am trying to set up a demo of 2 python projects, let's say A and LibA (A depends on LibA) I want to have a common tox job for both projects that always runs tox on A (so that I can see if a change on LibA breaks A)09:54
mhuis it possible to use the default tox-py* jobs for that? Maybe by overriding the zuul_working_dir var?09:55
mhuor should I rewrite a tox-like job?09:55
mhuseems like the var override is doing the trick, but if you have a better way to do it I'll gladly take it09:58
mordredmhu: no, I think the override of zuul_work_dir is the right way to go there10:05
mordredmhu: I think mnaser has a similar demo he was using at the vexxhost booth at ansiblefest - it seems like a really good demo case to make people get it10:06
mordredmhu: it's a slightly larger example - but https://opendev.org/openstack/shade/src/branch/master/.zuul.yaml#L1-L15 is us doing that same pattern in shade10:08
mhumordred, ah noice! I'm doing this for a blog post but that's something I could add to the OSS talk at the end of the month10:09
mordred++10:13
*** bhavikdbavishi has quit IRC10:45
openstackgerritMerged zuul/zuul master: Replace tox-py36 by tox-py37  https://review.opendev.org/68215810:47
toskymordred: thanks; I'm trying to write a test for that role (to push forward https://review.opendev.org/#/c/674334/) and being able to ignore one variable is useful10:50
*** avass has joined #zuul11:05
*** jpena is now known as jpena|lunch11:38
*** bolg has joined #zuul11:49
*** frickler has joined #zuul12:03
*** ianychoi has quit IRC12:10
*** ianychoi has joined #zuul12:13
*** fdegir has quit IRC12:21
*** fdegir has joined #zuul12:22
*** jpena|lunch is now known as jpena12:38
*** badboy has quit IRC12:41
*** fdegir has quit IRC12:45
*** fdegir has joined #zuul12:45
*** themroc has quit IRC13:11
*** avass has quit IRC13:34
*** michael-beaver has joined #zuul13:42
brendangallowayDoes a roles folder need to live in the root of a project in order for other projects to be able to import them?13:43
pabelangerbrendangalloway: https://zuul-ci.org/docs/zuul/user/config.html#attr-job.roles should help. Explains how we load roles in jobs13:45
brendangallowaypabelanger: Ok, so they won't be picked up if they're in playbook/roles?  That's where most of our roles live at the moment to make it easier to test the playbooks out-of-zuul13:48
brendangallowaypabelanger: Also, while you're online.  You previously advised to try a meta: connection_reset to help with our pxe boot reconnection issue.  That did not help, any other advice?13:49
pabelangerbrendangalloway: yah, top level roles is usually what I do, as that way can 'release' it to galaxy if wanted for projects outside of zuul. But inside roles/ folder also works13:52
pabelangerbrendangalloway: I think next step is to get packet trace (tcpdump) to better understand what is happening on network.13:53
brendangallowaypabelanger: Would symlinking roles->playbooks/roles fix the issue?13:53
pabelangershould, if in the same repo13:54
pabelangerroles next to playbooks should load without issue13:54
brendangallowaypabelanger: ok, we can get a packet capture going13:55
pabelanger++13:56
brendangallowayone final question.  We're trying to get a release pipeline going.  It's currently in a branch, but the branchmatcher is not returning that branch when it checks where to run14:00
fungibrendangalloway: release via git tags? tags are not directly related to branches in git, which leads to some conceptual challenges with the way they're often treated14:01
fungihttps://review.opendev.org/578557 is a proposal for guessing the most appropriate branch for a given tag, but use some additional feedback and confirmation that it would be useful to users to make that tenuous connection14:03
brendangallowayfungi: with git tags yes, to trigger /refs/tags/  as per the pipeline that ships with zuul14:04
fungier, could use some additional feedback14:04
fungiyeah, the short story there is that a tag can refer to a ref in the history of multiple branches or even none, so to pretend there's a particular branch for a tag is a bit of a stretch14:05
fungibut it's a stretch we could have zuul make, if there's some consensus that it's warranted by popular workflows14:05
brendangallowayThat could be an issue yes.  In our scenario the commit exists only in the one specific branch.  That's also the only branch where the pipeline is actually defined.14:07
brendangallowayIs it still expected to return a single match in such a scenario?14:09
fungiif the jobs you trigger from refs/tags/* don't have an explicit branch matcher in them, then they should run14:09
fungizuul ought to load job definitions from all available branches of the repository, so should be aware that the job is supposed to be triggered on that refname pattern14:10
brendangallowayexplicit branch matcher being a job.branches variable?14:11
fungiyeah, if the job definition includes an explicit branch matcher, then zuul will see that the tag event doesn't have any associated branch data and will decide the job is not supposed to be run14:11
brendangallowayNo branches variable in the job.  We actually tried to add one so that the job definition in that branch declares it should be run in that branch to try force it without any success14:13
fungithat'll have the reverse of the desired effect, since the tag ref won't be considered to match any branch14:14
fungiin opendev i think all our tag jobs are defined in a branchless repo... maybe the implicit branch match for the configuration (due to being in a branched repository) is having a similar effect?14:15
fungianyone happen to know if implicit branch matching will cause jobs triggered from tag refs not to be considered?14:16
brendangallowayok, that makes sense.  Even without the branch decleration the matchers trigger, then report not matching any of the branches14:16
brendangallowayI think the job is defined in maybe three of the branches (release branches implemented since zuul was turned on)14:17
fungii see in opendev's infrastructure manual we've documented: 'There are several notable exceptions for job configs that should remain in the central config repository [...] Jobs that are not “branch aware”. Typically these are jobs that are triggered by tag based events.' https://docs.openstack.org/infra/manual/creators.html#central-config-exceptions14:25
fungithis suggests to me that we likely defined tag-triggered jobs in a branchless repo as a workaround for that behavior14:28
AJaegerfungi: correct, that's why we left all the publish-to-pypi in project-config which is branchless14:28
brendangallowayok, that makes sense.  I'm guessing the tag-triggered jobs never need some sort of point-in-time release branch put in place then?14:34
openstackgerritJames E. Blair proposed zuul/zuul-registry master: Initial implementation  https://review.opendev.org/68395814:34
openstackgerritJames E. Blair proposed zuul/zuul-registry master: Fix container image build  https://review.opendev.org/68580814:34
fungibrendangalloway: by "point-in-time release branch" i assume you mean branch-specific behaviors in tag-triggered jobs... in a few cases we have logic embedded in the jobs themselves to guess a relevant branch based on some heuristic and alter behavior accordingly, but the https://review.opendev.org/578557 implementation sounds more like what you're talking about i think14:37
fungiif memory serves, that would provide a means for zuul to use its branch matcher logic in combination with a guessed branch for a tag, and apply the jobs configured in the branch it has associated with the tag, so that your tag-triggered release jobs can themselves branch over time14:38
brendangallowayWell branch specific in the sense of "that's what the builder system was doing in v1.1"14:39
fungiright, i believe that's what https://review.opendev.org/578557 would try to provide14:39
brendangallowayOk, that would be great14:40
fungimore reviews of that change, even at just a conceptual level, would be appreciated14:40
brendangallowayBut in them meantime tagged jobs only work when no branches are present?14:40
fungithat's what i'm coming to realize, yes14:40
*** bhavikdbavishi has joined #zuul14:41
brendangallowayok - using a tagged release was to 1) pass some info to the builder regarding some branding actions that need to be applied 2) Allow for build to be triggered without needing a dummy review or zuul enqueue14:42
fungiright now we're still assuming that different communities/organizations may have workflows where the heuristics we think could be a relevant way to guess the branch of a tag don't work, since the data model of git doesn't itself have any direct tie between those two sorts of handles14:42
brendangallowayNot sure how to provide those functions without a tag14:43
*** gtema_ has joined #zuul14:45
fungii'm not sure whether it's relevant for your jobs, but you could for example build those artifacts on each new branch state in the context of a branch-relevant triggering event and then merely anoint them with release-specific details when the corresponding ref for one is tagged14:45
mordredcorvus: I just left a useless comment on the second zuul-registry change. it was sort of stream of consciousenss from me as I was reviewing, then I realized I was wrong.14:46
brendangallowayfungi: The problem is we have a baseline product, that then has some branding actions applied at build time depending on which customer is next expecting a release - building for all customers every release will become burdensome14:47
fungibrendangalloway: but i agree, it would be nice not to have to jump through such hoops. one reason we're interested in this feature for opendev is that openstack periodically transitions its base test platform as newer operating systems become available, but maintains stable release branches for a long period of time. we'd like to be able to easily configure jobs so that tag-triggered release jobs on different14:48
fungibranches run on the platforms which were contemporary with when those branches were created14:48
brendangallowayOur artefacts are also really really big14:49
fungiinstead, currently, we generate some point releases on platforms which are newer than the ones those branches are otherwise running jobs on, which leaves us at a risk of having those release jobs spontaneously break due to changes in behavior for those platforms14:50
brendangallowayIt sounds like your work-in-progress solution is what we're hoping for, trying to determine what to do in the meantime14:50
clarkbbrendangalloway: you can have the job take different action based on the tag value itself. THis works as long as you don't need to change job details like nodeset14:51
fungiyeah, if the differences you need to apply are present in the correct repository branches then those will still be what's checked out on disk for the builds14:52
fungithough also if these jobs integrate multiple repositories, you still have the challenge of determining what state to check out for the other repositories which aren't the one which got the tag14:53
fungibasically, zuul should provide you with a reopsitory checked out to the tag ref14:54
*** jamesmcarthur has joined #zuul14:55
Shrewscorvus: i left a couple of comments on the initial registry change. none are -1 worthy14:55
Shrewsi like the abstract driver class  \o/14:55
corvusShrews: yay!14:55
brendangallowayThey job is also integrating multiple repos, but both should be branched at the same time (and hopefully merged at some point)14:55
corvusShrews: replied.  is it + worthy? :)14:59
Shrewscorvus: so, cat_objects() will concatenate those objects saved via put_object(), yeah?15:00
corvusShrews: yes (but after move_object)15:01
*** bhavikdbavishi has quit IRC15:02
corvusShrews: put: /uploads/uuid/1; move: uploads/uuid/1 -> /blobs/sha/1; cat /blobs/sha/1 -> blobs/sha/data15:02
Shrewscorvus: this is the exact situation we had in nodepool-builder where we were leaking objects. we often see the concat fail in rax (even after retries). I added code to sdk to manually cleanup those images, and also used the delete header. I don't see us handling that situation.15:02
corvusShrews: that's what the prune command is for15:02
Shrewswhere is this prune command? i must have missed it. can't find it offhand15:03
Shrewsoh, docker image prune ?15:05
corvusShrews: line 237 of storage.py15:05
corvusShrews: invoked by 'zuul-registry prune'15:05
corvusif something goes wrong with finalizing the upload, we'll end up with objects either in the uploads/ or blobs/ tree which aren't referenced by a valid manifest, so the prune will delete them15:06
corvusoh i just caught something....15:08
*** bhavikdbavishi has joined #zuul15:09
openstackgerritJames E. Blair proposed zuul/zuul-registry master: Initial implementation  https://review.opendev.org/68395815:10
openstackgerritJames E. Blair proposed zuul/zuul-registry master: Fix container image build  https://review.opendev.org/68580815:10
openstackgerritFabien Boucher proposed zuul/zuul master: WIP - Gitlab - Basic handling of merge_requests event  https://review.opendev.org/68599015:12
*** ianychoi has quit IRC15:20
Shrewscorvus: +2 with comment15:22
*** ianychoi has joined #zuul15:23
corvusShrews: yeah, and i think there's opportunity for making uploads/downloads from swift more efficient15:25
*** mattw4 has joined #zuul15:32
*** bolg has quit IRC15:34
*** jpena is now known as jpena|brb15:42
*** bhavikdbavishi1 has joined #zuul15:46
*** bhavikdbavishi has quit IRC15:48
*** bhavikdbavishi1 is now known as bhavikdbavishi15:48
brendangallowaypabelanger: we grabbed a packet capture.  ssh key exchange occurs, 5 encrypted packets are transferred and then there's a FYN.  That repeats in a loop until the job times out15:50
*** hashar has quit IRC15:55
*** brendangalloway has quit IRC16:01
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Revert "Use import_role on persistent-firewall"  https://review.opendev.org/68599616:11
clarkbcorvus: ^ fyi e-r shows occurences well after we updated the role16:11
*** gtema_ has quit IRC16:12
clarkbthe other idea I had was to change command to shell then maybe we can add more debugging like shell: iptables-save || echo "return value $?" ?16:12
*** jpena|brb is now known as jpena16:19
corvusclarkb: do we have the output?  or does everything look okay it's just a weird exit code?16:20
clarkbit says to look at output but we don't get any16:20
fungialso the modification you're proposing to the shell task would mask the failures16:20
clarkbfungi: we already mask the failures with failed_when: false16:20
fungiahh16:21
clarkbthis is why the jobs don't fail until they try to use the stdout from that command task16:21
fungishell: iptables-save || iptables -L16:21
fungimaybe?16:22
openstackgerritJimmy McArthur proposed zuul/zuul-website master: CSS fix for ul/li in FAQ  https://review.opendev.org/68600316:31
*** jpena is now known as jpena|off17:16
AJaegerteam, are we ready to merge https://review.opendev.org/#/c/567696/ - "Allow upload-logs to toggle compression of console logs" for zuul-jobs?17:24
AJaegerI'll approve in an hour if I hear nothing otherwise, it should not hit opendev and has been announced...17:27
*** igordc has joined #zuul17:37
*** jamesmcarthur has quit IRC17:38
*** jamesmcarthur has joined #zuul17:48
pabelangerSo, who do I need to bribe to help add pagination support to builds UI? Or date range support for searching17:58
*** jamesmcarthur has quit IRC17:59
*** jamesmcarthur has joined #zuul17:59
*** jamesmcarthur has quit IRC18:04
corvuspabelanger: you could start by checking if the rest api supports that, since that would be a pre-req for the js18:11
*** jamesmcarthur has joined #zuul18:19
*** jamesmcarthur has quit IRC18:23
tristanCpabelanger: corvus: pagination could be implemented using the skip argument (documented in http://zuul.opendev.org/openapi ). Date range is missing from the api.18:30
pabelangerOMG, that is a thing?18:30
pabelangerthat is awesome18:31
pabelangerHmm, openapi doesn't seem to work locally. Need to debug why18:34
pabelangerthat at least gives me a way to manually craft some urls18:34
*** fabdultf has joined #zuul18:35
tristanCpabelanger: what is not working?18:36
tristanCit seems like path parameter are not correctly handled by the execute button, but when using whitelabeled version, it works fine: http://zuul.openstack.org/openapi18:39
openstackgerritMerged zuul/zuul-jobs master: Revert "Use import_role on persistent-firewall"  https://review.opendev.org/68599618:42
*** jamesmcarthur has joined #zuul18:47
pabelangertristanC: yah, execute button isn't doing anything. Not sure why18:50
pabelangereven if I add a tenant18:50
openstackgerritMerged zuul/zuul-jobs master: Allow upload-logs to toggle compression of console logs  https://review.opendev.org/56769618:56
*** spsurya has quit IRC19:09
mnasermhu, mordred: sorry, i can make that repo public actually, it doesnt have anything magical19:13
mnaseri dont have a documented "how to do the demo" thing19:13
mnasermhu, mordred: but this is how i setup a very simple tox cross-repo job. https://www.irccloud.com/pastebin/fQdKYQC2/19:14
openstackgerritMohammed Naser proposed zuul/zuul-jobs master: Add unit tests for tox_instalL_sibling_packages  https://review.opendev.org/68396319:18
*** Miouge has quit IRC19:38
*** Miouge has joined #zuul19:38
openstackgerritMerged zuul/zuul-jobs master: Add unit tests for tox_instalL_sibling_packages  https://review.opendev.org/68396319:42
*** dkehn has joined #zuul19:48
*** fdegir has quit IRC19:54
*** fdegir has joined #zuul19:55
*** jamesmcarthur has quit IRC20:07
*** jamesmcarthur has joined #zuul20:16
*** fabdultf has quit IRC20:17
*** pcaruana has quit IRC20:33
*** openstackstatus has quit IRC21:32
*** openstackstatus has joined #zuul21:35
*** ChanServ sets mode: +v openstackstatus21:35
openstackgerritClark Boylan proposed zuul/zuul master: Always use change url from gerrit api data  https://review.opendev.org/68605421:47
openstackgerritClark Boylan proposed zuul/zuul master: Set url scheme on HTTP Gerrit events  https://review.opendev.org/68605422:13
*** sgw has quit IRC22:42
*** sgw has joined #zuul22:43
*** armstrongs has joined #zuul22:45
*** jamesmcarthur has quit IRC22:48
*** armstrongs has quit IRC22:55
*** tosky has quit IRC23:02
openstackgerritClark Boylan proposed zuul/zuul master: Set url scheme on HTTP Gerrit events  https://review.opendev.org/68605423:02
openstackgerritClark Boylan proposed zuul/zuul master: Handle new default of uncompressed job-output  https://review.opendev.org/68607223:02
*** jamesmcarthur has joined #zuul23:32
openstackgerritMerged zuul/zuul master: Handle new default of uncompressed job-output  https://review.opendev.org/68607223:54

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!