Wednesday, 2020-04-15

*** threestrands has joined #zuul00:00
clarkbwow so they just removed it00:02
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Add initial withCertManager input toggle  https://review.opendev.org/71884000:04
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Add gearman tls secret provided by cert-manager  https://review.opendev.org/71911000:08
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Add registry tls secret provided by cert-manager  https://review.opendev.org/71918500:16
*** cdearborn has quit IRC00:23
*** rfolco has quit IRC00:30
openstackgerritIan Wienand proposed zuul/zuul-jobs master: Document output variables  https://review.opendev.org/71970400:48
ianwtristanC: there was a couple in the stack https://review.opendev.org/#/q/topic:ensure-pip you didn't vote on -- was that on purpose or just didn't get to?01:00
openstackgerritMerged zuul/zuul-jobs master: ensure-pip: Add role  https://review.opendev.org/71763901:09
*** swest has quit IRC01:18
openstackgerritMerged zuul/nodepool master: Stop checking user_data in func test  https://review.opendev.org/72008801:27
openstackgerritIan Wienand proposed zuul/zuul-jobs master: Python roles: misc doc updates  https://review.opendev.org/72011101:27
*** swest has joined #zuul01:33
*** y2kenny has quit IRC01:36
*** Goneri has quit IRC01:37
openstackgerritMerged zuul/nodepool master: Add ZooKeeper TLS support  https://review.opendev.org/71273301:41
*** ysandeep|away is now known as ysandeep|rover02:11
*** zxiiro has quit IRC02:39
*** sanjayu__ has quit IRC02:47
*** ysandeep|rover is now known as ysandeep|BRB04:08
*** ysandeep|BRB is now known as ysandeep|rover04:23
*** bhavikdbavishi has joined #zuul04:26
*** bhavikdbavishi1 has joined #zuul04:29
*** bhavikdbavishi has quit IRC04:30
*** bhavikdbavishi1 is now known as bhavikdbavishi04:30
*** evrardjp has quit IRC04:37
*** evrardjp has joined #zuul04:37
openstackgerritIan Wienand proposed zuul/nodepool master: Test alternative username in unit tests  https://review.opendev.org/72010105:32
*** tobberydberg has joined #zuul05:45
*** threestrands has quit IRC05:54
*** bhavikdbavishi has quit IRC06:18
*** hashar has joined #zuul06:26
*** bhavikdbavishi has joined #zuul06:35
*** hashar has quit IRC06:42
*** gtema has joined #zuul06:43
*** dpawlik has joined #zuul06:49
*** sanjayu__ has joined #zuul06:55
*** hashar has joined #zuul07:07
*** jcapitao has joined #zuul07:14
*** tosky has joined #zuul07:23
*** rpittau|afk is now known as rpittau07:28
*** bhavikdbavishi has quit IRC07:47
*** ysandeep|rover is now known as ysandeep|lunch07:48
*** jpena|off is now known as jpena07:53
*** bhavikdbavishi has joined #zuul07:59
*** hashar has quit IRC08:02
msuszkoianw: Excuse me for tests missing from c/719191 I'm speeding though environment and process setup at the moment. (behind air gap)08:03
*** bhavikdbavishi has quit IRC08:22
*** bhavikdbavishi has joined #zuul08:30
*** bhavikdbavishi has quit IRC08:35
zbrAJaeger: re suse support on zuul-roles, i wonder who can commit to keep suse supported?08:39
*** ysandeep|lunch is now known as ysandeep|rover08:40
*** bhavikdbavishi has joined #zuul08:40
zbri would personally be happy to avoid having to spend extra time making roles work on suse.08:40
*** gtema has left #zuul08:40
AJaegerzbr: dirk and cmurphy (both not in this channel) are the best to talk to08:44
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: Improve 404 error message on download-logs.sh  https://review.opendev.org/72003508:52
openstackgerritSorin Sbarnea proposed zuul/zuul master: For pre blocks to wrap text  https://review.opendev.org/68153209:03
openstackgerritSorin Sbarnea proposed zuul/zuul master: Force pre blocks to wrap text  https://review.opendev.org/68153209:03
swestzuul-maint: anyone up for a review of https://review.opendev.org/#/c/701531/ ? Besides the issues that this solves it's also improving the startup time quite a bit :)09:10
tobiashtristanC or mordred: this would be an easy review which improves logging in nodepool: https://review.opendev.org/71656609:12
*** sgw has quit IRC09:12
*** hashar has joined #zuul09:14
openstackgerritTobias Henkel proposed zuul/zuul master: Add --validate-tenants option to zuul scheduler  https://review.opendev.org/54216009:19
avasstobiash: re https://review.opendev.org/#/c/653712/09:33
avasstobiash: I have a vague memory doing something like that and ansible_host/ansible_user would use the delegate hosts variables instead of the remote host.09:34
avassI could be wrong though09:34
tobiashavass: ansible_host stays the remote host09:34
avassalright09:34
tobiashwe use it like that in prod since a long time ;)09:35
avassoh :)09:35
avassCouldn't get a clear answer when looking around either09:35
avasstobiash: We can't test that in zuul-jobs right? I guess we don't have any ssh-enabled windows nodes09:36
tobiashavass: unfortunately no and we couldn't setup a third party ci yet09:37
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Support per branch change queues  https://review.opendev.org/71853109:55
openstackgerritMerged zuul/zuul-jobs master: Support ssh-enabled windows hosts in add-build-sshkey  https://review.opendev.org/65371210:00
openstackgerritMerged zuul/zuul master: Use ZK TLS in quickstart  https://review.opendev.org/71281710:05
*** rpittau is now known as rpittau|bbl10:23
*** jcapitao is now known as jcapitao_lunch10:38
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Move queue from pipeline to project  https://review.opendev.org/72018210:40
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Move queue from pipeline to project  https://review.opendev.org/72018210:42
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Move queue from pipeline to project  https://review.opendev.org/72018210:45
openstackgerritMerged zuul/zuul-jobs master: Improve 404 error message on download-logs.sh  https://review.opendev.org/72003510:46
*** ysandeep|rover is now known as ysandeep|coffee11:00
*** cdearborn has joined #zuul11:15
*** ysandeep|coffee is now known as ysandeep|rover11:17
*** jpena is now known as jpena|lunch11:30
openstackgerritTobias Henkel proposed zuul/zuul master: Support per branch change queues  https://review.opendev.org/71853111:39
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Move queue from pipeline to project  https://review.opendev.org/72018211:39
*** jcapitao_lunch is now known as jcapitao11:58
tristanCswest: tobiash: done and done :-)12:01
*** gtema has joined #zuul12:01
*** bhavikdbavishi has quit IRC12:03
tobiashtristanC: thanks!12:04
openstackgerritJan Kubovy proposed zuul/zuul master: Connect merger to Zookeeper  https://review.opendev.org/71622112:07
openstackgerritJan Kubovy proposed zuul/zuul master: Connect executor to Zookeeper  https://review.opendev.org/71626212:07
openstackgerritJan Kubovy proposed zuul/zuul master: Connect fingergw to Zookeeper  https://review.opendev.org/71687512:07
tristanCianw: oops i misread the ensure-pip role implementation, it seems like it install pip however pip is already there12:07
tristanCcorvus: it seems like a case of an ensure-* role that is not a noop when the thing it ensure is already there12:08
tristanCleft a comment about it in https://review.opendev.org/#/c/717663/2412:10
tobiashcorvus: since I implemented a small part of https://review.opendev.org/701531 (as I implemented the py35 workaround) I've reduced my vote on that to +1, it would be great if you could have a second look on that12:13
*** bhavikdbavishi has joined #zuul12:15
tobiashzuul-maint: I'd have an easy review that fixes timeouts waiting for logger issues with windows jobs: https://review.opendev.org/71797312:15
swesttristanC: thanks a lot!12:16
*** rfolco has joined #zuul12:17
openstackgerritTobias Henkel proposed zuul/zuul master: Support per branch change queues  https://review.opendev.org/71853112:19
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Move queue from pipeline to project  https://review.opendev.org/72018212:19
*** Goneri has joined #zuul12:21
tobiashtristanC: this would be an easy review as well: https://review.opendev.org/718397 (fixes disabled tls verification for test environments)12:28
tobiashtristanC: thanks!12:32
*** jpena|lunch is now known as jpena12:33
*** bhavikdbavishi has quit IRC12:36
openstackgerritMerged zuul/nodepool master: Add debug info on json parse error of image upload  https://review.opendev.org/71656612:39
openstackgerritTobias Henkel proposed zuul/nodepool master: Logs stats for nodepool automated cleanup  https://review.opendev.org/61407412:42
*** yolanda has quit IRC12:43
openstackgerritMerged zuul/zuul master: Ensure correct cleanup on repo update and reset  https://review.opendev.org/70153112:46
openstackgerritJan Kubovy proposed zuul/zuul master: Scheduler's pause/resume functionality  https://review.opendev.org/70973512:47
openstackgerritJan Kubovy proposed zuul/zuul master: Separate connection registries in tests  https://review.opendev.org/71295812:47
openstackgerritJan Kubovy proposed zuul/zuul master: Prepare Zookeeper for scale-out scheduler  https://review.opendev.org/71726912:47
openstackgerritJan Kubovy proposed zuul/zuul master: Improve typings for driver event ingestion  https://review.opendev.org/71729712:47
openstackgerritJan Kubovy proposed zuul/zuul master: WIP: Driver event ingestion  https://review.opendev.org/71729912:47
*** bhavikdbavishi has joined #zuul12:49
*** rpittau|bbl is now known as rpittau12:49
openstackgerritTobias Henkel proposed zuul/nodepool master: Logs stats for nodepool automated cleanup  https://review.opendev.org/61407412:51
*** gtema has quit IRC13:14
openstackgerritMerged zuul/nodepool master: k8s/OKD Provider: Don't Set ca_cert if TLS verification is skipped  https://review.opendev.org/71839713:15
openstackgerritTobias Henkel proposed zuul/zuul master: Add --validate-tenants option to zuul scheduler  https://review.opendev.org/54216013:22
andreykurilinhi folks! I have a question regarding secrets: if the secret should be defined in a project where it is used, why the name of secret should be unique per all projects?13:24
andreykurilinI mean there is project A that defines and uses secret with name XXX and there is another project B that would like to setup the secret with name XXX for it own purpose... zuul currently returns 2 errors to me (for project B): first one says that XXX secret is defined in another project and the second one says that I cannot use secret XXX since it should be defined in the same project...13:27
*** gtema has joined #zuul13:29
*** sgw has joined #zuul13:32
*** ysandeep|rover is now known as ysandeep|away13:32
*** gtema_ has joined #zuul13:32
*** gtema has quit IRC13:36
mordredandreykurilin: that's an excellent question. I think the first part is simply that all the other objects share a global namespace - I'm not sure how feasible it would be at the moment in the configparser to scope them to a project ... although your reasoning about their usage certainly makes logical sense. we'll probably want to ask corvus about that when he wakes up13:37
*** bhavikdbavishi has quit IRC13:37
corvusclarkb, AJaeger: see comments on https://review.opendev.org/71766313:43
andreykurilinmordred: good. at least that is not I'm doing something completely wrong :) thanks! PS: I reserved dockerhub_credentials secret name (devil) at openstack tenant13:43
AJaegercorvus: agreed, let's point it out to ianw as well ^13:44
corvusandreykurilin, mordred: i think that covers it -- they're just not special.  andreykurilin, but since they aren't special, you might consider sticking to the openstack naming scheme and prefixing that with your project name: https://docs.opendev.org/opendev/infra-manual/latest/drivers.html#consistent-naming-for-zuul-jobs13:45
corvusandreykurilin: for example, here's zuul's own dockerhub secret: https://opendev.org/zuul/zuul/src/branch/master/.zuul.yaml#L11113:47
corvustobiash: i don't remember the problem you mention on 70153113:49
andreykurilincorvus: It's mine! I was first! my precious...13:49
andreykurilinPS: I will rename it for sure to avoid confusions. I would name it right from the beginning but made an assumption that if it can be used only by project where it is defined, it should not follow naming convention since it is inner thing13:49
andreykurilinthanks13:50
*** gtema_ has left #zuul13:50
tobiashcorvus: you mean the problem this attempts to fix or the problem the old git version had?13:50
corvusandreykurilin: yeah, makes sense13:50
corvustobiash: something about breaking the py35 job?13:50
tobiashcorvus: the problem with the py35 job was that it used an old git versions that leaks empty directories, hence the workaround: https://review.opendev.org/#/c/701531/8/zuul/merger/merger.py line 315 (which is the part I added)13:51
corvusandreykurilin, mordred: it may be more intuitive to scope them to the project, but then we'll have two different kinds of config objects (scoped and unscoped), which is also a bit of complexity for users to know.  and if we ever figure out a way to safely let projects share secrets, that could be a problem.  all of that to say, i think we'd want to think carefully about scoping it.13:51
corvustobiash: ooooooooooooh! i get it, you're saying you don't feel entitled to a +2 vote on that because of your contribution.  thanks for letting us know, but i'm not too worried about that.  :)13:53
tobiashcorvus: yes, exactly ;)13:53
corvus(for a minute, i thought there was some other change with a py35 fix)13:54
tristanCyeah me too, got me worried for a bit :-)13:55
tobiashI just wanted to comply with the 2x +2 vote rule and my +2 in this case would be actually +1.7 or so ;)13:56
corvusit rounds up ;)13:56
corvustobiash: speaking of -- https://review.opendev.org/718708 could use a +313:56
tobiashcorvus: yeah, I rechecked it with the intention to +3 afterwards :)13:57
corvusand hey, it looks like the zk tls stuff merged :)13:57
tobiashyes :)13:57
corvusi think maybe the next steps are that we should start using that in opendev, then release a 3.x with that, and ask people to start using it13:58
*** y2kenny has joined #zuul13:58
tobiash++13:58
mordredcorvus: ++13:59
tobiashcorvus: re moving queue to project, this is a first stab which is a surprisingly small change and passes all but one tests so far: https://review.opendev.org/72018213:59
mordredcorvus: do you think it's worth trying to ansiblify our install? we've got the docker+afs thing blocking using docker, but I could update my ansible patch to just pip install zuul on the executor14:00
mordredcorvus: or would you rather not conflate the two goals for now?14:00
tobiashcorvus: we would then rebase the cyclic dep change on that and adapt it to leverage that14:00
zbrpart of ensure rename: https://review.opendev.org/#/c/720145/14:04
corvusmordred: hrm; if we're going to be stuck at the afs question for a while, maybe that is worth it14:08
corvusmordred: -> #opendev14:08
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Move queue from pipeline to project  https://review.opendev.org/72018214:10
y2kennyis there a way to post a URL when a job is queued?  I see the success and failure-url but not sure if there's one for job queued or started.  I noticed there's the web.status_url in zuul.conf but setting it doesn't seem to do what I thought it would do.14:18
corvusy2kenny: do you want to leave a message in gerrit when a job is queued?14:19
y2kennyyes14:19
y2kennypretty much pointing folks to zuul-web so that people can look at the streamer if they want14:21
AJaegery2kenny: you want an URL like http://zuul.opendev.org/t/zuul/status/change/720182,6 for https://review.opendev.org/720182 ?14:22
y2kennyAJaeger: oh yes!  I actually didn't know that type of pages exists14:23
y2kennyI always click into the stream log but the example you gave is even better14:23
AJaegery2kenny: click on the icon to the left of the change in the webui14:23
AJaegery2kenny: so, on http://zuul.opendev.org/t/zuul/status there's a Green circle with a tick mark besides 720182, click on it ;)14:24
AJaegerunfortunately not easy to find ;(14:24
openstackgerritMerged zuul/zuul master: OIDCAuthenticator: add capabilities, scope option  https://review.opendev.org/70227514:25
corvusy2kenny: zuul only reports to gerrit on entire items (changes), not individual jobs; so for things like that, you'll want to look at the pipeline config rather than the job config.  https://zuul-ci.org/docs/zuul/reference/pipeline_def.html#attr-pipeline.start should be close to what you describe14:28
corvusy2kenny: note also the one above "enqueue" though it will run that even if no jobs are run, so it may not be quite what you're looking for.14:28
y2kennycorvus: great thanks.  I will take a look14:30
*** ysandeep|away is now known as ysandeep14:31
y2kennyAJaeger: ah yes, I see it now.14:34
y2kennythanks for the tip14:35
openstackgerritMerged zuul/nodepool master: Add requires to zuul-quick-start job  https://review.opendev.org/71870814:36
openstackgerritSorin Sbarnea proposed zuul/zuul master: Force pre blocks to wrap text  https://review.opendev.org/68153214:53
openstackgerritSorin Sbarnea proposed zuul/zuul master: Force pre blocks to wrap text  https://review.opendev.org/68153214:55
zbrdoes anyone know why we print the stdout/stderr abbreviated output using <span> intead of <pre> inside renderFailedTask ?15:00
corvustristanC: ^15:01
zbrprinting console output as "text" seems weird to me, as we also loose the fixed-width font due to this.15:01
zbris quite funny because msg and exc are printed with pre, but not stdout/stderr.15:03
zbrasking because if there is no particular reason, I would like to make it a use <pre>.15:03
tristanCzbr: corvus: no particular reason15:04
openstackgerritSorin Sbarnea proposed zuul/zuul master: WIP: Improve display of stdout/stderr task errors  https://review.opendev.org/68153215:09
openstackgerritTobias Henkel proposed zuul/zuul master: Support per branch change queues  https://review.opendev.org/71853115:11
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Move queue from pipeline to project  https://review.opendev.org/72018215:11
*** ianychoi has quit IRC15:13
zbrtristanC: corvus: tranks, i will ping you when I have the desired rendering achieved.15:15
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Add zuul-preview service  https://review.opendev.org/71841915:16
y2kennyAre there any documentations on replacement fields like {change.number} {build.uuid}, etc?  I tried searching and couldn't find any.  I think I sumbled upon it buy searching the source and finding some test case15:17
y2kennyby*15:17
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Add zuul-operator-upload-image job  https://review.opendev.org/70886015:17
zbrit is really "env" lookup plugin banned from being used on zuul? what would a workaround for {{ lookup('env', 'HOME') }} ?15:22
clarkbcorvus: tristanC I guess the earlier ensure-pip changes didn't catch it because sf ci isn't yet using ensure-pip but does use ensure-tox?15:24
tristanCclarkb: that is correct15:24
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: test: refactor run tasks to include file  https://review.opendev.org/71815815:25
clarkbtristanC: ok just making sure we weren't missing something else there. Thanks for confirming15:25
clarkbzbr: I think it would only be blocked on localhost but should work on remotes? Unless lookup plugins are somethign that always runs within the ansible process on localhost15:26
zbrlookup env always runs on controller, well documented.15:26
tristanCclarkb: ftr here is the current list of files triggering tox job test: https://softwarefactory-project.io/cgit/third-party-ci-jobs/tree/zuul.d/legacy.yaml#n13915:26
tristanCand we are going to improve the third-party-ci-jobs we do on zuul-jobs shortly to improve and increase test coverage with kubernetes pods15:27
zbri am trying to make a role compatible with zuul, already got rid of a custom filter, but i reached the env one now.15:27
avasszbr: why do you need the home directory on the controll node?15:28
tristanCzbr: when would you need to know the HOME environment of the executor user?15:28
zbrlong story.... https://opendev.org/openstack/ansible-role-collect-logs/src/branch/master/defaults/main.yml#L315:29
*** Goneri has quit IRC15:30
zbri guess i need to use a zuul exposed var when present to avoid it.15:30
clarkbzbr: ya beacuse that won't be the correct path from the zuul executor standpoint necessarily15:30
clarkbI think in our case currently it would be if the lookup was allowed to run but nothing requires zuul's executor process to run with a user that has the same homedir as the remote test nodes15:31
*** Goneri has joined #zuul15:33
avassThis is ready if anyone want to take a look at it: https://review.opendev.org/#/c/709292/2315:42
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: dhall-diff: add new job  https://review.opendev.org/71869415:44
zbrclarkb: for that part case it was easy to use zuul_work_dir but i am afraid that there is a lot of lookup(env,...) usage along the role, not sure how to address the others.15:47
*** ianychoi has joined #zuul15:47
zbrprobably it would be easier if env would always return nothing instead of raising an exception.15:48
zbri already have cased with fallback to a default value.15:48
clarkbzbr: the issue is one of security concern so its better to error I think15:51
zbrclarkb: i will probably find out soon the extend of the issue. I wonder if ansible_env can be used instead.15:54
clarkbzbr: if ansible_env is populated as a fact from the remote host it should work15:55
zbryep, but the intended use was to load vars from the controller, as the playbook logic can vary based on them.15:56
clarkbzbr: in a zuul context that won't be valid15:57
zbrif it does not work, i will probably have to contain all unsafe lookups into a vars file which is loaded only when outside zuul.15:57
clarkbthe homedir of the controller is completely independent of the remote15:58
zbrhomedir issue already sorted, i was worried about the rest, but i think I have an idea about what can be done15:59
corvusi don't recall the security concern with env; if there isn't one, we may be able to allow it...16:00
zbrit can be a security issue if we populate stuff like PYPI_PASSWORD, or similar.16:00
avasscorvus: api keys16:00
openstackgerritJan Kubovy proposed zuul/zuul master: Scheduler's pause/resume functionality  https://review.opendev.org/70973516:01
openstackgerritMerged zuul/zuul-jobs master: Adds roles to install and run hashicorp packer  https://review.opendev.org/70929216:01
corvusavass: like running zuul-executor with that set globally?16:01
zbrmaybe a more friendly approach would be to whitelist/blacklist specific variables, so we would allow some to be exposed but not all.16:02
corvusi'd be scared to do that; i'd rather rely on trusted_ro_path to get private information to trusted jobs16:02
corvus(basically, the blocked ansible plugins are our first line of defense, but they're also our weakest)16:03
avasscorvus: right, that's probably only set for the executor process16:03
corvusyeah, i don't enven know if we pass through env vars to bwrap and ansible16:03
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: test: refactor run tasks to include file  https://review.opendev.org/71815816:04
corvusoh, yep we do pass them through16:04
corvusso yeah, you could run "SECRET_ENV_VAR=apikey zuul-executor", and it'll show up in jobs.  but i wouldn't recommend it, i'd recommend writing the secret to a file and using trusted_ro_paths to pull that file into the bwrap for trusted playbooks16:05
corvusat the very least, if we did want to consider allowing env, that means we should certainly discuss/announce it well in advance :)16:06
avasscorvus: I think we're using config interpolation with environment variables for an api key to gitlab :)16:06
tobiashI think if we want to allow that we should filter the env vars for a known set of vars before running ansible16:08
tristanCavass: corvus: that's all the case for ZUUL_DB_URI16:08
corvusavass: oh, so you're accidentally making your environment available to the executor16:08
corvusor rather to ansible16:08
corvusyou just want the executor to see that env var, but it's propagating into the job too, which is farther than you want16:08
tristanCalso* , it's actually a supported feature of zuul to support replacing secret in zuul.conf with environment variables16:08
avassyep16:09
tobiashwe're already setting up a distinct set of env vars for each ansible run: https://opendev.org/zuul/zuul/src/branch/master/zuul/executor/server.py#L212016:09
corvusright, but we start with what the executor sees16:09
tobiashso we should filter them or start from a clean set probably16:09
corvusyep16:09
corvusall of the variables we interpolate into the config are prefixed with ZUUL_ right?16:10
avassI think so16:10
openstackgerritTristan Cacqueray proposed zuul/nodepool master: config: add missing voluptuous required attributes  https://review.opendev.org/71909416:10
corvusso we could filter out ZUUL_ variables there to prevent env-config-var-interpolation from leaking into jobs16:11
tristanCavass: corvus: yes, at least those need to be filter out16:11
corvusideally before someone figures out how to work around the env plugin restriction since that's our only line of defense for that right now :)16:12
avassI can take a look at it later, doesn't sound like it should be hard to fix :)16:14
corvusavass: cool, thanks :)16:14
avassBut that does mean that we're probably able to configure ansible callbacks with env vars heh16:16
*** gtema has joined #zuul16:17
*** jcapitao has quit IRC16:20
*** gtema has quit IRC16:22
openstackgerritAlbin Vass proposed zuul/zuul master: Filter out environment config variables  https://review.opendev.org/72023816:23
avasscorvus: send you a private message16:31
*** rpittau is now known as rpittau|afk16:33
openstackgerritSorin Sbarnea proposed zuul/zuul master: WIP: Improve display of stdout/stderr task errors  https://review.opendev.org/68153216:36
*** evrardjp has quit IRC16:37
*** evrardjp has joined #zuul16:37
*** jpena is now known as jpena|off17:10
openstackgerritAlbin Vass proposed zuul/zuul master: Filter secret ZUUL_ env variables from ansible env  https://review.opendev.org/72023817:11
mnasershould we warn users if they push up a change where they have both zuul.d/ and .zuul.yaml in there?  maybe refuse to land those?17:13
mnaseri've been hit by this17:13
SpamapSmnaser: isn't there a defined order?17:15
SpamapS(that's what I would have assumed)17:15
mnaserSpamapS: maybe? but i think if you have zuul.d folder, .zuul.yaml seems to be completely ignored from my experience17:16
zbrclarkb: apparently we need to do something about env, take a look at https://zuul.opendev.org/t/openstack/build/81c14f81eadc4ba68cda4e8be11e782e17:17
zbr An unhandled exception occurred while templating '{{ zuul_work_dir | default(lookup('env', 'HOME')) }}/.quickstart'.17:17
zbrif this failed with exception, i have no idea how to make it work, apparently the lazy-rendering does not work in that case.17:18
tobiashmnaser, SpamapS: yes, there is a defined order but no notification when one file is ignored afaik17:18
fungiSpamapS: mnaser: yes, the documentation states the order they're considered in, but i agree it's potentially confusing17:18
tobiashI guess a warning in this case might make sense17:18
mnaserthere is a warning, but only in the zuul log17:18
mnaser"2020-04-14 11:25:21,394 WARNING zuul.TenantParser: Multiple configuration files in ...."17:19
tobiashmnaser: then it's probably easy to plumb this warning through to the user17:19
fungii think so far we've avoided providing non-fatal warnings in feedback on proposed changes17:19
tobiashwe have the mechanisms for that in place17:19
tobiash(see warnings attribute on the queue item or buildset)17:20
fungiahh, right that. i don't think we've generally propagated those into review comments though17:20
* mnaser is a little ENOTIME these days17:21
mnaseri can't get into this, but something to keep in mind i guess17:21
fungii could see making it a tenant configuration option though, whether to reject changes which would shadow a zuul config with another zuul config at a different path17:22
SpamapSA warning in the UI just like the job config errors would be ideal IMO.17:26
*** yolanda has joined #zuul17:27
openstackgerritJames E. Blair proposed zuul/nodepool master: Mark dib as deleting before erasing files from disk  https://review.opendev.org/72024717:29
corvusclarkb, fungi, mordred: ^17:29
fungithanks corvus!17:30
fungiSpamapS: you mean the alarm bell notification on the dashboard, that could be reasonable too yes17:30
*** y2kenny has quit IRC17:30
fungier, that was meant to be a question17:31
SpamapSfungi: yeah, I found those to be massively useful in cleaning up trash.17:31
tristanCcorvus: not sure why, but i now get a cryptic `This change depends on a change with an invalid configuration.` on https://review.opendev.org/718158 . Have this reach a limit of acceptable speculative change ?17:32
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Warn user when dynamic layout ignores zuul config  https://review.opendev.org/72024917:32
corvustristanC: you're probably hitting a long-standing bug we've been trying to track down17:32
tobiashmnaser, SpamapSthat should do it ^17:32
corvuswe added some more debugging logs for it; i'm unsure if opendev is running with that though17:32
corvustristanC: i have reached the limit of the number of fires i can fight at one time right now, so i can't check the logs :(17:33
tristanCcorvus: no worries, but i think this time it's different, previously it was 'unknown error' iirc17:33
tristanCoh and one more recheck made it through, it's building again17:34
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Check if pip is preinstalled before installing it  https://review.opendev.org/72025417:36
corvustristanC: ^ can you look at that17:37
tristanCsure17:38
*** gtema has joined #zuul17:47
*** dpawlik has quit IRC17:48
*** gtema has quit IRC17:51
clarkbtristanC: apparently python3-pip on centos8 does not install `pip`17:58
clarkbso that assumption is bad17:58
clarkbworking on a fix now17:58
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Check if pip is preinstalled before installing it  https://review.opendev.org/72025418:00
clarkbour testing actually has pretty good coverage of that stuff across distros looks like. Except that most of our images come preinstalled with pip18:00
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: test: refactor run tasks to include file  https://review.opendev.org/71815818:05
openstackgerritMerged zuul/nodepool master: Mark dib as deleting before erasing files from disk  https://review.opendev.org/72024718:26
AJaegerclarkb: is https://review.opendev.org/#/c/720257/1 addressing corvus' comment from https://review.opendev.org/#/c/717663/ ?18:34
openstackgerritTristan Cacqueray proposed zuul/nodepool master: config: add missing voluptuous required attributes  https://review.opendev.org/71909418:35
mordredAJaeger: https://review.opendev.org/720254 is18:35
AJaegermordred: wrong paste in my change - thanks18:36
openstackgerritTristan Cacqueray proposed zuul/zuul master: merger: replace custom message logger  https://review.opendev.org/72026618:54
openstackgerritTristan Cacqueray proposed zuul/nodepool master: config: add environment variable substitution  https://review.opendev.org/71959918:54
*** sassyn has joined #zuul19:02
sassynHi All Again,19:02
openstackgerritTristan Cacqueray proposed zuul/zuul master: URLTrigger driver time based  https://review.opendev.org/63556719:03
*** hashar has quit IRC19:04
sassynI have a question on the pipeline. Consider the following: User X push a change to x, the zuul trigger this change, run the job and on success will mark it as Verified +1. User Y go to the gerrit web interface and then do a code review +2 on this patch of user X. Zuul then will run the JOB and if all goes OK will merge the code.19:05
fungigo on19:06
sassynhi fungi :-)19:06
fungihi ;)19:07
sassynmy questions: 1. the test as you can see here running two times, for the verified and for the merge.  If I have patch X1, X2, X3, X4, X5 , it means I run it 5 times for verified, and then X1, X1+X2, X1+X2+X3, X1+X2+X3+X4, X1+X2+X3+X4+X5 = total 10 times19:08
fungithat sounds right. zuul is designed to test each patch so it can identify if one of them has a problem19:09
fungiand you can't assume that the test from when the patch was proposed will yield the same result later when it's approved19:10
*** bhavikdbavishi has joined #zuul19:10
sassynBut why I should run the X1...X5 in stand alone if I run it after19:10
fungibecause the state of the target branch in the repository may have changed over that time, and also any patches approved before it which haven't merged yet could introduce something which breaks that patch19:10
fungiyou don't *have* to run changes when they're first proposed, it's usually just done as feedback to reviewers19:11
fungier, don't have to run tests for patches when they're proposed19:11
fungifor most code review based workflows, reviewers do want to know if a proposed patch is passing tests though19:12
sassynmy problem is that my jobs takes a lot of time ~ 45 min.19:12
clarkbfwiw this is all configurable19:12
fungiour average job runtime in opendev is around an hour, and we have some jobs which take 3-4 hours to complete19:13
clarkbyou should be able to run the jobs when it makes the most sense for your repos19:13
fungisome of our tenants are also configured for our "gate" pipeline to supplant our "check" pipeline, so if jobs haven't completed in check yet when a change gets enqueued into gate, then the check jobs are aborted to free up resources19:13
AJaegercorvus: want to review https://review.opendev.org/#/c/720254/ and then change your vote on https://review.opendev.org/#/c/717663/? (the pip ensure zuul-jobs changes)?19:14
*** sanjayu__ has quit IRC19:14
fungisassyn: also we configure our gate pipeline to have a higher priority than check, so gate always gets all the available resources if it needs them, and check uses what's still available19:14
sassynbut this mean the gate will never check patches which not being verifed19:15
fungihow does it mean that?19:15
sassyngate will check only +2 reviewed +1 verified19:16
fungias clarkb says, that's configurable19:16
sassynothers patch in the Q, will wait19:16
clarkbAJaeger: corvus tristanC I can also depends on or rebase it such that tristanC's jobs will run. Though I think we can land the ensure-pip change first then recheck ensure-tox as nothing is using ensure-pip yet19:16
fungisome of our tenants have a gate pipeline which does not depend on there being an existing verified +119:16
fungisassyn: ^19:16
fungiso changes get enqueued straight into it the moment they're approved, start running jobs there, and abort any builds for that change in the check pipeline as they're no longer relevant19:17
fungisassyn: here's an example... https://opendev.org/opendev/project-config/src/branch/master/zuul.d/pipelines.yaml#L30-L6719:19
sassynthe critical issue, is that when a user X push a patch X, the job is running... and get verified. Then or in parallel other user Y also pushed a patch Y which got verified.  Now I need to submit patch X, and Reasbe patch Y (which will trigger again a verified JOB), and then do the submit.19:19
funginote the absence of a check for verified:1 and also the supercedes:check directive19:19
sassynYes19:20
sassynI read about it...19:20
fungisassyn: why do you need to rebase?19:20
sassyncause I'm in fast forward git repo19:20
AJaegerclarkb: let's land and recheck. I'll +2 and give corvus a chance to review...19:20
fungisassyn: oh, that sounds like a self-imposed choice for inefficiency in that case. you get a prettier git log, but at the expense of constant rebasing19:21
AJaegerclarkb: but go ahead and +A if you want19:21
fungisassyn: it's one of the reasons we use merge-if-necessary strategy for our projects19:21
fungithe history is still effectively linear if you ignore the single-change merge commit glue (and git log --no-merges is an easy way to filter those out when skimming history)19:22
sassynU know too much :-)19:23
sassynbut what if I want to keep fast forward19:23
sassynwhat would u recommend?19:24
fungisassyn: then you'll run a lot of additional jobs and get very little benefit from dependent pipelines19:24
sassyncan u please explain more?19:25
fungii would consider fast-forward mode in gerrit to be a (poor) alternative to what zuul does for you by ensuring changes are tested in the order they merge19:25
fungican you explain what perceived benefits you gain from gerrit forcing you to rebase changes if they merge in a different order than you propose them?19:26
sassynwell, I can't say19:27
sassynI need to ask the HEAD of the R&D19:27
sassynthis was his call19:27
fungiahh, basically most of zuul's features aren't much use if you're forced to use gerrit in that mode19:28
AJaegersassyn: education him/her19:28
fungilike for example if a change fails in a dependent pipeline, you're going to have to rebase everything, zuul won't be able to just restart testing without the failing commit19:28
*** bhavikdbavishi has quit IRC19:29
fungifast forward mode in gerrit seems like it would be a hindrance for any project with more than one developer working on more than one series of changes at a time19:30
fungizuul is designed to multiplex your development activity19:30
*** bhavikdbavishi has joined #zuul19:31
fungithis is effectively the entire reason it was developed19:31
sassynfungi I agree with u19:32
fungigerrit and git were also designed with that in mind, which is really why merge commits exist in the first place19:32
sassyntrue, fast-forward is like SVN19:33
sassynold school19:33
openstackgerritMerged zuul/nodepool master: diskimage.username setting was not read from configuration file  https://review.opendev.org/71919119:35
sassynbut If we will leave it aside for a moment, the verified +1 for patch A, and then Review +2 will trigger again the Job in merge if needed. not for fast forwarder19:35
sassynthis is what I understand19:37
openstackgerritMerged zuul/nodepool master: Test alternative username in unit tests  https://review.opendev.org/72010119:44
*** yolanda has quit IRC19:50
fungisassyn: yes, on zuul's side, it will try to merge the change to (its copy of) the branch and run jobs on it19:55
fungii know it also consults gerrit to find out if the change meets gerrit's requirements to submit, so it's possible that in ff-only mode gerrit may say any patchset which isn't parented to the branch tip isn't fit to submit, in which case zuul may just leave a verify -2 or similar and say something like "this change cannot be merged to the repository" (like it would for an actual merge conflict)19:57
mordredonce we have zuul-push, setting merge-mode to cherry-pick should allow both "clean tree" and taking advantage of zuul's multiplexing20:02
*** yolanda has joined #zuul20:03
fungigerrit's configured merge-mode becomes irrelevant once zuul is pushing the branch state into it directly, doesn't it?20:04
mordredfungi: a bit yeah20:08
mordredfungi: although if you have gerrit set to ff-only and zuul to merge, gerrit is going to grump at you20:09
*** jkt has quit IRC20:09
*** jkt has joined #zuul20:10
fungiyeah20:13
*** harrymichal has joined #zuul20:16
openstackgerritAlbin Vass proposed zuul/zuul master: Filter secret ZUUL_ env variables from ansible env  https://review.opendev.org/72023820:37
avass^ wanted to test that without running ansible to make it a bit faster but I couldn't figure out a good way of doing it20:39
sassynfungi : Thank You!20:39
sassynis it possible to do the a reverse check? so if A, B, C, D patch are pushed, I will run a job on  A + B + C +D, and not on A, A+B, A+B+C? so if A+B+C+D will pass, it will be merge? and if it fails, then I will run A+B+C... and so on..20:41
fungisassyn: i don't think there's any support for that in zuul, it's designed to make sure that every point in your git history passes configured jobs, and not skip testing some commits just because they're approved around the same point in time20:45
fungithis is so that you can more reliably git bisect and expect that every point you're checking was tested as it merged20:46
fungiand also so that if someone pulls from the repo in the middle of that batch of changes merging, they don't wind up with broken/untested source code20:47
clarkbya zuul doesn't do batching20:48
clarkbit is likely possible to write a pipeline manager that does though20:49
clarkbI'm not sure how you'd know when to create a batch. Maybe on a time interval? Basically every minute clump changes together or something (probably makes sense to make that user configurable)20:51
openstackgerritAlbin Vass proposed zuul/zuul master: Filter secret ZUUL_ env variables from ansible env  https://review.opendev.org/72023821:00
*** harrymichal has quit IRC21:07
openstackgerritAlbin Vass proposed zuul/zuul master: Filter secret ZUUL_ env variables from ansible env  https://review.opendev.org/72023821:10
*** bhavikdbavishi has quit IRC21:28
mordredsassyn: it's worth noting, in that case, zuul will run tests on a, a+b, a+b+c and a+b+c+d in parallel, so if a+b+c passes but a+b+c+d does not, you'll get a+b+c merged in the same amount of time (actually more quickly, since you don't have to wait for a+b+c+d to run,then fail, then a+b+c to run)21:34
avassmordred: that's if you have the resources to do that22:05
openstackgerritTristan Cacqueray proposed zuul/zuul master: merger: replace custom message logger  https://review.opendev.org/72026622:05
clarkbcorvus: ianw did you want to weigh in on https://review.opendev.org/#/c/720254/ and whether we should rebase ianw's stack on it or just merge and recheck ianw's stack?22:11
mordredavass: totally. but - you know - just buy more computers ;)22:13
corvusif the computers are cheaper than engineers, that approach may have some success with folks in control of budget.  however, sometimes computers are more expensive than people.22:14
corvusclarkb: did that get tested in sf.io?22:15
clarkbcorvus: not yet. There are two ways we can do that I think, rebase it into ianw's stack (or depends on it) or merge it then recheck ianw's change22:16
clarkbcorvus: ensure-pip has landed but none of the consumer role changes have so its semi safe to land it then test. But it means we may have ot make more changes later22:16
corvusi like the depends-on approach22:16
clarkbk22:16
clarkbworking on that now22:16
corvusclarkb: +1 on process grounds; i don't have enough stuff paged in to +2 the code, but trust you and the other reviewers on that.22:18
openstackgerritClark Boylan proposed zuul/zuul-jobs master: ensure-tox: use ensure-pip role  https://review.opendev.org/71766322:18
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Update Fedora to 31  https://review.opendev.org/71765722:18
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Make ubuntu-plain jobs voting  https://review.opendev.org/71970122:18
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Document output variables  https://review.opendev.org/71970422:18
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Python roles: misc doc updates  https://review.opendev.org/72011122:18
clarkbtristanC: ianw corvus AJaeger ^ fyi 3pci from sf.io should test 717663 with the don't install pip if it is installed checks now22:18
tristanCclarkb: we could also add ensure-pip to the files that trigger sf.io tox test job22:21
clarkbtristanC: ya until its consumed its probably less useful but that would be a good change once the rest of the stack there lands22:21
*** rfolco has quit IRC22:24
openstackgerritClark Boylan proposed zuul/zuul-jobs master: ensure-pip: export ensure_pip_virtualenv_command  https://review.opendev.org/71822422:34
openstackgerritClark Boylan proposed zuul/zuul-jobs master: fetch-zuul-cloner: use ensure-pip  https://review.opendev.org/71788222:34
openstackgerritClark Boylan proposed zuul/zuul-jobs master: fetch-subunit-output test: use ensure-pip  https://review.opendev.org/71822522:34
openstackgerritClark Boylan proposed zuul/zuul-jobs master: ensure-tox: use ensure-pip role  https://review.opendev.org/71766322:34
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Update Fedora to 31  https://review.opendev.org/71765722:34
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Make ubuntu-plain jobs voting  https://review.opendev.org/71970122:34
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Document output variables  https://review.opendev.org/71970422:34
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Python roles: misc doc updates  https://review.opendev.org/72011122:34
*** Goneri has quit IRC22:47
openstackgerritMerged zuul/zuul-jobs master: Check if pip is preinstalled before installing it  https://review.opendev.org/72025423:26
*** tosky has quit IRC23:39

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!