Friday, 2019-09-06

*** mattw4 has quit IRC00:06
*** jamesmcarthur has quit IRC00:06
*** jamesmcarthur has joined #zuul00:40
SpamapSAnybody know what's going on with the Zuul Kubernetes operator?00:45
SpamapSGetting tired of maintaining my spruce/yaml/bash/Makefile monstrosity.. wondering if I should divert that effort to working on the operator.00:46
ianwre testinfra, filed https://storyboard.openstack.org/#!/story/2006498 as i can't see what's up00:51
clarkbSpamapS: I think you want jeliu00:52
*** jamesmcarthur has quit IRC01:06
*** jamesmcarthur has joined #zuul01:06
SpamapSjeliu, clarkb: are there patches up? Storyboard tasks, anything I can use to help/track/etc. ?01:10
*** mattw4 has joined #zuul01:11
clarkbSpamapS: https://review.opendev.org/#/q/project:zuul/zuul-operator01:15
*** jamesmcarthur has quit IRC01:21
*** spsurya has joined #zuul01:28
*** jamesmcarthur has joined #zuul02:05
*** jamesmcarthur has quit IRC02:18
*** jamesmcarthur has joined #zuul02:20
*** jamesmcarthur has quit IRC02:26
*** jamesmcarthur_ has joined #zuul02:26
*** jamesmcarthur_ has quit IRC02:31
*** jamesmcarthur has joined #zuul02:34
*** jamesmcarthur has quit IRC03:34
*** jamesmcarthur has joined #zuul03:37
*** rfolco has quit IRC03:38
*** jamesmcarthur has quit IRC03:42
*** jamesmcarthur has joined #zuul04:05
*** igordc has joined #zuul04:10
*** jamesmcarthur has quit IRC04:12
*** igordc has quit IRC04:23
*** jank has joined #zuul04:48
*** jamesmcarthur has joined #zuul05:09
*** jamesmcarthur has quit IRC05:13
SpamapSclarkb:ty05:21
*** pcaruana has joined #zuul05:25
flaper87tristanC: do you have a custom executor image? The base one doesn't have kubectl/oc installed.05:33
*** swest has joined #zuul05:44
*** jamesmcarthur has joined #zuul05:49
*** jamesmcarthur has quit IRC05:54
*** mattw4 has quit IRC06:00
*** raukadah is now known as chandankumar06:03
*** jank has quit IRC06:17
*** jank has joined #zuul06:19
*** bjackman_ has joined #zuul06:20
*** bjackman_ has quit IRC06:46
*** jamesmcarthur has joined #zuul06:51
*** bjackman_ has joined #zuul06:54
*** jamesmcarthur has quit IRC06:55
*** jank has quit IRC07:00
*** threestrands has quit IRC07:10
*** saneax has joined #zuul07:21
*** jamesmcarthur has joined #zuul07:22
*** jpena|off is now known as jpena07:41
*** jamesmcarthur has quit IRC07:42
*** jank has joined #zuul07:49
*** pcaruana has quit IRC08:08
*** pcaruana has joined #zuul08:13
*** badboy has joined #zuul08:15
badboyhow do I change the logging level for Zuul and it's components?08:16
badboyunder [logger_zuul] change the level=DEBUG to level=INFO?08:16
*** tosky has joined #zuul08:49
*** NBorg has joined #zuul09:05
*** rfolco has joined #zuul09:07
*** NBorg has quit IRC11:08
*** jpena is now known as jpena|lunch11:36
tristanCflaper87: we use origin-clients provided by centos-release-openshift-origin3911:41
*** johnsom has quit IRC11:53
*** jpena|lunch has quit IRC11:54
*** johnsom has joined #zuul11:55
*** jpena has joined #zuul11:56
*** bjackman_ has quit IRC12:04
*** EmilienM is now known as EvilienM12:24
*** spsurya has quit IRC12:26
*** bjackman has joined #zuul12:37
*** badboy has quit IRC12:45
*** gtema_ has joined #zuul12:51
corvusSpamapS, clarkb, tristanC: i've just got word that jeliu's internship has ended and he won't have very much time to work on the operator; i think diverting attention/resources toward that would be prudent.  i think he was most recently working on getting the percona operator working.  the spec is here: https://zuul-ci.org/docs/zuul/developer/specs/kubernetes-operator.html13:01
*** gtema_ has quit IRC13:01
*** bjackman has quit IRC13:01
*** jank has quit IRC13:05
*** avass has joined #zuul13:06
*** jamesmcarthur has joined #zuul13:07
avasscorvus: Is there a reason why mirror-workspace-git-repos runs git reset --hard but not git clean -xdf?13:09
corvusavass: i don't know; are you running into a case where that's needed?  maybe you're reusing workspaces, but we just haven't done that much upstream so it was omitted...13:12
avasscorvus: Yep, since we're on static nodes the workspace are gonna be reused a lot13:13
avasscorvus: encountered a bug because some untracked files didn't get removed from a previous run13:13
corvusavass: it seems reasonable to propose a patch to add that, but also, i just woke up, so i may still be missing things :)13:14
avasscorvus: Added that locally for now. Just wanted to make sure I didn't miss something13:15
corvusavass: zuul's merger (which is designed to deal with all kinds of repo states) does a hard reset followed by a clean -xdf, so that sounds like another vote in favor of adding it :)13:15
avassYep :)13:16
*** jank has joined #zuul13:22
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Clean non-bare remote repos  https://review.opendev.org/68068913:42
pabelangerSpamapS: corvus: clarkb: tristanC: re operator, I can make some time to work on it, won't be 100% but I can start getting involved there. But if others want to help drive it, happy to see that too13:46
corvusi think there's enough work there for several folks to help out in bits, and the spec should keep us on the same page.  maybe if someone wants to claim tasks like "get pxc working", "get zk working", "write nodepool config sharding script", and "update operator crd to match spec" we could start a storyboard13:49
pabelangerYay, breaking it out into parts would work, like we did in zuulv3 spec.  Then figure out all the humans who want to do the parts13:52
pabelangerI can maybe look to creating for this weekend13:52
*** jank has quit IRC13:52
corvuspabelanger: do you have any idea why opendev uses "use-cached-repos" rather than "prepare-workspace-git" ?13:56
corvusavass: your change looks fine, but in looking at it, i noticed a hole in our testing, so i'm going to take a minute and see about closing that up13:58
fungiis that because we preinstall caches of our git repositories during image building?13:58
fungiis prepare-workspace-git supposed to also accommodate that?13:58
pabelangercorvus: no, feels like we just didn't do the migration after tobiash added it13:58
pabelangerwe do use prepare-workspace-git for zuul.a.c13:58
corvusfungi: i it is supposed to do that; it looks like a slightly more advanced version of use-cached-repos13:58
fungipabelanger: aha, that does sound like something we'd do ;)13:58
avasscorvus: alright13:58
corvusokay, so i think we should: 1) migrate opendev to that role; 2) switch the zuul-jobs testing to that role; 3) recheck avass's change, then merge13:59
pabelanger+113:59
fungisgtm13:59
corvusbecause it sure looks to me like zuul-jobs is testing use-cached-repos which is crazy since that's not even in that repo :)13:59
fungii concur, based on http://codesearch.openstack.org/?q=use-cached-repos14:02
*** hashar has joined #zuul14:05
*** jamesmcarthur has quit IRC14:07
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Update the base-roles test to use prepare-workspace-git  https://review.opendev.org/68070314:15
*** jamesmcarthur has joined #zuul14:15
corvusi think #1 and #2 can happen independently, so i'm going to go ahead and rebase avass's change on that ^ which should get it tested14:15
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Clean non-bare remote repos  https://review.opendev.org/68068914:16
*** avass has quit IRC14:22
openstackgerritTristan Cacqueray proposed zuul/zuul-base-jobs master: Add support for kubectl connection  https://review.opendev.org/68071114:31
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: Add prepare-workspace-openshift role  https://review.opendev.org/63140214:33
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: Add remove-zuul-sshkey  https://review.opendev.org/68071214:33
openstackgerritTristan Cacqueray proposed zuul/zuul-base-jobs master: Add support for kubectl connection  https://review.opendev.org/68071114:33
tristanCflaper87: here are some changes to make the base jobs work with kubectl connection: https://review.opendev.org/68071114:34
corvustristanC: is prepare-workspace-openshift openshift specific or could it be k8s?14:36
tristanCcorvus: it may works with k8s, but we are using the oc client that provides a convenient rsync wrapper14:37
corvusdoes oc work with plain k8s or does it require os?  (i know the openshift python module works with plain k8s)14:39
tristanCcorvus: it may very well work, i haven't tested it14:39
corvuswe should really get that k8s up in opendev for stuff like this :)14:39
tristanCcorvus: we are actually testing this service: https://pagure.io/software-factory/k1s14:40
*** mattw4 has joined #zuul14:42
corvustristanC: podman on the backend?14:43
tristanCcorvus: yes, podman14:43
corvusneat, i think i understand :)14:44
tristanCa bit of a hack, but we are looking forward swapping the runc driver by this new service :-)14:44
corvusi like that plan.  it's a good hack :)14:45
*** michael-beaver has joined #zuul14:49
*** mattw4 has quit IRC14:49
*** mattw4 has joined #zuul14:49
*** chandankumar is now known as raukadah14:54
*** mattw4 has quit IRC14:57
*** saneax has quit IRC15:09
corvusi think i have a lead on the task bug from yesterday -- i think it's because that last task was a handler15:10
openstackgerritJames E. Blair proposed zuul/zuul master: Record handler tasks in json job output  https://review.opendev.org/68072615:16
corvuspabelanger, clarkb: ^ mystery from yesterday solved15:16
*** noorul has joined #zuul15:33
noorulofosos: hi15:33
*** Diabelko has quit IRC15:36
*** Diabelko has joined #zuul15:36
*** mattw4 has joined #zuul15:51
clarkbcorvus: have a moment for quick rereview of https://review.opendev.org/#/c/679670/ the faq page?15:53
*** hashar has quit IRC15:54
corvusclarkb: +3 thx15:54
openstackgerritMerged zuul/zuul-website master: Add Zuul FAQ page  https://review.opendev.org/67967015:57
noorulIs zuul for opendev deployed and configured using gitops? If so can someone point me to the code?15:59
corvusnoorul: yes, but don't do what we do :)  we're moving to containers and will abandon our current system.  1 sec and i'll get links16:00
corvusnoorul: https://docs.openstack.org/infra/system-config/zuul.html16:00
noorulcorvus: By containers do you mean k8s operator?16:01
corvusnoorul: eventually, but we'd be happy just moving to docker16:01
corvusit's all puppet now16:01
noorulcorvus: What will you recommend for production?16:03
corvusyou have to keep in mind that when we deployed zuul initially, ansible itself was only 3 months old, and k8s hadn't been invented yet.  :)16:03
corvusnoorul: if i were setting up a new deployment today, i would use docker with upstream latest images (running on master)  based on the docker-compose file.  if you have a k8s, it should be fairly straightforward to translate that to k8s crd.16:04
clarkbya I'm not sure I would deploy a k8s just to run a zuul; however, if you plan to run tests in k8s or have k8s for other applications it wouldn't be a bad choice16:05
corvusnoorul: if you don't want to use containers at all, the windmill project is a fine choice for deploying directly on vms16:05
corvushttps://opendev.org/windmill/16:06
clarkbcorvus: probably ^ is a good FAQ page entry too?16:07
clarkb"Can zuul run in k8s?" YES "Does zuul have docker images?" YES16:07
corvusclarkb: ++16:07
openstackgerritClark Boylan proposed zuul/zuul-website master: Add faq entry about kubernetes and docker images  https://review.opendev.org/68074216:14
corvusAJaeger_: fyi, i was just reading the pdf thread -- not sure if you're aware of this option: https://zuul-ci.org/docs/zuul/user/config.html#attr-project.vars16:21
noorulHow will the user know via review system that corresponding job has started running?16:21
corvusnoorul: depends on the review system and the zuul pipeline configuration for the 'start' action16:22
corvusnoorul: https://zuul-ci.org/docs/zuul/user/config.html#attr-pipeline.start16:22
noorulcorvus: I see16:23
corvusnoorul: in opendev gerrit we do nothing in the check pipeline, but we leave a comment and clear the vote on gate.  in github, folks usually set the status.16:23
noorulcorvus: I think on stash also we need to set the status16:24
noorulcorvus: Otherwise users will be clueless16:24
noorulcorvus: s/stash/bitbucket16:24
corvusnoorul: seems reasonable; i think there are some doc changes to add sample pipelines for the different drivers.  would be good to add that to the bitbucket stack16:25
clarkbcorvus: I believe the gerrit one merged but I think there wa sstill some questions on the github one?16:26
* clarkb finds it16:26
clarkbhttps://review.opendev.org/#/c/672712/ is the github one16:27
clarkblooks like pabelanger has been reviewing it16:27
noorulAny idea when bitbucket patches will land in master?16:28
clarkbnoorul: the current review comments need to be addressed at least16:29
pabelangerclarkb: yah, github pipelines a little more tricky :(16:31
noorulUnable to reach out to ofosos :(16:34
*** pcaruana has quit IRC16:37
noorulDo we test the zuul containers in the pipeline?16:41
clarkbnoorul: the zuul quickstart job tests the containers16:43
*** jpena is now known as jpena|off16:48
*** noorul has quit IRC16:50
*** pcaruana has joined #zuul16:51
*** tosky has quit IRC16:58
ofososnoorul: what's up?17:05
ofososI'll have a look at the comments on the weekend.17:05
ofososHave a nice weekend everybody.17:05
ofososOff for a Schnitzel and beers.17:06
clarkbofosos: enjoy!17:06
*** hashar has joined #zuul17:15
clarkbhrm something wrong with the website promotion job? maybe that is what you meant about adding it to the artifacts?17:31
clarkbif we are promoting we need that artifact ya?17:31
corvusclarkb: that is not what i meant, but now that you mention it, it sounds very plausible17:32
clarkbzuul-website runs a post job17:32
clarkbso maybe a different problem17:32
corvusclarkb: is http://zuul.opendev.org/t/zuul/build/acbb494ef30f4bb6a3fba109928f738d the right build?17:33
clarkbya that ref is correct17:34
clarkbmaybe the afs vos release isn't working17:34
corvusclarkb: yeah, i see the files in afs, must not have released17:35
clarkbthe project.zuul volume is not locked for a release17:39
clarkbow I have to remember where we run the releasees from17:39
corvusclarkb: i think i've got something17:40
corvusclarkb: they run on afsdb0117:40
corvusthe log shows it getting up to starlingx but no further17:40
corvusclarkb: vos examine -localauth project.starlingx.readonly is hanging for me; let's switch to -infra17:41
clarkbk17:41
*** hashar has quit IRC17:42
*** jamesmcarthur has quit IRC17:45
*** jangutter has quit IRC17:54
*** hashar has joined #zuul18:11
corvusfaq is published!18:25
corvushttps://zuul-ci.org/docs/faq.html18:26
Shrewstobiash: clarkb: ooh, while trying help tobias with a test for https://review.opendev.org/671704 , i found an actual bug in the new code that we all missed.   yay tests!  :)18:49
clarkbthe log call ?18:51
Shrewsyeah. i left a comment18:51
*** igordc has joined #zuul18:53
clarkbcorvus: https://review.opendev.org/#/c/680498/1 failed tests because we test the heuristics that were removed18:57
clarkbcorvus: and that is parent for the handler fix18:57
clarkbcorvus: I can push up a new patchset if you want18:57
corvusclarkb: derp.  if you have it handy, please do; i'm warming up lunch19:04
clarkbk19:04
openstackgerritClark Boylan proposed zuul/zuul master: Web: rely on new attributes when determining task failure  https://review.opendev.org/68049819:06
openstackgerritClark Boylan proposed zuul/zuul master: Record handler tasks in json job output  https://review.opendev.org/68072619:06
clarkbdone19:07
clarkbI need to sort out lunc too and will then be running errands19:09
openstackgerritDavid Shrewsbury proposed zuul/nodepool master: Fix node failures when at volume quota  https://review.opendev.org/67170419:15
Shrewsclarkb: tobiash: that adds the test for tobias, but we might want to wait until he returns from vacation to validate it?? i don't think it's urgent19:17
*** hashar has quit IRC19:46
*** hashar has joined #zuul19:47
openstackgerritJeremy Stanley proposed zuul/zuul-website master: Update to page titles and Users  https://review.opendev.org/68045920:22
*** rfolco has quit IRC20:50
*** hashar has quit IRC21:14
*** hashar has joined #zuul21:21
corvus\o/  here's a milestone: 2019-09-06 14:46:42,836 DEBUG zuul.Pipeline.zuultest.check: [e: c4e8b6ad59f844c3b523a03ab663dd86] Event <GerritTriggerEvent pending-check localhost/test 1,1> for change <Change 0x7fd6412d0470 test 1,1> matched <GerritEventFilter types: pending-check uuid: zuul:check ignore_deletes: True> in pipeline <IndependentPipelineManager check>21:47
*** EvilienM is now known as EmilienM21:47
corvusthat's zuul using the checks api to enqueue an item in a pipeline21:48
corvusjust got that working in local testing21:48
fungithat's what i was going to guess from the keywords present21:48
fungiawesome!21:48
corvusthis is the pipeline config for that: http://paste.openstack.org/show/772113/21:49
corvuspaladox: ^21:49
paladox\o/21:49
paladoxyay!21:50
openstackgerritJames E. Blair proposed zuul/zuul master: WIP: Add support for the Gerrit checks plugin  https://review.opendev.org/68077821:50
corvusthat's my very un-finished work-in-progress code with bunches of comments and stuff, just to checkpoint that since i'm going to eod soon21:51
* fungi is excited21:53
fungihave a good weekend!21:53
*** hashar has quit IRC21:55
*** rfolco has joined #zuul22:38
*** michael-beaver has quit IRC22:38
*** mattw4 has quit IRC23:05
clarkbhttps://review.opendev.org/#/c/680498/2 passes tests now. I +2'd it23:07
clarkbif anyone else wants to review that it will unstick another fix in the console rendering23:08
paladoxi'm very excited when zuul takes over the ci on gerrit-review!23:10
paladoxwill be easier to tell your builds from the ui :)23:10

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!