Wednesday, 2019-07-03

*** jamesmcarthur has joined #zuul00:08
*** jamesmcarthur has quit IRC00:19
mordredcorvus: we don't have dockerhub tags because at the moment doing so would be manual ... and hard - because we don't tend to tag tip of master, and we don't have a good way to map from a git commit to a particular older image IIRC00:24
mordredcorvus: but it's been a hectic last month for me, so I could easily be wrong or misremembering00:25
corvusmordred: i agree; with our current state, we would need to have post jobs build docker images.  that's totally doable, it's just that no one has typed in the words to make that happen00:26
corvuser, tag jobs00:26
corvuser, release jobs00:26
corvuspromote for tip; release for tags.00:27
corvuswhich is how docs are set up for zuul-sphinx (and will be for all the projects when we move them to the zuul tenant)00:28
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add multi-node integration roles  https://review.opendev.org/66876700:33
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add multi-node integration roles  https://review.opendev.org/66876700:34
mordredcorvus: ah - yes - nod00:38
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add base integration roles  https://review.opendev.org/66806100:49
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add multi-node integration roles  https://review.opendev.org/66876700:49
*** panda has quit IRC01:03
*** panda has joined #zuul01:05
*** swest has quit IRC01:46
*** sgw has quit IRC01:51
*** swest has joined #zuul02:01
*** bhavikdbavishi has joined #zuul02:34
*** rlandy has quit IRC02:34
*** bhavikdbavishi1 has joined #zuul02:36
*** bhavikdbavishi has quit IRC02:38
*** bhavikdbavishi1 is now known as bhavikdbavishi02:38
*** rfolco has quit IRC02:47
*** jamesmcarthur has joined #zuul04:08
*** jamesmcarthur has quit IRC04:24
*** bolg has joined #zuul04:43
*** saneax has joined #zuul04:52
*** pcaruana has joined #zuul05:06
*** saneax has quit IRC05:14
*** wxy-xiyuan has quit IRC05:18
*** bolg has quit IRC05:33
*** igordc has quit IRC05:41
*** snapiri has quit IRC06:26
*** snapiri has joined #zuul06:26
*** hwangbo has quit IRC06:38
*** themroc has joined #zuul07:01
*** tosky has joined #zuul07:20
*** toabctl has joined #zuul07:30
*** hashar has joined #zuul07:35
*** jamesmcarthur has joined #zuul08:00
*** saneax has joined #zuul08:05
*** jamesmcarthur has quit IRC08:06
*** sshnaidm|afk is now known as sshnaidm|ruck09:19
*** tosky has quit IRC09:39
*** tosky has joined #zuul09:40
*** bhavikdbavishi has quit IRC09:42
*** hashar has quit IRC09:46
*** dmsimard9 has joined #zuul10:08
*** dmsimard has quit IRC10:08
*** saneax has quit IRC10:50
*** bkorren has joined #zuul11:11
bkorrenhi there, I'm trying to use 'zuul_return' from a job to customize the results URL Zuul posts to Gerrit - and its not working for me, can someone help me to figure out how to debug it?11:13
*** saneax has joined #zuul11:17
mordredbkorren: in your zuul, is there also a zuul_return in the base job that might be running later and overriding what your job is returning?11:31
mordredbkorren: (also, the yaml you sent in the rh email might be useful to eyes out here - mind if I put it in to paste.openstack.org to share it? or you can)11:32
AJaegermordred: good morning, want to review a zuul-jobs reorg, please? https://review.opendev.org/#/c/668763/11:35
mordredAJaeger: always11:35
bkorrenmordred, you can put it there, sure.11:35
AJaegerthanks, mordred11:35
mordredAJaeger: wow. that works? neat11:36
bkorrenmordred, could it be overriden bu the post-run playbook in the base job? I'm using the default one,...11:38
mordredhttp://paste.openstack.org/show/753809/ <-- this is the zuul_return invocation from bkorren11:38
mordredbkorren: yes - the post-run in the base job will run after the post-run in your job's playbook. we should see what corvus thinks when he gets up about how to maybe address that11:41
bkorrenmordred, got a build report here - https://ovirt-staging.softwarefactory-project.io/logs/56/256/15/check-patch/trigger-gate-job/bf0d24c/ara-report/11:41
bkorrenmordred, actually I'm not seeing a call to zuul_return in the post playbook11:42
mordredyeah - then that's definitely what's going on. you probably won't - because it tends to run after the logs have been uploaded11:43
mordredwhat we might want to do is return the log_url as an artifact_url ... but I'm still on first coffee so not having *awesome* thinking power yet :)11:45
openstackgerritMerged zuul/zuul-jobs master: Shard project-pipeline config  https://review.opendev.org/66876311:46
bkorrenmordred, I see, I'll have a look at the playbook/job source code if I'm missing something11:50
AJaegermordred: zuul is full of surprises ;)11:56
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Add base integration roles  https://review.opendev.org/66806111:56
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Add multi-node integration roles  https://review.opendev.org/66876711:56
*** cosss_ has quit IRC12:06
*** bhavikdbavishi has joined #zuul12:13
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Add multi-node integration roles  https://review.opendev.org/66876712:28
*** rlandy has joined #zuul12:33
*** dmsimard9 is now known as dmsimard12:43
bkorrenmordred, found the culprit - its the `uload-logs` role called from the `post.yaml` playbook - unfortunately there does not seem to be an easy way to make it just upload logs without also setting the the `log_url`13:00
*** bhavikdbavishi1 has joined #zuul13:00
bkorrenmordred, I can customize `post.yaml` in my tenant to make that happen but I'm not sure that would be smart in the long run13:01
*** bhavikdbavishi has quit IRC13:02
*** bhavikdbavishi1 is now known as bhavikdbavishi13:02
*** jeliu_ has joined #zuul13:40
*** bhavikdbavishi has quit IRC13:41
ofososGot a problem with nodepool. The docs read:13:47
ofosos`if Nodepool is running inside Kubernetes, this setting and the kube/config file may be omitted` w.r.t. Kubernetes provider13:48
ofososBut the source doesn't tolerate omitting this setting AFAICT https://github.com/openstack-infra/nodepool/blob/master/nodepool/driver/kubernetes/config.py13:48
tobiashcorvus: shall we land 661880? The full stack looks to be +2 now13:52
tobiash(cleanup-phase)13:52
corvusbkorren, mordred: i have an idea, but there are a couple of small things missing from zuul -- what you should be able to do is to return that as an *artifact* rather than a log_url.  then, when you look at the build page in zuul, it should show you a link to the artifact url.  The things missing from zuul are: the build page doesn't show artifact urls yet (the api does, so it's just a little JS), and, this13:54
corvuswill be a lot smoother when zuul reports the link to the build page rather than the log_url.13:54
corvustobiash: done, thx!13:55
tobiash:)13:55
corvusofosos: flaper87 just added that feature, maybe he can clarify13:56
*** sgw has joined #zuul13:57
corvusbkorren: in the mean time, you could use "success-url" and "failure-url" to override the link that zuul reports for the job: https://zuul-ci.org/docs/zuul/user/config.html#attr-job.success-url13:58
*** rfolco has joined #zuul14:01
bkorrencorvus, yeah, but that would make it static for the whole job, while I need it to be different in each run14:02
bkorrencorvus, I'm working on a solution though - I'm changing post.yaml a little so I could tell it not to set log_url and let the value I set through14:03
corvusbkorren: ah, i see you're templating values not available in job config14:03
corvusbkorren: interested in hacking a little javascript?14:04
bkorrencorvus, mordred : anyway, thanks for your help so far, I need to leave now; good day14:04
bkorrencorvus, I thought about doing a JS redirect; but that makes me feel dirty ;)14:04
corvusbkorren: i was wondering if you had some time to help with the js display of artifacts -- that's going to be the long-term solution to this, and it's pretty close14:05
corvusbkorren: anyway, if you do, let us know when you get back -- i don't mean to delay you :)14:06
*** bkorren has quit IRC14:09
AJaegercorvus: FYI, I updated your changes 668061 and 668767, the first is ready to merge now - the second has a failure in the job.14:13
corvusAJaeger: thanks, just looking at that -- the second isn't actually testing with multiple nodes yet -- i think my script is going to have to be more complicated.  it's interesting that only the persistent-firewall job failed -- maybe the rest just short-circuit because there's no "switch" node?14:14
AJaegercorvus: ah!14:16
AJaegerreally interesting ;(14:16
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add multi-node integration roles  https://review.opendev.org/66876714:23
corvusAJaeger: okay.  that's gonna burn some nodes.  :)14:24
AJaegeryeah ;(14:24
corvus5414:24
AJaegerwow, new record?14:25
corvusi think we've used more on some system-config changes (like in the 80s?)14:25
corvusbut the good news is that in the future, it shouldn't use any more than it did previously, possibly fewer, as long as only one role is being updated (the typical case)14:25
mordredcorvus: that's amazing14:26
corvusthis is only because we're adding a bunch at once14:26
AJaegercorvus: agreed14:26
corvusalso, if we don't like this approach, we can go back to testing all these roles in 2 jobs... i mostly wanted to see whether separating them as job-per-role was cleaner (made it more obvious that each was tested)14:28
corvusbut we will still have jobs that test multiple roles (for instance, the registry job tests a bunch of roles, because they all work together in sequence)14:29
mordredcorvus: so far I like the separation14:31
openstackgerritJeff Liu proposed zuul/zuul-operator master: Create tasks that spin up kubernetes and run the operator  https://review.opendev.org/66802914:31
corvusoh, i forgot to parent these to base-minimal, so one more rev on both patches14:32
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add base integration roles  https://review.opendev.org/66806114:33
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add multi-node integration roles  https://review.opendev.org/66876714:33
openstackgerritMerged zuul/zuul master: model: add cleanup-run to the job configuration  https://review.opendev.org/66188014:43
AJaegertristanC, I see no new documentation on https://review.opendev.org/#/c/661880/ . could you add some, please?14:47
corvusAJaeger: it's a followup14:49
corvus3 patch series, and all 3 are approved14:49
corvushttps://review.opendev.org/662147 is docs14:49
corvus(we held the series until everything was ready)14:50
*** igordc has joined #zuul14:50
openstackgerritMerged zuul/zuul master: executor: run cleanup playbook on stop  https://review.opendev.org/66188114:51
AJaegercorvus: good! Sorry, tristanC for missing that14:51
corvusi'd rather too many people check for docs than not enough :)14:52
AJaeger;)14:54
AJaegercorvus: https://review.opendev.org/668061 is failing with post_failures now, the rebase to base-minimal broke some jobs14:59
openstackgerritJeff Liu proposed zuul/zuul-operator master: Create tasks that spin up kubernetes and run the operator  https://review.opendev.org/66802915:02
*** mattw4 has joined #zuul15:06
*** mattw4 has quit IRC15:09
*** mattw4 has joined #zuul15:09
*** mattw4 has quit IRC15:11
*** mattw4 has joined #zuul15:12
openstackgerritMerged zuul/zuul master: docs: add cleanup-run documentation  https://review.opendev.org/66214715:16
corvusSpamapS: ^ cleanup landed!15:16
*** jamesmcarthur has joined #zuul15:21
*** jamesmcarthur has quit IRC15:21
*** jamesmcarthur has joined #zuul15:22
SpamapSssssaaahhhweeeet15:24
SpamapScorvus:all my dreams are coming true15:24
AJaeger;)15:25
SpamapSnow I just need EC2 nodepool builder and zuul pushes.15:25
*** hashar has joined #zuul15:30
mordredSpamapS: zuul push solves the universe15:38
mordredwell - except that github doesn't let apps push to protected branches, IIRC - so that'll need to be sorted out15:39
*** jangutter has quit IRC15:42
*** rfolco has quit IRC15:47
*** jeliu_ has quit IRC16:05
SpamapSmordred:well if you wanted to solve the universe you should have put that in the requirements!16:06
SpamapSFor the apps-can't-push thing.. I think the right thing to do is to yell at jlk16:06
mordredSpamapS: you mean nudge him with lovingkindness I'm sure16:08
SpamapSthat's not what yell means?16:08
SpamapSthis explains so much16:08
*** hwangbo has joined #zuul16:25
*** jeliu_ has joined #zuul16:31
openstackgerritKevin Carter (cloudnull) proposed zuul/zuul-jobs master: Add python3 detection to the sphinx role  https://review.opendev.org/66895416:33
*** jeliu_ has quit IRC16:35
*** jeliu_ has joined #zuul16:38
openstackgerritKevin Carter (cloudnull) proposed zuul/zuul-jobs master: Add python3 detection to the sphinx role  https://review.opendev.org/66895416:43
openstackgerritKevin Carter (cloudnull) proposed zuul/zuul-jobs master: Add python3 detection to the sphinx role  https://review.opendev.org/66895416:44
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add base integration roles  https://review.opendev.org/66806116:47
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add multi-node integration job  https://review.opendev.org/66876716:47
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add a script to make platform-specific versions of jobs  https://review.opendev.org/66895516:47
*** hashar has quit IRC16:48
corvusAJaeger, mordred: there's some amount of interaction between the roles in those test jobs, so i think it's going to be easiest to stick with them as two jobs rather than job-per-role.  but i think we can look at moving some of those roles out separately later -- probably starting with fetch-zuul-cloner (it should no longer need to be run under base-minimal, and i doubt anything relies on it).16:49
corvusAJaeger, mordred: i did update the multinode roles so they don't depend on base-minimal or the playbook from the other job anymore -- i don't think that should be necessary.  so at least that's a simpler job.16:49
corvus(but even for it, at least 2 of its roles interact)16:50
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add base role integration jobs  https://review.opendev.org/66806116:52
openstackgerritJeff Liu proposed zuul/zuul-operator master: Create tasks that spin up kubernetes and run the operator  https://review.opendev.org/66802916:52
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add multi-node integration jobs  https://review.opendev.org/66876716:52
*** jamesmcarthur has quit IRC16:56
*** jamesmcarthur has joined #zuul16:57
openstackgerritKevin Carter (cloudnull) proposed zuul/zuul-jobs master: Add python3 detection to the sphinx role  https://review.opendev.org/66895416:59
*** jamesmcarthur has quit IRC17:02
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add test job for ensure-sphinx  https://review.opendev.org/66895817:03
*** openstackgerrit has quit IRC17:04
*** openstackgerrit has joined #zuul17:13
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add a script to make platform-specific versions of jobs  https://review.opendev.org/66895517:13
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add base role integration jobs  https://review.opendev.org/66806117:13
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add multi-node integration jobs  https://review.opendev.org/66876717:13
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add test job for ensure-sphinx  https://review.opendev.org/66895817:20
*** jamesmcarthur has joined #zuul17:28
*** hwangbo has quit IRC17:44
*** jamesmcarthur has quit IRC17:44
*** hwangbo has joined #zuul17:44
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add base role integration jobs  https://review.opendev.org/66806117:44
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add multi-node integration jobs  https://review.opendev.org/66876717:44
*** igordc has quit IRC17:45
*** sshnaidm|ruck is now known as sshnaidm|afk17:53
*** rfolco has joined #zuul17:56
*** buumi has joined #zuul18:04
buumiHey, i was reading briefly about Zuul, do you think it would help with our situation. We've project A and B. If specific files are changed in Project A, our integration tests will fail. When Project A submodule in repository of Project B is updated, tests should pass again. Would Zuul CI make it possible to do so that submodule update of Project B18:09
buumiwould depend on change to Project A and both commits would be merged if integration tests pass after submodule update?18:09
AJaegermordred: want to review https://review.opendev.org/#/c/668955 for zuul-jobs, please?18:11
clarkbbuumi: yes with one current behavior requirement (simply because the other thing hasn't yet been implemented) zuul does a strict sequencing of changes. This means if A is queued before B and passes it will merge regardless of B's result. But B if queued after A will only merge if B works with A's changes18:11
clarkbthere has been discussion of doing atomic sets of merges across projects but that work hasn't been done yet. So if you don't need that you should be in a good spot18:13
buumiOk that's fine. And if A fails but later queued B will pass, it will then merge both A and B?18:13
clarkbbuumi: as long as B properly expresses its dependency on A zuul will require A pass and merge first yes18:13
clarkbthere are dependency tags you can set in commit messages to express that18:13
SpamapSbuumi:one caveat: submodules do not work well with Zuul18:14
SpamapSI've been down that road before, and it was extremely painful.18:14
clarkbSpamapS: thats largely a matter of updating the base jobs though right?18:14
clarkb(I mean I agree its painful, but should also be doable)18:14
buumiCool, i was worried if B fails to merge if A first fails or it just merges B, but that sounds fine. Need to experiment with it then18:15
SpamapSclarkb: In theory yes. submodules are just a hidden config file with hashes. But in practice, this has been a fail every time I touched it.18:15
clarkbSpamapS: I think that has more to do with submodules than with zuul :)18:15
buumiWe also had in the past rebase-submit strategy which i guess would cause some issues with submodules since commit SHA might change, but luckily now that's not an issue anymore18:16
SpamapSWhat did work, was just assembling a directory from two git working copies, with no submodules involved.18:16
SpamapSbuumi:I'd highly recommend that you do some basic tests. Submodules are rigid in the way they relate to git hashes, and Zuul is pretty opinionated about how it wants to assemble your git repos.18:17
SpamapS(And, IMO, just abandon submodules. They're probably not realy helping you)18:18
buumiOk, that's something to test then, thanks for help both of you!18:19
clarkbya I think the bit that is missing is updating the submodule hashes on related projects based on their hashes in the adjacent zuul managed repos18:19
SpamapSgit clone A ; git clone B ; ln -s ../A B/A18:19
SpamapSfar more predictable18:19
SpamapSclarkb:yeah, we had tried to spike on that a berzillion years ago (pirate cloud days) but IIRC there were enough problems we just started ripping submodules out and doing symlinks.18:20
clarkbSpamapS: were you doing it at the job level or in the zuul mergers?18:20
SpamapSjob18:20
*** mattw4 has quit IRC18:21
clarkbhrm I wouldn't expect that to be too difficult since zuul doesn't say "this is the sha you should have" so you could make a quick update based on the on disk repo state18:21
clarkbwas much harder in the past with v2 when zuul said "here is your sha(s)"18:21
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add base role integration jobs  https://review.opendev.org/66806118:21
SpamapSthe mergers were doing fine delivering the state of the repos. But that was one problem in particular. If somebody was submitting an update to the submodules.. what do we do? Because we don't want to test with the hashes they just submitted, but with the git state Zuul prepared.18:21
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add multi-node integration jobs  https://review.opendev.org/66876718:21
clarkbSpamapS: ah I think the problem there is/was github :P Gerrit will auto update those shas for you18:22
SpamapSso once we started ignoring the hashes.. we wondered "what are these even doing for us?"18:22
clarkbso you can test the future and never manually update the hashes18:22
SpamapSno it was on Gerrit18:22
SpamapSthese were not Gerrit auto submodules18:22
SpamapS(those had other problems ;)18:22
SpamapSanyway, I'm pretty sour on submodules. It's a misfeature of git, IMO.18:23
clarkbyup I agree and I think finally many others are starting to see it tht way too (golang doesn't use submodules for vendoring anymore aiui for example)18:24
*** buumi has quit IRC18:25
openstackgerritKevin Carter (cloudnull) proposed zuul/zuul-jobs master: Update the sphinx python package sets  https://review.opendev.org/66895418:28
SpamapSclarkb:yeah, golang has almost gotten to sane dependency management now. ;)18:32
*** hashar has joined #zuul18:46
*** themroc has quit IRC18:49
*** mattw4 has joined #zuul18:54
*** jeliu_ has quit IRC18:54
*** jeliu_ has joined #zuul19:00
*** jeliu_ has quit IRC19:05
*** igordc has joined #zuul19:05
*** mattw4 has quit IRC19:30
*** igordc has quit IRC19:31
*** mattw4 has joined #zuul19:41
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add base role integration jobs  https://review.opendev.org/66806119:45
*** mattw4 has quit IRC19:57
*** mattw4 has joined #zuul19:57
openstackgerritJeff Liu proposed zuul/zuul-operator master: Create tasks that spin up kubernetes and run the operator  https://review.opendev.org/66802920:40
* fungi is not here at the moment, but wholeheartedly agrees there's little practical benefit to submodules over just having multiple git repos and some instruction to users to clone or symlink repo b inside repo a20:41
*** igordc has joined #zuul21:14
*** pcaruana has quit IRC21:16
*** rfolco has quit IRC21:22
portdirecthey - is there is spec of proposal re `zuul/zuul-operator`21:43
portdirectsound interesting from the name, im starting to look seriously at deploying zuul in k8s21:44
clarkbportdirect: https://review.opendev.org/#/c/659180/21:44
portdirectclarkb: thanks dude! will read now :D21:44
portdirectat 1st pass this looks really nice - totally makes sense to reuse ansible operator here21:49
portdirectmy timeline means I'm gonna have to forge ahead with a `dumb` helm chart for now, but will certainly try and contribute in any way i can to this effort21:49
*** rlandy is now known as rlandy|bbl21:58
SpamapSportdirect: several of us have gone the "dumb" route (not helm, but.. one-off non-reusable k8s deployment)22:33
SpamapSLooking forward to the day when I replace my janky scripts with an operator. :)22:34
portdirectyeah - it really seems like an event driven application like zuul would benefit from the approach22:34
portdirectdo you have an example of a k8s deployment of zuul atm, that i could use for 'ahem' inspiration?22:35
portdirectin the meantime i was planning on just k8s'izing these: https://opendev.org/zuul/zuul/src/branch/master/doc/source/admin/examples/docker-compose.yaml22:37
*** tosky has quit IRC22:38
SpamapSportdirect: Mine is very environment specific, so sharing it wouldn't be productive23:02
portdirectGotcha, thanks SpamapS23:04
*** mattw4 has quit IRC23:21
openstackgerritJames E. Blair proposed zuul/zuul master: Fix multi-tenant caching of extra config files  https://review.opendev.org/66900823:34
openstackgerritJames E. Blair proposed zuul/zuul master: Fix multi-tenant caching of extra config files  https://review.opendev.org/66900823:37
*** saneax has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!