Tuesday, 2019-12-10

corvusannouncement emails sent00:00
corvusi'll work on the ansible email tomorrow...00:02
*** jamesmcarthur has joined #zuul00:28
*** Goneri has joined #zuul00:29
openstackgerritIan Wienand proposed zuul/nodepool master: Also build sibling container images  https://review.opendev.org/69739300:30
openstackgerritIan Wienand proposed zuul/nodepool master: [wip] move openstack testing to use containerised daemon  https://review.opendev.org/69346400:30
*** sshnaidm is now known as sshnaidm|afk00:35
*** rf0lc0 has joined #zuul00:51
*** rfolco has quit IRC00:51
SpamapScan somebody point me at an example of a jobs on a single project having a parent/child relationship (as in, 1 job runs, and then if it passes, the second one does)?00:52
SpamapSDocs don't seem to spell this out, only thing I can find is provides/requires... but I feel like there's a more natural way to do it.00:52
* SpamapS already forgetting how to configure zuul :-/00:53
corvusSpamapS: you want this one: https://zuul-ci.org/docs/zuul/user/config.html#attr-job.dependencies00:54
*** Goneri has quit IRC00:55
SpamapSthe elusive obvious01:00
SpamapScorvus:thanks, I am sure I skimmed past it 4 times01:00
corvusyeah, i think we need to add that to to the faq01:01
corvussomething like that should be more findable, and ideally be part of some narrative docs too01:01
jamesmcarthurYo Zuul! I'd like to tweet about the new Zuul release - congrats all btw!!01:03
jamesmcarthurAny chance I can get a rough quote on what it means in plain-speak and perhaps what's coming down the pike next?01:04
*** jamesmcarthur has quit IRC01:07
corvusjamesmcarthur: actually, this relase was mostly to flush out the queue so that we can make 2 more releases over the next couple of days adding and removing support for ansible versions... i think if you hold off for another day or 2, you can talk about adding support for ansible 2.9.  that's more interesting, and that way it doesn't get buried in 2 other releases.01:08
*** mhu has quit IRC01:15
*** jamesmcarthur has joined #zuul01:24
*** jamesmcarthur has joined #zuul01:24
jamesmcarthurApologies. I inadvertently disconnected, but clarkb gave me the scoop01:25
jamesmcarthurcorvus: let me know a bit ahead of time when the next release will be and we can tee up some tweets and make sure we're all in agreement01:25
*** jamesmcarthur has quit IRC01:32
*** igordc has quit IRC02:24
*** dmellado has quit IRC02:45
*** dmellado has joined #zuul02:48
*** bhavikdbavishi has joined #zuul02:54
*** bhavikdbavishi1 has joined #zuul02:57
*** bhavikdbavishi has quit IRC02:58
*** bhavikdbavishi1 is now known as bhavikdbavishi02:58
*** jamesmcarthur has joined #zuul03:40
ianwmnaser: https://zuul.opendev.org/t/zuul/build/7cf1dc94536f4c9c9bd59154782eba00/log/docker/nodepool_nodepool-builder_1.txt03:43
ianwso that's a nodepool-builder container building an image03:43
ianwdoesn't work with the full functional test yet, but it's a start :)03:44
mnaserianw: yay. I’m clearing out some stuff on my hand to be able to help out a bit more03:44
mnaserI’ve had a few interesting projects take up my time03:44
ianw:) may you live in interesting times03:45
*** jamesmcarthur has quit IRC04:32
*** raukadah is now known as chkumar|ruck05:23
*** saneax has joined #zuul06:00
*** jamesmcarthur has joined #zuul06:01
*** jamesmcarthur has quit IRC06:06
openstackgerritTobias Henkel proposed zuul/zuul master: enqueue: make trigger deprecated  https://review.opendev.org/69544606:34
*** sanjayu_ has joined #zuul06:37
*** saneax has quit IRC06:40
*** sanjayu__ has joined #zuul06:53
*** sanjayu_ has quit IRC06:55
*** themroc has joined #zuul07:26
*** pcaruana has joined #zuul07:39
openstackgerritMerged zuul/nodepool master: Dockerfile: add user to shadow file too  https://review.opendev.org/69364707:46
*** tosky has joined #zuul08:11
*** sanjayu__ has quit IRC08:19
*** sanjayu__ has joined #zuul08:19
*** jcapitao has joined #zuul08:21
*** hashar has joined #zuul08:43
*** jpena|off is now known as jpena08:53
*** sshnaidm|afk is now known as sshnaidm08:58
*** mhu has joined #zuul09:21
sshnaidmhi, how can I define job so that I can use it in different projects?09:24
*** spsurya has joined #zuul09:25
*** avass has joined #zuul09:32
openstackgerritFabien Boucher proposed zuul/zuul master: Pagure: remove connectors burden and simplify code  https://review.opendev.org/69613409:59
openstackgerritMatthieu Huin proposed zuul/zuul master: authentication config: add optional max_token_age, skew  https://review.opendev.org/64240810:04
*** bhavikdbavishi has quit IRC10:12
openstackgerritMerged zuul/zuul master: enqueue: make trigger deprecated  https://review.opendev.org/69544610:30
openstackgerritFabien Boucher proposed zuul/zuul master: Pagure - Refresh token and recheck sign when token regenerated  https://review.opendev.org/69820810:33
*** hashar has quit IRC10:48
*** jcapitao is now known as jcapitao|afk11:07
*** rf0lc0 has quit IRC11:40
*** bhavikdbavishi has joined #zuul12:17
*** bhavikdbavishi1 has joined #zuul12:20
*** bhavikdbavishi has quit IRC12:22
*** bhavikdbavishi1 is now known as bhavikdbavishi12:22
*** jpena is now known as jpena|lunch12:34
openstackgerritAlbin Vass proposed zuul/nodepool master: Keys must be defined for host-key-checking: false  https://review.opendev.org/69802912:34
*** spsurya has quit IRC12:35
avass^ that should fail since it only has the test without the fix12:35
avassDon't really like having to add host_key_checking as an argument to the test since it already has that information in the config file but couldn't figure out a nice way to access it.12:37
openstackgerritAlbin Vass proposed zuul/nodepool master: Keys must be defined for host-key-checking: false  https://review.opendev.org/69802912:48
*** jcapitao|afk is now known as jcapitao12:59
*** sanjayu__ has quit IRC13:01
*** rlandy has joined #zuul13:01
*** hashar has joined #zuul13:06
*** mhu has quit IRC13:07
*** jamesmcarthur has joined #zuul13:15
*** jamesmcarthur has quit IRC13:28
*** jamesmcarthur has joined #zuul13:30
fungisshnaidm: zuul provides ansible variables so you can identify what project triggered a build, and also provides correct source code state for the triggering repository, the repositories containing the job playbooks, and any repositories listed as required-projects, so making use of the context and source code zuul provides is most of what you need to think about in your cross-repository/shared job design13:31
fungialso where that job is defined (trusted config repo vs normal untrusted repo) and what it contains (secrets in particular) can influence whether that job is able to be shared and whether it can be run in specific sorts of pipelines or whether it can speculatively test changes to the job components13:33
*** jamesmcarthur has quit IRC13:34
fungianyway, depending on what you want to do it can be trivial or quite complicated13:34
*** jpena|lunch is now known as jpena13:37
*** Goneri has joined #zuul13:43
*** jamesmcarthur has joined #zuul13:46
avassuuh, I have a strange problem. A project that just got added isn't showing up in zuul after a full-reconfig and it's not reporting any errors. An ideas?13:51
avassIt shouldn't be a gerrit access problem since it uses the same rights as two other projects that are working13:52
fungiavass: can you elaborate on what you mean when you say "reporting errors" (bell icon on the web dashboard? scheduler daemon log?) and "showing up" (projects tab in the web dashboard? something else?)14:00
*** chkumar|ruck is now known as raukadah14:26
Petar_THi all.14:34
Petar_TI've a question on project dependencies and speculative merging.14:34
*** pcaruana has quit IRC14:36
*** hashar has quit IRC14:38
Petar_TIf you have projects Foo and Bar, and the former depends on the latter and they share a queue, if Bar currently has a running pipeline for a commit (let's call it A') and the pipeline for Foo is enqueued too, assuming that the dependency is only expressed via a required-projects entry, which commit of Bar will Zuul check out for Foo? HEAD on14:38
Petar_Tmaster, or A'?14:38
*** themroc has quit IRC14:55
openstackgerritMerged zuul/nodepool master: Aws cloud-image is referred to from pool labels section  https://review.opendev.org/69799814:58
*** igordc has joined #zuul14:58
*** pcaruana has joined #zuul15:01
*** jamesmcarthur has quit IRC15:03
tristanCPetar_T: iiuc it depends of the pipeline or Foo commit message15:03
tristanCPetar_T: if it's a dependent pipeline, e.g. gate, and if foo and bar share a queue and job have required-project, then when foo enter the gate, it will have A' in it's workspace15:04
tristanCPetar_T: if it's an indepdendent pipeline, e.g. check, then foo can have A' in its workspace if foo commit message mensions a 'Depends-On: https://location-of/A''15:05
fungiprojects don't really depend on other projects in zuul, jobs can depend on projects (meaning zuul should prepare and provide copies of them in the workspace), and changes can depend on other changes (either explicitly via depends-on commit message footers or implicitly based on the order in which they're enqueued into a dependent pipeline)15:05
openstackgerritMatthieu Huin proposed zuul/zuul master: authentication config: add optional token_expiry  https://review.opendev.org/64240815:09
openstackgerritMatthieu Huin proposed zuul/zuul master: [DNM][WIP] admin REST API: zuul-web integration  https://review.opendev.org/64353615:09
*** jamesmcarthur has joined #zuul15:15
*** jamesmcarthur has quit IRC15:15
*** jamesmcarthur_ has joined #zuul15:15
Petar_TtristanC: Thanks. That makes sense.15:16
Petar_Tfungi: Roger that. I'm still getting my terms straight.15:19
fungicool, mostly making sure you've got a grasp of the overall data model15:20
*** Petar_T has quit IRC15:24
pabelangerIs there an easy way to query zuul to check if a specific label is used by a job?15:33
avassfungi: no errors in the dashboard or when running zuul tenant-conf-check15:33
pabelangerWanting to remove one in nodepool, and try to ensure now jobs are using it15:33
avassfungi: it got fixed by restarting zuul completely15:33
fungiavass: did you look in the scheduler daemon log? could have been reporting a reconfiguration failure there15:38
openstackgerritMatthieu Huin proposed zuul/zuul master: authentication config: add optional max_validity_time, skew  https://review.opendev.org/64240815:41
openstackgerritMatthieu Huin proposed zuul/zuul master: [DNM][WIP] admin REST API: zuul-web integration  https://review.opendev.org/64353615:42
clarkbpabelanger: if you push removal of nodesets zuul will sanity check that. For labels its tricker because they arent verified config between zuul and nodepool.15:44
clarkbpabelanger: you can prpbably ask zuul for a list of all nodesets then scan that for your label?15:44
corvusi don't think we have a nodesets endpoint yet :(15:45
corvuspabelanger: but you can use a script like tobiash or tristanC were discussing yesterday to crawl the jobs+project pipelines to check for labels15:45
corvusi think they posted links if you check eavesdrop15:46
avassfungi: I did, but couldn't find anything15:46
corvusclarkb, pabelanger, fungi: how does this message look? https://etherpad.openstack.org/p/fyn4FVFo6v15:49
*** sugaar has quit IRC15:51
clarkbcorvus: lgtm15:53
corvusi'll send that to -announce15:53
fungislight vagueness in the first sentence15:54
fungimight help to say that ansible 2.5 has been unsupported by its authors or community or something for quite some time15:54
fungisince i first started reading it as indicating ansible 2.5 has been unsupported in zuul for quite some time, which then conflicted with the second part of the sentence15:55
fungithe rest lgtm15:56
*** Petar_T has joined #zuul15:58
Shrewsfolks might want a pointer to how to set the ansible version15:58
mordredcorvus: the words you say about adding 2.9 before removing 2.5 giving the widest range of possibilities makes me think that we should be sure to incorporate that pattern when we make our policy15:59
corvusShrews, fungi: updated, thanks; mordred ++16:00
fungilgtm!16:00
Shrewsditto16:01
*** jcapitao is now known as jcapitao|afk16:02
corvussent16:04
pabelanger+1 to announce16:08
Shrewscorvus: should we send a notice to our zuul users as well informing them of the default version update? we should probably point to the 2.8 porting guide so they can see if they need to update their job playbooks (https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.8.html)16:09
Shrewsi'm not sure if there are any changes in that guide that might affect them. scanning now...16:09
*** igordc has quit IRC16:11
Shrewssome of those changes seem noteworthy16:12
openstackgerritPaul Belanger proposed zuul/zuul master: Switch ansible_default to 2.8  https://review.opendev.org/67669516:12
openstackgerritPaul Belanger proposed zuul/zuul master: Support Ansible 2.9  https://review.opendev.org/67485416:12
pabelangerk, that should rebase away 2.5 version drop16:13
openstackgerritPaul Belanger proposed zuul/zuul master: Remove support for ansible 2.5  https://review.opendev.org/65043116:16
pabelangerzuul-maint: order of ansible 2.9 patchset updated^ Should match the email corvus just sent16:17
mordredpabelanger: ++16:19
mordredpabelanger: there's an issue on the drop-2.5 patch - otherwise stack looks good16:23
*** jcapitao|afk is now known as jcapitao16:24
Shrewspabelanger: mordred: none of those update the docs for https://zuul-ci.org/docs/zuul/user/config.html#attr-job.ansible-version that i can see16:24
pabelangerShrews: that should be generated dynamically, via ansible-manage16:26
pabelangermordred: looking16:26
Shrewspabelanger: oh, neat! i didn't realize that16:27
openstackgerritMonty Taylor proposed zuul/zuul master: Extract an abstract base Parser class  https://review.opendev.org/54561016:28
mordredShrews: ^^ I fixed your review comments16:28
* Shrews does not remember what that was16:28
openstackgerritPaul Belanger proposed zuul/zuul master: Switch ansible_default to 2.8  https://review.opendev.org/67669516:29
openstackgerritPaul Belanger proposed zuul/zuul master: Support Ansible 2.9  https://review.opendev.org/67485416:29
openstackgerritPaul Belanger proposed zuul/zuul master: Remove support for ansible 2.5  https://review.opendev.org/65043116:29
Shrewsmordred: oh, that one. i was stupid on my comments16:29
Shrewsmordred: i had left a +216:29
pabelangerunrelated, is there open review to update install-docker role to support centos-8?16:30
corvusjamesmcarthur_: see http://lists.zuul-ci.org/pipermail/zuul-announce/2019-December/000060.html  3.13.0 is coming up next, that's the one that adds ansible 2.9 support16:30
*** mhu has joined #zuul16:30
corvusmordred: lgtm, but holding +W until 3.1416:32
jamesmcarthur_corvus: thanks!16:33
mordredShrews: well, your headline comment was bogus - but the inlines seemed still accurate?16:33
corvusmordred: 676695 could use another +316:33
mordreddone!16:34
*** sugaar has joined #zuul16:36
*** sgw has joined #zuul16:39
*** jpena is now known as jpena|brb16:45
*** sanjayu__ has joined #zuul16:47
openstackgerritMatthieu Huin proposed zuul/zuul master: [DNM][WIP] admin REST API: zuul-web integration  https://review.opendev.org/64353617:08
*** sanjayu__ has quit IRC17:18
*** jpena|brb is now known as jpena17:24
openstackgerritMerged zuul/zuul master: Switch ansible_default to 2.8  https://review.opendev.org/67669517:24
*** tosky has quit IRC17:27
*** rlandy is now known as rlandy|brb17:36
openstackgerritMerged zuul/zuul master: Support Ansible 2.9  https://review.opendev.org/67485417:40
corvusdo we want to do an opendev restart sanity check before releasing that, or just go ahead and tag it?17:41
clarkbI'm happy to help with another restart, but will defer to others on their comfort level for deciding if that is necessary17:44
corvuspabelanger: ^ do you have thoughts about the risk level there?17:45
*** rlandy|brb is now known as rlandy17:45
pabelangerI'd be okay with a restart first, and smoke test of 2.9 job17:48
corvuscool, i'll get that started over in -infra17:48
pabelangerIf we can confirm say tox jobs works, we can then propose fixes to zuul-job as needed17:48
pabelanger++17:48
*** sshnaidm is now known as sshnaidm|afk17:50
*** jcapitao has quit IRC17:55
*** rf0lc0 has joined #zuul18:22
*** jamesmcarthur_ has quit IRC18:26
*** jamesmcarthur has joined #zuul18:26
*** jamesmcarthur has quit IRC18:36
*** jpena is now known as jpena|off18:42
*** bhavikdbavishi has quit IRC19:25
openstackgerritPaul Belanger proposed zuul/zuul master: Update tox-docs to ansible 2.9 job  https://review.opendev.org/69831619:30
fungiso i guess per the discussion in #openstack-infra we need something like a .encode() on these two: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/tox/library/tox_install_sibling_packages.py#L130-L13119:35
*** igordc has joined #zuul19:39
fungior maybe a default encoding a ways up on line 124 would solve it19:40
*** gmann is now known as gmann_afk19:40
pabelangercorvus: wonder if we should add an upgrade note about default switch to auto for 2.8 jobs19:50
corvuspabelanger: not a bad idea19:50
openstackgerritPaul Belanger proposed zuul/zuul master: Add upgrade note about ansible_python_interpreter  https://review.opendev.org/69831820:07
*** hashar has joined #zuul20:07
pabelangercorvus: fungi: clarkb: how is ^ for info to user20:07
pabelangeralso20:09
pabelangerhttps://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_047/698316/1/check/tox-pep8/0476433/ looks to have worked properly under ansible 2.920:09
pabelanger2019-12-10 19:57:05.784214 | Ansible Version: 2.9.220:09
pabelanger\o/20:09
*** pcaruana has quit IRC20:16
clarkbfungi: pabelanger I think the answer to that comment is yes20:20
corvuspabelanger: are you planning on looking into the tox-siblings issue?20:24
corvusfungi: or you?20:24
pabelangercorvus: I can, but will need a little time, maybe tomorrow20:24
fungicorvus: i can. just not sure the best way to test it. set an ansible version override on a change which runs a job we saw failing in opendev?20:24
pabelangerfungi: yah, you can set ansible-version: 2.820:25
fungii have a feeling that role itself needs some tests added to the zuul-jobs repo20:25
clarkbzuul tenant is still 2.8 default right?20:25
clarkbso should just be tested that way on push?20:25
pabelangeryes, 2.820:25
openstackgerritPaul Belanger proposed zuul/zuul master: Add upgrade note about ansible_python_interpreter  https://review.opendev.org/69831820:34
fungimore generally though, should we have something at https://opendev.org/zuul/zuul-jobs/src/branch/master/zuul-tests.d/python-jobs.yaml which exercises the tox-siblings role?20:35
*** Goneri has quit IRC20:38
corvusyeah, i think that would be nice.  maybe just a job with zuul and nodepool as siblings?20:42
corvuscould just inherit from tox-docs, then add both of those as required projects\20:45
corvusthat might be all that's necessary20:45
openstackgerritPaul Belanger proposed zuul/zuul master: Add additional info for executor.merge_jobs release note  https://review.opendev.org/69832420:59
pabelangertristanC: ^ update to your reno note about executor.merge_jobs. Had to dig into git logs to understand more21:01
*** hashar has quit IRC21:59
clarkbfungi: for fixing tox siblings I think we should change the mode of the file open instaed of doing explicit encodes22:01
clarkbfungi: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/tox/library/tox_install_sibling_packages.py#L124 becomes (mode='w', delete=False)22:01
clarkbthe default mode is w+b22:01
clarkbI think that reduces the python2 vs python3 checking we would have to do?22:02
corvusi'm starting on a gce nodepool driver with the hope that we'll be able to use it for the gerrit project22:14
pabelangerhow does the governance of gerrit project work? I'm assuming that is still mostly out of google?22:18
corvuspabelanger: there's a committee with google holding one vote more than 50%22:19
pabelangerneat! I never really look into stateholders of the project, aside from google being involved22:20
corvusgoogle is about half of the contributions at this point22:20
pabelangerthanks for info!22:21
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Fix python3 compat in tox siblings handling  https://review.opendev.org/69833522:44
clarkbfungi: pabelanger corvus ^ something like that maybe? The unittest confirms this is a fix for that particular function. Not sure if I got the job stuff correct though22:45
corvusclarkb: that looks like it should do it22:47
*** mhu has quit IRC22:49
*** gmann_afk is now known as gmann23:06
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Fix python3 compat in tox siblings handling  https://review.opendev.org/69833523:07
clarkbI think ^ should work now23:07
clarkbI used the wrong tox role var23:07
*** rlandy is now known as rlandy|bbl23:13
clarkbok now it fails due to lack of bindeping23:20
fungiyeah, needs libre2-dev i think23:25
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Fix python3 compat in tox siblings handling  https://review.opendev.org/69833523:27
clarkbfungi: yup23:27
fungioh, indeed, we just weren't doing the thing with the stuff23:30
clarkbit passes now23:33
clarkbwhich implies it works for python3 in the whole role too23:33
clarkband the unittest covers that specific subset of functioanlity23:33
*** avass has quit IRC23:33
*** Goneri has joined #zuul23:35
fungitests.unit.test_v3.TestDiskAccounting.test_disk_accountant_kills_job under py37 and tests.unit.test_v3.TestPrePlaybooks.test_pre_playbook_fail under py35 both timed out "waiting for Zuul to settle" on 69831823:36
fungiand that was merely a release notes addition, so i don't think the change is to blame23:39
*** Goneri has quit IRC23:40
*** sgw has quit IRC23:40
*** rf0lc0 has quit IRC23:47
openstackgerritJames E. Blair proposed zuul/nodepool master: WIP: Add Google Cloud provider  https://review.opendev.org/69834223:47
corvusthat's super-wip -- there are a few methods to implement still.  but it does implement the basic lifecycle -- create/delete23:48
*** rf0lc0 has joined #zuul23:49
clarkbcorvus: https://review.opendev.org/#/c/698335/ should be ready now23:49
corvusclarkb: +3 thanks!23:51
corvusmaybe tomorrow we'll remove the pin and see how things look, then make a release?23:51
corvusafter getting those relnotes changes merged23:51
clarkb++23:51

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!