Tuesday, 2020-05-26

« Monday, 2020-05-25 Index Wednesday, 2020-05-27 »
*** Goneri has quit IRC00:01
*** jamesmcarthur has quit IRC00:06
*** jamesmcarthur has joined #zuul00:22
*** jamesmcarthur has quit IRC00:30
*** jamesmcarthur has joined #zuul00:30
*** guillaumec has quit IRC00:30
*** iurygregory has quit IRC00:49
*** jamesmcarthur has quit IRC01:06
*** jamesmcarthur has joined #zuul01:07
*** jamesmcarthur has quit IRC01:08
*** jamesmcarthur has joined #zuul01:08
*** jamesmcarthur has quit IRC01:27
*** swest has quit IRC01:33
*** jamesmcarthur has joined #zuul01:37
*** swest has joined #zuul01:48
*** rfolco|rover has joined #zuul01:49
*** rfolco|rover has quit IRC01:54
*** jamesmcarthur has quit IRC01:58
*** jamesmcarthur has joined #zuul01:59
*** jamesmcarthur has quit IRC02:04
*** jamesmcarthur has joined #zuul02:37
*** jamesmcarthur has quit IRC02:49
*** cloudnull has quit IRC03:02
*** jamesmcarthur has joined #zuul03:17
*** jamesmcarthur has quit IRC03:25
*** jamesmcarthur has joined #zuul03:52
*** jamesmcarthur has quit IRC04:07
*** jamesmcarthur has joined #zuul04:08
*** ysandeep is now known as ysandeep|brb04:08
*** jamesmcarthur has quit IRC04:13
*** evrardjp has quit IRC04:33
*** evrardjp has joined #zuul04:33
*** jamesmcarthur has joined #zuul04:38
*** ysandeep|brb is now known as ysandeep04:38
*** threestrands has joined #zuul04:40
*** jamesmcarthur has quit IRC04:49
*** jamesmcarthur has joined #zuul04:50
*** jamesmcarthur has quit IRC04:55
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: test-base-roles: update include to import_playbook  https://review.opendev.org/73067405:05
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Make gentoo jobs nv  https://review.opendev.org/72864005:23
AJaegeravass, could you review this one, please? ^05:25
*** jamesmcarthur has joined #zuul05:26
openstackgerritMerged zuul/zuul-jobs master: test-base-roles: update include to import_playbook  https://review.opendev.org/73067405:33
*** jamesmcarthur has quit IRC05:38
openstackgerritSagi Shnaidman proposed zuul/zuul-jobs master: WIP Add ansible collection roles  https://review.opendev.org/73036005:49
*** jamesmcarthur has joined #zuul05:50
*** jamesmcarthur has quit IRC06:05
*** jamesmcarthur has joined #zuul06:06
*** jamesmcarthur has quit IRC06:11
*** jamesmcarthur has joined #zuul06:39
avassAJaeger: how come we need to make all gentoo jobs non-voting?06:51
avassmordred: you can use import with conditionals, but I guess that places the conditional on each task inside the import. the only place you really need include_* is for loops etc06:54
avassmordred: and the include should at least be updated to inclued_tasks, include_playbook etc06:54
avassmordred: and bare 'include:' is going away completely in 2.1206:56
avassmordred: oh, wait sorry  I need more coffee. Yes :)06:58
*** jamesmcarthur has quit IRC07:01
AJaegeravass: we use tools/update-test-platforms.py to propagate values and that does not allow single job non-voting and the rest voting for one platform07:03
AJaegeravass: let me change the script ;)07:06
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Make gentoo multinode job nv  https://review.opendev.org/72864007:11
AJaegeravass, ianw, could you review this, please? ^07:11
*** iurygregory has joined #zuul07:22
avassAJaeger: that seems like a better solution so other jobs that are currently working don't stop working :)07:24
AJaegeravass: agreed - but took me a bit ;)07:26
AJaegeravass: that change might be needed for some of yours to pass tests ;)07:27
*** jpena|off is now known as jpena07:27
*** tobiash has quit IRC07:28
*** jamesmcarthur has joined #zuul07:31
*** yolanda has joined #zuul07:32
*** jcapitao has joined #zuul07:34
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Remove install-* roles  https://review.opendev.org/71932207:35
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: bindep: update include to import_tasks  https://review.opendev.org/73066007:37
*** tosky has joined #zuul07:38
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: tox: update include to import_tasks  https://review.opendev.org/73067307:39
*** openstackstatus has quit IRC07:39
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: ensure-bazel: update include to include_tasks  https://review.opendev.org/73067207:40
*** openstackstatus has joined #zuul07:41
*** ChanServ sets mode: +v openstackstatus07:41
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: ensure-package-repositories: update include to include_tasks  https://review.opendev.org/73067107:41
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Rename test install role to ensure-  https://review.opendev.org/73072007:42
*** jamesmcarthur has quit IRC07:45
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: add-build-sshkey: update include to include_tasks  https://review.opendev.org/73067007:45
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Make gentoo multinode job nv  https://review.opendev.org/72864007:45
*** rpittau|afk is now known as rpittau07:46
avassis it possible to check why a driver is disabled somehow?07:50
avassand is it possible to re-enable it?07:50
avassfor some reason our SQL driver is disabled and a restart should get it working, but it would be great if we need to do a complete scheduler restart07:51
openstackgerritBogdan Dobrelya (bogdando) proposed zuul/zuul-jobs master: Support overrideable package_mirror  https://review.opendev.org/73060207:59
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: tox: update include to include_tasks  https://review.opendev.org/73067307:59
*** nils has joined #zuul08:00
*** armstrongs has joined #zuul08:00
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: bindep: update include to include_tasks  https://review.opendev.org/73066008:01
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: add-build-sshkey: update include to include_tasks  https://review.opendev.org/73067008:04
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-subunit-output: update include to import_tasks  https://review.opendev.org/73066808:05
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-subunit-output: update include to import_tasks  https://review.opendev.org/73066808:06
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: ensure-virtualenv: update include to inlude_tasks  https://review.opendev.org/73066908:07
*** tobiash has joined #zuul08:07
*** asaleh_ has joined #zuul08:08
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: ensure-podman: update include to include_tasks  https://review.opendev.org/73066708:08
*** armstrongs has quit IRC08:09
*** hashar has joined #zuul08:09
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: use-docker-mirror: update include to include_tasks  https://review.opendev.org/73066408:10
openstackgerritMerged zuul/zuul-jobs master: Make gentoo multinode job nv  https://review.opendev.org/72864008:12
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: configure-mirrors: update include to include_tasks  https://review.opendev.org/73066608:12
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: persistent-firewall: update include to include_tasks  https://review.opendev.org/73066508:14
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: multi-node-bridge: update include to include_tasks  https://review.opendev.org/73066208:16
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Remove tumbleweed from testing  https://review.opendev.org/73072708:16
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: ensure-pip: update include to import_tasks  https://review.opendev.org/73066108:19
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: ensure-pip: update include to include_tasks  https://review.opendev.org/73066108:20
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Fix deprecation warning from multinode tests  https://review.opendev.org/73047908:21
avassAJaeger, mordred: those should be a bit better now, I untangled them as well so it should be easier to merge them separately08:22
avasszuul-jobs-maint: it's the 26th of may, we should merge https://review.opendev.org/#/c/719322/ to remove the install-* roles completely08:23
AJaegeravass: thanks, I let mordred revisit them08:24
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: ensure-bazel: replace ignore_errors with failed_when  https://review.opendev.org/73073308:39
*** tobiash has quit IRC09:02
*** tobiash_ has joined #zuul09:02
*** ysandeep is now known as ysandeep|lunch09:08
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: ensure-bazel: replace ignore_errors with failed_when  https://review.opendev.org/73073309:16
*** iurygregory has quit IRC09:17
*** iurygregory has joined #zuul09:18
*** SotK has quit IRC09:24
*** tumble has joined #zuul09:25
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: multi-node-bridge: update include to include_tasks  https://review.opendev.org/73066209:37
*** jamesmcarthur has joined #zuul09:42
*** jamesmcarthur has quit IRC09:47
*** hashar has quit IRC09:48
*** tosky__ has joined #zuul09:50
*** tosky is now known as Guest6865809:50
*** tosky__ is now known as tosky09:50
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Add ensure-dnf-copr  https://review.opendev.org/73074309:59
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Add ensure-dnf-copr  https://review.opendev.org/73074310:02
*** ysandeep|lunch is now known as ysandeep10:05
*** threestrands has quit IRC10:05
avasstristanC: how about we do something like that ^ to ensure that dnf copr is present on the machine10:08
*** rpittau is now known as rpittau|bbl10:20
avassor should we just run a dnf install copr since there's not much logic in the role10:31
*** avass has quit IRC10:39
*** avass has joined #zuul10:48
*** jcapitao is now known as jcapitao_lunch11:05
AJaegertristanC, avass, do you know why Software Factory CI is failing at https://review.opendev.org/#/c/719322/ ?11:14
AJaegertristanC: the log file looks unrelated to me11:15
openstackgerritSagi Shnaidman proposed zuul/zuul-jobs master: WIP Add ansible collection roles  https://review.opendev.org/73036011:18
avassAJaeger: the pre-run is getting timed out11:21
avassAJaeger: so the docs are never generated it looks like11:22
*** jpena is now known as jpena|lunch11:30
avassAJaeger, tristanC: maybe we should wait with merging that and make sure it works for softwarefactory ci11:30
*** harrymichal has joined #zuul11:34
openstackgerritTristan Cacqueray proposed zuul/zuul master: requirements: add explicit reference to dateutil  https://review.opendev.org/73065411:37
AJaegeravass: yeah, let's wait for tristanC to comment about 719322, I don't want to break them11:38
tristanCavass: AJaeger: i'm looking now. it seems like we were not testing ensure-sphinx, and something is not working on our side: `bash: line 0: type: msgmerge: not found`11:43
*** jamesmcarthur has joined #zuul11:44
tristanCor rather the command after11:44
AJaegertristanC: "Failed to download metadata for repo 'AppStream'",11:48
tristanCAJaeger: yep, it seems like we had network issue, it seems to be solved now. sorry for the false positive!11:49
AJaegertristanC: glad to have that testing in general ;)11:49
*** jamesmcarthur has quit IRC11:50
AJaegertristanC: want to +2A that change ? ;)11:50
avasstristanC: nice to know it wasn't the change at least :)11:50
AJaegerzuul-jobs maintainer, please review https://review.opendev.org/729407 https://review.opendev.org/730500 https://review.opendev.org/730477 https://review.opendev.org/729520 https://review.opendev.org/72870811:50
openstackgerritMatthieu Huin proposed zuul/zuul master: CLI: Fix errors with the REST client  https://review.opendev.org/72806111:51
openstackgerritMatthieu Huin proposed zuul/zuul master: REST API: fix discrepancies between RPC and REST outputs for autohold  https://review.opendev.org/72807311:51
openstackgerritMatthieu Huin proposed zuul/zuul master: Add simple testing for Zuul CLI & REST API  https://review.opendev.org/72809811:51
openstackgerritMatthieu Huin proposed zuul/zuul master: REST API: improve tenant scoping of autohold, authorizations  https://review.opendev.org/72811811:51
openstackgerritMatthieu Huin proposed zuul/zuul master: CLI: add autohold-info, autohold-delete via REST  https://review.opendev.org/72841011:51
openstackgerritMatthieu Huin proposed zuul/zuul master: REST API: add promote endpoint  https://review.opendev.org/72848911:51
openstackgerritMatthieu Huin proposed zuul/zuul master: CLI: support promote via the REST API  https://review.opendev.org/72890711:51
openstackgerritMatthieu Huin proposed zuul/zuul master: [WIP] separate Zuul CLI, Zuul Web CLI  https://review.opendev.org/72923611:51
*** sshnaidm is now known as sshnaidm|afk11:54
*** rfolco|rover has joined #zuul11:55
*** hashar has joined #zuul11:55
openstackgerritMerged zuul/zuul-jobs master: Remove install-* roles  https://review.opendev.org/71932212:05
*** jcapitao_lunch is now known as jcapitao12:09
openstackgerritMerged zuul/zuul-jobs master: Add option to prefer https/ssl in configure-mirrors  https://review.opendev.org/72940712:09
openstackgerritMatthieu Huin proposed zuul/zuul master: Builds: add "held" attribute  https://review.opendev.org/72899412:12
*** rlandy has joined #zuul12:13
*** cloudnull has joined #zuul12:14
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add container and pod log in the test for ensure-kubernetes role  https://review.opendev.org/72792912:15
*** rpittau|bbl is now known as rpittau12:22
mordredavass: nice! all look great - +2 from me12:25
AJaegermordred: do you have time to review some more, please? https://review.opendev.org/730500 https://review.opendev.org/730477 https://review.opendev.org/729520 https://review.opendev.org/72870812:25
*** SotK has joined #zuul12:26
openstackgerritBogdan Dobrelya (bogdando) proposed zuul/zuul-jobs master: Support overrideable package_mirror  https://review.opendev.org/73060212:27
AJaegerthanks, mordred !12:27
mordredAJaeger: \o/12:28
openstackgerritBogdan Dobrelya (bogdando) proposed zuul/zuul-jobs master: Support overrideable package_mirror  https://review.opendev.org/73060212:28
AJaegermordred: podman uploading is failing on https://review.opendev.org/#/c/730720/ - didn't we fix that?12:29
AJaegermordred: https://zuul.opendev.org/t/zuul/build/70914e2896f14d5191521eb5696fd33512:29
*** rlandy is now known as rlandy|mtg12:31
mordredAJaeger: this seems like it might be a different one: https://zuul.opendev.org/t/zuul/build/70914e2896f14d5191521eb5696fd335/log/builder/podman/buildset_registry.txt#212-22812:33
mordredoh - wait - maybe that is the thing that we fixed12:33
*** jpena|lunch is now known as jpena12:34
avassmordred: no problem, I totally agree to your comments so it was worth fixing :)12:34
mordred\o/12:38
mordredcorvus: ^^ you might want to take a look at that zuul-registry thing. I'm looking at the code in question and it doesn't make sense to me how it would hit a file-not-found where it does - unless you had 2 things calling cat_objects at the same time, but that would also be dangerous in the preceeding while loop. is it perhaps possible that 2 _different_ things are trying to upload and then cat an object that12:41
mordredshare a chunk?12:41
openstackgerritMerged zuul/zuul-jobs master: packer: namespace test jobs correctly  https://review.opendev.org/73050012:41
openstackgerritMerged zuul/zuul-jobs master: ensure-pip: update include to include_tasks  https://review.opendev.org/73066112:43
openstackgerritMerged zuul/zuul-jobs master: ensure-package-repositories: fix loopvar collision  https://review.opendev.org/73047712:51
openstackgerritMerged zuul/zuul-jobs master: Do not interpolate values from tox --showconfig  https://review.opendev.org/72952012:51
openstackgerritMerged zuul/zuul-jobs master: bindep: update include to include_tasks  https://review.opendev.org/73066012:51
openstackgerritMerged zuul/zuul-jobs master: Add python3-devel to bindep  https://review.opendev.org/72870812:51
openstackgerritMerged zuul/zuul-jobs master: ensure-bazel: update include to include_tasks  https://review.opendev.org/73067212:51
AJaegerinfra-root, I see a promote failure in https://zuul.opendev.org/t/zuul/build/f1ac8fabf6b94f0798f0e79edf81f66f; error is "Status code was -1 and not [200]: Request failed: <urlopen error _ssl.c:629: The handshake operation timed out>" in "localhost: Query Zuul API for artifact information"; never seen this before. This is for docs, and other jobs will publish - but wanted to point this out in case12:52
AJaegerthre are problems with zuul.12:52
openstackgerritMerged zuul/zuul-jobs master: Add container and pod log in the test for ensure-kubernetes role  https://review.opendev.org/72792912:55
AJaegeravass: https://review.opendev.org/730671 is now in conflict12:56
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: multi-node-bridge: update include to include_tasks  https://review.opendev.org/73066212:56
avassAJaeger: fixing12:56
mordredavass: in that ^^ you only added "apply" to two of them12:57
mordredavass: a) what's apply? b) does it need to be done to common.yaml too?12:58
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: ensure-package-repositories: update include to include_tasks  https://review.opendev.org/73067112:58
avassmordred: oh :(12:58
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: multi-node-bridge: update include to include_tasks  https://review.opendev.org/73066212:58
mordredavass: and - it seems like maybe the first could be import - I think the jinja expansion should late-bind already12:59
mordredavass: I guess apply is just magic to say to apply those things in to the included tasks?12:59
avassmordred: you can't set keywords directly on include_tasks so you have to do {include_task: ..., apply: {environment: ...}} instead of {include: ... , environment: ...}13:00
avassmordred: yeah, it's for become etc too13:00
avassmordred: and yes common.yaml should probably be imported13:02
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: multi-node-bridge: update include to include_tasks  https://review.opendev.org/73066213:02
avassmordred: and for some reason import and include doesn't have the same syntax. I guess import is more like a block in a different file compared to include :)13:03
AJaegermordred: could you review https://review.opendev.org/#/c/730479/4 as well, please?13:05
AJaegermordred: one more: The stack at https://review.opendev.org/#/c/730322/ could use a good review as well, please13:06
*** sshnaidm|afk is now known as sshnaidm13:10
avassAJaeger: fetch-tox-output doesn't handle envdir correctly yet but I could probably push another change to fix that whenever I get time :)13:12
AJaegeravass: that would be nice ;)13:14
mordredavass: neat!13:17
*** rlandy|mtg is now known as rlandy13:19
avassoh...13:20
avassmordred, AJaeger: we need ansible2.7 for that :(13:24
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: multi-node-bridge: update include to include_tasks  https://review.opendev.org/73066213:26
openstackgerritMerged zuul/zuul-jobs master: tox: update include to include_tasks  https://review.opendev.org/73067313:29
openstackgerritMerged zuul/zuul-jobs master: add-build-sshkey: update include to include_tasks  https://review.opendev.org/73067013:29
corvusmordred, AJaeger: it kind of looks like that lock algorithm wasn't good enough, and it still raced.  we knew it had some problems, but i had hoped the chance of hitting the remaining race conditions would be very small.  i'll take another look at that.13:30
AJaegerthanks, corvus - and good morning to you!13:30
corvusAJaeger: likewise :)13:30
*** rlandy is now known as rlandy|mtg13:30
AJaegercorvus: do you want to review the tox stack at https://review.opendev.org/#/c/730334? and do we need an announcement for https://review.opendev.org/#/c/726830/23?13:32
corvusAJaeger: i very much doubt anyone other than openstack would have been using that, but an announcement just in case would be good.  we could probably have a short deprecation period though.13:33
mordredcorvus: sigh13:33
mordredcorvus: it's actually that we never landed the lock algorithm13:34
mordredcorvus: I just +Ad it13:34
corvusmordred: oh that would also explain it13:34
mordredcorvus: yeah - it seemed _very_ similar to before :)13:34
AJaegeravass: want to send an announcement for 726830? Give it one week deprecation...13:34
AJaegercorvus: ok. I haven't seen any openstack useage but I agree, let's be conservative13:35
avassAJaeger, corvus: sure, I'll send it later13:35
corvusAJaeger: yes, i also didn't expect any openstack usage; only that if there were any usage at all, that's where it would probably be :)13:35
AJaegermordred: once that's lock change lands, we can recheck a few changes ;)13:35
tobiash_corvus: I think the change queue and cdep stack should be ready for review (starting at 718531)13:36
corvustobiash_: ah great, i'll take a look after breakfast :)13:37
tobiash_corvus: thanks :)13:37
* tobiash_ agrees, breakfast is more important13:37
*** tobiash_ is now known as tobiash13:37
*** Goneri has joined #zuul13:42
openstackgerritJan Kubovy proposed zuul/zuul master: Scheduler's pause/resume functionality  https://review.opendev.org/70973513:44
openstackgerritJan Kubovy proposed zuul/zuul master: Separate connection registries in tests  https://review.opendev.org/71295813:44
openstackgerritJan Kubovy proposed zuul/zuul master: Prepare Zookeeper for scale-out scheduler  https://review.opendev.org/71726913:44
openstackgerritJan Kubovy proposed zuul/zuul master: Mandatory Zookeeper connection for ZuulWeb in tests  https://review.opendev.org/72125413:44
openstackgerritJan Kubovy proposed zuul/zuul master: Driver event ingestion  https://review.opendev.org/71729913:44
*** jamesmcarthur has joined #zuul13:46
tobiashzuul-maint: this is a refactoring of the github auth handling which will make it easier to do additional enhancements there in the future: https://review.opendev.org/710034 (I'm planning to implement lazy installation map priming which becomes more important with many app installations and many executors)13:47
tobiashmordred: mind an easy review that makes tox remote tests more stable? https://review.opendev.org/72889213:49
mordredtobiash: I love making things more stable13:50
tobiash:D13:50
AJaegeravass: https://review.opendev.org/730788 updates openstack-zuul-jobs for include, could you review that one, please?13:51
*** jamesmcarthur has quit IRC13:51
openstackgerritMerged zuul/zuul-registry master: Handle blob upload races  https://review.opendev.org/72592513:52
*** sgw has quit IRC13:52
openstackgerritTobias Henkel proposed zuul/zuul master: Revert "Revert "Create zuul/web/static on demand""  https://review.opendev.org/66310813:53
avassAJaeger: sure, on it!13:53
mordredtobiash: +2 with a comment13:54
AJaegerthx, avass13:54
tobiashmordred: that class is overridden in the tests hence there is a layer of indirection13:55
openstackgerritMerged zuul/zuul-jobs master: Fix deprecation warning from multinode tests  https://review.opendev.org/73047913:56
avassAJaeger: include_tasks doesn't have the static keyword, otherwise it looks good13:56
openstackgerritMerged zuul/zuul-jobs master: tox: empty envlist should behave like tox -e ALL  https://review.opendev.org/73032213:56
openstackgerritMerged zuul/zuul-jobs master: ensure-podman: update include to include_tasks  https://review.opendev.org/73066713:56
openstackgerritMerged zuul/zuul-jobs master: ensure-virtualenv: update include to inlude_tasks  https://review.opendev.org/73066913:56
mordredtobiash: oh - duh13:57
openstackgerritMatthieu Huin proposed zuul/zuul master: Builds: add "held" attribute  https://review.opendev.org/72899413:58
*** jamesmcarthur has joined #zuul14:00
*** yoctozepto8 has joined #zuul14:00
*** yoctozepto has quit IRC14:01
*** yoctozepto8 is now known as yoctozepto14:01
*** jamesmcarthur has quit IRC14:02
*** jamesmcarthur has joined #zuul14:02
*** rlandy|mtg is now known as rlandy14:04
avasscorvus: does zuul-jobs want https://review.opendev.org/#/c/728684/ or should I put that in one of my own repos? :)14:05
avasscorvus: still needs some polishing and tests etc though14:06
*** rpittau is now known as rpittau|brb14:10
corvusavass: well, zuul's job is to run some playbooks in a very particular way, and i have concerns about having a second implementation of running a playbook in a particular way.  either it's going to be an incomplete approximation, or it's going to be duplicate code.  there is, however, a third option which is what tristanC has been working on.  the parts of that that i think are most solid right now are the14:14
corvusjob freezing work.  that expands the api endpoints to support asking zuul to provide all the job configuration information so that a runner tool doesn't need to re-implement config parsing, inheritance, project-pipeline configuration, etc.  the parts that are the least settled are how to construct the inventory and how to handle pre/post playbooks.  that's what we're discussing in the spec currently.14:14
openstackgerritMerged zuul/zuul-jobs master: ensure-package-repositories: update include to include_tasks  https://review.opendev.org/73067114:14
corvusavass: so how about this:14:14
corvusavass: perhaps your tool could be based on the freeze-job api that's part of the zuul-runner work.  where your tool differs from zuul-runner is how you're handling the inventory (docker).  that's where a new tool could probably provide the most value, without having duplicate implementations of job construction14:15
corvusavass: (i am still curious about what you're doing with pre/post playbooks, particularly log uploads, etc)14:16
corvusavass: (or are you just dealing with one-layer jobs -- ie, only run the playbooks for the current job, not any parents?)14:16
avasscorvus: the latter, the original idea was just to have a tool to test the jobs in zuul-jobs locally without needing a zuul at all14:17
avasscorvus: just made it for fun as a proof of concept really. But I found it useful14:18
corvusavass: ok, that's a reasonable thing for zuul-jobs, since they don't tend to be deeply nested, so it's a much more limited use case than zuul-runner.  the not-having-a-zuul requirement nixes the idea of using the freeze-jobs api (though for the zuul-jobs repo, it does seem like using the opendev zuul would probably be sufficient)14:19
avasscorvus: yes :)14:19
avasscorvus: but I'm planning on somehow adding support for ansible debugging and --step, which you can't do in zuul14:22
corvusavass: i definitely see the utility, but i worry about the zuul project having a tool that runs zuul jobs, but not in exactly the same way as zuul, and also semi-duplicates zuul's logic.  i'd be much more comfortable with something in zuul repos that helped people run the ansible playbooks rather than running "zuul jobs".14:24
corvusi think it's the idea that it looks like it "runs a zuul job" but it doesn't do it the way that zuul does it that makes me worried.  if it were a tool that, say, "ran this playbook in a docker container", that might be different?14:26
avasscorvus: yeah, that's also why it's just a script and not a complete package.14:26
avasscorvus: yeah14:26
avasscorvus: I'll see if I can do something with zuul directly instead of imitating the behaviour :)14:27
corvusavass: with the api?  or the code?14:28
avasscorvus: well probably the code14:28
avasscorvus: but I guess that would require some work14:28
corvusavass: well, we've also said we don't really want zuul to be an end-user tool or provide any kind of api14:29
corvusit's a server-side application, and we need to be able to regularly refactor it without worrying about downstream users; and we also don't want to have to worry about people being able to install it everywhere14:29
corvusi think i'd be even more hesitant to suggest relying on zuul code itself for this14:30
avassah yeah that sounds reasonable14:32
*** jamesmcarthur has quit IRC14:33
avassI could probably modify it to just run a playbook against a container, but that would require the user to set up variables themselves though. Which would be slightly annoying14:33
corvusavass: i think you've found the sweet spot for local playbook development -- what your script does is pretty close to what i do, and i advocate other folks do, which is to just run the playbook you're working on.  i guess i'm wondering how little "reading zuul config" we could get away with.14:34
*** jamesmcarthur has joined #zuul14:34
avasscorvus: but, the tool could be that, with a companion script to build the inventory14:34
corvusavass: maybe it can set up the standard zuul.X variables, etc, but without reading the job config?14:34
avasscorvus: yeah14:36
corvus(basically i think my test would be whether a naive user would expect the tool to run the job exactly the way that zuul does)14:36
corvus(if it looks like it should, then we're in zuul-runner territory, but if it just looks like it runs a playbook, then that's different)14:37
corvussorry i have to run make breakfast, maybe other folks can weigh in14:37
avassI have to leave too. But I'll check in later :)14:38
*** rpittau|brb is now known as rpittau14:39
*** jamesmcarthur has quit IRC14:40
*** jamesmcarthur has joined #zuul14:44
*** iurygregory has quit IRC14:45
*** iurygregory has joined #zuul14:53
*** sgw has joined #zuul14:54
tristanCcorvus: note that mostly https://review.opendev.org/#/c/607078 was needed from the api to enable zuul-runner usage15:05
tristanCand we are still pretty much looking forward doing the rebase and all the necessary work to get that zuul-runner feature15:06
openstackgerritMerged zuul/zuul-jobs master: fetch-subunit-output: update include to import_tasks  https://review.opendev.org/73066815:06
openstackgerritMerged zuul/zuul master: Gitlab - allow to use the 'zuul enqueue' command  https://review.opendev.org/72866915:06
openstackgerritMerged zuul/zuul master: Increase wait time in remote tests  https://review.opendev.org/72889215:13
openstackgerritMatthieu Huin proposed zuul/zuul master: separate Zuul CLI, Zuul Web CLI  https://review.opendev.org/72923615:14
openstackgerritMatthieu Huin proposed zuul/zuul master: [WIP] Web UI: add "held" information, filtering in Builds page  https://review.opendev.org/72926515:17
openstackgerritMatthieu Huin proposed zuul/zuul master: [WIP] web UI: add Autoholds Requests page  https://review.opendev.org/72930715:17
*** ysandeep is now known as ysandeep|afk15:19
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Remove tumbleweed from testing  https://review.opendev.org/73072715:30
AJaegeranybody else wants to review the tox stack for zuul-jobs starting at https://review.opendev.org/#/c/730334/ ? Otherwise, I'll do a final review later and merge if ok.15:32
AJaegercorvus, mordred, https://review.opendev.org/730602 proposes to change configure_mirror , I'm not sure whether this is the right direction, please have a look15:33
*** sshnaidm is now known as sshnaidm|afk15:34
clarkbAJaeger: one important use there is if we were to use something like the intermediate registry but for packages. However, I think most packaging systems simply allow you to add an additional repo in addition to the existing set?15:36
clarkbhowever, symetry with pypi is nice I Guess15:37
mordredclarkb: yeah - they do, and things are selected by version15:37
AJaegerclarkb, mordred, see my comment regarding mirrors about https://zuul-ci.org/docs/zuul-jobs/mirror.html and https://review.opendev.org/#/c/677578/ - I wonder whether we want to wait for that implementation15:40
mordredAJaeger: it would be _awesome_ to get someone to finish that15:42
corvusi guess we'd just deprecate the new thing in 730602 when 677578 is done15:46
corvusit seems like a lot more work to implement 730602 and then deprecate it.  it would be great if Bogdan could just take over 677578 instead.15:46
corvuslet's ask15:47
mordred++15:47
*** Pilou has quit IRC15:47
corvusi left a comment15:49
*** Pilou has joined #zuul15:49
openstackgerritMerged zuul/zuul-jobs master: persistent-firewall: update include to include_tasks  https://review.opendev.org/73066515:50
openstackgerritMerged zuul/zuul-jobs master: use-docker-mirror: update include to include_tasks  https://review.opendev.org/73066415:50
*** rpittau is now known as rpittau|afk15:51
AJaegeravass: what are the next changes of yours that need review?15:53
*** Pilou has quit IRC15:54
*** ysandeep|afk is now known as ysandeep15:56
*** Pilou has joined #zuul15:56
openstackgerritMerged zuul/zuul-jobs master: configure-mirrors: update include to include_tasks  https://review.opendev.org/73066615:57
*** rlandy is now known as rlandy|mtg16:01
*** ysandeep is now known as ysandeep|afk16:02
*** jamesmcarthur has quit IRC16:05
*** EmilienM is now known as EmilienM|lunch16:20
*** EmilienM|lunch is now known as EmilienM16:20
*** hashar is now known as hasharAway16:36
*** rlandy|mtg is now known as rlandy16:39
*** jamesmcarthur has joined #zuul16:42
*** jpena is now known as jpena|off16:42
*** guillaumec has joined #zuul16:46
*** nils has quit IRC16:46
*** mnasiadka has quit IRC16:49
*** vblando has quit IRC16:50
*** rlandy is now known as rlandy|training16:51
*** ChrisShort has quit IRC16:51
*** ysandeep|afk is now known as ysandeep|away16:52
*** jamesmcarthur has quit IRC16:54
*** vblando has joined #zuul16:57
*** ChrisShort has joined #zuul16:57
*** mnasiadka has joined #zuul16:58
*** samccann has quit IRC16:58
*** ChrisShort has quit IRC17:03
*** vblando has quit IRC17:05
*** jamesmcarthur has joined #zuul17:06
*** mnasiadka has quit IRC17:07
*** samccann has joined #zuul17:07
*** Open10K8S has quit IRC17:08
*** ChrisShort has joined #zuul17:09
*** Open10K8S has joined #zuul17:13
*** ChrisShort has quit IRC17:15
avassAJaeger: I don't think I have anythin else ready to be merged yet except for https://review.opendev.org/#/c/730482/ if we want that17:16
*** wxy has quit IRC17:16
avassmordred: should we install os-testr in the venv to get subunit2html instead of relying on it being on the system?17:17
*** asaleh_ has quit IRC17:18
*** jcapitao has quit IRC17:21
*** wxy has joined #zuul17:21
*** mnasiadka has joined #zuul17:22
*** ChrisShort has joined #zuul17:22
*** vblando has joined #zuul17:22
mordredavass: yes - I think so - I think depending on it being pre-installed is definitely wrong17:30
mordredavass: I also think we should consider whether os-testr is the right home for and right place to get subunit2html from17:31
mordredit started life as a utility script we had copied in to our jenkins images. we copied it in to os-testr for "convenience"17:32
mordredbut maybe it should move to its own package - or we could just copy the script (it's standalone) into our role17:33
avassmordred: copying it into the role sounds reasonable unless it's used elsewhere as well17:33
mordredavass: it's used in a bunch of places in opendev - but manny of them seem like cargo-culted scripts that look for it in usr/os-testr-env/bin/subunit2html17:36
avassmordred: if we have it in the role we can stop looking for it as well :)17:37
mordredyeah. that said - we could also just install os-testr - it's not _super_ expensive of an install17:37
mordredI think I'd be happy with either approach :)17:37
avasscorvus: I'll see if I can make the tool less like zuul and still make sense. but being able to do zuul execution locally would be really nice and that could probably be part of the zuul-runner17:38
avassmordred: it's probably easier to just keep the script in the role, unless it's still being updated but I'm guessing it's not17:40
avassmordred: since installing os-testr in the role technically makes the role dependent on pip and that would require ensure-pip etc... depending on how extreme we want to be with making sure roles are self contained :)17:41
mordredavass: its last patch was Date:   Mon Dec 30 16:52:41 2019 +020017:43
mordredavass: before that, last substantive patch was Thu Nov 10 18:13:26 2016 +000017:43
mordredin fact, since it was imported into os-testr in 2015, there have been exactly 4 substantive patches17:44
avassmordred: huh17:44
openstackgerritGuillaume Chauvel proposed zuul/zuul master: Add tutorial tests  https://review.opendev.org/72819417:46
*** y2kenny has joined #zuul17:46
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-subunit-output: keep subunit2html.py in the role  https://review.opendev.org/73093017:51
avassmordred: I wonder if we can do something like that ^17:51
mordredavass: I don't see any reason why not17:53
openstackgerritSagi Shnaidman proposed zuul/zuul-jobs master: WIP Add ansible collection roles  https://review.opendev.org/73036017:55
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-subunit-output: keep subunit2html.py in the role  https://review.opendev.org/73093018:03
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-subunit-output: keep subunit2html.py in the role  https://review.opendev.org/73093018:10
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-tox-output: empty envlist should behave like tox -e ALL  https://review.opendev.org/73033418:23
AJaegermordred: can you review again, please? ^18:25
*** irclogbot_3 has quit IRC18:26
*** irclogbot_2 has joined #zuul18:31
avassjust have to confirm that using ansible_callbacks in zuul is really nice so far :)18:34
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-subunit-output: keep subunit2html.py in the role  https://review.opendev.org/73093018:59
*** dpawlik has joined #zuul19:04
*** dpawlik has quit IRC19:35
*** saneax has quit IRC19:39
openstackgerritMatthieu Huin proposed zuul/zuul master: separate Zuul CLI, Zuul Web CLI  https://review.opendev.org/72923619:41
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: fetch-subunit-output: keep subunit2html.py in the role  https://review.opendev.org/73093019:45
*** dpawlik has joined #zuul19:55
*** rlandy|training is now known as rlandy20:01
*** cloudnull has quit IRC20:12
guilhermesphello! Quick question: we've been trying to catch the nodesets from a tenant looking at zuul mysql tables and... I can't find any table or row logging the nodesets20:14
guilhermespam I looking at the wrong place?20:14
corvusguilhermesp: nope, there's nothing in mysql about nodesets.  that's all in zookeeper.  what's your goal?20:17
guilhermespwe are trying to catch which vm types from nodepool one of our  tenants is using20:18
guilhermespkinda of hard to filter stuff in mysql with build and buildsets20:19
corvusguilhermesp: the best bet for that is probably the scheduler log file20:19
corvus2020-05-26 06:25:05,039 INFO zuul.nodepool: [e: dc0ca93cd53f4446903152d834698d16] Nodeset <NodeSet openstack-single-node-bionic [<Node 0016773688 ('controller',):ubuntu-bionic>]> with 1 nodes was in use for 7270.749934911728 seconds for build <Build 83fdb36599ec412da9e23469a6d57b95 of ipa-tempest-ironic-inspector-src voting:False on <Worker ze09.openstack.org>> for project openstack/ironic-python-agent20:20
corvusguilhermesp: ^ something like that log line20:20
corvusguilhermesp: it doesn't have the tenant, but it does have the project (if the project in question is only in one tenant, then that should have what you need; if it's in more than one tenant, you may need to filter some more)20:21
*** irclogbot_2 has quit IRC20:22
guilhermesphum i see.. yeah that could work. But i wonder if would be a + to have that info, at least tenant/nodeset stored in db, dont you think?20:25
*** irclogbot_2 has joined #zuul20:26
corvusguilhermesp: maybe so, though until we have non-final buildsets stored in the db as well, it would be incomplete data for some purposes, so it's tricky.20:26
guilhermespi see corvus maybe if you can point out where I could possible contribute. Maybe i can try to start a work on implementing that info on db20:32
guilhermespand eventually mnaser could be helping me out on that  with his thoughts :P20:34
fungithat's part of the work on zuul v4/v5, since right now the relational database records are populated by a reporter, so that doesn't occur until a buildset completes20:42
fungiguilhermesp: https://zuul-ci.org/docs/zuul/reference/developer/specs/scale-out-scheduler.html#mandatory-sql-connection20:44
fungithat's the relevant part of the plan, i think20:44
corvusguilhermesp: you could probably start adding that now, but it's only going to report final results until the series at https://review.opendev.org/633501 lands.  and yeah, it's only going to apply to tenants with sql reporters until v4.20:44
fungiluckily, that seems to be step 1 on the roadmap for the scale-out scheduler spec20:45
fungiso probably not that far off20:46
mnaserguilhermesp: yeah -- i think for now until 633501 lands, it's ok to have at least non-final reporting (so inaccurate to a degree)21:00
mnaserand we already use the sql reporter by default because honestly most of the zuul web ui is broken if you don't use it :-p21:01
corvusyep.  it's just important to know that it won't be accurate for billing purposes :)21:09
*** kgz has joined #zuul21:17
*** hasharAway has quit IRC21:23
*** y2kenny has quit IRC21:23
*** cloudnull has joined #zuul21:43
clarkbstatsd does report resource usage though21:50
clarkbin a reasonably accurate manner I think21:51
*** jamesmcarthur_ has joined #zuul21:51
*** jamesmcarthur has quit IRC21:56
*** dpawlik has quit IRC21:56
*** tobiash has quit IRC22:33
*** tobiash has joined #zuul22:34
*** guillaumec has quit IRC22:48
*** tosky has quit IRC22:52
*** saneax has joined #zuul22:58
*** saneax has quit IRC22:58
openstackgerritClark Boylan proposed zuul/zuul-jobs master: DO NOT MERGE test base-test with no virtualenv perms modifications  https://review.opendev.org/68017823:00
*** threestrands has joined #zuul23:17
*** armstrongs has joined #zuul23:38
*** jamesmcarthur_ has quit IRC23:45
*** armstrongs has quit IRC23:47
« Monday, 2020-05-25 Index Wednesday, 2020-05-27 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!