| *** jamesmcarthur has joined #zuul | 00:03 | |
| *** jamesmcarthur has quit IRC | 00:07 | |
| sgw | fungi: or anyone! I want to add a non-standard command into zuul, specifically the "osc" command from OpenSUSE for interacting with OBS, there are repos/packages available for different OSes, so does this need to be done in a "pre" type of ansible ? | 00:09 |
|---|---|---|
| clarkb | sgw if the paclage is in the main distro repos you can add it to your bindep list | 00:11 |
| clarkb | assuming this is for jobs already running the bindep pre step | 00:11 |
| sgw | clarkb: unforunately not the correct version is available in the main distro repo's, I need to use the OBS repos to install osc from | 00:11 |
| sgw | is NOT available in main distro repo | 00:12 |
| clarkb | then for that I would add it as a pre step explicitly for thag | 00:12 |
| sgw | Ok, any examples of that? so I can quickly crib from it. | 00:13 |
| clarkb | Im not sureof any that add a repo and install a package from it | 00:14 |
| sgw | Ok, just seeing if there was a short-cut example, I will figure it out. | 00:14 |
| *** jamesmcarthur has joined #zuul | 00:21 | |
| fungi | yeah, i'm not overly familiar with opensuse and yast (or is it zypper?) package management and repository additions | 00:21 |
| fungi | but yeah if you know the shell commands to run you can always just create a role with the corresponding shell tasks and add that to one of the job's pre playbooks | 00:22 |
| sgw | fungi: yup that's what I am doing. My ansible is currently horrid, but I am getting there. | 00:23 |
| fungi | we likely have some debian/ubuntu apt examples, like the nodesource package repository roles | 00:23 |
| fungi | ensure-npm or whatever it's called | 00:24 |
| fungi | https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/install-nodejs/tasks/main.yaml | 00:25 |
| *** rlandy has quit IRC | 00:25 | |
| fungi | that one | 00:25 |
| fungi | not sure whether or which of those have opensuse corollaries | 00:26 |
| sgw | Thanks! | 00:29 |
| *** altlogbot_1 has joined #zuul | 00:30 | |
| *** zbr has quit IRC | 00:34 | |
| *** altlogbot_1 has quit IRC | 00:37 | |
| *** jamesmcarthur has quit IRC | 00:38 | |
| *** zbr has joined #zuul | 00:43 | |
| *** jamesmcarthur has joined #zuul | 00:47 | |
| *** jamesmcarthur has quit IRC | 00:52 | |
| *** jamesmcarthur has joined #zuul | 00:55 | |
| *** altlogbot_3 has joined #zuul | 01:08 | |
| *** jamesmcarthur has quit IRC | 01:09 | |
| *** jamesmcarthur has joined #zuul | 01:14 | |
| *** altlogbot_3 has quit IRC | 01:15 | |
| *** jamesmcarthur has quit IRC | 01:19 | |
| *** jamesmcarthur_ has joined #zuul | 01:21 | |
| *** jamesmcarthur_ has quit IRC | 01:23 | |
| *** jamesmcarthur has joined #zuul | 01:27 | |
| *** jamesmcarthur has quit IRC | 01:32 | |
| *** jamesmcarthur has joined #zuul | 01:36 | |
| *** jamesmcarthur has quit IRC | 01:41 | |
| *** jamesmcarthur has joined #zuul | 01:43 | |
| *** jamesmcarthur has quit IRC | 01:45 | |
| *** jamesmcarthur has joined #zuul | 01:50 | |
| *** bhavikdbavishi has joined #zuul | 01:54 | |
| *** jamesmcarthur has quit IRC | 01:54 | |
| *** bhavikdbavishi1 has joined #zuul | 01:57 | |
| *** bhavikdbavishi has quit IRC | 01:58 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 01:58 | |
| *** jamesmcarthur has joined #zuul | 02:07 | |
| *** jamesmcarthur has quit IRC | 02:09 | |
| *** jamesmcarthur has joined #zuul | 02:09 | |
| *** jamesmcarthur has quit IRC | 02:14 | |
| *** altlogbot_3 has joined #zuul | 02:20 | |
| *** irclogbot_3 has joined #zuul | 02:20 | |
| *** jamesmcarthur_ has joined #zuul | 02:22 | |
| *** bhavikdbavishi has quit IRC | 02:23 | |
| *** altlogbot_3 has quit IRC | 02:25 | |
| *** irclogbot_3 has quit IRC | 02:26 | |
| *** jamesmcarthur_ has quit IRC | 02:26 | |
| *** jamesmcarthur has joined #zuul | 02:56 | |
| *** jamesmcarthur has quit IRC | 03:19 | |
| *** jamesmcarthur has joined #zuul | 03:20 | |
| *** bhavikdbavishi has joined #zuul | 03:26 | |
| *** jeliu_ has joined #zuul | 03:48 | |
| *** njohnston has quit IRC | 03:53 | |
| *** irclogbot_2 has joined #zuul | 04:08 | |
| *** irclogbot_2 has quit IRC | 04:12 | |
| *** altlogbot_3 has joined #zuul | 04:14 | |
| *** altlogbot_3 has quit IRC | 04:17 | |
| *** jamesmcarthur has quit IRC | 04:32 | |
| *** jamesmcarthur has joined #zuul | 04:46 | |
| *** swest has joined #zuul | 04:59 | |
| *** altlogbot_2 has joined #zuul | 05:04 | |
| *** irclogbot_2 has joined #zuul | 05:08 | |
| *** raukadah is now known as chandankumar | 05:14 | |
| *** sanjayu_ has joined #zuul | 05:19 | |
| *** altlogbot_2 has quit IRC | 05:21 | |
| *** irclogbot_2 has quit IRC | 05:22 | |
| *** jeliu_ has quit IRC | 05:36 | |
| *** pcaruana has joined #zuul | 05:37 | |
| *** sanjayu_ has quit IRC | 05:41 | |
| *** saneax has joined #zuul | 05:47 | |
| *** jamesmcarthur has quit IRC | 05:50 | |
| *** pcaruana has quit IRC | 06:20 | |
| *** tosky has joined #zuul | 07:02 | |
| *** altlogbot_0 has joined #zuul | 07:04 | |
| mhu | corvus: I was on PTO yesterday, I can have a look today | 07:05 |
| mhu | exciting times ahead! | 07:05 |
| *** altlogbot_0 has quit IRC | 07:08 | |
| *** themroc has joined #zuul | 07:22 | |
| *** irclogbot_1 has joined #zuul | 07:41 | |
| *** irclogbot_1 has quit IRC | 07:44 | |
| *** pcaruana has joined #zuul | 07:54 | |
| *** hashar has joined #zuul | 08:09 | |
| *** irclogbot_0 has joined #zuul | 08:59 | |
| *** hwangbo has quit IRC | 09:00 | |
| *** irclogbot_0 has quit IRC | 09:02 | |
| *** altlogbot_0 has joined #zuul | 09:27 | |
| *** altlogbot_0 has quit IRC | 09:28 | |
| *** altlogbot_2 has joined #zuul | 09:31 | |
| *** altlogbot_2 has quit IRC | 09:34 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 09:42 | |
| *** bhavikdbavishi has quit IRC | 09:45 | |
| *** altlogbot_3 has joined #zuul | 09:46 | |
| *** altlogbot_3 has quit IRC | 09:50 | |
| *** hashar has quit IRC | 10:01 | |
| *** electrofelix has joined #zuul | 10:05 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: web: add tenant and project scoped, JWT-protected actions https://review.opendev.org/576907 | 10:11 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Allow operator to generate auth tokens through the CLI https://review.opendev.org/636197 | 10:11 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Zuul CLI: allow access via REST https://review.opendev.org/636315 | 10:11 |
| *** irclogbot_3 has joined #zuul | 10:25 | |
| *** irclogbot_3 has quit IRC | 10:26 | |
| *** hashar has joined #zuul | 10:33 | |
| *** hashar has quit IRC | 10:34 | |
| *** kklimonda has joined #zuul | 11:00 | |
| *** bhavikdbavishi has joined #zuul | 11:17 | |
| mordred | mhu: you picked a fun day to be on PTO :) | 11:47 |
| mhu | you don't say: | 11:48 |
| mhu | :) | 11:48 |
| * mordred hands mhu his blue suit and tie | 11:48 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Avoid confusing rsync errors when source folders are missing https://review.opendev.org/670044 | 11:50 |
| *** rfolco has joined #zuul | 11:57 | |
| Shrews | mordred: did you hand over the suite/tie using Notes? that's the only acceptable way to do that now | 12:20 |
| *** rlandy has joined #zuul | 12:24 | |
| *** bkorren has joined #zuul | 12:28 | |
| *** hashar has joined #zuul | 12:30 | |
| bkorren | Hi, I wonder if someone can help me figure out why this pipeline -> (https://ovirt-staging.softwarefactory-project.io/paste/show/4/) is not triggering when I put 'ci emulate-gate' in a Gerrit comment | 12:30 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Zuul CLI: allow access via REST https://review.opendev.org/636315 | 12:42 |
| *** tosky__ has joined #zuul | 12:44 | |
| *** tosky is now known as Guest87324 | 12:44 | |
| *** tosky__ is now known as tosky | 12:44 | |
| *** Guest87324 has quit IRC | 12:46 | |
| *** altlogbot_1 has joined #zuul | 13:12 | |
| *** altlogbot_1 has quit IRC | 13:16 | |
| *** irclogbot_2 has joined #zuul | 13:19 | |
| *** tosky__ has joined #zuul | 13:20 | |
| *** tosky has quit IRC | 13:20 | |
| *** tosky__ is now known as tosky | 13:20 | |
| *** irclogbot_2 has quit IRC | 13:22 | |
| *** tosky___ has joined #zuul | 13:44 | |
| *** yolanda has quit IRC | 13:46 | |
| bkorren | Hi there - I'm seeing some behavior of Zuul that I think may be a bug - as far as I can tell - Zuul's YAML syntax lets you define dependent pipelines that do not actually merge patches; This line in the code seems to indicate that the pipeline would not accept patches unless they are ready to be merged wither it would actually merge them or not -> https://opendev.org/zuul/zuul/src/branch/master/zuul/manager/dependent.py#L102 | 13:46 |
| *** tosky has quit IRC | 13:47 | |
| mordred | bkorren: well - the behavior of the dependent pipeline only makes sense if it's part of the merging action, because it's going to put things into a virtual shared queue and test various patches as-if the one ahead of them in the queue had landed. In a non-merging scenario this behavior would cause the patches being tested to be tested in the context of arbitrary other patches that are also in flight but without | 13:58 |
| mordred | the captive assurance that the constructed state is the state that would result from the completion of the previous in-flight gating actions | 13:58 |
| mordred | bkorren: so in that pipeline definition you have, I'd change manager to independent, even though you're trying to simulate gate activity, because if you're triggering these via comments there is no valid dependency relationship that zuul can create, and thus it'll just cause, at best, confusing and potentially invalid results | 13:59 |
| bkorren | mordred, I admit that not merging is not always useful in this context - but since actually making multi-project testing jobs can be difficult, as well is crafting patches to test their edge cases, I'd be nich if I could emulate the pipeline's actions without having the patches be merged | 14:00 |
| mordred | bkorren: the multi-project part will still work with the independent pipeline manager, as will explicit dependencies between patches via depends-on footers | 14:01 |
| mordred | bkorren: the dependent/independent in this case has to do with relationships between triggering events, not between changes | 14:01 |
| pabelanger | bkorren: is it the pipeline you want to test, or the jobs that run in the pipeline? because as mordred says, depends-on in commit message should be what you are looking for | 14:01 |
| pabelanger | then everything can be tested without merging | 14:02 |
| mordred | and dependent causes zuul to create patch relationships automatically based on the sequence it received the triggering event where otherwise there might not be an existing relationship | 14:02 |
| pabelanger | (unless you are dealing with trusted jobs) | 14:02 |
| bkorren | mordred, if I understand correctly, with independent - it will always check one patch against the 'master' of other projects, I would like to test multiple patches at once. | 14:02 |
| mordred | yah. what pabelanger said | 14:02 |
| mordred | bkorren: right - what you want to have happen will happen with the independent pipeline manager too | 14:02 |
| mordred | the independent pipeline manager will still do all of the patch-dependency relationship things you're wanting | 14:03 |
| pabelanger | yup | 14:03 |
| pabelanger | depends-on is super powerful, for pre-merge testing | 14:04 |
| bkorren | mordred - will it test patches together even if I don't explicitly set 'depends-on' ? | 14:04 |
| mordred | dependent is basically _only_ for the speculative execution bits needed for safe gating ... the word "dependent" in this case is, I think, causing some confusion | 14:04 |
| *** jamesmcarthur has joined #zuul | 14:04 | |
| *** jamesmcarthur has quit IRC | 14:05 | |
| *** jamesmcarthur has joined #zuul | 14:05 | |
| bkorren | pabelanger, mordred - do note that I am trying to test the pipeline and gate job in this case - the patches are crafted to cause various issues... | 14:05 |
| pabelanger | are the gate jobs different then check jobs? | 14:05 |
| mordred | bkorren: gotcha, no - you would have to set depends-on footers to tell zuul about that relationship | 14:05 |
| mordred | bkorren: let me make sure I understand what you're trying to do real quick ... | 14:06 |
| bkorren | mordred, ok | 14:06 |
| mordred | bkorren: you would like to manually trigger a set of patches using comment triggers and you would like for those to be tested in the context of each other based on the sequencing of comments added but for those patches to not be merged at the end of each test .. is that correct? | 14:07 |
| bkorren | mordred, yeah | 14:07 |
| bkorren | mordred, exactly | 14:07 |
| mordred | nod. so - in our world we tend to accomplish that use case by constructing those patches with depends-on footers and letting our check pipeline take care of it for us | 14:07 |
| bkorren | mordred, I see | 14:08 |
| mordred | I'm honestly not sure if the dependent pipeline manager can do the thing you're wanting - and I think that's a question for corvus when he awakens | 14:08 |
| mordred | (it's not a use case I've considered before, so I need to defer to a smarter person :) ) | 14:08 |
| bkorren | mordred, well, looking at the source is seems to me one could make it only do the 'canMerge' check if one configured the pipeline to actually merge, but maybe it affects other things... | 14:09 |
| mordred | but we frequently do that with patches with depends-on added to them - sometimes just with patches we mark as "do not merge" - just so we can verify behavior like your'e talking about ... but I do think clarifying the intent here is valuable either way | 14:10 |
| bkorren | mordred, ok, I get what you're saying | 14:11 |
| bkorren | mordred, one more thing - do I have to specify all the projects that are to be tested together in the job's 'required-projects' section, for then to ever get tested together? | 14:14 |
| bkorren | mordred, I thought that merely having patches together in a dependent queue at the same time would cause them to be tested together - but the results I get from merging some of my patches indicate that may not be the case... | 14:15 |
| *** tosky___ is now known as tosky | 14:15 | |
| pabelanger | you likely need to setup a shared queue: https://zuul-ci.org/docs/zuul/user/config.html#attr-project.%3Cpipeline%3E.queue | 14:16 |
| bkorren | pabelanger, AFAIK I did | 14:17 |
| pabelanger | for example, openstack has an integrated change queue where patches from nova, glace, keystone, etc need to be grouped together | 14:17 |
| pabelanger | bkorren: is there a place to look at the configuration? | 14:17 |
| bkorren | pabelanger, sure -> https://gerrit-staging.phx.ovirt.org/gitweb?p=ovirt-staging-zuul-config.git;a=blob;f=zuul.d/projects.yaml;h=33647b3d7147209ded72237635bc5554f68c30fe;hb=refs/heads/master#l10 | 14:19 |
| bkorren | pabelanger, basically I made all projects use both the 'ovirt-staging-gated-project' and the 'ost-gated-project' templates | 14:20 |
| bkorren | pabelanger, and 'ovirt-staging-gated-project' is setting the queue to the same value for all pipelines... | 14:21 |
| *** hashar has quit IRC | 14:21 | |
| pabelanger | bkorren: just looking through your configs | 14:25 |
| bkorren | pabelanger, I have to leave now, I'll send my question to the mailing-list later with hopes of getting an aswer there; thanks for all your help so far! | 14:25 |
| pabelanger | np | 14:26 |
| bkorren | pabelanger, uless there is something I did wrong that caught your eye quickly... | 14:26 |
| bkorren | portdirect, ok, thanks! | 14:26 |
| bkorren | pabelanger, ok, thanks! | 14:26 |
| portdirect | bkorren: I'll take credit for pabelanger's awesomeness anyday :D | 14:27 |
| *** bkorren has quit IRC | 14:30 | |
| mordred | portdirect: me too! | 14:33 |
| *** mattw4 has joined #zuul | 15:07 | |
| *** altlogbot_0 has joined #zuul | 15:11 | |
| *** altlogbot_0 has quit IRC | 15:12 | |
| *** yolanda has joined #zuul | 15:23 | |
| *** themroc has quit IRC | 15:26 | |
| *** hashar has joined #zuul | 15:26 | |
| *** mattw4 has quit IRC | 15:34 | |
| *** mattw4 has joined #zuul | 15:35 | |
| *** mattw4 has quit IRC | 15:39 | |
| *** hashar has quit IRC | 16:06 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add test_setup_skip role variable https://review.opendev.org/670119 | 16:24 |
| *** hwangbo has joined #zuul | 16:24 | |
| pabelanger | mordred: left a comment on 659180, but looks good in general | 16:34 |
| mordred | pabelanger: it's a good comment - responded - but tl;dr, I don't think we need to double-specify like that - we can let the majority of them passthrough directly - and should only need to double-specify the secret containing values | 16:41 |
| *** mattw4 has joined #zuul | 16:41 | |
| pabelanger | okay great, that's what I was hoping for | 16:46 |
| pabelanger | +2 | 16:47 |
| pabelanger | :D | 16:47 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add test_setup_skip role variable https://review.opendev.org/670119 | 16:47 |
| openstackgerrit | Andreas Jaeger proposed zuul/zuul-jobs master: Skip test-setup.sh in pep8 jobs https://review.opendev.org/670133 | 17:03 |
| pabelanger | AJaeger: left -1 and comment on ^ | 17:05 |
| AJaeger | pabelanger: I don't get it - why -1 that one and +2 the openstack-zuul-jobs one? | 17:05 |
| AJaeger | (oh, you didn't +2 that one) | 17:06 |
| pabelanger | AJaeger: +2 skip was false by default, so no change | 17:06 |
| pabelanger | also, haven't seen openstack one | 17:06 |
| AJaeger | pabelanger: skip by false is for *all* tox jobs. The true here is for pep8, that was the result of all the discussion... | 17:06 |
| pabelanger | AJaeger: where was discussion, I've missed that | 17:07 |
| pabelanger | however, | 17:07 |
| AJaeger | pabelanger: on #openstack-infra | 17:07 |
| pabelanger | if we change tox-pep8, that could break ansible.z.c | 17:07 |
| pabelanger | would need to look | 17:07 |
| AJaeger | pabelanger: let's discuss on openstack-infra, please | 17:07 |
| pabelanger | sure | 17:08 |
| *** hwangbo has quit IRC | 17:31 | |
| *** hwangbo has joined #zuul | 17:31 | |
| fungi | fwiw, discussion of changes to the zuul-jobs repo seems appropriate in here. i'm happy to provide a recap if anyone wants | 17:33 |
| pabelanger | unrelated to zuul-jobs change, upgrading to ansible 2.8.2 now for zuul-executors. So far, no issues reported against 2.8 for zuul jobs | 17:35 |
| AJaeger | fungi: you're right - sorry | 17:37 |
| *** bkorren has joined #zuul | 17:37 | |
| bkorren | pabelanger, are you here? | 17:39 |
| pabelanger | o/ | 17:40 |
| pabelanger | bkorren: I couldn't find an exmaple of the issue you mentioned, do you have a few reviews that show the merge issue? | 17:42 |
| bkorren | pabelanger, I have a couple - but I can try and make more | 17:42 |
| pabelanger | I couldn't find the zuul console log, since you updated the log url to point to jenkins | 17:43 |
| pabelanger | that includes zuul-info, which has the inventory file, which could contain more info | 17:43 |
| bkorren | pabelanger: here it is for the 2nd job that should've included both patches: https://ovirt-staging.softwarefactory-project.io/logs/64/264/8/gate-patch/ost-gate/a41bb77/zuul-info/inventory.yaml | 17:44 |
| bkorren | pabelanger, patches are: https://gerrit-staging.phx.ovirt.org/#/c/264/ ; https://gerrit-staging.phx.ovirt.org/#/c/273/ (by order of setting cr+2) | 17:45 |
| pabelanger | bkorren: how did you resolve the zuul-info url? | 17:46 |
| pabelanger | I can't see where that is logged | 17:46 |
| pabelanger | only jenkins info | 17:46 |
| bkorren | pabelanger, its always $server/logs/patch_no/queue/ID/job | 17:47 |
| bkorren | pabelanger, where patch number is like n gerrit URLS - last 2 digits and them full number | 17:47 |
| bkorren | pabelanger, I probably should make jenkins link back to that at some point, but no time for cosmetics now... | 17:48 |
| pabelanger | bkorren: Hmm, I actually would have expected 264 to fail | 17:50 |
| pabelanger | https://ovirt-staging.softwarefactory-project.io/logs/64/264/8/gate-patch/ost-gate/a41bb77/job-output.txt.gz#_2019-07-10_13_41_42_449936 | 17:50 |
| pabelanger | that was pre-run | 17:50 |
| pabelanger | so, jobs was retried? | 17:50 |
| pabelanger | also, I admit I don't fully understand the workflow of zuul triggering jobs in jenkins | 17:51 |
| bkorren | pabelanger, looking - that is kinda strange | 17:51 |
| bkorren | pabelanger, its not very complex - I just upload the source zuul prepared to somewhere jenkins can access it, trigger jenkins, and wait for it to finish | 17:52 |
| pabelanger | bkorren: sure, what bits does jenkins do? for example, that could also be done by zuul directly, but I assuming there is some sort of migration that needs to be done still? | 17:53 |
| bkorren | pabelanger, yeah, I have a whole bunch of logic in jenkins; migrating that would take a long long time; not to mention a user community that is used to reading the logs the way jenkins shows them.... | 17:54 |
| bkorren | pabelanger, right now that jenins job does nothing because I'm still debugging data fow and triggering from Zuul | 17:55 |
| pabelanger | okay, first step is to see why there was a module failure in zuul | 17:56 |
| pabelanger | because I suspect we are looking at old job | 17:56 |
| pabelanger | as for jenkins integration for zuul, it is a little redundant, given zuul and do all those bits, but it should still work. | 17:56 |
| pabelanger | bkorren: I am assuming, in the zuul job, you have logic to wait for jenkins job to finish? And zuul-jobs just does loop waiting for the results? | 17:57 |
| bkorren | pabelanger, even simpler, it runs a blocking jenkins CLI command that returns when the job finishes with a proper status code | 17:58 |
| mordred | that's nice and easy:) | 17:59 |
| bkorren | mordred, ? | 17:59 |
| mordred | (the blocking cli command) | 17:59 |
| pabelanger | https://ovirt-staging.softwarefactory-project.io/logs/64/264/8/gate-patch/ost-gate/ad1857b/job-output.txt.gz | 18:00 |
| pabelanger | bkorren: okay, that looks like right log | 18:00 |
| bkorren | pabelanger, yeah , but given the 1st failure, we may have "missed" the timing and the 1st patch finished getting merged already | 18:01 |
| pabelanger | bkorren: right, that is my thoughts too | 18:01 |
| bkorren | pabelanger, although I'm only seeing one patch in the inventory for the 1st run as well | 18:01 |
| pabelanger | bkorren: if patch a finished, and patch B retries, it likey has the right code sha1 from tip of master | 18:01 |
| bkorren | pabelanger, yeah, but note my comment about the inventory file for the 1st run - also I remember seen two parallel run in jenkins - so it means the 2nd one was triggered before the 1st one finished | 18:04 |
| pabelanger | bkorren: I see you are not using prepare-workspace(-git), can you share the code where you push git repos to node? | 18:04 |
| bkorren | pabelanger, also I'm a little puzzled about the 1st failure - its as if the 'pre.yaml' playbook (its the 'stock' one) did not upload the source to the node | 18:05 |
| pabelanger | bkorren: the 2nd one running before the 1st should be okay, zuul will have the correct repo state setup | 18:05 |
| pabelanger | and not merge, if in the same change queue | 18:05 |
| pabelanger | bkorren: maybe network issue in pre-run | 18:05 |
| bkorren | pabelanger, pre.yaml is running the 'prepare-workspace' rol - you can see playbook sources from the ara-report | 18:07 |
| *** irclogbot_1 has joined #zuul | 18:07 | |
| pabelanger | Oh, I see that now | 18:07 |
| pabelanger | Hmm, we don't log the sha1 info there that is right | 18:07 |
| pabelanger | I think we do for prepare-workspace-git | 18:07 |
| *** irclogbot_1 has quit IRC | 18:08 | |
| bkorren | pabelanger, we got that in the inventory file though, right | 18:08 |
| pabelanger | bkorren: so, when the 2 reviews are in the gate-patch pipeline, do you see them in the same change queue? eg: https://zuul.opendev.org/t/openstack/status you can see 'Integrated' queue | 18:09 |
| pabelanger | or tripleo queue | 18:09 |
| Shrews | corvus: swest: i left a cautionary tale/comment on https://review.opendev.org/667371 | 18:09 |
| bkorren | pabelanger, I think I did - but I may have missed that | 18:10 |
| bkorren | pabelanger, you know what, let me make a couple of fresh new patch and retry all of this from scratch | 18:10 |
| pabelanger | okay cool | 18:10 |
| pabelanger | I'll watch the UI | 18:10 |
| bkorren | pabelanger, don`t hold your breath... it'll take me a little while ;) | 18:12 |
| pabelanger | okay, send a ping when you are ready | 18:12 |
| pabelanger | bkorren: other way, is to look at zuul-scheduler logs, but you may not have access to them | 18:13 |
| bkorren | pabelanger, yeah, I do not; the downside of having someone else maintain Zuul for me... | 18:14 |
| pabelanger | bkorren: dmsimard maybe able to help | 18:16 |
| bkorren | pabelanger, well lets see what happens and 'wake him up' if we need to | 18:16 |
| bkorren | pabelanger, ok, got a couple of new patches: https://gerrit-staging.phx.ovirt.org/#/c/275/ ; https://gerrit-staging.phx.ovirt.org/#/c/276/ ; gonna vote to trigger the gate in a few | 18:19 |
| pabelanger | k | 18:22 |
| bkorren | pabelanger, here we go | 18:23 |
| bkorren | pabelanger, both in same queue | 18:23 |
| pabelanger | bkorren: yah, so that is right | 18:23 |
| pabelanger | 275 should merge before 276 | 18:23 |
| bkorren | pabelanger, so I should see 275 mentioned in the inventory file for 276? | 18:24 |
| pabelanger | also is https://gerrit-staging.phx.ovirt.org/ and http://jenkins-staging.ovirt.org/ is just test infra right? | 18:24 |
| bkorren | pabelanger, yeah | 18:24 |
| pabelanger | bkorren: I need to check, I know when depends-on is used, it is | 18:24 |
| bkorren | pabelanger, if it isn't I made some wrong basic assumptions in my playbooks and jobs... | 18:25 |
| pabelanger | well, 276 will contain commit of 275, because is a dependant pipeline | 18:26 |
| pabelanger | 276 failed for some reason | 18:26 |
| bkorren | pabelanger, do note that those are going to two different git repos... | 18:27 |
| bkorren | pabelanger, yeah I see the failure, strange, having a look | 18:27 |
| pabelanger | bkorren: right, so is job.required-projects setup for projectA and projectB? If so, then zuul will prepare repo state problem for both projects | 18:28 |
| pabelanger | if not, then I'd question why in a shared queue, if no code is shared | 18:28 |
| bkorren | pabelanger, I see so I need `required-projects` after all - I was intending to share the code in the form of a fallback package repo containing prebuild artifacts for stuff that was not being tested | 18:30 |
| bkorren | pabelanger, if I need to list all the projects in the job definition it means I can't let my users dynamically 'opt-in' to the system by dropping a .zuul.yaml file in their repos... | 18:31 |
| pabelanger | bkorren: yah, I don't believe that is dynamic right now | 18:32 |
| pabelanger | you need to create the list of projects before hand | 18:32 |
| bkorren | pabelanger, hmm.... that is a shame... | 18:32 |
| pabelanger | I _think_ corvus might have a patch up to make that more dynamic | 18:32 |
| pabelanger | looking | 18:32 |
| pabelanger | bkorren: https://review.opendev.org/613143/ | 18:33 |
| pabelanger | that is what I am thinking of, that may or may not do what you are looking for | 18:33 |
| pabelanger | bkorren: however, in openstack | 18:34 |
| pabelanger | a new project can create a new job, then parent to your base job (with common projects) and dynamically add their own | 18:34 |
| bkorren | pabelanger, hmm... the commit message isn't very detailed | 18:34 |
| pabelanger | that is used alot in the case of devstack or tempest | 18:34 |
| pabelanger | so, that might be a good way to do it too | 18:34 |
| hwangbo | Is there a way to limit what branches are checked for the startup cat jobs? | 18:35 |
| bkorren | pabelanger, but that would mean that other projects will not test the patches of the new project in their gate no? | 18:35 |
| bkorren | pabelanger, unless I'm not understanding well, is there an example of that I can see? | 18:36 |
| pabelanger | bkorren: right, it is a one way agreement. If you want all jobs to run a project, you'd move it into the base. In this case, it is more a policy issue, then some team oversees | 18:37 |
| corvus | bkorren, pabelanger: i don't think i understand the use-case here -- required-projects is an attribute of a job, it says that a certain job needs the source code for a project in order to run. for example, we can't run openstack without keystone, so many openstack jobs say "require-projects: keystone" | 18:37 |
| corvus | oh, yeah, if you're building a system with add-on projects, then what pabelanger says is spot on | 18:38 |
| pabelanger | hwangbo: for github driver, you can exclude unprotected branches | 18:39 |
| bkorren | corvus, pabelanger - the use case is that we have a gate that know how to test the whole of oVirt - and oVirt devs will opt-in to using the gate over time, with the system falling back to their last merged state if they didn't | 18:39 |
| hwangbo | pabelanger: anything for gerrit driver? | 18:39 |
| pabelanger | hwangbo: checking that now | 18:39 |
| pabelanger | been using github too much theses days :( | 18:40 |
| pabelanger | hwangbo: maybe corvus can confirm, but I don't believe there is a way to limit the branches to load configuration from with gerrit | 18:42 |
| bkorren | corvus, ideally if developer comes up with a new oVirt component - he can just add a .zuul.yaml file to his component to start having it be gated - bu liike like they would have to patch the gate job as well to list their project in `required-projects` | 18:42 |
| corvus | don't think so. i think we would welcome a non-driver-specific patch to do that (like "include_branches" or "exclude_branches") | 18:42 |
| pabelanger | bkorren: yah, I would personally prefer them to patch the base job, because what could stop any project of opt-in and break everything | 18:43 |
| corvus | bkorren: yeah, i think what pabelanger said about that is right -- they can do that, but it's one-way. you need the agreement of the larger team to make it two-way. | 18:43 |
| bkorren | pabelanger, corvus - ok, I see why that makes total sense for a huge project like OpenStack - in tiny little oVirt we can reach team consensus faster, so such separation of concerns is not so strictly needed; but I guess we can do it this way with some user education | 18:46 |
| hwangbo | pabelanger corvus: thanks for the clarification | 18:47 |
| corvus | bkorren: well, i don't think it's a matter of size, it's just location -- if everyone on your team is empowered to represent that consensus, then give them all the ability to approve the change to the base job to add in the new project | 18:48 |
| bkorren | pabelanger, bte I actually see now that I did get both patches listed in the inventory file - my playbook did not deal with that as well as it should have, but that is besides the point: https://ovirt-staging.softwarefactory-project.io/logs/76/276/1/gate-patch/ost-gate/67267d9/zuul-info/inventory.yaml | 18:49 |
| bkorren | pabelanger, corvus : so which is it now, do I need `required-projects` or not? | 18:50 |
| pabelanger | bkorren: I believe, in this case you'll see the project in inventory but https://zuul-ci.org/docs/zuul/user/jobs.html#var-zuul.projects.required is false | 18:55 |
| pabelanger | so, they were tested together, but usually not required | 18:55 |
| pabelanger | if job.required-projectrs was set, I'd expect that to always be true | 18:55 |
| fungi | that is likely an effect of your using the dependent pipeline model, so changes enqueued ahead of that one appear in its build inventory | 18:55 |
| bkorren | pabelanger, so that is exactly what I want to happen - so if that is how its supposed to work I'm a happy camper | 18:56 |
| *** jamesmcarthur has quit IRC | 18:57 | |
| pabelanger | bkorren: but it only happened because both entered the pipeline together. if you only did one, that info should be missing | 18:57 |
| pabelanger | which is what we seen before | 18:57 |
| bkorren | fungi, that was my assumption, pabelanger was helping me debug why I didn't see this happen in my 1t test of this | 18:57 |
| pabelanger | in this case, I think you do want required-projects all the time | 18:58 |
| pabelanger | to help ensure all projects are being checked out properly | 18:58 |
| bkorren | pabelanger, that is fine, since the job will know to get all the projects by other means - no need for the sources | 18:58 |
| fungi | however it can be racy, for example if you want change b tested on change a but you wait too long between triggering builds for a and b, then a can already have exited the pipeline by the time b's dependent changes are calculated | 19:00 |
| fungi | also with nearest-non-failing behavior, when you have failing builds in a change ahead of another, testing for the second will be restarted without the first, which may not be a side effect you're counting on there | 19:01 |
| fungi | much of the dependent pipeline design assumes changes will be merged if they succeed testing, and expects actual dependencies between changes to be declared explicitly so that, say, builds for change a failing doesn't cause change b's builds to succeed simply because they were rerun without a present | 19:03 |
| *** altlogbot_2 has joined #zuul | 19:04 | |
| *** altlogbot_2 has quit IRC | 19:04 | |
| *** jeliu_ has joined #zuul | 19:09 | |
| bkorren | fungi, that is all ok, I'll just need to have some locking in place to make sure that when a change is merged, no test can start running because the new merged code is built amd made available in a way that makes the tests use it | 19:10 |
| fungi | bkorren: i'm not really sure what that means, but before you determine that implicit change dependencies from a dependent pipeline are the answer to your problem, make sure you're okay with the heuristics described at https://zuul-ci.org/docs/zuul/user/gating.html#testing-in-parallel | 19:11 |
| bkorren | fungi, so that is a and b are not merged - I get tests for both; and if b is merged and a is not - a would be also tested with b because the build containing b had been made available for the test | 19:12 |
| fungi | ahh, so sounds like there's some sort of external state used by these jobs that zuul is unaware of? that's a more complicated model than i'm going to be able to wrap my head around, i think, but if it works then cool i guess | 19:15 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Create tasks that spin up kubernetes and run the operator https://review.opendev.org/668029 | 19:15 |
| bkorren | fungi, yeah - because the test actually needs RPMs - so on merge the RPMs are built and collected in a repo used by the test; and I only build RPMs for patches being tested 'on the fly' | 19:16 |
| bkorren | fungi, so basically yeah, the package repo is the external state | 19:17 |
| *** irclogbot_3 has joined #zuul | 19:21 | |
| *** irclogbot_3 has quit IRC | 19:22 | |
| bkorren | fungi, pabelanger , corvus : thanks for all your help! I gtg for nw, have a great day! | 19:23 |
| fungi | you too! | 19:24 |
| pabelanger | ++ | 19:25 |
| *** bkorren has quit IRC | 19:40 | |
| *** bhavikdbavishi has quit IRC | 19:46 | |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Create tasks that spin up kubernetes and run the operator https://review.opendev.org/668029 | 19:47 |
| *** openstackgerrit has quit IRC | 19:49 | |
| *** hwangbo has quit IRC | 19:53 | |
| *** hwangbo84 has joined #zuul | 19:53 | |
| daniel2 | Looking at the zuul docker compose example, when trying to set the files I get volume must be mapping not array: https://shafer.cc/paste/view/64072f21 | 19:55 |
| daniel2 | Still learning docker compose syntax, so just curious if my syntax is wrong (well it obviously is, but the example isn't very clear) | 19:55 |
| clarkb | daniel2: see https://docs.docker.com/compose/compose-file/compose-file-v2/#long-syntax for an example | 19:58 |
| clarkb | daniel2: I think you want volumes: | 19:58 |
| clarkb | er | 19:58 |
| clarkb | volumes:\n - sshkey: ./path | 19:58 |
| clarkb | and so on | 19:58 |
| daniel2 | That didnt work either | 19:58 |
| daniel2 | ERROR: In file './docker-compose.yaml', volume must be a mapping, not an array. | 19:59 |
| clarkb | oh remove the space between sshkey: and .path | 19:59 |
| clarkb | - sshkey:./path according to their examples | 19:59 |
| clarkb | or use the long format and spell everything out | 19:59 |
| daniel2 | clarkb: same error | 19:59 |
| daniel2 | I'll just remove that volumes section and add them to their spot in the service sections. | 20:00 |
| *** hwangbo84 has quit IRC | 20:13 | |
| *** altlogbot_3 has joined #zuul | 20:28 | |
| *** jamesmcarthur has joined #zuul | 20:29 | |
| *** altlogbot_3 has quit IRC | 20:32 | |
| *** saneax has quit IRC | 20:32 | |
| *** hwangbo has joined #zuul | 20:42 | |
| *** pcaruana has quit IRC | 20:48 | |
| *** openstackgerrit has joined #zuul | 20:58 | |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Create tasks that spin up kubernetes and run the operator https://review.opendev.org/668029 | 20:58 |
| *** mattw4 has quit IRC | 21:00 | |
| *** mattw4 has joined #zuul | 21:00 | |
| *** altlogbot_3 has joined #zuul | 21:05 | |
| *** altlogbot_3 has quit IRC | 21:08 | |
| *** altlogbot_2 has joined #zuul | 21:19 | |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Create tasks that spin up kubernetes and run the operator https://review.opendev.org/668029 | 21:21 |
| *** altlogbot_2 has quit IRC | 21:22 | |
| *** michael-beaver has joined #zuul | 21:24 | |
| pabelanger | jeliu_: left comment on ^ | 21:29 |
| pabelanger | I have to run now | 21:29 |
| jeliu_ | pabelanger thanks! | 21:31 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add install-devstack test job https://review.opendev.org/670195 | 21:36 |
| *** jeliu_ has quit IRC | 21:36 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Normalize test jobs yaml https://review.opendev.org/670198 | 21:44 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-authorized-keys test job https://review.opendev.org/670199 | 21:44 |
| *** irclogbot_1 has joined #zuul | 21:50 | |
| *** irclogbot_1 has quit IRC | 21:52 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-fileserver test job https://review.opendev.org/670201 | 21:55 |
| corvus | fungi: do you have a system where you can generate a throwaway 1024bit rsa gpg key? | 22:05 |
| corvus | fungi: because... and this may not come as a surprise... i have no idea how to convince my workstation to do that anymore | 22:05 |
| fungi | hah... let's find out | 22:05 |
| clarkb | in the future all bits will be >2048 | 22:06 |
| clarkb | thats why its so difficult right? | 22:06 |
| fungi | all bits are taco bell | 22:06 |
| clarkb | yum | 22:07 |
| corvus | no, it's difficult because: gpg: agent_genkey failed: Permission denied | 22:07 |
| corvus | as long as gpg will still import the 1024bit key, that should be fine | 22:07 |
| corvus | i'm just trying to write a job to test the add-gpg role | 22:07 |
| fungi | gpg2 on debian/sid can still do it | 22:08 |
| corvus | fungi: can you pastebin the ascii-armored private key? | 22:09 |
| fungi | pub rsa1024 2019-07-10 [SC] | 22:09 |
| fungi | yup, just a sec | 22:09 |
| corvus | fungi: many thanks :) | 22:09 |
| fungi | btw, all i needed to do was `mkdir test.gnupg&&gpg2 --homedir test.gnupg --full-generate-key` and then take the defaults other than specify 1024 instead of 3072 for the key size | 22:10 |
| corvus | yeah, i did all that, and just got errors because agent | 22:10 |
| fungi | oh, neat | 22:11 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-fileserver test job https://review.opendev.org/670201 | 22:14 |
| fungi | corvus: http://paste.openstack.org/show/754277/ | 22:14 |
| corvus | fungi: thanks! | 22:14 |
| fungi | that's with no symmetric encryption | 22:14 |
| fungi | corvus: for future reference, in https://docs.openstack.org/infra/system-config/signing.html we pass --pinentry-mode loopback | 22:15 |
| fungi | so that might be worth trying sometime | 22:16 |
| fungi | not sure if it still tries to connect to the agent though | 22:16 |
| corvus | oh that could help, thx | 22:16 |
| fungi | there's also a batch option, i think, which doesn't do pinentry at all | 22:17 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-fileserver test job https://review.opendev.org/670201 | 22:17 |
| *** rlandy is now known as rlandy|bbl | 22:19 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-authorized-keys test job https://review.opendev.org/670199 | 22:22 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-fileserver test job https://review.opendev.org/670201 | 22:22 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-gpgkey test job https://review.opendev.org/670206 | 22:22 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-launchpad-credentaials test job https://review.opendev.org/670207 | 22:27 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-sshkey test job https://review.opendev.org/670208 | 22:32 |
| corvus | that's the easy "a" roles :) | 22:34 |
| hwangbo | Can someone explain why the origin remote URL is set to 'file:///dev/null'? | 22:35 |
| hwangbo | For the executor workspaces* | 22:36 |
| corvus | hwangbo: https://zuul-ci.org/docs/zuul/user/jobs.html#git-repositories paragraph 2 | 22:37 |
| hwangbo | Missed that, thanks | 22:38 |
| corvus | hwangbo: the zuul repos represent a speculative future state of the origin; any other value we could put there would produce erroneous info (for example it might say we were ahead of master, when we are actually future-master) | 22:39 |
| fungi | also not including an origin causes some tools like pip to get confused | 22:41 |
| corvus | right, so it has to be there, but it also has to be something we can't fetch from | 22:41 |
| hwangbo | So after Zuul creates the speculative workspace and hands it off to the Ansible job, would there be an issue of adding a remote there? | 22:43 |
| corvus | hwangbo: why? | 22:43 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-fileserver test job https://review.opendev.org/670201 | 22:45 |
| fungi | the main issue i expect is that jobs could easily accidentally replace the prepared git repo state with one from the remote you've added | 22:45 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-gpgkey test job https://review.opendev.org/670206 | 22:47 |
| hwangbo | corvus: our builds make git fetch calls for some ancestry checking I think | 22:47 |
| clarkb | hwangbo: the entire ancestry (including the speculative state) is already in place on the repo | 22:48 |
| clarkb | you should be able to examine the log without doing updates/fetches | 22:48 |
| corvus | right, and doing a fetch is likely to produce incorrect results in the case of more than one change (for example, to see what changed in the commit under test, you need to compare to the origin/branch ref that zuul supplies, rather than a remote branch, otherwise you'll get too many changes) | 22:49 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-launchpad-credentaials test job https://review.opendev.org/670207 | 22:50 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add add-sshkey test job https://review.opendev.org/670208 | 22:51 |
| hwangbo | All right, thanks for the info everyone. I'll work toward removing using remotes in our build | 22:56 |
| *** altlogbot_1 has joined #zuul | 23:05 | |
| *** altlogbot_1 has quit IRC | 23:08 | |
| sgw | Hi guys, got the installation of osc working today with a couple of hiccups, but it's working, so thanks. next hiccup! I am trying to do an ansible find in the src_root and getting an error that it's not a vaild directory | 23:10 |
| *** tosky has quit IRC | 23:10 | |
| clarkb | sgw: can you share the task with us? | 23:13 |
| fungi | yeah, like could it simply be a case of variable expansion gone wrong? | 23:13 |
| fungi | or happening on the wrong host? | 23:13 |
| sgw | knowing me, probably wrong host! my file: paths: "{{ zuul.executor.src_root }}/opendev.org/starlingx/fault" | 23:14 |
| *** altlogbot_3 has joined #zuul | 23:15 | |
| clarkb | zuul.executor.src_root is going to be the src_root path on the executor which would need to run on localhost (which doesn't happen by default) | 23:15 |
| clarkb | dependong on what you are doing you may need to change that to the path on the remote node | 23:15 |
| * clarkb finds what that var is called | 23:15 | |
| *** jamesmcarthur has quit IRC | 23:16 | |
| clarkb | {{ zuul.project.src_dir }} ? | 23:16 |
| sgw | Sorry for the noise, I mis-understood the executor vs project variable sets | 23:17 |
| *** altlogbot_3 has quit IRC | 23:18 | |
| *** jamesmcarthur has joined #zuul | 23:19 | |
| *** altlogbot_1 has joined #zuul | 23:21 | |
| clarkb | sgw: the stuff under executor relates to the environment that is executing ansible on the remote host. It is possible to run jobs that only run on the executor (no remote host) but they are extremely limited in what they can do (mostly http requests are what we have done with it I think) | 23:22 |
| *** jamesmcarthur has quit IRC | 23:24 | |
| *** altlogbot_1 has quit IRC | 23:24 | |
| sgw | clarkb: I am testing with a local instance of zuul and gerrit running on the same machine, but I gather in different containers | 23:25 |
| clarkb | sgw: if this is with the quickstart then ya should be different containers | 23:25 |
| *** jamesmcarthur has joined #zuul | 23:25 | |
| sgw | yeah quickstart | 23:26 |
| *** altlogbot_2 has joined #zuul | 23:27 | |
| sgw | clarkb: yup src_dir was the right one, now to continue the debug into existance! | 23:30 |
| *** altlogbot_2 has quit IRC | 23:30 | |
| *** mattw4 has quit IRC | 23:32 | |
| *** michael-beaver has quit IRC | 23:34 | |
| *** jamesmcarthur has quit IRC | 23:43 | |
| *** jeliu_ has joined #zuul | 23:49 | |
| *** jeliu_ has quit IRC | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!