| *** tosky has quit IRC | 00:34 | |
| *** holser has quit IRC | 00:39 | |
| *** zenkuro has quit IRC | 02:00 | |
| *** bhavikdbavishi has joined #zuul | 02:34 | |
| *** bhavikdbavishi has quit IRC | 03:00 | |
| *** bhavikdbavishi has joined #zuul | 03:06 | |
| *** rfolco has quit IRC | 03:08 | |
| *** bhavikdbavishi has quit IRC | 03:57 | |
| *** bhavikdbavishi has joined #zuul | 03:57 | |
| *** bhavikdbavishi has quit IRC | 04:04 | |
| *** bhavikdbavishi has joined #zuul | 04:07 | |
| *** bhavikdbavishi1 has joined #zuul | 04:10 | |
| *** bhavikdbavishi has quit IRC | 04:12 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 04:12 | |
| *** bhavikdbavishi has quit IRC | 04:15 | |
| *** bhavikdbavishi has joined #zuul | 04:16 | |
| *** bhavikdbavishi has quit IRC | 04:20 | |
| *** bhavikdbavishi has joined #zuul | 04:21 | |
| *** ikhan has joined #zuul | 04:27 | |
| *** bhavikdbavishi has quit IRC | 04:34 | |
| *** bhavikdbavishi has joined #zuul | 04:35 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #zuul | 05:33 | |
| *** bhavikdbavishi has quit IRC | 05:42 | |
| *** ykarel has joined #zuul | 05:49 | |
| *** bhavikdbavishi has joined #zuul | 06:11 | |
| *** bhavikdbavishi1 has joined #zuul | 06:14 | |
| *** bhavikdbavishi has quit IRC | 06:16 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 06:16 | |
| *** ikhan has quit IRC | 06:27 | |
| *** bhavikdbavishi has quit IRC | 06:42 | |
| *** ikhan has joined #zuul | 07:05 | |
| *** ikhan has quit IRC | 07:10 | |
| *** ikhan has joined #zuul | 07:16 | |
| *** ikhan has quit IRC | 07:21 | |
| *** ykarel_ has joined #zuul | 07:34 | |
| *** ykarel has quit IRC | 07:37 | |
| *** ykarel_ is now known as ykarel | 07:52 | |
| *** ikhan has joined #zuul | 08:03 | |
| *** ikhan has quit IRC | 09:20 | |
| *** holser has joined #zuul | 09:55 | |
| *** ikhan has joined #zuul | 10:01 | |
| *** holser has quit IRC | 10:35 | |
| *** holser has joined #zuul | 10:46 | |
| *** holser has quit IRC | 10:47 | |
| *** holser_ has joined #zuul | 10:47 | |
| *** sshnaidm|afk is now known as sshnaidm|rover | 10:48 | |
| *** holser_ has quit IRC | 11:04 | |
| *** ikhan has quit IRC | 11:22 | |
| *** tosky has joined #zuul | 11:37 | |
| *** zenkuro has joined #zuul | 12:14 | |
| *** ikhan has joined #zuul | 12:18 | |
| *** ikhan has quit IRC | 12:22 | |
| *** rfolco has joined #zuul | 12:30 | |
| *** ikhan has joined #zuul | 12:56 | |
| *** bhavikdbavishi has joined #zuul | 13:07 | |
| *** sassyn has joined #zuul | 13:12 | |
| sassyn | Hi All | 13:12 |
|---|---|---|
| sassyn | I have a quick question regarding job.dependencies | 13:12 |
| *** iurygregory has quit IRC | 13:13 | |
| sassyn | Assume the following - I have a job name JobX. So when a commit comes in to repo RepoX JobX start to kicks in and run | 13:13 |
| sassyn | when JobX is finished I want to run a JobY | 13:14 |
| sassyn | I guess this can be done via the post run option | 13:14 |
| sassyn | Now consider this: When commit comes into RepoY a job Y kicks in. However I want to define that JobY need to run first JobX | 13:15 |
| sassyn | I gues I can do it via pre run option | 13:15 |
| sassyn | however I'm having trouble to understand what is the job.parent used for? | 13:19 |
| *** bhavikdbavishi has quit IRC | 13:21 | |
| sassyn | and what is the diff within job.requires/provide to dependencies? | 13:26 |
| sassyn | in any way I thinking of it I get into a loop. Cause if JobX run and then have a post run to run JobY all good. but if JobY need to run and run first JobX (which will run jobY) I ended with running (JobX+JobY)-+JobY | 13:32 |
| fungi | job.parent is for inheriting configuration from another job; job dependencies are a direct way of saying that one job needs to wait until another job runs; provides/requires is a more flexible alternative to job dependencies where you could have a variety of different jobs which provide the same thing and it allows you to make a job wait on that rather than waiting on some specific job to finish | 14:02 |
| sassyn | Hi fungi | 14:05 |
| sassyn | when u are saying job.parent is for inheriting configuration from another job what does it really mean? it mean that I inheriting the playbook as well? | 14:06 |
| sassyn | or just the configuration | 14:07 |
| sassyn | things like semaphore; nodeset; nodes; timeout: 3600; post-timeout: 3600; attempts: | 14:08 |
| sassyn | Ok I see in the docs now that inherits is everything I miss that | 14:10 |
| sassyn | so in jobB inherits JobA why is the different when using JobB dependence on JobA? | 14:11 |
| sassyn | what* not hwy* | 14:11 |
| sassyn | why* | 14:11 |
| *** ikhan has quit IRC | 14:21 | |
| fungi | if jobB declares jobA is its parent, then the build will be like running jobA except with the differences you've overridden in jobB | 14:34 |
| fungi | if jobB depends on jobA, then you have two separate builds: first a build of jobA runs, then after that gets a result a build of jobB can run | 14:35 |
| fungi | job.parent is used in defining the details of a job (for example if you have several jobs which are almost the same but with slight differences you don't have to redefine the common parts in each one), dependencies are used to define relationships between separate jobs | 14:36 |
| *** bhavikdbavishi has joined #zuul | 14:55 | |
| *** zenkuro has quit IRC | 15:09 | |
| *** zenkuro has joined #zuul | 15:09 | |
| *** ikhan has joined #zuul | 15:10 | |
| *** bhavikdbavishi has quit IRC | 15:31 | |
| *** ikhan has quit IRC | 15:33 | |
| sassyn | thank u fungi. | 15:39 |
| *** ykarel has quit IRC | 15:41 | |
| sassyn | So consider the following: RepoX define to run JobX. so when commit comes in to RepoX - JobX start and once finish it call to JobY [ JobX compile a code, JobY test the code). When a commit come to Repo Y ( the test code was changed) we need to run JobX and after JobY... how can I make it so without getting in a loop | 15:44 |
| *** noonedeadpunk_ is now known as noonedeadpunk | 16:01 | |
| fungi | job dependencies are declared where you run the jobs, not as part of the job definitions, so you just specify them in the project-pipeline for each project where you list them to run | 16:19 |
| sassyn | I read all the docs. I would love to get help. I can't think of a way of doing this without getting into loop | 16:19 |
| * fungi looks up an example | 16:20 | |
| sassyn | But if I set JobY as depend of JobX - when Job Y will run Job X will first run and then JobY | 16:21 |
| sassyn | But if I set in Job X that Job Y is a post -job I will get a state if JobX + JobY + JobY | 16:21 |
| sassyn | I want to get into a state when I commit to repoX jobX is run and then JobY, JobZ .... are running in parallel to check the code. | 16:22 |
| fungi | when you add jobs to the project-pipeline for repoX you tell it to run jobX and then jobY by saying that in that project-pipeline jobY requires jobX. | 16:22 |
| fungi | when you add jobs to the project-pipeline for repoY you tell it to run jobY and then jobX by saying that in that project-pipeline jobX requires jobY. | 16:23 |
| sassyn | If however I commit to RepoY I want to run first JobX and then JobY only. | 16:23 |
| fungi | here's an example in nodepool's check project-pipeline: https://opendev.org/zuul/nodepool/src/branch/master/.zuul.yaml#L375-L377 | 16:24 |
| fungi | so if a change is proposed for nodepool, then it runs zuul-quick-start only after running nodepool-build-image and some job providing zuul-container-image | 16:26 |
| *** ikhan has joined #zuul | 16:26 | |
| sassyn | I'm checking thank u | 16:26 |
| fungi | but that's specific to changes proposed for nodepool. those same job dependencies and requirements could be rearranged in zuul-client's check project-pipeline if we wanted | 16:26 |
| fungi | basically the requirements and dependencies for jobs run when proposing a change for repoX don't have to be the same as for changes to repoY, they're set independently for each repo | 16:28 |
| fungi | that's how you avoid cycles | 16:28 |
| sassyn | I'm trying to understand it... | 16:28 |
| fungi | here's a corresponding example from zuul's check project-pipeline: https://opendev.org/zuul/zuul/src/branch/master/.zuul.yaml#L262-L264 | 16:31 |
| fungi | you'll see the requires and dependencies are slightly different | 16:32 |
| fungi | so same job run in check for changes to nodepool and zuul repos, but with different requires and dependencies in each one | 16:32 |
| sassyn | I think I understand | 16:36 |
| sassyn | u declarer the setting in the job pipeline and not in the job itself... | 16:36 |
| sassyn | So JobX have dependencies and requires of it's own.... | 16:37 |
| sassyn | and JobY have requires and dependencies of it's own | 16:37 |
| *** bhavikdbavishi has joined #zuul | 16:40 | |
| *** iurygregory has joined #zuul | 16:41 | |
| fungi | yep | 16:50 |
| sassyn | OK | 16:52 |
| fungi | to be clear, it's project-pipelines (not sure what a job pipeline is) | 16:53 |
| sassyn | Yes That's what I meant | 16:53 |
| sassyn | but provide can't be on project-pipelines | 16:53 |
| fungi | and i was saying that jobX and jobY can have different requires and dependencies in the repoX project-pipeline compared to in the repoY project-pipeline | 16:53 |
| sassyn | understand! | 16:54 |
| *** hamalq has joined #zuul | 16:54 | |
| sassyn | my last question is if RepoX have JobY and JobZ and JobM etc... can all run in parallel ? | 16:54 |
| sassyn | So once RepoX is finished (code was compiles) all test starting | 16:55 |
| fungi | yes, normally builds run in parallel (so long as you have enough node resources to run them all), unless you declare requires/dependencies | 16:57 |
| fungi | and then it will still run the jobs it can in parallel as long as they're not waiting on results from another job which hasn't finished | 16:58 |
| sassyn | OK, let me try | 16:59 |
| *** hamalq has quit IRC | 16:59 | |
| sassyn | TODA (thank u in hebrew) | 16:59 |
| *** hamalq has joined #zuul | 17:00 | |
| sassyn | Last question :-) If I may fungi | 17:01 |
| fungi | you're welcome! | 17:01 |
| fungi | sure, ask | 17:01 |
| sassyn | I set in the pipleine JobX and then JobY under the project pipeline of project X. JobY is set to require JobX (where in the jobX i put the provide: JobX). This makes JobX run first and only when it finish JobY will start. If I put JobZ as well do I need to put require as well? | 17:06 |
| sassyn | does this mean JobX will run twise? | 17:08 |
| sassyn | see here https://pastebin.com/s6tns0MS | 17:12 |
| sassyn | ? | 17:29 |
| sassyn | Can I somehow update the document? | 17:29 |
| sassyn | It might help others as well | 17:30 |
| fungi | is there any relationship between jobZ and jobX or jobY? if not, then jobX and jobZ will start at roughly the same time, but jobY will still wait for jobX to complete before starting | 17:31 |
| fungi | if jobY and jobZ both declare dependencies on jobX, then jobX will run by itself, and both jobY and jobZ will start at roughly the same time once jobX completes | 17:33 |
| *** bhavikdbavishi has quit IRC | 17:33 | |
| fungi | also which document are you asking about updating? all of zuul's documentation is in the zuul git repository, under doc/source | 17:34 |
| sassyn | OK | 17:35 |
| sassyn | I will check them out maybe it is better to say about the job pipeline in the doc | 17:35 |
| sassyn | I understand what u are saying. but if I put JobY: | 17:36 |
| sassyn | The last thing I need to understand now is How to I pass ansible VARs from jobX to JobY.... | 17:37 |
| fungi | you can zuul-return values from a build when it completes, and reference those in a depending job | 17:47 |
| fungi | sassyn: https://zuul-ci.org/docs/zuul/reference/jobs.html#return-values | 17:48 |
| fungi | seems like that does exactly what you're asking for | 17:49 |
| sassyn | What about set_fact with cacheable | 17:50 |
| fungi | i think setting/caching facts only works within a build, not between multiple builds, but someone else would have to confirm | 17:52 |
| clarkb | fungi: sassyn yes that is the case since each build runs in a throwaway env | 17:53 |
| clarkb | that means facts set in pre run can persist through to post run if set to cacheable but then they are discarded | 17:53 |
| sassyn | let's give it a try | 17:54 |
| fungi | got it, so cached facts work between playbooks in the same build, but not from one build to another. for that you need zuul_return to have one build set variables which you reference in playbooks for a depending build | 17:54 |
| clarkb | fungi: yup | 17:55 |
| *** hamalq has quit IRC | 18:04 | |
| sassyn | clarkb it is not working in my case ..... in the pipeline there is JobX and then JobY - JobX as set facts a var name FS with cacheable, but JobY get an error that it doesn't know what it is. | 18:12 |
| sassyn | what I'm missing? | 18:12 |
| clarkb | sassyn: jobX and jobY are different builds so the facts don't cross between them | 18:12 |
| sassyn | understand. | 18:12 |
| sassyn | so only pre/post | 18:13 |
| clarkb | no, within one build | 18:13 |
| clarkb | (a build is a running instance of a job) | 18:13 |
| clarkb | JobX and JobY will have different builds as they are distnct | 18:13 |
| clarkb | to pass between builds you need to use zuul_return | 18:14 |
| sassyn | So if JobX have a pre run and post run then it will work with cacheable? | 18:15 |
| sassyn | bit not with JobX and JobY | 18:15 |
| sassyn | but* | 18:15 |
| clarkb | yes jobX runs in envX and jobY runs in envY. The fact caches for each build are per env | 18:15 |
| sassyn | OK | 18:16 |
| sassyn | Thank u very much! | 18:16 |
| fungi | sassyn: like we said before, if you want to pass values between build (that is, between runs of different jobs) use zuul_return | 18:20 |
| *** ikhan has quit IRC | 18:25 | |
| *** holser has joined #zuul | 18:28 | |
| *** holser has quit IRC | 18:34 | |
| *** holser_ has joined #zuul | 18:34 | |
| *** mgoddard has joined #zuul | 18:43 | |
| *** ikhan has joined #zuul | 18:45 | |
| *** holser_ has quit IRC | 19:11 | |
| *** holser has joined #zuul | 19:11 | |
| *** holser has quit IRC | 19:14 | |
| *** holser_ has joined #zuul | 19:14 | |
| *** ikhan has quit IRC | 19:14 | |
| sassyn | fungi thank u | 19:15 |
| sassyn | once I have setup in the JobX the follwing: | 19:15 |
| *** ikhan has joined #zuul | 19:16 | |
| sassyn | https://pastebin.pl/view/050fd701 | 19:16 |
| sassyn | How do I use it in the JobY ansible? | 19:16 |
| sassyn | by using zuul.trigger_build_number? | 19:16 |
| sassyn | for example | 19:16 |
| *** ikhan has quit IRC | 19:17 | |
| clarkb | sassyn: https://zuul-ci.org/docs/zuul/reference/jobs.html#return-values "Any values other than those in the zuul hierarchy will be supplied as Ansible variables to dependent jobs. These variables have less precedence than any other type of variable in Zuul, so be sure their names are not shared by any job variables." | 19:17 |
| sassyn | so just call the name as is? | 19:18 |
| clarkb | yup I think so | 19:18 |
| *** paladox has quit IRC | 19:19 | |
| sassyn | No it is not working | 19:21 |
| sassyn | The task includes an option with an undefined variable. The error was: 'mount_jenkins' is undefined | 19:21 |
| sassyn | Even if I have mount_jenkins set in the zuul_return: | 19:23 |
| sassyn | for example here https://pastebin.pl/view/050fd701 | 19:24 |
| sassyn | fungi maybe u know? | 19:24 |
| clarkb | can you check the job's inventory? also is jobY set up to run after jobX as a dependent job? | 19:25 |
| sassyn | JobY is requires of JobX | 19:28 |
| *** paladox has joined #zuul | 19:34 | |
| clarkb | so jobX provides foo and jobY requires foo? | 19:36 |
| clarkb | that should be sufficient. I would still double check the inventory to ensure that the variable pathing is correct | 19:37 |
| sassyn | https://pastebin.pl/view/941cf4ca | 19:41 |
| fungi | yes, if your variable is returned by jobX then it should appear in the inventory for jobY | 19:41 |
| sassyn | here it is the inventory.yaml | 19:42 |
| sassyn | for JOBY | 19:42 |
| clarkb | I agree none of the jenkins stuff seems to be in there | 19:49 |
| fungi | this is an example of where we use zuul_return in the zuul-jobs repo to pass buildset registry connection/credentials info to other builds: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/buildset-artifacts-location/tasks/main.yaml#L8-L15 | 19:53 |
| fungi | er, buildset artifact locations i mean | 19:54 |
| * fungi had multiple examples up | 19:55 | |
| sassyn | maybe delegate_to: localhost | 19:55 |
| sassyn | is the key? to make it work? | 19:55 |
| sassyn | so it will run on the executor and not on the node pool? | 19:57 |
| fungi | shouldn't matter, here's the buildset registry example: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/run-buildset-registry/tasks/main.yaml#L69-L72 | 19:59 |
| fungi | doesn't delegate_to | 19:59 |
| sassyn | OK | 20:01 |
| sassyn | Will try again now | 20:01 |
| sassyn | it is not working :-( | 20:15 |
| *** ikhan has joined #zuul | 20:16 | |
| *** ikhan has quit IRC | 20:21 | |
| fungi | here's an example inventory from a recent build result: https://zuul.opendev.org/t/zuul/build/92e660bb272f4ad896bda525ea88389f/log/zuul-info/inventory.yaml#24 | 20:24 |
| fungi | that's showing the buildset_registry returned from another job appearing in the inventory | 20:32 |
| sassyn | thansk | 20:32 |
| sassyn | tahnk u | 20:32 |
| fungi | so it's working for us | 20:32 |
| sassyn | yes | 20:33 |
| sassyn | but not for me :-( | 20:33 |
| sassyn | https://pastebin.com/pefJZUEm | 20:34 |
| sassyn | this is what I have in the repoX | 20:34 |
| sassyn | in the .zuul.yaml file | 20:34 |
| sassyn | and this is the jobs https://pastebin.com/R7UVkctw | 20:36 |
| sassyn | what do I miss? | 20:42 |
| sassyn | I'm lost :-( | 20:50 |
| *** rfolco has quit IRC | 20:51 | |
| sassyn | maybe the requires not working right? | 20:52 |
| *** rfolco has joined #zuul | 20:52 | |
| *** rfolco has quit IRC | 20:54 | |
| *** rfolco has joined #zuul | 20:54 | |
| *** akrpan-pure has joined #zuul | 20:56 | |
| akrpan-pure | Does anyone know where I could find documentation over how nodepool elements work? Like what directory structure a new element should be in? | 20:56 |
| akrpan-pure | Ope nvm, my bad, forgot that elements came from diskimage-builder | 20:57 |
| akrpan-pure | Found https://docs.openstack.org/diskimage-builder/latest/developer/developing_elements.html | 20:57 |
| *** rfolco has quit IRC | 20:59 | |
| sassyn | ping clarkb fungi | 21:07 |
| *** holser_ has quit IRC | 21:09 | |
| *** holser has joined #zuul | 21:12 | |
| sassyn | I put everything here: https://pastebin.com/6DZNbiUL | 21:12 |
| sassyn | maybe please u can take a look? | 21:13 |
| clarkb | sassyn: your provides and requires don't match up properly | 21:34 |
| clarkb | j vs J | 21:34 |
| clarkb | whether that is a transcription issue or an actual issue I can't say | 21:34 |
| sassyn | clarkb what do u mean? | 21:43 |
| sassyn | j vs J? | 21:43 |
| sassyn | hmm i see | 21:44 |
| sassyn | let me fix | 21:44 |
| clarkb | requires: JobX and provides: jobX | 21:44 |
| clarkb | they need to match, that key is used to pair things up | 21:44 |
| sassyn | it just a typo in the pastebin | 21:45 |
| sassyn | cause in real life i have different jobs names... due to security I change to a fake one | 21:45 |
| sassyn | in real life it is good | 21:45 |
| clarkb | right but that is all I have to go off of :) so I'm just correcting what I see is wrong | 21:45 |
| clarkb | otherwise that looks about right to me. | 21:46 |
| sassyn | but it still doesn't work :-( | 21:46 |
| sassyn | if u think the logical is good... besides the upper case lower case issue in pastebin | 21:46 |
| sassyn | thank u | 21:53 |
| sassyn | maybe fungi can think about some thing else? | 21:53 |
| clarkb | you might try inspecting your executor logs for zuul jobx to see if the zuul return is processed | 21:53 |
| clarkb | and then trace it from there | 21:53 |
| sassyn | nothing on the logs all seems good | 22:11 |
| sassyn | HELP! :( | 22:12 |
| sassyn | I'm running in debug mode so I have good logs | 22:12 |
| clarkb | you should haev logs for the zuul return itself right? to see that it runs properly? | 22:14 |
| sassyn | in the executor ? | 22:15 |
| clarkb | yes | 22:15 |
| sassyn | what should I grep in the file? | 22:16 |
| sassyn | i see some error like this: ges/ara/plugins/actions/ara_read.py) as it seems to be invalid: module' | 22:20 |
| clarkb | I think that one is expected | 22:21 |
| clarkb | if you grep the build uuid you will get all the logs associated to that build | 22:21 |
| sassyn | really thank you for the time and help | 22:23 |
| sassyn | i restarted the executor and clean up the logs | 22:23 |
| sassyn | and re run | 22:23 |
| sassyn | logs seems Ok | 22:29 |
| sassyn | in the web interface | 22:29 |
| sassyn | I can see that the ansible of Job X completed with no errors. | 22:29 |
| clarkb | the other thing to try is to make an explicit dependency rather than using provides and requires | 22:29 |
| clarkb | https://zuul-ci.org/docs/zuul/reference/job_def.html#attr-job.dependencies possible that is required for the zuul_return? | 22:30 |
| sassyn | trying with job.dependencies | 22:42 |
| sassyn | clarkb this was working! | 22:52 |
| *** hamalq has joined #zuul | 23:09 | |
| *** hamalq has quit IRC | 23:13 | |
| fungi | sassyn: awesome, so sounds like "requires" doesn't pull in zuul_return values like "dependencies" does? in that case, might be something we want to fix for better parity between the two mechanisms | 23:17 |
| sassyn | yes | 23:18 |
| sassyn | agree | 23:18 |
| sassyn | thank u | 23:18 |
| *** hamalq has joined #zuul | 23:25 | |
| *** hamalq has quit IRC | 23:29 | |
| *** holser has quit IRC | 23:30 | |
| fungi | sassyn: i've filed https://storyboard.openstack.org/#!/story/2008389 ( so we don't forget | 23:30 |
| fungi | "Support job.requires with zuul_return module" | 23:30 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!