| @weebull:matrix.org | Hi, just discovering Zuul. | 14:47 |
|---|---|---|
| I'm not very familiar with ansible, and wondering if anyone can recommend something to get me started that will give me enough to be able to use Zuul? I'm finding the normal use case (i.e. running the same thing on large numbers of machines) is confusing me. | ||
| @clarkb:matrix.org | Paul: maybe look at the "standard library" jobs and roles in the zuul-jobs repo: https://opendev.org/zuul/zuul-jobs. Then content in the playbooks and roles dirs. This is zuul specific so may help constrain Ansible to the sort of thing that is interesting here | 15:57 |
| @weebull:matrix.org | > <@clarkb:matrix.org> Paul: maybe look at the "standard library" jobs and roles in the zuul-jobs repo: https://opendev.org/zuul/zuul-jobs. Then content in the playbooks and roles dirs. This is zuul specific so may help constrain Ansible to the sort of thing that is interesting here | 16:00 |
| The jobs would be the files under `zuul.d/`? | ||
| @clarkb:matrix.org | Yes, and they will specify playbooks found under the playbooks/ for and the playbooks will use roles found under the roles/ dir. The stuff in zuul.d defining jobs is zuul specific but then the playbooks and roles are Ansible that zuul leverages | 16:02 |
| @weebull:matrix.org | Clark: Thanks, I think getting that hierarchy of jobs/playbooks/roles/tasks straight in my head is going to be key. | 16:08 |
| @weebull:matrix.org | At least I now know where the interface layer is. | 16:10 |
| @fungicide:matrix.org | Paul: if it helps, you can think of playbooks as the guts of what the job runs (per phase anyway), and roles are just ansible's construct for a modular reusable set of instructions, so playbooks can include roles and set variables for them instead of having all the tasks defined in the playbook itself | 16:14 |
| @fungicide:matrix.org | also it may be useful to start out just looking at playbooks which call a "command" or "shell" task, since ansible can act as a very lightweight wrapper around shell commands too | 16:15 |
| @weebull:matrix.org | fungi: So playbooks don't have to have roles in them, but they promote re-use? | 16:16 |
| @fungicide:matrix.org | yes, roles maye re-use far easier, but playbooks can just be a set of one or more explicit tasks too | 16:16 |
| @fungicide:matrix.org | * yes, roles make re-use far easier, but playbooks can just be a set of one or more explicit tasks too | 16:16 |
| @fungicide:matrix.org | and if you really want a job that just runs some shell script, that's basically one ansible task in an otherwise empty playbook | 16:17 |
| @weebull:matrix.org | So if I have my tests already scripted up for other CI systems, it may be an easy starting point to just have single task "shell" playbooks? | 16:17 |
| @fungicide:matrix.org | yes, in fact that's how we originally managed our migration from jenkins to ansible in the zuul 2.5-3.0 era | 16:18 |
| @weebull:matrix.org | That pretty much what I'm looking at except it's Gitlab CI I'm moving from. | 16:19 |
| Any major disadvantages with that sort of set-up? | ||
| @weebull:matrix.org | * That's pretty much what I'm looking at except it's Gitlab CI I'm moving from. | 16:21 |
| Any major disadvantages with that sort of set-up? | ||
| @fungicide:matrix.org | ansible is an extremely flexible language, so yeah you're basically not able to take advantage of a lot of its features if you only run shell scripts with it, but as a starting point for bootstrapping your jobs it's a fine way to go | 16:22 |
| @fungicide:matrix.org | the most things you move into roles and tasks and parameterize and template out, the more dynamic and composable you can make your jobs | 16:23 |
| @fungicide:matrix.org | * the more things you move into roles and tasks and parameterize and template out, the more dynamic and composable you can make your jobs | 16:23 |
| @weebull:matrix.org | Gives me a starting point at least, and I can worry about taking advantage of ansible later. I assume it's ok to come back here and ask more questions as I have them? ....or have I stumbled into a dev channel? | 16:24 |
| @fungicide:matrix.org | zuul is developed by its users, we don't really make a distinction between user questions and developer questions nor do we believe it's healthy to try to put up barriers between the people asking/answering them. so yes, please ask anything related to zuul in here or on the zuul-discuss mailing list | 16:28 |
| @weebull:matrix.org | I'm sure I'll be back then. Thanks for your help | 16:28 |
| @fungicide:matrix.org | my pleasure! | 16:28 |
| @weebull:matrix.org | One final thing that springs to mind... | 16:47 |
| With ansible as a configuration manager "roles" make perfect sense to me. I want this node to have the role of a mail server. Is the name a little misleading when it comes to zuul's use case though? Presumably you could have a "role" that did anything (launch the missiles) if it's a container of tasks? | ||
| @weebull:matrix.org | * One final thing that springs to mind... | 16:48 |
| With ansible as a configuration manager "roles" make perfect sense to me. I want this node to have the role of a mail server. Is the name a little misleading when it comes to zuul's use case though? Presumably you could have a _role_ that did anything (launch the missiles) if it's a container of tasks? | ||
| @weebull:matrix.org | * One final thing that springs to mind... | 16:48 |
| With ansible as a configuration manager _roles_ make perfect sense to me. I want this node to have the role of a mail server. Is the name a little misleading when it comes to zuul's use case though? Presumably you could have a _role_ that did anything (launch the missiles) if it's a container of tasks? | ||
| @jim:acmegating.com | Paul: you could think of it that way (and I usually do), but if you go *all the way* down the rabbit hole of "use your production playbooks in tests" which is something we like to think about a lot in Zuul, then the two cases become almost the same. | 16:56 |
| @fungicide:matrix.org | if your goal is to "test like production" then it's not as much of a mismatch as you might think, for example we use roles like this one in our tests for the opendev collaboratory's configuration management: https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/nameserver | 16:57 |
| @jim:acmegating.com | Paul: btw, the standard library of zuul jobs has a `run-test-command` job already defined, so you don't even need to write a playbook: https://zuul-ci.org/docs/zuul-jobs/general-jobs.html#job-run-test-command | 16:57 |
| @weebull:matrix.org | fungi: >if your goal is to "test like production" | 17:17 |
| So I'm in hardware design, not network based software. So "test" to me means compile and run simulations on an HPC grid, not commission nodes and deploy to them. | ||
| I think this is why ansible feels strange to me. I know it can do what I need, but all the docs and recipes are geared to problems I'm not trying to solve. My infrastructure isn't dynamic in that way. | ||
| @weebull:matrix.org | > <@jim:acmegating.com> Paul: btw, the standard library of zuul jobs has a `run-test-command` job already defined, so you don't even need to write a playbook: https://zuul-ci.org/docs/zuul-jobs/general-jobs.html#job-run-test-command | 17:17 |
| How handy! | ||
| @jim:acmegating.com | Makes sense. I see it as a spectrum, and Zuul supports both ends: on end you've got the "just run a shell script" approach; on the other you have "pretend these tests nodes are production servers". most people end up somewhere in the middle. a lot of the folks that i work with that do hardware testing end up with a bunch of roles for preparing test environments and then one to kick off a test run. having modularity in the test setup and post-processing/cleanup tends to be useful. | 17:21 |
| @weebull:matrix.org | I'm sure. Wasn't trying to be dismissive of the tech. I expect having a flexible language at that point is extremely useful in my case as much as others. The fact it can span the spectrum is great. | 17:23 |
| @jim:acmegating.com | Oh yeah I didn't take it as dismissive. :) Knowing your use case helps frame it. | 17:24 |
| @jim:acmegating.com | It's *super* flexible, so often the answer is "all of the above" :) | 17:25 |
| @jim:acmegating.com | My mindset is pretty close to yours though, and I have just come to terms with the idea that "role" means "collection of tasks" or "unit of modular reuse". :) | 17:26 |
| @weebull:matrix.org | Can roles use roles then? | 17:27 |
| @jim:acmegating.com | Yep | 17:27 |
| @jim:acmegating.com | (typically via the `include_role` task) | 17:27 |
| @weebull:matrix.org | ...and you'd build general hierarchy at that level rather than _jobs of jobs_ or _PBs of PBs_ or _tasks of tasks_? | 17:29 |
| @weebull:matrix.org | (Did that make sense?) | 17:30 |
| @jim:acmegating.com | Paul: I try to avoid that (but every approach has its uses). My favorite, unless there's a compelling reason otherwise, is to keep roles simple and focused on one thing, then have playbooks be simple lists of roles. then use Zuul job inheritance to combine those playbooks when it makes sense. This approach makes it really easy to make new jobs out of existing parts (just make a new playbook with a short list of roles with whatever difference is needed). and frequently jobs fit into a natural hierarchy, so Zuul's approach of combining playbooks works well there. The zuul-jobs repo is an example of all of that in action; let me grab some links. | 17:34 |
| @jim:acmegating.com | Paul: the `tox` job runs python unit tests -- here's the job: https://opendev.org/zuul/zuul-jobs/src/branch/master/zuul.d/python-jobs.yaml#L1-L48 and it's main playbook: https://opendev.org/zuul/zuul-jobs/src/branch/master/playbooks/tox/run.yaml | 17:36 |
| @jim:acmegating.com | Paul: it inherits from the `unittests` job defined here: https://opendev.org/zuul/zuul-jobs/src/branch/master/zuul.d/general-jobs.yaml#L9-L18 which runs this playbook first: https://opendev.org/zuul/zuul-jobs/src/branch/master/playbooks/unittests/pre.yaml | 17:37 |
| @jim:acmegating.com | * Paul: the `tox` job runs python unit tests -- here's the job: https://opendev.org/zuul/zuul-jobs/src/branch/master/zuul.d/python-jobs.yaml#L1-L48 and its main playbook: https://opendev.org/zuul/zuul-jobs/src/branch/master/playbooks/tox/run.yaml | 17:39 |
| @weebull:matrix.org | So `unittests` deals with environmental aspects, and `tox` the invocation. `tox` is effectively _inside_ `unittests`? | 17:41 |
| @jim:acmegating.com | Paul: yeah, pre- and post-playbooks layer like an onion (see the bullet list at the bottom of https://zuul-ci.org/docs/zuul/latest/config/job.html#job ) | 17:42 |
| @jim:acmegating.com | (So to tie this back to Ansible -- we have all the ways of dealing with modularity/composibility/reuse in Ansible plus one more -- the sequence of playbooks that Zuul runs) | 17:44 |
| @weebull:matrix.org | Presumably each stage (pre / post / main) runs parent first on the way in, and child first on the way out? | 17:45 |
| @weebull:matrix.org | (Sorry, I realise I'm asking a bunch of ansible questions on the Zuul channel.) | 17:45 |
| @jim:acmegating.com | yes -- except that "main" (or actually `run`) is only the child -- it overrides the parent run playbook if there is one. that's why the `unittest` job doesn't actually have a run playbook; it's left for jobs like `tox` to supply. | 17:46 |
| @weebull:matrix.org | With that I think it's probably time for me to go play. | 17:48 |
| @jim:acmegating.com | Paul: have fun! | 17:48 |
| @vlotorev:matrix.org | Hi, I'm confused how to stop/remove Docker containers on static nodes: I create NFS server in pre playbook and stop it in post playbook. When the job is aborted and re-enqueued post playbooks are not called (also logs are not collected). | 18:01 |
| @vlotorev:matrix.org | * Hi, I'm confused how to stop/remove Docker containers on static nodes: I create NFS server in pre playbook and stop it in post playbook. When the job is aborted and re-enqueued post playbooks are not called (also logs are not collected). This means container stays alive forever. | 18:01 |
| @vlotorev:matrix.org | So I'm wondering if I'm missing something and how others are cleaning up resources on static nodes. | 18:19 |
| There are cleanup playbooks, but those seem to be useful only for executor, as in post playbooks SSH key to nodes is removed and cleanup playbooks can't access nodes. | ||
| @vlotorev:matrix.org | * So I'm wondering if I'm missing something and how others are cleaning up resources on static nodes. | 18:19 |
| There are cleanup playbooks, but those seem to be useful only for executor, as in post playbooks SSH key for nodes is removed and cleanup playbooks can't access nodes. | ||
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!