| corvus | clarkb: thinking further on that, i don't think there's a blocker for creating a 'buildset' page in the webapp now. | 00:00 |
|---|---|---|
| corvus | i think we have all the infrastructure needed for that. | 00:00 |
| clarkb | do we need to record buildset uuids? | 00:01 |
| clarkb | builds have a uuid but not buildsets I don't think | 00:01 |
| corvus | clarkb: they do have uuids. there's a buildset table already, which i hope has that as a column | 00:01 |
| clarkb | ah | 00:01 |
| gundalow | OK, so I need to work out which variables represent the rest of that URL | 00:02 |
| gundalow | which is 1) PR 2) job | 00:02 |
| corvus | gundalow: to be clear, the checks tab at the top of the PR (with conversation, commits, checks, files changed) is new, and no one has written support in zuul for it yet | 00:07 |
| corvus | gundalow: the existing support is to set the status url of the checks comment-box-thingy at the bottom of the PR. note that users don't even see the checks comment-box-thingy unless they are logged in | 00:08 |
| *** clarkb has quit IRC | 00:09 | |
| *** elyezer has quit IRC | 00:10 | |
| gundalow | corvus: yup, asking about the details box at the bottom of the main page | 00:13 |
| corvus | gundalow: the log urls you have have the buildset at the *end* of the url, after the job, which means that there is no single url with a summary of all the jobs run for a pull request | 00:14 |
| corvus | gundalow: https://ansible.softwarefactory-project.io/logs/4/4/f5c6eca18c78cda01c9432e88b66dbde8a33aece/check/ansible-test-network-integration-vyos-py2/195a67f/ | 00:14 |
| corvus | gundalow: that's: /logs/pr[-2:]/pr/commit/pipeline/job/short_buildset_uuid | 00:15 |
| gundalow | oh, so link to https://ansible.softwarefactory-project.io/logs/4/4/f5c6eca18c78cda01c9432e88b66dbde8a33aece/check/ ? | 00:15 |
| corvus | gundalow: that location is all builds of all jobs for that pr | 00:16 |
| corvus | gundalow: if you 'recheck' you'll get another entry at https://ansible.softwarefactory-project.io/logs/4/4/f5c6eca18c78cda01c9432e88b66dbde8a33aece/check/ansible-test-network-integration-vyos-py2/ | 00:16 |
| gundalow | ah, so I do want with 195a67f/ as that's the last commit | 00:16 |
| corvus | well, it's the last build. if the commit on the pr changes, the f56e part will change | 00:17 |
| corvus | gundalow: https://ansible.softwarefactory-project.io/logs/4/4/f5c6eca18c78cda01c9432e88b66dbde8a33aece is probably the best you can do under the circumstances, but it's far from ideal. | 00:18 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: trigger-readthedocs: Move secret bits into a dict https://review.openstack.org/587767 | 00:19 |
| corvus | gundalow: the variables involved there are change.number and change.patchset | 00:19 |
| corvus | gundalow: whether or not the softwarefactory folks consider that a stable interface, i don't know. | 00:20 |
| corvus | gundalow: my personal recommendation would be to avoid doing this because i think that the log url should be treated as opaque data, and the user experience of browsing at that url is pretty confusing. if someone has some time to add a 'buildset' page to the zuul webapp, that would be a much better solution and would work here. | 00:21 |
| corvus | gundalow: but if you want to do this temporarily, those variable names should get you moving | 00:22 |
| gundalow | This is mainly for failing tests, we want to match Shippable's functionality, people know to look for logs at the end | 00:22 |
| gundalow | ie look at the end of https://github.com/ansible/ansible/pull/43576 | 00:22 |
| corvus | gundalow: well, i see nothing at the end because i'm not logged in to github | 00:23 |
| corvus | oh the little red "X" is a link to shippable | 00:24 |
| corvus | https://screenshots.firefox.com/1TstE3nuXtLDIdNC/github.com | 00:24 |
| corvus | that's the not-logged-in view | 00:24 |
| gundalow | Yup and the "Details" link to the right | 00:24 |
| corvus | i don't see a details link | 00:24 |
| gundalow | ah, interesting | 00:24 |
| gundalow | if you log in | 00:24 |
| gundalow | people doing Ansible development will be logged in | 00:25 |
| corvus | i actually almost never log in to github, even when i do use it (i open prs from the command line) | 00:26 |
| corvus | but you know your users and of course should tailor the experience for them. i only hope to convey that the check comment-box-thingy at the bottom has some serious and non-obvious ui issues :) | 00:27 |
| corvus | gundalow: anyway, there isn't a direct equivalent to that shippable link yet. that will be the buildset page in the zuul webapp, once someone writes that (and it shouldn't be too difficult) | 00:29 |
| corvus | the buildset page will be able to link to the appropriate builds of all the individual jobs for the latest run of the pr | 00:30 |
| gundalow | corvus: What's the `buildset` page? The right information is currently added as a comment: on `linters : SUCCESS in 1m 19s` | 00:31 |
| gundalow | where `linters` is a link | 00:32 |
| corvus | gundalow: that's only a single build of a single job; when you run two jobs you'll have 2 links there | 00:32 |
| gundalow | ah, I see | 00:34 |
| corvus | there's no way to set 2 links as a status in github, so we need a new page to link people to which has the summary of all the jobs | 00:34 |
| corvus | (however, once zuul has support for github's new checks api, the UI there is actually much more suited to zuul. take a look at the samples here https://github.com/ansible/ansible/pull/43576/checks -- that box on the left is exactly the way that zuul normally presents info) | 00:35 |
| corvus | progress on either the buildset page or the checks api would be helpful here | 00:36 |
| corvus | (we ultimately will need both of them i think) | 00:36 |
| gundalow | Yup, think you are right there | 00:46 |
| gundalow | We have contacted Shippable to see what their plans are for the new Checks API. One thing we haven't checked yet is if you can use Checks API to display information that isn't tied to a specific line in a file | 00:48 |
| gundalow | with `ansible-test` we generate junit files, so that should be easy to link to file:line | 00:48 |
| gundalow | corvus: What do you think we need to do to progress this? | 00:49 |
| gundalow | I don't think this blocks us putting it live for ansible/ansible | 00:51 |
| gundalow | Just a slightly different UI to shippable, though it's still clear where to look for issues | 01:01 |
| gundalow | On a different topic, one thing we might need to do is use ansible-test to build the `files:` would be great if that was dynamic, though don't think that's an option | 01:02 |
| *** EmilienM has quit IRC | 01:02 | |
| gundalow | as the parsing of `files:` seems to be done before much o fthe job runs at all | 01:02 |
| gundalow | `ansible-test` has a load of logic for following Python imports and working out given a git diff which tests need to be run | 01:03 |
| gundalow | ie if lib/ansible/modules/network/vyos/vyos_command.py is change, only run the vyos_command integration tests | 01:03 |
| *** EmilienM has joined #zuul | 01:03 | |
| gundalow | though if lib/ansible/module_utils/network/vyos/* is changed run all the vyos tests | 01:03 |
| gundalow | which is really useful as some of the full test suites are many hours per platforms | 01:04 |
| *** swest has quit IRC | 01:52 | |
| *** swest has joined #zuul | 02:16 | |
| *** clarkb has joined #zuul | 03:05 | |
| tobiash | gundalow: a dynamic files filter does not exist atm but you can leverage job hierarchies and skipping child jobs via zuul_return | 03:51 |
| tobiash | https://zuul-ci.org/docs/zuul/user/jobs.html#return-values | 03:52 |
| gundalow | tobiash: interesting. Can I trigger things to run via that before having to run the main job and waste time booting the nodesets that will not be used? | 04:35 |
| tobiash | gundalow: you can run many jobs per change and also define dependencies between them. So the idea could be to have a job that runs before all other jobs and decides which will be run. | 04:38 |
| tobiash | gundalow: like http://paste.openstack.org/show/727114/ | 04:40 |
| tobiash | that will make 'job-a' and 'job-b' run after 'decide-job' | 04:40 |
| tobiash | and if decide-job returns a list of child jobs to run via zuul_return the children zuul runs will be filtered by that list | 04:41 |
| tobiash | the job result is the skipped for a child that isn't run and there also won't be a vm booted for the skipped jobs | 04:41 |
| tobiash | with that you can build up a full directed acyclic graph of jobs to run | 04:44 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: trigger-readthedocs: fix typo https://review.openstack.org/588137 | 04:52 |
| gundalow | tobiash: interesting. Can one job return facts that another job uses? | 05:02 |
| tristanC | gundalow: iirc zuul_return data that are not named zuul are passed as variable to child jobs | 05:13 |
| tobiash | Yes, that's correct | 05:15 |
| gundalow | tristanC: tobiash wonder if something like this would work https://etherpad.openstack.org/p/gundalow | 05:20 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: trigger-readthedocs: fix typo https://review.openstack.org/588137 | 05:23 |
| tristanC | gundalow: not sure the list can be extended, it may get replaced by the last zuul_return if multiple list are set | 05:39 |
| tobiash | gundalow: added some comments to this etherpad | 05:47 |
| tobiash | I think the general approach can work if you obey these notes | 05:47 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: Debugging for readthedoc web ping https://review.openstack.org/588146 | 06:27 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul-jobs master: Require at least openstacksdk 0.17.1 https://review.openstack.org/588149 | 06:41 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul-jobs master: Fix comparison with wrong mime type https://review.openstack.org/588150 | 06:43 |
| *** snapiri has joined #zuul | 07:30 | |
| *** snapiri has quit IRC | 07:32 | |
| *** snapiri has joined #zuul | 07:32 | |
| *** gouthamr has quit IRC | 07:36 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul-jobs master: Fixup header/footer https://review.openstack.org/588163 | 07:44 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul-jobs master: Only link timestamps https://review.openstack.org/588164 | 07:44 |
| tobiash | corvus: this is a fix and a suggestion to htmlify that makes it work in our environment ^ | 07:45 |
| *** SotK has quit IRC | 07:45 | |
| *** eandersson has quit IRC | 07:46 | |
| tobiash | corvus: feel free to squash this into your change or just ignore it :) | 07:46 |
| *** leifmadsen_ has quit IRC | 07:47 | |
| *** leifmadsen has joined #zuul | 07:50 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Debugging for readthedoc web ping https://review.openstack.org/588146 | 07:50 |
| *** quiquell has joined #zuul | 08:09 | |
| quiquell | Good morning | 08:09 |
| quiquell | Have a question about how to reproduce a zuul run locally | 08:10 |
| quiquell | It's possible to startup a local zuul and point it to a specific change in gerrit towards my openstack ? | 08:10 |
| *** jpena|off is now known as jpena | 08:12 | |
| *** jlvillal has joined #zuul | 08:29 | |
| tobiash | quiquell: in principle that might be possible but it would be *much* work | 08:33 |
| quiquell | tobiash: ack | 08:35 |
| tobiash | corvus: I got a bug report from a user that zuul matches wrong project-templates under certain circumstances | 08:37 |
| *** electrofelix has joined #zuul | 08:37 | |
| tobiash | corvus: the analysis showed that he has two branches with config and pulling in different project templates | 08:37 |
| tobiash | corvus: like foo and foo-bar while a change to foo-bar matches both project pipelines | 08:38 |
| tobiash | corvus: it looks that the implied branch matcher doesn't do a full match | 08:38 |
| *** pcaruana has joined #zuul | 09:00 | |
| *** SotK has joined #zuul | 09:06 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: Revert "Debugging for readthedoc web ping" https://review.openstack.org/588186 | 09:18 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Revert "Debugging for readthedoc web ping" https://review.openstack.org/588186 | 10:04 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix wrong matched project template https://review.openstack.org/588201 | 10:19 |
| tobiash | corvus: this fixes that behavior ^ | 10:20 |
| *** panda|rover|bbl is now known as panda|rover | 10:47 | |
| *** jpena is now known as jpena|lunch | 11:04 | |
| *** maxamillion has quit IRC | 11:04 | |
| *** mattclay has quit IRC | 11:04 | |
| *** gregdek has quit IRC | 11:04 | |
| *** zxiiro has quit IRC | 11:04 | |
| *** samccann has joined #zuul | 11:15 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /{tenant}/pipelines route https://review.openstack.org/541521 | 11:25 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: scheduler: add job's parent name to the rpc job_list method https://review.openstack.org/573473 | 11:27 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /{tenant}/labels route https://review.openstack.org/553979 | 11:32 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /{tenant}/nodes route https://review.openstack.org/553998 | 11:32 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: dashboard: add /{tenant}/job.html page to display job details https://review.openstack.org/535545 | 11:44 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: dashboard: add /{tenant}/projects.html web page https://review.openstack.org/537870 | 11:44 |
| *** elyezer has joined #zuul | 12:02 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: dashboard: add /{tenant}/labels.html web page https://review.openstack.org/553980 | 12:15 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: dashboard: add /{tenant}/nodes.html web page https://review.openstack.org/553999 | 12:16 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: dashboard: add jobs graph rendering https://review.openstack.org/537869 | 12:16 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: dashboard: add project pipeline rendering https://review.openstack.org/537871 | 12:17 |
| *** zxiiro has joined #zuul | 13:03 | |
| *** maxamillion has joined #zuul | 13:28 | |
| *** jpena|lunch is now known as jpena | 13:33 | |
| *** quiquell is now known as quiquell|off | 13:43 | |
| *** quiquell|off has quit IRC | 13:48 | |
| *** shachar has joined #zuul | 13:53 | |
| *** shachar has quit IRC | 13:54 | |
| corvus | tobiash: unfortunately there's a whole lot of log files in openstack jobs with random timestamp formats. i think we can add support for them, but in the mean time, we should probably fall back on linking the whole line if there's no timestamp, so that we get something. | 13:59 |
| corvus | tobiash: and it might be good to structure that as a list of regexes or something, since we know there are a lot of formats | 14:00 |
| dmsimard | mordred: is it known that zuul_stream doesn't provide the runtime for every task ? | 14:34 |
| dmsimard | mordred: I was trying to compare how long it took to upload logs upstream vs rdo and there's no runtime for the log upload task at the very end of http://logs.openstack.org/93/582493/6/gate/tripleo-ci-centos-7-containers-multinode/2364aac/job-output.txt.gz but you can see runtime in some other tasks intermittently | 14:35 |
| mordred | dmsimard: I don't think it's known? | 14:35 |
| dmsimard | hmmm, I wonder why | 14:37 |
| dmsimard | mordred: there's no runtime here https://github.com/openstack-infra/zuul/blob/master/zuul/ansible/callback/zuul_stream.py#L430 | 14:37 |
| dmsimard | could it be as simple as that ? | 14:37 |
| dmsimard | or here https://github.com/openstack-infra/zuul/blob/master/zuul/ansible/callback/zuul_stream.py#L417-L423 | 14:39 |
| dmsimard | unrelated: clever use of format with dict keys, didn't realize you could do that and it just ignored other keys | 14:40 |
| pabelanger | we could also consider using | 14:40 |
| pabelanger | callback_whitelist = profile_tasks, timer | 14:40 |
| pabelanger | to give extra output on tasks | 14:40 |
| pabelanger | http://samdoran.com/performance-tuning-ansible-playbooks/ is a good write up | 14:40 |
| dmsimard | pabelanger: I thought about that too first but then profile_tasks adds lines to the output which would probably mangle the "tidy" output that zuul_stream does | 14:41 |
| dmsimard | or maybe it wouldn't, I dunno. | 14:41 |
| dmsimard | pabelanger: fwiw I really like this website which demonstrates what each callback does in practice: https://rndmh3ro.github.io/ | 14:42 |
| pabelanger | could I get a few reviews on https://review.openstack.org/557947/ adds test-emit-job-header so we can nodepool info | 14:53 |
| *** gouthamr has joined #zuul | 15:01 | |
| tobiash | corvus: how does the current htmlify work in openstack? There only the timestamp seems to be linked. | 15:12 |
| tobiash | does it do it more sophisticated or is it tailored to zuul logs? | 15:12 |
| clarkb | tobiash: it has whitelisted filenames that it knows it can parse then applies a set of regexes to make sense of them | 15:12 |
| clarkb | tobiash: it is fairly tairlored ot the logs that openstack ci produces for openstack services | 15:13 |
| tobiash | understood | 15:13 |
| clarkb | as for linking lines without timestamps the existing os-loganalyze tool will create a block back to the last timestamp iirc | 15:16 |
| clarkb | and if there are no timestamp at all then the whole thing is one giant block? | 15:17 |
| corvus | i think we can just have a list of regexes for different timestamp formats, and if one matches, use it. otherwise, link the whole line. | 15:24 |
| dmsimard | corvus: could it be as simple of simply linking log lines instead of relying on parsing and timestamps ? | 15:27 |
| corvus | dmsimard: i don't follow | 15:30 |
| dmsimard | logs.o.o/some/path/file?line=10 | 15:31 |
| corvus | dmsimard: what are you proposing as the hyperlink text? | 15:31 |
| dmsimard | hmmmm, it's probably not that easy in practice | 15:36 |
| dmsimard | :( | 15:36 |
| dmsimard | I went back and read how I had done it for ara and it's likely not appropriate to do for general purpose logging | 15:37 |
| dmsimard | but tl;dr in case it might be helpful anyway, ara uses pygment's htmlformatter http://pygments.org/docs/formatters/#HtmlFormatter | 15:39 |
| dmsimard | which wraps around the YAML pygments lexer | 15:40 |
| dmsimard | the lexer provides the syntax highlighting and the html formatter provides the table for line linking and highlighting | 15:40 |
| *** eandersson has joined #zuul | 16:10 | |
| tristanC | dmsimard: fwiw https://review.openstack.org/580891 adds multi-line selection to os-loganalyze, with anchor similar to github, e.g. job-output.txt#L23-L42 | 16:12 |
| *** jpena is now known as jpena|off | 16:30 | |
| *** jpena|off is now known as jpena | 16:31 | |
| *** jpena is now known as jpena|off | 16:32 | |
| corvus | tristanC: fwiw, it's looking like we may stop using os-loganalyze in openstack-infra -- so that we don't have to run a proxy to swift | 17:07 |
| corvus | tristanC: ^ moving that to #openstack-infra | 17:15 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Require at least openstacksdk 0.17.1 https://review.openstack.org/588149 | 17:48 |
| *** electrofelix has quit IRC | 18:02 | |
| *** rcarrillocruz has joined #zuul | 18:03 | |
| tobiash | hrm, we filled up 3*4 TB disk space with broken nodepool builds :/ | 18:10 |
| clarkb | tobiash: image builds? | 18:10 |
| tobiash | we had an dib-image which failed at every build by trying to download something that's not existing anymore | 18:10 |
| tobiash | and nodepool doesn't seem to do any cleanup after failed dib-image-builds | 18:11 |
| clarkb | tobiash: we built latest nodepool to aggressively rebuild on failures so you don't wait a day for the next attempt, but maybe we should add a backoff | 18:11 |
| clarkb | tobiash: dib is expected to clean up after itsel | 18:11 |
| clarkb | ianw: ^ might know if there are issues with that | 18:12 |
| tobiash | clarkb: well, rebuilding directly is ok, but not leaving the tmp dirs behind | 18:12 |
| tobiash | deleting that cruft takes ages... | 18:12 |
| pabelanger | I think nodepool will only clean up when we delete a image, if it fails, don't think nodepool does anything. Maybe we should | 18:14 |
| pabelanger | but, we only delete images today | 18:14 |
| tobiash | that's left behind: http://paste.openstack.org/show/727169/ | 18:16 |
| tobiash | plus the same directories as dib_image... | 18:16 |
| clarkb | pabelanger: yes dib is expected to clean up on failure | 18:16 |
| clarkb | it has a bunch of on exit handler code | 18:16 |
| clarkb | likely just a bug and need to update the handlers to clean more stuff up | 18:16 |
| tobiash | so maybe a dib upgrade fixes this | 18:16 |
| tobiash | I'm still on an ancient dib... | 18:17 |
| pabelanger | ++ | 18:19 |
| *** rcarrillocruz has quit IRC | 18:20 | |
| pabelanger | looking at nb02.o.o, I can also see some leaked builds | 18:21 |
| tobiash | pabelanger: but you're probably on the latest release | 18:22 |
| pabelanger | we likely could do the same thing in nodepool, we do for zuul-executor and lost builds dirs. Delete them on startup if any are found | 18:22 |
| pabelanger | tobiash: 22GB leaked for nb02.o.o | 18:24 |
| pabelanger | clarkb: ^ | 18:24 |
| clarkb | pabelanger: we should really fix dib | 18:24 |
| tobiash | lucky you, we leaked 12TB within one week | 18:24 |
| corvus | tobiash: can you try the latest dib and see if it's stil a problem? | 18:27 |
| tobiash | pabelanger: delete on startup is not sufficient if the builders live long | 18:28 |
| tobiash | corvus: yes, but probably not this evening | 18:28 |
| pabelanger | clarkb: tobiash I think we stop / start nodepool-builder during DIB build: https://nb02.openstack.org/ubuntu-bionic-0000000203.log | 18:28 |
| pabelanger | which caused the leak | 18:28 |
| pabelanger | trying to confirm | 18:29 |
| corvus | cool. let's wait for the results of that. | 18:29 |
| clarkb | pabelanger: ah that could be, possible that dib doesn't clean up properly when init reaps it | 18:30 |
| corvus | pabelanger: nodepool should tell dib to stop and wait for it. if not, that's a bug in nodepool. | 18:30 |
| tobiash | pabelanger: ok, the delete on start would fix your interrupted dib case | 18:30 |
| clarkb | corvus: I don't think it does, I think we fall back to init | 18:30 |
| corvus | then we should fix nodepool there | 18:30 |
| corvus | (i'm not opposed to eventually adding delete on startup, but we should only do so after we've covered all the cases where it shouldn't be necessary) | 18:31 |
| clarkb | ++ and also possibly have dib handle this better if it is leaking stuff in its on exit handlers | 18:32 |
| corvus | so first step is to identify any dib bugs and fix them. then identify any nodepool bugs and fix them. then add delete on startup. otherwise we'll just paper over a bunch of broken. | 18:32 |
| pabelanger | http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2018-07-23.log.html#t2018-07-23T18:49:18 | 18:32 |
| pabelanger | yah, Shrews looks to have restarted them | 18:32 |
| tobiash | corvus: a pod restart would make proper cleanup impossible | 18:32 |
| pabelanger | so, that explains one of the leaks | 18:32 |
| tobiash | corvus: but that could be handled in our init script as well | 18:32 |
| corvus | tobiash: i can't really help what other programs do to us. my goal is to make *our* program work as well as possible. | 18:33 |
| tobiash | corvus: yes, so I think I'll just add a cleanup in our startup script because in our environment the builder could get hard killed sometimes | 18:34 |
| corvus | tobiash: well, if you can help us identify the bug that would be great | 18:34 |
| corvus | tobiash: so upgrading dib and reporting whether it fixed the problem would help | 18:35 |
| tobiash | corvus: yes, that's still needed as nodepool filled up its disks without any restart ;) | 18:35 |
| tobiash | startup cleanup wouldn't have helped in my issue | 18:35 |
| corvus | right. again, my goal is to make sure that dib is working as designed, and nodepool are working as designed. if either of them leakes during ongoing operation, or on restart, we need to fix that. | 18:35 |
| corvus | when there are zero bugs in that code, i think we can add delete on startup to deal with unforseen situations. but if we add it early, we won't be able to fix our bugs. | 18:36 |
| tobiash | ++ | 18:36 |
| tobiash | corvus: ok, updated dib on one of my builders and at it at least successfully builds an image. Now I can test how it behaves if the build fails. | 19:45 |
| gundalow | Shrews: mordred corvus tristanC Ansible Contributor Summit signup info https://github.com/ansible/community/issues/247#issuecomment-410042603 (feel free to forward to whoever) | 19:54 |
| mordred | gundalow: done! thanks | 19:59 |
| tobiash | corvus: looks like a recent dib does a better job regarding cleanup :) | 20:01 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add HTMLify logs role https://review.openstack.org/588105 | 20:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add HTMLify logs role https://review.openstack.org/588105 | 20:12 |
| corvus | tobiash: i squashed your template fix into that patch ^ | 20:13 |
| tobiash | ++ | 20:35 |
| pabelanger | I wouldn't expect zuul to give a -1 on https://review.openstack.org/588387/ given depends-on, it removes the job in question | 20:50 |
| pabelanger | will dig more into it later, stepping away for some food | 20:51 |
| *** elyezer has quit IRC | 21:11 | |
| *** dmsimard has quit IRC | 21:24 | |
| corvus | pabelanger: you're right. but i have a theory -- it may be that there is more than one error (ie, something else refers to that job). but we only return the first error to the user, and that one from the second pass at layout generation (the one where we don't include the config-project change). here's the code: http://git.zuul-ci.org/cgit/zuul/tree/zuul/manager/__init__.py#n462 | 21:28 |
| corvus | pabelanger: so maybe we need to keep both layouts around, and if there are issues with the layout that includes trusted changes, report the error from that. otherwise, the error from the second layout. | 21:29 |
| corvus | should be relatively easy to add a test for this case. basically, have a job used in a config project and untrusted project. then make a change which removes the job, and have it depends-on a change that removes it from the config project. but leave the usage in the untrusted project. | 21:30 |
| ianw | tobiash: dib exit cleanup is a known point of some issue. it depends quite a lot on *when* the failure happens, as there's a few points we might exit and might not unwind mounts cleanly and then can leave things behind | 21:30 |
| ianw | patches welcome :) | 21:30 |
| *** dmsimard has joined #zuul | 21:32 | |
| *** samccann has quit IRC | 21:35 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Swift logs: don't allow links outside of the supplied path https://review.openstack.org/587580 | 22:19 |
| corvus | clarkb, tobiash: ^ that should take care of the symlink-in-tests problem | 22:19 |
| *** goern has quit IRC | 22:45 | |
| pabelanger | corvus: okay, thanks for the pointer. I'll see if I can reproduce the issue with a unit test, then try to patch zuul | 23:19 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul-jobs master: Swift logs: don't allow links outside of the supplied path https://review.openstack.org/587580 | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!