| *** jamielennox|away is now known as jamielennox | 01:06 | |
| *** persia has quit IRC | 04:17 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Change mutex to counting semaphore https://review.openstack.org/442563 | 05:51 |
|---|---|---|
| *** isaacb has joined #zuul | 06:24 | |
| *** bhavik1 has joined #zuul | 07:00 | |
| *** bhavik1 has quit IRC | 07:02 | |
| *** hashar has joined #zuul | 07:31 | |
| *** hashar has quit IRC | 07:36 | |
| *** hashar has joined #zuul | 07:42 | |
| openstackgerrit | Dirk Mueller proposed openstack-infra/nodepool master: Remove link to modindex https://review.openstack.org/415496 | 07:50 |
| *** bhavik1 has joined #zuul | 08:02 | |
| *** openstackgerrit has quit IRC | 08:03 | |
| *** bhavik1 has quit IRC | 08:39 | |
| *** hashar_ has joined #zuul | 08:52 | |
| *** hashar has quit IRC | 08:55 | |
| *** hashar_ is now known as hashar | 08:56 | |
| *** hashar is now known as hasharLunch | 11:04 | |
| *** adam_g has quit IRC | 11:45 | |
| *** adam_g has joined #zuul | 11:47 | |
| *** hasharLunch is now known as hashar | 11:52 | |
| *** adam_g has quit IRC | 12:35 | |
| *** adam_g has joined #zuul | 12:36 | |
| *** nt has quit IRC | 13:34 | |
| *** isaacb has quit IRC | 14:01 | |
| *** openstackgerrit has joined #zuul | 14:06 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Fix debian-jessie settings for devstack https://review.openstack.org/452491 | 14:06 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Remove unused timing constants https://review.openstack.org/450770 | 14:07 |
| jeblair | pabelanger: i'm happy to talk about timedatabase when you're ready | 14:27 |
| openstackgerrit | Merged openstack-infra/nodepool master: Remove link to modindex https://review.openstack.org/415496 | 14:50 |
| *** hashar has quit IRC | 15:19 | |
| *** nt has joined #zuul | 15:30 | |
| jeblair | rbergeron, Shrews: (cc clarkb) there was a person in #openstack-infra who is running into problems with nodepool documentation. i think it's an instructive case because the actual nodepool documentation doesn't explain what's going on. we have some infra-specific documentation that touches on it, but it still doesn't explain much (and when this person tried to follow that, they got themselves more confused because their problem didn't match ... | 16:42 |
| jeblair | ... the sort of problem we typically have in infra). at any rate, i think the sort of "this is what the builder does and how it works and these are what all the commands do" documentation is one of the things we're missing. i don't know if that's on the agenda for the doc work you're doing. if it's not, i'll be happy to write it, though i'm not going to get to it right away. | 16:42 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_queue_rate_limiting https://review.openstack.org/452835 | 16:51 |
| SpamapS | jeblair: ^^ do you have a minute to make a quick sanity check of the zuul.yaml file I ported there? The dependencies part is new for me, and also the hold-following-jobs doesn't seem to be working (builds ends up with all the jobs) | 16:52 |
| SpamapS | hold-following-changes I mean | 16:52 |
| SpamapS | hrm maybe that's not what I thought it was not that I get it out of my head | 16:52 |
| SpamapS | oh that seems right n/m | 16:53 |
| jeblair | SpamapS: will do in just a few. i'm finishing up poking at jesusaur's change | 16:56 |
| SpamapS | jeblair: thanks! | 16:56 |
| jeblair | SpamapS: i think the word "dependencies" and everything under it need to be indented an extra step. | 17:07 |
| SpamapS | oh derp | 17:07 |
| SpamapS | I screw that up in yaml a lot | 17:08 |
| SpamapS | looks visually right-ish | 17:08 |
| SpamapS | but definitely doesn't produce the desired result :-P | 17:08 |
| jeblair | yeah, I think this is something we can/should check for in configparser. it shows up as two keys inside of a dictionary, with one of them having a value of None. so I bet we can detect it. | 17:09 |
| SpamapS | And the other one has a list as the value | 17:09 |
| SpamapS | thanks for the extra eyes | 17:09 |
| SpamapS | that was in fact it (also didn't need hold-following-changes .. that was cargo cult fail) | 17:10 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_queue_rate_limiting https://review.openstack.org/452835 | 17:10 |
| jeblair | (this is also why i prefer the "always indent" style of yaml indentation, and we use it for all the documentation and examples -- i think knowing when it's optional vs required to indent yaml is an advanced skill :). i should write that up in the dev docs. | 17:15 |
| SpamapS | jeblair: I do wonder if there's room for something between yaml's flexibility and json's rigidity/non-readability. | 17:17 |
| SpamapS | Perhaps a yamlfmt would solve that. | 17:17 |
| SpamapS | just "run this on save of any yaml file" | 17:17 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_queue_rate_limiting_dependent https://review.openstack.org/452864 | 17:17 |
| mordred | SpamapS: well, I don't think actual yamlfmt sould solve it for us, since it does the opposite of the zuul yaml style. but it should be possible to write a yamlfmt that does emit zuul-style yaml | 17:41 |
| mordred | (assuming, of course, that by yamlfmt you mean https://github.com/mmlb/yamlfmt - which you might not) | 17:41 |
| SpamapS | oh there's already a yamlfmt? | 17:43 |
| SpamapS | I just made that up | 17:43 |
| mordred | the underlying library that yamlfmt wraps, ruamel.yaml, supports keyword arguments for setting indent style - so doing that as "indent=4, block_seq_indent=2" should get the zuul format (it's the block_seq_indent=2 that is the important part) | 17:45 |
| *** jamielennox is now known as jamielennox|away | 17:50 | |
| SpamapS | I will say, once I tied rustfmt to "run this whenever I save a rust file", I stopped noticing the differences and have just sort of learned the proper formatting by osmosis. | 17:50 |
| SpamapS | but this would be harder to setup | 17:51 |
| SpamapS | "when I'm yaml'ing for zuul..." | 17:51 |
| mordred | yah. for me that's honestly why gate jobs are personally helpful - I learn from the feedback loop and can _usually_ follow it for the most part in a given codebase, but if I get it wrong a friendly robot tells me and then I improve for next time | 17:55 |
| mordred | I've thought a format-on-save tool would be pleasing - but I have not found any such tools that can _emit_ code like I want it to look - whereas having tools that reject bad code for some reason seems easier :( | 17:56 |
| clarkb | I don't like it when tools magically fix things too | 17:58 |
| SpamapS | mordred: well ultimately the thing that finds bad code should be able to produce good code instead of nagging | 17:58 |
| mordred | SpamapS: one would certainly think so | 17:59 |
| clarkb | if you looked at the jenkins code base a couple years ago it was clearly edited by people with magical editor settings and was unreadable without them | 17:59 |
| mordred | SpamapS: but so far I have not found one that does so well for python | 17:59 |
| mordred | or, let me rephrase - I have not found one for python that I can bend to match _my_ preferences properly, thus I find all of them unusable :) | 17:59 |
| SpamapS | Yeah for python I just have the vim-flake8 plugin | 18:00 |
| SpamapS | f8 ... iterate on quickfix.. save | 18:00 |
| mordred | clarkb: ++ wow that stuff is hard to deal with | 18:00 |
| mordred | SpamapS: ++ | 18:00 |
| SpamapS | Also rustfmt is special. Because basically every piece of code you will see already had it run on it.. you never learn any other habit. | 18:01 |
| SpamapS | so it feels less like magic | 18:01 |
| SpamapS | it's too late for python and yaml to do that | 18:01 |
| mordred | yah - also, go and rust both started life out with this in mind | 18:01 |
| mordred | yup | 18:01 |
| mordred | I mean, python started us out with enforced indentation... | 18:02 |
| mordred | so you will never see a python program with incorrect indentation :) | 18:02 |
| SpamapS | Yeah, python is already pretty good even when it is bad. | 18:02 |
| mordred | yup | 18:02 |
| SpamapS | yaml tho.. | 18:02 |
| SpamapS | zomg have I seen bad yaml | 18:02 |
| mordred | leaving us to quibble over things like visual or non-visual indentation styles | 18:02 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 18:05 |
| jeblair | jesusaur: ^ i pushed up a new patchset | 18:06 |
| jesusaur | jeblair: cool, i'll take a look | 18:06 |
| jeblair | jesusaur: i was going to just leave comments, but it would have been gnarly and wasteful, so i'm doing this and i'll go leave comments on the diff to make sure i explain | 18:07 |
| SpamapS | hrm, test_crd_check_transitive is not cooperating. :-P | 18:07 |
| *** harlowja has joined #zuul | 18:12 | |
| jeblair | jesusaur: okay, i left explanatory comments on PS16. so if you do a side-by-side diff from PS16 to PS17, it will make the most sense. let me know if it does, in fact, make sense. :) | 18:23 |
| mordred | jeblair: if it doesn't make sense, should he also let you know that? :) | 18:24 |
| jeblair | mordred: he should let you know :) | 18:24 |
| mordred | jeblair: I walked right in to that one | 18:25 |
| jesusaur | jeblair: yeah, your comments on ps16 and comments in the tests all make sense :) thanks for fixing that up | 18:30 |
| jeblair | jesusaur: thank you :). mordred: you're off the hook. | 18:40 |
| mordred | THANK GOD | 18:42 |
| jesusaur | also, WHOO ALL THE (py27) TESTS PASS! | 18:42 |
| jeblair | jesusaur: i think before too long we need to change how the executor constructs git repos -- i think we're going to want to build on this change to save the exact git shas that the initial merge (triggered by the scheduler) uses for a change, and then pass those to the executor so it uses them. | 18:44 |
| jeblair | jesusaur: mostly we should do this so that jobs run by different executors run with exactly the same git repo content. the racy nature of the current system means they might not. | 18:45 |
| jeblair | jesusaur: but it will also have the effect of making it so that an executor should never have a merge failure (so that test will end up changing or being removed. but it's how things behave right now, so it's a good test.) | 18:46 |
| jesusaur | jeblair: yeah, i can see some benefit from passing the merge sha around to be sure everything is using the exact same version of the repo | 18:52 |
| *** hashar has joined #zuul | 19:09 | |
| pabelanger | jeblair: okay, back at desk. Regarding TimeDataBase(), what are your thoughts on hashing the Job objection in some matter, then using that as the file we store on disk? I am assuming we can make the Job objection unique | 19:27 |
| pabelanger | jeblair: I was also looking into dogpile.cache, but I am not sure that is the right path | 19:30 |
| *** harlowja has quit IRC | 19:37 | |
| jeblair | pabelanger: i think the unique key for this is probably 'job name, project, branch' | 19:42 |
| pabelanger | jeblair: tenant also? | 19:43 |
| jeblair | pabelanger: yes | 19:45 |
| pabelanger | jeblair: k, any preference on how to create the file on disk? concat, hash? | 19:45 |
| jeblair | [thinking] | 19:50 |
| clarkb | are you trying to persist the database? | 19:53 |
| jeblair | pabelanger: how about deep directories? .../tenant_name/connection_name/project_name/branch_or_ref_name/job_name.db (branch_or_ref_name may have '/' in it, making more directories; that should be okay). | 19:53 |
| clarkb | I wouldn't use dogpile for something that needs to persist | 19:53 |
| pabelanger | right, we only persist data over 10 runs | 19:55 |
| pabelanger | jeblair: sure, I can start with that | 19:55 |
| jeblair | pabelanger: probably should double check that the path we end up with is a subdirectory of the root, just in case someone manages to do something tricky with "../" in a branch name or somesuch | 19:56 |
| jeblair | (or job name) | 19:56 |
| pabelanger | ack | 19:57 |
| clarkb | path == os.path.abspath(path) | 19:59 |
| *** jamielennox|away is now known as jamielennox | 20:05 | |
| mordred | clarkb: you need realpath too | 20:26 |
| mordred | to expand fun symlink tricks | 20:27 |
| *** harlowja has joined #zuul | 20:53 | |
| jesusaur | hrm, it looks like the tox-linters zuul job may not be working as expected? | 20:59 |
| jesusaur | I see tracebacks and then 'commands succeeded' http://zuulv3-dev.openstack.org/logs/5142710d32a842e782a669414ac16637/console.log | 20:59 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 21:00 |
| clarkb | jesusaur: that looks like a docs job | 21:02 |
| clarkb | and if I am guessing we don't install all the deps to the venv tox target so autodoc fails to import | 21:03 |
| clarkb | hrm tox env doesn't override deps so it should get all the listed ones at least | 21:04 |
| jesusaur | oh, you're right, tox-linters and tox-docs both run the same thing | 21:06 |
| pabelanger | o.0 | 21:06 |
| pabelanger | that doesn't sound right | 21:06 |
| * jesusaur was wondering why tox-linters didn't catch a pep8 failure | 21:06 | |
| jesusaur | pabelanger: nope, it sure doesn't | 21:06 |
| pabelanger | ha | 21:07 |
| pabelanger | I see the issue | 21:07 |
| jesusaur | in .zuul.yaml? | 21:07 |
| pabelanger | copy pasta fail | 21:07 |
| pabelanger | ya | 21:07 |
| jesusaur | yep | 21:07 |
| clarkb | both jobs run the same playbook | 21:07 |
| clarkb | but also should figure out why autodoc is mad | 21:08 |
| jesusaur | oh, so 'run:' refers to a playbook? | 21:08 |
| pabelanger | line 34 should be : run: tox/linters | 21:08 |
| pabelanger | jesusaur: ya | 21:08 |
| jesusaur | ok, i see that now | 21:08 |
| jesusaur | i'll write up a quick change for that | 21:09 |
| pabelanger | I'd be in favour of making pre-run, run, post-run support file extensions. I think it would make it more readable to users | 21:10 |
| clarkb | looks like the docs issues are old code autodoc directives that weren't cleaned up when zuul got refactored | 21:10 |
| * clarkb pokes at cleaning that up | 21:10 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Make tox-linters run the tox/linters play https://review.openstack.org/452953 | 21:11 |
| clarkb | oh boy that actually means documenting all of zuuls new internals, I think I am not best person to do that | 21:11 |
| jesusaur | pabelanger: care to give that a review^? | 21:11 |
| clarkb | jeblair: doc/source/datamodel.rst is the area needing attention to clean up doc builds | 21:11 |
| clarkb | is it intentional that we don't record the job name in the logs dir structure? | 21:13 |
| pabelanger | I guess we also don't have warnerrors=true for pbr | 21:13 |
| pabelanger | clarkb: we have a patch up to include it | 21:14 |
| pabelanger | however, we haven't merged because of lookup pluging | 21:14 |
| pabelanger | https://review.openstack.org/#/c/441617/ | 21:14 |
| pabelanger | I could remove the lookup call, to at least land it | 21:15 |
| clarkb | pabelanger: they changed pbr and warnerrors is no longer a thing https://git.openstack.org/cgit/openstack-dev/pbr/commit/?id=72e8e42a34217e7f6847d221dc9751d0c350baa9 | 21:15 |
| clarkb | you have to set it with sphinx's config now aiui | 21:15 |
| pabelanger | neat | 21:15 |
| pabelanger | clarkb: actually, we can move forward on 441617 | 21:18 |
| pabelanger | let me update it | 21:18 |
| clarkb | I'm fiddling with the doc builds to see if I can get this to fail | 21:18 |
| clarkb | then will likely push up a mock up change with an attempt to make ti work | 21:18 |
| clarkb | but someone like jeblair will likely need to edit datamodel.rst | 21:18 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add net-info role https://review.openstack.org/441617 | 21:19 |
| pabelanger | we might need to restart zuul to pick up zuul.executor.hostname | 21:20 |
| pabelanger | clarkb: mind a +3 https://review.openstack.org/#/c/450894/ | 21:20 |
| pabelanger | nodepool-launcher rename | 21:20 |
| pabelanger | for puppet | 21:20 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add net-info role https://review.openstack.org/441617 | 21:28 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul feature/zuulv3: Cleanup docs builds and error on warnings https://review.openstack.org/452964 | 21:30 |
| clarkb | pabelanger: jesusaur ^ thats the fix for the docs. Jeblair you will likely want to make sure I didn't break anything on the docs side of the model | 21:30 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add net-info role https://review.openstack.org/441617 | 21:38 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Make tox-linters run the tox/linters play https://review.openstack.org/452953 | 21:40 |
| *** hashar has quit IRC | 21:51 | |
| pabelanger | clarkb: zuul information for jobs: http://zuulv3-dev.openstack.org/logs/44c6ac9aee174c8c8c8e8c190615a89f/console.log | 21:54 |
| pabelanger | feedback wanted | 21:54 |
| clarkb | pabelanger: taht seems to match up with what we have in jjb | 21:55 |
| clarkb | pabelanger: maybe for verbosity include the commands run to collect the data? | 21:55 |
| pabelanger | until we restart zuul, executor will be unknow | 21:55 |
| pabelanger | clarkb: most are ansible variables at this point. hostname is uname -a, but ya | 21:56 |
| pabelanger | I can fix that | 21:56 |
| clarkb | pabelanger: specifically the network neighbors one could be a little ambiguous | 21:58 |
| clarkb | pabelanger: oh we should include the image uuid near the image build date too | 22:00 |
| clarkb | or did we lose that capability with newer nodepool, hrm | 22:00 |
| pabelanger | will check | 22:01 |
| clarkb | ya we don't appear to have it anymore because we stopped sshing to the nodes to write that in | 22:01 |
| jeblair | there's some sort of zuul meeting now in #openstack-meeting-alt i think | 22:02 |
| pabelanger | clarkb: we likely need to append the info to our nodepool vars for zuul | 22:03 |
| clarkb | pabelanger: I also don't think its a blocker, just somethign to work towards while geting the current stuff in early | 22:04 |
| pabelanger | ya | 22:04 |
| jhesketh | Morning | 22:11 |
| jeblair | pabelanger: yeah, the node data structure in zk has a reference to the image, so we can look all that up within zuul and add it to ansible vars | 22:45 |
| pabelanger | jeblair: clarkb: we are just not collecting the subunit files from the worker, working on a patch | 22:59 |
| pabelanger | jeblair: agree | 22:59 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Collect test-results for tox jobs https://review.openstack.org/452991 | 23:07 |
| SpamapS | jeblair: apologies for missing zuul meeting. Did not sleep well last night and took a longer than expected power nap :-P | 23:17 |
| jeblair | SpamapS: np. sounds like you're ready to hack now. ;) | 23:19 |
| SpamapS | coffee in hand | 23:20 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Collect test-results for tox jobs https://review.openstack.org/452991 | 23:23 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Collect test-results for tox jobs https://review.openstack.org/452991 | 23:32 |
| pabelanger | okay, that gets the subunit files | 23:32 |
| pabelanger | but, if it is the right place in the playbook | 23:32 |
| pabelanger | I'll figure that out tomorrow | 23:32 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!