| *** rfolco has quit IRC | 00:31 | |
| *** openstackgerrit has joined #zuul | 04:29 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: executor: run cleanup playbook on stop https://review.opendev.org/661881 | 04:29 |
|---|---|---|
| *** altlogbot_1 has quit IRC | 04:36 | |
| *** altlogbot_0 has joined #zuul | 04:38 | |
| *** raukadah is now known as chandankumar | 04:41 | |
| *** sshnaidm is now known as sshnaidm|off | 05:03 | |
| *** pcaruana has joined #zuul | 05:08 | |
| *** swest has joined #zuul | 05:14 | |
| *** threestrands has joined #zuul | 05:28 | |
| *** sanjayu_ has joined #zuul | 05:34 | |
| *** sanjayu_ has quit IRC | 05:34 | |
| *** sanjayu_ has joined #zuul | 05:34 | |
| *** flepied has joined #zuul | 06:48 | |
| *** gtema has joined #zuul | 06:51 | |
| *** threestrands has quit IRC | 06:58 | |
| *** threestrands has joined #zuul | 07:03 | |
| *** jbryce has quit IRC | 07:41 | |
| *** mnaser has quit IRC | 07:41 | |
| *** timburke has quit IRC | 07:41 | |
| *** mugsie has quit IRC | 07:41 | |
| *** irclogbot_3 has quit IRC | 07:44 | |
| *** irclogbot_1 has joined #zuul | 07:46 | |
| *** EvilienM has quit IRC | 07:47 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: executor: run cleanup playbook on stop https://review.opendev.org/661881 | 07:49 |
| *** EvilienM has joined #zuul | 07:52 | |
| *** jbryce has joined #zuul | 07:53 | |
| *** mnaser has joined #zuul | 07:53 | |
| *** timburke has joined #zuul | 07:53 | |
| *** mugsie has joined #zuul | 07:53 | |
| *** jpena|off is now known as jpena | 07:55 | |
| *** threestrands has quit IRC | 07:58 | |
| *** yolanda has joined #zuul | 08:11 | |
| *** themroc has joined #zuul | 08:16 | |
| *** jangutter has joined #zuul | 08:25 | |
| *** themroc has quit IRC | 08:28 | |
| *** themroc has joined #zuul | 08:31 | |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 08:37 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: web: add tenant and project scoped, JWT-protected actions https://review.opendev.org/576907 | 08:50 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Allow operator to generate auth tokens through the CLI https://review.opendev.org/636197 | 08:50 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Zuul CLI: allow access via REST https://review.opendev.org/636315 | 08:50 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Add Authorization Rules configuration https://review.opendev.org/639855 | 08:50 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Web: plug the authorization engine https://review.opendev.org/640884 | 08:51 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Zuul Web: add /api/user/authorizations endpoint https://review.opendev.org/641099 | 08:51 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: authentication config: add optional token_expiry https://review.opendev.org/642408 | 08:51 |
| *** toabctl has left #zuul | 09:05 | |
| *** felixgcb_ has joined #zuul | 09:48 | |
| felixgcb_ | Hey :) Not sure if this is the right place to ask.. | 09:51 |
| felixgcb_ | is there a certain point in time when the "zuul.executor.log_root" points to a valid path? I'm trying to access is as part of a "post-run" job in order to send the log files to some API. But there is nothing there no matter what I try... | 09:52 |
| *** felixgcb_ has quit IRC | 10:13 | |
| *** felixgcb has joined #zuul | 10:14 | |
| *** aspiers has joined #zuul | 10:22 | |
| *** hashar has joined #zuul | 10:24 | |
| *** gtema has quit IRC | 10:43 | |
| *** MrCoder25 has joined #zuul | 10:56 | |
| *** MrCoder25 has quit IRC | 10:57 | |
| *** MrCoder25 has joined #zuul | 10:58 | |
| *** MrCoder25 has quit IRC | 11:00 | |
| *** jpena is now known as jpena|lunch | 11:31 | |
| felixgcb | Alternatively if somebody can share a zuul repo which accesses those logs from a post-run job, that would help, too :) | 11:38 |
| *** rfolco has joined #zuul | 11:58 | |
| *** gtema has joined #zuul | 12:05 | |
| *** rlandy has joined #zuul | 12:06 | |
| *** rfolco has quit IRC | 12:21 | |
| *** rfolco has joined #zuul | 12:24 | |
| *** themroc has quit IRC | 12:28 | |
| *** themroc has joined #zuul | 12:28 | |
| *** jpena|lunch is now known as jpena | 12:29 | |
| *** jbryce has quit IRC | 12:43 | |
| *** jbryce has joined #zuul | 12:46 | |
| dmsimard | felixgcb: codesearch has a few hits: http://codesearch.openstack.org/?q=zuul.executor.log_root&i=nope&files=&repos= | 12:49 |
| *** gtema has quit IRC | 12:51 | |
| *** EvilienM is now known as EmilienM | 13:24 | |
| *** mnaser has quit IRC | 13:34 | |
| *** themroc has quit IRC | 13:35 | |
| *** mnaser has joined #zuul | 13:37 | |
| *** panda has quit IRC | 13:39 | |
| *** panda has joined #zuul | 13:42 | |
| corvus | felixgcb: z.e.log_root points to a directory where zuul itself writes the console text and json logs. for any other logs to end up there, the job needs to fetch them. there are some roles in zuul-jobs to help with that (see fetch-output https://zuul-ci.org/docs/zuul-jobs/log-roles.html ) | 14:00 |
| corvus | felixgcb: so generally, we'd expect the site base job to run fetch-output along with a role to push the logs to an archive server (zuul-jobs also has roles to help with that), or perhaps the thing you are describing. | 14:01 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 14:40 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 14:59 |
| *** jamesmcarthur has joined #zuul | 15:01 | |
| *** jamesmcarthur_ has joined #zuul | 15:07 | |
| tobiash | corvus: we're starting to hit a scalability limit with the sequential event processing in the github event queue | 15:09 |
| SpamapS | tobiash:congratulations. ;) | 15:10 |
| clarkb | tobiash: I think we (opendev) are on the edge there with ansible's post merge testing not being able to take advantage of the commit to PR cache | 15:10 |
| clarkb | tobiash: is that where you see the problems too? | 15:10 |
| tobiash | corvus: I think I'll see if I can parallelize that using a threadpool while keepint the order they are injected into the scheduler queue | 15:10 |
| *** irclogbot_1 has quit IRC | 15:10 | |
| *** jamesmcarthur has quit IRC | 15:10 | |
| *** altlogbot_0 has quit IRC | 15:11 | |
| clarkb | basically we fall behind by about 45 minutes to an hour anytime ansible merges more than one change in succession | 15:11 |
| tobiash | we had an event queue of 1300 today where it took ~5s per event to gather all required metadata | 15:11 |
| tobiash | clarkb: if my idea works, this might solve your problem too | 15:11 |
| SpamapS | that sounds like you can definitely parallelize the information scraping, but, is the problem that you have to scrape it *after* the change ahead of it merges? | 15:12 |
| clarkb | the threadpool should reduce the cost of the scraping | 15:12 |
| fungi | yeah, the fact that zuul needs to check every pull request on a project for each of those events is painful. wish gh had a better api for that | 15:12 |
| *** irclogbot_1 has joined #zuul | 15:12 | |
| tobiash | my idea is to pop the events single threaded, put them into a threadpoolexecutor and put the futures in order into another queue from where the final zuul events are poped and pushed into the scheduler queue | 15:12 |
| *** altlogbot_2 has joined #zuul | 15:13 | |
| tobiash | what also would help is merging https://review.opendev.org/660800 and parent so we can better analize and further reduce the number of requests per event | 15:13 |
| tobiash | maybe we could also look into leveraging graphql | 15:14 |
| SpamapS | Yeah that sounds about right. But I'm asking, if you have 20 events in the queue, and while handling event 10, you merge something as the 21st, does that invalidate anything you already fetched to satisfy 11-20? | 15:14 |
| tobiash | but for now my top prio is the multithreaded information fetching in the github event queue | 15:14 |
| tobiash | my users were a little unhappy today :/ | 15:14 |
| clarkb | SpamapS: it shouldn't because git is a dag | 15:15 |
| clarkb | SpamapS: so earlier commits don't change | 15:15 |
| tobiash | SpamapS: that case should be (abd is I guess) handled later in the pipeline itself | 15:15 |
| clarkb | (and the struggles here are keeping track of commits) | 15:15 |
| SpamapS | Makes sense. | 15:18 |
| SpamapS | Why doesn't Gerrit suffer from this? | 15:24 |
| clarkb | because gerrit lets you lookup changes by commit sha | 15:25 |
| clarkb | github does not (you ahve to do a full scan of all PRs) | 15:25 |
| SpamapS | I'm really surprised the other CI platforms haven't also pushed for PR-by-sha | 15:26 |
| clarkb | SpamapS: jlk was saying their own internal slackbot team is pushing for it | 15:27 |
| clarkb | so it islikely to happen but unsure of when | 15:27 |
| SpamapS | Yeah, Hubot is pretty popular. | 15:27 |
| fungi | most ci platforms don't care about inter-change relationships | 15:27 |
| fungi | they get an event for a pull request and then retrieve the code referred to in the event | 15:28 |
| corvus | tobiash: your threadpool workaround sounds good | 15:28 |
| corvus | looking at the logging changes now | 15:28 |
| tobiash | thanks :) | 15:28 |
| SpamapS | fungi: yeah, and they can listen to the push events too, but then they just grab the branch and run | 15:28 |
| fungi | but yeah, a query-by-commit-id option in the gh api would solve much of this, i expect | 15:28 |
| corvus | do we have a cache index? | 15:29 |
| tobiash | SpamapS: one diffrrence is that gerrit spllies almost al needed information in the event while github does not and we need to do several requests to get status, reviews, ... | 15:29 |
| clarkb | we cache the commit -> PR mapping. The problem is it is based on the pre merge projected merge sha which changes when you merge multiple PRs in succession | 15:30 |
| corvus | (ie, do we maintain our own pr-by-sha index?) | 15:30 |
| clarkb | corvus: yes | 15:30 |
| corvus | gotcha, so we still have a lot of cache misses | 15:30 |
| clarkb | that coupled with ansible's post merge testing means we get a ton of PR events for shas that we've never seen before | 15:31 |
| corvus | (my irc connection is starting to lag) | 15:31 |
| corvus | https://review.opendev.org/660800 and parent lgtm | 15:33 |
| *** sanjayu_ has quit IRC | 15:36 | |
| corvus | quick question: i'm about to add a job to zuul-jobs which is not meant for public use, only internal testing of changes to zuul-jobs. we have a gate check that all jobs in zuul-jobs be included in the documentation. should i add an exception to that (so you set a tag on the job and it doesn't need to be included), or should i add a new section to the docs for jobs which test changes to this repo? | 15:37 |
| corvus | (basically -- would it be less confusing to hide the job from users by not including it in the documentation, or to include it but keep it separate so it's clear it shouldn't be used?) | 15:38 |
| AJaeger | corvus: new section with an intro "jobs for testing of zuul-jobs only" might be less confusing... | 15:39 |
| AJaeger | corvus: but no strong opinion, either way works for me | 15:39 |
| SpamapS | Do we update the cache when we hit the 'merge' button? | 15:40 |
| SpamapS | I bet that returns the final sha. | 15:40 |
| SpamapS | Or if nothing else, we could query it right then and shove that into the cache. | 15:40 |
| SpamapS | corvus: I suspect that job will be final, yes? | 15:41 |
| corvus | SpamapS: good idea :) | 15:41 |
| SpamapS | corvus: if so, I think that would be the right heuristic... final jobs shouldn't need docs since they can't be consumed. | 15:41 |
| corvus | well, they could still be consumed, but they couldn't be changed... | 15:42 |
| corvus | but we could throw an 'allowed-projects' in there to make it difficult to be consumed | 15:42 |
| SpamapS | Stack all the signals up. :) | 15:43 |
| SpamapS | But yeah, an explicit exception for the job is probably fine too. | 15:43 |
| *** chandankumar is now known as raukadah | 15:43 | |
| *** zbr has quit IRC | 15:47 | |
| fungi | corvus: i too think that being consistent about documenting zuul jobs used for testing the zuul-jobs repo would be more consistent and probably also clearer to people who come across them | 15:48 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 15:48 |
| corvus | thanks -- that adds a doc section and also sets allowed-projects :) | 15:49 |
| *** zbr has joined #zuul | 15:55 | |
| felixgcb | corvus: Thanks for your hint, I finally found it, I need to add "delegate_to: localhost" to be able to access those files | 15:56 |
| corvus | ah, yes, that's a path on the executor (localhost) | 15:57 |
| corvus | okay, the registry test job is all green now, i'll push up a final version | 16:03 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add a registry test job https://review.opendev.org/661327 | 16:05 |
| corvus | remote: https://review.opendev.org/662794 Add registry test to zuul-jobs | 16:05 |
| *** pwhalen has joined #zuul | 16:16 | |
| *** pwhalen has quit IRC | 16:21 | |
| corvus | Shrews: autohold changes are looking good; i left some comments | 16:24 |
| corvus | clarkb, fungi, AJaeger: https://review.opendev.org/661327 and https://review.opendev.org/662794 are ready | 16:24 |
| Shrews | corvus: cool. working on the last piece now (setting nodes from HOLD to USED). will look at your comments after lunch | 16:25 |
| *** panda has quit IRC | 16:35 | |
| *** panda has joined #zuul | 16:37 | |
| *** felixgcb has quit IRC | 16:51 | |
| *** jpena is now known as jpena|off | 17:04 | |
| Shrews | corvus: Good comments, thx. I did play devil's advocate on one of your points though. | 17:04 |
| Shrews | complete with example and everything! :) | 17:05 |
| *** hashar is now known as hasharDinner | 17:06 | |
| *** armstrongs has joined #zuul | 17:08 | |
| corvus | Shrews: ack; replied with suggestions for even more work! :) | 17:11 |
| armstrongs | hi guys has anyone had issues using lookup plugins with zuul. I am trying to set a fact that uses a lookup on a fedora nodepool machine and its telling me I don't have a required pip package. However it is on the target node. I also tried installing it on the zuul scheduler too but its still complaining. | 17:11 |
| armstrongs | I can run the same playbook on the target machine with no issue just via zuul | 17:12 |
| Shrews | corvus: joy | 17:12 |
| corvus | armstrongs: it will need to be installed on the zuul executor (that's where ansible runs, and lookup plugins run locally in ansible). but be aware that there are some restrictions on what lookup plugins can be used in untrusted jobs (but there are no restrictions for trusted jobs). | 17:14 |
| armstrongs | yeah i installed it on the zuul executor but its still complaining | 17:14 |
| armstrongs | my lookup is calling a vault secret | 17:14 |
| armstrongs | hashicorp vault that is | 17:15 |
| armstrongs | would that not work | 17:15 |
| armstrongs | its an untrusted project | 17:15 |
| corvus | armstrongs: there should be a clear error if it's hitting the zuul restriction; it shouldn't be an error about a missing package. | 17:16 |
| armstrongs | yeah its just saying "An unhandled exception occurred while running the lookup plugin 'hashi_vault'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Please pip install hvac to use the hashi_vault lookup module." | 17:16 |
| armstrongs | however the package is installed on the executor | 17:16 |
| corvus | armstrongs: oh! | 17:17 |
| corvus | armstrongs: it probably isn't installed in the virtualenvs zuul uses to run ansible | 17:17 |
| corvus | let me dig up a link | 17:17 |
| armstrongs | thanks | 17:17 |
| corvus | armstrongs: here we go -- https://zuul-ci.org/docs/zuul/admin/installation.html#ansible | 17:17 |
| corvus | armstrongs: so something like "ANSIBLE_EXTRA_PACKAGES=hvac zuul-manage-ansible" i think | 17:18 |
| armstrongs | thanks again will give that a go | 17:19 |
| corvus | armstrongs: (zuul manages its own installations of ansible inside virtualenvs in order to support multiple ansible versions) | 17:19 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Read multiple *.conf files from directory https://review.opendev.org/660977 | 17:20 |
| mordred | flaper87, corvus, tobiash, SpamapS : ^^ updated that to address review thigns and add some docs/release notes | 17:21 |
| mordred | I didn't add tests yet because my brain isn't smart enough to do that right now | 17:22 |
| *** rlandy has quit IRC | 17:22 | |
| *** rlandy has joined #zuul | 17:22 | |
| *** jamesmcarthur_ has quit IRC | 17:25 | |
| *** jamesmcarthur has joined #zuul | 17:29 | |
| *** jamesmcarthur_ has joined #zuul | 17:33 | |
| *** jamesmcarthur has quit IRC | 17:36 | |
| armstrongs | corvus: that worked thanks for the info | 17:49 |
| *** jamesmcarthur_ has quit IRC | 17:49 | |
| *** pwhalen has joined #zuul | 18:10 | |
| *** hasharDinner has quit IRC | 18:12 | |
| openstackgerrit | Jean-Philippe Evrard proposed zuul/zuul-jobs master: Explicitly store facts for promote https://review.opendev.org/662817 | 18:13 |
| *** themroc has joined #zuul | 18:23 | |
| *** themroc has quit IRC | 18:24 | |
| *** themroc has joined #zuul | 18:25 | |
| *** hasharDinner has joined #zuul | 18:26 | |
| *** hasharDinner is now known as hashar | 18:31 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Parallelize githib event processing https://review.opendev.org/662818 | 18:37 |
| tobiash | this is my attempt to parallelize the event processing ^ | 18:37 |
| tobiash | this passes all tests locally | 18:38 |
| *** pwhalen has quit IRC | 18:39 | |
| *** altlogbot_2 has quit IRC | 18:43 | |
| tobiash | corvus: your registry test job work is impressive :) | 18:45 |
| *** altlogbot_3 has joined #zuul | 18:45 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Parallelize github event processing https://review.opendev.org/662818 | 18:46 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add a registry test job https://review.opendev.org/661327 | 18:52 |
| *** themroc has quit IRC | 18:59 | |
| *** themroc has joined #zuul | 19:04 | |
| fungi | is anybody already working on the password lookup plugin todo noted at https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/run-buildset-registry/tasks/main.yaml#L28 ? | 19:06 |
| fungi | trying to get an idea of what's involved in making it allowed for the executor | 19:07 |
| fungi | that question is probably mostly for corvus since git blame says he added the todo comment | 19:09 |
| openstackgerrit | Jean-Philippe Evrard proposed zuul/zuul-jobs master: Append date to tag optionally https://review.opendev.org/662828 | 19:11 |
| *** pwhalen has joined #zuul | 19:14 | |
| corvus | fungi: not that i'm aware of; i think it just involves thinking about whether it can be exploited, and if it's okay, fiddling some knobs in zuul/ansible/ | 19:15 |
| fungi | yeah, i guess the biggest risk with it is the ability to write to arbitrary files... that may be enough of a reason to discount it? | 19:16 |
| fungi | if it's safe to use, the invocation we want there is presumably something like registry_password: "{{ lookup('password', '/dev/null') }}" | 19:17 |
| fungi | at least according to what's documented at https://docs.ansible.com/ansible/latest/plugins/lookup/password.html | 19:17 |
| fungi | looks like you can't write to an existing file (other than /dev/null) with it, though you can create new files and also read arbitrary files | 19:19 |
| fungi | target path has to be readable/writeable as the playbook user, doesn't automatically escalate privs | 19:20 |
| fungi | are these capabilities already possible via any of the allowed plugins? | 19:20 |
| fungi | (the reading of existing files and creating new files) | 19:21 |
| *** altlogbot_3 has quit IRC | 19:21 | |
| corvus | fungi: we have helper methods to verify the path is in the work dir; i think we'd want to use that here | 19:27 |
| *** altlogbot_0 has joined #zuul | 19:27 | |
| corvus | so with that, i think concerns about writing to files should be addressed | 19:27 |
| fungi | got it, so we do need to scope writes (and reads/) | 19:27 |
| fungi | er, and reads? | 19:28 |
| fungi | or is reading files okay? | 19:28 |
| *** armstrongs has quit IRC | 19:28 | |
| corvus | i think scope both | 19:29 |
| fungi | if there's a way to scope the path, i'd be fine just requiring it to be /dev/null and expecting it to be set as a fact or an ansible var | 19:29 |
| corvus | i don't think we need to require that it be /dev/null | 19:29 |
| fungi | and yeah, it's the same parameter for both reads and writes, which it does depends on how it's being invoked | 19:29 |
| corvus | i think as long as it's scoped to the workdir, reading/writing should be ok | 19:30 |
| fungi | have we done something similar with other plugins? | 19:30 |
| corvus | yeah, here's one: https://opendev.org/zuul/zuul/src/branch/master/zuul/ansible/base/lookup/ini.py | 19:30 |
| fungi | cool, thanks! | 19:30 |
| fungi | okay, so we just subclass the plugin basically and then shadow the methods we care about | 19:31 |
| fungi | not too bad | 19:31 |
| corvus | yep | 19:31 |
| corvus | mordred: I think zuul has delivered an answer to your question about testing https://review.opendev.org/660977 | 19:37 |
| *** pwhalen has quit IRC | 19:39 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Parallelize github event processing https://review.opendev.org/662818 | 19:49 |
| corvus | mordred: i threw a recheck on https://review.opendev.org/656879 -- it should be tested by the new registry job | 20:01 |
| corvus | infra-root, zuul-maint: i have a zuul/opendev crossover question -- the main unresolved question for moving the rest of the zuul repos to the new zuul opendev zuul tenant (that series of words really does make sense) is where to put the js tarballs -- we currently publish them to http://tarballs.openstack.org/zuul/zuul-content-latest.tar.gz -- should we continue doing that? is there some other way we | 20:05 |
| corvus | should publish the latest static js builds? should we deprecate those and tell people to use the docker image (or, at least, copy the content out of it?) should we set up tarballs.opendev.org and port the jobs over to use that? | 20:05 |
| mordred | corvus: so ... in order to do that, I think we'd need to support -c setting a root path for conf.d dir too - or an additional parameter that means taht | 20:05 |
| corvus | mordred: ah, yeah, we probably should -- if we can specify a file, we should probably also be able to specify a dir... | 20:06 |
| corvus | (i think just having -c accept a dir would be fine and the easier thing) | 20:06 |
| tobiash | corvus: do you know if someone is using the tarballs? | 20:06 |
| corvus | (er, i mean, have -c accept either a file or a dir) | 20:06 |
| mordred | corvus: how does the logic of "if -c option exists and is a dir, it's intended to be a base dir, otherwise it's a file | 20:06 |
| mordred | cool | 20:06 |
| *** jamesmcarthur has joined #zuul | 20:06 | |
| corvus | tobiash: i know that opendev is, but of course we'd be happy to change; i don't know if anyone else is. | 20:07 |
| tobiash | I wasn't even aware that they are there and could be used by folks | 20:07 |
| clarkb | tobiash: I think pabelanger is using wheels that end up on the tarballs server | 20:07 |
| mordred | corvus: I'd vote for deprecating/removing the tarballs and either publishing something to npm that could be used instead and/or pointing people to docker | 20:07 |
| corvus | clarkb: don't we publish wheels to pypi? | 20:07 |
| clarkb | and opendev consumes the js tarballs for our deployments | 20:07 |
| tobiash | clarkb: we publish the wheels now to pypi | 20:07 |
| clarkb | corvus: we do, but pabelanger wants per commit wheels | 20:07 |
| clarkb | tobiash: ^ | 20:07 |
| tobiash | and I guess pabelanger is already using them | 20:07 |
| corvus | clarkb: i'm almost certain that pabelanger is using pypi | 20:07 |
| tobiash | (from pypi) | 20:07 |
| corvus | clarkb: at least, that's what he says when he asks us to tag a release :) | 20:08 |
| mordred | clarkb, corvus: I think it woudl be nice to change how we're deploying opendev zuul for this anyway | 20:08 |
| clarkb | are we publishing wheels on every commit? I seem to recall the specific need was wheels on every commit so that windmill can avoid installing from source | 20:08 |
| tobiash | yes | 20:08 |
| clarkb | k | 20:08 |
| tobiash | clarkb: https://pypi.org/project/zuul/#history | 20:08 |
| mordred | so from an opendev-using-them perspective, I thnik it would be fine if they went away (coordinated with changing what we're doing) | 20:08 |
| webknjaz | It's recommended to always publish both sdist and wheel. | 20:08 |
| mordred | webknjaz: yah - we definitely do that | 20:09 |
| clarkb | tobiash: I misunderstood then and think we should stop doing that to pypi | 20:09 |
| webknjaz | nope | 20:09 |
| clarkb | we had to stop mirroring pypi because of projects doing that to pypi | 20:09 |
| *** mugsie_ has joined #zuul | 20:09 | |
| clarkb | and I'm not comfortable with being part of that problem | 20:09 |
| tobiash | clarkb: pabelanger wanted to be able to use pre-releases so we added pushing them to pypi | 20:09 |
| webknjaz | the idea is that sdist should have the whole source snapshot, while wheel is for built artifacts only | 20:10 |
| corvus | webknjaz: yes, we always do both | 20:10 |
| webknjaz | yep | 20:10 |
| clarkb | tobiash: ya I got that part of it, but thought we were using tarballs for the intermediate wheels. I don't like pushing every commit to pypi | 20:10 |
| clarkb | tobiash: its made pypi much less useable in a local/mirror context | 20:11 |
| webknjaz | as for publishing every commit, you should use https://test.pypi.org/project/zuul instead | 20:11 |
| clarkb | webknjaz: I think taht is for testing pypi/warehouse itself | 20:11 |
| clarkb | so pushing every commit there doesn't really help pabelanger | 20:11 |
| webknjaz | not really | 20:11 |
| webknjaz | you can use it for testing your flows | 20:12 |
| clarkb | right we don't need that | 20:12 |
| mordred | clarkb: but it might provide a way to push per-commit wheels to a location pabelanger can point pip to without scewing mirroring | 20:12 |
| webknjaz | i've configured that for molecule and ansible-lint and it keeps the publish pipeline healhy | 20:12 |
| clarkb | mordred: not sure how comfortable users should be installing zuul from test.pypi.org :) | 20:13 |
| webknjaz | is that for users? | 20:13 |
| clarkb | really I think the ideal there is if windmill needs per commit deployment support then doing it from source is what it should do | 20:13 |
| clarkb | webknjaz: yes, the need from pabelanger is to be able to deploy an arbitrary commit and currently it only deploys wheels | 20:13 |
| clarkb | (aiui) | 20:13 |
| mordred | clarkb: yah - the problem is that from source would involve installing all of the js tools on his servers - which is then a MUST larger deployment | 20:13 |
| corvus | i agree that as a project we should not target "test.pypi.org" as one of our release platforms, even for per-commit builds. | 20:13 |
| *** timburke_ has joined #zuul | 20:14 | |
| fungi | the only real concern i'd have with relying on wheels on pypi, images on dockerhub, js libs on npm, et cetera is that we've found occasions where having a backup copy of those artifacts is useful | 20:14 |
| mordred | if we don't want to push per-commit wheels to pypi, I *do* think we should publish per-commit wheels somewhere (What with suggesting zuul is good for CD consumption) - so maybe we should publish to $somewhere but do it as a full index? | 20:15 |
| *** timburke has quit IRC | 20:15 | |
| *** mugsie has quit IRC | 20:15 | |
| corvus | fungi: so tarballs.opendev.org as a backup artifact archive? | 20:15 |
| fungi | perhaps | 20:15 |
| fungi | also pypi doesn't like publishing signatures of packages | 20:15 |
| clarkb | fungi: ya that has come up a few times over the years in various python projects we've hosted/built | 20:16 |
| fungi | i do think that performing uploads tests to test.pypi.org could be useful, since the current rules baked into warehouse about whether a package is acceptable are somewhat inscrutable, but wouldn't want to rely on that as a publishing platform | 20:16 |
| mordred | fungi: ++ | 20:16 |
| fungi | they also say they won't go to the same degree of effort guarantee against content loss on test.pypi.org | 20:16 |
| clarkb | it also doesn't prevent the rules from changing and breaking you | 20:16 |
| webknjaz | exactly | 20:17 |
| fungi | sure, test uploads to test.pypi.org would be more of a canary | 20:17 |
| clarkb | depends on how often you are committing when they switch things over | 20:17 |
| fungi | but orthogonal to the current topic i think | 20:17 |
| clarkb | since if you don't commit for the week they switch you break anyway | 20:17 |
| mordred | well - the current parallel topics | 20:17 |
| corvus | looks like https://review.opendev.org/655474 is the change that started publishing per-commit wheels to pypi | 20:17 |
| mordred | since we've moved on to a second topic from covus' original question :) | 20:17 |
| clarkb | on the publish every commit topic: it does make me wonder if pypi has an official stance on that | 20:18 |
| fungi | some of you have moved on, i'm still catching up ;) | 20:18 |
| clarkb | I suppose if pypi says its ok we'll never turn opinion on that and may as well get in line | 20:18 |
| mordred | fungi: I think we just put it on hold for a second | 20:18 |
| clarkb | but if they'd rather not beacuse pypi is ingesting gigabytes of data every day now maybe we should be nice and stop too | 20:18 |
| webknjaz | clarkb: ask here https://discuss.python.org/c/packaging | 20:18 |
| clarkb | (zuul artifacts are bigger than I expected, I guess that is the js build results?) | 20:19 |
| mordred | yeah | 20:20 |
| fungi | presumably vendoring of vendoring of vendoring of javascript | 20:20 |
| mordred | built artifacts not vendoring | 20:21 |
| fungi | oh, we're not shipping copies of other people's libraries? | 20:21 |
| clarkb | going back to the original question, I think i agree wtih fungi that having a set of copies with hashes/signatures/someverification that we control is valuable so tarballs.opendev.org is probably an alias we should set up | 20:21 |
| fungi | or are you saying the js community simply has a more palatable term for the concept? ;) | 20:22 |
| clarkb | fungi: we are shipping compiled versions of those libraries for the various browser versions | 20:22 |
| mordred | no, not in their entirety. we're shipping minified versions of the relevant parts of other people's libraries. in some contexts one might call that "compiling" | 20:22 |
| clarkb | fungi: you compile jsx into javascript for firefox of this age and firefox of that age and mobile browsers and chrom* etc | 20:22 |
| fungi | okay, so like a statically-linked binary | 20:22 |
| mordred | yeah | 20:22 |
| fungi | still makes me a bit uncomfortable since we're shipping copies of (parts of) random libraries we don't produce and have no real means of alerting users to vulnerabilities in | 20:23 |
| clarkb | but despite having a tarballs archive we can/probably should also push peolpe to use the more official installation repository for their flavor of deployment (eg use docker hub and pypi) | 20:23 |
| fungi | but that ship has sailed i suppose | 20:23 |
| *** jamesmcarthur has quit IRC | 20:25 | |
| corvus | okay, sounds like tarballs.opendev.org is desirable for several reasons; since we publish js to tarballs.openstack.org currently, we may as well continue doing that for tarballs.opendev.org and consider deprecating that at some point in the future | 20:25 |
| mordred | corvus: ++ | 20:25 |
| clarkb | and then separately we can check if pypi has an opinion on per commit/daily uploads that have made mirroring pypi incredibly difficult and change our behavior if necessary | 20:25 |
| corvus | so blockers for zuul moving into its own tenant are: 1) create tarballs.opendev.org; 2) move existing publishing jobs to that | 20:25 |
| clarkb | corvus: can 2) be mostly a noop if we use the same host with an alias? | 20:26 |
| corvus | clarkb: unclear; we may need new jobs with new namespacing rules | 20:26 |
| mordred | well - we probably want to do $something to make the publication job tenant namespaced | 20:26 |
| corvus | (that's been true for most of the opendev "publication" style jobs so far) | 20:26 |
| mordred | yeah | 20:27 |
| corvus | mordred: maybe not tenant namespaced, but at least fully qualified project names | 20:27 |
| clarkb | eg new user with access to subtree and no other user can write to that and vice versa? | 20:27 |
| clarkb | wfm | 20:27 |
| corvus | eg like https://docs.opendev.org/ | 20:27 |
| clarkb | ya docs is set up that way via the kerberos users and acls | 20:27 |
| clarkb | iirc | 20:27 |
| corvus | so we'd probably end up with tarballs.opendev.org/zuul/zuul/zuul-content-latest.tar.gz | 20:28 |
| corvus | which is not the *most* number of "zuuls" i've seen in a path, but is very close. | 20:29 |
| clarkb | my zuul is in src/zuul/ so I have src/zuul/zuul/zuul/lib :) | 20:29 |
| *** themroc has quit IRC | 20:30 | |
| corvus | clarkb: you also have src/zuul/zuul/zuul/driver/zuul | 20:30 |
| mordred | I've got ~src/opendev.org/zuul/zuul/zuul | 20:30 |
| mordred | I've got ~/src/opendev.org/zuul/zuul/zuul | 20:31 |
| mordred | typing is hard | 20:31 |
| fungi | now i feel outdone since mine is just in ~/src/opendev.org/zuul/zuul/ | 20:32 |
| fungi | identical to mordred's now that i'm caught up on scrollback | 20:32 |
| corvus | (i have asked a followup conversation about implementation in #openstack-infra for those following along) | 20:33 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Read multiple *.conf files from directory https://review.opendev.org/660977 | 20:37 |
| mordred | corvus: maybe that will work ^^ | 20:37 |
| *** jamesmcarthur has joined #zuul | 20:37 | |
| mordred | corvus: I think that should handle -c doing dirs, adding a test and fixing the other broken one | 20:37 |
| *** altlogbot_0 has quit IRC | 20:45 | |
| clarkb | tobiash: the multithreaded github processing lgtm. I did leave some notes but I think that will likely make a big impact. | 20:59 |
| clarkb | the py35 test is unhappy too | 20:59 |
| *** rlandy is now known as rlandy|brb | 21:04 | |
| fungi | still trying to wrap my head around ansible's plugin module... i can't tell where the original methods we're shadowing in https://opendev.org/zuul/zuul/src/branch/master/zuul/ansible/base/lookup/ini.py are originally defined... traced all the way back to lib/ansible/plugins/__init__.py in ansible and it's just inheriting from abc.ABCMeta in the python stdlib which also does not have those methods | 21:05 |
| fungi | s/module/model/ | 21:05 |
| *** pcaruana has quit IRC | 21:07 | |
| mordred | fungi: you want to look in lib/ansible/plugins/lookup | 21:15 |
| corvus | fungi: https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/lookup/ini.py#L107 | 21:16 |
| mordred | fungi: the subdirs under plugins are special | 21:16 |
| fungi | yeah, but i don't see where the original read_properties() and read_ini() methods are defined for it | 21:16 |
| fungi | or are these new methods we're adding to the class? | 21:17 |
| fungi | not shadowing existing methods? | 21:17 |
| *** rlandy|brb is now known as rlandy | 21:18 | |
| *** jamesmcarthur has quit IRC | 21:19 | |
| corvus | fungi: why, they're all the way back in v2.3: https://github.com/ansible/ansible/blob/v2.3.0.0-1/lib/ansible/plugins/lookup/ini.py#L54 | 21:20 |
| corvus | i don't seem them in later versions | 21:20 |
| fungi | oh, so this is ansible v2.3 compat code we're carrying to work around leaks in older versions of the plugin not present in modern versions? | 21:20 |
| fungi | i did think to check back as far as v2.6 just to be sure i wasn't overlooking a supported version | 21:21 |
| corvus | fungi: we have a way of supporting different versions of our overrides for different ansible versions. since we're not using that here, there is a good chance that we actually just missed an implementation change and we accidentally lost protection. | 21:22 |
| corvus | (i haven't looked closely enough to say for certain) | 21:22 |
| fungi | got it. looks like that was added over 2 years ago, so sounds likely | 21:23 |
| corvus | (we have a directory for our overrides for each ansible version, most just symlink to the "base" directory, but if we needed different implementations, we would not do that and instead have different files) | 21:23 |
| fungi | okay, now i'm at least not nearly so uncertain about my ability to read python source code. i was second-guessing myself and thinking i was missing some sort of fancy object generator magic | 21:24 |
| corvus | a fair assumption with ansible :) | 21:24 |
| fungi | i'll consider this a lesson to not take provided examples at face value ;) | 21:25 |
| *** sshnaidm|off has quit IRC | 21:34 | |
| openstackgerrit | Merged zuul/zuul master: Set the DiskAccountant log as warning level https://review.opendev.org/660996 | 21:36 |
| *** sshnaidm has joined #zuul | 21:40 | |
| *** sshnaidm is now known as sshnaidm|off | 21:41 | |
| *** mattw4 has joined #zuul | 21:54 | |
| fungi | grr, the fact that this plugin just has a run() method and then relies on in-script private functions is making it hard to duplicate its path resolution behavior. already down in the weeds where it's indirectly calling into the jinja2.FileSystemLoader class | 21:57 |
| fungi | problem is that it wants to allow passing of "relative" paths, where the definition of "relative" is apparently being left up to jinja2 | 21:58 |
| fungi | (i think) | 21:58 |
| clarkb | fungi: it might just use relative paths direclty | 21:59 |
| clarkb | and they end up relative to the tasks cwd | 21:59 |
| fungi | not that i can see | 21:59 |
| fungi | lib/ansible/plugins/lookup/password.py identifies file paths by parsing a terms list passed into its initialization method | 22:00 |
| *** mattw4 has quit IRC | 22:01 | |
| fungi | to do that it calls a path_dwim() method of the _loader attribute from its parent LookupBase class | 22:01 |
| *** mattw4 has joined #zuul | 22:02 | |
| fungi | the LookupBase class seems to expect a loader parameter passed in explicitly at initialization or at least set at some point before use (its default value is None) | 22:03 |
| fungi | ahh, i went down the wrong road. it's in lib/ansible/parsing/dataloader.py | 22:03 |
| fungi | starting here https://github.com/ansible/ansible/blob/devel/lib/ansible/parsing/dataloader.py#L173 | 22:05 |
| fungi | doesn't looks like we currently import ansible.parsing in zuul | 22:07 |
| fungi | though the DataLoader class initialization takes no arguments, so i guess wouldn't be too hard to add | 22:09 |
| fungi | oh, nevermind, i can refer to it as an inherited class method | 22:11 |
| fungi | huh... https://opendev.org/zuul/zuul/src/branch/master/zuul/ansible/base/lookup/ doesn't make it at all clear that those files are symlinks | 22:20 |
| fungi | also what's with all the empty .pyi files? | 22:20 |
| clarkb | the empty .pyi files make the type checker happy iirc | 22:21 |
| *** hashar has quit IRC | 22:24 | |
| openstackgerrit | Jeremy Stanley proposed zuul/zuul master: Safely add Ansible password lookup plugin https://review.opendev.org/662870 | 22:26 |
| fungi | ^ wip for now as it presumably needs (positive and negative) tests | 22:26 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Read multiple *.conf files from directory https://review.opendev.org/660977 | 22:31 |
| mordred | corvus: this time I've learned that os.makedir is not how that is spelled | 22:32 |
| corvus | mordred: RTT on an airplane is not ideal is it? | 22:32 |
| mordred | corvus: it's not awesome :) | 22:33 |
| fungi | i curse myself every time i accidentally pass the -r flag to tox while i'm on an airplane (and destroy the testenvs i've safely cached before i departed) | 22:36 |
| *** zbr has quit IRC | 23:00 | |
| *** sanjayu_ has joined #zuul | 23:02 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Allow download-artifact to download multiple files https://review.opendev.org/662876 | 23:22 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Allow download-artifact to download multiple files https://review.opendev.org/662876 | 23:25 |
| openstackgerrit | Jeremy Stanley proposed zuul/zuul master: Safely add Ansible password lookup plugin https://review.opendev.org/662870 | 23:32 |
| *** jamesmcarthur has joined #zuul | 23:43 | |
| *** mattw4 has quit IRC | 23:44 | |
| *** jamesmcarthur has quit IRC | 23:49 | |
| *** jamesmcarthur has joined #zuul | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!