| *** Goneri has quit IRC | 00:53 | |
| *** rlandy has quit IRC | 02:15 | |
| openstackgerrit | Pierre-Louis Bonicoli proposed zuul/zuul master: Allow to reuse the code handling the branch cache https://review.opendev.org/756725 | 02:26 |
|---|---|---|
| openstackgerrit | Pierre-Louis Bonicoli proposed zuul/zuul master: gitlab: handle protected branches https://review.opendev.org/756900 | 02:31 |
| openstackgerrit | Pierre-Louis Bonicoli proposed zuul/zuul master: Gerrit & Pagure: reuse CachedBranchConnection https://review.opendev.org/758386 | 02:34 |
| openstackgerrit | Pierre-Louis Bonicoli proposed zuul/zuul master: gitlab: handle protected branches https://review.opendev.org/756900 | 02:36 |
| *** hamalq has quit IRC | 02:44 | |
| *** bhavikdbavishi has joined #zuul | 03:10 | |
| *** bhavikdbavishi1 has joined #zuul | 03:13 | |
| *** bhavikdbavishi has quit IRC | 03:15 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:15 | |
| *** tflink_ has joined #zuul | 03:18 | |
| *** tflink has quit IRC | 03:18 | |
| *** vorotech has joined #zuul | 04:16 | |
| *** bhavikdbavishi has quit IRC | 04:24 | |
| *** bhavikdbavishi has joined #zuul | 04:24 | |
| *** vishalmanchanda has joined #zuul | 04:58 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #zuul | 05:33 | |
| *** vorotech has quit IRC | 06:07 | |
| *** bhavikdbavishi has quit IRC | 06:11 | |
| *** bhavikdbavishi has joined #zuul | 06:12 | |
| *** vorotech has joined #zuul | 06:12 | |
| *** vorotech has quit IRC | 06:18 | |
| openstackgerrit | sean mooney proposed zuul/zuul-jobs master: make revoke sudo work if the zuul user is not used https://review.opendev.org/762082 | 06:54 |
| *** wuchunyang has joined #zuul | 07:02 | |
| *** bhavikdbavishi has quit IRC | 07:05 | |
| *** vorotech has joined #zuul | 07:08 | |
| *** bhavikdbavishi has joined #zuul | 07:23 | |
| *** tosky has joined #zuul | 07:32 | |
| *** jcapitao has joined #zuul | 07:40 | |
| *** bhavikdbavishi has quit IRC | 07:44 | |
| *** hashar has joined #zuul | 07:50 | |
| *** vorotech has quit IRC | 07:52 | |
| *** vorotech has joined #zuul | 07:54 | |
| *** vorotech has quit IRC | 07:57 | |
| *** vorotech has joined #zuul | 07:59 | |
| *** bhavikdbavishi has joined #zuul | 08:05 | |
| *** rpittau|afk is now known as rpittau | 08:08 | |
| *** vorotech has quit IRC | 08:14 | |
| *** vorotech has joined #zuul | 08:16 | |
| *** tosky has quit IRC | 08:23 | |
| *** jfoufas1 has joined #zuul | 08:29 | |
| *** jpena|off is now known as jpena | 08:56 | |
| *** SotK has quit IRC | 09:00 | |
| *** SotK has joined #zuul | 09:01 | |
| *** vorotech has quit IRC | 09:13 | |
| *** vorotech has joined #zuul | 09:22 | |
| *** hashar has quit IRC | 09:22 | |
| *** vorotech has quit IRC | 09:25 | |
| *** hashar has joined #zuul | 09:28 | |
| *** tosky has joined #zuul | 09:31 | |
| *** nils has joined #zuul | 09:39 | |
| *** zenkuro has joined #zuul | 09:40 | |
| *** wuchunyang has quit IRC | 10:10 | |
| *** hashar is now known as hasharLunch | 10:18 | |
| *** rpittau has quit IRC | 10:26 | |
| *** johnsom has quit IRC | 10:26 | |
| openstackgerrit | Ashley Bullock proposed zuul/zuul master: Add initial bitbucket cloud driver using webhooks https://review.opendev.org/759003 | 10:26 |
| *** rpittau has joined #zuul | 10:27 | |
| *** johnsom has joined #zuul | 10:28 | |
| *** wuchunyang has joined #zuul | 10:33 | |
| *** sshnaidm|rover is now known as sshnaidm|afk | 10:34 | |
| *** holser has joined #zuul | 10:36 | |
| *** wuchunyang has quit IRC | 10:37 | |
| *** paladox has quit IRC | 10:42 | |
| *** paladox has joined #zuul | 10:46 | |
| *** zenkuro has quit IRC | 11:08 | |
| *** bhavikdbavishi has quit IRC | 11:08 | |
| *** zenkuro has joined #zuul | 11:09 | |
| *** wuchunyang has joined #zuul | 11:40 | |
| *** bhavikdbavishi has joined #zuul | 11:48 | |
| *** bhavikdbavishi1 has joined #zuul | 11:57 | |
| *** bhavikdbavishi has quit IRC | 11:59 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 11:59 | |
| *** jcapitao has quit IRC | 12:02 | |
| *** jcapitao has joined #zuul | 12:03 | |
| *** jcapitao has quit IRC | 12:03 | |
| *** rfolco has joined #zuul | 12:08 | |
| *** rfolco has quit IRC | 12:21 | |
| *** zenkuro has quit IRC | 12:24 | |
| *** zenkuro has joined #zuul | 12:24 | |
| *** zenkuro has quit IRC | 12:26 | |
| *** zenkuro has joined #zuul | 12:26 | |
| *** holser has quit IRC | 12:27 | |
| *** jpena is now known as jpena|lunch | 12:35 | |
| *** wuchunyang has quit IRC | 12:39 | |
| *** hasharLunch is now known as hashar | 12:44 | |
| *** rlandy has joined #zuul | 12:50 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add optional support for circular dependencies https://review.opendev.org/685354 | 12:52 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add optional support for circular dependencies https://review.opendev.org/685354 | 12:54 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Check cycle items are mergeable before reporting https://review.opendev.org/743450 | 12:59 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Make reporting asynchronous https://review.opendev.org/691253 | 12:59 |
| *** sshnaidm|afk is now known as sshnaidm|rover | 13:00 | |
| *** jcapitao has joined #zuul | 13:12 | |
| *** zenkuro has quit IRC | 13:17 | |
| *** rfolco has joined #zuul | 13:32 | |
| *** bhavikdbavishi has quit IRC | 13:32 | |
| *** jpena|lunch is now known as jpena | 13:37 | |
| *** holser has joined #zuul | 13:37 | |
| *** Goneri has joined #zuul | 14:16 | |
| *** jcapitao has quit IRC | 14:24 | |
| *** jcapitao has joined #zuul | 14:25 | |
| *** jcapitao has quit IRC | 14:34 | |
| *** jcapitao has joined #zuul | 14:34 | |
| zbr | does zuul have anything similar to gerrit even stream? i am looking for a way to capture when builds start and end. | 15:06 |
| mhu | zbr, the mqtt reporter maybe? not sure if starting events are emitted though | 15:07 |
| zbr | the database reported clearly does not record starts, and also not doing any streaming. | 15:08 |
| zbr | i guess that even a http based poller could work if the client can specify event timestamp | 15:09 |
| fungi | the db records build start times after the fact, since it's a reporter (reporters act on completion of a buildset) | 15:11 |
| *** rpittau is now known as rpittau|bbl | 15:14 | |
| fungi | the mqtt reporter has the same behavior, so probably also not suitable for what you're wanting | 15:14 |
| fungi | "It is used to send MQTT message when items report." https://zuul-ci.org/docs/zuul/reference/drivers/mqtt.html | 15:15 |
| zbr | that page say: "The reporter action name, e.g.: 'start', 'success', 'failure', 'merge-failure', …" | 15:16 |
| zbr | i see a start there.... | 15:16 |
| fungi | that's for the buildset, so it fires on item enqueue | 15:17 |
| fungi | not on start of each build | 15:17 |
| fungi | e.g., the "starting gate jobs" message opendev's zuul comments on gerrit changes | 15:17 |
| fungi | that's done with a gerrit reporter start condition | 15:18 |
| fungi | i think there was previously talk about adding a more generic internal event stream interface similar to how statsd is handled now, but to potentially support other protocols | 15:19 |
| zbr | using gerrit comments is not quite the best approach, what if zuul does build from other systems like github/gitlab? | 15:19 |
| zbr | not to mention that the comments appear only when the entire buildset finishes, which can be hours later. | 15:19 |
| fungi | zbr: i'm not understanding your question. i was just explaining what a start condition for a reporter is by using an existing example you might have witnessed personally | 15:20 |
| fungi | because you asked about using the start condition | 15:20 |
| zbr | so basically the start event from mqtt means "buildset enqueue", which is very different that one would expect. | 15:21 |
| fungi | it provides feedback letting you know zuul has seen the event and acted on it by enqueuing the item | 15:21 |
| zbr | i would say that most people would count as "start" time the moment the race begun, not the time taken to walk towards the start line from the moment the race was approved. | 15:22 |
| zbr | which is also a very useful event | 15:22 |
| fungi | yeah, i'm not saying what you're asking about isn't useful, just saying that i don't think it exists yet and someone would need to work out how to add it. in previous discussions the closest thing we have to outwardly representing internal events is statsd integration, so extending that to something which could support other communication channels could provide a more detailed event stream than simple counters | 15:25 |
| fungi | do | 15:25 |
| zbr | fungi: so basically the only events supported by MQTT reporter are related to buildsets lifecycle, right? | 15:26 |
| fungi | correct | 15:26 |
| fungi | zuul's original feedback mechanism was reporting on changes, so as it's grown most of the transactional feedback which has been added has been built up around the reporter model | 15:26 |
| avass | I think some people using github wanted jobs reported individually as well | 15:27 |
| avass | and we've also got some interest for that internally at Volvo | 15:27 |
| fungi | it would be good to talk through what that data model would look like on the code review side with regards to signalling/conditions/acls | 15:28 |
| fungi | for the github example, i gather github requires you to predefine the ci results which allow or block merging, which wouldn't be possible with a dynamic set of builds | 15:29 |
| avass | you could still let the pipeline report and block on that | 15:30 |
| fungi | or do you imagine multiple tiers of reporting, separately for the individual builds and also for the buildset as a whole? | 15:30 |
| avass | but display results for individual jobs | 15:30 |
| avass | something like that | 15:30 |
| * zbr is trying to update the MQTT docs to make it clear. | 15:30 | |
| fungi | so the individual build result reporting would be entirely advisory, while the buildset result is what would register the gating signals to github? | 15:31 |
| avass | yeah | 15:31 |
| avass | we're not using github but that's what I understood other systems do and what webknjaz wanted | 15:31 |
| avass | and that would probably work for other usecases as well | 15:32 |
| *** zenkuro has joined #zuul | 15:33 | |
| avass | we just want to be able to report that data to a separate internally service for visualization | 15:35 |
| corvus | actually the 'start' reporter reports when the first build of the buildset requests nodes; it's a little different than it being enqueued. | 15:36 |
| webknjaz | @fungi: yep, something like that | 15:36 |
| corvus | avass: i think improvements to the sql db might be the best way to accomplish that. remember that in v4 it won't be a reporter any more, it'll be integral. so we can store more fine-grained data about more builds. | 15:37 |
| openstackgerrit | zbr proposed zuul/zuul master: docs: clarify MQTT messages https://review.opendev.org/762159 | 15:37 |
| avass | there was also an idea to just do that with an ansible callback somehow | 15:37 |
| fungi | corvus: thanks for the clarification, i forget that zuul has to analyze the item and its deps before it can decide on a node request | 15:38 |
| fungi | and that can indeed have its own delays | 15:38 |
| webknjaz | Another idea that crossed my mind is that on Zuul side, it'd be cool to have some view with jobs/logs similar to what all other CIs provide. Just this morning I was reading a feedback of how frustrating it is that boils down to the UI being a universal view into the database details rather than high-level concepts everybody is familiar with | 15:38 |
| fungi | webknjaz: examples would be great. zuul is the only ci system i've used for so many years i personally have no idea what sorts of interfaces other ci systems provide these days | 15:39 |
| avass | corvus: now I haven't worked with sql a lot, but what we want to do is report any events (job start,success,failure,abort) to a message queue that handles the rest | 15:40 |
| corvus | webknjaz: is this feedback accessible to other folks? | 15:41 |
| webknjaz | @fungi: @corvus: https://github.com/pypa/pip/pull/9107#issuecomment-724536732 | 15:41 |
| zbr | not sure if webknjaz was thinking about the same thing as me, but I kinda find the ab(use) of GUIDS in zuul far from user friendly. It produces a lot of extra content which is mostly noise. Which human would observe that one letter changed in an endless hash? | 15:42 |
| zbr | i know the technical reasons for hashes, but I doubt they deserve primetime in the UI (including urls) | 15:42 |
| webknjaz | Basically, I'd like to explore a separate/additional UI that would hide all the implementation details | 15:43 |
| corvus | webknjaz: i don't read that comment in that way. | 15:43 |
| corvus | webknjaz: it sounds like that person landed on a text log file instead of being directed to a buld or artifact page | 15:43 |
| webknjaz | I have a metafor for this: using Zuul feels like if a typical website, instead of asking for a user/password pair on log in required everyone to write a SQL query | 15:44 |
| corvus | it seems like once ianw pointed them at the build page they were a lot happier | 15:44 |
| zbr | webknjaz: considering my lack of success in improving the current ui even with minor changes, I would recommend you to start with very small improvements | 15:45 |
| corvus | webknjaz: i appreciate your intent to help, but that's not productive criticism. | 15:45 |
| corvus | webknjaz: that's a lot like your earlier statement that the ui was "terrible" :( | 15:45 |
| corvus | webknjaz: let's try to find out what's actually not working for people and improve it. concrete issues and solutions. | 15:46 |
| zbr | does anyone remember why mypy is pinned down? i run into some problems running locally and I supposed it may be related to me using py39 | 15:46 |
| corvus | for example, how did pfmoore end up on a text log? did something report that to the wrong place? or did they just end up looking at a non-representative example. | 15:47 |
| clarkb | zbr: I think there are notes in the change or commit log. I remember bumping it up as high as I could with minor edits to the code | 15:47 |
| webknjaz | @corvus: yes but still there's a huge disconnect between the typical Zuul users bubble and the outside world. Good UX usually means that the user wouldn't get lost there when seeing the new UI, having to explain every bit to new folks each time is by itself a good signal that something should be improved radically. I myself get lost 50% of time I end up in that UI. | 15:48 |
| corvus | webknjaz: i'm not convinced that pfmoore actually saw the ui | 15:48 |
| zbr | clarkb: thanks, i am looking now if another bump with minor tuning would fix it. maybe we can also add a py39 job? | 15:48 |
| avass | corvus: I think one way to start is to report invidual jobs to github somehow | 15:48 |
| corvus | webknjaz: "There's only one link to an example of output that I could find." that's what pfmoore said. that doesn't say "i saw zuul used for real on my repo and followed the link it reported and had a problem" | 15:49 |
| corvus | avass: i would welcome a concrete proposal for how to accomplish that with all the caveats that are in the existing doc from tobiash. | 15:50 |
| corvus | i said as much the last time we talked about this, but i haven't seen anything show up on the mailing list yet. | 15:50 |
| avass | I was waiting for webknjaz to send something :) | 15:50 |
| corvus | avass: and again, i'm pretty sure that three people have joined this conversation and are bringing their own pet peeves to it | 15:50 |
| corvus | that's fine | 15:50 |
| corvus | but let's be up front about it | 15:50 |
| corvus | if avass wants individual github job reports, that's great, but let's not think that's going to solve pfmoore's problem that they found an example text log somewhere | 15:51 |
| avass | I'm not sure how that would happend either. Unless he caught an unfinished buildset and clicked on a finished build | 15:52 |
| corvus | i mean, the comment came before the project was even set up | 15:53 |
| fungi | or found a project which had a misconfigured zuul reporting on it or something | 15:53 |
| corvus | so i have no idea what they found | 15:53 |
| corvus | could have just googled or something | 15:54 |
| *** rpittau|bbl is now known as rpittau | 15:54 | |
| webknjaz | @corvus you're right, I must be projecting my own impression on what Paul wrote there. But there was a similar sentiment in 2 issues that led to the PR: we had to fight for the change to even give it a try. And yes, job matrix reporting is separate from this. I haven't had a chance to write down all my thoughts on this yet. | 15:55 |
| avass | corvus: it did? (ianw ran some rechecks before that) | 15:56 |
| avass | maybe pfmoore looked at the links ianw posted: https://github.com/pypa/pip/pull/9107#issuecomment-722890728 | 15:58 |
| corvus | avass: maybe. anyway, for me the comment reads much more like the latter than a build page. | 15:58 |
| avass | yeah | 15:58 |
| *** zenkuro has quit IRC | 16:00 | |
| openstackgerrit | Ashley Bullock proposed zuul/zuul master: Add initial bitbucket cloud driver using webhooks https://review.opendev.org/759003 | 16:04 |
| *** jfoufas1 has quit IRC | 16:06 | |
| *** vishalmanchanda has quit IRC | 16:06 | |
| corvus | avass, webknjaz: on this page, i see the individual job(s) listed: https://github.com/pypa/pip/pull/9107/checks?check_run_id=1378708174 | 16:10 |
| corvus | basically, if we reported jobs as individual checks, they would show up on the left | 16:10 |
| corvus | because of the compromise we made, you have to click "pypa/check" and then the jobs show up in the middle of the page | 16:11 |
| zbr | ianw: corvus: please check again https://review.opendev.org/#/c/734833/ and tell me if last version is ok. | 16:11 |
| corvus | i'll grant it's a little deficient compared to other systems (but only because zuul is doing more) | 16:11 |
| corvus | and if we could improve it for that use case, it's worth considering | 16:11 |
| corvus | but the direct job links are there; we're not asking folks to go searching through buildset indexes or anything to find their jobs | 16:12 |
| avass | yes and that could stay which is what would allow maintainers to select which check should block merging a PR | 16:12 |
| corvus | avass: yeah the "do both" option? | 16:12 |
| avass | yep, which is what other CI services do afaik | 16:13 |
| avass | that way the individual jobs are more visible which would probably cause less confusion | 16:13 |
| corvus | right, my point is that the status quo isn't *terrible* -- they are there, the ui just isn't as slick as it could be | 16:14 |
| fungi | how does that play out with the need for github "checks" to all be predefined? just don't register the individual jobs as checks, or are those other ci systems relying on job lists being static? | 16:14 |
| corvus | anyway, i think it's a good idea, and look forward to webknjaz's write up so we can analyze/agree on it and move forward | 16:14 |
| avass | fungi: I don't think all jobs has to be predefined | 16:15 |
| corvus | fungi: they only need to be predefined in order to require them as branch protection | 16:15 |
| avass | at least that's not my experience | 16:15 |
| fungi | ahh, okay | 16:15 |
| * zbr thinks analogy around half-full or half-empty glass is needed. | 16:15 | |
| corvus | fungi: so in order to tell github "require gate before allowing a change to merge" there has to be a check called "gate" that has run -- so you need a stable name | 16:15 |
| avass | corvus: though what we want isn't github support. we just have some users that want data reported when the builds finish instead of when the buildset finishes. but it's not a high priority in any way :) | 16:17 |
| corvus | fungi: iiuc webknjaz suggestion is/will-be that we report the pipeline as a check, and also the individual jobs as a check. a user configuring the github branch protection would tell github to require "gate" and ignore all the other checks it knows about. | 16:17 |
| zbr | imho, what I find confusing is that I cannot easily get to the main-build-output when I click the build. instead i get a full dashboard which can be overwhelming | 16:17 |
| fungi | zbr: which output is the main build output? an artifact? | 16:17 |
| avass | I think the idea was also that the invidual checks for the jobs would be namespaces with the pipeline name so they didn't override eachother | 16:17 |
| corvus | avass: right, there's a fundamental disconnect with that request and we should dig into what you really need and why to figure it out. the fundamental issue is that in zuul, it's not over till it's over (speculative execution makes that necessary). | 16:18 |
| zbr | lets think about very basic example: tox jobs. regardless if the job is success or failure, if you clicked on it you are likely to want to see tox output right away, without extra digging. | 16:18 |
| zbr | if it was a failure is bit better as we do display the abridge failed task. | 16:18 |
| corvus | zbr: when i click "pypa-pip-py38" on that check output page, i get to the summary page for that build | 16:18 |
| avass | corvus: right since that would technically only work in an independent pipeline unless you're okay with a 'finished' job starting again. | 16:18 |
| corvus | this page: https://github.com/pypa/pip/pull/9107/checks?check_run_id=1378708174 leads here: https://zuul.opendev.org/t/pypa/build/3a977302d0264abbb9326102eedb32a4 | 16:18 |
| zbr | but if it passed, the new user will struggle to find the tox output in the output. | 16:19 |
| zbr | when analyzing UX, try to put the hat of someone what has no knowledge of zuul. (can be hard) but try. | 16:19 |
| corvus | zbr: can i summarize your feedback as suggesting that the 'console' tab is not apparent/discoverable? | 16:19 |
| zbr | while console tab is visible, what user will see is not a "terminal console with tox output" when it clicks it. | 16:20 |
| zbr | it will see lot of ansible tasks, which they have no clue about. finding the right task is ticky, most of the time I use "search in page" | 16:21 |
| avass | imo the ansible tasks are close enough to github workflow tasks so it shouldn't be too hard to figure out | 16:22 |
| zbr | after clicking on console, you get a huge table, where you need to scroll to a particular task, which is not highligted in any way, it is just called "Run tox". | 16:22 |
| clarkb | avass: zuul isn't parsing the stdout though :P | 16:23 |
| zbr | i never seen 30-ish tasks on github actions, usually there are 3-4 of them. | 16:23 |
| corvus | zbr: well, that's on us as job authors | 16:24 |
| corvus | we told ansible to do 30 things | 16:24 |
| zbr | i know it would be quite tricky to determine which tasks are meaningful when you have a success. for failures is easier. | 16:24 |
| corvus | if we want it simpler, we could just write 4 ansible tasks | 16:24 |
| zbr | i think we need to think about some kind of nesting/collapsing, in order to reduce apparent complexity. | 16:24 |
| fungi | what i don't understand is why would the user have any more understanding of a giant heap of text output than they would for the tasks which make up the job they ran | 16:25 |
| corvus | zbr: you know the playbooks are expand/collapseable? and there are 4 of them on the job i linked? | 16:25 |
| zbr | fungi: because user likely already run tox locally, he knows what he would get from tox. | 16:25 |
| avass | zbr: isn't that because github VMs already have a lot of software already installed and zuul jobs install it during the job and display it in the console instead? | 16:25 |
| corvus | here's the problem: | 16:25 |
| zbr | the expectation of the user is that CI would somehow provide a similar experience with local command line run. | 16:26 |
| corvus | we're running ensure-pip in the run playbook | 16:26 |
| corvus | it should be in pre | 16:26 |
| corvus | if we did that, then only the tasks that are relevant would be displayed | 16:26 |
| tobiash | also ensure-tox could be moved to the pre-run | 16:26 |
| corvus | tobiash: ++ | 16:26 |
| fungi | zbr: so the idea is that job authors should be able to flag specific tasks as "expand by default" in the console view or included in the summary view regardless of outcome? | 16:26 |
| zbr | it is true that the idea of all-in-once images from github does simplify the pipeline definition. | 16:27 |
| zbr | fungi: yep, i think something like this. | 16:27 |
| tobiash | can this be done with tags? | 16:27 |
| fungi | i don't know what all-in-once images github provides. can you explain? | 16:27 |
| zbr | as job author you are likely to know what task is likely to be the "core" of your job. | 16:27 |
| avass | fungi: anything you could wish for :) | 16:28 |
| zbr | fungi: for example github pre-installs stuff like docker, podman, and lots of other stuff on its images, they are just there. | 16:28 |
| corvus | zbr, tobiash: if we re-arranged those playbooks as they should be, that job would look like this: https://imgur.com/a/YmzzyMh | 16:28 |
| fungi | got it. well in theory zuul deployments can have that too | 16:28 |
| tobiash | corvus: looks much better | 16:29 |
| fungi | no reason you can't make a "docker" nodeset and a "podman" nodeset and whatever | 16:29 |
| tobiash | and as side effect the job is more stable :) | 16:29 |
| avass | corvus: ++ | 16:29 |
| corvus | i think if a zuul deployment were to target github users, they absolutely should have those thing pre-installed on images. | 16:29 |
| fungi | but if you want to test different versions of "docker" than that image provides, suddenly installing docker in a pre playbook becomes more flexible | 16:29 |
| corvus | note that opendev is not targeting github users | 16:29 |
| zbr | fat-images are ticky for mentioned resons and also because to do them right you need a huge amount of testings, something we kinda lacked so far. | 16:30 |
| fungi | yeah, opendev in particular is trying to maintain a minimal number of per-distro-release node types to keep maintenance burden low | 16:30 |
| corvus | opendev's users value reproducibility from scratch on vanilla images very highly, so we're moving to bare images more and more | 16:30 |
| zbr | also github has only one distro to support, much easier for them | 16:31 |
| zbr | i would focus our attention on improving the display of tasks as a first step | 16:31 |
| * avass putting some comments on the PR so this leads to something | 16:31 | |
| fungi | opendev's largest user, openstack, has job frameworks which want to test that they can correctly install those things, so having them preinstalled works against that goal | 16:31 |
| corvus | avass: on the pr? | 16:32 |
| fungi | also different frameworks which disagree on how and from where those same tools should be installed | 16:32 |
| avass | corvus: https://github.com/pypa/pip/pull/9107 but I guess it's just the ensure-tox | 16:32 |
| zbr | I bet 99% of users do not need/want to see all the noisy 11 tasks before "tox: Run tox" one. | 16:32 |
| corvus | avass: i'm not sure that putting a comment on a PR in the pip project is going to lead to opendev restructuring its jobs? | 16:32 |
| corvus | zbr: of course not, that's why we have pre-playbooks and why they are hidden by default | 16:33 |
| corvus | zbr: the solution is to move those tasks to pre-playbooks | 16:33 |
| fungi | zbr: at least not until they propose changes to their project which could influence how those tasks run | 16:33 |
| avass | corvus: that job was in the repo | 16:33 |
| avass | corvus: normally ensure-tox would have been in a pre-run | 16:33 |
| corvus | avass: ack, then i agree, pr comment makes sense :) | 16:34 |
| corvus | zbr: it doesn't require any new ui work. it requires using the tools we already have correctly | 16:34 |
| corvus | zbr: let me put it another way: what you want already exists. wish granted! | 16:34 |
| zbr | and maybe we can combine it with a change that allows us to define a task that is included in summary even for succesful runs? | 16:35 |
| zbr | so user will get to output without clicking anything. | 16:36 |
| corvus | zbr: that's a good idea worth exploring | 16:36 |
| corvus | zbr: though, as an alternative, what about just dropping the summary page and having the user land on the 'console' tab? | 16:36 |
| corvus | s/summary page/summary tab/ | 16:37 |
| zbr | corvus: tbh, what is the value of the summary page? those two stats lines? | 16:37 |
| avass | zbr: or output for any failing task | 16:38 |
| zbr | but at the same time, I find the header part as taking far too much screen real-estate | 16:38 |
| clarkb | also artifact links | 16:39 |
| clarkb | I use the artifact links a lot | 16:39 |
| corvus | clarkb: those are in a tab now | 16:39 |
| clarkb | oh heh shows how much I've done since the pf updates | 16:39 |
| fungi | the summary page used to have some additional details which have now moved up into a more persistent area of the build result ui | 16:39 |
| fungi | so i agree it's serving less of a distinct purpose now | 16:40 |
| webknjaz | What @zbr wrote is something that I've been failing to communicate clearly. Basically, GitHub users expect that if they add a shell command, they'll see that command executed in the log just if it was a local run. They don't need to know about the implementation details (Ansible etc). Other CI providers hide or collapse their infrastructure scripts (you usually need to enable some debug mode to see them). But then, it puts what the project | 16:40 |
| webknjaz | maintainers wrote in the CI config on the spotlight: this is what is important to them *most of the time*. All the extra Ansible-specific details are just noise in this case. They may be necessary occasionally if everything goes wrong heavily but not very often. If I tell the CI to run tox, I expect to see exactly that: tox command with args logged + it's output + maybe a flag if it failed... | 16:40 |
| corvus | yeah, i think the value is to signal "everything's ok!" or "here's what probably went wrong". that predates the console tab though, and the console tab also tries to guess what went wrong (but it shows you the full output), so it's worth considering whether both are still necessary. | 16:40 |
| zbr | i do not think that header should take ~40$ of screen width (4K full screen). The effective list of tasks starts on the second half of the scren. | 16:40 |
| corvus | webknjaz: i agree. i believe that the zuul software and job authors also agree, and that we have the tools to do that but are not using them correctly now. | 16:41 |
| *** iurygregory has quit IRC | 16:42 | |
| corvus | webknjaz: i think the most zuul-ish way to achieve that would be to move "boring" actions out of the run playbook into pre-run playbooks so that the console tab shows only the interesting actions. then i think we should think about making the console tab the default. | 16:43 |
| fungi | sounds like the other desire is possibly auto-expanding one or more tasks even if they succeed | 16:43 |
| corvus | fungi: ++ (zbr) | 16:44 |
| webknjaz | Yeah, the console tab seems important | 16:44 |
| fungi | so for example if clicking on a job result took you to something like https://zuul.opendev.org/t/zuul/build/be42d72ce6e046aca46787ab485dbcff/console but with the "tox: Run tox" task expanded by default? | 16:44 |
| * zbr feel happy as it seems to be some kind of agreement regarding what to change the UI. | 16:45 | |
| avass | maybe install-siblings should be part of the pre-run. that's not really running any tests and is something you'd expect to succeed | 16:45 |
| zbr | maybe we can add an ansible tag "expanded" to declare its status in UI. | 16:45 |
| corvus | also, we might look at roles like "tox" and try to simplify them. maybe we shouldn't do "record file location" and "create a tempfile" and "remove tempfile" all as individual ansible tasks; maybe we should make some shell snippets to do some of that to keep the noise down. | 16:46 |
| webknjaz | @fungi other CIs dynamically expand each task during its execution and collapse them back on success. If all the tasks succeed, they all are collapsed (which is okay if they have meaningful names). If a task fails, it usually stays expanded | 16:46 |
| fungi | webknjaz: got it. so that's what the build result has for zuul's console tab as well. failed tasks are left expanded by default | 16:46 |
| zbr | webknjaz: this report is generated after the run. for live output we only have a standard streamed console. | 16:46 |
| corvus | fun fact: we actually could do this live if we wanted :) | 16:47 |
| webknjaz | I'm just sharing how other things work :) | 16:47 |
| corvus | it would be some fancy js programming, but all the pieces are there | 16:47 |
| corvus | (the system was designed to support that eventually) | 16:47 |
| avass | corvus: do you mean the console tab? because I'm not sure how that would work | 16:49 |
| webknjaz | I feel like we need to get some of Zuulers watch a few builds in various CIs so that everyone could be on the same page, because retelling in own words loses may be imprecise | 16:49 |
| fungi | the other challenge i see is that our per-task view lacks the timestamping and line numbers/clickable reference anchors we get from the log viewer. i don't know whether we have sufficient metadata to be able to add those features to the console stdout/stderr lines but it would be nice to have consistency there if possible | 16:49 |
| corvus | fungi, zbr: maybe we should keep the "expand this task even on success" idea in reserve for now -- ie, let's try out the other ideas first; because i think there's an argument to be made that if we get the rest of the ui right, then users won't need/want successful items expanded (it would be obvious which thing to expand if they really want to see it) | 16:49 |
| corvus | webknjaz: i think most of us have seen other systems :) | 16:49 |
| * avass does not miss the jenkins days | 16:50 | |
| webknjaz | Oh, I just got an impression that some of the assumptions about the other systems are a bit off or maybe just outdated | 16:51 |
| fungi | i have seen the results of other systems from time to time, but i don't generally "watch" ci systems in action, i have work of my own to be doing while the ci systems do theirs. also it doesn't help that github doesn't show ci details to anyone who isn't logged in | 16:51 |
| webknjaz | Yeah, that's weird. But other systems are public | 16:52 |
| fungi | usually i'm working on (at least) ten different things so if i push a change for something i'll switch gears to a different activity and check back later to find out what results the ci system gave. i can't imagine having time to sit and watch jobs as they unfold unless i'm debugging the jobs themselves | 16:52 |
| webknjaz | Fair enough | 16:53 |
| fungi | i think a lot of zuul users work similarly, so it hasn't been developed as a source of live entertainment ;) | 16:53 |
| corvus | we also don't seek to explicitly copy any other system | 16:54 |
| corvus | happy to borrow as things see fit | 16:54 |
| corvus | but our goal is not to out-travis travis :) | 16:54 |
| avass | webknjaz: it also seems to me that other systems are focusing on different things. zuul is trying to scale up CI while other systems only seem to manage a single repo at a time and run changes independently and let users run regression tests after they're merged. | 16:54 |
| corvus | the fact that we can do what avass said *and* it turns out we also have a console log data structure that lets us display data similarly to other ci systems (if only we tweak it a bit to improve the signal/noise ratio) is pretty cool. :) | 16:55 |
| corvus | but we have that data because it was driven by the underlying use case (multi-system coordination), not because we wanted to mimic travis | 16:56 |
| webknjaz | There's cases when something is only reproducible in CI and you have to push updates to Git to trigger it and then you watch it reach the failing step to see if your change fixed it which is not uncommon | 16:56 |
| avass | our answer when someone asks us how we handle regression is almost always "we don't have regression" | 16:57 |
| corvus | avass: we should put that one the hompage | 16:57 |
| avass | :) | 16:57 |
| avass | webknjaz: yes and those errors should be investigated and fixed :) | 17:00 |
| *** sshnaidm|rover is now known as sshnaidm|off | 17:01 | |
| fungi | webknjaz: even then, i trust that the build result will probably contain the data i need to discover what's broken there. and if it doesn't i can push up another change to add more detail/gather more log files and configs... but i don't usually watch it live i just iterate on it in between juggling other activities | 17:01 |
| webknjaz | re: expand all —there's one case where it's useful: sometimes some env/transitive dep changes and breaks the build w/o changes to the project itself. In this case, I often open two builds an need to compare the logs. But it's not happening too often so instead of the autoexpand I'd suggest a UI button for that | 17:01 |
| clarkb | webknjaz: that case and ones like it are why I'm the oddity that lives in the text logs. I rarely use the rendered versions | 17:02 |
| clarkb | I find that I can manipulate text far more effectively. Grep/diff/etc. I'm always lost when dealing with travis type results | 17:03 |
| fungi | i do certainly perform side-by-side comparison of job logs, but that doesn't rely on live streaming | 17:03 |
| clarkb | but I also liekly debug "why did this job fail in a weird way" more than most | 17:04 |
| webknjaz | @fungi: when the failure happens pretty fast after the build starts, the context switching to other tasks contribute to loosing focus so it may be easier to keep rechecking the build | 17:04 |
| fungi | webknjaz: that's fair. i'm also accustomed to a ci environment which is often backlogged and so don't expect my builds to start immediately | 17:05 |
| zbr | clarkb: fungi: any chance to make the Makefile a reality? https://review.opendev.org/#/c/723837/ | 17:08 |
| *** hamalq has joined #zuul | 17:10 | |
| avass | corvus: btw are updates to zuul-helm appreciated or is the focus more on the operator. I've almost finished up our helm chart for deploying zuul/nodepool/zookeeper and might be able to push to opendev from that | 17:10 |
| *** hamalq has quit IRC | 17:10 | |
| clarkb | zbr: I'm not sure we should merge that unless we intend to use it since it may regress otherwise. That is basically why my makefile from 2014 never merged | 17:10 |
| *** hamalq has joined #zuul | 17:11 | |
| corvus | avass: i think helm updates would be welcome; i stopped using it though, and am unsure about mnaser. but i'd be happy to take a look if you push something up | 17:11 |
| clarkb | zbr: also I'd prefer that if we used make we didn't continue to depend on tox | 17:12 |
| clarkb | at least my original motivation was to remove the tox dep | 17:12 |
| clarkb | (and there were many erason for thati ncluding it was impossible toget bug fixes in tox, tox chagned a lot and broke things, and so on) | 17:12 |
| *** rlandy is now known as rlandy|brb | 17:12 | |
| corvus | [i have to afk for a few hours] | 17:13 |
| avass | corvus: sure, some parts might be a bit rough. but I'll see if I can get something up :) | 17:13 |
| zbr | clarkb: i think that ditching tox is a totally different aspect and i am personally against it. | 17:14 |
| clarkb | zbr: in that case lets not add a makefile | 17:14 |
| clarkb | its just noise to do that | 17:14 |
| clarkb | tox and make fill the same niche for python projects | 17:14 |
| zbr | not really, now I cannot lint all. there is a huge divide between the python part and the js part. | 17:14 |
| avass | I guess the biggest part would be a zookeeper subchart with tls certificate generated by a job pod unless it already exists | 17:14 |
| clarkb | zbr: tox runs a script that builds the js stuff. Is that broken? can we fix that if so? | 17:15 |
| clarkb | zbr: its the install_command in tox.ini and the script is tools/pip.s | 17:16 |
| clarkb | *tools/pip.sh | 17:16 |
| zbr | tox linters does not run yarn. if you agree to run it, i can stick it there and I am ok. | 17:17 |
| zbr | at this moment this was the only bit i wanted, the next one was to run start yarn commands like local UI. but somehow i missed to add it to this PR. | 17:17 |
| clarkb | does it need to run yarn to run the linters properly? | 17:18 |
| clarkb | I feel like I'm missing something | 17:18 |
| zbr | "yarn lint" is not included in "tox -e linters" | 17:18 |
| clarkb | ok thats a completely separate conversation to adding a makefile | 17:18 |
| zbr | if this was by design or accident, i do not know. | 17:18 |
| clarkb | why not add yarn lint to linters and start a conversation about it? | 17:18 |
| zbr | it would ok for me, but take a look at https://opendev.org/zuul/zuul/src/branch/master/web/package.json#L48-L55 | 17:20 |
| zbr | we have several other development commands that are not easily accesible. | 17:20 |
| zbr | we could add them to the Makefile and have a single cli launcher. | 17:20 |
| clarkb | you can also add them to tox | 17:20 |
| clarkb | whcih is the current "cli launcher" | 17:20 |
| clarkb | (personally I don't think all of the commands need to be exposed a layer above, if you need something specific running yarn direclty should be doable) | 17:21 |
| zbr | based on the docs, we have two now: tox and yarn from inside web/ folder. | 17:21 |
| zbr | i would not mind using tox, but i am not sure how to name the fake environments for calling yarn tasks. | 17:22 |
| zbr | tox was never designed to be a generic build tool. | 17:22 |
| clarkb | well in this case you aren't relying on make build features either, just phony targets to run commands which tox does just fine | 17:22 |
| *** nils has quit IRC | 17:23 | |
| zbr | are you ok if I add: "start" and "build" fake tox envs, and also yarn lint to existing linters? | 17:23 |
| zbr | i can also populate the "description" fields. | 17:24 |
| clarkb | no as mentioned above I would leave out the more specific ones from tox and make | 17:24 |
| corvus | web devs use yarn. we don't need tox for that. | 17:24 |
| clarkb | I think adding the linter to linters is fine | 17:24 |
| corvus | (if you can't handle running 'yarn start' you shouldn't be doing js dev) | 17:24 |
| zbr | you need to run it from inside web/ folder, it does not work from root of project. | 17:25 |
| corvus | yep | 17:26 |
| zbr | i never listed js on my resume, but somehow I ended up making few PRs on js area for zuul. | 17:26 |
| corvus | zbr: and you learned to use yarn :) | 17:26 |
| zbr | yep | 17:26 |
| corvus | now you are js dev | 17:26 |
| corvus | if you used tox you wouldn't have that important js dev skill :) | 17:26 |
| corvus | don't deny anyone else the opportunity to learn from which you already benefitted :) | 17:27 |
| corvus | in all seriousness: yarn is the right tool for 'yarn start' 'yarn add' etc. we shouldn't mask it with tox. | 17:27 |
| zbr | corvus: yep, but I lose the opportunity to force-fed Makefile experience to others | 17:27 |
| tristanC | it sounds better to tell contributor when to use tox or yarn instead of hiding that fact in makefile or toxfile | 17:28 |
| corvus | zbr: you make a compelling argument ;) | 17:28 |
| corvus | anyway, sorry, really have to run now | 17:28 |
| *** jcapitao has quit IRC | 17:28 | |
| *** iurygregory has joined #zuul | 17:33 | |
| *** rlandy|brb is now known as rlandy | 17:47 | |
| *** hashar has quit IRC | 17:59 | |
| openstackgerrit | zbr proposed zuul/zuul master: Bump mypy https://review.opendev.org/762187 | 18:14 |
| *** rpittau is now known as rpittau|afk | 18:20 | |
| openstackgerrit | zbr proposed zuul/zuul master: Bump mypy https://review.opendev.org/762187 | 18:27 |
| zbr | any reason why we do not have a base tox-py39 job? i see an openstack-tox-py39 one. | 18:31 |
| openstackgerrit | zbr proposed zuul/zuul-jobs master: Add tox-py39 job https://review.opendev.org/762192 | 18:34 |
| avass | zbr: I don't think there's any specific reason not to | 18:36 |
| openstackgerrit | zbr proposed zuul/zuul-jobs master: Add tox-py39 job https://review.opendev.org/762192 | 18:50 |
| *** jpena is now known as jpena|off | 18:52 | |
| avass | zbr: was there any more 208 changes? | 19:02 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: revoke-sudo: delete cloud-config sudoers file https://review.opendev.org/762198 | 19:05 |
| avass | i remember there being a change about that ^ but I don't remember why that wasn't merged | 19:06 |
| avass | supporting cloud-config labels in nodepool would be nice :) | 19:06 |
| *** rf0lc0 has joined #zuul | 19:26 | |
| *** hashar has joined #zuul | 19:27 | |
| *** rfolco has quit IRC | 19:27 | |
| openstackgerrit | zbr proposed zuul/zuul-jobs master: Add tox-py39 job https://review.opendev.org/762192 | 19:29 |
| *** gmann is now known as gmann_lunch | 19:29 | |
| openstackgerrit | zbr proposed zuul/zuul-jobs master: More E208 (final) https://review.opendev.org/761297 | 19:35 |
| openstackgerrit | Albin Vass proposed zuul/nodepool master: WIP: Digitalocean driver https://review.opendev.org/759559 | 19:37 |
| openstackgerrit | Albin Vass proposed zuul/nodepool master: WIP: Digitalocean driver https://review.opendev.org/759559 | 19:38 |
| ianw | corvus: speaking of UI, there's a small stack of things @ https://review.opendev.org/#/c/758534/, https://review.opendev.org/#/c/758530/4 https://review.opendev.org/#/c/751140/15 that have been sitting for a while | 20:08 |
| ianw | i haven't got back to re-testing the api mocking felixedel proposed, will try to do | 20:09 |
| corvus | ianw: thanks, i'll take a look at those soon | 20:10 |
| *** iurygregory has quit IRC | 20:25 | |
| *** tosky has quit IRC | 20:42 | |
| *** tosky has joined #zuul | 20:42 | |
| *** zenkuro has joined #zuul | 20:44 | |
| openstackgerrit | Albin Vass proposed zuul/nodepool master: Reorganize drivers into separate documents https://review.opendev.org/762208 | 20:52 |
| avass | corvus: wanna take a look at that ^ if you got time over? | 20:53 |
| *** hashar has quit IRC | 20:55 | |
| *** tflink_ is now known as tflink | 20:56 | |
| openstackgerrit | Albin Vass proposed zuul/nodepool master: WIP: Digitalocean driver https://review.opendev.org/759559 | 21:02 |
| openstackgerrit | Albin Vass proposed zuul/nodepool master: Fix urllib3 dependency https://review.opendev.org/762210 | 21:29 |
| avass | looks like a new urllib3 version broke ci | 21:33 |
| *** iurygregory has joined #zuul | 21:51 | |
| corvus | avass: ^ q from ianw | 22:08 |
| ianw | yeah, i'm looking but it's not clear to me what the dep chain is | 22:09 |
| corvus | ianw: you're wondering why requests shows up at all? at least via openstacksdk | 22:10 |
| corvus | (and probably 6 other cloud client apis too) | 22:10 |
| ianw | ok, so that's what pins it to 2.24.0? | 22:11 |
| corvus | i don't know about a pin | 22:11 |
| corvus | ianw: it looks like 2.24.0 is latest | 22:11 |
| corvus | so just a dep on requests would pull that in | 22:12 |
| ianw | oh, right, so urllib3 have made a release that is incompatible with the latest requests | 22:12 |
| corvus | ianw: that's my understanding | 22:13 |
| corvus | (latest requests is 2.24.0 and is several months old; urllib3 1.26.0 is 2 hours old) | 22:13 |
| corvus | this is, to my recollection, not the first time urllib3 has broken requests | 22:13 |
| ianw | if only there was some sort of system where you could run cross-project ci to avoid this sort of thing | 22:13 |
| corvus | if only... | 22:14 |
| ianw | ok, i would probably appreciate a comment in there so we can unwind it when required, let me see if someone has a issue already | 22:15 |
| corvus | ianw: me too, i just left a review to that effect. also, maybe it should be a !=1.26.0 exclusion? | 22:15 |
| corvus | (under the assumption that the world is about to blow up for everyone and be fixed in a few hours) | 22:16 |
| ianw | doesn't seem there's an issue @ https://github.com/psf/requests/issues | 22:16 |
| corvus | avass: i like your doc change in principal and will look more closely at it after the preview job returns | 22:17 |
| corvus | principle even | 22:17 |
| ianw | https://github.com/psf/requests/pull/5651 | 22:21 |
| ianw | looks like they know about it | 22:21 |
| ianw | i guess this will just be a PITA for everyone for a bit :/ | 22:23 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Fix urllib3 dependency https://review.opendev.org/762210 | 22:25 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Fix urllib3 dependency https://review.opendev.org/762210 | 22:27 |
| *** gmann_lunch is now known as gmann | 22:27 | |
| corvus | ianw: i treated your contribution as an implied +2 and +3d it | 22:30 |
| ianw | cool ; i fear the problem will get wider quickly though | 22:33 |
| *** iurygregory has quit IRC | 22:47 | |
| *** iurygregory has joined #zuul | 22:49 | |
| *** ianychoi has quit IRC | 22:52 | |
| *** ianychoi has joined #zuul | 22:52 | |
| *** tosky has quit IRC | 22:55 | |
| openstackgerrit | Merged zuul/zuul-sphinx master: Switch to zuul-release-python https://review.opendev.org/682366 | 23:01 |
| openstackgerrit | Merged zuul/zuul-sphinx master: Ignore __pycache__ directory when looking at roles https://review.opendev.org/715764 | 23:01 |
| *** rlandy has quit IRC | 23:03 | |
| openstackgerrit | Merged zuul/nodepool master: Fix urllib3 dependency https://review.opendev.org/762210 | 23:57 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!