| clarkb | jkt: we have seen them leak in the past. Nodepool attempts to clear out leaked ports but its best effort iirc | 00:02 |
|---|---|---|
| openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: Fix incorrect variable name for promote-docker-image https://review.opendev.org/724442 | 00:05 |
| mnaser | clarkb: that _should_ work, i have to grab some food, so feel free to hijack the review if you need to make small revisions to get it to land | 00:05 |
| mnaser | ill beb ack in 30ish | 00:05 |
| clarkb | mnaser: that looks good to me based on your debuigging so far | 00:06 |
| *** Goneri has quit IRC | 00:14 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Fix incorrect variable name for promote-docker-image https://review.opendev.org/724442 | 00:15 |
| jkt | clarkb: here's my current status with one VM running and four ports being used: http://paste.openstack.org/show/792907/ | 00:18 |
| jkt | clarkb: I believe that port #3 (8b227f1c-8dda-4006-8a68-0bdc833639c0, ip_address='172.16.202.2') belongs to the router and that it's just some misleading output that it's shown as DOWN | 00:19 |
| clarkb | jkt: if you do a port show on the ports you'll get more info about its prupose | 00:20 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Add git name and email for quickstart executor https://review.opendev.org/724096 | 00:29 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: container functional test : allow to specify elements-dir https://review.opendev.org/724452 | 01:45 |
| *** swest has quit IRC | 01:59 | |
| *** swest has joined #zuul | 02:13 | |
| *** bhavikdbavishi has joined #zuul | 02:15 | |
| *** bhavikdbavishi has quit IRC | 02:21 | |
| *** cdearborn has quit IRC | 02:23 | |
| *** bhavikdbavishi has joined #zuul | 02:31 | |
| *** bhavikdbavishi has quit IRC | 02:44 | |
| *** bhavikdbavishi has joined #zuul | 02:44 | |
| *** bhavikdbavishi1 has joined #zuul | 02:51 | |
| *** bhavikdbavishi has quit IRC | 02:53 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 02:53 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Allow disabling build-log-retention https://review.opendev.org/723782 | 02:59 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: use local openstack-ci-core PPA key https://review.opendev.org/724462 | 03:23 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: use local openstack-ci-core PPA key https://review.opendev.org/724462 | 03:42 |
| openstackgerrit | Merged zuul/nodepool master: Switch to fedora-30 for the openshift integration job https://review.opendev.org/686737 | 03:46 |
| *** bhavikdbavishi has quit IRC | 04:23 | |
| *** evrardjp has quit IRC | 04:35 | |
| *** evrardjp has joined #zuul | 04:36 | |
| *** sgw has quit IRC | 04:36 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: use local openstack-ci-core PPA key https://review.opendev.org/724462 | 04:36 |
| *** bhavikdbavishi has joined #zuul | 04:40 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: use local openstack-ci-core PPA key https://review.opendev.org/724462 | 05:28 |
| *** reiterative has quit IRC | 05:41 | |
| *** reiterative has joined #zuul | 05:42 | |
| *** ysandeep|away is now known as ysandeep | 05:42 | |
| *** bhavikdbavishi has quit IRC | 06:01 | |
| *** bhavikdbavishi has joined #zuul | 06:02 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: use local openstack-ci-core PPA key https://review.opendev.org/724462 | 06:14 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: use local openstack-ci-core PPA key https://review.opendev.org/724462 | 06:42 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: use local openstack-ci-core PPA key https://review.opendev.org/724462 | 07:03 |
| *** bhavikdbavishi has quit IRC | 07:06 | |
| *** jcapitao has joined #zuul | 07:08 | |
| *** PrinzElvis has joined #zuul | 07:13 | |
| zbr | can we do something about https://review.opendev.org/#/c/703053/ ? | 07:18 |
| zbr | that workaround is more than 6mo old and I doubt Docker Inc will ever fix it, better to have working roles | 07:20 |
| *** tosky has joined #zuul | 07:28 | |
| *** rpittau|afk is now known as rpittau | 07:29 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: use local openstack-ci-core PPA key https://review.opendev.org/724462 | 07:32 |
| *** avass has joined #zuul | 07:43 | |
| *** bhavikdbavishi has joined #zuul | 07:56 | |
| *** jpena|off is now known as jpena | 07:57 | |
| *** PrinzElvis has quit IRC | 08:07 | |
| *** PrinzElvis has joined #zuul | 08:08 | |
| *** ysandeep is now known as ysandeep|lunch | 08:08 | |
| *** PrinzElvis has quit IRC | 08:11 | |
| *** PrinzElvis has joined #zuul | 08:12 | |
| *** yolanda has joined #zuul | 08:41 | |
| avass | does the "Depends-On" tag in the commit message make zuul always check out that other project, even though that project is not specified with "required-projects:"? Looks like one of our users did that. Seems dangerous because the next change would have failed, or used an outdated version of the required-project. | 08:44 |
| avass | I have a feeling that there should be a check like "if dependent_change.project not in (project, required_projects): error()" | 08:54 |
| *** masterpe has quit IRC | 08:56 | |
| *** ysandeep|lunch is now known as ysandeep | 09:04 | |
| *** masterpe has joined #zuul | 09:11 | |
| *** bhavikdbavishi has quit IRC | 09:13 | |
| *** bhavikdbavishi has joined #zuul | 09:17 | |
| AJaeger | avass: it only checks it out if there is required-projects. If there isn't, Zuul will just enforce the order | 09:20 |
| AJaeger | avass: adding such a check would break for example policy changes that we do also with depends-on. Here the depends-on just ensures the changes merge in order - but are tested indepentent | 09:21 |
| avass | AJaeger: are you sure? because the project was present on the machine and they were able to use the zuul.projects[canonical_name].src_dir | 09:24 |
| avass | AJaeger: I'll take another look at that change to see that I didn't miss something | 09:24 |
| avass | mnaser: Sorry! | 09:24 |
| avass | AJaeger: Yeah they're at least able to use the zuul.projects variable, and the tests look like they should't work if the repo wasn't present | 09:27 |
| avass | AJaeger: it is a static node though so the project could have left from another change | 09:28 |
| avass | AJaeger: but I see a reason why we want to be able to load a dependent change even though it's not a required project | 09:37 |
| avass | AJaeger: say we updated parent a parent job in project A, project B needs the updated parent to work so you add Depends-On: change-in-project-A, but I guess this also causes zuul to push project A to the remote and set the relevant zuul.projects variables | 09:39 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Only push required projects https://review.opendev.org/724626 | 09:45 |
| avass | AJaeger: I guess something like this would be the fix :) | 09:45 |
| avass | at least partially | 09:45 |
| AJaeger | avass: best discuss with corvus on what he thought | 09:48 |
| avass | yeah I want more input on that | 09:49 |
| openstackgerrit | Sorin Sbarnea zbr proposed zuul/zuul-jobs master: ensure-docker: workaround for centos-8 conflicts https://review.opendev.org/703053 | 10:04 |
| *** guillaumec has joined #zuul | 10:11 | |
| *** rpittau is now known as rpittau|bbl | 10:21 | |
| openstackgerrit | Sorin Sbarnea zbr proposed zuul/zuul master: WIP: Enable ANSI rendering on stdout/stderr https://review.opendev.org/716251 | 10:21 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Only push required projects https://review.opendev.org/724626 | 10:24 |
| *** wxy has quit IRC | 10:36 | |
| *** zxiiro has quit IRC | 10:42 | |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Do not add dependent changes project to zuul.projects https://review.opendev.org/724633 | 10:42 |
| avass | I think that^ fixes it | 10:46 |
| zbr | avass: AJaeger what do you think about the makefile helper at https://review.opendev.org/#/c/723837/ ? | 10:49 |
| *** bhavikdbavishi has quit IRC | 10:57 | |
| avass | zbr: I'll have to look later, have to make some food | 10:57 |
| zbr | avass: thanks, going to do the same, try it if, it gives a nice output w/o params. | 10:58 |
| *** jcapitao is now known as jcapitao_lunch | 11:09 | |
| *** ysandeep is now known as ysandeep|brb | 11:18 | |
| *** sugaar has quit IRC | 11:21 | |
| *** sugaar has joined #zuul | 11:26 | |
| *** bhavikdbavishi has joined #zuul | 11:31 | |
| AJaeger | zbr: I'm rarely building Zuul myself but if, it might be handy. I left a few comments | 11:31 |
| zbr | AJaeger: my goal was to help newbies, those that want to do just a simple hack. | 11:34 |
| *** jpena is now known as jpena|lunch | 11:36 | |
| AJaeger | thx | 11:36 |
| zbr | in fact i want to also write a contributing documentation bit because there is none aimed at this yet, where I would also mention the makefile. | 11:42 |
| zbr | there is nothing about contributing/developer on https://zuul-ci.org/docs/zuul/index.html homepage | 11:42 |
| zbr | is about user, admin, but no developer/hacker. | 11:43 |
| *** ysandeep|brb is now known as ysandeep | 11:54 | |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Do not add dependent changes project to zuul.projects https://review.opendev.org/724633 | 12:07 |
| *** hashar has joined #zuul | 12:10 | |
| *** jcapitao_lunch is now known as jcapitao | 12:18 | |
| *** rpittau|bbl is now known as rpittau | 12:22 | |
| *** rlandy has joined #zuul | 12:22 | |
| *** jpena|lunch is now known as jpena | 12:38 | |
| avass | is this supposed to be empty? https://zuul-ci.org/docs/zuul/tutorials/user.html | 12:43 |
| avass | ah, I guess it actually is | 12:45 |
| *** dpawlik has joined #zuul | 12:46 | |
| *** dpawlik has joined #zuul | 12:46 | |
| tristanC | avass: there is a change that adds content here: https://review.opendev.org/#/c/702487/ | 12:46 |
| avass | tristanC: nice, I'll take a look at it and throw some comments on it. would be nice to have something like that when we onboard new teams :) | 12:50 |
| tristanC | avass: feedback would be greatly appreciated, thanks! | 12:52 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Install skopeo in container images https://review.opendev.org/724344 | 13:05 |
| mordred | tristanC: great idea ^^ thanks - I think that makes it much simpler | 13:06 |
| *** jcapitao has quit IRC | 13:06 | |
| *** jcapitao has joined #zuul | 13:07 | |
| mordred | tristanC: also - I think that patch looks great! | 13:08 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-website master: Switch website to Gatsby https://review.opendev.org/717371 | 13:27 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-website master: Add blog to website https://review.opendev.org/724648 | 13:27 |
| mordred | mnaser, tristanC: I updated that to not be WIP and split the blog-post out of it | 13:27 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-website master: Switch website to Gatsby https://review.opendev.org/717371 | 13:30 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-website master: Add blog to website https://review.opendev.org/724648 | 13:33 |
| avass | mordred: oooh, a zuul blog | 13:35 |
| mordred | yeah - tristanC's patch to add the software-factory blog content to the docs reminded me that we had the gatsby and blog work outstanding | 13:36 |
| avass | I like the zuul-is-cool/index.md ;) | 13:36 |
| *** bhavikdbavishi has quit IRC | 13:46 | |
| *** ysandeep is now known as ysandeep|afk | 13:56 | |
| corvus | avass: pushing extra projects with depends-on was intentional; that's why there's a 'required' flag in the projects list | 13:56 |
| corvus | zbr: if you search for 'developer' on that index page you linked, you'll find the developer's guide. | 13:58 |
| zbr | corvus: i found it few hours ago, but that was only when using site search, browser search does not find any mention of "developer" of "contributing", two very common keywords. | 13:59 |
| avass | corvus: that seems dangerous | 14:00 |
| corvus | zbr: i don't know what to say, it's on my screen: https://imgur.com/a/bfNveiu | 14:01 |
| fungi | avass: can you expand on the exploit scenario you have in mind which makes it dangerous (i assume you mean "insecure)? | 14:02 |
| *** sgw has joined #zuul | 14:02 | |
| corvus | zbr: that used to be a top-level guide: user, admin, developer. but then documentation experts said that was wrong, so we rearranged it. maybe we were right to start with and we should switch it back. | 14:02 |
| avass | fungi, corvus: not insecure but for static nodes the required project could be left from an earlier change testing it with an our of sync repo, making the change succeed when it should have failed | 14:03 |
| avass | out of sync* | 14:03 |
| zbr | corvus: that is amazing, firefox search is broken. it works searching for other texts on the same page but "devel" does not find anything. | 14:03 |
| avass | and I we don't want to start deleting the workspaces since we have a repo that is ~5GB :( | 14:04 |
| corvus | avass: well, if you're re-using workspaces they could be left from a different job | 14:04 |
| fungi | avass: why would a job suddenly choose to use that repository? it's just some files pushed to disk | 14:04 |
| zbr | indeed, only some words are found by firefox search, looks like 50/50 chance | 14:04 |
| zbr | proved to be a PBKAC issue, somehow the whole words was enabled. | 14:05 |
| avass | corvus, fungi: wait.. no the following changes would probably fail since they wouldn't have the zuul.projects[] entry for that repo | 14:05 |
| avass | still, what's the reason we push non-required projects to the remote? | 14:06 |
| corvus | avass: basically because the user asked us to by setting depends-on :) | 14:06 |
| fungi | pushed to the node in case the job wants it for some reason | 14:07 |
| avass | corvus, fungi: just get the feeling that the depends-on and required-projects should be separate | 14:07 |
| corvus | avass: i think the idea was to support jobs that switched between using published software and development software based on whether there was a depends-on. of course, that's problematic for gating projects, but could be useful. | 14:07 |
| corvus | they are separate | 14:08 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: docs: move components to reference https://review.opendev.org/708686 | 14:08 |
| corvus | a project that appears in the 'zuul.projects' list that was not in 'required-projects' will not have the 'required' flag set | 14:08 |
| zbr | corvus: any reason why the Scheduler info is on the root of the developer guide? to me it seems as it should a sub page, like the others. | 14:08 |
| corvus | zbr: no reason | 14:08 |
| corvus | zbr: that really shouldn't even be documented there | 14:09 |
| corvus | no one is ever going to instantiate a Scheduler object | 14:09 |
| zbr | corvus: ok, because I would be happy to put some generic info on root, on how to run various tests. | 14:09 |
| tristanC | corvus: iirc we adapted the `four sections` documentation structure to fit the operator vs user vision for zuul documentation, but i think that needs more work to make it convenient. | 14:10 |
| corvus | zbr: like this? https://opendev.org/zuul/zuul/src/branch/master/TESTING.rst | 14:10 |
| avass | corvus: I can see why having a depends-on to load updated job configuration is wanted, but not why we should push a job-library to the remote | 14:10 |
| *** Goneri has joined #zuul | 14:10 | |
| avass | corvus, fungi: anyway the problem is that a new user added a "Depends-On:" footer since they wanted to test two projects together, they forgot to add the "required-projects" attribute so the following change would have failed. | 14:11 |
| corvus | tristanC: yeah. i think maybe we need to pull the developer guide up to the top level too, then have those sections under it. | 14:11 |
| avass | so the repository would have been in a sort of broken state, depends on how you see it I guess | 14:13 |
| corvus | avass: yeah, i agree that what you propose makes logical sense. it might be worth examining our jobs in opendev to see if we really make use of the dynamically-add-a-project feature. if it turns out not to be useful, perhaps we could consider changing to the approach you recommend. | 14:13 |
| zbr | corvus: yes, very similar to that. Why is this page not included under developer guide? in fact there is a Testing one there which is totally different. | 14:13 |
| corvus | avass: however, you may also want to make that job more robust, because even if zuul behaved the way you suggest, you could have repos on disk left over from other jobs or previous versions of the job. i might suggest adding something in a pre-playbook which deleted any repos which are not in zuul.projects with required=true. | 14:14 |
| corvus | zbr: yep, the developer guide needs work; none of that is by design and improvements are welcome | 14:15 |
| avass | coruvs: no I don't think the following jobs would have succeeded since the "zuul.projects['project-b']" entry wouldn't exist | 14:15 |
| avass | corvus: | 14:15 |
| avass | I have to leave for some hours, I'll be back later | 14:15 |
| zbr | corvus: sure, but i need to ask questions before proposing some improvements. | 14:15 |
| zbr | corvus: for example it would very useful to know if https://review.opendev.org/#/c/723837/ would be welcomed or not, as I can be used to simplify the documentation a lot. | 14:16 |
| corvus | avass: i agree the following jobs wouldn't have succeeded -- my suggestion is that if you delete repos that are on disk but not in zuul.projects with required=true, then you would have deleted project-b from the *first* build. in other words, you have the tools today to make sure that that first change doesn't pass just by adding a depends-on without required-projects. | 14:17 |
| zbr | instead of writing long and boring commands about how to install build tools,..., we could document `make coffee` commands. | 14:18 |
| corvus | zbr: i don't know if people are interested in a makefile, but i do know that even when we chose to make something simple and easy, we should still document what that thing is doing. we should not simply delete all of the knowledge that is in https://opendev.org/zuul/zuul/src/branch/master/TESTING.rst -- it belongs in the documentation because developers need to understand how things work, even if there's a | 14:20 |
| corvus | simpler way. | 14:20 |
| tristanC | zbr: i'd be in favor of documenting (and teaching) the correct commands, as this is more instructive than an abstract `make` command. | 14:20 |
| fungi | i guess the concern that makefile is attempting to address is that the web build relies on yarn instead of tox? | 14:20 |
| *** hashar_ has joined #zuul | 14:21 | |
| tristanC | though having a Makefile lgtm to me as it's usefull | 14:21 |
| *** hashar has quit IRC | 14:23 | |
| zbr | fungi: basically yes, but to describe it better is to have a single command/entry-point for most testing commands, one that is not biased towards tox, yarn,... or whatever may use in the future. | 14:27 |
| zbr | one good example is that linting is split between the tox and yarn now. | 14:27 |
| zbr | i do not have any particular love for make syntax, but I cannot beat the popularity of `make ...` among developers. | 14:28 |
| fungi | don't get me wrong, i rather like make, just want to understand better whether this is really reducing or actually increasing the maintenance burden | 14:29 |
| zbr | i used other tools (scons, waf, doit, cmake, even bash), but none is so ubiquitous | 14:29 |
| zbr | i doubt we would have to update it often, i really doubt. | 14:30 |
| zbr | i consider it as "part of documentation" | 14:30 |
| *** guillaumec has quit IRC | 14:40 | |
| mnaser | mordred, clarkb, avass: ok i just had a promote job fail again even after yesterday's fix | 14:54 |
| mnaser | https://zuul.opendev.org/t/vexxhost/build/e817c7336f264e79b97930e8c7cea2b2 | 14:54 |
| * mnaser digs some more.. | 14:54 | |
| mnaser | looks like zuul promote's are failing too so its not limited to my uaage | 14:56 |
| AJaeger | mnaser: could you review https://review.opendev.org/#/c/724132/ , please? Let's not merge until the issue above is fixed | 14:56 |
| AJaeger | avass: did you send the ensure- change mail to zuul-announce? | 14:57 |
| mnaser | AJaeger: i believe so :) | 14:58 |
| corvus | mnaser: yeah, that change and a zuul change failed on getting the dockerhub token | 14:59 |
| mnaser | corvus: yeah i fixed something yesterday thinking it was the next task that was silently failing -- https://review.opendev.org/#/c/724442/ | 14:59 |
| mnaser | but clearly it's not that.. | 15:00 |
| mnaser | it can clearly push to dockerhub with no problems though | 15:00 |
| mnaser | as the review_xxx tag showed up | 15:00 |
| corvus | yeah, though the process is slightly different for promoting | 15:00 |
| corvus | (the auth process) | 15:01 |
| mnaser | right yes, so rather tha relying on the local file and docker doing the auth, we get a token via api | 15:01 |
| corvus | mnaser: oh i think i see it | 15:01 |
| corvus | mnaser: see my comment on https://review.opendev.org/724442 | 15:01 |
| mnaser | ah darn, yes! | 15:02 |
| corvus | patch in progress | 15:02 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Fix another incorrect variable name for promote-docker-image https://review.opendev.org/724686 | 15:03 |
| corvus | AJaeger, mnaser: ^ | 15:03 |
| mordred | +A | 15:04 |
| mordred | corvus: so once that lands, re-enqueue the promote yeah? | 15:06 |
| openstackgerrit | Paul Albertella proposed zuul/zuul-jobs master: Add Bazel build and ensure roles https://review.opendev.org/693513 | 15:06 |
| mnaser | it sounds like zuul will do its own cleanup later, right? | 15:06 |
| mnaser | for the stale review_xx images | 15:06 |
| corvus | mordred: mnaser yes to both | 15:10 |
| avass | AJager, corvus: Yeah, I sent an updated mail, did you get it? | 15:11 |
| avass | corvus: Yeah, the problem with deleting repos is that we want to keep a continuous cache on the nodes since some of the repos are very large | 15:14 |
| avass | corvus: because of legacy issues | 15:14 |
| corvus | avass: right, but you want to delete the repo which was in the depends-on but not in required-projects. you can do that. | 15:15 |
| corvus | it's not ideal to push a repo to the remote node and then delete it, but it would at least keep that job from succeeding. :/ | 15:15 |
| AJaeger | avass: http://lists.zuul-ci.org/pipermail/zuul-announce/ does not show anything yet. corvus, did you approve it? | 15:16 |
| corvus | and protect against any other repo being left on the node between jobs | 15:16 |
| avass | corvus: but then we would have to check if there's any job running for that repo on that label | 15:16 |
| avass | corvus, AJaeger: I could send another one with update dates | 15:16 |
| corvus | AJaeger, avass: i approved it now | 15:16 |
| openstackgerrit | Merged zuul/zuul-jobs master: Fix another incorrect variable name for promote-docker-image https://review.opendev.org/724686 | 15:17 |
| avass | corvus: unless I'm not just understanding what you mean | 15:17 |
| AJaeger | avass: we're good since corvus approved - thanks | 15:17 |
| corvus | avass: i don't follow -- i'm proposing a simple and automated way to prevent not only the case you are concerned about, but also the danger of repos being left on the node from other jobs, or older versions of jobs. | 15:18 |
| corvus | avass: simply do this: "for each repo on disk, if that repo is in zuul.projects and required=true: continue; otherwise: delete" | 15:18 |
| corvus | avass: that will make the repos on disk correspond exactly to the required-projects of the running job | 15:19 |
| avass | corvus: for that specific job yes, but not for jobs running for the repo we just deleted | 15:19 |
| clarkb | corvus: avass could you use a /opt/git cache type setup like we use and hardlink that into /home/zuul/src. Then you can always delete /home/zuul/src then let git repo setup for the next job set up /home/zuul/src again from the cache | 15:20 |
| corvus | clarkb: that's a good idea | 15:20 |
| clarkb | it seems to work pretty well for opendev but all those nodes are single use. I expect it would map to multiuse nodes reasonably well though | 15:20 |
| avass | clarkb, corvus: yes but the 5GB repos that just needs to run a linter starts taking a very long time because it needs to clone the repo as well | 15:20 |
| clarkb | avass: thats basically free as long as it is on the same fs | 15:21 |
| corvus | avass: i'm really confused :( | 15:21 |
| clarkb | due to hardlinking | 15:21 |
| corvus | avass: i don't understand " for that specific job yes, but not for jobs running for the repo we just deleted" | 15:21 |
| avass | uuh, I missed the link | 15:21 |
| avass | clarkb: that sounds like a good idea actually | 15:22 |
| corvus | i thought the issue is that you want, for each job running on the node, for the checked out repos to be exactly the repos in required-projects | 15:22 |
| mordred | avass, clarkb: for your setup where you have a known set of static nodes - you could also set up /opt/git to be a location that gerrit replicates to - then the zuul role that will clone/hard-link from /opt/git as part of pushing the repos can do it efficiently, and you cn also keep the /opt/git cache up to date as you land patches so that you minimize the amount of data being rsynced from the zuul executor | 15:24 |
| avass | corvus: the problem does not happen at the moment since we only test against project in zuul.projects at the moment | 15:24 |
| corvus | avass, clarkb: regarding the cache idea, i think if you put them in /opt/git and that's on the same filesystem as /home/zuul (or whatever user), and you use the git based prepare-workspace roles, it should all be automatic | 15:25 |
| avass | corvus: it is but I believe it actually clones it | 15:25 |
| corvus | avass: can you dereference some of those "it"s? :) | 15:26 |
| avass | corvus: but testing together with another project using depends-on without required-projects allows the user to merge the repo in a broken state | 15:26 |
| avass | corvus: it's automatic :) | 15:27 |
| avass | corvus: let me check | 15:27 |
| corvus | i am completely lost | 15:27 |
| corvus | i'm pretty sure i understand what you want | 15:27 |
| openstackgerrit | Paul Albertella proposed zuul/zuul-jobs master: Add Bazel build and ensure roles https://review.opendev.org/693513 | 15:27 |
| corvus | avass: i have to run to a meeting now, maybe we can continue later | 15:27 |
| avass | corvus: I'll try to come up with a scenario for it :) | 15:28 |
| corvus | avass: just to double check, you know about the "required" attribute i mentioned, right? https://zuul-ci.org/docs/zuul/reference/jobs.html#var-zuul.projects.required | 15:28 |
| avass | corvus: yes | 15:28 |
| corvus | ok. let's chat in a bit | 15:29 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Install skopeo in container images https://review.opendev.org/724344 | 15:29 |
| avass | clarkb, corvus: prepare-workspace-git clones the cache to the workspace: https://review.opendev.org/gitweb?p=zuul/zuul-jobs.git;a=blob;f=roles/prepare-workspace-git/tasks/main.yaml;h=9c5de2146ddc28d87a57b818fb75bd5a5719ee9c;hb=refs/heads/master | 15:30 |
| *** bhavikdbavishi has joined #zuul | 15:31 | |
| avass | clarkb: using a hard link would require us to set up multiple caches since we have multiple users to be able to run multiple jobs on the nodes | 15:31 |
| avass | clarkb: but could work | 15:31 |
| fungi | hardlinking is done at an object level under the hood of git itself | 15:32 |
| fungi | you don't explicitly hardlink anything | 15:32 |
| fungi | you just clone from the file path using git and it sets up hardlinks for any objects it needs | 15:32 |
| fungi | so it's basically copy-on-write git clones | 15:32 |
| avass | fungi: ok, I didn't know that :) | 15:32 |
| fungi | and effectively instantaneous since there is no data getting copied | 15:33 |
| avass | fungi: I bet that's specific for linux though | 15:33 |
| fungi | and also doesn't use additional disk space for any of the objects which are duplicated across the clones | 15:33 |
| avass | fungi: we have another (horrible) set-up for some windows jobs | 15:33 |
| fungi | it might not do that on windows, but i thought the newer windows filesystems had hardlinks | 15:34 |
| avass | not if you need to test on windows 7 :) | 15:34 |
| fungi | oh, indeed | 15:34 |
| fungi | https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createhardlinka | 15:35 |
| fungi | not sure if ntfs on win7 supports that | 15:35 |
| fungi | nor if windows builds of git make use of it | 15:35 |
| fungi | so would probably need to be tested | 15:35 |
| avass | yeah we already need to create new workspaces for the same user on our windows machines because of software that doesn't like to be automated or licensing issues | 15:36 |
| avass | and the cloning for that takes a while | 15:37 |
| fungi | this reminds me of why i sought different work after years of dealing with windows professionally ;) | 15:40 |
| fungi | i don't envy you | 15:40 |
| avass | I can see why ;) we are moving towards linux very fast though, it's just that we need to support software that's 10 years or older | 15:41 |
| fungi | yeah, there was a point at which i realized that taking all my microsoft software experience off my resume was the best choice i ever made | 15:46 |
| * mordred tries not to mention his experience as a C# developer very often | 15:51 | |
| reiterative | avass I think I've addressed your review comments on https://review.opendev.org/#/c/693513/ now - get_url choked on the checksum for the bazel installer at first, because it has the filename appended :-) | 15:53 |
| avass | reiterative: cool, I'll take a look in a moment! | 15:53 |
| avass | fungin, mordred: Even though I'm stuck with supporting windows sometimes, I'm quite happy with this being my first job ;) | 15:56 |
| avass | but I'd love to work on opensource full time | 15:56 |
| *** jcapitao has quit IRC | 16:03 | |
| *** jcapitao has joined #zuul | 16:03 | |
| fungi | oh, this is your first job? congrats! | 16:03 |
| fungi | and yeah, my first full-time job was supporting a mix of windows nt 3.5 and sco "open"server 5... i can totally relate | 16:04 |
| fungi | it was a good stepping-stone though, and i did manage to convince them to start using some linux before i left for greener pastures | 16:05 |
| avass | does this make sense? https://etherpad.opendev.org/p/depends-on_ | 16:07 |
| *** rpittau is now known as rpittau|afk | 16:07 | |
| corvus | avass: okay, i'm back; taking a look at the etherpad now | 16:08 |
| avass | corvus: ah good timing :) | 16:08 |
| *** zxiiro has joined #zuul | 16:09 | |
| *** hashar_ has quit IRC | 16:10 | |
| corvus | avass: yes, i think that makes sense. now, aside from the inefficiency of cloning, is there anything in the section below line 27 that is undesirable? | 16:11 |
| avass | corvus: not really | 16:11 |
| avass | at least not that I can think of | 16:11 |
| corvus | avass: yeah, me neither. so i think that could be a solution. | 16:14 |
| corvus | avass: clarkb's solution is also good; though that looks more like the "Currently" section, except that example.org/A won't be in a problen state for change a2, it would not be present. but change a1 would still succeed unless you did something to remove the non-required repo. | 16:15 |
| corvus | avass: you can combine the two, and remove non-required projects and still clone from the cached repos so you get hard-links whenever cloning needs to happen | 16:15 |
| avass | corvus: I mean, combining them would be best | 16:15 |
| avass | corvus: but then we'd have to update the cache all the time wouldn't we? | 16:16 |
| corvus | avass: yeah, you'd want something to periodically update it. could be continuous by using gerrit replication, or could be an hourly/daily cron job | 16:16 |
| corvus | avass: the cache doesn't have to be current, only recent. zuul will make sure the final clone in /home/zuul/src is current. | 16:17 |
| avass | corvus: yeah | 16:17 |
| avass | corvus: Still, being able to use Depends-On to test one change against another project seems wrong. | 16:18 |
| corvus | avass: a fourth option is that we could add a job attribute to tell zuul not to include any non-required repos. then change a1 will not pass because B isn't in zuul.projects, and then if you also did clarkb's solution of deleting /home/zuul/src then you would avoid having extra repos leak from other jobs or if you removed a repo from required-projects. | 16:18 |
| corvus | avass: testing one change against another project is exactly what it's supposed to be used for :) | 16:19 |
| tristanC | corvus: would it makes sense to have a 'use-intermediate-registry' role so that a zuul-registry can be used without buildset registry? | 16:22 |
| avass | corvus: I feel like it's good if you don't want to wait for another change to merge, or to make sure your change don't merge before the other one. but not for testing against a separate repo with just one change because of what I described in that etherpad | 16:22 |
| tristanC | otherwise it seems like using the 'use-buildset-registry' role would work, but the name is a bit missleading... | 16:22 |
| tristanC | and bonus question, the use-buildset-registry is restarting docker without when condition, and because we don't have docker, this is going to cause quite a bit of noise... would it be possible to not restart docker when docker is not installed? | 16:24 |
| avass | corvus: because the next change would also need a depends-on, or add the other repository to required-projects to succeed. | 16:25 |
| avass | corvus: or if you add a check to see if zuul.projects['example.org/B'] is defined to do this other thing. but only sometimes when we add a Depends-On footer to a commit. | 16:27 |
| avass | corvus: I just don't see a usecase for it, but I can see it causing errors | 16:28 |
| tristanC | iiuc, to use an intermediate or buildset registry with rootless podman, we only needs the modify_registries_conf and user-config tasks... so perhaps we could add a `configure-podman-registry` role which takes the buildset or intermediary secret as a variable? | 16:29 |
| *** evrardjp has quit IRC | 16:35 | |
| *** evrardjp has joined #zuul | 16:36 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: DNM Run builder tests on expanded node https://review.opendev.org/724079 | 16:37 |
| corvus | tristanC: the intermediate registry can't be used directly because the images in it have names based on the build uuid, not the "real" names; the pull-from-intermediate-registry role copies the images from intermediate to buildset and renames them to their correct names. | 16:38 |
| tristanC | corvus: hmm, can't the intermediate registry be used with real name? | 16:40 |
| corvus | avass: yeah, it's possible there are no uses of that feature and we could restrict it to required-projects only. i think it's worth asking that question on zuul-discuss to see if we can find users of it. the tox siblings role only looks at 'required' projects, so it is compatible with the behavior you want. | 16:41 |
| corvus | tristanC: no, it's job is to hold all of the random check pipeline builds, so there's no "zuul:latest" it's all "zuul:123abcdef" | 16:42 |
| tristanC | corvus: our use-case is for a periodic pipeline to manage container update (e.g. one job build and push the image to the intermediate registry, then other jobs test the update, and a promote job push the image to quay.io) | 16:43 |
| corvus | (but, from the perspective of the job that built it, zuul:1234abcdef was intended to be zuul:latest at the time. so when pull-from-intermediate-registry runs, it copies zuul:123abcdef into the buildset registry as zuul:latest") | 16:43 |
| tristanC | using the buildset.uuid as an intermediate ref would work | 16:44 |
| mordred | tristanC, corvus, clarkb: https://review.opendev.org/#/c/724344/ is green | 16:45 |
| corvus | tristanC: that should work; or you could build it around gating and have check/gate/promote | 16:45 |
| mordred | tristanC: (and thanks again for the idea about just copying the executable - I think that wound up being much nicer) | 16:47 |
| tristanC | mordred: nice, that's good to hear :) | 16:47 |
| tristanC | mordred: but it seems like this is not tested, and that could break if the skopeo requires an extra lib in the future... should we add a simple 'run --rm zuul/zuul-executor skopeo --version' test to the container build job just to be sure? | 16:49 |
| avass | reiterative: re https://review.opendev.org/#/c/693513/20 just one small thing, I think it would be good to use the 'mode' parameter on get_url instead of 'chmod +x' for each distro/os | 16:49 |
| mordred | tristanC: I agree - it would be good to add a test - although I thnik they'd have to rewrite skopeo to be in something other than go for it to grow a library dependency | 16:50 |
| clarkb | mordred: left a question on that change | 16:50 |
| avass | mordred, tristanC: yeah go doesn't have runtime dependencies right? only thing that could error is if we change from glibc to musl | 16:51 |
| tristanC | mordred: not necessarly, if docker.io/opendevorg/python-builder doesn't provides the good SONAME used by /kubic:/libcontainers:/stable/Debian_10/ , then running skopeo won't work | 16:51 |
| avass | I believe | 16:52 |
| mordred | clarkb: yeah - we could drop that - it's leftover from a previous version whee I was copying to constructed keyring | 16:52 |
| tristanC | avass: mordred: yes, libc, or any of the libs currently linked, like libgpgme.so.11, libdevmapper.so.1.02 or libgcc_s.so.1 | 16:53 |
| mordred | tristanC: http://paste.openstack.org/show/792963/ that's the ldd output from skopeo | 16:53 |
| mordred | I mean - python-builder and python-base would have to be out of sync for that issue to arise - and they'd have to be out of sync with each on a different major version of the debian base image | 16:54 |
| tristanC | mordred: what version is this? mine has quite a few more from fedora | 16:54 |
| clarkb | mordred: except for maybe libgpg since you install gpg ? | 16:54 |
| mordred | no - but those are all installed from the distro - and we install skopeo from the distro too | 16:55 |
| mordred | and it's the same base distro | 16:55 |
| mordred | so the source of the so libs is the same in both images | 16:55 |
| clarkb | mordred: ya but we only install the libs in the builder | 16:55 |
| mordred | we don't install libs | 16:55 |
| clarkb | mordred: we do gpg? | 16:55 |
| mordred | oh wow | 16:56 |
| mordred | actually - wtaf? | 16:56 |
| mordred | ok - I agree - this is problematic | 16:56 |
| tristanC | mordred: iiuc, even golang base can pulls extra ELF requirements for shared c libraries | 16:56 |
| clarkb | everything but gpg looks fine | 16:56 |
| clarkb | but gpg stands out as a weird exception there | 16:56 |
| mordred | yeah. I'm going to back to installing the package in the final image from apt | 16:56 |
| mordred | if it's pulling in lib depends, that's not acceptable from a "just copy the binary" pov | 16:56 |
| zbr | mordred: clarkb: sorry to nag again about the pre-wrapping change https://review.opendev.org/#/c/723603/ but i am getting dizzy scrolling on dependencies | 16:57 |
| mordred | (and it is) | 16:57 |
| mordred | tristanC: that's skopeo installed from kubic on a focal test node | 16:57 |
| mordred | (that produced that ldd) | 16:57 |
| mordred | but - the libgpgme - which the apt install DID install as a dependency - I think blows the straight-copy approach | 16:58 |
| clarkb | zbr: that applies to live streams of the consoles and the log rendering when jobs are complete? you don't happen to have a build I can use to test it in the built dashboard? | 16:58 |
| tristanC | mordred: i haven't seen issue with copying go binaries, as long as they are built using the same toolchain as the dest environment | 16:58 |
| tristanC | and simply checking execve works is enough to tell if the correct soname are present | 16:59 |
| clarkb | nevermind I think I found one for the live streamed console | 16:59 |
| clarkb | (pick a tempest job :) ) | 16:59 |
| avass | corvus: wanna +3 this ? https://review.opendev.org/#/c/723640/ | 16:59 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Install skopeo in container images https://review.opendev.org/724344 | 16:59 |
| zbr | clarkb: i addresses all instances of using <pre>, so both summary view and the console. i doubt it has anything to affect streaming because streaming does not use pre. | 17:00 |
| clarkb | oh streaming was already wrapping then | 17:00 |
| clarkb | I guess I should find a different example | 17:00 |
| mordred | tristanC: yeah - me either - except that in this case there sure does seem to be an ldd depend on libgpgme.so.11 - and libgpgme is installed by apt and is not in the target image | 17:00 |
| zbr | streaming already supports ANSI and if I remember well uses spans what do wrap correctly. | 17:00 |
| mordred | tristanC: I'm very surprised by that | 17:00 |
| zbr | and tbw, I got the ANSI working for console too, but I need to. merge the react app update first. | 17:01 |
| zbr | i will show some examples tomorrow. | 17:01 |
| tristanC | mordred: left a comment about having Debian_10 in the repo location | 17:02 |
| zbr | the real question is if we do have a specific place where we do not want to wrap content on <pre>, i personally doubt. | 17:02 |
| clarkb | zbr: I think there is a bug. My comment on the change links to a job log and the line numbers are wrapping too | 17:03 |
| clarkb | zbr: so instead of line 155 its line 1\n5\n\n5 | 17:03 |
| corvus | clarkb, zbr: i don't think we should implement wrapping, and now that i see that we have apparently already merged a change to do that in one place, i think we should revert it for further discussion. | 17:05 |
| mordred | tristanC: root@a8ef1daabbe9:/usr/src# ./skopeo --version | 17:05 |
| mordred | ./skopeo: error while loading shared libraries: libgpgme.so.11: cannot open shared object file: No such file or directory | 17:05 |
| mordred | tristanC: that's what happens when I install skopeo from apt, copy the binary, then remove skopeo from apt and run autoremove (to get rid of the deps) | 17:05 |
| tristanC | mordred: well yes, you would have to add libgpgme to the bindep | 17:05 |
| mordred | so somehow they really have made a go binary that is doing dynamic linking to system libs | 17:06 |
| openstackgerrit | Paul Albertella proposed zuul/zuul-jobs master: Add Bazel build and ensure roles https://review.opendev.org/693513 | 17:06 |
| corvus | mordred, tobiash, zbr: (since i -1'd https://review.opendev.org/681532 i would have liked a chance to weigh in before we merged it :) | 17:06 |
| mordred | yeah - and that's just starting to get weird | 17:06 |
| corvus | zbr: is there any other way to avoid the trackpad accidentally scrolls issue without using pre? | 17:06 |
| zbr | is not only a tracpad issue, is an UX issue | 17:07 |
| zbr | what if your terminal would do the same, forcing you to scroll to long lines? | 17:07 |
| corvus | zbr: i agree about that; in my comment on your latest change i explained why i think the horizontal scrolling ux is better | 17:07 |
| corvus | zbr: journalctl does that and i love it | 17:07 |
| corvus | zbr: when i don't use journalctl, i go to great lengths to avoid wrapping | 17:08 |
| zbr | ... one of the few people loving journalctl :D | 17:08 |
| corvus | because when i look at long debug lines, i need them to be on one line so i can scan them | 17:08 |
| tristanC | corvus: `| less -S` is useful to get that horizontal scroll | 17:08 |
| zbr | ok, in that case I will do a survery, i am curious what openstack developers prefer. | 17:08 |
| corvus | tristanC: yes that! thanks! :) | 17:09 |
| zbr | defaults should be about bringing best experience overall, if someone has a unique preference, they can use custom-css injenction, something I was using for many months. | 17:10 |
| corvus | and yes, the fact that i praised journalctl for doing that is significant :) | 17:10 |
| clarkb | I personally prefer the wrapping. When you get really long lines the browser doesn't do horizontal scrolling well and you end up either scrolling slowly or losing your place | 17:10 |
| tobiash | corvus: oops sorry do you want a revert? | 17:10 |
| clarkb | granted thats mostly a browser issue but our logs often expose it | 17:10 |
| corvus | and to be clear, i'm not saying that the current implementation is perfect; parts of it annoy me too | 17:11 |
| corvus | so i'm not advocating horizontal scroll in a small window (i actually prefer horizontall scroll the whole page) | 17:11 |
| corvus | but i do think wrapping log lines is worse ux than horizontal scroll | 17:12 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: DNM Run builder tests on multi-numa-ubuntu-bionic node https://review.opendev.org/724079 | 17:13 |
| zbr | if zuul would have had user preferences I would have attempted to make it configurable. i am really curious what users will prefer when being allowed to compare the two. | 17:15 |
| corvus | zbr, clarkb: https://zuul.opendev.org/t/zuul/build/baf99edfb2b94e7999b713c7722d4d9f/log/container_logs/executor.log | 17:15 |
| zbr | my guess is 80/20 in preference to enable wrapping. | 17:15 |
| mordred | I wish there was a good way to support both based on user preference - or even toggleable depending on content - there are times when lack of wrapping is important to me - and times when lack of wrapping makes life super hard (I see that on openstacksdk tests where the log line is some absurdly long rest json blob) - and I don't know that I believe there is a "right" answer that applies to all of the output | 17:15 |
| corvus | i think that's the ideal -- full window horizontal scrolling | 17:15 |
| mordred | forms | 17:15 |
| corvus | mordred: i'm curious when wrapping is desirable? | 17:16 |
| *** guillaumec has joined #zuul | 17:16 | |
| corvus | zbr is advocating it based on the trackpoint doing two dimensional scrolling | 17:16 |
| *** bhavikdbavishi has quit IRC | 17:17 | |
| clarkb | corvus: when you want to see all the info at once without scrolling | 17:17 |
| tristanC | corvus: when the lines are not too long, for example when they can fit in two screen width, it can be valuable to see the whole content at once | 17:17 |
| mordred | corvus: https://zuul.opendev.org/t/openstack/build/61c9ee9337a64e7ba5d65616f05a9fec/log/job-output.txt#45527 | 17:18 |
| *** bhavikdbavishi has joined #zuul | 17:18 | |
| mordred | corvus: that would be much easier to deal with if it was wrapped | 17:18 |
| mordred | looking for another better example though | 17:18 |
| corvus | mordred: if you were looking for find, say, what version of dogpile.cache was installed, but if you wanted to see "ansible finish; ansible installed; ansible start" by scanning down vertically, it would be very hard | 17:19 |
| fungi | most of the times that wrapping is desirable for me is when i don't want to have to actually interact with the window... so if it all fits in my (usually fairly large) browser window without needing either horizontal or vertical scrolling, that's great. as soon as i need to scroll vertically though, i might as well scroll horizontally too. the only time horizontal scrolling itself becomes challenging is when | 17:20 |
| fungi | you have to vertically scroll to find the horizontal scrollbar, then horizontally scroll blind to roughly the columns you think you need to be at, then veritcally scroll again to get back to the lines you were looking at | 17:20 |
| corvus | fungi: yeah, that's one of the reasons to consider full-screen horizontal scrolling distinct from tiny-window horizontal scrolling | 17:21 |
| mordred | corvus: right - that's why I'm saying I think that both scrolling and not-scrolling can be valuable tools depending on circumstances | 17:22 |
| fungi | if browsers did a better job of scrolling using keypresses, it wouldn't be such a bother, but lots of sites break keyboard interaction | 17:22 |
| corvus | mordred: agreed | 17:22 |
| corvus | mordred: but possibly even from one line to the next :) | 17:22 |
| mordred | yup | 17:22 |
| mordred | I kind of think a toggle could be super useful | 17:22 |
| mordred | that would wrap or not-wrap the container on the fly | 17:23 |
| corvus | i could dig that | 17:23 |
| tristanC | it seems like most cli output are better viewed with word wrap, but application logs are better viewed with horizontal scroll | 17:23 |
| fungi | maintaining a cursor anchor of sorts so your content doesn't jump around when you toggle that option could be tough, but maybe not critical to support | 17:24 |
| *** jpena is now known as jpena|off | 17:24 | |
| tristanC | fungi: there is usually a `scrollIntoView` javascript function that can be used for that | 17:24 |
| corvus | tristanC: that may be a good generalization -- probably true 90% of the time? :) | 17:24 |
| mordred | tristanC: I've got logs from openstacksdk that sometimes are *absurdly* long and a single line - and make scanning in a non-wrapped way almost impossible to do - I can't find an example right now, but they usually happen when I'm madly debugging something | 17:24 |
| tristanC | mordred: isn't that an issue with the sdk that shouldn't emit such line? | 17:25 |
| fungi | debug line containing a massive http request parameter blob, maybe | 17:26 |
| mordred | tristanC: no - in this case it's the line that the sdk should emit - it's an http request debug entry and the catalog returned is ... long | 17:26 |
| fungi | good guess ;) | 17:26 |
| mordred | splitting it across multiople lines (which is what keystoneauth does in some cases) is actually quite noisy and makes the logs hard to read in the general case | 17:26 |
| mordred | so it really is circumstance-specific as to whether you want the wrapping | 17:27 |
| mordred | having it all on one logical line (since. it's a logical thought) allows you to wrap it if you need to - but if it's pre-split it's harder to combine if it the split version is noise | 17:27 |
| corvus | mordred, zbr, tobiash: let's avoid reverting the change that has merged already (wrapping in the task output) and treat it as an experiment. but i'm opposed to forced wrapping in the log viewer; in fact, i spent about a day working to make the output look exactly like it does now, so that's no accident. if we want to undo that work, we're going to need a good reason. i think i'll need a user toggle to get | 17:27 |
| corvus | behind wrapping there. | 17:27 |
| corvus | (by the log viewer, i mean what happens when you click on a log under the "logs" tab; the existing change already merged applies to the popup in the 'console' tab) | 17:28 |
| fungi | fwiw, i felt like doing it on the results page snippet was a good compromise (precisely because all the page content usually fits in my, admittedly largeish, browser window) | 17:29 |
| zbr | ok, and meanwhile we can fix reported bugs, and investigate more the issue. | 17:29 |
| corvus | zbr: yeah, you'd need to solve the line number problem even with a user toggle | 17:29 |
| corvus | zbr: and you could probably save the user toggle as a cookie so you don't have to keep switching back and forth. the status page saves the filter as a cookie. | 17:30 |
| zbr | i need to see it... damn, how much I hate that we cannot share the site-preview links. | 17:30 |
| corvus | zbr: take the build uuid that clarkb pasted in his comment and navigate to that build, then navigate to the log | 17:30 |
| zbr | i seen that someone was adding a feature controlable with a cookie but i never seen the option to toggle it. | 17:30 |
| zbr | doing it now... | 17:30 |
| mordred | zbr: we have a toggle that interacts with a cookie on the status page - so you can look at that for inspiration | 17:31 |
| mordred | oh - nevermind- it's just the search bar that is cookie - not the expand-by-default toggle | 17:31 |
| zbr | we will have to create some kind of profile / settings page where to put settings | 17:32 |
| mordred | but - the search bar does | 17:32 |
| zbr | as probably the list of options will grow, and hiding them on various pages is not ideal | 17:32 |
| corvus | zbr: i think because of the use case mordred points out (wating to change wrapping on the fly), i would just put it at the top of any page where it could apply, and have it persist and apply to all of them | 17:33 |
| fungi | well, unless the "wrap" toggle is sticky via a cookie and persists to your next view | 17:33 |
| corvus | i don't think a user settings page is necessary | 17:33 |
| avass | corvus: I wonder how EU would look at those cookies, they're not really "strictly necessary" for the site to work ;) | 17:34 |
| corvus | zbr: you could just check it and never touch it again. :) mordred can check it and uncheck it as he sees fit. and i can just (almost) never check it. :) | 17:34 |
| zbr | i am thinking at the notification icon the rightop as in deal place for such thing. | 17:34 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: use-buildset-registry: add Fedora certifacts vars https://review.opendev.org/724717 | 17:34 |
| avass | but I can't say I'm an expert at cookie laws | 17:34 |
| zbr | but if the settings is sticky, it would acceptable to spam each page with the option. | 17:34 |
| zbr | cookies are not fobidden, tracking cookies are. | 17:35 |
| corvus | zbr: i'd put it near the refresh/auto reload. ie, the page header, not the site header. (because it shouldn't appear on pages like the builds table) | 17:35 |
| avass | zbr: they're not forbidden but you have to inform the user that your site is using cookies if the cookies are not stricly necessary for the site to work | 17:36 |
| zbr | and here is how i ruin my plan to avoid javascript world till i retire | 17:36 |
| *** dpawlik has quit IRC | 17:38 | |
| fungi | avass: from what i''m reading, exemptions to the consent requirement include "Technical cookies strictly necessary for the provision of the service. These include preference cookies, session cookies, load balancing, etc." | 17:43 |
| fungi | "...preference cookies..." | 17:43 |
| fungi | so the definition of "strictly necessary" seems like it includes optional site features | 17:44 |
| fungi | everything about consent requirements appear to be specific to third-party tracking cookies, non-anonymized statistical behavior tracking, et cetera | 17:45 |
| avass | fungi: huh, good to know | 17:46 |
| fungi | i am not a lawyer though, so this is not legal advice ;) | 17:46 |
| avass | I guess 'strictly necessary' is different if you're a lawmaker or a developer ;) | 17:46 |
| fungi | courts rarely interpret words the same way computer programmers do, and so this sort of misunderstanding is a common point of debate in legal discussion lists about computer software | 17:47 |
| fungi | the law is (almost) never about absolutes and literal interpretations | 17:48 |
| zbr | lets close the cookie subject, we already have the _ga cookie on https://zuul.opendev.org | 17:49 |
| zbr | it is set on the bare domain, applies to any site | 17:50 |
| fungi | *we* don't set that cookie on https://zuul.opendev.org/ | 17:50 |
| *** jcapitao has quit IRC | 17:52 | |
| fungi | i'm checking now to see if i have one, but i don't think i do | 17:52 |
| fungi | zbr: are you maybe thinking of openstack.org? | 17:53 |
| mordred | corvus: I'm still getting a puppet error on lists in the gate: https://75ab7718a1ac933a54f7-6414947d36de3e18491bc440889da30b.ssl.cf2.rackcdn.com/724682/6/check/system-config-run-lists/6f39839/bridge.openstack.org/ara-report/result/513b6b1c-6952-4838-8f60-64288f83627a/ | 18:19 |
| mordred | corvus: sorry | 18:20 |
| mordred | clarkb: ^^ | 18:20 |
| mordred | clarkb: and it's because you fixed listpassword in a different change: https://review.opendev.org/#/c/724356/3 | 18:20 |
| mordred | fungi: ^^ would you mind reviewing that? | 18:20 |
| mordred | GAH | 18:20 |
| mordred | sorry wrong channel | 18:20 |
| zbr | anyone brave to talk about create-react-app update via https://review.opendev.org/#/c/716305/ ? | 18:24 |
| mordred | zbr: it still has a bug that needs to be solved | 18:25 |
| mordred | zbr: if zuul has a config error, it is broken | 18:25 |
| avass | mordred: maybe -1 workflow that so no-one merges it | 18:25 |
| zbr | where is this bug? | 18:26 |
| avass | mordred: with some information, since it looks very ready otherwise | 18:26 |
| zbr | ahh, right, i do not see the alert button when looking at openstack tenant | 18:27 |
| mordred | avass, zbr : I just left a -2 and a description of how to currently see the issue | 18:27 |
| mordred | the local tenant on SF - which is the multi-tenant test job - has config errors | 18:28 |
| mordred | and shows the dashboard break | 18:28 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Validate ansible extra packages https://review.opendev.org/724110 | 18:28 |
| mordred | I'm currently stumped - I have no idea what's wrong | 18:28 |
| mordred | but I think other than that it's ready to go :) | 18:28 |
| mordred | corvus, clarkb, tristanC, tobiash: https://review.opendev.org/#/c/724079/ is green - it ran on an IPv6-only cloud. I think that means that https://review.opendev.org/#/c/722339/ is ready to go | 18:30 |
| mordred | avass: you might also find that interesting - or you might not :) | 18:30 |
| avass | mordred: I've pretty much never touched react, but I do like challenges ;) | 18:31 |
| mordred | avass: :) | 18:31 |
| mordred | corvus, tristanC: also - https://review.opendev.org/#/c/724344/ - skopeo patch works now - and should avoid the soname issue tristanC pointed out earlier (although - thanks - I *never* would have imagined that) | 18:31 |
| avass | oh, unless you meant the multiarch image builds | 18:32 |
| avass | I'll take a look at that :) | 18:32 |
| avass | mordred: the complexity of the build docker image task feels like it should be a custom action plugin | 18:36 |
| avass | or maybe it's just me that don't like reading that much jinja | 18:36 |
| tristanC | mordred: both lgtm | 18:41 |
| tristanC | may i request some zuul-base-jobs and zuul-jobs review to enable kubectl based connection usage: https://review.opendev.org/680712 and https://review.opendev.org/716298 | 18:43 |
| avass | tristanC: I'll take a look in a minute | 18:45 |
| mordred | avass: so much jinja :) | 18:54 |
| *** rlandy is now known as rlandy|biab | 19:00 | |
| mordred | tristanC: replied to your question on https://review.opendev.org/#/c/722339/ | 19:09 |
| mordred | tristanC: also - as best I can tell, buildah does not yet fully support doing this, although I believe support is in work | 19:10 |
| *** bhavikdbavishi has quit IRC | 19:11 | |
| mordred | tristanC: (if it had, I would have been very tempted to just write this to use buildah in the first place and say "if you want multi-arch images this is the story" - because unlike normal docker build - where I can see users being upset if they say "please build this docker container" and then we use not-docker and it's different- I thnk there can not be very many pre-existing expectations for how multi-arch | 19:12 |
| mordred | images get built :) ) | 19:12 |
| mordred | tristanC: which si all to say - if you learn how to do the same thing with buildah, please let me know :) | 19:12 |
| * mordred isn't crazy about how complicated that buildx approach is - nor the duplication of registry config information due to the registry interactions happening inside of the builder images using buildkitd | 19:19 | |
| *** bhavikdbavishi has joined #zuul | 19:19 | |
| masterpe | Is there a howto how you install zuul on a ubuntu system? | 19:23 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Validate ansible extra packages https://review.opendev.org/724110 | 19:24 |
| avass | masterpe: there should be | 19:24 |
| clarkb | quickstart should run on ubuntu, I believe we test it on ubuntu and the more in depth instructions cover a number of distros I think | 19:25 |
| *** bhavikdbavishi has quit IRC | 19:25 | |
| masterpe | till now I only could find the quick-start | 19:25 |
| masterpe | with docker | 19:25 |
| mordred | corvus: do you want to review the multi-arch patch? it's got 2x+2 - but I'm not sure if you want to review before it lands | 19:26 |
| clarkb | I guess install from.scratch is leap, fedora and centos | 19:26 |
| masterpe | But that is with via docker, is there something without docker? | 19:26 |
| avass | masterpe, clakb: I was very sure I saw one once | 19:26 |
| *** bhavikdbavishi has joined #zuul | 19:27 | |
| clarkb | masterpe: https://zuul-ci.org/docs/zuul/howtos/zuul_install.html#installation | 19:27 |
| clarkb | the from scratch docs dont use docker | 19:27 |
| *** hashar has joined #zuul | 19:28 | |
| clarkb | you would do apt-get install -y $(bindep -b compile) for ubuntu in that step | 19:29 |
| clarkb | the other steps should be fairly distroo indeoendent | 19:29 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: use-buildset-registry: remove unused test library from remarshal https://review.opendev.org/724735 | 19:30 |
| avass | clarkb, masterpe: right I always get confused by the layout of "Zuul from scratch" | 19:31 |
| *** rlandy|biab is now known as rlandy | 19:32 | |
| masterpe | I will give Zuul from scratch a try | 19:33 |
| avass | masterpe, clakb: I do remember having to build the json part of zuul-web for some reason on ubuntu though, can't remember why | 19:34 |
| avass | masterpe: if your zuul-web isn't working as it should later, make sure it got built :) | 19:35 |
| *** cloudnull2 has joined #zuul | 19:37 | |
| *** cloudnull has quit IRC | 19:38 | |
| *** cloudnull2 is now known as cloudnull | 19:38 | |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Validate ansible extra packages https://review.opendev.org/724110 | 19:39 |
| reiterative | I think https://review.opendev.org/#/c/693513/ is just about ready to go now, but it's missing a Workflow label. Could someone do the honours? | 19:46 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Validate ansible extra packages https://review.opendev.org/724110 | 19:46 |
| *** bhavikdbavishi has quit IRC | 19:47 | |
| corvus | mordred: yeah, i'll review it (at least to make sure i have the latest version in my brain) | 19:51 |
| *** saneax has quit IRC | 19:51 | |
| avass | reiterative: we usually require two maintainers to review it before we merge something, unless it's something simple like a quick doc fix. :) | 19:52 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Add git name and email for quickstart executor https://review.opendev.org/724096 | 19:52 |
| mordred | corvus: I think we still have work to do on it - but I think it's sufficient to get images built | 19:53 |
| corvus | mordred: what do you want to improve? | 19:56 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Add git name and email for quickstart executor https://review.opendev.org/724096 | 20:00 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Control log archive and user preservation with vars https://review.opendev.org/701381 | 20:01 |
| corvus | mordred: question on 722339 | 20:02 |
| mordred | corvus: we still need to plumb in registry config so that it will pull from buildset registry | 20:05 |
| mordred | corvus: right now it is not appropriate for consuming depends-on | 20:05 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Control log archive and user preservation with vars https://review.opendev.org/701381 | 20:05 |
| mordred | corvus: ah - your question is the thing I think we need to improvie | 20:06 |
| avass | corvus: re ^, I just realized I completely misunderstood what you meant. I should probably not work on things when I'm too tired | 20:06 |
| mordred | corvus: there'sa. bunch of work to allow us to push the results of the build to the buildset registry | 20:06 |
| avass | like 22:06 right now :) | 20:06 |
| mordred | corvus: but in order to configure the builder to pull things _from_ the buildset registry when it's building (the mirror config stuff) - we need to write a buildkit.toml file - which is similar to but different from the container config used by buildah/podman | 20:07 |
| avass | corvus: I have a feeling that all unarchive/synchronize tasks between remote and executor should behave that way | 20:07 |
| mordred | corvus: pushing to the buildset registry works because we're doing an explicit push with the buildset registry appended to the location | 20:08 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Control log archive and user preservation with vars https://review.opendev.org/701381 | 20:09 |
| corvus | mordred: what do you think about holding the merge of that until it's ready? | 20:18 |
| corvus | mordred: i feel like "depends-on will silently not work" may be a bad trap here | 20:19 |
| mordred | corvus: could do - I wasn't sure what the relative urgency was to get multi-arch nodepool images built | 20:19 |
| corvus | mordred: i'd say it's pretty urgent for opendev, but it's not urgent for zuul-jobs :/ | 20:20 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: Add configure-podman-registry role https://review.opendev.org/724744 | 20:22 |
| mordred | corvus: do the current test jobs test that we can consume images from the intermediate registry when we build a new image? | 20:22 |
| corvus | mordred: they're supposed to, so i'm interested in seeing why they didn't fail | 20:23 |
| mordred | corvus: ah - FROM docker.io/library/debian:testing | 20:24 |
| mordred | corvus: so - the jobs test that we can docker pull on the builder host | 20:24 |
| mordred | but don't build a downstream image FROM the upstream image | 20:24 |
| tristanC | corvus: not sure how it works in opendev, but to use the zuul-registry with podman on fedora, i had to set the CA used to signed the registry cert for the buildset_registry.cert value. | 20:25 |
| mordred | corvus: that said - now that I think about it - it's _possible_ this does Just Work | 20:26 |
| mordred | corvus: let me see if I can cook up a synthetic example locally real quick | 20:26 |
| tristanC | In the end we were able to push intermediary image using the configure-podman-registry role proposed in https://review.opendev.org/724744 . Not very happy about the code duplication, but that's a first milestone for us! | 20:27 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: DNM Check to see if images from intermediate work https://review.opendev.org/724751 | 20:31 |
| corvus | tristanC: that's great, but i have some questions on that change :) | 20:32 |
| mordred | corvus: ^^ I think that should show whether images from intermediate registry work | 20:32 |
| mordred | corvus: (but double check me that should work) | 20:33 |
| corvus | tristanC: notably, use-buildset-registry should already set everything up for podman/containers, so i'm curious why you can't use that directly, and if it doesn't work, what could we do to fix it. | 20:33 |
| tristanC | corvus: use-buildset-registry should work, but it seems like designed for buildset-registry, and it does docker/podman/cri-o all at once | 20:39 |
| corvus | tristanC: oh, you're trying to push to the intermediate registry; why not use the push-to-intermediate-registry role? | 20:40 |
| corvus | tristanC: and yes, it does do all of those things, but that shouldn't be a problem? that way it can be used in a base job that doesn't need to know exactly how an image is being built | 20:42 |
| tristanC | corvus: here is the playbook for our image-update job: https://softwarefactory-project.io/r/gitweb?p=zuul-images-jobs.git;a=blob;f=playbooks/image/update.yaml;h=0bd8ac81f4b4505edffb155de9f9a1cca6685e28;hb=feee5fa32e39841970f1ea1e684d7d13a0b8180b | 20:42 |
| corvus | mordred: i think that test is valid | 20:43 |
| tristanC | corvus: push-to-intermediate-registry seems to be doing `skopeo copy docker://... docker://`, but when the image is built with podman, then the source is `container-storage:` | 20:46 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Write a buildkitd config file pointing to buildset registry https://review.opendev.org/724757 | 20:46 |
| mordred | corvus: cool. if it is - then maybe that ^^ should work | 20:46 |
| tristanC | corvus: and we don't use a buildset registry, only an intermediary registry, so doing `podman build`, then `podman push` works well, and it's only two tasks | 20:47 |
| mordred | corvus: although - now that I think about it - I think we should likely move the setup stuff into use-buildset-registry | 20:47 |
| corvus | mordred: yeah, but we need the builder, right? | 20:47 |
| mordred | corvus: yah - but we could have use-buildset-registry do the "are there arches in the images list" detection and then do the buildx-setup tasks | 20:47 |
| mordred | corvus: so that it's in a position to configure the buildx builder at the same time | 20:48 |
| tristanC | corvus: using `use-buildset-registry` seems overkill, it does things we don't want like restart a docker service, and i'm not sure why it works with a registry.cert where we need a registry.ca value | 20:48 |
| corvus | tristanC: i'd still love to have a discussion about how you could use the gating workflow | 20:48 |
| corvus | i'm a little concerned about overloading the zuul-jobs registry system to support a second non-gating workflow | 20:49 |
| tristanC | corvus: we'd be happy to talk about this use case, here is the documentation so far: https://tree.taiga.io/project/morucci-software-factory/us/3419 and https://softwarefactory-project.io/cgit/zuul-images-jobs/tree/README.md | 20:50 |
| tristanC | it's not exactly a gate, but rather a periodic pipeline to update image and push good new layers | 20:52 |
| corvus | tristanC: right, i think we're proving out the gate process, why not use that? we're using it for zuul, opendev, mnaser is using it for vexxhost, openstack is starting to use it... | 20:53 |
| corvus | swift is using it | 20:53 |
| *** flaper87 has quit IRC | 20:53 | |
| *** johanssone has quit IRC | 20:53 | |
| *** openstackgerrit has quit IRC | 20:53 | |
| *** johanssone has joined #zuul | 20:54 | |
| mordred | corvus: the test of pulling from intermediate in multiarch failed as we expected: https://zuul.opendev.org/t/zuul/build/7977d9d05428466aa607936d54ea0383 | 20:55 |
| corvus | mordred: excellent! :( | 20:55 |
| mordred | corvus: so - at least the hunch about how things work or don't work turned out to be correct | 20:55 |
| fungi | i gather the kolla devs are currently looking at redoing their image building jobs to take advantage of the buildset and intermediate registry workflow | 20:55 |
| mordred | corvus: now let's see if that buildktd.toml fixed it | 20:55 |
| mordred | fungi: woot! that's great news | 20:56 |
| fungi | at least that seemed to be their resolution after a recent discussion about their broken old legacy converted jobs | 20:56 |
| tristanC | we also would like to use that gate process, but we don't have a gate since our job are not tied to a change, they run periodically | 21:18 |
| tristanC | (and we are aiming for a pipeline that also works for diskimage) | 21:25 |
| *** hashar has quit IRC | 21:27 | |
| mordred | corvus: ok - my first stab at writing a buildkitd.toml did not work | 21:33 |
| mordred | corvus: and - it doesn't seem like it decided to provide any information as to WHY | 21:33 |
| mordred | oh wait | 21:34 |
| *** openstackgerrit has joined #zuul | 21:34 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Write a buildkitd config file pointing to buildset registry https://review.opendev.org/724757 | 21:34 |
| mordred | corvus: in fact, it DID provide logs, and they were clear, and I can verify that it read the config file I put in :) | 21:34 |
| mordred | https://6858d98d7a501e0358b0-54baced7b3de4d4a8f29448d2e5adbbf.ssl.cf2.rackcdn.com/724757/3/check/zuul-jobs-test-registry-docker-multiarch/9badf51/builder/docker/buildx_buildkit_mybuilder0.txt | 21:35 |
| mordred | and if you look at the interdiff between the last 2 patchsets of 724757 - I agree with the buildx builder that it was an error :) | 21:35 |
| corvus | mordred: agree that looks legit | 21:36 |
| mordred | corvus: I'm a little annoyed it didn;t know to ignore my copy-pasta error and do what I meant | 21:36 |
| mordred | all of the rest of you know that by now | 21:36 |
| corvus | mordred: i feel toml is not living up to its billing | 21:37 |
| mordred | corvus: ikr | 21:37 |
| mordred | corvus: I thought tom was solving all of the world's problems by inventing a new and ugly config file format | 21:37 |
| mordred | I don't feel like this solved my problems despite the use of a bunch of square brackets! | 21:38 |
| mordred | corvus: so - back to the previous thing - I think I've convinced myself that we do need a reorg of some of the new buildx stuff | 21:39 |
| mordred | but I think I've also convinced myself that can wait | 21:39 |
| mordred | as it should not have end-user facing impact | 21:39 |
| mordred | at the very least, I think we can have use-buildset-registry just write an /etc/buildkit/buildkitd.toml file (editing it using the same approach as the containers.conf file) | 21:40 |
| mordred | and then if we want to keep buildx set up in the build role, we can just copy the file in place | 21:40 |
| corvus | oh that's a good idea | 21:40 |
| mordred | because it's also potentially a useful file in case someone wanted to use buildkit on the build host | 21:40 |
| mordred | (I'm also not sure whether dockerd with USE_BUILDKIT=1 would read /etc/buildkit for config...) | 21:41 |
| mordred | corvus: 2020-04-30 21:44:08.302296 | builder | #4 [linux/amd64 internal] load metadata for docker.io/upstream/image:latest | 21:44 |
| mordred | 2020-04-30 21:44:09.503583 | builder | #4 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed | 21:44 |
| mordred | 2 | 21:44 |
| mordred | corvus: sad panda | 21:44 |
| mordred | corvus: I don't think I have enough brain pellets to further debug that today | 21:45 |
| mordred | I'll pick it up in the morning when my pellet trough has hopefully been replenished | 21:45 |
| corvus | mordred: kk | 21:45 |
| mordred | corvus: https://github.com/moby/buildkit/issues/606#issuecomment-569969872 | 21:48 |
| mordred | corvus: it's possible there's something the buildkit may be doing that's non-standard? | 21:48 |
| mordred | https://add65db26ee2f5c0e620-8e6e263cc7d5bd2680bb2429d6dd2093.ssl.cf1.rackcdn.com/724757/4/check/zuul-jobs-test-registry-docker-multiarch/2eb566e/ is the logs | 21:49 |
| mordred | corvus: 21:44:09.503583 is when it fails to pull ... which corresponds to some lines here: | 21:50 |
| mordred | https://add65db26ee2f5c0e620-8e6e263cc7d5bd2680bb2429d6dd2093.ssl.cf1.rackcdn.com/724757/4/check/zuul-jobs-test-registry-docker-multiarch/2eb566e/builder/docker/buildset_registry.txt | 21:50 |
| mordred | 2020-04-30 21:44:08,353 INFO registry.authz: Authenticate level read | 21:50 |
| corvus | mordred: maybe run the buildset registry with debug so we can see what the uris are | 21:51 |
| mordred | corvus: got a cantrip for that handy? | 21:53 |
| mordred | corvus: oh wait - I'm supposed to set username/password for the buildset registry aren't I? | 21:54 |
| *** rfolco has quit IRC | 22:37 | |
| tristanC | mordred: that should be defined by https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/build-docker-image/tasks/main.yaml#L5 | 22:51 |
| *** tosky has quit IRC | 23:11 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] add plain nodes https://review.opendev.org/724776 | 23:15 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] suse python2 pip support https://review.opendev.org/724777 | 23:15 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!