| openstackgerrit | Emilien Macchi proposed openstack-infra/zuul-jobs master: zuul-cloner-shim: load python from the system https://review.openstack.org/522093 | 03:12 |
|---|---|---|
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool feature/zuulv3: Reorg non detailed instance listing columns https://review.openstack.org/522103 | 03:41 |
| *** jkilpatr has quit IRC | 03:43 | |
| *** bhavik1 has joined #zuul | 05:26 | |
| *** openstackgerrit has quit IRC | 06:33 | |
| *** bhavik1 has quit IRC | 06:45 | |
| *** xinliang has quit IRC | 06:58 | |
| *** openstackgerrit has joined #zuul | 07:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Setup virtualenv for zuul-cloner https://review.openstack.org/522145 | 07:13 |
| *** xinliang has joined #zuul | 07:14 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Setup virtualenv for zuul-cloner https://review.openstack.org/522145 | 07:20 |
| *** isaacb has joined #zuul | 07:23 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use username from node information if available https://review.openstack.org/453983 | 07:23 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Rename ssh_port to connection_port https://review.openstack.org/500799 | 07:23 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use connection type supplied from nodepool https://review.openstack.org/501976 | 07:23 |
| *** hashar has joined #zuul | 07:31 | |
| *** threestrands has quit IRC | 07:47 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/jobs route https://review.openstack.org/503270 | 08:08 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/builds route https://review.openstack.org/466561 | 08:08 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add Cache-Control to static files https://review.openstack.org/522163 | 08:08 |
| openstackgerrit | Rui Chen proposed openstack-infra/nodepool feature/zuulv3: Apply floating ip for node according to configuration https://review.openstack.org/518875 | 08:34 |
| *** isaacb has quit IRC | 09:44 | |
| *** electrofelix has joined #zuul | 10:25 | |
| *** hashar is now known as hasharAway | 10:54 | |
| *** jkilpatr has joined #zuul | 12:15 | |
| *** pbrobinson has quit IRC | 12:29 | |
| *** pbrobinson has joined #zuul | 12:31 | |
| *** weshay_pto is now known as weshay | 12:58 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Apply floating ip for node according to configuration https://review.openstack.org/518875 | 13:38 |
| tristanC | so i'm setting up an openstack tenant to run a third-party-ci on zuul-jobs with container slaves... just got the first run that failed with the bindep role here: https://review.openstack.org/522213 | 13:43 |
| *** jasondotstar has quit IRC | 13:46 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: unittests: make bindep role optional https://review.openstack.org/522255 | 13:56 |
| *** smyers has quit IRC | 14:04 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: revoke-sudo: only revoke when zuul is sudoer https://review.openstack.org/522261 | 14:05 |
| *** smyers has joined #zuul | 14:11 | |
| *** jasondotstar has joined #zuul | 14:15 | |
| dmsimard | tristanC: can you add it to the zuul-jobs pad ? | 14:17 |
| tristanC | dmsimard: it's already there, i'm adding review to proposed review | 14:19 |
| dmsimard | tristanC: ok | 14:19 |
| dmsimard | tristanC: where's the bindep failure? I don't see it | 14:20 |
| tristanC | dmsimard: it's on PS2, right before i added the depends-on 522255 | 14:21 |
| tristanC | btw, the third-party-ci-config is https://softwarefactory-project.io/r/gitweb?p=third-party-ci-config.git;a=tree | 14:21 |
| dmsimard | ERROR: The executable /tmp/bindepkxXV7q/venv/bin/python2 could not be run: [Errno 13] Permission denied | 14:22 |
| dmsimard | weird ? | 14:22 |
| tristanC | and the test-unittests job is in https://softwarefactory-project.io/r/gitweb?p=third-party-ci-jobs.git;a=tree | 14:22 |
| dmsimard | tristanC: could that be related to the systemd tmpdir I told you about ? | 14:22 |
| tristanC | dmsimard: /tmp is noexec in this container | 14:22 |
| *** jasondotstar has quit IRC | 14:23 | |
| dmsimard | tristanC: hmm, you should probably match the container rules with the bubblewrap ones ? | 14:23 |
| dmsimard | I don't think it's a bad assumption to say we can run things from /tmp | 14:24 |
| tristanC | dmsimard: sure, just have to update L270 of https://review.openstack.org/#/c/468753/26/nodepool/driver/oci/provider.py | 14:26 |
| tristanC | dmsimard: things is, container likely do not want to use bindep to run unittest | 14:26 |
| dmsimard | bindep is a mechanism to install deps that might be required for python libraries installed by tox (which will then run unit tests) | 14:28 |
| dmsimard | Think like installing libxml/libxslt/openssl for crypto compilation | 14:28 |
| dmsimard | Are you expecting nothing to be installed in the container ? Maybe I'm not clear on the use case, I thought it was probably a VM replacement | 14:29 |
| tristanC | dmsimard: well yes, those oci containers are immutable, / is ro | 14:30 |
| *** jasondotstar has joined #zuul | 14:31 | |
| tristanC | i got to go now, be back tomorrow | 14:33 |
| tristanC | forgot to mention, the openstack tenant is setup like this: https://softwarefactory-project.io/r/gitweb?p=config.git;a=blob;f=zuulV3/openstack.yaml;hb=HEAD | 14:34 |
| *** jasondotstar has quit IRC | 14:35 | |
| *** jasondotstar has joined #zuul | 14:45 | |
| *** _ari_ has quit IRC | 14:49 | |
| *** _ari_ has joined #zuul | 15:02 | |
| openstackgerrit | Jan Kundrát proposed openstack-infra/zuul master: Prepare correct refspec on new Gerrit https://review.openstack.org/439057 | 15:21 |
| *** bhavik1 has joined #zuul | 15:43 | |
| *** bhavik1 has quit IRC | 15:49 | |
| openstackgerrit | Sean McGinnis proposed openstack-infra/zuul-jobs master: Update upload-npm credentials secret https://review.openstack.org/522284 | 15:49 |
| SpamapS | dmsimard: tmp might be mounted noexec? | 16:04 |
| dmsimard | SpamapS: that's what ended up being his problem, yes, but I'm not understanding the container use case properly I think | 16:05 |
| dmsimard | SpamapS: I thought the goal of the container driver was to be able to run fully-fledged jobs without an OpenStack cloud easily -- not sure what's the extent of the things we can run if the container is super locked down | 16:06 |
| dmsimard | We're "bastardizing" the usage of a container here, don't get me wrong -- a container is supposed to be immutable, but it doesn't need to be | 16:06 |
| *** haint has quit IRC | 16:07 | |
| dmsimard | It could mount volumes from from the bubblewrap environment or something, I dunno. | 16:07 |
| SpamapS | dmsimard: I think the idea is to have the container image ready to rock and roll and just start it up with some git volumed in, and run an executable. | 16:08 |
| dmsimard | SpamapS: in the general world, yes ? but in the context of CI, that executable is going to read and write things | 16:08 |
| dmsimard | whether it's tox creating the venv and installing packages before running tests or other things | 16:09 |
| SpamapS | IMO we don't want "a container" from nodepool. We want a space to run a container. | 16:09 |
| SpamapS | Because most cases want to start by building an image with the code on it. | 16:09 |
| dmsimard | I want something that isn't an OpenStack VM and I thought that the OCI driver was going to provide that :P | 16:09 |
| dmsimard | Having an openstack cloud to run zuul/nodepool against is a huge barrier to entry | 16:10 |
| SpamapS | dmsimard: I have it on my todo to make a nodepool driver that provides k8s namespaces. | 16:10 |
| dmsimard | that requires standing up a k8s which is probably a smaller barrier to entry than an openstack cloud | 16:11 |
| dmsimard | OCI containers are simple stupid | 16:11 |
| SpamapS | dmsimard: also another good use for containers is to get out of the untrusted prison.... so maybe there is a case for the mutable container. | 16:11 |
| dmsimard | it wouldn't scale very well, but with a mutable container option, I could technically have an all-in-one node with everything in it without requiring a cloud | 16:12 |
| dmsimard | and I think that could be cool, even if just to allow people to stand up and try out zuul/nodepool easily | 16:13 |
| SpamapS | Yeah... another way to look at it though, is to explode the number of clouds you can run jobs on. | 16:13 |
| SpamapS | So add AWS, GCE, VMware, Azure, Kubernetes, Docker Swarm... etc. etc. | 16:14 |
| SpamapS | because honestly.. allinone is really Jenkins's wheelhouse. | 16:14 |
| dmsimard | SpamapS: why do you think jenkins is popular ? :) | 16:14 |
| dmsimard | zuul/nodepool doesn't need to be complicated or expensive to run | 16:14 |
| SpamapS | I think an allinone that you're playing with could be fine with mostly trusted jobs. | 16:19 |
| *** nguyentrihai has joined #zuul | 16:20 | |
| SpamapS | dmsimard: can we back up. /tmp was noexec.. is that something OCI defines always? | 16:22 |
| dmsimard | SpamapS: the policy appears explicit, tristanC mentioned line 270 of this https://review.openstack.org/#/c/468753/26/nodepool/driver/oci/provider.py | 16:23 |
| SpamapS | dmsimard: awesome | 16:28 |
| SpamapS | so we can fix that :) | 16:28 |
| SpamapS | Now we need to step up one level, and get rid of the idea that nodes are always SSH targets. | 16:30 |
| dmsimard | mordred said he was going to start a thread on openstack-infra about that topic | 16:30 |
| dmsimard | ಠ_ಠ | 16:30 |
| rcarrillocruz | there's some work already fo that | 16:30 |
| rcarrillocruz | tobiash started plumbin ansible_connection | 16:30 |
| rcarrillocruz | plumbing | 16:30 |
| SpamapS | Yeah nodepool shouldn't necessarily know that though. | 16:31 |
| jeblair | okay, i'm going to take one more stab at this. how zuul and nodepool should handle containers is a really important conversation to have, but it's really complicated. if we want to release v3.0, we do not have time to have it now. so let us, please, as a group, choose to work on one thing or the other. | 16:31 |
| SpamapS | But definitely should be able to say "Here's connection info, type: x" | 16:31 |
| jeblair | SpamapS: my understanding is that you really want v3.0 released? | 16:32 |
| SpamapS | I want all the things, and perfection, and ponies. SO many ponies. | 16:32 |
| dmsimard | jeblair: we got into that discussion because tristanC is currently testing zuul-jobs from a third party perspective on the experimental OCI driver | 16:33 |
| dmsimard | We don't need to shift all the manpower to containers at all :) | 16:33 |
| jeblair | well, there is no guarantee that what tristanC is doing is going to end up in zuul in its current state | 16:33 |
| jeblair | that's a risk tristanC knows | 16:34 |
| jeblair | i would prefer if we could all spend at least some of our time working to get v3.0 out the door | 16:34 |
| jeblair | since that's a goal we've agreed on | 16:34 |
| SpamapS | jeblair: the problem I'm running into at this very moment, is two fold. 1) No v3.0 == no respect from demo watchers. 2) Users in my org are very wary of OpenStack, despite having access to a pretty stable OpenStack with thousands of HV's, and the first question they ask is "can I use it on ...." would like to (k8s, aws, gce, *)" | 16:34 |
| rcarrillocruz | my undestanding is that nodepool drivers was not a short-term goal for release anyways | 16:34 |
| jeblair | SpamapS: i'm starting to spend more time telling people that we only just need a little more work to get v3.0 out the door than i am actually working on zuul | 16:35 |
| SpamapS | so yes, (1) is the first step, and since (1) is sort of out of my control in many ways, I tend to jump to (2) :-/ | 16:35 |
| jeblair | SpamapS: it's not out of your control. there are certainly things you can do to get v3.0 out the door | 16:35 |
| SpamapS | If I can pick up a specific v3.0 task, I will have some time to do that this week. | 16:35 |
| mnaser | i've been a bit out of it for the past few days. was there a fix put in to limit the number of dynamic reconfigs in zuul v3? | 16:37 |
| jeblair | and in the mean time, it would be really great if we could put a pin in the container thing so we can finish this and get to the point where we can have a design conversation. we've been talking about it in snippets for 2 years, so i know it's not going to be short. :) | 16:37 |
| jeblair | mnaser: no | 16:37 |
| mnaser | jeblair: ok cool, i'll stick to 5 reconfig changes at a time | 16:37 |
| dmsimard | SpamapS: I find the "Users in my org are very wary of OpenStack" argument quite unfortunate, having experienced it first hand with ARA | 16:39 |
| dmsimard | SpamapS: I've had folks tell me they wouldn't even try ara because of the openstack affiliation | 16:39 |
| rcarrillocruz | same with Ansible engineering, lol | 16:39 |
| electrofelix | I wish more users in orgs I know were willing to add any tests at all, never mind the level of testing that openstack has... | 16:40 |
| SpamapS | dmsimard: the users in my org have had OpenStack for 4 years. They're about ready to start giving in to their 2nd system syndrome. :-/ | 16:40 |
| dmsimard | SpamapS: where is that again? Godaddy? | 16:41 |
| SpamapS | dmsimard: aye | 16:41 |
| SpamapS | IMO we have a really great cloud. It's a little quirky... requiring unique server names, and limiting the name field to 15 chars.. also requiring AZ to be specified. But for the most part, it's fast, robust, and works quite well despite being stuck on Liberty. | 16:42 |
| rcarrillocruz | electrofelix: that's kind of the argument I give 'ok, you don't like openstack cos you heard is hard, just private, whatever, but PLEASE look at just the CI, the tooling is AWESOME' | 16:42 |
| jeblair | SpamapS: i've got 2 things on my pre-3.0 list i can hand to you if you want: abstract job attribute / protected flag. abstract is half-implemented here: https://review.openstack.org/504809 | 16:42 |
| electrofelix | rcarrillocruz: ;-) | 16:42 |
| SpamapS | Most of what people don't like about it has to do with the restrictions placed on images (you have to ask really nice to be able to upload your own images and agree to update your images for security problems) | 16:42 |
| dmsimard | SpamapS: didn't godaddy close their openstack cloud ? | 16:42 |
| clarkb | dmsimard: only the public stuff aiui | 16:42 |
| dmsimard | ohhhhh | 16:43 |
| clarkb | so you can't show up with a credit card anymore | 16:43 |
| SpamapS | yeah and the public one was... a bit misguided in some ways | 16:43 |
| electrofelix | jeblair: do you have time to chat about zuul supporting pushing back merge commits? not a 3.0 thing :D, but the more I can show things are more likely to be possible (or at least more sensible) on v3 instead of hacking on v2, the better chance I have to getting some time allocated to it | 16:43 |
| SpamapS | Oddly enough, it was basically fronted by something very similar to mordred's oaktree | 16:44 |
| SpamapS | They saw the ... challenges with supporting OpenStack's APIs. ;) | 16:44 |
| SpamapS | jeblair: Are those attributes already in spec somewhere? Not sure I remember reading about them. | 16:45 |
| SpamapS | electrofelix: haha ... you did see jeblair's rebuke of our container bikeshedding right? ;-) | 16:46 |
| jeblair | SpamapS: they aren't -- they just came up in conversation here and in -infra as we found they were things that would be useful during the conversion | 16:46 |
| SpamapS | electrofelix: You can tell your management it's never going to happen on v2. :) | 16:46 |
| electrofelix | SpamapS: my managements response would be "sure couldn't you just stick something in and land locally..." | 16:47 |
| rcarrillocruz | lol | 16:48 |
| SpamapS | electrofelix: debt is only useful when it leads to significant returns. What is the benefit of doing that locally that its worth taking on local patch debt? | 16:48 |
| electrofelix | you speak sense, you must not be in management :p | 16:48 |
| SpamapS | zuul pushing is something I want to see on the 3.1 list... but what I really want to see is lots of new names on the 3.0 authors list. :) | 16:49 |
| SpamapS | electrofelix: they keep trying to pull me into that mess.. have been able to avoid it thus far... but .. every grey hair I get seems to be a beacon to management that "he's almost dead enough inside to be one of us!" | 16:49 |
| electrofelix | lol | 16:49 |
| jeblair | electrofelix: you're talking about pushing zuul refs for workers to pull, right? not pushing the result to the final branch. (which i'm guessing is what SpamapS is interpreting) | 16:50 |
| SpamapS | I thought we were talking about pushing the result. | 16:50 |
| electrofelix | jeblair: nope, this time it's about the push the result to the final branch, I'll have to circle back on zuulv3 -> jenkins fun later | 16:51 |
| jeblair | oh neat | 16:51 |
| SpamapS | jeblair: are there notes on protected/abstract I can view? (In the patch maybe? Haven't looked yet) | 16:51 |
| jeblair | electrofelix: then yes, like SpamapS, i am in support of that after v3.0 | 16:51 |
| *** nguyentrihai has quit IRC | 16:52 | |
| electrofelix | jeblair: we're building docker images in the gate and trying to save and publish those in a post pipeline instead of rebuilding just in case something changed dependency wise between when the gate started and the change passed and triggered a post merge pipeline | 16:52 |
| jeblair | abstract flag (do not run this job) for base jobs that will be inherited from but not run | 16:52 |
| jeblair | protected flag (inherit only within this project) | 16:52 |
| jeblair | SpamapS: ^ that's the entirety of my notes on the subject | 16:53 |
| electrofelix | but that leads to some interesting git metadata being recorded unless we can push back the merge commit | 16:53 |
| electrofelix | I suspect the limitation on push back is: when enabling push back, any dependent changes within a given project must be merged on the same zuul executor so that they will have correct parents in their history should they also pass | 16:54 |
| electrofelix | Is that correct? | 16:55 |
| electrofelix | is there a story on that, at the very least if I can push management to see that there are enough things on the horizon that directly solve our issues, I might be able to get some time to just work on zuul in general | 16:56 |
| jeblair | electrofelix: that does seem likely to be an issue. so the solution will probably involve more than "just pushing". | 16:56 |
| jeblair | electrofelix: i don't think much if any work on this has been done yet, so starting a spec or story would be a good idea | 16:56 |
| SpamapS | jeblair: fantastic, I can go with those notes. :) | 16:57 |
| jeblair | electrofelix: i can say it's something we've long desired and we can engage on design and implementation after v3.0 release. maybe after container design. :) | 16:57 |
| SpamapS | I feel like that shouldn't be an issue. If you have the result of the merge that you tested, you should be able to just push that. | 16:58 |
| electrofelix | jeblair: ok, that's at least a starting point, and yes, it means likely keeping track of which merger (or is it called executor now?) node a project is associated with | 16:58 |
| SpamapS | I figured it was more of an access and interface issue. We might need to change zuul's access levels in Gerrit to be able to directly push vs. tell Gerrit to merge. | 16:59 |
| jeblair | SpamapS: i think electrofelix is saying that if zuul creates a merge commit for change A, and change B depends on A, different mergers would create different merge commits for A, so whichever merge commit for A ends up pushed to the source, *that specific commit* needs to be in the history for B in order to push B. | 16:59 |
| jeblair | electrofelix: or it could mean ensuring all mergers can communicate with each other, and when merging B, first pull from the merger which handled A rather than repeating that merge | 17:00 |
| jeblair | incidentally, that's related to a solution SpamapS and i discussed for avoiding repeated mergers on executors | 17:00 |
| electrofelix | jeblair: yes, that would be more complex, but also less likely to risk a minor dos | 17:00 |
| SpamapS | Yeah I see that, however, for the use case electrofelix and I have.. we only actually care that the sha is stable for the last thing that merges. | 17:00 |
| jeblair | so quite possibly the solution to both could be "let mergers and executors ssh to each other" | 17:00 |
| jeblair | SpamapS: right but i don't think you would be able to push B with the wrong parent | 17:01 |
| SpamapS | it would be fantastic to have all the artifacts.. and we maybe stretch ourselves to get that done some day.. but I'm good with only having the last one as a first step. | 17:01 |
| jeblair | this probably warrants some testing with git to double check :) | 17:01 |
| SpamapS | jeblair: well right now, with a dependent meanager, we "click merge" in the order of landing, right? | 17:02 |
| electrofelix | hmm, I think for our use case we'd need to solve for gate queues of up to 5 changes in flight | 17:02 |
| pabelanger | much backscroll | 17:02 |
| jeblair | SpamapS: yeah, that causes gerrit to create merge commits of its own if necessary. they aren't the same ones that zuul creates. | 17:02 |
| SpamapS | So we're basically just using gerrit as the way we communicate. We could very easily make the scheduler do that.. and basically say "hey scheduler I'm done and I want to push" and it can say "push it all" or "actually what you have already got pushed" | 17:04 |
| jeblair | SpamapS: it's fine to push a merge commit from zuul to gerrit. but if you push a second commit on top of that (say because you had 2 changes in the gate simultaneously), that second commit has to have the same git parent as what is currently in the tree. | 17:04 |
| SpamapS | but yeah | 17:04 |
| SpamapS | --> 3.1 | 17:04 |
| SpamapS | this deserves a nice solid design. | 17:04 |
| jeblair | ya. we can probably stop here with "this is desirable, there are issues, let's talk about it later." :) | 17:04 |
| jeblair | (i'm fairly certain it is doable though) | 17:05 |
| electrofelix | I think we might be willing to run with a POC in this area | 17:05 |
| electrofelix | and deal with changing it based on feedback and real world results | 17:06 |
| electrofelix | SpamapS jeblair: think the most scale-able solution would be to favour using the same merger for changes for the same project up a limit, and then switching to having a different merger get the latest tree from the first merger. Either would be good enough as a starting point for an initial pass | 17:17 |
| dmsimard | electrofelix: we're able to keep up with the upstream gate with 3 mergers | 17:18 |
| dmsimard | electrofelix: (as third party CI) | 17:19 |
| dmsimard | electrofelix: the main problem is that they are "single threaded" so if you only have one merger and it's stuck in a rebase hell of 20 patchsets with different depends-on mixed in there, it's going to be busy for a while | 17:19 |
| jeblair | electrofelix: i lean toward thinking we should just always have mergers fetch previous merges from other mergers since (a) gearman makes it hard to schedule things like that, (b) it can be used to solve the executor double-merge problem, and (c) you said we should do it anyway :) | 17:19 |
| jeblair | electrofelix: but it's early days -- that's just my first instinct. :) | 17:20 |
| dmsimard | iirc there's also code in zuul so that the merger doesn't needlessly merge everything if it's for a project it doesn't care about | 17:20 |
| electrofelix | jeblair: seems like a good starting point, the other step would be an optimization to keep down traffic for a really big set up (if someone was to run Zuul-as-a-Service for GitHub it might be relevant, but not critical) | 17:21 |
| SpamapS | Yeah I like that plan for its efficiency win. I dislike it for its resiliency requirements | 17:22 |
| electrofelix | SpamapS: as in we should always have the resulting merge synced to some set of nodes to ensure any of them could perform the push back? | 17:23 |
| SpamapS | or push them into the source | 17:24 |
| jeblair | i'm not in favor of pushing temporary merges into the source -- they take up a lot of space quick and you have to think about how they might get propogated | 17:27 |
| SpamapS | ZUUL_REF | 17:27 |
| SpamapS | err | 17:27 |
| SpamapS | errant enter press | 17:27 |
| electrofelix | SpamapS: Also I think that introduces a race unless you make sure the mergers wait until it's been pushed to a suitably hidden branch | 17:27 |
| SpamapS | ZUUL_REF may yet have a place in mergers eh? :-P | 17:27 |
| jeblair | apparently so. i haven't managed to complety exise it yet. :| | 17:28 |
| SpamapS | I mean, it's still easier to say "all the mergers can talk to eachother" than it is to say "all the test nodes can talk to the mergers" | 17:28 |
| jeblair | yep | 17:29 |
| SpamapS | electrofelix: the way around having this, btw, is to log your build artifact ID, and scrape it out of the log. | 17:29 |
| SpamapS | not the prettiest thing, but it is doable. | 17:29 |
| SpamapS | you can even just drop a single file in the logs dir that is the ID, so it's not parsing and stuff like that. | 17:30 |
| electrofelix | SpamapS: not sure I'm following? we currently override the metadata with some additional layers in the docker images added during the post merge phase to reference the correct git sha1 | 17:30 |
| electrofelix | but it means we're chasing assumptions on what teams/people are doing with git metadata in their projects | 17:31 |
| SpamapS | electrofelix: so you want to build a docker image in gate, and then in post, use that same exact set of bits, yes? The way you can do that is to have the gate make a UUID for the docker image, and put that somewhere that your post job can find by finding the change's last passing log. | 17:31 |
| SpamapS | and then your post job can tag the commit with the docker image ID and then people who pull the branch can fetch the docker image that was tested in the gate with that commit. | 17:32 |
| electrofelix | SpamapS: that's what we're doing, it's just that we end up needing to wrap the image with additional git metadata to override what teams stored in the LABEL and ENV, trying to spot when this is necessary seems to be a bit of a challenge | 17:32 |
| rcarrillocruz | so | 17:32 |
| rcarrillocruz | i just added the first network operating system VM to my zuul | 17:33 |
| rcarrillocruz | been playing with ovs on xenial up till now | 17:33 |
| SpamapS | electrofelix: kk, so you have found the cracks in the workaround. ;) | 17:33 |
| rcarrillocruz | and i'm hitting what we were discussing yesterday | 17:33 |
| rcarrillocruz | http://paste.openstack.org/show/627125/ | 17:33 |
| rcarrillocruz | on non-linux boxes, the implicit gather_facts fail | 17:33 |
| rcarrillocruz | as that assumes on the other end python | 17:34 |
| electrofelix | SpamapS: yeap, and would love to stop trying to paper over them :D | 17:34 |
| rcarrillocruz | although, wait, i think i'm hitting here a plain ssh | 17:34 |
| rcarrillocruz | probably wrong user | 17:34 |
| * rcarrillocruz pokes at logs for ansible_user | 17:35 | |
| rcarrillocruz | yeah...so the appliance has 'vyos' user, i think the real issue here is the executor connects remotely against root | 17:36 |
| pabelanger | I think tobiash has a patch up to support that in nodepool? | 17:37 |
| pabelanger | shouldn't be root however | 17:37 |
| rcarrillocruz | yeah, been keeping an eye on it | 17:38 |
| rcarrillocruz | it is for me, look at a fact when gathering facts on xenial | 17:39 |
| rcarrillocruz | Ansible output: b' "ansible_user_id": "root",' | 17:39 |
| rcarrillocruz | oh well | 17:39 |
| rcarrillocruz | i have default_username as root on executor | 17:39 |
| rcarrillocruz | zuul.conf | 17:39 |
| rcarrillocruz | that explains | 17:40 |
| jeblair | yep | 17:40 |
| jeblair | rcarrillocruz: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/executor/server.py?h=feature/zuulv3#n910 | 17:40 |
| rcarrillocruz | so, as default username here is zuul, do i assume right that infra DIB images just create a 'zuul' user on xenial, fedora, etc with the proper zuul key on authorized_keys ? | 17:42 |
| rcarrillocruz | i use cloud-images for xenial | 17:42 |
| pabelanger | jeblair: is there anything else I should get together for https://review.openstack.org/521324/ ? Thanks to dmsimard for a more detailed explaination | 17:42 |
| pabelanger | rcarrillocruz: right | 17:43 |
| pabelanger | the root user we now use glean to setup SSH keys on the nodes, but zuul user we still bake in info | 17:44 |
| rcarrillocruz | k, i tried to use DIB for xenial, but had all sorts of fun with glean. As my provider pool has multiple networks (for testing multiple NIC things on our modules), half of the time the images would not be reachable probably cos NIC getting booted in different order | 17:44 |
| rcarrillocruz | erm, the instances i meant | 17:45 |
| rcarrillocruz | sooo... i think i'll just create zuul user on both xenial and vyos, take snapshot, retry | 17:45 |
| rcarrillocruz | i really want to get to teh point of failing facts gatherinjg | 17:46 |
| pabelanger | ++ | 17:48 |
| *** dmsimard is now known as dmsimard|afk | 17:53 | |
| *** jasondotstar has quit IRC | 17:57 | |
| EmilienM | when defining a job layout with irrelevant-files, does irrelevant-files merge with the parent job layout? | 17:59 |
| EmilienM | or the latest wins? | 17:59 |
| EmilienM | not sure my question is clear enough | 17:59 |
| jeblair | EmilienM: not really either -- irrelevant files causes that particular job variant to either match or not | 18:02 |
| jeblair | EmilienM: so you can't put irrelevant-files on a child job to cause the parent not to match | 18:03 |
| jeblair | (it would only cause the child not to match, but the parent still would) | 18:03 |
| EmilienM | jeblair: ok so child are checked first (from the youngest to the oldest) | 18:04 |
| EmilienM | but each list is taken in account | 18:04 |
| EmilienM | so I can maintain a top level list for all jobs | 18:04 |
| EmilienM | and having childs having their own list | 18:04 |
| EmilienM | for specifics | 18:05 |
| EmilienM | is that correct? | 18:05 |
| rcarrillocruz | http://paste.openstack.org/show/627128/ | 18:07 |
| rcarrillocruz | /sad trombone | 18:07 |
| jeblair | EmilienM: probably -- if you want to mock up a quick etherpad example, i can confirm :) | 18:08 |
| jeblair | rcarrillocruz: yay! successful failure! | 18:08 |
| rcarrillocruz | so yeah, i need to wait for tobiash change to land the ansible_connection change on https://review.openstack.org/#/c/501976/ , then push a patch to put a conditional, if no ssh then don't gather facts | 18:08 |
| EmilienM | jeblair: I'll show you a patch I'm doing | 18:08 |
| rcarrillocruz | lol, \o/ weee | 18:08 |
| EmilienM | shortly | 18:08 |
| rcarrillocruz | jeblair: does that sound as a good plan ^ | 18:09 |
| jeblair | rcarrillocruz: yep. you could probably hard-code some hacks in right now if you wanted to try to get a couple steps ahead :) | 18:10 |
| rcarrillocruz | yeah, feel like it :D | 18:10 |
| *** jasondotstar has joined #zuul | 18:18 | |
| EmilienM | jeblair: https://review.openstack.org/#/c/522321 is what I'm trying to achieve | 18:20 |
| jeblair | EmilienM: that's a long list -- are you sure 'files' wouldn't be better than 'irrelevant-files' ? | 18:22 |
| EmilienM | jeblair: I'm sure | 18:22 |
| EmilienM | jeblair: I spent all my morning thinking about it | 18:22 |
| EmilienM | and the list of files is too huge | 18:22 |
| EmilienM | jeblair: the list of irrelevant-files won't move every day | 18:23 |
| EmilienM | while 'files' will | 18:23 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Fix wrong dir for doc/requirements.txt https://review.openstack.org/522327 | 18:23 |
| jeblair | EmilienM: okay, so to take tripleo-ci-centos-7-ovb-fakeha-caserver as an example | 18:24 |
| EmilienM | jeblair: my question for you is, rrelevant-files defined in the parent job will still work? | 18:24 |
| jeblair | EmilienM: since you're doing parent/child, what zuul will do is see whether tripleo-ci-centos-7-ovb-fakeha-caserver matches. if it doesn't, the job won't run. if it does, then it will try to find its parent, tripleo-ci-ovb. if it doesn't match, the job won't run. if it does, it will. | 18:25 |
| EmilienM | ok | 18:25 |
| jeblair | EmilienM: i think you've found the one way to 'combine' irrelevant-files :) | 18:26 |
| EmilienM | jeblair: just to make sure, let's say I try to patch releasenotes and manifests/profile/base/aodh - can you confirm ovb job won't run? | 18:27 |
| jeblair | EmilienM: be careful though, if you were to add another variant of tripleo-ci-ovb (like for a stable branch or something), as long as any one matched the job would run. so if you need to do that, you may need to duplicate the irrelevant-files setting (or, add another layer to the hierarchy) | 18:27 |
| EmilienM | my guess is no, since manifests/profile/base/aodh is part of the child list and releasenote is part of the parent list | 18:27 |
| EmilienM | in fact, let's try that | 18:28 |
| EmilienM | so we're sure :) | 18:28 |
| EmilienM | https://review.openstack.org/522342 | 18:29 |
| EmilienM | jeblair: ok it didn't work | 18:30 |
| jeblair | EmilienM: i'm not 100% sure without drawing a big diagram, but from the sound of it, i'm not sure your example will work | 18:30 |
| EmilienM | jeblair: it didn't work, ovb are still running now | 18:31 |
| EmilienM | jeblair: even if I patch a file part of the irrelevant_files | 18:32 |
| jeblair | EmilienM: because manifests/profile/base/aodh is not an irrelevant file for tripleo-ci-centos-7-ovb-fakeha-caserver, so it matches. and releasenotes is not an irrelevant file for tripleo-ci-ovb, so it matches. | 18:32 |
| EmilienM | jeblair: I'm interested by tripleo-ci-centos-7-ovb-ha-oooq | 18:33 |
| jeblair | that doesn't have any irrelevant files? | 18:33 |
| EmilienM | jeblair: wait so how can I maintain a list for all child jobs | 18:33 |
| EmilienM | jeblair: I want the irrelevant files list be in a parent job layout, for all OVB jos | 18:34 |
| EmilienM | jobs* | 18:34 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Set success-url for sphinx-docs to html https://review.openstack.org/522017 | 18:34 |
| EmilienM | so I don't maintain the same list 12 times | 18:34 |
| jeblair | EmilienM: releasenotes is not listed in irrelevant-files for triplo-ci-ovb, so it matches | 18:34 |
| jeblair | EmilienM: if you proposed a change with only the aodh profile file touched, it would not match and not run | 18:35 |
| EmilienM | jeblair: but it's defined in the parent | 18:35 |
| EmilienM | so the parent list isn't taken in account :( | 18:35 |
| jeblair | EmilienM: tripleo-ci-ovb is the paret | 18:35 |
| EmilienM | jeblair: tripleo-ci-ovb has a parent | 18:35 |
| jeblair | EmilienM: legacy-dsvm-base. it has no irrelevant files, i hope. | 18:36 |
| EmilienM | jeblair: https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/base.yaml#L88-L96 | 18:36 |
| EmilienM | jeblair: it does :) | 18:36 |
| jeblair | EmilienM: that's tripleo-ci-dsvm, not legacy-dsvm-base | 18:36 |
| jeblair | EmilienM: from your patch: 22 parent: legacy-dsvm-base | 18:36 |
| EmilienM | oh | 18:37 |
| EmilienM | so I did a typo | 18:37 |
| EmilienM | I need to use tripleo-ci-dsvm | 18:37 |
| EmilienM | let's try again | 18:37 |
| EmilienM | jeblair: still doesn't work, OVB jobs are running | 18:38 |
| EmilienM | https://review.openstack.org/#/c/522321/3/zuul.d/ovb-jobs.yaml | 18:38 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Only run whereto if htaccess file exists https://review.openstack.org/521996 | 18:39 |
| jeblair | EmilienM: what files are you touching in your test patch? | 18:39 |
| EmilienM | jeblair: https://review.openstack.org/#/c/522342/ | 18:39 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Update fetch sphinx output to use sphinx vars https://review.openstack.org/521590 | 18:39 |
| jeblair | EmilienM: releasenotes matches tripleo-ci-ovb, and aodh matches the parent | 18:40 |
| EmilienM | jeblair: so we can't have inheritance on irrelevant files list | 18:41 |
| EmilienM | that would have been cool | 18:41 |
| EmilienM | another option is to add the irrelevant_files from the parent into the child | 18:42 |
| EmilienM | but it's duplicating things :( | 18:42 |
| jeblair | EmilienM: none of the matchers combine on inheritance, and irrelevant-files never does what you expect since it's backwards :|. files behaves sort of like how you would expect though. | 18:42 |
| jeblair | EmilienM: yeah, i think that's the best solution | 18:42 |
| EmilienM | k | 18:42 |
| *** mwhahaha has joined #zuul | 18:42 | |
| EmilienM | jeblair: is what I'm asking a feature or something well known and by design? | 18:43 |
| jeblair | (basically, when you put two irrelevant-files sections together, the set of files that can trigger the job get bigger. when you put two files sections together they get smaller) | 18:43 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Setup virtualenv for zuul-cloner https://review.openstack.org/522145 | 18:44 |
| jeblair | EmilienM: the way the matchers work in general is by design (it's an important part of how variants work). but i don't think much thought went into how that affects irrelevant-files in particular. i think we could consider special casing it, though i don't have any immediate ideas for how. | 18:46 |
| EmilienM | ok | 18:48 |
| EmilienM | jeblair: I confirm it works now. Thanks | 18:55 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Fix wrong dir for doc/requirements.txt https://review.openstack.org/522327 | 18:56 |
| *** jesusaur has quit IRC | 18:58 | |
| *** electrofelix has quit IRC | 19:19 | |
| *** jesusaur has joined #zuul | 19:20 | |
| rcarrillocruz | jeblair: if i inherit a job, are nodesets additive? meaning , will the child nodeset nodes be added to the parent or replaced altogether ? | 19:56 |
| jeblair | rcarrillocruz: replaced | 20:15 |
| rcarrillocruz | Thx | 20:16 |
| *** jkilpatr has quit IRC | 20:37 | |
| *** jkilpatr has joined #zuul | 21:14 | |
| *** threestrands has joined #zuul | 21:45 | |
| *** threestrands has joined #zuul | 21:45 | |
| rcarrillocruz | https://github.com/rcarrillocruz-org/ansible-fork/pull/8 | 22:39 |
| rcarrillocruz | WOOTZ | 22:39 |
| rcarrillocruz | first network operating getting test suite on zuul v3! | 22:39 |
| pabelanger | rcarrillocruz: \o/ | 22:45 |
| *** hasharAway has quit IRC | 22:45 | |
| rcarrillocruz | s/getting/running | 22:45 |
| rcarrillocruz | super hackish | 22:46 |
| rcarrillocruz | had noop the gather facts implicit role | 22:46 |
| rcarrillocruz | and bake the key on the bastions | 22:46 |
| jeblair | rcarrillocruz: but we have an idea of what we need to do so that's great! | 22:48 |
| jeblair | rcarrillocruz: i will celebrate your success by eating a large turkey | 22:48 |
| rcarrillocruz | ++ , enjoy thanksgiving :P | 22:49 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!