| corvus | mordred, tristanC: also, i suspect there may be an issue with the new depends syntax. i created a complex series here: https://review.openstack.org/#/q/status:open+topic:mailman-templates | 00:00 |
|---|---|---|
| tristanC | mordred: heh, unfortunately i'm not convinced this would work as per the pr comments. So I'm on hold regarding swagger doc generation | 00:00 |
| corvus | mordred, tristanC: and i'm getting an error a couple of changes into it. i only *just* got to the point where the change at the bottom of the stack is correct, so i haven't started digging into it yet. i've only just flagged it as a potential problem that needs investigating. | 00:00 |
| mordred | corvus, tristanC: OH - also, we need to blacklist sphinx 1.6.6 - openstack blacklisted it via constraints, but that doesn't apply to zuul | 00:01 |
| corvus | so, keep that in mind as you use depends-on urls for the next little bit | 00:01 |
| corvus | (i mean, as with any new feature, we should verify it's correct anyway :) | 00:01 |
| tristanC | mordred: the stack looks good to me, I'm curious what else could be added to the info endpoint. For example could we declare a "zuul-doc-url" in zuul.conf and then link it on the right side of the nav bar if present? | 00:01 |
| corvus | it'd be nice for zuul to self-host its documentation | 00:03 |
| mordred | tristanC: maybe? not sure... for the most part I think that, if we wind up accepting that endpoint we should still endeavor to minimize use of it | 00:03 |
| mordred | and what corvus said - I'd like for docs specifically to just be part of the standard web api, so hopefully we can just add them and not need any additional info exposed | 00:04 |
| mordred | corvus: nevermind my comments about sphinx 1.6.6 ... I was reading some errors incorrectly | 00:04 |
| tristanC | corvus: on https://review.openstack.org/#/c/535852/, it doesn't seems like 535851 was added to the zuul.items | 00:06 |
| corvus | tristanC: that would seem to confirm a problem :) | 00:06 |
| corvus | hrm, the git dependency is outdated... | 00:07 |
| corvus | maybe i should push a rebase | 00:07 |
| corvus | okay done | 00:08 |
| tristanC | corvus: shouldn't the scheduler fail early if a depends-on failed to merge? | 00:08 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Update docs to use sphinx-build and doc/requirements https://review.openstack.org/536881 | 00:08 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Use reno for release notes https://review.openstack.org/262597 | 00:08 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Switch to stestr https://review.openstack.org/536882 | 00:08 |
| corvus | tristanC: yeah -- the most recent failures there were due to the depends-on failing to merge | 00:08 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Remove --no-subunit-trace argument https://review.openstack.org/536883 | 00:08 |
| corvus | tristanC: actually, you may have been looking at old results -- i uploaded those before depends-on URL was in production | 00:09 |
| corvus | mordred: i generally like your zuul-web stack, but i'll review it in depth when i have more time. i did leave a quick question on the .json one. | 00:10 |
| tristanC | may i ask for review of https://review.openstack.org/#/c/535542/ and https://review.openstack.org/#/c/535576/ | 00:10 |
| tristanC | small patches to harden the executor | 00:11 |
| mordred | corvus: awesome. thanks - as long as you're generally cool with it, I'll deal with comments and see about getting unit tests added tomorrow | 00:11 |
| corvus | mordred: and is it okay if we continue to defer reno until after v3.0 release? though i would like it to be the first thing we merge after the v3.0 release. ie, i don't think we can encompass the v3.0 changes in release notes, but after 3.0, we should draw a hard line and do them for everything. | 00:11 |
| mordred | corvus: yah, totally | 00:12 |
| mordred | corvus: in fact, that stack is good to merge up to the one before the reno patch, and it should be fine sitting there untouched until post-release | 00:12 |
| mordred | corvus: there's a similar stack for nodepool, and same story I think | 00:13 |
| corvus | mordred: cool; i left an admin -2 on reno for now | 00:13 |
| mordred | cool. | 00:14 |
| mordred | tristanC: +2 on both | 00:16 |
| corvus | tristanC: can you elaborate on the get_mime issue? i'm not immediately seeing it. | 00:16 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Convert from legacy to native devstack job https://review.openstack.org/535899 | 00:17 |
| tristanC | corvus: there is a long history of memory corruption in libmagic that happens to be used by the get_mime option | 00:17 |
| corvus | tristanC: gotcha | 00:18 |
| corvus | both +3 | 00:19 |
| tristanC | sweet, thanks! | 00:19 |
| corvus | tristanC, mordred: oh hey, after that rebase, those changes are showing up with the correct dependencies on the status page. so that may have just been me botching it after all. | 00:20 |
| openstackgerrit | Merged openstack-infra/zuul master: ansible: honor command no_log module attribute https://review.openstack.org/535542 | 00:36 |
| *** sshnaidm is now known as sshnaidm|off | 00:38 | |
| openstackgerrit | Merged openstack-infra/zuul master: executor: block stat get_mime on localhost https://review.openstack.org/535576 | 00:38 |
| mordred | corvus: \o/ | 01:20 |
| *** elyezer has quit IRC | 01:57 | |
| *** elyezer has joined #zuul | 02:03 | |
| tristanC | fwiw, we just finished the migration to zuulv3, sf-project.io doesn't run zuulv2 anymore | 02:10 |
| dmsimard | tristanC: woot | 02:16 |
| *** harlowja has quit IRC | 02:20 | |
| *** Wei_Liu has joined #zuul | 02:21 | |
| tristanC | dmsimard: note that it includes the new job page, e.g. https://softwarefactory-project.io/zuul3/local/job.html?job_name=tox-py35 | 02:22 |
| mordred | tristanC: \o/ | 02:24 |
| *** yolanda has quit IRC | 02:30 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul master: Add support for dumping queues from a status.json file https://review.openstack.org/536622 | 02:42 |
| *** haint93 has joined #zuul | 02:55 | |
| *** haint_ has quit IRC | 02:58 | |
| *** jaianshu has joined #zuul | 04:11 | |
| *** rlandy has quit IRC | 04:25 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool master: openstack: convert rate to float https://review.openstack.org/537250 | 04:30 |
| tobiash | tristanC: the job page is cool :) | 05:05 |
| tobiash | tristanC: a minor but probably too complicated enhancement would be to sphinx render the descriptions | 05:06 |
| tristanC | tobiash: it helps to answer the "where is the job defined" question i kept on getting | 05:13 |
| tristanC | (in the job list, there is a context location button) | 05:13 |
| tristanC | also users can now query dynamically the jobs' variables | 05:14 |
| *** harlowja has joined #zuul | 05:19 | |
| *** elyezer has quit IRC | 05:53 | |
| *** elyezer has joined #zuul | 05:55 | |
| openstackgerrit | Merged openstack-infra/nodepool master: handler: fix support for handler without launch_manager https://review.openstack.org/535550 | 05:55 |
| *** threestrands has joined #zuul | 06:39 | |
| tobiash | tristanC: +2 with comment on https://review.openstack.org/#/c/537250/1 | 06:48 |
| *** threestrands has quit IRC | 06:49 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool master: openstack: convert rate to float https://review.openstack.org/537250 | 06:54 |
| AJaeger | Shrews: https://review.openstack.org/537284 redirects the zuulv3 docs | 07:20 |
| *** AJaeger has quit IRC | 08:03 | |
| *** AJaeger has joined #zuul | 08:10 | |
| *** jpena|off is now known as jpena | 08:45 | |
| *** jaianshu_ has joined #zuul | 09:03 | |
| *** jaianshu has quit IRC | 09:05 | |
| *** sshnaidm|off is now known as sshnaidm | 09:05 | |
| *** harlowja has quit IRC | 09:07 | |
| *** Wei_Liu has quit IRC | 09:10 | |
| *** Wei_Liu has joined #zuul | 09:10 | |
| *** electrofelix has joined #zuul | 10:04 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix builds queued forever after failure to get node request https://review.openstack.org/537335 | 10:10 |
| *** elyezer has quit IRC | 10:24 | |
| *** elyezer has joined #zuul | 10:25 | |
| *** jaianshu__ has joined #zuul | 10:29 | |
| *** jaianshu_ has quit IRC | 10:32 | |
| *** elyezer has quit IRC | 10:40 | |
| *** elyezer has joined #zuul | 10:43 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] zuul autohold: allow filtering per commit https://review.openstack.org/536993 | 10:46 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: zuul autohold: allow filtering per commit https://review.openstack.org/536993 | 11:20 |
| *** elyezer has quit IRC | 11:28 | |
| *** elyezer has joined #zuul | 11:32 | |
| *** jkilpatr has quit IRC | 11:33 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Update docs to use sphinx-build and doc/requirements https://review.openstack.org/536881 | 11:52 |
| *** jkilpatr has joined #zuul | 11:55 | |
| *** jaianshu__ has quit IRC | 11:56 | |
| *** Wei_Liu has quit IRC | 11:57 | |
| dmsimard | tobiash: do you really need to use sphinx if all you want to do is render rst ? | 12:03 |
| dmsimard | tobiash: I mean, the description formatting doesn't have to be heavy with full sphinx | 12:04 |
| dmsimard | I don't have a strong opinion, just wondering | 12:04 |
| tristanC | not sure how to prevent local file include from rst rendering too | 12:07 |
| Shrews | AJaeger: ah, excellent | 12:18 |
| Shrews | mordred: If you get a spare moment, I got the native devstack jobs for nodepool to run (https://review.openstack.org/535899) but I'm not quite sure what to look for in the -odepool-functional-py35-src job to make sure that it's using the git versions of the required-projects. | 12:29 |
| Shrews | I would have thought searching for 'siblings' in http://logs.openstack.org/99/535899/10/check/nodepool-functional-py35-src/e261150/job-output.txt.gz would reveal something, but that term does not appear | 12:33 |
| Shrews | oh, this isn't actually a tox job. hrm | 12:35 |
| *** jpena is now known as jpena|lunch | 12:36 | |
| mordred | Shrews: so - http://paste.openstack.org/show/652002 | 12:44 |
| mordred | Shrews: I think you need to add the LIBS_FROM_GIT line to ge the devstack machinery to do the right thing with the install - but then the pbr freeze line will print out the versions of things installed in the nodepool venv | 12:45 |
| *** elyezer has quit IRC | 12:49 | |
| *** elyezer has joined #zuul | 12:50 | |
| Shrews | mordred: *nod* thx | 12:52 |
| tobiash | dmsimard: I don't know if sphinx would be necessary for that but we have zuul markup (a sphinx plugin) in the descriptions | 12:53 |
| mordred | tobiash, dmsimard: yah - we've talked about leveraging sphinx so that we can get the zuul-sphinx markup and also the links between things. that said - we've also talked about potentially needing to do something else | 12:56 |
| mordred | tobiash, dmsimard, tristanC: because having zuul itself run sphinx to get the content might be a bit too heavy - tough to say | 12:57 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Convert from legacy to native devstack job https://review.openstack.org/535899 | 12:58 |
| mordred | otoh - we've got the merger/executor farm and we already have it read content from the repos via the gearman jobs - so it might be possible to get clever and have those jobs also render the rst in some manner when they read the config | 12:59 |
| Shrews | mordred: i think 'vars' needs to be 'devstack_localrc', but you got me on the right track i think. thx | 12:59 |
| mordred | Shrews: yup. you are totally right | 13:00 |
| Shrews | how rare is that??? | 13:00 |
| mordred | :) | 13:00 |
| Shrews | pabelanger: at some point, i'd like to pick your brain on all of the image pause vars in our tests (e.g., NODEPOOL_PAUSE_CENTOS_7_DIB). i'm not quite sure how those are supposed to be used b/c right now, none of them except for trusty are ever going to be unpaused | 13:02 |
| mordred | tristanC: perhaps we should add $something to indicate a browseable location - would probably have to be in the connection config - so that you could make "zuul-jobs/zuul.yaml@master" be a link to the source | 13:03 |
| mordred | Shrews: speaking of dib things - I think I saw a dib patch for removing fedora 25 - so we should probably go ahead and remove the fedora 25 entries in there | 13:03 |
| Shrews | pabelanger: for example, see the old code in https://review.openstack.org/#/c/535899/10/playbooks/nodepool-functional-py35-src/run.yaml ... elif [ "" == "-debian" ] will never equate to true | 13:04 |
| Shrews | mordred: i think all of that needs reworking anyway, but one thing at a time | 13:04 |
| Shrews | (see the above brokenness as an example) | 13:05 |
| mordred | Shrews: ++ | 13:05 |
| tristanC | mordred: would that be added to the info endpoint? | 13:06 |
| mordred | tristanC: I think it might have to be added in from connection_config somehow in the jobs endpoint itself | 13:07 |
| mordred | tristanC: since the url could be different - like in openstack's zuul, jobs defined in ansible/ansile should have the link be https://github.com/ansible/ansible but jobs defined in zuul should link to https://git.openstack.org/cgit/openstack-infra/zuul | 13:08 |
| mordred | tristanC: (my first thought was info) | 13:09 |
| tristanC | mordred: git browseable url per connection in zuul.conf, and /info endpoint so that webpage knows how to create href link, perhaps using {project-name} substitution | 13:11 |
| mordred | tristanC: I think if we had the zuul.conf setting, you should be able to add the browseable url in the list here: https://review.openstack.org/#/c/535545/1/zuul/rpclistener.py | 13:15 |
| mordred | tristanC: and yah, we'd almost certainly need to have some substitutions defined to be able to put links in the zuul.conf | 13:16 |
| tristanC | mordred: also, i'm thinking about adding a "project" page to display the pipeline configuration and a graph for job dependencies | 13:19 |
| tobiash | yay, I have a zuul bug which is triggered by another zuul bug which is triggered by an edge case... | 13:23 |
| tobiash | that chain leads to a gate ignoring its trigger | 13:24 |
| mordred | tristanC: nice | 13:24 |
| mordred | tobiash: oh - btw - I've got a fix for the test failure in your github zuul-web patch locally | 13:24 |
| tobiash | mordred: you mean the remove webapp change? | 13:25 |
| mordred | yah | 13:25 |
| tobiash | ah cool | 13:25 |
| tobiash | feel free to amend my change | 13:26 |
| mordred | awesome. fix coming | 13:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Remove webapp https://review.openstack.org/536780 | 13:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Register .json routes with and without suffix https://review.openstack.org/537010 | 13:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Add /info and /{tenant}/info route to zuul-web https://review.openstack.org/537011 | 13:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Add support for configuring graphite_url https://review.openstack.org/537012 | 13:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Move WebInfo config processing into zuul.model https://review.openstack.org/537013 | 13:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Add facility for plugins to register web routes https://review.openstack.org/537014 | 13:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Update docs to use sphinx-build https://review.openstack.org/536881 | 13:31 |
| *** jpena|lunch is now known as jpena | 13:38 | |
| *** andreaf has joined #zuul | 13:42 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: zuul autohold: allow filtering per commit https://review.openstack.org/536993 | 13:42 |
| *** elyezer has quit IRC | 13:43 | |
| *** rlandy has joined #zuul | 13:44 | |
| Shrews | I've gone back a couple of years in the git history of nodepool... I can't see that this support for testing multiple images has ever actually been used in that time period. Did we perhaps have separate jobs for per image at one point? I think we're just carrying around unused fluff at this point. | 13:45 |
| *** elyezer has joined #zuul | 13:46 | |
| Shrews | that would make the comments in the 'if' section of http://git.openstack.org/cgit/openstack-infra/nodepool/tree/playbooks/nodepool-functional-py35-src/run.yaml#n87 at least sensible | 13:49 |
| tristanC | my blog post just got published: http://rdoproject.org/blog/2018/01/CICD-workflow-with-zuul/ | 13:50 |
| clarkb | Shrews: ya we did an ubuntu job and a fedora job and so on iirc | 13:58 |
| clarkb | Shrews: and paused the rest of the images that werent being tested | 13:59 |
| *** weshay|rover is now known as weshay|mtg | 14:00 | |
| Shrews | clarkb: ok, that explains it then. we should decide to either bring back those jobs or remove this unused code. i vote for the latter since we've been running that way for so long anyway | 14:03 |
| clarkb | Shrews: I think pabelanger did the original setup his feedback would be good | 14:04 |
| clarkb | I think dib may use the tests too? | 14:05 |
| Shrews | ++ | 14:05 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Convert from legacy to native devstack job https://review.openstack.org/535899 | 14:16 |
| mordred | Shrews: oh - right. what clarkb said | 14:28 |
| mordred | Shrews: the legacy job you're starting from has some old jjb variables already expanded | 14:28 |
| mordred | Shrews: dib has legacy nodepool dsvm jobs too: legacy-dsvm-nodepool-redhat-src, legacy-dsvm-nodepool-ubuntu-src, legacy-dsvm-nodepool-opensuse-src, legacy-dsvm-nodepool-debian-src | 14:30 |
| *** yolanda has joined #zuul | 14:32 | |
| mordred | Shrews: so I think maybe we should add a job variable that defaults to 'ubuntu' - then we can add nodepool-functional-py35-redhat-src that has parent nodepool-functional-py35-src and just overrides whatever the job variable is | 14:32 |
| mordred | and then we can update the dib pipeline config to use the new nodepool jobs too | 14:32 |
| mordred | Shrews: just left a few inline comments related to ^^ on the change | 14:39 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Convert from legacy to native devstack job https://review.openstack.org/535899 | 14:39 |
| mordred | they're on PS12 though | 14:40 |
| Shrews | mordred: ++ | 14:41 |
| Shrews | mordred: that new var would likely go on the base job though, but i see what you're driving at | 14:42 |
| Shrews | me likey | 14:42 |
| Shrews | i might do that separately though | 14:44 |
| mordred | coolio | 14:47 |
| corvus | Shrews: nodepool only uses one image. the other images are there for the dib jobs. | 14:50 |
| Shrews | corvus: yep, mordred and clarkb clued me in to that | 14:51 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Support DIB image selection in functional jobs https://review.openstack.org/537408 | 15:00 |
| Shrews | mordred: something like that ^^^ | 15:00 |
| Shrews | assuming that works | 15:00 |
| mordred | Shrews: yah. *waves hands* | 15:04 |
| pabelanger | Shrews: yah, I noticed that last night about variables my sell in nodepool-dsvm jobs. I was going to propose a few patches to see how to fix that, we mostly us it from glean and diskimage-builder | 15:05 |
| Shrews | pabelanger: i think 537408 is going to make that easy for the other repos | 15:06 |
| pabelanger | cool, I'll look today | 15:06 |
| Shrews | clarkb: another test_failed_provider flap seen today http://logs.openstack.org/99/535899/10/check/tox-cover/d8cd6e0/testr_results.html.gz | 15:07 |
| Shrews | haven't had time to look myself yet | 15:07 |
| *** sshnaidm is now known as sshnaidm|bbl | 15:19 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix builds queued forever after failure to get node request https://review.openstack.org/537335 | 15:20 |
| *** sshnaidm|bbl has quit IRC | 15:23 | |
| *** Guest28399 is now known as mgagne | 15:33 | |
| *** mgagne has joined #zuul | 15:33 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: zuul autohold: allow filtering per commit https://review.openstack.org/536993 | 15:35 |
| *** yolanda has quit IRC | 15:37 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix cross source dependencies for github https://review.openstack.org/537428 | 15:44 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix runtime stats reporting for noop job https://review.openstack.org/537432 | 15:51 |
| *** sshnaidm|bbl has joined #zuul | 16:03 | |
| *** bhavik1 has joined #zuul | 16:12 | |
| Shrews | I wish my brain could think as fast as my fingers typed | 16:15 |
| *** sshnaidm|bbl is now known as sshnaidm | 16:17 | |
| *** bhavik1 has quit IRC | 16:19 | |
| mordred | Shrews: good luck with that | 16:21 |
| openstackgerrit | Merged openstack-infra/nodepool master: Add a plugin interface for drivers https://review.openstack.org/535551 | 16:41 |
| openstackgerrit | Merged openstack-infra/nodepool master: builder: do not cleanup image for driver not managing image https://review.openstack.org/535552 | 16:42 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Convert from legacy to native devstack job https://review.openstack.org/535899 | 16:45 |
| corvus | mordred: i'm wondering if we could use the override-checkout feauture to address the job branch problem. so in the ansible case, when we attach the shade-integration job, we would say in the project-pipeline variant: "required-projects: - {name: openstack-infra/shade, override-checkout: master}, {name: openstack-dev/devstack, override-checkout: master}". or you could put the devstack override on the shade | 16:46 |
| corvus | job. | 16:46 |
| corvus | mordred: it's a little more verbose, but it's also more specific (as it lets you set that mapping for individual projects in very specific conditions). it ties two things together which perhaps don't need to be, otoh, maybe they should be tied together. if you're specifically checking out the 'master' branch of some repo, seems like maybe you want the master branch versions of any jobs defined in that repo? | 16:48 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fall back on parent jobs from default branch https://review.openstack.org/536998 | 16:50 |
| mordred | corvus: hrm. | 16:53 |
| mordred | corvus: let me write a quick patch to see if I understand what you're saying | 16:53 |
| mordred | corvus: like this? https://review.openstack.org/536159 Add devel branches and override-checkout for ansible-devel job | 16:57 |
| corvus | mordred: yep. (or in a similar case, where you don't have a convenient job to modify, you could do the same in the project-pipeline config of the job) | 17:02 |
| corvus | mordred: also, fyi, branches takes a list, so you don't have to regex that | 17:02 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: WIP: Fall back on default branch for jobs https://review.openstack.org/537457 | 17:05 |
| mordred | corvus: yah. I did branches as a list the first time, and then tried switching to regex while I was trying to figure out why it wasn't matching | 17:07 |
| corvus | ack | 17:07 |
| mordred | corvus: I should put it back to list | 17:07 |
| corvus | mordred: how does that approach feel to you? | 17:07 |
| mordred | feels totally fine- I think being explicit about which branches we want in this job is a good thing | 17:12 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Add facility for plugins to register web routes https://review.openstack.org/537014 | 17:13 |
| mordred | corvus: that doesn't solve the whole thing though, right? like, we still need something in zuul to be able to find the devstack parent job definition from a devel patch? | 17:16 |
| corvus | mordred: oh sorry, what i'm mulling is that we use this for that as well. so not only does this mean "checkout shade master on ansible devel patches" but it also means "when looking for jobs defined in the shade repo, pretend this is a change to master" | 17:21 |
| corvus | it *sounds* like they shouldn't be related if you think about the implementation, but in all the cases i can think of in practice, you probably want both things. also, if you had a slightly incorrect but entirely reasonable mental-model of zuul, you might think that if you told zuul to checkout the master branch of devstack, you would also be telling it to use the jobs which are defined there. | 17:23 |
| mordred | corvus: ah - yes. this now all makes sense to me | 17:26 |
| mordred | corvus: and yes, I agree -I think that that patch above describes a state of the world I'm desiring and zuul taking those actions I think makes a good amount of sense | 17:27 |
| mordred | corvus: that said - idea for a variation on your 'fall back on default branch' patch ... what if instead of falling back to default branch on not matching- we added a special case for matching default branches/HEAD ... that is, when considering branch matches, we would consider ansible/ansible@devel and openstack-dev/devstack@master to match because devel is HEAD of ansible/ansible and master is HEAD of | 17:31 |
| mordred | openstack-dev/devstack | 17:31 |
| mordred | (might be a terrible idea - mostly just thinking out loud) | 17:32 |
| *** myoung is now known as myoung|lunch | 17:34 | |
| *** yolanda has joined #zuul | 17:35 | |
| corvus | mordred: yeah, i think that's a contender. zuul's default-branch is basically intended to mean "HEAD", so with your idea, we could update the implied branch matchers to say "{master OR default-branch}" in the case that master is the default branch of that repo, and we set a flag on the change which says "this change is for devel, but devel is the default-branch of this repo" and that flag would cause the | 17:38 |
| corvus | branch matcher to return true. | 17:38 |
| corvus | mordred: that is probably simpler to implement than any of the other ideas so far :) | 17:39 |
| corvus | it's even more of a special case though. i think it's a really good match for the ansible case, it lets everything sort of slot in very nicely. i don't think it would have solved the nodepool feature/zuulv3 case, where we wanted to run a devstack job on feature/zuulv3, which isn't the default branch. | 17:41 |
| mordred | corvus: yes. I wholeheartedly agree | 17:41 |
| *** xinliang has quit IRC | 18:00 | |
| *** myoung|lunch is now known as myoung | 18:04 | |
| *** xinliang has joined #zuul | 18:04 | |
| *** xinliang has quit IRC | 18:04 | |
| *** xinliang has joined #zuul | 18:04 | |
| *** jpena is now known as jpena|off | 18:12 | |
| *** electrofelix has quit IRC | 18:32 | |
| tobiash | mordred: is there a reason why nodepool wants spinx < 1.6? | 18:59 |
| mordred | tobiash: not anything explicit that I'm aware of | 19:00 |
| mordred | tobiash: zuul has the same line - would be worth trying and seeing if later will work | 19:01 |
| pabelanger | +1 for uncapping | 19:02 |
| pabelanger | any change we could get a few reviews on https://review.openstack.org/535817/ would be good to use that now that we lost infracloud. will help clean up zookeeper entries | 19:03 |
| corvus | Shrews: what do you think about splitting 'erase' into a new command? it feel weird to have "info --erase" :) | 19:05 |
| corvus | (but also +2 on that as is) | 19:06 |
| tobiash | +1 for erase command, I had the same feeling during review but didn't want to block this because of that | 19:08 |
| mordred | corvus, tobiash, pabelanger: it has 3x +2 - shall we land it but come back as a follow up and shift it to its own command? | 19:13 |
| tobiash | Fine for me | 19:13 |
| tobiash | Just curious while we're at nodepool and lost clouds. Are there plans to use different sized nodes to improve resource utilization? | 19:15 |
| mordred | I think now that we have the better quota stuff it's a thing we can think about | 19:17 |
| mordred | I don't know that we've made any specific plans to do so in infra yet -- but there are definitely a host of jobs (pep8, sphinx, tarball generation) that would do fine on substantially smaller instances | 19:18 |
| tobiash | We use 2 core nodes by default and bigger ones where needed | 19:21 |
| pabelanger | When I lasted talked with clarkb about it, I think we agreed we might be limited by ipv4 addresses in some clouds, even if we could get more instances online | 19:21 |
| pabelanger | I guess another reason to get ipv6 working on all OSes :) | 19:21 |
| mordred | pabelanger: and on all clouds :) | 19:22 |
| *** jkilpatr has quit IRC | 19:22 | |
| pabelanger | I was thinking about hacking on glean again, to atleast get ipv6 working on centos / fedora | 19:24 |
| pabelanger | and know we still want to try getting it working again on citycloud | 19:24 |
| pabelanger | but haven't found the time | 19:24 |
| mordred | pabelanger: yah. tell me about it | 19:25 |
| corvus | mordred: ya i think we can land it | 19:26 |
| corvus | which you've done :) | 19:26 |
| Shrews | I can split it into another command if we want that. | 19:33 |
| *** jkilpatr has joined #zuul | 19:37 | |
| *** harlowja has joined #zuul | 19:38 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Add provider info command https://review.openstack.org/535817 | 19:39 |
| Shrews | Anyone feel like approving a very easy review that fixes a spurious test failure? https://review.openstack.org/536540 | 20:11 |
| Shrews | I will give you a cookie in exchange | 20:12 |
| pabelanger | cookies! | 20:13 |
| pabelanger | +3 | 20:13 |
| jlk | oh geez | 20:14 |
| pabelanger | double +3 | 20:14 |
| Shrews | wow | 20:14 |
| jlk | we all raced each other | 20:14 |
| Shrews | i'm going to offer cookies more often! | 20:14 |
| pabelanger | triple | 20:14 |
| jlk | WORKFLOW INTENSIFIES | 20:14 |
| pabelanger | nice | 20:14 |
| jlk | Shrews: are you going to be at PTG? | 20:14 |
| Shrews | jlk: unfortunately, not this time | 20:14 |
| jlk | then I shall not accept cookie delivery there | 20:15 |
| *** dtruong has joined #zuul | 20:20 | |
| mordred | while people are in the mood for easy reviews: https://review.openstack.org/#/c/536636/ and https://review.openstack.org/#/c/536854/ are a couple of cleanup patches | 20:21 |
| mordred | unfortunately I do not have any cookies to offer, but can probably find booze somewhere | 20:22 |
| pabelanger | cookies and booze, this day is getting better all the time | 20:24 |
| jlk | cookies and booze is the name of my free association jazz jam band. | 20:24 |
| Shrews | corvus: i went ahead and asked a question in your Depends-On email that I think most folks are going to wonder about. | 20:25 |
| openstackgerrit | Merged openstack-infra/nodepool master: Fix race in test_provider_removal https://review.openstack.org/536540 | 20:31 |
| mordred | Shrews: I have responded to your email to the list | 20:31 |
| pabelanger | speaking of new syntax, when I click on new URL depends on in gerritUI, it opens in a new tab. Not sure if that is my browser or not. But old depends-on would just stay in current tab | 20:33 |
| pabelanger | but, that is not a zuul issue | 20:33 |
| Shrews | mordred: i acknowledge your response and raise you one "thank you" | 20:33 |
| mordred | pabelanger: I experience the same behavior | 20:35 |
| mordred | pabelanger: although I am finding it preferrable so far - since the old behavior did a search then a redirect, so it was always extra effort to double-back-button to get to the original change | 20:35 |
| mordred | pabelanger: but it's also weird | 20:36 |
| mordred | because habits :) | 20:36 |
| pabelanger | ah, I think I see why | 20:37 |
| pabelanger | <a href="https://review.openstack.org/536461/" target="_blank" rel="nofollow">https://review.openstack.org/536461/</a> | 20:37 |
| pabelanger | new syntax | 20:37 |
| pabelanger | old syntax doesn't have https://, just / | 20:38 |
| mordred | pabelanger: so I guess gerrit's behavior is to put discovered links in with target="_blank" rel="nofollow" ? | 20:39 |
| pabelanger | <a href="/#/q/Icf22643f17ea0500ceef38f493bd98d49bd9a3bd">Icf22643f17ea0500ceef38f493bd98d49bd9a3bd</a> is old | 20:39 |
| pabelanger | mordred: guess so, would need to look at JS | 20:40 |
| openstackgerrit | Merged openstack-infra/nodepool master: Remove shade_inner_exceptions https://review.openstack.org/536636 | 20:40 |
| openstackgerrit | Merged openstack-infra/nodepool master: Set app_name and app_version https://review.openstack.org/536854 | 20:40 |
| pabelanger | but, also happy to try new method for a day or so | 20:40 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Use override-checkout to select jobs https://review.openstack.org/537655 | 21:06 |
| corvus | mordred: that's that option | 21:06 |
| corvus | if we feel like overloading override-checkout like that is weird, it'd be pretty easy to split that out into another attr | 21:07 |
| *** abadger1999 has quit IRC | 21:23 | |
| *** abadger2000 has joined #zuul | 21:23 | |
| *** abadger2000 is now known as abadger1999 | 21:24 | |
| corvus | mordred, Shrews: i also replied to myself again on the ML with a further analysis of the implications of that; let me know what you think. | 21:27 |
| *** weshay|mtg is now known as weshay|rover | 21:34 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Split out erasing from 'info' command into 'erase' https://review.openstack.org/537663 | 21:36 |
| Shrews | corvus: ack. fyi, there ^ is the requested command split | 21:36 |
| Shrews | pabelanger: fyi for you too ^^ | 21:39 |
| * Shrews is have a "Dude, where's my build?" moment for one of his nodepool changes | 21:40 | |
| Shrews | is having* | 21:40 |
| *** threestrands has joined #zuul | 21:41 | |
| *** sshnaidm has quit IRC | 21:42 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Support DIB image selection in functional jobs https://review.openstack.org/537408 | 21:44 |
| *** threestrands_ has joined #zuul | 21:44 | |
| *** threestrands_ has quit IRC | 21:44 | |
| *** threestrands_ has joined #zuul | 21:44 | |
| Shrews | mordred: \o/... got the -src nodepool job correctly using the tips of the other repos. thx again | 21:47 |
| *** threestrands has quit IRC | 21:47 | |
| corvus | 21:49 < openstackgerrit> James E. Blair proposed openstack-infra/project-config master: Add zuul-website repo https://review.openstack.org/537670 | 21:49 |
| corvus | fyi ^ | 21:49 |
| pabelanger | great | 21:56 |
| pabelanger | corvus: left question | 21:59 |
| corvus | pabelanger: yep, forgot git add, thx | 22:02 |
| pabelanger | +2 | 22:04 |
| Shrews | corvus: mordred: Why is it necessary to specify override-checkout for shade and devstack in the required-projects bit in https://review.openstack.org/#/c/536159/3/.zuul.yaml ? | 22:07 |
| Shrews | oh, that's parented to something different there. those job names are long and all similar :) | 22:09 |
| Shrews | i thought the parent was doing the override-checkout too, but not so much | 22:10 |
| Shrews | oh, but one of the parents eventually does it too. hrm. | 22:11 |
| Shrews | oy | 22:12 |
| Shrews | congrats. this has hurt my brain | 22:12 |
| mordred | Shrews: woot! | 22:23 |
| mordred | corvus: I don't think it seems like an overloading of that config-item - I think most of the time people will be thinking about what branch of a repo they want and will just expect jobs to flow from that naturally | 22:25 |
| mordred | corvus: but other people might have different thoughts :) | 22:25 |
| *** sshnaidm has joined #zuul | 22:31 | |
| pabelanger | seeing in fingergw.log: http://paste.openstack.org/show/652851/ | 22:36 |
| pabelanger | suspect we need to add some additional logging | 22:36 |
| pabelanger | but, also possible we have hostname issue again on an executor | 22:37 |
| pabelanger | I don't think we have fixed that properly yet | 22:37 |
| pabelanger | I think we also talked about just using IP addresses | 22:37 |
| mordred | pabelanger: I'd honestly love it if we could fix the hostname issue once and for all | 22:42 |
| mordred | pabelanger: I mean, it seems like a thing we should have the powers to do | 22:42 |
| mordred | (also, a node does not necessarily know its IP address - such as in the case of floating ips - so an executor can't just report it's IP back) | 22:43 |
| pabelanger | mordred: agree, I am going to spend some time tomorrow looking into it. I know there was some discussion about how it would work, will read up tonight | 22:43 |
| mordred | pabelanger: I just did ansible 'ze*' -m shell -a 'hostname --fqdn' | 22:43 |
| pabelanger | I think we might be missing search openstack.org in our /etc/resolv.conf files, because when we get ze08 for hostname, we don't know what domain to add | 22:44 |
| mordred | pabelanger: and everything is showing as having the correct hostname right now | 22:44 |
| pabelanger | good | 22:44 |
| mordred | pabelanger: ansible 'ze*' -m shell -a 'hostname' on the other hand returns ze09 for ze09, and fully qualified for everythig else | 22:44 |
| pabelanger | yah, think ze09 is right, and other wrong? | 22:45 |
| pabelanger | or is it the other way around | 22:45 |
| pabelanger | $ cat /etc/resolv.conf | 22:45 |
| pabelanger | nameserver 127.0.0.1 | 22:45 |
| pabelanger | that is on zuul01.o.o | 22:45 |
| pabelanger | so, if we did get ze09, I don't think we'd append openstack.org to it | 22:46 |
| pabelanger | due to missing search command | 22:46 |
| pabelanger | ansible 'ze*' -m shell -a 'netstat -na | grep \:7900' | 22:49 |
| pabelanger | is also good | 22:49 |
| corvus | pabelanger: i believe we agreed that we just need to fix the hostnames on most of the executors | 22:54 |
| corvus | also, this is an -infra discussion :) | 22:54 |
| pabelanger | +1 | 22:55 |
| mordred | corvus: reading your email, Ithink I've imagined another weird edge-case | 23:35 |
| corvus | mordred: good, i like imagining them ahead of time | 23:36 |
| mordred | corvus: what if ansible/ansible wanted to define a job, "ansible-tox-py27" with parent: tox-py27 | 23:36 |
| corvus | mordred: tox-py27 is in an unbranched repo, so its single definition matches all branches | 23:37 |
| mordred | AH. gotcha | 23:37 |
| mordred | k. well - just for sake of argument - what if they wanted to make an 'ansible-functional-devstack' parent: devstack | 23:38 |
| mordred | would they do override-checkout: master on the job and then required-projects: ansible: override-checkout: devel ? | 23:38 |
| mordred | (this is a pathological example - just poking to see if I undersatnd the things) | 23:38 |
| corvus | mordred: i'd probably just do required-projects: devstack, override-branch: master -- basically like your shade job | 23:39 |
| corvus | mordred: assuming that's defined on ansible/ansible@devel, then it's basically "if you want to run the ansible-functional-devstack job on the devel branch, here is the devstack branch you need to use with it" | 23:40 |
| corvus | mordred: oh, but i'm guessing you're getting at: what if they wanted to write that job for other folks to use | 23:40 |
| corvus | like, make it available for openstack projects | 23:40 |
| *** rlandy is now known as rlandy|bbl | 23:41 | |
| mordred | yah. | 23:41 |
| corvus | lemme work through that :) | 23:41 |
| mordred | to be fair though - I think an answer could be "make an ansible-zuul-jobs repo with a master branch and nothing else" | 23:41 |
| corvus | mordred: yes, or set an explicit branch matcher for all branches, or turn off implied branch matchers with the pragma, or pragma an implied branch matcher of devel+master ... that solves the issue of mapping ansible's devel branch to, say, kolla's master. but as long as "make a change in the devstack repo" is not an option, we'd still probably have the underlying problem of mapping ansible's devel to a | 23:43 |
| corvus | devstack branch for the parent:devstack bit. | 23:43 |
| mordred | yah | 23:44 |
| corvus | mordred: https://etherpad.openstack.org/p/OM4LTzKVXQ | 23:46 |
| corvus | mordred: that would let that job work on the devel branch of ansible, and the master branch of kolla unchanged. | 23:47 |
| corvus | mordred: what you can't do at that point though, is run that job on the stable/pike branch of kolla | 23:47 |
| corvus | mordred: but i think a big reason for that is -- what does it mean to do that? | 23:48 |
| corvus | mordred: i've added a section at the bottom for one way to make that work for kolla@stable/pike. but that's basically saying "we're going to use the stable/pike devstack branch, which somehow works with the devel branch of ansible". | 23:49 |
| corvus | mordred: if that's a sensible combination, then that works. :) | 23:49 |
| mordred | corvus: yah. that works and makes sense even | 23:50 |
| corvus | i'm pretty sure the change as written supports all of this | 23:50 |
| corvus | mordred: my guess is that for bidirectional jobs, the required-projects form is going to be better suited. for unidirectional jobs, like the nodepool feature/zuulv3 issue, the job.override-branch form would be easier and more suitable. | 23:51 |
| mordred | ++ | 23:55 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!