| SpamapS | I think we may need to use the lower level non-react-aware library to do it right. | 00:06 |
|---|---|---|
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: connections: always register the scheduler https://review.openstack.org/637444 | 00:37 |
| *** ruffian_sheep has joined #zuul | 01:17 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: connections: always register the scheduler https://review.openstack.org/637444 | 01:20 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: connections: remove unused getSchema procedure https://review.openstack.org/637445 | 01:46 |
| *** sdake has joined #zuul | 02:00 | |
| *** sdake has quit IRC | 02:18 | |
| *** sdake has joined #zuul | 02:21 | |
| *** sdake has quit IRC | 02:44 | |
| ruffian_sheep | Can anyone tell me what is the reason? http://paste.openstack.org/show/745241/ | 03:22 |
| ruffian_sheep | I can use the cmd :git clone ssh://hjy@review.openstack.org:29418/openstack-dev/sandbox /var/lib/zuul/executor-git/review.openstack.org/openstack-dev/sandbox to clone . | 03:22 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: fixtures: use default _cleanUp procedure https://review.openstack.org/637449 | 03:23 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: config: add playbooks to job.toDict() https://review.openstack.org/621343 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Add API endpoint to get frozen jobs https://review.openstack.org/607077 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Get executor job params https://review.openstack.org/607078 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Separate out executor server from runner https://review.openstack.org/607079 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: implement prep-workspace https://review.openstack.org/607082 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: web: add /connections route https://review.openstack.org/631703 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: use connections endpoint for prepare-workspace https://review.openstack.org/631704 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add execute sub-command https://review.openstack.org/630944 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add support for depends-on https://review.openstack.org/632064 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add quick-start integration test https://review.openstack.org/635701 | 04:37 |
| ruffian_sheep | I meet some probelm can anyone help me ? The log showed me taht git.exc.GitCommandError: Cmd('git') failed due to: exit code(-13) | 05:55 |
| *** chandankumar is now known as chkumar|ruck | 06:17 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 06:20 |
| tobiash | ruffian_sheep: maybe you miss the sshkey config in zuul.conf: https://zuul-ci.org/docs/zuul/admin/drivers/gerrit.html#attr-%3Cgerrit%20connection%3E.sshkey | 06:21 |
| ruffian_sheep | tobiash:T have set it in the zuul.conf. http://paste.openstack.org/show/745241/ | 06:34 |
| *** sdake has joined #zuul | 06:38 | |
| ruffian_sheep | tobiash:Just like you answer the baiyi.I can git clone this by myself.But it cannot be cloned in this way.I have checked that I set the ssh public key in https://review.openstack.org/#/settings/ssh-keys . | 06:40 |
| ruffian_sheep | tobiash:I read the log you told with baiyi in the link :eavesdrop.openstack.org/irclogs/%23zuul/%23zuul.2017-11-30.log.html | 06:41 |
| ruffian_sheep | tobiash:I also checked the permission of the files.The path /var/lib/zuul/executor-git/ and the file in the document is all belong to zuul:zuul. | 06:43 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /connections route https://review.openstack.org/631703 | 06:46 |
| *** AJaeger has quit IRC | 06:55 | |
| *** AJaeger has joined #zuul | 07:01 | |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Add new merger job to get role definitions from a repository https://review.openstack.org/637181 | 07:04 |
| *** sanjayu__ has joined #zuul | 07:17 | |
| *** quiquell|off is now known as quiquell|rover | 07:23 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 07:24 |
| *** sdake has quit IRC | 07:30 | |
| *** quiquell|rover is now known as quiquell|rover|b | 07:36 | |
| *** pcaruana has joined #zuul | 07:43 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 07:48 |
| *** sshnaidm|off is now known as sshnaidm | 07:56 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 08:02 |
| *** quiquell|rover|b is now known as quiquell|rovee | 08:05 | |
| *** quiquell|rovee is now known as quiquell|rover | 08:05 | |
| *** panda|off is now known as panda | 08:09 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 08:12 |
| *** badboy has joined #zuul | 08:20 | |
| *** gtema has joined #zuul | 08:22 | |
| *** themroc has joined #zuul | 08:23 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 08:43 |
| zbr | SpamapS: thanks for the heads on ANSI, let me know if I can help with it. maybe you can share with me an easy way to test/benchmark it. | 08:47 |
| *** jpena|off is now known as jpena | 08:52 | |
| *** ruffian_sheep has quit IRC | 08:52 | |
| *** pcaruana|afk| has joined #zuul | 09:01 | |
| quiquell|rover | tobiash, tristanC: Do you know if zuul will override "files" section if job is run in periodic trigger ? | 09:02 |
| *** pcaruana has quit IRC | 09:02 | |
| tobiash | quiquell|rover: hrm, I guess it won't match | 09:02 |
| quiquell|rover | tobiash: feels like a bug | 09:03 |
| quiquell|rover | tobiash: I suppose periodic has to ignore files | 09:03 |
| tobiash | quiquell|rover: maybe, let's discuss this when corvus is around | 09:03 |
| quiquell|rover | tobiash: going to reproduce | 09:04 |
| quiquell|rover | jpena: ^ | 09:08 |
| quiquell|rover | tobiash: files: [] can help workaround this ? | 09:12 |
| *** sdake has joined #zuul | 09:14 | |
| zbr | zuul console uses websockets and does not work behind squid. does anyone knows how to configure squid to make it work? | 09:15 |
| zbr | for the moment i will just add zuul.openstack.org to the exclude list for proxy config. | 09:20 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 09:20 |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: wip: Jaeger tracing PoC https://review.openstack.org/637505 | 09:42 |
| openstackgerrit | Simon Westphahl proposed openstack-infra/nodepool master: wip: Jaeger tracing PoC https://review.openstack.org/637506 | 09:42 |
| *** electrofelix has joined #zuul | 09:42 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 09:47 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool master: WIP: Second try to fix single cloud config reload https://review.openstack.org/608695 | 09:50 |
| *** pcaruana|afk| has quit IRC | 10:01 | |
| *** pcaruana has joined #zuul | 10:07 | |
| openstackgerrit | Jakub Bielecki proposed openstack-infra/nodepool master: doc bugfix for static provider https://review.openstack.org/637518 | 10:32 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: URLTrigger driver time based - artifact change jobs triggering driver https://review.openstack.org/635567 | 11:05 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: URLTrigger driver time based - artifact change jobs triggering driver https://review.openstack.org/635567 | 11:06 |
| *** hashar has joined #zuul | 11:12 | |
| *** hashar is now known as hasharAway | 11:12 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: URLTrigger driver time based - artifact change jobs triggering driver https://review.openstack.org/635567 | 11:13 |
| *** sdake has quit IRC | 11:16 | |
| *** sdake has joined #zuul | 11:20 | |
| *** goern has joined #zuul | 11:31 | |
| *** jpena is now known as jpena|brb | 11:42 | |
| quiquell|rover | tobiash: confirmed at local zuul | 12:02 |
| quiquell|rover | tobiash: periodic with files does not work | 12:02 |
| *** sdake has quit IRC | 12:12 | |
| *** sdake has joined #zuul | 12:16 | |
| quiquell|rover | tobiash: https://storyboard.openstack.org/#!/story/2005040 | 12:26 |
| *** rlandy has joined #zuul | 12:54 | |
| *** jpena|brb is now known as jpena | 13:27 | |
| *** sdake has quit IRC | 13:28 | |
| *** quiquell|rover is now known as quique|rover|eat | 13:28 | |
| *** sdake has joined #zuul | 13:56 | |
| *** quique|rover|eat is now known as quiquell|rover | 14:09 | |
| fungi | quiquell|rover: how would you expect a files filter to work with a periodic trigger, out of curiosity? | 14:23 |
| fungi | (files filters are based on the list of files modified by the triggering change, and there is no "change" associated with a periodic trigger) | 14:23 |
| quiquell|rover | fungi: I expect it to run, you can have changes at those same files but comming from RPMs promoted | 14:24 |
| fungi | quiquell|rover: ahh, i see. more generally, should triggers which don't provide a list of files not take the files filters for jobs into account? | 14:24 |
| quiquell|rover | fungi: yep | 14:25 |
| quiquell|rover | fungi: Timer trigger have to ignore files | 14:25 |
| fungi | (as opposed to literally checking to see if any listed files match, which they never will) | 14:25 |
| fungi | doesn't sound like a bug, but does sound like a potentially useful behavior change | 14:25 |
| quiquell|rover | fungi: Yep that's why I send the email, is not a bug since this was not even a feature :-) | 14:26 |
| *** gtema has quit IRC | 14:26 | |
| fungi | cool, thanks! | 14:26 |
| quiquell|rover | fungi: If I have a free brain cycle I will try to put some unit test + fix | 14:27 |
| quiquell|rover | fungi: doesn't look to difficult | 14:27 |
| quiquell|rover | s/to/too/ | 14:27 |
| *** chkumar|ruck is now known as chandankumar | 14:28 | |
| odyssey4me | has anyone else seen this pop up? https://gist.github.com/odyssey4me/666909262135edb1aeed37ef548bfe4b - it seems related to nodepool/openstacksdk changes cc mordred | 14:29 |
| mordred | odyssey4me: Proxy object has no attribute servers basically means the sdk failed to create a compute adapter - probably due to issues with discovery - looking real quick | 14:35 |
| mordred | odyssey4me: is this using rax public? | 14:35 |
| odyssey4me | mordred yep, rax public cloud serves multiple providers - I can check to see whether this is also happening on a standard openstack cloud | 14:36 |
| mordred | odyssey4me: I mean - it should work - we use rax public for openstack, and I just did a quick test script | 14:37 |
| *** jkt_ is now known as jkt | 14:37 | |
| odyssey4me | mordred hmm, well - what's not very handy is that the exception does not provide any information to help me know which provider the node is from, or what request it is... r anything | 14:40 |
| odyssey4me | so I have no way of knowing whether the log is repeating the same error for a single node, or whether it's a bunch of the | 14:40 |
| odyssey4me | *them | 14:40 |
| mordred | odyssey4me: yeah. sorry about that - it's a pretty terrible error message | 14:41 |
| * mordred is looking for a good way to improve that real quick | 14:42 | |
| pabelanger | grr | 14:51 |
| pabelanger | just hit: https://review.openstack.org/636764/ | 14:51 |
| pabelanger | sorr | 14:52 |
| pabelanger | https://review.openstack.org/637023/ | 14:52 |
| pabelanger | well, the stack at ^ | 14:52 |
| pabelanger | I've just +3'd | 14:52 |
| pabelanger | Would not object to a new zuul release this week :) | 14:53 |
| odyssey4me | mordred ok, so it seems that I have multiple nodes stuck in a 'deleting' state across multiple public cloud regions - IAD, ORD, LON - some have been stuck for over 15 days :p | 14:54 |
| mordred | odyssey4me: awesome | 14:55 |
| mordred | odyssey4me: I think I have an idea of how to at least make that error suck less | 14:55 |
| *** sdake has quit IRC | 14:58 | |
| fungi | odyssey4me: seems like nova's delete api method is merely a means of indicating that you would appreciate deletion of an instance some day | 14:59 |
| * fungi has sadly seen this fairly often as well | 15:00 | |
| odyssey4me | fungi heh, well, the thing is that nova has never received the request by the looks of things... the state in openstack is active | 15:00 |
| fungi | ohh | 15:00 |
| fungi | i guess nodepool retrying to delete them doesn't help either since it's probably just repeating the same call | 15:02 |
| Shrews | all: i was supposed to return today, but i cannot keep my head up due to some horrible illness that has decided to afflict me today. imma go back to bed | 15:04 |
| mordred | odyssey4me: remote: https://review.openstack.org/637563 Bail earlier on a version mismatch for a supported service | 15:05 |
| mordred | odyssey4me: that won't fix the underlying issue - but it should at least improve error logging to try to find the issue | 15:05 |
| mordred | Shrews: go to bed! | 15:06 |
| mordred | odyssey4me: is it possible that nodepool process is running with an older version of keystoneauth loaded? | 15:06 |
| mordred | odyssey4me: we fixed a rackspace issue in 3.11.2 (which your freeze shows you're running) | 15:07 |
| *** badboy has quit IRC | 15:07 | |
| tobiash | corvus, mordred, fungi: not sure if you got an email notification, maybe you want to have a look at https://storyboard.openstack.org/#!/story/2005037 | 15:09 |
| fungi | Shrews: thanks for the check-in, and hope you get feeling better! | 15:10 |
| fungi | tobiash: thanks, i should remember to set up e-mail notification for projects i follow in sb | 15:10 |
| *** quiquell|rover is now known as quiquell|off | 15:11 | |
| fungi | tobiash: oh, it's a private story... yeah there's a shortcoming in sb where it doesn't send notifications when private stories are created. corvus tried to start on a fix for it but got stuck on inscrutable test failures. sb's testing needs a severe overhaul | 15:11 |
| openstackgerrit | Merged openstack-infra/zuul master: Make UnsafeTag self registering https://review.openstack.org/637023 | 15:20 |
| fungi | corvus: tobiash: dmsimard: on a related note, how do you feel about switching story 2003552 to public? it's been sitting there for nearly 6 months and doesn't describe an actual vulnerability, just a class of potential ones | 15:21 |
| tobiash | I guess we could open it | 15:23 |
| *** nilashishc has joined #zuul | 15:24 | |
| *** nilashishc has quit IRC | 15:30 | |
| fungi | the associated hardening feature in ara was public as of august 24 | 15:38 |
| *** sanjayu__ has quit IRC | 15:44 | |
| *** sdake has joined #zuul | 15:46 | |
| odyssey4me | mordred possible, although it'd have to be grabbing that through py2 somehow | 15:46 |
| dmsimard | fungi: it can be made public as far as I am concerned | 15:48 |
| fungi | thanks dmsimard! | 15:48 |
| *** jamesmcarthur has joined #zuul | 16:01 | |
| *** iurygregory has joined #zuul | 16:07 | |
| *** pcaruana has quit IRC | 16:10 | |
| jlk | clarkb: corvus: did the caching of PR to head SHA go live yet? Have y'all seen any queue improvements? | 16:31 |
| clarkb | jlk: it did and yes, but we discovered an issue with it | 16:31 |
| jlk | oh boy! | 16:31 |
| jlk | was it the statuses on merged PRs? | 16:32 |
| clarkb | jlk: ansible runs post merge testing and merges create new commits. Our old expiration method was to remove closed PRs from the table | 16:32 |
| clarkb | yup | 16:32 |
| jlk | right, so you get a status event but it's on a closed PR | 16:32 |
| clarkb | overall it seemed to help but we were still getting hit by those post merge statuses so I think new plan is to update to a LRU (with possible time based expiry) cache | 16:32 |
| clarkb | ya | 16:32 |
| jlk | yeah, easy enough to mark it as a non-open PR in the cache, but less easy to decide when to purge the cache. | 16:33 |
| jlk | what's "LRU" mean? | 16:33 |
| clarkb | least recently used | 16:33 |
| clarkb | so we'll set some upper bound size and when we hit that in the table evict the entries for the entry that hasn't been used recently compared to the others | 16:34 |
| *** cognifloyd has joined #zuul | 16:34 | |
| *** themroc has quit IRC | 16:34 | |
| jlk | that seems reasonable, but more complicated code wise | 16:35 |
| jlk | Wouldn't it be more fun to just cache unbound, and then award a prize to the event that OOMs the box? | 16:36 |
| clarkb | jlk: the automatic merge event | 16:38 |
| clarkb | and ya it is more complicated, I don't think we've started writing the change yet | 16:39 |
| jlk | At least you know the complexity is worth it, if you're seeing queue improvement already | 16:39 |
| clarkb | ya current github queue length is 0, but we still log occurences >0 for largish periods of time, the improvement is we are able to recover unlike before | 16:40 |
| jlk | 👍 | 16:41 |
| *** nilashishc has joined #zuul | 16:42 | |
| *** nilashishc has quit IRC | 16:46 | |
| *** panda is now known as panda|off | 16:58 | |
| fungi | it's up to 118 as of just now | 17:00 |
| *** sdake has quit IRC | 17:00 | |
| fungi | started climbing around 15 minutes ago | 17:00 |
| *** sdake has joined #zuul | 17:05 | |
| odyssey4me | mordred ok, so I did a little clean up to make sure we're running keystoneauth1==3.11.2 but no dice... if this is all due to version negotation, then perhaps I can just set the version in clouds.yaml and move on with life? | 17:08 |
| *** gtema has joined #zuul | 17:08 | |
| clarkb | fungi: ya it is going to jump anytime ansible merges changes | 17:22 |
| clarkb | fungi: until we get the updated cache behavior implemented | 17:22 |
| fungi | and now up to 172. yeah it does eventually burn back down at least | 17:23 |
| *** nilashishc has joined #zuul | 17:30 | |
| *** nilashishc has left #zuul | 17:30 | |
| corvus | clarkb: want to abandon https://review.openstack.org/636751 ? | 17:43 |
| clarkb | done | 17:43 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Update docker image roles https://review.openstack.org/637387 | 17:51 |
| openstackgerrit | Merged openstack-infra/zuul master: Add spec for multi ansible version support https://review.openstack.org/623927 | 17:56 |
| *** gtema has quit IRC | 17:56 | |
| *** jpena is now known as jpena|off | 18:09 | |
| openstackgerrit | Merged openstack-infra/zuul master: web: prevent status update loop in background https://review.openstack.org/636343 | 18:18 |
| odyssey4me | mordred well, I did a pip3 install --upgrade of openstacksdk, which brought some more things to the party, and now all seems well... | 18:20 |
| odyssey4me | I updated the gist with the python packages changes, in case you're interested and may see something | 18:21 |
| *** sdake_ has joined #zuul | 18:29 | |
| *** sdake has quit IRC | 18:29 | |
| jlk | Anecdotally it does appear that Travis right now is not triggering CI on Draft PRs | 18:34 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Load buildset_registry in intermediate pull https://review.openstack.org/637603 | 18:48 |
| openstackgerrit | Merged openstack-infra/zuul master: connections: always register the scheduler https://review.openstack.org/637444 | 18:54 |
| jkt | do you think that it would be reasonable to add a change's sha1 into zuul variables somewhere? | 18:56 |
| jlk | ... that's not already there somewhere? | 18:57 |
| jkt | my use case it that I want to cache artifacts, so I would like to add a post-run trusted job which checks if a job has produced ~/artifacts/{{ zuul.project.name }}/$SHA1 , and if it exists, use that | 18:57 |
| jkt | so that a malicious job can only poison cache for its own sha1 | 18:57 |
| clarkb | perhaps under the zuul.projects list? and ya doesn't seem we write that down in the inventory | 18:57 |
| openstackgerrit | Merged openstack-infra/zuul master: connections: remove unused getSchema procedure https://review.openstack.org/637445 | 18:58 |
| jkt | jlk: it doesn't appear there when I check that variable from inside a check job | 18:58 |
| jlk | huh. I figured that was like base functionality of a CI system, provide the sha it's working from. | 18:58 |
| jkt | it "doesn't have to" because of $magic :), mainly because all correct refs are already set up by the executor | 18:58 |
| corvus | jkt: given your use case, i think the sha may not be sufficient and you may want to take a different approach | 18:58 |
| jlk | jkt: yeah, but I've seen other CI systems provide it as an ENV variable in the execution environment. | 19:00 |
| corvus | jlk: remember that the thing zuul is testing may not exist, it may be N changes across Y projects in the future. we could provide the sha of the most recent commit in that change (the sha of the change under test) but that may not be what's checked out. because of that, so far, it hasn't been a useful thing to add to the inventory. | 19:00 |
| jlk | I just _assumed_ that Zuul did the same. | 19:00 |
| corvus | jkt: are you familiar with the work i'm doing on container artifact caching? | 19:00 |
| jkt | corvus: I am not | 19:00 |
| corvus | jkt: let me dig up some stuff | 19:00 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Rename project to project_name in getPullBySha https://review.openstack.org/637218 | 19:01 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Test GithubShaCache https://review.openstack.org/637228 | 19:01 |
| clarkb | corvus: tobiash ^ addressed your comments re project_name | 19:01 |
| jkt | corvus: okay, I know that it could be a transient merge, fair enough | 19:01 |
| jlk | corvus: right, I think it would be more useful on post-merge jobs, like "now that you've tested and merged this thing, run a job to build an artifact and upload it somewhere" | 19:01 |
| corvus | jlk: in the case of a post-merge event, the sha is provided | 19:02 |
| corvus | jkt: here's an email i wrote which describes it -- http://lists.zuul-ci.org/pipermail/zuul-discuss/2019-January/000718.html | 19:02 |
| jkt | I'm doing this for a repo which essentially only contains (as git submodules) references to other git repos, repos that I cannot gate on because they are from an external project from github | 19:02 |
| * jkt reads | 19:02 | |
| corvus | jkt: (we're going with alternative #2, ftr) | 19:03 |
| corvus | jkt: basically, with the combination of provides/requires and zuul artifacts, you can get artifacts built for a previous change in a patch series. | 19:04 |
| corvus | (so you get speculative artifacts via git commit parents, and depends-on footers) | 19:04 |
| jkt | yeah, the artifacts and provides/requires, I asked about these last week | 19:07 |
| jkt | what I figured that I could use "now" is a simple logic driven by the git commit hash of the repo which provides these images/artifacts | 19:08 |
| corvus | jkt: oh, sorry, all of the stuff in zuul to support this has landed | 19:08 |
| corvus | the only thing i'm still working on is the roles to do this for a docker registry | 19:08 |
| jkt | yup, if the version which ends up getting commited is different (perhaps due to gerrit and zuul merging in a different way, or producing a commit with a different date etc), there would be a cache miss | 19:09 |
| jkt | I'm still not that fluent with Ansible, and my zuul server is centos7 without all these deps for a JS/webapp build, so I sticked with a release of Zuul so far, but that's not a big deal | 19:10 |
| corvus | oh, ok. it'll probably be either later this week or early next before we do a release with this, mostly because i at least want to see it proven to work once before we do so, to make sure we haven't missed anything :) | 19:10 |
| jkt | I wanted to go with just copying the artifacts somewhere, similarly to a log server, but not actually prefixed by the usual change number prefix | 19:10 |
| jkt | so that I have a repo which consumes these artifacts, and I could just wget foo/{{ zuul.job }}/{{ zuul.projects.THAT_SUBMODULE.sha1 }}/deps.tar | 19:11 |
| corvus | https://zuul-ci.org/docs/zuul-jobs/roles.html#role-pull-from-intermediate-registry and https://zuul-ci.org/docs/zuul-jobs/roles.html#role-push-to-intermediate-registry are the roles that implement the push/pull using artifacts | 19:12 |
| jkt | yeah, it indeed sounds like an interesting feature -- it's just that for me, this is not docker, so I will have to provision a custom registry and what not | 19:13 |
| jkt | compared to that, a predictable name of the artifact and a simple wget is much, much easier IMHO | 19:13 |
| corvus | it should work with a static fileserver too | 19:13 |
| corvus | jkt: yeah, this is totally doable with a simple wget :) | 19:13 |
| jkt | there is also one more build when a change is merged | 19:14 |
| corvus | jkt: if you also hook in a "promote" pipeline, so that deps.tar gets promoted to a known location on each merge, then your "pull" role becomes "is there a deps.tar artifact variable set due to provides/requires? if so, pull from its url. if not, pull from the standard location" | 19:14 |
| corvus | jkt: exactly, yes, that's the promote part. | 19:14 |
| jkt | corvus: but how do I access the original change from promote? | 19:14 |
| corvus | jkt: but you don't have to make another build in that case | 19:15 |
| *** electrofelix has quit IRC | 19:15 | |
| jkt | trying ot understand that | 19:15 |
| corvus | jkt: you can have your gate pipeline push its intermediate build to a location that's keyed based on the change, and then have the promote job pull from there and push to the publication url | 19:15 |
| corvus | jkt: the example i have for that (again, for docker images) is https://zuul-ci.org/docs/zuul-jobs/roles.html#role-upload-docker-image and https://zuul-ci.org/docs/zuul-jobs/roles.html#role-promote-docker-image | 19:16 |
| jkt | corvus: do you have an example of a promote job which does that, and showing the interaction between gate and promote? I digged through some real-world jobs visible at openstack's zuul status dashboard | 19:17 |
| jkt | but I got lost in all these indirections | 19:17 |
| corvus | jkt: yes, we're using the promote stuff in production now (we're building in gate, and publishing in promote). the zuul docker image build jobs use that | 19:17 |
| jkt | (these were not using provides/requires, sure, just release tarballs, but I got lost nonetheless) | 19:17 |
| corvus | only the docker images are using promote right now, no one has backported tarballs to it yet | 19:18 |
| *** jamesmcarthur has quit IRC | 19:19 | |
| jkt | corvus: I thought that "promote" is a fully independent pipeline which only has access to the state of the repo, not to a gate job which "caused" it | 19:20 |
| corvus | jkt: unlike our "post" pipeline, "promote" is triggered by a change-merged event, so it has the change which "caused" it available, but not the resulting commit sha. | 19:21 |
| jkt | ah, doh! | 19:21 |
| jkt | thanks, that looks like that missing piece | 19:21 |
| corvus | jkt: so it's not suitable for building an artifact where it needs to be tracable to the underlying commit sha (because it will have used zuul's speculative sha, not the real one that landed). but if you don't need that, it's okay. | 19:22 |
| corvus | i think it's good enough for our (openstack and zuul) docker images and documentation builds at least. openstack may want to continue to build its tarballs post-merge depending on considerations like gpg signing, etc. | 19:23 |
| jkt | to be honest, I have not been using "gate" so far -- "check" was enough for me. This obviously requires "gate", but that's not really a problem | 19:23 |
| jkt | (apart from that extra Workflow label :) ) | 19:23 |
| corvus | yeah, this depends on being sure that something landed, and that what landed was exactly what was tested/produced | 19:24 |
| jkt | corvus: where do I see this promote thing in action? I grepped zuul-jobs and openstack-infra/project-config, but it seems like something that might be getting used from project's own .zuul.yaml, perhaps? | 19:27 |
| jkt | corvus: okay, I found and example in Zuul's own zuul.yaml :) | 19:29 |
| jkt | corvus: thanks for this, I'll take a deeper look into this and ask tomorrow if there are any more missing pieces | 19:30 |
| *** jamesmcarthur has joined #zuul | 19:33 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Switch to LRU + TTL based sha to PR cache https://review.openstack.org/637615 | 19:40 |
| clarkb | corvus: tobiash jlk ^ first pass at switching to LRU + TTL based cache. Hopefully the test suite makes it easy to see that this works :) | 19:41 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Load buildset_registry in intermediate pull https://review.openstack.org/637603 | 19:43 |
| clarkb | fwiw cachetools is MIT licensed, relatively small and well written so I don't think we should try and write our own version of an LRU ttl based cache | 19:43 |
| clarkb | SpamapS: ^ you were interested in this too I think re caches | 19:45 |
| clarkb | I don't know if 3 hours is long enough fwiw. I figured we could tune that in review | 19:46 |
| corvus | clarkb: why lru+ttl? why not just lru? | 19:47 |
| corvus | i feel like ttl is basically "try to guess how long jobs run" | 19:47 |
| clarkb | corvus: spamaps was concerned we'd go stale in the multi commit case iirc | 19:48 |
| clarkb | er one sha to many PRs case? because we'd cache the first then enver add others | 19:48 |
| clarkb | I suppose that behavior may actually be better for zuul since it doesn't know how to handle multiples | 19:48 |
| corvus | i thought i responded to that in a way which i thought addressed the concern | 19:48 |
| clarkb | oh I missed it if so | 19:49 |
| clarkb | its a simple change to switch to an LRU. Basically have to update the cache type and update the test for removal | 19:49 |
| corvus | i guess i will go dig it up and copy it into my review :( | 19:49 |
| *** sdake_ has quit IRC | 19:54 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Switch to LRU based sha to PR cache https://review.openstack.org/637615 | 19:59 |
| clarkb | that is an LRU version | 19:59 |
| *** cognifloyd has quit IRC | 20:00 | |
| SpamapS | Yeah I believe corvus addressed that. | 20:17 |
| SpamapS | When we were talking it through I came to the conclusion that LRU and the update-from-events should be plenty. | 20:17 |
| SpamapS | (I think anyway) | 20:18 |
| clarkb | in any case both version are pushed now (with LRU being latest patchset) and the test suite is useful for that transition even with minor tweaks to handle the behavior change | 20:19 |
| *** hasharAway has quit IRC | 21:31 | |
| *** hasharAway has joined #zuul | 21:31 | |
| clarkb | anyone willing to give the lru cache stack at https://review.openstack.org/637615 a look? Mostly wondering if people like the approach (and if so maybe we can get it in soon?) | 21:49 |
| *** sdake has joined #zuul | 22:32 | |
| *** sdake has quit IRC | 22:46 | |
| *** sdake has joined #zuul | 22:49 | |
| corvus | clarkb: do you want to give https://review.openstack.org/636705 another look? | 23:07 |
| clarkb | looks like just a rebase for the cache addition? | 23:08 |
| clarkb | I've approved it | 23:08 |
| corvus | clarkb: i'm +2 on your whole cache stack | 23:11 |
| clarkb | thanks | 23:11 |
| *** sdake has quit IRC | 23:24 | |
| *** sdake has joined #zuul | 23:24 | |
| *** sdake has quit IRC | 23:32 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Fix build-docker-image when using buildset_registry https://review.openstack.org/637650 | 23:33 |
| openstackgerrit | Merged openstack-infra/zuul master: Re-use the github PR object when fetching reviews https://review.openstack.org/636705 | 23:36 |
| openstackgerrit | Merged openstack-infra/zuul master: Add comment about extra issues request https://review.openstack.org/636706 | 23:36 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Don't request PR issue data https://review.openstack.org/636728 | 23:40 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Don't request PR issue data https://review.openstack.org/636728 | 23:41 |
| clarkb | I think I managed to push the rebase clean then fix what I think is the issue (though the tests may still be broken) | 23:42 |
| *** spsurya has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!