Wednesday, 2018-09-05

*** jesusaur has quit IRC00:12
*** jesusaur has joined #zuul00:19
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: prometheus: add options to start the server and process collector  https://review.openstack.org/59920900:25
*** zigo has joined #zuul01:09
*** threestrands has joined #zuul02:22
*** rlandy|bbl is now known as rlandy02:45
openstackgerritMerged openstack-infra/nodepool master: uncap sphinx  https://review.openstack.org/59974205:16
*** shachar is now known as snapiri05:16
openstackgerritMerged openstack-infra/nodepool master: Add list of metrics provided to statsd  https://review.openstack.org/59023305:41
*** jiapei has joined #zuul05:48
openstackgerritMerged openstack-infra/nodepool master: Update static nodes in Zookeeper on config change  https://review.openstack.org/59815606:23
*** pcaruana has joined #zuul06:23
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul-jobs master: Add a role to upload logs to Amazon's S3 service  https://review.openstack.org/59993106:30
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: prometheus: add options to start the server and process collector  https://review.openstack.org/59920906:40
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: prometheus: add existing metrics  https://review.openstack.org/59921006:40
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: prometheus: add decorators to measure scheduler latency  https://review.openstack.org/59921106:40
*** rlandy has quit IRC06:45
openstackgerritMarkus Hosch proposed openstack-infra/nodepool master: Add metric for image build result  https://review.openstack.org/59041206:49
*** hashar has joined #zuul06:58
SpamapSfascinating..07:15
SpamapSif you put cloudflare in front of your zuul API, it will deny encrypt_secret.py07:15
SpamapSTIL07:15
SpamapSguessing it's the generic/lame user agent07:16
openstackgerritMarkus Hosch proposed openstack-infra/nodepool master: Add metric for image build result  https://review.openstack.org/59041207:39
*** jpena|off is now known as jpena07:45
*** goern has joined #zuul07:57
*** corvus has quit IRC08:05
*** electrofelix has joined #zuul08:13
*** threestrands has quit IRC08:26
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: web: add tenant-scoped, JWT-protected actions  https://review.openstack.org/57690708:27
*** sshnaidm|afk is now known as sshnaidm08:36
*** pcaruana has quit IRC08:50
*** hashar is now known as hasharAway09:43
*** pcaruana has joined #zuul09:58
*** Diabelko has joined #zuul11:04
*** jpena is now known as jpena|lunch11:36
*** sshnaidm has quit IRC11:43
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Do not abort node launch if failed node cannot be deleted  https://review.openstack.org/58985411:44
*** zigo has quit IRC11:45
*** sshnaidm has joined #zuul11:46
*** odyssey4me has quit IRC11:53
*** odyssey4me has joined #zuul11:55
*** zigo has joined #zuul12:08
*** rlandy has joined #zuul12:33
*** jpena|lunch is now known as jpena12:41
*** ssbarnea has quit IRC12:59
*** bhavikdbavishi has joined #zuul13:00
*** ssbarnea has joined #zuul13:01
*** ssbarnea has quit IRC13:01
*** samccann has joined #zuul13:03
*** hasharAway is now known as hashar13:07
*** bhavikdbavishi1 has joined #zuul13:11
*** bhavikdbavishi has quit IRC13:12
*** bhavikdbavishi1 is now known as bhavikdbavishi13:12
*** bhavikdbavishi has quit IRC13:17
openstackgerritJeremy Stanley proposed openstack-infra/zuul-website master: Add a "start" redirect  https://review.openstack.org/59969913:19
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Do not abort node launch if failed node cannot be deleted  https://review.openstack.org/58985413:54
*** hashar is now known as hasharAway14:06
tobiashclarkb, corvus: the cherrypy fix has been merged so the next cherrypy release should be fine again14:14
ShrewsSo, https://review.openstack.org/597302 and children have multiple +2's all the way through the stack (I just approved the parent of that one). 302 has one -1 from jhesketh about variable naming, but is willing to defer to the group. What do we want to do with that?14:33
ShrewsWe can always approve the stack and fix vars after, if we want.14:34
ShrewsI think corvus wanted these in before ptg, iirc14:35
Shrewsand he is gleefully holidaying  :)14:35
* Shrews super jealous14:35
Shrewsi think i may just approve the stack and leave the var name bikeshedding for later14:37
tobiashShrews: just wanted to suggest that14:38
openstackgerritSimon Westphahl proposed openstack-infra/nodepool master: Improve static provider to check non-ssh hosts  https://review.openstack.org/60007914:38
openstackgerritSimon Westphahl proposed openstack-infra/nodepool master: Fix missing node state refresh in static driver  https://review.openstack.org/60008014:38
openstackgerritSimon Westphahl proposed openstack-infra/nodepool master: Fix use of wrong label static test fixture  https://review.openstack.org/60008114:38
openstackgerritSimon Westphahl proposed openstack-infra/nodepool master: Worker and interfaces for checking node liveness  https://review.openstack.org/60008214:38
openstackgerritSimon Westphahl proposed openstack-infra/nodepool master: Implement liveness probe for static driver  https://review.openstack.org/60008314:38
openstackgerritSimon Westphahl proposed openstack-infra/nodepool master: Cleanup of leaked resource for static driver  https://review.openstack.org/60008414:39
openstackgerritMarkus Hosch proposed openstack-infra/zuul master: Reload tenant in case of new project branches  https://review.openstack.org/60008814:48
openstackgerritMarkus Hosch proposed openstack-infra/zuul master: Reload tenant in case of new project branches  https://review.openstack.org/60008814:49
openstackgerritMerged openstack-infra/zuul master: Add private key storage migration  https://review.openstack.org/59221314:51
*** ssbarnea|bkp2 has joined #zuul14:53
ssbarnea|bkp2clarkb fungi : https://review.openstack.org/#/c/598222/ should take 10s to read and review.14:55
fungissbarnea|bkp2: indeed, http://logs.openstack.org/22/598222/3/check/zuul-build-dashboard/c4c9c24/npm/html/status.html looks like it dtrt14:58
fungithough to be fair, it took me more than 10s to pull up the draft rendering alone ;)14:59
ssbarnea|bkp2fungi cool,i didn't know there was a way to check the UI by using the logs.14:59
ssbarnea|bkp2i tested using a browser styler exptension only.14:59
openstackgerritMerged openstack-infra/zuul master: Change project.private_key to private_secrets_key  https://review.openstack.org/59730215:00
fungissbarnea|bkp2: yeah, it's the zuul-build-dashboard job result link (obviously)15:01
*** corvus has joined #zuul15:02
openstackgerritMerged openstack-infra/zuul master: Create per-project ssh keys  https://review.openstack.org/59771115:03
openstackgerritMerged openstack-infra/zuul master: Provide per-project ssh key to executor  https://review.openstack.org/59772715:04
openstackgerritMerged openstack-infra/zuul master: Serve project SSH keys and document  https://review.openstack.org/59906315:07
openstackgerritMerged openstack-infra/zuul master: Display correct cursor when hovering over patchset header  https://review.openstack.org/59822215:21
*** pcaruana has quit IRC15:25
tristanCSpamapS: could you please comment on https://review.openstack.org/599209 ?15:40
openstackgerritMerged openstack-infra/zuul master: Add container spec  https://review.openstack.org/56013615:42
*** hasharAway is now known as hashar15:59
*** jiapei has quit IRC16:22
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: add-build-sshkey: Remove only the master key  https://review.openstack.org/59907316:26
*** hashar has quit IRC16:48
corvustristanC: where's the data storage for grafana + prometheus?16:53
tristanCcorvus: prometheus comes with it's own timeserie database, but it can export to other backend like gnocchi for example16:54
corvustristanC: internally in a prometheus server then?16:55
tristanCcorvus: yes, it's a service that polls data from target16:56
corvustristanC: so statsd client -> prometheus endpoint; statsd+graphite -> prometheus server.16:56
tristanCcorvus: the prometheus_client is just a server to serve the current metrics, it does not accumulate or do any computation16:56
corvustristanC: have you used https://github.com/prometheus/statsd_exporter ?16:58
tristanCcorvus: no i haven't, it seems like it would act as middleware between statsd push and prometheus pull16:59
tristanCcorvus: which seems less efficient than having zuul services expose the metrics directly17:00
corvustristanC: i think you could point zuul at it and get all the data into prometheus today; might let you work with the full stack before comitting to it17:00
bramweltirc17:01
corvustristanC: there's no reason it should be inefficient.  it's not very different than statsd itself.  it's basically a prometheus backend for statsd.  it offloads the http serving from the application.17:02
*** hashar has joined #zuul17:03
corvustristanC: but i hear your point that prometheus as a system is simpler to set up, while that's adding another component.17:04
corvusi mostly suggest it as a way to start using prometheus now17:04
tristanCcorvus: it's a good suggestion, i actually didn't knew statsd_exporter. the other thing i like with poll is that you get a nice of running service and a red light on the one unreachable17:08
corvustristanC: yeah, that's certainly the one thing you don't get with push (unless you have a heartbeat)17:08
tristanCa nice list of*17:08
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Add missing README.rst for roles  https://review.openstack.org/60012817:10
*** hashar has quit IRC17:10
*** hashar has joined #zuul17:11
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Turn of missing docs warnings temporarily  https://review.openstack.org/60013217:14
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Add missing README.rst for roles  https://review.openstack.org/60012817:16
*** hashar has quit IRC17:17
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Add missing README.rst for roles  https://review.openstack.org/60012817:19
openstackgerritJames E. Blair proposed openstack-infra/zuul-sphinx master: Ignore non-directory entries in roles/  https://review.openstack.org/60013517:19
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Revert "Turn of missing docs warnings temporarily"  https://review.openstack.org/60013717:20
*** sshnaidm is now known as sshnaidm|afk17:32
*** electrofelix has quit IRC17:32
*** jpena is now known as jpena|off17:40
*** pcaruana has joined #zuul17:52
tobiashcorvus, tristanC: we're using statsd_exporter with prometheus and it kinda works17:55
tobiashbut it has its glitches17:55
bramweltSo I've got Zuul running, and a Gerrit instance, and I'm trying to figure out how to debug my zuul.yaml configuration as nothing shows up in the web-ui17:55
tobiashwe need a ton of mapping rules to map the statsd metrics to something useful in prometheus17:55
tobiashthat also involves complex regex matching due to a non-optimal structure of the nodepool metrics17:56
corvusbramwelt: can you be more specific about what's missing in the web-ui?17:56
bramweltIf I have a simple zuul.yaml file with just '- topic:\nname: quickstart', I can see it listed in the ui, but as soon as I add the source with gerrit and set config-projects nothing it all disappears.17:56
corvustobiash: this is good info :)17:56
bramweltcorvus: the tenents.html page displays, but the table is empty17:57
SpamapStristanC: commented17:58
tobiashcorvus: so it is usable but with quite some effort to get it running17:58
bramweltAs a side note zuul-ci.org is down for me...17:58
corvusbramwelt: can you paste your main.yaml?  (the zuul tenant config file)17:58
bramweltYep17:58
corvusbramwelt: zuul-ci.org seems to be working for me17:59
bramwelthttps://paste.fedoraproject.org/paste/1HFP8wRAcgKrG-FXnThH9Q17:59
tobiashso I personally would love to have a pluggable metrics system in zuul (similar to the drivers interface) to be able to use native metrics for prometheus17:59
bramweltcorvus: It may be my dns resolver.17:59
tobiashcorvus: but I also can understand your position and could live without it18:00
SpamapStobiash: I'm coming to this late but.. https://github.com/prometheus/statsd_exporter should give you that, no?18:00
*** openstackgerrit has quit IRC18:00
tobiashSpamapS: that's what we're using18:00
SpamapSso just have zuul send the statsd packets it already knows how to send, to that.18:00
corvusbramwelt: so the 'workshop' tenant doesn't show up in the tenant list.... is there any useful information in the scheduler's log file?18:01
tobiashSpamapS: it's not *that* easy18:01
*** hashar has joined #zuul18:02
tobiashSpamapS: you need a mapping rule for every metric you want in prometheus that parses values out of parts of the metric name18:02
tobiashSpamapS: we even contributed to it to make it possible to use it with nodepool: https://github.com/prometheus/statsd_exporter/pull/13618:02
bramweltcorvus: Hmm, this wasn't happening before but now I see it says unknown host key for Gerrit18:02
clarkbbramwelt: corvus: if this is the opendaylight server the sshd there is broken18:03
clarkbyou hvae to manually add the rsa host key to known hosts yourself so that zuul will talk to it as the ecdsa or e25519 key is broken on that version of gerrit18:03
bramweltclarkb: Ah, no, but that could be related.18:04
*** hashar is now known as hasharAway18:04
bramweltI've changed some config in my testing..so let me get back to you all where I've gotten this to a sane state.18:04
corvusbramwelt: great, let us know if we can help more18:05
bramweltThanks!18:05
bramweltWill do18:05
corvusbramwelt: ("ssh -p 29418 $gerrit_hostname -i /path/to/key ls-projects" run as the zuul user is usually a good connectivity test)18:06
corvusthough if you run into the problem clarkb describes, that won't show it since openssh and paramiko have different key preferences18:07
bramweltright, and there doesn't seem to be a host_key setting like there is for nodepool18:07
bramweltPerhaps this is a better question. What is the minimum configuration needed in a 'config-projects' repo? A single zuul.yaml file with a job definition?18:28
bramweltI haven't been able to find the docs on standing up a zuul project. There's quite a bit on running and configuring zuul though :)18:29
corvusbramwelt: technically... nothing i think :)   a reasonable potentially useful minimum might be a pipeline definition.18:29
corvusbramwelt: yes... unfortunately, no one has written that yet18:29
bramweltThat's okay, I'm hoping to contribute it :)18:29
corvus(that's what's supposed to come next in the zuul-from-scratch document)18:29
corvusbramwelt: that would be great :)18:29
bramweltOkay, so just '---\n- pipeline:\ncheck ...'?18:30
corvusbramwelt: yep.  you can omit the '---' bit.18:30
bramweltRight. i'll give that a shot18:30
corvusbramwelt: https://zuul-ci.org/docs/zuul/user/config.html#pipeline  is a good base18:31
bramweltYeah, that's what I'm using. :)18:31
tobiashtristanC, corvus: added my point of view to 59920918:33
corvustobiash: it's a shame statsd_exporter isn't more automatic.  i don't understand why it can't just handle the metrics just like statsd does (it prefixes the different metric names to different hierarchies, like "statsd_timers")18:36
bramweltHuh, I crashed gearman18:36
bramwelt(not the internal one)18:36
tobiashcorvus: because the data models are different. By default it creates a metric per statsd metric18:37
bramwelthttps://paste.fedoraproject.org/paste/oQi5EtLhOEEh21mlKP1YyA18:37
tobiashbut in zuul use case this doesn't get us anywhere because many parts of the metric name actually need to be labels in prometheus18:37
corvustobiash: i see18:38
tobiashand it has no possibility to know which elements of the path is just a metric name and which element should be a label18:38
bramweltFor context, I'm trying to get Zuul spun up with docker-compose. Had to use an external german container as I believe either forking wasn't happening in the process, or my configuration was off. :)18:38
tobiashthat's highly dependent on the metrics and data itself18:38
tobiashe.g. in nodepool you have nodepool.launch.provider.<provider>.<az>.<result>18:39
tobiashthat needs to be mapped to nodepool_launch_by_provider with the labels <provider>, <az> and result18:39
tobiashto make that more complicated nodepool emits ready or error.foo as <result> which breaks simple globbing18:40
corvustobiash: i'm not sure it's worth our time to make an abstraction layer for stats.  that sounds like a lot of work for little gain.  it may be better to pick one and stick with it.18:40
tobiashcorvus: I meant *if* we go and support statsd and prometheus we should not mix it but create an abstraction layer18:41
corvustobiash: yeah, i agree we shouldn't mix it; i'm just saying i don't think supporting both is worthwhile.  we have better things to do than split our development and review effort supporting multiple systems there.18:42
tobiashcorvus: so If the decision is to stick with statsd I can live with it, but in that case we should have some reference mapping rules in zuul18:42
tobiashI'd be happy to provide what we have in mapping rules18:43
corvusand, ftr, i'm willing to consider switching if it seems like a good idea18:43
tobiashand we should think about some little restructure of the metric names to better support the mapping18:43
corvusbramwelt: i haven't used docker-compose, but i'm not sure why there would be a problem forking... any ideas tobiash?18:44
* tobiash reads backscroll18:44
mordredI feel like jlk or spamaps had used docker-compose successfully in the past18:44
clarkbI've got nginx running at least one forked process in a docker compose managed container here18:45
tobiashwe used docker-compose too back with zuulv218:45
jlkI did, a while ago. It's been a while though.18:45
tobiashbramwelt which gearman server do you use?18:46
bramweltRight, I figure it was just my config then, as I'm using the same zuul.conf for all the services.18:46
tobiashthere are a ton of them out there18:46
bramweltjust kendu/gearman18:46
bramweltI believe it installs and runs whatever package is in Ubuntu18:47
pabelangerone reason to consider switching away from statsd is https://github.com/etsy/statsd doesn't really seem to be active any more18:47
pabelangereg: is this project dead: https://github.com/etsy/statsd/issues/64618:47
corvuspabelanger: in many cases, that's a feature, not a bug :)18:47
tobiashcorvus: I think if the decision is to use *one* metric system we should stick with statsd. I personally would love to switch to prometheus but that would close the door for everything else. While using prometheus with statsd is possible with some effort.18:47
pabelangercorvus: agree. I just think in the neat future, with dependency changes to npm, it might become harder to install statsd.  So far, it does still work18:48
corvusbramwelt: it should be okay to use the same zuul.conf for all services18:49
tobiashbramwelt: back then I used gearmand but it's really much easier to use start the gearman server together with the zuul-scheduler18:49
tobiashbramwelt: further only the python gear server supports the ssl encryption and client certificates afaik18:49
mordredI thought Shrews added ssl support to the gearmand server a while back18:50
Shrewsi seem to recall doing that for geard (not gearmand)18:51
Shrewsalso, hi mordred18:51
mordredShrews: ohai18:51
Shrewsoh, did i also add it to gearmand??? i forget, but that may be why i did it for geard18:52
tobiashat least I couldn't get ssl to work with zuul3 and gearmand so I switched to geard with zuul318:52
mordredShrews: I thought that's why you added it to the python ... but we'd be talking 6 years ago or so IIRC18:53
Shrewsmordred: i don't remember what i did yesterday18:53
mordredalso - there was a long while before SpamapS took it over that gearmand had a bunch of unrelased changed18:54
mordredchanges18:54
tobiashwhile we're talking about gearman, I created a change to gear that solves a infinite merger freeze in case the scheduler vm crashed and the scheduler gets rescheduled on a different vm18:54
tobiashhttps://review.openstack.org/59956718:54
tobiashand https://review.openstack.org/599573 would be the zuul part of it18:55
Shrewsmordred: ah, i did the cli part:  https://bazaar.launchpad.net/~tangent-trunk/gearmand/1.2/revision/81918:57
Shrewspretty sure brian added wolfssl support18:58
mordredShrews: so long ago18:59
tobiashcorvus, tristanC: these are our current zuul statsd_exporter mappings: http://paste.openstack.org/show/729542/19:10
tobiashand these for nodepool: http://paste.openstack.org/show/729541/19:11
tobiashso if our decision will be to stick with statsd I'd like to have them maintained in zuul so we can have them in parallel and run a statsd_exporter as a sidecar container to zuul with the upstream maintained mappings19:12
clarkbtobiash: mordred pabelanger can we get a second review on https://review.openstack.org/#/c/600135/1 to fix a bug with the wraning that was added there19:19
mordredtobiash: that seems like a reasonable possibility to me19:19
mordredclarkb: done19:20
clarkbtyty19:20
*** openstackgerrit has joined #zuul19:28
openstackgerritMerged openstack-infra/zuul-sphinx master: Ignore non-directory entries in roles/  https://review.openstack.org/60013519:28
clarkbhttps://review.openstack.org/#/c/600137/1 should be mergeable now that ^ is in19:29
clarkbnevermind apparenrly we need to release zuul-sphinx first19:30
corvusclarkb: how about i tag that as 0.2.519:34
*** AJaeger has joined #zuul19:34
clarkbwfm ajaeger asking about in in #openstack-infra too19:34
clarkbI think if we make the release then we don't need 600137 and the change it reverts at all19:34
corvusoh that hasn't merged yet? :)19:35
clarkbindeed19:35
corvustag pushed19:35
AJaegercorvus: no, failed first time ;(19:35
AJaegerok, I'll rebase and abandon...19:35
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Add missing README.rst for roles  https://review.openstack.org/60012819:35
AJaegeruntil 600128 starts testing, we should be good - zuul is very busy ;(19:36
SpamapSbramwelt: regarding the forked geard.. I had similar issues but it's probably just that you don't have a logging config for it.19:37
Shrewsmordred: oh! i have a present for you: https://hub.docker.com/u/zuul/dashboard/19:37
SpamapSgearmand proper I believe is still missing native TLS support and thus I prefer the internal geard.19:37
mordredSpamapS: that's super cool!19:38
mordredgah19:38
mordredShrews: that's super cool!19:38
SpamapSmordred: and no, I haven't used docker compose.. but I'm learning it right now for a side project that I want to convert to kubernetes. :)19:38
mordredShrews: we're pushing those images on comits now?19:39
Shrewsmordred: yup19:39
mordred\o/19:39
tobiashmordred: I get 404 on that19:39
mordredtobiash: you get 404 on the dockerhub link?19:40
Shrewsmordred: but we could not use protected/final because of the job var. we use allowed-projects based on corvus' suggestion19:40
tobiashyes19:40
mordredtobiash: what about for https://hub.docker.com/r/zuul/ ?19:40
corvusShrews, mordred: yeah, we need to rework that job, but it's good enough for nodepool/zuul now19:40
tobiashmordred: that works19:41
tobiashmordred: is that dashboard link some owner stuff?19:41
mordredmight be - yeah19:41
Shrewstobiash: probably (i'm logged in as the owner now)19:41
mordredme too19:41
tobiashI don't even have an account there ;)19:42
Shrewsi don't think you need one to pull an image19:42
clarkbya they have anonymous downloads19:43
clarkbopenstack infra relies on that with our proxies caching the images all over19:44
tobiashyes, pulling works19:46
AJaegerzuul-sphinx 0.2.5 is on pypi now...19:48
pabelangeroh, speaking of zuul containers, I think https://review.openstack.org/597224/ removes some left over apk indexes in images19:52
pabelangerShrews: mordred: clarkb: ^19:52
mordredpabelanger: +A19:53
pabelangerYay19:53
*** hasharAway has quit IRC20:00
dmsimardmordred: welcome back? :)20:02
mordreddmsimard: thanks! :)20:04
openstackgerritMerged openstack/pbrx master: Don't store apk indexes in cache  https://review.openstack.org/59722420:13
corvusbramwelt: are you using the images on dockerhub?  https://hub.docker.com/r/zuul/20:14
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Remove restriction on add_host  https://review.openstack.org/60020520:35
mordredcorvus, clarkb: ^^20:35
*** pcaruana has quit IRC20:40
*** sshnaidm|afk has quit IRC20:54
*** sshnaidm has joined #zuul21:00
*** sshnaidm has quit IRC21:01
*** samccann has quit IRC21:14
*** jesusaur has quit IRC21:22
*** jesusaur has joined #zuul21:23
*** rfolco has quit IRC21:52
bramweltcorvus: Uh..no. I looked and those never showed up.21:59
bramweltOh, they just got built huh? :)21:59
corvusbramwelt: they're brand new, we just started publishing them.  but they should be published on every commit.  they may be helpful  (or they may throw a wrench in your plans).  depending on how adventurous you are feeling.  :)21:59
bramweltHaha, awesome! Can you point me to the Dockerfile for them?22:00
bramweltEr wait, I think I see they're built through Python. :)22:02
*** sshnaidm has joined #zuul22:06
*** sshnaidm is now known as sshnaidm|off22:08
openstackgerritMerged openstack-infra/zuul-jobs master: Add missing README.rst for roles  https://review.openstack.org/60012823:34

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!