| dmsimard | fungi: ah, well, I meant we probably want a more detailed version that reflects all the things that the other one doesn't | 00:05 |
|---|---|---|
| fungi | oh, got it | 00:05 |
| dmsimard | Makes me notice I didn't mention gearman in the "not pictured" | 00:09 |
| SpamapS | Hm | 00:18 |
| SpamapS | is there a formal way for roles to specify bindeps? | 00:18 |
| SpamapS | I think it would be kind of cool if there was | 00:19 |
| SpamapS | I am making an S3 log uploader.. and S3 is particularly stupid about directory indexes.. | 00:19 |
| SpamapS | so I am going to use `tree` to generate the directory index... but that's not usually installed. | 00:19 |
| clarkb | SpamapS: have you seen the swift indexes? I imagine it is similar | 00:20 |
| SpamapS | I didn't look too closely | 00:20 |
| SpamapS | I am actually finding some things that claim to be able to generate them using JS that walks the S3 bucket | 00:20 |
| clarkb | this generates them at upload time | 00:21 |
| SpamapS | yeah that would be better | 00:23 |
| SpamapS | just need something simple | 00:23 |
| * SpamapS peeks | 00:23 | |
| SpamapS | I copied my role from upload-logs-swift so should be obvious no? | 00:23 |
| SpamapS | oooohhh, the python does it | 00:25 |
| SpamapS | wow that's kind of.. wow | 00:25 |
| ianw | SpamapS: i've got a series out on that, downwards from https://review.openstack.org/#/c/592341/ ... and the next logical step there would be to actually create the html file from a template, rather than inline | 00:28 |
| SpamapS | https://github.com/rufuspollock/s3-bucket-listing works | 00:28 |
| SpamapS | ianw: eh? | 00:29 |
| SpamapS | ahh got it | 00:29 |
| SpamapS | http://zuullogsgdmnyco.s3.us-west-2.amazonaws.com/5/5/9da1db91fc18db832dfe87b0289a21af230c06f2/check/test-base/640379f/index.html?prefix=5/5/9da1db91fc18db832dfe87b0289a21af230c06f2/check/test-base/640379f/ | 00:29 |
| ianw | right, if you wanted to do something similar i mean, but ensure you're using bootstrap or whatever to make it look nice(r) | 00:30 |
| ianw | which would be terrible coded inline in python | 00:30 |
| SpamapS | this is pretty cool | 00:30 |
| SpamapS | I think this will work perfectly | 00:30 |
| SpamapS | and just requires that you put it in as the error/index html | 00:31 |
| *** threestrands has joined #zuul | 00:32 | |
| *** threestrands has quit IRC | 00:32 | |
| *** threestrands has joined #zuul | 00:32 | |
| ianw | not terribly wget-able though | 00:32 |
| *** spsurya has quit IRC | 00:35 | |
| *** fbouliane has quit IRC | 00:59 | |
| *** fbouliane has joined #zuul | 00:59 | |
| *** goern has quit IRC | 01:47 | |
| SpamapS | ianw: not sure that matters, but it's a fair point. | 01:51 |
| ianw | well that's why i've been banging on a bit about a download script anyway | 01:59 |
| ianw | when you've got a mass of interrelated log files like a devstack run, it makes it *much* easier if you've just got it on disk | 01:59 |
| clarkb | grep is a powerful tool | 02:00 |
| ianw | much better than ctrl-f in a web browser, for sure :) | 02:00 |
| *** bhavikdbavishi has joined #zuul | 02:36 | |
| SpamapS | Yeah I can see that. I'm honestly underwhelmed by this approach now that I've been poking at it | 03:21 |
| SpamapS | it's kinda neat that it can generate the listing from xml... | 03:22 |
| SpamapS | but there are a bunch of "this works, this doesn't" trade-offs | 03:22 |
| SpamapS | lspci | 03:22 |
| SpamapS | derrrp | 03:22 |
| *** jimi|ansible has joined #zuul | 03:29 | |
| *** toabctl has quit IRC | 03:48 | |
| SpamapS | aaand I just found http.server | 03:52 |
| SpamapS | which will serve me up a dir listing of . | 03:52 |
| SpamapS | which is entirely suitable | 03:52 |
| *** toabctl has joined #zuul | 03:57 | |
| *** spsurya has joined #zuul | 04:10 | |
| *** rlandy has quit IRC | 04:16 | |
| *** swest has quit IRC | 05:08 | |
| *** ssbarnea|bkp has quit IRC | 05:22 | |
| *** swest has joined #zuul | 05:25 | |
| *** nchakrab_ has joined #zuul | 05:36 | |
| SpamapS | hrm | 05:40 |
| SpamapS | seems that set-zuul-log-path-fact has changed a lot lately and makes it nearly impossible to have a status-url that matches it. :-/ | 05:40 |
| SpamapS | well, hopefully we have a build page to land on soon and I can stop the silliness anyway :-P | 05:48 |
| *** pcaruana has joined #zuul | 06:13 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool master: Fix label name in reported label stats https://review.openstack.org/600659 | 06:22 |
| *** bhavikdbavishi has quit IRC | 06:23 | |
| *** bhavikdbavishi has joined #zuul | 06:24 | |
| *** hashar has joined #zuul | 06:43 | |
| *** goern has joined #zuul | 06:49 | |
| *** goern has joined #zuul | 06:49 | |
| goern | dmsimard :) | 06:55 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: WIP: Match tag items against containing branches https://review.openstack.org/578557 | 07:38 |
| *** jpena|off is now known as jpena | 07:51 | |
| *** fbo has joined #zuul | 08:06 | |
| *** ssbarnea|bkp has joined #zuul | 08:11 | |
| *** electrofelix has joined #zuul | 08:14 | |
| *** threestrands has quit IRC | 08:33 | |
| *** ssbarnea|bkp has quit IRC | 08:43 | |
| *** ssbarnea|bkp has joined #zuul | 08:54 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: WIP: Match tag items against containing branches https://review.openstack.org/578557 | 10:27 |
| *** bhavikdbavishi has quit IRC | 10:38 | |
| *** jpena is now known as jpena|lunch | 11:01 | |
| *** EmilienM is now known as EvilienM | 11:25 | |
| *** ssbarnea|bkp has quit IRC | 11:43 | |
| dmsimard | goern: ohai, I'll be around to chat in ~45m if you're still there | 11:54 |
| *** jpena|lunch is now known as jpena | 11:54 | |
| dmsimard | Has anyone worked on or thought about a gitlab driver ? At first glance their python lib looks well maintained | 12:12 |
| *** nchakrab_ has quit IRC | 12:12 | |
| *** nchakrab has joined #zuul | 12:13 | |
| *** nchakrab has quit IRC | 12:22 | |
| *** rlandy has joined #zuul | 12:40 | |
| *** hashar is now known as hasharAway | 12:44 | |
| *** ssbarnea|bkp has joined #zuul | 13:00 | |
| *** elyezer__ has quit IRC | 13:01 | |
| openstackgerrit | Markus Hosch proposed openstack-infra/zuul master: Reload tenant in case of new project branches https://review.openstack.org/600088 | 13:03 |
| *** myoung has joined #zuul | 13:13 | |
| *** nchakrab has joined #zuul | 13:28 | |
| *** nchakrab_ has joined #zuul | 13:39 | |
| *** nchakrab has quit IRC | 13:41 | |
| *** nchakrab has joined #zuul | 13:48 | |
| *** nchakrab_ has quit IRC | 13:52 | |
| *** TheJulia is now known as needssleep | 14:00 | |
| tobiash | dmsimard: as far as I know not, I remember someone asking the same questions a few months ago | 14:02 |
| tobiash | dmsimard: if you want you could start implementing this | 14:02 |
| dmsimard | tobiash: yeah, I might try a proof of concept to help understand how much work it is | 14:03 |
| tobiash | dmsimard: I think the workflow is similar to github so I would start with that as a reference | 14:03 |
| dmsimard | agreed that it would likely be similar to github | 14:03 |
| dmsimard | I remember that SpamapS and jlk worked on the github integration for such a long time though :p | 14:04 |
| tobiash | dmsimard: sure, it's not a one liner, but back then there was only gerrit and there were many concept issues to solve due to the different work models | 14:04 |
| tobiash | so I agree that it will be much work but I think still easier than getting github working in the first place | 14:05 |
| openstackgerrit | Merged openstack-infra/nodepool master: Add metric for image build result https://review.openstack.org/590412 | 14:06 |
| dmsimard | tobiash: ++ | 14:07 |
| *** nchakrab has quit IRC | 14:28 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Improve static provider to check non-ssh hosts https://review.openstack.org/600079 | 14:32 |
| *** pcaruana has quit IRC | 14:33 | |
| dmsimard | Saw a lot of familiar names attending the AnsibleFest contributor meetup, I just added my name :D | 14:35 |
| goern | dmsimard, hey man | 14:35 |
| openstackgerrit | Merged openstack-infra/nodepool master: Fix missing node state refresh in static driver https://review.openstack.org/600080 | 14:35 |
| openstackgerrit | Merged openstack-infra/nodepool master: Fix use of wrong label static test fixture https://review.openstack.org/600081 | 14:35 |
| *** xbend has joined #zuul | 14:37 | |
| *** samccann has joined #zuul | 14:40 | |
| corvus | SpamapS, clarkb: could i trouble you for a re-review of https://review.openstack.org/599073 ? | 14:48 |
| jlk | dmsimard: the GitHub driver work took a lot of time because as dmsimard said, it was shoehorned in where there was only gerrit before, during the v2 days. With v3 the structure is much more friendly for another driver, and now you have a pile of GitHub code to take inspiration from. | 15:05 |
| jlk | Really the hardest parts are going to be the signaling. Letting Zuul know when something needs to happen, and reporting from zuul back to Gitlab. | 15:06 |
| fungi | also gitlab has a lot more in common with github than with gerrit | 15:26 |
| fungi | so yes, seems like it shouldn't be a ton of effort now | 15:27 |
| SpamapS | dmsimard: the github integration was difficult because it was happening in concert with zuulv3's early dev. | 15:38 |
| SpamapS | I think a third driver would be much much simpler to write now. | 15:38 |
| SpamapS | corvus: will take a look at that shortly | 15:39 |
| *** ssbarnea|bkp has quit IRC | 15:43 | |
| corvus | tristanC: will you be in berlin? maybe statsd/prometheus should be a forum topic? | 15:49 |
| clarkb | corvus: re 599073 I like that getting the length out of the struct apparently tripped E741 | 15:50 |
| corvus | clarkb: yeah. it's a bong rule. | 15:52 |
| *** nchakrab has joined #zuul | 16:05 | |
| *** ssbarnea|bkp has joined #zuul | 16:23 | |
| *** nchakrab has quit IRC | 16:28 | |
| corvus | we just restarted openstack-infra's zuul with master. it looks like it sucessfully performed the key database migration and generated per-project ssh keys. | 16:33 |
| *** hasharAway is now known as hasharDinner | 16:43 | |
| pabelanger | so, playing around with nodepool-builder container today | 16:49 |
| pabelanger | managed to get it started it seems | 16:49 |
| pabelanger | docker run -u 1001:1001 -v /etc/nodepool:/etc/nodepool -v /opt/nodepool:/opt/nodepool -v /var/log/nodepool:/var/log/nodepool -h nodepool-builder zuul/nodepool-builder nodepool-builder -d -l /etc/nodepool/builder-logging.conf | 16:49 |
| pabelanger | but, need to still expose zookeeper to it | 16:49 |
| clarkb | pabelanger: -l for /etc/nodepool/builder-logging.conf ? | 16:52 |
| pabelanger | yah | 16:52 |
| corvus | wow that command is just the word 'nodepool' over and over again :) | 16:53 |
| clarkb | is't that for metadata? | 16:53 |
| pabelanger | clarkb: no, because 2 logging files on disk right now 1 for builder another for launcher | 16:53 |
| pabelanger | so needed to pass -l | 16:53 |
| Shrews | corvus: well, nodepooling is hard without nodepool and its nodepool things. | 16:53 |
| Shrews | also... nodepool | 16:53 |
| clarkb | pabelanger: still not sure I understand, nodepool doesn't know to read docker metadata, is pbrx doing something fancy there? | 16:54 |
| pabelanger | clarkb: which metadata are you referring too? I haven't really done much docker myself | 16:54 |
| clarkb | pabelanger: docker run -l == docker run --label which sets metadata | 16:55 |
| clarkb | things like foo=bar or in this case just a valueless flag | 16:55 |
| pabelanger | clarkb: oh, the -l there is for nodepool-builder | 16:55 |
| pabelanger | not docker flag | 16:55 |
| clarkb | oh its part of the command derp | 16:55 |
| pabelanger | yah | 16:55 |
| pabelanger | entry point in container is /usr/bin/dumb-init right now | 16:56 |
| clarkb | pabelanger: I wasn't expecting that because the docker image sets the command | 16:56 |
| clarkb | but that makes sense now | 16:56 |
| pabelanger | so, need to pass fully command / args into docker | 16:56 |
| clarkb | pabelanger: pbrx builds images with commands iirc | 16:56 |
| pabelanger | clarkb: okay, I might be derping something | 16:56 |
| clarkb | this is why there is a 'zuul' container different than zuul-base | 16:56 |
| Shrews | i think the entry point is the command as setup in setup.cfg. but it must be overridden to be useful | 16:57 |
| pabelanger | well, if I start the container, nodepool-builder will run, but try to create daemon pid, so I try to pass -d flag, but then dumb-init complains | 16:57 |
| pabelanger | and also need to change -l | 16:57 |
| clarkb | ya if you have different logging configs you may have to override that value, the pid thing should work as long as the image has /var/run/nodepool/ in it which I doubt it does | 16:58 |
| pabelanger | yah, I didn't mount that path into docker, but not sure pid inside docker will help? | 16:59 |
| clarkb | pabelanger: depends on whether or not simple-init would make use of it | 16:59 |
| clarkb | (chances are it doesn't) | 16:59 |
| *** myoung is now known as myoung|dr | 16:59 | |
| pabelanger | yah, would have to defer to docker experts, so far -d started properly | 16:59 |
| clarkb | this does make me wonder if it is worth having command specific containers instead of an install container that you customize as above | 17:00 |
| *** jpena is now known as jpena|off | 17:01 | |
| clarkb | or maybe we update nodepool to have saner defaults | 17:01 |
| pabelanger | yah, I could see some sort of $ARGS variable with sane default, then give operator option to override with docker -e (think that's flag) | 17:03 |
| clarkb | or detect if running in container and don't daemon by default | 17:03 |
| clarkb | and the log config issue is one that I think corvus has wanted to solve regardless and just always have sane logging config by default | 17:04 |
| corvus | yep | 17:08 |
| clarkb | pabelanger: the builder container needs to be privileged too right? | 17:09 |
| clarkb | pabelanger: otherwise dib can't mount things and use loopback devices iirc | 17:09 |
| pabelanger | clarkb: Yup, just haven't started doing builds yet | 17:11 |
| *** nchakrab has joined #zuul | 17:13 | |
| *** nchakrab_ has joined #zuul | 17:14 | |
| *** nchakrab has quit IRC | 17:14 | |
| *** nchakrab has joined #zuul | 17:16 | |
| *** nchakrab_ has quit IRC | 17:18 | |
| *** ssbarnea|bkp2 has joined #zuul | 17:24 | |
| *** ssbarnea|bkp has quit IRC | 17:26 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: add-build-sshkey: Remove only the master key https://review.openstack.org/599073 | 17:29 |
| pabelanger | ha | 17:30 |
| pabelanger | don't think dib likes the alpine images | 17:31 |
| pabelanger | or we missed bindep dependencies specific for diskimage-builder | 17:31 |
| pabelanger | http://paste.openstack.org/show/729702/ | 17:31 |
| electrofelix | seeing some images appear at https://hub.docker.com/u/zuul/, is this the start of some official docker images? | 17:34 |
| pabelanger | electrofelix: yup | 17:35 |
| pabelanger | I'm testing zuul/nodepool-builder today | 17:35 |
| pabelanger | but having issues with DIB dependencies I think | 17:35 |
| pabelanger | docker_image from ansible worked properly however | 17:36 |
| clarkb | pabelanger: pbrx uses bindep to install deps. We may not have the bindep deps configured properly for alpine to run dib | 17:37 |
| clarkb | pabelanger: alpine may not even have the packages necessary to run eg yum or debootstrap | 17:37 |
| clarkb | it does have debootstrap in the repos but not yum | 17:37 |
| electrofelix | pabelanger: looks like you're going to save me a bunch of time though as I was going to be looking at what was needed to run nodepool from containers next week ;-) | 17:44 |
| corvus | electrofelix: we'll be doing the same thing next week for openstack-infra at the PTG | 17:45 |
| corvus | electrofelix: Shrews and mordred have set up jobs which build zuul/nodepool container images from every commit and push them to dockerhub, so those should automatically track master | 17:45 |
| electrofelix | well looking forward to experimenting with them as we look to start really getting on the road to v3 | 17:46 |
| *** dkranz has joined #zuul | 17:55 | |
| *** electrofelix has quit IRC | 17:55 | |
| pabelanger | clarkb: yah, I'll have to look at pbrx and see if it loads bindep from diskimage-builder repo or if it expects them in nodepool. Hopefully the first, so we don't dupe dependencies | 17:56 |
| *** electrofelix has joined #zuul | 17:56 | |
| clarkb | pabelanger: just nodepool's I think | 17:56 |
| pabelanger | boo | 17:57 |
| pabelanger | but yah, quick fix would be to add them into nodepool bindep, then see how to stack containers in pbrx | 17:57 |
| *** electrofelix has quit IRC | 17:59 | |
| *** nchakrab has quit IRC | 18:10 | |
| *** pwhalen has joined #zuul | 18:19 | |
| *** pwhalen has joined #zuul | 18:19 | |
| *** nchakrab has joined #zuul | 18:25 | |
| *** nchakrab has quit IRC | 18:30 | |
| SpamapS | hrm.... struggling with console streaming | 18:39 |
| SpamapS | http://paste.openstack.org/show/729705/ | 18:40 |
| SpamapS | also zuul_console doesn't seem to be working and that is fairly complicated to debug :-P | 18:42 |
| corvus | SpamapS: what version of cherrypy are you using? | 18:44 |
| SpamapS | corvus: checking | 18:46 |
| SpamapS | corvus: 18.0.0 ? | 18:48 |
| corvus | SpamapS: pin to <18 | 18:48 |
| SpamapS | oh I can probably just rebuild my zuul-web containers | 18:49 |
| SpamapS | In fact I think I can switch to the dockerhub ones now | 18:49 |
| SpamapS | (been pbrx building) | 18:49 |
| corvus | latest master has that pin, but we don't have a release with it yet | 18:49 |
| corvus | SpamapS: cool, that should work | 18:49 |
| SpamapS | so zuul-web:latest == master? | 18:49 |
| corvus | yep | 18:50 |
| SpamapS | well, zuul/zuul-web:latest I presume | 18:50 |
| SpamapS | haven't tried them yet | 18:50 |
| corvus | SpamapS: https://review.openstack.org/599502 has the cherrypy deets | 18:50 |
| corvus | ooh. i bet we could do a real integration test with cherrypy | 18:51 |
| corvus | remote: https://review.openstack.org/600846 Add cherrypy to third-party ci | 18:52 |
| *** EvilienM is now known as EmilienM | 19:33 | |
| *** hasharDinner is now known as hashar | 19:55 | |
| pabelanger | mordred: so, for pbrx, how do you think it will be best to create a nodepool user / group within the nodepool container images? Rather then using the root user inside the container, this is mostly for trying to use the -u flag in docker run command | 20:33 |
| pabelanger | seems like using the uid is the only way right now, to have container logs have the right host file permissions for host nodepool user | 20:33 |
| pabelanger | same goes for HOME directory, default in contain I think is /root, but on host system, I've setup /var/lib/nodepool which contains things like clouds.yaml | 20:35 |
| *** samccann has quit IRC | 20:56 | |
| *** ssbarnea|bkp2 has quit IRC | 21:19 | |
| pabelanger | cool, managed to get docker container under systemd | 21:40 |
| pabelanger | not sure if the right way, but is working :D | 21:40 |
| *** myoung|dr is now known as myoung | 21:49 | |
| *** rfolco has quit IRC | 22:15 | |
| tristanC | corvus: i won't be attending the ptg physically, is it possible to join it remotely? | 22:21 |
| corvus | tristanC: we'll be doing a lot of stuff in irc and gerrit, hopefully should be easy to follow along. but if this is about my earlier question about prometheus, i was asking about the summit in berlin | 22:23 |
| corvus | tristanC: at the ptg, we're going to be trying to use more ansible to run our control plane, including running zuul+nodepool from the container images we're publishing, possibly triggered by zuul CD -- so you'd probably find that interesting | 22:24 |
| corvus | tristanC: we may be able to use someone's laptop to connect to pbx.openstack.org | 22:25 |
| *** hashar has quit IRC | 22:25 | |
| tristanC | corvus: my bad, got summit and ptg mixed up. i'm still waiting for approval for berlin, but i may be there yes | 22:26 |
| corvus | cool, that might be a good place to talk about that then | 22:27 |
| tristanC | i could work on the metric driver interface earlier than that, but it could wait for the summit too | 22:30 |
| tristanC | corvus: you also mention possibly landing the zuul react interface at the ptg, is this still planned? | 22:32 |
| corvus | tristanC: possibly, or maybe right after the ptg? i think it depends on how well we fell we could respond if something doesn't work | 22:40 |
| corvus | tristanC: after that, i'd like to try to get some container stuff merged. between that, and all the events coming up, the berlin summit is pretty close. :) | 22:40 |
| *** rlandy has quit IRC | 22:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!