| *** annabelleB has joined #zuul | 00:07 | |
| *** annabelleB has quit IRC | 00:20 | |
| *** dmsimard has joined #zuul | 00:20 | |
| *** annabelleB has joined #zuul | 00:23 | |
| *** annabelleB has quit IRC | 00:29 | |
| *** ianychoi has quit IRC | 00:49 | |
| *** ianychoi has joined #zuul | 00:50 | |
| pabelanger | dkehn: looks like bug in gear when using workers | 01:19 |
|---|---|---|
| pabelanger | encoding / decoding | 01:19 |
| pabelanger | SpamapS: tobiash: I just seen something odd with github and depends-on, if I first create my PR and zuul runs jobs, everything is okay. But then I --ammend commit message with depends-on, force push, zuul will not setup depends-on. I had to manually edit the first comment of the PR, and add it there to get it to work | 01:20 |
| pabelanger | wanted to see if you've seen that before | 01:20 |
| dkehn | pabelanger: yepper it is, it appear that the internal gearman is not python v3 compat | 01:32 |
| dkehn | root@zuul:~# lsof -n |grep 4730|grep LISTEN | 01:32 |
| dkehn | zuul-sche 28034 zuul 11u IPv4 30260948 0t0 TCP *:4730 (LISTEN) | 01:32 |
| dkehn | zuul-sche 28034 28035 zuul 11u IPv4 30260948 0t0 TCP *:4730 (LISTEN) | 01:32 |
| dkehn | zuul-sche 28034 28038 zuul 11u IPv4 30260948 0t0 TCP *:4730 (LISTEN) | 01:32 |
| dkehn | pabelanger: I’m assuming the the work around is running separate gearman server | 01:33 |
| dkehn | pabelanger: created an issue https://storyboard.openstack.org/#!/story/2003826 | 01:42 |
| *** rlandy|bbl is now known as rlandy | 01:46 | |
| pabelanger | dkehn: great, will take a peak more in the morning | 01:51 |
| SpamapS | pabelanger: correct, the Depends-On must be in the *PR* description, not the commit message. | 02:06 |
| SpamapS | pabelanger: that was a compromise because in GitHub PR's are not as tied to the commits they encompass. | 02:06 |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: [wip] add instance boot metadata https://review.openstack.org/604248 | 02:48 |
| openstackgerrit | Matthew Thode proposed openstack-infra/zuul-jobs master: notify needs a command https://review.openstack.org/604249 | 03:03 |
| *** eventingmonkey has quit IRC | 03:07 | |
| *** eventingmonkey has joined #zuul | 03:07 | |
| openstackgerrit | Merged openstack-infra/zuul master: web: add /{tenant}/pipelines route https://review.openstack.org/541521 | 03:34 |
| *** dkehn has quit IRC | 03:36 | |
| *** jamielennox has quit IRC | 03:37 | |
| *** jamielennox has joined #zuul | 03:38 | |
| *** sdake has quit IRC | 03:38 | |
| *** sdake has joined #zuul | 03:43 | |
| *** dkehn has joined #zuul | 03:47 | |
| *** bhavikdbavishi has joined #zuul | 04:00 | |
| *** jhesketh_ has joined #zuul | 04:32 | |
| *** sshnaidm|pto has joined #zuul | 04:34 | |
| *** bhavikdbavishi has quit IRC | 04:35 | |
| *** bhavikdbavishi1 has joined #zuul | 04:35 | |
| *** jhesketh has quit IRC | 04:36 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 04:38 | |
| *** rfolco has quit IRC | 04:41 | |
| *** rlandy has quit IRC | 05:10 | |
| *** jhesketh_ is now known as jhesketh | 05:25 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: wip: web: fix /project/{template} templates config returned value https://review.openstack.org/604264 | 05:34 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: wip: web: fix /project/{project} templates config returned value https://review.openstack.org/604264 | 05:35 |
| tristanC | mordred: corvus: it seems like project templates config are missing information to be effectively useful from the /project/{project} rest api, see ^ | 05:36 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add project page https://review.openstack.org/604266 | 05:37 |
| tristanC | here is a workaround L33 of https://review.openstack.org/#/c/604266/1/web/src/pages/Project.jsx | 05:38 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-sphinx master: Add example and type options to attributes https://review.openstack.org/604267 | 05:55 |
| *** sshnaidm|pto has quit IRC | 06:06 | |
| *** hashar has joined #zuul | 06:15 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: notify needs a command https://review.openstack.org/604249 | 06:20 |
| *** chkumar|off is now known as chkumar|ruck | 06:23 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add instance boot properties https://review.openstack.org/604248 | 06:34 |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: [wip] start using zuul-sphinx for configuration layout https://review.openstack.org/604274 | 06:34 |
| *** bhavikdbavishi has quit IRC | 06:35 | |
| *** bhavikdbavishi1 has joined #zuul | 06:35 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 06:38 | |
| openstackgerrit | Slawek Kaplonski proposed openstack-infra/zuul-jobs master: Add patch port between neutron ext bridge and infra bridge https://review.openstack.org/604285 | 07:30 |
| *** jpena|off is now known as jpena | 07:48 | |
| openstackgerrit | Slawek Kaplonski proposed openstack-infra/zuul-jobs master: Add patch port between neutron ext bridge and infra bridge https://review.openstack.org/604285 | 08:20 |
| *** quiquell|rover has joined #zuul | 08:50 | |
| quiquell|rover | AJaeger or anyone: can we re-run only the gate jobs on a very urgent review ? | 08:51 |
| AJaeger | quiquell|rover: an infra-root can do this manually for you - neither you nor me has those permissions | 09:00 |
| quiquell|rover | AJaeger: ack, thanks | 09:01 |
| ianw | quiquell|rover: what do you need re-run? | 09:08 |
| quiquell|rover | ianw: we were facing infra issues at gate jobs, we want to rerun only gate jobs here to merge it ASAP | 09:09 |
| quiquell|rover | ianw: this is the review https://review.openstack.org/#/c/603323/ | 09:10 |
| quiquell|rover | ianw: we have the tripleo gate queue purged to merge this to be able to have promotions | 09:10 |
| quiquell|rover | ianw: now we have rebased to re-run it | 09:11 |
| ianw | quiquell|rover: i can re-enqueue it to the gate for you (let's move to infra) | 09:12 |
| *** bhavikdbavishi has quit IRC | 09:14 | |
| *** quiquell|rover is now known as quique|rover|afk | 10:26 | |
| *** quique|rover|afk is now known as quiquell|rover | 10:29 | |
| *** jpena is now known as jpena|lunch | 11:23 | |
| *** chkumar|ruck is now known as chkumar|off | 11:46 | |
| *** dkehn has quit IRC | 12:05 | |
| *** dkehn has joined #zuul | 12:09 | |
| *** rfolco has joined #zuul | 12:15 | |
| *** jpena|lunch is now known as jpena | 12:26 | |
| *** rlandy has joined #zuul | 12:32 | |
| *** hashar has quit IRC | 12:32 | |
| *** quiquell|rover is now known as quique|rover|lch | 12:32 | |
| *** EmilienM is now known as EvilienM | 12:44 | |
| *** samccann has joined #zuul | 12:51 | |
| *** panda|off is now known as panda | 12:53 | |
| *** quique|rover|lch is now known as quiquell|rover | 12:55 | |
| *** hashar has joined #zuul | 13:03 | |
| *** quiquell|rover is now known as quique|rover|afk | 13:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: DMN - debugging prepare-workspace failure https://review.openstack.org/604359 | 13:17 |
| SpamapS | DMN == Do Merge Now ? ;) | 13:23 |
| SpamapS | harumph. I think we need an /etc/zuul/conf.d. My kubernetes deploy needs to pull secrets from 3 different places now.. very hard to get that all in to one config file without basically turning to actual config management.. which seems heavy for .. one config file :-P | 13:26 |
| * SpamapS is janking it up with a bash script that builds zuul.conf from conf.d but... lame. ;) | 13:26 | |
| pabelanger | SpamapS: more like Do Merge Now, this coffee to my face | 13:27 |
| mordred | SpamapS: I kept meaning to propose a shift from ini format to yaml format, cause the "this one file is in ini format" thing started to bother me - maybe it's a decent time to ponder both? | 13:36 |
| pabelanger | I can't seem to remember this morning, but trusted jobs are able to access files outside of the work directory on the executor right? Where untrusted jobs can't. | 13:38 |
| pabelanger | eg: https://ansible-network.softwarefactory-project.io/logs/an/24/24/4a0420d708997e0c9067c0428f8cb33a4930ccfe/check/ansible-test-sanity/1ad120c/job-output.txt.gz#_2018-09-21_13_22_48_178655 | 13:38 |
| pabelanger | we recently moved prepare-workspace to untrusted playbook, but I _think_ because ansible/ansible is using symlinks, zuul is now failing it because of our security for untrusted | 13:38 |
| mordred | pabelanger: yes, untrusted jobs cannot access files outside of the workspace | 13:39 |
| pabelanger | k, right now I can't figure out which files are outside. Will just using prepare-workspace role | 13:40 |
| pabelanger | I suspect something something symlnk | 13:40 |
| *** quique|rover|afk is now known as quiquell|rover | 13:41 | |
| *** openstackgerrit has quit IRC | 14:07 | |
| pabelanger | mordred: do you happen to know where I can find an example dict() format of cloud in: https://git.zuul-ci.org/cgit/zuul-jobs/tree/roles/upload-logs-swift/library/zuul_swift_upload.py#n418 | 14:07 |
| pabelanger | for some reason, I think the secret I am passing into that, is not formatted correctly, resulting in auth errors | 14:08 |
| *** quiquell|rover is now known as quique|rover|off | 14:14 | |
| tobiash | SpamapS: hey, we're doing the same ;) | 14:17 |
| SpamapS | tobiash: :-D | 14:19 |
| SpamapS | tobiash: I *very* much hope to circle back and turn this all into a Helm chart. | 14:19 |
| SpamapS | Though one thing I've got entangled with local choices is that I use terraform to set up the MySQL DB credentials. | 14:20 |
| Shrews | mysql and not drizzle????? shame! | 14:21 |
| * Shrews misses drizzle | 14:22 | |
| Shrews | and C++ | 14:22 |
| mordred | ++ | 14:27 |
| mordred | Shrews: drizzle would actually make a GREAT in-container database engine - what with its ability to be completely configured by env vars and its lack of need for any additional files | 14:28 |
| mordred | Shrews: it's almost like our 'databasefor the cloud' tagline was just 10 years too early | 14:28 |
| *** annabelleB has joined #zuul | 14:51 | |
| *** sshnaidm|pto has joined #zuul | 14:56 | |
| *** sshnaidm|pto has quit IRC | 15:02 | |
| *** hashar is now known as hasharAway | 15:18 | |
| pabelanger | okay, finally figured out why upload-logs-swift wasn't working, I didn't use --strip for my password encrypt | 15:24 |
| pabelanger | https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/a0b4156a37f9453eb4ec7db5422272df/logs_45/45/8d13b58b7114c0ae2150a1fc49113516f0b2487e/check/tox-linters/651434b/ | 15:25 |
| pabelanger | woot! | 15:25 |
| pabelanger | however, something is wrong with tox-docs upload | 15:25 |
| pabelanger | https://object-storage-ca-ymq-1.vexxhost.net/v1/a0b4156a37f9453eb4ec7db5422272df/logs_45/45/8d13b58b7114c0ae2150a1fc49113516f0b2487e/check/tox-docs/html/ | 15:26 |
| pabelanger | I get NoSuchKey | 15:26 |
| rcarrillocruz | oh yah | 15:26 |
| rcarrillocruz | that tripped me up for my aws key | 15:26 |
| rcarrillocruz | heh | 15:26 |
| rcarrillocruz | ossum | 15:26 |
| rcarrillocruz | mnaser: i just switched my ansiblefest demo to use vexxhost , damn...it's faaast | 15:27 |
| mordred | rcarrillocruz: \o/ | 15:38 |
| rcarrillocruz | like, same play, it finishes 3 minutes earlier compared to AWS | 15:39 |
| rcarrillocruz | lulz | 15:39 |
| mordred | tobiash: did you happen to test that openstacksdk change for scale benefits yet? I'm mostly just curious | 15:46 |
| tobiash | mordred: I took a day off yesterday and planned to to that this evening | 15:47 |
| mordred | tobiash: ok, cool. if it helps, I think we should plumb in a way to configure the size of that executor pool | 15:54 |
| tobiash | mordred: yes, but I think 5 is a good starting point for our size of deployment | 15:55 |
| mordred | ++ | 15:58 |
| SpamapS | Shrews: we're rewriting Drizzle in Rust | 16:03 |
| SpamapS | it's called FoRizzle | 16:03 |
| SpamapS | we wrote FoRizzle from Drizzle in Rizzle my devizlle | 16:04 |
| SpamapS | mordred: we don't need drizzle for cloud anymore. Clouds have their own databases and you should totally submit to the will of BigTable and Dynamo | 16:06 |
| * SpamapS throws up while typing it | 16:06 | |
| mordred | SpamapS: hah | 16:07 |
| mordred | SpamapS: well, to be fair, I was using the word "cloud" to mean the opposite of what it means, but in the way that the 'cloud native' folks have decided to redefine it - which is to mean "mostly-stateless container based execution environment" | 16:07 |
| SpamapS | :) | 16:08 |
| SpamapS | question: in a post/release type pipeline, what zuul var will have the git sha? | 16:09 |
| clarkb | I didn't realize drizzle solved the stateless database problem :P I remember it fixing a bunch of other issues though | 16:09 |
| *** openstackgerrit has joined #zuul | 16:09 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: upload-logs-swift: Add trailing slash to zuul_return https://review.openstack.org/604403 | 16:09 |
| clarkb | SpamapS: my trick for figuring that out is to look at the zuul inventory for jobs that have run | 16:09 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: WIP - Pagure driver https://review.openstack.org/604404 | 16:09 |
| SpamapS | zuul.newrev ? | 16:09 |
| SpamapS | clarkb: me too ;) | 16:10 |
| SpamapS | but I haven't added the "write inventory" role back in for mine yet, so I don't have those yet | 16:10 |
| clarkb | you can always look at zuuls/openstacks | 16:10 |
| pabelanger | clarkb: mordred: corvus: SpamapS: mind a review of 604403^, so appending of success-url works for upload-logs-swift role | 16:10 |
| clarkb | http://logs.openstack.org/19/19be7612e73dd3aeea038e1fe3c273a0a40cefee/post/publish-openstack-python-branch-tarball/bb66d2f/zuul-info/inventory.yaml is an example | 16:11 |
| clarkb | SpamapS: zuul.newrev appares to be the answer | 16:11 |
| mordred | clarkb: the main thing it did that makes it nice(r) for container execution context, is that it doesn't need the additional bootstrapping mysql does - so all you need is the executable (and shared libs) installed, and "drizzled" on the command line TOTALLY works. additionally, everything you could set in my.cnf could also be done via command line OR via env vars | 16:12 |
| mordred | so "docker run -v /var/lib/drizzle:/var/lib/drizzle drizzled" should totally work without any additional bootstrap orchestration or config management or whatnot | 16:13 |
| mordred | that ability didn't seem to impress a ton of people at the time, but I think now that people are doing single-process execution environments, more people would find it pleasant | 16:14 |
| clarkb | oh ya I remember. I appreciated it because it made it very easy to use for things like testing | 16:15 |
| mordred | yah. that is what I also liked about it | 16:15 |
| SpamapS | clarkb: indeed it is | 16:15 |
| *** caphrim00_ has joined #zuul | 16:21 | |
| tobiash | mordred: just testing and with creating jobs for 200 nodes at once the queue depth stays constantly at 0 now (where it was before at >50 which caused node failures) | 16:40 |
| tobiash | mordred: but that test also reveals that nodepool is not very efficient in processing many requests at once (I think mainly because it's iterating over all znodes several times in this process) | 16:41 |
| tobiash | but there also was already a discussion to maybe cache the nodes znode tree (there is a recipe for that in kazoo) and iterate over the local cache | 16:42 |
| tobiash | mordred: so it looks like your patch works and solves the api call contention for us :) | 16:47 |
| *** jpena is now known as jpena|off | 16:54 | |
| Shrews | yes, the znode iteration is not optimal | 17:02 |
| *** spsurya has quit IRC | 17:05 | |
| tobiash | Shrews: but I think it should be possible and save to iterate over cached znodes. as we do separate ckecks and locks on the znodes we really operate on | 17:05 |
| tobiash | s/save/safe | 17:05 |
| mordred | tobiash: yay for solving the api call contention! | 17:13 |
| tobiash | mordred: that were 200 vms in about 10 minutes and a single provider | 17:13 |
| tobiash | and no node failure | 17:13 |
| mordred | that's excellent | 17:14 |
| clarkb | mordred: tobiash was the issue that we were serializing and waiting for every api call to finish before running the next? | 17:14 |
| mordred | yah | 17:14 |
| tobiash | yes | 17:14 |
| *** caphrim00_ has quit IRC | 17:38 | |
| *** caphrim007 has joined #zuul | 17:39 | |
| pabelanger | anybody up for a friday afternoon review: https://review.openstack.org/604403/ this is to attempt to fix the trailing slash for zuul_return and upload-logs-swift: https://github.com/ansible-network/ansible-zuul-jobs/pull/45 | 17:45 |
| pabelanger | you can see the tox-docs jobs is not correct | 17:45 |
| *** ssbarnea|bkp has joined #zuul | 17:49 | |
| mnaser | rcarrillocruz: w00t, let me know if there will be a video or something | 17:53 |
| mnaser | i'd love to attend ansiblefest but i have to go to OSDN and it would be very tight | 17:53 |
| *** panda has quit IRC | 18:08 | |
| *** ssbarnea|bkp has quit IRC | 18:10 | |
| *** panda has joined #zuul | 18:10 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: upload-logs-swift: Add trailing slash to zuul_return https://review.openstack.org/604403 | 18:29 |
| pabelanger | Hmm, so something seems to be incorrect with swift container, https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/a0b4156a37f9453eb4ec7db5422272df/logs_45/45/8d13b58b7114c0ae2150a1fc49113516f0b2487e/check/tox-docs/0bc7e99/ doesn't seem to apply index.html properly and raises NoSuchKey | 18:41 |
| pabelanger | but https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/a0b4156a37f9453eb4ec7db5422272df/logs_45/45/8d13b58b7114c0ae2150a1fc49113516f0b2487e/check/tox-docs/0bc7e99/index.html work | 18:41 |
| pabelanger | $ openstack container set --property web-listings=true logs_45 | 18:46 |
| pabelanger | gets directory listings back | 18:46 |
| pabelanger | which helps a little | 18:46 |
| *** annabelleB has quit IRC | 19:01 | |
| *** annabelleB has joined #zuul | 19:13 | |
| mordred | pabelanger: I thought the stuff corvus had been working on had been making the index listings and uploading them rather than relying on swif to make them for us | 19:22 |
| timburke | pabelanger: https://object-storage-ca-ymq-1.vexxhost.net/swift/info show swift.version 12.2.5 -- given that swift is on 2.19.0, i'd say they're probably on radosgw... and i have no idea how much of staticweb that supports | 19:22 |
| mordred | yah - vexxhost is on radosgw ... although I know we tested uploading to both - so I'm curious why it's not working for pabelanger but was working in infra | 19:23 |
| pabelanger | the odd thing is, 1 jobs works as expected, the other doesn't. See jobs in https://github.com/ansible-network/ansible-zuul-jobs/pull/45 | 19:25 |
| mordred | pabelanger: https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/86bbbcfa8ad043109d2d7af530225c72/logs_78/587178/1/check/openstack-zuul-jobs-linters/bfe33e0/ and https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/86bbbcfa8ad043109d2d7af530225c72/logs_78/587178/1/check/openstack-zuul-jobs-linters/156539c/ | 19:25 |
| pabelanger | yup, linters for for us too | 19:25 |
| mordred | pabelanger: OH | 19:25 |
| pabelanger | it is tox-docs that is failing | 19:25 |
| pabelanger | well, not rendering propelry | 19:26 |
| mordred | yeah. bcause that one is uploading its own index.html | 19:26 |
| pabelanger | Oh | 19:26 |
| mordred | rather than the generated directory listing | 19:26 |
| pabelanger | odd, because if you enter index.html, it does work properly | 19:27 |
| pabelanger | it seems like web-index isn't applied for some reason | 19:27 |
| mordred | maybe because it's a 'subdirectory' container | 19:27 |
| mordred | pabelanger: I don't think the docs index.html is working right either, even when it's direct - https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/a0b4156a37f9453eb4ec7db5422272df/logs_45/45/8d13b58b7114c0ae2150a1fc49113516f0b2487e/check/tox-docs/0bc7e99/html/index.html doesn't quite look right | 19:30 |
| pabelanger | Yah, odd indeed. I'll see if I can reproduce the issue manually | 19:41 |
| mordred | cool. I don't have that whole stack paged in - but corvus will be back on monday and he might be more help | 19:46 |
| *** rlandy is now known as rlandy|brb | 19:47 | |
| pabelanger | ++ Yah, we likely won't go live on Friday anyways :) | 19:49 |
| mordred | pabelanger: chicken ;) | 19:52 |
| *** rlandy|brb is now known as rlandy | 20:16 | |
| *** annabelleB has quit IRC | 20:36 | |
| *** annabelleB has joined #zuul | 20:43 | |
| *** samccann has quit IRC | 20:45 | |
| *** caphrim007 has quit IRC | 20:54 | |
| *** hasharAway has quit IRC | 21:30 | |
| *** annabelleB has quit IRC | 21:42 | |
| *** annabelleB has joined #zuul | 21:55 | |
| *** rlandy has quit IRC | 21:56 | |
| *** annabelleB has quit IRC | 21:58 | |
| openstackgerrit | Matthew Thode proposed openstack-infra/zuul-jobs master: add Gentoo vars for ovs https://review.openstack.org/604491 | 22:36 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!