Friday, 2018-08-17

openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: Enable Python 2.7 unit tests  https://review.openstack.org/59276700:22
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: Enable Python 2.7 unit tests  https://review.openstack.org/59276700:23
SpamapSmordred: ohthat doeslook very useful.00:24
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: DNM: verify tox-py27 works  https://review.openstack.org/59277100:34
*** elyezer has quit IRC00:43
*** elyezer has joined #zuul00:49
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: small python2/3 update  https://review.openstack.org/59277400:56
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: Bump ansible test version to 2.5  https://review.openstack.org/59277601:20
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: Use collections Sequence for FileList  https://review.openstack.org/59277801:34
*** NeilSun has joined #zuul01:41
*** ianychoi has joined #zuul01:47
*** rlandy|bbl is now known as rlandy02:41
*** rlandy has quit IRC02:42
*** ssbarnea has joined #zuul03:28
*** pwhalen has quit IRC04:09
*** ssbarnea has quit IRC04:12
*** ssbarnea has joined #zuul04:21
openstackgerritMerged openstack-infra/zuul-jobs master: Enable Python 2.7 unit tests  https://review.openstack.org/59276705:07
*** elyezer has quit IRC05:26
*** elyezer has joined #zuul05:28
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: WIP web: rewrite interface in react  https://review.openstack.org/59160405:51
tristanCcorvus: last PS squashed the browser router, however i couldn't find a better solution for sub-directory than the one documented in the installation.rst file (e.g. dashboard files need to be rebuilt to update the static files location)05:54
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Small python2/3 update  https://review.openstack.org/59277405:57
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: Bump ansible test version to 2.5  https://review.openstack.org/59277605:57
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Use collections Sequence for FileList  https://review.openstack.org/59277805:57
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Cleanup temporary directories  https://review.openstack.org/59234005:57
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Create a download script  https://review.openstack.org/59234105:57
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Turn FileList into a context manager  https://review.openstack.org/59285005:57
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Keep the FileList in the indexer class  https://review.openstack.org/59285105:57
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Make indexer more generic  https://review.openstack.org/59285205:57
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Add a unicode file  https://review.openstack.org/59285305:57
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Create a download script  https://review.openstack.org/59234106:05
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Add a unicode file  https://review.openstack.org/59285306:05
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: WIP web: rewrite interface in react  https://review.openstack.org/59160406:14
tristanCcorvus: makes me wonder if it's even possible to serve a html5 path based web application from arbritary location without rebuild.06:18
*** darkwisebear has joined #zuul06:20
tristanCperhaps using the /.well-known folder to store a manifest?06:25
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: WIP web: rewrite interface in react  https://review.openstack.org/59160406:47
*** pcaruana has joined #zuul06:54
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: WIP web: rewrite interface in react  https://review.openstack.org/59160407:22
*** jpena|off is now known as jpena07:46
*** electrofelix has joined #zuul07:59
*** elyezer has quit IRC08:01
*** elyezer has joined #zuul08:02
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Make indexer more generic  https://review.openstack.org/59285208:33
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Create a download script  https://review.openstack.org/59234108:33
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Add a unicode file  https://review.openstack.org/59285308:33
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: upload-logs-swift: Stub out dry run in the uploader  https://review.openstack.org/59292908:33
ianwjhesketh: tfr!  ^ i think maybe stubbing out the uploader makes the dry run easier, and we can have the url while we index?08:33
jheskethianw: sounds good, but I'll have to look later sorry (in meetings then heading out)08:41
ianwjhesketh: np, i'm out too :)  we know how to party on a friday night!08:45
jheskethhaha :-)08:45
cmurphyif doing code reviews is not your idea of a great friday night i don't know what is08:50
*** darkwisebear has quit IRC08:54
*** panda|off is now known as panda09:08
*** elyezer has quit IRC09:12
*** elyezer has joined #zuul09:14
mordredcmurphy: ikr?09:43
*** elyezer has quit IRC09:54
*** jiapei has joined #zuul09:56
*** elyezer has joined #zuul09:57
*** EmilienM is now known as EvilienM10:15
*** elyezer has quit IRC10:23
*** elyezer has joined #zuul10:25
*** jpena is now known as jpena|lunch11:04
*** darkwisebear has joined #zuul11:25
*** darkwisebear has quit IRC11:33
*** darkwisebear has joined #zuul11:34
*** jpena|lunch is now known as jpena12:00
rcarrillocruzhey folks12:06
rcarrillocruzso12:06
rcarrillocruzhttps://ansible.softwarefactory-project.io/logs/7/7/bc3f0e852d069096ae27ff3c8b2d7bc3a83f6447/check/ansible-role-tests-vqfx-devel-py2/7784813/job-output.txt.gz12:07
rcarrillocruztrying to run a network module from executor, and failing miserably12:07
rcarrillocruzi'm seeing in the zuul source code that you override the action plugins to block them12:07
rcarrillocruzis it possible to create a venv on executor, install ansible there and run 'ansible -m <my network module> " from there, providing the job is defined in an untrusted repo12:08
rcarrillocruz?12:08
rcarrillocruznot clear to me what things are allowed/disallowed12:09
Shrewsrcarrillocruz: generally, any module that access the local executor is not allowed12:16
Shrewsrcarrillocruz: see commit msg in http://git.openstack.org/cgit/openstack-infra/zuul/commit/?id=c231d939ea84c21fe587478006c18f22410e975112:16
Shrewsbut it should work on a remote node, i would think12:17
rcarrillocruzyeah, that folder is what i was referring to12:17
rcarrillocruznow12:17
rcarrillocruzi don't see command there12:17
rcarrillocruzso could i use on local executor ?12:17
Shrewscommand is in actiongeneral folder12:18
rcarrillocruz /sadtrombone12:18
rcarrillocruzbut is not blocked there12:19
rcarrillocruzjust adds a task args12:19
rcarrillocruzand calls super12:19
rcarrillocruzwhich i assume ends up running library/command ?12:19
rcarrillocruzi know the library/command thing is to allow the streaming output stuff, now i'm not sure if that module also controls what can/cannot be run on executor12:21
* rcarrillocruz looks at check_command method, looks like a winner12:22
rcarrillocruzbah, that's just to point to use <command> module, instead of shelling out12:22
Shrewswe do host blocking somewhere too12:23
Shrewsiirc12:23
Shrewsbut i'm not sure if/how/where command would be blocked12:24
rcarrillocruzso here's the scenario12:26
rcarrillocruzwe have nodepool nodes that are appliances, that SURPRISE, mostly do not support cloud-init12:26
rcarrillocruzso i baked the zuul executor pub key on them12:26
rcarrillocruzwhat i had in mind was:12:26
Shrewsoh, maybe in zuul/ansible/action/normal.py12:27
rcarrillocruza job with two nodes, a fedora as controller and the appliance. The job would generate a pubkey, bake it on the appliance from the executor (as the executor has the privkey needed), checkout the PR on controller and run the integration test from there with the newly generated keys12:27
rcarrillocruzthat's the reason why i tried to run 'junos_config' from executor, since that allowed me to push a command onthe appliance12:27
rcarrillocruzif i can create a venv on executor, put ansible in there and run 'ansible -m junos_config' i'd be golden12:28
rcarrillocruzif not, the only think i can think of is storing the private key as secret, dump it on the job in the controller and do that dance, but i'd like to avoid dumpoing priv keys on node for obvious sec reasons12:29
rcarrillocruzShrews: that looks like it, reading12:29
mordredrcarrillocruz: so - we prevent all local code execution on the executor for security reasons - which would include executing ansible from a venv12:33
mordredrcarrillocruz: that said, a trusted job can run code from the executor12:33
rcarrillocruzso12:33
rcarrillocruz1. create the job as trusted12:33
rcarrillocruz2. do the secrets thing, but anyway, jobs tha thandle secrets are also trusted12:33
rcarrillocruzaccurate ^ ?12:34
mordredwell, jobs that handle secrets aren't trusted - but they are restricted from having speculative job config12:34
mordredrcarrillocruz: you might want to use 2 though12:35
mordredbecause in 1 you're using the ansible that zuul is running with12:35
mordredand with 2 you could have ansible from git on the controller node12:35
mordredit depends on whether it's important to be able to run junos_config from master or from a depends-on12:35
mordredrcarrillocruz: also - we could discuss whether nor not junos_config (and similar modules) need to be blocked12:36
mordredwe blocked them to be safe, not really having much context on whether they would be safe or not12:37
mordredrcarrillocruz: all that said - this also sounds like some setup work that might want to find its way into the base job12:39
*** rlandy has joined #zuul12:39
rcarrillocruzok... i think i may: 1. rebuild images with an arbitrary pubkey (non zuul executor, since it can't be used), 2. create a secret with the private key 3. Create a job that essentially dumps the priv key on the controller, creates an inventory to point to the appliance and does a final 'ansible-playbook test.yaml'12:39
mordredrcarrillocruz: for normal ssh hosts we actually do a dance where we create a new per-job ssh key, use the executor ssh key to put the per-job ssh key on the remote node, then remove the executor ssh key from the remote node12:40
mordredrcarrillocruz: yah- that's probably a great way to move forward12:40
rcarrillocruzawesome, thx folks12:40
* rcarrillocruz back to building image12:40
*** TheJulia is now known as needssleep12:55
*** jiapei has quit IRC12:55
Shrewsmordred: i plan on writing a pbrx job to publish images today (based off the loci job) and testing it with zuul or nodepool. Do we have a zuul dockerhub account? I can test with mine if not12:57
mordredShrews: yes we do!12:57
Shrewsossum possum12:57
mordredShrews: you'll find hub.docker.com: openstackzuul in the passwords file - along with the password12:59
*** samccann has joined #zuul13:17
*** elyezer has quit IRC13:20
*** elyezer has joined #zuul13:22
*** chmouel has joined #zuul13:30
*** chmouel has quit IRC13:35
*** electrofelix has quit IRC13:36
*** chmouel has joined #zuul13:38
*** elyezer has quit IRC13:40
*** elyezer has joined #zuul13:41
*** chmouel has quit IRC13:45
*** chmouel has joined #zuul13:46
*** chmouel has quit IRC13:52
*** chmouel has joined #zuul13:55
darkwisebearShrews: What's your opinion on ianw's comment in https://review.openstack.org/590412: Should nodepool-builder report success/failure as a gauge value to statsd or should we just count successful/failed builds as we do with the other stats in nodepool-builder/launcher?13:55
*** chmouel has quit IRC14:01
Shrewsdarkwisebear: after a quick look, seems to me gauge might be better (even the code a few lines up for image_build uses gauge)14:04
*** chmouel has joined #zuul14:04
darkwisebearShrews: Oh, does it? I should double-check if the docs I wrote are consistent :-[14:05
Shrewswe could probably change the image_update stats from incr to gauge as i'm not seeing a good reason for that atm14:06
*** elyezer has quit IRC14:07
darkwisebearok, I will then a) check the docs, b) use gauge instead of a counter, c) propose a change for image_update14:08
darkwisebearhowever, this will be a breaking change. Should we preserver the old stat in order to be backwards-compatible?14:08
*** elyezer has joined #zuul14:08
*** chmouel has quit IRC14:09
Shrewsdarkwisebear: we should probably get some input from clarkb/pabelanger on the statsd stuff14:10
*** chmouel has joined #zuul14:12
corvusrcarrillocruz, mordred: it sounds like you have a solution, but for completeness, it's possible to have a partially trusted job -- you can have a trusted parent job do things on the executor like installing ssh keys, then its untrusted child job can do the rest.14:15
*** chmouel has quit IRC14:16
rcarrillocruzyeah. so actully, writing up a trusted job that does all the ssh keys dance as a pre-run within zuul-config (trusted repo) and the untrusted job runing the actual tests living on ansible-zuul-jobs14:17
corvusyep, that's an option14:17
mordred++14:18
*** chmouel has joined #zuul14:18
Shrewshrm, interesting. we don't seem to capture builder statsd data14:21
corvusShrews: may need to add a firewall entry14:22
Shrewsi mean, i don't see any grafana dashboard setup for it14:23
Shrewsbut, i am also grafana illiterate14:23
*** chmouel has quit IRC14:23
corvusShrews: i'd look by grepping for likely metric keys in project-config/grafana14:24
*** chmouel has joined #zuul14:25
corvusdarkwisebear, Shrews, ianw: responded on 590412 with more info14:27
Shrewscorvus: yeah, that's were i looked. no relevant keys14:29
Shrewsbut i am now late for an appointment. bbl14:29
*** elyezer has quit IRC14:56
*** elyezer has joined #zuul14:58
*** elyezer has quit IRC15:13
*** elyezer has joined #zuul15:17
*** elyezer has quit IRC15:21
*** elyezer has joined #zuul15:23
*** chmouel has quit IRC15:28
*** elyezer has quit IRC15:35
*** elyezer has joined #zuul15:37
*** darkwisebear has quit IRC15:40
*** elyezer has quit IRC15:50
*** elyezer has joined #zuul15:52
pabelangerShrews: when you have spare time, do you mind helping look at https://review.openstack.org/584488/ where I could better update the fake driver to validate infinite retries?  Struggling to think of best way still16:07
Shrewspabelanger: i'll try to take a look a bit later16:21
pabelangergreat, thanks16:21
*** rcarrillocruz has quit IRC16:24
*** jpena is now known as jpena|off16:40
*** NeilSun has quit IRC17:04
mnaserzuul doesn't really do much of a testing matrix at the moment, or does it?17:07
*** myoung is now known as myoung|bbl17:08
mnaserodyssey4me pushed up a really cool interesting patch but i feel like the jobs.yml could be so much cleaner if there was something along those lines -- https://review.openstack.org/#/c/551791/17:08
mnasermaybe templated job names?17:08
clarkbmnaser: its somewhat intentional aiui, otherwise you end up like ironic with ~800 jobs and not realize it17:09
mnaserah okay17:09
odyssey4memnaser: heh, I proposed that idea some time ago - and the suggestion was to build a tool to do it, but that zuul should keep it's current mechanism17:09
odyssey4meuntil you mentioned it, I had forgotten about it17:09
mnaserzuul.d/project.py which is executed and yaml output is parsed? :>17:10
odyssey4meIt should be relavitvely trivial to build a small CLI tool to convert a source matrix config into a zuul job config17:10
mnaseryup, we'll probalby need to write one with all those scenarios to keep it sync'd17:10
odyssey4methen the job output could still be inspected before merging, preventing job explosion by mistake17:10
mnaserodyssey4me: maybe repurpose openstack proposal bot for this type of thing17:11
odyssey4meI figure you'd have to define some base jobs - then the matrix would be generated with everything using those as parents17:11
mnaseri feel like you're our resident expert with using it by now :p17:11
odyssey4meI may just figure out a little tool for it and publish it to pypi. It all depends on how much rage maintaining the jobs induces. ;)17:13
fungicorvus: sounds like the summit organizers are trying to finalize project update and onboarding session schedules rsn and haven't heard whether the zuul project is going to want either/both. did you get an e-mail from kendall nelson with links to the form?17:35
corvusfungi: yes i did!17:37
fungiokay, just making sure it didn't slip through the cracks17:38
corvusfungi: well it did that too17:38
fungisounds like they're hoping to try to iron out scheduling them mondayish maybe17:38
corvusfungi, clarkb, mordred, Shrews, tobiash, SpamapS, pabelanger, jhesketh, anyone else...  do you think we should have a summit project update and/or onboarding session?17:38
fungithat is to say, this monday they're hoping to slot them into the schedule17:38
corvusi guess i should ask everyone in channel *except* those people about the onboarding session :)17:40
fungii see the project update as a no-brainer. this is an opportunity to say what's transpired in the first ~6 months of being a separate project under the osf at least17:40
mordredyah17:40
mordredI think update is a good idea17:40
mordredproject onboarding I'm less sure would be a good use of time. I could see a zuul-office-hours thing where people can come ask us complicated zuul quesions as being more broadly beneficial17:41
corvusmy feeling about onboarding is we probably don't have an army of developers who don't know how to get started, *but* we may have folks here who do some work but would like to talk about other areas they aren't as familiar with....17:41
corvusmordred: yeah...17:42
corvusi wonder if i can ask them if the onboarding session could morph into a developer/user office hours session?17:42
mordredyeah17:42
corvusmordred: ok if i put your name on the form as a speaker?17:44
mordredabsolutely17:44
mordredalthough I've already gotten my speaker free-entry-pass17:44
*** chmouel has joined #zuul17:45
corvusform filled and email sent17:49
*** chmouel has quit IRC17:55
pabelangerI think it is a good idea, but unsure if I'll be attending summit, to far in future.17:56
pabelangerI'm also not going to be attending PTG this time around, but happy to participate remotely17:57
openstackgerritDavid Shrewsbury proposed openstack/pbrx master: WIP: Add image push job  https://review.openstack.org/59311917:58
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: WIP: test container push job  https://review.openstack.org/59312518:16
Shrewscorvus: what's the rule on where secrets reside?18:18
Shrewstop-level playbook?18:18
Shrewsoh, nm18:19
openstackgerritMerged openstack-infra/zuul-jobs master: upload-logs-swift: Small python2/3 update  https://review.openstack.org/59277418:21
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: WIP: test container push job  https://review.openstack.org/59312518:21
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: WIP: test container push job  https://review.openstack.org/59312518:23
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: WIP: test container push job  https://review.openstack.org/59312518:25
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: WIP: test container push job  https://review.openstack.org/59312518:28
Shrewshelps to encrypt with the correct project key, i suppose18:28
Shrews"Unable to freeze job graph: Pre-review pipeline check does not allow post-review job zuul-push-container-images"18:30
Shrewstranslation, anyone?18:30
pabelangerShrews: we don't allow secrets in check pipeline, because somebody could just echo it18:31
pabelangerneeds to be in post18:31
Shrewsah18:31
Shrewspabelanger: any way to test that *before* merging then?18:34
pabelangerShrews: not really, when I was writing base jobs for pypi publishing, there was a staging server I tested against, using non produciton secrets18:35
pabelangerbut unsure if docker has something like that18:36
pabelangerbut mostly with secrets, we land then test18:36
pabelangerthen iterate if needed18:36
*** rlandy is now known as rlandy|afk18:48
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: DNM: Add container push job  https://review.openstack.org/59312518:55
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: DNM: Add container push job  https://review.openstack.org/59312518:56
openstackgerritMerged openstack-infra/zuul-jobs master: Bump ansible test version to 2.5  https://review.openstack.org/59277619:02
*** elyezer has quit IRC19:13
*** elyezer has joined #zuul19:14
*** elyezer has quit IRC19:19
*** elyezer has joined #zuul19:25
*** chmouel has joined #zuul19:44
*** chmouel has quit IRC19:55
*** chmouel has joined #zuul19:56
*** chmouel has quit IRC19:59
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Add tox-with-sudo job  https://review.openstack.org/59315020:00
*** chmouel has joined #zuul20:01
*** chmouel has quit IRC20:09
*** elyezer has quit IRC20:11
*** chmouel has joined #zuul20:12
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: htmlify: fix py3 gzip issue  https://review.openstack.org/59316520:13
*** elyezer has joined #zuul20:16
openstackgerritMerged openstack-infra/zuul-jobs master: upload-logs-swift: Use collections Sequence for FileList  https://review.openstack.org/59277820:18
*** rlandy|afk is now known as rlandy20:36
*** samccann has quit IRC20:56
*** chmouel has quit IRC21:13
*** jianghuaw_ has quit IRC21:20
*** chmouel has joined #zuul21:29
SpamapShmmm21:42
SpamapSwhen I configure zuul-scheduler for TLS it seems like it deadlocks21:42
SpamapS2018-08-17 14:37:35,440 DEBUG zuul.MergeClient: Connecting to gearman at localhost:473021:42
SpamapS2018-08-17 14:37:35,442 DEBUG zuul.MergeClient: Waiting for gearman21:42
SpamapSlocalhost:4730 isn't responding to connections (it's listening but responding with nothing)21:42
*** chmouel has quit IRC21:42
SpamapSn/m ... problem is the cert file isn't readable by the gearman server process21:43
*** chmouel has joined #zuul21:49
openstackgerritMerged openstack-infra/zuul-jobs master: htmlify: fix py3 gzip issue  https://review.openstack.org/59316521:58
*** elyezer has quit IRC21:59
*** elyezer has joined #zuul21:59
mordredSpamapS: that might be important22:13
*** chmouel has quit IRC22:14
SpamapSmordred: seems far fetched but yeah, might be22:14
SpamapS:)22:14
*** elyezer has quit IRC22:25
*** elyezer has joined #zuul22:28
*** elyezer has quit IRC22:56
*** elyezer has joined #zuul22:56
*** rlandy has quit IRC23:27
*** chmouel has joined #zuul23:43

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!