Saturday, 2017-09-09

pabelangermordred: zuul_site_traceroute_host?00:00
pabelangerI see that in site-variables.yaml00:00
jeblairmordred: fwiw, that worked for me now.  so let's just keep an eye out.00:00
mordredpabelanger: yup. that would be it00:01
mordredpabelanger: roles/validate-host/defaults/main.yaml - the defaults are backwards00:02
pabelangermordred: okay, I'll let you push up the patch :)00:04
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Use zuul_site_traceroute_host properly  https://review.openstack.org/50219700:04
mordredpabelanger: I tried to stage it as a three step rename ... but I clearly failed00:05
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Remove reference to unused variable  https://review.openstack.org/50219900:05
mordredpabelanger: we'll have to force-merge the first one00:06
pabelangerkk00:06
mordredand sorry about that00:06
mordredpabelanger: that line was SUPPOSED to be 'zuul_traceroute_host: "{{ zuul_site_tracereoute_host|default(omit) }}"'00:06
mordredso that we could seamlessly roll out the change00:06
jlkhrm, does zanata-cli have it's own secret it needs?00:06
pabelangermordred: ah, ya00:07
mordredturns out 'zuul_site_traceroute_host: "{{ zuul_tracereoute_host|default(omit) }}"' is backwards00:07
mordredjlk: yah - and the script probably assumes that secret exists already on the proposal slave as managed by puppet00:07
jlkyup.00:07
jlkso... I might have to punt on this since I don't think we have that secret prepared.00:07
mordredkk. I can pick it up00:08
jlkI'll leave some hints00:08
mordredwoot00:08
jlkor maybe I can do a partial WIP that you could pick up and run with00:09
pabelangerya, I can help with secret bits too00:09
jlkcan you list multiple parents?00:11
mordredjlk: nope. single inheritance only00:12
jlkhrm, okay. So this needs both the parts that copy slave scripts AND the parts that set up the tarball secrets00:12
jlkhrm.00:12
jlkI can just call that as a pre I suppose00:12
mordredjlk: finding a happy medium between jobs reusing/sharing roles and and reusing jobs is an interesting balance ...00:12
jlkthe script copy00:12
mordredjlk: yah00:13
jlkahaha crap00:14
pabelangermordred: ya, force merge on 502197 needed00:14
jlkthe pre playbook from proposal has a task to write out the ssh key, which we wouldn't have unless we made that a parent.00:14
openstackgerritMerged openstack-infra/zuul-jobs master: Use zuul_site_traceroute_host properly  https://review.openstack.org/50219700:15
pabelangerjlk: is there an etherpad of job structure?00:15
jlkpabelanger: not yet. THe job I'm looking at requires slave-scripts prepared (which exists in the proposal pre playbook), and it needs a secret to upload tarballs. That said, I _think_ I can just make the translation upload job it's own thing that has the parent on the tarball secret00:16
mordredjlk: yah - I think whatyou said there sounds right00:16
jlkhow exactly do we do 'post-run' jobs?00:17
jlksomething that takes the place of a 'publisher' ?00:17
mordredjlk: post-run playbooks00:18
jlkah, so we can't link jobs?00:18
jlkit's the post-run that needs the tarball secret00:18
mordredjlk: jobs can totally depend on other jobs (it's a full DAG even)00:18
mordredjlk: but currently there is no pass-artifact-between-jobs00:18
jlkah right00:19
jlkthat's the catch00:19
mordredjlk: in general we've been tackling this via just making independent jobs that re-use roles and secrets00:19
jlkso what I said doesn't exactly work00:19
jlkthis job needs elements from two distinct parents. Unless I copy-pasta00:19
jlkoh, I suppose I can just copy the secret. Derp.00:19
mordredwell - and the secret is a thing you can refer to by name00:19
jlkso long as the secret doesn't need any preparation00:19
mordredso you can just have two jobs that request the same secret00:20
jlkunless that secret needs some prep, like writing it to the filesystem in some way00:20
mordredwell - if it does, that can go into a role and you can add that to a pre-playbook for the job or something perhaps?00:20
jlkoh yeah.00:20
mordredjlk: (it seems like you got one of the fun ones! it took pabelanger and I a couple of weeks to figure out python tarball publishing originally :) )00:21
jlkhah, that's what I'm cribbing a bit from00:21
mordred++00:21
mordredit's the right thing to crib from00:21
jlkthe publish-openstack-artifact job has no run,  just a secret. Is that...00:22
jlkhrm.00:22
jlkit has a post-run00:22
mordredyah. is uses the secret in the post-run00:22
jlkthe hell? where is the add-fileserver role located?00:23
pabelangerjlk: mordred: ya, i really found it helpful when we all got into etherpad, and whiteboard'd what the job looked like00:24
jlkit's referenced in playbooks/publish/openstack-artifacts.yaml00:24
pabelangerwe when through a few iterations, but eventually got it00:24
pabelangerjlk: zuul-jobs00:24
jlkoh00:25
clarkbjlk: ya I've been struggling with that too. Makes me wonder if we shouldn't allow for scoped roles00:26
clarkbzuul-jobs/add-fileserver or something00:26
pabelangerclarkb: why not?00:26
clarkbpabelanger: I think theo nly thing that might prevent that from working reliably is ansible role lookups00:27
clarkbI have no idea if that would just work00:27
clarkb(also it potentially breaks inheritance?00:27
pabelangerwe could do role: zuul-jobs.add-fileserver in playbook, to better indicate where it came from. But we'd need to make some changes to support that00:27
mordredit could be done - but I think there are some fuller things to discuss in that area00:27
pabelangerya, I agree it is confusing in some cases00:28
pabelangerespecially when first looking at stuff00:28
clarkbmy biggest problem is git grep doesn't work00:29
clarkbso then you basically go now what00:29
clarkb(and usually results in find/proper grep)00:29
pabelangerya, the reason to have the role in zuul-jobs, is to make it available to others. Moving it into project-config, would be a problem for downstream things00:29
pabelangerI like the idea of namespacing roles with project they come from, since that is just a directory name local to playbook00:30
pabelangersomething to dive into at PTG for sure00:31
jlkI think it just means that in SCM it would be like git.openstack.org/openstack-infra/zuul-jobs/zuul-jobs/roles/foo00:32
jlkthe zuul-jobs repo would have a dir of "zuul-jobs"00:32
jlkor we could alter how they get put onto the filesystem/search path00:32
pabelangerya, IIRC, we are doing symlinking for roles into jobdir00:32
clarkbif you set your search path to include ../ it would allow you to avoid needing redundant zuul-jobs dirs right?00:33
pabelangerso, that is possible00:33
clarkb(but maybe that gives access to things you don't want to expose?)00:33
jlkargh00:34
jlkhahah00:34
jlkthis publish-artifacts-to-fileserver role is _almost_ what I need00:34
jlkexcept it only publishes what's in the artifacts/ directory00:34
jlkI need it to publish ... hrm.00:35
*** jkilpatr has quit IRC00:35
jlkI wonder if I can prep this so that it takes in the translation-source/**/*.pot files and puts them in the artifacts/ dir?00:35
pabelangerjlk: yup, I do something the same for afs-docs job00:37
pabelangerso, likely we just need to rework existing jobs to use artifacts folder now for pubishing to tarballs.o.o00:38
pabelangermordred: jeblair: great, release jobs working again00:41
pabelangerthank you00:41
jlkAre we dealing at all with publishing to static.openstack.org, the logs/$LOG_PATH stuff?00:44
pabelangerya, see upload-logs role in zuul-jobs00:45
jlkbut is it basically automatic by using base?00:46
pabelangerYup, we have a post-run playbook from base jobs00:46
pabelangerthat is in project-config00:46
jlkargh this is odd. zuul_fileserver_project_path is used by publish-artifacts-to-fileserver, but it's set by add_host in roles/add-fileserver/tasks/main.yaml00:47
jlkso if you want to overload where the content gets sent....00:47
jeblairjlk: where do you need to send content (and what content)?00:51
jlk"source: 'translation-source/**/*.pot'"   "target: 'tarballs/'"00:51
jlkto the tarballs.openstack.org host00:51
jeblairwhy do we put that on tarballs...?00:51
jlk¯\_(ツ)_/¯00:51
jlktranslations are dark art.00:52
clarkbI think we stage them for push to zanata00:52
pabelangerhttp://tarballs.openstack.org/translation-source/00:52
clarkbas pushing to zanata requires adminy creds00:52
jlkmaybe not00:52
jeblairmaybe we should rethink this as push to zanata from executor?00:53
clarkband generating .pots is potentially arbitrary python00:53
jlkbecause the playbook already has a zanata push00:53
jeblairjlk: oh, hrm00:53
clarkbjlk: in that case I am stumpted00:53
pabelangerso, you should be able to sync pots file to artifacts/translation-source folder on executor, and our existing artifact publisher will push up changes00:53
jlkthe playbook does a zanata push and then this upload is done as a post job00:53
jeblairokay, i'm going to need an etherpad so i'll start one00:53
jeblairhttps://etherpad.openstack.org/p/XIWAMDDcXB00:53
pabelangerty00:53
jlkI was trying to figure out if anything else uses the 'tarballs/' path, but the existing artifact upload things we've done already just seem to put them into a subdir of the project name00:54
pabelangerjlk: ya, possible we need to expose that to user00:55
pabelangerwhich we did recently with afs_publisher_target00:55
jlkpabelanger: right, I was thinking about that, which is how I found that the destination path is currently "exposed" as a host var to the added fileserver host00:56
jlkso we could make that a variable with a default in the play that does the add_host00:57
jlkor something else.00:57
jeblairwe need to be careful not to let cloudkitty publish something into nova's dir00:58
jlkah right.00:58
jlkso letting it be host overridden is dangerous.00:58
jlkmaybe we need a copy-pasta that publishes to tarballs/00:58
jlkor maybe yell at translations and stop dumping them all in tarballs/ ?00:59
pabelangerjeblair: ya01:00
jeblairjlk, pabelanger, clarkb: https://review.openstack.org/309560 is the most relevant commit i can find about this01:01
jlkI'm trying to find where any of these translation files might wind up on the tarballs host, and I'm failing01:02
jlkoh01:02
*** harlowja has quit IRC01:02
jeblairjlk: that makes me think that proceeding with your plan of working out a way to use the tarball publishing stuff is necessary.  at least it seems that publishing of pot files is an end-goal; it's not an intermediary step or anything (and explains why nothing consumes it)01:03
jlkbut...01:03
pabelangerjlk: jeblair: with 309560 in mind, if we set path: /src/static/tarballs/translate-source for http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.yaml#n609 it should be the same publisher using add-fileserver role01:03
pabelangerbecause we append project.short_name01:03
jlkI'm really confused then by line 14 of the etherpad01:04
pabelangerthat is /src/static/tarballs on our server01:05
jlkpabelanger: you mean duplicating the secret and making it a new secret name?01:05
jlkoooh damn I think I get it now01:05
pabelangerjlk: you can use the same secret, but overload path in playbook01:05
jeblairyeah site:tarballs.openstack.org gets you as far as 'static.openstack.org:/src/static'.  target:tarballs/ gets you 'static.openstack.org:/src/static/tarballs'.  then the rest of the path comes from 'source' because that's copied over.01:05
jlkbecause of how things work, it's going to use SOME of the source path as the destination path01:05
jeblairjlk: yep01:06
pabelangerya01:06
jlkgross01:06
*** fbo has quit IRC01:06
jeblairjenkins compatability :)01:06
pabelangerhehe01:06
jlkwill a playbook var set in zuul.yaml override the secret var?01:07
jlkI thought secrets win, so that they couldn't be altered?01:07
pabelangerHmm01:08
jeblairhttps://docs.openstack.org/infra/zuul/feature/zuulv3/user/jobs.html#variables01:08
jlk(so that you couldn't trick zuul into copying the secret somewhere else by changing a host var)01:08
jeblairjlk: secrets beat job vars01:08
jlkyeah, so changing the host name at the job level isn''t going to work01:09
pabelangerYa, so because a secret path will be hard to overright01:09
pabelangeroverride*01:09
pabelangerthis is one case where mordred change to break out non-secret data is better01:09
jeblair(intentionally -- that's the cloudkitty/nova thing in action)01:09
pabelangeryup01:09
jeblairpabelanger: well, it's not strictly better, it's different01:09
pabelangerjeblair: ya, different use case for sure01:09
jlkso there is a way to do this01:10
jlkin the role, it's kind of ugly, let me etherpad a moment01:10
pabelangerafs_publish_target for afs-docs would be a way to do this i think. tarballs_publish_target variable, but jeblair cloudkitty / nova comment is still issue01:11
pabelangermaybe we'd still append project.short_name01:11
jeblairzuul_fileserver_project_path is set by one role and used by another; it's possible to change that at any point in between, so really the security here is just based on the fact that these are only used inside of a trusted playbook01:14
jeblairso i think it would be okay for add-fileserver to take an extra parameter in building it, or for us to pass something different to the publish-artifacts-to-fileserver role.01:15
jlkSee line 5301:15
jeblairjlk: ya01:15
jlkyou could allow for an override, and it'll be used if set, otherwise fall back to the secret provided "path" variable.01:15
jeblairi think my question at the moment is -- why was zuul_fileserver_project_path put in the add_host task01:15
pabelangerjeblair: ya, I think that needs to be moved01:16
jlkbecause it's used as a variable for that host? In the other role01:16
pabelangerI believe mordred added that logic01:16
jlkcommit fa7507fd98b284adaf80791739742cbc73791c9d01:17
jlkmaybe it was done so that downstream roles don't need to access the secret directly?01:18
jeblairwhat about instead of overriding the whole path, have an optional section between the secret.path and the project.short_name?01:18
jlkI'm not entirely sure why ti's a secret anyway01:18
jeblairthat feels a little closer (for better or worse) to the idea that the root path is part of the secret01:18
jlkcould do that yeah01:18
jlkalter the secret to have multiple vars01:19
jeblairit helps me to think of 'secret' as 'bundle of related information'01:19
jlkbase_path: srv/static01:19
jeblairjlk: i wasn't thinking of adding the inserted path to the secret, but rather to the add-fileserver role01:19
jlktarget_path: tarballs/01:19
jlkah01:19
jlkreduce the secret to just path: /srv/static/   ?01:20
jlkand then let the publish-artifacts role have a default path mid-path of tarballs/  ?01:20
jeblairjlk: that's an option, but we don't have to (our translation-source is under tarballs).  so the secret can stay as is (it's the "tarball site root"), then artifact publishing has no mid-path, and translation publishing has 'translation-source/' as a mid-path.01:21
jlkoooh right01:21
pabelangerya, I like the idea of keeping secret.path with tarballs then mid-path option01:22
jlkokay, I somehow was lost on a tangent, thinking we needed to change the tarballs/ part of that01:22
jeblairi put lines 45-49 in etherpad01:23
jeblairi think the hardest part will be naming the mid-path variable.01:23
jlkwill synchronize be okay with /srv/static/tarballs///<project>    ?01:23
jeblairjlk: we can default it to '/' and drop a / from the path (see what i did in line 49)01:24
jlkthat would require that the user always add a / to the right part on a mid path01:24
jlkseems dangerous.01:24
jeblairhrm.  maybe there's some jinja magic we could use.01:25
jeblairif mid path doesn't end with / or something01:25
jlkthere is.01:25
jlkI'll cook something up01:25
jeblaircool01:25
jeblairspeaking of cook... :)01:25
pabelangerjeblair: off-topic, on git push of tag, will zuul post pipeline be used for a project?01:27
jlkjeblair: see line 58.01:30
jlkzuul_fileserver_project_path: "{{ fileserver.path) }}{{ '/' + fileserver_something_path if fileserver_something_path is defined else ''}}/{{ zuul.project.short_name }}"01:30
jeblairjlk: nice01:31
jeblairpabelanger: i believe we have configured our post pipelines to only run on branch updates, not tag pushes01:31
jeblair        - event: ref-updated01:31
jeblair          ref: ^(?!refs/).*$01:31
jeblairthat's gerrit-event-speak for branches only.01:32
jeblair(gerrit strips refs/ from the ref name which something is pushed to a branch)01:32
jeblair/which/when/01:32
pabelangerjeblair: okay, maybe I'm just tired, but how are we building docs for tags ? http://files.openstack.org/docs/nova/01:32
pabelangerI can't see anything in our current release pipelines01:33
jeblairpabelanger: we have a tag pipeline01:33
jlkwhat do we think of "fileserver_leading_path"   as the name for the part between the base and the end ?01:33
jeblairjlk: better than anything i've come up with so far :)01:33
jeblairpabelanger: look at the contents of the .root-marker file to find out what job published a directory01:34
pabelangerjeblair: Oh, so that is a different job01:34
pabelanger{name}-releasenotes'01:34
pabelangerwe don't use unified-docs for tags it seems01:34
pabelangerthat only runs in post today01:35
pabelangerwhich, is odd, because we have tag logic01:35
pabelangerso, in that case https://docs.openstack.org/sandbox/ is good01:37
pabelangerI can quickly create a new patch and see if we publish01:37
pabelangermaybe for the morning01:37
jlkhrm, do we have examples of translating "{template-name}" into zuul variables?01:46
jeblairjlk: you can skip that one i think; it should just be jjb debug info01:47
jlkit seems to be passed as an argument to the script01:48
jlkJOBNAME01:48
jlkoh but not used, wtf.01:48
jlkinteresting.01:49
pabelangerk, I need to get some food01:52
openstackgerritJesse Keating proposed openstack-infra/zuul-jobs master: Add content to support translation jobs [WIP]  https://review.openstack.org/50220702:12
jlkokay two WIPs uploaded02:14
dmsimarddid mordred ever send that email ?03:33
*** jkilpatr has joined #zuul10:51
*** jkilpatr has quit IRC11:13
*** olaph1 is now known as olaph12:28
Shrewsmordred: is vexxhost the only provider where we get ipv6 nodes? having a devil of a time getting 502137 to get one12:48
Shrewsnot only do we have this ipv6 issue, but seems there is also a race in the streaming tests where the build uuid is not found (separate issue)12:49
Shrewsso this has been fun12:49
Shrewsone last recheck, then finding better things to do with my Sat morning12:57
*** jkilpatr has joined #zuul13:31
clarkbShrews: rackspace too13:52
*** jkilpatr has quit IRC13:58
*** pleia2 has quit IRC14:11
*** pleia2 has joined #zuul14:18
dmsimardShrews: you could do a fake review that creates 10 jobs that all run unit tests and then do rechecks14:19
dmsimardShrews: you're bound to run into one on each recheck :P14:19
dmsimardIf any cores are around, need reviews on the /etc/ci/mirror_info.sh stuff -- they will fix the v3 fedora devstack jobs: https://review.openstack.org/#/c/501578/ and https://review.openstack.org/#/c/501954/14:33
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add node label to inventory  https://review.openstack.org/50224814:55
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add node label to inventory  https://review.openstack.org/50224815:31
fungihrm... i'm probably pretty behind on what's going on, but shouldn't 500201 have enqueued into the gate when mordred approved it?16:40
fungiits depends-on was already merged some several days prior16:40
mordredfungi: v2 is doing a check job first - stale check I believe16:43
fungiyeah, but once the verify +1 got added, it should have enqueued right?16:44
fungiat least if we're going for behavior parity w/ 2.5.x16:45
mordredfungi: oh - shade has no gate jobs in v316:51
mordredfungi: it's still gating in v216:51
mordredbecause shade has devstack gate jobs, which means we can't move all of its jobs over yet16:51
fungiaha, right-o16:52
*** olaph1 has joined #zuul17:25
*** olaph has quit IRC17:26
*** olaph has joined #zuul17:28
*** olaph1 has quit IRC17:29
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Clean up obsolete TODO for _setGitSsh()  https://review.openstack.org/50092519:07
openstackgerritMerged openstack-infra/zuul-jobs master: Remove reference to unused variable  https://review.openstack.org/50219919:41
*** olaph has quit IRC19:52
*** olaph has joined #zuul19:53
*** olaph1 has joined #zuul19:59
*** olaph has quit IRC19:59
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Add zuul_log_url to emit-job-header  https://review.openstack.org/50227022:37
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Rename log_path and generate-log-url  https://review.openstack.org/50227122:50
*** olaph1 is now known as olaph23:00
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Override tox requirments with zuul git repos  https://review.openstack.org/48971923:02

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!