Wednesday, 2019-08-14

*** igordc has joined #zuul00:09
*** mattw4 has quit IRC00:16
*** fdegir has quit IRC00:23
*** fdegir has joined #zuul00:24
*** spsurya has joined #zuul01:05
*** saneax has quit IRC01:19
*** bhavikdbavishi has joined #zuul01:33
*** bhavikdbavishi has quit IRC01:38
*** bhavikdbavishi has joined #zuul01:50
*** rfolco has quit IRC01:52
*** bhavikdbavishi has quit IRC02:05
*** saneax has joined #zuul02:05
*** sshnaidm|afk has quit IRC03:10
*** sshnaidm|afk has joined #zuul03:23
*** bhavikdbavishi has joined #zuul03:27
*** raukadah is now known as chkumar|ruck04:33
*** igordc has quit IRC05:02
*** bjackman has joined #zuul05:56
*** themroc has joined #zuul07:26
*** jpena|off is now known as jpena07:39
*** jangutter has joined #zuul08:24
*** hwangbo has quit IRC08:53
mnasiadkais there a way to use github project in job.required-projects, when the main project is on opendev gerrit?09:17
*** sshnaidm|afk is now known as sshnaidm09:23
*** bjackman has quit IRC11:04
*** bjackman has joined #zuul11:19
*** jpena is now known as jpena|lunch11:35
openstackgerritJean-Philippe Evrard proposed zuul/zuul master: Revert "Expose date time as facts"  https://review.opendev.org/67639311:37
*** bjackman has quit IRC11:49
*** bjackman has joined #zuul11:52
mordredmnasiadka: https://opendev.org/openstack/openstacksdk/src/branch/master/.zuul.yaml#L323-L324 is an example of doing that12:13
mordredmnasiadka: it requires adding the github repo to the zuul tenant config file12:13
mordredmnasiadka: and obviously the opendev zuul isn't going to be gating the github project, so care should be taken - but depending on what you're doing, it can be very helpful12:14
mnasiadkamordred: I just need ceph-ansible code being cloned in one of kolla-ansible jobs, what's the downside of just doing git clone in bash scripts run by Zuul?12:15
*** jangutter_ has joined #zuul12:19
*** jangutter has quit IRC12:20
*** rlandy has joined #zuul12:22
*** rlandy is now known as rlandy|rover12:22
*** rfolco has joined #zuul12:26
mordredmnasiadka: cloning things from the internet in jobs is inherently unstable - because of the internet :)12:30
mnasiadkamordred: ok, got the point :)12:30
mordredmnasiadka: when the repo comes in via zuul, the zuul mergers will attempt to clone it, and will retry things if there are temporary issues, etc12:30
mordredalso - if it's in required-projects, you can do depends-on with PRs to the project, if needed12:31
*** jpena|lunch is now known as jpena12:31
mnasiadkamordred: ok, sounds fancy - let me find the zuul tenant config file and create a PR...12:32
mnasiadkamordred: no configuration needed on github project side?12:32
mordrednope - it's a public git repo :)12:33
mordredmnasiadka: https://opendev.org/openstack/project-config/src/branch/master/zuul/main.yaml#L1541-L155912:34
*** jangutter_ has quit IRC12:39
*** jangutter has joined #zuul12:39
openstackgerritTristan Cacqueray proposed zuul/zuul master: executor: resolve build root real path  https://review.opendev.org/67640412:50
tristanCftr, zuul-3.10.0 doesn't work when /var/lib/zuul is a symlink and executor.job_dir is not set without https://review.opendev.org/67640412:53
openstackgerritTristan Cacqueray proposed zuul/zuul master: config: add tenant.toDict() method and REST endpoint  https://review.opendev.org/62134412:55
tristanCto be more precise, executor can't find roles from zuul-jobs and jobs fail with retry_limit12:55
*** AJaeger_ is now known as AJaeger12:57
*** jeliu_ has joined #zuul13:11
*** bjackman_ has joined #zuul13:15
*** bjackman has quit IRC13:17
pandaI'm having problem testing a change from a bare role. the role is added to a job as required_project: and as role: and it' included in the run: playbook. The inclusion of the role with a tasks_from: seems to not find the task file.13:19
pandais it even possible to check and gate the role with zuul itself ?13:29
pandamaybe more clear: if I want a job to check a change in a role that is used in the job itself13:34
pandais it possible ?13:34
*** bjackman_ has quit IRC13:35
pandais the role cloned with or without the change I want to test ?13:37
pandaI see in inventory theyr're using the same src_dir, so if the role is prepared after, may master checkout overrides the patch13:38
pabelangerpanda: we have something in zuul-jobs now, but unsure where it is documented. It is something corvus has been driving13:39
pandaooh, so it's not currently possible ?13:40
pabelangerhttps://zuul-ci.org/docs/zuul-jobs/policy.html#testing13:40
pandapabelanger: OpenDev only ... I would need it for rdo ...13:42
pandapabelanger: and the role is not even a zuul-jobs, is from tripleo13:43
pabelangerpanda: you'd need to setup the same test framework13:44
pabelangersetup minimal base job, then parent testing to it13:45
pandapabelanger: so make rdo read zuul-tests.d, then setting up a minimal base job and ... I'm not sure I understood the last one.13:48
pabelangerpanda: for the test to work, you shouldn't run the role, when you are testing the role. So for that, the base minimal job, will not include it, then your new test job needs to be setup to parent to minimal base13:50
pabelangerif that makes sense13:50
pabelangerhttps://opendev.org/zuul/zuul-jobs/src/branch/master/zuul-tests.d/general-roles-jobs.yaml#L9813:52
pabelangeryou can see, that parents to base-minimal which has a lot of the roles from base missing13:52
tristanCpanda: we can help you with that, we actually have a story for this sprint to document zuul-tests.d use-case: https://tree.taiga.io/project/morucci-software-factory/us/277513:53
pabelangerwhat is zob?13:58
pabelanger'zob testing'13:59
pandapabelanger: tristanC I see ... so the child job is not using the roles: in your case because you have the role directly in the same repository.14:01
tristanCpabelanger: i think it's a shortcut for zuul jobs14:03
pandato test a role in tripleo that is used in base jobs in rdo ... we would need to enable the zuul-test.d lookup ..and14:03
tristanCpanda: and define testing job that have a file matcher for the role14:04
*** noorul has joined #zuul14:04
pandatristanC: in the zuul-tests.d in the role repo ?14:05
noorulhi all14:05
tristanCpanda: you can also make that job inherit from tripleo-ci-base and use the match-on-config-updates attribute so that the test job also run when tripleo-ci-base definition is changed14:05
noorulI see that in Zuul documentation there is not mention of GCP driver. Is there any work in progress for this?14:05
tristanCpanda: yes, add role testing job in the same repos as the role is located14:05
pabelangernoorul: I don't think once was started14:06
pabelangerone*14:06
pabelangerI think azure is14:06
pandatristanC: that is probably the main problem. The role must be tested in rdo14:06
noorulHow nodepool behaves in the case of static driver?14:07
tristanCpanda: that's ok, we can load the zuul-tests.d only from rdo's zuul, or let's name it zuul-tests-rdo.d14:07
noorulWill it ensure that one node is used by only one task at a time?14:07
pabelangernoorul: yes, you can set a limit14:07
pabelangerhttps://zuul-ci.org/docs/nodepool/configuration.html#static-driver14:07
pabelangerhttps://zuul-ci.org/docs/nodepool/configuration.html#attr-providers.max-concurrency is the setting14:08
noorulThank you!14:08
tristanCpanda: if you want to do that now, please let us start a review to add such documentation to zuul/doc/source/user/howtos so that we can take notes14:08
pandatristanC: so, add a dir zuul-tests-rdo.d in the repo, add the registry-login-integration job in that dir, that will inherit from a base rdo job and test the role ?14:09
pandatristanC: I should have done it three weeks ago ...14:09
tristanCpanda: alright, let me write down now a draft for the process in a review14:10
pandatristanC: my saviour !14:10
tristanCpanda: actually, that taiga story was initially planned for last sprint too :)14:10
pandatristanC: just to be more specific, this is the change I have to test on the repo https://review.opendev.org/673481. and this is hte job I've tried to run in both config and rdo-jobs https://review.rdoproject.org/r/2159314:11
pandathe goal is to reuse a role that deals with docker registry to make all job pre-login to rdo registry14:11
pandabut using the role from tripleo, I want to be sure that any change to the role are tested with the jobs14:12
pandapabelanger: thanks for your hints.14:23
clarkbnote that if you are running those jobs jn opendev talking through the mirror proxies that is currently done jn the clear. I made bot of that with cloudnull when he added the new rdo registry to those mirrors14:24
clarkbthere is work to deploy them with https but not all regions have hadtheir mirror replaced yet (so is in progress)14:26
pandaclarkb: oh, because even if thery are intheritd from jobs in rdo , they are run upstream  ...14:28
pandatristanC: clarkb: I mean in opendev ... mmhh, that may be a complication for the nodes14:28
clarkbYes. I specifically called this out as that registry didnt appear fully public14:29
clarkbbut was told it was fine at the time14:29
pabelangerdmellado: o/14:29
dmelladohi pabelanger o/14:29
openstackgerritTristan Cacqueray proposed zuul/zuul master: WIP: docs: add test jobs howto  https://review.opendev.org/67642414:30
pandaclarkb: the job will not access the registry anyway, it will not have the correct credentials, we're probably going to mock the registry and it will be good enough.14:30
pandatristanC: clarkb but I'm worried by the fact that the job will not be tested in rdo environemnt14:30
pandathe integration will differ from the final job we're going to run14:31
tristanCpanda: hum, that sounds more tricky than just testing a job and or role, perhaps the draft ( https://review.opendev.org/676424 ) could use a dedicated section for testing base component used by child jobs accross multiple tenant...14:32
noorulIn openstack project there are two pipelines, one is check and another one is gate. Is it possible to use different set of tasks for check and gates?14:35
pabelangernoorul: yup, we usually try to keep the same jobs in both, but sometime non-voting jobs in check are unstable. So they are left out of gate14:36
pandatristanC: hold on, so is there a specific reason why the job defined here https://review.rdoproject.org/r/21593 is not running the container role with the change in https://review.opendev.org/673481 ?14:38
openstackgerritTristan Cacqueray proposed zuul/zuul master: WIP: docs: add test jobs howto  https://review.opendev.org/67642414:39
tristanCpanda: you mean this error: https://logs.rdoproject.org/93/21593/21/check/registry-login-integration/c89fd2e/job-output.txt.gz#_2019-08-14_12_03_46_544226 ?14:42
pandatristanC: yes14:42
pabelangercould we promote https://opendev.org/zuul/zuul/src/branch/master/tools/zuul-changes.py to be installable in zuul wheel? Or some other method to save queues to disk? This will help avoid the step of manually copying it to zuul server14:43
*** noorul has quit IRC14:46
pandatristanC: crap, I think I understand why, there's a job with the same name defined in config, that's the job that is running14:47
tristanCpanda: also, where is playbooks/integration/registry-login.yaml ?14:48
openstackgerritPaul Belanger proposed zuul/zuul master: Update zuul-changes.py to python3 only  https://review.opendev.org/67642914:49
pandatristanC: forgot to git add ... and didn't realize it was missing because the job was running. ... well at elast I know that config jobs take precedence over other configuration s...14:49
openstackgerritMonty Taylor proposed zuul/zuul-jobs master: Switch to fetch-sphinx-tarball for tox-docs  https://review.opendev.org/67643014:49
pabelangerzuul-maint: should be easier review^14:49
pandatristanC: added it now, changed the name, and retrying.14:49
* panda facepalm14:49
pandatristanC: this is all because we can't test in config :)14:50
tristanCpanda: shouldn't this job be hosted in opendev, using a zuul-rdo-tests.d that will be loaded in rdo?14:50
tristanCpanda: or do you prefer if rdo report result as a regular third party job14:51
tristanCpanda: right now, https://review.rdoproject.org/r/#/c/21593/ says that it will gate, but it doesn't seems like this can gate, only opendev can gate openstack/ansible-role-container-registry14:51
tristanCor i'm missing something :)14:52
pandatristanC: yes, well, with the former, you're not running in the rdo environemnt, so you get only partial results, with the latter, you can't really stop breaking changes to be merged14:52
pandatristanC: I think I'd prefer to have real results over a real stopper taht decides on partial results.14:54
pandawith the first choice, we can blame the approver :)14:54
pandatristanC: the new patch worked ... renaming the job and adding the playbook.14:55
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Return preview artifact in fetch-sphinx-output  https://review.opendev.org/67643714:59
openstackgerritTristan Cacqueray proposed zuul/zuul master: WIP: docs: add test jobs howto  https://review.opendev.org/67642415:00
tristanCpanda: could you please review https://review.opendev.org/676424 , it should contains the instructions for what you did15:00
pabelangertobiash: left comment on https://review.opendev.org/674034/15:02
*** chkumar|ruck is now known as raukadah15:05
*** jangutter has quit IRC15:07
openstackgerritMonty Taylor proposed zuul/zuul master: Add appending yaml log plugin  https://review.opendev.org/62325615:08
openstackgerritMonty Taylor proposed zuul/zuul master: Process yaml log files if they exist  https://review.opendev.org/67624615:08
openstackgerritMonty Taylor proposed zuul/zuul master: Write errors from ansible execution into yaml log  https://review.opendev.org/67625015:08
tobiashpabelanger: I'm open to drop py2 there15:08
mordredcorvus: it helps when the new callback plugin actually has symlinks in the various ansible dir. :)15:08
pabelangertobiash: k, I only noticed it because of https://review.opendev.org/676429/15:09
pandatristanC: is match-on-config-update a shortcut for files: - zuul.d/myjobconfiguration.yaml ?15:09
mordredpabelanger: how about let's rebase 676429 on 674034?15:10
mordredpabelanger: (I was thinking of hitting +A on 674034 just now)15:10
pabelangermordred: wfm15:10
tristanCpanda: it's not a file matcher, it will match when the parent job definition change, e.g. timeout, run, roles... any of the parent attributes15:10
openstackgerritMerged zuul/zuul master: executor: resolve build root real path  https://review.opendev.org/67640415:11
pandatristanC: uh, proactive testing, you launch the jobs that may be affected by your definition change. that's very cool15:13
*** ianychoi has joined #zuul15:20
corvusmordred: regarding your change at https://review.opendev.org/676430 (and https://review.opendev.org/676432 ) it does look like it will work (which is great, that means we're halfway to making promote work for everyone) -- but there's one concern to work through: it doesn't support nearly as many role variables as fetch-sphinx-output, so any jobs which are setting those will bork.  otoh, the *job* doesn't15:21
corvusactually *document* any of the role variables as job variables.15:21
tristanCpanda: it is indeed, that's the reason why we wanted to documented that previous sprint15:21
corvusmordred: so i think the question there is -- is it okay to change the "tox-docs" job in zuul-jobs in such a way that will cause it to ignore all the implied role variables?15:22
corvusAJaeger: ^ fyi15:22
mordredcorvus: I think those are *highly* unlikely to be used by anyone outside of openstack publishing jobs15:26
corvusmordred: you're not worried about non-openstack users of tox-docs?15:26
corvusmordred: oh sorry15:26
corvusi misread15:26
corvusmordred: you mean that the only folks likely to tweak those vars are openstack publishing15:27
mordredyeah. the uses of those vars I can see are all direct uses of the role itself15:27
corvusmordred: yeah, i agree with that15:27
mordredrather than via tox-docs15:27
mordredso I thni kthe tox-docs change is safe - and we can deal with the other variables when we make new publish jobs for openstack15:27
corvusmordred: should we send something out to zuul-discuss or zuul-annonce first though?  because if there's someone out there that's using tox-docs and has found those variables, we'll break them and we have no way of knowing15:28
corvus(fwiw, i have looked at the inheritance hierarchy of tox-docs in the openstack tenant and confirmed that there are no jobs which inherit from it and set those vars; that doesn't cover project-pipeline variants, but that seems unlikely)15:29
mordredcorvus: yes, probably so. in the meantime - we COULD just *add* fetch-sphinx-tarball15:29
mordredcorvus: I just searched for those variables15:30
mordredand found them only set in publication playbooks15:30
mordredhttp://codesearch.openstack.org/?q=sphinx_output_suffix&i=nope&files=&repos= and http://codesearch.openstack.org/?q=sphinx_output_src&i=nope&files=&repos=15:30
corvusmordred: how about you email zuul-annonce telling folks that we're going to switch the implementation under tox-docs in 2 weeks (to better support promote pipelines) and that if they're using any role variables that it'll be a problem.  then we merge that change in 2 weeks.  meanwhile, i think the change to the fetch-sphinx-output role can cover the immediate preview url need.    how's that sound for a plan?15:32
corvus(i'm not super keen on adding the second fetch role to the job because we'll end up pulling the data back to the executor twice)15:34
mordredcorvus: ++15:34
fboHi, is it normal that on a post job the repo on the test node get an origin set on "origin  file:///dev/null" ?15:40
clarkbfbo: yes15:41
clarkbfbo: Zuul curated repos don't have true origins since they tend to be forward looking (post may be an exception to this but zuul doesn't differentiate)15:41
clarkband to avoid pollution of the repo during a job via a remote update zuul sets an origin (because some tools want one) but set it to dev null to avoid it being a source for unwanted commits15:42
fboalright, ok I understand, the tool I run in post expects a valid remote. But I can set it prior to run the tool15:43
clarkbfbo: you should consider if that invalidates the job results15:43
mordredfbo: yeah - hopefully the tool isn't pulling from remotes or such :)15:44
clarkbfbo: zuul ensures that all branch heads are set to the correct ref for the job that is running15:44
clarkband if something demands a valid remote it may change the heads or other repo state15:44
fbobut could it make sense to provide an option to have zuul set the correct remote ?15:45
fbothe tool is fedpkg, and it looks at the remote to tell/send to koji (fedora build system) where is the commit published.15:46
mordredah - that's an interesting use case15:46
fbonothing is done with the local repo except looking at the origin.15:46
*** themroc has quit IRC15:49
*** mattw4 has joined #zuul15:52
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: Correct version of ansible from test-requirements.txt  https://review.opendev.org/67644816:06
pabelangerwould be nice to have option to toggle it for post job16:08
clarkbpabelanger: it isn't necessarily safe to do that in a post job either though16:08
clarkbsince a post job is tied to a specific commit16:08
pabelangerclarkb: isn't that promote?16:09
clarkbpabelanger: it is both. Post jobs run against a merge commit16:09
mordredwell - *may* be tied to a specific commit16:09
mordreddepending on whether supercedent is being used or not16:09
* pabelanger nod16:10
*** jpena is now known as jpena|off16:10
pabelangerwhen this last came up, I was running code from git location on disk manually or zuul could. Solution was, don't do that :)16:10
pabelangeronly do it it rare conditions, when zuul is down for some reason16:10
*** bhavikdbavishi has quit IRC16:13
fungifbo: there isn't necessarily a "correct" remote, since the repository is pushed onto the node16:14
fungiat least from a zuul job node's perspective16:15
clarkbzuul does know the canonical repo location from its perspective though16:16
fbofungi: yeah but zuul knows the remote loc of the repo, I was surprised to find it set to /dev/null. I undertand the why, but it bring a limitation. At least for my usecase :)16:17
pabelangerfbo: you can fix it via prepare-workspace too16:18
fboimo that should be configurable and keep the default like it is today16:18
fbopabelanger: just added a task in my role https://pagure.io/zuul-distro-jobs/c/3cd4aedd0e6087775e36679dbb55c75f30646c74?branch=master16:19
fungiyeah, zuul knows *a* remote which it used to get the code it pushed, but whether that's the "correct" remote is a more existential discussion16:21
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: WIP: Allow ensure-tox to upgrade tox version  https://review.opendev.org/67276016:28
openstackgerritJeff Liu proposed zuul/zuul-operator master: Create zookeeper operator and zuul CR to k8s test  https://review.opendev.org/67645816:35
pabelangerso, we just upgraded to zuul 3.10.0, but having issue getting web working16:36
pabelangerThe path '/t/ansible/status' was not found. is from cherrypy16:36
openstackgerritMerged zuul/zuul master: Make tenant and pipeline optional in zuul-changes  https://review.opendev.org/67403416:36
pabelangerhttps://dashboard.zuul.ansible.com/t/ansible/status16:36
pabelangerzuul-maint: any suggestions where to look?16:38
pabelanger^16:38
clarkbpabelanger: did you update your js too? (I think those tend to need to update together on big updates?)16:38
pabelangerclarkb: what ever was installed via whell16:39
pabelangerwheel*16:39
pabelangerhttps://dashboard.zuul.ansible.com/api/info works and I can see github using it16:40
pabelangerso, just UI seems to be issue16:40
*** fungi has quit IRC16:42
*** fungi has joined #zuul16:43
corvuspabelanger, mordred: i think we found yesterday that 3.10.0 wasn't correctly built with the js16:46
corvusmordred was working on fixing the build jobs16:47
corvuswhen that happens we can either rebuild 3.10.0 or issue 3.10.116:47
pabelangerOh16:47
corvusmordred: where'd you get on that?16:48
pabelangerokay, I'll revert16:48
pabelangercorvus: is it safe just to revert zuul-web or do I need everything?16:48
corvuspabelanger: i bet zuul-web will be ok16:48
pabelangerk, let me try that16:49
pabelangerokay, back16:49
pabelangerso, that is likely it16:50
pabelangerI'd vote for 3.10.116:50
pabelangerlet me reply to ML too, so others are aware16:50
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: Allow ensure-tox to upgrade tox version  https://review.opendev.org/67646416:52
pabelangercorvus: mordred: yes, that looks like the issue static/static is missing16:54
corvusi'm trying to figure out what we're missing from the release job16:55
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: POC: Enable tox-molecule on ensure-tox  https://review.opendev.org/67276016:55
mordredcorvus: sorry - got distracted - looking in to it now16:56
pabelangeryah, same16:58
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: Allow ensure-tox to upgrade tox version  https://review.opendev.org/67646416:58
pabelangermaybe yarn is missing?16:58
pabelangerhttps://logs.opendev.org/62/6254985dd726b8df8f4b73ef148dbec9ea7c48d9/release/opendev-release-python/4acda91/job-output.txt.gz#_2019-08-13_18_15_57_74196116:59
pabelangerhttps://logs.opendev.org/62/6254985dd726b8df8f4b73ef148dbec9ea7c48d9/release/opendev-release-python/4acda91/job-output.txt.gz#_2019-08-13_18_16_01_291661 confirms it is empty16:59
corvuswhere was release-zuul-python defined?17:00
corvusthat's the job we used to run17:00
corvusah it's in openstack/project-config17:00
pabelangeryah17:00
openstackgerritMonty Taylor proposed zuul/zuul master: Install yarn before building python artifacts  https://review.opendev.org/67646617:01
pabelangerso, wrong job?17:01
corvusmordred: comment on that17:01
pabelangermordred: should we just move https://opendev.org/openstack/project-config/src/branch/master/playbooks/zuul-tarball/pre.yaml intree?17:02
mordredyah - was just fixing that :)17:02
openstackgerritMonty Taylor proposed zuul/zuul master: Install yarn before building python artifacts  https://review.opendev.org/67646617:02
pabelangerand revoke sudo17:02
corvusi'm not sure about revoke sudo17:03
corvusyeah, it's in run17:03
corvuswe don't need it in pre17:03
pabelangerwfm17:03
mordred++17:03
mordredcorvus: this should let us make new artifacts ... I think we want a followup to do what we were discussing yesterday and build tarballs without built javascript but wheels with17:04
corvusmordred: i think this is still incomplete17:04
pabelangerhowever17:04
corvusmordred: agree re followup17:04
pabelangeropendev-release-python is what we use for release pipeline and promote17:04
pabelangerso we also need them there too (yarn)17:04
corvusyeah that ^17:04
corvusso this will fix the branch-tip artifacts17:04
corvuspabelanger's thing will fix the released ones17:04
pabelangeryah17:05
corvuspabelanger: given that you aren't using the pypi branch-tip releases, can we stop releasing there on every commit?17:05
pabelangercorvus: we still use them, for nodepool we were on specific commit17:05
corvushrm.17:06
pabelangerI mean, we can drop the pre-release but it was to avoid asking for a new release (tag)17:06
corvusyeah, it's just it makes the pypi page look terrible17:07
corvushave you tried to use it?17:07
pabelangerthey were hidden before, did that change?17:07
pabelangeror thought they were hidden17:08
corvuspabelanger: https://pypi.org/project/zuul/#history17:08
openstackgerritMonty Taylor proposed zuul/zuul master: Install yarn before building python artifacts  https://review.opendev.org/67646617:08
mordredcorvus, pabelanger: how's that look now?17:08
corvusmordred: makes sense17:09
pabelangerwe missed promote, but that is what we are discussing now17:09
corvuspabelanger: no, nothing in promote needs to change to fix the problem17:10
pabelangercorvus: it is helpful to have a wheel per commit, means we don't have to change a lot of tooling to install a commit we want.  But also understand it is ungly17:10
pabelangerugly*17:10
pabelangerso, we can drop for now if want17:10
pabelangercorvus: mordred: +317:11
pabelangeralso, if we do 3.10.1, we need renonote right?17:14
corvusyep, can't have a release without a reason :)17:14
*** spsurya has quit IRC17:14
pabelangermordred: want me to add that, in follow up?17:14
corvuspabelanger: yes please :)17:18
pabelangerk17:19
openstackgerritPaul Belanger proposed zuul/zuul master: Add release note for yarn dependencies missing  https://review.opendev.org/67646817:25
pabelangercorvus: mordred: ^17:26
corvuswe should be able to verify all of that is working before we tag the release17:26
pabelangeragree17:26
pabelangermaybe add check to error if they are missing in future too17:27
mordredpabelanger, corvus: crappit. there's still one more thing we'r emissing17:29
mordredwe need to set release_python - one sec, patch coming17:29
corvuspabelanger: how about you *don't* make your change depend on mordred's17:30
mordred:)17:30
openstackgerritMonty Taylor proposed zuul/zuul master: Install yarn before building python artifacts  https://review.opendev.org/67646617:32
openstackgerritMonty Taylor proposed zuul/zuul master: Build wheels with javascript and tarballs without  https://review.opendev.org/67647017:32
corvusmordred: oh, for the py3 thing17:32
mordredyeah17:32
mordredotherwise the wheels are wrong17:32
mordredalso - there's a stab at the more complex thing of not putting javascript into source tarballs17:33
corvusmordred: cool, do we want to hold that for after this fix release?17:33
mordredprobably so, yes17:33
mordredI think we should warn people about that, just in case17:33
corvusyeah.  maybe a -discuss post is warranted17:34
corvusmordred: the good news is that the "py2" wheel looks like it is the right size: https://71dff8999a9cb00d39ca-d5fbed16a2dc3218104e32d2a6839727.ssl.cf2.rackcdn.com/676466/3/gate/zuul-build-python-release/d239f76/artifacts/ :)17:34
mordredyay!17:35
mordredsending the -announce post now for the other thing17:35
mordredcorvus: the original job in project-config also set node_version: 10 ... do we care?17:38
mordredI can update this one more time real quick17:38
corvusmordred: we seem to still use that everywhere, so probably so17:38
corvusfor the build-javascript-content-tarball job17:39
mordredkk. one sec - I can also suck in pabelanger's release note patch too17:39
corvusand maybe updating that is another followup change17:39
mordredcorvus: wait - updating what is another followup change?17:40
corvusmordred: changing the node version?17:41
corvusor is 10 the better one to use?17:41
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Sync fetch-sphinx-tarball with fetch-sphinx-output  https://review.opendev.org/67647217:41
mordred10 is the better one - it's an override of the older default version in install-npm17:41
mordredwhich defaults to node version 617:43
pabelangersorry, stepped away17:43
pabelangermordred: go for it17:43
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Partial sync fetch-sphinx-tarball with fetch-sphinx-output  https://review.opendev.org/67647217:44
openstackgerritMonty Taylor proposed zuul/zuul master: Install yarn before building python artifacts  https://review.opendev.org/67646617:44
openstackgerritMonty Taylor proposed zuul/zuul master: Build wheels with javascript and tarballs without  https://review.opendev.org/67647017:44
clarkbya I updated the node version across the board for zuul not too long ago to 10 because one job was using 6 another 8 and another 1017:44
mordredpabelanger, corvus: THAT should finally be accurate17:44
pabelangerlulz17:44
clarkband I wanted to eliminate any potential behavior differences while I was debugging a problem17:44
mordredyeah17:44
pabelanger+217:44
mordredclarkb: https://review.opendev.org/#/c/676466 look ok to you?17:45
clarkbya not approving because distracted this week17:46
mordredtotes17:46
corvusokay, that's approved.  when it lands, we'll inspect the wheel/tarball and if it looks good, cut a .117:47
corvuspabelanger: also, it looks like we already stopped doing the pre-release stuff on pypi17:47
corvusafter this release, i kinda want to go in and 'hide' all those17:48
pabelangercorvus: okay, thats fine. usually is really an issue when we want to get something from master, but unable to tag release for some reason. Hopefully things are getting more stable the longer we do this :)17:48
openstackgerritMonty Taylor proposed zuul/zuul master: Build wheels with javascript and tarballs without  https://review.opendev.org/67647017:49
mordredpabelanger: ++17:56
openstackgerritMonty Taylor proposed zuul/zuul-jobs master: Log errors better in case of unknown REST errors  https://review.opendev.org/67647617:57
openstackgerritMerged zuul/zuul-jobs master: Return preview artifact in fetch-sphinx-output  https://review.opendev.org/67643718:00
openstackgerritMerged zuul/zuul-jobs master: Partial sync fetch-sphinx-tarball with fetch-sphinx-output  https://review.opendev.org/67647218:00
*** rlandy|rover is now known as rlandy|rover|brb18:10
fungimordred: corvus: it's failing zuul-build-python-release in check18:22
fungioh i guess 676466 is the one we need for the patch release, not 67647018:23
*** rlandy|rover|brb is now known as rlandy|rover18:26
openstackgerritMerged zuul/zuul master: Install yarn before building python artifacts  https://review.opendev.org/67646618:31
pabelangerhttps://logs.opendev.org/66/676466/5/promote/opendev-promote-python/24e32e2/job-output.txt.gz18:37
pabelangerpromote failed18:37
pabelangerhttps://zuul.opendev.org/api/tenant/zuul/builds?change=676466&patchset=5&pipeline=gate&job_name=build-python-release18:40
pabelangeris empty18:40
pabelangerbut download-artifact requires some value?18:40
*** mattw4 has quit IRC18:41
pabelangercorvus: mordred: do you know why that would be empty?18:41
pabelangerhttps://opendev.org/zuul/zuul-jobs/src/branch/master/roles/download-artifact/tasks/main.yaml#L5 seems to expect a result18:41
*** mattw4 has joined #zuul18:44
mordredyeah - it's looking for artifacts from build-python-release -- but we replaced that with zuul-build-python-release -- one sec19:05
pabelangertobiash: github driver seems faster, but also could just be excited about improvements I know about19:08
openstackgerritMonty Taylor proposed zuul/zuul master: Override the job name from which we promote  https://review.opendev.org/67649419:09
mordredpabelanger, fungi: ^^19:09
tobiashpabelanger: it is definitely faster in our deployment19:10
pabelangertobiash: \o/19:10
pabelangermordred: +219:11
pabelangertobiash: only think we are waiting for now, is new github3.py release. Happy to move to next thing and not worry about github now19:12
openstackgerritMonty Taylor proposed zuul/zuul master: Override the job name from which we promote  https://review.opendev.org/67649419:14
openstackgerritMonty Taylor proposed zuul/zuul master: Build wheels with javascript and tarballs without  https://review.opendev.org/67647019:14
mordredfungi: ^^ that should fix the other thing19:14
mordredcorvus: whence you en-backen ... https://zuul.opendev.org/t/zuul/build/cfbe3f43bd604e509cc1b196cce66f15/log/job-output.txt#340 has an error message that shows as SUCCESS in the console https://zuul.opendev.org/t/zuul/build/cfbe3f43bd604e509cc1b196cce66f15/console19:19
openstackgerritJeff Liu proposed zuul/zuul-operator master: Create zookeeper operator and zuul CR to k8s test  https://review.opendev.org/67645819:36
corvusback; catching up19:44
mordredcorvus: we did some things, and also some other things19:47
mordredcorvus: mostly just ran around throwing our hands in the air19:48
corvusmordred: ah yep -- https://review.opendev.org/676494 is almost the solution but i think you missed a thing19:49
mordredcorvus: HAHAHAHAHAHA19:50
mordredcorvus: that wasn't so much like overriding as just redefining to the same value19:50
openstackgerritMonty Taylor proposed zuul/zuul master: Override the job name from which we promote  https://review.opendev.org/67649419:50
corvusmordred: yeah, as overrides go, it wasn't the most effective19:51
mordredcorvus: I thnik for the console thing above it's because the fake thing we inject into the log for errors like that doesn't have a status with it?19:51
corvuspabelanger: want to +3 https://review.opendev.org/67649419:51
pabelangerdone!19:51
mordredcorvus: but maybe we shoudlnt' spend too much energy on fixing it until we're all the way to the yaml - bcause I thnik defining a data structure for "this is what an injected fake message should look like" is probably important there so that we can also inject the messages from the executor19:52
mordredcorvus: speaking of - https://logs.opendev.org/56/623256/7/check/tox-py36/eaa1c6d/testr_results.html.gz -19:54
mordredyaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/new:ansible.parsing.yaml.objects.AnsibleUnicode'19:54
mordredcorvus: we've got a trick somewhere already that lets us dump yaml in python3 without putting in unicode objects right?19:54
fungii have some, just a sec19:58
mordredfungi: I thnik the trick in this case might be that I nede to import from zuul.lib.yamlutil instead of yaml?19:59
fungihrm, yeah i thought i had an example handy but didn't find it20:00
* pabelanger starts reading AWS driver docs for nodepool20:00
openstackgerritMonty Taylor proposed zuul/zuul master: Add appending yaml log plugin  https://review.opendev.org/62325620:02
fungimordred: not sure, looking at yamlutil.py it seems to mostly just be dealing with optional yamllib availability20:03
fungithe dumper doesn't look like it does anything fancy20:04
mordredhrm. yeah20:04
mordredmaybe just the difference between safe_dump and dump?20:04
mordredyeah. I think that's it20:05
mordredsafe_dump(data, stream=None, **kwds)20:05
mordred    Serialize a Python object into a YAML stream.20:05
mordred    Produce only basic YAML tags.20:05
fungioh, got it20:05
fungiyeah i guess that will do it20:06
mordredwe'll see :)20:06
fungianyway, using the class from zuul.lib.yamlutil seems like a good idea regardless for consistency20:06
*** saneax has quit IRC20:07
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Revert "Partial sync fetch-sphinx-tarball with fetch-sphinx-output"  https://review.opendev.org/67651120:13
corvusmordred: yeah, i think this is fundamentally a different problem than the other one, but the solution will textually conflict, so we may as well solve them together?20:19
corvusmordred: basically, i think you identified a place where we create a fake task record, but we forgot to set "failed: true" on it.20:20
corvusthat's a simple enough fix20:20
mordredyeah20:20
corvusmordred: hrm, now that i think about it, maybe we should just go ahead and fix that quickly then base the yaml appending stuff on that?20:21
mordredcorvus: or - actually ...20:21
mordredcorvus: I don't think that's actually the issue here20:21
corvusmordred: just because i think this is a one-liner, and we're not sure how quickly we can roll out the yaml thing20:21
mordredI think this is passthrough of incomplete data20:22
corvusmordred: hrm.  aiui, https://zuul.opendev.org/t/zuul/build/cfbe3f43bd604e509cc1b196cce66f15/log/job-output.json#6332 should have "failed": true  under tasks.hosts.ubuntu-bionic20:22
mordredcorvus: yup20:23
mordredbut I don't think we're creating that20:23
corvusoh neat20:23
mordredyeah. "neat"20:25
mordredI'm staring at it to see if there's any "good" way to do a fix in a couple of lines20:25
corvusmordred: do you know which method gets called for that?20:25
mordredthe json plugin is very simple - it calls v2_runner_on_ok for both20:26
mordredsince all we're really doing is passing through the structure20:26
corvusmordred: ah, so maybe we need to fan that out and add in a failed field if it isn't there?20:26
mordredbut what I think we might want to do instead is make actual methods20:26
mordredyeah20:26
corvusmordred: maybe it makes sense to have all 4 methods and store all 4 things, then we can make sure our statuses match ansible's20:27
corvusright now, i think we're actually missing unreachable :)20:27
mordredyeah20:27
mordredand something like for host in self.results[-1]['tasks'][-1]['hosts'].values(): host.setfault('failed', True)20:28
mordredin v2_runner_on_ok20:28
mordredgah20:28
mordredin v2_runner_on_failed20:29
corvusmordred: how about just  host.setfault('failed', True) ?20:29
mordredshould there be skipped=true and unreachable=true too then?20:29
corvusmordred: we don't need to set the failed attribute on successful hosts20:29
mordredah - good point20:29
openstackgerritMerged zuul/zuul-jobs master: Revert "Partial sync fetch-sphinx-tarball with fetch-sphinx-output"  https://review.opendev.org/67651120:30
*** saneax has joined #zuul20:31
corvusmordred: and yeah, i think either that, or we should use one var (eg 'result')20:31
openstackgerritMonty Taylor proposed zuul/zuul master: Set failed, unreachable, skipped statuses in json plugin  https://review.opendev.org/67651620:33
mordredcorvus: ^^ somethign liek that?20:33
corvusmordred: if i'm reading the really roundabout code in the upstream json plugin correctly, it should set "failed: true" and "skipped: true" on the host result object20:35
mordredoh yeah?20:36
openstackgerritMerged zuul/zuul master: Override the job name from which we promote  https://review.opendev.org/67649420:36
corvusmordred: so i thkn your change mirrors the upstream behavior, except that i have no idea what happens on unreachable20:36
mordredoh - gotcha. so set each of those20:36
corvusmordred: so maybe we don't need to handle unreachable because something already does?  i just don't know what20:37
corvusmordred: https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/json.py#L13920:39
corvusi interpret that as "if the name of the method that is being called is 'v2_runner_on_failed' then set 'failed: True' on the host result20:40
corvusditto for skipped20:40
corvusso maybe the host result already has a field for unreachable?20:40
mordredmaybe so?20:40
mordredor maybe you only get a host if the host got reached?20:40
corvusmordred: oh interesting20:41
corvusmordred: so i think we're good on failed/skipped -- can you set up a local test to figure out unreachable?  (i'm guessing if you just put a bogus thing in inventory...)20:41
mordredcorvus: yeah20:42
corvuscool, i'm going to go do the swift thing, biab.20:42
mordredcorvus: cool. I'm about to EOD so I'll do the local test in the morning20:42
mordred(well, I'm about to do a phone call - but when it's done I'll be EOD - so same result)20:43
corvusack20:43
pabelangerso, just happen to look at tower release notes, new release today. And https://nvd.nist.gov/vuln/detail/CVE-2019-12439 was linked as security issue for bubblewrap20:46
pabelangerlooking at ubuntu bionic, I don't see any fix published yet20:47
*** rfolco has quit IRC20:51
pabelangerseems low chance20:53
pabelangerlikely why the delay on release20:53
pabelangerhowever, first needs local SSH access to executor20:53
mordredcorvus: yes - unreachable=True is on the host record http://paste.openstack.org/show/757014/20:55
openstackgerritMonty Taylor proposed zuul/zuul master: Set failed, unreachable, skipped statuses in json plugin  https://review.opendev.org/67651620:57
pabelangermordred: corvus: release job looks correct now: https://logs.opendev.org/94/676494/3/gate/zuul-build-python-release/eaefe32/artifacts/20:57
pabelangerand promote worked20:57
mordredcorvus: but also there was an error in the first patch20:57
mordredpabelanger: woot20:57
pabelangerthink we are ready to tag20:57
pabelangerconfirmed, wheel is fine now20:59
pabelangerOh, looks like we'll pick up 2 commits too21:08
pabelangerhttps://review.opendev.org/676466/ and https://review.opendev.org/676404/21:09
*** saneax has quit IRC21:10
*** jeliu_ has quit IRC21:16
pabelangercorvus: if you don't mind sending ping if you tag today, I'll then upgrade to 3.10.1 for zuul.a.c. For now, I have to #dadops for a bit21:18
pabelangerthanks!21:18
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Log errors better in case of unknown REST errors  https://review.opendev.org/67647621:22
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Retry more operations  https://review.opendev.org/67651821:22
corvusmordred, pabelanger: ^ i've tested that in rax and can confirm that it doesn't break the case where everything works :)21:22
corvusmordred: (i also fixed a couple of things with your earlier patch)21:23
corvusmordred: that's great news on the unreachable thing; i think 676516 should take care of it!  it'll be easier to update the javascript once that's in production, so i say we wait for that for the next step21:25
corvuspabelanger: i will start tagging .1 now21:25
corvus3aa36dd2d141a018ad5ba26146d979d4ac0d3eea will be the commit21:25
pabelangercorvus: ack thanks, will look at swift stuff once back21:26
mordredcorvus: fwiw - since you're already importing openstacksdk - the iterate_timeout function is available if you want to use it21:36
mordredcorvus: openstack.utils.iterate_timeout21:36
mordredcorvus: (I like what you have - just mentioning it's availability)21:37
corvusmordred: oh neat.  well, i thought about using iterate_timeout but i thought this approach might be more suitable (with the built-in exception handler, it makes the calling site much simpler (at the expense of throwing in a lambda))21:40
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Retry more operations  https://review.opendev.org/67651821:42
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Retry more operations  https://review.opendev.org/67651821:42
corvusmordred: okay, i think i have added the required whitespace :)21:42
mordred+@21:46
*** rfolco has joined #zuul21:56
*** mattw4 has quit IRC21:57
*** mattw4 has joined #zuul21:58
corvusi sent out an announcement email, so i think that concludes the "forgot the javascript in the release" fire drill :)22:06
*** rlandy|rover has quit IRC23:12
*** sshnaidm is now known as sshnaidm|afk23:38
pabelanger\o/23:54
pabelangerstarting upgrades now23:54
pabelangeroh, I'll start in a few hours, periodic jobs about to fire :)23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!