| clarkb | jkt: what you are describing seems to be very similar and basically we only want compressed file to be set with a compressed type others shouldb't be so we get them back as expected | 00:01 |
|---|---|---|
| jkt | now that's it, I think, thanks! | 00:01 |
| jkt | confirming, this fixed my problem, thanks a lot | 00:13 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 00:22 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-pip: export ensure_pip_virtualenv_command https://review.opendev.org/718224 | 00:22 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] fetch-zuul-cloner: use ensure-pip https://review.opendev.org/717882 | 00:22 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] use ensure-pip in fetch-subunit-output test https://review.opendev.org/718225 | 00:23 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-tox: use ensure-pip role https://review.opendev.org/717663 | 00:23 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 00:23 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: make idempotent and update testing https://review.opendev.org/718284 | 00:23 |
| *** zxiiro has joined #zuul | 00:23 | |
| *** armstrongs has joined #zuul | 00:25 | |
| *** armstrongs has quit IRC | 00:29 | |
| *** ysandeep|away is now known as ysandeep|rover | 01:17 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: make idempotent and update testing https://review.opendev.org/718284 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-pip: export ensure_pip_virtualenv_command https://review.opendev.org/718224 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] fetch-zuul-cloner: use ensure-pip https://review.opendev.org/717882 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] use ensure-pip in fetch-subunit-output test https://review.opendev.org/718225 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-tox: use ensure-pip role https://review.opendev.org/717663 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 01:20 |
| *** swest has quit IRC | 01:27 | |
| *** swest has joined #zuul | 01:42 | |
| *** ysandeep|rover is now known as ysandeep|brb | 02:10 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-tox: use ensure-pip role https://review.opendev.org/717663 | 02:30 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 02:30 |
| *** rlandy has quit IRC | 02:37 | |
| *** Goneri has quit IRC | 02:40 | |
| *** zxiiro has quit IRC | 03:27 | |
| *** bhavikdbavishi has joined #zuul | 03:35 | |
| *** bhavikdbavishi1 has joined #zuul | 03:38 | |
| *** bhavikdbavishi has quit IRC | 03:39 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:39 | |
| *** ysandeep|brb is now known as ysandeep|rover | 03:42 | |
| *** cdearborn has quit IRC | 03:59 | |
| *** evrardjp has quit IRC | 04:36 | |
| *** evrardjp has joined #zuul | 04:37 | |
| *** sgw has quit IRC | 05:13 | |
| *** bhavikdbavishi has quit IRC | 05:27 | |
| *** y2kenny has quit IRC | 05:29 | |
| *** igordc has quit IRC | 05:32 | |
| *** bhavikdbavishi has joined #zuul | 05:34 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 06:05 |
| *** bhavikdbavishi has quit IRC | 06:12 | |
| *** bhavikdbavishi has joined #zuul | 06:27 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 06:27 |
| *** gtema has joined #zuul | 06:30 | |
| *** gtema has quit IRC | 06:38 | |
| *** gtema has joined #zuul | 06:40 | |
| *** gtema has quit IRC | 06:45 | |
| *** gtema has joined #zuul | 06:45 | |
| *** jcapitao has joined #zuul | 06:54 | |
| *** avass is now known as Guest28731 | 06:56 | |
| *** avass has joined #zuul | 06:56 | |
| *** rpittau|afk is now known as rpittau | 07:17 | |
| *** ysandeep|rover is now known as ysandeep|lunch | 07:24 | |
| *** bhavikdbavishi has quit IRC | 07:27 | |
| *** sshnaidm|afk is now known as sshnaidm | 07:42 | |
| *** gtema has quit IRC | 07:44 | |
| *** bhavikdbavishi has joined #zuul | 07:49 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 07:52 |
| *** gtema has joined #zuul | 07:58 | |
| *** tosky has joined #zuul | 08:00 | |
| *** dmellado has quit IRC | 08:01 | |
| *** gtema has quit IRC | 08:04 | |
| *** dmellado has joined #zuul | 08:05 | |
| *** bhavikdbavishi has quit IRC | 08:05 | |
| *** gtema has joined #zuul | 08:07 | |
| *** dpawlik has joined #zuul | 08:08 | |
| *** bhavikdbavishi has joined #zuul | 08:16 | |
| *** ysandeep|lunch is now known as ysandeep|rover | 08:29 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 08:48 |
| *** harrymichal has joined #zuul | 09:02 | |
| jkt | can I stop all other jobs that are queued or running for the current change when a given "most significant" job fails? | 09:24 |
| jkt | I know about https://zuul-ci.org/docs/zuul/reference/job_def.html#attr-job.hold-following-changes , but I like the paralelization | 09:24 |
| *** harrymichal has quit IRC | 09:24 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 09:26 |
| jkt | ah, it seems that I am actually looking for https://zuul-ci.org/docs/zuul/reference/project_def.html#attr-project.%3Cpipeline%3E.fail-fast | 09:26 |
| *** bhavikdbavishi has quit IRC | 10:11 | |
| andreykurilin | hi folks! The case project X defines a job, project Y inherits it. Project X would like to change job and Project Y would like to use old version of the job for stable branches. How the things should be done? `required-projects: [{"name": "ProjectX", "override-checkout": "old-tag"}]` doesn't help | 10:12 |
| jkt | aaaaargh, looks like the virtual env installation of Ansible on centos7 with SCL-provided Python doesn't really work: | 10:15 |
| jkt | Ansible output: b'/var/lib/zuul/ansible-bin/2.8/bin/python3: error while loading shared libraries: libpython3.6m.so.rh-python36-1.0: cannot open shared object file: No such file or directory' | 10:15 |
| *** harrymichal has joined #zuul | 10:15 | |
| *** bhavikdbavishi has joined #zuul | 10:15 | |
| jkt | which is weird because I can exec that /var/lib/zuul/ansible-bin/2.8/bin/python3 just fine from my shell, and `ldd` shows that the path to that lib is given as an absolute path, so... | 10:18 |
| jkt | ah, but of course bwrap doesn't allow that path | 10:19 |
| *** rpittau is now known as rpittau|bbl | 10:24 | |
| *** harrymichal has quit IRC | 10:28 | |
| *** ysandeep|rover is now known as ysandeep|afk | 10:30 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: WIP: Support per branch change queues https://review.opendev.org/718381 | 10:36 |
| jkt | enabling that via bwrap was easy, now next milestone for this Zuul 3.8.1 -> 3.18 update: ModuleNotFoundError: | 10:52 |
| jkt | No module named 'openstack' | 10:52 |
| jkt | via upload-logs-swift | 10:52 |
| *** sshnaidm is now known as sshnaidm|afk | 10:56 | |
| jkt | how do I "cleanly" specify that a role requires a certain package from pip? | 10:58 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 10:59 |
| jkt | right, so this is apparently only because my initial installation failed due to that lib path filtering | 11:07 |
| *** jcapitao is now known as jcapitao_lunch | 11:07 | |
| jkt | I see that it's installing the openstack package now, nice | 11:08 |
| jkt | hmmm, so the installation fails: http://paste.openstack.org/show/791798/ | 11:21 |
| AJaeger | andreykurilin: copy the job into project Y and add it to the branches you need it one | 11:25 |
| jkt | oh yeah, let's try a random update for https://github.com/pypa/pip/issues/4216 , this box cannot possibly get any more broken | 11:27 |
| openstackgerrit | Benjamin Schanzel proposed zuul/nodepool master: k8s/OKD Provider: Don't Set ca_cert if TLS verification is skipped https://review.opendev.org/718397 | 11:28 |
| andreykurilin | AJaeger: so there is no way to avoid duplication? and `override-checkout` works after playbooks and roles are loaded, right? | 11:28 |
| AJaeger | andreykurilin: I don't think override-checkout will help with job definitions. I have no other idea but maybe others have once they are awake | 11:30 |
| *** ysandeep|afk is now known as ysandeep|rover | 11:31 | |
| *** bhavikdbavishi has quit IRC | 11:46 | |
| andreykurilin | AJaeger: thanks! one more question: Does override-checkout affects zuul role files? | 11:49 |
| AJaeger | andreykurilin: I don't know directly, best ask again later | 11:50 |
| andreykurilin | AJaeger: anyway, you helped a lot. thanks! | 11:50 |
| jkt | this starts getting interesting -- apparently Zuul installs setuptools-28.8.0 for all of its /var/lib/zuul/ansible-bin/2.9/lib/python3.6/site-packages/ , using PIP 9 for that | 12:01 |
| jkt | even though if I do something like `python -m venv WTF` with my host's python3, I get a newer PIP | 12:02 |
| webknjaz | When `virtualenv` (not `venv`) is used, it comes with a certain version of pip. If you want newer pip in fresh envs, virtualenv should be upgraded. | 12:04 |
| webknjaz | stdlib `venv` uses what's bundled in `ensurepip` that usually comes as a part of the CPython distribution | 12:05 |
| jkt | webknjaz: OK, this is centos7 with Python 3.6 from RH's SCL, so a pretty ancient thing. Let me try updating host's virtualenv as well in addition to pip and setuptools | 12:08 |
| webknjaz | pip/virtualenv upgrade won't affect what's inside of the new/fresh virtualenvs. Only outer env. | 12:09 |
| *** jcapitao_lunch is now known as jcapitao | 12:14 | |
| *** Goneri has joined #zuul | 12:16 | |
| openstackgerrit | Albin Vass proposed zuul/zuul master: WIP: Enables whitelisting and configuring callbacks https://review.opendev.org/717260 | 12:16 |
| jkt | webknjaz: thanks a lot, that did the trick | 12:17 |
| jkt | webknjaz: would it make sense to extend zuul's zuul/lib/ansible-config.conf with a dep on new enough virtualenv? Or is that actually too late and should Zuul itself depend on that one? | 12:18 |
| avass | corvus: thoughts on 717260 so far? | 12:18 |
| *** rlandy has joined #zuul | 12:19 | |
| *** sshnaidm|afk is now known as sshnaidm | 12:19 | |
| *** rpittau|bbl is now known as rpittau | 12:23 | |
| jkt | webknjaz: I don't understand this -- Zuul's own CI builds run on Ubuntu Xenial, and they have virtualenv=15.0.1-something, right? https://packages.ubuntu.com/xenial-updates/virtualenv | 12:23 |
| jkt | webknjaz: how come that my system (which used 15.1.0) did not work? | 12:23 |
| jkt | ah, perhaps because that one doesn't use systemwide installation of virtualenv, but rather gets it via tox | 12:25 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: WIP: Enables whitelisting and configuring callbacks https://review.opendev.org/717260 | 12:26 |
| *** rlandy has quit IRC | 12:31 | |
| *** rlandy has joined #zuul | 12:31 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 12:32 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add zuul-registry deployment https://review.opendev.org/710650 | 12:36 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add zuul-preview service https://review.opendev.org/718419 | 12:36 |
| webknjaz | @jkt: `tox` uses `virtualenv` from the same env where it's installed, it's not "inside". btw AFAIR the latest tox requires virtualenv to be v16+ | 12:43 |
| openstackgerrit | Jan Kundrát proposed zuul/zuul master: Ensure we use recent enough virtualenv https://review.opendev.org/718425 | 12:45 |
| jkt | webknjaz: can you please check https://review.opendev.org/718425 to make sure I got you right? | 12:45 |
| *** gtema has quit IRC | 12:47 | |
| webknjaz | I don't see tox there, is it the same env? | 12:47 |
| *** gtema has joined #zuul | 12:49 | |
| jkt | webknjaz: the error message is from a real deployment on el7 + system installation of the virtualenv package from RH's SCL Python distribution | 12:49 |
| jkt | I checked upstream's virtualenv release history, and I have to say I have no idea which version pulls in "recent enough" setuptools to not break | 12:50 |
| jkt | and given that the CI uses "whatever is latest", it made sense to me to pull that as a requirement, given that I know that 15.1.0 doesn't work for me, and that Ubuntu ships even older releases | 12:50 |
| webknjaz | I mean, I'm not sure where tox is installed. But if it's there, it's probably OK. | 12:59 |
| *** saneax has quit IRC | 13:01 | |
| *** sgw has joined #zuul | 13:02 | |
| *** cdearborn has joined #zuul | 13:13 | |
| zbr | webknjaz: if you are referring to ZAC instance, i am aware that there is an outdated tox there, probably pre-backed by pabelanger | 13:14 |
| zbr | in fact that is the reason why I am working on https://review.opendev.org/#/c/690057/ -- so we can enforce it to upgrade or to install extra deps | 13:15 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Support per branch change queues https://review.opendev.org/718381 | 13:23 |
| *** rlandy is now known as rlandy|pto | 13:24 | |
| *** ysandeep|rover is now known as ysandeep|away | 13:25 | |
| *** hashar has joined #zuul | 13:28 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 13:35 |
| *** gtema has quit IRC | 13:40 | |
| corvus | avass: that looks nearly ready -- i have one question (just for my own curiosity), and one suggested change. then we'll need docs. | 13:51 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Report retried builds in a build set via mqtt. https://review.opendev.org/632727 | 13:52 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Report retried builds via sql reporter. https://review.opendev.org/633501 | 13:52 |
| *** gtema has joined #zuul | 13:53 | |
| *** gtema has quit IRC | 13:56 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add --validate-tenants option to zuul scheduler https://review.opendev.org/542160 | 13:58 |
| avass | corvus: huh, I had to double check the callback_plugins path | 14:07 |
| avass | corvus: seems it is added by default, must have been something else I changed at the same time that made it work | 14:08 |
| tobiash | corvus: I've responded on 718381, what do you think? | 14:11 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: WIP: Enables whitelisting and configuring callbacks https://review.opendev.org/717260 | 14:20 |
| corvus | tobiash: at least i'm consistent :) | 14:32 |
| tobiash | corvus: yeah, I totally agree that it should be allowed optionally | 14:32 |
| tobiash | corvus: I've responded a proposal to the tenant config just a minute ago to that change | 14:33 |
| tobiash | that would be consistent with the circular deps change then | 14:34 |
| *** bhavikdbavishi has joined #zuul | 14:38 | |
| fungi | jkt: why did you need to use scl? i thought the latest centos 7.x release added python3 directly to the main archive | 14:40 |
| corvus | tobiash: i'm afraid i may have just added more questions and things to think about :/ | 14:41 |
| *** bhavikdbavishi1 has joined #zuul | 14:41 | |
| corvus | tobiash: did we decide that circular deps needs to be in the tenant config because projects with two different settings for circular deps are incompatible? | 14:43 |
| *** bhavikdbavishi has quit IRC | 14:43 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 14:43 | |
| AJaeger | here's a change for chaning the way we remove the ssh-key in zuul-base-jobs, first for base-test. It looks sane to me but wanted to give others a chance to chime in. I'll +2 now and will later +A if nobody objects - https://review.opendev.org/#/c/708871/4 Reviews welcome (and feel free to +A) | 14:43 |
| tobiash | corvus: I think we decided this for cdep because it felt naturally allowing it in the tenant config | 14:43 |
| corvus | avass: i sent you another private message :) | 14:43 |
| *** y2kenny has joined #zuul | 14:45 | |
| *** igordc has joined #zuul | 14:45 | |
| corvus | tobiash: ah, i guess we should figure out which is the most natural thing here; considering that we support different change queues within a tenant, and it's plausible that one group of projects may want split-branch queues and another may want shared-branch queues, it doesn't seem straightforward to me. i think we'll need to think about whether it's possible to support both inside of a tenant, or if that's | 14:46 |
| corvus | too difficult and we just need to make it a tenant setting... | 14:46 |
| tobiash | corvus: re in-repo for per branch change queues. I think that's even easier than the tenant config as it can be handled entirely in buildChangeQueues in the dependent manager | 14:46 |
| tobiash | corvus: the tenant config way would support both as it's overridable per project | 14:46 |
| tobiash | at least it's already handled like this in the current cdep implementation | 14:47 |
| corvus | tobiash: yeah, but you need to ask your administrator to override it :) | 14:47 |
| tobiash | yes | 14:47 |
| corvus | tobiash: hrm. do you think that's the right decision for cdep? | 14:47 |
| corvus | tobiash: should both of these be in-repo instead of in tenant config? | 14:47 |
| tobiash | corvus: thinking deeper about that I think it makes sense | 14:48 |
| tobiash | corvus: so I guess it would look like this: https://etherpad.openstack.org/p/uQ5XkWptud | 14:49 |
| tobiash | and only allowed in a config project | 14:49 |
| corvus | tobiash: but what happens if 2 projects have different settings for either of those? | 14:51 |
| AJaeger | can we say "allowed-projects" with a regex like "allowed-projects: ^openstack/.*"? | 14:51 |
| tobiash | corvus: when specifying the queue in the config project it should take precedence no matter which settings? | 14:51 |
| corvus | AJaeger: i don't think so | 14:52 |
| corvus | tobiash: that sort-of happens now, as long as you list the config-project first, since it's first-one-wins | 14:52 |
| AJaeger | corvus: ok | 14:52 |
| y2kenny | is there a recommanded way to write to the inventory or add to zuul.resources inside a job/playbook? (let say I use the k8s/openshift driver's namespace type and created pods "manually/not by nodepool" inside the job/playbook.) | 14:53 |
| tobiash | corvus: yes, so I think that makes sense. So when one wants to use queue per branch he must omit the queue in the config project | 14:54 |
| corvus | tobiash: well, we need to support setting queue-per-branch entirely in the config project too | 14:56 |
| corvus | tobiash: (because some sites don't want to use any untrusted in-repo config) | 14:56 |
| tobiash | hrm, yes, that's also true for some of our projects | 14:56 |
| tobiash | corvus: what about a 'change-queue-per-branch' setting instead of an 'allow-change-queue-per-branch'? | 14:57 |
| tobiash | and having the queue then being impicit integrated-<branch> in this case? | 14:57 |
| tobiash | I think that might more sense from projects point of view | 14:58 |
| corvus | tobiash: i think having the 'change-queue-per-branch' setting makes sense, but i'm concerned about that meaning an implicit queue name | 14:59 |
| corvus | i think it would make more sense if it were automatic, sort of like how you have it implemented now | 14:59 |
| tobiash | the name is not used anywhere I think so we could just have multiple 'integrated' queues then | 14:59 |
| corvus | tobiash: but i'm still not sure what the behavior should be if two projects have different settings | 15:00 |
| tobiash | I think we'd need some sort of precedence here | 15:01 |
| tobiash | as the per-branch attribute does match the queue itself more than the project | 15:01 |
| tobiash | ideally we'd create a queue object and reference that from the projects | 15:01 |
| tobiash | similar to semaphore | 15:01 |
| corvus | tobiash: i was just thinking the same thing | 15:02 |
| corvus | tobiash: i need to get some breakfast, biab :) | 15:03 |
| tobiash | wow, design discussions pre-breakfast | 15:03 |
| tobiash | I'm impressed :) | 15:03 |
| *** zxiiro has joined #zuul | 15:04 | |
| fungi | time is an illusion, breakfast time doubly so? | 15:11 |
| * fungi has already done six impossible things before breakfast | 15:12 | |
| *** bhavikdbavishi has quit IRC | 15:14 | |
| *** bhavikdbavishi has joined #zuul | 15:16 | |
| jkt | fungi: this host has been around for a few years already, I'm sure there was no recent enough python3 in main repos at the time of installation | 15:34 |
| y2kenny | is modifying zuul.resources/inventory inside a playbook not a good idea? | 15:34 |
| fungi | jkt: oh, got it, you're running an older centos 7 and not upgrading it? | 15:35 |
| clarkb | y2kenny: its probably better to use ansible's add_host module | 15:35 |
| clarkb | y2kenny: thats give you a programmatic interface so if details on the backend change the interface will accomodate that | 15:35 |
| fungi | y2kenny: it sounds like a layer violation to me, but what are you hoping to accomplish that way? stash things in it your playbooks will use, or alter zuul's behavior? | 15:35 |
| y2kenny | clarkb: would that update zuul.resources (or I can make add_host to register a change to zuul.resources?) | 15:36 |
| jkt | fungi: more like "I did not even know about that news" | 15:36 |
| jkt | see, I would have gone all the way to el8 if I was installing from scratch | 15:37 |
| fungi | jkt: heh, yeah i think it's centos 7.7 they started including python 3.6 directly so that folks have a better migration path to centos 8 | 15:37 |
| clarkb | y2kenny: it can update host vars for the new host as well as add hosts to groups. So depending on how you want to update that data structure would potentially have the equivalent result | 15:37 |
| jkt | I'm afraid my "migration path" includes full reinstallation and finding out which of my ansible playbooks need some desparete updating | 15:38 |
| jkt | I don't really do CI/CD of the infra :) | 15:38 |
| y2kenny | clarkb, fungi: I am thinking of creating pods using nodepool k8s driver with namespace type | 15:38 |
| clarkb | y2kenny: ya for that I think add_host is probably the correct option | 15:38 |
| clarkb | then the playbooks can operate on those resources after they are created and added | 15:39 |
| y2kenny | so that I can have more flexible definition. After the launch of the pods, I still want to use them similar to pods created by nodepool | 15:39 |
| y2kenny | clarkb: cool, I will give that a try | 15:39 |
| y2kenny | thanks! | 15:39 |
| *** bhavikdbavishi has quit IRC | 15:46 | |
| openstackgerrit | Albin Vass proposed zuul/zuul master: WIP: Enables whitelisting and configuring callbacks https://review.opendev.org/717260 | 16:03 |
| *** sshnaidm is now known as sshnaidm|afk | 16:04 | |
| *** hashar is now known as hasharAway | 16:05 | |
| *** hasharAway is now known as hashar | 16:23 | |
| *** rpittau is now known as rpittau|afk | 16:23 | |
| *** dpawlik has quit IRC | 16:29 | |
| tristanC | zuul-maint : similarly to the zuul.conf, i've added variable substitution for the registry configuration so that it is easier to deploy in kubernetes. Could you please have a look at https://review.opendev.org/710644 | 16:30 |
| *** jcapitao has quit IRC | 16:32 | |
| *** bhavikdbavishi has joined #zuul | 16:36 | |
| *** evrardjp has quit IRC | 16:37 | |
| *** evrardjp has joined #zuul | 16:37 | |
| *** bhavikdbavishi1 has joined #zuul | 16:39 | |
| tristanC | zuul-maint : and i've been integrating the zuul-registry and the zuul-preview service in the operator, the changes are verified by zuul, could you please review https://review.opendev.org/#/c/718419 and https://review.opendev.org/710650 . Thanks in advance! | 16:39 |
| *** bhavikdbavishi has quit IRC | 16:41 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 16:41 | |
| openstackgerrit | Merged zuul/zuul-base-jobs master: Remove ssh key in base-test cleanup run. https://review.opendev.org/708871 | 16:47 |
| openstackgerrit | James E. Blair proposed zuul/zuul-preview master: Handle SSL proxying and other fixes https://review.opendev.org/718517 | 16:49 |
| corvus | mordred, mnaser, tristanC: ^ running that image locally makes site.925bfe37815144d0859f260605d5fb98.zuul.zuul-preview.opendev.org work for me | 16:49 |
| openstackgerrit | James E. Blair proposed zuul/zuul-preview master: Handle SSL proxying and other fixes https://review.opendev.org/718517 | 16:50 |
| corvus | oops, silly comment edit | 16:50 |
| corvus | mordred: ^ | 16:50 |
| mordred | corvus: +2 | 16:51 |
| mordred | clarkb: ^^ | 16:51 |
| clarkb | on the dockerignore change I may be overthinking this but having the dockerfile change implies you want new things doesn't it? | 16:54 |
| clarkb | otherwise why change the dockerfile | 16:54 |
| corvus | clarkb: yeah, but docker already knows how to detect that things in the dockerfile are changed | 16:54 |
| corvus | clarkb: aiui, this should just affect the COPY statements | 16:54 |
| corvus | clarkb: which are important because we have "COPY ." in there | 16:55 |
| clarkb | right but you end up with a new layer either way? | 16:55 |
| clarkb | because you're building an image? | 16:55 |
| clarkb | I guess I don't see how it changes anything. | 16:55 |
| corvus | clarkb: when i added "a2enmod ssl" to the dockerfile, it recompiled the c++ code because of the "COPY ." that happens before the recompile. | 16:56 |
| corvus | clarkb: with this change, i can change the dockerfile and it won't do that anymore, it will just start rebuilding at the "RUN a2enmod" layer, which is later | 16:56 |
| clarkb | I see so this is specficially to prevent rebuilds of c++ if changing other aspects of the image | 16:57 |
| corvus | yep | 16:57 |
| corvus | (i'm actually wondering if i can add vhost.conf in there it doesn't affect the earlier layer, but it does affect a later layer, but there's an explicit copy for that... i'm experimenting) | 16:58 |
| corvus | nope, that doesn't work -- even the explicit "COPY vhost.conf" won't work if vhost.conf is in the dockerfile | 16:59 |
| corvus | so i think that's about all we can do right now :) | 16:59 |
| corvus | er s/dockerfile/dockerignore/ two lines up ^ | 17:00 |
| clarkb | tristanC: question for you on https://review.opendev.org/#/c/710644/5 | 17:02 |
| tristanC | clarkb: it's because yaml.load output is of type Any. To get a proper type, we would need to type the config schemas. | 17:06 |
| clarkb | and that is beacuse its not runtime checked ya? | 17:06 |
| clarkb | it just seems really weird to me that we haev a specific return type and even rely on that at line 316, but don't check it | 17:07 |
| clarkb | (seems to really devalue the purpose of type checking here) | 17:07 |
| mordred | clarkb: yeah - but from python's pov, yaml.load could return anything | 17:07 |
| mordred | we know that the contents of the yaml file are expeted to be a dict | 17:07 |
| clarkb | mordred: ya tristanC's comment has clarified that for me. I still think that devalues type checking that fuinction fwiw | 17:08 |
| mordred | but the python typing can't know that in this context | 17:08 |
| tristanC | clarkb: i didn't mind to type check the config, adding the function type definition enables typechecking it's internal code | 17:08 |
| tristanC | clarkb: for example changing any variables of the inner lambda or list comprehension will likely result in a useful type error | 17:10 |
| tristanC | mordred: actually yaml content can be a list or literal value too, not just a dict :) | 17:10 |
| clarkb | tristanC: yes, but in this case we require a dict because of line 316 | 17:11 |
| clarkb | tristanC: which is why this jumped out to me | 17:11 |
| clarkb | because literally the next line of code dependso n the type being specific | 17:11 |
| clarkb | (but we aren't checking that) | 17:11 |
| corvus | what happens if you change line 383 to say "-> typing.Dict" ? | 17:11 |
| openstackgerrit | Merged zuul/zuul-preview master: Handle SSL proxying and other fixes https://review.opendev.org/718517 | 17:11 |
| clarkb | tristanC: and yes checking thei nputs has value too. Its just as I mentioned this stood out to me because of what we do on the next line | 17:13 |
| tristanC | corvus: that would fails the yaml_load: `Returning Any from function declared to return "Dict[Any, Any]"` | 17:14 |
| clarkb | I've approved the change after tristanC and mordred's explanation | 17:14 |
| corvus | tristanC, mordred: so is there some way to 'coerce' the type? because clarkb is right, while it's true that yaml_load can return anything, the only valid return value from this function is a dict | 17:15 |
| tristanC | corvus: iiuc the mypy check implementation, if we add : if isinstance(yaml_load_output, dict): return ... then the output can be typed as Dict | 17:16 |
| tristanC | clarkb: corvus: i agree that's odd to return Any and then expect a Dict, but that is a limitation of dynamic language like python. If we want proper type, then we need extra work to ensure the types are correct | 17:17 |
| corvus | tristanC: if you feel like spending a minute on that, i'd be interested to see what it looks like. i'm not promising i'll love it or want to merge it here or elsewhere, but it might make clarkb and i feel better about type annotations :) | 17:17 |
| clarkb | ya it would be helpful if we as a user of the tpye system could say " we get it this type isn't actually set until runtime, but we expect type foo. Is the rest of the program consistent with that expectation" | 17:18 |
| clarkb | because that has value imo | 17:18 |
| corvus | tristanC: yes, we're in full agreement there, and i really like static typed languages. i think the difference is that i'm still not convinced all the extra work for type annotations in python is worth it (because we keep hitting cases where the design of the annotation system is fundamentally at odds with the design of the language). but i'm trying to keep an open mind. :) | 17:20 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Demonstrate how to cast Any into Dict for load_config https://review.opendev.org/718525 | 17:20 |
| fungi | as much as i might sometimes wish python were c, it's... not | 17:21 |
| tristanC | fungi: iirc c doesn't check subtype either, so it's not exactly preferable... | 17:21 |
| fungi | yeah, true on that point | 17:22 |
| tristanC | corvus: yeah i agree. The extra safety mypy provides is often not worthy to get the type check right | 17:22 |
| clarkb | well C does the assume the user provided type is the case thing | 17:22 |
| corvus | tristanC: so mypi will follow the AST and see that isinstance ensures it's a dict? | 17:22 |
| clarkb | which still has value | 17:22 |
| tristanC | corvus: yeah | 17:22 |
| corvus | tristanC: that's pretty neat :) | 17:22 |
| mordred | yeah - that's actually cool :) | 17:23 |
| clarkb | you can still get it wrong, but it does catch a number of isses around typing when you check assumed types | 17:23 |
| tristanC | corvus: which is kind of crazy, however that makes its implementation quite hard to understand... In some case it wasn't not able to process the AST as expected | 17:23 |
| fungi | it's tempting me to give it a spin on some of my simpler personal projects which handle outside data | 17:23 |
| corvus | tristanC: given that a robust implementation should really be doing that check anyway (we do in zuul, obviously), that seems like a good solution to clarkb's concern | 17:23 |
| mordred | ++ | 17:24 |
| clarkb | corvus: ya I've +2'd it | 17:24 |
| tristanC | fungi: this recent article shows how modern language does type check on external data: https://codetalk.io/posts/2020-04-05-common-json-patterns-in-haskell-rust-and-javascript.html | 17:24 |
| fungi | tristanC: thanks! | 17:25 |
| mordred | #[serde(skip_serializing_if = "Option::is_none")] | 17:26 |
| mordred | I like that from teh rust | 17:26 |
| tristanC | fungi: and btw, the dhall implementation of rust and haskell does provides the derive mechanism to express the config types in their native language | 17:26 |
| fungi | it's not all that far from ye olden days of validating structs in network protocol data | 17:26 |
| fungi | now that i think about it, i deal with packed structs in some of my software which implements a full telnet option negotiation stack | 17:27 |
| fungi | so not that olden | 17:27 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Use explicit provides/requires for container jobs https://review.opendev.org/717767 | 17:29 |
| mnaser | so -- http://site.925bfe37815144d0859f260605d5fb98.zuul.zuul-preview.opendev.org is up (based on the DNM change with the move to gatsby) | 17:39 |
| mnaser | can i get comments so that if we reach a good final state, i can start splitting to multiple digestable changes? | 17:40 |
| mnaser | (i was thinking of moving the add-blog part to a seperate change too) | 17:40 |
| mnaser | the change in question: https://review.opendev.org/#/c/717371/ | 17:41 |
| mnaser | cc zuul-maint ^ :) | 17:41 |
| mordred | mnaser: yeah - I think that's a good idea - because that way we can add an actual blog content in that change - and it'll both show how to add the feature and be digestible | 17:41 |
| mnaser | i think overall right now if i can get a review in "this looks ok" or "this looks off compared to the normal site" | 17:41 |
| mordred | zuul-maint: if you haven't played with gatsby yet, I recommend looking at https://review.opendev.org/#/c/717371/12/gatsby-node.js ... but don't block too much on not fully understanding it | 17:42 |
| fungi | mnaser: woah, a blog! | 17:42 |
| fungi | "Zuul is pretty good" | 17:42 |
| mordred | it's a core piece of how gatsby drives compile time things and provides them as data to the code | 17:42 |
| fungi | hard to disagree | 17:43 |
| mnaser | fungi: had to put _some_ filler code :) | 17:43 |
| fungi | it's certainly better than lorem ipsum | 17:43 |
| mordred | it took me a few iterations to wrap my head around how the graphql works in my gatsby website | 17:43 |
| clarkb | our of curiousity why are the pages like users and docs not markdown? | 17:52 |
| clarkb | also what does the routing? I see that the blog is explicitly set to /blog in gatsby-node.js but the things in pages/ don't appear to be explicitly routed. Is that implicit for the pages/ dir? | 18:02 |
| mnaser | imho, pages like that which have the potential to evenutally become things with logos or fancy formatting shouldnt be in markdown | 18:03 |
| clarkb | content wise the community page is missing the infrastructure donors and supporting companies info | 18:07 |
| clarkb | the documentation navbar doesn't drop down with shortcuts into specific docs like current website does | 18:07 |
| clarkb | oh wait the company stuff is there its just below a gray bar | 18:08 |
| clarkb | I would remove that gray bar, it makes it feel like the bottom of the page | 18:08 |
| * clarkb tries to find where to leave those comments in the source | 18:10 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: WIP: Support per branch change queues https://review.opendev.org/718531 | 18:13 |
| tobiash | corvus: this is an attempt with a queue config item and passes tests locally ^ | 18:13 |
| *** avass has quit IRC | 18:17 | |
| *** bhavikdbavishi has quit IRC | 18:17 | |
| *** avass has joined #zuul | 18:17 | |
| clarkb | ok I gave up trying to find the cause of the last thing I found and just left a best guess comment | 18:27 |
| clarkb | mnaser: my feedback is in the change now | 18:27 |
| *** hashar has quit IRC | 18:33 | |
| *** hashar has joined #zuul | 18:36 | |
| *** rlandy|pto is now known as rlandy | 18:37 | |
| mordred | clarkb: yeah - re js vs markdown - markdown really works best when it's a thing with a structure where there can be multiple instances of it each with some data - because you need to make the display template and then map the markdown content into it | 18:39 |
| clarkb | mordred: yup thats our users and documentation pages :) | 18:40 |
| clarkb | they are basically list of things with header and content | 18:40 |
| mordred | THAT SAID - once we get to the point where we've got netlify-cms plugged in to this, we might want to make a few more things markdown even if it's somewhat weird - so that people can edit TTW | 18:40 |
| mordred | clarkb: yah - we could totally develop those to be driven by markdown | 18:41 |
| mordred | might be a good follow on task when someone wants to poke at it | 18:41 |
| clarkb | mnaser: structurally you may want to git mv the assets rather than adding them new. That may reduce git repo size | 18:42 |
| clarkb | er mordred ^ sorry mnaser | 18:46 |
| clarkb | bah | 18:46 |
| clarkb | I'm all confused with my irc channels right now | 18:46 |
| clarkb | apologies | 18:46 |
| *** gtema has joined #zuul | 18:52 | |
| *** gtema has quit IRC | 18:59 | |
| *** gtema has joined #zuul | 19:06 | |
| *** gtema has quit IRC | 19:16 | |
| *** y2kenny has quit IRC | 19:31 | |
| *** igordc has quit IRC | 20:17 | |
| *** igordc has joined #zuul | 20:18 | |
| *** hashar has quit IRC | 20:19 | |
| *** rlandy has quit IRC | 20:46 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: tox: Don't inline python warnings https://review.opendev.org/718554 | 21:18 |
| corvus | mordred: ^ i'm mostly interested in landing the test data; i'm open to hearing what folks actually want out of the job :) | 21:18 |
| corvus | mnaser, tobiash: ^ fyi | 21:21 |
| tobiash | corvus: lgtm with a slight nit | 21:27 |
| tobiash | I think it makes sense to exclude the python warnings as they often are not really fixable | 21:31 |
| fungi | definitely when the problem is a dependency of your dependency complaining about how your dependency is invoking it | 21:44 |
| fungi | which happens with some regularity unfortunately | 21:44 |
| fungi | (especially if it's newer versions of python warning you than some transitive dependency of yours is using a deprecated function, though maybe that's a clue to get involved in the maintenance of your dependencies or switch to an alternative which keeps on top of changes in new python releases) | 21:45 |
| corvus | fungi: i think the file path check will catch that -- so we'll only see warnings from files in the current repo | 22:00 |
| fungi | ahh, yep probably | 22:04 |
| corvus | tobiash: replied | 22:04 |
| corvus | so i think the warnings are going to be addressible by the folks in the repo; but they may still not want to address them (or they may be some innocuous side effect of a unit test or something) | 22:06 |
| mnaser | hmm | 22:55 |
| mnaser | are we allowed to write files inside the zuul executor? | 22:55 |
| mnaser | i'm looking at upload-git-mirror and i suspect we _might_ be able to just run that inside the executor? | 22:55 |
| mnaser | we're just adding things to ~/.ssh/config (but that happens inside bwrap?), update known hosts (that's bound to happen with any ansible run) and then run a git push | 22:55 |
| mnaser | is there a good way of testing that theory? | 22:57 |
| *** saneax has joined #zuul | 23:01 | |
| *** threestrands has joined #zuul | 23:10 | |
| *** sanjayu_ has joined #zuul | 23:11 | |
| *** saneax has quit IRC | 23:14 | |
| fungi | mnaser: i though upload-git-mirror already ran on the executor in opendev | 23:15 |
| fungi | note that ssh can be controlled through other methods than writing to its config files | 23:15 |
| mnaser | fungi: zuul-jobs seem to indicate that there is no 'empty nodeset' | 23:16 |
| fungi | huh, looking | 23:16 |
| mnaser | fungi: https://opendev.org/zuul/zuul-jobs/src/branch/master/zuul.d/general-jobs.yaml#L43-L74 | 23:16 |
| fungi | huh, indeed, here's a build for a job parented to it, using an actual node: http://zuul.opendev.org/t/openstack/build/2d6a1db95eb642d98af242f4c598a482/log/zuul-info/inventory.yaml | 23:18 |
| mnaser | i guess we can test this by writing a job that uses a node, setups a repo on it, and then runs upload-git-mirror (on localhost to nodepool vm) | 23:18 |
| fungi | and yeah, thinking through it, we'd need some place to stick the private key | 23:18 |
| mnaser | not able to write anything on the executor i assume? | 23:19 |
| mnaser | wonder if we can add something to ssh agent? | 23:19 |
| fungi | oh, there is a role which can add keys to ssh-agent, yeah | 23:19 |
| mnaser | i assume that ssh-agent runs inside bwrap (aka you cant have another job grab creds?) | 23:20 |
| *** tosky has quit IRC | 23:20 | |
| fungi | i'm thinking of the ssh agent used to load the temporary per-node ssh keys that ansible uses to connect to them, so it's at least running ssh from within the bwrap yeah | 23:24 |
| fungi | someone on the west coast may still have more braincells left capable of making suggestions there, i'm fairly tapped out for the evening | 23:28 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!