| @michael_kelly_anet:matrix.org | Hmm. Yea, looks like right now if your node image happens to have git-lfs installed even this will break entirely because it tries to download the objects and can't | 02:53 |
|---|---|---|
| -@gerrit:opendev.org- Michael Kelly proposed: [zuul/zuul] 869784: manager: Fix TODO in executeJobs() https://review.opendev.org/c/zuul/zuul/+/869784 | 06:03 | |
| -@gerrit:opendev.org- Michael Kelly proposed: [zuul/zuul] 869784: manager: Fix TODO in executeJobs() https://review.opendev.org/c/zuul/zuul/+/869784 | 06:04 | |
| -@gerrit:opendev.org- Michael Kelly proposed: | 06:18 | |
| - [zuul/zuul] 869785: manager: Remove unncessary job_graph check from executeJobs() https://review.opendev.org/c/zuul/zuul/+/869785 | ||
| - [zuul/zuul] 869786: model: Eliminate semaphore_handler parameter on findJobsTo*() https://review.opendev.org/c/zuul/zuul/+/869786 | ||
| -@gerrit:opendev.org- Michael Kelly proposed: [zuul/zuul] 869786: model: Eliminate semaphore_handler parameter on findJobsTo*() https://review.opendev.org/c/zuul/zuul/+/869786 | 06:28 | |
| -@gerrit:opendev.org- Michael Kelly proposed: [zuul/zuul-jobs] 869787: prepare-workspace-git: Skip LFS checkout when mirroring repos https://review.opendev.org/c/zuul/zuul-jobs/+/869787 | 07:02 | |
| -@gerrit:opendev.org- Michael Kelly proposed: [zuul/zuul-jobs] 869787: prepare-workspace-git: Skip LFS checkout when mirroring repos https://review.opendev.org/c/zuul/zuul-jobs/+/869787 | 08:15 | |
| -@gerrit:opendev.org- Michael Kelly proposed: [zuul/zuul-jobs] 869787: prepare-workspace-git: Skip LFS checkout when mirroring repos https://review.opendev.org/c/zuul/zuul-jobs/+/869787 | 08:20 | |
| -@gerrit:opendev.org- Michael Kelly proposed: [zuul/zuul-jobs] 869787: prepare-workspace-git: Skip LFS checkout when mirroring repos https://review.opendev.org/c/zuul/zuul-jobs/+/869787 | 08:24 | |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/nodepool] 869771: Add release notes for 8.0.1 https://review.opendev.org/c/zuul/nodepool/+/869771 | 09:32 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/nodepool] 866549: Correct documentation for image upload metric https://review.opendev.org/c/zuul/nodepool/+/866549 | 09:43 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [zuul/zuul] 867137: Update openshift client install version https://review.opendev.org/c/zuul/zuul/+/867137 | 10:22 | |
| @mbecker12:matrix.org | Hi! I've been looking into the openshiftpods driver recently and stumbled across the max-pods quota and how it's evaluated. | 10:27 |
| I can see that it boils down to the listNodes() function in provider.py. What I don't get is the [FakeServer class](https://opendev.org/zuul/nodepool/src/branch/master/nodepool/driver/openshiftpods/provider.py#L50). What's the purpose of that? To me it looks like it eventually just resolves to a counter of instances which should [coincide with the number of items in self.k8s_client.list_namespaced_pod()](https://opendev.org/zuul/nodepool/src/branch/master/nodepool/driver/openshiftpods/provider.py#L73-L75). | ||
| @mbecker12:matrix.org | Also, in the openshiftpods driver, the valid_names derived from [pod_names] (https://opendev.org/zuul/nodepool/src/branch/master/nodepool/driver/openshiftpods/provider.py#L38) are set to be the keys of the label definition. I don't really understand why this is done this way. Shouldn't it be something like label.name instead? | 10:28 |
| @westphahl:matrix.org | corvus: Clark is it intentional that with the switch to nox we now get the full log output from the test runs and not just the failed ones? job output is now in the ~250mb range | 14:00 |
| @jim:acmegating.com | swest: nope! :) | 14:32 |
| -@gerrit:opendev.org- Fabien Boucher proposed: [zuul/zuul] 869829: Fix scheduleFilesChanges fallback to target branch ref https://review.opendev.org/c/zuul/zuul/+/869829 | 14:46 | |
| @westphahl:matrix.org | corvus: k, good :) passing `--suppress-attachments` to stestr doesn't seem to make any difference. I don't quite understand how this was working before | 14:48 |
| -@gerrit:opendev.org- Fabien Boucher proposed: [zuul/zuul] 869559: [gitlab driver] Remove usage of MR diff_refs attribute https://review.opendev.org/c/zuul/zuul/+/869559 | 14:49 | |
| @clarkb:matrix.org | > <@westphahl:matrix.org> corvus: k, good :) passing `--suppress-attachments` to stestr doesn't seem to make any difference. I don't quite understand how this was working before | 15:00 |
| It has to do with the OS_*_CAPTURE env vars iirc. Maybe those aren't being set correctly by the noxfile helper function | ||
| @jim:acmegating.com | yes, i suspect we missed something with the variables | 15:03 |
| @westphahl:matrix.org | corvus: just curious: how do those `OS_*_CAPTURE` vars interact with stestr? | 15:06 |
| @jim:acmegating.com | swest: we do some stuff in the test framework setup in `__init__.py` with them | 15:08 |
| @jim:acmegating.com | er base.py | 15:09 |
| @jim:acmegating.com | https://opendev.org/zuul/zuul/src/branch/master/tests/base.py#L4308 is the general area | 15:09 |
| @westphahl:matrix.org | yeah I think I'm missing the link how that interacts with stestr | 15:10 |
| @clarkb:matrix.org | We manipulate the data stream by capturing the info and then decide whether or not to add it to the subunit stream | 15:10 |
| @clarkb:matrix.org | If stdout and logs are captured and then not forwarded along they never leave the test process and stestr doesn't see them | 15:10 |
| @clarkb:matrix.org | If they are written to stdout or stderr or explicitly in the subunit stream then stestr sees the data | 15:11 |
| @jim:acmegating.com | and we only attach them if there's an error https://opendev.org/zuul/zuul/src/branch/master/tests/base.py#L4322 | 15:11 |
| @westphahl:matrix.org | k, now I get it. still seems weird that passing `--suppress-attachments` to stestr doesn't hide the output on that level | 15:12 |
| @jim:acmegating.com | * and we only attach logs if there's an error https://opendev.org/zuul/zuul/src/branch/master/tests/base.py#L4322 | 15:12 |
| @jim:acmegating.com | the problem may be that nothing is being captured and attached and it's all just going to stdout/err directly | 15:13 |
| @clarkb:matrix.org | making a new session that sets env vars and echos out the capture var values using `session.run("bash", "-c", "env")` shows the values being set | 15:58 |
| @clarkb:matrix.org | and doing some simple testing locally the vars seem to be set | 16:05 |
| @clarkb:matrix.org | makes me think the issue isn't with the var setup but with something internal to the test suite | 16:05 |
| @clarkb:matrix.org | If you look in the log it is actually capturing stderr and reporting the captured stderr back properly. What seems to be making it through is just the log output | 16:11 |
| @clarkb:matrix.org | Ok some logs were leaking through in the old zuul jobs too. But much fewer were doing this. Whatever has changed has caused a much larger range of logs to leak through | 16:19 |
| @clarkb:matrix.org | Logging is properly captured in a random test I modified (one that doesn't need zookeeper and friends for simplicity). I think that means our simple test cases are ok and something must be side effecting us | 16:26 |
| -@gerrit:opendev.org- Fabien Boucher proposed: [zuul/zuul] 869559: [gitlab driver] Remove usage of MR diff_refs attribute https://review.opendev.org/c/zuul/zuul/+/869559 | 16:30 | |
| @clarkb:matrix.org | Running just `tests.unit.test_circular_dependencies.TestGerritCircularDependencies.test_crd_cycle` reproduces the issue | 17:01 |
| @clarkb:matrix.org | at the time we try to add our special log handler a stderr handler already exists in the logger | 17:13 |
| @clarkb:matrix.org | possibly because we create class level loggers which get evaluated at code load time before we execute the setUp() method | 17:14 |
| @jim:acmegating.com | zuul-maint: how is this for a nodepool release? commit a4c4f0afcca01cde9a4be2a3fbf813fcca2bedd5 (HEAD -> master, tag: 8.0.1, origin/master) | 17:21 |
| @jim:acmegating.com | that will give us an easy revert target before we land the openstack state machine change | 17:21 |
| @clarkb:matrix.org | The issue is the warning for lacking the rsa skip check thing. We run that super early and calling logging.warning() produces a stderr handler on the root logger | 17:43 |
| @michael_kelly_anet:matrix.org | Anyone up for giving me some reviews on https://review.opendev.org/c/zuul/zuul-jobs/+/869787 ? | 17:47 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 869862: Cleanup test logging https://review.opendev.org/c/zuul/zuul/+/869862 | 17:54 | |
| @clarkb:matrix.org | corvus: swest ^ | 17:54 |
| @clarkb:matrix.org | corvus: I think they formalized the unsafe thing and I'm guessing the old flag went away: https://github.com/pyca/cryptography/pull/7667/files | 17:57 |
| @jim:acmegating.com | Clark: so we need to set the new flag, yeah? | 17:59 |
| @clarkb:matrix.org | that is my hunch | 17:59 |
| @westphahl:matrix.org | Clark: +2, it's interesting though that wenn you run the tests with tox the logs will be correctly captured. So I doubt this has to do with a changed ssl version | 18:04 |
| @jim:acmegating.com | swest: any chance your tox env has an old cryptography installed in its venv? | 18:04 |
| @westphahl:matrix.org | yeah I was about to write that my tox env might be outdated | 18:05 |
| @jim:acmegating.com | Clark: swest i'll do the followup | 18:06 |
| @westphahl:matrix.org | ftr. my tox env was outdated. same issue with tox now | 18:07 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 869865: Use unsafe_skip_rsa_key_validation with cryptography https://review.opendev.org/c/zuul/zuul/+/869865 | 18:12 | |
| @fungicide:matrix.org | yay, so entirely unrelated to nox | 18:13 |
| @jim:acmegating.com | looks like 39.0.0 was released on jan 1, so timing is coincidence | 18:13 |
| @clarkb:matrix.org | Is that flag valid with openssl 1.x? The old comment mentioned openssl 3.0.0. many distros are still not on 3.0.0 | 18:14 |
| @clarkb:matrix.org | Hopefully cryptography handles it for both | 18:14 |
| @fungicide:matrix.org | unit testing ought to tell us if not | 18:15 |
| @clarkb:matrix.org | Oh ya the 3.8 job runs in focal and is 1.x | 18:15 |
| @jim:acmegating.com | Clark: i believe in both cases we're telling cryptography not to perform validation (which it does with openssl), so regardless of the openssl version, we're just not invoking the routine either way | 18:15 |
| @jim:acmegating.com | (basically, it would be okay to do it on the old version, too slow on the new version, and since we don't care, we just don't do it regardless) | 18:16 |
| @clarkb:matrix.org | Got it | 18:16 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 869862: Cleanup test logging https://review.opendev.org/c/zuul/zuul/+/869862 | 18:37 | |
| -@gerrit:opendev.org- Clark Boylan proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 869865: Use unsafe_skip_rsa_key_validation with cryptography https://review.opendev.org/c/zuul/zuul/+/869865 | 18:37 | |
| @clarkb:matrix.org | I messed up a linter thing... should be good now in that stack | 18:37 |
| @clarkb:matrix.org | Looking at the complaints about leaked sockets one of the sockets is of type dgram. Ive had the fakestatsd (which gets shutdown cleanly) print out its port number and it doesn't match the one that the warning complains about | 18:53 |
| @clarkb:matrix.org | Those warnings are probably worthy of investigation too | 18:53 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 869870: Unvendor kazoo locks recipes https://review.opendev.org/c/zuul/zuul/+/869870 | 19:17 | |
| @clarkb:matrix.org | I wasn't able to figure out those leaks. But did determine we could fix one set of warnings this way ^ | 19:17 |
| @jpew:matrix.org | Is there a way to make zuul only load configuration from and process changes from at a single branch in a repository? | 22:51 |
| @jpew:matrix.org | We're trying to easy a repo into use Zuul on a branch, but it's commenting on changes for other branches, which we don't want | 22:52 |
| @jpew:matrix.org | (specifically, complaining when there are merge conflicts) | 22:52 |
| @clarkb:matrix.org | jpew: https://zuul-ci.org/docs/zuul/latest/tenants.html#attr-tenant.untrusted-projects.%3Cproject%3E.include-branches | 22:56 |
| @jpew:matrix.org | Clark: We did that, but it still commented on changes on other branches :/ | 22:57 |
| @clarkb:matrix.org | and you reloaded the config after editing that? It isn't automatic like the in repo configs | 22:57 |
| @clarkb:matrix.org | Its possible the merge checking is happening super early though since zuul generally needs to merge things to get any config | 22:58 |
| @jpew:matrix.org | Ya, because the project wasn't even part of zuul before | 22:58 |
| @clarkb:matrix.org | (that is probably a bug if that is what is happening) | 22:58 |
| @jpew:matrix.org | Ya, it was definetly doing that | 23:00 |
| @clarkb:matrix.org | the easiest way to confirm that is probably to grep the event id out of your scheduler and merger logs and then trace that to see if merging is happening before include branch list is consulted | 23:04 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!