| @jim:acmegating.com | dancywmf: some general thoughts that may be helpful: | 00:59 |
|---|---|---|
| 1) you could raise your ulimit :) | ||
| 2) you might find it easiest to just run one test at a time; that may be sufficient for this work. | ||
| 3) feel free to use the upstream ci system to deal with running all the other tests. upload as many patchsets as you need; that's what it's there for. | ||
| in short: you might be able to save time/effort by just running one or two tests locally then once your local change looks good, uploading the result for confirmation everything else is still good | ||
| @mhuin:matrix.org | > <@dancywmf:matrix.org> corvus: Today I attempted to run the Zuul test suite just to make sure I can get a clean run before making modifications to it. I set up a fresh Debian 12 VM for it. It took a while but I eventually got the test suite running (by running 'nox' and/or stestr directly). After running for quite a long time it hung. The most recent notable error prior to the hang was "OSError: [Errno 24] Too many open files: '/tmp/tmpahhywq1o/zuul-test/builds/124ce4c890394846bec6376c605a3b6a/trusted/project_0/review.example.com/common-config/.git/logs/refs/remotes'". I ran `lsof` on the system and found a python proces with all 1024 file descriptors used. 679/1024 of the open fds were pipes (FIFO). 302/1024 were UDP sockets. Looking around in Gerrit I see that this is appears to be semi-known problem? | 06:54 |
| if it's any help, I've put up a few notes a while ago about setting up a test env. I think they're still valid, except I'd choose a base OS where python 3.11 or above is the default python https://www.softwarefactory-project.io/hacking-zuul-for-developers-running-unit-tests.html | ||
| @dancywmf:matrix.org | So we don't consider this situation to be a gug? | 15:35 |
| @dancywmf:matrix.org | *bug? | 15:35 |
| @dancywmf:matrix.org | Feels like a bug to me | 15:35 |
| @dancywmf:matrix.org | mhu: Thanks! That looks like more or less what I did. | 15:37 |
| @dancywmf:matrix.org | (except for altering the ulimit, of course) | 15:37 |
| @dancywmf:matrix.org | Anyway, I'm not stuck. Just wanted to inquire about it. | 15:43 |
| @fungicide:matrix.org | if there's a descriptor leak then that might be a bug worth fixing, but it's more likely just a combination of how the tests are designed and their concurrency | 16:36 |
| @clarkb:matrix.org | The udp sockets are for statsd testing iirc and the git lib "leaks" pipes (I think it actually leaks processes and that keeps pipes open and then init will clean them up eventually?) | 16:43 |
| @clarkb:matrix.org | they are both essentially short term "leaks" that show up during testing because we create many resources | 16:43 |
| -@gerrit:opendev.org- Ahmon Dancy proposed: [zuul/zuul] 899425: gitlabconnection.py: Handle 404 on unapprove https://review.opendev.org/c/zuul/zuul/+/899425 | 21:20 | |
| @dancywmf:matrix.org | Hey Dan! | 22:53 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 23:54 | |
| - [zuul/zuul] 900828: Fix flaky deduplication test https://review.opendev.org/c/zuul/zuul/+/900828 | ||
| - [zuul/zuul] 900829: Restore lingering deduplicated builds https://review.opendev.org/c/zuul/zuul/+/900829 | ||
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!