| opendevreview | James E. Blair proposed zuul/zuul master: Add ExecutorApi https://review.opendev.org/c/zuul/zuul/+/770902 | 03:42 |
|---|---|---|
| opendevreview | James E. Blair proposed zuul/zuul master: Change zone handling in ExecutorApi https://review.opendev.org/c/zuul/zuul/+/787833 | 03:42 |
| opendevreview | James E. Blair proposed zuul/zuul master: Switch to string constants in BuildRequest https://review.opendev.org/c/zuul/zuul/+/791849 | 03:42 |
| opendevreview | James E. Blair proposed zuul/zuul master: Clean up Executor API build request locking and add tests https://review.opendev.org/c/zuul/zuul/+/788624 | 03:42 |
| opendevreview | James E. Blair proposed zuul/zuul master: Fix race with watches in ExecutorAPI https://review.opendev.org/c/zuul/zuul/+/792300 | 03:42 |
| opendevreview | James E. Blair proposed zuul/zuul master: Execute builds via ZooKeeper https://review.opendev.org/c/zuul/zuul/+/788988 | 03:42 |
| opendevreview | James E. Blair proposed zuul/zuul master: Move build request cleanup from executor to scheduler https://review.opendev.org/c/zuul/zuul/+/794687 | 03:42 |
| opendevreview | James E. Blair proposed zuul/zuul master: Re-register components after ZK reconnect https://review.opendev.org/c/zuul/zuul/+/796582 | 03:43 |
| opendevreview | James E. Blair proposed zuul/zuul master: Handle errors in the executor main loop https://review.opendev.org/c/zuul/zuul/+/796583 | 03:43 |
| *** marios is now known as marios|ruck | 05:04 | |
| opendevreview | Merged zuul/zuul master: Correct URL in matrix howto https://review.opendev.org/c/zuul/zuul/+/796328 | 06:36 |
| *** rpittau|afk is now known as rpittau | 07:15 | |
| *** jpena|off is now known as jpena | 07:33 | |
| opendevreview | Matthieu Huin proposed zuul/zuul master: [DNM] Tests: add non-voting unit testing for Python 3.10 https://review.opendev.org/c/zuul/zuul/+/796270 | 08:45 |
| *** sshnaidm|afk is now known as sshnaidm | 08:50 | |
| opendevreview | Matthieu Huin proposed zuul/zuul master: [DNM] Tests: add non-voting unit testing for Python 3.10 https://review.opendev.org/c/zuul/zuul/+/796270 | 10:02 |
| *** bhagyashris_ is now known as bhagyashris | 10:27 | |
| *** jpena is now known as jpena|lunch | 11:32 | |
| *** hashar is now known as Guest2387 | 12:24 | |
| *** hashar is now known as Guest2388 | 12:26 | |
| *** jpena|lunch is now known as jpena | 12:30 | |
| mhuin | Hey, does anybody know which version of OpenSSL in installed on the nodes used by tox jobs for the zuul project? | 12:46 |
| mordred | mhuin: looks like 1.1.1-1ubuntu2.1~18.04.9 | 13:19 |
| mhuin | thanks mordred | 13:20 |
| mordred | mhuin: fwiw - I went to the raw json log of the most recent failed test run for you there ^^ https://7494b51f1867d83e2df3-c514cadbc1265d8f1c8d131346aa619b.ssl.cf5.rackcdn.com/796270/11/check/zuul-tox-py310/1a75c14/job-output.json | 13:20 |
| mordred | mhuin: and did a search for "libssl-dev" | 13:20 |
| mhuin | I'm asking because I'm seeing SSL failures with the gear library when testing with python 3.10 | 13:21 |
| mordred | it's running on bionic - you might want to make that job run on focal instead fwiw - since you're aiming at 3.10 | 13:21 |
| mordred | hrm weird | 13:21 |
| mhuin | yep ... but there were some changes in 3.10 regarding the ssl module, I'm looking into it atm | 13:22 |
| mhuin | I saw that OpenSSL 1.1.1 and above were required, so I wanted to eliminate that possible problem | 13:22 |
| mordred | yeah - you should have 1.1.1 in that job | 13:23 |
| mhuin | so that's not it | 13:24 |
| avass[m] | mhuin: I think there was some issue with gear, ssl and newer python versions | 13:27 |
| mhuin | avass[m], seems so yeah | 13:28 |
| avass[m] | trying to find the conversation | 13:28 |
| mhuin | I also see that the last tag on gear is 14 commits / 1 year ago | 13:28 |
| avass[m] | mhuin: looks like clarb was involved, maybe he remembers something about that? | 13:32 |
| mhuin | avass[m], mordred I'm gonna try running the test again on gear's master branch. I've run gear's unit tests on py310 from master and from 0.15.1 - seems like some problems were fixed since the last tag | 13:33 |
| mhuin | (tox py310 fails on 0.15.1, but passes on master) | 13:33 |
| mordred | mhuin: so perhaps gear needs a release to pick up 3.10 enablement? | 13:34 |
| mhuin | mordred, possibly, let me run the test again to confirm first | 13:35 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: [DNM] Tests: add non-voting unit testing for Python 3.10 https://review.opendev.org/c/zuul/zuul/+/796270 | 13:38 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: [DNM] Tests: add non-voting unit testing for Python 3.10 https://review.opendev.org/c/zuul/zuul/+/796270 | 13:41 |
| guillaumec | mhuin: it could be related to this series https://review.opendev.org/c/opendev/gear/+/784083 , https://review.opendev.org/c/zuul/zuul/+/780261 | 13:42 |
| guillaumec | but as your job is using bionic and not focal, it may not | 13:43 |
| mhuin | I need to check if py3.10 is bundled in focal | 14:04 |
| fungi | mhuin: mordred: yeah, i wanted to polish gear up and get a new version tagged, but stalled getting python 3.9 testing working for it | 14:12 |
| fungi | i need to check again and see if we ever managed to solve that and get the tests added | 14:13 |
| mordred | mhuin: it does not look liek 3.10 is in focal - just 3.9 | 14:15 |
| fungi | yeah, okay my change to add python 3.9 testing did finally merge last month | 14:15 |
| fungi | so we could probably release it in the current state unless there's anything else out for review that needs to merge on it first | 14:15 |
| mhuin | fungi, could we wait until we're sure it's working fine with python 3.10? | 14:16 |
| mhuin | the current build seems ok so far: https://zuul.opendev.org/t/zuul/stream/8247b6e3389b4f8797d759dcbe30d1d8?logfile=console.log | 14:16 |
| fungi | mhuin: when does 3.10 release? | 14:17 |
| fungi | though if you're looking for a platform with available packages, we could add the experimental suite on a debian-bullseye node and apt install python3.10 there | 14:17 |
| fungi | https://packages.debian.org/python3.10 | 14:17 |
| mhuin | fungi: this october IIUC, see https://fedoraproject.org/wiki/Changes/Python3.10 | 14:18 |
| mhuin | it's going to be the default python version in fedora 35 | 14:18 |
| fungi | mhuin: so you want to wait to tag a new gear release until october? | 14:18 |
| mhuin | fungi, oh no, I just want to wait until I get a verified+1 on https://review.opendev.org/c/zuul/zuul/+/796270 :) | 14:19 |
| fungi | oh, got it | 14:19 |
| mhuin | we don't even have to merge it, but it sure would help distro packagers | 14:20 |
| fungi | yeah, i expect it'll be fine, getting 3.9 working was a big hurdle but that's atypical | 14:20 |
| fungi | big mess around changes to ssl/tls | 14:21 |
| fungi | which we finally got sorted out | 14:21 |
| mhuin | it's mostly simple stuff so far, like moving some classes to collections.abc | 14:21 |
| mhuin | but it's everywhere | 14:21 |
| mhuin | I've opened so many pull requests in the last 48 hours | 14:21 |
| fungi | i don't doubt it, i just wish tox made it easier to manage complex PYTHON_WARNINGS filter lists | 14:22 |
| fungi | though even with that, the necessary granularity makes it a game of whack-a-mole to keep the list updated with whatever deprecation warnings crop up in your dependencies | 14:23 |
| fungi | and worse, in the packaging toolchain, pip/setuptools vendor all manner of crap chock full of deprecations | 14:24 |
| fungi | and it seems to change with every new version of those packages | 14:24 |
| fungi | so trying to spot deprecated things in your own code is sufficiently nontrivial that almost no projects bother to fix them until (sometimes long) after an interpreter release comes along that breaks hard on removal of things which were deprecated for years | 14:26 |
| mhuin | aaaand we have a build success \o/ | 14:34 |
| mhuin | fungi: tag away! | 14:34 |
| fungi | mhuin: thanks, i'll start summarizing the recent changes for the release announcement in that case | 14:49 |
| mhuin | fungi, if that implies a patch on gear, please let me know, I'll add it as a Depends-On on the py310 tests for zuul | 14:50 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Tests: add non-voting unit testing for Python 3.10 https://review.opendev.org/c/zuul/zuul/+/796270 | 14:51 |
| fungi | mhuin: likely not, i just need to look through commit messages and (if we have them) release notes, and then put together a summary for the release announcement e-mail | 14:52 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Tests: add non-voting unit testing for Python 3.10 https://review.opendev.org/c/zuul/zuul/+/796270 | 14:55 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Tests: add non-voting unit testing for Python 3.10 https://review.opendev.org/c/zuul/zuul/+/796270 | 14:59 |
| opendevreview | Matthieu Huin proposed zuul/nodepool master: [DNM] Add unit testing with python 3.10 https://review.opendev.org/c/zuul/nodepool/+/796691 | 15:06 |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Tests: add non-voting unit testing for Python 3.10 https://review.opendev.org/c/zuul/zuul/+/796270 | 15:09 |
| fungi | digging into it now, we haven't used reno with gear, so i'll just base the announcement on a summary of commit messages | 15:14 |
| mhuin | there have been 14 commits since last release IIRC | 15:14 |
| fungi | mhuin: since it's an opendev project, i'll move further discussion for the release to #opendev so as not to clutter up #zuul, if you'd like to /join (or feel free to follow the channel logs on https://meetings.opendev.org/irclogs/ ) | 15:16 |
| mhuin | got it, I'll check for the announcement on the chan, thanks | 15:16 |
| fungi | you're welcome | 15:17 |
| opendevreview | Matthieu Huin proposed zuul/nodepool master: [DNM] Add unit testing with python 3.10 https://review.opendev.org/c/zuul/nodepool/+/796691 | 15:18 |
| *** marios|ruck is now known as marios|out | 15:45 | |
| opendevreview | Matthieu Huin proposed zuul/zuul master: Tests: add non-voting unit testing for Python 3.10 https://review.opendev.org/c/zuul/zuul/+/796270 | 15:47 |
| opendevreview | Matthieu Huin proposed zuul/nodepool master: [DNM] Add unit testing with python 3.10 https://review.opendev.org/c/zuul/nodepool/+/796691 | 15:48 |
| fungi | mhuin: you were right, i wanted to push a final change before tagging, see https://review.opendev.org/796704 (it doesn't touch the application source at all though, just docs and packaging) | 16:08 |
| *** rpittau is now known as rpittau|afk | 16:11 | |
| corvus | mhuin, fungi: we're like this close to completely removing gear from zuul -- would it be possible to just continue using the current version until that's done? | 16:42 |
| corvus | if gear needs a release for other reasons, then i'd like to just pin the version of gear in zuul. we should be able to have it completely removed very soon, especially if we're not spending time trying to get it to work with new ssl versions. if we're still using gearman in october, something has gone very wrong. | 16:47 |
| fungi | corvus: sure, i more or less expected this to be the last release opendev would support, and then we're retire or offer it to others if they wanted to continue to maintain it for something unrelated to zuul | 16:50 |
| corvus | fungi: makes sense; i just don't want a moving target for the next few weeks (frozen for security release, then literally the nexit thing i want to merge is the removal of gearman for build requests). so if you want to go ahead and do that now, that's fine, let's merge a pin in zuul. but i think all this started because mhuin wants to update gear in zuul, so i don't want us all working at cross-purposes :) | 16:53 |
| *** jpena is now known as jpena|off | 16:59 | |
| corvus | mhuin: ^ thoughts? | 17:11 |
| fungi | i'd certainly prefer not to complicate things for zuul, so yes at a minimum agree we ought to defer releasing another version of gear until zuul 4.6.0 is behind us, or even hold off releasing it until zuul 5.0.0 and we can basically call gear "complete" from opendev's perspective | 17:14 |
| corvus | i'm fine with a pin too; we could merge that right now; i just don't want to do that if that blocks mhuin, and the only reason we're releasing gear is to unblock mhuin :) | 17:15 |
| fungi | i think hashar had requested a new gear tag back closer to the beginning of the year, though now i don't recall why exactly | 17:16 |
| mordred | yeah - I remember hashar requesting that ... also don't remember why | 17:17 |
| fungi | that was what led me initially down the path of trying to get it to work with python 3.9 since i was hesitant to tag it without support for the latest interpreter release at the time | 17:17 |
| avass[m] | also wasn't the idea to stick to depends-on etc for development until 4.6 is released? | 17:18 |
| fungi | yeah, that's why i'm suggesting even if we do pin it we wait until after 4.6.0, just so as not to further complicate anything leading up to the security roll-up | 17:19 |
| corvus | that wfm too | 17:20 |
| fungi | i'd really rather not create more complexity between now and then, now that i've been reminded | 17:20 |
| corvus | tobiash: most of those optimizations lgtm; i think one of them maybe we could skip just because it's focused on gearman and by the time we're ready to merge that optimization, we'll be ready to merge the change that removes it. the other's i admin -2d, but will flip to +2 after 4.6 | 17:28 |
| *** ricolin_ is now known as ricolin | 17:49 | |
| mhuin | corvus: the sf team owns the fedora package for python-gear, so technically we can simply add patches supporting py310 to the spec. It'd be easy since these patches are already merged upstream so no need to rework them as a new release approaches | 20:46 |
| mhuin | Ultimately I just wanted to check what was the bare minimum to get zuul 4.x to run on rawhide, so we know what packages may need patching as well | 20:49 |
| mhuin | we have until october anyway ... | 20:49 |
| mhuin | There's some testing for rpms that rely on zuul's rpm being in a working state on rawhide, but nothing that can't be circumvented without a --force option | 20:52 |
| mhuin | with* | 20:53 |
| corvus | mhuin: okay. does this work for you? 1) pin gear<0.16.0 in zuul now [include that pin in the 4.6.0 release of zuul]. 2) release gear 0.16.0. and if we haven't removed gear completely in a month or so, look at un-pinning gear in zuul? | 20:55 |
| corvus | fungi: ^ i think if we are going to pin gear in zuul and release a new gear, that doing the pin before 4.6.0 is a good idea, that way people who install 4.6.0 from packages after gear 0.16.0 is released aren't surprised | 20:55 |
| corvus | er, sorry i should clarify i meant step #2 above to happen after 4.6.0 is released | 20:56 |
| mordred | oops. forgot to rejoin ... | 21:09 |
| *** ChanServ sets mode: +o corvus | 21:09 | |
| *** corvus was kicked by corvus (Kicked by @jim:acmegating.com : Testing) | 21:10 | |
| fungi | corvus: if i'm reading correctly, that would work for rawhide to be able to package gear, but not then the gear in rawhide wouldn't be supported by zuul... and ultimately we expect to have zuul not depend on gear by the point where it becomes necessary? | 21:29 |
| corvus | fungi: yes, and if i'm wrong, gear 0.16 is standing by and we can remove the pin in zuul to use it | 21:29 |
| corvus | i don't know if that's useful to mhuin or not; but it achieves my goal of not having to think about gear while i'm trying to get rid of it | 21:31 |
| corvus | (i honestly don't understand all the packaging nuance here, so i'm relying on mhuin for feedback) | 21:32 |
| fungi | my concern is "rpms that rely on zuul's rpm being in a working state on rawhide" means that either they need to patch zuul or gear (or both) no matter what we do, if we're going to declare that zuul doesn't support the gear release which works on python>=3.9 (or really >3.5 according to its trove classifiers, but is really broken on 3.9 and up) | 21:32 |
| corvus | that sounds plausible. is that bad? | 21:33 |
| fungi | by pinning zuul to gear <0.16 we're effectively saying "don't use zuul on python 3.9 right now" which i don't think is necessarily bad, but does require people who feel they need to make it work on 3.9 and later take on some responsibility for that one way or another | 21:34 |
| fungi | from our end, we hope to have zuul not depend on gear "real soon now" so it's neither here nor there | 21:35 |
| fungi | zuul will work with python 3.9 when it drops its dependency on gear | 21:35 |
| corvus | right, and i think that's weeks away | 21:36 |
| fungi | yeah, i think that's a good upstream decision. we need mhuin to say which of the available options are more convenient for him, and maybe find some middle ground if there's a problem to solve | 21:37 |
| fungi | from my perspective, i'm fine not releasing a new gear until after zuul 5.0.0 | 21:37 |
| fungi | if that's what's sanest for us | 21:38 |
| corvus | earlier doesn't bother me as long as zuul pins :) | 21:38 |
| fungi | yeah, the question is does releasing a new version of gear that zuul claims it doesn't support solve mhuin's dilemma | 21:39 |
| corvus | agreed | 21:39 |
| fungi | does he want new gear so that current zuul will be compatible with newer python? if so, then the pin is probably no better than not tagging gear | 21:40 |
| fungi | and not tagging gear plus not pinning it in zuul is less work all around | 21:40 |
| opendevreview | James E. Blair proposed zuul/zuul master: Re-register components after ZK reconnect https://review.opendev.org/c/zuul/zuul/+/796582 | 22:54 |
| opendevreview | James E. Blair proposed zuul/zuul master: Add ExecutorApi https://review.opendev.org/c/zuul/zuul/+/770902 | 22:54 |
| opendevreview | James E. Blair proposed zuul/zuul master: Change zone handling in ExecutorApi https://review.opendev.org/c/zuul/zuul/+/787833 | 22:54 |
| opendevreview | James E. Blair proposed zuul/zuul master: Switch to string constants in BuildRequest https://review.opendev.org/c/zuul/zuul/+/791849 | 22:54 |
| opendevreview | James E. Blair proposed zuul/zuul master: Clean up Executor API build request locking and add tests https://review.opendev.org/c/zuul/zuul/+/788624 | 22:54 |
| opendevreview | James E. Blair proposed zuul/zuul master: Fix race with watches in ExecutorAPI https://review.opendev.org/c/zuul/zuul/+/792300 | 22:54 |
| opendevreview | James E. Blair proposed zuul/zuul master: Execute builds via ZooKeeper https://review.opendev.org/c/zuul/zuul/+/788988 | 22:54 |
| opendevreview | James E. Blair proposed zuul/zuul master: Move build request cleanup from executor to scheduler https://review.opendev.org/c/zuul/zuul/+/794687 | 22:54 |
| opendevreview | James E. Blair proposed zuul/zuul master: Handle errors in the executor main loop https://review.opendev.org/c/zuul/zuul/+/796583 | 22:54 |
| opendevreview | James E. Blair proposed zuul/zuul master: Add debug info for merger component test https://review.opendev.org/c/zuul/zuul/+/796739 | 22:54 |
| mhuin | corvus: the end goal is to have zuul packaged (and working) when fedora 35 releases. I think we can live with python-gear's rpm packaging the current version (0.15.1 IIRC), and we'll add the needed patches that are already in gear's master branch to support python 3.10 | 23:38 |
| mhuin | and if by october we can remove the dependency to python-gear in zuul's rpm spec file, all the better | 23:39 |
| fungi | mhuin: ideally then fedora 35 will release with zuul 5.x.x which won't need gear at all | 23:39 |
| fungi | or what you just said, yep | 23:39 |
| fungi | cool so seems like we can just avoid capping gear in zuul if we also avoid tagging a new release of gear before zuul drops it | 23:40 |
| mhuin | fungi, I think the minimum would be adding tests on python3.10 - but the patch can be trivially rebased as needed I think, so it probably doesn't need to be merged if you don't want to add an extra test to save time and resources | 23:41 |
| mhuin | fungi, yeah if you don't need the tag, don't do it. I think we can manage on our side. I can make sure of it tomorrow (it's 1:45AM and I really shouldn't be talking about packaging right now) | 23:43 |
| fungi | um, yeah, get some sleep! | 23:44 |
| fungi | happy to pick up discussion tomorrow | 23:44 |
| fungi | as for py310 testing of zuul, we can probably merge the addition in a few weeks when the gear dep is dropped from the source tree, even before the 5.0.0 release is finalized | 23:46 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!