Monday, 2025-01-06

*** jhorstmann is now known as Guest509601:44
*** elodilles_pto is now known as elodilles08:34
*** marios_ is now known as marios12:47
fungiwe've gone a week without any backup failure notifications, so i guess the prune helped speed up weekly validation runs13:23
*** sfinucan is now known as stephenfin14:06
clarkbnice15:49
clarkbI need to focus on some beginning of the year paperwork stuff this week. I'll also work on getting a meeting agenda out today. If I can work through that stuff then maybe I can boot a noble paste late this week?15:53
fungisounds good, i have similar tasks distracting me at the moment16:05
fungiclarkb: if we want to consider switching bindep over to pyproject.toml, it might make sense to couple that with the stack of fixes/updates we have for testing it on noble and whatnot16:13
clarkboh ya that seems like a good tool to switch over16:15
clarkbdoing my monday morning local package updates updated xfce. It set my background image to the default for xfce overriding what I Had before. I discovered a fun bug setting it back again. It seems that if I have the settings manager fullscreened over the desktop it won't change the image. But if I reduce its size so that part of thei mage is actually rendered it will16:18
clarkbxfce has taken a stance on "if a tree falls in a forest and no one is around to hear it?" philosophical questions16:18
fungiamusing16:19
clarkbfungi: ubuntu ports afs mirror is all happy now right? I can clear that off the meeting agenda?16:28
fungichecking16:29
fungiyeah, last updated 2 hours ago16:29
fungiit's all good16:29
fungilooks like mirror.epel hasn't updated for a couple of days, but the only other stale mirrors are empty volumes we never got around to deleting16:30
fungirsync: failed to connect to pubmirror3.math.uh.edu (129.7.128.191): Connection timed out (110)16:31
clarkbgreat I'll drop that from the meeting agenda. We can add back if epel mirroring is persistently sad16:32
funginoonedeadpunk: ^ you were the one to update the epel mirroring to that source a few years ago in https://review.opendev.org/862793 so maybe you have suggestions or contacts who can say whether we should move off it to something else?16:33
fungii wonder if we should simply revert back to pubmirror1.math.uh.edu16:34
fungii don't see any of the math.uh.edu mirrors listed at https://mirrormanager.fedoraproject.org/mirrors/EPEL currently16:38
clarkbosuosl might be a good choice as we know who to talk to there16:39
clarkbI also know who to takl to at the psu mirror16:39
fungialso dfw.mirror.rackspace.com claims to support rsync16:39
fungiand i can confirm it's listening on the port at least16:40
clarkbI think we started using rax's mirrors but they often fell behind on updates? but we know who to talk to there and it should be very local16:40
noonedeadpunkfungi: ugh, good question16:40
noonedeadpunkI can totally recall that and that some communication happened in IRC about it...16:40
noonedeadpunkand I think it was still tibbs who suggested it....16:41
noonedeadpunkso note was still relevant after an update16:42
fungiyeah, i'm wondering if they took all their mirrors offline16:43
fungieven https://math.uh.edu/ itself times out in my browser16:43
fungicould be they restructured all the university of houston network resources or something16:44
clarkbmaybe they decided to stop teaching math16:48
fungizing!16:49
fungiclarkb: any objection to me rebasing your 816741 for bindep pyproject.toml on top of my stack of test updates? right now it's not passing and i think it would once they merge16:51
clarkbfungi: that would be great thanks16:54
clarkbI've got a first draft of a meeting agenda on the wiki now. Introduces planning for service coordinator election timing and a beginning of the year "meetup" (spiritual successor to the pre ptg we had last year)16:57
clarkblet me know if anything important is missing from there and I can edit/add/remove as necessary. I won't send it out until end of day today to give time for feedback on that16:58
opendevreviewJeremy Stanley proposed opendev/bindep master: Update for newer Python, Tox and Hacking  https://review.opendev.org/c/opendev/bindep/+/93219017:12
opendevreviewJeremy Stanley proposed opendev/bindep master: Support and prefer unittest.mock  https://review.opendev.org/c/opendev/bindep/+/87873317:12
opendevreviewJeremy Stanley proposed opendev/bindep master: Fix Python version guessing logic for Debian Sid  https://review.opendev.org/c/opendev/bindep/+/93219117:12
opendevreviewJeremy Stanley proposed opendev/bindep master: Add testing for Debian testing and unstable  https://review.opendev.org/c/opendev/bindep/+/93219217:12
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674117:12
fungireworked the series while i was in there, and simplified the pyproject.toml change a bunch17:12
fungithough i suspect it's going to fail py36 testing, so this may be the point at which we ask whether we should drop support for that from bindep too17:13
fungiwhich i think also means dropping bionic testing17:15
clarkbwhy drop focal and jammy? is that just due to testing upper and lower bounds17:15
fungiyeah17:15
fungimaybe we can get by with just using an older min setuptools version for now17:18
corvusclarkb: fungi hi! happy new year!  i'm declaring scrollback bankruptcy from before the holidays...  has anyone set up a docker image mirror job yet?17:18
fungii don't *think* so17:18
corvusi'll get started on that then17:20
clarkbI am not aware of one17:21
clarkbthere was more discussion about options with openstack helm folks but I haven't heard if tehy did anything with that new role17:22
corvusdid we decide to have jobs in the opendev tenant and put images in the opendev quay.io org for everyone, or did we want images needed by the zuul project to be handled by the zuul tenant and zuul-ci quay.io org?17:27
clarkbI want to say we said it was ok for common images (like mariadb) to be hosted by opendev but that the less common things should probably be project specific so that we (opendev) don't act as arbiters for that content17:28
corvusack; i'll propose a change with all the images i think zuul-quick-start needs and we can prune those down and i'll move the excess to the zuul tenant17:29
fungihttps://www.openwall.com/lists/oss-security/2025/01/06/1 is a great example of the sorts of bugs that continue to make nested kvm dicey17:34
clarkbare general protection faults generally considered security concerns? Not sure what the impact is that would lead to it being on that list17:35
clarkb(I would expect things to crash in the hypervisor and the host to kill the related processes and move on but maybe cleanup is dangerous?)17:35
funginot sure, mostly just observing yet another situation where certain guest kernels can crash when doing nested kvm17:37
clarkb++17:37
fungii meant "dicey" from the perspective of random job node crashes which we have no way to debug without access to the host17:38
opendevreviewJames E. Blair proposed opendev/system-config master: Add jobs to mirror dockerhub images  https://review.opendev.org/c/opendev/system-config/+/93850817:44
corvusi put that in system-config since it already had a secret, but we could make a new secret and move it17:45
clarkbcorvus: I guess using a new secret might allow us to scope downt he access. but nothing is using that secret today so it should be fine for now17:47
clarkbthere was a zuul syntax error fwiw17:47
clarkbcorvus: also you should add the python images17:47
corvus++17:48
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674117:59
opendevreviewJames E. Blair proposed opendev/system-config master: Add jobs to mirror dockerhub images  https://review.opendev.org/c/opendev/system-config/+/93850818:06
clarkbI guess I can add ^ to the meetin agenda too18:06
fungilooks like pyproject.toml use with pbr under python 3.6 isn't going to work without more fiddling in pbr itself, i think? https://zuul.opendev.org/t/opendev/build/16d00668281b4857a8c933e0a063b307/log/job-output.txt#615-61718:34
clarkbfungi: thats the thing that requires you to not use develop18:35
clarkbfungi: I worked around that in pbr with usedevelop=false for that specific environment18:35
clarkbits the whole pep660 support in pip but not in setuptools for python3.6 versions of those two tools problem18:36
fungiright, maybe nox does editable by default then?18:36
clarkbfungi: its in the noxfile.py: session.install("-e", ".")18:37
clarkbI think we can either try to lookup the session python version and drop the -e or add a new session that matches 3.6 specifically and install . without the -e18:38
clarkboh no we can't redefined tests() so need to check the version and drop the -e18:40
fungii guess you do it like a normal python version check you'd do in other python code18:41
clarkbhttps://nox.thea.codes/en/stable/config.html#nox.sessions.Session.python18:41
clarkbyup exactly something like if session.python == '3.6': no -e else: use -e18:41
clarkbor just stop using -e entirely18:43
clarkbthe benefits for bindep are minimal since it installs so quickly you can just reinstall it18:43
clarkbin theory we could write a little wrapper function for session.install too that did those checks then we could share that around all the nox projects18:51
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674118:52
fungii'd rather just clean up the workarounds once everything requires python 3.718:53
clarkbfungi: I think that comparison doesn't work when you're running nox interactively because session.python will be "3" in that case (matching the decorator parameter value). Things are setup that way so that you can run it locally and it will just use your default python3 version. but then in CI (or if you are explicit locally) you can force the python version separately18:55
clarkbfungi: maybe it should be if session.python != "3" and session.python <= "3.6" ?18:55
clarkbwe can wait to see what CI says about this first though. But ya if you just run nox -s tests locally I think it will not use -e ?18:56
clarkbassuming your have a newer python18:56
clarkbcorvus: I think now there is an indentation issue?19:08
opendevreviewJames E. Blair proposed opendev/system-config master: Add jobs to mirror dockerhub images  https://review.opendev.org/c/opendev/system-config/+/93850819:11
corvusoops19:11
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674119:32
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674119:51
fungiincreasingly worried that newer setuptools is requiring a [project] table in pyproject.toml, but not sure why it didn't surface in the pbr changes if so19:54
fungithough before i go making assumptions, i'm trying to figure out why this didn't do an editable install: https://zuul.opendev.org/t/opendev/build/1841505ad6364d2883c2a1f76eddfefe/log/job-output.txt#62220:04
clarkbI guess whatever the python version is is <=3.6?20:05
fungioh, 3.12 < 3.6 yep!20:05
* fungi grumbles20:05
clarkbfungi: you should be able to print debug there. re [project] in pyproject.toml that is really odd considering the pbr changes never hit that20:05
clarkbmaybe setting py-modules=[] overrode that?20:06
fungibut pbr does that in its own packaging and didn't hit the same error20:06
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674120:07
clarkbright I'm wondering if telling setuptools to not determine the moduels skops the need for a [project] block20:09
fungii already have another edit ready for moving just about everything out of setup.cfg, but really hoping it's not necessary20:09
clarkbbecause maybe module auto detection requires a project name or something?20:09
fungiunfortunately as soon as there's a [project] table a lot of tools assume all metadata is either set there or declared as dynamic20:09
clarkbmaybe it is enough to make things dynamic?20:10
fungiit's a list of around 10 different keys20:11
fungii'll push that up wip just to see what we get20:11
clarkband I guess all of these changes are newer than when we first starting poking at this with bindep and pbr20:11
opendevreviewJeremy Stanley proposed opendev/bindep master: Evacuate most metadata out of setup.cfg  https://review.opendev.org/c/opendev/bindep/+/93852020:12
clarkbfungi: note the key used in pbr pyproject.toml is py-modules not packages20:15
clarkblooks like the first pyproject.toml change for bindep is still failing20:15
fungithe error indicates both are synonymous, but maybe they behave subtly differently20:16
clarkbI don't see py-modules in the error output. Maybe you're looking at the docs?20:17
opendevreviewJames E. Blair proposed opendev/system-config master: Add jobs to mirror dockerhub images  https://review.opendev.org/c/opendev/system-config/+/93850820:17
fungithe original error about being a flat package with multiple modules, saying you should override setuptools' module search function in one of several ways20:18
clarkbah20:19
clarkbthe py36 job did pass20:19
fungiyeah, it's been passing20:19
fungiwhich suggests we're dealing with a behavior change in newer setuptools20:19
clarkbso ya this must be in newer setuptools. But PBR did test py312 so thats odd20:19
clarkbI think we may also need a __main__.py for when bindep is used as a module (that apepars to by what failed in unittesting?)20:22
clarkbno maybe its something else. Says bindep --version failed in your pyproject takes over change20:23
clarkbwith no output so maybe that change is simply not correct for installing the console scipt?20:23
fungiwell, the latest revision of 816741 is at least doing editable mode for py312, so there's one less question to resolve20:24
fungithe old setup.cfg still specified the entrypoint as bindep = bindep.__main__:main20:25
fungii didn't change that, just moved it into pyproject.toml20:26
fungiand yes, bindep already has a __main__.py20:26
fungibut maybe it's not getting included in the package or something20:28
fungiif it's bindep --version failing, i suspect we're not installing pbr (listed in requirements.txt), so maybe the dynamic set for dependencies isn't working as expected20:30
opendevreviewJeremy Stanley proposed opendev/bindep master: Evacuate most metadata out of setup.cfg  https://review.opendev.org/c/opendev/bindep/+/93852020:35
fungicurious to see if that changes the picture20:35
fungiyeah, running build is failing to include any of the actual project python files20:47
fungilike pbr isn't actually supplying the manifest20:48
fungiso it's actually the entire project missing20:49
opendevreviewJeremy Stanley proposed opendev/bindep master: Evacuate most metadata out of setup.cfg  https://review.opendev.org/c/opendev/bindep/+/93852020:54
opendevreviewJeremy Stanley proposed opendev/bindep master: Move stray release note into the correct location  https://review.opendev.org/c/opendev/bindep/+/93852220:54
clarkbfungi: thats odd because it did work when we originally pushed the code and pbr itself is able to usepbr to package itself20:55
fungiyeah, agreed20:56
clarkbis it possibly related to not requiring setuptools >=64?20:56
clarkbI wonder if it works if you do that20:57
fungiseems unlikely, since it would use the newest available version, but worth a try20:57
clarkbdoes it log what version it is trying to use /me looks20:58
clarkbdoesn't look like it20:58
fungino, pip is very thorough about hiding all possible output from build backend stuff20:58
clarkbthe only other thought I've got is maybe require latest pbr21:00
clarkbsince pbr itself is using the latest version of pbr when packaging itself21:00
fungihuh, okay now that's passing for 3.12 but not 3.621:05
fungilooks like it's missing the entrypoint script21:06
opendevreviewJeremy Stanley proposed opendev/bindep master: Evacuate most metadata out of setup.cfg  https://review.opendev.org/c/opendev/bindep/+/93852021:08
clarkbanother event horizon between setuptools releases and pyproject.toml support?21:08
clarkbI really hope not because having one was plenty21:08
fungiyeah, i think so, that's another pep that landed around the 3.6-3.7 timeframe21:08
clarkbI guess the main thing to sort out now is why pbr is ok so that we're comfortable releasing pbr?21:11
fungiyeah, that's making me a little more weird, like maybe pbr testing still isn't entirely using the patched version21:11
fungiscarily, the latest 938520 is going all green21:12
clarkbpbr does refer to itself using backend-path21:12
clarkbrather than intsalling pbr via requires. Possibly something is happening via that to make it just work?21:13
fungiyeah, so 938520 is testing clean21:13
clarkbfungi: https://review.opendev.org/c/opendev/bindep/+/816741/11 this one passed except for py3621:14
clarkbthat implies to me that with a working noxfile to disable -e installs for py36 the rest of the content from ps11 should be fine?21:15
clarkbfungi: can we revert to the ps11 state but with the updated noxfile?21:15
fungiyeah, let's try splitting the rest of that into the later wip change, just a sec21:16
clarkbhrm build release also failed21:17
clarkberror: Multiple top-level packages discovered in a flat-layout: ['bindep', 'playbooks', 'releasenotes'].21:17
clarkbbut why didn't that fail when running py312 unittests?21:17
fungifwiw, teh only difference between ps 11-15 is tools.setuptools py-modules vs packages, i wonder if they're really not synonymous21:18
clarkbits certainly looking that way21:18
clarkbits almost like older platforms only respect the one key and newer platforms only respect the other21:20
clarkbI wonder if we can set them both and get away with both sets of platforms being happy21:20
clarkb(but also thats just a hack and my guessing at the problem)21:20
opendevreviewJeremy Stanley proposed opendev/bindep master: Move stray release note into the correct location  https://review.opendev.org/c/opendev/bindep/+/93852221:20
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674121:20
opendevreviewJeremy Stanley proposed opendev/bindep master: Evacuate most metadata out of setup.cfg  https://review.opendev.org/c/opendev/bindep/+/93852021:21
clarkbthe error says py_modules or packages21:21
clarkbmaybe py-modules is not correct (it is what pbr used though)21:21
clarkbperhaps py-modules is old/deprecated and that explains it working on the older platform though21:21
fungiscarily, https://stackoverflow.com/questions/76663521/where-does-py-modules-go-in-pyproject-toml-setuptools has both doing different things21:22
fungihttps://setuptools.pypa.io/en/latest/userguide/package_discovery.html#automatic-discovery21:23
clarkbI remember reading those docs a year or two ago and feeling just as confused when I was done /me checks if the feeling is any different this time around21:24
fungiyeah, i'm descending into the same madness again21:25
clarkbat a high level I feel like using the src/ layout would probably fix things for us21:26
clarkbbut we'd have to fix all our imports within the project too likely21:26
fungiwe'd have to change the contents of all our git repos to move everything into a subdirectory, yes21:27
clarkbhttps://setuptools.pypa.io/en/latest/userguide/package_discovery.html#custom-discovery seems like what we want if we want to keep supporting the existing structure but the docs there don't really help me undersatnd how to configure it21:28
clarkb"disable implicit namespaces" in their example doesn't map to the human readable text in the paragraph above21:29
clarkbor maybe it does but in a non obvious to me manner21:29
clarkbthat said you updated your change to set the pcakage to bindep right? maybe thats the minimal change we need?21:30
clarkbfungi: in your latest patchset you renamed tools.setuptools to tool.setuptools. fwiw pbr uses tools.setuptools but the docs you just linked to use tool.setuptools. Maybe that is the problem?21:31
fungioh, huh...21:31
clarkbrather than py-modules vs packages vs py_modules could it be the header name lookup instead21:31
clarkbif so in theory we can set both values maybe?21:31
clarkband have a two sections that are redundant with one another21:32
fungifwiw https://packaging.python.org/en/latest/guides/writing-pyproject-toml/ says it's "tool" not "tools"21:32
clarkbbut also your latest ps fails py312 and not py3621:33
clarkbps11 was the other way around. This is super confusing21:33
fungihttps://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#setuptools-specific-configuration also says "tool.setuptools"21:34
clarkbEmit a warning when ``[tools.setuptools]`` is present in ``pyproject.toml`` and will be ignored. -- by :user:`SnoopJ` (#4150)21:35
clarkbtools.setuptools appears to be the old thing and tool.setuptools is the new thing. I suspect that py312 is working without that section entirely21:35
fungihuh21:35
clarkbbut when you "fix" it to use the correct name tool.setuptools it breaks21:35
clarkbbut then release-build still fails21:35
clarkbok theory: when you drop the tool*section entirely it makes things "work" possibly because we don't need to pip install -e . when we're running all the code relative to . anyway21:37
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674121:37
opendevreviewJeremy Stanley proposed opendev/bindep master: Evacuate most metadata out of setup.cfg  https://review.opendev.org/c/opendev/bindep/+/93852021:37
fungiseeing what happens when it's removed21:37
clarkbbut then release build breaks because the resulting package build is of nebulous utility as a package21:37
clarkbbut ya lets reset to not having that section at all and see what ahppens21:37
fungiincreasingly happy i keep bubbling the unrelated fixes ahead of these two21:38
clarkbI'm going to pop out for a bike ride now to take advantage of some nice weather. Interested to see what you learn. I'll also get the meeting agenda out sometime after I return and before the end of my day today. Feel free to leave notes here if you want additional edits.21:38
clarkbI think I'll add a note about corvus' change to mirror container images21:39
fungisounds good. catch you later!21:39
clarkbfungi: do we also need to WIP the pbr release until we understand tools.setuptools weirdness better?21:39
fungiit's a pre-release, so i'm not worried about it21:39
fungiand it's still waiting on an openstack release team discussion about independent model deliverables being allowed pre-releases21:40
clarkback21:40
fungiif we decide pbr needs another commit, i'll either update the pre-release request or make a new one depending on what lands first21:41
fungiand if we decide it doesn't need any new commits, then no adjustments are needed to the pre-release request anyway21:41
fungiyeaj, looks like it's build-python-release that's going to be unhappy about things21:52
Clark[m]That probably implies the install we do in unit tests isn't really installing anything and we're relying on the local code path instead from th git checkout21:54
fungieditable installs, yes21:55
fungiwe used to force at least one test to be non-editable in order to catch that, if memory serves21:55
fungimaybe build-python-release is serving that purpose now21:56
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674121:58
opendevreviewJeremy Stanley proposed opendev/bindep master: Evacuate most metadata out of setup.cfg  https://review.opendev.org/c/opendev/bindep/+/93852021:58
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674122:03
opendevreviewJeremy Stanley proposed opendev/bindep master: Evacuate most metadata out of setup.cfg  https://review.opendev.org/c/opendev/bindep/+/93852022:03
opendevreviewJeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support  https://review.opendev.org/c/opendev/bindep/+/81674122:11
opendevreviewJeremy Stanley proposed opendev/bindep master: Evacuate most metadata out of setup.cfg  https://review.opendev.org/c/opendev/bindep/+/93852022:11
fungii'm not sure i love that solution, but it's working22:23
fungii guess i should stack another change dropping py36 support to see how it works out22:23
Clark[m]fungi: do we run build python release in PBR? If nnot adding that as a sanity check might be a good idea23:39
Clark[m]I'll look at the fix above soon23:39

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!