| *** blarnath is now known as d34dh0r53 | 07:01 | |
| fungi | https://twitter.com/abbycabs/status/1618000064240291840 might be something some of our leaders have examples for | 12:41 |
|---|---|---|
| *** dasm|off is now known as dasm | 13:59 | |
| slaweq | Hi @gmann, I'm not feeling well today and I will not be attending the meeting today. Sorry. | 15:24 |
| gmann | slaweq: ack, please take rest. | 15:39 |
| gmann | tc-members: weekly meeting here in ~5 min from now | 15:54 |
| gmann | #startmeeting tc | 16:00 |
| opendevmeet | Meeting started Wed Jan 25 16:00:17 2023 UTC and is due to finish in 60 minutes. The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| opendevmeet | The meeting name has been set to 'tc' | 16:00 |
| gmann | #topic Roll call | 16:00 |
| gmann | o/ | 16:00 |
| dansmith | o/ | 16:00 |
| knikolla[m] | o/ | 16:00 |
| rosmaita | o/ | 16:01 |
| JayF | o/ | 16:02 |
| gmann | let's start | 16:03 |
| gmann | #link https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Agenda_Suggestions | 16:03 |
| gmann | today agenda ^^ | 16:03 |
| gmann | #topic Follow up on past action items | 16:03 |
| gmann | two action item from last meeting | 16:03 |
| gmann | gmann to send email to PTLs on openstack-discuss about checking PyPi maintainers list for their projects | 16:03 |
| gmann | done | 16:03 |
| noonedeadpunk | o/ | 16:03 |
| gmann | #link https://lists.openstack.org/pipermail/openstack-discuss/2023-January/031848.html | 16:03 |
| gmann | we will talk about it in next topic | 16:04 |
| gmann | gmann to reachout to release team to check about Zaqar gate fix by 25th Jan as deadline to consider it for this cycle release | 16:04 |
| gmann | this also done, we will talk in detail in next topics | 16:04 |
| gmann | #topic Gate health check | 16:05 |
| gmann | any news on gate | 16:05 |
| dansmith | unhealthy | 16:05 |
| dansmith | the skip of the test that was eating the large images merged which helped a lot | 16:05 |
| dansmith | I've got that fix up for the test and the unskip now | 16:05 |
| dansmith | but nova at least is still struggling to merge stuff | 16:05 |
| gmann | ok, I will take a look to the fix, thanks | 16:06 |
| dansmith | we've got a functional failure that manifests fairly often, which I don't think we've tracked down | 16:06 |
| dansmith | and we still have a lot of failures, usually around volume stuff hitting in the tempest jobs, which probably affect other people | 16:06 |
| gmann | yeah | 16:06 |
| JayF | I don't think any of the gate issues for the last couple of weeks have impacted Ironic; FWIW. We're still in about as good of shape as we've been all year. | 16:06 |
| spotz[m] | o/ | 16:06 |
| rosmaita | i'm seeing problems in ussuri and train grenade and tempest jobs, something about not being able to build bcrypt because Rust isn't available | 16:06 |
| dansmith | not sure if those are cinder or nova | 16:06 |
| fungi | i did notice cinder/glance/nova are all having a tough time getting the ossa-2023-002 backports landed | 16:06 |
| gmann | I saw someone reported in qa channel also about volume state issue | 16:07 |
| rosmaita | fungi: exactamundo | 16:07 |
| dansmith | yeah | 16:07 |
| fungi | rosmaita: missing rust build dependency is a sign that pyca/cryptography doesn't have a suitable wheel for that platform with the requested version | 16:07 |
| gmann | rosmaita: train, ussuri are not supposed to run grenade as mandatory as they are in EM state | 16:07 |
| rosmaita | gmann: how about tempest? | 16:08 |
| gmann | one recent tempest master change broke the stable/wallaby EM testing which tempest master does not support so I need to pin old compatible tempest there | 16:08 |
| rosmaita | i guess for EM, only unit and pep8? | 16:08 |
| gmann | rosmaita: those should pass. | 16:08 |
| gmann | tempest integration tests are required | 16:08 |
| gmann | please ping me failure link in qa and I will have a look | 16:08 |
| rosmaita | dang | 16:09 |
| rosmaita | ok, will do | 16:09 |
| gmann | but stable/wallaby is broken as reported by gibi in nova channel and I will push fix today | 16:09 |
| gmann | this cycle seems not good for gate health not just tox4 but many more unstable things | 16:10 |
| noonedeadpunk | tbh integrated queue doesn't look optimal for cases with vulnarabilities | 16:10 |
| dansmith | gmann: yeah | 16:10 |
| noonedeadpunk | As when you need to merge smth fast and withing this queue patches are waiting for merge and depending on everything else that's going on around | 16:10 |
| dansmith | noonedeadpunk: but CVEs can be manually applied as long as the patch is available, which is why the distributors get advance warning, and even people deploying from source can do that | 16:11 |
| noonedeadpunk | and in times of intermittent things or just loaded zuul workers it's becoming nightmare to land things | 16:11 |
| fungi | yes, this is exactly why our advisories don't say "upgrade to this release" but rather "here's a link to the patches, even if they're not merged yet" | 16:11 |
| JayF | I think it's hard to make the argument that it's OK that it can take a while to make our git-shipped version of software secure, even if the patches are available. | 16:12 |
| noonedeadpunk | Well, pip can't really pull from gerrit, can it? | 16:12 |
| noonedeadpunk | As it needs to fetch refs first that he knows nothing about | 16:12 |
| dansmith | to me, it's hard to say "we forced this patch in that didn't pass tests, but we're sure it's better" | 16:13 |
| dansmith | especially if that has the potential to 100% break everyone if it's wrong, instead of the partial failures that occur with the general gate instability | 16:13 |
| noonedeadpunk | I'm not saying not passing tests, what I'm saying is - in case of some random failure on another project we're to wait for another 5 hours | 16:14 |
| noonedeadpunk | and hoping that nothing else will happen on queue | 16:14 |
| fungi | yes, but to reiterate (for the i can't recall how many times now) we publish packages to pypi for ease of retrieval by testing systems and distributors. that some deployment projects are relying on packages on pypi for production use is a serious concern | 16:14 |
| dansmith | meh, the patch is available | 16:14 |
| dansmith | fungi: right | 16:15 |
| gmann | ok any other gate failure/concern to discuss? | 16:16 |
| noonedeadpunk | well, for me as operator doing `apt upgrade` and getting regression for neutron, which happened like 3 times last 4 releases, is quite serious concern not to use distro packages.. .anyway | 16:16 |
| noonedeadpunk | I think we can move forward:) | 16:18 |
| gmann | let's hope we get gate green there :) that is ultimate goal | 16:18 |
| gmann | moving to next topic | 16:18 |
| gmann | #topic Cleanup of PyPI maintainer list for OpenStack Projects | 16:19 |
| gmann | as discussed in previous meetings, sent email about audit to openstack-discuss #link https://lists.openstack.org/pipermail/openstack-discuss/2023-January/031848.html | 16:20 |
| gmann | also created a etherpad to track the audit results #link https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup | 16:20 |
| gmann | I can see 3-4 projects done audit and added the result in etherpad | 16:20 |
| gmann | there is some discussion on ML about PyPi maintainers cleanup, hope you have read that | 16:20 |
| gmann | any opinion on that? | 16:21 |
| JayF | I'm not surprised we get some pushback on it. I don't think it changes that we're going down the right path. | 16:21 |
| noonedeadpunk | Well, I think I can kind of relate on maintainers unwilling to fully give out credentials for projects they've just moved under openinfra umbrella | 16:22 |
| gmann | as we discussed earlier also, communication to additional maintainer is the key and convey the intension behind that | 16:22 |
| gmann | noonedeadpunk: yes that is the case. | 16:22 |
| noonedeadpunk | also if foundation tries to use our infra as hosting for projects | 16:22 |
| gmann | I will not be surprise to see that feedback more as audit goes | 16:22 |
| noonedeadpunk | as - "we can prodide you ci and infra stack but you will need to give us access and remove yourself" sounds indeed not great and more as a catch | 16:23 |
| knikolla[m] | it's also worth considering the fact that PyPI is not the only place we publish things to, and having additional maintainers can put things out of sync. | 16:24 |
| gmann | JayF: agree, we stay with the cleanup decision otherwise cases like xstatic* is more dangerous | 16:24 |
| noonedeadpunk | but from technical prespective I still don't really see why there should be humans involved | 16:24 |
| gmann | knikolla[m]: true | 16:24 |
| JayF | So I think it's fair to say nobody has a technical concern; but there is a perceptional concern. | 16:25 |
| knikolla[m] | from playing a bit around with it, the API is very limited. | 16:25 |
| noonedeadpunk | yeah and how to deal with that feeling of "takeover" I'm not sure to be frank | 16:25 |
| gmann | noonedeadpunk: but at same time, in open source its governance and OpenStack umbrella which has accountability and there is no such things "I am owner of this project/repo" | 16:25 |
| JayF | noonedeadpunk: yeah, the problem is the individual hooked up to pypi alongside openstackci feels a false sense of ownership :/ | 16:26 |
| JayF | noonedeadpunk: understandable, but in terms of governance it's clear that it's owned by the community, and openstackci user is the rep of that | 16:26 |
| knikolla[m] | gmann: ++, the way I see it, when you cede your project to OpenStack's governance the owner is effectively OpenStack, not you. | 16:26 |
| gmann | true | 16:26 |
| noonedeadpunk | gmann: I totally agree with you. | 16:27 |
| gmann | I think we all are in same page here. let's continue the discussion on ML and add your opinion there. we continue this audit then cleanup as planned | 16:27 |
| fungi | just to touch on the "foundation" point, the "foundaton" isn't mandating this, it's an openstack community decision about official openstack deliverables | 16:28 |
| gmann | also reachout to your known projects to plan the audit | 16:28 |
| gmann | fungi: yeah | 16:28 |
| knikolla[m] | If more scripts are needed, I'm happy to write them. That was fun. | 16:28 |
| gmann | knikolla[m]: +1, that listing projects are really helpful. | 16:28 |
| fungi | the foundation isn't telling projects they need to hand over credentials, far from it | 16:28 |
| gmann | yeah its openstack community and governance | 16:29 |
| fungi | it's not required for hosting a project on opendev either | 16:29 |
| noonedeadpunk | ok, then maybe we should also review what we're saying in docs regarding moving existing project under openstack umbrella? | 16:29 |
| clarkb | a foundation employee (me) pointed out one of your packages had been hijacked. But I didn't mandate you do anything at all. I did suggest that one path forward would be to officially hand over control of that packge to the hijackers and retire it in opendev though | 16:29 |
| clarkb | Also note I noticed this because I'm subscribed to openstack project events in github. Something anyone can do. No special access as foundation employee or opendev admin required | 16:30 |
| knikolla[m] | ++fungi and clarkb. This is an initiative from TC, not OpenInfra. | 16:30 |
| gmann | 'moving existing project under openstack umbrella?' ? which one. i did not get this fully | 16:30 |
| gmann | clarkb: yes | 16:31 |
| noonedeadpunk | I was thinking about this page https://governance.openstack.org/tc/reference/new-projects-requirements.html or smth related about adding new projects (or emerging projects) under umbrella | 16:32 |
| noonedeadpunk | I was talking overall about process and managing expectations of maintainers | 16:33 |
| fungi | "Releases of OpenStack deliverables are handled by the OpenStack Release Management team through the openstack/releases repository. Official projects are expected to relinquish direct tagging (and branch creation) rights in their Gerrit ACLs once their release jobs are functional." | 16:33 |
| fungi | that could be expanded on, sure | 16:33 |
| gmann | noonedeadpunk: sure, there is no harm to mention it there to be explicit and if new project creators does know about it | 16:33 |
| knikolla[m] | ++ | 16:34 |
| gmann | noonedeadpunk: do you want to push the doc change? | 16:35 |
| noonedeadpunk | to address things like `remove the project creator from their own project just for contributing it to OpenStack` | 16:35 |
| noonedeadpunk | gmann: yep, can do this | 16:35 |
| knikolla[m] | awesome | 16:35 |
| gmann | cool, thanks | 16:35 |
| gmann | let's move to next topic | 16:35 |
| gmann | #topic Less Active projects status: | 16:35 |
| gmann | Zaqar | 16:35 |
| gmann | Zaqar (Zaqar deliverable) Gate is green, bete version is released | 16:36 |
| gmann | #link https://review.opendev.org/c/openstack/releases/+/871399 | 16:36 |
| gmann | but | 16:36 |
| fungi | awesome, i'll abandon my change. thanks! | 16:36 |
| gmann | Zaqar-ui, python-zaqarclient tox4 issue fixes are up but not yet merged | 16:36 |
| gmann | #link https://review.opendev.org/q/topic:zaqar-gate-fix | 16:36 |
| gmann | these tox4 failure are not just this project but many project -ui/tempest plugins, client repo might not be fixed yet | 16:37 |
| gmann | even there is only PTL active (active on ping) there I feel we can continue with the Zaqar to be released in this cycle | 16:37 |
| gmann | not to mark as inactive | 16:37 |
| fungi | yep, i just abandoned | 16:38 |
| gmann | but keep monitoring the situation | 16:38 |
| fungi | excellent outcome | 16:38 |
| gmann | any objection on the plan? ^^ | 16:38 |
| gmann | no reply seems no objection :) | 16:40 |
| gmann | I will convey the same to release team | 16:40 |
| gmann | fungi: +1 on abandon the patch | 16:40 |
| gmann | Mistral | 16:40 |
| gmann | Gate is green, Beta version is released and all good now | 16:40 |
| *** gthiemon1e is now known as gthiemonge | 16:40 | |
| gmann | #link https://review.opendev.org/c/openstack/releases/+/869470 | 16:40 |
| gmann | #link https://review.opendev.org/c/openstack/releases/+/869448 | 16:40 |
| gmann | Governance patch to deprecate Mistral release is abandon | 16:41 |
| gmann | #link https://review.opendev.org/c/openstack/governance/+/866562 | 16:41 |
| gmann | with that, we left with no inactive projects. | 16:41 |
| noonedeadpunk | \o/ | 16:42 |
| gmann | but I agree that we stretched the monitoring/marking of less active projects beyond deadlines which is not good for release team planning | 16:42 |
| JayF | I wonder if there's something we can do to be proactive for Bobcat | 16:42 |
| gmann | that is something we need to improve 'detect and decide on such project before m-2' | 16:42 |
| JayF | we know these are skeleton crewed projects (and there are likely others); it might be benefitial for us to reach out to them much earlier | 16:42 |
| gmann | yeah, we should do that | 16:43 |
| JayF | if I can crack the nut on grading how active a project is, that could help us quantify which projects need help | 16:43 |
| gmann | I will add this topic for vPTG discussion | 16:43 |
| gmann | JayF: perfect and that can help to proceed/discuss further | 16:43 |
| gmann | anything else on this topic? | 16:44 |
| gmann | #topic Recurring tasks check | 16:45 |
| gmann | Bare 'recheck' state | 16:45 |
| gmann | #link https://etherpad.opendev.org/p/recheck-weekly-summary | 16:45 |
| gmann | slaweq is not present today but he added summary in above etherpad | 16:45 |
| gmann | data seems much better than last week | 16:45 |
| gmann | #topic Open Reviews | 16:46 |
| gmann | #link https://review.opendev.org/q/projects:openstack/governance+is:open | 16:46 |
| gmann | three open reviews, two of them are waiting on project-config change | 16:46 |
| gmann | this need more reviews, please check #link https://review.opendev.org/c/openstack/governance/+/871302 | 16:46 |
| gmann | and that is all from today agenda | 16:47 |
| gmann | we will have out next meeting on Feb 1st which will be a video call on zoom. | 16:47 |
| gmann | we have ~13 min left if anything else to discuss? otherwise we can close the meeitng. | 16:47 |
| JayF | I'll note I'm early on working on my item for this cycle | 16:48 |
| JayF | I strongly welcome any input here: https://etherpad.opendev.org/p/project-health-check | 16:48 |
| JayF | Trying to get ideas on what to measure and why before I start thinking about how and writing it up | 16:48 |
| JayF | so if you have opinions please toss them in the etherpad | 16:48 |
| gmann | thanks. will check it | 16:48 |
| fungi | i'm getting a strong feeling of déjà vu. check the earlier lists of criteria we tried to measure for project health if you haven't already, as well as reasons why we eventually deemed those untenable | 16:50 |
| JayF | fungi: ack; will look for them. | 16:50 |
| fungi | should be able to dredge them out of the governance git history | 16:51 |
| fungi | and look at tc meeting minutes from around those changes landing | 16:51 |
| gmann | I will say that was more of checking every project health and making decision and i noticed TC did without discussing it with projects team :) | 16:51 |
| spotz[m] | Yeah also maybe the chat logs? | 16:51 |
| gmann | but now situation changes where we are strugling with even having single active maintainers | 16:51 |
| fungi | also remember that there are 9 tc members and approximately 10x that many project teams | 16:52 |
| knikolla[m] | would be interesting to compare our standards for what an inactive projects looks like then vs now. | 16:52 |
| gmann | so instead of 'go and check every projects' we can go with 'observe these number of things and then take that project to monitor and discussion table' | 16:52 |
| dansmith | fungi: yeah... | 16:52 |
| knikolla[m] | "project only has 5 cores" vs "oh, we're lucky the the only core fixed the gate" | 16:53 |
| gmann | yeah | 16:53 |
| JayF | Yeah I don't think the idea is to program this into a terminator bot or anything | 16:53 |
| JayF | but just give us some kind of reliable way to indicate to people externally that a project needs help | 16:53 |
| gmann | yes | 16:54 |
| JayF | e.g. maybe OVHCloud steps up for Mistral earlier if we have it in yellow on a dashboard | 16:54 |
| * dansmith is skeptical it will ever be anything remotely like "reliable" | 16:54 | |
| gmann | yeah, that is good example | 16:54 |
| gmann | anyways let's put the ideas in etherpad | 16:54 |
| JayF | Including skepticisms too :D | 16:54 |
| gmann | ok, i think that is all for today. thanks everyone for joining | 16:56 |
| gmann | #endmeeting | 16:56 |
| opendevmeet | Meeting ended Wed Jan 25 16:56:08 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:56 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/tc/2023/tc.2023-01-25-16.00.html | 16:56 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/tc/2023/tc.2023-01-25-16.00.txt | 16:56 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/tc/2023/tc.2023-01-25-16.00.log.html | 16:56 |
| arne_wiebalck | thanks gmann ! | 16:56 |
| clarkb | I put some quick notes in #openstac-qa about the bcrypt issue. Hopefully that helps | 16:56 |
| knikolla[m] | thanks! | 16:56 |
| rosmaita | clarkb: ty | 16:57 |
| *** dasm is now known as dasm|off | 21:51 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!