| opendevreview | Ghanshyam proposed openstack/project-team-guide master: Clarify Extended Maintenance branch testing and support policy https://review.opendev.org/c/openstack/project-team-guide/+/861141 | 06:40 |
|---|---|---|
| *** pojadhav is now known as pojadhav|afk | 07:49 | |
| frickler | fyi I added the current list of config-error affected projects to https://etherpad.opendev.org/p/zuul-config-error-openstack in order to show the scope of the issue | 08:11 |
| *** pojadhav|afk is now known as pojadhav | 08:16 | |
| noonedeadpunk | gmann: sure, will do this today | 09:02 |
| *** dasm|off is now known as dasm | 13:00 | |
| *** knikolla[m] is now known as knikolla | 13:26 | |
| *** blarnath is now known as d34dh0r53 | 13:32 | |
| gmann | frickler: thanks | 14:42 |
| gmann | tc-members: weekly meeting here in an ~9 min from now | 14:51 |
| jungleboyj | \o/ | 14:54 |
| gmann | noonedeadpunk: thanks | 14:55 |
| gmann | #startmeeting tc | 15:00 |
| opendevmeet | Meeting started Thu Oct 13 15:00:14 2022 UTC and is due to finish in 60 minutes. The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'tc' | 15:00 |
| JayF | o/ | 15:00 |
| gmann | #topic Roll call | 15:00 |
| gmann | o/ | 15:00 |
| noonedeadpunk | o/ | 15:00 |
| knikolla | o/ | 15:00 |
| slaweq | o/ | 15:00 |
| dansmith | o/ | 15:00 |
| gmann | in Absence section today | 15:01 |
| gmann | arne_wiebalck will miss the meeting on Oct 13 | 15:01 |
| gmann | rosmaita will miss Oct 13 | 15:01 |
| gmann | let's start | 15:02 |
| gmann | #link https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting | 15:02 |
| gmann | today agenda ^^ | 15:02 |
| spotz | o/ | 15:02 |
| gmann | #topic Follow up on past action items | 15:02 |
| gmann | gmann to update the wording the EM branch status and reality of maintenance policy/expectation | 15:02 |
| gmann | I proposed patch yesterday #link https://review.opendev.org/c/openstack/project-team-guide/+/861141 | 15:03 |
| gmann | please review ^^ | 15:03 |
| gmann | #topic Gate health check | 15:03 |
| gmann | any news on gate? | 15:03 |
| dansmith | so, I have seen a bunch of POST_FAILUREs lately | 15:03 |
| gmann | ohk | 15:03 |
| dansmith | I don't have specific pointers, I might be able to find some, but they also seem undebuggable since there are no logs | 15:04 |
| dansmith | I dunno if anyone else has been seeing that, or what might be causing it | 15:04 |
| slaweq | nope, I didn't, at least not in neutron related projects | 15:04 |
| dansmith | here's an example:https://zuul.opendev.org/t/openstack/build/3c40559f664543359c0109b28dc07656 | 15:05 |
| dansmith | anyway, I'll start collecting links for next week if I keep seeing them | 15:05 |
| gmann | seems nothing showing in console also | 15:06 |
| dansmith | there was one run where there were like six jobs that all POST_FAILUREd, so it seemed systemic | 15:06 |
| noonedeadpunk | it's actually interesting, as logs are present | 15:06 |
| noonedeadpunk | and there're not much of stuff to be executed afterwards | 15:07 |
| dansmith | yeah, but I didn't see any reasons for the failure.. some of the other ones were just no logs, which are hard to debug | 15:07 |
| slaweq | maybe it's timeing out on sending logs to swift? | 15:07 |
| slaweq | I think we had such kind of issue few weeks ago in Neutron functional tests job | 15:07 |
| noonedeadpunk | Well when there're not logs it's related to swift highly likely. | 15:07 |
| dansmith | in the no logs case, perhaps | 15:07 |
| dansmith | anyway, maybe just something to keep an eye out for | 15:08 |
| noonedeadpunk | We had issues when upload logs to swift took more then 30 mins | 15:08 |
| noonedeadpunk | And it was some specific provider just being slow | 15:08 |
| gmann | yeah | 15:08 |
| fungi | failures during log upload can be challenging to expose, since it's a chicken-and-egg problem (zuul relies on being able to serve the logs to provide results as it doesn't store that data elsewhere) | 15:08 |
| dansmith | yup | 15:08 |
| noonedeadpunk | But eventually even with logs present it can be same - as timeout for post jobs is 30 mins | 15:08 |
| fungi | but if someone has a recent list of ones that are suspect, i can try to find causes (likely tracebacks from zuul) in the executor service logs | 15:09 |
| gmann | one I can see 'process-stackviz: pip' role taking 10 min | 15:09 |
| gmann | but not sure if that is causing timeout | 15:09 |
| fungi | there was a recent complaint about aodhclient hitting timeouts from pip's dep solver taking too long because they don't use constraints. maybe the problem is more widespread (i don't know if the stackviz installation is constrained) | 15:10 |
| noonedeadpunk | I actually don't think it's timeout for mentioned example - it ended at 00:16:19 and post jobs started 3 mins earlier | 15:10 |
| gmann | usually it takes 20-30 sec | 15:11 |
| gmann | fungi: that is not constraint i think, it is latest published one we use? | 15:11 |
| fungi | gmann: it has dependencies though, right? | 15:12 |
| gmann | yeah | 15:12 |
| fungi | those are what i'm talking about possibly taking time to dep solve if they aren't constrained | 15:12 |
| gmann | checked some other result and it was ok | 15:13 |
| fungi | anyway, expect that there may be multiple causes for the observed failures, but we can try to classify them and divide/conquer | 15:13 |
| gmann | yeah, let's check and debug later if it is occurring more | 15:13 |
| gmann | any other failure observed in gate? | 15:13 |
| gmann | Bare 'recheck' state | 15:14 |
| gmann | #link https://etherpad.opendev.org/p/recheck-weekly-summary | 15:14 |
| gmann | slaweq: please go ahead | 15:14 |
| slaweq | things are good | 15:14 |
| slaweq | what is IMO worth to mention is fact that we have less and less teams with 100% of bare rechecks in last 7 days | 15:15 |
| gmann | nice | 15:15 |
| slaweq | so it is improving slowly :) | 15:15 |
| gmann | great | 15:15 |
| gmann | Zuul config error | 15:16 |
| gmann | #link https://etherpad.opendev.org/p/zuul-config-error-openstack | 15:16 |
| JayF | Do we have any documentation on not doing bare rechecks, that can be sent to contributors who do bare rechecks? | 15:16 |
| gmann | frickler added the effected projects for zuul config error in etherpad | 15:16 |
| JayF | I continue to work, with other Ironic contributors, to get these config errors wrapped up in Ironic. | 15:16 |
| gmann | JayF: yes, this one #link https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures | 15:17 |
| gmann | knikolla: please go ahead if any updates | 15:17 |
| knikolla | i'm updating the etherpad as i go. pushed patches for zaqar and senlin today | 15:17 |
| fungi | be aware the configuration may be on any branch, that list in the pad isn't differentiating between branches but the details in the zuul webui will tell you the project+branch+file | 15:17 |
| knikolla | starting with projects who have errors in the master branch | 15:18 |
| gmann | +1, nice | 15:18 |
| knikolla | adjutant's gate had been broken for a year, so pushed a fix for that as well. | 15:18 |
| fungi | broken for... a year?!? | 15:18 |
| knikolla | yes. the requirements.txt specificed django <2.3 | 15:18 |
| fungi | doesn't that mean the project is just de facto retired then? | 15:18 |
| gmann | knikolla: thanks | 15:18 |
| knikolla | upper contraind specified == 3.2 | 15:19 |
| gmann | there is only one maintainer (PTL) there, you can add him in review to get it merge | 15:19 |
| noonedeadpunk | django version? | 15:19 |
| noonedeadpunk | ah yes, sorry, missed | 15:19 |
| knikolla | will do. i'm using this as an exercise in getting to know the ptl of the smaller teams | 15:20 |
| gmann | perfect | 15:20 |
| fungi | not so gentle reminder, projects with development blocked by broken job configuration which is unaddressed for a very long time should be a strong signal that it can just be retired | 15:20 |
| gmann | let's continue that and everyone can pick up few projects to fix in their available time | 15:20 |
| gmann | we have new volutneer to took over this project, let's see how it goes now. | 15:21 |
| knikolla | fungi: totally agree | 15:21 |
| fungi | keeping track of those situations would be a good idea, even if someone steps in to fix the testing for them | 15:21 |
| gmann | we had long time that there was no maintainer so broken gate is very much possible | 15:21 |
| gmann | sure | 15:21 |
| gmann | anything else on zuul config error or gate health ? | 15:21 |
| gmann | thanks knikolla for helping here | 15:22 |
| noonedeadpunk | yeah, but adjutant was working nicely if u-c for django - we were testing it in osa | 15:22 |
| gmann | ack | 15:22 |
| fungi | it just had no development for a year straight, and no maintainers to address any bugs in it | 15:22 |
| noonedeadpunk | we will go back to zun then now | 15:23 |
| gmann | true but as we have new maintainer let's wait for this cycle how it goes | 15:23 |
| noonedeadpunk | +1 | 15:23 |
| gmann | #topic 2023.1 cycle PTG Planning | 15:23 |
| gmann | #link https://etherpad.opendev.org/p/tc-leaders-interaction-2023-1 | 15:24 |
| gmann | #link https://etherpad.opendev.org/p/tc-2023-1-ptg | 15:24 |
| gmann | please add the topics in the etherpad, by friday I might try to schedule the present topics | 15:24 |
| gmann | one news: I talked to kubernetes steering committee members to attend the TC PTG session like we have in past couple of PTG | 15:25 |
| gmann | and two members Tim and Christoph agree to attend the session on Friday 21 Oct 16:00 - 17:00 UTC | 15:26 |
| knikolla | awesome! | 15:26 |
| gmann | feel free to add the topic you would like to discuss with them - #link https://etherpad.opendev.org/p/tc-2023-1-ptg#L72 | 15:26 |
| JayF | I might add a topic w/r/t fungi's point from earlier | 15:26 |
| JayF | about more proactively identifying projects that are unmaintained or insufficiently maintained | 15:27 |
| slaweq | ++ | 15:27 |
| gmann | sure, we can discuss that | 15:27 |
| spotz | I suspect some of the others might be already headed to Detroit that Friday | 15:27 |
| gmann | As discussed in last PTG, one thing we did for that is emerging and in-active project things | 15:28 |
| gmann | JayF: this one #link https://governance.openstack.org/tc/reference/emerging-technology-and-inactive-projects.html | 15:28 |
| gmann | but it will be good to discuss it further | 15:28 |
| gmann | we have enough slot for TC at least as per the current topics present in etherpad so feel free to add more | 15:29 |
| gmann | but try to add before Friday central time 4-5PM | 15:29 |
| gmann | Schedule 'operator hours' | 15:30 |
| spotz | I can speak a little on this | 15:30 |
| gmann | we have ~12 project signed up for operator hours which is good | 15:30 |
| gmann | i sent ML reminder also | 15:30 |
| gmann | spotz: please go ahead | 15:30 |
| gmann | #link https://lists.openstack.org/pipermail/openstack-discuss/2022-October/030790.html | 15:31 |
| spotz | So we've put a link to all the operator hours in the main ops etherpad. Kendall puttogether a blog which has been mailed out to Large Scale and Public Cloud SiG Members as well as attendees from past OPS Meetups. We've also tweeted and retweeted | 15:31 |
| gmann | yeah, +1 | 15:32 |
| gmann | many tweeting it. I did this week. spreading the information will be very helpful | 15:32 |
| spotz | They've been invited to attend the enviro-sus meeting Monday morning for an intro to the PTG type session and on THursday we hope to get feedback for the future though there are more sessions on Friday | 15:32 |
| gmann | #link https://etherpad.opendev.org/p/oct2022-ptg-openstack-ops | 15:33 |
| gmann | central etherpad ^^ | 15:33 |
| gmann | spotz: thanks, please ask them to join IRC channel also in case they find any difficulties to join PTG or switch to projects operator-hours | 15:33 |
| fungi | i've also been reaching out where it makes sense. brought all those sessions up during the scs-tech meeting earlier today, for example | 15:34 |
| gmann | +1 | 15:34 |
| gmann | fungi: thanks | 15:34 |
| fungi | there seemed to be interest in the large-scale and public cloud sig meetings too | 15:34 |
| gmann | especially if they join #openinfra-events channel there we can help them for joining issues or so | 15:35 |
| gmann | great | 15:35 |
| spotz | gmann I'll mention that to Kendall for the Monday morning session | 15:35 |
| spotz | I'll see her tonight | 15:35 |
| gmann | spotz: cool, thanks | 15:35 |
| gmann | anything else on PTG topic? | 15:36 |
| gmann | #topic 2023.1 cycle Technical Election & Leaderless projects | 15:36 |
| gmann | one thing left in this, appointment of Zun project PTL. | 15:37 |
| gmann | #link https://review.opendev.org/c/openstack/governance/+/860759 | 15:37 |
| gmann | hongbin volunteer to server as PTL and patch has good amount of required votes. it needs to be wait until 15 oct and I will merge if no negative fedback | 15:37 |
| gmann | #topic Open Reviews | 15:38 |
| gmann | #link https://review.opendev.org/q/projects:openstack/governance+is:open | 15:38 |
| gmann | we need review in this #link https://review.opendev.org/c/openstack/governance/+/860599 | 15:38 |
| noonedeadpunk | actually, ^ that is good topic | 15:39 |
| noonedeadpunk | or well, quite valid comment to it | 15:39 |
| gmann | discuss it in PTG? | 15:40 |
| noonedeadpunk | on top of that we should actully decide what OS grenade job for N+2 should run? | 15:40 |
| gmann | sure, we can discuss it there next week. I will add it. thanks | 15:40 |
| noonedeadpunk | as it's either leaving py3.8 on 2023.1 or backporting py3.10 to Y | 15:40 |
| noonedeadpunk | and focal vs yammy | 15:41 |
| noonedeadpunk | *jammy | 15:41 |
| dansmith | mmm, yammy | 15:41 |
| fungi | yummy yams | 15:41 |
| gmann | :) | 15:41 |
| dansmith | I think we have to run it on focal right? | 15:41 |
| dansmith | much easier to do that than anything else I think | 15:41 |
| noonedeadpunk | For me - yes | 15:41 |
| gmann | yammy could be good name though | 15:41 |
| dansmith | but we should discuss next week | 15:41 |
| gmann | yeah, let's discuss and clarify the things accordingly in doc | 15:42 |
| gmann | all other open reviews are in good shape | 15:42 |
| noonedeadpunk | yeah, that's why I always mix the first letter, as yammy really way better name :D | 15:42 |
| slaweq | actually I have a question about https://review.opendev.org/c/openstack/governance/+/836888 | 15:42 |
| gmann | :) | 15:42 |
| slaweq | should we maybe find new volunter who would work on this? | 15:42 |
| spotz | I can't read Jammy without thinking of the dog I had with that name:( | 15:42 |
| slaweq | or abandon it maybe? | 15:42 |
| gmann | next week we will be in PTG, so I will send the meeting cancel on ML | 15:42 |
| gmann | slaweq: yeah, good point | 15:43 |
| *** tkajinam is now known as Guest2971 | 15:43 | |
| gmann | jungleboyj: not sure if you will be able to update it or work on it? if no then either of us can pick it | 15:43 |
| jungleboyj | Sorry, got pulled into another meeting. | 15:43 |
| jungleboyj | Assume this is related to the review of the User Survey stuff? | 15:45 |
| gmann | jungleboyj: yes | 15:45 |
| jungleboyj | Ok. I was planning to look at that this week before the PTG. Other fires started. | 15:46 |
| gmann | jungleboyj: thanks. ping us if you need help | 15:46 |
| jungleboyj | I am going to try to look at it tomorrow or during the PTG next week so I can wrap it up. | 15:46 |
| jungleboyj | gmann: I will. Apologies. | 15:46 |
| gmann | cool | 15:46 |
| slaweq | thx jungleboyj++ | 15:46 |
| gmann | np! | 15:46 |
| gmann | ok with next week meeting cancel, our next weekly meeting will be on Oct 27. | 15:46 |
| gmann | with that, that is all for today meeting | 15:46 |
| opendevreview | Merged openstack/governance master: Add project for managing zuul jobs for charms https://review.opendev.org/c/openstack/governance/+/861044 | 15:47 |
| jungleboyj | ++ | 15:47 |
| slaweq | o/ | 15:47 |
| gmann | if nothing else to discuss then let's close it little early (13 min before) | 15:47 |
| spotz | woohoo | 15:47 |
| gmann | thanks everyone for joining. | 15:48 |
| gmann | #endmeeting | 15:48 |
| opendevmeet | Meeting ended Thu Oct 13 15:48:08 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:48 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/tc/2022/tc.2022-10-13-15.00.html | 15:48 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/tc/2022/tc.2022-10-13-15.00.txt | 15:48 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/tc/2022/tc.2022-10-13-15.00.log.html | 15:48 |
| JayF | thanks, have a good one o/ | 15:48 |
| spotz | Thanks gmann and everyone! | 15:48 |
| knikolla | see you all next week virtually! | 15:48 |
| jungleboyj | Thanks! | 15:48 |
| opendevreview | Ghanshyam proposed openstack/project-team-guide master: Clarify Extended Maintenance branch testing and support policy https://review.opendev.org/c/openstack/project-team-guide/+/861141 | 15:58 |
| *** Guest2868 is now known as diablo_rojo | 17:59 | |
| *** dasm is now known as dasm|off | 23:52 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!