| opendevreview | Tony Breeds proposed openstack/governance master: Update with 2023.2 Election results. https://review.opendev.org/c/openstack/governance/+/874042 | 00:19 |
|---|---|---|
| opendevreview | Tony Breeds proposed openstack/governance master: Update with 2023.2 Election results. https://review.opendev.org/c/openstack/governance/+/874042 | 00:28 |
| gmann | tonyb: thanks. +1 | 00:44 |
| *** dasm|off is now known as dasm | 14:00 | |
| *** blarnath is now known as d34dh0r53 | 14:26 | |
| bauzas | man, the gate is horrendous those days | 14:43 |
| bauzas | I understand that the TC is feeling that the gate becomes better, from from my personal POV, sorry no :( | 14:44 |
| dansmith | bauzas: it's definitely better than it was a week ago, but the volume failures haven't improved lately, which is what nova feels the most generally | 14:45 |
| bauzas | I also see some cirros-0.5.2 problems now | 14:46 |
| bauzas | and not only volume-related problems | 14:46 |
| bauzas | I don't feel those are particular to nova | 14:46 |
| dansmith | well, I looked at that one you posted and I guess I'm not sure it's not related to volume stuff. the volume got stuck too right? | 14:47 |
| bauzas | we continue to have volume timeouts yes | 14:47 |
| bauzas | but we also have ssh guest failures due to multiple reasons | 14:47 |
| dansmith | oh, are you talking about a cirros guest kernel panic? | 14:47 |
| bauzas | one can be the dhcp client not being able to get a lease | 14:47 |
| bauzas | other being cirros panicking, yes, or segfaulting | 14:48 |
| dansmith | I've definitely seen those, that's different than what I saw you post (before) that | 14:48 |
| dansmith | yeah, but you were opposed to trying to bump the cirros version? | 14:48 |
| dansmith | (we should probably chat about this in -qa) | 14:48 |
| bauzas | no, I'm not opposed, I even have an open change against nova | 14:48 |
| dansmith | oh, cool | 14:48 |
| bauzas | for bumping to 0.6.1 | 14:48 |
| dansmith | oh sorry, I guess that was sean | 14:49 |
| bauzas | let's discuss it back in -nova if you want, but I wanted to convince the TC that we're unfortunately not done yet :) | 14:49 |
| bauzas | Today is FF, and I still have a shitty ton of accepted changes that I need to recheck due to CI failures | 14:50 |
| dansmith | if you find someone that thinks we're done, let me know :) | 14:50 |
| bauzas | I'm like able to merge one single patch per day :) | 14:50 |
| bauzas | (not joking) | 14:50 |
| dansmith | I hear you, I've been fighting for weeks | 14:50 |
| dansmith | (as you know) | 14:50 |
| fungi | we did merge a change to start caching the latest cirros version on our node images, so that's already in place | 14:57 |
| dansmith | cool, I think we should make nova-next run with that for a while and see how it goes | 14:59 |
| dansmith | that's our bleeding edge job | 14:59 |
| bauzas | ok, then I should modify https://review.opendev.org/c/openstack/nova/+/873934 | 14:59 |
| fungi | we're also dropping cirros versions older than 0.5.2 from the on-image cache since opensearch indicated no recent/frequent jobs were using other versions | 15:00 |
| fungi | (devstack will still fetch them from the internet if a job does request some other version of course) | 15:01 |
| dansmith | bauzas: modify it how? | 15:01 |
| bauzas | dansmith: to update to 0.6.2 ? | 15:02 |
| dansmith | oh sure okay | 15:02 |
| * dansmith didn't look up what the actual current version is | 15:02 | |
| dansmith | I assumed 0.6.1 :) | 15:03 |
| fungi | oh, is there a 0.6.2 now? | 15:03 |
| dansmith | bauzas: see^ :) | 15:03 |
| bauzas | nevermind https://github.com/cirros-dev/cirros/releases | 15:04 |
| * bauzas is lost sometimes | 15:04 | |
| * fungi is lost most times | 15:04 | |
| dansmith | where are we? | 15:04 |
| dansmith | bauzas: I hadn't actually looked, but did you see that job failed on network things? | 15:05 |
| dansmith | failed to ssh to the guest | 15:05 |
| bauzas | yes, we have a couple of usual suspects | 15:05 |
| bauzas | for the dhcp lease thing, we have a tracker | 15:06 |
| dansmith | well, I'm just saying, that's three cases where it looks like the guest didn't get an IP | 15:06 |
| dansmith | which, if related the dhcp client change, would be a *higher* incidence of that behavior than we currently see, I'm sure | 15:06 |
| dansmith | I just rechecked | 15:07 |
| bauzas | yup, we only see a SSH timeout | 15:07 |
| bauzas | but from what I can read, sometimes we have the guest failing to access the metadata, sometimes the lease acquisition not there before the job times out, sometimes the guest panicking | 15:08 |
| dansmith | right but we see guest panics on the console I think | 15:09 |
| dansmith | bauzas: at least one of those fails are on volume things where we spend a lot of time trying to clean up the volume long after the ssh failure | 15:10 |
| dansmith | anyway, we should take this out of -tc I think, let's go back to nova for all this | 15:10 |
| bauzas | ++ | 15:10 |
| fungi | are the panics consistently about the same problem, or are they random panic reasons? | 15:10 |
| fungi | but yeah, this is not the channel to dig into it | 15:11 |
| *** dasm is now known as Guest5046 | 16:46 | |
| *** dasm is now known as Guest5052 | 18:10 | |
| gmann | tc-members: need more vote on these https://review.opendev.org/c/openstack/governance/+/872233 https://review.opendev.org/c/openstack/governance/+/873440 | 18:48 |
| opendevreview | Merged openstack/governance master: Add guidelines about naming versions of the OpenStack projects https://review.opendev.org/c/openstack/governance/+/872769 | 18:58 |
| *** dasm_ is now known as dasm|off | 23:55 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!