Monday, 2024-12-16

Sandzwerg[m]That might be, community images are certainly something newer: It was quiet a while ago but I still remember that they were introduced01:27
opendevreviewcid proposed openstack/ironic master: [WIP] Migrates Inspector rules over to Ironic  https://review.opendev.org/c/openstack/ironic/+/91830308:08
rpittaugood morning ironic! o/08:53
Sandzwerg[m]Morning o/10:52
fricklerrpittau: I forgot whether there were any plans to just drop these bugfix branches? I do remember there was some discussion to drop unmaintained branches for bifrost. since this is now the largest remaining block of config errors, please let me know how we could best make progress on this https://zuul.opendev.org/t/openstack/config-errors?project=openstack%2Fbifrost&skip=0&limit=5011:30
rpittauhey frickler we regularly remove old bugfix branches, but I think we forgot the bifrost ones!11:53
rpittauI'll take care of that11:53
fricklercool, thx12:07
rpittaufrickler: should be fine now, but please let me know if zuul still reports errors for that12:12
fricklerrpittau: wow, that went fast. only 5 errors remaining for unmaintained branches, but those can also be handled by the usual eol process if needed. thanks for your help12:30
rpittauanytime :)13:08
TheJuliagood morning14:23
cardoemorning all14:31
rpittau#startmeeting ironic15:00
opendevmeetMeeting started Mon Dec 16 15:00:48 2024 UTC and is due to finish in 60 minutes.  The chair is rpittau. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
opendevmeetThe meeting name has been set to 'ironic'15:00
iurygregoryo/15:00
rpittauHello everyone!15:00
rpittauWelcome to our weekly meeting!15:00
rpittauThe meeting agenda can be found here:15:00
rpittauhttps://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_December_16.2C_202415:00
rpittauwaiting for people to join :)15:01
TheJuliaGood morning15:01
rpittauwe don't have a lot of things to discuss, so I'll just start15:03
rpittau#topic Announcements/Reminders15:03
TheJuliaSounds good15:03
rpittau#info Standing reminder to review patches tagged ironic-week-prio and to hashtag any patches ready for review with ironic-week-prio15:03
rpittau#link https://tinyurl.com/ironic-weekly-prio-dash15:03
rpittauI'll have alook during the week, some outstanding patches there15:03
JayFThose ironic tempest patches from Adam could really use a look. I know he's ready to make a bunch more of them but want some idea that the consensus is that they're good15:04
rpittauack15:04
cardoeI'll get back to reviews this week.15:04
cardoeI've got a handful for ruff stuff that I haven't tagged with ironic-week-prio but I can.15:05
JayFPlease do, I'll make a commitment to go through all the tagged patches today15:05
cardoeI've been looking for other lints to enable that match style / behavior we're using. Most of the other OpenStack projects are a bit inconsistent.15:05
* TheJulia suddenly dreads massive style conflicts15:06
rpittaummm let's try to keep consistent as much as possible15:07
JayFYeah. We likely want to get all the lint changes we want done and as few patches as possible so that the rebase pain is minimal15:07
JayF**in as few patches 15:07
opendevreviewDoug Goldstein proposed openstack/ironic master: move imports to top of file for lints  https://review.opendev.org/c/openstack/ironic/+/93727115:07
opendevreviewDoug Goldstein proposed openstack/ironic master: enable ruff in pre-commit with some initial lints  https://review.opendev.org/c/openstack/ironic/+/93727215:07
rpittauok, anything else to add for the weekly prios?15:08
rpittaugood!15:09
rpittau#info 2025.1 Epoxy Release Schedule15:09
rpittau#link https://releases.openstack.org/epoxy/schedule.html15:09
rpittauwe're at R-1515:09
rpittaunothing special to mention, we're close to mid dev cycle!15:09
cardoeSo I wanted to ask, do you want me to touch ironic-inspector? And anything going through deprecation?15:09
TheJuliaouch, time slowly disappearing15:09
rpittaucardoe: I don't think it's necessary15:09
rpittauTheJulia: not so slowly :/15:09
rpittau#info This is the last meeting of 2024!15:10
rpittauI'll be around until the 24th, but mostly busy with downstream stuff15:11
rpittauanything else to remind/announce?15:11
TheJuliaI'm gone as soon as I post an initial POC patch for artifacts from container registries15:11
rpittauI expect the next 3 weeks to be quiet :)15:12
cardoeI've got an internal release deadline but once I'm over that hump I'll be working on sushy more.15:13
rpittauok, moving on15:13
rpittau#topic Discussions 15:13
rpittauI have only one thing15:13
rpittau#info CentOS Stream 10 is GA https://blog.centos.org/2024/12/introducing-centos-stream-10/ , I've started looking into migration for our stuff (not urgent)15:13
JayFI would strongly recommend keeping that out of CI as long as possible15:14
rpittauI can confirm that ironic works well on CS10, at least according to metal3 CI :)15:14
JayFThat requires x86-64-v3 which some of our donor clouds do not support15:14
TheJuliaThere is a dual issue here, it is reasonable to push forward, but specific flags also get disabled at times15:15
rpittauJayF: I don't plan to have any patch up before the end of the cycle15:15
rpittauI doubt will see CS10 nodes soon15:15
JayFWe've already seen CI breakage caused by packages built against x86-64-v2, some Wheels built against Alma 9 broke in some jobs. I talked to upstream and got them to pull those broken Wheels.15:15
JayFYeah, and I think it's actually a somewhat reasonable decision, we've just got to be careful about how we go about it.15:15
TheJulia++15:15
rpittauyep15:15
JayFI'd hate for us to land something that passes on 2 out of three clouds just because we got "lucky"15:15
TheJuliaa middle ground might be if we can get some way/pattern to match jobs to clouds15:18
TheJuliaor clouds with specific capabilities15:18
JayFI also wonder for IPA testing. We can somehow get qemu to emulate newer CPUs than it's actually running on15:18
JayFI don't know if that's possible, but would be easy enough to check and to fix if so15:18
TheJuliaJayF: not... entirely, but maybe?15:19
TheJuliaWorth looking into, I guess15:20
rpittauanyway, I'll start slowly testing it locally, but until we have at least a  mention on CS10 nodes it's probably not worth going any deeper15:21
rpittauanything else to discuss today?15:22
rpittauI guess we can amend the bug deputy for the next 3 weeks :)15:23
JayFI've got nothing. And like many others, this is my last week in the office for 2024.15:23
rpittaualriht15:23
rpittaujust a reminder that next meeting will be on January 6th15:24
rpittauif there's nothing else we can close it here15:24
rpittauthank you all for a great 2024! :)15:24
rpittau#endmeeting15:24
opendevmeetMeeting ended Mon Dec 16 15:24:53 2024 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:24
opendevmeetMinutes:        https://meetings.opendev.org/meetings/ironic/2024/ironic.2024-12-16-15.00.html15:24
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/ironic/2024/ironic.2024-12-16-15.00.txt15:24
opendevmeetLog:            https://meetings.opendev.org/meetings/ironic/2024/ironic.2024-12-16-15.00.log.html15:24
cardoeTheJulia (and anyone else interested in the network side): As far as neutron + ironic goes, we're currently internally focused on some of that. We're working through that trunking stuff we discussed and how we can best upstream this.15:27
TheJuliacardoe:  Please refresh my memory, which trunking stuffs?15:28
cardoeSo we define a trunk port and then sub-ports for tagged VLANs. How to create the cloud-init metadata that maps to the physical port and to create the vlan tagged interfaces in the OS.15:29
cardoehttps://bugs.launchpad.net/nova/+bug/209118515:29
TheJuliaright15:31
TheJuliaso in essence, the issue is a bug in the lack of nova processing through that data. I guess the ironic virt driver adds a further wrinkle to it15:39
TheJuliaon a plus side, it does seem like "just a bug" with how it is framed15:39
cardoeI lost that bug / issue of the nova vs ironic issue that needed to get re-visited.15:42
cardoeBut I'm sure I'll run into it again.15:42
TheJuliaThe nova.virt.ironic driver does some further data processing to handle port groups15:54
TheJuliaso honestly, I'd try to just treat it as bugs to be fixed15:55
rpittaugood night! o/17:00
opendevreviewJay Faulkner proposed openstack/ironic-python-agent master: Remove dependency on ironic-lib  https://review.opendev.org/c/openstack/ironic-python-agent/+/93774318:17
adam-metal3Hello Ironic, is it just me or does others have issues acessing opendev gerrit via ssh?18:24
adam-metal3I was assuming it is some config thing but I can't even clone a repo18:24
iurygregoryadam-metal3, let me try doing a fresh clone 1min18:29
adam-metal3iurygregory, thanks18:31
iurygregoryadam-metal3, https://paste.opendev.org/show/bx7lLtOOvDby0wlsAfzl/ got this message when trying18:33
iurygregorydoing a `git clone git@github.com:openstack/ironic.git` works without any problem for me18:34
adam-metal3I was trying this way: git clone "ssh://<username>@review.opendev.org:29418/openstack/sushy-tools"18:35
adam-metal3this was going sideways for me18:35
iurygregorylet me try here18:36
iurygregoryadam-metal3, it worked for me O.o18:36
adam-metal3O_O18:36
iurygregorydo you have some config in ~/.ssh/config ?18:36
iurygregorythis is what I have  https://paste.opendev.org/show/bnBvA9yUo5VaS3XRXYhW/ 18:37
JayFI'm down an absolute crazy rabbithole cc: cid18:38
adam-metal3yes except this part PubkeyAcceptedKeyTypes 18:38
iurygregoryI only have this part because of the type of key I'm using I think18:39
JayFopenstack --os-cloud=valid-devstack-config baremetal node list # works18:39
JayFbaremetal --os-cloud=valid-devstack-config node list # fails with errors  about getting an auth_url, that appears to be due to auth type defaulting to none18:39
JayFbut if I force it to auth-type:password, like openstackclient does directly, it complains about not having a domain set18:39
JayFI think there's something subtly broken and I can't put my finger on it, and likely nobody has noticed because almost everyone prefixes with `openstack ` in integrated scenarios18:40
adam-metal3iurygregory, I don't get it I have removed my stuff then put it back and now it works18:42
iurygregoryadam-metal3, O.O WOOT?!18:43
adam-metal3it is super strange as I had a handfull of repos already with branches that I have pushed some time ago so it wasn't a new env18:43
TheJuliaadam-metal3: I've occasionally had similar issues with v6 routing breaking.18:44
iurygregoryit's weird indeed =( 18:44
adam-metal3TheJulia now that you mention I have received error messages with IPV6 addresses e.g. Cloning into 'sushy-tools'...18:45
adam-metal3Received disconnect from 2604:e100:1:0:f816:3eff:fe52:22de port 29418:2: Too many authentication failures: 718:45
adam-metal3Disconnected from 2604:e100:1:0:f816:3eff:fe52:22de port 2941818:45
adam-metal3fatal: Could not read from remote repository.18:45
iurygregoryIPv6 to blame :D18:45
opendevreviewAdam Rozman proposed openstack/ironic-python-agent master: integrate oslo imageutils low level format handling  https://review.opendev.org/c/openstack/ironic-python-agent/+/93780418:45
TheJuliaoh, wow, that is different18:46
adam-metal3iurygregory, thanks for the help I still have no issue what fixed it but now it works18:46
TheJuliawell, worked but auth failures. Interesting18:46
JayFadam-metal3: you sure you need that in IPA?18:47
iurygregoryadam-metal3, np =) 18:48
JayFadam-metal3: I was under the impression a bugfix for that was ironic-only18:48
adam-metal3JayF: I will push it for Ironic too, I am not sure if we can and up in the gpt+someotherformat situation I just noticed that the code follows the same logic so I thought it might worth to fix it there too18:50
JayFWell, I'm confused a bit18:51
JayFbecause I can't see a case where IPA should write an image that detects as iso18:51
JayFand plus the complexity being added there means it's extremely likely future oslo.utils changes will break us18:51
JayFIronic might see ISOs due to ramdisk driver, but IPA shouldn't18:52
adam-metal3but could we detect gpt + something else? 18:52
JayFI'd need to see a valid image that happens on in order to +2 that change18:52
JayFI would assume anything that tested as both is insecure in lieu of that evidence18:52
JayFif that makes sense?18:52
JayFI don't wanna make this loose because it might be needed, it's a security feature18:53
adam-metal3Yeah I am fine if it doesn't land in IPA but jsut FYI I didn't remove any security functionality if it finds gpt + some other format it runs the check for both. Anyhow I will go and tomorrow push up the Ironic fix, it is quite late here 18:54
JayFIs there a reason that code can't go into oslo.utils?18:54
JayFI just really wanna make sure I understand this; I lost a month of my life to a CVE around all this once already 😅18:55
adam-metal3From comments I have read in oslo utils I understood nova has these gpt images and the "formats" property was introduced AFAIK for other users who might need to handle multiple formats so it seemd to me that what we have run into is the "edge case" from oslo perspective18:57
JayFSo, I think part of the problem is I don't understand the edge case18:58
JayFThere's a good chance your change is right, I just don't understand the use case of when it happens18:58
JayFe.g. is it /actually a good idea/ to deploy an image that detects as two things rather than failing18:58
JayFand if it detects that way, why18:58
adam-metal3simple iso image that has a gpt partition that what we have given and it produced the error18:58
JayFwas the intent to write that ISO image to a disk?18:59
JayFOr was it being used for ramdisk driver (NOT IPA; ramdisk driver) 18:59
JayFIPA should only see images that are to be written to a disk; which to me seems like ISO is generally invalid19:00
JayFbut IMBW19:00
JayF(we can also pick this up async, I know it's late there)19:00
adam-metal3no that is why I am saying that I am fine if the code doesn't land in IPA I didnä see it in IPA I just noticed the logic is the same so thought I make this also handle it I am not disagreeing with you :D19:00
JayFack, okay19:00
JayFyeah, keeping the code in IPA as simple as "is it safe and sensible to write to disk" is preferred imo19:00
JayFand IPA only even runs a check when we are converting, and defaults are such that the conversion to raw/gpt happens on the conductor19:01
adam-metal3sounds reasonable, thanks for the explanation I will be back with the other fix soon, until then o/19:06
*** awb_ is now known as awb20:52
JayFhooray, remove-ironic-lib-from-ipa change is passing integration tests; I just need to port over the unit tests from ironic-lib and it'll be ready for review21:28
iurygregoryok I think I found the the place where the funny issue happens when doing firmware updates https://opendev.org/openstack/ironic/src/branch/master/ironic/drivers/modules/redfish/boot.py#L718 https://paste.opendev.org/show/bdf8ZjY9DXtJhzYGKTDm/  should we just try to add some extra logic in case we know that a firmware update was requested and let it wait a bit an retry to get the managers? 21:49
opendevreviewAdam McArthur proposed openstack/ironic-tempest-plugin master: Microversion Test Generator  https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/93720623:14
opendevreviewAdam McArthur proposed openstack/ironic-tempest-plugin master: Testing bad microversions on v1/allocations  https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/93721323:14
opendevreviewAdam McArthur proposed openstack/ironic-tempest-plugin master: Testing bad microversions on v1/nodes/{uuid}/firmware  https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/93721423:14

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!