| rpittau | good morning ironic! o/ | 06:47 |
|---|---|---|
| rpittau | if anyone has a moment please +W https://review.opendev.org/c/openstack/metalsmith/+/892965 so we can release metalsmith :) | 08:52 |
| mmalchuk | good morning ironic! o/ | 09:46 |
| opendevreview | Harald Jensås proposed openstack/ironic master: devstack - fix IPv6 ping https://review.opendev.org/c/openstack/ironic/+/893151 | 10:48 |
| iurygregory | good morning Ironic | 11:35 |
| iurygregory | rpittau, if you have some time https://review.opendev.org/c/openstack/python-ironicclient/+/891560 | 11:49 |
| opendevreview | Verification of a change to openstack/metalsmith master failed: Remove CentOS Stream 8 jobs https://review.opendev.org/c/openstack/metalsmith/+/892965 | 12:39 |
| rpittau | of course | 12:46 |
| rpittau | iurygregory: I'll add it to my list | 12:47 |
| iurygregory | rpittau, tks! (if I recall JayF is holding the release so we can include it) | 12:47 |
| rpittau | I'll try to get to it before EOD | 12:48 |
| iurygregory | ty! | 12:51 |
| TheJulia | good morning | 13:33 |
| rpittau | good morning TheJulia :) | 13:35 |
| masghar | Good morning! | 13:39 |
| iurygregory | good morning TheJulia =) | 13:40 |
| masghar | I have a question: I am trying to move the local-link-connection inspection hook into ironic (link to inspector implementation: https://opendev.org/openstack/ironic-inspector/src/branch/master/ironic_inspector/plugins/local_link_connection.py) | 13:41 |
| masghar | And my ramdisk is not collecting any LLDP information for my VM's network interface. I tried increasing the timeout to 180s, but same result | 13:42 |
| masghar | Is there some kind of switch configuration I should do in bifrost? | 13:42 |
| iurygregory | I think you need pa-collect-lldp=1 as kernel parameter | 13:43 |
| iurygregory | ipa-collect-lldp=1 * | 13:43 |
| masghar | let me try that, thanks | 13:44 |
| iurygregory | check the ironic.conf under [inspector] extra_kernel_params, to see what is there | 13:45 |
| masghar | I added ipa-inspection-collectors=lldp; and now also added ipa-collect-lldp=1 | 13:50 |
| iurygregory | fingers crossed | 13:51 |
| masghar | no change | 13:56 |
| opendevreview | Merged openstack/ironic master: devstack - fix IPv6 ping https://review.opendev.org/c/openstack/ironic/+/893151 | 13:57 |
| rpittau | masghar: now that I see that, shouldn't that be lldp_basic ? | 13:58 |
| rpittau | also I think the ipa-collect-lldp kernel param is deprecated in favor of the collectors | 13:58 |
| masghar | yes, I added just to be sure. lldp_basic was another hook, let me see | 13:59 |
| rpittau | it says indeed lldp in the docs https://docs.openstack.org/ironic-python-agent/latest/admin/how_it_works.html | 14:01 |
| masghar | Yes, the lldp-basic hook will not be migrated as per the spec: https://specs.openstack.org/openstack/ironic-specs/specs/approved/merge-inspector.html#inspection-hooks | 14:03 |
| masghar | (fwiw, the eth0 interface on the VM is part of the virbr0 bridge) | 14:06 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Enable OVN CI https://review.opendev.org/c/openstack/ironic/+/885087 | 14:19 |
| opendevreview | Rafal Lewandowski proposed openstack/bifrost master: Fix for lack of log rotation in Bifrost https://review.opendev.org/c/openstack/bifrost/+/893195 | 14:23 |
| rpittau | masghar: I'm not sure that will work in bifrost with libvirt, maybe TheJulia or hjensas can give a hint here ? | 14:25 |
| TheJulia | huh, what? | 14:25 |
| rpittau | :D | 14:25 |
| TheJulia | I didn't do it! | 14:25 |
| * TheJulia hides | 14:25 | |
| masghar | :P | 14:26 |
| TheJulia | so the lldp hook just won't get data in CI | 14:26 |
| opendevreview | Rafal Lewandowski proposed openstack/bifrost master: Fix for lack of log rotation in Bifrost https://review.opendev.org/c/openstack/bifrost/+/893195 | 14:26 |
| TheJulia | since there is no lldp, the kernel param is still a think afaik | 14:26 |
| masghar | I am not trying anything on CI, just running it on a local bifrost setup | 14:27 |
| TheJulia | still won't collect data, you need an actual physical switch | 14:27 |
| masghar | ah right! In that case I should introduce that in my setup somehow... | 14:28 |
| masghar | a virtual switch? | 14:29 |
| masghar | but you said actual physical switch | 14:29 |
| TheJulia | yeah, real physical enterprise switches put lldp bridgeport data packet updates to the wire | 14:32 |
| TheJulia | virtual switches, not really | 14:32 |
| masghar | alright, I should maybe try to get my hands on one then | 14:40 |
| masghar | thank you! | 14:40 |
| TheJulia | oh where oh where did my brain go | 15:31 |
| TheJulia | folks, reminder. When we cut releases upstream, we need to update release_mappings.py | 15:37 |
| TheJulia | for numbered versions | 15:37 |
| rpittau | mmm I think I've never done that for bugfix, only for stable | 15:40 |
| rpittau | should we that for bugfix branches too ? | 15:40 |
| TheJulia | on master at least | 15:41 |
| TheJulia | it doesn't need to land on the branch | 15:41 |
| rpittau | ok | 15:42 |
| TheJulia | the upgrade modeling needs to be able to relate numbered versions | 15:42 |
| iurygregory | we used to add the ones for bugfix https://github.com/openstack/ironic/blob/master/ironic/common/release_mappings.py#L534 | 15:43 |
| iurygregory | but seems like we missed some of the 22.* | 15:44 |
| TheJulia | yeah | 15:44 |
| rpittau | good night! o/ | 15:50 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Utilize the JSON-RPC port https://review.opendev.org/c/openstack/ironic/+/879215 | 15:56 |
| opendevreview | Mahnoor Asghar proposed openstack/ironic master: [WIP] Add inspection hooks https://review.opendev.org/c/openstack/ironic/+/892661 | 16:09 |
| opendevreview | Merged openstack/metalsmith master: Remove CentOS Stream 8 jobs https://review.opendev.org/c/openstack/metalsmith/+/892965 | 16:12 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Correct bindep.txt entries for bookworm https://review.opendev.org/c/openstack/ironic/+/893225 | 16:31 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Transiton to Storage resource from SimpleStorage https://review.opendev.org/c/openstack/ironic/+/892252 | 16:36 |
| TheJulia | stevebaker[m]: quick review for you when you: https://review.opendev.org/c/openstack/diskimage-builder/+/893196 | 16:39 |
| JayF | nice, that patch seems to have fixed the py311 unit tests, they are running now in zuul | 16:41 |
| JayF | fun | 16:41 |
| JayF | I'm curious if we'll pass :) | 16:41 |
| JayF | almost certainly yes as I'm sure I've accidentally run them on 3.11 locally | 16:42 |
| mmalchuk | folks, please review: https://review.opendev.org/q/hashtag:ironic-week-prio+(status:open+OR+status:merged)+topic:extend-dib-checksum | 16:46 |
| JayF | mmalchuk: you sure that DIB_CHECKSUM change you made got backported that far? | 16:47 |
| JayF | mmalchuk: just sanity checking | 16:47 |
| mmalchuk | JayF sure | 16:47 |
| mmalchuk | they stacked | 16:47 |
| JayF | I'm saying for https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/892926/ | 16:48 |
| JayF | that adds support for other checksums | 16:48 |
| JayF | that was added in a DIB change | 16:48 |
| JayF | are you sure that DIB change got backported and was released for those releases? | 16:48 |
| mmalchuk | I did DIB change | 16:48 |
| JayF | it's gotta be released, too, I think, for the backports to land | 16:48 |
| mmalchuk | also ask for tag | 16:49 |
| JayF | yeah, basically once that release happens | 16:49 |
| JayF | the bump should happen in openstack/requirements for the constraints file for that stable/ version | 16:49 |
| JayF | then we can put it in IPA-B | 16:49 |
| JayF | otherwise if we release IPA-B before DIB, we release broken DIB_CHECKSUM support | 16:49 |
| TheJulia | so the underlying conundrum is I think dib's branches are more for convenience | 16:49 |
| TheJulia | I'm not sure we would release from them | 16:50 |
| mmalchuk | yep, this need also urgent fix :https://review.opendev.org/c/openstack/diskimage-builder/+/893196 | 16:50 |
| JayF | Yeah I have +1 on it but I don't have core in DIB | 16:50 |
| JayF | TheJulia: mmalchuk: Then I'm not convinced we should backport it since we do release stable/ versions of IPA-B | 16:50 |
| mmalchuk | np, waiting for Steve | 16:50 |
| clarkb | dib doesn't branch fwiw | 16:51 |
| mmalchuk | it would be nice to land code in the git. yep, dib have a master only | 16:52 |
| clarkb | mmalchuk: why are you asking for tags if there is an unmerged urgent fix? | 16:52 |
| JayF | So that means we do have to reject the extend DIB_CHECKSUM patches in stable/ in IPA-B | 16:52 |
| mmalchuk | clarkb proposed today only | 16:52 |
| clarkb | ack | 16:52 |
| mmalchuk | JayF no! not affected | 16:53 |
| mmalchuk | JayF there is no async in IPA-B | 16:53 |
| JayF | I'm not sure what you mean; but I put a comment on the change so if I'm wrong please put it there and I'll have another look | 16:53 |
| JayF | I want to be defensive here so we don't release an IPA-B that just won't work :( | 16:53 |
| mmalchuk | DIB uses async, IPA-B not | 16:53 |
| JayF | what does async mean in this case | 16:53 |
| mmalchuk | exec in bash with & and use wait call | 16:54 |
| mmalchuk | async everywhere async) | 16:54 |
| TheJulia | *but* ipa-b can consume latest dib... | 16:54 |
| mmalchuk | TheJulia ipa-и | 16:55 |
| JayF | TheJulia: https://github.com/openstack/ironic-python-agent-builder/blob/stable/2023.1/tox.ini#L13 | 16:55 |
| mmalchuk | TheJulia ipa-b uses own call to checksums | 16:55 |
| JayF | TheJulia: in theory, yes. in CI, we tie it to the constaints for that release | 16:55 |
| mmalchuk | here https://review.opendev.org/q/hashtag:ironic-week-prio+(status:open+OR+status:merged)+topic:extend-dib-checksum | 16:55 |
| JayF | mmalchuk: am I misremembering? Wasn't there a change to make DIB_CHECKSUM work in diskimage-builder proper? | 16:56 |
| JayF | yeah this https://github.com/openstack/diskimage-builder/commit/43e47f1912ce07e65181355d4f5c52298a8fba28 | 16:56 |
| JayF | doesn't https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/892913 depend on ^^^ | 16:56 |
| mmalchuk | JayF yes it uses async and this only in dib | 16:57 |
| mmalchuk | ipa-b use: https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/881299 | 16:57 |
| TheJulia | yeah, those are disjointed operations | 16:57 |
| JayF | oh boy | 16:57 |
| mmalchuk | and there is no async | 16:57 |
| JayF | so we use the DIB_CHECKSUM var that DIB uses | 16:58 |
| JayF | and reimplement it ourself | 16:58 |
| mmalchuk | sure) | 16:58 |
| JayF | and want to backport it to DIB releases that wouldn't support those values outside of ipa-b? | 16:58 |
| JayF | er, ^ is not exactly right but I mean, we want to backport it to releases where the matching DIB wouldn't understand, e.g. DIB_CHECKSUM=sha256? | 16:58 |
| mmalchuk | Clark ask me to) | 16:58 |
| JayF | I'm just struggling with what this means for a backport | 16:59 |
| JayF | because IPA-B mostly presents a DIB-like interface, and references DIB docs | 16:59 |
| JayF | this change getting backported would somewhat invalidate that on stable branches | 16:59 |
| JayF | but because the feature itself is new, DIB_CHECKSUM=blah would've been completely ignored before, now we do a thing | 16:59 |
| JayF | it's probably OK? | 16:59 |
| TheJulia | I don't think it is entirely new, I think previously it just did md5 ages ago | 17:00 |
| mmalchuk | ok | 17:00 |
| JayF | ah | 17:00 |
| TheJulia | but it is a bit fuzzy and most likely okay | 17:00 |
| JayF | I'm basically on the border | 17:00 |
| mmalchuk | we later add DIB_CHECKSUM=sha512 or whatever and drop MD5 | 17:00 |
| JayF | which we usually default to "yes" if we're on the border | 17:00 |
| JayF | mmalchuk: that change will not be backportable :D | 17:00 |
| mmalchuk | but we need to | 17:01 |
| mmalchuk | because DIB don't have branches | 17:01 |
| JayF | ipa-builder, as configured and treated right now | 17:01 |
| JayF | when it branches | 17:01 |
| JayF | is stickied to the version of DIB in upper-constraints.txt for that release | 17:02 |
| JayF | so e.g. stable/2023.1 IPA-B would *never* install a DIB version with changes to DIB_CHECKSUM=md5 gone *unless* someone forced it to | 17:02 |
| JayF | and it's this way for a reason, it's not arbitrary | 17:02 |
| JayF | because you may need to make an IPA ramdisk for stable/victoria, using distros from the era of stable/victoria which have been retired from master-dib | 17:03 |
| mmalchuk | victoria not interested, but xena | 17:04 |
| JayF | either way, you get the idea :) we can't break the old versions against old versions of dib | 17:04 |
| JayF | and we don't test old versions against new versions of dib | 17:04 |
| JayF | if that's wrong, we can talk about changing it, but I am not sure I'm clever enough to find a better way to organize it :) | 17:05 |
| clarkb | old dib does both md5 and sha256. FWIW my understanding is that md5 is still acceptable for data integrity checks. I undersatnd that people want to kill it to remove any confusion or misapplication but I'm pretty sure its fine for this particular use case | 17:07 |
| clarkb | its there so that when you upload it to a cloud provider or copy it from one host to another you can ensure bits don't flip | 17:07 |
| JayF | clarkb: very yes, although mmalchuk left :) | 17:07 |
| TheJulia | stevebaker[m]: re-review if your up to it https://review.opendev.org/c/openstack/ironic-python-agent/+/888794 | 17:09 |
| TheJulia | clarkb: I've got folks that just want to have a default for all md5 to be disabled. | 17:10 |
| clarkb | side note: these hashes are largely useless when uploading to glance because glance ends up converting images in many cases and the bits change so you have no way of checking the cloud actually got the correct bits in many cases | 17:11 |
| iurygregory | JayF, if you have time today https://review.opendev.org/c/openstack/python-ironicclient/+/891560 so we can move with the release later =) | 17:11 |
| JayF | I'm in review mode right now, good timing | 17:12 |
| iurygregory | :D | 17:12 |
| JayF | you just got to the top of the list | 17:12 |
| TheJulia | clarkb: for ramdisk/initramfs, it doesn't, and the checksums can still be compared afaik | 17:12 |
| clarkb | thats nice | 17:12 |
| TheJulia | its not like a qcow2 it can re-pack | 17:12 |
| clarkb | to be fair I'm only aware of one instance where we had a bad upload corrupt an image so incidence is low. but after that happened I tried to determien if we could guard against it by comparing hashes post upload and realized you can't | 17:13 |
| clarkb | makes sense that simpler "lower level" images like ramdisks wouldn't have conversion issues the same way though | 17:13 |
| TheJulia | aiui, the whole driver is also so one can take checksums and compare them later on for the downloaded artifacts to make sure MITM attack hasn't occured | 17:14 |
| opendevreview | Verification of a change to openstack/ironic stable/wallaby failed: [iRMC] Fix parse_driver_info bug enforcing SNMP v3 under FIPS mode https://review.opendev.org/c/openstack/ironic/+/885246 | 17:15 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Add missing release mappings for 22.0 and 22.1 https://review.opendev.org/c/openstack/ironic/+/893232 | 17:33 |
| opendevreview | Merged openstack/ironic-python-agent-builder stable/zed: Add libaio engine for fio on Centos/Rhel9 https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/886489 | 17:51 |
| stevebaker[m] | good morning | 18:05 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Add missing release mappings for 22.0 and 22.1 https://review.opendev.org/c/openstack/ironic/+/893232 | 18:05 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: Intentional break to investigate grub2 boot ops https://review.opendev.org/c/openstack/ironic/+/893238 | 18:05 |
| TheJulia | good morning stevebaker[m] | 18:08 |
| JayF | https://review.opendev.org/c/openstack/ironic/+/893225 fixes those annoying py3.11 timeouts | 18:20 |
| JayF | and hey, our unit tests pass on 3.11 \o/ | 18:20 |
| TheJulia | so with mysql == sadness, but with mariadb == happiness? | 18:22 |
| JayF | no, mysql-client doesn't exist on debian | 18:22 |
| JayF | the timeouts were because bindep was bailing super early | 18:22 |
| TheJulia | oh, that explains it only lasting 1 minute 30 seconds | 18:23 |
| JayF | they have 'default-mysql-client' and 'virtual-mysql-client' which appear to both point to mariadb-client | 18:23 |
| JayF | and I decided to just make it clear in bindep.txt what was happening | 18:23 |
| TheJulia | cool cool | 18:26 |
| TheJulia | Thanks! | 18:26 |
| opendevreview | Merged openstack/ironic-python-agent-builder stable/zed: Add checksum generation support https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/892925 | 18:34 |
| opendevreview | Merged openstack/ironic-python-agent stable/2023.1: Handle the node being locked https://review.opendev.org/c/openstack/ironic-python-agent/+/892593 | 18:40 |
| opendevreview | Merged openstack/python-ironicclient master: Firmware Interface support https://review.opendev.org/c/openstack/python-ironicclient/+/891560 | 18:42 |
| JayF | Has anyone run ironic-ui recently? | 18:49 |
| JayF | It's in the list of things that need a release, it's got no meaningful commits in a year+ | 18:49 |
| * JayF created the release patch and -W -1 it himself here https://review.opendev.org/c/openstack/releases/+/893274 | 18:56 | |
| TheJulia | run, dunno. but we haven't gotten reports that it is outright broken afaik | 18:56 |
| JayF | I don't feel good about just rubber stamping a release | 18:57 |
| JayF | but I also don't feel good if it works and we don't do anything with it | 18:58 |
| jrosser | i run it on Z | 18:58 |
| jrosser | and 2023.1 in the lab | 18:59 |
| opendevreview | Jay Faulkner proposed openstack/ironic-ui master: DNM: Dummy change testing CI https://review.opendev.org/c/openstack/ironic-ui/+/893275 | 18:59 |
| TheJulia | Then zigo's change at least passing seems okay | 18:59 |
| JayF | jrosser: you mind putting that comment on https://review.opendev.org/c/openstack/releases/+/893274 ? | 19:00 |
| JayF | jrosser: not sure how much you gerrit tbh if that's a hardship I can proxy it | 19:00 |
| jrosser | done :) | 19:01 |
| * TheJulia goes back to reading a very long document while waiting for a CI job to fail. | 19:01 | |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Fully monkey patch eventlet for consistent behavior https://review.opendev.org/c/openstack/ironic/+/887996 | 20:08 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: DNM: Testing Nova-Ironic driver lessee change. https://review.opendev.org/c/openstack/ironic/+/889122 | 20:21 |
| opendevreview | Julia Kreger proposed openstack/ironic master: PXE: Remove DHCP option 210 from being set https://review.opendev.org/c/openstack/ironic/+/893286 | 20:30 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Enable OVN CI https://review.opendev.org/c/openstack/ironic/+/885087 | 20:37 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: Test grub+ovn-dhcp-opt-210 https://review.opendev.org/c/openstack/ironic/+/893287 | 20:37 |
| opendevreview | Julia Kreger proposed openstack/ironic master: CI: Fix our internal MTU settings https://review.opendev.org/c/openstack/ironic/+/893112 | 20:46 |
| opendevreview | Merged openstack/ironic-python-agent-builder stable/yoga: Add checksum generation support https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/892966 | 20:53 |
| opendevreview | Jay Faulkner proposed openstack/bifrost master: DNM: Verify bifrost jobs can be run w/zuul on ansible 8 https://review.opendev.org/c/openstack/bifrost/+/893291 | 21:24 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: DNM: Testing Ironic jobs w/Zuul running ansible 8 https://review.opendev.org/c/openstack/ironic/+/893295 | 22:30 |
| JayF | FYI: all voting bifrost jobs passed w/ansible 8, and the Ironic jobs look on their way to the same (already have some passing, the others got kicked off successfully so no apparent impact) | 23:21 |
| JayF | they are gonna change the default on monday so I thought it prudent to test :) | 23:21 |
| TheJulia | Awesome | 23:22 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!