| Continuity | Morning nova has a max build setting, does ironic have a similar setting to set how many machines ironic can attempt to build at the same time | 07:59 |
|---|---|---|
| dtantsur | Continuity: I believe so, let me check | 11:28 |
| Continuity | Thanks, having done some reading it looks like it relies on the max_concurrent_build of nova, but per conductor, so a max_concurrent_build in nova of 2 would be 6 if you have three conductors | 11:29 |
| Continuity | Unless im misreading | 11:29 |
| dtantsur | https://docs.openstack.org/ironic/latest/configuration/config.html#conductor.max_concurrent_deploy | 11:29 |
| TheJulia | ... Funny thing, we've actually had someone hit that ;) | 13:48 |
| cardoe | morning ironic | 14:36 |
| TheJulia | good morning | 14:46 |
| * TheJulia cries due to 4k | 14:46 | |
| cardoe | I missed your patch on that so I'm out of the loop. | 14:48 |
| TheJulia | tl;dr we're starting to see hardware which has raid controllers which apparently only support 4k virtual sector size | 14:49 |
| TheJulia | with LBA, and that renders GPT and all partitioning incompatible with 512 byte based layouts | 14:49 |
| cardoe | Lovely | 14:53 |
| opendevreview | Doug Goldstein proposed openstack/sushy master: add a link to the release notes in the README https://review.opendev.org/c/openstack/sushy/+/927598 | 14:55 |
| cardoe | Could we make a release of sushy-oem-idrac? | 14:57 |
| TheJulia | Hmmm... I wonder if anyone is available to cut that release right now | 15:01 |
| cardoe | Like no maintainer or you mean today? Cause there's no rush from me. There's just not been a release in 2 years and there's 2 fixes in there that I use and install from git rather than a release. | 15:04 |
| TheJulia | cardoe: today, specifically | 15:24 |
| TheJulia | I think Riccardo or Iury would be the ones who need to do so | 15:25 |
| JayF | I think I have access to do so as well, but I haven't worked a lot on that project so I would definitely prefer someone more familiar | 15:29 |
| JayF | Or idk, I might have lost that access when I stopped being ptl? Because I don't think I'm technically a release liaison... | 15:29 |
| TheJulia | yeah, you have to be a release liaison | 15:29 |
| dtantsur | JayF, TheJulia, I think Riccardo gave ~ everyone access to releasing sushy-oem-idrac | 16:18 |
| dtantsur | it's pity that gerrit UI is no longer useful to check groups and access rights | 16:19 |
| dtantsur | ah, wait, it's just hidden in another tab | 16:19 |
| dtantsur | https://review.opendev.org/admin/groups/51dc1df6f2ef25fe66385fca6f72c8feac3559dd,members | 16:19 |
| dtantsur | includes ironic-core | 16:20 |
| * dtantsur sighs and deletes Ilya | 16:20 | |
| dtantsur | I can do the release but please remind me on Monday: I don't want to do releasing after 6pm on Friday | 16:21 |
| dtantsur | cardoe: ^^ | 16:23 |
| cardoe | Will do. Thank you dtantsur | 16:23 |
| cardoe | So for an addition of the Ports interface to sushy, should I make a blueprint to discuss the addition? I’m just not sure if it needs the interop profile to change to say it’s an endpoint that is hit. | 17:49 |
| JayF | Ironic doesn't use blueprints; our typical process is file a bug in launchpad, tag it `rfe` and put a description in there | 17:50 |
| JayF | if the description is detailed enough, you can often get it approved without a spec by asking in the next ironic meeting; if it gets tagged `rfe-approved` it's good to go | 17:51 |
| JayF | alternatively, if it's not detailed enough or the feature is complex enough to want more detail, it'll be tagged `needs-spec` and you'll have to put a spec in ironic-specs | 17:51 |
| JayF | generally speaking we are OK with detailed RFE bugs for things that aren't like, revolutionary new features or things that need lots of planning/coordination | 17:51 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: add pyproject.toml to support pip 23.1 https://review.opendev.org/c/openstack/ironic/+/927544 | 17:59 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Enable WSGI module entrypoint for Ironic https://review.opendev.org/c/openstack/ironic/+/927635 | 17:59 |
| cardoe | JayF: not sure if it requires even a spec or rfe... basically I wanna bring in https://www.dmtf.org/sites/default/files/standards/documents/DSP0268_2024.2.html#port-1130 | 18:19 |
| JayF | I'd | 18:20 |
| JayF | **I'd just file the bug, it's trivial to file an rfe bug, and if you think it doesn't need much detail that's your call :D | 18:20 |
| cardoe | Sure. My questions are really what's the version of Redfish we're tracking so I use the correct version. Does it require a change to the interop profile? | 18:21 |
| cardoe | But I'll make the bug. | 18:21 |
| JayF | Those are *exactly* the kind of questions that generally come up when discussing an RFE | 18:31 |
| JayF | it's less for us about wanting the paperwork, and more about wanting the conversation and to get that conversation documented | 18:31 |
| JayF | I have taken the ironic-specs template and used it downstream because the headers worked really well as a "checklist" to ensure you've thought about all he high level impacts of a change. | 18:32 |
| cardoe | Yeah like I've wanted the sushy library to say which version of the redfish spec it tracks. | 18:33 |
| JayF | I think the reality of it is that it supports as many as we see? lol | 18:33 |
| cardoe | Well no. Each source file maps to a specific version of each model. | 18:34 |
| * JayF is showing his hand-waving level knowledge of sushy | 18:34 | |
| cardoe | So like above my link was to Redfish Model Spec 2024.2 which includes Port v1.13.0 with backwards compat to Port v1.3.0 | 18:34 |
| cardoe | For example sushy uses EthernetInterface v1.4.0 per the comment in that source file... # https://redfish.dmtf.org/schemas/EthernetInterface.v1_4_0.json | 18:35 |
| cardoe | But Redfish 2024.2 has EthernetInterface v1.12.2 https://www.dmtf.org/sites/default/files/standards/documents/DSP0268_2024.2.html#ethernetinterface-1122 | 18:37 |
| cardoe | EthernetInterface v1.4.0 is from Redfish 2017.2 | 18:37 |
| cardoe | EthernetInterface v1.7.0+ uses the Port interface. And https://bugs.launchpad.net/sushy/+bug/2075979 is a non-bug by using EthernetInterface v1.10.0 and newer. | 18:39 |
| cardoe | You see how you just proved that I need to make a RFE bug.... | 18:41 |
| JayF | yeah, like I said, I think it's as much about the process of thinking it through moreso than creating paperwork | 18:43 |
| cardoe | Like I mentioned yesterday, we have our own Redfish library for dealing with our gear. It's got a bunch of tests and it hits hardware. | 18:44 |
| cardoe | I'm refactoring it to be a shim over sushy (and sushy-oem-idrac) | 18:45 |
| cardoe | And looking to add all the support into sushy proper and upstream it. | 18:46 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Support Automatic Lessee from instance metadata https://review.opendev.org/c/openstack/ironic/+/924887 | 18:46 |
| cardoe | I commented on your other two ironic's JayF | 18:51 |
| * cid nips out for the week. o/ | 19:03 | |
| JayF | already replied to one of the comments | 19:04 |
| JayF | really until I get an "ironic-api can run in wsgi" flag in devstack, I'm nervous about merging that | 19:04 |
| TheJulia | .... I thought it was by default | 19:15 |
| JayF | I know in devstack, restart ir-api does a thing | 19:18 |
| JayF | so I assumed we ran the ironic-api process | 19:18 |
| JayF | ope, you're 100% right | 19:18 |
| JayF | tyvm for that pointer | 19:19 |
| JayF | ExecStart = /usr/bin/uwsgi --procname-prefix ironic-api --ini /etc/ironic/ironic-uwsgi.ini --venv /opt/stack/data/venv | 19:19 |
| JayF | aha, makes sense | 19:19 |
| JayF | that means I should be able to completely close the loop on testing in those patches; score | 19:19 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Enable WSGI module entrypoint for Ironic https://review.opendev.org/c/openstack/ironic/+/927635 | 19:28 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: add pyproject.toml to support pip 23.1 https://review.opendev.org/c/openstack/ironic/+/927544 | 19:28 |
| JayF | that should change our configuration to use the new module I added | 19:28 |
| JayF | I'm going to see if it passes CI, but the more I think about it, the more I think the path should be ironic.api.wsgi:application, since we're launching an ironic api | 19:28 |
| JayF | but I'm mainly curious if it works, moving it around if it does is trivial | 19:28 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: [doc] Add instructions on making big fake-BM nodes https://review.opendev.org/c/openstack/ironic/+/927645 | 19:39 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: [doc] Add instructions on making big fake-BM nodes https://review.opendev.org/c/openstack/ironic/+/927645 | 19:49 |
| JayF | adamcarthur5: ^ would that have been useful for the issue we troubleshooted today | 19:49 |
| adamcarthur5 | Hmm... I think so? But it would be something that I would expect as a new-user I wouldn't need to mess with. | 19:50 |
| adamcarthur5 | So I doubt I would look at that documentation directly? | 19:50 |
| JayF | You should fix that habit at the end :D at least look for the doc, we have a lot of troubleshooting stuff in our docs | 20:00 |
| JayF | and the protip is to use your preferred git search (I use ripgrep) to find stuff in the docs | 20:01 |
| JayF | especially the contributor docs though, we've done a pretty good job of keeping those updates | 20:01 |
| JayF | *updated | 20:01 |
| TheJulia | So, regarding my recent 4k block device size pain, I've posted a change to at least begin to open a path. https://review.opendev.org/c/openstack/diskimage-builder/+/927550 I'm really wondering if we should add smarts/capabilities into ironic to enable forcing/asserting | 20:44 |
| TheJulia | for devices, and maybe massaging out, dunno | 20:44 |
| TheJulia | massaging out the stream of data/disk image on disk | 20:44 |
| TheJulia | That may not be realistic | 20:44 |
| TheJulia | I *guess* the other thing is how in the world would an operator discover their hardware *requires* 4k blocks ?!? | 20:44 |
| JayF | I know this isn't a good answer for users, but like, geeze we bend over backwards to keep backwards compat | 20:47 |
| JayF | it's really annoying when we get burned by some other technology not being :( | 20:48 |
| TheJulia | Truthfully, it boils down to "GPT starts at the second logical block" | 20:51 |
| TheJulia | and the byte size just being different | 20:51 |
| JayF | I'm just surprised that the tooling doesn't handle that | 20:54 |
| JayF | hmmm | 20:54 |
| TheJulia | how can it? | 20:54 |
| JayF | I guess whole disk images make that hilariously hard | 20:54 |
| JayF | TheJulia: I was in "just run parted [blah]" mindset, lol | 20:54 |
| TheJulia | heh, yeah | 20:54 |
| TheJulia | those tools get it based upon the underlying block device | 20:54 |
| JayF | is there a way to make a one-size-fits-all whole disk image these days? | 20:55 |
| TheJulia | nope, there is not | 20:55 |
| JayF | even if it means giving up a few k at the front of the disk | 20:55 |
| JayF | even UEFI only? even if we toss BIOS out? | 20:55 |
| TheJulia | I talked to one of the large operators and their answer is they just force everything to 512 bytes in their envioronment | 20:55 |
| JayF | this is absolutely miserable | 20:55 |
| TheJulia | nope, even with uefi | 20:55 |
| TheJulia | if you have a 4k block disk, firmware doesn't recognize the the partition table because it is looking at the wrong bytes on disk | 20:56 |
| TheJulia | We think of it as a software problem, why can't it just look at the 512 byte area, but that is not how the firmware is designed | 20:57 |
| TheJulia | nor the standard | 20:57 |
| TheJulia | Truthfully, bios is actually kind of better for this | 20:57 |
| JayF | well that's where my head was | 20:57 |
| TheJulia | but maybe not | 20:57 |
| JayF | hybrid-MBR style uefi | 20:57 |
| TheJulia | I don't remember the pointer location formatting style for bios booting | 20:57 |
| JayF | I think that was a microsoftism though? was that ever standardized? | 20:57 |
| TheJulia | if it is a block or bytes into disk | 20:58 |
| TheJulia | I think it has to be aligned, so it is likely block address which means it wouldn't work either | 20:58 |
| JayF | ah, that makes sense | 20:58 |
| TheJulia | we have the ability to build those sorts of hybrid uefi/bios images with dib, fwiw | 20:59 |
| JayF | so if you translated it (not saying this exists in the real world anywherre) | 20:59 |
| JayF | it implies we'd end up with a 4k MBR instead of a 512byte mbr | 20:59 |
| TheJulia | yup | 20:59 |
| TheJulia | no idea if 4k disks are even supportable with MBR style booting either, it might just work if the target is properly aligned | 21:00 |
| TheJulia | which it *should* be since I beleive O_DIRECT is used for that write | 21:00 |
| JayF | yeah that's what I'm saying, just mentally extending the model :) | 21:00 |
| TheJulia | so, fun thought, mathmatically, GPT is too big to have a 512 byte and a 4096 byte record structure on disk | 21:01 |
| TheJulia | or maybe antifun "oh noes!" | 21:01 |
| TheJulia | https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_entries_(LBA_2%E2%80%9333) | 21:02 |
| TheJulia | 2-33 *on* 512 bytes, its like 2-5 with 4k blocks | 21:03 |
| JayF | so basically some hardware vendor invented their own crap that's not even standardized | 21:03 |
| JayF | and we've been handed the pieces | 21:03 |
| TheJulia | eh, not really, just nobody cared about cross-compatability | 21:04 |
| * JayF cares :) | 21:04 | |
| JayF | nobody at $vendor cares :D | 21:04 |
| TheJulia | well, nobody cared in whatever working group cooked up 4k disks | 21:05 |
| JayF | if we find out whatever group it is, we should leave a bag of flaming hdd on their porch /s | 21:05 |
| TheJulia | magnesium case HDDs right? | 21:05 |
| TheJulia | There would likely need to also *not* be much alunimun if there is magnesium | 21:08 |
| JayF | I think the devil there is in the idiom or something like that :D | 21:10 |
| TheJulia | heh | 21:11 |
| TheJulia | oh, covering HDDs in rainbows?! | 21:11 |
| TheJulia | lots of rainbows! | 21:13 |
| JayF | Yeah, but you can't find the partition table at the end of the rainbow | 21:16 |
| TheJulia | True, sadly. | 21:17 |
| * TheJulia wonders about the hdparm --set-sector-size option.... | 21:20 | |
| JayF | block size only really impacts spinny disks, yah? | 21:23 |
| TheJulia | outstanding question though, how does one discover they need to use a different disk image... | 21:23 |
| * TheJulia suspects maybe we need to evaluate root_device and record it as a trait if it is 4k | 21:23 | |
| JayF | This is a place where my knowledge of old school hardware sometimes pollutes my knowledge about current one :) | 21:23 |
| JayF | well my thought was more, if we can find a universal way to flip the setting | 21:24 |
| JayF | default to flipping it | 21:24 |
| TheJulia | nah, flash/nvme also impacted | 21:24 |
| JayF | and if someone disables it, they can have the pieces | 21:24 |
| JayF | ack | 21:24 |
| TheJulia | because it is fundimentanlly the smallest chunk which can be read or written | 21:24 |
| JayF | so would be interesting to see if hdparm/nvme/etc gives us a route to disable (and if in the real world it works) | 21:24 |
| TheJulia | NobodyCam mentioned he has something in his downstream hardware manager, I wonder if it is hdparm or not | 21:25 |
| TheJulia | I'll try to ask him later, I think I'm going to call it a day soon and go to the post office | 21:27 |
| JayF | same, I want to leave some hours so I can see if my nova patch gets a fresh review from non-USians on Monday | 21:27 |
| JayF | in fact... o/ | 21:27 |
| TheJulia | have a great weekend! | 21:27 |
| JayF | have a good long weekend! | 21:27 |
| TheJulia | :) | 21:28 |
| cardoe | So does that mean it can be bad idea o’clock? OCI format for images. With ability to package one up for multiple platforms and multiple block sizes. You download only the correct one. | 23:53 |
| * TheJulia glances at whiskey | 23:54 | |
| TheJulia | That actually might not be a bad idea, tbh | 23:54 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!