| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent-builder master: Update pip version in dib source install https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/949974 | 00:02 |
|---|---|---|
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent-builder master: Build CS9 DIB IPA ramdisk with python 3.12 https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/950152 | 00:06 |
| opendevreview | Merged openstack/ironic master: Fix agent get_XXX_steps retries from being treated as not fresh agents https://review.opendev.org/c/openstack/ironic/+/949785 | 01:29 |
| opendevreview | Verification of a change to openstack/ironic master failed: Allow to unprovision instance from service wait states https://review.opendev.org/c/openstack/ironic/+/948301 | 02:44 |
| rpittau | good morning ironic! o/ | 07:05 |
| rpittau | so we're just removing tinyipa testing | 07:06 |
| rpittau | I'll stop my work on compiling python3.12 on tinycore then :) | 07:06 |
| opendevreview | Merged openstack/ironic-python-agent-builder master: Remove tinyipa CI testing https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/950235 | 08:04 |
| opendevreview | Verification of a change to openstack/ironic master failed: CI: Reconfigure jobs to minimize tinyipa usage https://review.opendev.org/c/openstack/ironic/+/950192 | 08:26 |
| opendevreview | Verification of a change to openstack/ironic master failed: CI: Remove the multinode grenade job https://review.opendev.org/c/openstack/ironic/+/950224 | 08:26 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: [WIP] Run metal3 integration job using UEFI boot (default) https://review.opendev.org/c/openstack/ironic/+/939694 | 08:29 |
| abongale | Good morning Ironic! | 08:51 |
| opendevreview | Pierre Riteau proposed openstack/bifrost master: Add support for downloading CentOS Stream 10 image https://review.opendev.org/c/openstack/bifrost/+/950286 | 08:55 |
| opendevreview | Verification of a change to openstack/ironic master failed: Allow to unprovision instance from service wait states https://review.opendev.org/c/openstack/ironic/+/948301 | 11:43 |
| opendevreview | Pierre Riteau proposed openstack/networking-generic-switch stable/2025.1: Correctly set use_keys when key_file is set https://review.opendev.org/c/openstack/networking-generic-switch/+/950297 | 11:54 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-python-agent-builder master: Build CS9 DIB IPA ramdisk with python 3.12 https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/950152 | 12:03 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-python-agent-builder master: Build CS9 DIB IPA ramdisk with python 3.12 https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/950152 | 12:03 |
| opendevreview | Verification of a change to openstack/ironic master failed: CI: Reconfigure jobs to minimize tinyipa usage https://review.opendev.org/c/openstack/ironic/+/950192 | 12:16 |
| opendevreview | Verification of a change to openstack/ironic master failed: Fix redfish driver URL parsing https://review.opendev.org/c/openstack/ironic/+/949597 | 12:55 |
| TheJulia | rpittau: yeah, why go through the hassle at this point | 12:57 |
| TheJulia | Also, good morning folks | 12:57 |
| rpittau | TheJulia: good morning! :) | 12:57 |
| rpittau | sounds good to me | 12:57 |
| * TheJulia feels very old today after doing a bunch of landscaping work over the weekend | 12:58 | |
| opendevreview | Julia Kreger proposed openstack/ironic master: CI: Reconfigure jobs to minimize tinyipa usage https://review.opendev.org/c/openstack/ironic/+/950192 | 13:03 |
| Sandzwerg[m] | <TheJulia> "Sandzwerg: my theory is that..." <- You were absolutely correct. It was a MBR and not a GPT partition layout. Coworker fixed it and it worked directly. It's a bit puzzling that it worked before. Thank you very much for your help | 13:10 |
| TheJulia | Sandzwerg[m]: that is awesome news | 13:24 |
| opendevreview | Julia Kreger proposed openstack/ironic stable/2025.1: Fix agent get_XXX_steps retries from being treated as not fresh agents https://review.opendev.org/c/openstack/ironic/+/950308 | 13:26 |
| opendevreview | Julia Kreger proposed openstack/ironic stable/2024.2: Fix agent get_XXX_steps retries from being treated as not fresh agents https://review.opendev.org/c/openstack/ironic/+/950309 | 13:27 |
| opendevreview | Julia Kreger proposed openstack/ironic stable/2024.1: Fix agent get_XXX_steps retries from being treated as not fresh agents https://review.opendev.org/c/openstack/ironic/+/950310 | 13:28 |
| Sandzwerg[m] | <TheJulia> "Sandzwerg: that is awesome news" <- Yes indeed. The only question I'd have left is, would it make sense to make that issue more clear? So is it possible to detect the MBR and bubble that up as invalid? Or is that something that shouldn't happen anyways so no exceptions needed | 13:34 |
| TheJulia | It likely would, if you wouldn't mind creating a bug in launchpad that would be good and hopefully someone can pick it up soon | 13:41 |
| opendevreview | Julia Kreger proposed openstack/ironic master: CI: Remove the multinode grenade job https://review.opendev.org/c/openstack/ironic/+/950224 | 14:09 |
| JayF | oh no :( | 14:09 |
| JayF | someone on list is having the utf8mb3 vs mb4 issue | 14:10 |
| JayF | like we had to deal with | 14:10 |
| JayF | except in neutron | 14:10 |
| * JayF tries to unearth that patchset and associated bug | 14:10 | |
| TheJulia | ugh | 14:21 |
| TheJulia | looks like the multinode job is close to being tinyipa free, but networking is not working and we're going to have to figure out a way to connect the nodes with routed traffic at this point, or maybe bridged, Also oddly... they show in a down state when they are up. | 14:22 |
| JayF | MTU issues? | 14:27 |
| JayF | Or just who the hell knows issues :| | 14:27 |
| JayF | I would greatly appreciate reviews on https://review.opendev.org/c/openstack/ironic-specs/+/945642 -- I am trying to arrange resources around this and eventlet migration, and getting at least an indicator of core buy-in from the work items as listed would be a huge, huge help | 14:33 |
| TheJulia | JayF: Well, definitely a bit of addressing issues | 14:40 |
| TheJulia | It might all be addressing actually since the bridge shows tx/rx on each, the issue more so seems to be non-unique addressing | 14:42 |
| TheJulia | since each conductor process is self-referring | 14:42 |
| JayF | meeting in 5 jfyi, I will run it if nobody else wants to :) | 14:54 |
| rpittau | JayF: please go ahead :) | 14:58 |
| JayF | #startmeeting Ironic | 15:00 |
| opendevmeet | Meeting started Mon May 19 15:00:13 2025 UTC and is due to finish in 60 minutes. The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'ironic' | 15:00 |
| dtantsur | o/ | 15:00 |
| iurygregory | o/ | 15:00 |
| JayF | Welcome to the weekly Ironic meeting, I'm Jay, I'll be your host today :D | 15:00 |
| kubajj | o/ | 15:00 |
| rpittau | o/ | 15:00 |
| frickler | \o | 15:00 |
| cid | o/ | 15:00 |
| JayF | As always our meetings are operated under the OpenInfra Code of conduct | 15:00 |
| JayF | #note Standing reminder to review patches tagged ironic-week-prio and to hashtag any patches ready for review with ironic-week-prio: https://tinyurl.com/ironic-weekly-prio-dash | 15:00 |
| JayF | #note It's R-19 week! 2025.2 Flamingo Release Schedule https://releases.openstack.org/flamingo/schedule.html | 15:01 |
| JayF | Honestly this is usually the point where I'd say we'll wait for quorum | 15:01 |
| JayF | but I think we're already loaded in :D | 15:01 |
| JayF | #topic Working Group Updates: Standalone Networking | 15:01 |
| masghar | (o/) | 15:01 |
| JayF | Any update on Standalone Networking work? | 15:01 |
| dtantsur | not from me. alegacy? | 15:01 |
| JayF | I'm going to assume there's nothing to update; we can revisit if folks come around later. | 15:03 |
| JayF | #topic Discussion Topics: Phasing off Python 3.9 in Tests | 15:03 |
| JayF | There are several related PRs linked on this topic | 15:03 |
| JayF | #link https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/950152 use Python 3.12 in DIB CS9 image | 15:03 |
| JayF | #link https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/950235 remove tinycore tests (does not support Python > 3.9) | 15:03 |
| JayF | #link https://review.opendev.org/c/openstack/bifrost/+/949861 pinning UCs in bifrost CS9 jobs | 15:04 |
| rpittau | yeah they just represent what's currently happening | 15:04 |
| rpittau | python 3.9 support in UCs is not going to last for long | 15:04 |
| JayF | It seems like, at least to me, there's general consensus we're likely better off removing our need for a radically tiny ramdisk than trying to chase tinyipa or a replacement forever | 15:04 |
| JayF | Now would be a great time to make noise if you don't agree :) | 15:04 |
| rpittau | it is really a huge effort to use anything higher than pythgon 3. 9 in tinycore at the moment | 15:04 |
| JayF | My research into using gentoo could get us a smaller one, but not smaller enough to prevent our need to rework the tests | 15:05 |
| dtantsur | Is there any official position by the TCL team? | 15:05 |
| rpittau | meaning that you need to compile it yourself or use a tool like pyenv, which is still a big effort due to the chroot spaghetti stuff | 15:05 |
| JayF | https://forum.tinycorelinux.net/index.php/board,31.0.html they appear to release approximately once a year | 15:06 |
| JayF | and the most recent update was 4/26/2025 with no reference to python | 15:06 |
| rpittau | yep | 15:06 |
| JayF | I'll note I also don't know tinycore well enough to know if that is separate | 15:06 |
| rpittau | fun fact: python 3.12 for tinycore exists only for ARM | 15:06 |
| JayF | we can run tests under arm :P | 15:06 |
| rpittau | me shrugs | 15:06 |
| JayF | I prefer the approach of getting multinode working and trying to rework our tests to use realistic ramdisks | 15:06 |
| JayF | it moves us closer to what our customers use | 15:07 |
| rpittau | sounds great to me | 15:07 |
| JayF | gets us into the business of making our software/CI better rather than chasing a distro which I think none of us love :) | 15:07 |
| rpittau | I used to love it, now it's yet another Stockholm syndrome I have :D | 15:07 |
| JayF | It's (seemingly) one of those things that's really cool as a 1 time project; really painful to maintain for years | 15:08 |
| rpittau | I agree | 15:08 |
| JayF | So is there anything else to talk about around the discussion topic of python 3.9? I'll give a couple minutes | 15:08 |
| JayF | Aight, next topic | 15:11 |
| JayF | #topic Bug Deputy Updates | 15:11 |
| JayF | 1 new bug: https://bugs.launchpad.net/ironic/+bug/2110916 | 15:11 |
| iurygregory | o/ | 15:11 |
| JayF | 1 new RFE: https://bugs.launchpad.net/ironic/+bug/2110694 | 15:11 |
| JayF | 3 bugs closed, 3 triaged | 15:11 |
| JayF | Whoever was the bug deputy feel free to speak up with more detail :) | 15:11 |
| iurygregory | it was me | 15:11 |
| JayF | Also we need a bug deputy for next week | 15:11 |
| JayF | \p | 15:11 |
| iurygregory | so last wed Michael Sherman (not sure about his irc nick) created this rfe | 15:12 |
| JayF | The RFE seems sensible, I just wanna make sure it's unique :D | 15:12 |
| JayF | Yeah the real hard part of the RFE is around handing credentials and uploading the image to glance | 15:13 |
| iurygregory | based on the details, seems like we retired, so maybe he wants to work on it? | 15:13 |
| iurygregory | wondering how we should reach out to him about it | 15:13 |
| JayF | I am +1 to the feature existing, a little nervous about someone implementing it specless | 15:13 |
| iurygregory | agree | 15:13 |
| iurygregory | I can add the needs-spec to it | 15:14 |
| iurygregory | and mention we briefly talked at the weekly meeting | 15:14 |
| dtantsur | Yeah, especially since there is a prior art already | 15:14 |
| TheJulia | o/ | 15:15 |
| TheJulia | sorry, got super distracted this morning | 15:15 |
| iurygregory | np TheJulia o/ | 15:15 |
| JayF | Sounds like needs-spec is the answer, and I think he was already down that path. | 15:15 |
| JayF | \o | 15:15 |
| frickler | nick should be shermanm[m] | 15:15 |
| JayF | Do we have someone who wants to bug deputy this week? | 15:16 |
| TheJulia | I can watch the bugs this week | 15:16 |
| JayF | I'll note for US-ians: many of us will have Monday off next week (memorial day). So take this into consideration when volunteering | 15:16 |
| iurygregory | frickler, ack | 15:16 |
| JayF | Skipping RFE review, we just kinda did that in the bug deputy section | 15:16 |
| JayF | #note Next bug deputy: Julia | 15:16 |
| JayF | #topic Open Discussion | 15:16 |
| JayF | re: next week's Monday being Memorial Day in the US, would we like to keep the meeting? | 15:16 |
| JayF | I will not be here. | 15:17 |
| TheJulia | I should not be here next monday :) | 15:17 |
| TheJulia | I may take Friday off as well, who knows! | 15:17 |
| JayF | I'd be +1 to just cancelilng next week | 15:17 |
| rpittau | fine to me | 15:18 |
| dtantsur | I'll be here but I'm fine with cancelling | 15:18 |
| iurygregory | ++ | 15:18 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: CI Science - Expand the multinode job https://review.opendev.org/c/openstack/ironic/+/950206 | 15:19 |
| iurygregory | JayF, you want to send an email or should I? | 15:19 |
| JayF | #note Meeting next week cancelled due to US Memorial Day holiday. Next meeting June 2 (if my math is right :D) | 15:19 |
| cid | cancel, ++ | 15:19 |
| JayF | iurygregory: I'll do it | 15:19 |
| iurygregory | ack | 15:19 |
| JayF | Anything else for Open Discussion? | 15:19 |
| rpittau | CI related | 15:20 |
| rpittau | something odd with metal3 jobs | 15:20 |
| rpittau | not sure if anyeone else noticed but there are auth issues with sushy | 15:20 |
| rpittau | like 2025-05-19 13:20:52.997 1 WARNING sushy.connector [None req-f7d87eb6-999d-4248-a780-1a044a68911a - - - - - -] Session authentication appears to have been lost at some point in time. Connectivity may have been lost during a prior session refresh. Attempting to re-authenticate.: sushy.exceptions.AccessError: HTTP POST https://192.168.111.1:8000/redfish/v1/SessionService/Sessions returned code 401. Base.1.0.GeneralError: | 15:21 |
| rpittau | Authorization required Extended information: [{'@odata.type': '/redfish/v1/$metadata#Message.1.0.0.Message', 'MessageId': 'Base.1.0.GeneralError'}] | 15:21 |
| rpittau | please check the other jobs as well | 15:21 |
| rpittau | just want to exclude it's Python 3.12 only | 15:21 |
| rpittau | if anyone wants to have a look see the ironic logs here https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3ed/openstack/3ed3085139af4072b86cd6c990706577/controller/before_pivoting/ironic.log | 15:22 |
| TheJulia | That is a super weird edge to fall into to get that error | 15:23 |
| TheJulia | :( | 15:23 |
| rpittau | yeah | 15:23 |
| rpittau | I don't get it | 15:23 |
| rpittau | it was working until yesterday? | 15:23 |
| JayF | so we should check for requirements/constraints changes? | 15:23 |
| JayF | https://opendev.org/openstack/requirements/commits/branch/master looks like a lot of bumps for internal libraries two days ago | 15:24 |
| JayF | inc keystoneauth1/keystonemiddleware | 15:24 |
| JayF | might be a place to dig from, if you could lock to the u-c from three days ago and see if it repros | 15:24 |
| rpittau | latest uc updated sushy to 5.6.0 | 15:25 |
| TheJulia | well, if memory serves the way to get into that case is you think you've authenticated, but in the process you lost connectivity *or* the remote side never confirmed your session | 15:25 |
| TheJulia | so so when you try to use what you believe is valid, then you error and you need to reauthenticate, the code *should* be re-authenticating | 15:26 |
| rpittau | sushytools logs look very weird | 15:27 |
| rpittau | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3ed/openstack/3ed3085139af4072b86cd6c990706577/controller/before_pivoting/sushy-tools.log | 15:27 |
| TheJulia | maybe time to add more debug logging to sushy-tools ? I'm guessing that is all re-auth requests but why are we getting 401s being sent back | 15:28 |
| TheJulia | that seems like a bug | 15:28 |
| rpittau | yeah | 15:28 |
| rpittau | I have to drop in like 10 minutes, I can get another look tomorrow if no one gets to it before | 15:29 |
| JayF | May I suggest we continue this troubleshooting async/outside of meeting thne? | 15:29 |
| iurygregory | ++ | 15:30 |
| rpittau | yep | 15:30 |
| TheJulia | yeah, wrap the the meeting :) | 15:30 |
| JayF | #endmeeting | 15:30 |
| opendevmeet | Meeting ended Mon May 19 15:30:58 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:30 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/ironic/2025/ironic.2025-05-19-15.00.html | 15:30 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/ironic/2025/ironic.2025-05-19-15.00.txt | 15:30 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/ironic/2025/ironic.2025-05-19-15.00.log.html | 15:30 |
| JayF | Thanks o/ | 15:31 |
| rpittau | thank you! | 15:31 |
| TheJulia | rpittau: is metal3 failing on every run now? | 15:33 |
| rpittau | I'm not sure, most likely | 15:34 |
| rpittau | I was going to suggest to put it non voting during the troubleshooting, just to be sure | 15:34 |
| TheJulia | I think that likely makes sense | 15:34 |
| TheJulia | I'll look at another redfish job to see if we're seeing any similar behavior | 15:34 |
| TheJulia | hmm, nothing in one of the standalone jobs although we also don't capture the sushy-tools runtime logs | 15:37 |
| TheJulia | at least, to its own file | 15:37 |
| rpittau | let me add the non-voting patch up before I disappear :) | 15:38 |
| TheJulia | cool cool, thanks! | 15:39 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: [CI] metal3 non voting for investigation https://review.opendev.org/c/openstack/ironic/+/950332 | 15:40 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: metal3 job back to voting https://review.opendev.org/c/openstack/ironic/+/950333 | 15:41 |
| rpittau | and the revert ^ | 15:41 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: [CI] metal3 job back to voting https://review.opendev.org/c/openstack/ironic/+/950333 | 15:41 |
| rpittau | btw this is logs of sushy-tools in a passing job https://7e4ed52e60c14914c16f-0507f0d56ad72621551127593d6d8a94.ssl.cf1.rackcdn.com/openstack/8d57878811b3421aa1c918d2368de6a0/controller/before_pivoting/sushy-tools.log | 15:42 |
| TheJulia | oh, i bet this is something with the libvirt python bindings | 15:46 |
| TheJulia | I bet sushy-tools is crashing out horribly | 15:46 |
| opendevreview | Julia Kreger proposed openstack/sushy-tools master: DNM: Test commit https://review.opendev.org/c/openstack/sushy-tools/+/950334 | 15:47 |
| rpittau | it does not look to be Python 3.12 as the passing job also has that | 15:50 |
| TheJulia | Doesn't the metal3 job run on centos? | 15:52 |
| rpittau | TheJulia: nope, on ubuntu | 15:52 |
| rpittau | the ironic image is CentOS though | 15:52 |
| TheJulia | hmm | 15:52 |
| TheJulia | so that should include sushy-tools as the hypervisor | 15:52 |
| rpittau | so the difference I see here seems to be the sushy version | 15:53 |
| rpittau | and I see this in the failing job | 15:53 |
| rpittau | ERROR ironic.conductor.verify [None req-f7d87eb6-999d-4248-a780-1a044a68911a - - - - - -] Failed to get power state for node d9671975-c368-46c8-88d6-dfe8843752aa. Error: maximum recursion depth exceeded: RecursionError: maximum recursion depth exceeded | 15:53 |
| dtantsur | oh, this rings bells | 15:53 |
| TheJulia | Re-authentication is failing hardcore | 15:53 |
| TheJulia | since sushy-tools is also returning 401s | 15:53 |
| rpittau | http://pastebin.com/YtsQZZWd | 15:54 |
| TheJulia | one thing to note, in the passing job, for each request we see libvirt driver logging | 15:54 |
| dtantsur | https://bugs.launchpad.net/sushy/+bug/2101868 | 15:54 |
| TheJulia | in the log you pasted a link to, we don't see that, we just see it go direct to a 401 | 15:54 |
| rpittau | dtantsur: there you go :) | 15:54 |
| rpittau | that's exactly what's happening | 15:54 |
| TheJulia | sure looks like it | 15:55 |
| rpittau | that's in the new sushy version I think | 15:55 |
| rpittau | ok maybe not only there | 15:57 |
| rpittau | yeah version is the same both passing and failing jobs | 15:58 |
| dtantsur | https://192.168.111.1:8000/redfish/v1/SessionService/Sessions seems to be the correct URL though | 16:00 |
| dtantsur | so it's triggered by something else? | 16:00 |
| TheJulia | The immediate 401 is rather suspect to me | 16:00 |
| TheJulia | at least in the sushy-tools logging | 16:00 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: CI Science - Expand the multinode job https://review.opendev.org/c/openstack/ironic/+/950206 | 16:00 |
| dtantsur | Could not discover the Session service path, falling back to /redfish/v1/SessionService/Sessions.: sushy.exceptions.MissingAttributeError: The attribute Links/Sessions/@data.id is missing from the resource /redfish/v1 | 16:01 |
| dtantsur | Received representation of Sushy /redfish/v1: None | 16:02 |
| dtantsur | it shouldn't be None for sure | 16:02 |
| rpittau | mmm that's teh same as in the good job though | 16:02 |
| TheJulia | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3ed/openstack/3ed3085139af4072b86cd6c990706577/controller/before_pivoting/sushy-tools.log (not working) vs (working) https://7e4ed52e60c14914c16f-0507f0d56ad72621551127593d6d8a94.ssl.cf1.rackcdn.com/openstack/8d57878811b3421aa1c918d2368de6a0/controller/before_pivoting/sushy-tools.log | 16:03 |
| dtantsur | I hope that /redfish/v1 vs /redfish/v1/ is not a critical difference.. | 16:04 |
| JayF | We finally reached the point where sushy-tools can emulate the quirkiness of real hardware /s | 16:04 |
| rpittau | mmm ok so | 16:05 |
| rpittau | the difference is that in the failing job we're not falling back to Basic auth | 16:05 |
| dtantsur | JayF: truth | 16:05 |
| rpittau | for the BMC auth | 16:05 |
| rpittau | working -> Falling back to "Basic" authentication as we have been unable to authenticate to the BMC. | 16:05 |
| TheJulia | JayF: oh.. my. | 16:06 |
| dtantsur | no changes to sushy, no changes to sushy-tools.. dunno, new version of requests? | 16:06 |
| rpittau | mmm no seems the same | 16:08 |
| Continuity | JayF: is that a good thing.... :D | 16:09 |
| TheJulia | That being said, I'll -2 any change to get sushy-tools to emulate an idrac10 | 16:10 |
| TheJulia | well, okay, -1 requesting high bandwidth discussion | 16:10 |
| rpittau | lol | 16:11 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: CI Science - Expand the multinode job https://review.opendev.org/c/openstack/ironic/+/950206 | 16:14 |
| Continuity | So, noddy question time, I have need of being able to boot a specific baremetal node sometimes. Usually we just let the scheduler pick a node, but on the odd times I need to pick a specific one, does anyone have any idea how that can be accomplished? On a VM its pretty straight forward | 16:16 |
| rpittau | gaaaa metal3 worked today | 16:16 |
| rpittau | https://zuul.opendev.org/t/openstack/build/20300fb75ca3493da8889aea43fb8adc | 16:16 |
| rpittau | I smell a fishy sneaky bug in sushy/sushy-tools | 16:17 |
| JayF | Continuity: it's a reflection of how if 47 people read the same spec, they'll have 48 ideas on how to implement it (who knows where that bonus one came from :D) | 16:18 |
| Continuity | https://xkcd.com/927/ | 16:19 |
| Continuity | :D | 16:19 |
| dtantsur | Continuity: I think node UUID is hypervisor ID so you can filter on that | 16:20 |
| dtantsur | I haven't done any Nova for years though | 16:20 |
| Continuity | Yeah it seems to work on Nova for VMs not having much luck for BM though. | 16:20 |
| Continuity | Ill have another poke tomorrow | 16:20 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: CI Science - Expand the multinode job https://review.opendev.org/c/openstack/ironic/+/950206 | 16:28 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: CI Science - Expand the multinode job https://review.opendev.org/c/openstack/ironic/+/950206 | 16:48 |
| TheJulia | hopefully this one will do it | 16:48 |
| TheJulia | Continuity: yeah, hypervisor UUIDs match baremetal uuids. Often it is flavor mismatching that causes issues, fwiw. | 16:48 |
| TheJulia | okay, tempest is running, time will tell | 17:17 |
| TheJulia | gah, failing. At least its a data point | 17:26 |
| opendevreview | Merged openstack/ironic-tempest-plugin master: Remove available nodes configuration https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/950238 | 17:37 |
| opendevreview | Merged openstack/ironic master: Inspection throws exception on CPU-less systems https://review.opendev.org/c/openstack/ironic/+/949090 | 17:38 |
| opendevreview | Merged openstack/ironic-tempest-plugin master: Adding basic microversion tests to CI https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/943086 | 17:50 |
| TheJulia | JayF: I took a look at https://review.opendev.org/c/openstack/ironic-specs/+/945642 - It looks good to me, I +1'ed it but I suspect your good at this point, basically all my comments are clarity/position data and I could easily change it to +2 if you are not planning ot revise in short order | 18:32 |
| JayF | as long as I'm not begging folks to +2 when the time comes, I'll revise the minor stuff :) | 18:33 |
| JayF | as I mentioned to cid, I may not be the person implementing this so I wanna include as much context and clarity as possible | 18:33 |
| * cid đź‘€ | 18:34 | |
| JayF | I really need some shorthand to reference you without pinging you :D | 18:34 |
| cid | Not at all :D | 18:35 |
| TheJulia | heh, I chimed in on a few of those items | 18:35 |
| cid | I just heard my name and hopped in :) | 18:35 |
| JayF | Now I just think you are Joe Hendry :P ( https://www.youtube.com/shorts/zu-FBbaQLmk - a pro wrestling reference) | 18:38 |
| * TheJulia is sooooooo confused and just goes back to things and stuff | 18:43 | |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: CI Science - Expand the multinode job https://review.opendev.org/c/openstack/ironic/+/950206 | 18:43 |
| cid | :D | 18:54 |
| TheJulia | JayF: if we want to reduce the tempest runs which report the length is too long, then https://review.opendev.org/c/openstack/ironic/+/942355 would be a good starting point | 18:55 |
| opendevreview | Merged openstack/ironic master: [CI] metal3 non voting for investigation https://review.opendev.org/c/openstack/ironic/+/950332 | 19:04 |
| Continuity | TheJulia: Thanks, I managed to get it to work, appreciate it | 19:57 |
| opendevreview | Merged openstack/ironic master: CI: cleanup outdated rbac scope enforce jobs https://review.opendev.org/c/openstack/ironic/+/949912 | 20:11 |
| opendevreview | Merged openstack/ironic master: ci: set a longer nova tempest build interval https://review.opendev.org/c/openstack/ironic/+/942355 | 20:11 |
| opendevreview | cid proposed openstack/ironic master: doc: Make port binding failure configurably fatal https://review.opendev.org/c/openstack/ironic/+/950363 | 20:36 |
| iurygregory | https://github.com/dell/iDRAC-Redfish-Scripting/issues/324 | 20:41 |
| JayF | Nice | 20:42 |
| TheJulia | iurygregory: I wonder if there is a real internal engineering partner manager we can reach out to... | 20:44 |
| iurygregory | TheJulia, same =( | 20:44 |
| iurygregory | now I'm into hacking sushy to try to make it work =D | 20:44 |
| iurygregory | weird thing is that ironic doesn't give any error, just the one about cleaning timeout.. | 20:45 |
| TheJulia | see,s ;ole we | 20:46 |
| TheJulia | blah | 20:46 |
| JayF | if nothing else, it's a bug that we don't notice and fail | 20:46 |
| JayF | tell the kitty I say hello Julia :P | 20:46 |
| TheJulia | seems like we need to (a) surface the failure then (b) figure out if we can have a semi-compliant way to go forward | 20:46 |
| TheJulia | JayF: heh, nah, was me being on the wrong keys without looking | 20:47 |
| iurygregory | yup, agree 100% | 20:47 |
| iurygregory | rebuilding my bifrost to use master, since we merged sushy-oem-idrac into sushy | 20:48 |
| TheJulia | stevebaker[m]: https://review.opendev.org/c/openstack/networking-generic-switch/+/949834 looks good code wise, but I'm struggling to sort of wrap my head around the registrations. If you could add more ocntext that would be awesome | 20:49 |
| stevebaker[m] | TheJulia: Thanks, I'll add some comments to the subscribe function. I've replied to the comment about config | 21:02 |
| opendevreview | Julia Kreger proposed openstack/ironic master: CI: Reconfigure jobs to minimize tinyipa usage https://review.opendev.org/c/openstack/ironic/+/950192 | 21:09 |
| opendevreview | Julia Kreger proposed openstack/ironic master: CI: Remove the multinode grenade job https://review.opendev.org/c/openstack/ironic/+/950224 | 21:09 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: CI Science - Expand the multinode job https://review.opendev.org/c/openstack/ironic/+/950206 | 21:09 |
| TheJulia | stevebaker[m]: ack, thanks | 21:09 |
| opendevreview | Julia Kreger proposed openstack/ironic master: WIP Patch configdrive metadata https://review.opendev.org/c/openstack/ironic/+/946677 | 22:00 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: PTG: Move CI job descriptions into yaml https://review.opendev.org/c/openstack/ironic/+/946584 | 22:06 |
| Sandzwerg[m] | <Continuity> "So, noddy question time, I..." <- `openstack server create ... --availability-zone ::<node-id> test-instance-on-node-SOMETHING´ should work. You might need to state the AVZ itself as well (I usually do so I'm not 100% sure if it's okay to omit it) | 22:34 |
| Sandzwerg[m] | ah to late | 22:35 |
| iurygregory | TheJulia, JayF https://github.com/dell/iDRAC-Redfish-Scripting/issues/324#issuecomment-2892410726 | 22:38 |
| iurygregory | I'm surprised they said we need to use a specific device for VirtualMedia.. | 22:40 |
| TheJulia | So… I guess the days of caring about dmtf standard are no so now? | 22:40 |
| TheJulia | Sandzwerg[m]: it happens :) | 22:40 |
| JayF | iurygregory: I replied | 22:42 |
| iurygregory | and ofc https://paste.opendev.org/show/bl92mF2hr5DxEMGViDw0/ | 22:43 |
| iurygregory | both devices have CD/DVD, so why I can't use number 2?! | 22:43 |
| iurygregory | TheJulia, yeah =) | 22:43 |
| JayF | iurygregory: you can reply to that point, I wanted my voice in that issue for my own reasons :) | 22:43 |
| iurygregory | yeah will mention this, since he said we need to use a specific device... | 22:44 |
| TheJulia | how do we know the right specific device #question_of_the_day | 22:46 |
| JayF | When I've done evaluation and talked to vendors, I've not even really entertained suggestions that we should use device X and it works | 22:46 |
| JayF | if you give me a device, and don't tell me in the schema not to use it, it better work | 22:47 |
| TheJulia | wut is going on with their etags?! | 22:47 |
| iurygregory | ironic tries the first one it sees, and it doesn't fail | 22:49 |
| iurygregory | but according to them "you need device 1", I'm going to hardcode the path in my setup to test after dinne for sure | 22:50 |
| iurygregory | I would expect the BMC saying " ERROR bla bla bla, you cant attach iso here" | 22:50 |
| iurygregory | Am I a bad person for asking if they have something in their documentation about it? :X | 22:51 |
| TheJulia | @iurygregory your a very good person doing so | 22:55 |
| JayF | iurygregory: it's in everyone's best interest (including Dell; whether they realize it or not), that these are as well documented and work as "normal" as a redfish bmc can | 22:56 |
| JayF | iurygregory: thank you for bringing those things up, please do so very loudly and publicaly so inaction is as obvious as action :) | 22:56 |
| Sandzwerg[m] | Yes please keep on doing to good work, so I don't have to sit in meetings with vendors where they don't understand that their strange work-arounds don't work for us. | 23:00 |
| JayF | Sandzwerg[m]: only by sitting in those meetings and demanding better will we change the ecosystem | 23:01 |
| JayF | Sandzwerg[m]: to that end; if you're a Dell customer, I'd suggest letting your account manager know about that github issue; that's a nice start :D | 23:01 |
| iurygregory | JayF, yeah, my next step is asking on the redfish forum =) | 23:01 |
| JayF | Sandzwerg[m]: I'm not saying to be obnoxious, I'm saying to do enough to increment the "customer ask" field on their internal jira about API compatibility by 1 :D | 23:02 |
| * TheJulia wonders if mraineri might have opinions ;) | 23:03 | |
| Sandzwerg[m] | Oh yeah I'll for sure poke them about this. If I remember tomorrow when I'm back at my work setup, feel free to poke me. Need to create the bug then as well. | 23:06 |
| iurygregory | I can say that I'm not surprised https://paste.opendev.org/show/bYV0weY9bdTKbpj6f9so/ | 23:20 |
| iurygregory | 1- get one time boot setting 2- insert the media on device1 3- Set next one time boot for Vmedia 4- get one time boot setting again multiple times | 23:22 |
| iurygregory | I would expect to see something different of None for " | 23:22 |
| iurygregory | - Current next reboot, one time boot setting is: None" | 23:22 |
| iurygregory | I need coffee ... | 23:22 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Automated cleaning by runbook https://review.opendev.org/c/openstack/ironic/+/945259 | 23:27 |
| opendevreview | cid proposed openstack/ironic master: Fix IPA external inspection callback url override https://review.opendev.org/c/openstack/ironic/+/949521 | 23:56 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!