| opendevreview | Merged openstack/ironic master: CI: Make metal3 non-voting https://review.opendev.org/c/openstack/ironic/+/943347 | 00:06 |
|---|---|---|
| TheJulia | sweet | 00:07 |
| opendevreview | Julia Kreger proposed openstack/ironic master: network testing: hooking in an external network simulator https://review.opendev.org/c/openstack/ironic/+/942298 | 00:36 |
| opendevreview | Julia Kreger proposed openstack/ironic master: WIP: Add network simulator support for force10 os 10 https://review.opendev.org/c/openstack/ironic/+/943345 | 00:36 |
| opendevreview | Verification of a change to openstack/ironic master failed: Add systemd provider for console containers https://review.opendev.org/c/openstack/ironic/+/941614 | 00:54 |
| opendevreview | Verification of a change to openstack/ironic master failed: Implement drivers redfish-graphical, fake-graphical https://review.opendev.org/c/openstack/ironic/+/941615 | 00:56 |
| opendevreview | Verification of a change to openstack/ironic master failed: Add vnc-container image build https://review.opendev.org/c/openstack/ironic/+/942017 | 01:06 |
| opendevreview | Merged openstack/ironic master: Restore recompile of dnsmasq https://review.opendev.org/c/openstack/ironic/+/943339 | 03:40 |
| JayF | [the recheck fairy has visited several patches including rest of console + IPA + IPA-B fixes] | 04:05 |
| opendevreview | Verification of a change to openstack/ironic-python-agent master failed: Fix the way qemu-img is called with prlimits https://review.opendev.org/c/openstack/ironic-python-agent/+/942690 | 04:16 |
| opendevreview | Merged openstack/ironic master: Add systemd provider for console containers https://review.opendev.org/c/openstack/ironic/+/941614 | 04:22 |
| opendevreview | Merged openstack/ironic master: Implement drivers redfish-graphical, fake-graphical https://review.opendev.org/c/openstack/ironic/+/941615 | 04:22 |
| opendevreview | Verification of a change to openstack/ironic master failed: Trivial: Enable disabling tftp setup https://review.opendev.org/c/openstack/ironic/+/941742 | 04:30 |
| opendevreview | Merged openstack/ironic-python-agent-builder master: More reliable TinyIPA builds with network retries https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/942369 | 04:30 |
| opendevreview | Merged openstack/ironic master: Trivial: Enable disabling tftp setup https://review.opendev.org/c/openstack/ironic/+/941742 | 06:54 |
| opendevreview | Merged openstack/ironic master: Add vnc-container image build https://review.opendev.org/c/openstack/ironic/+/942017 | 07:09 |
| opendevreview | Merged openstack/ironic master: Implement graphical console read-only support https://review.opendev.org/c/openstack/ironic/+/942300 | 07:16 |
| opendevreview | Merged openstack/ironic-python-agent master: Lockout agent command results if a token is received https://review.opendev.org/c/openstack/ironic-python-agent/+/942010 | 07:33 |
| opendevreview | Merged openstack/ironic-python-agent master: Missing __init__.py file https://review.opendev.org/c/openstack/ironic-python-agent/+/943178 | 07:33 |
| rpittau | good morning ironic! o/ | 07:39 |
| mdfr | good morning | 08:29 |
| opendevreview | Merged openstack/ironic master: CI: Extend default timeouts slightly https://review.opendev.org/c/openstack/ironic/+/942633 | 08:33 |
| frickler | hello ironicians, I'm currently fighting with installing a node via redfish vmedia + raid, image was built with dib according to the docs. the install itself finishes fine, but then the boot fails into grub rescue and it is lacking the mdraid module. does that ring a bell for someone? | 08:48 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: [WIP] Run metal3 integration job using UEFI boot (default) https://review.opendev.org/c/openstack/ironic/+/939694 | 09:58 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Use bigger partition as work dir for metal3 job https://review.opendev.org/c/openstack/ironic/+/943374 | 10:38 |
| opendevreview | Merged openstack/ironic-python-agent-builder master: fix: Adds bsdextrautils package for Debian which provides hexdump command https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/943334 | 11:01 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: [WIP] Run metal3 integration job using UEFI boot (default) https://review.opendev.org/c/openstack/ironic/+/939694 | 11:03 |
| frickler | so my issue seems to be related to https://storyboard.openstack.org/#!/story/2009794 , but I'm not sure how efibootmgr works, should grub not even be installed on the image, then? | 11:35 |
| opendevreview | Merged openstack/ironic-python-agent master: Fix the way qemu-img is called with prlimits https://review.opendev.org/c/openstack/ironic-python-agent/+/942690 | 12:31 |
| dtantsur | \o/ | 12:32 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic-python-agent-builder stable/2024.2: More reliable TinyIPA builds with network retries https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/943380 | 12:32 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic-python-agent-builder stable/2024.1: More reliable TinyIPA builds with network retries https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/943381 | 12:33 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic-python-agent stable/2024.2: Fix the way qemu-img is called with prlimits https://review.opendev.org/c/openstack/ironic-python-agent/+/943382 | 12:34 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic-python-agent stable/2024.1: Fix the way qemu-img is called with prlimits https://review.opendev.org/c/openstack/ironic-python-agent/+/943383 | 12:35 |
| frickler | ok, looks like I was missing the grub2 element, which looks like it results in an incomplete grub installation | 12:56 |
| frickler | dtantsur: with that streak of things merging, could you also have another look at https://review.opendev.org/c/openstack/ironic/+/942787 please? | 12:57 |
| dtantsur | sure, approved | 12:59 |
| rpittau | anything else to add to the cycle highlights? https://review.opendev.org/c/openstack/releases/+/943229 | 13:37 |
| keekz | how can i make a new version of https://opendev.org/openstack/ironic-python-agent-builder ? the most recent one in pip is 5.4.0 from august 2024 | 13:50 |
| rpittau | keekz: new release is coming soon | 13:50 |
| rpittau | I was actually preparing it now :) | 13:50 |
| keekz | oh cool, thanks :) | 13:53 |
| opendevreview | Kaifeng Wang proposed openstack/ironic master: Ironic doesn't update local_link_connection when changed https://review.opendev.org/c/openstack/ironic/+/943395 | 14:11 |
| opendevreview | Merged openstack/ironic master: Make floppy images more floppy https://review.opendev.org/c/openstack/ironic/+/942787 | 14:14 |
| * TheJulia yawns | 14:17 | |
| rpittau | TheJulia: good morning! | 14:21 |
| rpittau | I saw that metal3 integration job is now non-voting, it was kind of a surprise, my understanding was that the issue with the disk size was not a pressing one | 14:21 |
| rpittau | anyway! I added a patch that should work around that, please have a look when you have a moment https://review.opendev.org/c/openstack/ironic/+/943374 thanks! | 14:21 |
| rpittau | JayF, dtantsur, cardoe, iurygregory_ FYI ^ | 14:21 |
| TheJulia | rpittau: we found multiple jobs where it was out of space | 14:22 |
| TheJulia | well, examples | 14:22 |
| TheJulia | so, we we're left with few options to enable other CI stabilization patches to land. :( | 14:22 |
| dtantsur | The mid-term solution may be to switch to BMO end-to-end tests instead of the integration test | 14:23 |
| TheJulia | rpittau: another factor is also, JayF and I won't be around the next few days to help push stuff along, since we will both be at SCaLE | 14:24 |
| opendevreview | Dr. Jens Harbott proposed openstack/ironic stable/2024.2: Make floppy images more floppy https://review.opendev.org/c/openstack/ironic/+/943396 | 14:26 |
| opendevreview | Merged openstack/ironic-python-agent-builder master: Change to maximum compression level when using lzma https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/940602 | 14:41 |
| cardoe | rpittau: I'm a -1 until we can fix up the test overall cause we keep band aiding stuff and then it eats us. It needs to fast fail IMHO. | 15:04 |
| cardoe | Cause it ran out of disk space creating the virtual BMC VMs and then proceeded to wait 2400 seconds for the first node to be available and THEN waited another 2400 seconds for the second node to be available. | 15:05 |
| cardoe | Those waits don't happen in parallel so we're eating up CI resources for a minimum of an hour and 20 minutes when there's a failure. | 15:06 |
| cardoe | We've got a convo on the ML right now about demand for CI resources and projects needing to look at their jobs. | 15:07 |
| opendevreview | Merged openstack/ironic master: Enable trunk plugin for tinyipa-multinode https://review.opendev.org/c/openstack/ironic/+/941023 | 15:14 |
| rpittau | cardoe: what you mean by "fix the test overall" ? the only issue there is the disk space, the rest works just fine | 15:14 |
| JayF | Yeah, I don't think it's the usual failure mode for it to take that long, just a weird failure mode because it ran out of space | 15:18 |
| rpittau | cardoe: btw that patch is not a band-aid, there are no band-aids in the metal3 integration job at the moment, we're actually not even running tests there, just verifying that the nodes are up and deployed | 15:18 |
| rpittau | which is the minimum necessary there | 15:19 |
| cardoe | JayF: I mean it's not the first time this year that we've talked about the metal3 job taking almost 3 hours to run when it's failed much earlier and it got marked as non-voting. | 15:20 |
| cardoe | or I should say this release cycle | 15:21 |
| JayF | Aiui, that was caused by a race condition in their release process. That's been fixed now. | 15:22 |
| rpittau | cardoe: show me where the metal3 job runs for almost 3 hours please :) | 15:22 |
| rpittau | in case of success the average runtime is 30-40 minutes, 50 at max | 15:22 |
| rpittau | in case of failure we're around 1h50 | 15:22 |
| JayF | I guess I'm not being that hard on the metal3 job because compared to our actual tempest jobs it's pretty stable | 15:22 |
| rpittau | at max, usually it fails in about 1h | 15:22 |
| cardoe | My point is integration tests aren't where you are doing detailed troubleshooting of a component. So if it fails on IPMI, just fast fail out. | 15:23 |
| cardoe | But I'll +1 once it hits a rackspace worker | 15:25 |
| opendevreview | Merged openstack/ironic-python-agent-builder master: Prevent service restart while in 'rescued' state https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/941610 | 15:37 |
| opendevreview | Verification of a change to openstack/ironic-python-agent stable/2024.2 failed: Fix the way qemu-img is called with prlimits https://review.opendev.org/c/openstack/ironic-python-agent/+/943382 | 15:58 |
| opendevreview | Julia Kreger proposed openstack/ironic master: WIP: Add network simulator support for force10 os 10 https://review.opendev.org/c/openstack/ironic/+/943345 | 16:00 |
| TheJulia | i'd appreciate another reviewer to take a look at https://review.opendev.org/c/openstack/networking-generic-switch/+/942160?usp=search | 16:08 |
| cardoe | https://www.irccloud.com/pastebin/oRV5ksgy/ | 16:16 |
| cardoe | https://review.opendev.org/c/openstack/ironic-python-agent/+/943382 | 16:16 |
| JayF | maybe3 something weird in error handling for the ipa-b change from cid? | 16:18 |
| JayF | but others succeeded so I'd assume this is just not the prettiest return for a network failure that couldn't retry out | 16:19 |
| * cid heh, looks like it's from that fix | 16:26 | |
| cid | cardoe: what log file is that from? | 16:27 |
| cardoe | https://zuul.opendev.org/t/openstack/build/85e6776052e847008a546a7d1fb29f9c right on that console | 16:28 |
| cid | Alright, tks | 16:29 |
| * cid found no logs though | 16:40 | |
| opendevreview | Satoshi Shirosaka proposed openstack/ironic-python-agent-builder master: WIP Test Podman for container-based cleaning https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/943348 | 16:57 |
| opendevreview | Satoshi Shirosaka proposed openstack/ironic-python-agent-builder master: WIP Test Podman for container-based cleaning https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/943348 | 17:03 |
| opendevreview | Merged openstack/ironic-python-agent master: Remove pre-victoria cycle agent token transition upgrade support https://review.opendev.org/c/openstack/ironic-python-agent/+/941565 | 17:05 |
| opendevreview | Merged openstack/ironic-python-agent stable/2024.1: Fix the way qemu-img is called with prlimits https://review.opendev.org/c/openstack/ironic-python-agent/+/943383 | 17:05 |
| opendevreview | Merged openstack/networking-generic-switch master: don't use paramiko's get_fingerprint (md5) https://review.opendev.org/c/openstack/networking-generic-switch/+/942160 | 18:59 |
| opendevreview | Vasyl Saienko proposed openstack/ironic master: Do not review debug multinode https://review.opendev.org/c/openstack/ironic/+/942725 | 19:32 |
| opendevreview | Vasyl Saienko proposed openstack/ironic master: Do not review debug multinode https://review.opendev.org/c/openstack/ironic/+/942725 | 19:35 |
| Sandzwerg[m] | <cardoe> "https://github.com/rackerlabs/..." <- Glad you got it working. Most of the work was done by https://github.com/killermoehre I mostly tried to keep it running afterwards. | 19:46 |
| opendevreview | Vasyl Saienko proposed openstack/ironic master: Do not review debug multinode https://review.opendev.org/c/openstack/ironic/+/942725 | 19:59 |
| opendevreview | Steve Baker proposed openstack/ironic master: Documentation for graphical consoles https://review.opendev.org/c/openstack/ironic/+/943122 | 20:10 |
| cardoe | networking-baremetal just makes sure that neutron is aware of the physical_network that each of the ports are on to help to binding correct? | 21:33 |
| cardoe | and portgroups | 21:34 |
| opendevreview | Satoshi Shirosaka proposed openstack/ironic-python-agent-builder master: WIP Test Podman for container-based cleaning https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/943348 | 22:20 |
| opendevreview | Satoshi Shirosaka proposed openstack/ironic-python-agent-builder master: WIP Test Podman for container-based cleaning https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/943348 | 23:32 |
| opendevreview | Satoshi Shirosaka proposed openstack/ironic-python-agent master: WIP Add ContainerHardwareManager https://review.opendev.org/c/openstack/ironic-python-agent/+/941714 | 23:32 |
| TheJulia | cardoe: well, there is some other stuff there, but in the broadest general terms, yeah | 23:51 |
| cardoe | yeah it talks to the switches to get link as well it seems like | 23:52 |
| cardoe | So having an issue where its registering agents in the neutron agent lists for nodes that I don't have. | 23:52 |
| TheJulia | nodes that it can see on a different account, perhaps? | 23:56 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!