| *** Kangie_ is now known as Kangie | 01:17 | |
| rpittau | good morning ironic! o/ | 08:22 |
|---|---|---|
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: Update default ansible version https://review.opendev.org/c/openstack/bifrost/+/903136 | 08:53 |
| dtantsur | JayF: I wonder at which point the health of the requests library becomes an even more serious issue than eventlet.. | 09:47 |
| drannou | Hello ironic, linked to dtantsur feedback, I made a simple usb devices inventory detection https://review.opendev.org/c/openstack/ironic-python-agent/+/902769 . Except I limited the discovery to vendor, product and handle, do you think that I should put more informations ? | 10:05 |
| dtantsur | It sounds like a good start to me. What is there more to discover? | 10:05 |
| dtantsur | drannou: left a review, just minor things | 10:26 |
| drannou | more to discover => we can store a lot of things, but not interesting in my POV | 10:30 |
| dtantsur | yeah, let's start from what you have | 10:31 |
| dtantsur | (I'm looking forward to learning what handle is :D) | 10:31 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic master: Test redfish with reduced sushy-tools feature set https://review.opendev.org/c/openstack/ironic/+/903361 | 10:39 |
| dtantsur | rpittau: morning! looks like bifrost dhcp jobs have been broken since https://review.opendev.org/c/openstack/bifrost/+/875750 although I have no clue how that would cause it.. | 10:59 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: DNM Revert "Support ansible-collections-openstack 2 and later" https://review.opendev.org/c/openstack/bifrost/+/903694 | 11:01 |
| dtantsur | trying in the gate ^^ | 11:01 |
| opendevreview | Merged openstack/ironic-python-agent master: docs: improve rootpwd password generation command https://review.opendev.org/c/openstack/ironic-python-agent/+/899380 | 11:27 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic-python-agent master: Reformat and update the section on injecting root credentials https://review.opendev.org/c/openstack/ironic-python-agent/+/903750 | 11:34 |
| rpittau | dtantsur: that's suspicious indeed, let's see how the revert goes | 11:36 |
| iurygregory | morning Ironic | 12:01 |
| opendevreview | Merged openstack/bifrost stable/2023.2: Fix key-order ansible errors https://review.opendev.org/c/openstack/bifrost/+/902039 | 12:41 |
| rpittau | dtantsur: I'm looking at the history of the dhcp jobs, are you sure that's the commit that broke the jobs? I see some success for about another week after the merge | 13:29 |
| dtantsur | rpittau: which dates are you looking at? | 13:29 |
| dtantsur | (I'm definitely not sure) | 13:29 |
| rpittau | I'm looking at master branch between end of february and beginning of march | 13:29 |
| rpittau | I see dhcjp jobs passing until march 8th | 13:30 |
| dtantsur | March 8th is when the patch merged | 13:30 |
| dtantsur | 08.03.2023 21:23 | 13:30 |
| rpittau | mmm yeah | 13:32 |
| rpittau | centos dhcp passed in the revert, but not jammy | 13:32 |
| dtantsur | some jobs fail because they run out of disk space | 13:33 |
| rpittau | OSError: [Errno 28] No space left on device | 13:33 |
| rpittau | yeah | 13:33 |
| dtantsur | yep | 13:33 |
| dtantsur | okay, so the migration to the new collection could be the cause... weird | 13:33 |
| dtantsur | can it be that they're not passing something? | 13:33 |
| rpittau | is it bigger? | 13:33 |
| dtantsur | I'm more worried about the other failure | 13:34 |
| dtantsur | IIRC nodes fail to boot from network, dnsmasq rejects their requests | 13:34 |
| rpittau | oh I see | 13:34 |
| rpittau | @dt | 13:49 |
| rpittau | ehm... | 13:49 |
| rpittau | dtantsur: from the logs I can't really see the difference, but I'm ok with the revert as it looks like dhcp works there | 13:51 |
| *** dking is now known as Guest10467 | 13:51 | |
| *** Guest10467 is now known as dking | 13:51 | |
| rpittau | at least on centos9, but jammy has the disk space issue | 13:52 |
| dking | Good morning. Does anybody know if there's anything in the cleaning steps which checks a device to see if it is a removable media, like a USB, before attempting to clean it? | 13:53 |
| dking | We had a server where somebody accidentally left a USB stick in it, and it failed on a custom clean step I have, but it looks like it was already cleaned before I got to it. So I'm assuming that we don't check that. I see checks for virtual media and other things, but I don't see anything for USB. | 13:56 |
| dtantsur | rpittau: I'm not a fan of proceeding with this revert. We'll get ourselves into other problems this way. | 14:01 |
| dtantsur | dking: yeah, we don't exclude any writeable devices by default. | 14:01 |
| dking | dtantsur: Thank you. should we? | 14:02 |
| rpittau | dtantsur: ok, then I think we should collect at least the dnsmasq config to see if we can find anything | 14:02 |
| dtantsur | dking: I'm not sure. Can we even reliably distinguish removeable devices? Probably a good topic for the next IRC meeting. | 14:02 |
| dking | dtantsur: Well, there is /sys/class/block/*/removable. If it's not just a simply crazy thought, maybe I'll bring it up at some point. | 14:05 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [DNM] collect dnsmasq config https://review.opendev.org/c/openstack/bifrost/+/903755 | 14:08 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: Collect lshw output in json format https://review.opendev.org/c/openstack/bifrost/+/890408 | 14:38 |
| TheJulia | good morning | 14:54 |
| dtantsur | morning TheJulia | 14:54 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-inspector master: [WIP] Handle LLDP parse Unicode error https://review.opendev.org/c/openstack/ironic-inspector/+/903760 | 14:56 |
| * TheJulia really wishes we had published arm artifacts | 15:03 | |
| dtantsur | TheJulia: we have IPA images, what are you looking for? | 15:08 |
| TheJulia | oooh, we do, hiding under ironic-python-agent-builder | 15:09 |
| dtantsur | yep. because they're kinda experimental (i.e. don't run in the CI) | 15:09 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [DNM] collect dnsmasq config https://review.opendev.org/c/openstack/bifrost/+/903755 | 16:04 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-inspector master: [WIP] Handle LLDP parse Unicode error https://review.opendev.org/c/openstack/ironic-inspector/+/903760 | 16:10 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-inspector master: [WIP] Handle LLDP parse Unicode error https://review.opendev.org/c/openstack/ironic-inspector/+/903760 | 16:12 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-inspector master: Handle LLDP parse Unicode error https://review.opendev.org/c/openstack/ironic-inspector/+/903760 | 16:20 |
| rpittau | bye everyone! have a great weekend! o/ | 16:55 |
| dtantsur | have a nice weekend folks | 16:59 |
| JayF | I'm taking some PTO for most of the day; I'm around for a bit right now if you need me but will be gone for the day in a bit. | 17:28 |
| JayF | dking: dtantsur: re: not wiping removable devices, that could be a useful feature for rpi support for Ironic, as well, since many "PXE network boot" implementations for rpi involve putting a network boot rom on a usb key | 17:29 |
| JayF | dking: dtantsur: although, as with all optional cleaning features, we'd want to default safe (clean) and opt-in to unsafe (not-clean) | 17:29 |
| dking | JayF: That sounds good to me. It might not hurt, though, to consider which is less surprising. Are people more likely to use a removable drive and wonder why it wasn't being cleaned, or are they more likely to be surprised when their USB drive is wiped by default. I would go with the feature with least surprise. | 17:48 |
| JayF | I think for the average Ironic use case it'd be more like | 17:48 |
| JayF | "Holy $@%#! someone plugged a removable storage device into my server!?!?" | 17:48 |
| JayF | :) | 17:49 |
| dking | I would tend to agree. I can't imagine that people are more likely to plug USB drives in and just expect them to get cleaned. | 17:49 |
| dking | For my local stuff, I'm skipping the cleaning but logging a warning. | 17:49 |
| JayF | So maybe it's a ternary option? | 17:50 |
| JayF | removable_device_action = [wipe,ignore,kaboom] and default to kaboom | 17:50 |
| JayF | so cleanfail if a removable device in, as a security guard | 17:50 |
| JayF | I will be -1 and borderline -2 to a change that just doesn't wipe a device hooked up because it's marked as removable | 17:50 |
| dking | I'm fine with that. I like the fail early, fail loudly principle. Sometimes, I log errors like "This thing isn't expected, so I'm failing. If you wanted it, do X" | 17:52 |
| JayF | We had a hardware manager back when I did OnMetal which basically, we had a piece of metadata in the node which dictated what sku it was | 17:53 |
| JayF | and we basically did an inventory check "this should have X ram sticks of Y capacity, these CPUs, these disks, etc etc" | 17:53 |
| JayF | and if any of it was missing/not what was expected, we would fail cleaning and address it | 17:53 |
| JayF | you find a lot of flakey hardware that way (I was surprised at the number of cases of "this ram stick shows up 5 out of 10 boots") | 17:54 |
| dking | Yeah, I think that's a good pattern. We aim for that also. When we get past some of the current milestones, I want to go back and add checks against the specs in NetBox. | 17:56 |
| opendevreview | Merged openstack/bifrost master: Update default ansible version https://review.opendev.org/c/openstack/bifrost/+/903136 | 21:29 |
| opendevreview | Iury Gregory Melo Ferreira proposed openstack/ironic master: RedfishFirmwareInterface - Unit Tests & More logs https://review.opendev.org/c/openstack/ironic/+/903379 | 23:04 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!