Wednesday, 2025-01-22

TheJuliaSo, as far as I'm aware, we never built any framework to get secret data into the agent00:30
TheJulia... what if we used the secret token as the key to enable secret data to be encrypted on the conductor side, which the agent can then decrypt as needed. Thoughts?00:31
TheJuliain a perfect world, the agent is running an https server00:34
TheJuliathat message then goes over https so it is doubly encrypted, my whole worry is logs and just making it harder to decode it out00:34
opendevreviewMerged openstack/ironic master: Fix typo calling save_and_reraise_exception  https://review.opendev.org/c/openstack/ironic/+/93970401:10
JayFDon't we already have automatic TLS between the agent and the conductor? I thought that was mutual TLS and not just certs on both sides but I'm not 100% sure02:12
TheJuliait is auto04:09
rpittaugood morning ironic! o/07:33
kubajjgood morning, rpittau, I have a quick question. For the backport, should I just add the release note to the change, squash the commits together or do some other git magic?09:13
rpittaukubajj: you can cherry pick both commits into one, either manually squashing them or letting cherry-pick do that for you, and then remember to check that both commits are mentioned in the commit message as "cherry picked from"09:24
opendevreviewJakub Jelinek proposed openstack/ironic-python-agent stable/2024.2: Fix RAID volume name  https://review.opendev.org/c/openstack/ironic-python-agent/+/93970010:35
opendevreviewJulia Kreger proposed openstack/ironic master: ci: fix ipv6 CI job  https://review.opendev.org/c/openstack/ironic/+/93973214:17
opendevreviewJakub Jelinek proposed openstack/ironic-python-agent master: Fix errors in the function erase_devices_express  https://review.opendev.org/c/openstack/ironic-python-agent/+/93980314:17
kubajjThis ^ is a little bit of a weird error we came across today. I reported the bug and figured out that it can be fixed easily. Let me know if you want me to change anything. I am still a bit confused about that function.14:19
TheJuliakubajj: weeeird14:27
TheJuliakubajj: I'm super curious how metadata cleaning failed to generate that though14:27
kubajjTheJulia: indeed14:27
TheJuliaI'm like "wait, how did that fail" and got to "oh, the change only makes sense with a pass then fail14:28
kubajjTheJulia: exactly, it passes the first try and then it fails the second one14:29
kubajjbut isn't the first try block doing something only for nvmes?14:30
TheJuliaIndeed, the only way I could really see it happening is if _is_nvme(dev) returns false14:37
TheJuliaand destroy_disk_metadata then fails14:37
TheJuliaI guess I could see that starting to go sideways... if the device is like a read only block device provided by a storage fabric14:38
kubajjTheJulia: for us it failed with a hard drive (we think the disk is broken, because it throws wipefs: error: /dev/sda: probing initialization failed: No such device or address)15:05
TheJuliaYeah, a detected device which has been offlined.15:06
TheJuliawe likely need to backport that change then15:07
TheJuliaBecause that is a thing which happens15:07
kubajjTheJulia: I was expecting that (learned my lesson and included a release note for this one without a follow up), can do the backport15:12
TheJuliacool cool15:15
TheJuliathat works15:15
rpittaugood night! o/16:01
kubajjTheJulia: am I safe to recheck? There seems to have been a problem with the cloning of ipmitool (fatal: expected 'packfile')16:16
TheJuliafirst I've heard of this issue, do you have a link?16:16
JayFthat failure usually indicates a network issue cloning down git for the IPA ramdisk build16:18
opendevreviewVerification of a change to openstack/ironic-python-agent stable/2024.2 failed: Fix RAID volume name  https://review.opendev.org/c/openstack/ironic-python-agent/+/93970016:18
JayFI haven't seen the exact error, but that matches that pattern16:18
kubajjTheJulia: https://zuul.opendev.org/t/openstack/build/336fb98a492d40509947af9c600cf93416:20
JayFkubajj: that needs a recheck, it's a network issue16:21
JayFHTTP 504 pulling from codeberg.org16:21
JayFwell, "network issue" I am assuming it's temporary, I haven't tried the clone locally16:21
TheJuliayeah16:21
kubajjjust tried to clone it locally and worked, so hopefully will pass this time16:22
TheJuliahmm, ipv6 job is being misbehaving16:40
opendevreviewJulia Kreger proposed openstack/ironic master: WIP - A very early wip of bootc deployment on the ironic side  https://review.opendev.org/c/openstack/ironic/+/93789716:55
JayFjfyi cid and I will be doing a pair review + maybe demo of inspection rules monday after the meeting19:25
JayFwould be a good time to get context+understanding so we can get this over the line19:26
cid++++++ * 1019:26
opendevreviewJulia Kreger proposed openstack/ironic master: ci: fix ipv6 CI job  https://review.opendev.org/c/openstack/ironic/+/93973219:56
TheJuliaokay, I'm hopeful regarding ^19:56
rm_work[m]during a clean, can you provide multiple steps? like this is my proposed example of a full firmware update run using multiple hardware managers: https://paste.openstack.org/show/bcLs4DoiQkplTVFRRoLI/21:43
rm_work[m]the example on the firmware-updates doc page for some reason is doing multiple node clean / node service commands, even within the same interface, but it seems like given the data types of the schema I could provide them all in one go?21:44
rm_work[m]and could I go even further and if I wanted to flash multiple nic or disk firmwares, could I just provide the same component twice in the same settings block, with different urls and other params?21:46
TheJuliaso, I'm becoming convinced the v6 stuff is just not going to be happy with edk2/ovmf images with defaults, states... and the pile of open issues21:48
TheJuliarm_work[m]: yeah, idea wise, it can be a giant list if youw ant21:49
TheJuliarm_work[m]: keep in mind, steps have to be available to the state21:49
rm_work[m]i'm not totally sure what that second one means21:50
TheJuliaso, with manual clean you provide a priority, as long as the priorities are separate I don't think it dedupes just by name21:50
TheJuliasince it is about making a list by step priority when you do manual cleaning21:50
TheJuliabasically what I meant was the step has to exist and be available in the returned list of steps by the hardware manager for when you trigger it21:51
TheJuliaso a hardware manager, in theory could say "this is automatic" and "this is manual" based upon this overall phase with cleaning specifically. Service is all user driven21:52
TheJuliaDeploy wise, a hardware manager could expose additional steps, I just don't think we'ev ever tried it outside of maybe some of the downstream users21:52
TheJuliaAnyway, I'm going to step away from the computer since I'm almost to the end of my workday and I need to scream about ipv621:53
rm_work[m]hmm, so as long as you do a clean and specify the hardware manager for at least one step, it will be "triggered" and then we could just do whatever we want without even providing any settings, which, yeah that makes sense21:54
rm_work[m]I suppose settings is just arbitrary stuff that gets passed to the step in the manager and parsed in whatever way we code it21:54
rm_work[m]anyway cool thanks, screaming about ipv6 is also a hobby of mine periodically so have fun :D21:55
TheJuliarm_work[m]: actually, if the hardware manager is loaded, when ironic collects steps, it will poll the additional hardware manager as well for available steps21:56
rm_work[m]ah, so even if it isn't provided in the clean json at all, it can still do thing?21:56
rm_work[m]which seems... not like something we should be doing likely, but still, interesting21:56
TheJuliaThis is how you can mash in automatic steps for automatic actions, its when your doing manual cleaning or changes is when only what you specify to execute will be triggered, but that step of course has to be available and visible based upon the flow your executing within. Flow being servicing, cleaning, or deploying21:56
rm_work[m]OH, so this is how we make things just "always happen" on clean21:57
TheJuliaall goes back to that balancing the forces :)21:57
TheJuliaexactly!21:57
rm_work[m]ok, very very useful, somehow I didn't think about that21:57
rm_work[m]we were trying to figure out how to set up automation to always trigger the right cleaning steps but it's just... this :D21:57
rm_work[m]thanks o721:58
TheJuliaThe hardware manager can literally make that decision, except for the baked in clean steps21:58
TheJuliabut you can also override it's default value to deprioritize or disable it21:58
TheJuliaThink of it like a swiss army knife full of options.21:58
rm_work[m]yeah since we'll need to be building a custom IPA image anyway for our custom hardware manager, we can tweak those as well if necessary21:59
TheJuliaexactly21:59
rm_work[m]do you mean code changes to existing hardware managers, or is it actually just a config somewhere even for those as far as priority/enabled?21:59
TheJuliaThere is an ability to put in an ironic.conf entry which overrides the default values which get assigned22:01
TheJuliato reprioritize a step value or even disable a step22:01
TheJuliaI don't have it off the top of my head, but it is in ironic.conf22:01
TheJuliaso if you look at the example, it will talk about priority values22:02
TheJuliafor steps specifically22:02
TheJuliaanyway, wife is now requesting walk time and I still need to use impolite words to describe ipv6 + edk2/ovmf22:02
opendevreviewJulia Kreger proposed openstack/ironic master: ci: fix ipv6 CI job  https://review.opendev.org/c/openstack/ironic/+/93973222:34
JayFrm_work[m]: worth noting the flip is possible: you set zero as the priority, it won't be added to automated cleaning but will still be available for manual cleaning22:47
-opendevstatus- NOTICE: The Gerrit service on review.opendev.org will be offline momentarily for a restart to put some database compaction config changes into effect, and will return within a few minutes22:54
opendevreviewcid proposed openstack/ironic master: DB: inspection rules migration  https://review.opendev.org/c/openstack/ironic/+/93931822:54
opendevreviewcid proposed openstack/ironic master: Apply Rules: inspection rules migration  https://review.opendev.org/c/openstack/ironic/+/93921822:54
opendevreviewcid proposed openstack/ironic master: API/Testing: Inspection rules migration  https://review.opendev.org/c/openstack/ironic/+/93921722:54
opendevreviewJulia Kreger proposed openstack/ironic master: ci: fix ipv6 CI job  https://review.opendev.org/c/openstack/ironic/+/93973223:46

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!