Tuesday, 2022-11-29

vanougood morning ironic01:37
opendevreviewMerged openstack/ironic bugfix/19.0: Add SNMPv3 authentication functionality  https://review.opendev.org/c/openstack/ironic/+/86446504:32
opendevreviewMerged openstack/ironic bugfix/18.1: Add SNMPv3 authentication functionality  https://review.opendev.org/c/openstack/ironic/+/86446605:24
opendevreviewSONG SHUKUN proposed openstack/ironic bugfix/19.0: Add support auth protocols for iRMC  https://review.opendev.org/c/openstack/ironic/+/86594607:36
opendevreviewSONG SHUKUN proposed openstack/ironic stable/xena: Add support auth protocols for iRMC  https://review.opendev.org/c/openstack/ironic/+/86594707:42
opendevreviewSONG SHUKUN proposed openstack/ironic bugfix/18.1: Add support auth protocols for iRMC  https://review.opendev.org/c/openstack/ironic/+/86595007:47
opendevreviewSONG SHUKUN proposed openstack/ironic stable/wallaby: Add support auth protocols for iRMC  https://review.opendev.org/c/openstack/ironic/+/86595107:49
arne_wiebalckGood morning, Ironic!07:55
rpittaugood morning ironic! o/08:31
jandershey vanou arne_wiebalck rpittau and Ironic o/08:39
rpittauhey janders :008:46
kubajjGood morning everyone 09:34
rpittauhey kubajj :)09:39
dtantsurTheJulia: wdyt about silencing oslo_policy logs by default? (cc janders)09:47
dtantsurjanders: we can actually override logs filtering in ironic-image, but I wonder what TheJulia thinks about it09:47
jandersdtantsur ACK. And TY. Makes sense!09:47
vanouHi arne_wiebalck rpittau janders kubajj dtantsur o/09:52
rpittauhey vanou :)09:52
opendevreviewRiccardo Pittau proposed openstack/bifrost master: Use ansible 6.x  https://review.opendev.org/c/openstack/bifrost/+/86596910:21
dtantsurjanders, TheJulia, hmm, apparently I thought about it already: https://opendev.org/openstack/ironic/commit/7748b154b0477e42cd0937c9f52cbed33e7b7c8e10:29
dtantsurjanders: don't we have this patch ^^?10:29
dtantsurit should go as far as 18.1..10:30
dtantsurI have some local logs from a bugzilla that don't have oslo_policy debug logging10:31
dtantsurlooking at the bug, it's OCP 4.7, which is ancient and not supported any more10:32
opendevreviewMerged openstack/ironic-inspector stable/ussuri: Fix port id vs uuid in ValidateInterfacesHook  https://review.opendev.org/c/openstack/ironic-inspector/+/86362611:07
opendevreviewkamlesh chauvhan proposed openstack/ironic-tempest-plugin master: OOB inspection timeout fix for iDRAC  https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/86580311:46
iurygregorygood morning Ironic o/11:48
jandershey iurygregory13:22
jandersajya I have a quick iDRAC question - are you aware on any limits on number of characters in the hostname (or the URL) of the virtual media source http(s) server? stevebaker[m] and I spent a fair bit of time troubleshooting something related today and it seems that the difference between a URL that works and the one that does not work is the length13:23
jandersIt's an R640 with iDRAC 6.00.30.0013:24
jandersstevebaker[m] we may want re-test on 5.10.x iDRAC firmware on rpittau's advice - it seems more stable13:30
opendevreviewRiccardo Pittau proposed openstack/bifrost master: Use ansible 6.x  https://review.opendev.org/c/openstack/bifrost/+/86596913:33
ajyajanders: What length did you try? Not aware if there is a limit, but it's likely that internally there is a data type restricting it. What errors do you see?13:34
janderssample error: https://paste.openstack.org/show/bsGMi4kPonUH2njQQmNk/13:35
janderslooks like up to 64 chars worked (just for the hostname, not the whole URL)13:36
jandersbeyond that did not13:36
janderswill PM you some details (not sharing here due to hostnames)13:36
opendevreviewHarald Jensås proposed openstack/ironic master: benchmark: Create allocations for nodes  https://review.opendev.org/c/openstack/ironic/+/86598813:37
opendevreviewHarald Jensås proposed openstack/ironic master: Use association_proxy for ports node_uuid  https://review.opendev.org/c/openstack/ironic/+/86293313:45
opendevreviewHarald Jensås proposed openstack/ironic master: Use association_proxy for port groups node_uuid  https://review.opendev.org/c/openstack/ironic/+/86478113:45
opendevreviewHarald Jensås proposed openstack/ironic master: Use association_proxy for node chassis_uuid  https://review.opendev.org/c/openstack/ironic/+/86480213:45
opendevreviewHarald Jensås proposed openstack/ironic master: Use association_proxy for node allocation_uuid  https://review.opendev.org/c/openstack/ironic/+/86598913:45
ajyajanders, stevebaker I'm suspecting this will be same on all iDRAC versions, especially, in older versions if still observed in newer. Indeed there could be some data type limiting host to 64 chars. Why 64 chars? Maybe because hostnames often can't be more than 64 chars, and here it does not distinguish between subdomains.14:01
ajyaWhat is happening - it probably gets silently truncated and then can't find the file as it's not the same URL anymore. I will check with firmware team if they can fix this. Are there any BZs or Jiras I can link to? Also I found this - https://bugzilla.redhat.com/show_bug.cgi?id=1956421 that looks similar issue14:02
hjensasTheJulia: I added chassis and allocation to benchmark and added patches to use association_proxy SQL join for chassis_uuid and allocation_uuid. benchmark stats here: https://paste.opendev.org/show/bR0EaLEjG5NVr8Kp1f9r/14:04
hjensasTheJulia: it is ever so slightly slower when fields are used, but also quite a bit of savings when no fields are used.14:06
TheJuliaa 100th of a second is... in the territory of a standard deviation14:13
TheJulianot bad at all14:13
TheJuliahjensas: it is 66% faster, which is still not bad at all14:15
hjensasTheJulia: ~35% faster-ish, it taks ~66% of the previous time. No? 14:19
hjensasstill, not bad.14:19
TheJuliait could be 66ish of the previous time14:20
TheJuliaI'm still waking up and just starting to sip coffee14:20
TheJuliafixing my ratio compare is 40%14:21
TheJuliaat least in terms of nodes per second14:21
opendevreviewHarald Jensås proposed openstack/ironic master: benchmark: Create allocations for nodes  https://review.opendev.org/c/openstack/ironic/+/86598815:03
* dtantsur is wondering at which point of the ongoing benchmarking we get to "rewrite the conductor in rust" :D15:09
JayFdtantsur: when we get past the low hanging fruit. Right now it's like we have a whole harvest on the ground waiting to be picked up :P15:14
TheJuliaI think the rules require that we take a break between those two actions, and *then* enjoy a delicous cup of pumpkin spiced coffee15:15
dtantsura very good idea!15:25
JayFis disliking "pumpkin spice *" an unpopular opinion/15:26
dtantsurseems so15:26
TheJuliaheh15:26
TheJuliaI left the pumpkin spiced coffee at the RV which I won't see for like a month15:27
TheJuliastevebaker[m]: numerous small fixes posted to https://review.opendev.org/q/project:x%252Fproliantutils hopefully that should shed some clarity. If you want to kick that downstream bug over to me, feel free15:28
rpittauTheJulia: we may want to merge this first https://review.opendev.org/c/x/proliantutils/+/86595615:30
TheJuliastevebaker[m]: absolutely *no idea* when they will be merged15:30
TheJuliarpittau: on master, sure15:30
TheJuliaThis stuff goes all the way back to wallaby15:30
rpittauah right15:30
opendevreviewJay Faulkner proposed openstack/ironic-lib master: No longer override install_command in tox.ini  https://review.opendev.org/c/openstack/ironic-lib/+/86603215:38
opendevreviewJay Faulkner proposed openstack/python-ironicclient master: No longer override install_command in tox.ini  https://review.opendev.org/c/openstack/python-ironicclient/+/86603315:39
JayF^^ there was a post on the mail list about how this is a misconfiguration15:41
JayFso I fixed it15:41
opendevreviewRiccardo Pittau proposed openstack/bifrost master: Fix deprecated module ansible lint error  https://review.opendev.org/c/openstack/bifrost/+/86604016:03
rpittauneed to split, see ya all tomorrow! o/16:03
TheJuliado I get to feel old because I remember the last time we did fixes to install_command in tox.ini?16:07
dtantsurthen please welcome me in the old club :)16:08
* TheJulia starts a club, makes hot chocolate and passes it around16:13
kubajjCan I join the club even if I don't remember it?16:13
TheJuliayes!16:13
kubajjI'll come for the hot chocolate then16:14
dtantsuryummy!16:32
opendevreviewJay Faulkner proposed openstack/ironic master: DB & Object layer for of shards & node.shard_key  https://review.opendev.org/c/openstack/ironic/+/86423616:35
TheJuliaHas anyone thought of ironic specific topics for Open Infra Summit?16:41
dtantsurSummit specifically? not really (esp. since my participation is a very wide open question)16:42
dtantsuryou mean, Forum?16:42
TheJuliaSummit16:42
TheJuliaMaybe a better question: What topics might be interesting?16:42
dtantsurAs a track chair, I should probably not provide specific suggestions :)16:43
TheJuliaoh! Congrats! I was not aware!16:43
dtantsur(I'm also the only track chair, apparently, since my peer withdrew)16:44
TheJuliaugh16:44
opendevreviewJay Faulkner proposed openstack/ironic master: DB & Object layer for node.shard  https://review.opendev.org/c/openstack/ironic/+/86423616:57
TheJuliaJayF: anything else on the shard key spec that you think needs to change based upon your work so far?16:58
TheJuliabeyond the 6 comments on there?16:58
JayFI've been trying to put comments on there as I go17:00
TheJuliaack ack17:01
JayFTheJulia: while you're working on shards; mind looking @ something w/me?17:04
JayFTheJulia: Trying to get my sharding PR fixed up (no more Shard table/object) and I'm getting head-scratching unit test failures17:04
JayFlike I broke the whole node object in the API 17:04
JayFhttps://review.opendev.org/c/openstack/ironic/+/864236 broken code is here17:04
JayFonly hint I have it 17:05
JayF**is17:05
JayF    webtest.app.AppError: Bad response: 500 Internal Server Error (not 200 OK or 3xx redirect for http://localhost/v1/nodes)17:05
JayFb'{"error_message": "{\\"faultcode\\": \\"Server\\", \\"faultstring\\": \\"\'Node\' object has no attribute \'shardstorage_interface\'\\", \\"debuginfo\\": null}"}'17:05
TheJuliathat looks like a solid typo17:05
JayFtell that to grep17:06
JayFif it's an err; I don't know where it comes from17:06
TheJuliahttps://review.opendev.org/c/openstack/ironic/+/864236/7/ironic/api/controllers/v1/node.py17:06
TheJuliamissing typo second half17:06
TheJuliaerr17:06
TheJuliamissing comma17:06
JayFI've only looked at that code 47 times without seeing that lolsob17:06
JayFmight be a little optimistic; but with the no-mo-caching, I'm really going to try and get this hooked up completely today17:07
JayFso we need to get that spec landed so I can land this stuf lol17:07
* TheJulia shrugs about need to have something before something else17:08
* TheJulia is laid back17:08
TheJuliaor at least... tries to be17:08
JayFThere's "laid back" and there's "yolo landing API microversions without the in-review spec landing"17:10
JayFIDK exactly where my line is, and I think that's just over it17:10
* TheJulia shrugs17:25
TheJuliawe've done worse before, trust me :)17:25
TheJuliaspeaking of things, has anyone heard of status on service role stuffs?\17:28
opendevreviewJay Faulkner proposed openstack/ironic master: DB & Object layer for node.shard  https://review.opendev.org/c/openstack/ironic/+/86423617:36
dtantsurfolks, I'm on PTO until end of week, see you on Monday17:48
kubajjTheJulia: if I add something to the conf/inspector.py do I need to tell ironic somewhere to load this value or is this handled on its own? I am can't find a change similar to what I am trying to do17:59
opendevreviewJay Faulkner proposed openstack/ironic master: DB & Object layer for node.shard  https://review.opendev.org/c/openstack/ironic/+/86423618:03
opendevreviewJakub Jelinek proposed openstack/ironic master: WIP: Enable alternative storage for inventory  https://review.opendev.org/c/openstack/ironic/+/86605618:30
kubajjIs Swift used for anything in Ironic now?19:15
opendevreviewRiccardo Pittau proposed openstack/bifrost master: Fix deprecated module ansible lint error  https://review.opendev.org/c/openstack/bifrost/+/86604019:16
opendevreviewJulia Kreger proposed openstack/ironic-specs master: Add a shard key  https://review.opendev.org/c/openstack/ironic-specs/+/86180319:20
TheJuliakubajj: temporary object storage19:21
TheJuliakubajj: if it is in the list, it is automatically available on the CONF object 19:21
kubajjTheJulia: that is quite cool19:22
TheJuliathere is a method call someplace towards the end of the file like register_opts19:22
TheJuliathat does the magic :)19:22
JayFTheJulia: OK; so my code (and the comment on the spec) said I was going to use node.shard19:24
JayFTheJulia: do you feel strongly about node.shard_key? I can rework my patch for the [needs a second hand to count] time :P 19:24
TheJuliaoh19:26
TheJuliaI thought you meant you used node.shard_key19:27
TheJuliauhhh.... shard is fine I guess19:27
JayFI neverr was good at the game of "telephone" :P 19:27
JayFlol19:27
* TheJulia tries to think of reasons against it19:27
JayFshard_key is a bad name because it's a key by definition19:27
JayFwe don't say "conductor_group_key"19:27
TheJuliayeah19:27
TheJuliayeah, makes sense19:28
JayFthat's at least how I landed on `node.shard` 19:28
TheJuliamicrowaving some lunch and I'll fix the spec19:30
JayFI'm excited, DB+Obj layer done, going to plug up the API then time to test19:30
opendevreviewJulia Kreger proposed openstack/ironic-specs master: Add a shard key  https://review.opendev.org/c/openstack/ironic-specs/+/86180319:39
stevebaker[m]good morning19:57
stevebaker[m]TheJulia: I've reviewed those proliantutils changes19:57
TheJuliadtantsur: do you want me to just merge the inspector sqlalchemy changes at this point since you seem to want the first patch to have the fixes in the second patch... and in order to do it I have to cult over basically the entire content of the second patch.... soo....20:10
TheJuliastevebaker[m]: thanks20:10
opendevreviewJulia Kreger proposed openstack/ironic stable/zed: Change boot_interface order of iRMC driver  https://review.opendev.org/c/openstack/ironic/+/86599820:19
JayFTheJulia: I put a -1 on ^^^, I'm available for the next 2-3 hours if you wanna try and talk me out of it :)20:54
stevebaker[m]TheJulia: I think my esp.img is correct because grub boots, but it drops into the grub shell, like it can't find grub.cfg https://pasteboard.co/83Is3ylp2jqa.png21:59
stevebaker[m]should it be in /boot/grub2/grub.cfg rather than /boot/grub/grub.cfg?21:59
JayFyes21:59
JayFhmm, or not22:00
JayFon my machine; grub2; my files are at /boot/grub/grub.cfg22:00
stevebaker[m]oh, its an ironic.conf option!22:01
JayFoh, nice!22:02
stevebaker[m]this is for the boot iso that ironic builds for virtual-media-boot22:04
stevebaker[m]it worked \o/22:19
TheJuliaJayF:  wrt that change, to me it kind of makes sense given that zed is still fairly new, and otherwise you can run into misconfigurations where you need entirely separate config or node specific overrides to try and navigate22:49
TheJuliaJayF: in other words, not super friendly for operators who seek consistency22:50
JayFI figured that'd be the answer; but what about the operators who want it configured that way? We're going to break them with minimal notice (hope they read changelogs)22:51
TheJuliastevebaker[m]: \o/22:51
JayFand if nobody in their right mind wants it configured that way; why is it supported in master?22:51
JayFAFAICT; we're just setting into stone a preference -- one that I hold as well -- but it's hard to justify that for already-released software22:52
stevebaker[m]metal3 and bifrost set grub_config_path to =EFI/{{ efi_distro }}/grub.cfg, but I just tried grub_config_path=EFI/BOOT/grub.cfg and it worked, so a distro agnostic UEFI default seems to exist22:53
stevebaker[m]JayF: I think its a throwback to Legacy BIOS defaults, we should change that now that UEFI is the default22:54
JayFI mean, I'm onboard with that for master; I just question the wisdom of breaking already-deployed Zed operators who may prefer the current default22:54
JayFI'm all for a loose stable policy; but causing people who could be currently working to have to reconfigure to maintain existing behavior seems really, really unfriendly to me22:54
TheJuliaso, the conundrum is FJ is actually asking me for it22:55
JayF(in context of a stable branch)22:55
stevebaker[m](soz I was talking about a different thing)22:55
TheJuliaheh22:55
JayFstevebaker[m]: oooh22:55
TheJuliaI think their visibility/testing of it has lagged behind release22:55
JayFstevebaker[m]: makes sense; although I prefer the EFI-is-boot-partition setup; but most distros don't :P 22:55
JayFTheJulia: I mean, is it true that: 1) There is a use case for the existing, default configuration. 2) Users who want that configuration would need to reconfigure when updating to latest-stable if we pushed that change?22:56
TheJuliathat being said the gray area for stable is operator friendly driver fixes..... soo... worth raising the question :)22:56
JayFTheJulia: I said I wasn't sure where my line was, I think ^^^^ that might be it for stable lol22:56
TheJuliathe existing default config doesn't work if pxe is not enabled22:56
TheJuliathat is the conundrum22:57
stevebaker[m]JayF: I believe the interface order is wrong to the point of it being broken by default22:57
TheJuliaI think people have been routing around it without actually going "hey, this is broken"22:57
stevebaker[m]oh, if no pxe22:57
JayFCould we do something like, detect the broken configuration and flip behavior only in that case? That's probably not worth it from complexity22:57
TheJuliawe would have no way to really know if it was intentional or not22:58
TheJuliaand we would be enabling driver features if their config was intentional and one person didn't talk to the ironic admin22:58
JayFIs there a case/configuration, that makes sense for a user to have, that would break if we deployed this change?22:58
JayFthat's the fulcrum of my concern22:58
TheJulianot really, since we're just changing the supported matrix23:00
TheJulianot the actual configured preference23:00
TheJuliaand we're adding ipxe to that list, and the first option remains the same which is a vendor preferred method23:00
TheJuliaif we removed an option from that list, that would be very breaking23:01
JayFwhat /is/ boot.IRMCPXEBoot and how is it different than usual pxe boot? /me goes to ask the code23:01
JayFit just backs up bios config for use in cleaning, interesting23:02
TheJuliayeah, and they don't want to adapt to the formal bios interface either :(23:02
JayFWhy do we give them a choice?23:02
JayFWe have a place for drivers that don't wanna work in our model. 23:02
TheJuliathat code was from *long* before we had the bios interface23:03
JayFso the only breaky situation I can think of with this23:03
JayFis if I intentionally want pxe, not ipxe23:03
JayFand I customized my pxe template, but not my ipxe template23:03
JayFyeah?23:03
TheJuliaand you were *expecting the buggy behavior even though your enabled_boot_interfaces parameter was set to ipxe, pxe, magical-virtual-media-323:04
TheJuliathat enabled list always take precedence as long as it is supported23:04
JayFso there's no way, outside of disabling pxe globally or merging 865998, to get that boot IF to use pxe?23:05
TheJuliato use ipxe23:05
TheJuliaand correct23:05
TheJuliajust... ipxe23:05
JayFokay, so if we merged this, I'm looking at the changelog and am concerned it's not ... noisy enough23:06
TheJuliaheh23:06
JayFis there any way we can make that more directive+clear to operators of zed/other stable branches?23:06
* JayF brb plumbers here23:07
TheJuliafunny you said that, because my draft review comments I somehow never posted were basically like "wow, this is a bit much"23:07
TheJuliaJayF: likely... less words actually, the reno is very verbose23:07
TheJuliaat least, that is my thought23:07
JayFSo lets make this a clearer release note23:12
JayFand then *immediately* cut a **minor** (not patch) release of Ironic23:13
JayFto ensure it gets shipped as a minor release bump 23:13
TheJuliasounds good to me23:13
TheJuliaopenstack release team might argue... depending on their mood and the phase of the moon23:14
JayFnah, elodilles suggested we do a minor release at one point b/c we added a dep23:15
JayFso I suspect there'll be some appreciation that we thought about it at all :P 23:15
JayFTheJulia: https://review.opendev.org/c/openstack/ironic/+/865998/1#message-93140ce6955d15a0eacb26e5d697bfc7e1875d3b23:15
TheJuliaack, I'll rev it tomorrow23:19
TheJuliagoing to go paint now23:19

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!