Monday, 2024-02-05

opendevreviewSteve Baker proposed openstack/sushy-tools master: Add virtual-media-boot to openstack driver  https://review.opendev.org/c/openstack/sushy-tools/+/90676802:54
adamcarthur5After talking with Jay and Julia, they helped figure out that I need to run OS_CLOUD=devstack-system-admin before running "openstack baremetal" commands in devstack to get it to work. I went to try it again tonight and it did not work. Am I forgetting a step that I maybe completed but forgot about?04:14
adamcarthur5I have gone through the commands I ran in the command history but cannot see anything of note04:18
adamcarthur5ls04:19
JayFmake sure you didn't ever source the openrc file04:21
JayFif you did get a fresh env04:21
adamcarthur5Yeah, so I opened a fresh ssh instance, and just went straight to running the command, no luck04:26
JayFyou have that cloud defined in /etc/openstack/clouds.yaml (iirc) (?)04:27
adamcarthur5Yeah. Looking through the keystone logs it seems that there is a problem with SQL? something to investigate tomorrow. My two ideas.04:31
adamcarthur5- Someone (a hacker) is crawling for exposed devstack instances (I think this is unlikely but incase its something that happens i'll include it) and doing "interesting things"04:31
adamcarthur5-  My VPN is doing something interesting / breaking when snapshots are created04:31
adamcarthur5Well, I thought I would test the latter by trying to rollback to a snapshot from just after running ./stack.sh but that has created more errors. Time to roll back to pre-stack I think :^)04:34
opendevreviewAdam McArthur proposed openstack/ironic-prometheus-exporter master: [codespell] Adding Tox Target for Codespell  https://review.opendev.org/c/openstack/ironic-prometheus-exporter/+/90677804:44
opendevreviewAdam McArthur proposed openstack/ironic-prometheus-exporter master: [codespell] Adding CI target for Tox Codespell  https://review.opendev.org/c/openstack/ironic-prometheus-exporter/+/90677904:44
JayFyeah I was kinda worried about your exposure to getting botted with a public-ip'd devstack instance05:07
JayFI'm going to bed o/05:07
opendevreviewAdam McArthur proposed openstack/ironic master: [codespell] Fixing Spelling Mistakes  https://review.opendev.org/c/openstack/ironic/+/90680605:23
opendevreviewAdam McArthur proposed openstack/ironic master: [codespell] Adding Tox Target for Codespell  https://review.opendev.org/c/openstack/ironic/+/90680705:23
opendevreviewAdam McArthur proposed openstack/ironic master: [codespell] Adding CI target for Tox Codespell  https://review.opendev.org/c/openstack/ironic/+/90680805:23
opendevreviewAdam McArthur proposed openstack/networking-baremetal master: [codespell] Fixing Spelling Mistakes  https://review.opendev.org/c/openstack/networking-baremetal/+/90678805:26
opendevreviewAdam McArthur proposed openstack/networking-baremetal master: [codespell] Fixing Spelling Mistakes  https://review.opendev.org/c/openstack/networking-baremetal/+/90678805:30
opendevreviewAdam McArthur proposed openstack/networking-baremetal master: [codespell] Adding Tox Target for Codespell  https://review.opendev.org/c/openstack/networking-baremetal/+/90678905:30
opendevreviewAdam McArthur proposed openstack/networking-baremetal master: [codespell] Adding CI target for Tox Codespell  https://review.opendev.org/c/openstack/networking-baremetal/+/90679005:30
adam-metal3Hello Ironic! Is this the best IRC channel to ask about sushy-tools/emulator in relation to OVB or is there a better one? I have a hard time finding sushy specific IRC .07:48
rpittaugood morning ironic! o/07:50
*** jph0 is now known as jph08:25
dtantsuradam-metal3: generally, sushy stuff is here10:09
adam-metal3dtantsur, thanks do you happened to know what is correct way to supply OpenStack username and password for sushy-emulator if I would like to use the "Systems resource driver: OpenStack" ?10:14
dtantsuradam-metal3: https://docs.openstack.org/sushy-tools/latest/user/dynamic-emulator.html#systems-resource-driver-openstack it expects a populated clouds.yaml (standard openstack credentials storage) and you pass the cloud name10:15
adam-metal3dtantsur, thanks!10:16
iurygregorygood morning Ironic10:37
masgharGood morning!10:44
opendevreviewVerification of a change to openstack/ironic bugfix/22.0 failed: Make sure we eject media from DVD when CD is requested  https://review.opendev.org/c/openstack/ironic/+/89933711:17
opendevreviewMerged openstack/sushy master: Force constraints when installing a package during tox test  https://review.opendev.org/c/openstack/sushy/+/90746111:49
opendevreviewMerged openstack/ironic stable/2023.1: Fix service role support  https://review.opendev.org/c/openstack/ironic/+/90726912:21
opendevreviewMerged openstack/ironic master: Basic support for OVN VTEP switches  https://review.opendev.org/c/openstack/ironic/+/90056812:40
opendevreviewMerged openstack/ironic-python-agent stable/2023.1: preserve/handle config drives on 4k block devices  https://review.opendev.org/c/openstack/ironic-python-agent/+/89373612:44
iurygregoryFYI I will miss the weekly meeting today13:03
TheJuliagoodmorning13:59
TheJuliaerr, good morning13:59
* TheJulia clearly needs to start sipping the coffee13:59
rpittauI second that and I'll go grab another espresso quickly14:05
TheJuliaso, regarding the VTEP change, any chance we can dial down the requirement to jsonschema 4.16 ? 4.19 is not in Centos9 Stream yet so things are breaking14:20
iurygregorysounds good to me14:23
JayFI set the requirement where I did to make it pass reliably on all python versions14:40
JayFAs long as CI is happy, I will be14:40
JayFBut we have to have support for something that's only in the newer ones, I think it was the ability to set dependent fields14:41
JayFCan someone moderate the meeting this morning? 14:44
TheJuliawas there something specific that we went to use that was only in 4.19 ?14:44
* TheJulia pokes packages.ubuntu.com14:47
TheJulialooks like launchpad is also down14:49
JayFTheJulia: the ability to set dependent fields iirc14:50
JayFThere is something in the schema for the VTe0 ports that we don't have anywhere else, that didn't work reliably between older and newer versions14:50
JayFI'll note that since then, I seen some of Iury changes that set a schema URL explicitly, and I wonder if that might also be a solution14:51
TheJuliai guess the underlying challenge is to support installation on a binary distribution, I wonder if we need to revisit the logic there to be able to dial it to slightly older so it is in distributions14:54
TheJuliafairly up to date debian 4.1014:56
JayFI'm saying I painted with a broad brush14:57
TheJuliaunderstood14:57
JayFbecause upgrading requirements IMO is a positive thing unless there's a hitch (like this)14:57
JayFand I can probably be more targetted now that I know there's a hitch14:57
JayFI guess I'll run the meeting in lieu of volunteers, I may not be around afterwards, woke up feeling a bit ill14:57
TheJuliadebian unstable is still 4.1014:58
JayFI suspect >=4 is fine14:58
TheJuliaI can run it, go get some rest14:58
TheJuliaI'm just in another meeting right now14:58
JayFeh if it's just me and you it's going to be a lot of #topic #topic #topic #end14:58
JayFlol14:58
TheJuliait is likely going to wrap any minute14:58
TheJuliatrue, lack of quorum14:58
JayFif it's just me and you maybe I bail the meeting with exactly that message14:59
JayFunless there's a pressing item on genda14:59
* TheJulia starts the coffee maker14:59
JayF#startmeeting Ironic15:00
opendevmeetMeeting started Mon Feb  5 15:00:13 2024 UTC and is due to finish in 60 minutes.  The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
opendevmeetThe meeting name has been set to 'ironic'15:00
TheJuliao/15:00
* TheJulia hands out coffee to everyone awake15:00
JayFDo we have quorum?15:00
JayFo/15:00
masgharo/15:00
TheJuliaIs there some holiday we didn't get a memo on?15:00
JayFNot today afaik15:00
* TheJulia suspects we don't have quorum15:00
JayFUnless there is objection, I'm going to close the meeting for lack of quorum. I see nothing pressing on https://wiki.openstack.org/wiki/Meetings/Ironic to make me wanna push forward with 3 folks15:01
TheJuliaWell, we can just punt and if there is a specific topic or decision we can handle it as needed15:01
TheJulia+1 to lack of quorum/end the meeting for today15:01
masghar++, agreed15:01
JayF#info Only three Ironic contributors for meeting, cancelling for lack of quorum.15:03
JayF#endmeeting15:03
opendevmeetMeeting ended Mon Feb  5 15:03:07 2024 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:03
opendevmeetMinutes:        https://meetings.opendev.org/meetings/ironic/2024/ironic.2024-02-05-15.00.html15:03
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/ironic/2024/ironic.2024-02-05-15.00.txt15:03
opendevmeetLog:            https://meetings.opendev.org/meetings/ironic/2024/ironic.2024-02-05-15.00.log.html15:03
opendevreviewJulia Kreger proposed openstack/ironic master: packaging: reduce jsonschema revision  https://review.opendev.org/c/openstack/ironic/+/90775915:29
TheJuliaadded ironic-week-prio flag to ^15:38
rpittauTheJulia: just FYI ubuntu jammy has 3.2.015:58
TheJuliayeah, 3.2.0 fails16:18
TheJulia3.2.0 was released nov 18, 201916:19
rpittauyep16:22
TheJuliaI'm not sure we want to take on the additional complexity to get 3.2.0 working, since ubuntu is sort of not moving it forward anyway16:24
rpittauI don't think it's worth either, and there are multiple ways to work around that16:28
TheJuliaYeah16:28
JayFThe thing is I'm not sure four is actually required to make it work... I will probably take a timeboxed look at this later16:29
TheJulia3.2.0 was the last release before 4.0.016:29
TheJulia4.0.0 itself was yanked, 4.0.1 was uploaded16:29
*** ravlew is now known as Guest171516:29
rpittaugood night! o/17:13
opendevreviewMerged openstack/networking-generic-switch master: [codespell] Adding Tox Target for Codespell  https://review.opendev.org/c/openstack/networking-generic-switch/+/90679217:41
AcormierAtJunipWe hit a pretty serious Neutron networking issue related to baremetal nodes when using OVN: https://bugs.launchpad.net/neutron/+bug/1995078. I believe this issue prevents anyone from operating an HA Openstack/Ironic environment with the networking generic switch driver.18:11
AcormierAtJunipThe short of it is that Neutron has a different scheduling mechanism for external/baremetal ports than it does for the L3 router interface.  If these two ports in OVN land on a different chassis (odds are they will) the packets will get dropped.18:11
opendevreviewDmitry Tantsur proposed openstack/ironic master: Extract common code for managers with periodics and workers  https://review.opendev.org/c/openstack/ironic/+/90789818:11
TheJuliaAcormierAtJunip: yeouch18:12
AcormierAtJunipJust wanted to call this out for posterity.  It was very difficult for me to reach the bottom of what issue we were facing.  The symptom was that no ARP response was being generated by OVN for the internal gateway.18:13
dtantsurCould you file a bug against ironic? Sounds like at least it has to be tracked.18:13
dtantsuror s/ironic/networking-generic-switch/18:13
JayFI'm adding NGS as impacted18:13
AcormierAtJunipI can do that.  I figured that since OVN seems to be taking over as default network driver this would be pretty critical for Ironic18:14
TheJulia... I wonder how we might fix this.18:15
dtantsurJayF: I'd rather have a baremetal-specific bug tbh18:15
JayFdtantsur: then file what you want?18:15
JayFI don't know what you're looking for unless we have code changes to coordinate18:15
dtantsurI don't understand it even remotely enough18:15
JayFYeah that's why I'm not filing a bug and just making it as Ironic-y18:15
AcormierAtJunipThere is a PR which is kind of abandoned: https://review.opendev.org/c/openstack/neutron/+/87203318:15
dtantsurwhich is largely why I'd want it written18:15
JayFthis is wildly out of my wheelhouse of Ironic knowledge :)18:15
AcormierAtJunipBut I BELIEVE would fix the issue18:16
JayFthe OVN VTEP stuff was all taken from one AcormierAtJunip's original patch lol18:16
JayFlaunchpad is having a bad day18:16
AcormierAtJunipThe owner of that issue is Rodolfo Alonso, which I believe is a lead for Neutron18:16
JayFI think I added NGS as impacted but couldn't get a comment through18:16
TheJulialaunchpad is definitely having a very bad day18:17
AcormierAtJunipWas kind of hoping we might be able to convey the importance of this issue to Ironic's functionality.  Assuming networking baremetal nodes with VMs is a priority18:17
JayFI mentioned it in Neutron IRC18:18
JayFwe'll see what folks say18:18
JayFI don't have much technical help to offer but I can poke people and ask them nicely to do things :D 18:18
AcormierAtJunipThat's about as much I can hope, appreciate it.18:19
TheJuliaAcormierAtJunip: Thank you for raising visibility18:21
TheJuliaI've let some downstream folks know as well18:21
AcormierAtJunipOf course18:22
TheJuliaoh where oh where did I put my brain18:23
opendevreviewMerged openstack/sushy master: [codespell] Adding Tox Target for Codespell  https://review.opendev.org/c/openstack/sushy/+/90679818:53
opendevreviewMerged openstack/ironic master: packaging: reduce jsonschema revision  https://review.opendev.org/c/openstack/ironic/+/90775919:16
opendevreviewMerged openstack/ironic master: tox: Drop envdir  https://review.opendev.org/c/openstack/ironic/+/90756019:16
opendevreviewMerged openstack/sushy master: [codespell] Adding CI target for Tox Codespell  https://review.opendev.org/c/openstack/sushy/+/90679919:24
opendevreviewMerged openstack/ironic stable/2023.1: [Backport] Fixes Raid creation in iLO6 and other BMC with latest schema  https://review.opendev.org/c/openstack/ironic/+/90406919:40
opendevreviewMerged openstack/sushy master: Handle session-uri in body  https://review.opendev.org/c/openstack/sushy/+/86796920:55

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!