| cardoe | What’s the ironic-guest-metadata patch? I was reading through the runbook change and it seems like it would benefit from allowing disable_ramdisk no? | 04:25 |
|---|---|---|
| kubajj | Good morning o/ | 04:35 |
| rpittau | good morning ironic! o/ | 06:33 |
| janders | hey rpittau o/ | 06:44 |
| rpittau | hey janders :) | 06:44 |
| rpittau | dtantsur, JayF, TheJulia, iurygregory: any backport for 2023.1 that we want to merge soon? It's going to be unmaintained soon https://review.opendev.org/c/openstack/releases/+/925347 | 07:26 |
| masghar | JayF will also take a look after finishing my current task :) and good morning ironic! | 08:11 |
| opendevreview | Merged openstack/ironic bugfix/26.0: Update .gitreview for bugfix/26.0 https://review.opendev.org/c/openstack/ironic/+/925410 | 08:15 |
| dtantsur | rpittau: time flies.. I don't have anything | 09:48 |
| rpittau | it does indeed :/ | 09:48 |
| rpittau | thanks! | 09:48 |
| cid | Good afternoon Ironic. | 11:10 |
| cid | cardoe: I will appreciate any feedback. You can include it on the change. | 11:12 |
| cid | Though, I believe the change currently allows disable_ramdisk? Not sure | 11:13 |
| cardoe | So I didn’t put it on there cause it seemed like a feature creep request. | 11:14 |
| cid | It's even in the specification, so if it's indeed not there (which I confirming), that needs a fix. | 11:15 |
| cid | The metadata patch: https://review.opendev.org/c/openstack/nova/+/923910 | 11:15 |
| cardoe | The client doesn’t support sending that flag. I was trying to add it in https://review.opendev.org/c/openstack/python-ironicclient/+/924895 | 11:17 |
| iurygregory | morning ironic | 11:22 |
| iurygregory | rpittau, let me try to check a few things here | 11:22 |
| iurygregory | rpittau, do we have releases for ipa? I see we have 3 open patches (CI gave -1) and maybe the only open patch in ironic https://review.opendev.org/c/openstack/ironic/+/921884 is a fix for that ... | 11:49 |
| iurygregory | but we wouldn't need ci fix for a release in ironic | 11:49 |
| cid | cardoe: a runbook has a `disable_ramdisk` field which will be set by an admin and the value is retreived at the api level. | 12:12 |
| cid | This patch https://review.opendev.org/c/openstack/python-ironicclient/+/924895 has to do with manual provisioning. | 12:12 |
| * cid when using arbitrary steps | 12:14 | |
| rpittau | iurygregory: yep, I saw that, I meant something that needs to be backported other than that CI patch | 12:26 |
| iurygregory | yeah, I've looked but couldn't find anything =) | 12:29 |
| cardoe | So I know there was some discussion previously on macOS / homebrew but I've gone ahead and added python-ironicclient into the homebrew package for openstack cli. Oddly only python-ironicclient breaks with Python 3.12 so I'm submitting a fix to homebrew to pin stuff at Python 3.11 for now. | 13:12 |
| iurygregory | cardoe, I don't think support for 3.12 landed https://github.com/openstack/python-ironicclient/blob/master/setup.cfg | 13:16 |
| rpittau | cardoe: how does that break? instead of inning it would be better to fix it since Python 3.12 is supposed to be supported in the next OS dev cycle | 13:16 |
| cardoe | So "openstack baremetal" claims the command doesn't exist. I've gotta dig into why. | 13:18 |
| rpittau | cardoe: that's not ironicclient though, that's openstack cli | 13:19 |
| cardoe | https://github.com/Homebrew/homebrew-core/pull/179231 is what I've submitted. I'm just trying to make sure homebrew users (we've got a bunch at work) can use ironic commands. | 13:19 |
| cardoe | okay openstack cli doesn't see the baremetal command when installed with Python 3.12 | 13:19 |
| TheJulia | good morning | 13:20 |
| iurygregory | good morning TheJulia =) | 13:21 |
| cardoe | good morning | 13:21 |
| cardoe | iurygregory: the classifiers unfortunately don't mean anything to the dependency resolver | 13:22 |
| iurygregory | but it means is not 100% tested or fully supported yet if i remember | 13:23 |
| rpittau | cardoe: I tested OSC with Py 3.12.4 and it looks good to me, well except for the missing 'pkg_resources', but I don't see the error of baremetal command missing | 13:24 |
| cardoe | So I can make it happen on one machine I've got but then it doesn't happen on the other. | 13:25 |
| TheJulia | fun | 13:25 |
| rpittau | I'm afraid that does not depend on either OSC or ironicclient then :) | 13:25 |
| cardoe | Using Python 3.11 on the problematic machine makes it work. | 13:25 |
| cardoe | Since Python 3.12 isn't officially supported, I figured I'd pin the homebrew package back to 3.11. | 13:26 |
| cardoe | Just wanted to get users a working "openstack baremetal" out of the box on macOS. | 13:27 |
| rpittau | cardoe: it makes sense, all considered Python 3.12 is not supported yet, but the issue you're seeing does not look related | 13:27 |
| cardoe | I'm just throwing print()'s in here trying to figure out why | 13:28 |
| rpittau | cardoe: since you're there probably good to increase openstackclient to version 6.6.1 | 13:30 |
| clarkb | is it possible that issue is related to the simplejson issue that affected the CI jobs using osc? | 14:22 |
| clarkb | maybe you've got simplejson installed via something else and then it works, but if not explicitly included it breaks? | 14:23 |
| clarkb | I think 6.6.1 should include the fix for that though | 14:27 |
| clarkb | oh the package uses 6.6.0. So ya maybe updating to 6.6.1 will fix it? | 14:28 |
| rpittau | it should, I was thinking the same but cardoe said that it worked for him in a different machine | 14:28 |
| rpittau | it's worth trying | 14:28 |
| clarkb | rpittau: ya maybe that machine has simplejson installed via something else | 14:28 |
| rpittau | right | 14:28 |
| clarkb | homebrew doesn't isolate python package installs | 14:29 |
| cardoe | I'll give it a whirl. | 14:29 |
| clarkb | what I do personally is create virtualenvs for the tools I use then symlink commands from a dir in my path to the virtualenv bin/$command paths | 14:30 |
| cardoe | That's how my setup is. But alas just trying to lower the barrier to entry for others. | 15:17 |
| rpittau | good night! o/ | 15:32 |
| iurygregory | TheJulia, do you still have the pxe boots big sticker? | 17:37 |
| TheJulia | iurygregory: I do | 17:38 |
| TheJulia | iurygregory: if you send me an address, I can mail one to you | 17:38 |
| iurygregory | TheJulia, can you mail to Amy? :D | 17:39 |
| iurygregory | I will be in Suwon and she can probably give me (it would be cheaper to mail =) ) | 17:39 |
| TheJulia | yes | 17:39 |
| iurygregory | awesome! tks :D | 17:40 |
| cardoe | clarkb: it does. The packages are installed in their own "Cellar" directory. | 17:44 |
| cardoe | What I noticed is that "pkg_resources" isn't available in the openstackclient Cellar | 17:44 |
| JayF | "big sticker" | 17:48 |
| * JayF is thinking life-sized | 17:48 | |
| iurygregory | JayF, nooooooo lol | 17:49 |
| iurygregory | it would be awesome but it wouldn't be ok to put on my car :D | 17:50 |
| cardoe | clarkb: I can confirm that on Python 3.12, when it makes the venv in the Cellar directory it specifies "--without-pip" | 17:53 |
| cardoe | The box that "worked" used Python 3.12 and used --with-pip | 17:53 |
| cardoe | The default in 3.12 switched to --without-pip which is why pkg_resources is missing | 17:53 |
| cardoe | I'm guessing early in the 3.12 cycle the homebrew folks overrode that when I tested this out. | 17:54 |
| cardoe | I had 3.12.0_2 on that box | 17:54 |
| cardoe | 6.6.1 didn't fix it | 17:57 |
| cardoe | And I can break it on my working box by using a newer Python 3.12 | 17:58 |
| clarkb | it is possible you need 6.6.1 for the simplejson fix as well as also installing setuptools for pkg_resources | 18:26 |
| cardoe | Just chased down a fun IPA issue. Two interfaces were configured on the provisioning network. The one we expected and then another one was just left on there. Had intermittent network issues with the IPA commands on that box since I believe conductor connects back to IPA. We looked at traffic on the switch and both MACs replied in the ARP for the IP that IPA got. | 18:38 |
| cardoe | https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt specifically "arp_ignore" | 18:39 |
| cardoe | The default is to reply back on any interface that saw the packet. | 18:41 |
| cardoe | dhcp from the IPA image is putting the interface in promiscuous mode and it sees the packet and replies back it seems like | 18:42 |
| TheJulia | JayF: a life sized sticker... HMMMMMmmmm | 18:49 |
| TheJulia | JayF: My HOA may have an issue with how this idea would pan out | 18:49 |
| cardoe | oh this is a fun chain actually. not sure if you guys would want my tweak in ipa builder. But setting arp_ignore=1 fixes the issue. | 19:17 |
| cid | o/ | 20:34 |
| JayF | cardoe: I'm really interested about the OS machinations causing that situation | 20:54 |
| JayF | cardoe: honestly just with my "interested in cool weird linux networking things" hat on as much as an ironic one :D | 20:54 |
| clarkb | just off the top of my head arp should only respond if an address is configured on that interface. That would imply all of those addresses are getting dhcp leases. But then it would also imply the leases are for the same IP? | 20:57 |
| clarkb | definitely a curious situation from a "linux networking behavior" perspective | 20:57 |
| cardoe | So you would think clarkb but that’s not the case in Linux. It replies to all. | 20:58 |
| clarkb | til | 20:58 |
| cardoe | See the kernel doc I linked. | 20:58 |
| cardoe | So it’s a timing thing. | 20:58 |
| clarkb | huh arp_ignore = 1 is the behavior I would have expected to eb default | 20:59 |
| iurygregory | anyone has access to Dell PowerEdge XR11 ? to check some redfish response ? | 20:59 |
| JayF | dude you're not gettin' a dell (from me) :D | 21:00 |
| iurygregory | I'm really wondering if the ID for Systems on it is really 1 instead of System.Embedded.1 | 21:00 |
| clarkb | cardoe: I guess the primary issue is that both interfaces are on the same network segment so see arps originate for that l2 span | 21:01 |
| cardoe | So in playing around we had 4 interfaces plugged into the provisioning network. And randomly the box got a read timeout when getting its token setup. And then would keep failing on heartbeat cause it didn’t have a token. | 21:01 |
| clarkb | with multiple interfaces on different network segments this behavior wouldn't create those issues (though packets might still get delivered via unexpected paths) | 21:03 |
| cardoe | IPA during that token setup had already told conductor its IP. And conductor arped it and all 4 answered. So it connected back over a different interface. | 21:03 |
| JayF | cardoe: I can tell you that for onmetal, we explicitly only brought up one half of the bond in the prov/rescue/cleaning networks | 21:04 |
| JayF | cardoe: to avoid these kinds of issues | 21:04 |
| cardoe | Yep. That’s what we’re doing as well. | 21:04 |
| cardoe | Just not doing that at the moment. | 21:04 |
| JayF | using (some of) the same code Brad and Clif wrote ten years ago, I bet :D | 21:04 |
| cardoe | Maybe. Wanna go as stock as possible. So I’m pretty judicious with the axe. | 21:05 |
| cardoe | Also Brad McConnell? | 21:06 |
| JayF | Well, he was the network architect on the project | 21:07 |
| JayF | but I'm talking about morgabra + clif, the devs on the onmetal team | 21:07 |
| JayF | who wrote the ML2 plugin piece for both v1 and v2 style switches... in at least one of the cases IIRC it's just sshing in and running a script brad McC (NOC) gave us lol | 21:08 |
| * JayF wonders if you still run https://github.com/rackerlabs/arsenal/ + the cache_image patch | 21:11 | |
| JayF | probably not | 21:11 |
| JayF | python 2.7 last supported version, I hope not lol | 21:12 |
| *** awb_ is now known as awb | 23:34 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!