| JayF | It occurs to me that there is an option 3 for fixing this: reverting the module removals from ironic lib. It might require us to conditionally import the option registration though, but I'm not 100% sure | 00:19 |
|---|---|---|
| JayF | Obviously we could then release another ironic lib that doesn't completely just blow up when paired with Dalmatian stuff | 00:20 |
| cardoe | I think you can mark a release as yanked in pypi and people will avoid it. | 00:25 |
| cardoe | As far as the RBAC it feels like that’ll bite me as well. My plan is to make all the node as project scoped to a DC team. Leaving system scope to OpenStack admins. | 00:25 |
| cardoe | Is that reasonable or no? | 00:26 |
| JayF | I think that's what our model intends, but it seems like that will clash with project manager type users using ironic-ui unless we revisit our default policy there -- and at a minimum allow custom policy to let project-scoped users see some info there. | 00:27 |
| * JayF afk | 00:27 | |
| cardoe | Yeah I was gonna have someone try to work on that. | 00:27 |
| cardoe | We’re also looking at using skyline. | 00:27 |
| opendevreview | Merged openstack/ironic master: Disable deep image inspection for kernel/initramfs https://review.opendev.org/c/openstack/ironic/+/935359 | 01:05 |
| opendevreview | Verification of a change to openstack/ironic master failed: Remove default override for config options policy_file https://review.opendev.org/c/openstack/ironic/+/934569 | 01:09 |
| *** jroll09 is now known as jroll0 | 07:36 | |
| rpittau | good morning ironic! o/ | 08:04 |
| rpittau | JayF: re ironic-lib release: yep, didn't really think through the code removal breaking change..... sorry about that :/ | 08:08 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Use OVN and OVS from OS packages in CI https://review.opendev.org/c/openstack/ironic/+/935628 | 08:46 |
| *** dking is now known as Guest404 | 10:22 | |
| opendevreview | Kamil Gustab proposed openstack/sushy master: Adds attributes for IPs for Ethernet Interfaces. https://review.opendev.org/c/openstack/sushy/+/930814 | 11:13 |
| opendevreview | Kamil Gustab proposed openstack/sushy master: Adds attributes for IPs for Ethernet Interfaces. https://review.opendev.org/c/openstack/sushy/+/930814 | 11:15 |
| opendevreview | Kamil Gustab proposed openstack/sushy master: Adds attributes for IPs for Ethernet Interfaces. https://review.opendev.org/c/openstack/sushy/+/930814 | 11:16 |
| opendevreview | Kamil Gustab proposed openstack/sushy master: Adding attributes for IPs for Ethernet Interfaces. https://review.opendev.org/c/openstack/sushy/+/930814 | 11:17 |
| cardoe | rpittau: since you poked some tenks. Will you look at https://review.opendev.org/c/openstack/tenks/+/907112 ? That’ll free up a chain that’s already workflowed. | 13:02 |
| rpittau | cardoe: thanks, not sure why I ignored that patch :D | 13:02 |
| cardoe | I don’t have +2 on that repo and I was going to look at the config. I suspect it’s using a group other than an ironic one for permissions. If so should I make it use the ironic one or just augment the existing? | 13:04 |
| rpittau | cardoe: hold on, I'll fix that now | 13:05 |
| cardoe | Jay had mentioned that was guess why. | 13:05 |
| rpittau | cardoe: you should have +2 there now | 13:05 |
| rpittau | errr nvm | 13:06 |
| rpittau | wrong one | 13:06 |
| rpittau | ok fixed | 13:06 |
| *** Guest404 is now known as dking | 13:17 | |
| opendevreview | Verification of a change to openstack/ironic master failed: Agent deploy: account for disable_power_off https://review.opendev.org/c/openstack/ironic/+/934637 | 13:19 |
| olafb | Any kayobe experts here who can point me to an example of how to configure and run the seed hypervisor and seed vm on the ansible control node? | 13:30 |
| dtantsur | I wonder if they have an IRC channel for their own | 13:37 |
| dtantsur | olafb: the kayobe docs suggest using #openstack-kolla | 13:38 |
| olafb | dtantsur: thanks, I'll head over there. | 13:39 |
| opendevreview | Merged openstack/tenks master: [codespell] Fixing Spelling Mistakes https://review.opendev.org/c/openstack/tenks/+/907112 | 14:00 |
| opendevreview | Merged openstack/tenks master: [codespell] Adding Tox Target for Codespell https://review.opendev.org/c/openstack/tenks/+/907113 | 14:00 |
| cardoe | rpittau: I've got +2 now. Thanks! | 14:13 |
| opendevreview | Merged openstack/tenks master: [codespell] Adding CI target for Tox Codespell https://review.opendev.org/c/openstack/tenks/+/907114 | 14:15 |
| cardoe | So a couple of weeks ago someone on my team mentioned in here that ironic can go down silently when it has problems with the interfaces during bring up. And I think Julia confirmed that. Well just as an update, it's not going down silently. It's logging out the "oslo_service" logger which is silenced in about every config I've seen. | 14:16 |
| JayF | Re: yesterday's issue with the release, we should double check to make sure that installing IPA from source in IPA builder uses constraints | 14:35 |
| JayF | I'll do that once I get started this morning after my meetings | 14:36 |
| opendevreview | Verification of a change to openstack/ironic master failed: IPMI power: account for disable_power_off https://review.opendev.org/c/openstack/ironic/+/932624 | 14:43 |
| dtantsur | rpittau, iurygregory, could either of you W+1 https://review.opendev.org/c/openstack/ironic-python-agent/+/934234 please? Got 2x +2 | 14:56 |
| rpittau | done | 14:56 |
| dtantsur | Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit | 14:57 |
| dtantsur | I wonder if metal3 could pull the registry from quay.io instead | 14:57 |
| JayF | This is something that's happening to multiple projects. The rate limits were drastically reduced. I'm thinking it might be wise to make metal3 non-voting until we find some solution | 14:58 |
| JayF | We can ask core reviewers to specifically check to make sure that we have at least one pass on that job | 14:58 |
| opendevreview | Verification of a change to openstack/ironic master failed: Remove default override for config options policy_file https://review.opendev.org/c/openstack/ironic/+/934569 | 14:59 |
| rpittau | dtantsur: I have a deja vu, wasn't that discussed already in metal3? | 15:00 |
| opendevreview | Merged openstack/ironic master: Refactoring: simplify tear_down_agent https://review.opendev.org/c/openstack/ironic/+/934956 | 15:00 |
| TheJulia | good morning | 15:02 |
| dtantsur | rpittau: we definitely migrated many dependencies, but presumably not the registry? | 15:16 |
| opendevreview | Merged openstack/ironic master: Inspection: account for disable_power_off https://review.opendev.org/c/openstack/ironic/+/934233 | 15:17 |
| rpittau | mmm right | 15:17 |
| dtantsur | JayF: looks like a few patches have passed the gate shortly after that job failed | 15:18 |
| rpittau | the issue with the rate limit is quite rare, at least I don't remember last time it happened before this one | 15:19 |
| rpittau | we have more frequent timeouts with ansible download :D | 15:20 |
| TheJulia | JayF: replied on https://review.opendev.org/c/openstack/ironic-specs/+/933612, to be clear we're inventing exactly 0% of that example schema. It is all prior art from the noted container earlier in the spec. | 15:28 |
| TheJulia | A quick review on https://review.opendev.org/c/openstack/networking-baremetal/+/933149 would be appreciated. I'd like to get it off my in progress column. | 15:29 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Use OVN and OVS from OS packages in CI https://review.opendev.org/c/openstack/ironic/+/935628 | 15:40 |
| opendevreview | Verification of a change to openstack/ironic master failed: IPMI power: account for disable_power_off https://review.opendev.org/c/openstack/ironic/+/932624 | 15:45 |
| TheJulia | So, I was talking with harald downstream around one of the networking challenges, around preferred porting, and what if we had some way to denote a preferred trait on a port which if on a flavor would focus on using that port for port binding | 16:06 |
| opendevreview | Merged openstack/ironic-python-agent master: Add a command to lock down the agent https://review.opendev.org/c/openstack/ironic-python-agent/+/934234 | 16:20 |
| TheJulia | err, it is stored on each port in advance of that point... | 16:22 |
| cardoe | Where can I drop a link for you guys to review at next week's call? (I'll be on PTO the next 2 weeks). It's a link to Gerrit for all Ironic patches that have been +2 & +W but haven't landed. I think that'd be worth while to peek at each week. | 16:23 |
| rpittau | cardoe: etherpad or pastebin is fine | 16:27 |
| TheJulia | cardoe: meeting agenda could also work i guess ? | 16:37 |
| rpittau | and yeah add the link to the meeting agenda :) | 16:38 |
| mbrandt | Hey, for the review (https://review.opendev.org/c/openstack/ironic-python-agent/+/934330) I have a question. Should I change it to "if not ipv6.startwith('::')" or is it ok? I also looked in the ipmitool code and I think the 255 comes from the code in which the struct ip6_addr_statuses is called. Here NULL returns 0xFF which is 255. | 16:39 |
| mbrandt | But of course it makes more sense to be checked only "if not '::'" ^^ | 16:39 |
| TheJulia | Greetings mbrandt. So my takeaway when I left the comment was yes, that felt like a bug in ipmitool in that it was returning 255 from that data stucture. I guess my worry and suggestion is to just use startswith is to disregard the rest of the line on a match | 16:43 |
| TheJulia | in case anyone "fixes" that weird behavior in ipmitool | 16:43 |
| mbrandt | Yeah true its really weird. Ok thank you, ill do it :) | 16:44 |
| TheJulia | yeah, it felt mind bending to look at. I'm glad someone else looked at the ipmi code as well. :) | 16:48 |
| opendevreview | Maximilian Brandt proposed openstack/ironic-python-agent master: Inventoried MAC address for only ipv6 addresses https://review.opendev.org/c/openstack/ironic-python-agent/+/934330 | 16:51 |
| opendevreview | Verification of a change to openstack/ironic master failed: Agent deploy: account for disable_power_off https://review.opendev.org/c/openstack/ironic/+/934637 | 16:52 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Use alternative registry for metal3 job https://review.opendev.org/c/openstack/ironic/+/935895 | 17:01 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Use alternative registry for metal3 job https://review.opendev.org/c/openstack/ironic/+/935895 | 17:02 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Use quay.io registry image for metal3 job https://review.opendev.org/c/openstack/ironic/+/935895 | 17:02 |
| rpittau | JayF, dtantsur, giving quay.io a try here ^ | 17:03 |
| TheJulia | quay++ | 17:06 |
| rpittau | :) | 17:09 |
| dtantsur | rpittau: nice! if it works, we should propose it upstream | 17:09 |
| rpittau | yup :) | 17:09 |
| TheJulia | https://github.com/openstack/ironic/commit/e8f6fdd56d7eccbdeba3ff3d1101a8dc6a8be72f <-- turns out we did add a way to do upfront declaritive mapping | 17:26 |
| rpittau | good night! o/ | 17:34 |
| opendevreview | Merged openstack/networking-generic-switch stable/2023.2: Do not make actual device changes in bind_port() https://review.opendev.org/c/openstack/networking-generic-switch/+/896112 | 17:38 |
| JayF | So it doesn't look like releases team is willing to even entertain a discussion about yanking the release, because policy says if constraints work it's okay | 17:39 |
| JayF | even if in reality that's not the truth for many of our users | 17:39 |
| JayF | so I suggest we either cut an early bugfix release, or revert the library removals and release another ironic-lib | 17:39 |
| JayF | since it's gotta be our action to fix the users | 17:39 |
| JayF | I guess we could also commit a upper bound to the stable branch requirements.txt, but I think that's also against policy? | 17:40 |
| opendevreview | Merged openstack/ironic master: Remove default override for config options policy_file https://review.opendev.org/c/openstack/ironic/+/934569 | 17:54 |
| cardoe | mbrandt / TheJulia: I think should look at using the ip_address module and using its is_link_local and other property checks on the address. You can then normalize it for ipmiyool. | 18:19 |
| TheJulia | That semi-feels like overkill, given it is just a sanity check of if there is a value or not, the base issue is the format ipmitool spits out is "::/255" as of the version when ipmitool's repo was frozen | 18:25 |
| JayF | Ramdisk deploy interface works integrated with nova, yeah? And a rebuild can replace the ramdisk in place? /me hopes | 19:01 |
| TheJulia | it just uses the kernel/ramdisk, but I don't know if you can change the image uuid to change it | 19:04 |
| opendevreview | Verification of a change to openstack/ironic master failed: Agent deploy: account for disable_power_off https://review.opendev.org/c/openstack/ironic/+/934637 | 19:12 |
| JayF | ack; I'll mark that as an unknown to be tested then | 19:15 |
| opendevreview | Julia Kreger proposed openstack/ironic master: trivial: remove xclarity remenent https://review.opendev.org/c/openstack/ironic/+/935973 | 19:27 |
| TheJulia | JayF: what I'm saying is rebuild is effectively, for ramdisk, just a reboot unless you can change the image_source as part of the request on nova. I just don't know at that level. | 21:07 |
| JayF | I *think* you can, but I'll dig in | 21:07 |
| TheJulia | Granted, ramdisk is one those super useful "just works" drivers for people who are in very specific patterns of operation and may also not be using nova at all. | 21:08 |
| JayF | honestly I just wanted to make sure I wasn't tilting at windmills trying to get it to work integrated at all :) | 21:08 |
| TheJulia | I think people have done it, honestly... its all a blur | 21:08 |
| JayF | I'll bring it in focus in the coming weeks, then :) | 21:19 |
| JayF | My downstream is excited about the prospect of skipping the extra reboot | 21:19 |
| TheJulia | oh my | 21:25 |
| TheJulia | good luck! | 21:25 |
| TheJulia | I need to do bootable containery things and such | 21:25 |
| TheJulia | if downstream is quiet next week :) | 21:25 |
| TheJulia | JayF: going back to the topic of ironic-lib, I guess the biggest question is "what is the easiest path" at this point | 21:28 |
| TheJulia | JayF: and to be honest, happy to push buttons either way, but I'm definitely on the lazy consensus for easiest path given my current workload. | 21:28 |
| TheJulia | So I filed a new bug regarding vif binding modeling. https://bugs.launchpad.net/ironic/+bug/2089315 is a little vague, but it just seems wrong that there is no knob to de-preference pxe enabled boot itnerfaces. | 21:29 |
| TheJulia | interfaces | 21:29 |
| JayF | TheJulia: IMO; the "easy route" is doing an early bugfix release, it'd only be a couple weeks early | 21:30 |
| JayF | we already have the long term fix in motion | 21:30 |
| JayF | rpittau: Are you game to cut the bugfix release early to resolve the ironic-lib stuff? | 21:30 |
| TheJulia | then lets get them proposed and someone who is flagged as a release manager to approve it | 21:30 |
| JayF | I was going to at least ask someone who consumes those first :) | 21:30 |
| TheJulia | Stepping out to find lunch and head back up the mountain | 21:30 |
| TheJulia | JayF: reasonable | 21:30 |
| JayF | enjoy o/ | 21:30 |
| TheJulia | my brain is also semi-soup since I'm sort of re-writing our admin/multitenancy docs into actual more network clarity based docs which still need quite a bit of theory as well | 21:31 |
| TheJulia | because omg the gap which exists there | 21:31 |
| JayF | cid: cardoe: other cores; can you toss a +2 on https://review.opendev.org/c/openstack/ironic/+/935895 if you're cool with it? Otherwise I'll land as single core as a CI fix | 21:32 |
| JayF | TheJulia: That will be a great improvement, and a heavy lift. Thank you :) | 21:32 |
| TheJulia | I guess we got used to just explaining the highlights and filling in the gaps | 21:36 |
| TheJulia | we should have been pointing at docs | 21:36 |
| TheJulia | gah! | 21:36 |
| JayF | yeah, similarly part of why adamcarthur5 is digging into api-ref/client stuff is I found something undocumented in api-ref -- it'll be nice to have that rounded out and automated | 21:42 |
| JayF | plus the bonus test coverage we're adding in tempest is super exciting. More tempest testing that /is not scenario stuff that tends to flake/ +++++ | 21:42 |
| cardoe | That's a real pretty link on the agenda. Credit to you JayF for 99% of it. | 22:33 |
| JayF | I did waht | 22:33 |
| JayF | :blink: | 22:33 |
| cardoe | I added a link that's Workflow+1 and mergeable but not yet merged for all Ironic projects. | 22:33 |
| cardoe | I just took your opened on all Ironic projects query and tweaked it slightly. | 22:33 |
| JayF | you mean on the whiteboard? or what? | 22:34 |
| JayF | I'm so confused | 22:34 |
| JayF | oh, I see | 22:34 |
| cardoe | But I think we should at least review those once a week cause they'd be a good source of "CI is broken" or "hrm... this should have landed?" type items. | 22:34 |
| JayF | I have a folder on my bookmarks toolbar with: 1) that, 2) the equivalent for DIB, 3) the equivalent for oslo | 22:37 |
| JayF | usually every other day I open all three and look for anything recently updated and unreviewed | 22:37 |
| JayF | closing the loop re: ironic-lib | 23:59 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!