| opendevreview | Jacob Anders proposed openstack/ironic bugfix/19.0: Modify do_node_verify to avoid state machine stuck https://review.opendev.org/c/openstack/ironic/+/860308 | 00:01 |
|---|---|---|
| janders | ^ I think this is it, I don't believe we had iDRAC verify_steps in Xena (which is why the other patch has only gone down to yoga) | 00:02 |
| janders | stable/zed and bugfix/21.0 already have this fix - it merged before these branches were forked from master I suppose | 00:06 |
| opendevreview | Merged openstack/ironic-inspector stable/xena: CI: Zuul no longer respects queue https://review.opendev.org/c/openstack/ironic-inspector/+/860162 | 00:38 |
| JayF | TheJulia: just making a note here that they were talking in the opendev channel about weird mirror breakages. We should probably make sure we check the dib logs to make sure that was a properly built image | 00:48 |
| JayF | I'm not actually here but wanted to put that somewhere where it would be in the morning for when I am here 🙈 | 00:48 |
| arne_wiebalck | Good morning, Ironic! | 06:16 |
| opendevreview | Merged openstack/ironic bugfix/20.2: Modify do_node_verify to avoid state machine stuck https://review.opendev.org/c/openstack/ironic/+/860307 | 09:53 |
| opendevreview | Merged openstack/ironic stable/yoga: Modify do_node_verify to avoid state machine stuck https://review.opendev.org/c/openstack/ironic/+/860306 | 10:10 |
| opendevreview | Merged openstack/ironic bugfix/19.0: Modify do_node_verify to avoid state machine stuck https://review.opendev.org/c/openstack/ironic/+/860308 | 10:10 |
| iurygregory | good morning ironic | 11:17 |
| TheJulia | good morning | 13:21 |
| iurygregory | morning TheJulia | 13:29 |
| dtantsur | morning iurygregory, TheJulia | 13:40 |
| * TheJulia did some dib reviews and feels good | 13:57 | |
| opendevreview | Julia Kreger proposed openstack/ironic-python-agent-builder stable/xena: dhcp-all-interfaces: let NetworkManager doit. https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/860432 | 14:07 |
| opendevreview | Julia Kreger proposed openstack/ironic-python-agent-builder stable/wallaby: dhcp-all-interfaces: let NetworkManager doit. https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/860433 | 14:07 |
| opendevreview | Julia Kreger proposed openstack/ironic-python-agent stable/wallaby: CI: Zuul no longer respects queue param https://review.opendev.org/c/openstack/ironic-python-agent/+/860187 | 14:13 |
| * TheJulia ponders inspector | 14:32 | |
| JayF | TheJulia: thank you kindly :) | 14:37 |
| TheJulia | so I think I grok what is happening with inspector | 14:47 |
| TheJulia | and I compared successes/failures, and I can see what is helping us substantially | 14:47 |
| JayF | If it's simple, what's the basic gist of what's going on? | 14:49 |
| TheJulia | basically, we have some code in our devstack lib which goes "oh, we're on a cloud provider where our jobs are likely to fail with a full blown ramdisk, switch to tinycore!" | 14:49 |
| TheJulia | in this case, the opposite is occuring, we're on the default, the passing runs are on tinycore | 14:50 |
| TheJulia | so I'm thinking of just making that job tinycore since it runs with a more resource constrainted setup | 14:50 |
| JayF | makes sense | 14:53 |
| JayF | tinycore ramdisks also I think seem to change less over time | 14:53 |
| JayF | compared to the ones based on cloud images | 14:53 |
| JayF | (is that a correct-seeming observation?) | 14:53 |
| opendevreview | Julia Kreger proposed openstack/ironic-inspector stable/wallaby: CI: Zuul no longer respects queue https://review.opendev.org/c/openstack/ironic-inspector/+/860163 | 14:54 |
| TheJulia | basically yes | 14:54 |
| TheJulia | so it makes a lot of sense to use a full ramdisk early on | 14:54 |
| TheJulia | but then more sense to dial back to tiny more as default as things are stable since every time we update that image or merge a fix, our stock images can grow due to updates | 14:54 |
| TheJulia | ... or new firmware as we've learned :( | 14:55 |
| JayF | that makes a lot of sense | 14:55 |
| TheJulia | (we've had a few "lets remove excess firmware" passes over the last few years, the stock image is something like 450MB of firmware images) | 14:55 |
| JayF | I know we leave some firmware in for real-hardware-reasons | 14:55 |
| TheJulia | yeah, we don't strip nics out for that very reason | 14:56 |
| JayF | but I wonder if we should add a flag to dib and like remove all firmware in thsoe cases, for VM/test usage | 14:56 |
| TheJulia | there *is* one we did, but it alone was like 196MB | 14:56 |
| TheJulia | and it is almost unheard of outside of a specific vertical | 14:56 |
| TheJulia | so, that would be an uphill fight, and tons of folks pull the stock images for their CI on real hardware instead of rebuilding.... soo... yeah :( | 14:57 |
| JayF | last time you did one of those changes; I looked at the firmware my two 10gb `atlantic`-driver NICs used and it was bananas | 14:57 |
| JayF | TheJulia: I don't mean for images we publish; we don't use the iamges from the CI run for publishing, we run a second, separate build in post | 14:57 |
| JayF | TheJulia: so we could test with a small image; then for publishing flip that value and push a full fat one | 14:57 |
| JayF | I'm not saying we should do it; but the way it's arranged it should be possible | 14:57 |
| TheJulia | oh, we already kind of do that actually... we would just have to build it each time | 14:57 |
| JayF | oh; I get what you're saying | 14:58 |
| TheJulia | the conundrum then is mirrors and CDNs | 14:58 |
| JayF | I lived in IPA-only world so long I forgot !-src jobs existed | 14:58 |
| TheJulia | c'est la vie :) | 14:58 |
| TheJulia | ironic-cores: Quick review for queue stuffs: Yes, passing CI: https://review.opendev.org/c/openstack/ironic-lib/+/860172 | 15:05 |
| TheJulia | I'm happy to +2 it, but I won't workflow it as I changed it | 15:06 |
| TheJulia | https://zuul.opendev.org/t/openstack/build/482c46005949488d8a8dbb2a14dc14a4 is... fun | 15:10 |
| TheJulia | it *looks* like it tried to fire up a kvm config on the host which it went all "NOPE" | 15:11 |
| dtantsur | TheJulia: the regex part is likely wrong | 15:11 |
| TheJulia | https://33782c12fa31ec23ee50-cbc72fe775ff576ae1ecbe350e73c360.ssl.cf2.rackcdn.com/860172/2/check/ironic-lib-partition-bios-ipmi-src/140721d/testr_results.html | 15:12 |
| dtantsur | yeah, but it shouldn't be necessary if we configure the jobs right | 15:13 |
| dtantsur | so you're hiding a wrong configuration | 15:13 |
| dtantsur | maybe it's not worth bothering with since it's a stable branch.. | 15:13 |
| TheJulia | we've had this come up all over the place | 15:13 |
| TheJulia | so its pervasive or it is broken... | 15:13 |
| dtantsur | yeah, but we never changed the regex | 15:13 |
| dtantsur | we have skip logic in the plugin | 15:13 |
| TheJulia | umm.... I'm fairly sure we've updated the regex | 15:13 |
| TheJulia | but, I'll go look at the plugin | 15:14 |
| JayF | I remember that too, but as an aside | 15:14 |
| JayF | Zed is released! Congrats everyone on #18 for Ironic | 15:14 |
| dtantsur | \o/ | 15:14 |
| JayF | 9 years of Ironic as part of openstack | 15:14 |
| JayF | we need to have a decade celly for the b release lol | 15:14 |
| dtantsur | true | 15:14 |
| TheJulia | so the job's skip logic is if it is only configured, in this case it *is* configured | 15:18 |
| * dtantsur is wondering if we didn't backport something to youga | 15:18 | |
| dtantsur | yoga | 15:18 |
| TheJulia | oookay | 15:22 |
| TheJulia | I see what is going on | 15:22 |
| TheJulia | for some reason the job is in UEFI mode | 15:22 |
| TheJulia | and this is one of the cases where we're running the same test needlessly still | 15:22 |
| dtantsur | it's the default, yeah | 15:22 |
| TheJulia | but the job is bios... | 15:22 |
| dtantsur | I think I fixed it in master, but likely no yoga | 15:22 |
| TheJulia | and the artifact is bios based | 15:22 |
| TheJulia | so when it goes to boot, it can't | 15:22 |
| TheJulia | since its wholedisk image is cirros | 15:22 |
| dtantsur | right | 15:23 |
| TheJulia | yup | 15:23 |
| TheJulia | riccardo set it to bios in update jobs names 6 months ago | 15:23 |
| opendevreview | Julia Kreger proposed openstack/ironic-lib stable/yoga: CI: Zuul no longer respects queue param https://review.opendev.org/c/openstack/ironic-lib/+/860172 | 15:28 |
| TheJulia | JayF: https://review.opendev.org/c/openstack/ironic/+/858903 I +2'ed this, fwiw | 15:39 |
| JayF | I backported it; you wrote the original change; why does it matter if I workflow it? | 15:39 |
| JayF | like, having someone else entirely workflow it makes sense | 15:40 |
| TheJulia | exactly | 15:40 |
| JayF | but we're both attached to that PR | 15:40 |
| TheJulia | yup | 15:40 |
| * TheJulia shrugs | 15:40 | |
| TheJulia | I'd likely just workflow it to be honest, and if there are no objections I'm happy to do so | 15:40 |
| * JayF is OK with it | 15:40 | |
| * TheJulia has a sudden urge to channel a pirate | 15:41 | |
| JayF | I don't think I pay for the pirate channel with my cable :P | 15:42 |
| TheJulia | it would be a disney channel where the pirates of the carrabian movies just loop | 15:42 |
| arne_wiebalck | with auto-registration enabled, there can be situations where nodes booting up are registered which should not (e.g. test samples) ... I don't think Ironic has anyway to configure the node_not_found hook for a subset of nodes somehow ... has it? | 15:43 |
| TheJulia | It would solve the need for "Arrrr!" | 15:43 |
| arne_wiebalck | (where the subset is nodes unknown to Ironic) | 15:43 |
| TheJulia | arne_wiebalck: it does not... but if there is a good way to filter them out by identifying them, i guess it would make sense to carry it | 15:43 |
| TheJulia | node_not_found_hook is just some code too.... | 15:44 |
| * TheJulia thinks the parameter is the method name... | 15:44 | |
| * TheJulia may also be wrong | 15:44 | |
| arne_wiebalck | TheJulia: thanks ... yeah, I was thinking to have some exclude list based on criteria ... (vague alert!) | 15:44 |
| arne_wiebalck | TheJulia: like, before the hook is called, the exclude regex is checked ... sth along these lines | 15:45 |
| TheJulia | I could see it making a lot of sense | 15:45 |
| TheJulia | And coming from an operator, it would definitely make much more sense | 15:45 |
| arne_wiebalck | I am hesitant to have some auto clean up :-D | 15:45 |
| TheJulia | vauge being permitted there as long as docs exist in the patch ;) | 15:45 |
| arne_wiebalck | heh | 15:45 |
| arne_wiebalck | ok, we will try to come up with a proposal ... thanks! | 15:46 |
| TheJulia | easiest code review of the day: https://review.opendev.org/c/openstack/ironic-inspector/+/860278 | 15:47 |
| TheJulia | bahahaha I just found a race condition | 16:00 |
| JayF | In our tests? | 16:10 |
| TheJulia | I *suspect* it is an artifact of how sqlalchemy works | 16:10 |
| TheJulia | I got interrupted by a manager needing insight on how to do something in jira | 16:10 |
| TheJulia | minor bug + how sqlalchemy works | 16:22 |
| TheJulia | oh, well bad test actually I think | 16:46 |
| opendevreview | Merged openstack/ironic stable/yoga: Redfish: Consider password part of the session cache https://review.opendev.org/c/openstack/ironic/+/858903 | 17:18 |
| opendevreview | Merged openstack/ironic-inspector master: Add missing space to configuration help message https://review.opendev.org/c/openstack/ironic-inspector/+/860278 | 17:18 |
| JayF | https://review.opendev.org/c/openstack/ironic-inspector/+/860163 needs someone other than julia or I to land it, ideally | 18:17 |
| JayF | https://review.opendev.org/c/openstack/ironic/+/860281 is in dire need of review from vanou and others who are knowledgable on irmc | 18:50 |
| JayF | otherwise I'll need to revert the irmc fixes out to perform a stable release | 18:50 |
| TheJulia | I can email the folks at FJ | 18:51 |
| JayF | please do; feel free to CC: my jay at jvf dot cc address? | 18:51 |
| TheJulia | I won't be able to do that, it will be on another thread with their management folks | 18:52 |
| TheJulia | downstream thread, that is | 18:52 |
| JayF | ack | 18:52 |
| TheJulia | sent | 18:58 |
| * TheJulia goes back to the horror of sqlalchemy | 19:00 | |
| TheJulia | JayF: fyi, your not going to get anything back until likely early next week. It is a holiday this week in China | 19:17 |
| TheJulia | I think I'm going to likely have to swap out to enginefacade | 19:18 |
| JayF | acknowledge | 19:19 |
| TheJulia | well, at least I'm making good progress on enginefacading it | 20:29 |
| TheJulia | 91 failing tests, hundreds passing | 20:29 |
| JayF | In anticipation of retiring branches next week; I'm going to start with rocky/stein/train and investigating and abandoning patches there since there was already agreement to retire those at some point in the past | 20:58 |
| TheJulia | sounds good to me | 21:37 |
| opendevreview | Julia Kreger proposed openstack/ironic-inspector master: WIP: Use declarative reader/writer separation https://review.opendev.org/c/openstack/ironic-inspector/+/463768 | 21:52 |
| TheJulia | well, 2 foreign key index errors | 21:52 |
| TheJulia | but overall, *much* better | 21:52 |
| JayF | nice, thank you | 22:12 |
| ashinclouds[m] | And you know, it looks like we actually really test then… | 22:12 |
| ashinclouds[m] | Anyway, more tomorrow | 22:12 |
| janders | good morning Ironic o/ | 23:05 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!