Thursday, 2022-01-27

NobodyCammorning janders o/00:10
TheJuliao/ janders 00:32
TheJuliahow goes it?00:32
jandershey NobodyCam TheJulia o/00:38
jandersbit hectic - 4.10 code freeze in 2 days, but we're managing! :)00:39
opendevreviewVanou Ishii proposed openstack/ironic master: Fix failure of create_vfat_image function  https://review.opendev.org/c/openstack/ironic/+/82657603:09
opendevreviewVanou Ishii proposed openstack/ironic master: Fix failure of create_vfat_image function  https://review.opendev.org/c/openstack/ironic/+/82657603:55
arne_wiebalckGood morning, Ironic!07:28
arne_wiebalckdtantsur: after some back and forth, https://review.opendev.org/c/openstack/ironic-python-agent/+/826026 is back in the (Zuul) game ... would be great if we could get this in, then do i) the refactor and ii) the backports (if we decide to do it)08:11
rpittaugood morning ironic! o/08:26
rpittauarne_wiebalck: I had a look and it looks ok, also as backport-candidate, and I 100% agree with dtantsur about the extensions.image import, need to revisit something there09:13
arne_wiebalckrpittau: thanks09:51
arne_wiebalckrpittau:  re extensions.image, I was thinking you would notice this after you have done all the efi/raid refactoring09:52
arne_wiebalckrpittau: I propose to do this in a followup09:52
rpittauheh I have to get back to that, this week is a bit busy :/10:14
dtantsurmorning folks10:27
dtantsurarne_wiebalck: re https://storyboard.openstack.org/#!/story/2009805: batching is what many ironic consumers do, I'm aware of metal3 and tripleo.10:28
dtantsurso yeah, totally +1 on that10:28
dtantsur(oh, and ironic-inspector has batching too)10:29
dtantsurnow, failing vs delaying is a good question, and may even be an option for operators10:29
arne_wiebalckdtantsur: thanks for checking10:29
dtantsurarne_wiebalck: the RFE as it is written now needs technical details or even a spec10:30
arne_wiebalckdtantsur: yes10:30
arne_wiebalckdtantsur: if someone has ideas on *how* to do it, I may be able to a way to have it done :)10:31
arne_wiebalck*to find a way10:31
arne_wiebalckdtantsur: unless we come up with a plan how to do it until then, we could also briefly discuss this at the ptg10:51
dtantsurarne_wiebalck: we could add a new DB API, something like, nodes_in_status_count(['cleaning', 'clean wait'])10:52
arne_wiebalckdtantsur: so, upon a new clean request, we would call this and then fail or stall (somehow) ?10:56
iurygregorygood morning Ironic10:58
arne_wiebalckhey iurygregory o/10:58
iurygregoryarne_wiebalck, o/10:58
rpittauhey iurygregory dtantsur :)10:59
dtantsurarne_wiebalck: yeah. how to stall is a good question.11:00
dtantsurarne_wiebalck: we *might* need new state machines states. like 'clean delayed' or something11:00
arne_wiebalckdtantsur: what do clients do when they get a 503 with a retry header?11:04
* arne_wiebalck is trying to outsource the problem :)11:04
dtantsurarne_wiebalck: depends on the client? I don't think it's a good fit here11:04
arne_wiebalckdtantsur: ok11:04
dtantsurcleaning takes from minutes to hours, no client will retry that long11:04
dtantsurwe'll also need to solve the issue with automated cleaning11:05
dtantsurwe cannot really refuse deprovisioning11:05
arne_wiebalckright, we need to distinguish these11:05
dtantsurthis is why I'm thinking of a new state11:06
dtantsura node can be "parked" in "clean delayed"/"inspect delayed" until it has a slot11:06
arne_wiebalckyes11:07
dtantsurwhich can be checked in a periodic task on conductor11:07
dtantsurit will also be easy for an operator to understand why their nodes are not booting into the ramdisk11:07
arne_wiebalcka state would also be sth which an operator would understand11:07
arne_wiebalck:-D11:07
arne_wiebalckI will add our thoughts to the story11:08
dtantsurarne_wiebalck: another question: do we want to treat all booting steps (cleaning, deploy, inspection) together or separately?11:09
arne_wiebalckdtantsur: the initial motivation was to prevent irreversible damage11:10
arne_wiebalckdtantsur: therefore cleaning came to mind first, deploying is equally "dangerous", inspection should be less damaging I would think11:11
arne_wiebalckdtantsur: along those lines, what about deleting nodes from Ironic?11:12
dtantsurplease elaborate, wdym by damage?11:12
arne_wiebalckdtantsur: remove instances, erase disks, ...11:12
* arne_wiebalck goes for a quick lunch11:12
dtantsurarne_wiebalck: I'm not sure I'm following. even with batching, the damage will still be done.11:12
dtantsurI was under assumptions you're trying to prevent a lot of PXE bootings from saturating your network11:13
arne_wiebalckdtantsur: right, there will still be damage, but it will be limited or can be stopped; if Ironic accepts 10k clean reqs and boots 10k nodes into cleaning, all instances could be gone in minutes; if you batch this into 100 max at any moment, you will have a chance to step in and stop it11:32
dtantsurhmm, interesting11:32
arne_wiebalckdtantsur: not saturating or limiting the network is an additional selling point I have not thought about :)11:34
*** sshnaidm|afk is now known as sshnaidm11:35
opendevreviewAmeya Raut proposed openstack/ironic-tempest-plugin master: WIP : Add iDRAC management cleaning steps tests  https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/82664611:52
opendevreviewRiccardo Pittau proposed openstack/bifrost master: Enable epel repository only when needed  https://review.opendev.org/c/openstack/bifrost/+/82644211:58
opendevreviewRiccardo Pittau proposed openstack/bifrost master: Enable epel repository only when needed  https://review.opendev.org/c/openstack/bifrost/+/82644213:40
dtantsurFYI https://review.opendev.org/c/openstack/glance/+/82668414:28
TheJulia++14:30
opendevreviewAija Jauntēva proposed openstack/ironic master: Fix DevStack plugin ipxe-snponly-x86_64.efi name  https://review.opendev.org/c/openstack/ironic/+/82669415:02
opendevreviewAmeya Raut proposed openstack/ironic-tempest-plugin master: Add iDRAC management cleaning steps tests  https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/82664615:03
arne_wiebalckdtantsur: if you could give https://review.opendev.org/c/openstack/ironic-python-agent/+/826026 another thumps-up, I would start looking into backports (and refactor), so that we can finally close that chapter :)15:30
dtantsurlooking15:32
dtantsuryep, +A15:33
arne_wiebalckthanks!15:34
opendevreviewDmitry Tantsur proposed openstack/ironic master: [WIP] Explicit parameter to distinguish partition/whole-disk images  https://review.opendev.org/c/openstack/ironic/+/82530515:37
TheJuliaso random thought, w/r/t power arch. I wonder if it might be time to deprecate PReP partition support16:21
dtantsurTheJulia: is it not needed any more?16:31
TheJuliaI don't think so, I think the latest generation no longer supports it16:31
TheJuliaI *think*16:31
dtantsurwould be good to get a confirmation16:32
TheJuliayeah16:32
TheJuliaoh where oh where is that email address16:33
dtantsurI definitely don't mind removing so much code that none of us can even test16:36
rpittaugood night! o/17:06
TheJuliadtantsur, I've copied you on an email to mike who authored it originally17:07
dtantsuryep, thanks!17:09
dtantsursee you tomorrow folks o/17:09
opendevreviewMerged openstack/ironic-python-agent master: SoftwareRAID: Use efibootmgr (and drop grub2-install)  https://review.opendev.org/c/openstack/ironic-python-agent/+/82602617:13
*** pmannidi is now known as pmannidi|Gone17:29
arne_wiebalck\o/ (this should hopefully mark the end of grub2-install for s/w RAID and makes it use the same paths as any other UEFI deployment)17:33
TheJuliathe world will be a better place17:35
arne_wiebalck:-D17:37
iurygregoryarne_wiebalck, \o/17:48
arne_wiebalckiurygregory: :)17:48
iurygregorythe wolrd will be a better place and we are one step closer to take over the galaxy :D17:48
arne_wiebalckbye everyone, see you tomorrow o/17:49
opendevreviewJulia Kreger proposed openstack/ironic-tempest-plugin master: WIP: An idea for rbac positive/negative testing  https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/81916517:51
iurygregorybye arne_wiebalck o/17:51
TheJuliastevebaker[m]: hjensas: w/r/t that bz yesterday, it magically worked this morning...18:12
*** sshnaidm is now known as sshnaidm|afk18:13
hjensasTheJulia: that's good news for sanity.18:23
TheJuliahjensas: yeah18:24
TheJuliaunfortunately it sunk two days of my brain thinking I was absolutely starting to loose it18:25
opendevreviewMerged openstack/ironic master: Fix DevStack plugin ipxe-snponly-x86_64.efi name  https://review.opendev.org/c/openstack/ironic/+/82669418:26
hjensasTheJulia: feel free to redirect the rear stuff to me in case they raise something. I know you have a lot on your plate.18:31
TheJuliahjensas: thanks, I looked at it mainly because I remembered talking to them when they developed the feature originally18:33
TheJuliaand kind of kept going "what?!?"18:33
TheJuliastevebaker[m]: I'm starting to think ddt....19:04
TheJuliastevebaker[m]: which would allow us to fork lift a lot out of ironic today, into ironic-tempest-plugin and remodel it I guess19:20
TheJuliathe question ebcomes how to model "this should work, but not that and that, but then this other case should also work"19:20
stevebaker[m]Yeah ddt is great if you know up front that it can implement your most complex test19:33
TheJuliathe whole unknown is how to represent that matrix I guess19:41
NobodyCamGood AfterNoon Ironic'ers20:28
NobodyCamTheJulia: I wanted to check with you on https://review.opendev.org/c/openstack/nova/+/813897 ... looks to be DOA?20:29
iurygregorygood afternoon NobodyCam o/20:31
NobodyCamhey hey iurygregory :) happy almost Friday20:32
iurygregoryhappy almost Friday \o/20:32
NobodyCamheheheh20:32
NobodyCamglad to back? saw some great photos on "the" twitter 20:33
iurygregorythe PTO was quite good :D20:39
iurygregoryenergy restored =)20:39
NobodyCam\o/20:41
TheJuliaNobodyCam: ENOTIME21:12
NobodyCamTheJulia: +++ completely understand, wanted to check21:13
NobodyCamHope things are going well today!21:13
TheJulialots of long discussions21:13
NobodyCam:)21:13
TheJuliaNobodyCam: if you feel that would solve your issue, I would highly recommend commenting and the version you would love to see a fix in21:59
NobodyCam++22:00
TheJuliaGranted, it may be an EM version, often these are the sort of bugs we just go ahead and backport through em versions to older versions we're supporting for long term support22:00

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!