Tuesday, 2021-05-18

NobodyCamit's a little hack and slash but something kinda like:00:02
NobodyCamhttps://www.irccloud.com/pastebin/XtQf24AV/00:02
*** uzumaki has joined #openstack-ironic01:23
*** gyee has quit IRC01:31
*** k_mouza has joined #openstack-ironic01:58
*** k_mouza has quit IRC01:58
*** rloo has quit IRC02:01
*** mkrai has joined #openstack-ironic02:36
*** sdanni has quit IRC03:46
*** paras333_ has joined #openstack-ironic03:50
*** paras333_ has quit IRC03:55
*** zaneb has quit IRC04:07
*** zaneb has joined #openstack-ironic04:07
*** uzumaki has quit IRC04:45
*** uzumaki has joined #openstack-ironic05:50
*** uzumaki has quit IRC06:01
*** jawad_axd has joined #openstack-ironic06:13
arne_wiebalckGood morning, ironic!06:39
iurygregorygood morning arne_wiebalck and Ironic o/06:47
arne_wiebalckhey iurygregory o/06:51
*** rpittau|afk is now known as rpittau07:27
rpittaugood morning ironic! o/07:27
*** mkrai has quit IRC07:29
iurygregorywelcome back rpittau o/07:33
arne_wiebalckhey rpittau o/07:33
rpittauhey iurygregory :)07:33
iurygregoryhow was the PTO? =)07:33
*** dougsz has joined #openstack-ironic07:40
rpittauit was good, the weather has been nice besides one little storm last friday and strong wind all over the weekend :)07:44
*** tosky has joined #openstack-ironic07:46
*** ociuhandu has joined #openstack-ironic07:49
iurygregorywow07:52
*** derekh has joined #openstack-ironic07:59
*** k_mouza has joined #openstack-ironic07:59
*** k_mouza has quit IRC08:04
*** lucasagomes has joined #openstack-ironic08:05
openstackgerritRiccardo Pittau proposed openstack/ironic-python-agent-builder master: Update requirements for diskimage-builder  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/79190208:09
jandershey arne_wiebalck iurygregory rpittau and Ironic o/08:22
rpittauhey janders :)08:22
arne_wiebalckhey janders o/08:23
iurygregoryhey janders o/08:23
openstackgerritRiccardo Pittau proposed openstack/sushy master: [trivial] add wallaby versions to release notes  https://review.opendev.org/c/openstack/sushy/+/79190408:25
openstackgerritMerged openstack/python-ironicclient master: setup.cfg: Replace dashes with underscores  https://review.opendev.org/c/openstack/python-ironicclient/+/79107808:42
*** ociuhandu has quit IRC08:55
*** ociuhandu has joined #openstack-ironic08:56
*** ociuhandu has quit IRC08:57
*** ociuhandu has joined #openstack-ironic08:57
*** strigazi has joined #openstack-ironic08:59
*** tkajinam has quit IRC09:02
*** mkrai has joined #openstack-ironic09:02
*** tkajinam has joined #openstack-ironic09:03
openstackgerritMerged openstack/sushy-tools master: setup.cfg: Replace dashes with underscores  https://review.opendev.org/c/openstack/sushy-tools/+/79139209:07
openstackgerritRiccardo Pittau proposed openstack/ironic master: Update min version of tox to use allowlist  https://review.opendev.org/c/openstack/ironic/+/79190909:15
*** alexmcleod has joined #openstack-ironic09:35
jandersoops... I my house lost power (likely due to heavy winds in the area). Running on whatever phone battery I had, will likely drop out soon09:40
iurygregoryjanders, good luck =)09:56
*** ociuhandu has quit IRC09:56
*** ociuhandu has joined #openstack-ironic09:57
*** ociuhandu has quit IRC09:58
*** ociuhandu has joined #openstack-ironic09:58
*** dtantsur|afk is now known as dtantsur10:14
dtantsurmorning/afternoon ironic10:14
iurygregorygood afternoon dtantsur o/10:15
rpittauhey dtantsur :)10:15
*** k_mouza has joined #openstack-ironic10:20
*** Qianbiao has joined #openstack-ironic10:24
*** MahnoorAsghar has joined #openstack-ironic10:40
*** dougsz has quit IRC10:54
*** ociuhandu has quit IRC10:56
*** ociuhandu has joined #openstack-ironic10:57
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Do not serialize command_params  https://review.opendev.org/c/openstack/ironic-python-agent/+/79179810:59
*** mkrai_ has joined #openstack-ironic10:59
*** mkrai has quit IRC11:00
*** ociuhandu has quit IRC11:00
*** ociuhandu has joined #openstack-ironic11:00
*** ociuhandu has quit IRC11:00
*** ociuhandu has joined #openstack-ironic11:01
*** ociuhandu has quit IRC11:02
*** ociuhandu has joined #openstack-ironic11:02
*** ociuhandu has quit IRC11:02
*** ociuhandu has joined #openstack-ironic11:03
*** ociuhandu has quit IRC11:08
*** dougsz has joined #openstack-ironic11:09
openstackgerritMerged openstack/ironic master: Rework the user guide  https://review.opendev.org/c/openstack/ironic/+/79038711:13
MahnoorAsgharHey guys!11:23
*** mkrai_ has quit IRC11:23
*** ociuhandu has joined #openstack-ironic11:23
*** ociuhandu has quit IRC11:25
*** ociuhandu has joined #openstack-ironic11:25
MahnoorAsghar:dtantsur I was looking at my PR in the metal3 docs, and wanted to ask you about a comment. Do you have a moment?11:27
dtantsursure11:27
MahnoorAsgharYou proposed an idea - making the ironic drivers support disk numbers in addition to full names. It seems like a great approach11:28
MahnoorAsghar(https://github.com/metal3-io/metal3-docs/pull/148#discussion_r633644800)11:28
MahnoorAsgharDo you think it would be a feasible way to go about it?11:29
dtantsurYeah, it seems so11:30
MahnoorAsgharShall I make another RFE for review, or update the same one?11:33
dtantsuras you find convenient11:33
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Stop accepting duplicated configdrive  https://review.opendev.org/c/openstack/ironic-python-agent/+/79047111:34
*** Qianbiao has quit IRC11:34
MahnoorAsghardtantsur: I will update the same RFE then. thanks ^-^11:35
*** ociuhandu has quit IRC11:36
*** ociuhandu has joined #openstack-ironic11:37
*** ociuhandu has quit IRC11:42
*** ociuhandu has joined #openstack-ironic11:45
openstackgerritMerged openstack/ironic stable/wallaby: Delete unavailable py2 package  https://review.opendev.org/c/openstack/ironic/+/79043911:46
*** ociuhandu has quit IRC11:51
*** ociuhandu has joined #openstack-ironic12:04
openstackgerritDmitry Tantsur proposed openstack/ironic-lib master: Raise qemu-img memory limit to 2 GiB  https://review.opendev.org/c/openstack/ironic-lib/+/79193212:23
*** ociuhandu has quit IRC12:27
*** ociuhandu has joined #openstack-ironic12:28
*** ociuhandu has quit IRC12:33
*** ociuhandu has joined #openstack-ironic12:39
iurygregorydtantsur, question about allocation API, if I want to create an allocation for a specific node I can just set candidate_nodes [node_uuid]  right?12:42
dtantsuriurygregory: yep12:42
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Enable out-of-order writes when writing whole disk images  https://review.opendev.org/c/openstack/ironic-python-agent/+/79193412:42
dtantsurTheJulia: dunno if you have experience with ^^ but seems reasonable per docs12:42
iurygregorydtantsur, I think if found why we saw the wrong secure boot in other node :D12:42
*** alexmcleod has quit IRC12:42
dtantsurmmm?12:42
TheJuliadtantsur: depending on the contents, it may not be ideal, but for a clean image I guess it would be okay12:43
iurygregoryin all 3 allocations we are setting candidate_nodes to all 3 masters12:43
dtantsurTheJulia: morning! The documentation says "permit out-of-order writes", so I assume qemu-img does the right thing?12:43
dtantsurI don't have a way to test it, but I remember somebody complaining on the meetup about writing really large sparse whole disk iamges12:43
TheJuliadtantsur: well, qemu-img tries to do a sequential write, but that is less of a big deal with solid state storage12:44
TheJuliastill helps from the standpoint of appending blocks together in io operations, but qemu buffers and tries to build a later sequential write12:44
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Rewrite write_image.sh in Python  https://review.opendev.org/c/openstack/ironic-python-agent/+/79193512:45
dtantsuralso, finally ^^^12:45
TheJuliareally large sparse whole disk images should efficiently write...  unless things are not in order. The QCOW2 format is also evil in that regard because it looks for free space to inject stuff instead of just growing the file12:45
TheJuliahence out of order12:46
*** alexmcleod has joined #openstack-ironic12:46
dtantsuryep12:46
dtantsurthat could be their problem12:46
TheJuliaLikely, if they worked on it as a vm12:46
dtantsurthat's a common way to build whole disk images, as it seems12:47
dtantsurlike, more command than I assumed12:47
dtantsursigh, which command12:47
dtantsurcommon12:47
TheJuliaone way, unfortuantely12:48
dtantsurwell, some people do use DIB :)12:48
TheJuliaand some people just use the vendor images out there and who knows how those have been created12:49
dtantsurwell, RH images are creating using kickstart in a VM12:49
TheJuliado we know of *anyone* using postgres with ironic?12:51
dtantsurnot personally12:51
dtantsuriurygregory: O___O12:51
dtantsurthis is... bad. it only hasn't blown up so far because masters are pretty identical..12:52
*** uzumaki has joined #openstack-ironic12:52
openstackgerritBob Fournier proposed openstack/ironic master: Include bios registry fields in bios API  https://review.opendev.org/c/openstack/ironic/+/78670712:52
openstackgerritBob Fournier proposed openstack/ironic master: Retrieve BIOS registry from sushy  https://review.opendev.org/c/openstack/ironic/+/79195612:53
TheJuliai ask about postgres because the utilities to get indexes in oslo_db seem broken12:53
dtantsurspeaking of allocation, https://review.opendev.org/c/openstack/ironic/+/790396 will give us a much nicer user guide #winkwink #notasalespitchatall12:53
dtantsurTheJulia: not wow :(12:54
TheJuliaI'm not going to burn anymore time on it, aside from just report the bug and note we'll only be super kind on mysql deployments12:54
dtantsurI guess this is fair12:55
openstackgerritDmitry Tantsur proposed openstack/ironic-lib master: convert_image: add flags required for IPA  https://review.opendev.org/c/openstack/ironic-lib/+/79195712:55
iurygregorydtantsur, at least seems possible to me "master-0" has allocation_id 1 (this allocation has "name":"master-2" and 3 candidate nodes :D12:55
dtantsurthat's a mess12:56
iurygregorymaster-2 has allocation_id 2 and the allocation has "master-0" in the name and all 3 candidate nodes12:56
dtantsurwe probably shouldn't even use allocations if we need 1-1 mapping12:56
TheJuliaerr, I just said I wasn't going to, looks like a sqlalchemy issue12:57
iurygregoryworkaround would be have a specific allocation for each node (which doesn't make much sense I think?)12:57
*** rloo has joined #openstack-ironic12:58
openstackgerritBob Fournier proposed openstack/ironic master: Retrieve BIOS registry from sushy  https://review.opendev.org/c/openstack/ironic/+/79195613:02
dtantsuriurygregory: yeah13:02
dtantsurmaybe just stop using allocations?13:02
iurygregorylet's burn with fire :D13:02
dtantsur\o/13:03
*** MahnoorAsghar has quit IRC13:04
arne_wiebalckTheJulia: Our DB colleagues are looking into what Ironic is doing on the database. I pointed them also to your post, which brings optimisations we do not run in our deployment yet.13:05
arne_wiebalckTheJulia: They have set up some performance monitoring tooling to analyze the queries.13:06
arne_wiebalckTheJulia: But given the code will change with your work, I am not sure how relevant the current data is.13:06
arne_wiebalckTheJulia: I was also thinking about experiments, like switching off power sync for an hour, to see the impact on the DB to pin-point what to tackle.13:07
arne_wiebalckTheJulia: So, if you have thoughts/ideas on what to try or how to proceed, let me know.13:07
openstackgerritJulia Kreger proposed openstack/ironic master: Add additional node indexes  https://review.opendev.org/c/openstack/ironic/+/78862513:13
arne_wiebalckheh13:25
TheJuliaI don't know, it would be interesting to see how much power sync impacts honestly. We've looked at that before wondering what could we do to try and make things better there, but that is a really hard "nut to crack"13:26
arne_wiebalckRight. I think it would be nice to see how much power sync is doing13:27
arne_wiebalcklike, switch it off and observe13:27
TheJuliaAt least at current, your mostly an ipmi shop if memory serves13:27
TheJuliawhich means super cpu intensive13:27
arne_wiebalckyes13:27
* TheJulia hopes her manual index check code works on postgres13:28
TheJulia(which is modeled on the ye-olde tenant_id->project_id changes13:29
TheJulia)13:29
*** ociuhandu has quit IRC13:29
*** tzumainn has joined #openstack-ironic13:34
*** mnasiadka has quit IRC13:35
openstackgerritRiccardo Pittau proposed openstack/sushy-cli master: Retire sushy-cli - Step 2  https://review.opendev.org/c/openstack/sushy-cli/+/79196613:36
*** mnasiadka has joined #openstack-ironic13:37
*** coreycb has quit IRC13:37
*** rh-jlabarre has quit IRC13:37
*** rh-jlabarre has joined #openstack-ironic13:37
*** ociuhandu has joined #openstack-ironic13:38
*** coreycb has joined #openstack-ironic13:38
openstackgerritJulia Kreger proposed openstack/ironic master: Add additional node indexes  https://review.opendev.org/c/openstack/ironic/+/78862513:39
TheJuliaI gave it my best to avoid making postgres admins do more work :(13:41
*** paras333_ has joined #openstack-ironic13:45
TheJuliawell, postgres servers. On a plus side, the index should still build quickly13:45
*** timeu has quit IRC13:54
*** timeu has joined #openstack-ironic13:54
openstackgerritJulia Kreger proposed openstack/ironic master: Add additional node indexes  https://review.opendev.org/c/openstack/ironic/+/78862513:55
TheJuliaokay, it passes all the database unit tests just fine now. Updated docs and reno to reflect greater mysql friendlyness13:55
*** openstack has joined #openstack-ironic14:09
*** ChanServ sets mode: +o openstack14:09
*** uzumaki has quit IRC14:10
*** openstackstatus has joined #openstack-ironic14:11
*** ChanServ sets mode: +v openstackstatus14:11
*** k_mouza has quit IRC14:14
*** k_mouza_ has joined #openstack-ironic14:14
*** Qianbiao has joined #openstack-ironic14:20
openstackgerritDmitry Tantsur proposed openstack/ironic master: Make configdrive in instance_info even if it's a dict  https://review.opendev.org/c/openstack/ironic/+/79198714:33
*** k_mouza_ has quit IRC14:34
*** k_mouza has joined #openstack-ironic14:34
NobodyCamGood morning ironic’ers14:35
rpiosoGood morning :)14:36
rpiosoNobodyCam: o/14:36
TheJuliagood morning!14:36
iurygregorymorning NobodyCam rpioso o/14:36
NobodyCamGood morning14:37
openstackgerritDmitry Tantsur proposed openstack/ironic master: Delay rendering configdrive  https://review.opendev.org/c/openstack/ironic/+/79047214:37
*** ricolin has quit IRC14:41
rpiosoiurygregory, TheJulia: Hey!14:43
*** lmcgann has joined #openstack-ironic14:46
openstackgerritMerged openstack/ironic-python-agent master: Do not serialize command_params  https://review.opendev.org/c/openstack/ironic-python-agent/+/79179814:58
*** jamesdenton has joined #openstack-ironic14:59
*** jamesden_ has quit IRC15:02
openstackgerritDmitry Tantsur proposed openstack/ironic master: Delay rendering configdrive  https://review.opendev.org/c/openstack/ironic/+/79047215:02
*** Qianbiao has quit IRC15:19
*** zul has joined #openstack-ironic15:22
*** k_mouza has quit IRC15:25
*** k_mouza has joined #openstack-ironic15:25
*** thiagop has joined #openstack-ironic15:29
*** jamesden_ has joined #openstack-ironic15:33
*** ociuhandu has quit IRC15:34
JayFAre we still doing the review jams (next one in ~1 hour 24 minutes)? If so, is there a place folks queue up patches for it? If so I'll add it to the whiteboard15:35
*** jamesdenton has quit IRC15:36
*** ociuhandu has joined #openstack-ironic15:40
*** ociuhandu has quit IRC15:45
*** tosky_ has joined #openstack-ironic15:49
*** tosky has quit IRC15:52
*** ociuhandu has joined #openstack-ironic15:55
TheJuliaJayF: add items to the agenda on the meetings page15:59
JayFack, makes sense15:59
TheJuliahttps://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_next_meeting15:59
JayFI'll put that info in the whiteboard15:59
TheJuliadown under current agenda15:59
TheJuliaat the very bottom15:59
TheJuliayay for the wiki anchors linking not doing what is expected16:00
TheJulia(and also I load that url fairly often)16:00
*** ociuhandu has quit IRC16:00
*** lucasagomes has quit IRC16:02
*** ociuhandu has joined #openstack-ironic16:11
*** jamesdenton has joined #openstack-ironic16:12
*** ociuhandu has quit IRC16:14
*** ociuhandu has joined #openstack-ironic16:14
openstackgerritDmitry Tantsur proposed openstack/ironic master: Trivial: fix unfinished message in iDRAC boot  https://review.opendev.org/c/openstack/ironic/+/79200716:14
dtantsuriurygregory: ^^16:14
*** ociuhandu has quit IRC16:14
*** jamesden_ has quit IRC16:15
iurygregorydtantsur, ack16:15
*** jamesdenton has quit IRC16:18
*** ociuhandu has joined #openstack-ironic16:19
*** ociuhandu has quit IRC16:20
arne_wiebalckbye everyone o/16:24
*** jamesdenton has joined #openstack-ironic16:24
*** dougsz has quit IRC16:28
*** ociuhandu has joined #openstack-ironic16:30
iurygregorybye arne_wiebalck o/16:32
*** ociuhandu has quit IRC16:36
*** k_mouza has quit IRC16:41
*** ociuhandu has joined #openstack-ironic16:42
*** k_mouza has joined #openstack-ironic16:44
rpittaubye everyone! o/16:45
*** rpittau is now known as rpittau|afk16:45
*** k_mouza has quit IRC16:45
*** k_mouza has joined #openstack-ironic16:46
*** ociuhandu has quit IRC16:49
*** ociuhandu has joined #openstack-ironic16:49
TheJuliaJayF: o/17:02
JayFThere was nothing on the agenda, I thought?17:02
*** thiagop has quit IRC17:02
*** derekh has quit IRC17:05
*** ociuhandu has quit IRC17:14
openstackgerritBob Fournier proposed openstack/ironic master: Retrieve BIOS registry from sushy  https://review.opendev.org/c/openstack/ironic/+/79195617:22
openstackgerritBob Fournier proposed openstack/ironic master: Include bios registry fields in bios API  https://review.opendev.org/c/openstack/ironic/+/78670717:37
openstackgerritBob Fournier proposed openstack/ironic master: Retrieve BIOS registry from sushy  https://review.opendev.org/c/openstack/ironic/+/79195617:40
*** dtantsur is now known as dtantsur|afk17:43
dtantsur|afko/17:43
NobodyCamanyone know a clean way to tell when placement has finished updating for an Ironic node? hitting a situation where ironic and nova say the node is free but placement is lagging behind18:03
NobodyCamNight to all that have signed off18:07
*** rloo has quit IRC18:19
*** rloo has joined #openstack-ironic18:21
TheJuliaNobodyCam: not that I've figured out or really looked into18:23
NobodyCam++ wish Jay had been able to do: https://review.opendev.org/c/openstack/nova-specs/+/423872/18:24
TheJuliaso part of the conundrum is nova-compute has a lock that holds up a huge chunk of activity while it is looping through18:25
TheJuliaso it may be past that point, but it is working on a thousand nodes, and that whole interaction is locked18:26
NobodyCam++ I'm needing to notify a scheduling system that runs above Openstack when the node is actually usable again18:28
TheJuliaI mean if placement is not the source of scheduling truth...18:32
TheJuliayou could always just move the lock down to the node level and syncronize one at a time18:32
*** k_mouza has quit IRC19:13
*** k_mouza has joined #openstack-ironic19:14
*** k_mouza has quit IRC19:14
*** tosky_ is now known as tosky19:25
*** rloo has quit IRC19:37
*** rloo has joined #openstack-ironic19:39
*** ociuhandu has joined #openstack-ironic20:27
*** k_mouza has joined #openstack-ironic20:29
*** k_mouza has quit IRC20:33
*** ociuhandu has quit IRC20:35
*** k_mouza has joined #openstack-ironic20:59
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic-specs master: Event Subscription Spec  https://review.opendev.org/c/openstack/ironic-specs/+/78574220:59
*** k_mouza has quit IRC21:03
*** Paras has quit IRC21:25
*** rloo has quit IRC22:10
*** dustinc has joined #openstack-ironic22:13
TheJuliarofl, I have sqlalchemy generating the right sql, *but* whenI let it run.... I get a fully populated node :(22:35
*** rcernin has joined #openstack-ironic22:36
*** uzumaki has joined #openstack-ironic22:44
*** rcernin has quit IRC22:44
*** rcernin has joined #openstack-ironic22:45
*** k_mouza has joined #openstack-ironic22:59
*** k_mouza has quit IRC23:04
*** tosky has quit IRC23:05
*** uzumaki has quit IRC23:32

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!