Monday, 2020-11-23

*** tosky has quit IRC00:06
*** ociuhandu has joined #openstack-ironic00:11
*** ociuhandu has quit IRC00:23
*** zzzeek has quit IRC00:28
*** zzzeek has joined #openstack-ironic00:32
-openstackstatus- NOTICE: Our Gerrit upgrade maintenance has concluded successfully; please see the maintenance wrap-up announcement for additional details: http://lists.opendev.org/pipermail/service-announce/2020-November/000014.html00:54
TheJuliaI guess we should make a list of "you should be on these mailing lists" list01:10
*** rcernin has quit IRC01:57
*** rcernin has joined #openstack-ironic01:58
*** rcernin has quit IRC02:05
*** rcernin has joined #openstack-ironic02:05
*** Qianbiao has joined #openstack-ironic02:06
*** zzzeek has quit IRC02:53
*** zzzeek has joined #openstack-ironic02:54
*** ricolin has quit IRC03:07
*** rcernin has quit IRC03:47
*** ociuhandu has joined #openstack-ironic03:47
*** rcernin has joined #openstack-ironic03:48
*** ociuhandu has quit IRC03:52
*** mkrai has joined #openstack-ironic04:05
*** openstackgerrit has joined #openstack-ironic04:11
openstackgerritSteve Baker proposed openstack/ironic master: JSON conversion followup change  https://review.opendev.org/c/openstack/ironic/+/76327904:11
openstackgerritSteve Baker proposed openstack/ironic master: Improve object_to_dict arguments  https://review.opendev.org/c/openstack/ironic/+/76350104:11
openstackgerritSteve Baker proposed openstack/ironic master: Test patching booleans with string values  https://review.opendev.org/c/openstack/ironic/+/76371304:11
openstackgerritJacob Anders proposed openstack/ironic master: Use OOB inspection to fetch MACs for IB inspection  https://review.opendev.org/c/openstack/ironic/+/75094304:28
openstackgerritJacob Anders proposed openstack/ironic-inspector master: Use OOB inspection to fetch MACs for IB inspection  https://review.opendev.org/c/openstack/ironic-inspector/+/75899405:02
*** ociuhandu has joined #openstack-ironic05:04
*** ociuhandu has quit IRC05:08
*** ricolin has joined #openstack-ironic05:42
openstackgerritankit proposed openstack/ironic-tempest-plugin master: Adding firmware update standalone gate  https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/76334005:46
*** Qianbiao has quit IRC05:49
openstackgerritJacob Anders proposed openstack/ironic-inspector master: Use OOB inspection to fetch MACs for IB inspection  https://review.opendev.org/c/openstack/ironic-inspector/+/75899406:17
arne_wiebalckGood morning, ironic!06:39
*** mkrai has quit IRC06:45
*** Qianbiao has joined #openstack-ironic06:49
*** ociuhandu has joined #openstack-ironic06:52
*** mkrai has joined #openstack-ironic06:54
*** abdysn has joined #openstack-ironic06:56
*** ociuhandu has quit IRC06:56
jandersgood morning arne_wiebalck06:58
jandershow was your weekend?06:58
arne_wiebalckHey janders o/06:58
arne_wiebalckCalm :) And yours?06:58
jandersquiet on Saturday, surprisingly busy on Sunday - but good! :)06:59
arne_wiebalckoh!07:02
* arne_wiebalck has opened gerrit07:03
arne_wiebalckwill need some time to get used to I guess07:04
*** iurygregory has joined #openstack-ironic07:06
iurygregorygood morning Ironic!07:07
arne_wiebalckhey iurygregory o/07:07
iurygregoryarne_wiebalck, o/07:08
*** ociuhandu has joined #openstack-ironic07:14
*** ociuhandu has quit IRC07:19
iurygregorythe new gerrit interface is a bit strange =(07:21
iurygregoryI miss the colors07:21
*** rcernin has quit IRC07:24
*** rpittau|afk is now known as rpittau07:29
rpittaugood morning ironic! o/07:29
iurygregorymorning rpittau o/07:30
rpittauhey iurygregory :)07:30
*** ociuhandu has joined #openstack-ironic07:31
rpittauthe new gerrit feels..... empty07:34
iurygregoryyeah07:35
iurygregoryhello darkness my old friend...07:35
rpittauI guess it will take some time to get used to it, hopefully there's a dark theme07:36
iurygregoryYup!07:36
iurygregoryI'm hoping for a dark theme also07:36
*** ociuhandu has quit IRC07:39
*** AshuKumar_ has joined #openstack-ironic07:42
*** rcernin has joined #openstack-ironic07:50
iurygregoryyay today we will get the results for outreachy07:57
*** rcernin has quit IRC08:08
jandersgood morning rpittau and iurygregory08:29
rpittauhey janders :)08:30
iurygregoryhey janders o/08:30
jandershow was your weekend? :)08:30
jandersI share the sentiment about the new gerrit colour scheme :)08:30
iurygregoryvery busy XD08:30
jandersgetting ready for the big trip?08:30
iurygregoryyeah08:30
iurygregoryand today I need to go out pick up my long term residence card08:31
openstackgerritRiccardo Pittau proposed openstack/ironic master: Convert ironic-tempest-ipa-wholedisk-bios-agent_ipmitool to dib  https://review.opendev.org/c/openstack/ironic/+/73475908:32
*** tosky has joined #openstack-ironic08:38
rpittauoh gosh it will take me ages to get used to the new interface :/08:40
* rpittau feels old08:40
rpittauthere is a dark theme! \o/08:41
*** rcernin has joined #openstack-ironic08:43
openstackgerritRiccardo Pittau proposed openstack/ironic master: Removing 2 non-voting jobs from gate  https://review.opendev.org/c/openstack/ironic/+/76375208:46
iurygregory=O08:46
iurygregoryrpittau, share!08:46
rpittauiurygregory: if you go to settings, top-right corener, on the first page there a Dark them (alpha)08:47
iurygregoryrpittau++08:48
iurygregoryI also agree it will take some time to get used to the new interface...08:48
iurygregoryit's.. strange08:48
*** dougsz has joined #openstack-ironic08:49
rpittaunow I need to understand how to have a better view of the zuul jobs08:52
rpittaurpioso: when you get a chance please have a look at https://review.opendev.org/c/x/sushy-oem-idrac/+/727260 https://review.opendev.org/c/x/sushy-oem-idrac/+/758365 thanks!08:54
*** lucasagomes has joined #openstack-ironic08:56
*** derekh has joined #openstack-ironic08:57
*** mgoddard has joined #openstack-ironic08:58
openstackgerritvinay50muddu proposed openstack/ironic master: Add support to manage certificates in iLO  https://review.opendev.org/c/openstack/ironic/+/76057309:01
*** ociuhandu has joined #openstack-ironic09:01
*** ociuhandu has quit IRC09:20
*** dougsz has quit IRC09:22
*** rcernin has quit IRC09:23
openstackgerritvinay50muddu proposed openstack/ironic master: Add support to manage certificates in iLO  https://review.opendev.org/c/openstack/ironic/+/76057309:28
*** abdysn has quit IRC09:32
*** dougsz has joined #openstack-ironic09:38
*** mkrai has quit IRC09:47
*** dtantsur|afk is now known as dtantsur09:50
dtantsurgood morning ironic09:50
dtantsurmaybe unpopular opinion, but modulo a few rough edges, I like the new gerrit more09:50
*** ociuhandu has joined #openstack-ironic09:51
*** mkrai has joined #openstack-ironic09:52
*** ricolin has quit IRC09:53
iurygregorymorning dtantsur o/09:55
iurygregorydtantsur, can you share what did you like more in the new gerrit? XD09:56
dtantsuriurygregory: seems faster, more compact09:58
iurygregorythat's why they removed the colors =P lol09:58
rpittauso far I'm just getting lost in it :D09:58
iurygregory209:59
*** k_mouza has joined #openstack-ironic09:59
dtantsuryeah, I'm still getting used to it as well09:59
iurygregorywondering where is the video 101 gerrit new version LOL09:59
*** ociuhandu has quit IRC09:59
dtantsurmmm maybe on their web site? or am I too optimistic all of a sudden?10:00
iurygregory+1 to optimistic10:00
*** ociuhandu has joined #openstack-ironic10:15
*** ociuhandu has quit IRC10:22
*** mkrai has quit IRC10:28
openstackgerritRiccardo Pittau proposed openstack/tenks master: Increase verbosity of ansible lint command  https://review.opendev.org/c/openstack/tenks/+/76376510:28
*** ociuhandu has joined #openstack-ironic10:34
dtantsurFYI https://etherpad.opendev.org/p/gerrit-3.2-post-upgrade-notes10:36
*** iurygregory has quit IRC10:39
*** ociuhandu has quit IRC10:42
*** abdysn has joined #openstack-ironic10:44
*** mkrai has joined #openstack-ironic10:50
rpittauoh cool thanks10:51
*** thiagop has joined #openstack-ironic11:01
openstackgerritMerged openstack/bifrost master: Support UEFI in bifrost-cli and document it  https://review.opendev.org/c/openstack/bifrost/+/76255511:15
*** derekh has quit IRC11:16
openstackgerritKaifeng Wang proposed openstack/ironic-specs master: Snapshot support  https://review.opendev.org/c/openstack/ironic-specs/+/74693511:23
openstackgerritKaifeng Wang proposed openstack/ironic-specs master: Snapshot support  https://review.opendev.org/c/openstack/ironic-specs/+/74693511:25
*** derekh has joined #openstack-ironic11:44
*** mkrai has quit IRC11:58
*** iurygregory has joined #openstack-ironic11:58
openstackgerritMerged openstack/ironic-python-agent master: Remove the unused coding style modules  https://review.opendev.org/c/openstack/ironic-python-agent/+/75979212:04
iurygregoryI'm back o/12:08
*** ociuhandu has joined #openstack-ironic12:08
iurygregoryfunny that the internet decided to not work while I was out lol12:08
*** derekh has quit IRC12:13
*** ociuhandu has quit IRC12:13
*** ociuhandu has joined #openstack-ironic12:20
*** anuradha1904 has joined #openstack-ironic12:21
*** ociuhandu has quit IRC12:25
openstackgerritMerged openstack/ironic-inspector master: Use correct Node id attribute  https://review.opendev.org/c/openstack/ironic-inspector/+/76366412:27
*** ociuhandu has joined #openstack-ironic12:39
openstackgerritDmitry Tantsur proposed openstack/ironic-inspector stable/victoria: Use correct Node id attribute  https://review.opendev.org/c/openstack/ironic-inspector/+/76372912:39
*** ociuhandu has quit IRC12:44
iurygregorythe new url's are a bit strange hehe13:07
openstackgerritBob Fournier proposed openstack/ironic-python-agent master: Bring up VLAN interfaces and include in introspection report  https://review.opendev.org/c/openstack/ironic-python-agent/+/76057013:13
*** ociuhandu has joined #openstack-ironic13:18
*** ociuhandu has quit IRC13:18
*** ociuhandu has joined #openstack-ironic13:20
dtantsuriurygregory: I find it handy that they include a project name now, more context when pasting a link13:22
iurygregorydtantsur, yeah, I just need to get used to it =)13:22
dtantsuryep. a lot of getting used to.13:22
*** ociuhandu has quit IRC13:24
bfourniedtantsur: I guess draft comments on dashboard are a lot more obvious now, never realized I had a dozen review comments that were never submitted :-(13:33
dtantsuryep, I had to cleanup like 20 patches :D13:33
iurygregorywow13:34
iurygregory=O13:34
*** ociuhandu has joined #openstack-ironic13:36
rpittauI admit that after the first impact I'm liking it13:38
dtantsurTheJulia: we should probably have victoria releases in the near future13:38
dtantsurat least ironic has a lot of changes13:39
dtantsurbifrost too13:40
openstackgerritRiccardo Pittau proposed openstack/ironic master: Removing 2 non-voting jobs from gate  https://review.opendev.org/c/openstack/ironic/+/76375213:42
*** ociuhandu has quit IRC13:44
openstackgerritMerged openstack/ironic master: Remove from_dict function from context  https://review.opendev.org/c/openstack/ironic/+/76301613:44
rpittaudtantsur: as far as I can see the the standalone job is failing on test_software_raid, it doesn't look as bad to be honest in terms of failure frequency, it's usually just a time out13:48
dtantsurtime outs \o/13:48
rpittauheh13:49
dtantsursame for the multinode job, and we keep it voting13:49
rpittaualright13:49
rpittauI guess I'll mnove it back13:49
rpittaulikewise for redfish13:50
rpittauunless we want to keep it non-voting for other reasons, but it looks pretty stable13:50
openstackgerritRiccardo Pittau proposed openstack/ironic master: Make standalone jobs voting again  https://review.opendev.org/c/openstack/ironic/+/76375213:52
*** paras333 has joined #openstack-ironic13:54
TheJuliagood morning14:08
dtantsurmorning TheJulia14:09
TheJuliadtantsur: ++, I was actually thinking about htat late last week14:09
rpittaugood morning TheJulia :)14:09
dtantsurnice! let's find a volunteer to propose releases14:09
iurygregoryo/14:09
iurygregoryme14:09
iurygregorygood morning TheJulia =)14:10
TheJuliaiurygregory: excellent!14:10
TheJuliaoh my the new gerrit interface really looks like I'm going to need to reorganize my desk14:10
dtantsurthanks iurygregory :) as always, ironic, inspector, IPA and bifrost are the first candidates14:10
dtantsurheh14:10
TheJuliaI guess it means it is a good week to do so since I'm only working today and tomorrow14:10
*** ociuhandu has joined #openstack-ironic14:10
iurygregorydtantsur, ack, we are waitting for more patches or I can go ahead?14:10
dtantsuriurygregory: could you take a look if there is something close to approving or merging?14:11
iurygregoryTheJulia, take your time and a few more coffees =)14:11
iurygregorydtantsur, looking now at the 4 projects14:11
TheJuliaiurygregory: oh of course14:11
TheJuliacoffee is brewing now14:11
* TheJulia seriously needs it this morning14:11
* dtantsur has earl grey with orange peel and vanille14:14
* TheJulia turns on the dark theme to match her soul14:15
iurygregorydark theme ++14:16
dtantsurthe dark theme looks nice at first glance14:16
rpittauit's easy on the eyes at least14:16
dtantsurcan I apply it to the whole firefox?14:16
rpittauwould love that...14:17
dtantsurall dark themes I've found so far are too black14:18
dtantsurI'd love this grey color14:18
*** rloo has joined #openstack-ironic14:19
rpittaugmail has a couple of decent ones14:19
TheJuliaYeah, a number of the screens don't fit on my page view monitor :(14:20
*** ociuhandu has quit IRC14:20
TheJuliaohhh ahh14:23
TheJuliagraphical glitching in my browser14:23
*** mkrai has joined #openstack-ironic14:24
*** ociuhandu has joined #openstack-ironic14:24
*** ociuhandu has quit IRC14:24
* TheJulia gets coffee14:25
*** ociuhandu has joined #openstack-ironic14:26
iurygregoryday1 of new gerrit interface - glitches - checked14:28
rpittauit's not glitches, it's graphical features14:28
TheJuliaam I not the only one getting glitching?14:28
iurygregoryhahaha14:28
rpittaume too twice, I restarted firefox and all seems good14:28
TheJuliaugh14:28
iurygregoryinclude me in this list =D14:28
TheJuliafun14:28
TheJuliachome14:28
TheJuliaso.. yeah.14:29
iurygregoryrpittau, have you found a way to show CI results without having to click on the Zuul comments?14:29
*** ociuhandu has quit IRC14:30
rpittauiurygregory: nope, still looking for that14:31
*** derekh has joined #openstack-ironic14:31
iurygregoryok so I'm not alone \o/14:31
TheJuliathat was likely custom skinning14:31
iurygregorywe are in this together \o/14:31
TheJuliaI can see a lot of positives, but I sense I may end up seeing if I can get gertty running again14:31
TheJuliaI do like the new url format to changes14:33
TheJuliaspeaking of https://review.opendev.org/c/openstack/ironic-python-agent/+/762453 <-- wants reviews :)14:33
openstackgerritRiccardo Pittau proposed openstack/sushy-tools master: [WIP] Support processors schema  https://review.opendev.org/c/openstack/sushy-tools/+/76190914:34
*** bdodd has joined #openstack-ironic14:34
*** ociuhandu has joined #openstack-ironic14:36
TheJuliawow, like after I get the clitching in gerrit, it continues14:36
*** ociuhandu has quit IRC14:36
TheJuliain other sites, it is funky14:36
iurygregory=(14:36
*** rpittau is now known as rpittau|brb14:36
*** ociuhandu has joined #openstack-ironic14:37
* TheJulia loves kaifeng's comment on the new gerrit look14:37
*** stendulker has joined #openstack-ironic14:38
*** AshuKumar_ has quit IRC14:40
TheJuliastendulker: I'm guessing your commenting on ilo ci right now?14:40
stendulkerTheJulia: Yes.14:41
TheJuliaok14:41
*** ociuhandu_ has joined #openstack-ironic14:41
*** ociuhandu_ has quit IRC14:41
stendulkerThis can be added into announcement section in Today's meeting.14:41
*** ociuhandu_ has joined #openstack-ironic14:41
TheJuliaFeel free to bring it up or I'll try to remember, or just add it to the agenda14:42
TheJuliaI'm checking to see what has merged14:42
stendulkerWe have shifted CI to India this year and during furlough period they perform electrical maintenance.14:42
TheJuliaerr14:42
TheJuliaI'll note it14:42
*** ociuhandu has quit IRC14:42
stendulkerThanks TheJulia14:43
*** tzumainn has joined #openstack-ironic14:44
TheJuliaerr, I'd actually approve a patch at the moment if it wasn't for the glitching in my current browser window :(14:45
TheJuliabfournie: It might help to rephrase the title on https://review.opendev.org/c/openstack/ironic-python-agent/+/760570, seems like (without digging into the code) it is more about connected vlan detection ?14:48
iurygregorywuuut Dell CI is running on Ubuntu Xenial14:50
iurygregoryO.o14:50
dtantsurTheJulia: re gertty: you need to change to basic auth14:50
iurygregoryrpioso, can you please check with Dell CI folks about this? =)14:52
bfournieTheJulia: well it also provides an option to define the vlan to bring up, which doesn't use lldp detection logic but I can work on the title14:52
iurygregorythe logs I was looking at https://review.opendev.org/c/openstack/ironic/+/76301614:52
TheJuliahmm, intruiging14:53
TheJuliabfournie: would be the use case be cards that will pxe boot on a specific vlan instead of the default vlan?14:53
*** Goneri has joined #openstack-ironic14:55
TheJulialooks like we have someone that thought they could apply changes like many do with github14:56
TheJuliain a series of patches :(14:57
TheJuliaerr14:57
TheJuliaseries of commits14:57
bfournieTheJulia: main use case is to bring up vlan interfaces and allow them to get an IP via dhcp-all-interfaces and include these interfaces in introspection report, interfaces/IPs in inspection report can then be consumed e.g. by metal3 to use in CSR (certificate signing request)14:57
rpiosoiurygregory: It is well known. We are working on upgrading to Focal.15:00
TheJulia#startmeeting ironic15:00
rpiosoerbarr: ^^^15:00
openstackMeeting started Mon Nov 23 15:00:26 2020 UTC and is due to finish in 60 minutes.  The chair is TheJulia. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: ironic)"15:00
openstackThe meeting name has been set to 'ironic'15:00
iurygregoryo/15:00
stendulkero/15:00
rpioso\o15:00
rlooo/15:00
ajyao/15:00
TheJuliaGood morning everyone!15:00
arne_wiebalcko/15:00
*** rpittau|brb is now known as rpittau15:01
rpittauo/15:01
TheJuliaOur agenda can be found on the wiki, this week we have a number of items.15:01
TheJulia#link https://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_next_meeting15:01
TheJulia#topic Announcements / Reminders15:01
*** openstack changes topic to "Announcements / Reminders (Meeting topic: ironic)"15:01
TheJuliaFirst up!15:01
TheJulia#info The TC has approved a new cycle goal which we did not incorporate into our planning to basically drive the migration to YAML only policy files.15:02
TheJulia#link https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html15:02
TheJuliaSecondly!15:02
dtantsuro/15:02
TheJulia#info iLO CI will be down for approximately 3 weeks due to a site power shutdown during the end of the year starting around ?December 18th?15:03
TheJuliaLast, but not least, at least for me.15:03
TheJuliaA reminder for everyone that this is a holiday week in the states. Many people tend to take some or this entire week off. And on that subject, I'll only be around Today and Tomorrow for this week.15:04
TheJuliaDos anyone have anything to raise/announce/remind us of?15:04
iurygregoryenjoy the time off TheJulia =)15:04
dtantsur++15:04
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Copy any configuration from the virtual media  https://review.opendev.org/c/openstack/ironic-python-agent/+/76320715:05
TheJuliaI'll be writing for nonorimo15:05
iurygregorywe can probably mention we will have 2 new contributors since outreachy project was approved?15:05
TheJulia(National Novel Writers Month)15:05
*** mkrai has quit IRC15:05
TheJuliaiurygregory: I hadn't even gotten to that email yet, if you wouldn't mind making the annoucement15:05
iurygregoryor wait for next monday since we will have the official email =)15:05
TheJuliais it embargoed?15:06
*** mkrai has joined #openstack-ironic15:06
openstackgerritvinay50muddu proposed openstack/ironic master: Add clean/deploy steps to manage certificates  https://review.opendev.org/c/openstack/ironic/+/76379115:06
TheJuliaor we can just be vague15:06
iurygregoryI only got the one saying that All interns are approved =)15:06
TheJuliasweet15:06
TheJuliaOkay then15:06
iurygregoryso we will have 2 new contributors15:06
TheJulia#info We will have 2 new contirbutors for this current Outreachy cycle.15:07
TheJuliaAnyway, I guess we can proceed if nobody has anything else?15:07
iurygregory++15:07
TheJuliaiurygregory: Your taking all of december off right?15:07
iurygregoryYes15:07
TheJuliak15:08
* TheJulia makes a mental note to send an email today15:08
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] inject TLS certificate when using virtual media  https://review.opendev.org/c/openstack/ironic/+/75842715:08
TheJulia#topic Review action items from the prior week.15:09
stendulkerTheJulia: I will also be off for 2 weeks starting from Dec 18th.15:09
*** openstack changes topic to "Review action items from the prior week. (Meeting topic: ironic)"15:09
TheJuliastendulker: thanks!15:09
TheJuliaWe had one action item last week which was to get with stevebaker and start moving the API patches forward. Thanks everyone who helped push that forward!15:09
TheJulia#topic Review subteam status reports15:10
*** openstack changes topic to "Review subteam status reports (Meeting topic: ironic)"15:10
TheJulia#link https://etherpad.openstack.org/p/IronicWhiteBoard15:10
TheJuliaStarting around line 26015:10
*** kaifeng has joined #openstack-ironic15:11
TheJuliaIf everyone would update relevent sections on the etherpad15:11
arne_wiebalckFor the UEFI topic, is the plan to also include the re-factoring we skipped/postponed at the time for UEFI RAID? IIRC, this was around grub2install vs efibootmgr ... or is UEFI happy enough without this? ,)15:14
arne_wiebalck;)15:14
dtantsurit's probably worth at least taking a look15:15
arne_wiebalckdtantsur: ++15:15
TheJuliaI was pondering that and I'm kind of wondering if we get that for free15:15
TheJuliaalready, but I may not completely understand the problem from the raid perspective15:15
arne_wiebalckwe wanted to get the UEFI code merged, and I was not keen on refactoring it at the time (b/c this would have required a lot of re-testing)15:17
TheJuliaarne_wiebalck: do you have a patch?15:18
arne_wiebalckTheJulia: to refactor?15:18
TheJuliayeah, sounds like one was started?15:18
arne_wiebalckTheJulia: no15:18
TheJuliaokay, well... hmm15:18
TheJuliaI still think it may cover the case, but I'll need to look back at the comments and maybe discuss it further with you15:19
arne_wiebalckTheJulia: ok!15:19
TheJuliaoh! it likely works for partition images now, but not wholedisk15:19
TheJuliaTaht is likely fairly easy to follow-up with15:19
dtantsurif so, it needs fixing15:19
TheJuliaI think it is explictly cased out because it has to be handled differently15:19
TheJuliafundimentally the patches I've been working on are bugfix soft of fixes since we're doing the wrong thing with the agent today and causing pain for deployments with partition iamges.15:20
arne_wiebalckIIRC, it was mostly to not call grub2 anymore but efibootmgr instead (iurygregory was working on this area at the time, so things were a bit in flux)15:21
* TheJulia adds notes int the etherpad15:21
arne_wiebalckat the moment we have a big IF somewhere :-D15:21
iurygregorywe do15:21
iurygregorywhen calling the install_bootloader if I do remember15:21
arne_wiebalckiurygregory: right15:22
TheJuliayeah15:22
openstackgerritMerged openstack/ironic-inspector stable/victoria: Use correct Node id attribute  https://review.opendev.org/c/openstack/ironic-inspector/+/76372915:22
TheJuliaAnyway, Is everyone good to proceed?15:22
iurygregory++15:22
openstackgerritJohn Garbutt proposed openstack/networking-generic-switch master: Add ngs_manage_vlans configuration  https://review.opendev.org/c/openstack/networking-generic-switch/+/76379415:22
dtantsuryep15:22
arne_wiebalckrpioso: rpittau: we should probably discuss how to move the interop profiles fwd15:22
TheJulia#topic Deciding on priorities for the coming week15:22
*** openstack changes topic to "Deciding on priorities for the coming week (Meeting topic: ironic)"15:22
TheJulia#link https://etherpad.openstack.org/p/IronicWhiteBoard15:23
TheJuliaStarting at line 12115:23
TheJuliaLooks like we got some stuff merged last week, and also the API work stevebaker largely merged which is absolutely awesome15:23
* TheJulia removes merged items15:23
TheJuliaas for items to add, looks like items are appearing on line 196 and below15:25
dtantsuryep, I've added a few things15:26
TheJuliaDoes anyone have any objections on adding these items?15:27
dtantsurbtw we should have announced that most of the wsme removal has merged15:27
dtantsurthose who haven't been following it may experience shock next time they look at the API layer :D15:28
rloodidn't the wallaby priorities get approved?15:28
rlooline 12215:28
rloo(still trying to get used to this new UI...)15:28
TheJuliaI just added two more related to the new cycle goal, which is mostly cookie cutter across repositories, the later step that community is starting to talk about is going to be more work15:28
TheJuliarloo: good catch, removed15:29
TheJuliadtantsur: ++ although haven't even seen that yet :(15:29
TheJuliadtantsur: shock, and sudden waves of happiness? :)15:29
dtantsurwell, yes15:29
dtantsurstevebaker has succeeded in reducing the amount of copy-paste in the API code very substantially15:29
dtantsurlike, 50% or more15:29
TheJuliaIf there are no objections, then we should move on15:30
rloogreat job stevebaker!!!15:30
TheJulia++15:30
TheJuliaI'll move the proposed items in after the meeting15:30
TheJuliaAnyway, onward!15:31
TheJulia#topic Discussion15:31
*** openstack changes topic to "Discussion (Meeting topic: ironic)"15:31
TheJuliaWe have two-ish items for discussion. The first is ultimately sourced by a downstream bug I've got. We've observed a converged network/storage fabric adapter basically overloading the UEFI nvram boot entries list15:31
TheJuliaAnd while the card is well meaning, and it is definitely the wrong behavior for the card, an question of if the community would be receptive to a flag that says "ignore bootloader installation failures"?15:32
dtantsurnot yet, at least15:32
TheJuliaAny thoughts/concerns?15:32
dtantsurI mean, does it mean the the OS simply won't boot?15:32
dtantsuror will boot depending on how the stars align?15:33
TheJuliaThe OS boots, because the card firmware goes and hunts down all the passible UEFI images15:33
TheJuliapossible15:33
TheJuliawhich seems crazy, but it is doing it for all of the luns it is providing15:33
TheJuliaas well, so the machine ultimately does boot, it is just the nvram update that is blowing up15:33
TheJuliaWe should be able to identify that15:33
iurygregory++15:33
dtantsurand then the custom enables discovery in ironic-inspector and the machine starts booting into IPA?15:33
dtantsurs/custom/customer/15:34
dtantsurwhy cannot we add a boot entry? too many entries? hardware just rejects that?15:34
TheJuliadtantsur: no, the card wants to boot to disk instead of network15:34
TheJuliadtantsur: The table is full, literally overloaded to the point we cannot add new entries15:34
TheJuliaand we have no way of knowing what we can safely remove15:34
iurygregoryouch15:34
*** mkrai has quit IRC15:34
TheJuliayeah...15:35
dtantsurcan we try finding the corresponding disk entry?15:35
*** mkrai has joined #openstack-ironic15:35
stendulkercant we add explicit entry for the device that was provisioned and remove all other entries?15:35
dtantsurnote that linux probably won't work normally on such system anyway15:36
TheJuliapossibly, but maybe not since they are all oriented to the way the UEFI firmware views the hardware15:36
TheJuliastendulker: that is another possibility15:36
dtantsurbecause at least Red Hat shim tries to recover an UEFI record under some conditions15:36
arne_wiebalckwhy would we need to preserve existing entries?15:36
dtantsurvery interesting reading: https://github.com/rhboot/shim/blob/a1170bb00a116783cc6623b403e785d86b2f97d7/README.fallback#L33-L4615:36
TheJuliaarne_wiebalck: ++15:36
* dtantsur knows a lot of weird stuff like that now15:36
TheJuliaYup, that is basically what the card is doing, populating all of the paths that seem possible with bootlaoders :\15:37
TheJuliaand it may not even hold past reboot, that is the other frustrating thing15:37
TheJuliasince they can pull/re-insert the card with out changing anything and the list of uefi boot targets explodes in nvram15:38
dtantsurwow15:38
dtantsurokay, it does sound like a skip_bootloader flag with an appropriate documentation note (use at your own risk)15:38
TheJuliaThat is kind of what I was thinking15:38
TheJuliaanyway, one more discussion topic and two RFEs15:39
dtantsurif we had this problem on my laptop, we could remove "LENOVO CLOUD" :D15:39
TheJuliaarne_wiebalck: premeptive ask on baremetal sig, anything to cover15:39
* TheJulia blinks at dtantsur 15:39
dtantsurI'm serious, I have this entry15:39
TheJuliaSecond discussion topic!15:39
TheJuliadtantsur: I'm afraid15:39
arne_wiebalckTheJulia: Next meeting #3 Tue Dec 8, 2020 at 2pm UTC with rpioso on interop profiles, otherwise NTR15:39
TheJuliaAs previously noted the TC has approved an additional effort for this cycle to migrate json policy files to drive the migration of JSON policy files to YAML. This is viewed as a stepping stone for the secure RBAC work that is starting to be discussed which may bring many projects many headaches15:40
TheJulia#link https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html15:40
dtantsurJust Do It (tm)15:41
TheJuliaWell, patches for that are up :)15:41
iurygregoryI was about to say that15:41
TheJuliaLance has propsed a series of patches converting our policies to the secure rbac model, however without additional testing and that is going to drive additional discussion. I guess the question I have is should we add this to the project level cycle priorities?15:42
dtantsurI assume we have to?15:43
TheJuliaFWIW, those patches were on ironic, not ironic-inspector.15:43
TheJuliadtantsur: I suspect yes, I'm also curious how receptive will people be if we push forward on RBAC model changes15:43
rlooyes, i think to support community priorities, etc, let's update our priorities to reflect that. Assuming we want to do that work this cycle...15:44
dtantsurI think it's a question for operators, not for us15:44
TheJuliadtantsur: Good point, I know some operators want it, I guess we need to map out the entire scope of what is imapcted and what needs to be touched in the larger case15:45
TheJuliaI've already started tinking in that direction so I can keep heading in that direction.15:45
TheJuliaAnyway, we should move on if there are no last minute discussion thoughts?15:45
TheJulia#topic Baremetal SIG15:45
*** openstack changes topic to "Baremetal SIG (Meeting topic: ironic)"15:45
TheJulia#info Next Meeting - December 8th at 2PM UTC15:46
TheJulia#link https://etherpad.opendev.org/p/bare-metal-sig15:46
TheJulia#topic RFE Review15:46
*** openstack changes topic to "RFE Review (Meeting topic: ironic)"15:46
* TheJulia hands the microphone to dtantsur 15:46
dtantsurthank you15:46
dtantsur#link https://storyboard.openstack.org/#!/story/2008380 Configdrive with virtual media ramdisk ISO deploy15:46
dtantsurthe ramdisk deploy does not have support for configdrive15:47
dtantsurI have a feeling that we can make it work for redfish-virtual-media15:47
dtantsursince a lot of hardware (I tested 2 Dells and 1 HPE) seems to have a USB virtual media slot in addition to ISO15:47
TheJuliaI guess we would end up with maybe an additional boot interface method or two to facilitate this?15:47
TheJuliaor would we try to entirely do this inside of the boot interface?15:48
dtantsurI haven't looked in depth what it involves15:48
TheJuliaOkay15:48
dtantsurlikely an extension of the boot interface indeed15:48
dtantsurunless we can roll it into prepare_instance15:48
dtantsurwhich is not unlikely, now that I'm thinking about it15:48
TheJuliaI guess I'd kind of like to see patches to better understand. It seems logical to me for the most part15:48
TheJuliaI'd prefer not to roll it in to prepare_instance directly15:49
TheJuliaAnyway I'm not opposed, I like the idea. Anyone else have any thoughts on this one?15:49
dtantsuryeah, I need to see patches myself :) I want to have a prototype this week15:49
TheJuliaOkay, seems like we should try to revisit this next week :)15:50
TheJuliabut otherwise, I like the idea15:50
Qianbiaodtantsur why not just use virtual media it self15:50
dtantsurQianbiao: with the ramdisk deploy we'd like to avoid changing the provided ISO15:50
Qianbiaoi see.15:50
*** abdysn has quit IRC15:51
TheJuliaI think the idea is also attach to have it be natively detected15:51
TheJuliaas a device with a label that is read15:51
Qianbiaomaybe we can provide an option, if user desired to use iso15:51
TheJuliawhich kind of makes sense15:51
dtantsurQianbiao: your hardware doesn't have a USB slot?15:51
QianbiaoNot sure, will ask15:51
dtantsurplease check, it's useful input for us15:51
TheJuliaNext RFE?15:52
Qianbiaosure15:52
dtantsur#link https://storyboard.openstack.org/#!/story/2008366 BMC event framework15:52
dtantsurokay, this qualifies for a loong spec, but I'd like some early input :)15:52
dtantsurand note that I may not be able to work on it in the near future15:52
dtantsurdepending on how the priorities align downstream15:52
TheJuliaThis may also funnel into "Maybe hack out another API service"15:52
TheJuliaI like the idea a lot15:53
dtantsurI still think the complexity of maintaining another service does not justify the benefits15:53
dtantsurbut we may discuss :)15:53
TheJuliaAnd truthfully there are a number of ideas there, but it does solve a conundrum and help operators keep their BMC's further insulated by adding intermediate mechanisms15:53
* dtantsur gives everyone a chance to scroll through15:54
TheJuliadtantsur: well, it could still be in the project and just be an increadible skimmed down API, but as you also pointed out it could just be an operating mode flag.15:54
TheJuliamuch as we have lookup today15:54
TheJuliaIt is a lot to take in, I guess it helps a lot that I've already read the earlier version15:55
dtantsurright15:55
dtantsurI'm looking for yay/nay/wtf reactions15:56
rpittauyay from me15:56
rpittau:)15:56
TheJuliaI'm basically yay because we've had operators ask about similar things before15:56
iurygregoryyay for me also15:56
TheJuliathe whole unfortunate thing is the nature of redfish events and the mechanics there in15:56
TheJuliaI do like the idea of maybe later storing, but with that comes great risk and would likely be what operators expect behavior/mechanics wise15:57
TheJuliaWell, then!15:57
TheJuliaI think we can move on!15:57
dtantsuryep, I welcome any comments later on15:57
* TheJulia moves on because people can still yay/nay/wtf in open discussion15:57
TheJulia#topic Open Discussion15:57
*** openstack changes topic to "Open Discussion (Meeting topic: ironic)"15:57
TheJuliaSo 3 minutes left, we've covered a LOT!15:57
TheJuliaDoes anyone have anything else to dsicuss?15:58
rpittaucrickets say no15:59
TheJuliaSeems like it15:59
TheJuliaWell, thanks everyone!15:59
iurygregoryty15:59
rpittauthanks!15:59
TheJuliaHave a wonderful week!15:59
arne_wiebalckthanks TheJulia16:00
kaifengthannks Julia16:00
dtantsuro/16:00
*** stendulker has quit IRC16:00
TheJulia#endmeeting16:00
*** openstack changes topic to "Bare Metal Provisioning | Status: http://bit.ly/ironic-whiteboard | Docs: http://docs.openstack.org/ironic/ | Bugs: https://storyboard.openstack.org/#!/project_group/75 | Contributors are generally present between 6 AM and 12 AM UTC, If we do not answer, please feel free to pose questions to openstack-discuss mailing list."16:00
openstackMeeting ended Mon Nov 23 16:00:29 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-11-23-15.00.html16:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-11-23-15.00.txt16:00
openstackLog:            http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-11-23-15.00.log.html16:00
Qianbiaothanks TheJulia16:00
arne_wiebalckdtantsur: for the BMC event framework, how about adding an example or two of specific events you're thinking of?16:01
TheJuliaSo I'm not sure that makes sense, and the reason is becasue it would be a pass-through16:01
dtantsurI need to see myself how they look16:01
TheJuliathe spec has some of it formatted, but it is just a payload body keyed off the target URL with a long lived "known" token16:02
dtantsursomething among the lines of "I'm overheating, send help"16:02
arne_wiebalckIs that sth you expect the user to deal with?16:03
TheJuliaThink "Tiny Toons" and then think "raid set go down the hooolleeee"16:03
rpittaulol16:04
TheJuliaOperator can, if they have the framework16:04
arne_wiebalckSome/many of such events/alerts are available from within the node (like PSU failure, high temp, ...) and would therefore be picked up and reported with the usual alarms from the node.16:04
arne_wiebalckI am thinking SEL.16:05
TheJuliaMaybe it is best to think of these notifications maybe replacing snmp trap alarms?16:05
TheJuliaYeah, these would include SEL sorts of items16:05
arne_wiebalckAre these accessible from within the node via redfish?16:06
TheJuliano16:06
arne_wiebalckWith ipmi they are.16:06
TheJuliathe configuration to set them are though16:06
TheJuliathe redfish BMC, at least the last time I looked at eventing, didn't have any requirement to retain the event in the API16:06
* arne_wiebalck has not looked at redfishtool yet16:06
TheJuliaI think there is a table defined for those that want to expose the SEL list16:07
TheJuliaIt has been like 2... or 3 years sinec I looked at redfish eventing16:08
arne_wiebalckcollectd, for instance, has an IPMI plugin to pick up sensor data16:08
clarkbTheJulia: re graphical gliches: I wonder if you've got hardware acceleration enabled in your browser? if you're on linux that is supposed to be super flaky ( and I turn it off in firefox)16:08
clarkbalso I haven't run into that at all in all the testing or post upgrade work we did so mroe info on that if you can quantify it would be good (maybe screenshots?)16:09
arne_wiebalckTheJulia: or it had, not sure this is still maintained16:09
TheJuliaclarkb: Yeah, it is chrome so surely accelerated16:11
TheJuliaclarkb: I'll try to get some screen shots, it may be interesting!16:11
clarkbthanks, and sorry for the trouble16:11
TheJuliaarne_wiebalck:  dunno, the notifications spec actually had a section if memory serves why people should basically use notifications16:11
TheJuliainstaead of tryign to poll16:12
*** mgoddard has quit IRC16:12
TheJuliaAnyway, two things down, 32767 items left fo rthe day16:12
iurygregoryyay now we have the emails \o/16:12
TheJuliahuh?16:12
iurygregoryTheJulia, outreachy =)16:12
Qianbiaodtantsur ibmc has "/redfish/v1/Managers/1/VirtualMedia/USBStick"16:13
Qianbiaoso it supports virtual usb16:13
iurygregoryI hope it has the actions to insert/eject =)16:14
*** mgoddard has joined #openstack-ironic16:15
iurygregoryI saw in some cases the hardware has the "endpoint" but doesn't support the actions16:15
Qianbiaoiurygregory :) yes, it support connect16:18
iurygregorynice =)16:18
*** Qianbiao has quit IRC16:20
TheJuliaclarkb: any preferred place for me to put screen shots?16:22
rpittauiurygregory: mmm weird, eject and insert should be mandatory in the redfish schema16:22
clarkbTheJulia: not really. imgur is both good and terrible at the same time. Not sure if there are better options16:23
TheJuliaafaik they are not16:23
iurygregoryrpittau, maybe they just have the "VirtualMedia" but since it doesn't support you can't see the actions available?16:23
TheJuliarpittau: what annosy me is those that did things like <objecttype>Insert and <objecttype>Eject16:23
iurygregoryor maybe the vendor understood that way in a specific firmware?16:24
iurygregoryquestions...16:24
TheJuliaclarkb: https://drive.google.com/drive/folders/1hsaygA6fGHMN3dAZ_VdIJD_sZ6RJJcIe16:28
TheJuliaI'm not far from rebooting. I can't see any of my browser tabs in any of my windows now and glitching has extended to windows that never touched gerrit :(16:29
rpittauTheJulia: yes, you're right, I remembered wrong :/16:29
clarkbTheJulia: I'm looking at that screenshot and it seems fine? Also its early so I may just be missing it16:30
clarkbbut if they actually are clean then that would point more to the hardware acceleration I think16:30
TheJuliathey uploaded out of order16:30
TheJuliaStarted with text not aligning, clicked through a few screens and review text disappeared16:31
clarkboh ha need to look at more of them16:31
clarkbthanks16:31
TheJuliathen clicked back and forth and got no diff and then some text and then finally all my tabs are missing16:31
TheJuliathey are there, just can't see them :\16:31
TheJuliait is super bizzar :(16:32
clarkbother thoughts: disable hardware acceleration if it is enabled just to rule it in/out. Try incognito mode to rule out browser extensions16:33
TheJuliaI think incognito mode goes without acceleration16:33
TheJuliaand it doesn't seem to be plagued by the glitching16:33
TheJuliabut it hasn't touched gerrit yet16:33
clarkbcool, if we can anrrow it down like that then we should be able to file a bug upstream16:34
TheJuliaerr, and glitching started upon loading16:34
clarkb:(16:34
* TheJulia kills all the browser pids16:34
iurygregory++ for using kill16:35
clarkbalso if hardware acceleration is disabled try enabling it? the internets suggest that it could go both ways depending on hardware/platform/drivers/etc16:36
TheJuliaYeah, killed hardware acceleration and it  *seems* to be behaving16:36
TheJuliaYeah, seems to be behaving for now16:38
*** mkrai has quit IRC16:39
clarkbok that is good info to have16:39
clarkbI think we can still file a bug upstream but they may say its the browser's fault give that16:39
clarkbbut at least we have a workaround16:39
TheJuliaYeah, well, the fact multiple people on multiple browsers have reported glitching...16:40
TheJuliaYeah.16:40
TheJuliaAnyway, coffee, food16:40
TheJuliaand then five billion emails16:41
*** ociuhandu_ has quit IRC16:43
*** ociuhandu has joined #openstack-ironic16:43
*** sdanni has joined #openstack-ironic16:44
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Allow inserting SSH public key for dynamic-login  https://review.opendev.org/c/openstack/bifrost/+/76380616:50
*** ociuhandu has quit IRC16:53
*** lucasagomes has quit IRC17:01
rpittaugood night o/17:02
*** rpittau is now known as rpittau|afk17:03
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Inject TLS certificate when using virtual media  https://review.opendev.org/c/openstack/ironic/+/75842717:08
dtantsurTheJulia: I'm making the last attempt on DHCP-less before declaring the ramdisk side broken and probably pulling it from the docs :(17:13
dtantsurI don't know how glean works for infra folks, I cannot make it work17:13
dtantsuron a debian image it does not even start by default, but there is cloud-init....17:17
dtantsurokay, manually running it and doing `systemctl restart networking` actually got me an IP address17:20
*** ociuhandu has joined #openstack-ironic17:21
*** ociuhandu has quit IRC17:21
dtantsurianw: hey! not sure about your timezone, but will you have a chance to chat about glean?17:21
TheJuliaugh, ugh, ugh, and ugh17:23
TheJuliadtantsur: hopefully you'll have detailed notes handy so we can pick it back up to fix it17:26
dtantsurI have proposed patches for the parts I understood17:28
dtantsurI don't have anything useful for my "try everything" attempts right now17:28
TheJuliaI don't remember what OS ilya used, but I remember being kind of "oh, itneresting"'17:29
TheJuliaI have the video files someplace here, I should be able to find them.17:29
dtantsurdunno, when I picked this up, the ironic side was also hopelessly broken17:29
TheJuliayeah, the back and forth in changes in review from the demo code is likely where things went sideways in that case17:30
openstackgerritBob Fournier proposed openstack/ironic master: Add vendor_passthru method for virtual media  https://review.opendev.org/c/openstack/ironic/+/76300717:31
*** ociuhandu has joined #openstack-ironic17:32
dtantsurTheJulia: my last hope is that somebody who knows glean will just tell me "oh, you're obviously missing $thingname"17:32
*** dougsz has quit IRC17:35
TheJulia:(17:35
dtantsuralso, why cloud-init in my IPA???17:36
TheJuliaWUT?!?17:36
dtantsuryep17:36
dtantsurI've built a debian image, and it has cloud-init happily running on start-up17:36
TheJuliaugh17:36
dtantsureven though 1) we remove cloud-init, 2) the simple-init element disables cloud-init17:37
*** ociuhandu has quit IRC17:37
TheJuliacomputers, why do you hate us?17:38
dtantsurindeed17:39
*** mgoddard has quit IRC17:54
arne_wiebalckftarasenko: unless I am missing sth, you have 3 patches up for the ironic-python-agent re "Support using LABEL as identifier for rootfs" with different change IDs and different content ... if this is correct, could you abandon the ones you dont' want to follow up on?17:56
*** rloo has quit IRC17:57
iurygregorybugs everywhere =(18:03
TheJuliaDefects or imperfections?18:04
iurygregorymaybe both (it depends) XD18:12
*** k_mouza has quit IRC18:13
NobodyCamGood Morning Ironic folks.. Happy Holiday week Monday18:19
iurygregorygood morning NobodyCam o/18:25
*** kaifeng has quit IRC18:26
JayFWhat is "assignee" in the new gerrit intended for? Person actively working the change? Person actively reviewing the change?18:29
iurygregoryJayF, https://etherpad.opendev.org/p/gerrit-3.2-post-upgrade-notes18:31
iurygregorymaybe it helps =)18:31
JayFthat's perfect, ty18:39
openstackgerritMerged openstack/ironic stable/victoria: CI: update bindep for centos-8 py36 job changes  https://review.opendev.org/c/openstack/ironic/+/75597018:46
openstackgerritMerged openstack/ironic stable/victoria: Fixes the issue that instance bond port can't get IP address  https://review.opendev.org/c/openstack/ironic/+/76261718:46
*** dtantsur is now known as dtantsur|afk18:49
dtantsur|afko/18:49
iurygregoryyay now I can push the releases for victoria18:49
arne_wiebalckbye everyone o/18:49
iurygregoryafter some dinner =D18:49
TheJulia\o/18:49
iurygregorybye o/18:49
sdanniHi all! I'm developing a keylime-agent element for ironic python agent builder. I test building an image with the keylime-agent element, but dib-run-parts doesn't run the keylime-agent install.d. It skips that element and finishes building. I wonder if I miss some steps to develop an element. I created a directory inside dib/ for keylime-agent, implemented scripts of install.d, element-deps and other files just like18:53
sdanniironic-python-agent-ramdisk.18:53
TheJuliasdanni: typically it is just file name formatting that throws it18:54
TheJuliaso you may want to just go ahead and post your work in progress so others can review it18:54
TheJuliaIt might be something that is standing out as obvious to those of us with those... umm.. ?battle scars?18:54
*** zzzeek has quit IRC18:55
sdanniTheJulia, thanks! I'll use a git-review to post my work18:56
*** zzzeek has joined #openstack-ironic18:57
JayFthe other trick: install.d/ has a script in it, and it's not +x, it's ignor3ed19:11
JayFthat's the one that always gets me19:11
TheJuliaOh yeah, that is a good one19:18
sdanniJayF: do you mean scripts like 60-ironic-python-agent-ramdisk?19:22
JayFyes19:23
JayFif they are not chmod +x dib won't run 'em19:23
sdanniJayF, I see! I'll try it again19:24
JayFI do agree with TheJulia that you should push WIP code up as soon as possible though19:24
JayFor else folks helping you have to stab in the dark at it :)19:24
*** shreya11 has left #openstack-ironic19:25
TheJuliaFor some reason all I think of today is cartoon references19:25
*** mgoddard has joined #openstack-ironic19:26
*** zzzeek has quit IRC19:27
sdanniJaF: yeah, I'm going to post it soon. It's my first time to use gerrit so I'm learning how to link this change with a storyboard19:27
TheJuliaIt is all in the commit message :)19:28
sdannisorry for typo :(: JayF19:28
JayFhttps://wiki.openstack.org/wiki/GitCommitMessages19:28
JayFeveryone loves a verbose commit message! :D19:28
*** zzzeek has joined #openstack-ironic19:29
sdannicool!19:29
TheJuliasdanni: typos happen, what is really word when people begin to swap fun around... <-- example, and not entirely intentional. :)19:29
TheJuliai.e. word/fun are swapped19:29
TheJuliaor could be in different languages. That is fun too19:29
sdannihaha, fun stuff!19:30
*** hjensas has quit IRC19:31
*** hjensas has joined #openstack-ironic19:31
*** dougsz has joined #openstack-ironic19:37
openstackgerritMerged openstack/ironic-python-agent master: Make mdadm a soft requirement  https://review.opendev.org/c/openstack/ironic-python-agent/+/76366119:38
*** mgoddard has quit IRC19:38
openstackgerritDanni Shi proposed openstack/ironic-python-agent-builder master: Add a keylime-agent dib element  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/76383719:49
*** ociuhandu has joined #openstack-ironic19:51
*** dougsz has quit IRC19:52
*** zzzeek has quit IRC19:53
*** zzzeek has joined #openstack-ironic19:55
sdanniTheJulia, JayF: hi! chmod +x to install.d script did work. I've posted the WIP code. Feel free to make comments!19:55
JayFgreat!19:55
TheJulia\o/19:56
TheJuliaawesome19:56
sdanniyeah19:56
sdannithe code is still under testing19:56
sdanninot sure when it will crash...19:57
TheJuliawhen it will crash!?? eek19:58
-openstackstatus- NOTICE: The Gerrit service on review.opendev.org is being restarted quickly to troubleshoot an unexpected error condition, downtime should be less than 5 minutes20:00
*** openstackgerrit has quit IRC20:02
sdannifingers crossed!20:03
*** k_mouza has joined #openstack-ironic20:14
*** jhesketh has quit IRC20:16
*** k_mouza has quit IRC20:19
*** jhesketh has joined #openstack-ironic20:23
*** zzzeek has quit IRC20:30
*** zzzeek has joined #openstack-ironic20:31
*** juanoterocas has joined #openstack-ironic20:33
juanoterocasHi, perhaps this is a basic question, but I need some help setting up ironic with microstack, is the ironic.conf the same file to configure ironic conductor?20:35
ianwdtantsur: hey, i'm in UTC+10; i can chat or another option is to have it as a topic in the infra meeting for a wider exposure20:35
juanoterocasinfra meeting?20:39
juanoterocassorry idk if this message is for me20:39
*** anuradha1904 has quit IRC20:40
*** ociuhandu has quit IRC20:50
*** derekh has quit IRC21:01
*** derekh has joined #openstack-ironic21:02
eanderssonIs it not possible to have a build step that reboots?21:07
TheJuliajuanoterocas: ironic api and conductor can have different config files but should be fine with the same, the documentation details the settings that must align between the two21:16
*** derekh has quit IRC21:16
TheJuliaeandersson: should be totally possible21:16
TheJuliawhat seems to be the issue21:17
* TheJulia asks while pizza is cooking21:17
eanderssonIn Victoria after restart it tries to connect to the IPA agent and then fails21:17
TheJuliaAhh... yes21:17
eandersson> Max retries exceeded with url: /v1/commands/?wait=false21:17
*** juanoterocas has quit IRC21:17
TheJuliaso what is triggering the reboot?21:17
eanderssonWe wrote a custom hardware manager21:17
TheJuliaahh21:17
TheJuliaokay21:18
TheJuliaso yes, a reboot is totally possible, the thing is the conductor HAS to know about it21:18
TheJuliawhat your encountering is the agent token is gone21:18
eanderssonWe set 'reboot_requested': True,21:18
TheJuliaerr, gone on the agent side and can't be re-obtained unless the existing token is wiped out21:18
eanderssonand that is what is handling the restart21:18
TheJuliaso your either hitting a bug, or you've got custom code setting that state and triggering the reboot?21:18
eanderssonIt fails almost instantly (1 minute) after the restart has begun21:19
TheJuliayup21:19
TheJuliabecause once the token has been set, they both require it. The agent won't even fire up without the token to give it commands21:19
eanderssonAre you sure it has to do with the token, because the IPA isn't running because it takes 5-10 minutes to reboot21:19
TheJuliahttps://docs.openstack.org/ironic/latest/admin/agent-token.html21:19
TheJuliano, not in that case21:20
TheJuliahmm... do you know where in the ironic code it is raising the max retries error following up21:20
TheJuliait sounds like it is trying to handle a command from the original state21:20
TheJuliaor original reboot sequence21:20
eandersson> ERROR ironic.conductor.utils [req-X None None] Node Y failed deploy step {'step': 'update_firmware', 'priority': 10, 'interface': 'deploy', 'reboot_requested': True}. Error: Connection to agent failed: Failed to connect to the agent running on node Z for invoking command deploy.execute_deploy_step. Error: HTTPSConnectionPool(host='IP',21:21
eanderssonport=9999): Max retries exceeded with21:21
TheJuliaagent side update_firmware deploy_step?21:22
TheJuliai guess that is obvious, interface is deploy21:23
eanderssonYea21:23
TheJuliahmmm21:23
*** rcernin has joined #openstack-ironic21:23
eanderssonTo me it seems that it instantly fails because it expects that the IPA agent is up and running, which can't be true because it is currently rebooting.21:23
TheJuliaso it is the step in the hardware manager that triggers the actual reboot?21:25
eanderssonYea21:25
*** ociuhandu has joined #openstack-ironic21:25
eanderssonHonestly at the moment the upgrade script is a bash that has an echo and exit 0 in it21:25
TheJuliayeah... this is a bug21:25
TheJuliahttps://github.com/openstack/ironic/blob/master/ironic/drivers/modules/agent_base.py#L1146-L114821:26
TheJuliaexcept your likely dying here https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/agent_base.py#L108621:26
TheJuliano command status to get mid-reboot21:32
TheJulialikely your conductor is processing another heartbeat or another one was recorded which was inflight and queued up21:32
eanderssonInteresting. We will move this to a clean job for now.21:33
eanderssonAssuming of course that it wont be having the same issue there21:33
*** ociuhandu has quit IRC21:45
eanderssonAnother fun one.. why would erase_devices_metadata fail during cleaning, but not during build/deploy?21:48
eandersson> Failed to erase the metadata on the device(s): "/dev/sda3": Unexpected error while running command.21:48
*** sean-k-mooney has joined #openstack-ironic21:55
TheJuliabecause it is an automatic step for cleaning21:56
TheJuliaonly21:56
TheJuliaeandersson: software raid?21:56
* TheJulia wonders if post pizza nap can be a thing today21:57
eanderssonOh! I didn't actually confirm if it was run during deploy, but21:57
eanderssonhttps://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/hardware.py#L175221:57
eanderssonI don't think these host have raid.21:57
TheJuliaoh, heh21:57
TheJuliainteresting....21:57
TheJuliaI wonder why it failed on that partition...21:58
sean-k-mooneyquick question what is the best way to promt ironic to see if it can reconnecto to the ipmi interface on a node. my switch reset its vlan config so ironc lost conenctivity to the nodes and set them to maintance mode. if i just unset mainatnce mode will that be enough or is tthere a way i can check if the ipmi driver can connect before i do that?21:58
TheJuliaI guess  maybe it didn't get cleaned through cleaning?21:58
*** ociuhandu has joined #openstack-ironic21:58
TheJuliasean-k-mooney: what version of ironic?21:58
TheJuliasean-k-mooney: you should be able to remove them from maintenance mode and ironic should update the power state as it polls through them over the next five minutes21:59
sean-k-mooneyussuri i think21:59
TheJuliaThere is a way to force it to re-check the power state but it is not surfacing in my brain right now21:59
TheJuliayeah, you just need to unset it, and _should_ do it itself if memory serves21:59
sean-k-mooneyok cool22:00
TheJuliais there an error reported in last_error?22:00
sean-k-mooneyya ill pastbin it one sec22:00
sean-k-mooneywell there is an error in the node show22:00
TheJuliabut nodes have a power state?22:01
sean-k-mooneyhttp://paste.openstack.org/show/800342/22:01
TheJuliayeah, ironic should pull that node out of that state in <15 minutes or something like that22:02
sean-k-mooneyhttp://paste.openstack.org/show/800343/ that is the full show22:02
TheJuliabut you can manually remove maintenance mode and it should record powers tate22:02
sean-k-mooneyok i have a 50% chance that i tagged the port with the correct vlan22:02
sean-k-mooneyif not it sthe other port on each node that is used for ipmi :)22:02
*** ociuhandu has quit IRC22:02
TheJuliaif you remove maintenance and ask it to lookup the boot device, that should tell you since it will wait for the call to come back22:03
TheJuliathat way you can figure out if the tag is right or wrong really quickly22:04
sean-k-mooneyopenstack baremetal node boot device show <node uuid> right22:04
sean-k-mooneysee all the pro hacks when you ask in the right place22:05
TheJuliaI don't know that one off the top of my head22:05
sean-k-mooneythanks ill give that a try22:05
*** iurygregory has quit IRC22:05
sean-k-mooneyTheJulia: thats what osc was suggesting22:05
sean-k-mooneyso if its not that its an osc help bug22:05
TheJuliacool, that won't work for every driver unfortunately22:05
TheJuliabut for ipmi it definitely will22:06
*** iurygregory has joined #openstack-ironic22:06
sean-k-mooneyi guess i could just exec into the condcutor container and use ipmi too and see if i can connect too22:06
TheJuliathat is basically what the command does22:07
sean-k-mooneythe condcutor is where ipmitool is invoked form right?22:07
TheJuliaasks it for bootparam 5 or something like that22:07
TheJuliasean-k-mooney: yeah22:07
sean-k-mooneycool wasnet sure if it was there or nova_compute_ironic but im pretty sure that just calls the ironci rest api22:08
sean-k-mooneyhehe and with a 50% chance of being right i was wrong22:11
sean-k-mooneybut i can now list the boot device so it should be working again22:12
sean-k-mooney... i almost closed my swtich config without hittin save config... thats why my swich lost its config the last time22:13
sean-k-mooneyits both a blessing an a curse that enterpise switches do that22:14
*** iurygregory has quit IRC22:20
*** ociuhandu has joined #openstack-ironic22:22
*** ociuhandu has quit IRC22:26
*** iurygregory has joined #openstack-ironic22:35
*** rloo has joined #openstack-ironic22:44
*** rloo has quit IRC22:49
*** rloo has joined #openstack-ironic22:50
*** rloo has quit IRC22:51
*** rloo has joined #openstack-ironic22:52
iurygregoryfunny, redfish on idrac8 can send response without body and this makes sushi throw an Exception =( https://opendev.org/openstack/sushy/src/branch/master/sushy/exceptions.py#L10022:56
rpiosoiurygregory: Which iDRAC firmware version?22:58
iurygregoryrpioso, idrac8 2.75.75.7522:58
iurygregorywe found on an R730xd22:58
iurygregorywhen trying virtualmedia boot22:59
rpiosoiurygregory: Plain old vmedia boot or DHCP-less?23:00
rpiosoiurygregory: What's the BIOS version?23:00
TheJuliaWow, it is board nominations time already23:01
*** iurygregory_ has joined #openstack-ironic23:02
TheJuliasean-k-mooney: for years I was in the habit of always trying to reboot switches if I could after making major config changes in maintenance windows, just to be sure it would come up in a happy state before I closed out the window.23:02
iurygregory_my connection is strange today =(23:03
TheJuliaiurygregory_: a response without a body? a 202 or 204?23:03
iurygregory_TheJulia, well it's a 401 from Session... but on idrac9 the response has a body on idrac8 it doesn't23:04
*** iurygregory has quit IRC23:04
iurygregory_need to get full logs from conductor to see what happen in the machine23:04
iurygregory_finally they sent the full logs -.-' going to look tomorrow morning23:06
iurygregory_midnight already and I'm not going to look at logs without coffee =D23:07
*** iurygregory_ is now known as iurygregory23:08
TheJuliaiurygregory_: we should double check the HTTP RFCs because I'm not sure 4xx requires a body23:08
iurygregorynoted23:08
iurygregorywill look at it in the morning to make sure =)23:09
TheJuliaiurygregory: was the email you indicated an email to the outreachy intern that emailed us?23:11
iurygregoryTheJulia, nope, but it's my email on gerrit (the default one)23:12
iurygregoryand also on the storyboard https://storyboard.openstack.org/#!/story/200819323:13
TheJuliaOkay, I'm going to reply to the other email that came in a little while ago in the morning. I'm running out of spoons23:14
TheJuliaeandersson: if you can backtrack through you rlogs and see if the command to reboot is carrying the same request id as the status check that timed out. If so then we know it is just stacking commands up on heartbeat and that is a separate problem than what I'm thinking and needs to be handled a little differentlyu23:19
iurygregorynp o/23:20
iurygregorygood night23:20
* TheJulia will likely wander back, just... brains23:20
TheJuliaand I need to start thinking about dinner in a little while23:20
jandersgood morning Ironic o/23:30
*** ociuhandu has joined #openstack-ironic23:48
*** ociuhandu has quit IRC23:53

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!