Monday, 2020-07-13

*** diurnalist has quit IRC00:42
*** janders has joined #openstack-ironic00:55
*** ociuhandu has joined #openstack-ironic01:33
*** ociuhandu has quit IRC01:38
*** ricolin has joined #openstack-ironic02:33
*** ociuhandu has joined #openstack-ironic03:10
*** ociuhandu has quit IRC03:14
*** mkrai has joined #openstack-ironic03:24
*** janders has quit IRC03:27
*** diurnalist has joined #openstack-ironic04:40
*** ricolin_ has joined #openstack-ironic04:56
*** ricolin has quit IRC04:57
*** abdysn has joined #openstack-ironic05:02
*** Abdallahyas has joined #openstack-ironic05:07
*** Abdallahyas has quit IRC05:09
*** Abdallahyas has joined #openstack-ironic05:10
*** abdysn has quit IRC05:11
*** diurnalist has quit IRC05:45
*** rcernin has quit IRC06:02
*** rcernin has joined #openstack-ironic06:11
arne_wiebalckGood morning, ironic!06:59
*** mkrai has quit IRC07:03
*** yolanda has joined #openstack-ironic07:07
openstackgerritMerged openstack/networking-generic-switch master: change segment id type to int  https://review.opendev.org/74017107:15
iurygregorygood morning arne_wiebalck and Ironic!07:21
arne_wiebalckhey iurygregory o/07:22
iurygregoryif anyone has patches for ipa please recheck07:24
*** dougsz has joined #openstack-ironic07:31
*** abdysn has joined #openstack-ironic07:34
*** Abdallahyas has quit IRC07:37
*** dougsz has quit IRC07:38
*** janders has joined #openstack-ironic07:49
*** lucasagomes has joined #openstack-ironic07:56
*** mkrai has joined #openstack-ironic07:58
*** mkrai has quit IRC08:04
*** mkrai_ has joined #openstack-ironic08:05
*** rcernin has quit IRC08:06
openstackgerritDoug Szumski proposed openstack/ironic-python-agent master: Fix bootloader install issue with MDRAID  https://review.opendev.org/74033108:07
*** ociuhandu has joined #openstack-ironic08:10
*** ociuhandu has quit IRC08:16
*** mkrai has joined #openstack-ironic08:23
*** mkrai_ has quit IRC08:24
*** dtantsur|afk is now known as dtantsur08:34
dtantsurmorning ironic08:34
jandersdtantsur: g'day! :)08:38
dtantsurjanders: hey-hey, how is your 2nd week starting? :)08:38
jandersit's looking good, just testing my "patch" on the beaker machine08:40
iurygregorygood morning dtantsur08:45
iurygregoryhey janders o/08:45
jandershey iurygregory08:45
iurygregorydtantsur, FYI stable/train is broken but we have a fix08:45
dtantsurack, anything to review?08:45
iurygregoryhttps://review.opendev.org/#/c/740529/08:45
patchbotpatch 740529 - ironic (stable/train) - add tempest boot_mode config - 1 patch set08:46
iurygregorythis is the most important atm08:46
iurygregoryI noticed on saturday08:46
*** mkrai has quit IRC08:46
*** mkrai_ has joined #openstack-ironic08:47
dtantsuriurygregory: +A thanks08:47
iurygregorydtantsur, np08:47
iurygregoryI'm looking at the failures in ironic-lib08:48
dtantsurprobably DIB still broken? <- a wild guess08:48
iurygregoryI think so08:49
iurygregorythe error is from diskimage-builder08:49
*** ociuhandu has joined #openstack-ironic08:50
iurygregoryI'm wondering here why on the experimental job for ipa we have IRONIC_INSPECTOR_BUILD_RAMDISK True but we don't even run inspector on the experimental jobs08:53
iurygregoryO.o08:53
iurygregoryalso wondering if we still need the experimental jobs...08:53
*** abdysn has quit IRC08:55
*** abdysn has joined #openstack-ironic08:55
*** Abdallahyas has joined #openstack-ironic08:57
dtantsurOo08:58
dtantsuryeah, no idea08:58
iurygregoryfor me they seem to be covered by the jobs we run08:58
iurygregoryonly thing got my attention was that one job uses VM with 256 ram08:59
iurygregoryXD08:59
iurygregoryI'm not even sure if it will work hehe08:59
*** derekh has joined #openstack-ironic09:00
iurygregoryand both experimental are testing partition-ipmi-iscsi-tinypa09:00
iurygregoryone has bios in the name (but both will use bios since is the default lol)09:00
*** abdysn has quit IRC09:00
*** Lucas_Gray has joined #openstack-ironic09:01
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic-python-agent master: Refactor ipa-jobs  https://review.opendev.org/74064109:02
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic-python-agent-builder master: Update ipa jobs  https://review.opendev.org/74051909:05
janderswhile testing my little libselinux patch on the beaker machine I'm again hitting the mariadb suspected-bug09:07
jandershow to go about this? on one hand I'm not trying to fix up the mariadb stuff just yet... but on the other hand with this issue ongoing I can't really prove my fix is valid09:07
iurygregoryjanders, can you try downgrade the package?09:07
jandersthe mariadb one?09:07
iurygregoryyeah09:08
dtantsurjanders: make sure the packages are installed correctly; then switch to centos 809:08
jandersok!09:08
jandersthank you dtantsur09:08
jandersgiven I'm reaching the DB stage I reckon packages are all good now, I will kick rebuild to c8 in beaker09:09
*** sshnaidm|afk is now known as sshnaidm09:09
*** alexmcleod has joined #openstack-ironic09:18
*** k_mouza has joined #openstack-ironic09:21
*** Abdallahyas has quit IRC09:25
*** abdysn has joined #openstack-ironic09:26
*** aedc_ has joined #openstack-ironic09:27
openstackgerritDmitry Tantsur proposed openstack/ironic master: Decompose the core deploy step on iscsi and ansible deploy  https://review.opendev.org/73118009:36
*** rcernin has joined #openstack-ironic09:53
arne_wiebalckftarasenko: I checked the code and retried once more, and now I see the EFI partition being created on all holder disks (as expected). In my  initial test, the RAID was degraded, not sure this is related or if there is sth wrong with my test node.09:59
arne_wiebalckftarasenko: From what I see atm, the EFI partitions are created correctly.09:59
ftarasenkoarne_wiebalck: I've rechecked and seems that everything is correct and EFI partitions created where required10:00
arne_wiebalckftarasenko: thanks!10:01
*** ricolin_ has quit IRC10:01
ftarasenkoThanks. BTW, one more question. Why don't use partition LABEL as rootfs_uuid? It's easy to define unique LABEL before image build.10:01
*** rcernin has quit IRC10:03
openstackgerritMerged openstack/ironic stable/train: add tempest boot_mode config  https://review.opendev.org/74052910:04
*** rcernin has joined #openstack-ironic10:07
arne_wiebalckftarasenko: I know why it did not create the ESP everywhere in my case: the RAID was degraded, so the code to get the holder disks did not get all disks and it only created the ESP on the ones reported by mdadm. Still not sure if this is an issue with my node or a general issue: we do not have UEFI RAID at scale yet, only on a test node.10:08
arne_wiebalckftarasenko: Did you ever notice degraded RAIDs after deployment with UEFI?10:09
arne_wiebalckftarasenko: For your label question: there is probably no specific reason, it simply uses the uuid as this was used in the case of partition images IIRC.10:11
*** janders has quit IRC10:11
ftarasenkoarne_wiebalck: I've seen degraded RAID while deployment, but in syncs correctly after boot. Never tried to troubleshoot.10:11
arne_wiebalckftarasenko: degraded as in "not sync'ed yet" or as in "missing device"?10:12
ftarasenkoarne_wiebalck: not synced10:12
arne_wiebalckftarasenko: right10:13
arne_wiebalckftarasenko: this is normal I'd think10:13
arne_wiebalckftarasenko: I saw a missing device10:13
*** alexmcleod has quit IRC10:14
*** janders has joined #openstack-ironic10:26
*** rcernin has quit IRC10:40
*** Lucas_Gray has quit IRC10:59
*** Lucas_Gray has joined #openstack-ironic11:00
*** alexmcleod has joined #openstack-ironic11:03
dtantsurdo I get it right that our CI is unstable/broken again?11:19
*** ricolin_ has joined #openstack-ironic11:23
jandersI'm testing  my bifrost "patch" on C8 (with full updates)11:24
jandersI hit this11:24
jandershttp://paste.openstack.org/show/795873/11:24
jandershave you guys seen anything similar before?11:24
jandersI tried backing out my patch (even though I thought chances of it causing this are near-zero as that code path should only trigger on Fedora)11:25
jandersno change11:25
dtantsur'gcc': No such file or directory11:25
dtantsurWe forget to install gcc somewhere :)11:25
jandersdtantsur well spotted and thank you! :)11:26
dtantsurjanders: a patch is welcome too ^^11:26
jandersno worries, can do11:26
dtantsurvery few people try bifrost on a REALLY clean machine apparently :)11:27
* dtantsur has ansible playbooks to run on each new environment11:27
janderswe dont care about people who dont have gcc installed, right? :)11:27
dtantsurwe do, we're just not these people :)11:27
jandershaha :) okay11:27
jandersokay I asked this last week but I didn't quite understand the response11:28
janderswhen I have a fix commited and ready for review pending successful testing11:28
jandersdo I push this commit in my branch to the repo and then git clone that on the test server?11:28
jandersI don't want to submit the code for review, but it would be handy to have it pushed so I can just clone and test11:29
janderswhat's the best way to achieve this outcome?11:30
dtantsurjanders: if you really, REALLY don't want anyone to see your WIP patch, you can push or rsync directly from your laptop to the machine11:30
dtantsurhowever, this case is rare. if you're just not sure the patch works, submit it for review and mark as Workflow-1.11:30
jandersok!11:30
dtantsurthen if it works - remove Workflow, if it does not - update11:30
jandersnow I got it, will try this now11:31
dtantsurhonestly, I personally have no problems uploading really raw patches to gerrit for my convenience :)11:31
jandersthank you dtantsur11:31
*** mkrai_ has quit IRC11:33
*** tkajinam has quit IRC11:37
openstackgerritJacob Anders proposed openstack/bifrost master: Removing libselinux-python package from Fedora dependencies  https://review.opendev.org/74067811:41
jandersokay... now how do I mark it as "workflow-1"?11:42
dtantsurjanders: click "Reply..." you'll see it11:43
jandersokay... it was hiding there... done11:43
jandersdoes this look right to you?11:44
dtantsurjanders: yep. marked +2 since the change looks correct from my perspective.11:50
dtantsurunrelated: this is ridiculous https://storyboard.openstack.org/#!/story/200791811:50
jandersthat's a lot of log lines...11:51
dtantsuryep :(11:53
dtantsurand it's not a trivial issue to fix: we cache these registries per session, but we create a new session per each action....11:54
jandershttp://paste.openstack.org/show/795875/11:54
janderson to the next problem11:54
jandersdoes this (looks like failed compile) ring any bells?11:54
dtantsurjanders: we don't install python3-devel too? Oo11:54
dtantsurthis starts being weird11:55
dtantsurmmm, no, we should be installing it11:55
dtantsurjanders: check the logs before that, did we install python3-devel before?11:55
dtantsurit was supposed to have happened very early11:55
janderschecking if I followed the exact same install procedure on both my FC32 and Beaker C811:58
jandersmy FC32 had python3-devel installed before I started playing with bifrost12:00
jandersc8 took off as well as soon I pre-installed python3-devel12:00
jandersI will read through dependencies in detail12:01
dtantsurpython3-devel is included twice in bifrost, it absolutely has to get installed12:01
dtantsur(twice: in bindep and in dependencies)12:02
jandershttp://paste.openstack.org/show/795877/12:07
jandersyeah  I just came to the same conclusion12:08
*** rh-jelabarre has joined #openstack-ironic12:12
janderswhat I haven't found yet is what is pulling in those tasks12:12
*** rh-jelabarre has quit IRC12:12
*** rh-jelabarre has joined #openstack-ironic12:12
jandershowever having said that, I have a working bifrost/ironic on C8 on the test node, so I'm happy with my mini-patch12:14
jandersshould I remove the "workflow-1" tag from it?12:15
iurygregoryI would say we need a releasenote12:20
*** mkrai has joined #openstack-ironic12:25
*** derekh has quit IRC12:26
jandersok! will do that tomorrow :)12:28
iurygregorysure =)12:29
jandersI haven't managed to find what part of the code is calling playbooks such as playbooks/roles/bifrost-ironic-install/defaults/required_defaults_RedHat_family.yml - if you have time, pointers will be welcome - will have a look tomorrow12:34
jandersI'm wrapping up, see you guys tomorrow o/12:36
jandershave a good one12:36
*** mkrai has quit IRC12:59
zombieJuliagood morning13:06
*** zombieJulia is now known as TheJulia13:06
iurygregorygood morning TheJulia =)13:08
*** derekh has joined #openstack-ironic13:09
TheJuliaI hope everyone had a wonderful weekend13:09
iurygregoryalmost13:10
dtantsurmorning TheJulia13:13
dtantsura pretty good one indeed. you?13:13
TheJuliaEh, didn't really do much besides start to get parts in for my cnc projects and started assembly13:14
iurygregoryDmitry had a nice dinner =D13:14
TheJuliaI saw13:15
* TheJulia goes and gets coffee which will hopefully help wake her up13:15
iurygregorydtantsur, do you think we need to backport for stein and older branches https://review.opendev.org/#/c/740529/ ?13:17
iurygregorygerrit is sleeping ? O.o I did a cherry-pick and nothing pop up here O.o13:20
dtantsuriurygregory: I did :) and a nice long walk along the Rhine, which I didn't photo because I've lost the habit of taking my phone with me on city walks13:22
iurygregorydtantsur, awesome!13:22
dtantsurah, the CI breakage is because of the new stevedore release13:23
dtantsurokay, then we just sit and wait until 3.0.1 is in u-c13:23
iurygregoryouch13:23
iurygregorywoot this is strange13:24
iurygregoryhttps://review.opendev.org/#/c/740692/ doesn't show any ipa-tempest-X-bios-ipmi-direct-tinyipa-src on https://zuul.openstack.org/status13:25
iurygregoryO.o13:25
*** aedc_ has quit IRC13:26
*** Goneri has joined #openstack-ironic13:26
patchbotpatch 740529 - ironic (stable/train) - add tempest boot_mode config (MERGED) - 1 patch set13:27
patchbotpatch 740692 - ironic-python-agent (bugfix/6.2) - Update jobs - 1 patch set13:27
*** sean-k-mooney has quit IRC13:27
*** bdodd has quit IRC13:27
*** irclogbot_3 has quit IRC13:27
dtantsurPSA folks ^^^13:27
*** sean-k-mooney has joined #openstack-ironic13:28
*** bdodd has joined #openstack-ironic13:28
*** irclogbot_3 has joined #openstack-ironic13:28
iurygregoryPSA? o.o13:28
TheJulialooks like our CI failures are killing us13:34
iurygregoryat least we fixed stable/train for ironic13:34
TheJuliadtantsur: uggggghhhh13:34
iurygregorybefore the BOOM13:34
iurygregoryXD13:34
*** abdysn has quit IRC13:35
*** tzumainn has joined #openstack-ironic13:42
dtantsuriurygregory: we probably need to backport CI-fixing changes everywhere13:56
iurygregorydtantsur, ack13:56
dtantsurwe need https://review.opendev.org/#/c/740681/ or wait for the 3.0.1 release to get unblocked13:58
patchbotpatch 740681 - requirements - Don't use stevedore 3.0.0 which breaks Glance gate - 1 patch set13:58
TheJulialooks like we need to be really good keeping a list for backports14:07
TheJuliaI'd hate to say it... possibly even a spreadsheet or something14:08
iurygregoryTheJulia, today when I was looking at the ipa jobs, I was wondering if we still need the experimental jobs...14:10
iurygregorythe only big difference is that one of the jobs runs with 256 MB14:11
iurygregoryfor VM_SPECS14:11
dtantsurTheJulia: I've seen some projects (oslo?) using a custom label in gerrit for potential backports14:11
iurygregoryI know some projects have Backport-Candidate14:13
iurygregoryfor example barbican14:13
iurygregoryand they also have a nice dashboard14:14
iurygregoryhttps://review.opendev.org/#/dashboard/?foreach=(project:openstack/barbican%20OR%20project:openstack/python-barbicanclient%20OR%20project:openstack/castellan%20OR%20project:openstack/barbican-specs)%20status:open&title=Barbican%20Review%20Dashboard&Needs%20Approval=NOT%20label:Workflow%3E=1%20NOT%20label:Workflow%3C=-1%20label:Verified%3E=1%20NOT%20owner:self%20label:Code-Review%3E=2%20NOT%20label:Code-Review-1%20bran14:14
iurygregorych:master%20NOT%20label:Code-Review%3C=-2&Bug%20Fixes%20without%20Negative%20Feedback=NOT%20label:Workflow%3E=1%20NOT%20label:Workflow%3C=-1%20label:Verified%3E=1%20NOT%20owner:self%20NOT%20label:Code-Review%3C=-1%20NOT%20label:Code-Review%3E=2%20limit:50%20branch:master%20topic:^bug.*&Blueprints%20without%20Negative%20Feedback=NOT%20label:Workflow%3E=1%20NOT%20label:Workflow%3C=-1%20label:Verified%3E=1%20NOT%20owner:14:14
iurygregoryself%20NOT%20label:Code-Review%3C=-1%20NOT%20label:Code-Review%3E=2%20limit:50%20branch:master%20topic:^bp.*&Without%20Negative%20Feedback=NOT%20label:Workflow%3E=1%20NOT%20label:Workflow%3C=-1%20label:Verified%3E=1%20NOT%20owner:self%20NOT%20label:Code-Review%3C=-1%20NOT%20label:Code-Review%3E=2%20limit:50%20branch:master%20NOT%20topic:^bug.*%20NOT%20topic:^bp.*&With%20Negative%20Feedback=NOT%20label:Workflow%3C=-1%214:14
iurygregory0NOT%20label:Verified%3C=-1%20NOT%20owner:self%20label:Code-Review-1%20limit:20%20branch:master&Needs%20Reverify=label:Verified%3C=-1%20branch:master%20NOT%20label:Code-Review%3C=-1%20NOT%20label:Workflow%3C=-1&5%20Days%20Without%20Feedback=NOT%20label:Workflow%3E=1%20NOT%20label:Workflow%3C=-1%20label:Verified%3E=1%20NOT%20owner:self%20NOT%20label:Code-Review%3C=2%20age:5d%20branch:master&Own%20Patches=owner:self&Pat14:14
iurygregoryches%20-2'd=label:Code-Review%3C=-2,self&Stable%20Branches=branch:^stable/.*14:14
iurygregorydamm I should paste the tiny url14:14
iurygregoryhttps://tinyurl.com/yd3x48fl14:14
TheJuliadtantsur: I see it needing to be more than just a label because there is should it be backported, can it be backported, and is an entirely new fix needed, or not-applicable across a matrix of branches14:14
dtantsurwe could start with a label though14:15
TheJuliaI'm likely thinking one level higher for tracking, not reviewer visibility14:15
TheJuliaTrue14:15
dtantsurI'm afraid that maintaining such a spreadsheet would be too much work to keep up with it constantly14:15
TheJuliaWe've just seen how getting people to use consistant labels14:16
dtantsurremember when we had a trello? :)14:16
TheJuliaoh yeah14:16
TheJuliaWe've got... how many bug fixes sitting in review right now?14:16
TheJulialike 8?14:16
TheJulia9?14:16
dtantsursomething like that. at least.14:16
TheJuliaI guess I should.. at a minimum, once they merge upstream make sure we have backports or are moving forward in that direction14:17
dtantsurwe could use storyboard boards, but again, maintaining them will be a burden, which I don't want to fall on you (or me)14:17
TheJuliabecause it is a matter of catching it too14:17
TheJuliayeah, we need "easy"14:17
dtantsurso, maybe Backport: -1 = not backportable, +1 - to backport, +2 - backported?14:18
iurygregory^ makes sense to me14:18
TheJuliayeah, that does seem nice and simple14:22
TheJuliajust need to make sure we try and create the backports for all the branches at one time14:22
TheJuliajohnthetubaguy: Hey, Curious if there is a desire to revisit https://review.opendev.org/#/c/699953/? I didn't see your reply back in December14:26
patchbotpatch 699953 - ironic - WIP: make port binding fatel - 3 patch sets14:26
TheJuliaugh, looks like my grocery delivery is going to be stupidly early14:28
*** rloo has joined #openstack-ironic14:29
iurygregorythat's good I assume?14:31
TheJuliayeah, except look like they will be here in 20 minutes14:31
dtantsurhere they're call it "over-punctual" :D I think my wife even took her time at some point to explain them that "over-punctual" means "NOT punctual"14:31
TheJuliadtantsur: Did we discuss the step priority overrides topic sufficently last time we met?14:31
iurygregoryhehehehe14:32
TheJuliaYeah, I'm now the next delivery on the tracking14:32
dtantsurTheJulia: I think we discussed it enough, I've updated the story with a nicer format and more explanations14:33
iurygregory\o/14:33
TheJuliadtantsur: should we revisit for rfe review?14:33
dtantsurTheJulia: yeah, maybe a short revisit14:33
TheJuliak14:37
*** beekneemech is now known as bnemec14:38
*** diurnalist has joined #openstack-ironic14:45
*** Goneri has quit IRC14:50
*** uzumaki has joined #openstack-ironic14:52
uzumakimorning ironic! o/14:52
dtantsurhi uzumaki14:53
uzumakihow's it going dmitry?14:54
uzumakiI'm so spent today.. urghh.. mondays are the worst14:54
dtantsurMondays are not great indeed, I'm not sure where has the most of the day gone14:55
dtantsurthe weekend was nice though14:55
dtantsuruzumaki: https://twitter.com/morgantales/status/128203254159625420814:56
uzumakiyeah and before you realize, your eyes are feeling flaky.. your brain is giving high latency reponse with so much packet loss, and half the funciton calls are returning error codes..14:56
uzumakiOhhh! many many congratulations!\o/ \o/ \o/ <confetti> <horns> <cheers> <party popper> <firecrackers> dtantsur14:57
dtantsurheh, thanks :D14:57
dtantsuruzumaki: but yep, packet loss is especially a problem on Monday14:57
uzumakidtantsur, it really is, happened 3 times today, I forget what I wasn going to do, and I realize that after 15-20mins14:58
dtantsuronly 3 times? that would be my regular day >_<14:58
TheJulia*sigh* looks like the driver is intentionally waiting until after 8 AM to pull up14:58
uzumakihah! Not to mention my sleep cycles are so bad these days.. 2am-7pm, instead of the 6/7am-9/10pm14:59
uzumakiTheJulia, time bomb? xD14:59
TheJuliabasically14:59
dtantsurTheJulia: do you want me to start or add a co-chair just in case?14:59
TheJuliathey are on my street now14:59
TheJuliadtantsur: I'll start it in a moment14:59
TheJuliaand I'll co chair you and then will likely pull up14:59
uzumakiTheJulia, lol14:59
*** kaifeng has joined #openstack-ironic14:59
uzumakico chair what? how can it go both ways?15:00
TheJulia#startmeeting ironic15:00
openstackMeeting started Mon Jul 13 15:00:06 2020 UTC and is due to finish in 60 minutes.  The chair is TheJulia. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: ironic)"15:00
openstackThe meeting name has been set to 'ironic'15:00
iurygregoryo/15:00
TheJulia#chair dtantsur15:00
dtantsuro/15:00
openstackCurrent chairs: TheJulia dtantsur15:00
rpioso\o15:00
bdoddo/15:00
uzumakioh, the meeting. yayyy \o/15:00
ajyao/15:00
TheJuliaand they walked up15:00
dtantsur#link https://wiki.openstack.org/wiki/Meetings/Ironic is our agenda15:00
kaifengo/15:01
dtantsur#topic Announcements / Reminder15:01
*** openstack changes topic to "Announcements / Reminder (Meeting topic: ironic)"15:01
rlooo/15:01
uzumakidtantsur, we are ironic and ironic is us15:01
dtantsur#info The main news: we have done our first batch of releases according to the new model15:01
arne_wiebalcko/15:01
dtantsurThe bugfix/X.Y branches have been created, but the devstack jobs need some care on them15:01
dtantsur(bifrost seems to work normally, good job bifrost!)15:02
iurygregorywe probably need to pin on the zuul config...15:02
iurygregoryfor the jobs we have the bugfix branches15:02
dtantsurset some overrides, I guess15:03
mgoddard\o15:03
dtantsur#link https://www.eventbrite.com/e/opendev-hardware-automation-registration-104569991660 OpenDev: Hardware Automation15:03
iurygregorydtantsur, exactly15:03
dtantsurthis ^^ event is coming up next week. I'm not sure if there is a schedule already.15:03
*** Goneri has joined #openstack-ironic15:03
dtantsurah15:03
dtantsur#link https://www.openstack.org/events/opendev-2020/opendev-schedule-2 The schedule15:03
uzumakiwho's arranging/presenting this? dtantsur15:04
dtantsuruzumaki: this is a Foundation event, it's not really about presentations, more about discussions15:04
TheJuliauzumaki: it is more like a forum discussion15:05
dtantsurso not one person talking, but rather one moderator, a few experts and a bunch of random people who care15:05
uzumakioh! I see, must be cool, I might attend this one!15:05
dtantsuruzumaki: I highly recommend, if it plays nicely with your timezone15:05
uzumakibunch of random people who care? THAT'S ME!15:05
TheJuliaso some central topics/questions and the hope is discussion results15:05
rpiosoSounds interesting. Has it been in the works for a while?15:05
dtantsurrpioso: it has, yes. we've already had one such opendev event centered around large-scale deployments15:06
dtantsurTheJulia: you back and with stashes full of yummies?15:06
TheJuliadtantsur: putting stuff away now15:07
dtantsuranything else to announce or remind us of? (except for staying safe, washing hands and wearing masks?)15:07
TheJuliaJust for everyone to please take a little time to review this week. If we need to do ad-hoc review jams, I'm happy to host15:07
uzumakidid someone say jams?15:08
dtantsur:D15:08
dtantsurit's a good idea, actually15:08
iurygregorysounds good to me15:08
uzumakias long as there's jam15:08
dtantsurlet's get back to it in open discussions15:09
dtantsur#topic Review subteam status reports (capped at ten minutes)15:09
*** openstack changes topic to "Review subteam status reports (capped at ten minutes) (Meeting topic: ironic)"15:09
dtantsur#link https://etherpad.opendev.org/p/IronicWhiteBoard starting around like 29915:09
dtantsurThanks everyone who participated in the bug triaging, great progress!15:09
dtantsurI think as the next step we should clean up the RFE list a bit. Maybe propose some of RFE Review on the upcoming meetings.15:10
TheJuliaokay all done with putting things away15:12
uzumaki248? that's a lot of RFEs :O15:12
TheJuliadtantsur: ++15:12
* dtantsur throws TheJulia the remote control15:12
TheJuliaHeh15:12
dtantsuruzumaki: exactly, there is no way we're going to finish all of them. and we probably shouldn't.15:12
TheJuliaWe don't retire RFEs really because they are all valid ideas, but often we get 2-3 of them per major thing15:12
uzumakiit's just too much =O15:12
TheJuliaor things get resolved as bug fixes and we don't realize it15:13
dtantsurhappens ^^^15:13
dtantsurI've closed a bunch of duplicate RFEs already15:13
TheJuliaYup, anyway!15:13
uzumakican I help?15:13
dtantsuruzumaki: it may be a bit non-trivial without a lot of context. starting with bug triaging may be easier.15:14
uzumakiokay!15:14
dtantsuruzumaki: we've got some words written about it: https://docs.openstack.org/ironic/latest/contributor/bugs.html#triaging-guide15:14
uzumakitell me where to shoot15:14
uzumakigot it15:14
TheJuliauzumaki: but! if you want to ask questions in channel as related to RFEs, we may be able to provide additional context if you don't know off the top of your head. The thing is, it will go slow that way15:14
TheJuliaiurygregory: hey, is ngs + zuulv3 + grenade still problematic?15:15
iurygregoryTheJulia, totally =(15:15
iurygregoryI think people are afraid of grenade hehe XD15:15
TheJuliaiurygregory: should we just establish a time for a few people to jump on a call and try to work through it15:16
TheJuliaand yes, grenade has been the source of great sorrow15:16
uzumakiTheJulia, I realize that, that's why I was hesitant in asking.. I'll start with bug triage for a few days, let's see where that takes me15:16
iurygregoryTheJulia, sounds good, I can put up a doodle and send on openstack-discuss?15:16
TheJuliauzumaki: awesome :)15:16
uzumakiTheJulia, grenades are known to be that way "sources of pain"15:16
TheJuliaheh15:16
TheJuliaindeed15:16
dtantsurnot untrue15:16
TheJuliaarne_wiebalck: re scale is the only thing your aware of just memory consumption right now on the conductors?15:17
arne_wiebalckTheJulia: yes15:17
TheJuliaI guess standalone auth for bifrost is "Coming Soon"15:17
TheJuliaarne_wiebalck: mind if I update the item to note that?15:17
arne_wiebalckTheJulia: ofc not :)15:17
* uzumaki wonders why dtantsur always uses the fancy double negatives to make a point xD15:18
dtantsuruzumaki: double negations are normal in Russian, using them in English adds some spice15:19
uzumakidtantsur, if from spice you mean getting people to read it twice, yes, it does the trick15:19
dtantsurI mean precisely that, yes :)15:19
uzumakixD15:19
* iurygregory also uses double negatives =X15:20
* TheJulia unfortunately is not in the habit of double negatives15:20
TheJuliaRamdisk TLS unchanged?15:20
TheJuliaiurygregory: could you add links to the update as you propose the changes to simplify and reduce job load?15:21
* uzumaki remembers the Martin Luther speech when he says "I'm not unmindful.." that's a really spicy double negative15:21
TheJuliaAny update on compatability profiles?15:21
iurygregoryTheJulia, sure15:21
dtantsurTheJulia: ramdisk TLS unchanged15:21
dtantsurgot distracted with the power off + agent token issue15:22
TheJuliaI saw that was still a WIP which is why I didn't add that to the items to review this week15:23
TheJuliaI guess I'm good to proceed, just curious if there is an update for redfish compatability profiles since the current update is undated.15:23
TheJuliaarne_wiebalck, rpioso ^^^15:23
dtantsurI hope to finish it today-tomorrow15:23
TheJuliadtantsur: ack15:24
iurygregoryTheJulia, added to the weekly priorities15:24
TheJuliaiurygregory: beyond what is already proposed?15:24
TheJuliaAre we good to proceed for priorities for the week?15:24
arne_wiebalckTheJulia: re redfish, no, I did not get round to work on this yet15:24
iurygregoryI added to both status and the WP15:25
TheJuliaonward to priorities!15:26
TheJulia#topic Deciding on priorities for the coming week15:26
*** openstack changes topic to "Deciding on priorities for the coming week (Meeting topic: ironic)"15:26
TheJulia#link https://etherpad.opendev.org/p/IronicWhiteBoard15:27
TheJuliaStarting at line 12915:27
rpiosoarne_wiebalck, TheJulia: We're continuing to work on the Redfish Interoperability profile for sushy. Reviewed the progress with the team late last week. Initially focusing on the ServiceRoot and ComputerSystem resources. Once those are satisfactory-ish, we would like to demo. ETA ~1 week.15:27
* TheJulia removes merged items15:27
* rpioso will update the white board to reflect ^^^15:27
TheJuliarpioso: thanks15:27
rpiosoTheJulia: :-)15:28
TheJuliaSo I have a huge list of proposed items stariting at line  19715:29
TheJuliaany questions/concerns/items missing?15:29
uzumakiso many CI failures X(15:30
arne_wiebalckTheJulia: For the RAM patch, did you manage to test this with a real node?15:30
arne_wiebalckTheJulia: The code looks good to, I would just like to see what happens IRL :)15:31
TheJuliaarne_wiebalck: I haven't had time to15:33
TheJuliaI'm fairly sure on the result that I've noted being the case though, but yeah, would be good to get it loaded up and run the machine out of ram15:33
TheJuliaIs everyone good with the list?15:35
iurygregoryI'm15:35
* TheJulia starts brewing coffee for everyone15:35
dtantsurthe list looks good to me15:35
TheJuliaa lot of stuff is just CI failures or are almost there, so a recheck or quick review may be all that is needed on some of the patches15:36
TheJuliaAnyway! Shall we move on?15:36
dtantsur++15:37
TheJuliaWe don't seem to have any discussion topics, so I guess we would jump directly to the Baremetal SIG15:37
TheJulia#topic Baremetal SIG15:37
*** openstack changes topic to "Baremetal SIG (Meeting topic: ironic)"15:37
TheJuliaarne_wiebalck: Anything to discuss or to let people know about?15:37
arne_wiebalckTheJulia: no15:37
TheJuliaokay then!15:37
TheJuliaOnward to RFE Review!15:38
TheJulia#topic RFE Review15:38
*** openstack changes topic to "RFE Review (Meeting topic: ironic)"15:38
TheJuliadtantsur wanted us to quickly revisit his RFE from our last meeting15:38
dtantsur#link https://storyboard.openstack.org/#!/story/161801415:38
dtantsurI've updated it with a somewhat nicer format and more words15:38
TheJuliaSounds good to me15:39
TheJuliaWell, reads is more appropriate15:39
TheJuliaDoes anyone have any questions regarding this item?15:39
* TheJulia gets out the automated crickets15:40
iurygregorynone on my side15:41
*** gyee has joined #openstack-ironic15:41
TheJuliadtantsur: I think lazy consensus grants you rfe-approved15:42
dtantsur\o/15:42
openstackgerritAija Jaunteva proposed openstack/ironic-specs master: [RFE] Operationally efficient and fast-track bare metal hardware provisioning  https://review.opendev.org/74072115:42
dtantsurbtw, this work item is not too complex, so if anybody wants to take it to practice - be my guest15:42
TheJuliadtantsur: feel free to add rfe-approved to the item15:42
rlooseems like it'd be better to have one or two people indicating that they approve. can i suggest julia and iurygregory?15:42
iurygregoryI will add15:43
rloothx iurygregory!15:43
TheJuliaiurygregory: thanks15:43
TheJuliaWell then, onward to open discussion!15:43
TheJulia#topic Open Discussion15:44
*** openstack changes topic to "Open Discussion (Meeting topic: ironic)"15:44
TheJuliaDoes anyone have anything to bring up/discuss?15:44
iurygregoryyay it worked \o/15:44
kaifengwrt the steps override, could we have this on the driver fields?15:44
kaifengconfiguration options not always handy15:45
TheJuliakaifeng: I feel like that is likely a logical next step15:46
rpiosoTheJulia: We have updated the RFE https://storyboard.openstack.org/#!/story/2003594.15:46
TheJuliaI see.15:46
rpiosoTheJulia: A spec has been posted for review: https://review.opendev.org/#/c/740721/15:46
patchbotpatch 740721 - ironic-specs - [RFE] Operationally efficient and fast-track bare ... - 1 patch set15:46
TheJuliarpioso: We saw, that I think is going to require some time to digest since it seems to be mixing terms from other areas under a generic umbrella, so we may need to wrap our heads around what is each specific area it touches upon15:48
*** dsneddon has joined #openstack-ironic15:48
dtantsuryep, the very first request: formulate everything in driver-specific terms15:48
dtantsurI initially assumed you were about to rewrite half of ironic :)15:49
rpiosoTheJulia: I think you'll find it doesn't touch much and is optional :-)15:49
rpiosoTheJulia: LOL. We don't have that much ambition ;-)15:49
TheJuliarpioso: I mean in terms of word selection to describe the feature. I had the same reaction as dtantsur and that is going to take some time to grok15:49
TheJuliaI think the old feature name was rather succinct, personally15:50
TheJuliaI feel like we should add the spec to the list of items to review15:51
TheJuliaAny objections?15:51
rpiosoTheJulia: We feel it was too vendor specific. The change makes it available to other drivers and addresses a couple of operator pain points.15:51
*** dking has joined #openstack-ironic15:51
TheJuliarpioso: It would be good for more vendors to have similar behavior. Anyway, as I noted, it is going to take a little time to digest15:52
rpiosoTheJulia: And they will be able to if they'd like. It's generic. Implementation of the low-level interactions are vendor-specific.15:53
TheJuliarpioso: indeed15:53
rpiosoTheJulia: We've focused on the parts that we believe are of most general interest. Few words are spent on iDRAC-specific stuffs. We're trying to be considerate of the reviewers' time.15:54
TheJuliarpioso: I'm a little worried that it will have the reverse effect. Anyway, since I didn't have any objections I'll add it to the list15:55
TheJuliaAnyone else have anything to bring up before we wrap up the meeting?15:56
rpiosoTheJulia: Thank you so much!15:56
* TheJulia hears the autocricket 4000 make cricketing sounds15:57
rlooCRICKKKKKKet15:58
TheJuliaThanks everyone!15:58
TheJuliarloo: ++15:58
rloo:D15:58
TheJulia#endmeeting15:58
*** openstack changes topic to "Bare Metal Provisioning | Status: http://bit.ly/ironic-whiteboard | Docs: http://docs.openstack.org/ironic/ | Bugs: https://storyboard.openstack.org/#!/project_group/75 | Contributors are generally present between 6 AM and 12 AM UTC, If we do not answer, please feel free to pose questions to openstack-discuss mailing list."15:58
openstackMeeting ended Mon Jul 13 15:58:43 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:58
openstackMinutes:        http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-07-13-15.00.html15:58
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-07-13-15.00.txt15:58
openstackLog:            http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-07-13-15.00.log.html15:58
* kaifeng just successfully stopped his "not so smart" console to close the window15:59
TheJuliakaifeng: *blink* *blink*16:00
TheJuliakaifeng: that sounds... bad16:00
*** lucasagomes has quit IRC16:02
kaifengTheJulia: i have clarified some on the storyboard of automatic console, hope that makes the proposal clear16:04
TheJuliakaifeng: awesome, would you like us to revisit it next week?16:04
kaifengTheJulia: that would be awesome! actually we are doing same thing at nova side at downstream16:05
kaifengbut it has risk, as lock contention exists16:06
TheJuliakaifeng: do you have the link available by chance?16:06
kaifengyep  https://storyboard.openstack.org/#!/story/200787616:07
TheJuliaokay, done16:09
kaifengTheJulia: thank you!16:09
*** Lucas_Gray has quit IRC16:13
openstackgerritAija Jaunteva proposed openstack/ironic-specs master: [RFE] Operationally efficient and fast-track bare metal hardware provisioning  https://review.opendev.org/74072116:28
*** k_mouza has quit IRC16:33
dtantsurthe upper-constraints update has landed, we should be able to recheck16:45
TheJulia\o/16:47
TheJuliarecheck party16:47
*** tzumainn has quit IRC16:53
*** tzumainn has joined #openstack-ironic16:58
*** derekh has quit IRC17:00
*** sean-k-mooney has quit IRC17:01
*** johnthetubaguy has quit IRC17:02
*** ociuhandu has quit IRC17:03
*** johnthetubaguy has joined #openstack-ironic17:03
arne_wiebalckbye everyone o/17:04
openstackgerritJulia Kreger proposed openstack/ironic master: Add knob for read-only and "erase_devices"  https://review.opendev.org/74073117:04
*** uzumaki has quit IRC17:11
*** tzumainn has quit IRC17:13
*** dtantsur is now known as dtantsur|afk17:15
openstackgerritAija Jaunteva proposed openstack/ironic-specs master: [WIP] Operationally efficient and fast-track bare metal hardware provisioning  https://review.opendev.org/74072117:33
*** tzumainn has joined #openstack-ironic17:35
*** ricolin_ has quit IRC17:40
openstackgerritJulia Kreger proposed openstack/ironic master: Add ``ipxe_template`` override option  https://review.opendev.org/73984118:15
TheJuliaOkay, I think I've got the bad gateway issue figured out. Looks like the same job is also sensitive to timeouts... its taking like 60+% of the time getting the OS up and then we're loosing time due to looking up the baremetal endpoint lookup.18:32
*** kaifeng has quit IRC18:39
openstackgerritJulia Kreger proposed openstack/ironic-inspector master: Respond so a ContentType is included  https://review.opendev.org/74034418:44
openstackgerritJulia Kreger proposed openstack/ironic-inspector master: Make inspector base CI job use 2 CPUs  https://review.opendev.org/74074618:48
*** diurnalist has quit IRC18:49
openstackgerritJulia Kreger proposed openstack/ironic-inspector master: Make inspector base CI job use 2 CPUs  https://review.opendev.org/74074618:49
TheJulia^^^ should address MOST of the timeout issues with inspector + ramdisks18:50
openstackgerritJulia Kreger proposed openstack/ironic-inspector stable/ussuri: Make inspector base CI job use 2 CPUs  https://review.opendev.org/74074718:54
TheJuliaussuri cherry pick, that is likely going to take care of all of our inspector stability issues18:56
openstackgerritBob Fournier proposed openstack/ironic stable/train: DRAC: Drives conversion from JBOD to RAID  https://review.opendev.org/73850918:59
*** diurnalist has joined #openstack-ironic19:12
*** ociuhandu has joined #openstack-ironic19:12
*** ociuhandu has quit IRC19:17
iurygregoryTheJulia, +219:20
*** diurnalist has quit IRC19:26
TheJuliawoot19:33
*** diurnalist has joined #openstack-ironic19:40
*** shadower has quit IRC19:52
*** shadower has joined #openstack-ironic19:55
TheJuliaSo looks like we'll also want to backport my bad gateway fix to the ussuri stable branch20:25
TheJuliabut that one is slightly more controversial in my mind, so best to let that get through review20:26
*** Lucas_Gray has joined #openstack-ironic20:26
*** diurnalist has quit IRC20:32
*** diurnalist has joined #openstack-ironic20:40
openstackgerritVerification of a change to openstack/ironic failed: Use min_command_interval when ironic does IPMI retries  https://review.opendev.org/74046620:43
*** johnthetubaguy has quit IRC20:46
*** johnthetubaguy has joined #openstack-ironic20:47
jandersg'day Ironic! o/21:01
TheJuliaGood morning janders21:17
*** JayF has quit IRC21:23
*** JayF has joined #openstack-ironic21:24
openstackgerritJulia Kreger proposed openstack/ironic master: Change non-tinyipa jobs to use multiple cores  https://review.opendev.org/74082021:30
openstackgerritBob Fournier proposed openstack/ironic stable/train: DRAC: Drives conversion from JBOD to RAID  https://review.opendev.org/73850921:34
openstackgerritVerification of a change to openstack/ironic failed: Do not validate driver on changing non-driver fields  https://review.opendev.org/73969921:44
TheJuliadtantsur|afk: looks like it would be a bad idea to move forward fail if in transient state...21:47
TheJuliait will require reworking the hash ring init21:48
TheJuliawhich is, in itself, is doable, but just moving the init of it up seems have to disturbed the balacne int he force and I now have ten unit tests failing with errors implying the hash ring is not established yet it is being established.... so I kind of thing it would be a good idea to just leave it unless someone hits an issue in that window21:49
TheJuliaand two failures upon devstack standup. Maybe now is a good time to call it a day21:51
TheJulialooks like stevedore 3.1.0 is also breaking22:00
* TheJulia updates the whiteboard to indicate CI is red22:01
TheJuliahttps://review.opendev.org/#/c/740730/22:10
patchbotpatch 740730 - python-openstackclient - Fix compatibility issue in 5.3 - 2 patch sets22:10
*** k_mouza has joined #openstack-ironic22:12
jandersnewbie question: adding release notes to my mini-patch https://review.opendev.org/#/c/740678/1 would it be right to say I only need prelude, features and fixes sections and I can leave the rest blank?22:29
patchbotpatch 740678 - bifrost - Removing libselinux-python package from Fedora dep... - 1 patch set22:29
iurygregoryjanders, you only need fixes22:38
iurygregoryhttps://github.com/openstack/bifrost/search?q=fixes&unscoped_q=fixes22:40
jandersdoco states "This section is always included" in the prelude section - I suppose I can just ignore this in my case?22:41
jandersthanks heaps iurygregory22:41
iurygregoryprelude is used when we are doing a release22:41
iurygregoryyou can remove all sections and use fixes22:41
openstackgerritVerification of a change to openstack/ironic failed: Allow deleting nodes with a broken driver  https://review.opendev.org/73970222:48
*** rloo has quit IRC22:48
*** tkajinam has joined #openstack-ironic22:54
*** rcernin has joined #openstack-ironic23:05
*** rcernin has quit IRC23:05
*** rcernin has joined #openstack-ironic23:08
*** diurnalist has quit IRC23:13
*** k_mouza has quit IRC23:17
openstackgerritVerification of a change to openstack/ironic failed: Decompose the core deploy step on iscsi and ansible deploy  https://review.opendev.org/73118023:18
*** hoonetorg has quit IRC23:24
*** k_mouza has joined #openstack-ironic23:30
*** k_mouza has quit IRC23:34
*** hoonetorg has joined #openstack-ironic23:39
*** k_mouza has joined #openstack-ironic23:40
*** diurnalist has joined #openstack-ironic23:44
*** k_mouza has quit IRC23:45
*** k_mouza has joined #openstack-ironic23:48
*** k_mouza has quit IRC23:53
*** k_mouza has joined #openstack-ironic23:57

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!