Friday, 2020-10-09

*** k_mouza has quit IRC00:04
*** Goneri has quit IRC00:15
*** k_mouza has joined #openstack-ironic00:22
*** k_mouza has quit IRC00:26
*** k_mouza has joined #openstack-ironic00:32
*** k_mouza has quit IRC00:36
*** JamesBenson has quit IRC00:42
*** JamesBenson has joined #openstack-ironic00:43
*** JamesBenson has quit IRC00:47
*** larsks has quit IRC00:50
*** larsks has joined #openstack-ironic00:54
*** zzzeek has quit IRC01:10
*** zzzeek has joined #openstack-ironic01:11
*** k_mouza has joined #openstack-ironic01:13
*** outbrito has joined #openstack-ironic01:17
*** k_mouza has quit IRC01:17
*** JamesBenson has joined #openstack-ironic01:20
*** JamesBenson has quit IRC01:24
*** outbrito has quit IRC01:25
*** k_mouza has joined #openstack-ironic01:26
*** k_mouza has quit IRC01:30
*** Qianbiao has joined #openstack-ironic01:33
openstackgerritMichelle (Shelly) Mutu-Grigg proposed openstack/ironic master: Corrected merge conflicts  https://review.opendev.org/75694201:43
*** smutugrigg has joined #openstack-ironic01:46
*** k_mouza has joined #openstack-ironic01:57
smutugrigghi, Michelle (Outreachy applicant here) have just committed a update on a last commit as there were conflicts with another commit.  Would anyone be able to have a look and let me know if I have resolved the issues? https://review.opendev.org/#/c/756942/02:01
patchbotpatch 756942 - ironic - Corrected merge conflicts - 1 patch set02:01
*** k_mouza has quit IRC02:01
*** smutugrigg has quit IRC02:04
janders@smutugrigg thank you for the patch Michelle. This is continuation of https://review.opendev.org/#/c/756708/ correct?02:13
patchbotpatch 756708 - ironic - Update .rst files to remove --os-baremetal-api-ver... - 1 patch set02:13
jandersIt seems to me it came through as a new change02:15
jandersif I understand correctly it should have been another revision of the same change02:17
janders@smutugrigg have a look at https://docs.opendev.org/opendev/infra-manual/latest/sandbox.html and https://gerrit-review.googlesource.com/Documentation/intro-gerrit-walkthrough.html#_reworking_the_change - "git commit --amend" should allow you to submit it as a revision of the existing change02:17
janders@smutugrigg it may be worthwhile walking through this tutorial https://docs.opendev.org/opendev/infra-manual/latest/sandbox.html start to finish, I think it gives a good overview of the process02:19
jandershope this helps :)02:22
*** k_mouza has joined #openstack-ironic02:23
*** k_mouza has quit IRC02:27
*** JamesBenson has joined #openstack-ironic02:30
*** JamesBenson has quit IRC02:34
*** k_mouza has joined #openstack-ironic02:51
*** k_mouza has quit IRC02:55
*** JamesBenson has joined #openstack-ironic03:05
*** JamesBenson has quit IRC03:10
*** rcernin has quit IRC03:17
*** Qianbiao has quit IRC03:29
*** smutugrigg has joined #openstack-ironic03:39
*** rcernin has joined #openstack-ironic03:55
*** rcernin has quit IRC03:56
*** ricolin_ has joined #openstack-ironic04:01
*** rcernin has joined #openstack-ironic04:03
*** uzumaki has quit IRC04:05
*** tzumainn has quit IRC04:12
*** pmannidi has quit IRC04:16
*** mpsairam has joined #openstack-ironic04:16
openstackgerritManuel Schönlaub proposed openstack/sushy master: Adds basic support for TaskService to retrieve task information.  https://review.opendev.org/75690604:19
openstackgerritManuel Schönlaub proposed openstack/sushy master: Adds basic support for TaskService to retrieve task information.  https://review.opendev.org/75690604:21
*** Qianbiao has joined #openstack-ironic04:21
openstackgerritManuel Schönlaub proposed openstack/sushy master: Adds basic support for TaskService to retrieve task information.  https://review.opendev.org/75690604:23
*** zzzeek has quit IRC05:02
*** smutugrigg has quit IRC05:04
*** zzzeek has joined #openstack-ironic05:05
*** Qianbiao has quit IRC05:18
*** zzzeek has quit IRC05:42
*** zzzeek has joined #openstack-ironic05:43
*** rpittau|afk is now known as rpittau05:43
rpittaugood morning ironic! o/05:43
*** kafilat has joined #openstack-ironic05:46
*** sshnaidm is now known as sshnaidm|off06:06
*** zzzeek has quit IRC06:11
*** zzzeek has joined #openstack-ironic06:12
arne_wiebalckGood morning, rpittau and ironic!06:34
*** tkajinam has quit IRC06:42
*** tkajinam has joined #openstack-ironic06:42
*** Akki58 has joined #openstack-ironic06:55
Akki58Hello All,06:55
Akki58and goes into second time booting and gets stuck there in "Probing" state ( Seen on node's console) BUT according to openstack horizon, it is up and running and according to "openstack baremetal node show", it is in "Active" state.06:55
Akki58again and then comes up, the "spawning" state on openstack horizon  goes into ERROR.06:55
Akki58for callback for node 75210cc4-ad98-442d-ace1-89ce69467580"06:55
*** jtomasek has joined #openstack-ironic06:59
rpittauhey arne_wiebalck :)07:00
rpittauAkki58: hey, good morning, so is it just a problem on horizon ui ? or you can't access the node? you should check the console logs if possible, and also conductor logs07:03
Akki58Hi rpittau, Good morning. Thank you for your reply. When it is up on horizon, the baremetal console is stuck on "Probing EDD (edd=off to disable) ... ok07:07
rpittauAkki58: would be also useful to have a look at the ironic-python-agent ramdisk logs, check all of those I mentioned, you will probably get some hints on what's going on07:08
*** uzumaki has joined #openstack-ironic07:10
*** smutugrigg has joined #openstack-ironic07:11
iurygregorygood morning arne_wiebalck rpittau and Ironic07:22
iurygregoryhappy friday07:22
rpittauhey iurygregory :)07:22
rpittauHappy Friday!07:22
*** Qianbiao has joined #openstack-ironic07:31
Akki58rpittau, I searched for logs and found no errors in ironic-conductor:07:36
Akki58According to ironic-conductor it says:07:37
Akki58INFO ironic.conductor.deployments [req-6c388902-2862-4756-a1cb-b67984962205 - - - - -] Successfully deployed node 75210cc4-ad98-442d-ace1-89ce69467580 with instance 712633e5-9631-41df-af0f-beedade68a36.07:37
smutugrigggood morning iurygregory07:37
Akki58Also i can't find ironic-python-agent ramdisk logs. Can you please guide me how to get those?07:41
smutugriggiurygregory, could I request some assistance please?07:43
iurygregorysmutugrigg, morning, sure07:44
smutugriggI had merge conflicts in my commit, that I think I have resolved, but recieved a build failed from Dell07:44
smutugriggHow do I work out if I have corrected the merge conflicts or not?07:45
*** rcernin has quit IRC07:45
iurygregoryoh you submitted another change correct?07:46
*** tosky has joined #openstack-ironic07:46
smutugriggI thought I amended, but I may have submitted a second change07:46
iurygregorybefore was https://review.opendev.org/#/c/756708/ and you sent another patch https://review.opendev.org/#/c/756942/ because of the conflict you had07:46
patchbotpatch 756708 - ironic - Update .rst files to remove --os-baremetal-api-ver... - 1 patch set07:46
patchbotpatch 756942 - ironic - Corrected merge conflicts - 1 patch set07:46
iurygregoryno worries =)07:46
smutugrigg:)07:47
iurygregorylet me check here07:47
smutugriggthanks07:47
iurygregorysmutugrigg, seems like you fixed the merge conflicts you had, I'm adding a few comments that you will need address ok?07:57
smutugriggfantastic - thank you07:57
rpittauwell inspector grenade is broken08:00
*** lucasagomes has joined #openstack-ironic08:10
iurygregorysmutugrigg, I've posted the comments =)08:14
smutugriggThanks for the detail iurygregory, much appreciated.  Will work on this.08:17
iurygregorysmutugrigg, if you have any doubts on how to ammend the change feel free to ask08:17
smutugriggI think I might need some help to make sure I do it correctly08:18
smutugriggWill come back to the chat once I am there08:18
iurygregorycool =)08:18
*** k_mouza has joined #openstack-ironic08:20
*** derekh has joined #openstack-ironic08:28
openstackgerritRiccardo Pittau proposed openstack/ironic-inspector master: Use direct as IRONIC_DEFAULT_DEPLOY_INTERFACE for grenade  https://review.opendev.org/75670508:28
*** k_mouza_ has joined #openstack-ironic08:28
*** k_mouza has quit IRC08:31
iurygregoryrpittau, I loved the "recheck sqlalchemy shenanigan"08:40
rpittauheh08:42
*** dtantsur|afk is now known as dtantsur08:44
smutugriggiurygregory, hello o/08:51
*** ociuhandu has joined #openstack-ironic08:51
iurygregorysmutugrigg, o/08:51
smutugriggI think I have now completed the updates and merged all the chnages from the master into my branch08:51
smutugriggI have also commited using git commit -a --amend08:52
smutugriggwhichc shows me openstack-ironic-task-40964 197e3c051] Update .rst files remove -os-baremetal-api-version08:52
smutugrigg Date: Fri Oct 9 21:47:05 2020 +130008:52
*** Akki58 has quit IRC08:52
smutugriggShould I be OK to submit git review?08:52
smutugriggqit = git08:53
iurygregoryhave you added Task and Story in the commit message?08:53
smutugriggyes I have08:53
iurygregorycool08:53
iurygregorygo ahead and do git review08:53
smutugriggsame message as from 756708 with minor additions08:53
iurygregoryok =)08:53
dtantsurmorning ironic08:55
iurygregorymorning dtantsur08:55
rpittauhey dtantsur :)08:55
iurygregorystandalone job hate us all08:59
*** ricolin_ has quit IRC09:04
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Do not silently swallow errors in the write_image deploy step  https://review.opendev.org/75703709:24
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: DNM break prepare_image  https://review.opendev.org/75703809:25
openstackgerritRiccardo Pittau proposed openstack/ironic master: Replace retrying with tenacity  https://review.opendev.org/37657409:26
*** jtomasek has quit IRC09:29
*** fmuyassarov has joined #openstack-ironic09:33
smutugrigghi iurygregory o/09:36
fmuyassarovHi @dtantsur, I have asked earlier a way to avoid disk cleaning as part of deprovisioning process. And I remember you suggested to disable it by setting erase_devices_metadata_priority and erase_devices_priority to zero. And my question is, can I avoid disk cleaning by setting automated_clean to False instead of setting 0 values for erase* fields? Thanks.09:40
iurygregorysmutugrigg, hi09:42
QianbiaoHello ironic.09:48
dtantsurfmuyassarov: yes, you can09:48
fmuyassarovThanks!09:50
Qianbiaohello, is there a command can transfer node status from error to manageable again?09:51
dtantsurQianbiao: check here https://docs.openstack.org/ironic/latest/contributor/states.html09:52
openstackgerritMichelle (Shelly) Mutu-Grigg proposed openstack/ironic master: Update .rst files remove -os-baremetal-api-version  https://review.opendev.org/75670809:53
smutugriggiurygregory, so I think I have completed the update09:54
smutugriggbut only to https://review.opendev.org/75670809:54
patchbotpatch 756708 - ironic - Update .rst files remove -os-baremetal-api-version - 2 patch sets09:54
smutugriggI have merged the requested changes from https://review.opendev.org/756942 as well09:54
patchbotpatch 756942 - ironic - Corrected merge conflicts - 1 patch set09:54
smutugrigghad a bit of trouble with rebasing ;)09:55
Qianbiaodtantsur i try delete, but it's not allowed.09:55
dtantsurQianbiao: you mean, undeploy?09:55
Qianbiaoyes09:56
Qianbiaohttp://paste.openstack.org/show/798901/ ^^09:56
*** lucasagomes has quit IRC09:56
*** lucasagomes has joined #openstack-ironic09:57
*** lucasagomes has quit IRC09:57
*** lucasagomes has joined #openstack-ironic09:58
iurygregorysmutugrigg, cool! I will look at it09:58
smutugriggthanks heaps iurygregory, very much appreciated09:59
Qianbiaodtantsur sorry, my bad, there has a "undeploy" command, i was using delete.09:59
dtantsurdelete literally deletes the node, it's not quite what you want10:00
Qianbiaoin the status machine picture, it show delete :)10:00
Qianbiaoso. i try to run delete command.10:00
fmuyassarov@dtantsur, sorry I forgot to mention one thing.In our setup erase-devices-metadata-priority will have some non-zero value. So, even though erase-devices-metadata-priority has a value greater than zero, I still can avoid disk cleaning by having false for automated-clean right?10:02
dtantsurfmuyassarov: yep, automated_clean=False makes the whole cleaning process completely no-op10:02
fmuyassarovGreat. Thank you very much!10:02
*** fmuyassarov has quit IRC10:06
rpittaufixed inspector grenade -> https://review.opendev.org/75670510:13
patchbotpatch 756705 - ironic-inspector - Use direct as IRONIC_DEFAULT_DEPLOY_INTERFACE for ... - 3 patch sets10:13
openstackgerritlikui proposed openstack/ironic-inspector master: bump py37 to py38 in tox.ini  https://review.opendev.org/75706910:15
iurygregoryrpittau, awesome10:16
iurygregorysmutugrigg, only a few more things and it should be good =)10:19
iurygregoryif you have problems running pep8, docs locally let me know10:20
smutugriggthanks iurygregory for your patience, I will get there in the end10:23
iurygregorysmutugrigg, yw! good luck =)10:23
openstackgerritlikui proposed openstack/ironic-inspector master: bump py37 to py38 in tox.ini  https://review.opendev.org/75706910:31
iurygregorythe commit message doesn't make sense .-.10:33
*** Qianbiao has quit IRC10:36
smutugrigghi iurygregory o/10:48
iurygregorysmutugrigg, o/10:48
smutugriggfor the pep810:48
iurygregorysure10:48
smutugriggis the command 'tox -epep8 docs'?10:48
iurygregorytox -r -vv -e pep810:49
smutugriggthank you10:49
iurygregoryand for docs is tox -r -vv -e docs10:49
smutugriggah ha, thank you10:49
*** priteau has joined #openstack-ironic10:49
*** rcernin has joined #openstack-ironic10:50
*** zzzeek has quit IRC10:52
openstackgerritMerged openstack/bifrost master: Use upper constraints when installing Bifrost requirements  https://review.opendev.org/75675410:53
*** zzzeek has joined #openstack-ironic10:54
*** Qianbiao has joined #openstack-ironic10:56
smutugrigghi iurygregory, could I request some help with some diagnosis please?10:56
iurygregorysmutugrigg, sure10:56
smutugriggpep8 is passing, however docs is erroring10:56
smutugriggInvocationError for command /home/michelle/openstack/ironic/.tox/docs/bin/sphinx-build -b html -W doc/source doc/build/html (exited with code 2)10:56
iurygregorycan you paste the error in http://paste.openstack.org/10:56
iurygregoryoh seems like you are missing some packages10:57
smutugriggpasted to URL as well10:57
smutugriggdid I miss a command during setup?10:57
iurygregorymaybe, what OS are you using?10:58
smutugriggubuntu 20.04.110:58
iurygregoryhttps://www.sphinx-doc.org/en/master/usage/installation.html10:59
iurygregorytry installing python3-sphinx10:59
smutugriggok10:59
iurygregoryand re run the tox for docs10:59
smutugrigghi iurygregory, same error :(11:04
smutugriggAlso says11:04
smutugriggRSVG converter command 'rsvg-convert' cannot be run. Check the rsvg_converter_bin setting11:04
dtantsursmutugrigg: you need to install it, it's librsvg2-tools on Fedora, for example11:05
smutugriggAh ha, thank you dtantsur, will attempt11:06
iurygregoryyeah11:06
iurygregorylibrsvg package probably11:06
dtantsuriurygregory: could you check https://review.opendev.org/#/c/757037/ please? it's nasty enough to require a new V release.11:10
patchbotpatch 757037 - ironic-python-agent - Do not silently swallow errors in the write_image ... - 1 patch set11:10
iurygregorydtantsur, wow11:10
smutugriggthank you kindly iurygregory & dtantsur, it was librsvg2-bin11:11
iurygregorysmutugrigg, cool!11:11
smutugriggall tox commands now green with cngrats11:11
smutugriggyah!11:11
smutugriggsmall wins11:11
iurygregory\o/11:11
iurygregorydtantsur, lgtm +2 (will wait for the CI)11:12
* iurygregory goes for lunch11:12
*** zzzeek has quit IRC11:17
*** zzzeek has joined #openstack-ironic11:18
arne_wiebalckFYI reducing the IPA image size seems to address some OOM situations in grub (this is with UEFI bootmode). YMMV as the grub file we use is not brand new.11:26
*** jtomasek has joined #openstack-ironic11:31
*** rcernin has quit IRC11:31
*** shreya11 has quit IRC11:34
*** dking has quit IRC11:34
*** bdodd has quit IRC11:34
*** jamesdenton has quit IRC11:34
*** rajinir has quit IRC11:34
*** shreya11 has joined #openstack-ironic11:34
*** outbrito has joined #openstack-ironic11:38
*** bdodd has joined #openstack-ironic11:38
*** dking has joined #openstack-ironic11:40
*** jamesdenton has joined #openstack-ironic11:40
*** rajinir has joined #openstack-ironic11:40
openstackgerritMerged openstack/ironic-inspector master: Use direct as IRONIC_DEFAULT_DEPLOY_INTERFACE for grenade  https://review.opendev.org/75670511:47
openstackgerritMichelle (Shelly) Mutu-Grigg proposed openstack/ironic master: Corrected merge conflicts  https://review.opendev.org/75694211:48
openstackgerritMichelle (Shelly) Mutu-Grigg proposed openstack/ironic master: Update .rst files  https://review.opendev.org/75670811:48
smutugrigghi iurygregory o/11:49
smutugriggfor some reason I could not abandon the https://review.opendev.org/756942 commit and commited it with https://review.opendev.org/75670811:50
patchbotpatch 756942 - ironic - Corrected merge conflicts - 2 patch sets11:50
patchbotpatch 756708 - ironic - Update .rst files - 3 patch sets11:50
smutugriggI hope this is OK.  All tox unit tests passing11:51
iurygregorysmutugrigg, it's because you are using https://review.opendev.org/#/c/756942/ as base for https://review.opendev.org/#/c/756708/11:55
patchbotpatch 756942 - ironic - Corrected merge conflicts - 2 patch sets11:55
patchbotpatch 756708 - ironic - Update .rst files - 3 patch sets11:55
iurygregoryto fix this in the branch you are working on https://review.opendev.org/#/c/756708 you need to run git rebase -i origin/master11:56
patchbotpatch 756708 - ironic - Update .rst files - 3 patch sets11:56
iurygregoryand fix any problems it shows when doing the rebase11:56
rpittauarne_wiebalck: that's probably because of the reduced memory need12:10
arne_wiebalckrpittau: Yes. But newer versions of grub or different f/w may handle this differently.12:12
rpittautrue12:12
arne_wiebalckrpittau: And I a not sure there is an actual limit of the amount of RAM. Seems like grub does not have such a limit.12:12
arne_wiebalckrpittau: All this may also have a completely different cause as I observed the same node with the same image sometimes working, sometimes failing.12:13
arne_wiebalckrpittau: Could be sth with the PXE server or some memory on the node that is not freed properly ...12:14
rpittauyeah, I was not thinking of a limit on grub side, but more on hwo the image is uncompressed and transfered12:14
*** Qianbiao has quit IRC12:22
openstackgerritMichelle (Shelly) Mutu-Grigg proposed openstack/ironic master: Update .rst files  https://review.opendev.org/75670812:22
*** Qianbiao has joined #openstack-ironic12:22
smutugrigghi iurygregory, I think I finally have it....hopefully12:24
*** smutugrigg has quit IRC12:29
*** vilitho has joined #openstack-ironic12:33
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent stable/victoria: Do not silently swallow errors in the write_image deploy step  https://review.opendev.org/75709112:34
*** jtomasek has quit IRC12:44
QianbiaoHello, when undeploy and redeploy node frequency, the arp table is not refresh, is this normal.12:46
Qianbiao^^^ http://paste.openstack.org/show/798909/12:46
*** lmcgann has joined #openstack-ironic12:50
rpittauclosing early today, have a great weekend everyone! o/13:03
*** rpittau is now known as rpittau|afk13:03
arne_wiebalckbye rpittau|afk o/13:03
*** outbrito has quit IRC13:09
*** uzumaki has quit IRC13:18
*** tzumainn has joined #openstack-ironic13:19
iurygregorybye rpittau|afk13:24
openstackgerritMerged openstack/ironic-python-agent master: Do not silently swallow errors in the write_image deploy step  https://review.opendev.org/75703713:31
vilithoHello everyone, my name is Valerie from Nigeria and I'm an Outreachy applicant. For my first task, I picked #40969. Its the one that requires that I validate bifrosts installation documentation. I am doing this using via the WSL feature on windows and i have run into a libvirt problem. From browsing online i realize its very likely because i am running ubuntu virtually but so far i haven't been able to fix this problem and i'd really13:35
vilithoappreciate any help /pointers. Thanks :D13:35
*** Goneri has joined #openstack-ironic13:36
vilithoI am doing this using ubuntu*13:37
Qianbiao<vilitho> wsl2 or wsl?13:38
Qianbiaowsl is not a original ubuntu i think.13:38
QianbiaoSome package is not supported on wsl13:39
vilithowsl213:39
vilithoRight. But if i were to now install ubuntu on vmware for example, that should fix it right?13:44
vilithoI'll just test that out and see.13:46
*** ociuhandu has quit IRC14:11
*** ociuhandu has joined #openstack-ironic14:12
*** jtomasek has joined #openstack-ironic14:12
*** jtomasek has quit IRC14:12
*** vilitho has quit IRC14:16
*** vilitho has joined #openstack-ironic14:16
iurygregoryHi vilitho welcome!14:25
*** k_mouza_ has quit IRC14:25
iurygregoryI would say it would be more easy if you have a machine with Ubuntu / Fedora / CentOS to validate14:26
*** k_mouza has joined #openstack-ironic14:26
openstackgerritJulia Kreger proposed openstack/ironic master: Log the ports we bind in flat  https://review.opendev.org/75711514:28
* TheJulia does the kahn type scream because we don't log the vifs in rescue14:29
openstackgerritMerged openstack/ironic stable/ussuri: Increase memory of tinyipa vms  https://review.opendev.org/74763314:34
openstackgerritMark Goddard proposed openstack/bifrost stable/victoria: Use upper constraints when installing Bifrost requirements  https://review.opendev.org/75711614:37
openstackgerritMark Goddard proposed openstack/bifrost stable/victoria: Use upper constraints when installing Bifrost requirements  https://review.opendev.org/75711614:38
openstackgerritMark Goddard proposed openstack/bifrost stable/ussuri: Use upper constraints when installing Bifrost requirements  https://review.opendev.org/75711714:39
openstackgerritMark Goddard proposed openstack/bifrost stable/train: Use upper constraints when installing Bifrost requirements  https://review.opendev.org/75711814:41
openstackgerritMark Goddard proposed openstack/bifrost stable/ussuri: Use upper constraints when installing Bifrost requirements  https://review.opendev.org/75711714:41
openstackgerritMark Goddard proposed openstack/ironic master: docs: Add information on post-branch release tasks for bifrost  https://review.opendev.org/75712014:50
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Do not use RPC continue_node_deploy/clean in agent_base  https://review.opendev.org/75635414:51
*** k_mouza has quit IRC14:52
*** k_mouza has joined #openstack-ironic15:04
TheJuliadid we ever validate if cirros _could_ boot to a raidset?15:05
TheJuliaI'm guessing lots of nope15:05
dtantsurTheJulia: you mean, software RAID? did not work the last time I tried.15:06
TheJuliathat is what I figured15:07
TheJuliaI think I know why rescue keeps failing on victoria15:07
dtantsurdo I want to know? :)15:07
TheJuliathe deploy is ending up on the raidset15:07
* TheJulia is tryign to figure out why it is sitll around or still being used15:07
dtantsuryeah, we should be removing it15:07
* dtantsur looks at agent_base.py and wants to scream15:08
TheJulialooks like we're orphaning raid_config :\15:11
dtantsurwe what? Oo15:11
TheJuliaAnd I don't think we're cleaning so it stays around :(15:12
* dtantsur ponders declaring wallaby a technical debt cycle15:12
TheJuliaI was already thinking along those lines, tbh15:12
dtantsurexcept that everyone wants active steps.. which will make everything a much bigger mess15:12
TheJuliaugh15:12
dtantsurI'll probably keep refactoring the guts of the agent deploy so that it's not so bad in the end15:13
dtantsursad we cannot remove the iscsi deploy right now15:13
TheJuliaugh, yeah, cleaning is off15:14
* TheJulia looks at the tempest plugin15:14
* dtantsur wonders how many more regressions he'll introduce in the process15:15
TheJuliaheh, I have a note saying we should do this in the tempest job15:17
iurygregoryhehehe15:18
* TheJulia turns on cleaning15:19
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Do not use RPC continue_node_deploy/clean in agent_base  https://review.opendev.org/75635415:27
openstackgerritJulia Kreger proposed openstack/ironic master: CI: Turn on cleaning on standalone jobs  https://review.opendev.org/75713215:27
TheJulia^ -- is likely the source of many of our standalone failures... if not ALL of ours present standalone failures15:28
*** k_mouza has quit IRC15:28
iurygregoryyay15:30
*** zaneb has quit IRC15:32
*** zaneb has joined #openstack-ironic15:33
TheJuliaeven _IF_ we explicitly called for the raidset to be nuked from high orbit by the tempest test, cleaning would still need to be enabled for it to work. We may need to change it so it is just a metadata erase though15:34
openstackgerritVerification of a change to openstack/ironic-python-agent failed: Do not silently swallow errors in the write_image deploy step  https://review.opendev.org/75709115:34
TheJuliaI _think_ that nukes everything with arne_wiebalck's recent chagnes15:34
*** vilitho has quit IRC15:35
*** k_mouza has joined #openstack-ironic15:35
* arne_wiebalck wakes up15:35
openstackgerritMark Goddard proposed openstack/ironic master: docs: Add information on post-branch release tasks for bifrost  https://review.opendev.org/75712015:35
* TheJulia puts coffee in front of arne_wiebalck 15:35
iurygregoryarne_wiebalck will send cookies \o/15:35
TheJuliawait, is he baking cookies?!?15:35
arne_wiebalcksuch a nice community!15:35
iurygregoryhe will \o/15:35
iurygregoryarne_wiebalck is a nice person =D15:35
TheJulia++15:35
arne_wiebalckha, iurygregory, getting cookies is not that easy! ;)15:36
arne_wiebalckmy recent changes broke sth?15:36
dtantsurTheJulia: if you have a second: https://review.opendev.org/#/c/757114/15:36
iurygregoryarne_wiebalck, you are only going to share the cookies with zuul?!15:36
patchbotpatch 757114 - x/ironic-staging-drivers (stable/victoria) - Prepare stable/victoria - 1 patch set15:36
iurygregory=OO15:36
arne_wiebalckzuul tests my patches15:36
TheJuliaarne_wiebalck: no, Your change likely reduced the risk of a failure, we have a race condition in the standalone job sequence where we're creating a raid set in one job, not running cleaning, and then provisioning the next job on the machine15:38
TheJuliaerr, same node15:38
TheJuliawhich ends up choosing the already created and present raid set15:38
dtantsurTheJulia: automated cleaning does not remove software RAID (by design)15:39
dtantsurbut I think we have some code to run manual cleaning to wipe it?15:39
*** k_mouza has quit IRC15:39
TheJuliaerrr15:40
TheJuliawell this stinks15:40
dtantsurheh15:40
arne_wiebalckwould it make sense to make RAID cleaning part of automatic cleaning?15:40
TheJuliathe tempest test likely needs to explicitly tear it down15:40
arne_wiebalckwe discussed this twice at least ...15:40
TheJuliaI'm going to move the raid test to the very last test15:41
TheJuliaas a first step15:41
*** k_mouza has joined #openstack-ironic15:41
arne_wiebalckthat sounds like a good work around15:41
TheJuliaThat should help get CI more reliable in the mean time15:41
TheJuliadtantsur: are software raid sets still being preserved if target_raid_config == {}15:42
arne_wiebalckTheJulia: yes15:42
arne_wiebalckTheJulia: as the cleaning step is not run15:42
dtantsurTheJulia: just like hardware RAID - yes15:42
arne_wiebalckyeah, this ^^ is the reason for what we do, analogy with h/w RAID15:43
arne_wiebalckmaybe it is time to break this15:43
TheJuliano, tempest needs to ask for the teardown15:43
arne_wiebalckbreak the analogy, heal the code15:43
TheJuliaerr, its a separate file :\15:45
arne_wiebalckre scale tests: it seems I am not able to drive the conductor into an OOM situtation in our current setup (this is without the RAM guard patch, 3 controllers with 8GB each)15:48
arne_wiebalckI just launched 200 nodes in parallel and the conductors were not impressed15:48
arne_wiebalckthe only way to squeeze it now, is to go from 3 controllers to 1 ...15:49
* TheJulia pokes and prods tempest chagnes15:49
TheJuliaconductor: "Foolish mortals! I shall deploy quickly!"15:50
arne_wiebalckconductor: "Doing tests, huh? No problem for me! But wait when you do this in prod again .." (evil laugh)15:51
* arne_wiebalck thinks it is time to call it a week when you make your processes speak ...15:52
*** k_mouza has quit IRC15:55
*** lucasagomes has quit IRC15:56
dtantsurarne_wiebalck: make processes speak? easy! https://pypi.org/project/jazzit/15:56
openstackgerritJulia Kreger proposed openstack/ironic-tempest-plugin master: WIP Tear down raid sets  https://review.opendev.org/75714115:57
arne_wiebalckdtantsur: ha ha ha, this is hilarious!15:57
*** k_mouza has joined #openstack-ironic15:57
dtantsurright? we should totally add it to unit tests!15:57
TheJuliaI think we need like little concurency tracking raspberry pi's with LCD screens that then start laughing evilly15:57
arne_wiebalckthere was a proposal some time ago to translate log messages into noises15:58
TheJuliathat is kind of amusing15:58
arne_wiebalcknormal operation would be some background humming15:58
arne_wiebalckwhen sth happens the frequency would change15:58
arne_wiebalckthis way, you can monitor things with your ears15:58
TheJulia++15:58
arne_wiebalckand use your eyes to continue writing code :-D15:59
TheJuliaor just wire it to play things like the Red Alert klaxon15:59
dtantsuron my previous job someone installed a buzzer that went off when a build was broken15:59
dtantsurit did not survive for long :)15:59
TheJuliaheh16:00
TheJuliadtantsur: something like https://www.youtube.com/watch?v=kHyWYA1Nvq8 ?16:01
arne_wiebalckyeah, no real alarms, just some background humming with occasional knock tones16:01
TheJuliaknock tones?!? like "*whiirrrrrrlllll* *knock knock knock*16:01
TheJulia"16:01
JayFThat episode of enterprise was triggering16:01
JayF"That new thing is annoying, shut it up, make it go away forever"16:02
JayF[5 minutes later]16:02
dtantsurTheJulia: louder :)16:02
JayF"Wow, the new thing saved us!"16:02
dtantsurmore like an emergency vehicle thing :)16:02
JayFit's like every ops or security person in an IT shop when their "annoying processes" finally pay off and prevent shenanigans :D16:02
dtantsurJayF: was about to say "security in IT" :)16:02
*** priteau has quit IRC16:06
*** k_mouza has quit IRC16:06
openstackgerritVerification of a change to openstack/ironic failed: Fix inspection for idrac  https://review.opendev.org/75597216:16
openstackgerritJulia Kreger proposed openstack/metalsmith master: Add a Centos8 UEFI based job  https://review.opendev.org/74831516:20
TheJuliasurprising zuul doesn't fail jobs that are missing referenced yaml files....16:20
*** ociuhandu has quit IRC16:21
*** Qianbiao has quit IRC16:25
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Don't run os-prober from grub2-mkconfig  https://review.opendev.org/75230816:26
*** dtantsur is now known as dtantsur|afk16:26
dtantsur|afkhave a great weekend folks16:26
TheJuliao/ dtantsur|afk have a wonderful weekend16:29
*** gyee has joined #openstack-ironic16:32
*** tosky has quit IRC16:35
TheJuliaarne_wiebalck: replied to your comment on efi content preservation. It might be good to refactor, but I think right now the best thing to do is to focus on the partition image raidless case and then refactor from the basis of trying to support all the cases a bit more cleanly16:35
TheJuliaI think if we can get into a place where we consider it a bad image if we're running grub-install, then we're doing good16:35
*** priteau has joined #openstack-ironic16:41
arne_wiebalckTheJulia: yes ... my point was mostly we that dive into that function for md devices, just to (hopefully) do nothing and I thought it might be more readable/clearer if we could make the 'if' clearly state that this is for partition images only ... from what I see, the code should work as is, also in the case of s/w RAID.16:43
TheJuliaunfortunately I don't think we know the image type16:43
TheJuliahitting there in this case would almost exclusively be a partition image case16:44
TheJuliaor a whole disk that is really broken16:44
*** ViniciusCoelho has joined #openstack-ironic16:46
arne_wiebalckbut we could always add "and not hardware.is_md_device(device)", for instance16:48
TheJuliathat is exactly what I was thinking of doing16:48
TheJuliaI've only gotten derailed nicely this morning :)16:49
TheJulia3 people messaging me with-in 10 minutes... seems... suspect :)16:49
arne_wiebalckthey plot sth ...16:49
arne_wiebalckbye everyone, have a good week-end o/16:52
TheJuliaEnjoy!16:53
TheJuliahave a wonderful weekend16:53
*** derekh has quit IRC16:55
dkingFor DIB elements, I notice that a lot of them set their variables in scripts inside of "environment.d" directories. However, I don't see that documented as a valid directory. Am I missing the documentation, or is there a better place to set variables?17:00
dkingI started looking because I wanted to be sure that a variable was set in time to be used by source-repositories. I see that runs a script in extra-data.d.17:02
dkingNevermind. I see that environment.d is mentioned further down in the documentation. Sorry.17:03
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Support partition image efi contents  https://review.opendev.org/74804917:14
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Add fstab pointer to EFI partition  https://review.opendev.org/74826017:14
TheJuliadking: I think it is loaded, or at least was loaded, it is not afaik part of the process of building and the defined directories for such17:16
TheJuliaoh, okay17:16
TheJulia:)17:16
arne_wiebalckTheJulia: For the fstab entry, I think the UUID for the ESP from the image and the one we create for md devices is the same. I can verify, but I think this is why the newly created EFI partitions are mounted into /boot/efi. So, I *think* we would end up with twice the same line (which does not hurt, it is just not very useful).17:42
*** priteau has quit IRC17:58
*** priteau has joined #openstack-ironic18:03
*** priteau has quit IRC18:08
TheJuliait _should_ be fine, I think.18:16
TheJuliaI guess I could always check the contents and replace the line :\18:17
TheJuliaanyway, CI should run and it should pull in centos8.2 when it finally gets to that job and we will either see success or great sadness18:17
*** tkajinam has quit IRC18:25
*** tzumainn has quit IRC18:31
*** tosky has joined #openstack-ironic18:32
*** k_mouza has joined #openstack-ironic18:32
*** tzumainn has joined #openstack-ironic18:34
arne_wiebalckTheJulia: I just checked, on our nodes with s/w RAID, the UUID of the ESP that comes with the image is the same as the UUID which is on the ESPs we created during the RAID deployment. Since the ESP is mounted by UUID in the fstab, the nodes mount (one of the) ESP of the holder disks, i.e. one of the partitions we created. The patch would duplicated the line in the fstab, I guess.18:38
* arne_wiebalck really really goes now :)18:40
*** priteau has joined #openstack-ironic18:42
*** priteau has quit IRC18:48
TheJulia:)18:51
rpiosoiurygregory, JayF: I would appreciate another core reviewer look at https://review.opendev.org/#/c/750605/. It is a cherry pick, which TheJulia has Code-Review+2.19:13
patchbotpatch 750605 - ironic (stable/train) - DRAC: Fix RAID create_config clean step - 3 patch sets19:13
*** priteau has joined #openstack-ironic19:14
*** k_mouza has quit IRC19:20
erbarrTheJulia, hey sorry I didn't get a notification about your message but rpioso let me know, I think the log server must have gotten full yesterday but got cleared in the overnight deleting of old builds, then the mirroring was successful after that so the logs are working now for me, can you check to see if you can reach?19:24
*** ociuhandu has joined #openstack-ironic19:24
*** priteau has quit IRC19:27
*** zzzeek has quit IRC19:49
*** ociuhandu has quit IRC19:50
*** zzzeek has joined #openstack-ironic19:51
TheJuliaerbarr: I'll recheck the job that I'm curious about and I'll go from there20:04
TheJuliaerbarr: thanks!20:04
erbarrno problem, thanks for the notice20:05
* TheJulia wonders if anyone has made an easy to setup 3rd party CI tool yet20:08
*** bfournie has quit IRC20:42
rpiosoJayF: Thank you!20:48
JayFnp20:49
*** ociuhandu has joined #openstack-ironic20:51
*** bfournie has joined #openstack-ironic21:00
clarkbsoftware factory is often suggested21:09
clarkbTheJulia: ^ that was for you21:09
*** lmcgann has quit IRC21:10
*** ociuhandu has quit IRC21:13
*** ViniciusCoelho has quit IRC21:18
*** ociuhandu has joined #openstack-ironic21:18
*** ociuhandu has quit IRC21:25
*** kafilat has quit IRC21:54
openstackgerritVerification of a change to openstack/ironic failed: DRAC: Fix RAID create_config clean step  https://review.opendev.org/75060522:10
openstackgerritVerification of a change to openstack/ironic failed: DRAC: Fix a failure to create virtual disk bug  https://review.opendev.org/75589522:10
*** tosky has quit IRC23:26
TheJuliaclarkb: thanks23:46

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!