Monday, 2019-07-29

*** dsneddon has joined #openstack-ironic00:50
*** dsneddon has quit IRC00:55
*** dsneddon has joined #openstack-ironic00:56
*** igordc has quit IRC01:24
*** dsneddon has quit IRC01:42
*** dsneddon has joined #openstack-ironic02:07
*** dsneddon has quit IRC02:33
*** rcernin has quit IRC02:45
*** rcernin has joined #openstack-ironic02:47
*** dsneddon has joined #openstack-ironic02:58
*** ricolin_ has joined #openstack-ironic03:06
*** dsneddon has quit IRC03:21
*** sabedevops has joined #openstack-ironic04:31
*** sabedevops has quit IRC04:37
openstackgerritMerged openstack/ironic-prometheus-exporter master: build universal wheels  https://review.opendev.org/67294105:17
*** brtknr has quit IRC05:17
*** brtknr has joined #openstack-ironic05:19
*** rachit7 has joined #openstack-ironic05:19
*** whoami-rajat has joined #openstack-ironic05:22
*** zaneb has quit IRC05:39
*** zbitter has joined #openstack-ironic05:39
*** e0ne has joined #openstack-ironic05:41
*** e0ne has quit IRC06:11
*** mkrai has joined #openstack-ironic06:36
*** etingof has joined #openstack-ironic06:39
*** tesseract has joined #openstack-ironic07:08
*** iurygregory has joined #openstack-ironic07:11
*** rcernin has quit IRC07:11
*** rpittau|afk is now known as rpittau07:11
rpittaugood morning ironic! o/07:12
iurygregorygood morning rpittau and Ironic o/07:14
rpittauhey iurygregory :)07:14
*** tssurya has joined #openstack-ironic07:14
*** pcaruana has joined #openstack-ironic07:31
*** lucasagomes has joined #openstack-ironic07:41
*** dsneddon has joined #openstack-ironic07:50
*** dsneddon has quit IRC07:55
*** iurygregory has quit IRC08:01
*** dougsz has joined #openstack-ironic08:12
*** iurygregory has joined #openstack-ironic08:31
*** brtknr has quit IRC08:33
*** brtknr has joined #openstack-ironic08:36
*** yolanda has joined #openstack-ironic08:42
*** hwoarang__ has quit IRC08:43
*** hwoarang has joined #openstack-ironic08:43
*** gkadam has joined #openstack-ironic08:44
*** Goneri has joined #openstack-ironic08:45
*** mkrai has quit IRC08:46
*** derekh has joined #openstack-ironic08:53
*** dtantsur|afk is now known as dtantsur08:58
*** tesseract has quit IRC08:58
dtantsurmorning08:58
*** dsneddon has joined #openstack-ironic09:00
*** dsneddon has quit IRC09:05
*** mkrai has joined #openstack-ironic09:07
*** e0ne has joined #openstack-ironic09:08
iurygregorymorning dtantsur09:11
*** hemanth_n has joined #openstack-ironic09:14
*** trident has quit IRC09:16
*** trident has joined #openstack-ironic09:17
*** tesseract has joined #openstack-ironic09:20
*** csmart has quit IRC09:22
*** ajya[m] has quit IRC09:22
*** altlogbot_2 has quit IRC09:24
*** altlogbot_2 has joined #openstack-ironic09:25
*** ricolin_ has quit IRC09:29
*** csmart has joined #openstack-ironic09:34
openstackgerritDmitry Tantsur proposed openstack/ironic master: Add an option to abort cleaning and deployment if node is in maintenance  https://review.opendev.org/36682809:37
*** S4ren has joined #openstack-ironic09:39
openstackgerritDmitry Tantsur proposed openstack/ironic master: Allow configuring global deploy and rescue kernel/ramdisk  https://review.opendev.org/67298909:47
*** Goneri has quit IRC09:49
*** Goneri has joined #openstack-ironic10:04
dtantsurTheJulia: I have some bad news: apparently reloading configuration with SIGHUP doesn't work because oslo.service shutdowns on SIGHUP :(10:04
dtantsurokay, false alert, I may be misinterpreting the logs10:07
openstackgerritDmitry Tantsur proposed openstack/ironic master: Allow configuring global deploy and rescue kernel/ramdisk  https://review.opendev.org/67298910:08
*** Lucas_Gray has joined #openstack-ironic10:21
*** ajya[m] has joined #openstack-ironic10:26
*** brtknr has quit IRC10:30
*** brtknr has joined #openstack-ironic10:32
*** brtknr has quit IRC10:34
*** Wryhder has joined #openstack-ironic10:35
*** brtknr has joined #openstack-ironic10:35
*** Lucas_Gray has quit IRC10:35
*** brtknr has quit IRC10:36
*** Wryhder is now known as Lucas_Gray10:36
*** jawad_axd has joined #openstack-ironic10:38
*** yaawang has quit IRC10:40
*** brtknr has joined #openstack-ironic10:45
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Add an option to abort cleaning and deployment if node is in maintenance  https://review.opendev.org/36682810:56
*** mkrai has quit IRC10:57
*** mkrai has joined #openstack-ironic10:57
*** gkadam has quit IRC11:02
*** brtknr has quit IRC11:04
openstackgerritShivanand Tendulker proposed openstack/ironic master: Fixes power-on failure for 'ilo' hardware type  https://review.opendev.org/67300811:05
*** brtknr has joined #openstack-ironic11:05
*** brtknr has quit IRC11:06
*** brtknr has joined #openstack-ironic11:06
*** brtknr has quit IRC11:07
*** brtknr has joined #openstack-ironic11:08
*** hemanth_n has quit IRC11:10
*** Lucas_Gray has quit IRC11:15
*** gkadam has joined #openstack-ironic11:17
*** brtknr has quit IRC11:20
*** brtknr has joined #openstack-ironic11:21
*** brtknr has quit IRC11:22
*** brtknr has joined #openstack-ironic11:23
*** brtknr has quit IRC11:23
*** brtknr has joined #openstack-ironic11:24
*** brtknr has quit IRC11:24
*** brtknr has joined #openstack-ironic11:25
*** Lucas_Gray has joined #openstack-ironic11:26
*** jaypipes has joined #openstack-ironic11:26
*** Lucas_Gray has quit IRC11:28
*** Lucas_Gray has joined #openstack-ironic11:33
TheJuliadtantsur: Oh.... good. I almost didn't get out of bed11:36
TheJuliapillow or sighup... pillow or sighup11:36
*** gkadam has quit IRC11:42
*** derekh has quit IRC11:43
*** gkadam has joined #openstack-ironic11:44
*** rh-jelabarre has joined #openstack-ironic12:02
TheJuliaDoes anyone have any thoughts on backporting https://review.opendev.org/#/c/670697 ?12:07
patchbotpatch 670697 - ironic - IPA does not boot up after cleaning reboot for 're... (MERGED) - 2 patch sets12:07
TheJuliahttps://review.opendev.org/#/c/654038 would love some reviews12:13
patchbotpatch 654038 - ironic - Option to send all portgroup data - 7 patch sets12:13
*** mkrai has quit IRC12:13
TheJuliaxXraphXx: I believe https://review.opendev.org/#/c/669906/ needs to be backported?12:16
patchbotpatch 669906 - ironic - Ansible module: fix deployment for private and/or ... (MERGED) - 1 patch set12:16
TheJuliahttps://review.opendev.org/#/c/666296/4 would also love some reviews12:17
patchbotpatch 666296 - sushy-tools - Add state sharing and persistence mechanism - 4 patch sets12:17
*** jcoufal has joined #openstack-ironic12:22
xXraphXxTheJulia: yep, we can backport it. Doing. Question, how far in the past should we backport ? Queens or Rocky ?12:24
TheJuliaxXraphXx: As far back makes sense as a consumer. So if you want to backport it all the way to ocata, we'll merge it, but it will never be officially released12:26
openstackgerritraphael.glon proposed openstack/ironic stable/stein: Ansible module: fix deployment for private and/or shared images  https://review.opendev.org/67328312:28
TheJuliaNobodyCam: If you review  https://review.opendev.org/#/c/662057/, I'll make you guys pizza when I'm back home in socal.12:29
patchbotpatch 662057 - bifrost - Fix troubleshooting docs - SSH access to CoreOS - 2 patch sets12:29
openstackgerritraphael.glon proposed openstack/ironic stable/rocky: Ansible module: fix deployment for private and/or shared images  https://review.opendev.org/67328412:29
*** ricolin_ has joined #openstack-ironic12:30
*** rachit7 has quit IRC12:31
TheJuliaxXraphXx: I've added those two patches to the etherpad so they have some review visibility12:38
xXraphXxok thanks12:42
TheJuliaetingof: by chance did you investigate the standalone job failure on https://review.opendev.org/#/c/638453/ ?12:46
patchbotpatch 638453 - ironic - Add Redfish Virtual Media Boot support - 40 patch sets12:46
etingofTheJulia, "Unable to stat device /dev/vda1 after attempting to verify 3 times", hmmm12:48
TheJuliaetingof: on an iscsi deploy?12:48
TheJuliaactually12:49
TheJuliathat wouldn't matter12:49
etingofperhaps, it's in ironic.drivers.modules.agent12:49
TheJuliaThat CI bug is getting annoying12:49
TheJuliaYeah, it is telling the agent to get started12:49
TheJuliaand boom12:49
etingofis it a famous bug?12:50
TheJuliaIts a bug where the agent doesn't see the freshly partitioned partition12:51
etingofright, why that happens?12:52
TheJuliaMy theory is disk IO latency because the rescans occur using O_DIRECT, so I bet they have code that only allows it to try for so long...12:55
TheJuliaSo if a chance is made via iscsi, there is no guarantee it is on the platter yet.12:56
TheJuliaAlternatively, the device could be locked. If locked those programs will typically skip the device12:56
* TheJulia changes locations12:57
etingofmay be more rescans then? the bread and butter of software engineers12:57
rpittauTheJulia: I think it makes sense to backport https://review.opendev.org/#/c/670697 , just wondering how clean that would be older than Rocky releases13:00
patchbotpatch 670697 - ironic - IPA does not boot up after cleaning reboot for 're... (MERGED) - 2 patch sets13:00
TheJuliaetingof: more rescans likely won't fix it, I think we already try ten times. Maybe if we backed-off between them. As a crazy idea. What if we had the conductor call for a sync() or fsync() after the iscsi write process completes?13:06
TheJuliaIt would be performance impacting because that call is blocking and depending on activity level the conductor could be very busy.... but we don't recommend iscsi for high usage environments.13:06
TheJulias/high/heavy/13:06
*** jroll has quit IRC13:07
*** jawad_axd has quit IRC13:08
*** jroll has joined #openstack-ironic13:08
etingofwould it help of the agent would do sync() instead? is not it its local drive after all?13:08
etingofor it won't help because the data may be still on the conductor side?13:09
TheJuliaetingof: not really, because it has no idea what is coming over the iscsi connection13:09
TheJuliaif that file system table update or even the io packets are still streaming down, its not going to know it13:10
etingofso that's the essence of the racing here13:10
etingofwe do fdisk over iscsi and then proceed locally13:11
etingofwhat if the agent would mount its own iscsi target?13:11
TheJuliaor we write a whole disk image and the device is locked13:11
etingofin the latter case it seems reasonable to back off13:12
TheJuliaetingof: won't change the scenario13:12
etingofso why the agent wants to see the partition table?13:14
*** S4ren has quit IRC13:15
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic master: CI Documentation  https://review.opendev.org/67277813:18
TheJuliaetingof: determine the root/boot partition uuids13:24
etingofmay be conductor could communicate that?13:25
*** rloo has joined #openstack-ironic13:26
TheJuliaThat woud be majorly incompatible retooling13:29
etingofprotocol change? or what do you mean - retooling?13:30
TheJuliaso the conductor may know nothing, and it may just be streaming a whole disk image13:35
TheJuliait can stop, rescan, but even then until it's buffers are flushed out complete, that data is not completetly on the remote host13:35
TheJuliaso that remote host can't proceed, and even the conductor wants that data if a direct to disk write-out is occuring, it has to scan and then orcustrate the needful13:36
*** whoami-rajat has quit IRC13:42
etingofit seems the conductor can read the partition table out of its local cache even if the data is not landed on the plates yet... but, yeah, it's not clear how to determine when the agent could proceed with its things...13:52
TheJuliaWell, problem is, even if the conductor tries to refresh and read it, it is direct_io which means cache bypassing in thisc ase13:58
TheJuliacase13:58
openstackgerritRiccardo Pittau proposed openstack/ironic stable/stein: IPA does not boot up after cleaning reboot for 'redfish' bios interface  https://review.opendev.org/67329614:01
*** S4ren has joined #openstack-ironic14:04
etingofah, O_DIRECT it is, yeah14:06
rpittauTheJulia: just verified, https://review.opendev.org/670697 is not a clean merge even from Rocky14:08
patchbotpatch 670697 - ironic - IPA does not boot up after cleaning reboot for 're... (MERGED) - 2 patch sets14:08
dtantsurTheJulia: hmm, I have a feeling that using SIGHUP does make ironic unusable: https://review.opendev.org/#/c/672989/ ...14:15
patchbotpatch 672989 - ironic - Allow configuring global deploy and rescue kernel/... - 3 patch sets14:15
* dtantsur changes for restart_service for now14:16
openstackgerritDmitry Tantsur proposed openstack/ironic master: Allow configuring global deploy and rescue kernel/ramdisk  https://review.opendev.org/67298914:16
*** Lucas_Gray has quit IRC14:17
*** Lucas_Gray has joined #openstack-ironic14:17
dtantsurOUCH14:20
dtantsurfolks, our CI seems to have gone RED. all IPMI jobs.14:20
dtantsurmmm, not all, bifrost and metalsmith seem unaffected14:21
dtantsurNode did not reach manageable state in 120 seconds14:21
*** S4ren has quit IRC14:23
*** S4ren has joined #openstack-ironic14:23
dtantsuretingof: do you know why we still start vbmc the old way? http://logs.openstack.org/84/673284/1/check/ironic-dsvm-standalone/bd45089/controller/logs/ironic-bm-logs/virtualbmc_log.txt.gz#_2019-07-29_12_57_42_99814:24
dtantsuralso this is suspicious http://logs.openstack.org/84/673284/1/check/ironic-dsvm-standalone/bd45089/controller/logs/ironic-bm-logs/virtualbmc_log.txt.gz#_2019-07-29_12_59_09_30414:24
etingofdtantsur, that's the fallback, it occurs is the mother-daemon is dead14:25
dtantsuretingof: right, but wasn't it fixed?14:25
dtantsurhttps://opendev.org/x/pyghmi/commit/eb736283606fa5120f47012f917b945141a722f4 is a suspect for the breakage14:25
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic master: CI Documentation  https://review.opendev.org/67277814:25
etingofdtantsur, I think we changed devstack ironic to invoke the master daemon long ago14:26
*** mkrai has joined #openstack-ironic14:31
openstackgerritDmitry Tantsur proposed openstack/ironic master: [DNM] Try blocking pyghmi 1.4.0  https://review.opendev.org/67330114:34
TheJuliaugh, I had a feeling there were CI issues this morning, but didn't look yet :(14:37
dtantsurTheJulia: https://storyboard.openstack.org/#!/story/2006301 I had a patch above to test it14:38
TheJuliaooh, ouch14:39
TheJuliabreaking pyghmi change?14:39
dtantsurI need to check again, u-c has pyghmi 1.3.0..14:40
dtantsurand we're bypassing u-c, so nice of us!14:41
TheJuliawoot14:41
*** mrtmr has joined #openstack-ironic14:42
*** sabedevops has joined #openstack-ironic14:45
openstackgerritDmitry Tantsur proposed openstack/ironic master: CI: stop using pyghmi from git master  https://review.opendev.org/67330114:45
dtantsurTheJulia: this may be the fix for now ^^^14:45
*** jaypipes has quit IRC14:46
dtantsurholy cow, now we have a storn14:46
dtantsur* storm14:46
mrtmrhi Im trying to do make a RAID disk deployment with bifrost. but ı couldnt find a any implementation in bifrost about RAID so I tried to make with ironic raid configuration after dynamic enrollment step in bifrost I go to openstack cli add raid conf file and then I go on with dynamic deployment step in bifrost but while in deploying step image part14:49
mrtmrition14:49
TheJuliadtantsur: like.... flying cows?14:49
TheJuliakind of storm?14:49
TheJuliamrtmr: interesting14:50
*** Lucas_Gray has quit IRC14:50
mrtmrcouldnt find any partition14:50
mrtmris there a way to do raid conf with bifrost14:51
*** Lucas_Gray has joined #openstack-ironic14:51
TheJuliayou'd have to directly invoke openstack cli commands because those fields are not available to be set via the ansible modules14:52
dtantsurTheJulia: yeah, if we had cows around, they would totally fly14:53
TheJuliamrtmr: mainly because nobody has had a need to extend the ansible module to suppor thte raid configuration fields. As for finding the partition, could you elaborate a little more so we can understand what your encountering?14:53
dtantsurI don't think openstacksdk even supports our RAID.. largely because of the crazy non-restful things we are doing :(14:54
TheJuliaI believe the fields should be able to be set... but no explicit extra help14:57
etingofit if's in Germany, I'd rather expect pigs flying by15:00
*** sbroderick has joined #openstack-ironic15:01
TheJuliaheh15:01
*** sabedevops has quit IRC15:01
iurygregorypigs flying seems to be cool15:01
etingofpigs flying fast might get hot15:02
iurygregorysince its 42C maybe you would get free bacon dtantsur XD15:03
dtantsurlol15:03
dtantsurfortunately it was "just" 33 today, and getting colder15:03
iurygregoryawesome I would say XD15:03
iurygregorywe are with 2815:04
* rpittau heppy for the wind keeping the temp "low", not happy about the fires15:04
iurygregorydtantsur, I think they merged a fix in pyghmi15:05
iurygregoryoh no its the patch that broke us15:06
*** Lucas_Gray has quit IRC15:06
iurygregorylol the only have jobs for pep-8 and unit tests O.o that would explain a lot hehehe15:06
*** Lucas_Gray has joined #openstack-ironic15:07
*** stendulker has joined #openstack-ironic15:07
mrtmrhttp://paste.openstack.org/show/755077/15:18
mrtmrI listed all what did I in this steps15:18
*** sbroderick has quit IRC15:20
openstackgerritIlya Etingof proposed openstack/sushy master: Disable HTTP connection pooling  https://review.opendev.org/67331215:24
*** Lucas_Gray has quit IRC15:26
*** tssurya has quit IRC15:26
*** mkrai has quit IRC15:27
*** e0ne has quit IRC15:30
mrtmrdtantsur TheJulia http://paste.openstack.org/show/755077/ i listed all things what did i :blush:15:31
*** sabedevops has joined #openstack-ironic15:31
*** dsneddon has joined #openstack-ironic15:31
openstackgerritRiccardo Pittau proposed openstack/ironic-inspector master: [WIP] use openstacksdk for ironic module  https://review.opendev.org/67217915:33
NobodyCamGood Morning Ironic'ers15:37
NobodyCamTheJulia:15:38
NobodyCamI'll take a look!15:38
*** jjohnson42 has joined #openstack-ironic15:38
jjohnson42dtantsur, sorry about that15:38
*** gyee has joined #openstack-ironic15:39
*** Lucas_Gray has joined #openstack-ironic15:41
*** mkrai has joined #openstack-ironic15:41
*** mkrai has quit IRC15:43
TheJuliaNobodyCam: thanks15:44
NobodyCam:) hope you had a good weekend15:45
dtantsurjjohnson42: no worries, things happen15:47
*** lucasagomes has quit IRC15:47
jjohnson42dtantsur, 1.4.1 will be tagged shortly with the two lines added to restore 1.3.0 capability on bmc side15:48
dtantsurmany thanks!15:48
jjohnson42dtantsur, incidentally, I was driven to do the change due to the emergence of new servers without support for HMAC-SHA1-9615:48
jjohnson42so in ipmitool speak, those servers require -C 17 argument15:48
jjohnson42pyghmi as a client now tries for HMAC-SHA256-128 and then falls back to HMAC-SHA1 to accomodate those servers15:49
jjohnson42as a BMC, I think 1.4.2 will add properly identifying the requestors integrity/auth algorithm and using SHA256 or SHA1 correctly in response to the requestor15:50
jjohnson42which would allow tests for default (-C 3) and the ones that require -C 1715:50
dtantsurcool15:51
jjohnson42it's all frankly a bit silly, since HMAC transform renders the SHA1 weaknesses moot, but I understand why some circles that's hard to explain why SHA1 is 'dangerous' in digest but not HMAC15:52
openstackgerritDmitry Tantsur proposed openstack/ironic master: CI: stop using pyghmi from git master  https://review.opendev.org/67330115:53
dtantsurTheJulia: found another place where we clone pyghmi ^^15:53
dtantsurjjohnson42: yeah, doesn't sound like an easy conversation :D15:53
*** derekh has joined #openstack-ironic15:54
jjohnson42dtantsur, master and 1.4.1 should be available now/presently with the added two lines to fix the missing lib/len values for cipher suite 3 as a bmc15:58
*** rh-jelabarre has quit IRC15:58
dtantsurthx again!15:59
rpittaubye all! good night! o/16:04
*** rpittau is now known as rpittau|afk16:04
*** sabedevops has quit IRC16:10
*** sabedevops has joined #openstack-ironic16:12
mrtmrdo you have any suggestion about raid configuration? http://paste.openstack.org/show/755077/16:12
*** stendulker has quit IRC16:15
*** Lucas_Gray has quit IRC16:20
*** rh-jelabarre has joined #openstack-ironic16:21
*** Goneri has quit IRC16:24
*** tesseract has quit IRC16:28
dtantsurmrtmr: ipmi doesn't have RAID on its own, the "agent" RAID interface assumes you have a hardware-specific RAID implementation on your deploy ramdisk. Is it the case? (if unsure, say no)16:36
*** sabedevops has quit IRC16:37
*** dtantsur is now known as dtantsur|afk16:44
dtantsur|afk\o16:44
*** S4ren has quit IRC16:47
*** sabedevops has joined #openstack-ironic16:50
*** sthussey has joined #openstack-ironic16:51
openstackgerritPavlo Shchelokovskyy proposed openstack/ironic master: Use HTTPProxyToWSGI middleware from oslo  https://review.opendev.org/67281916:58
*** igordc has joined #openstack-ironic16:59
*** dougsz has quit IRC16:59
*** derekh has quit IRC17:00
mrtmrdtantsur|afk what should I do RAID interface option, in ironic master branch document, it say "for Software RAID, this would be agent." on the workflow section17:05
mrtmrhttps://opendev.org/openstack/ironic/src/branch/master/doc/source/admin/raid.rst17:06
mrtmrı follow this document17:06
*** e0ne has joined #openstack-ironic17:14
*** e0ne has quit IRC17:20
*** sabedevops has quit IRC17:22
*** ricolin_ has quit IRC17:24
*** sabedevops has joined #openstack-ironic17:36
TheJuliamrtmr: You would use agent, and you'd pass configuration information to tell the agent to build a software raid set17:38
*** whoami-rajat has joined #openstack-ironic17:39
TheJuliamrtmr: oh! so your not invoking the actual raid configuraiton step, your just wanting it to happen upon deploy?17:40
*** sabedevops has quit IRC17:42
openstackgerritRiccardo Pittau proposed openstack/ironic-inspector master: [WIP] use openstacksdk for ironic module  https://review.opendev.org/67217917:46
*** trident has quit IRC17:47
*** trident has joined #openstack-ironic17:51
mrtmrTheJulia http://paste.openstack.org/show/755077/ I follow this steps, I set the raid conf with that --target-raid-config param before deployment playbook but while in deploying my node doesdnt know /dev/sda or /dev/md0. Im doing someting wrong or missing steps17:55
*** sabedevops has joined #openstack-ironic17:57
mrtmrI thought if i set root_device param with that /dev/md0 in openstack cli it could work  but nothing I got same error (error massage is in the paste)17:58
*** jjohnson42 has quit IRC18:10
*** jjohnson42 has joined #openstack-ironic18:10
*** sabedevops has quit IRC18:33
*** sabedevops has joined #openstack-ironic18:34
TheJuliathat won't work if the device is not set yet18:36
TheJuliaerr, created18:36
TheJuliahmm18:36
openstackgerritJulia Kreger proposed openstack/ironic master: Change debug to error for heartbeats  https://review.opendev.org/64944318:41
TheJuliamjturek: o/ looks like powerkvm ci is having issues18:52
TheJuliahttps://oplab9.parqtec.unicamp.br/pub/ppc64el/openstack/ironic/01/673301/3/check-ironic/tempest-dsvm-ironic-agent_ipmitool/973713a/console.html18:52
TheJuliaerr18:52
* TheJulia blinks at the url18:52
openstackgerritMerged openstack/ironic-prometheus-exporter master: Payload Timestamp Metric  https://review.opendev.org/67151418:58
openstackgerritMerged openstack/ironic-prometheus-exporter master: Voltage Metrics  https://review.opendev.org/67160918:58
*** mrtmr has quit IRC19:01
openstackgerritJulia Kreger proposed openstack/ironic master: Enable no IP address to be returned  https://review.opendev.org/49937619:10
*** e0ne has joined #openstack-ironic19:22
*** jcoufal_ has joined #openstack-ironic19:28
*** jcoufal has quit IRC19:31
*** sabedevops has quit IRC19:31
*** jcoufal_ has quit IRC19:34
TheJuliahmm... mrtmr timed out :(19:50
openstackgerritMerged openstack/ironic master: Remove deprecated Neutron authentication options  https://review.opendev.org/67297120:17
*** jjohnson2_ has joined #openstack-ironic20:32
openstackgerritMerged openstack/ironic master: Fix potential race condition on node power on and reboot  https://review.opendev.org/63383320:34
*** jjohnson42 has quit IRC20:35
*** jjohnson2_ has quit IRC20:39
*** whoami-rajat has quit IRC20:39
*** e0ne has quit IRC20:46
*** trident has quit IRC20:49
*** eandersson has joined #openstack-ironic20:49
*** trident has joined #openstack-ironic20:52
*** betherly has joined #openstack-ironic21:42
*** betherly has quit IRC21:47
*** bdodd has quit IRC21:47
*** betherly has joined #openstack-ironic21:48
*** betherly has quit IRC21:53
*** tripul0noob has joined #openstack-ironic22:09
tripul0noobHi all.22:10
tripul0noobI am using tripleo and it seems a few machines are in a state of:" clean wait" after introspection.22:10
tripul0noobSeems that it was to clean the disks for me, but likely timed out.22:10
tripul0noobNow i'm wondering how I can reset that state of the few failed physical machines and/or redeploy without clean option.22:10
openstackgerritMerged openstack/ironic master: CI: stop using pyghmi from git master  https://review.opendev.org/67330122:11
tripul0noobShould I just be able to remove the node, re-add and re-introspect as has been proposed? Or will it likely enter the same state? :-/22:11
tripul0noobbug #1786109 seems to be a candidate for this.22:12
openstackbug 1786109 in tripleo "did not reach state "available", the state is "clean wait", error: None'" [Medium,Triaged] https://launchpad.net/bugs/1786109 - Assigned to wes hayutin (weshayutin)22:12
*** tripul0noob is now known as xarlos22:28
*** xarlos is now known as tripul0noob22:28
tripul0noob.22:28
tonybThis is kinda an overlap question with TripleO but apart from https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/node_discovery.html are there any docs on using auto-discovery?22:39
tonybI *kinda* thought if I configured tripleo/ironic for it all I really needed to do was boot a node and assuming is came up on the network the 'inspector' boot-target *should* work and have the node "do the right thing" but as I read $link it seems like I have the wrong idea22:40
tonybtripul0noob: I'd say it'll end up in the same state22:41
tonybtripul0noob: if this is a dev instance where security matters less you can disablr cleaning IIUC22:41
tripul0noobtonyb: It's a new deployment to go into lab. Looking at the result it seems /dev/sdb2 as a read only filesystem (it's a raid 1 config with just two disks)22:43
tripul0noobtonyb: There's nothing of any securty implications on the hardware at the moment. I would be happy to disable clean if I know how best to do this.22:44
tonybtripul0noob: https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/cleaning.html22:45
tripul0noob(I have re-attempted introspection, but alas the same result)22:45
tonybtripul0noob: Basically set [DEFAULT]/clean_nodes = False in the undercloud and reinstall/update22:45
tripul0noobI thought that would be the case. Wasn't sure if there was a --no-clean parameter somewhere.22:46
tripul0noobThat link you sent looks good (my google-fu seems to have taken me elsewhere!) so i'll see if I Can try a manual clean. If that fails, i'll reinstall the undercloud (seems like a big undertaking for such a small job)22:47
tonybtripul0noob: Yeah22:47
tonybtripul0noob: the *right* thing to do would eb to get the cleanign working but I certainly can't hepl with that22:47
tonybtripul0noob: this is just a bandaid so you can make progress and then come back to this later22:48
tonybtripul0noob: as you can do manual cleaning at any stage you and iterate on the hardware then until you figure out what the problem is22:48
tripul0noobI am attempting a manual clean now.22:50
tripul0noobSeems weird it's referring to a read only filesystem though.22:50
tonybtripul0noob: Yeah It may bee your RAID config is b0rked somehow22:50
*** rcernin has joined #openstack-ironic23:02
*** dsneddon has quit IRC23:03
tripul0noobtonyb: Entirely possible. Been swapping stuff about.23:05
tripul0noobThough, not on this one that I know of.23:05
tripul0noobI wonder if previous UEFI deployments are getting in the way here.23:15
tripul0noobFigurative shotgun to the face aught to sort that out.23:16
*** dsneddon has joined #openstack-ironic23:31
tripul0noobwell. That didn't seem to work either.23:45
*** dsneddon has quit IRC23:45
tripul0noobI'm going to bed. Will have a fresh look in the morning.23:45
tripul0noobThanks for help, learned much as always.23:45
*** tripul0noob has left #openstack-ironic23:46
mrdaMorning Ironic23:48
*** sthussey has quit IRC23:51

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!