Friday, 2020-09-18

openstackgerritYushiro FURUKAWA proposed openstack/ironic master: Support iRMC hardware type again  https://review.opendev.org/75258700:08
*** ricolin has quit IRC00:22
*** dsneddon has quit IRC00:31
*** Qianbiao has quit IRC00:54
*** gyee has quit IRC01:03
*** dsneddon has joined #openstack-ironic01:08
*** VisBits has joined #openstack-ironic01:15
*** mushrushu_ has joined #openstack-ironic01:17
*** VisBits_ has quit IRC01:19
*** mushrushu has quit IRC01:19
*** mushrushu_ is now known as mushrushu01:19
*** zzzeek has quit IRC01:21
*** zzzeek has joined #openstack-ironic01:22
*** rloo has quit IRC01:26
*** gregwork has quit IRC01:27
*** Qianbiao has joined #openstack-ironic01:41
*** dsneddon has quit IRC01:44
*** dsneddon has joined #openstack-ironic01:46
*** rcernin has quit IRC02:29
*** vmud213 has joined #openstack-ironic02:49
*** zzzeek has quit IRC02:54
*** zzzeek has joined #openstack-ironic02:54
*** zzzeek has quit IRC03:01
*** zzzeek has joined #openstack-ironic03:04
*** rcernin has joined #openstack-ironic03:15
*** Qianbiao has quit IRC03:15
*** Qianbiao has joined #openstack-ironic03:15
*** rh-jelabarre has quit IRC03:20
*** ricolin has joined #openstack-ironic03:23
*** mkrai has joined #openstack-ironic03:25
*** VisBits_ has joined #openstack-ironic03:39
*** ricolin_ has joined #openstack-ironic03:42
*** VisBits has quit IRC03:42
*** Qianbiao has quit IRC04:06
*** mkrai has quit IRC04:06
*** mkrai_ has joined #openstack-ironic04:06
*** VisBits has joined #openstack-ironic04:06
openstackgerritRichard G. Pioso proposed openstack/ironic master: Fix redfish BIOS to use @Redfish.SettingsApplyTime  https://review.opendev.org/75261404:08
*** VisBits_ has quit IRC04:09
*** ianychoi has quit IRC04:29
*** mkrai_ has quit IRC04:32
*** mkrai has joined #openstack-ironic04:34
*** Qianbiao has joined #openstack-ironic04:41
*** tzumainn has quit IRC04:45
*** mkrai has quit IRC04:50
*** mkrai_ has joined #openstack-ironic04:50
openstackgerritRichard G. Pioso proposed openstack/ironic master: Add Redfish BIOS interface to idrac HW type  https://review.opendev.org/74924005:14
*** zzzeek has quit IRC05:19
*** zzzeek has joined #openstack-ironic05:19
*** Qianbiao has quit IRC05:22
openstackgerritRichard G. Pioso proposed openstack/ironic master: Add Redfish BIOS interface to idrac HW type  https://review.opendev.org/74924005:22
openstackgerritRichard G. Pioso proposed openstack/ironic master: Add Redfish BIOS interface to idrac HW type  https://review.opendev.org/74924005:23
*** vmud213 has quit IRC05:28
*** vmud213 has joined #openstack-ironic05:31
arne_wiebalckGood morning, ironic!05:50
*** Qianbiao has joined #openstack-ironic06:25
*** mkrai_ has quit IRC06:40
*** mkrai_ has joined #openstack-ironic06:40
vmud213Good morning Ironic, arne_wiebalck06:40
arne_wiebalckGood morning vmud213 o/06:41
iurygregorygood morning arne_wiebalck vmud213 and Ironic! happy friday06:44
arne_wiebalckhey iurygregory o/06:45
vmud213Good morning iurygregory o/06:45
iurygregoryo/06:45
iurygregoryo/06:45
*** jawad_axd has joined #openstack-ironic06:45
* iurygregory grabs coffee06:46
*** rajinir has quit IRC06:49
*** rajinir has joined #openstack-ironic06:52
*** portdirect has quit IRC06:53
*** seongsoocho has quit IRC06:53
*** guilhermesp has quit IRC06:53
*** andrein has quit IRC06:53
*** phrobb has quit IRC06:53
*** johnsom has quit IRC06:53
*** vdrok has quit IRC06:53
*** rpittau|afk has quit IRC06:53
*** pas-ha has quit IRC06:53
*** donnyd has quit IRC06:53
*** sri_ has quit IRC06:53
*** buhman has quit IRC06:53
*** TheJulia has quit IRC06:53
*** ildikov has quit IRC06:53
*** andrein has joined #openstack-ironic06:55
*** phrobb has joined #openstack-ironic06:55
*** vdrok has joined #openstack-ironic06:56
*** ildikov has joined #openstack-ironic06:56
*** portdirect has joined #openstack-ironic06:56
*** seongsoocho has joined #openstack-ironic06:56
*** guilhermesp has joined #openstack-ironic06:56
*** rpittau|afk has joined #openstack-ironic06:56
*** TheJulia has joined #openstack-ironic06:57
*** donnyd has joined #openstack-ironic06:57
*** johnsom has joined #openstack-ironic06:57
*** buhman has joined #openstack-ironic06:58
*** sri_ has joined #openstack-ironic06:58
*** mkrai_ has quit IRC07:06
iurygregoryarne_wiebalck, if you have a minute today https://review.opendev.org/#/c/705030/ we need to merge this change of the community goal o/07:08
patchbotpatch 705030 - ironic - Native zuulv3 grenade multinode multitenant - 35 patch sets07:08
*** pas-ha has joined #openstack-ironic07:09
openstackgerritzhen proposed openstack/ironic stable/ussuri: Allow configuring IPMI cipher suite  https://review.opendev.org/75263307:10
iurygregoryI don't remember an agreement that we can backport this... O.o07:11
*** derekh has joined #openstack-ironic07:20
*** tosky has joined #openstack-ironic07:25
*** tremble has joined #openstack-ironic07:27
arne_wiebalckI'd think strictly speaking this is a feature. But given that upgrading ipmitool may leave you with no access to your nodes, this may warrant an exception. There was no decision on this yet, from what I saw.07:37
*** ociuhandu has joined #openstack-ironic07:45
iurygregoryarne_wiebalck, yup!07:46
*** lucasagomes has joined #openstack-ironic08:14
*** ricolin_ has quit IRC08:22
*** dtantsur|afk is now known as dtantsur08:27
dtantsurmorning ironic08:28
iurygregorymorning dtantsur o/08:29
*** Qianbiao has quit IRC08:30
* dtantsur has learned that bifrost downgrade does not work :)08:30
dtantsurjanders: hey, please let me know if https://review.opendev.org/#/c/752411/ fixed you deploy-dynamic struggles08:31
patchbotpatch 752411 - bifrost - Gather facts on localhost before running {enroll,d... - 1 patch set08:31
jandersgood morning Ironic o/08:34
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Gather facts on localhost before running {enroll,deploy}-dynamic  https://review.opendev.org/75241108:38
*** k_mouza has joined #openstack-ironic08:41
*** Qianbiao has joined #openstack-ironic08:42
*** derekh has quit IRC08:55
*** mkrai has joined #openstack-ironic09:01
*** k_mouza has quit IRC09:04
*** k_mouza has joined #openstack-ironic09:12
*** k_mouza has quit IRC09:16
*** k_mouza has joined #openstack-ironic09:18
*** derekh has joined #openstack-ironic09:26
*** dking has quit IRC09:28
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Fix internal_ip detection in {enroll,deploy}-dynamic  https://review.opendev.org/75241109:51
dtantsurjanders: try this please ^^09:51
*** uzumaki has joined #openstack-ironic09:51
*** vmud213 has quit IRC09:54
*** rcernin has quit IRC10:09
jandersACK, on it10:13
openstackgerritDmitry Tantsur proposed openstack/bifrost master: bifrost-cli: do not provide any release if non can be detected  https://review.opendev.org/75265810:14
jandersdtantsur success! ( http://paste.openstack.org/show/798057/ )10:16
jandersnicely done10:16
dtantsurcool, thanks for testing. now let's hope the CI agrees :)10:16
janderson that positive note - have a great weekend everyone!10:20
jandersSee you next week Ironic o/10:20
dtantsuro/10:20
iurygregorybye janders enjoy the weekend o/10:23
*** mkrai has quit IRC10:34
*** mkrai_ has joined #openstack-ironic10:34
* dtantsur is impressed that test-bifrost.sh simply does not work on stable/ussuri outside of the CI, and nobody has complained10:39
dtantsurspeaking of which, could I get some reviews on https://review.opendev.org/#/c/752473/ please?10:40
patchbotpatch 752473 - bifrost - Ensure that repositories are owned by the running ... - 2 patch sets10:40
iurygregorydtantsur, I will look after lunch10:40
dtantsurthx10:40
*** rcernin has joined #openstack-ironic10:44
*** rcernin has quit IRC10:54
Qianbiaohello ironic.11:10
Qianbiaohi, if i put inspect dhcp on an isolate vlan.11:10
Qianbiaothen how inspect IPA communicate with ironic-inspector11:10
openstackgerritDmitry Tantsur proposed openstack/bifrost master: bifrost-configdrives-dynamic: automatically find ed25519 SSH keys  https://review.opendev.org/75267311:15
*** rcernin has joined #openstack-ironic11:18
*** rcernin has quit IRC11:23
*** k_mouza has quit IRC11:45
*** rh-jelabarre has joined #openstack-ironic11:46
openstackgerritVerification of a change to openstack/ironic failed: Handle an older agent with agent_token  https://review.opendev.org/74958511:47
*** k_mouza has joined #openstack-ironic11:51
*** rcernin has joined #openstack-ironic11:54
openstackgerritMerged openstack/metalsmith master: Rework jobs so failed devstack does not retry the job  https://review.opendev.org/74975811:55
openstackgerritMerged openstack/metalsmith master: Change metalsmith nodeset to Focal  https://review.opendev.org/75048911:55
*** k_mouza has quit IRC11:55
*** dking has joined #openstack-ironic11:56
*** k_mouza has joined #openstack-ironic11:58
*** rcernin has quit IRC11:59
*** ricolin_ has joined #openstack-ironic12:02
openstackgerritDmitry Tantsur proposed openstack/metalsmith stable/victoria: Rework jobs so failed devstack does not retry the job  https://review.opendev.org/75268012:27
*** fgofurov_ has joined #openstack-ironic12:30
dtantsurCould someone please request releases for IPA-builder and virtualbmc?12:35
*** k_mouza_ has joined #openstack-ironic12:40
*** k_mouza has quit IRC12:42
*** Goneri has joined #openstack-ironic12:42
iurygregorydtantsur, sure!12:44
openstackgerritDmitry Tantsur proposed openstack/bifrost master: [PoC] A naive attempt at creating an upgrade job  https://review.opendev.org/75037612:46
openstackgerritVerification of a change to openstack/ironic failed: Do not assume that prepare_image is the last command to run  https://review.opendev.org/74991312:47
dtantsurthe upgrade job works locally but fails in the CI. computers were clearly a mistake.12:48
*** VisBits has quit IRC12:52
*** VisBits has joined #openstack-ironic12:54
openstackgerritDmitry Tantsur proposed openstack/ironic master: Deprecate the iscsi deploy interface  https://review.opendev.org/75020412:57
iurygregoryyou were trying in a fresh env?13:06
*** nam-est has joined #openstack-ironic13:09
*** cdearborn has joined #openstack-ironic13:11
dtantsuriurygregory: yep, a new VM13:15
openstackgerritDmitry Tantsur proposed openstack/ironic master: Deprecate the iscsi deploy interface  https://review.opendev.org/75020413:16
*** k_mouza_ has quit IRC13:18
*** tosky_ has joined #openstack-ironic13:18
*** tosky is now known as Guest8304713:18
*** tosky_ is now known as tosky13:18
iurygregorydtantsur, damm =(13:20
*** k_mouza has joined #openstack-ironic13:22
*** uzumaki has quit IRC13:23
nam-estHi all, I have one question13:24
openstackgerritDmitry Tantsur proposed openstack/bifrost master: [PoC] A naive attempt at creating an upgrade job  https://review.opendev.org/75037613:25
nam-estDoes any know which commits created the version 15.1.1.dev16 of Ironic?13:26
dtantsurnam-est: the versions that pbr generate are not exactly trivial to understand. this is the 16th commit after 15.1.0.13:26
nam-estWhat is pbr? Sorry for my ignorance13:27
dtantsurnam-est: pbr is a library that we (as the whole openstack) use to manage our python packages13:27
dtantsurit handles some boilerplate tasks that distutils don't handle, like generating versions from git tags13:28
nam-estdtantsur: I see.thanks for your answer and explanation13:29
dtantsurnam-est: if you have a system with the ironic package installed, `pbr sha ironic` will tell you the git hash13:30
dtantsurinfo even better. e.g. on my in-progress patch it outputs `ironic 15.2.1.dev94 pre-release 98426f00b`13:30
dtantsur(I mean, `pbr info ironic`)13:30
TheJuliagood morning13:31
iurygregorygood morning TheJulia =)13:31
TheJuliaSorry about being missing yesterday, sometimes stuff happens and takes far longer to get a replacement found and the work started to have the failed item replaced13:31
dtantsurmorning TheJulia, happy Monday13:31
TheJuliawait, this is monday?!?13:31
iurygregoryhappy what dtantsur ?!13:32
dtantsuroh lol13:32
TheJuliaNOOOOOOO!!!!!!13:32
iurygregoryNOOOOOOOOO [2]13:32
dtantsurwell, you now know my state :)13:32
dtantsuryeah, happy Friday all13:32
* dtantsur probably needs another week in the mountains13:32
iurygregorydtantsur, I think you need to finish work early today...13:32
TheJuliadtantsur: ++13:32
dtantsursee? iurygregory permitted!13:32
TheJuliaSo what did I miss yesterday?13:32
* dtantsur tries to remember yesterday13:32
iurygregorydtantsur, I'm not shadower ...13:33
iurygregorybut I think he will agree XD13:33
dtantsurheh13:33
dtantsurTheJulia: I remember cursing ansible and having some meetings13:34
dtantsurthat's all my memory holds13:34
TheJuliamemory overwise overflowed and yeah13:34
TheJuliaand then you lost the word for cat or something13:34
TheJuliabecaues overflow13:34
nam-estdtantsur: Found it. `pbr sha ironic` is very userful. It is the commit 079b22a80. Thank you very much, dtantsur13:35
dtantsuryou're welcome13:35
openstackgerritVerification of a change to openstack/ironic-inspector failed: Power off before inspection  https://review.opendev.org/74984513:35
dtantsursigh13:36
iurygregoryit's friday and zuul is off already13:36
dtantsurthe inspector and IPA CIs are very unstable, ironic ussuri is pretty much dead13:36
TheJuliaare we in the typical end of cycle CI hates us period?13:36
iurygregoryyes13:36
dtantsurironic (master) and bifrost seem to behave13:36
iurygregorybecause we are close to release13:36
iurygregory=D13:37
TheJuliais it just timeoutes?13:37
TheJuliatimeouts13:37
dtantsurhaven't looked at ussuri13:37
dtantsurIPA was, I think, DIB images taking really long to run13:37
shadoweriurygregory, dtantsur: I'm not sure it's my place to jump in here as a non-contributor but yes, Fridays are for leaving early and having a good weekend \o/13:37
dtantsurinspector - our favourite non-standalone job13:37
dtantsurshadower: thanks :)13:37
*** rloo has joined #openstack-ironic13:37
TheJuliaI've seen some tox docs jobs have been observed as timing out recently13:37
TheJuliathere has been a whole discussion on it13:37
dtantsurand by the way, everyone is welcome here, as long as you behave (better than zuul)13:37
TheJuliashadower: ++++13:37
shadowerlol13:37
TheJuliadtantsur: ++++13:38
dtantsurTheJulia: I could use your input on this ansible mind-love: https://review.opendev.org/#/c/752411/13:38
patchbotpatch 752411 - bifrost - Fix internal_ip detection in {enroll,deploy}-dynamic - 3 patch sets13:38
dtantsurit fixes the problems janders had, but my ansible-fu is still not perfect13:38
TheJuliadtantsur: still in my email and sipping coffee, give me about 20 minutes and I should be able to context switch13:39
dtantsurenjoy your coffee13:39
dtantsurspeaking of which, we've got pumpkin-spice syrup, I should give it a go :)13:39
TheJuliathe iRMC folks emailed me worried we would remove their driver and stated that they almost have CI back up13:39
TheJuliaohhhh ahhh13:39
TheJuliaI told them not to stress and to stay safe and that we understood delays right now due to the pandemic.13:40
dtantsurI guess they have further delays.. I remember having a conversation with one contributor a week or two ago13:40
*** k_mouza has quit IRC13:40
TheJuliayeah13:40
TheJuliaGetting things "done" is interesting13:41
dtantsurrewrite it in go \o/13:41
* dtantsur ducks13:41
TheJulialike... parts to repair my car have a weird availability so I actually selected different parts entirely because of new delays13:41
TheJuliaDid tyler and his ports with pxe options appear in IRC yesterday?13:43
dtantsurI don't recall that, but, as I mentioned, my memory is not to be trusted13:44
TheJuliak13:44
dtantsurif anybody has free cycles (haha), we should probably update bifrost to use Focal as the primary nodeset for jobs, leaving Bionic in a non-voting job13:47
derekhUsing the standalone client, shouldn't I be able to add multiple --driver-info ? or am I doing something wrong? http://paste.openstack.org/show/798069/13:50
*** Qianbiao has quit IRC13:50
iurygregoryderekh, it's a bit wrong13:50
iurygregory--driver-info redfish_username=admin,redfish_password=pass test13:50
*** k_mouza has joined #openstack-ironic13:50
iurygregorytry like this to see how it goes13:51
derekhiurygregory: ahhh, gotcha, thanks13:51
*** k_mouza has quit IRC13:55
*** rcernin has joined #openstack-ironic13:55
derekhhmm, that doesn't look right13:56
derekh| driver_info            | {'redfish_username': 'admin,redfish_password=******'} |13:56
*** fmuyassarov has joined #openstack-ironic13:56
TheJuliano, that doesn't look right13:59
*** rcernin has quit IRC14:00
iurygregory--driver-info <key=value>14:01
iurygregory    Key/value pair used by the driver, such as out-of-band management credentials. Can be specified multiple times. (the way you were doing before seems correct after reading this)14:01
*** k_mouza has joined #openstack-ironic14:02
derekhlooks like the same problem came up in other clients in the passed https://review.opendev.org/#/c/459115/114:02
patchbotpatch 459115 - os-client-config - Make _fix_argv() somewhat compatible with Argparse... (MERGED) - 1 patch set14:02
iurygregoryif you are doing a request with post is a single dict with multiple key/value14:02
dtantsurI've fixed it in openstacksdk recently14:04
dtantsurbut maybe it hasn't been released yet?14:04
derekhdtantsur: this https://review.opendev.org/#/c/749903/ , checking if I have it14:05
patchbotpatch 749903 - openstacksdk - Fix a bogus error in config loader when using seve... (MERGED) - 1 patch set14:05
iurygregoryderekh, in CI logs I see we use like the first case you mentioned14:06
TheJuliadtantsur: regarding the gather facts change. Did ansible stop the auto collection for localhost?14:06
iurygregoryhttps://zuul.opendev.org/t/openstack/build/e287ae5d46274d508b306b79cf90de27/log/job-output.txt  --driver-info ipmi_address=10.176.193.84 --driver-info ipmi_username=admin --driver-info ipmi_password=password --driver-info ipmi_port=623014:06
dtantsurTheJulia: not sure, honestly14:06
TheJuliabut you added it and things are now happier, I guess I'm trying to understand why things were not just working to begin with14:07
derekhdtantsur: thanks, that was it  https://review.opendev.org/#/c/749903/ <-- iurygregory14:09
patchbotpatch 749903 - openstacksdk - Fix a bogus error in config loader when using seve... (MERGED) - 1 patch set14:09
iurygregoryaha =)14:09
dtantsurTheJulia: to be clear, I'm not sure which part of the patch is more important14:10
TheJuliadtantsur: so did some tablescans in my brain14:14
TheJuliait was default to always collect a long time ago14:14
TheJuliaat some point that changd so depending on how you invoked the run resulted in if facts got collected14:14
TheJuliaso I've +2'ed the change, I think it is good to land.14:14
dtantsurat least it shouldn't hurt14:16
openstackgerritJulia Kreger proposed openstack/ironic stable/ussuri: Allow configuring IPMI cipher suite  https://review.opendev.org/75263314:22
TheJuliadtantsur: ^^^14:22
dtantsurTheJulia: I wonder if we should add an 'issues' section to the release note, reflecting what ttx and I discussed this morning14:22
dtantsur(see #openstack-release if you haven't already)14:22
TheJuliaI have not...14:23
*** trandles has quit IRC14:25
TheJuliaThe TC has already taken the stand that it is acceptable to backport "gray" area items of this nature to support operators. I think you basically rehashed the same discussion14:25
TheJuliaissues may make sense to go ahead and note in case they go "this looks like a feature!"14:26
TheJuliaand decide to be unreasonable14:26
dtantsurTheJulia: yep, what I'd like to propose is a release note update to make it clear how this feature can be usable as a bug fix14:27
dtantsur(and to warn the operators about the potential problem)14:27
TheJuliaso I already partially augmented, I can just add an issues item to provide additional background context14:28
dtantsur++14:28
dtantsurbloody upgrade jobs, what's wrong with you?14:28
iurygregorydtantsur, ipa-builder 2.2.0 patch up (I've pushed 2.2.0 becasue we added the TLS support)14:29
dtantsuryep, thx14:29
iurygregorylooking at vbmc now14:29
*** mkrai_ has quit IRC14:34
openstackgerritJulia Kreger proposed openstack/ironic stable/ussuri: Allow configuring IPMI cipher suite  https://review.opendev.org/75263314:34
TheJuliadtantsur: ^^^14:34
dtantsurthx, will look after another pathetic attempt to make the upgrade job work..14:35
TheJulia:(14:35
iurygregorynot much on virtualbmc, most updates on l-c docs so I've pushed 2.1.114:35
TheJuliacould be worse, I figured out why centos8 change for metalsmith was breaking14:35
iurygregoryTheJulia, I think someone already pushed a backport for it ...14:36
iurygregoryoh ok it was an update =)14:36
TheJuliaos-prober apparently goes on a vacation searching for disks and chewing up CPU resources14:36
*** mkrai has joined #openstack-ironic14:36
TheJuliaand people have reported fedora 30/31 bugs indicating it can spin for 15-20+ minutes14:37
dtantsurlovely14:37
TheJuliayeah14:37
dtantsurand in the upgrade jobs upgrade happens, but for some reason ironic starts with old code14:38
dtantsurI couldn't reproduced it locally14:38
*** fmuyassarov has quit IRC14:38
TheJuliaI put up a wip change on ipa to give that a try.... I'm not entirely sure that is a "good idea" though since the grub option to use doesn't seem documented14:38
*** Qianbiao has joined #openstack-ironic14:43
QianbiaoHello,ironic.14:45
TheJuliagood morning14:45
Qianbiaohi, when i move inspect network to a special vlan.14:45
Qianbiaoipxe could not startup. it keeps using pxe.14:45
Qianbiaoany idea?14:45
Qianbiaohttp://paste.openstack.org/show/798073/14:46
Qianbiaowhen move inspect dhcp to normal network without vlan, it can ipxe.14:46
TheJulialooks like your chain loading is failing to work into ipxe14:46
TheJuliaI suspect your return tftp packet is not making it past any firewalling14:47
TheJuliaare you using ubuntu?14:47
Qianbiaoyes14:47
TheJuliaso they did remove the auto-insert of the connection tracker helper for tftp14:47
TheJuliaif you load the kmod, I think you'll be good14:47
Qianbiaoyes enable-tftp is not configed.14:47
Qianbiaobut what concerns me is that when without vlan, it can ipxe14:48
Qianbiaowhen dnsmasq listen to a vlan interface.14:48
Qianbiaoit can not ipxe14:48
TheJuliathat doesn't really make sense14:50
openstackgerritVerification of a change to openstack/ironic-python-agent failed: Generate a TLS certificate and send it to ironic  https://review.opendev.org/74993014:50
TheJuliabecause the reply is clearly getting through with the instruction to chainload, the undionly.kpxe binary14:50
TheJuliait seems tftp is where things are going sideways to load that :\14:50
*** Qianbiao has quit IRC14:51
openstackgerritJulia Kreger proposed openstack/ironic stable/train: Allow configuring IPMI cipher suite  https://review.opendev.org/75271014:51
*** Qianbiao has joined #openstack-ironic14:52
TheJuliatremble: ^^^14:52
dtantsurTheJulia: do you recall why bifrost uses xinetd rather than built-in dnsmasq tftp?14:52
TheJuliaxinetd has some logic to guard the back-end binary against abuse14:54
TheJuliaalso I don't think i managed to get the dnsmasq tftp service to behave14:55
*** fgofurov_ has quit IRC14:55
Qianbiao<TheJulia> ipxe requires tftp too?14:55
Qianbiaoi thought it directly visit http14:56
TheJuliaQianbiao: depends on your Network cards14:56
TheJuliaQianbiao: if your network cards HAVE ipxe embedded, they can go direct to HTTP14:56
Qianbiaook i see14:56
TheJuliaif they don't, they have to load ipxe from tftp14:56
TheJuliaIn that case, you only end up sending ~600kb-1.4MB over tftp instead of hundreds of megabytes14:56
Qianbiaogot14:57
Qianbiaoi will try enabled tftp in dnsmasq14:57
*** ianychoi has joined #openstack-ironic14:59
dtantsurTheJulia: metal3 uses dnsmasq, I think, hence my question15:02
dtantsurone less service for bifrost => faster deploy, less complexity15:02
TheJuliaahh, yeah I think it is fine to give it a shot with the dnsmasq service15:03
* dtantsur dives into in-band deploy steps again because https://storyboard.openstack.org/#!/story/200816715:03
TheJuliait never changed because it just worked15:03
*** mkrai has quit IRC15:05
*** Qianbiao has quit IRC15:05
TheJuliadtantsur: that seems super familiar to the error I was hitting on checking prepare_image command status when no other prior commands had executed15:08
dtantsurwell, there is a previous command in this case.. just.. it's considered done too early15:09
dtantsurwe.. have.. a race.. between.. two calls to RPC continue_node_deploy....15:10
* dtantsur doesn't even have energy to flip a table15:10
dtantsurSuccessfully released exclusive lock for heartbeat on node b5c31dba-759b-4784-9e5e-708bba0ad46c (lock was held 37.81 sec)15:15
dtantsurhow many???15:15
*** mkrai has joined #openstack-ironic15:15
dtantsurah, iscsi deploy. sure.15:16
*** Qianbiao has joined #openstack-ironic15:17
QianbiaoHmm, i found it already has a tftp process.15:20
Qianbiao"/usr/sbin/in.tftpd --listen --user tftp --address :69 --secure /var/lib/tftpboot"15:20
QianbiaoI do not know who start it, it watches wrong folder seems15:20
*** mkrai has quit IRC15:22
*** mkrai_ has joined #openstack-ironic15:22
* tremble thanks TheJulia 15:29
TheJuliadtantsur: I guess the lock is retrying when it maybe should retry less or kill itself it is hangs open for too long?15:30
dtantsurTheJulia: that's exactly the direction I'm going right now :)15:30
TheJuliaexcellent15:32
*** trandles has joined #openstack-ironic15:33
TheJuliaGood morning tremble15:33
TheJuliaerr15:33
TheJuliatrandles:15:33
* TheJulia tries again15:33
TheJuliaGood morning trandles15:33
trandlesgood morning TheJulia15:33
trandleslol15:33
trandlesIt's Friday, almost there, hold it together ;)15:33
* tremble bids TheJulia a good afternoon and hopes she has a good weekend.15:34
trandlesI rebooted my workstation 2 hours ago and forgot to fire up the IRC client again...15:34
TheJuliaWednesday our water heater decided it no longer wanted to be a water heater... Most of thursday was spent cleaning...15:34
* trandles wonders what else he's forgetting15:34
TheJuliaand now we have a wildfire under national forest service command15:34
TheJuliaso updates are not frequent15:34
trandlesyikes15:34
TheJuliayeah :(15:35
openstackgerritDmitry Tantsur proposed openstack/ironic master: Do not retry locking when heartbeating  https://review.opendev.org/75271615:35
dtantsur^^^15:35
dtantsurugh, wildfire--15:35
TheJuliaWe just pulled the cushions in from the courtyard because ash is now falling from the sky.15:35
trandleslast fall our water heater did the same, still have some minor drywall repair to do in the utility closet15:35
*** tremble has quit IRC15:35
trandlesok that's stressful15:36
dtantsurTheJulia: thinking of it, it doesn't eliminate the race completely15:37
dtantsura heartbeat can still arrive between the previous heartbeat and the continue_node_deploy call it caused15:38
dtantsurand the only cure is probably to get rid of the continue_node_deploy RPC...15:38
dtantsurwhich is risky because it's used by drivers....15:39
*** tkajinam has quit IRC15:44
QianbiaoTheJulia dtantsur thanks for the hint.15:49
Qianbiaoit's due to tftp-hpa service is start.15:49
Qianbiaoand xinetd.d failed to bind port.15:49
Qianbiaowe should disable tftp-hpa when install15:49
*** jawad_axd has quit IRC15:49
*** ricolin_ has quit IRC15:50
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: When reporting that agent is busy, report the executed command  https://review.opendev.org/75271915:53
*** lucasagomes has quit IRC15:54
*** uzumaki has joined #openstack-ironic15:55
*** rcernin has joined #openstack-ironic15:56
QianbiaoHello, TheJulia, the "/dev/ram0" does not exists issue occurs during load inspect ramdisk too now.15:56
QianbiaoIt 's because the way i build ramdisk is wrong?15:57
TheJuliaQianbiao: download and double check hashes of hwat is in glance veruses what you can verify from upstream15:57
TheJuliaoh, your building your own15:57
TheJuliahmmm15:57
QianbiaoI did follow diskimage-build doc15:57
TheJuliadid you use ironic-python-agent-builder?15:57
Qianbiaoyes15:58
TheJuliafor your agent ramdisk?15:58
TheJuliahmmmmmm15:58
QianbiaoI set element path15:58
TheJuliathat doesn't make sense15:58
QianbiaoI need to build it myself.15:58
Qianbiaobecause our node need to support IB port. i need mellanox element to support it.15:58
TheJuliaand I guess continue_node_deploy is not waiting on a lock or blocked on a lock :\15:59
QianbiaoIt's an issue from IPA?16:00
QianbiaoTheJulia I built the ramdisk on a ubuntu server, does this matters?16:01
QianbiaoAnd the inspect, it does not allways failed, it success before.16:02
*** rcernin has quit IRC16:03
dtantsuras promised, bailing out earlier. have a great weekend!16:10
*** dtantsur is now known as dtantsur|afk16:10
*** mkrai_ has quit IRC16:19
*** ociuhandu_ has joined #openstack-ironic16:23
*** ociuhandu has quit IRC16:27
*** ociuhandu_ has quit IRC16:28
*** k_mouza has quit IRC16:29
rpiosoTheJulia: I refactored the iDRAC Redfish BIOS into two changes. One is a bug fix and the other a new feature. The bug fix _could_  be considered for back porting.16:31
rpiosoTheJulia: The new feature depends on the bug fix.16:32
rpiosoTheJulia: Please let me know if that's an agreeable approach. If not, I'll revert.16:33
*** ociuhandu has joined #openstack-ironic16:33
TheJuliarpioso: that _sounds_ reasonable16:33
rpiosobfournie: ^^^16:33
rpiosoTheJulia: :-)16:33
rpiosoTheJulia: agreeable, reasonable, and _merge-able_ ;-)16:34
* rpioso grabs a meal16:35
TheJuliafood sounds like a really good idea16:35
TheJuliabut also it is only 9:30 am16:35
TheJuliaQianbiao: it doesn't seem like an issue with IPA. I guess I'm wondering "what changed"16:37
*** ociuhandu has quit IRC16:37
QianbiaoTheJulia I do not know where to start now.16:38
Qianbiaoany hint where i can begin to find out the issue.16:38
openstackgerritJay Faulkner proposed openstack/ironic-python-agent-builder master: Advertise HTTPS in IPA ramdisk without Ironic assist  https://review.opendev.org/75272816:38
JayFrloo: ^ Fixes the issue I referenced to you downstream16:38
TheJuliathe way to get the can't find ramdisk is maybe when the ramdisk uploaded is bad16:38
TheJuliacontent wise or maybe was the wrong file?16:38
TheJuliamaybe download the file from glance and extrat it?16:38
TheJuliaextract16:39
* JayF currently living in the house of TLS-pain16:39
Qianbiao<TheJulia> so, i should start from the image.16:40
* TheJulia expects someone has a literal house named that16:40
Qianbiaomake sure the image is right not broken.16:40
*** k_mouza has joined #openstack-ironic16:40
Qianbiaook will try.16:40
TheJuliaQianbiao: exactly16:40
* TheJulia expects only Klingon is spoken in the house of TLS-pain16:41
*** k_mouza has quit IRC16:44
dkingWhat are the options available for disk cleaning? Will ATA secure erase be used for all disk types? Is there a way to know what will be run or what has been run?16:50
JayFHey, that's completely configurable.16:51
*** k_mouza has joined #openstack-ironic16:51
JayFthere are two erase "methods" at a high level: erase_device_metadata, which wipes off partition tables, boot loaders and other basic metadata, and generally operates near instantaneously16:51
dkingIs it configurable using a custom hardware manager, or can it be customized with the generic one?16:51
JayFand erase_devices, which uses ATA secure erase if available, or shed if not16:52
JayFand you can totally override either or both of those methods with a custom hardware manager16:52
JayFit's literally the use case that was originally written for years and years ago :D16:52
JayFhttps://opendev.org/openstack/ironic-python-agent/src/branch/master/ironic_python_agent/hardware.py#L679 this is the "generic" erase_devices". If you notice, it calls out to `erase_block_device` for each block device.16:53
JayFSo you have two choices -- override "erase_devices" and write a method that can erase all devices on system, or override "erase_block_device" and have that erase devices it can support, and raise an unsupported method exception if it can't, so you can build fallbacks, or mix-and-match hardware managers for different hardware16:54
JayFthe best place to start is here: https://opendev.org/openstack/ironic-python-agent/src/branch/master/examples -- these contain a couple of example custom hardware managers. You probably want to follow the "vendor-device" example, and by just putting either an `erase_devices` or `erase_block_device` method on that hardawre manager, and advertising "higher" than generic for hardware16:55
JayFsupport, IPA will use your method instead16:55
JayFthis is also pretty well documented here: https://docs.openstack.org/ironic-python-agent/latest/contributor/hardware_managers.html#:~:text=Hardware%20managers%20are%20how%20IPA,methods%20compatible%20with%20most%20hardware.16:55
*** k_mouza has quit IRC16:55
JayFgood luck!16:55
dkingThank you very much!16:56
JayFno problem; good luck!16:57
JayFHappy to help if you hit a wall too, I've been down this road before16:57
*** Qianbiao has quit IRC16:58
*** Qianbiao has joined #openstack-ironic17:00
*** derekh has quit IRC17:01
*** k_mouza has joined #openstack-ironic17:04
JayFTheJulia: if you have a second, trivial review here: https://review.opendev.org/#/c/752728/17:05
patchbotpatch 752728 - ironic-python-agent-builder - Advertise HTTPS in IPA ramdisk without Ironic assist - 1 patch set17:05
JayFwould save me from having to backport that patch downstream :D17:05
*** k_mouza has quit IRC17:09
*** iurygregory has quit IRC17:10
*** k_mouza has joined #openstack-ironic17:14
*** k_mouza has quit IRC17:19
*** uzumaki has quit IRC17:26
*** k_mouza has joined #openstack-ironic17:26
*** uzumaki has joined #openstack-ironic17:26
*** k_mouza has quit IRC17:30
*** k_mouza has joined #openstack-ironic17:34
*** gyee has joined #openstack-ironic17:35
*** uzumaki has quit IRC17:38
*** k_mouza has quit IRC17:39
*** uzumaki has joined #openstack-ironic17:40
*** k_mouza has joined #openstack-ironic17:42
*** k_mouza has quit IRC17:47
*** k_mouza has joined #openstack-ironic17:50
*** uzumaki has quit IRC17:52
*** k_mouza has quit IRC17:55
openstackgerritBob Fournier proposed openstack/ironic master: Handle a Redfish error response when turning power off  https://review.opendev.org/75258217:57
*** jawad_axd has joined #openstack-ironic18:11
*** jawad_axd has quit IRC18:15
*** irclogbot_0 has quit IRC18:19
*** irclogbot_3 has joined #openstack-ironic18:23
TheJuliaI think janders is interested in supporting blkdiscard as well, but I think we'll want to do a quick review of the code and mechanics to make sure we don't do anything horribly. I thought of adding trim support a long time ago but it looked just as bad as just zeroing out  the device in terms of how long it could/would take :(18:24
*** irclogbot_3 has quit IRC18:31
*** Qianbiao has quit IRC18:31
*** jawad_axd has joined #openstack-ironic18:32
*** irclogbot_3 has joined #openstack-ironic18:35
*** jawad_axd has quit IRC18:36
*** k_mouza has joined #openstack-ironic18:51
*** jawad_axd has joined #openstack-ironic18:52
*** k_mouza has quit IRC18:56
*** jawad_axd has quit IRC18:56
*** uzumaki has joined #openstack-ironic19:02
*** k_mouza has joined #openstack-ironic19:21
*** k_mouza has quit IRC19:26
openstackgerritJulia Kreger proposed openstack/ironic master: Guard conductor from consuming all of the ram  https://review.opendev.org/72648319:29
openstackgerritMerged openstack/ironic-python-agent-builder master: Advertise HTTPS in IPA ramdisk without Ironic assist  https://review.opendev.org/75272819:43
openstackgerritMerged openstack/bifrost master: Fix internal_ip detection in {enroll,deploy}-dynamic  https://review.opendev.org/75241119:43
*** k_mouza has joined #openstack-ironic19:47
openstackgerritMerged openstack/ironic-python-agent-builder master: tinyipa: use python 3 for building on all systems  https://review.opendev.org/75176919:49
openstackgerritMerged openstack/ironic master: Clarify HPE Edgeline support  https://review.opendev.org/75188119:50
*** k_mouza has quit IRC19:51
openstackgerritMerged openstack/ironic master: Detail iPXE + LACP troubleshooting information  https://review.opendev.org/74378720:05
*** zzzeek has quit IRC20:06
*** uzumaki has quit IRC20:28
*** k_mouza has joined #openstack-ironic20:30
*** k_mouza has quit IRC20:34
*** zzzeek has joined #openstack-ironic20:42
*** zzzeek has quit IRC20:48
*** zzzeek has joined #openstack-ironic20:54
TheJuliarpioso: if you could trim down the release note on https://review.opendev.org/#/c/752614/1, it would be much appreciated. I think you have way too much detail in it which results in the reader quickly thinking this is a feature when you look at the code it is more of a bug fix.20:57
patchbotpatch 752614 - ironic - Fix redfish BIOS to use @Redfish.SettingsApplyTime - 1 patch set20:57
*** zzzeek has quit IRC20:59
*** ociuhandu has joined #openstack-ironic21:02
*** zzzeek has joined #openstack-ironic21:03
TheJuliarpioso: also, https://review.opendev.org/#/c/749240's release note needs a little massaging, mostly splitting it down since the feature is one piece of information, the what is functionally a formal declaration of support with information about interface ordering and priorities is secondary to the key fact of the feature (and the audience reading trying to grok the important details).21:04
patchbotpatch 749240 - ironic - Add Redfish BIOS interface to idrac HW type - 8 patch sets21:04
*** zzzeek has quit IRC21:05
*** zzzeek has joined #openstack-ironic21:06
*** ociuhandu has quit IRC21:06
rpiosoTheJulia: I'll work on addressing your feedback on https://review.opendev.org/#/c/752614/ now.21:07
patchbotpatch 752614 - ironic - Fix redfish BIOS to use @Redfish.SettingsApplyTime - 1 patch set21:07
JayFTheJulia: I think someone said something about an ipa-builder release for TLS support? If so, you likely want https://review.opendev.org/752728 in that or as a patch release21:09
patchbotpatch 752728 - ironic-python-agent-builder - Advertise HTTPS in IPA ramdisk without Ironic assist (MERGED) - 1 patch set21:09
JayFTheJulia: advise me if I need to backport it, and I will21:09
TheJuliaJayF: I did not on the highlights21:10
rpiosoRegarding https://review.opendev.org/#/c/749240's release note, it is modeled after this one: https://review.opendev.org/#/c/681745/3/releasenotes/notes/idrac-add-redfish-inspect-support-ce74bd3d4a97b588.yaml21:10
patchbotpatch 749240 - ironic - Add Redfish BIOS interface to idrac HW type - 8 patch sets21:10
patchbotpatch 681745 - ironic - Add Redfish inspect interface to idrac HW type (MERGED) - 3 patch sets21:10
TheJuliait is just latest release JayF21:11
JayFTheJulia: ack, do you want that backported then or not? I consume ipa-builder directly from git so I don't need it; but if folks are going to consume that support I'd rather them get the more ergonomic version21:11
TheJuliaJayF: there is no backport branch for ipab21:12
JayFack21:12
TheJuliarpioso: hmm, then disregard my second comment I guess21:12
*** zzzeek has quit IRC21:13
*** zzzeek has joined #openstack-ironic21:16
*** zzzeek has quit IRC21:23
*** zzzeek has joined #openstack-ironic21:24
*** rcernin has joined #openstack-ironic21:27
openstackgerritVerification of a change to openstack/sushy failed: Make message parsing more resilient  https://review.opendev.org/74892721:33
*** rcernin has quit IRC21:35
*** rcernin has joined #openstack-ironic21:36
*** Goneri has quit IRC21:39
*** rcernin has quit IRC21:42
rpiosoTheJulia: Which comment should I disregard, the one about the first paragraph or the one about the second?21:44
TheJuliacdearborn: hey! https://review.opendev.org/#/c/749619/17 looking fairly good but it is going to take a monday morning with very strong coffee to go through the management unit tests. One thing, I would <3 an update to the utils to explain the lambda usage since largely it seems bizzar to need it, I feel like I need to understand.21:49
patchbotpatch 749619 - ironic - Redfish driver firmware update - 17 patch sets21:49
TheJuliarpioso: Disregard the comment on the second paragraph21:50
TheJuliacdearborn: also, just noticed the patch for which that one is parent. I'm slightly confused by idrac-redfish having a separate patch, maybe the primary reno just needs to be muted or soemthing. Anyway, coffee monday.21:51
rpiosoTheJulia: +1 re: comment on the 2d paragraph. While I'm very clearly not an English professor (shucks, I placed out of my engineering program's English course requirement and never looked back), I believe the language in the first paragraph is grammatically correct. Just saying. That said, I'm interested in this merging, so ...21:53
openstackgerritMerged openstack/ironic master: Adds ilo-uefi-https boot interface to ilo5  https://review.opendev.org/74550121:53
TheJuliarpioso: it is more about style and information conveyance for the target audience21:54
TheJulialike looking at cdearborn's change that I just tagged him on, he is expressing what is being added first and that is the preferred style21:55
TheJuliaugh, this reminds me I need to write the prelude soon21:55
TheJuliagah!21:55
rpiosoTheJulia: Have we cut our final release of sushy for Victoria?21:56
rpioso3.4.0 back on September 7?21:56
TheJulia<narator style="movie intro">In a world of bare metal provisioning systems, one system...<...21:57
TheJuliarpioso: yes. Library release freeze21:57
cdearbornTheJulia: yeah, the idrac-redfish patch is kinda weird.  The code works fine without it.  The main reason for it is that the code that auto generates the cleaning step tables in the docs does not pick up inherited cleaning steps. I assumed this was intended to make the driver explicitly declare support for the cleaning step.21:57
cdearbornTheJulia: since there were a fair number of idrac specific changes, i broke it out in a separate patch21:58
TheJuliawe can bugfix it, but it won't really included in until after the press release and all goes out unless one of the release team folks approves it anyway21:58
TheJuliacdearborn: hmm... interesting21:59
rpiosoTheJulia: Gotcha :-)22:00
*** rcernin has joined #openstack-ironic22:00
TheJuliarpioso: if we really really really need to get the library updated to 3.4.1 with the fix, add it to the meeting agenda and if nobody screams during the meeting we'll need to do a post to the ml and see if anyone screams22:01
rpiosocdearborn: Please remind me, do we really, really need https://review.opendev.org/#/c/748927/ for firmware update?22:03
patchbotpatch 748927 - sushy - Make message parsing more resilient - 3 patch sets22:03
cdearbornrpioso - welllllllll - i would say no, at least not for Dell hardware.  i made those changes to get message parsing to work for Dell hardware, but, it turns out that Dell hardware returns already parsed messages in addition to the non-parsed ones.  And, it doesn't return any substitution parameters to substitute in.  The end result is that the pre-parsed messages are much better messages than the ones we can22:09
cdearbornbuild with the message parsing.  As a result, i made the code use the preparsed messages if they are returned by the BMC.  So in the end, we don't need that patch for Dell hardware22:09
TheJuliasounds like problem solved?22:10
*** rloo has quit IRC22:11
cdearbornTheJulia, thx for the review! I beefed up the release note based on feedback from Steve Baker.  Will  trim it down a bit22:13
TheJuliacdearborn: much appreciated. I tend to lean toward shorter notes just because of the large quantity of them22:13
rpiosocdearborn, TheJulia: Thank you! And, yes, problem solved for now :-)22:14
openstackgerritRichard G. Pioso proposed openstack/ironic master: Add Redfish BIOS interface to idrac HW type  https://review.opendev.org/74924022:15
rpiosoTheJulia: I modified the first paragraph in the release note to be more like cdearborn's.22:16
cdearbornhave a great weekend all22:19
*** cdearborn is now known as cdearborn|afk22:20
*** zzzeek has quit IRC22:25
*** zzzeek has joined #openstack-ironic22:26
*** zzzeek has quit IRC22:30
*** zzzeek has joined #openstack-ironic22:31
TheJuliaMuch appreciated! Have a wonderful weekend everyone!22:54
openstackgerritVerification of a change to openstack/ironic failed: Native zuulv3 grenade multinode multitenant  https://review.opendev.org/70503023:00
*** rcernin has quit IRC23:06
*** zzzeek has quit IRC23:08
*** patchbot has quit IRC23:10
*** zzzeek has joined #openstack-ironic23:11
rpiosoTheJulia: Thank you, and you, too!23:13
*** zzzeek has quit IRC23:16
*** zzzeek has joined #openstack-ironic23:17
*** patchbot has joined #openstack-ironic23:18
*** tosky has quit IRC23:31
*** rcernin has joined #openstack-ironic23:39
*** Goneri has joined #openstack-ironic23:44

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!