Tuesday, 2021-11-16

opendevreviewSteve Baker proposed openstack/ironic-python-agent stable/wallaby: Fix UEFI record regex  https://review.opendev.org/c/openstack/ironic-python-agent/+/81701401:40
opendevreviewSteve Baker proposed openstack/ironic-python-agent bugfix/8.1: Fix UEFI record regex  https://review.opendev.org/c/openstack/ironic-python-agent/+/81747001:46
opendevreviewSteve Baker proposed openstack/ironic-python-agent bugfix/8.1: Fix UEFI record regex  https://review.opendev.org/c/openstack/ironic-python-agent/+/81747001:53
opendevreviewSteve Baker proposed openstack/ironic-python-agent stable/victoria: Fix UEFI record regex  https://review.opendev.org/c/openstack/ironic-python-agent/+/81701801:59
opendevreviewSteve Baker proposed openstack/ironic-python-agent stable/ussuri: Fix UEFI record regex  https://review.opendev.org/c/openstack/ironic-python-agent/+/81702102:03
opendevreviewSteve Baker proposed openstack/ironic-python-agent stable/train: Fix UEFI record regex  https://review.opendev.org/c/openstack/ironic-python-agent/+/81702602:07
opendevreviewMerged openstack/ironic master: [iRMC] Convert the type of irmc_port to int  https://review.opendev.org/c/openstack/ironic/+/81715404:26
opendevreviewMerged openstack/ironic-python-agent stable/victoria: Output verbose info from efibootmgr  https://review.opendev.org/c/openstack/ironic-python-agent/+/81701606:05
opendevreviewMerged openstack/ironic-python-agent stable/victoria: Delete EFI boot entry duplicate labels first  https://review.opendev.org/c/openstack/ironic-python-agent/+/81701706:08
arne_wiebalckGood morning, Ironic!07:49
muellerbegood morning arne_wiebalck07:49
arne_wiebalckhey muellerbe o/07:53
iurygregorygood morning arne_wiebalck and Ironic o/07:55
arne_wiebalckhey iurygregory o/07:55
rpittaugood morning ironic! o/07:57
iurygregorymorning rpittau o/08:00
rpittauhey iurygregory :)08:01
arne_wiebalckhey rpittau o/08:01
arne_wiebalckiurygregory: dtantsur: from mraineri 's answer here https://redfishforum.com/thread/565/weak-etags, I would say sushy should not remove the qualifier (see also dtantsur 's intuition :)08:04
arne_wiebalckiurygregory: dtantsur: that leaves still two options: a) remove the stripping (may break timeu 's hardware) or b) use both tags in the response's header ... thoughts?08:06
iurygregoryarne_wiebalck, a can cause problems for Lenovo HW, b I'm ok but it seems weird sending both etags..., maybe option c? c - since ironic has logic to identify vendor we can probably add an option to insert_media in sushy https://github.com/openstack/sushy/blob/master/sushy/resources/manager/virtual_media.py#L110-L111 that would tell if we need to strip the W/ or not ....08:17
iurygregoryby default the variable wouldn't remove anything from the ETag, and in ironic we can add a logic for lenovo as a vendor to strip when using vmedia08:19
iurygregoryofc I'm not against option b =) (but I'm not sure how BMC's will handle in case we are sending two values... 08:20
arne_wiebalckiurygregory: sending two values is standard compliant08:28
iurygregoryoh \o/08:28
iurygregoryok cool :D08:28
arne_wiebalckiurygregory: and it I tested this with my hardware08:28
iurygregory<insert happy dance gif>08:29
arne_wiebalckiurygregory: it just looks like the big gun08:29
iurygregorykinda =)08:29
arne_wiebalckiurygregory: doing per vendor massaging of the header seems worse, though08:29
iurygregorylike in brazil we would say you are using a bazooka to kill an ant :D08:29
arne_wiebalckiurygregory: so, maybe b) is not such a bad option after all08:29
arne_wiebalckiurygregory: heh08:30
iurygregoryyeah, I was worried if some BMCs will complain...08:30
arne_wiebalckiurygregory: let's see what dtantsur thinks once he comes online08:30
iurygregorybecause... standards :D08:30
rpittau"standards"08:30
arne_wiebalckiurygregory: oh, totally possible this will create new issues08:30
iurygregoryrpittau, yeah I forgot the " " :D08:31
rpittau:)08:31
iurygregoryarne_wiebalck, yup =)08:31
dtantsurarne_wiebalck: well, what mraineri suggests contradicts the ETag RFC08:47
dtantsurI'm fine with sending both, but I'm worried about the message they are sending08:48
arne_wiebalckdtantsur: yes08:48
arne_wiebalckdtantsur: I agree08:48
arne_wiebalck"weak Etags on If-* should never match"08:48
* dtantsur answers there08:50
arne_wiebalck+1, I was about to ask you :)08:50
dtantsurdone08:54
opendevreviewDmitry Tantsur proposed openstack/bifrost master: [WIP] iPXE fallback script  https://review.opendev.org/c/openstack/bifrost/+/81800008:58
arne_wiebalckdtantsur: thanks10:01
arne_wiebalckdtantsur: I suggest none the less to put up a patch for sushy to send back both etags (even if temporary): while weak etags should not be used, we seem to have two cases already where this is the case and we have two different behaviours :-S10:03
dtantsurheh, yeah10:04
dtantsuromg, why is bifrost job succeeding on failure? O__o10:04
arne_wiebalckError: SUCCESS10:04
iurygregoryWOOT?!10:05
dtantsurI think it's a regression in https://opendev.org/openstack/bifrost/commit/b31bc66726ddcff11678a77cce8b14f629a1b19d10:05
iurygregoryI'm trying to recall the reason why TheJulia needed this10:07
dtantsurI think it's kinda wrong, EXIT should catch it10:07
dtantsurI'll propose a revert (which may fail because tinyIPA fails with the DNS problem)10:07
iurygregoryouch =(10:07
opendevreviewDmitry Tantsur proposed openstack/bifrost master: Revert "Explicitly trap on ERR"  https://review.opendev.org/c/openstack/bifrost/+/81797810:09
dtantsurTheJulia: ^^^10:09
dtantsurwe have tinyIPA basically broken because of eventlet: https://zuul.opendev.org/t/openstack/build/a2c04cad4f054f5a958148a91d4f03ed/log/logs/testvm1_console.log#145810:10
rpittau\o/10:12
iurygregoryofc eventlet =(10:15
* dtantsur opens eventlet source10:22
dtantsurhttps://github.com/eventlet/eventlet/pull/73710:39
dtantsurnow I need to do groceries shopping and then figure out how to test it10:40
dtantsurcc janders 10:40
jandersdtantsur NICE! :)12:25
jandersif anyone has time, I would really appreciate reviews on https://review.opendev.org/c/openstack/sushy/+/81713712:25
jandersw/r/t eventlet CI issue this is exactly what I ran into the other day12:27
janders(just didn't get far looking into eventlet code)12:27
iurygregoryjanders, feel free to add ironic-week-prio as a hashtag in your patch so it will be in the prio list for review =)12:28
jandersiurygregory thank you, done!12:30
janderssee you tomorrow Ironic o/12:31
iurygregorybye janders o/ (see you Thursday :D )12:32
TheJuliadtantsur: I think without catching it was failing and not being caught at all13:28
TheJuliadtantsur: I *think*13:28
TheJuliadtantsur: stevebaker[m] was fighting it when he spotted it, I just happened to whip up a patch first13:28
* TheJulia wonders if the McDonalds is open13:29
iurygregoryTheJulia, fyi regarding the tinyipa problem in ussuri/train, maybe we can try using --insecure? dtantsur proposed this in the meeting yesterday =)13:33
TheJuliaiurygregory: insecure is really unacceptable imho13:33
TheJuliabecause MitM attacks could occur on code in our repositories13:34
iurygregoryoh ok =)13:34
TheJuliaAlso, can't patch old IPA-builder 13:34
iurygregoryoh right =( we didn't have ipa-b with branches at that time...13:35
TheJuliaand infra flat out told us "absolutely not" when we asked if we could go back and branch to do a 2.2.x fix to address some of the issues we were having a while back13:35
TheJuliayeah13:35
TheJuliawell, I think it was more the openstack release team13:35
TheJuliathan infra13:35
TheJuliaI bet infra would have just done it13:35
iurygregoryprobably...13:35
TheJuliaAnyway, I need coffee and to eat13:35
TheJuliaI got 5 hours of sleep \o/13:35
TheJuliaAmongst trucks13:35
iurygregorycoffee and food ++13:35
iurygregoryyeah I saw the email and twitter =(13:36
iurygregorygood luck!13:36
TheJuliaiurygregory: Yeah, I guess I'm getting all my broken down in some strange city credits for life13:47
dtantsurTheJulia: IIRC the tinyIPA build is already insecure13:49
dtantsurgood morning13:49
dtantsurhence my suggestion13:49
TheJuliadtantsur: are we explicitly disabling cert verification anywhere else?13:49
dtantsurI *think* (double-checking now) TCL repos don't use TLS13:50
TheJuliaI guess the conundrum beyond that is we would have to hack 10.x and 12.x support in side by side into master branch13:50
dtantsurTheJulia: https://opendev.org/openstack/ironic-python-agent-builder/src/branch/master/tinyipa/tc-mirror.sh#L1213:50
TheJuliawoot13:51
dtantsurthat's one of the reasons we don't recommend tinyIPA in production13:51
TheJuliaI always viewed it as drivers, but that is definitely another solid reason13:52
Alex-devHi! I'm trying to deploy OS (centos stream) via ironic (tripleo setup Victoria) and got error:13:59
Alex-devFailed to provision instance 2318a720-783c-41bc-8842-cc793a7cf389: Deploy step deploy.write_image failed on node 95442d75-44f8-4783-ba24-783d1d008eae. Error performing deploy_step write_image: Command execution failed: Unexpected error while running command.\nCommand: blkid /dev/sda --match-tag UUID --match-tag PARTUUID\nExit code: 213:59
rpittauAlex-dev: that's a known error13:59
rpittauthis should fix the issue https://review.opendev.org/c/openstack/ironic-lib/+/81792214:00
Alex-devThanks! I will try14:00
rpittauoh wait that's centos stream though, so should be compatible with the long variable names14:01
rpittauAlex-dev can you post the full log somewhere ?14:01
rpittauAlex-dev: just to confirm, are you using a centos stream based ipa ramdisk ?14:01
Alex-devI seen in ipmi console centos stream. in which log file I can find info about it? ironic, ironic-conductor, nova?14:03
rpittauAlex-dev: the ironic-python-agent ramdisk logs, you can see how to get them here: https://docs.openstack.org/ironic/latest/admin/troubleshooting.html#retrieving-logs-from-the-deploy-ramdisk14:05
rpittauAlex-dev: that looks like the same error from centos 7, but since you mentioned centos stream it may be something different. At least see if you can post the conductor logs somewhere, as a start14:09
Alex-devconductor logs: https://drive.google.com/file/d/1V1rMD7bGwkK13wira8gNTH403e5B9UZi/view?usp=sharing14:15
TheJuliaI *think* another diesel broke down in the parking lot14:15
Alex-dev@rpittau ^14:15
* TheJulia hears telltale signs of air break problems14:15
TheJuliabrake14:16
rpittauAlex-dev: I don't see any errors there, where did you get the original error from ?14:18
* TheJulia attempts to patiently wait for the mobile mechanic14:21
Alex-devrpittau: in nova-conductor and nova-compute logs, I will share it now14:25
TheJuliaso it is exposing last_error14:27
TheJuliaand that gets reset if cleaning is enabled and it rolls back through successfully14:27
* TheJulia has this sneaking suspicion14:28
Alex-devrpittau: https://drive.google.com/drive/folders/1pZn04cVcJskD3JpgXJ5PQnS5kMg5TUUr?usp=sharing14:28
rpittauAlex-dev: ok, I can see the error there, it's taken from ironic conductor but I can't see the same in the ironic conductor logs, they seem cut14:32
rpittauwould be great to have the full ironic conductor logs plus understand if the deploy image is using centos 714:32
Alex-devWhere I can get actual builded ipa images for update it ? this error expected only on centos 7? I want try again on latest images14:37
TheJuliahttps://tarballs.opendev.org/openstack/ironic-python-agent/dib/14:38
TheJuliaAlex-dev_: ^^14:40
TheJulianote the dates14:40
TheJuliaspecifically we had to stop building centos7 a long time ago because we dropped python 2.x support14:40
rpittauAlex-dev: that should happen only on centos 7, yes14:40
* TheJulia looses her mind14:41
Alex-dev_ok, I will use ipa-centos8-stable-victoria.tar.gz. tryed and get back14:41
TheJuliaokay14:42
* TheJulia resumes waiting for the mechanic14:42
* TheJulia is getting super antsy 14:42
* TheJulia wants to get home... badly14:42
TheJuliaAnyone taking odds on if I'll get home today?14:44
rpittauTheJulia: I truly hope you get home today :)14:45
TheJuliaI sure hope so. :(14:57
TheJuliaI can't eat any more McDonalds14:57
TheJuliaOhhh, mechanic has arrived14:57
iurygregory\o/ yay15:08
* TheJulia hopes by lunch time to be on the road15:20
* TheJulia hopes15:20
* TheJulia has been hoping a lot15:20
TheJuliaIf y'all haven't figured it out, your like my only human contact right now15:23
* dtantsur is an owl contact15:28
dtantsurTheJulia: fingers crossed that it's resolved this time!15:28
iurygregoryowl contact :D 15:29
TheJuliaOwls are awesome15:31
dtantsur++ can confirm (or cat confirm?)15:31
TheJuliaConfirm... at the moment. I think15:32
TheJuliaKittah overlord was loaded into the car and taken home today.15:33
dtantsurgood to be a kitty!15:35
*** dviroel is now known as dviroel_15:37
TheJuliawoot, two more bolts to come off, then four to go back on15:41
TheJuliaAnd then resume the belt replacement15:42
opendevreviewArne Wiebalck proposed openstack/sushy master: Handle weak Etags  https://review.opendev.org/c/openstack/sushy/+/81811015:45
opendevreviewArne Wiebalck proposed openstack/sushy master: Use Etag in precondition header when setting boot options  https://review.opendev.org/c/openstack/sushy/+/81811415:47
arne_wiebalckdtantsur: iurygregory ^^ returning both etag flavors and using them for patch (boot options only)15:49
iurygregoryarne_wiebalck, ack =D15:49
opendevreviewMerged openstack/ironic-lib master: Restore blkid compatibility with Centos 7  https://review.opendev.org/c/openstack/ironic-lib/+/81792215:50
opendevreviewRiccardo Pittau proposed openstack/ironic-lib stable/xena: Restore blkid compatibility with Centos 7  https://review.opendev.org/c/openstack/ironic-lib/+/81798315:54
opendevreviewRiccardo Pittau proposed openstack/ironic-lib stable/wallaby: Restore blkid compatibility with Centos 7  https://review.opendev.org/c/openstack/ironic-lib/+/81798415:55
opendevreviewDmitry Tantsur proposed openstack/ironic-python-agent master: Move manage_uefi from the image extension to a public location  https://review.opendev.org/c/openstack/ironic-python-agent/+/81565116:58
dtantsurarne_wiebalck: could you check that I've applied your changes correctly ^^^?16:58
arne_wiebalckdtantsur: I will!17:00
arne_wiebalckdtantsur: etags, so my vendor's implementation is correct when it comes to the redfish standard ... I just wonder why it took me so long to realise what to send back ;)17:18
*** sshnaidm is now known as sshnaidm|afk17:33
dtantsursigh17:33
arne_wiebalcktake two standards, combine them with some implementation creativity and you end up with a nice explosive result :)17:42
arne_wiebalckbye everyone o/17:44
dtantsuryeah..17:45
dtantsurI'm going as well, see you17:45
*** tosky is now known as Guest605418:05
*** tosky_ is now known as tosky18:05
*** sshnaidm|afk is now known as sshnaidm18:39
rpittaugood night! o/18:45
*** tosky_ is now known as tosky18:48
stevebaker[m]morning19:32
stevebaker[m]TheJulia: I see you're still having A Time19:32
stevebaker[m]New baremetal SIG video is live! https://www.youtube.com/watch?v=wy7T8WuYDMY22:11
stevebaker[m]Once again I had to find a new video editor when the current one broke after a fedora upgrade22:11
stevebaker[m]So far its been openshot -> flowblade -> kdenlive22:12
stevebaker[m]anyway, arne_wiebalck ^^22:12
*** tosky is now known as Guest607022:42
*** tosky_ is now known as tosky22:42
*** tosky is now known as Guest607323:07
*** tosky_ is now known as tosky23:07

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!