Thursday, 2021-09-16

opendevreviewSteve Baker proposed openstack/ironic master: Add missing mode setting on pxe created directories  https://review.opendev.org/c/openstack/ironic/+/80929401:54
opendevreviewSteve Baker proposed openstack/ironic master: WIP chmod fetched pxe kernel/ramdisk files  https://review.opendev.org/c/openstack/ironic/+/80929501:54
*** pmannidi|AFK is now known as pmannidi02:09
*** pmannidi is now known as pmannidi|brb05:57
iurygregorygood morning Ironic06:03
arne_wiebalckGood morning iurygregory and Ironic!07:02
iurygregorymorning arne_wiebalck o/07:03
*** pmannidi|brb is now known as pmannidi07:22
opendevreviewMerged openstack/python-ironicclient master: Fix distribution compatability for configdrive build  https://review.opendev.org/c/openstack/python-ironicclient/+/80925807:41
ajyaGood morning, if there is any 2nd core reviewer who could take a look at these: https://review.opendev.org/c/openstack/ironic/+/804112 https://review.opendev.org/c/openstack/ironic/+/804848 07:57
opendevreviewMerged openstack/ironic stable/ussuri: Fix Redfish jobs by pinning sushy-tools  https://review.opendev.org/c/openstack/ironic/+/80927508:00
*** pmannidi is now known as pmannidi|AFK08:10
opendevreviewZhouHao proposed openstack/ironic master: [iRMC] Avoid repeatedly resuming clean after creating raid configuration  https://review.opendev.org/c/openstack/ironic/+/80902210:39
opendevreviewZhouHao proposed openstack/ironic master: [iRMC] Set polling after RAID is built  https://review.opendev.org/c/openstack/ironic/+/80902310:39
opendevreviewMerged openstack/ironic master: Fix RAID steps for non-RAID in idrac-redfish  https://review.opendev.org/c/openstack/ironic/+/80411211:31
opendevreviewMerged openstack/ironic master: Fix clear foreign config in idrac-redfish RAID  https://review.opendev.org/c/openstack/ironic/+/80484812:27
opendevreviewVerification of a change to openstack/ironic-python-agent-builder master failed: Support non-genisoimage binaries  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/80929912:47
opendevreviewJulia Kreger proposed openstack/ironic master: Attempt to reduce odds of neutron port startup failure  https://review.opendev.org/c/openstack/ironic/+/80941513:39
TheJuliagood morning13:39
TheJuliaso I am hoping ^^^ might reduce some of our erronious failures in the gate. I've seen a few instances where our jobs have failed on the neutron port setup recently and neutron is kind of known not to be the fastest in that area13:40
iurygregorygood morning TheJulia 13:53
iurygregoryTheJulia, do we think we can just go ahead with ngs and networking-baremetal release? (for ipa-builder I think we can wait for https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/809299 )13:56
TheJuliaiurygregory: can you take a look at https://review.opendev.org/c/openstack/ironic/+/809145 <-- stable backport just need an extra approval13:58
iurygregorysure looking now13:59
TheJuliaiurygregory: yeah, ngs and networking-baremetal sound reasonable13:59
TheJulialooks like we need another core review on https://review.opendev.org/c/openstack/ironic/+/806859 that is not Iury14:02
TheJuliaiurygregory: I'm going to workflow https://review.opendev.org/c/openstack/ironic/+/80928814:06
TheJuliaif there are no objections14:06
iurygregoryTheJulia, I can +W https://review.opendev.org/c/openstack/ironic/+/809288 (since is just a fix for our CI)14:07
TheJuliaiurygregory: works for me14:07
iurygregoryI just saw your comment and it makes sense14:07
TheJuliayeouch https://2d9a473fba18604e747c-da887b185713bfa3a6995c0b44f5e463.ssl.cf2.rackcdn.com/803855/10/check/ironic-tempest-ipa-partition-pxe_ipmitool/1de197f/controller/logs/syslog.txt <-- indcates we hit 1.1MB free in one job14:08
TheJuliaI guess it took out mysqld :\14:10
iurygregory=(14:17
opendevreviewJulia Kreger proposed openstack/ironic stable/train: Fix Redfish jobs by pinning sushy-tools  https://review.opendev.org/c/openstack/ironic/+/80928514:17
TheJuliaugh, had to lower the mysql version even further14:17
TheJuliahmm, multiple hits on the neutron tap not being up yet14:23
TheJuliaoh! I see what is going on with neutron14:29
TheJuliaIt is crashing14:29
iurygregoryneutron stop crashing, we will give you cookies :D14:30
TheJuliahttps://paste.opendev.org/show/809374/14:36
TheJuliait is really funky14:36
iurygregoryoh god =(14:38
TheJuliaThe agent code hasn't changed in months14:47
TheJuliaerr, wrong agent14:49
TheJulialooks like it has changed some recently14:50
opendevreviewIury Gregory Melo Ferreira proposed openstack/ironic bugfix/18.1: Fix driver task pattern to reduce periodic db load  https://review.opendev.org/c/openstack/ironic/+/80937615:24
opendevreviewJulia Kreger proposed openstack/ironic master: CI: set Neutron OVS of_inactivity_probe for CI load  https://review.opendev.org/c/openstack/ironic/+/80942615:30
opendevreviewMerged openstack/ironic stable/ussuri: Use shim-signed on Ubuntu, shim is empty now  https://review.opendev.org/c/openstack/ironic/+/80914515:43
opendevreviewJulia Kreger proposed openstack/ironic master: CI: set Neutron OVS of_inactivity_probe for CI load  https://review.opendev.org/c/openstack/ironic/+/80942615:55
TheJuliaiurygregory: ^^ now with bashate fix :(15:58
opendevreviewMerged openstack/ironic master: Dial back gate job memory allocation  https://review.opendev.org/c/openstack/ironic/+/80928816:08
opendevreviewJulia Kreger proposed openstack/ironic master: CI: set Neutron OVS of_inactivity_probe for CI load  https://review.opendev.org/c/openstack/ironic/+/80942616:38
opendevreviewMerged openstack/ironic-python-agent-builder master: Support non-genisoimage binaries  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/80929916:39
opendevreviewMerged openstack/ironic stable/wallaby: Fix driver task pattern to reduce periodic db load  https://review.opendev.org/c/openstack/ironic/+/80898316:51
iurygregoryTheJulia, ack I will keep an eye on it16:59
opendevreviewJulia Kreger proposed openstack/ironic master: CI: set Neutron OVS of_inactivity_probe for CI load  https://review.opendev.org/c/openstack/ironic/+/80942617:33
opendevreviewMerged openstack/ironic master: Expand the driver contributor documentation  https://review.opendev.org/c/openstack/ironic/+/80728418:09
opendevreviewMerged openstack/ironic master: Remove images from the OS profiler docs  https://review.opendev.org/c/openstack/ironic/+/80732618:09
opendevreviewJulia Kreger proposed openstack/python-ironicclient stable/xena: Fix distribution compatability for configdrive build  https://review.opendev.org/c/openstack/python-ironicclient/+/80938018:11
opendevreviewJulia Kreger proposed openstack/python-ironicclient stable/wallaby: Fix distribution compatability for configdrive build  https://review.opendev.org/c/openstack/python-ironicclient/+/80938118:12
opendevreviewVerification of a change to openstack/ironic bugfix/18.1 failed: Fix driver task pattern to reduce periodic db load  https://review.opendev.org/c/openstack/ironic/+/80937618:28
opendevreviewJulia Kreger proposed openstack/ironic master: CI: set Neutron OVS of_inactivity_probe for CI load  https://review.opendev.org/c/openstack/ironic/+/80942618:35
bfournieHi TheJulia, so for https://review.opendev.org/c/openstack/ironic/+/802389, we haven't found this particular patch to be helpful in that it didn't resolve the issue of the boot order changing after reboot. We recently isolated that to a particular (deprecated) hardware model for that vendor, the new h/w did not have the issue. I'll update the story. I can iterate on the patch if you think it would be useful in other 18:37
TheJuliabfournie: sounds like in that case, the patch should just be abandoned18:38
bfournieTheJulia: ok, will do. Sorry you spent time reviewing it, I should have updated the story last week, will do it now18:40
opendevreviewJulia Kreger proposed openstack/ironic stable/train: Fix Redfish jobs by pinning sushy-tools  https://review.opendev.org/c/openstack/ironic/+/80928518:40
TheJuliabfournie: no worries!18:41
TheJuliaIt *does* raise some legitimate questions, fwiw18:41
TheJuliaand a path we likely ought to explore18:41
bfournieTheJulia: yeah, but we haven't really seen an reproducible issue that setting the device first helps 18:42
TheJuliaI feel like we're starting to encounter increased vendor specific "weirdness"18:44
bfournieTheJulia: yep18:50
opendevreviewMerged openstack/bifrost master: Use TLS for virtual media when TLS is enabled  https://review.opendev.org/c/openstack/bifrost/+/80587419:06
TheJuliastevebaker: hmmmm https://zuul.opendev.org/t/openstack/build/cf1c28a9e4b14bccbd00197c1fa74113 <-- shim missing19:59
opendevreviewJulia Kreger proposed openstack/ironic stable/train: Use shim-signed on Ubuntu, shim is empty now  https://review.opendev.org/c/openstack/ironic/+/80914620:11
stevebakerTheJulia: is that what ^^ fixes?20:24
TheJuliaahh, right20:25
TheJuliachasing a few too many unhappy jobs20:25
stevebakerI'll land them both as soon as they're green20:31
TheJuliaok, another is https://review.opendev.org/c/openstack/ironic/+/809426 which I just rechecked20:34
TheJuliatl;dr this may be our "our port never appeared" issue in ironic20:35
TheJuliaI've chased down 4-5 separate failures in the past 24 hours which were all basically that issue and the tl;dr is neutron-openvswitch-agent exits with a RuntimeError20:36
TheJuliawhich causes it never to appear for the dhcp agent in ovs, which *boom*20:36
stevebakerTheJulia: I have another tricky thing which might need your help. Rabi has a change to switch ironic and neutron to using basic auth on the undercloud. featureset001 was green on that change until my change to switch to uefi boot landed. Nodes deploy but ssh times out. https://review.opendev.org/c/openstack/tripleo-heat-templates/+/794912 https://review.rdoproject.org/zuul/build/2bed88354c8c4f18baeb0e8a90aa7eef20:46
TheJuliastevebaker: would it be okay to look tomorrow?20:49
stevebakerTheJulia: absolutely20:49
TheJuliastevebaker: okay, I'm shifting my brain into some srbac stuffs and then heading to the store.... I ordered a compressed air dryer attachment with the wrong fitting size, so I need to get an adapter before I can cut some steel tomorrow night.20:51
stevebakerbadass20:51
TheJuliaI think at some point, I'm going to include presentation photos of "don't be like julia"20:52
TheJulia"don't weld in shorts"20:52
stevebakerhaha!20:55
opendevreviewSamuel Zuk proposed openstack/ironic-specs master: Power control intermediary spec proposal  https://review.opendev.org/c/openstack/ironic-specs/+/80880421:18
TheJuliastevebaker: you might want to take a look at https://review.opendev.org/c/openstack/tripleo-heat-templates/+/78157121:35
opendevreviewMerged openstack/ironic stable/train: Fix Redfish jobs by pinning sushy-tools  https://review.opendev.org/c/openstack/ironic/+/80928523:21
opendevreviewMerged openstack/ironic stable/train: Use shim-signed on Ubuntu, shim is empty now  https://review.opendev.org/c/openstack/ironic/+/80914623:21
*** pmannidi|AFK is now known as pmannidi23:47

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!