Thursday, 2022-09-08

opendevreviewIury Gregory Melo Ferreira proposed openstack/ironic-ui master: DNM - Testing CI before Zed release  https://review.opendev.org/c/openstack/ironic-ui/+/85635301:33
opendevreviewIury Gregory Melo Ferreira proposed openstack/ironic-python-agent-builder master: DNM - Testing CI before Zed release  https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/85635401:33
opendevreviewIury Gregory Melo Ferreira proposed openstack/ironic-prometheus-exporter master: DNM - Testing CI before Zed release  https://review.opendev.org/c/openstack/ironic-prometheus-exporter/+/85635501:33
opendevreviewIury Gregory Melo Ferreira proposed openstack/networking-generic-switch master: DNM - Testing CI before Zed release  https://review.opendev.org/c/openstack/networking-generic-switch/+/85635601:33
opendevreviewIury Gregory Melo Ferreira proposed openstack/networking-baremetal master: DNM - Testing CI before Zed release  https://review.opendev.org/c/openstack/networking-baremetal/+/85635701:33
opendevreviewIury Gregory Melo Ferreira proposed openstack/bifrost master: DNM - Testing CI before Zed release  https://review.opendev.org/c/openstack/bifrost/+/85635801:35
opendevreviewOpenStack Release Bot proposed openstack/ironic-lib stable/zed: Update .gitreview for stable/zed  https://review.opendev.org/c/openstack/ironic-lib/+/85641207:52
opendevreviewOpenStack Release Bot proposed openstack/ironic-lib stable/zed: Update TOX_CONSTRAINTS_FILE for stable/zed  https://review.opendev.org/c/openstack/ironic-lib/+/85641307:52
opendevreviewOpenStack Release Bot proposed openstack/ironic-lib master: Add Python3 antelope unit tests  https://review.opendev.org/c/openstack/ironic-lib/+/85641407:52
opendevreviewOpenStack Release Bot proposed openstack/metalsmith stable/zed: Update .gitreview for stable/zed  https://review.opendev.org/c/openstack/metalsmith/+/85641507:52
opendevreviewOpenStack Release Bot proposed openstack/metalsmith stable/zed: Update TOX_CONSTRAINTS_FILE for stable/zed  https://review.opendev.org/c/openstack/metalsmith/+/85641607:52
opendevreviewOpenStack Release Bot proposed openstack/metalsmith master: Update master for stable/zed  https://review.opendev.org/c/openstack/metalsmith/+/85641707:52
opendevreviewOpenStack Release Bot proposed openstack/metalsmith master: Add Python3 antelope unit tests  https://review.opendev.org/c/openstack/metalsmith/+/85641807:52
opendevreviewOpenStack Release Bot proposed openstack/python-ironic-inspector-client stable/zed: Update .gitreview for stable/zed  https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/85641907:52
opendevreviewOpenStack Release Bot proposed openstack/python-ironic-inspector-client stable/zed: Update TOX_CONSTRAINTS_FILE for stable/zed  https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/85642007:52
opendevreviewOpenStack Release Bot proposed openstack/python-ironic-inspector-client master: Update master for stable/zed  https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/85642107:52
opendevreviewOpenStack Release Bot proposed openstack/python-ironic-inspector-client master: Add Python3 antelope unit tests  https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/85642207:52
opendevreviewOpenStack Release Bot proposed openstack/python-ironicclient stable/zed: Update .gitreview for stable/zed  https://review.opendev.org/c/openstack/python-ironicclient/+/85642307:52
opendevreviewOpenStack Release Bot proposed openstack/python-ironicclient stable/zed: Update TOX_CONSTRAINTS_FILE for stable/zed  https://review.opendev.org/c/openstack/python-ironicclient/+/85642407:52
opendevreviewOpenStack Release Bot proposed openstack/python-ironicclient master: Update master for stable/zed  https://review.opendev.org/c/openstack/python-ironicclient/+/85642507:52
opendevreviewOpenStack Release Bot proposed openstack/python-ironicclient master: Add Python3 antelope unit tests  https://review.opendev.org/c/openstack/python-ironicclient/+/85642607:52
opendevreviewOpenStack Release Bot proposed openstack/sushy stable/zed: Update .gitreview for stable/zed  https://review.opendev.org/c/openstack/sushy/+/85642707:53
opendevreviewOpenStack Release Bot proposed openstack/sushy stable/zed: Update TOX_CONSTRAINTS_FILE for stable/zed  https://review.opendev.org/c/openstack/sushy/+/85642807:53
opendevreviewOpenStack Release Bot proposed openstack/sushy master: Update master for stable/zed  https://review.opendev.org/c/openstack/sushy/+/85642907:53
opendevreviewOpenStack Release Bot proposed openstack/sushy master: Add Python3 antelope unit tests  https://review.opendev.org/c/openstack/sushy/+/85643007:53
adam-rozmanGood day Ironic!08:02
opendevreviewHayato Morita proposed openstack/ironic master: Add warning to notify requirements for vmedia boot  https://review.opendev.org/c/openstack/ironic/+/85473508:12
opendevreviewStephen Finucane proposed openstack/ironic master: tests: Add a WarningsFixture  https://review.opendev.org/c/openstack/ironic/+/85634510:00
opendevreviewStephen Finucane proposed openstack/ironic master: db: Resolve SAWarning warnings  https://review.opendev.org/c/openstack/ironic/+/85634910:00
opendevreviewStephen Finucane proposed openstack/ironic master: tests: Enable SQLAlchemy 2.0 deprecation warnings  https://review.opendev.org/c/openstack/ironic/+/85634610:00
opendevreviewStephen Finucane proposed openstack/ironic master: tests: Replace invalid UUIDs  https://review.opendev.org/c/openstack/ironic/+/85634710:00
opendevreviewStephen Finucane proposed openstack/ironic master: tests: Comment out prints  https://review.opendev.org/c/openstack/ironic/+/85634810:00
dtantsurTheJulia, iurygregory, JayF, I'm pondering automating getting event logs from BMCs if possible10:11
dtantsurmaybe akin to our ramdisk logs collection10:11
dtantsura lot of issues with virtual media could be made much more debuggable10:11
iurygregorygood morning ironic11:09
iurygregorydtantsur, that makes a lot of sense I would say11:09
dtantsurmraineri: hi! before I dive too deep into specs and forums, do you happen to remember if Redfish exposes the BMC event log? The thing that contains, among other things, reasons for failed virtual media connection?11:21
dtantsurfg11:30
dtantsurdamn11:30
opendevreviewDmitry Tantsur proposed openstack/ironic-inspector master: Fix compatibility with oslo.db 12.1.0  https://review.opendev.org/c/openstack/ironic-inspector/+/85648811:43
dtantsurthis does not fix the problem for me locally :(11:44
dtantsurideas welcome11:44
opendevreviewMerged openstack/sushy stable/zed: Update .gitreview for stable/zed  https://review.opendev.org/c/openstack/sushy/+/85642711:56
opendevreviewMerged openstack/sushy stable/zed: Update TOX_CONSTRAINTS_FILE for stable/zed  https://review.opendev.org/c/openstack/sushy/+/85642811:56
opendevreviewMerged openstack/ironic-lib stable/zed: Update .gitreview for stable/zed  https://review.opendev.org/c/openstack/ironic-lib/+/85641212:02
opendevreviewMerged openstack/ironic-lib stable/zed: Update TOX_CONSTRAINTS_FILE for stable/zed  https://review.opendev.org/c/openstack/ironic-lib/+/85641312:02
opendevreviewMerged openstack/sushy master: Update master for stable/zed  https://review.opendev.org/c/openstack/sushy/+/85642912:13
iurygregorygoing to test locally12:36
ajyaHi iurygregory , if I remember correctly, few cycles ago you were looking at tempest+bifrost. What's the status there?12:49
iurygregoryajya, due to downstream priorities I wasn't able to move forward on that .-.12:52
ajyaiurygregory: what are the thing necessary for this/why this combo isn't working?12:53
iurygregoryajya, I wasn't able to full test, my wip was just installing tempest+ironic plugin and trying to generate the necessary config (I didn't attempt to run tests at that time)12:54
ajyaiurygregory: ok, thanks12:55
opendevreviewMerged openstack/sushy master: Add Python3 antelope unit tests  https://review.opendev.org/c/openstack/sushy/+/85643013:03
smachiHello ironic13:05
smachiTheJulia I have added necessary comments on below path can you please have a look into it13:06
smachihttps://review.opendev.org/c/openstack/ironic-tempest-plugin/+/84870913:06
mraineridtantsur: Yes, you can get log information from a BMC via the LogService resource beneath a Manager resource; however, I don't know if you can reliably find events specific to virtual media connection failures13:09
mraineriIt might depend on the specific implementation, but I can see some vendors may not want to put those types of events in their logs since it can be very noisy in cases where the user's network is unstable13:10
dtantsurhmm, I see. worth experimenting with, I guess.13:16
dtantsurthanks!13:17
mraineriYou're welcome!13:17
opendevreviewMerged openstack/ironic-lib master: Add Python3 antelope unit tests  https://review.opendev.org/c/openstack/ironic-lib/+/85641413:20
opendevreviewMerged openstack/python-ironicclient master: Update master for stable/zed  https://review.opendev.org/c/openstack/python-ironicclient/+/85642513:20
TheJuliaGOOD MORNING13:32
opendevreviewMerged openstack/python-ironicclient master: Add Python3 antelope unit tests  https://review.opendev.org/c/openstack/python-ironicclient/+/85642613:36
opendevreviewMerged openstack/python-ironicclient stable/zed: Update .gitreview for stable/zed  https://review.opendev.org/c/openstack/python-ironicclient/+/85642313:36
opendevreviewMerged openstack/python-ironicclient stable/zed: Update TOX_CONSTRAINTS_FILE for stable/zed  https://review.opendev.org/c/openstack/python-ironicclient/+/85642413:36
TheJuliaugh, tired and air quality is kind of not great due to a big fire near by13:42
TheJuliaI'm not terribly a fan of just blocking out and ignoring SQLAlchemy warnings until later :(13:54
opendevreviewStephen Finucane proposed openstack/ironic-inspector master: Fix compatibility with oslo.db 12.1.0  https://review.opendev.org/c/openstack/ironic-inspector/+/85648814:04
kamlesh6808cHi Julia , Would you please help to second review https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/85362114:39
dtantsurgood mornign TheJulia 15:16
dtantsurthe inspector workaround is passing unit tests: https://review.opendev.org/c/openstack/ironic-inspector/+/85648815:16
TheJuliaapproved15:19
TheJuliaI'm banging my head against just making it 2.0 compatible...15:19
TheJuliamaking decent progress considering the changes, so far15:19
JayFdtantsur++ I'm on board with that, especially if logging mechanisms are more robust in Redfish than ipmi :D 15:20
* JayF has still never laid hands personally on a redfish machine15:20
TheJuliathey are kind of nice15:21
TheJuliathat is as long as you've queried the thing holding the information you need :)15:21
JayFyeah from observation it seems like (surprise!) inconsistencies between vendors are the most painful part?15:21
dtantsurby far15:23
dtantsurat least we're past things like cipher suite negotiation...15:23
* JayF remembers backporting a patch about cipher suites in sushy recently15:24
dtantsurmmmmmm? I think we had a discussion about hashing passwords, but it was on the Ironic side15:25
JayFhttps://review.opendev.org/c/openstack/sushy/+/855555 hey look, there it is for stable/train waiting on a review lol15:26
JayF> Do not send compression as acceptable encoding15:26
JayFmorning-brain mixed encoding and encryption 15:27
TheJuliathe sarcasm there is... exceptional.15:27
JayFHey, so I think I'm likely to start streaming some of my upstream work (e.g. on twitch)15:27
JayFdoes anyone have any specific interest? if so I'll take that into consideration when picking a time15:28
* JayF was going to start by doing bug triage that way15:28
* dtantsur feels like some of his work could go to onlyfans....15:28
mallikJayF: could you please take a look at the ilo patch https://review.opendev.org/c/openstack/ironic/+/85369015:28
JayFit's on my list now, but you'll need more than just me to land it15:29
mallikdtantsur: could you please take a look at the ilo patch https://review.opendev.org/c/openstack/ironic/+/85369015:29
* TheJulia hopes revised locking code does similar thing as before and prevents the random edge case we've seen sporatically in logs15:30
mallikJayF and dtantsur, I have addressed both of your comments in this patch15:30
JayFso > hpe-gen9-dsvm-ironic-standalone-wholedisk-uefi-ilo-virtual-media-direct-python315:31
JayFis just forever broken?15:31
mallikyes15:32
JayFWhy still run it / leave it voting? 15:32
TheJuliastevebaker did mention a... sushy/proliantutils incompatability he found the other day, I told him to email Nisha15:32
TheJulia\o/ only 6 unit tests fail now15:32
mallikthe 2 tests failing are due to infra is what our team triaged it and we are going to work on to resolve it. these 2 tests have been failing for sometime and nothing to do with the current patch. I will check with Nisha for the details.15:36
JayFIf there's an upstream breakage, and you put it in a story/task, you don't have to fix it alone :)15:36
JayFI have a vested interest in keeping ilo/proliantutils/sushy all operating happily too :D 15:36
mallikok 15:36
* TheJulia is still stumped why do_allocate gets called twice15:39
JayFTheJulia: my entire tomorrow is open, if you wanna toss some time on it to pair on any of this ... I'll spend more time today trying to catch up to you in problem understanding :D15:40
TheJuliaJayF: ack, I have a blood draw in the morning... down the mountain15:40
TheJuliaso I'll be back around noon I think15:40
* JayF has to knock out the last story he owes Armada project today15:40
JayFWork in github actions for a while if you want to love zuul more :D 15:41
TheJuliaheh15:41
TheJuliaThat is... I have a blood draw down the mountain if we don't get an order to evacuate15:41
JayFget that big bus fueled up and rarin' to go15:42
TheJuliawheeeeeee: https://www.google.com/maps/d/u/1/viewer?mid=1JNa6NxSir7mKFQA3h_vMfflv31gS5Gs&ll=33.619314211393586%2C-116.80587582714843&z=1115:42
TheJuliaI think we've got a half tank of diesel, way more than enough to get me down the hill into town15:43
JayFy'all stay safe. Leave earlier than ordered if you must15:43
JayFthat's way more imporatant than sqlalchemy lol15:43
JayFselect * from location where condition != 'burny';15:44
TheJulialol15:44
TheJuliathe locals say we're in a super safe spot anyway15:44
TheJuliabut yeah15:44
*** dviroel is now known as dviroel_15:46
TheJuliaAll, with regards to the pending election, you need to be activated to receive emails from CIVS.16:03
TheJuliaPlease see: https://lists.openstack.org/pipermail/openstack-discuss/2022-September/030405.html16:03
TheJuliapolling starts *monday*16:03
dtantsurmmm, inconvenient.. I wonder how many people will miss the message16:06
TheJuliawho knows16:06
TheJuliaIt has been a think for a while16:06
dtantsuryou're probably right, my email is already activated :)16:07
TheJuliaOkay, fixed my underlying issue... but still weirdness with some duplicate method calls16:09
JayFHow many ATCs in OpenStack don't follow the list16:11
JayFthat's wildly unfortunate :( 16:11
TheJuliaUnfortunately CIVS got flagged for spamming a while back16:12
JayFI figured, it's not ideal but it's not anyones' fault either16:16
opendevreviewMerged openstack/ironic-inspector master: Fix compatibility with oslo.db 12.1.0  https://review.opendev.org/c/openstack/ironic-inspector/+/85648816:34
TheJuliajust a constraint of the times I guess16:45
dtantsurlemme guess: the oslo.db patches should be backported to all bugfix branches in Zed?16:45
TheJuliadtantsur: likely yes16:46
opendevreviewDmitry Tantsur proposed openstack/ironic-inspector bugfix/11.0: Fix compatibility with oslo.db 12.1.0  https://review.opendev.org/c/openstack/ironic-inspector/+/85650516:46
TheJuliaI *susupect* fixes for 2.0 won't be ready until after we cut Zed, but I don't think anything in it will be backwards incompatible16:46
opendevreviewDmitry Tantsur proposed openstack/ironic-inspector bugfix/10.9: Fix compatibility with oslo.db 12.1.0  https://review.opendev.org/c/openstack/ironic-inspector/+/85650616:46
dtantsuryeah, I'm mostly worried about upper-constraints in the Zed branches16:46
TheJulialikewise, if it has 2 somehow, then I'm going to cry lots16:46
dtantsurI **hope** not16:47
TheJuliabut... this is just more time and a few weird methods which somehow get called twice... apparently deep inside taskmanager16:47
opendevreviewDmitry Tantsur proposed openstack/ironic bugfix/21.0: Fix compatibility with oslo.db 12.1.0  https://review.opendev.org/c/openstack/ironic/+/85650716:47
opendevreviewDmitry Tantsur proposed openstack/ironic bugfix/20.2: Fix compatibility with oslo.db 12.1.0  https://review.opendev.org/c/openstack/ironic/+/85650816:47
JayFdtantsur: +2, I trust the V+1 to validate that change16:52
JayFthanks16:52
JayFsame for inspector fixes16:53
TheJuliaokay.... I understand what the unit tests are seeing with autocommit off16:54
TheJuliawhy is the huge, giant, unknown question16:54
dtantsur:D16:54
* TheJulia has resorted to adding numerous print statements 16:55
dtantsurI'm growing more and more understanding towards people who don't like ORMs :)16:55
TheJuliaheh16:55
dtantsurthe ironic-inspector-client CI is long broken, isn't it?16:56
TheJuliauhh... It feels like it has broken every cycle with a minor thing16:58
iurygregorydtantsur, not really16:59
iurygregoryhttps://review.opendev.org/c/openstack/python-ironic-inspector-client/+/855110 16:59
iurygregoryit was working before we did the release16:59
dtantsurhuuuh16:59
iurygregory:D16:59
dtantsurthen why does the functional job fail now16:59
dtantsurmistery...16:59
iurygregory*magic*17:00
iurygregorywondering if is because we are still fixing the oslo.db problem...17:00
dtantsurpossibly17:00
dtantsursqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked17:03
dtantsurW17:04
dtantsurT17:04
dtantsurF17:04
dtantsurLet's hope it's the oslo.db thing indeed17:04
iurygregoryEveryone, for TC and PTL elections since civs is used there is an action required to be able to vote https://lists.openstack.org/pipermail/openstack-discuss/2022-September/030405.html17:04
JayFJulia posted that earlier too, we should probably try to get it in chat in most of the timezones17:05
JayFI'll post something in here before my EOD17:05
dtantsurAn interesting fact: the antelope unit test set has py3.10 voting. I think it was non-voting in zed?17:05
iurygregorydtantsur, correct...17:05
iurygregoryand it was using the new ubuntu if I recall17:06
iurygregoryor maybe it was just an attempt at that time17:06
* dtantsur is curious how rpittau's work on supporting jammy in bifrost is doing17:08
opendevreviewMerged openstack/python-ironicclient master: remove unicode from code  https://review.opendev.org/c/openstack/python-ironicclient/+/85447717:08
dtantsuranyone willing to try revive https://review.opendev.org/c/openstack/bifrost/+/848828/ before Riccardo comes back?17:08
JayFdtantsur: I'll add it to my list, no promises, but I might be able to find some time tomorrow17:09
JayFdepends on how much progress I make on this final armada thing before I go full stacker again :D 17:09
iurygregorydtantsur, I can check now17:09
iurygregoryor attempt to make it work :D17:09
dtantsurthx! I've issues a recheck to get the logs again17:09
JayFbtw since you all are here, if you ever need to ask for help with something async outside of IRC, feel free to add it to the incoming section here https://etherpad.opendev.org/p/JayFTODO17:10
opendevreviewMerged openstack/python-ironic-inspector-client master: Update master for stable/zed  https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/85642117:10
iurygregoryJayF, ack17:10
iurygregorydtantsur, ok, I will keep an eye on it (I'm checking why bifrost and IPE are broken :D)17:11
opendevreviewMerged openstack/metalsmith master: Update master for stable/zed  https://review.opendev.org/c/openstack/metalsmith/+/85641717:13
dtantsuronce https://review.opendev.org/c/openstack/sushy-tools/+/844126/ merges, could someone please request a sushy-tools release?17:13
dtantsurI got a request downstream to get it out (for another patch)17:13
opendevreviewNisha Agarwal proposed openstack/ironic master: Adds create_csr and add_https_certificate clean step  https://review.opendev.org/c/openstack/ironic/+/80448617:16
iurygregorydtantsur, I'm working on releases today, I can push a patch17:16
dtantsur++17:17
opendevreviewNisha Agarwal proposed openstack/ironic master: Adds create_csr and add_https_certificate clean step  https://review.opendev.org/c/openstack/ironic/+/80448617:22
dtantsursee you tomorrow folks o/17:24
opendevreviewMerged openstack/sushy-tools master: Add Chassis to ServiceRoot  https://review.opendev.org/c/openstack/sushy-tools/+/84412617:42
iurygregorysushy-tools release https://review.opendev.org/c/openstack/releases/+/856554 17:49
opendevreviewMerged openstack/metalsmith master: Add Python3 antelope unit tests  https://review.opendev.org/c/openstack/metalsmith/+/85641818:37
opendevreviewMerged openstack/metalsmith stable/zed: Update .gitreview for stable/zed  https://review.opendev.org/c/openstack/metalsmith/+/85641518:37
opendevreviewMerged openstack/metalsmith stable/zed: Update TOX_CONSTRAINTS_FILE for stable/zed  https://review.opendev.org/c/openstack/metalsmith/+/85641618:37
opendevreviewMerged openstack/ironic-python-agent stable/yoga: Fix software raid output poisoning  https://review.opendev.org/c/openstack/ironic-python-agent/+/85480518:42
TheJuliaoh where oh where did I put my braiiiinnnns18:43
TheJuliahttps://docs.sqlalchemy.org/en/14/dialects/sqlite.html#transaction-isolation-level-autocommit <--- screaming intensifies.... lots21:10
opendevreviewJulia Kreger proposed openstack/ironic master: WIP: Fix SQLAlchemy 2.0 Compatability  https://review.opendev.org/c/openstack/ironic/+/85633621:26
TheJuliaJayF: stevebaker[m]: I think I've got locking licked. And I think I now understand why some of the other behavior21:27
TheJuliadtantsur: ^^^21:27
TheJuliaat least, when autocommit is turned off21:27
JayFwhat was the 'eureka' moment of it?21:27
TheJuliareading about sqlite driver behavior and meshing that knowledge with autocommit behavior in sqlalchemy21:27
TheJuliabasically the take away is: sqlite consistency in unit testing basically kind of relies upon autocommit. Turn off autocommit... and well.. yeah :\21:29
JayFOH UNIT TESTS USE SQLITE21:30
JayFholy cow, we trust that to mimic mysql behavior!?21:31
TheJuliaalso transactional behavior is turned off with the default....21:31
* JayF just lost a lot of confidence in our tests21:31
TheJuliawell, some tests do, but yeah. The *real* test is going to be seeing what happens with that change in CI and then adjusting from there21:31
JayFyeah, tempest is still real lol21:32
TheJuliayup21:32
TheJuliaso anyway, I feel a bit more sane now21:32
TheJulia... or insane21:32
JayFthe unit test issues you were seeing make a lot more sense now21:32
JayFthe previous job, where we were wranging sqlalchemy: almost all of it was targetting sqlite21:32
TheJuliaI'm really not sure, but I think I need to go to the market21:32
JayFand I suspect these are on the same razors edge of pain21:32
TheJuliayeah, likely21:32
JayFcongrats on finding it \o/21:33
TheJuliaI was finally like "why does it not have the value I suspect, and why does it seem to have the old value other times... WUT?!"21:33
TheJuliaand that led me to the docs21:33
TheJuliaand then it clicked, and yeah.21:33
TheJuliawhat does one eat for a weekend when a small hurricane is coming up one side of a coast, and fire on the other side of the road?!21:35
JayFdrive thru on the way outta there21:39
TheJuliaI think we're going to stay put, it is just a huge headache to head down the mountain21:40
TheJuliaand we're in a nice safe cove it seems21:40

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!