| shukun | JayF: Could you please +2 this patch (https://review.opendev.org/c/openstack/ironic/+/850553) again so that I can continue the backport process to the rest branch? | 01:15 |
|---|---|---|
| vanou | good morning ironic | 01:36 |
| opendevreview | SONG SHUKUN proposed openstack/ironic bugfix/21.0: Add support auth protocols for iRMC https://review.opendev.org/c/openstack/ironic/+/862449 | 01:51 |
| opendevreview | SONG SHUKUN proposed openstack/ironic bugfix/21.0: Add support auth protocols for iRMC https://review.opendev.org/c/openstack/ironic/+/862449 | 01:57 |
| *** Guest0 is now known as osmanlicilegi | 04:11 | |
| opendevreview | Jacob Anders proposed openstack/sushy master: Improve resiliency of eTag handling https://review.opendev.org/c/openstack/sushy/+/856123 | 04:40 |
| rpittau | good morning ironic! o/ | 08:03 |
| dtantsur | hmm, I see `sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked` in the latest metal3 CI run | 09:20 |
| dtantsur | I wonder if we broke something. TheJulia ^^ | 09:20 |
| dtantsur | the first traceback is https://paste.opendev.org/show/b9nTqbw0m7HF7QZrsA1i/, then repeats for every action | 09:22 |
| dtantsur | yep, trivial to reproduce locally :( | 09:25 |
| dtantsur | reverting all 3 phases of SQLA2 migration fixes the problem. uh-oh. | 09:29 |
| dtantsur | TheJulia, JayF, we need to find something urgenly, else the whole SQLA2 work need to be redone :( | 09:30 |
| dtantsur | I assume we don't close some transaction, but I don't know which | 09:35 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic master: Do not disable autocommit until we fully migrate https://review.opendev.org/c/openstack/ironic/+/862476 | 09:43 |
| dtantsur | good news, I think I complete revert is not needed. Bad news ^^^ | 09:43 |
| dtantsur | TheJulia, rpittau, ^^ | 09:43 |
| rpittau | ah... ok | 09:44 |
| dtantsur | I think it boils down to some dbapi calls still opening a transaction without properly finishing it | 09:45 |
| dtantsur | which is okay for mysql, but not for sqlite | 09:45 |
| dtantsur | I wonder if we need a CI job with sqlite *somehow* | 09:46 |
| opendevreview | Verification of a change to openstack/ironic master failed: Cross test sushy with python 3.10 https://review.opendev.org/c/openstack/ironic/+/862140 | 09:49 |
| rpittau | I wonder if I can hook up a metal3 job with the ironic-image based on source directly here | 09:50 |
| dtantsur | that would be ideal (but make sure to run it on our infra, not the metal3's one) | 09:50 |
| rpittau | yeah | 09:52 |
| opendevreview | Jacob Anders proposed openstack/sushy master: Improve resiliency of eTag handling https://review.opendev.org/c/openstack/sushy/+/856123 | 10:11 |
| iurygregory | good morning Ironic | 12:15 |
| opendevreview | Jacob Anders proposed openstack/sushy master: [WIP] Retry BootSourceOverride request when SettingsURI is read-only https://review.opendev.org/c/openstack/sushy/+/856597 | 12:57 |
| TheJulia | Good morning | 13:19 |
| TheJulia | dtantsur: oh lovely :( | 13:20 |
| TheJulia | dtantsur: or a commit that hasn't called yet I guess | 13:20 |
| TheJulia | I remember checking everything to making sure we closed things out, but maybe there is more. | 13:22 |
| TheJulia | oh wow, that halted very early on | 13:22 |
| TheJulia | dtantsur: is that log with debug turned on? | 13:30 |
| *** rcastillo_ is now known as rcastillo | 13:30 | |
| TheJulia | dtantsur: no nodes, just start process correct? | 13:37 |
| TheJulia | I *suspect* it might be register_conductor | 13:39 |
| dtantsur | TheJulia: I assume it's because touch_conductor was not updated | 14:38 |
| dtantsur | good morning | 14:38 |
| JayF | What kind of tests are those failing on metal3? Is it trying to setup an Ironic server only backed by sqlite? | 14:38 |
| JayF | Or just unit tests that cover a case that ours don't? | 14:38 |
| TheJulia | so orm style of ops is still supported, and enginefacade is supposed to close things out, but I think it is a situation where we end up handing back and object that keeps the transaction open | 14:39 |
| TheJulia | just some style of ops did not explicitly need to change nor did we spot in unit testing, and heartbeat ops is definitely an area where I could see it breaking | 14:40 |
| JayF | ack; good stuff then, nice to find these corners in CI then (even if not /our/ CI) | 14:40 |
| TheJulia | JayF: starting the service with just sqlite outside of unit test | 14:40 |
| JayF | Do we document that as supported? (if metal3 operates this way; we obviously support it -- it'd be cool if we documented it) | 14:41 |
| JayF | and probably should also have a CI job that tests it in our gate, too | 14:41 |
| TheJulia | yeah, realistically we're going to need a single process sqllite job | 14:42 |
| TheJulia | That is likely the only upfront config which would have caught this | 14:42 |
| * TheJulia rebuilds tox env since it didn't want to be happy to run unit tests | 14:42 | |
| dtantsur | JayF: no write transactions work | 14:47 |
| dtantsur | basically, something (I presume the conductor keepalive loop) locks the database and never unlocks it | 14:48 |
| JayF | Yeah I was mainly trying to tease out why this didn't show up in Ironic CI | 14:48 |
| JayF | because we shouldn't be breaking downstream integrations | 14:48 |
| JayF | and, apparently, we should advertise you can use Ironic with sqlite | 14:48 |
| dtantsur | because mysql is fine with transactions opened forever, apparently? | 14:48 |
| JayF | mysql, depending on how it's configured, implicitly closes transactions in some cases | 14:48 |
| dtantsur | I don't think it's a sqlite-specific problem, it can be an actual bug if we don't commit transactions | 14:48 |
| * TheJulia blinks | 14:49 | |
| TheJulia | I think the issue is the transaction gets started, and can live onward if we return a query object | 14:50 |
| TheJulia | it could also be singleprocess is a contributing factor | 14:50 |
| JayF | sqlite also doesn't so simultaneous writes | 14:51 |
| dtantsur | The issue can be reproduced even without any API accesses | 14:51 |
| JayF | so having a txn hang open is guaranteed fatal | 14:51 |
| dtantsur | just start and wait a few seconds | 14:51 |
| JayF | whereas in mysql, it's not limited that way | 14:51 |
| JayF | it'd be possible we'd have a dangling write cxn in a place that isn't called much | 14:51 |
| JayF | therefore not breaking mysql | 14:51 |
| * JayF thinks this fits with TheJulia's hypothesis of register_conductor or some other method called on start | 14:52 | |
| * TheJulia suspects a breaking version of pip dropped | 14:52 | |
| JayF | Meeting in ~5 minutes | 14:54 |
| TheJulia | https://paste.opendev.org/show/817308/ | 14:57 |
| dtantsur | wut | 15:00 |
| TheJulia | yup... I bumped my requirements locally too just to see if that was it | 15:00 |
| TheJulia | installing py39 now | 15:01 |
| JayF | #startmeeting ironic | 15:01 |
| opendevmeet | Meeting started Mon Oct 24 15:01:44 2022 UTC and is due to finish in 60 minutes. The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:01 |
| opendevmeet | The meeting name has been set to 'ironic' | 15:01 |
| TheJulia | o/ | 15:01 |
| matfechner | o/ | 15:01 |
| ebbex | o/ | 15:01 |
| JayF | Good morning everyone. Welcome to our meeting. | 15:01 |
| ajya | o/ | 15:02 |
| hjensas | o/ | 15:02 |
| JayF | Gonna be honest and say I probably dropped the ball and should've cancelled this; I'm doubtful we have much to talk about so I'm going to move fast. | 15:02 |
| iurygregory | o/ | 15:02 |
| dtantsur | o/ | 15:02 |
| rpittau | o/ | 15:02 |
| JayF | #note We had a productive PTG last week. Thank you to everyone who particiapted! | 15:02 |
| dtantsur | live fast, endmeeting early | 15:02 |
| JayF | There are no action items from last meeting that were not handled in a PTG session. | 15:02 |
| JayF | Normally we'd also say at this point to check the whiteboard for workstream status; but we haven't updated any of that for the PTG. | 15:03 |
| TheJulia | dtantsur: ++ | 15:03 |
| JayF | So I am going to just keep going as any information there is more out of date than scrolling up 10 lines in IRC :) | 15:03 |
| TheJulia | I would suggest updating the whiteboard | 15:03 |
| rloo | o/ | 15:03 |
| TheJulia | some stuff naturally rolls, some stuff is done and can be struck-trough and removed soon | 15:03 |
| JayF | It's my intention to do that very soon as part of the documenation of workstreams | 15:04 |
| JayF | #action JayF to ensure whiteboard gets updated when he makes 2023.1 Ironic workstreams spec | 15:04 |
| JayF | Moving on | 15:04 |
| JayF | Just a reminder if anyone needs code reviewed, please add hashtag ironic-week-prio and/or mention it here | 15:04 |
| JayF | I don't believe there's anything outstanding, but if there is, please do tag it. I'll be reviewing all tagged, up to date PRs this afternoon. | 15:05 |
| dtantsur | https://review.opendev.org/c/openstack/ironic/+/862476 is the mosti mportant one | 15:05 |
| JayF | ack | 15:05 |
| ebbex | I've got a bunch for bifrost here. https://review.opendev.org/q/topic:deps , but there's a big question as to what to do with suse. | 15:06 |
| dtantsur | ebbex: already deprecated => remove support for good? | 15:06 |
| ebbex | Fine by me, all in favor of removing suse support? | 15:06 |
| JayF | If it's already been deprecated, removal is the next step | 15:07 |
| dtantsur | https://docs.openstack.org/bifrost/latest/install/index.html#supported-operating-systems does not even list suse any more | 15:07 |
| rpittau | let's remove it | 15:07 |
| dtantsur | I think I just was too lazy to remove the code later on :) | 15:07 |
| rpittau | heh me too :P | 15:07 |
| JayF | That sounds like the clear consensus. | 15:07 |
| ebbex | Cool. I'll have it cleaned out instead. | 15:08 |
| JayF | #action ebbex To remove suse-supporting code from bifrost, it's already deprecated and scheduled to be removed. | 15:08 |
| JayF | thanks for bringing that up and volunteering to clean it up | 15:08 |
| JayF | if there are no other patches to discuss; moving on | 15:08 |
| ebbex | np :) | 15:08 |
| JayF | Baremetal SIG: it was decided at PTG the BM SIG would move to quarterly, and be booked well in advance with calendar invites sent out. | 15:09 |
| JayF | Does someone want to own the action to schedule that? I'd suggest booking one for Q1 2023 as our next one | 15:09 |
| dtantsur | arne_wiebalck is the natural candidate, but I can do it if he doesn't have time | 15:09 |
| dtantsur | there an internal something that we need to clear up first before I know my schedule for Q1 | 15:10 |
| JayF | that sounds good; we should follow up next week if nobody has taken action yet | 15:10 |
| JayF | thanks :) | 15:10 |
| JayF | There are no RFEs for review. | 15:10 |
| dtantsur | you can put the action on me for now, keeping in mind that it probably won't happen before next week | 15:10 |
| dtantsur | * by next week | 15:10 |
| JayF | #action dtantsur or arne_wiebalck to book Baremetal SIG meeting for Q1 2023 sometime in the next couple of weeks. | 15:11 |
| dtantsur | yep | 15:11 |
| JayF | There are no pre-agenda'd open discussion items | 15:11 |
| JayF | Are there any topics for open discussion? I'll give a couple minutes for anyone to speak up before ending the meeting. | 15:11 |
| TheJulia | I did talk to a larger operator w/r/t the ironic/nova-compute stuffs | 15:13 |
| JayF | Anything interesting come outta the chat? | 15:13 |
| TheJulia | basically, I could tell there was an undertone that they were *really* not pleased by the availability and the locked interaction behavior and the fact they will now need to self-implement HA, but that the overall tradeoff from the amount of work/effort they face *today* would offset that as the lost hypervisor records today are *far* more painful than what they view that work to be | 15:14 |
| JayF | I think that's generally reflective of how most of us feel about it, yeah? | 15:14 |
| JayF | Not ideal, but the best we can do inside the nova model and orders of magnitude better than things today | 15:15 |
| TheJulia | They believe that based on what I wrote in the spec that I tossed up for ironic, that it should generally "just work" moving forward | 15:15 |
| TheJulia | JayF: yeah, I think so as well | 15:15 |
| JayF | That's a good report, thank you. | 15:15 |
| JayF | Everyone being slightly unhappy is the usual sign of true compromise :) | 15:16 |
| JayF | I'm going to close the meeting out if there's nothing else. | 15:16 |
| TheJulia | #link https://review.opendev.org/c/openstack/ironic-specs/+/861803 | 15:16 |
| TheJulia | JayF: johnthetubaguy: sorry, but I had to get the interaction out of my head | 15:16 |
| JayF | Reading that is on my list today :). It's going to be a spec writing/reading day :). | 15:16 |
| TheJulia | but nova-side is going to be important/needed as well | 15:16 |
| TheJulia | johnthetubaguy: I'd also appreciate a review of that spec if you have the time, since I took a shot at how I percieve the nova-compute service upgrade to execute | 15:17 |
| JayF | Thanks for taking the first swing at that spec; we'll review and enhance it | 15:20 |
| JayF | #endmeeting | 15:20 |
| opendevmeet | Meeting ended Mon Oct 24 15:20:08 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:20 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/ironic/2022/ironic.2022-10-24-15.01.html | 15:20 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/ironic/2022/ironic.2022-10-24-15.01.txt | 15:20 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/ironic/2022/ironic.2022-10-24-15.01.log.html | 15:20 |
| opendevreview | Verification of a change to openstack/ironic master failed: Add ironic-grenade-skip-level Job https://review.opendev.org/c/openstack/ironic/+/836966 | 15:21 |
| TheJulia | so py39 works just fine, I'm suspecting a dependency dropped that breaks 38 | 15:25 |
| JayF | Would you all be OK if I just documented "possible workstreams for 2023.1 Antelope" and "completed workstreams for 2023.1 Antelope" and refer people to the whiteboard to see what is in-progress? | 15:31 |
| opendevreview | Verification of a change to openstack/ironic master failed: Do not disable autocommit until we fully migrate https://review.opendev.org/c/openstack/ironic/+/862476 | 15:34 |
| JayF | that failed in unit tests(!) | 15:35 |
| JayF | The conflict is caused by: | 15:35 |
| JayF | The user requested SQLAlchemy>=1.4.0 | 15:35 |
| JayF | The user requested (constraint) sqlalchemy===1.4.4 | 15:35 |
| JayF | TheJulia: ^ is this the pip breakage you were referring to? | 15:35 |
| JayF | if so, it's impacting gate :| | 15:35 |
| JayF | https://zuul.opendev.org/t/openstack/build/dcf1bfcc6b40496badc4d9d37267ec08 | 15:35 |
| JayF | that is a brutal zuul failure line on that fix patch | 15:36 |
| TheJulia | JayF: seems reasonable, although we've often stalled on items which have built up on the whiteboard | 15:36 |
| TheJulia | think, "oh, this came up and it was more important, I'll try to get back ot the other thing) | 15:37 |
| JayF | TheJulia: I forsee a "Is anyone working on:" topic with a handful of subtopics for next week | 15:37 |
| JayF | TheJulia: for things that were lost mid-stream | 15:37 |
| TheJulia | yah | 15:38 |
| JayF | What the hell is going on with the gate? | 15:40 |
| JayF | breaking pip version totally seems possible | 15:41 |
| JayF | but we can't be the only folks hurting with this | 15:41 |
| opendevreview | Ebbex proposed openstack/bifrost master: Remove remaining traces of Suse https://review.opendev.org/c/openstack/bifrost/+/861541 | 15:51 |
| TheJulia | dtantsur: are you in a place to quickly test a patch? | 15:55 |
| TheJulia | with sqlite? | 15:55 |
| * TheJulia crosses her fingers | 16:09 | |
| dtantsur | TheJulia: to an extent (I can test if things work at all, but not the whole metal3 flow) | 16:10 |
| TheJulia | that should be fine | 16:11 |
| TheJulia | running unit tests now | 16:11 |
| rpittau | bye everyone o/ | 16:13 |
| JayF | Issue with gate appears to be infrastructural; pypi is fronted by a CDN and if the CDN can't reach backend servers; it gives out of date info that blows up the gate | 16:14 |
| JayF | (per clark in #opendev) | 16:14 |
| TheJulia | sweet! | 16:15 |
| opendevreview | Julia Kreger proposed openstack/ironic master: WIP: Sqllite fix maybe? https://review.opendev.org/c/openstack/ironic/+/862506 | 16:18 |
| dtantsur | TheJulia: I wonder if you need to update touch_conductor as well | 16:19 |
| TheJulia | possibly | 16:19 |
| TheJulia | likely next | 16:19 |
| * TheJulia will be pulling it back up after the current test run | 16:20 | |
| TheJulia | I don't think it returned a query object | 16:20 |
| TheJulia | but I shall see! | 16:20 |
| opendevreview | Ebbex proposed openstack/bifrost master: Switching netstat to ss in report https://review.opendev.org/c/openstack/bifrost/+/861542 | 16:21 |
| dtantsur | TheJulia: with your patch still crashes in touch_conductor :( | 16:21 |
| TheJulia | same backtrace? | 16:21 |
| dtantsur | yeah | 16:22 |
| TheJulia | Interesting... | 16:22 |
| TheJulia | what gets called *before* touch_conductor | 16:22 |
| opendevreview | Ebbex proposed openstack/bifrost master: Fix initial python/venv dependencies https://review.opendev.org/c/openstack/bifrost/+/861534 | 16:23 |
| opendevreview | Ebbex proposed openstack/bifrost master: Install git-core in prep-for-install https://review.opendev.org/c/openstack/bifrost/+/861535 | 16:23 |
| opendevreview | Ebbex proposed openstack/bifrost master: Remove unused iniparse python system dependency https://review.opendev.org/c/openstack/bifrost/+/862391 | 16:23 |
| opendevreview | Ebbex proposed openstack/bifrost master: Remove pymysql from system dependencies https://review.opendev.org/c/openstack/bifrost/+/861537 | 16:23 |
| opendevreview | Ebbex proposed openstack/bifrost master: Install passlib to venv (htpasswd) https://review.opendev.org/c/openstack/bifrost/+/861536 | 16:23 |
| opendevreview | Ebbex proposed openstack/bifrost master: Install firewall to venv (redhat) https://review.opendev.org/c/openstack/bifrost/+/861538 | 16:23 |
| TheJulia | touch_conductor should be fine, fwiw | 16:24 |
| opendevreview | Julia Kreger proposed openstack/ironic master: WIP: Sqllite fix maybe? https://review.opendev.org/c/openstack/ironic/+/862506 | 16:32 |
| TheJulia | maybe?! | 16:32 |
| TheJulia | dtantsur: give ^^ a quick spin if you can | 16:33 |
| dtantsur | same :( | 16:36 |
| TheJulia | ugh! | 16:36 |
| dtantsur | you can reproduce by ironic running with | 16:38 |
| dtantsur | [DEFAULT] | 16:38 |
| dtantsur | auth_strategy = noauth | 16:38 |
| dtantsur | debug = True | 16:38 |
| dtantsur | rpc_transport = none | 16:38 |
| dtantsur | enabled_hardware_types = fake-hardware | 16:38 |
| dtantsur | enabled_boot_interfaces = fake | 16:38 |
| dtantsur | enabled_deploy_interfaces = fake | 16:38 |
| TheJulia | thanks | 16:42 |
| * TheJulia wonders if it is get_conductor | 16:45 | |
| TheJulia | err, no, that doesn't do it | 16:47 |
| TheJulia | dtantsur: so... it is running, no deadlocks | 17:11 |
| TheJulia | yet! | 17:11 |
| dtantsur | huh? with your patch? | 17:12 |
| TheJulia | yeah | 17:12 |
| TheJulia | my local repo state and current dependencies | 17:13 |
| TheJulia | periodics are triggering | 17:13 |
| TheJulia | dtantsur: can I get a link to the job log where it is failing? | 17:14 |
| dtantsur | TheJulia: sure, fetch the tarball from https://jenkins.nordix.org/job/metal3_ironic_image_main_integration_test_ubuntu/22/ | 17:14 |
| TheJulia | https://paste.opendev.org/show/byaFQonINexfnQ2FeBrR/ <-- what it is running with locally | 17:15 |
| dtantsur | I think I have a freshly built venv | 17:21 |
| dtantsur | anyway, time to go unfortunately | 17:21 |
| dtantsur | TheJulia: do you feel particularly bad about re-enabling autocommit for now and keeping experimenting? | 17:21 |
| dtantsur | metal3 is kinda stuck | 17:21 |
| JayF | We already have that change approved | 17:22 |
| dtantsur | ooops | 17:22 |
| JayF | gate is in pypi-outage-hell | 17:22 |
| dtantsur | oooooooops | 17:22 |
| JayF | see #opendev | 17:22 |
| TheJulia | dtantsur: just to confirm, this is single process yes? | 17:23 |
| dtantsur | true | 17:23 |
| dtantsur | so yeah, I'm using the `ironic` executable | 17:23 |
| TheJulia | okay, weird | 17:25 |
| TheJulia | oh, there we go | 17:25 |
| TheJulia | finally | 17:25 |
| TheJulia | it failed after the second round of periodics | 17:25 |
| dtantsur | \o/ | 17:26 |
| dtantsur | on this positive note I'll wish you a nice evening | 17:27 |
| TheJulia | goodnight! | 17:27 |
| JayF | It's not just me or firefox, right? The code review dashboards on the ironic whiteboard are completely fubar now | 17:34 |
| TheJulia | looks fine to me | 17:36 |
| TheJulia | oh... the dashboards | 17:36 |
| TheJulia | I'm not sure I've used the classic dashboards in ages | 17:36 |
| TheJulia | some of the super old, like create db aeva originally did break in one of the gerrit upgrades, but I thought we nuked them | 17:38 |
| JayF | Yeah I think it's all broken | 17:57 |
| JayF | I tried to remake them in the "new" way | 17:57 |
| JayF | and they broke again | 17:57 |
| JayF | I'm going to remove them ... | 17:57 |
| TheJulia | yeah, we mainly went to hashtag + combined queries | 18:08 |
| TheJulia | 2.5 minutes.. | 18:32 |
| TheJulia | Okay, there are a couple differet things going on | 21:30 |
| TheJulia | 1) OperationalError as raised by sqlite3's adapter is not an exception it knows to auto-retry on | 21:30 |
| TheJulia | 2) it *actually works* every once and a while. If you drop the heartbeat interval too low, it never works | 21:35 |
| JayF | I'm trying to add | 21:47 |
| JayF | > Cleaning up RAID created by tenants | 21:47 |
| JayF | to my document | 21:47 |
| JayF | but it's unclear to me what the actual specific work is here? | 21:47 |
| JayF | I think it's tl;dr: 1) we added support for skipping cleaning of sw raid member devices 2) that needs to clean out RAID created by tenants, even if that is disabled (?) | 21:48 |
| JayF | but that sounds wrong to me, and I think I must be missing something | 21:48 |
| JayF | dtantsur: ftarasenko: ^ if you could clarify for me what the work is for clean out raid created by tenants, it's not clear to me and the PTG notes arenot wonderful :( | 21:49 |
| JayF | I'm going to push with a known-probably-bad summary before I EOD; you can also comment on that PR (or push up a better summary) if you'd prefer | 21:49 |
| opendevreview | Jay Faulkner proposed openstack/ironic-specs master: Add Ironic work items for 2023.1 https://review.opendev.org/c/openstack/ironic-specs/+/862538 | 22:00 |
| TheJulia | w/r/t sqlite: What if we only support in-memory.... | 22:07 |
| TheJulia | well, that would solve metal3 | 22:08 |
| TheJulia | it wouldn't solve standalone without autocommit | 22:08 |
| * TheJulia suspect metal3 creates a pile of sqlalchemy objects that eventually manage to autocommit through | 22:08 | |
| TheJulia | well, ironic in metal3 | 22:08 |
| JayF | iurygregory: please remove the -1 or clarify in https://review.opendev.org/c/openstack/releases/+/847933 | 22:10 |
| TheJulia | I *think* the only path forward to possibly making it more sane is to restructure db queries so we no longer use orm model query format | 22:14 |
| TheJulia | even then... pagination explicitly requires it | 22:14 |
| TheJulia | for now. | 22:14 |
| TheJulia | basically, we need a mode where we can have a singular reader/writer operation | 22:22 |
| TheJulia | globally, OR can use the db connection pooling behavior | 22:22 |
| TheJulia | and... each $thing we do is a new file open | 22:23 |
| TheJulia | which is locking | 22:23 |
| JayF | It seems a little bananas to me that we ever agreed that Ironic's service could run under sqlite limitations | 22:24 |
| TheJulia | example: just after the periodic trigger: https://paste.opendev.org/show/bYQLLbacafpsfba8oIsQ/ | 22:24 |
| TheJulia | I think it was always implicit, less explicitly stated | 22:24 |
| TheJulia | and I think "because heat can" | 22:24 |
| TheJulia | but I think heat does it in-memory | 22:24 |
| JayF | I mean, it ships that way with metal3 | 22:25 |
| JayF | which means we either explicitly support it or we dunk on a major external integration | 22:25 |
| JayF | in-memory sqlite is a whole different ball of wax to on disk | 22:25 |
| TheJulia | yeah | 22:26 |
| TheJulia | and in metal3 it is ephemeral | 22:26 |
| JayF | dtantsur: we should probably talk about metal3 moving to sqlite in memory or off sqlite altogether | 22:26 |
| TheJulia | so.... there really is no need to file back it, I think | 22:26 |
| JayF | dtantsur: this is only going to get worse | 22:26 |
| * JayF hands anyone who wants it a mysql container image for docker | 22:27 | |
| JayF | :P | 22:27 |
| TheJulia | Yeah, periodics restart, 3 new connections after everything closes out | 22:27 |
| TheJulia | \o/ | 22:27 |
| * TheJulia dances | 22:27 | |
| TheJulia | and not in a good way :) | 22:27 |
| TheJulia | the added pain with it all is sessions... which create transactions are auto-created by enginefacade even if we don't want/need them | 22:32 |
| TheJulia | and that compounds things, they are supposed to auto-close out as well, but I'm fairly sure on my theory that the orphaned returned element holds the session... i.e. model_query calls. | 22:34 |
| opendevreview | Jay Faulkner proposed openstack/ironic-specs master: Add Ironic work items for 2023.1 https://review.opendev.org/c/openstack/ironic-specs/+/862538 | 22:44 |
| opendevreview | Jay Faulkner proposed openstack/ironic-specs master: Add Ironic work items for 2023.1 https://review.opendev.org/c/openstack/ironic-specs/+/862538 | 22:51 |
| opendevreview | Verification of a change to openstack/ironic master failed: Do not disable autocommit until we fully migrate https://review.opendev.org/c/openstack/ironic/+/862476 | 23:14 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!