| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: Don't actually heartbeat with sqlite! https://review.opendev.org/c/openstack/ironic/+/888506 | 01:09 |
|---|---|---|
| TheJulia | so, that might be a bit extreme, but there is a point where everyone should just use a full database. | 01:09 |
| opendevreview | Julia Kreger proposed openstack/ironic master: DNM: Don't actually heartbeat with sqlite! https://review.opendev.org/c/openstack/ironic/+/888506 | 04:53 |
| dtantsur | TheJulia: I don't think it's extreme to disable conductor heartbeating for an architecture that is very explicitly single-conductor. That being said, retries will probably solve most of our issues. | 08:07 |
| JayF | I wonder if that conductor heartbeat getting stale will have side effects we didn't consider | 08:09 |
| JayF | Is metal3 single conductor or single process? | 08:27 |
| JayF | it's still separate api/cond, yeah | 08:27 |
| dtantsur | single process | 08:45 |
| dtantsur | metal3 does support the split mode though | 08:45 |
| dtantsur | but that will require mariadb | 08:46 |
| dtantsur | one of our problems (our = metal3) is that we don't really prescribe any architecture, we support too many options | 08:47 |
| dtantsur | I'm trying to fix that with https://github.com/dtantsur/ironic-operator but no idea when I'm able to finish that | 08:48 |
| dtantsur | JayF: ^^ | 08:48 |
| JayF | that doesn't sound familiar at all | 08:48 |
| JayF | lol | 08:48 |
| dtantsur | :D | 08:48 |
| dtantsur | .. maybe I should decide it's my last day before the PTO, and I should just hack on this operator ... | 08:49 |
| JayF | YES | 08:49 |
| JayF | As PTL of Ironic; I'm assigning you an urgent task | 08:49 |
| JayF | to go do whatever you think is best today ;) | 08:49 |
| dtantsur | \o/ | 08:49 |
| JayF | I assign myself that task every morning lol | 08:51 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic master: Document caveats of running with SQLite https://review.opendev.org/c/openstack/ironic/+/888529 | 09:24 |
| dtantsur | JayF, TheJulia ^^^ | 09:24 |
| dtantsur | I wonder if there is still value of having different guides for RH/Debian | 09:26 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic master: Stop splitting installation docs per distros https://review.opendev.org/c/openstack/ironic/+/888531 | 09:38 |
| dtantsur | okay, I wanted to do it for a long time ^^ | 09:38 |
| JayF | dtantsur: fwiw; if we wanted to support migrations on sqlite, if we wanted to pick a release to start I think we can | 09:55 |
| JayF | dtantsur: I have the knowledge on what can/can't be done and how to get around them on sqlite | 09:55 |
| * JayF used to work on a sqlite-only project that had a ton of alembic migrations | 09:55 | |
| JayF | dtantsur: > This is not a typo! A full path requires 4 slashes. | 09:56 |
| JayF | dtantsur: I needed this knowledge last month, I spent way too long learning that the hard way :D | 09:56 |
| dtantsur | We've all been there :D | 10:15 |
| dtantsur | JayF: re sqlite migrations: yes, I'd like to have this option. | 10:15 |
| JayF | dtantsur: post to the mailing list about it then and gather consensus, I'm happy to have the discussion | 10:18 |
| dtantsur | JayF: I'll be out for the next 2 weeks, don't want to start any large discussions | 10:18 |
| JayF | only real hilarious bit would be, adding more migrations testing right now seems like the stupidest possible thing we could do lol | 10:18 |
| JayF | dtantsur: that's the best time! Just set off the smoke bomb and run ;) | 10:18 |
| dtantsur | :D | 10:18 |
| JayF | seriously, I don't think it should be too controversial; maybe even -nv a job to validate it | 10:21 |
| JayF | only real pain is sqlite migrations in alembic can sometimes end up as "Do N^2 things for N rows" | 10:22 |
| JayF | so they are not fast, but they also are not typically large so it mostly is a wash | 10:22 |
| JayF | (for migrations unsupported by sqlite; alembic does a batch migration where it basically recreates the table under a new name and schema, copies over the data, and then replaces the old table with the new) | 10:22 |
| dtantsur | I think it's fair to say that sqlite migrations will always be offline ones, with downtime | 10:25 |
| dtantsur | and that the database sizes will be modest | 10:25 |
| JayF | you know what | 10:32 |
| JayF | we shoudl wait until post-sqla2.0 to do this | 10:32 |
| JayF | because otherwise we'll have a crapton of 1.4/2.0 conditionals in the same way as we do for the WAL event stuff, I think | 10:32 |
| JayF | that would also put it at the start of a slurp, which I doubt sqlite users care about but still is sensible | 10:32 |
| opendevreview | Merged openstack/bifrost master: Consolidate ubuntu/debian required_defaults https://review.opendev.org/c/openstack/bifrost/+/888444 | 10:33 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic master: Document caveats of running with SQLite https://review.opendev.org/c/openstack/ironic/+/888529 | 10:36 |
| dtantsur | JayF: agreed | 10:36 |
| JayF | dtantsur: then go agree with my TC resolution to set a deadline on it, as was mostly consensus at the forum ;) https://review.opendev.org/c/openstack/governance/+/887083 | 10:38 |
| opendevreview | Merged openstack/bifrost master: sgabios-bin is a subpackage of sgabios https://review.opendev.org/c/openstack/bifrost/+/888445 | 10:45 |
| opendevreview | Merged openstack/bifrost stable/xena: Skip unnecessary SDK get_machine calls https://review.opendev.org/c/openstack/bifrost/+/883118 | 10:45 |
| dtantsur | Could someone please merge https://review.opendev.org/c/openstack/bifrost/+/887613 ? | 12:21 |
| dtantsur | It is very annoying to test inspection because of it | 12:21 |
| JayF | +2 if it's still blocking you just workflow it and we can revert it if anyone disagrees 😁 | 12:58 |
| TheJulia | JayF: mostly just hash ring resets which I think I've identified and... addressed | 13:23 |
| TheJulia | Also, good morning everyone! | 13:23 |
| TheJulia | well, looks like retry logic and turning off heartbeating seems to be the magic trick | 13:29 |
| TheJulia | we'll need to retry lock handling as well it looks like based upon my latest change | 13:29 |
| TheJulia | dtantsur: replied to most of your comments, basically agree, none of those patches were in final form | 13:36 |
| TheJulia | I got a result where I watched the heartbeater somehow just line up perfectly to thrash everything into the ground which started me on the "just disable the heartbeater" path | 13:36 |
| TheJulia | Remarkably, looking at the latest log, we still had an incoming request for a lock error, and had an earlier retry occur, so... I think we're on the "right" path (even though I'll restack that one first on master and go from there. | 13:37 |
| dtantsur | meanwhile, I got a first successful Ironic installation using https://github.com/dtantsur/ironic-operator \o/ | 13:59 |
| dtantsur | it's very limited and cannot be used for e2e deployments, but it's something! | 13:59 |
| opendevreview | Verification of a change to openstack/bifrost master failed: Allow nginx to access images again on systems with SELinux https://review.opendev.org/c/openstack/bifrost/+/887613 | 14:53 |
| dtantsur | See you on July 31st, folks o/ I'll turn off my IRC bouncer at some point, please use email. | 14:55 |
| TheJulia | oh wow, have a great time off | 14:59 |
| * TheJulia needs to take some PTO off soon | 15:02 | |
| NobodyCam | Good Morning OpenStack Folks... And of course..... | 15:19 |
| NobodyCam | TGIF! | 15:19 |
| TheJulia | good morning NobodyCam | 15:21 |
| TheJulia | \o/ no more wedding this weekend /o\ | 15:21 |
| NobodyCam | O/ morning TheJulia | 15:21 |
| NobodyCam | oh that's good (because of the heat) and .... | 15:21 |
| TheJulia | groom crashed his motorcycle | 15:22 |
| TheJulia | :( | 15:22 |
| TheJulia | so he needs to heal first | 15:22 |
| NobodyCam | oh no :( | 15:22 |
| TheJulia | and... well.. 119F was the updated forecast as of last night :( | 15:23 |
| TheJulia | This weekend is going to be "not fun" | 15:23 |
| NobodyCam | Hope he'll recover quickly | 15:23 |
| TheJulia | I suspect he will | 15:23 |
| NobodyCam | yea, we're planing on heading out early (for us) on Saturday | 15:23 |
| NobodyCam | \o/ for recovery | 15:24 |
| TheJulia | makes sense | 15:25 |
| TheJulia | sooo many things need to be touched when you start to monkey with the hash ring | 15:42 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Don't actually heartbeat with sqlite https://review.opendev.org/c/openstack/ironic/+/888506 | 16:22 |
| TheJulia | almost have a meta retry decorator | 17:31 |
| TheJulia | just have to remember how to twist tenacity's arm to my will | 17:32 |
| *** tzumainn__ is now known as tzumainn | 17:47 | |
| *** janders6 is now known as janders | 20:34 | |
| opendevreview | Merged openstack/bifrost master: Reduce the libvirt/qemu packages list https://review.opendev.org/c/openstack/bifrost/+/888446 | 23:48 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!