| kevko | bbezak: will you have a time to fix that kolla-ansible upgrade CI jobs ? | 07:57 |
|---|---|---|
| kevko | morning \o | 07:57 |
| opendevreview | Roman Krček proposed openstack/kolla-ansible master: Split ipv4 and ipv6 systemctl config https://review.opendev.org/c/openstack/kolla-ansible/+/905831 | 08:07 |
| kevko | frickler: can u check please my reply comment in my designate patch if you have a min ? | 08:27 |
| bbezak | morning, kevko: yeah I'll look into that today | 08:29 |
| opendevreview | Rafal Lewandowski proposed openstack/kolla-ansible master: [WIP] Enable ML2/OVN and distributed FIP by default https://review.opendev.org/c/openstack/kolla-ansible/+/904959 | 08:37 |
| opendevreview | Ivan Halomi proposed openstack/kolla-ansible master: Refactor of getting list of containers that will also prepare function for migration scenario https://review.opendev.org/c/openstack/kolla-ansible/+/905837 | 08:48 |
| kevko | bbezak: thanks | 09:17 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible master: Configuring nova-compute to support vendordata over configdrive. https://review.opendev.org/c/openstack/kolla-ansible/+/905843 | 09:56 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible master: Configuring nova-compute to support vendordata over configdrive. https://review.opendev.org/c/openstack/kolla-ansible/+/905843 | 10:01 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Disable instace usage audit https://review.opendev.org/c/openstack/kolla-ansible/+/905502 | 10:50 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Test designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 10:50 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible master: Fix OpenSearch upgrade tasks idempotency https://review.opendev.org/c/openstack/kolla-ansible/+/905851 | 10:58 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Disable instace usage audit https://review.opendev.org/c/openstack/kolla-ansible/+/905502 | 11:03 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Test designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 11:03 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix neutron DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/905852 | 11:03 |
| opendevreview | Ivan Halomi proposed openstack/kolla-ansible master: Refactor of getting list of containers that will also prepare function for migration scenario https://review.opendev.org/c/openstack/kolla-ansible/+/905837 | 11:27 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix neutron DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/905852 | 11:27 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Test designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 11:27 |
| *** ravlew is now known as Guest14423 | 11:33 | |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Enable instance usage audit only when ceilometer is enabled https://review.opendev.org/c/openstack/kolla-ansible/+/905502 | 11:34 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix neutron DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/905852 | 11:34 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Test designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 11:34 |
| *** ravlew2 is now known as ravlew | 11:37 | |
| opendevreview | Bartosz Bezak proposed openstack/kolla-ansible master: use docker_custom_config override for Kolla CI upgrade jobs https://review.opendev.org/c/openstack/kolla-ansible/+/905858 | 11:51 |
| opendevreview | Bartosz Bezak proposed openstack/kolla master: Fix openstack CADF audit maps and installation https://review.opendev.org/c/openstack/kolla/+/904576 | 11:52 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix neutron DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/905852 | 12:35 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Test designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 12:35 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test neutron DNS integration and designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 13:02 |
| opendevreview | Uwe Jäger proposed openstack/kolla-ansible master: Update skyline configuration to enable sso https://review.opendev.org/c/openstack/kolla-ansible/+/905860 | 13:02 |
| opendevreview | Uwe Jäger proposed openstack/kolla-ansible master: Allow customizing of skyline configuration https://review.opendev.org/c/openstack/kolla-ansible/+/905861 | 13:03 |
| opendevreview | Uwe Jäger proposed openstack/kolla-ansible master: Update skyline configuration to enable sso https://review.opendev.org/c/openstack/kolla-ansible/+/905860 | 13:15 |
| opendevreview | Bartosz Bezak proposed openstack/kolla-ansible master: use docker_custom_config override for Kolla CI upgrade jobs https://review.opendev.org/c/openstack/kolla-ansible/+/905858 | 13:37 |
| bbezak | mgoddard mnasiadka bbezak frickler kevko SvenKieske mmalchuk gkoper jangutter jsuazo jovial osmanlicilegi mattcrees Kolla meeting in 10 minutes | 13:50 |
| bbezak | #startmeeting kolla | 14:00 |
| opendevmeet | Meeting started Wed Jan 17 14:00:17 2024 UTC and is due to finish in 60 minutes. The chair is bbezak. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 14:00 |
| bbezak | #topic rollcall | 14:00 |
| mmalchuk | \o | 14:00 |
| SvenKieske | o/ | 14:00 |
| mhiner | o/ | 14:00 |
| frickler | \o | 14:00 |
| jangutter | o/ | 14:01 |
| mattcrees | o/ | 14:02 |
| bbezak | #topic agenda | 14:02 |
| bbezak | * Announcements | 14:02 |
| bbezak | * Review action items from the last meeting | 14:02 |
| bbezak | * CI status | 14:02 |
| bbezak | * Release tasks | 14:02 |
| bbezak | * Regular stable releases (first meeting in a month) | 14:02 |
| bbezak | * Current cycle planning | 14:02 |
| bbezak | * Additional agenda (from whiteboard) | 14:02 |
| bbezak | * Open discussion | 14:02 |
| bbezak | #topic CI status | 14:03 |
| bbezak | Kolla master is red | 14:03 |
| bbezak | fixing it for now with https://review.opendev.org/c/openstack/kolla-ansible/+/905858 | 14:03 |
| bbezak | however I don't think it is a final approach | 14:03 |
| bbezak | as we're using the same var for configuring docker daemon and container images source for kolla-ansible. That maybe not the best approach | 14:04 |
| frickler | do you have a patch up in kolla to verify your fix? | 14:04 |
| bbezak | it is only seen in Kolla CI invoked k-a upgrade jobs | 14:04 |
| SvenKieske | sounds like a good observation to me | 14:04 |
| bbezak | https://review.opendev.org/c/openstack/kolla/+/904576 | 14:05 |
| frickler | thx. and I agree splitting vars sounds reasonable | 14:05 |
| bbezak | upgrade jobs finished there, however I've updated commit message, so we need to wait a bit for them to finish second time https://zuul.opendev.org/t/openstack/status#904576 | 14:05 |
| bbezak | hopefully we can fix it soon for now | 14:06 |
| bbezak | and come with more general follow up next | 14:07 |
| bbezak | #topic Release tasks | 14:07 |
| bbezak | as we wanted to go back to monthly releases, I've pushed most of stable backports last week | 14:07 |
| bbezak | and I want to wait with monthly release after this CI fix from above | 14:08 |
| bbezak | will raise release patch as soon as we merge it | 14:08 |
| bbezak | I think at some point we need to update this spreadsheet - https://docs.google.com/spreadsheets/d/1MS2KEhr5yMXEjD5ywU8peJomBNT0Va8bN8wy8hvodVY/edit#gid=0 | 14:09 |
| kevko | \o | 14:09 |
| bbezak | will add a note for it | 14:10 |
| frickler | what's in that sheet? I don't like to open a google doc | 14:10 |
| SvenKieske | I didn't even know this spreadsheet is a thing, until now. | 14:10 |
| bbezak | link to it is in the kolla whiteboard | 14:11 |
| bbezak | https://etherpad.opendev.org/p/KollaWhiteBoard#L156 | 14:11 |
| bbezak | I'll ask mnasiadka about it when he is back in couple of weeks | 14:11 |
| bbezak | #topic Current cycle planning | 14:12 |
| bbezak | looking at caracal priorities. I've seen that ravlew started working on his ovn by default enablement | 14:13 |
| bbezak | of course that also need a CI revamp a bit, so it'll take some time | 14:13 |
| frickler | and /me started discussing about details a bit ;) | 14:13 |
| bbezak | cool | 14:14 |
| frickler | #link https://review.opendev.org/c/openstack/kolla-ansible/+/904959 | 14:14 |
| frickler | just for reference | 14:14 |
| bbezak | thx frickler | 14:14 |
| bbezak | just pasted to the whiteboard https://etherpad.opendev.org/p/KollaWhiteBoard#L233 | 14:15 |
| bbezak | if anybody have already some patches for priorities please put them in the whiteboard | 14:15 |
| mattcrees | I've got a couple of bugfixes on the go, will add them there | 14:16 |
| bbezak | just add your nickname there that we're aware who put them :) | 14:17 |
| mattcrees | Sure thing | 14:18 |
| SvenKieske | I put https://review.opendev.org/c/openstack/kolla-ansible/+/882497 there additionally, want to introduce shellcheck to bring some sanity to our bash scripts ;) need to resolve conflicts and tests though (again) | 14:19 |
| bbezak | great, if anobody have something already half-baked from priorities and wanted to discuss the best approach please post questions in irc/changes | 14:19 |
| SvenKieske | reviews would still be nice, also some openstack core test now fails, which I still haven't figured out why | 14:19 |
| bbezak | thx SvenKieske. I haven't look into that change yet, will do some reading | 14:21 |
| bbezak | #topic Additional agenda (from whiteboard) | 14:21 |
| bbezak | frickler Stop/disable/cleanup services | 14:22 |
| SvenKieske | some users wanted to extend the skyline configurability, I added patches they supplied to the whiteboard just now | 14:22 |
| frickler | mnasiadka helpfully added some links to prior art. I need to check those | 14:22 |
| bbezak | thx SvenKieske | 14:23 |
| frickler | nothing more to add regarding cleanup from me at this point | 14:23 |
| bbezak | that would be nice feature, thx frickler | 14:23 |
| bbezak | frickler Deploy split glance services | 14:24 |
| bbezak | some volunteer needed for that | 14:24 |
| bbezak | will add it to priorities | 14:24 |
| SvenKieske | I'm sorry, but can someone point me to the correct line on our whiteboard for that? I'm unable to find it.. (need some sleep I guess :D) | 14:25 |
| frickler | I would do that, but if anyone wants to help, that would be nice | 14:25 |
| frickler | L66 | 14:25 |
| SvenKieske | found it, ty | 14:25 |
| bbezak | https://etherpad.opendev.org/p/KollaWhiteBoard#L66 | 14:25 |
| frickler | so I'll add that to the priorities list, too. seems it is getting pretty crowded | 14:27 |
| bbezak | I don't think we should add regular bugfixes to priorites | 14:27 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/yoga: Switch IPA builds to CentOS Stream 9 for yoga https://review.opendev.org/c/openstack/kayobe/+/903242 | 14:27 |
| bbezak | just ping reviewers and cores to review :) | 14:28 |
| frickler | well it may depend on how review intensive they are | 14:28 |
| frickler | but I agree in general | 14:28 |
| mattcrees | Fair enough, I'll take them back off | 14:28 |
| frickler | also set RP+1 | 14:29 |
| bbezak | exactly | 14:29 |
| bbezak | L70 we already covered | 14:29 |
| bbezak | mhiner Container engine migration | 14:30 |
| mhiner | I would to reach some agreement about the CI tests | 14:31 |
| frickler | I'll need to look at the patch mentioned there and the logs | 14:31 |
| frickler | #link https://review.opendev.org/c/openstack/kolla-ansible/+/836941 | 14:31 |
| SvenKieske | in general you can also ping me for reviews, I'll scream if it's too much ;) | 14:31 |
| mhiner | alright | 14:32 |
| mhiner | additionally i would like for someone to take a look at ansible/roles/container-engine-migration/tasks/cleanup.yml | 14:33 |
| bbezak | I haven't look into that change too much unfortunately. lack of logs after migration is pretty important to look into | 14:33 |
| bbezak | will try to look into it | 14:33 |
| mhiner | first task which deletes old container engine files produces massive logs (80MB, 300K lines) | 14:33 |
| frickler | using no_log seems reasonable in that case | 14:34 |
| mhiner | is it okay? for now I added no_log: true to suppress it | 14:34 |
| SvenKieske | +1 | 14:34 |
| bbezak | looks pretty scary if some task before will not work as intended :) | 14:35 |
| SvenKieske | yeah, the best option would be if this could be filtered or split up somehow, but I don't have looked into the details if that is even possible? | 14:36 |
| frickler | the other option would be to leave cleanup as a manual task I guess? just don't add another -yes-i-really-mean-to-copy-this-option-from-the-docs-without-understanding-it | 14:36 |
| bbezak | as a CLI command | 14:37 |
| mhiner | that is also a option | 14:37 |
| mhiner | all the volumes will be moved so theoretically all that is left are some configs from the old container engine | 14:37 |
| frickler | if it is just some configs, why 300k lines? | 14:39 |
| SvenKieske | can we split this cleanup task in multiple smaller tasks? maybe it's only a few "cleanup" items that account for the majority of the 300k lines? | 14:40 |
| mhiner | yeah, it's not just configs, my bad | 14:40 |
| mhiner | I still have those logs saved and majority of lines come from deletion in /var/lib/docker/overlay2/ | 14:41 |
| bbezak | I've added RP1 for this change | 14:42 |
| SvenKieske | I guess one line per file/symlink? maybe we can collapse this somehow | 14:42 |
| mhiner | yes, that's the case | 14:42 |
| frickler | use "command: rf -rf" in this case instead of "file: state: absent", even if that's non-ansiblic? | 14:43 |
| bbezak | well, removal of whole directory | 14:43 |
| bbezak | above would solve it | 14:43 |
| bbezak | I mean the current engine main path | 14:43 |
| frickler | anyway, maybe we can continue this discussion on the review after having looked at this in more detail? | 14:44 |
| bbezak | if that's what we want to achieve | 14:44 |
| bbezak | yeah | 14:44 |
| bbezak | #topic Open discussion | 14:44 |
| SvenKieske | yeah, I was also thinking about something like `find -bla -delete` and only logging errors from that | 14:45 |
| frickler | I have one topic regarding yoga eol | 14:45 |
| frickler | iiuc so far we have been waiting for the release team to finish the automation for the new "unmaintained" state | 14:46 |
| frickler | but it seems we could move to eol right now regardless of that, do we want that? or does someone still want to keep yoga around? | 14:46 |
| bbezak | we still got several clouds on yoga. however definitely we should go to EOL yoga in this cycle | 14:47 |
| bbezak | or next at most | 14:48 |
| bbezak | imho | 14:48 |
| bbezak | as yoga is a stepping stone for CS8/RL9 | 14:48 |
| SvenKieske | I think there are still some patches for yoga in flight, I guess that list could get updated and we should maybe ask to not propose any new patches for yoga then? | 14:48 |
| bbezak | and such migration is taking a while | 14:48 |
| SvenKieske | on the other hand I don't know if there is actual harm of patches being proposed and/or merged in yoga | 14:48 |
| bbezak | yoga is somewhat a LTS release from CS8/RL9 migration perspective | 14:49 |
| SvenKieske | to me it feels it should be the other way around, honestly. Maybe I don't have a good enough understanding of the release process. There should be three phases (like debian): | 14:49 |
| frickler | imo it is just useless extra work for reviewers, but I can also just ignore reviews for that branch | 14:50 |
| bbezak | it's rather a supporting burden | 14:50 |
| bbezak | but good point frickler, we should do EOL of yoga soon'ish | 14:50 |
| SvenKieske | 1. declare feature/patch freeze (no new patches proposed); maybe add a tag for that on the branch 2. merge existing patches 3. declare EOL via Tag | 14:50 |
| bbezak | I need to check of status of unmaintained branches | 14:51 |
| SvenKieske | currently we do: 1. merge existing patches 2. go back to 1. until there are no patches anymore 3. declare EOL via Tag | 14:51 |
| frickler | those three things should happen in quick succession IMO, no need to make explicit phases for each | 14:51 |
| opendevreview | Pierre Riteau proposed openstack/kolla-ansible master: Drop more remnants of install_type https://review.opendev.org/c/openstack/kolla-ansible/+/905880 | 14:52 |
| SvenKieske | well I guess a problem is that new patches are faster to propose than existing patches getting merged | 14:52 |
| bbezak | ok, we've got that on our radar then | 14:53 |
| frickler | that just needs some concerted reviewer effort. another option is to just abandon unmerged patches | 14:53 |
| bbezak | yeah, we're usually either merging or abandoning unmerged patches when doing EOL | 14:53 |
| bbezak | and release team is checking if there are no patches awaiting in queue | 14:54 |
| SvenKieske | There should be a clear cutoff date or other criteria as a patch submitter so I can know if my patch will be merged, no? | 14:54 |
| bbezak | ok, anything else to discuss? | 14:54 |
| frickler | so maybe I can propose a release patch for that, but wip it until more feedback comes in and maybe mnasiadka is back, too | 14:54 |
| bbezak | yeah, I don't want to do EOL without PTL around | 14:54 |
| bbezak | ok, thx frickler | 14:55 |
| SvenKieske | maybe it's better if I pester the release team with this stuff? | 14:55 |
| SvenKieske | I find it problematic, that there is this kind of no-mans-land/twilight zone, where it's not clear if patches will be merged or get abandoned last minute. | 14:56 |
| frickler | usually the predicted dates in the release schedule work well. the current change in methods has sadly lead to delays | 14:56 |
| frickler | you never can be sure whether a patch gets merged or rejected anyway | 14:57 |
| SvenKieske | alright, if it's only a one-off event I'm fine with keeping everything like it is. I'll watch what happens the next EOL release cycle then :) | 14:57 |
| bbezak | ok, let's wait for making decisions for PTL to be back from vacation. I'll add a note for that | 14:57 |
| bbezak | ok I think we're done for today, thank your for joining | 14:58 |
| bbezak | #endmeeting | 14:58 |
| opendevmeet | Meeting ended Wed Jan 17 14:58:24 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:58 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-01-17-14.00.html | 14:58 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-01-17-14.00.txt | 14:58 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-01-17-14.00.log.html | 14:58 |
| SvenKieske | thank you for taking over for the PTL :) | 14:58 |
| frickler | thx bbezak | 14:58 |
| bbezak | np | 14:59 |
| mmalchuk | thanks bbezak | 15:00 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible master: Configuring nova-compute to support vendordata over configdrive. https://review.opendev.org/c/openstack/kolla-ansible/+/905843 | 15:21 |
| opendevreview | Uwe Jäger proposed openstack/kolla-ansible master: Update skyline configuration to enable sso https://review.opendev.org/c/openstack/kolla-ansible/+/905860 | 15:57 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix neutron DNS integration https://review.opendev.org/c/openstack/kolla-ansible/+/905852 | 16:04 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test neutron DNS integration and designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 16:04 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/yoga: Switch IPA builds to CentOS Stream 9 for yoga https://review.opendev.org/c/openstack/kayobe/+/903242 | 16:47 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test neutron DNS integration and designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 16:55 |
| opendevreview | Piotr Parczewski proposed openstack/kolla-ansible master: Adjust Ceph metrics scrape interval in Prometheus https://review.opendev.org/c/openstack/kolla-ansible/+/902129 | 20:40 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible master: Configure nova-compute to support exposing vendordata over configdrive https://review.opendev.org/c/openstack/kolla-ansible/+/905843 | 22:00 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!