mnasiadka | frickler, bbezak, kevko, SvenKieske[m]: https://review.opendev.org/c/openstack/kolla/+/915952 - OpenSearch 2.13 fixed | 05:50 |
---|---|---|
opendevreview | Michal Nasiadka proposed openstack/kolla master: Add ovn-bgp-agent / FRR / Horizon BGPVPN dashboard https://review.opendev.org/c/openstack/kolla/+/891617 | 05:55 |
opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: DNM: Debug Octavia cascade delete fail https://review.opendev.org/c/openstack/kolla-ansible/+/916037 | 06:12 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: horizon/neutron: Add FWaaS plugin https://review.opendev.org/c/openstack/kolla/+/914855 | 06:15 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: horizon/neutron: Add FWaaS plugin https://review.opendev.org/c/openstack/kolla/+/914855 | 06:16 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: prometheus: use inovex/prometheus-libvirt-exporter https://review.opendev.org/c/openstack/kolla/+/909507 | 06:19 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: kolla-toolbox: Bump ansible-core to 2.16 https://review.opendev.org/c/openstack/kolla/+/914744 | 06:20 |
SvenKieske | o/ from a new computer device | 07:52 |
SvenKieske | rebooting.. | 08:18 |
SvenKieske | interesting, don't we still support python 3.8? | 08:52 |
frickler | why should we? that was last used in focal iirc | 08:54 |
SvenKieske | because I get an error on a focal based install; just wanted to check "tox -e linters" and can't seem to find a suitable ansible version that supports python 3.8? | 08:55 |
SvenKieske | need to upgrade that wsl stuff I guess.. | 08:55 |
SvenKieske | at least it seems windows linux stuff does support the same dist upgrade mechanism as upstream | 08:57 |
mnasiadka | we don't support focal :) | 08:57 |
SvenKieske | frickler: I just thought that python 3.8. was in general still supported in wider openstack, I don't pay usually attention because I run almost latest python via fedora | 08:57 |
SvenKieske | but my replacement machine here is windows based and the default linux vm there seems to be still stuck on focal :D | 08:58 |
SvenKieske | apt upgrade works though | 08:58 |
opendevreview | Merged openstack/kolla master: Fix OpenSearch file ownership issues https://review.opendev.org/c/openstack/kolla/+/915981 | 09:16 |
opendevreview | Michal Nasiadka proposed openstack/kolla stable/2023.2: Fix OpenSearch file ownership issues https://review.opendev.org/c/openstack/kolla/+/916081 | 09:23 |
opendevreview | Michal Nasiadka proposed openstack/kolla stable/2023.1: Fix OpenSearch file ownership issues https://review.opendev.org/c/openstack/kolla/+/916082 | 09:25 |
opendevreview | Michal Nasiadka proposed openstack/kolla stable/zed: Fix OpenSearch file ownership issues https://review.opendev.org/c/openstack/kolla/+/916083 | 09:25 |
SvenKieske | mnasiadka: https://bugs.launchpad.net/kolla/+bug/2061877 this looks like an upstream bug, no? is this reported at opensearch already? | 09:25 |
SvenKieske | I agree with fixing it in the mean time though :) thanks | 09:25 |
mnasiadka | SvenKieske: it is - https://github.com/opensearch-project/OpenSearch-Dashboards/issues/6484 | 09:27 |
mnasiadka | but I doubt they are going to fix that before 2.14 | 09:27 |
mnasiadka | so that's why we fix that on our side | 09:28 |
SvenKieske | sure, thanks for the link | 09:28 |
SvenKieske | mhm now flake8 fails locally, I guess that's a local only problem. | 09:44 |
mnasiadka | frickler: https://review.opendev.org/c/openstack/kolla-ansible/+/915435 - codespell k-a part looks good as well now | 09:51 |
SvenKieske | do we backport this? | 09:55 |
opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Patch service-cert-copy role to be used w/o HAProxy https://review.opendev.org/c/openstack/kolla-ansible/+/915901 | 10:00 |
opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Patch service-cert-copy role to be used w/o HAProxy https://review.opendev.org/c/openstack/kolla-ansible/+/915901 | 10:00 |
mnasiadka | SvenKieske: probably not, lot of changes to my taste ;) | 10:28 |
mnasiadka | Or maybe for the sake of renos | 10:29 |
SvenKieske | but doesn't this have the same backporting issues like introducing any other linter? weird to say the least. | 10:32 |
matusjenca | Hello. Can someone have look at this small patch? It's related to the Redis and ProxySQL TLS patches I made earlier https://review.opendev.org/c/openstack/kolla-ansible/+/915901 | 10:45 |
matusjenca | Thanks in advance. | 10:45 |
opendevreview | Verification of a change to openstack/kolla master failed: Revert "CI/Master only: pin opensearch{-dashboards}" https://review.opendev.org/c/openstack/kolla/+/915952 | 10:53 |
SvenKieske | mnasiadka: ah just reread your opensearch debugging story from yesterday :D..glad it's "only" an upstream owner problem | 11:20 |
opendevreview | Grzegorz Bialas proposed openstack/kolla master: add scaphandre image https://review.opendev.org/c/openstack/kolla/+/914646 | 11:30 |
opendevreview | Merged openstack/kolla master: kolla-toolbox: Bump ansible-core to 2.16 https://review.opendev.org/c/openstack/kolla/+/914744 | 12:25 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: kolla-toolbox: Use py311 on rpm distros https://review.opendev.org/c/openstack/kolla/+/916102 | 12:32 |
opendevreview | Merged openstack/kolla master: prometheus: use inovex/prometheus-libvirt-exporter https://review.opendev.org/c/openstack/kolla/+/909507 | 12:35 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: kolla-toolbox: Use py311 on rpm distros https://review.opendev.org/c/openstack/kolla/+/916102 | 12:40 |
SvenKieske | the python 3.12 experimental jobs where removed again? Sorry I was on a hackathon and my laptop broke down so maybe I missed something. | 12:53 |
SvenKieske | we do have a meeting in a few minutes, don't we? :) | 12:53 |
mnasiadka | yeah | 12:56 |
mnasiadka | mgoddard mnasiadka bbezak frickler kevko SvenKieske mmalchuk gkoper jangutter jsuazo jovial osmanlicilegi mattcrees dougszu - meeting in 4 minutes | 12:56 |
mnasiadka | #startmeeting kolla | 13:00 |
opendevmeet | Meeting started Wed Apr 17 13:00:01 2024 UTC and is due to finish in 60 minutes. The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot. | 13:00 |
opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 13:00 |
opendevmeet | The meeting name has been set to 'kolla' | 13:00 |
mnasiadka | #topic rollcall | 13:00 |
mnasiadka | o/ | 13:00 |
mhiner | o/ | 13:00 |
ihalomi | \o | 13:00 |
SvenKieske | o/ | 13:00 |
frickler | \o | 13:01 |
mnasiadka | #topic agenda | 13:02 |
mnasiadka | * CI status | 13:02 |
mnasiadka | * Release tasks | 13:02 |
mnasiadka | * Regular stable releases (first meeting in a month) | 13:02 |
mnasiadka | * Current cycle planning | 13:02 |
mnasiadka | * Additional agenda (from whiteboard) | 13:02 |
mnasiadka | * Open discussion | 13:02 |
mnasiadka | #topic CI status | 13:02 |
mnasiadka | Basically all green - apart the gnocchi issue that we fixed (and Kolla is now using Caracal sources for building) | 13:03 |
mnasiadka | And Octavia has problems tearing down an amphora LB - claims Nova did not free up the port | 13:03 |
mnasiadka | I'm having a slow look into that | 13:03 |
mnasiadka | #topic Release tasks | 13:03 |
mnasiadka | So basically - we should start preparing Kolla and Kolla-Ansible for branching | 13:04 |
mnasiadka | I'd like to squeeze ovn-bgp-agent in | 13:04 |
mnasiadka | There are patches for adding Scaphandre (energy consumption monitoring for processes/instances) which would be also nice to get in | 13:04 |
mnasiadka | So I'll be running around asking for reviews surely and forming up a queue of RP+1 things that need to get in | 13:05 |
mnasiadka | I'll also create a list on the Whiteboard - so we can track that properly | 13:05 |
SvenKieske | nice, I guess we would also happily review scaphandre, as we already use that downstream | 13:06 |
frickler | +1 | 13:06 |
mnasiadka | Goodie | 13:06 |
mnasiadka | Let's move on | 13:06 |
mnasiadka | #topic Current cycle planning | 13:06 |
mnasiadka | vPTG was last week, I'll formulate some priorities for D cycle - but for now the focus is getting closer to releasing Caracal | 13:07 |
mnasiadka | From another front - our meeting is 13:00 UTC now - any volunteer to update the meetings page for summer daylight savings period? | 13:07 |
frickler | didn't we do that? I updated the channel topic at least, can also push another patch | 13:09 |
SvenKieske | I'm currently sorting out my local dev setup issues, else I would say I can do it, but I have various issues due to my fallback machine being windows based :( | 13:10 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: kolla-toolbox: Use py311 on rpm distros https://review.opendev.org/c/openstack/kolla/+/916102 | 13:10 |
opendevreview | Dawud proposed openstack/kolla master: Change logic for plugins in grafana image build https://review.opendev.org/c/openstack/kolla/+/913184 | 13:10 |
mnasiadka | frickler: https://meetings.opendev.org/#Kolla_Team_Meeting says 14:00 UTC | 13:11 |
frickler | ok, will update | 13:11 |
mnasiadka | thanks! | 13:11 |
mnasiadka | #topic Additional agenda (from whiteboard) | 13:11 |
mnasiadka | Let's go through that, it's loooong | 13:11 |
mnasiadka | (SvenKieske): https://review.opendev.org/c/openstack/kolla-ansible/+/909912/comment/2da918f8_2a5d1097/ is a complete copy of "service-cert-copy" role ok for one task, or should it rather be extended instead? | 13:11 |
mnasiadka | SvenKieske: I think there's already separate patch for that? | 13:12 |
SvenKieske | that was actually addressed I believe, there is a patchset | 13:12 |
SvenKieske | yes | 13:12 |
SvenKieske | but it could get some reviews :) | 13:12 |
mnasiadka | (SvenKieske): cores: please review the following: | 13:12 |
mnasiadka | ovn-exporter https://review.opendev.org/c/openstack/kolla-ansible/+/855498 | 13:12 |
mnasiadka | skyline SSO https://review.opendev.org/c/openstack/kolla-ansible/+/905860 | 13:12 |
mnasiadka | neutron service limit https://review.opendev.org/c/openstack/kolla-ansible/+/877776 | 13:12 |
mnasiadka | add new sysctl role: https://review.opendev.org/c/openstack/kolla-ansible/+/912351 | 13:12 |
mnasiadka | cell0 VIP change fix https://review.opendev.org/c/openstack/kolla-ansible/+/910924 | 13:12 |
mnasiadka | oh boy | 13:12 |
frickler | and that's already only half of what was there on Monday ;) | 13:13 |
SvenKieske | skyline, sysctl and afaik neutron service limit are rather uncontroversial imho :) | 13:13 |
SvenKieske | especially the skyline stuff would be nice for downstream users using that, also it would be nice for the new contributors seeing there patches being merged sometime :D | 13:14 |
SvenKieske | the sysctl is nice to have i guess, not that important | 13:14 |
mnasiadka | ok, I think all of them except ovn-exporter have my +2 | 13:14 |
SvenKieske | the cell0 stuff depends on how much we expect kolla vip to change | 13:15 |
mnasiadka | ovn-exporter is failing all over the place | 13:15 |
SvenKieske | yeah I need to triple check something about ovn, but my local test env has currently issues :( will report back | 13:15 |
mnasiadka | usually we don't, but there's no sense in breaking people doing that, if there's a contributor willing to fix it | 13:15 |
mnasiadka | ok then, next list | 13:16 |
mnasiadka | (mhiner) status of adding python3-docker to ospbo repository | 13:16 |
mnasiadka | please review: https://review.opendev.org/c/openstack/kolla-ansible/+/911417 | 13:16 |
mnasiadka | and its following change: https://review.opendev.org/c/openstack/kolla-ansible/+/911505 | 13:16 |
mnasiadka | plus the podman fix: https://review.opendev.org/c/openstack/kolla-ansible/+/912521 | 13:16 |
mnasiadka | this one is also ready for reviews, it fails only because of docker--py 5.0.3 version on Debian: https://review.opendev.org/c/openstack/kolla-ansible/+/908295 | 13:16 |
mhiner | Podman fix is merged, thanks for that | 13:16 |
opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add backend TLS between MariaDB and ProxySQL https://review.opendev.org/c/openstack/kolla-ansible/+/909912 | 13:17 |
opendevreview | Matt Crees proposed openstack/kayobe master: Docs: add a link to the gerrit review dashboard https://review.opendev.org/c/openstack/kayobe/+/916110 | 13:17 |
ihalomi | few weeks ago we had discussion about adding python3-docker to ospbo repository or changing the way it is installed on the newest debian | 13:18 |
ihalomi | any news about it? | 13:18 |
mnasiadka | ihalomi: can you reply to the comments in 911417? | 13:18 |
mnasiadka | and resolve those that are answered ; | 13:18 |
mnasiadka | ;) | 13:18 |
SvenKieske | also https://review.opendev.org/c/openstack/kolla-ansible/+/911417 needs a manual rebase, just checked | 13:18 |
ihalomi | mnasiadka: i did, i was waiting for SvenKieske to close it or add more comments, since they are his suggestions | 13:19 |
mnasiadka | So what about 908295 ? | 13:19 |
SvenKieske | I did an initial review of that, I'll do another round today. | 13:20 |
mnasiadka | It's failing | 13:21 |
SvenKieske | it's quite some amount of code | 13:21 |
ihalomi | the tests on 908295 is failing only because on debian there is no way to install newer version of docker | 13:21 |
ihalomi | we talked about this few weeks ago that on debian we dont use pip to install docker for some reason | 13:21 |
SvenKieske | ah it would've been nice to note that on the change, if you know the reason. | 13:21 |
mnasiadka | well, we don't use venvs for some reason | 13:22 |
SvenKieske | that being said I faintly recall the discussion, wasn't there some agreement to make this the same for all distros? not sure what the outcome was, I remember some experimental patch in that regard at least. | 13:22 |
mnasiadka | anyway, it seems like this is not going to make it to C, unless somebody has a lot of time to make the CI work in venvs (because Debian without venv will install python3-docker deb package) | 13:23 |
ihalomi | there was 2 suggestions, one to make it same for all distros and other was to add python3-docker to repository we use for podman | 13:23 |
opendevreview | Roman Krček proposed openstack/kolla-ansible master: Optimise config https://review.opendev.org/c/openstack/kolla-ansible/+/745164 | 13:24 |
opendevreview | Roman Krček proposed openstack/kolla-ansible master: Refactor services' check-containers and optimise https://review.opendev.org/c/openstack/kolla-ansible/+/773243 | 13:24 |
mnasiadka | the same for all distros == use venv for CI, but then we would need to force users to use venv | 13:24 |
mnasiadka | Debian does not support using pip to install package globally | 13:24 |
ihalomi | look at approach for podman https://github.com/openstack/ansible-collection-kolla/blob/master/roles/podman_sdk/tasks/main.yml#L4 it adds this osbpo repository and install podman from there for debian | 13:25 |
frickler | that's the future for other distros, too | 13:25 |
ihalomi | then we would have control over which version of docker or podman we use also for non pip installs | 13:26 |
SvenKieske | afaik kevko said he can fix osbpo, no problem? (for the short term) but he's not here today it seems | 13:26 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: kolla-toolbox: Use py311 on rpm distros https://review.opendev.org/c/openstack/kolla/+/916102 | 13:27 |
opendevreview | Roman Krček proposed openstack/kolla-ansible master: Refactor services' check-containers and optimise https://review.opendev.org/c/openstack/kolla-ansible/+/773243 | 13:27 |
mnasiadka | well, then you need to chase kevko | 13:28 |
mnasiadka | let's move on | 13:28 |
mnasiadka | (r-krcek) | 13:28 |
mnasiadka | thoughts on https://review.opendev.org/c/openstack/kolla-ansible/+/914997 ? | 13:28 |
opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Patch service-cert-copy role to be used w/o HAProxy https://review.opendev.org/c/openstack/kolla-ansible/+/915901 | 13:29 |
mnasiadka | It's failing for now, I rechecked that - should be probably better | 13:30 |
mnasiadka | and the last one is | 13:30 |
mnasiadka | (mgoddard): Masakari CI is passing on all branches back to stable/zed. Can we de-deprecate it? | 13:30 |
SvenKieske | well tbh I went - quickly - over the linked markdown document and the performance benefit really doesn't seem to stand out. could you maybe provide some more numbers there? afaik even some tests regressed in performance? | 13:30 |
SvenKieske | I don't doubt there might be performance benefits, but I think it should be possible to show them in a more convincing fashion :) | 13:31 |
mnasiadka | yeah, I asked in the review | 13:31 |
mnasiadka | so, Masakari | 13:31 |
mnasiadka | I think it's about this: https://review.opendev.org/c/openstack/kolla-ansible/+/914877 | 13:31 |
mnasiadka | And then reverting this: https://review.opendev.org/c/openstack/kolla-ansible/+/901193 | 13:31 |
frickler | so shpc want's to have that downstream and support it? | 13:32 |
mnasiadka | One of our customers is using that extensively and we have stepped up to make sure that project upstream is viable | 13:37 |
opendevreview | Merged openstack/kolla stable/2023.2: Fix OpenSearch file ownership issues https://review.opendev.org/c/openstack/kolla/+/916081 | 13:37 |
frickler | ok, I don't mind doing that, will add to my review list | 13:39 |
mnasiadka | Ok then, lists of patches are done | 13:39 |
opendevreview | Verification of a change to openstack/kolla stable/2023.1 failed: Fix OpenSearch file ownership issues https://review.opendev.org/c/openstack/kolla/+/916082 | 13:39 |
mnasiadka | #topic Open discussion | 13:39 |
mnasiadka | anybody? | 13:39 |
r-krcek | Hi. I was here at the wrong time and missed my spot for the performace upgrade regarding the use or filters. | 13:40 |
r-krcek | Any thoughts on this https://review.opendev.org/c/openstack/kolla-ansible/+/914997 | 13:41 |
mnasiadka | we discussed that shortly, first of all it needs to pass CI - second of all - we'd like some marketing information how much faster is that | 13:41 |
mnasiadka | (in Kolla-Ansible case, not some general case) | 13:41 |
SvenKieske | sure, as I said above - and also commented on the patchset - maybe run some more stats to show the real benefit there. also please account for caching et cetera. I'm missing some basic description describing the benchmark environment (virtual machine, software stack with versions et cetera). good benchmarking is very very hard. | 13:42 |
r-krcek | Okay, I will try to come up with some "real-world" scenario and put the specific number into the comments of the patchset. | 13:43 |
SvenKieske | doesn't even need to be real world, but should - in the best case - be reproducible, not just some fluctuation in local dev env :) | 13:43 |
SvenKieske | I usually refer to this when it comes to benchmarking, it has many do's and dont's and many pitfalls covered: https://www.brendangregg.com/activebenchmarking.html | 13:45 |
SvenKieske | and you don't need to do all that is advised there, but it can give you some ideas what to look for :) | 13:45 |
r-krcek | Thanks for the tip :) | 13:46 |
SvenKieske | thanks, looking forward to a v2 of that performance document :) better performance is surely greatly appreciated by all :) | 13:47 |
SvenKieske | you can ping me also most of the time here if you have a question or general feedback in this regard. I did quite some benchmarking in the past and did many of the common errors myself :D | 13:48 |
r-krcek | Thank you, that would be great. I'll be sure to reach out, once I have some more questions/updates. :) | 13:49 |
mnasiadka | good | 13:50 |
mnasiadka | ok then, let's finish for today | 13:50 |
mnasiadka | Thanks for coming! | 13:51 |
mnasiadka | #endmeeting | 13:51 |
opendevmeet | Meeting ended Wed Apr 17 13:51:04 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 13:51 |
opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-04-17-13.00.html | 13:51 |
opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-04-17-13.00.txt | 13:51 |
opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-04-17-13.00.log.html | 13:51 |
frickler | thx mnasiadka | 13:51 |
SvenKieske | r-krcek: here are some debug tools being mentioned which can record execution time of certain stuff in ansible, maybe you already know them, they are quite popular I think: https://www.redhat.com/sysadmin/faster-ansible-playbook-execution | 13:52 |
PrzemekK | How prevent kolla-ansible create entries in /etc/hosts ? I want to use dns server instead | 14:03 |
mnasiadka | PrzemekK: https://github.com/openstack/ansible-collection-kolla/blob/877f91f500897640494b4e5672345da8a710d3d1/roles/etc_hosts/defaults/main.yml#L3 | 14:05 |
PrzemekK | During upgrade from 2023.1 to 2023.2 i can see that there are some issues that repeats in every eviroment example https://bugs.launchpad.net/kolla-ansible/+bug/2060823 or prometheus salt is "too long" after merge old with new passwords . There is need to reset docker volume for Rabbitmq queues needs to be manually upgraded to cluster . There is need to delete docker volume for grafana/gnocchi | 14:09 |
PrzemekK | rabbitmq upgrade process trying to check enabled features but images are still 2023.1 and it fails | 14:10 |
PrzemekK | thx mnasiadka | 14:12 |
opendevreview | Martin Hiner proposed openstack/kolla-ansible master: Add container engine migration scenario https://review.opendev.org/c/openstack/kolla-ansible/+/836941 | 14:43 |
opendevreview | Merged openstack/kolla stable/zed: Fix OpenSearch file ownership issues https://review.opendev.org/c/openstack/kolla/+/916083 | 14:46 |
PrzemekK | could You merge this to master/bobcat Refactor external ceph . Its adds one ceph cluster but with different pools/users/cluster names https://review.opendev.org/c/openstack/kolla-ansible/+/907166 | 14:46 |
opendevreview | Ivan Halomi proposed openstack/kolla-ansible master: Refactor of kolla_container_facts https://review.opendev.org/c/openstack/kolla-ansible/+/911417 | 14:58 |
opendevreview | Merged openstack/kolla master: Revert "CI/Master only: pin opensearch{-dashboards}" https://review.opendev.org/c/openstack/kolla/+/915952 | 15:39 |
opendevreview | Roman Krček proposed openstack/kolla-ansible master: Refactor services' check-containers and optimise https://review.opendev.org/c/openstack/kolla-ansible/+/773243 | 16:26 |
opendevreview | Merged openstack/kolla stable/2023.1: Fix OpenSearch file ownership issues https://review.opendev.org/c/openstack/kolla/+/916082 | 16:32 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: kolla-toolbox: Use py311 on rpm distros https://review.opendev.org/c/openstack/kolla/+/916102 | 17:43 |
opendevreview | Maksim Malchuk proposed openstack/kayobe master: Fix default place for Ubuntu Apt keyrings https://review.opendev.org/c/openstack/kayobe/+/916138 | 19:48 |
opendevreview | Maksim Malchuk proposed openstack/ansible-collection-kolla master: Use deb822 format for Apt repositories on Ubuntu and drop apt-key https://review.opendev.org/c/openstack/ansible-collection-kolla/+/916143 | 21:48 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!