Wednesday, 2024-04-17

mnasiadkafrickler, bbezak, kevko, SvenKieske[m]: https://review.opendev.org/c/openstack/kolla/+/915952 - OpenSearch 2.13 fixed05:50
opendevreviewMichal Nasiadka proposed openstack/kolla master: Add ovn-bgp-agent / FRR / Horizon BGPVPN dashboard  https://review.opendev.org/c/openstack/kolla/+/89161705:55
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: DNM: Debug Octavia cascade delete fail  https://review.opendev.org/c/openstack/kolla-ansible/+/91603706:12
opendevreviewMichal Nasiadka proposed openstack/kolla master: horizon/neutron: Add FWaaS plugin  https://review.opendev.org/c/openstack/kolla/+/91485506:15
opendevreviewMichal Nasiadka proposed openstack/kolla master: horizon/neutron: Add FWaaS plugin  https://review.opendev.org/c/openstack/kolla/+/91485506:16
opendevreviewMichal Nasiadka proposed openstack/kolla master: prometheus: use inovex/prometheus-libvirt-exporter  https://review.opendev.org/c/openstack/kolla/+/90950706:19
opendevreviewMichal Nasiadka proposed openstack/kolla master: kolla-toolbox: Bump ansible-core to 2.16  https://review.opendev.org/c/openstack/kolla/+/91474406:20
SvenKieskeo/ from a new computer device07:52
SvenKieskerebooting..08:18
SvenKieskeinteresting, don't we still support python 3.8?08:52
fricklerwhy should we? that was last used in focal iirc08:54
SvenKieskebecause I get an error on a focal based install; just wanted to check "tox -e linters" and can't seem to find a suitable ansible version that supports python 3.8?08:55
SvenKieskeneed to upgrade that wsl stuff I guess..08:55
SvenKieskeat least it seems windows linux stuff does support the same dist upgrade mechanism as upstream08:57
mnasiadkawe don't support focal :)08:57
SvenKieskefrickler: I just thought that python 3.8. was in general still supported in wider openstack, I don't pay usually attention because I run almost latest python via fedora08:57
SvenKieskebut my replacement machine here is windows based and the default linux vm there seems to be still stuck on focal :D08:58
SvenKieskeapt upgrade works though08:58
opendevreviewMerged openstack/kolla master: Fix OpenSearch file ownership issues  https://review.opendev.org/c/openstack/kolla/+/91598109:16
opendevreviewMichal Nasiadka proposed openstack/kolla stable/2023.2: Fix OpenSearch file ownership issues  https://review.opendev.org/c/openstack/kolla/+/91608109:23
opendevreviewMichal Nasiadka proposed openstack/kolla stable/2023.1: Fix OpenSearch file ownership issues  https://review.opendev.org/c/openstack/kolla/+/91608209:25
opendevreviewMichal Nasiadka proposed openstack/kolla stable/zed: Fix OpenSearch file ownership issues  https://review.opendev.org/c/openstack/kolla/+/91608309:25
SvenKieskemnasiadka: https://bugs.launchpad.net/kolla/+bug/2061877 this looks like an upstream bug, no? is this reported at opensearch already?09:25
SvenKieskeI agree with fixing it in the mean time though :) thanks09:25
mnasiadkaSvenKieske: it is - https://github.com/opensearch-project/OpenSearch-Dashboards/issues/648409:27
mnasiadkabut I doubt they are going to fix that before 2.1409:27
mnasiadkaso that's why we fix that on our side09:28
SvenKieskesure, thanks for the link09:28
SvenKieskemhm now flake8 fails locally, I guess that's a local only problem.09:44
mnasiadkafrickler: https://review.opendev.org/c/openstack/kolla-ansible/+/915435 - codespell k-a part looks good as well now09:51
SvenKieskedo we backport this?09:55
opendevreviewMatúš Jenča proposed openstack/kolla-ansible master: Patch service-cert-copy role to be used w/o HAProxy  https://review.opendev.org/c/openstack/kolla-ansible/+/91590110:00
opendevreviewMatúš Jenča proposed openstack/kolla-ansible master: Patch service-cert-copy role to be used w/o HAProxy  https://review.opendev.org/c/openstack/kolla-ansible/+/91590110:00
mnasiadkaSvenKieske: probably not, lot of changes to my taste ;)10:28
mnasiadkaOr maybe for the sake of renos10:29
SvenKieskebut doesn't this have the same backporting issues like introducing any other linter? weird to say the least.10:32
matusjencaHello. Can someone have look at this small patch? It's related to the Redis and ProxySQL TLS patches I made earlier https://review.opendev.org/c/openstack/kolla-ansible/+/91590110:45
matusjencaThanks in advance.10:45
opendevreviewVerification of a change to openstack/kolla master failed: Revert "CI/Master only: pin opensearch{-dashboards}"  https://review.opendev.org/c/openstack/kolla/+/91595210:53
SvenKieskemnasiadka: ah just reread your opensearch debugging story from yesterday :D..glad it's "only" an upstream owner problem11:20
opendevreviewGrzegorz Bialas proposed openstack/kolla master: add scaphandre image  https://review.opendev.org/c/openstack/kolla/+/91464611:30
opendevreviewMerged openstack/kolla master: kolla-toolbox: Bump ansible-core to 2.16  https://review.opendev.org/c/openstack/kolla/+/91474412:25
opendevreviewMichal Nasiadka proposed openstack/kolla master: WIP: kolla-toolbox: Use py311 on rpm distros  https://review.opendev.org/c/openstack/kolla/+/91610212:32
opendevreviewMerged openstack/kolla master: prometheus: use inovex/prometheus-libvirt-exporter  https://review.opendev.org/c/openstack/kolla/+/90950712:35
opendevreviewMichal Nasiadka proposed openstack/kolla master: WIP: kolla-toolbox: Use py311 on rpm distros  https://review.opendev.org/c/openstack/kolla/+/91610212:40
SvenKieskethe python 3.12 experimental jobs where removed again? Sorry I was on a hackathon and my laptop broke down so maybe I missed something.12:53
SvenKieskewe do have a meeting in a few minutes, don't we? :)12:53
mnasiadkayeah12:56
mnasiadkamgoddard mnasiadka bbezak frickler kevko SvenKieske mmalchuk gkoper jangutter jsuazo jovial osmanlicilegi mattcrees dougszu - meeting in 4 minutes12:56
mnasiadka#startmeeting kolla13:00
opendevmeetMeeting started Wed Apr 17 13:00:01 2024 UTC and is due to finish in 60 minutes.  The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot.13:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:00
opendevmeetThe meeting name has been set to 'kolla'13:00
mnasiadka#topic rollcall13:00
mnasiadkao/13:00
mhinero/13:00
ihalomi\o13:00
SvenKieskeo/13:00
frickler\o13:01
mnasiadka#topic agenda13:02
mnasiadka* CI status13:02
mnasiadka* Release tasks13:02
mnasiadka* Regular stable releases (first meeting in a month)13:02
mnasiadka* Current cycle planning13:02
mnasiadka* Additional agenda (from whiteboard)13:02
mnasiadka* Open discussion13:02
mnasiadka#topic CI status13:02
mnasiadkaBasically all green - apart the gnocchi issue that we fixed (and Kolla is now using Caracal sources for building)13:03
mnasiadkaAnd Octavia has problems tearing down an amphora LB - claims Nova did not free up the port13:03
mnasiadkaI'm having a slow look into that13:03
mnasiadka#topic Release tasks13:03
mnasiadkaSo basically - we should start preparing Kolla and Kolla-Ansible for branching13:04
mnasiadkaI'd like to squeeze ovn-bgp-agent in13:04
mnasiadkaThere are patches for adding Scaphandre (energy consumption monitoring for processes/instances) which would be also nice to get in13:04
mnasiadkaSo I'll be running around asking for reviews surely and forming up a queue of RP+1 things that need to get in13:05
mnasiadkaI'll also create a list on the Whiteboard - so we can track that properly13:05
SvenKieskenice, I guess we would also happily review scaphandre, as we already use that downstream13:06
frickler+113:06
mnasiadkaGoodie13:06
mnasiadkaLet's move on13:06
mnasiadka#topic Current cycle planning13:06
mnasiadkavPTG was last week, I'll formulate some priorities for D cycle - but for now the focus is getting closer to releasing Caracal13:07
mnasiadkaFrom another front - our meeting is 13:00 UTC now - any volunteer to update the meetings page for summer daylight savings period?13:07
fricklerdidn't we do that? I updated the channel topic at least, can also push another patch13:09
SvenKieskeI'm currently sorting out my local dev setup issues, else I would say I can do it, but I have various issues due to my fallback machine being windows based :(13:10
opendevreviewMichal Nasiadka proposed openstack/kolla master: WIP: kolla-toolbox: Use py311 on rpm distros  https://review.opendev.org/c/openstack/kolla/+/91610213:10
opendevreviewDawud proposed openstack/kolla master: Change logic for plugins in grafana image build  https://review.opendev.org/c/openstack/kolla/+/91318413:10
mnasiadkafrickler: https://meetings.opendev.org/#Kolla_Team_Meeting says 14:00 UTC13:11
fricklerok, will update13:11
mnasiadkathanks!13:11
mnasiadka#topic Additional agenda (from whiteboard)13:11
mnasiadkaLet's go through that, it's loooong13:11
mnasiadka(SvenKieske): https://review.opendev.org/c/openstack/kolla-ansible/+/909912/comment/2da918f8_2a5d1097/ is a complete copy of "service-cert-copy" role ok for one task, or should it rather be extended instead?13:11
mnasiadkaSvenKieske: I think there's already separate patch for that?13:12
SvenKieskethat was actually addressed I believe, there is a patchset13:12
SvenKieskeyes13:12
SvenKieskebut it could get some reviews :)13:12
mnasiadka(SvenKieske): cores: please review the following:13:12
mnasiadka ovn-exporter https://review.opendev.org/c/openstack/kolla-ansible/+/85549813:12
mnasiadkaskyline SSO https://review.opendev.org/c/openstack/kolla-ansible/+/90586013:12
mnasiadkaneutron service limit https://review.opendev.org/c/openstack/kolla-ansible/+/87777613:12
mnasiadkaadd new sysctl role: https://review.opendev.org/c/openstack/kolla-ansible/+/91235113:12
mnasiadkacell0 VIP change fix https://review.opendev.org/c/openstack/kolla-ansible/+/91092413:12
mnasiadkaoh boy13:12
fricklerand that's already only half of what was there on Monday ;)13:13
SvenKieskeskyline, sysctl and afaik neutron service limit are rather uncontroversial imho :)13:13
SvenKieskeespecially the skyline stuff would be nice for downstream users using that, also it would be nice for the new contributors seeing there patches being merged sometime :D13:14
SvenKieskethe sysctl is nice to have i guess, not that important13:14
mnasiadkaok, I think all of them except ovn-exporter have my +213:14
SvenKieskethe cell0 stuff depends on how much we expect kolla vip to change13:15
mnasiadkaovn-exporter is failing all over the place13:15
SvenKieskeyeah I need to triple check something about ovn, but my local test env has currently issues :( will report back13:15
mnasiadkausually we don't, but there's no sense in breaking people doing that, if there's a contributor willing to fix it13:15
mnasiadkaok then, next list13:16
mnasiadka(mhiner) status of adding python3-docker to ospbo repository13:16
mnasiadkaplease review: https://review.opendev.org/c/openstack/kolla-ansible/+/91141713:16
mnasiadkaand its following change: https://review.opendev.org/c/openstack/kolla-ansible/+/91150513:16
mnasiadkaplus the podman fix: https://review.opendev.org/c/openstack/kolla-ansible/+/91252113:16
mnasiadkathis one is also ready for reviews, it fails only because of docker--py 5.0.3 version on Debian: https://review.opendev.org/c/openstack/kolla-ansible/+/90829513:16
mhinerPodman fix is merged, thanks for that13:16
opendevreviewMatúš Jenča proposed openstack/kolla-ansible master: Add backend TLS between MariaDB and ProxySQL  https://review.opendev.org/c/openstack/kolla-ansible/+/90991213:17
opendevreviewMatt Crees proposed openstack/kayobe master: Docs: add a link to the gerrit review dashboard  https://review.opendev.org/c/openstack/kayobe/+/91611013:17
ihalomifew weeks ago we had discussion about adding python3-docker to ospbo repository or changing the way it is installed on the newest debian 13:18
ihalomiany news about it?13:18
mnasiadkaihalomi: can you reply to the comments in 911417?13:18
mnasiadkaand resolve those that are answered ;13:18
mnasiadka;)13:18
SvenKieskealso https://review.opendev.org/c/openstack/kolla-ansible/+/911417 needs a manual rebase, just checked13:18
ihalomimnasiadka: i did, i was waiting for SvenKieske to close it or add more comments, since they are his suggestions 13:19
mnasiadkaSo what about 908295 ?13:19
SvenKieskeI did an initial review of that, I'll do another round today.13:20
mnasiadkaIt's failing13:21
SvenKieskeit's quite some amount of code13:21
ihalomithe tests on 908295 is failing only because on debian there is no way to install newer version of docker13:21
ihalomiwe talked about this few weeks ago that on debian we dont use pip to install docker for some reason13:21
SvenKieskeah it would've been nice to note that on the change, if you know the reason.13:21
mnasiadkawell, we don't use venvs for some reason13:22
SvenKieskethat being said I faintly recall the discussion, wasn't there some agreement to make this the same for all distros? not sure what the outcome was, I remember some experimental patch in that regard at least.13:22
mnasiadkaanyway, it seems like this is not going to make it to C, unless somebody has a lot of time to make the CI work in venvs (because Debian without venv will install python3-docker deb package)13:23
ihalomithere was 2 suggestions, one to make it same for all distros and other was to add python3-docker to repository we use for podman 13:23
opendevreviewRoman Krček proposed openstack/kolla-ansible master: Optimise config  https://review.opendev.org/c/openstack/kolla-ansible/+/74516413:24
opendevreviewRoman Krček proposed openstack/kolla-ansible master: Refactor services' check-containers and optimise  https://review.opendev.org/c/openstack/kolla-ansible/+/77324313:24
mnasiadkathe same for all distros == use venv for CI, but then we would need to force users to use venv13:24
mnasiadkaDebian does not support using pip to install package globally13:24
ihalomilook at approach for podman https://github.com/openstack/ansible-collection-kolla/blob/master/roles/podman_sdk/tasks/main.yml#L4 it adds this osbpo repository and install podman from there for debian 13:25
fricklerthat's the future for other distros, too13:25
ihalomithen we would have control over which version of docker or podman we use also for non pip installs 13:26
SvenKieskeafaik kevko said he can fix osbpo, no problem? (for the short term) but he's not here today it seems13:26
opendevreviewMichal Nasiadka proposed openstack/kolla master: WIP: kolla-toolbox: Use py311 on rpm distros  https://review.opendev.org/c/openstack/kolla/+/91610213:27
opendevreviewRoman Krček proposed openstack/kolla-ansible master: Refactor services' check-containers and optimise  https://review.opendev.org/c/openstack/kolla-ansible/+/77324313:27
mnasiadkawell, then you need to chase kevko 13:28
mnasiadkalet's move on13:28
mnasiadka(r-krcek)13:28
mnasiadkathoughts on https://review.opendev.org/c/openstack/kolla-ansible/+/914997 ?13:28
opendevreviewMatúš Jenča proposed openstack/kolla-ansible master: Patch service-cert-copy role to be used w/o HAProxy  https://review.opendev.org/c/openstack/kolla-ansible/+/91590113:29
mnasiadkaIt's failing for now, I rechecked that - should be probably better13:30
mnasiadkaand the last one is13:30
mnasiadka(mgoddard): Masakari CI is passing on all branches back to stable/zed. Can we de-deprecate it?13:30
SvenKieskewell tbh I went - quickly - over the linked markdown document and the performance benefit really doesn't seem to stand out. could you maybe provide some more numbers there? afaik even some tests regressed in performance?13:30
SvenKieskeI don't doubt there might be performance benefits, but I think it should be possible to show them in a more convincing fashion :)13:31
mnasiadkayeah, I asked in the review13:31
mnasiadkaso, Masakari13:31
mnasiadkaI think it's about this: https://review.opendev.org/c/openstack/kolla-ansible/+/91487713:31
mnasiadkaAnd then reverting this: https://review.opendev.org/c/openstack/kolla-ansible/+/90119313:31
fricklerso shpc want's to have that downstream and support it?13:32
mnasiadkaOne of our customers is using that extensively and we have stepped up to make sure that project upstream is viable13:37
opendevreviewMerged openstack/kolla stable/2023.2: Fix OpenSearch file ownership issues  https://review.opendev.org/c/openstack/kolla/+/91608113:37
fricklerok, I don't mind doing that, will add to my review list13:39
mnasiadkaOk then, lists of patches are done13:39
opendevreviewVerification of a change to openstack/kolla stable/2023.1 failed: Fix OpenSearch file ownership issues  https://review.opendev.org/c/openstack/kolla/+/91608213:39
mnasiadka#topic Open discussion13:39
mnasiadkaanybody?13:39
r-krcekHi. I was here at the wrong time and missed my spot for the performace upgrade regarding the use or filters. 13:40
r-krcekAny thoughts on this https://review.opendev.org/c/openstack/kolla-ansible/+/91499713:41
mnasiadkawe discussed that shortly, first of all it needs to pass CI - second of all - we'd like some marketing information how much faster is that13:41
mnasiadka(in Kolla-Ansible case, not some general case)13:41
SvenKieskesure, as I said above - and also commented on the patchset - maybe run some more stats to show the real benefit there. also please account for caching et cetera. I'm missing some basic description describing the benchmark environment (virtual machine, software stack with versions et cetera). good benchmarking is very very hard.13:42
r-krcekOkay, I will try to come up with some "real-world" scenario and put the specific number into the comments of the patchset.13:43
SvenKieskedoesn't even need to be real world, but should - in the best case - be reproducible, not just some fluctuation in local dev env :)13:43
SvenKieskeI usually refer to this when it comes to benchmarking, it has many do's and dont's and many pitfalls covered: https://www.brendangregg.com/activebenchmarking.html13:45
SvenKieskeand you don't need to do all that is advised there, but it can give you some ideas what to look for :)13:45
r-krcekThanks for the tip :) 13:46
SvenKieskethanks, looking forward to a v2 of that performance document :) better performance is surely greatly appreciated by all :)13:47
SvenKieskeyou can ping me also most of the time here if you have a question or general feedback in this regard. I did quite some benchmarking in the past and did many of the common errors myself :D13:48
r-krcekThank you, that would be great. I'll be sure to reach out, once I have some more questions/updates. :) 13:49
mnasiadkagood13:50
mnasiadkaok then, let's finish for today13:50
mnasiadkaThanks for coming!13:51
mnasiadka#endmeeting13:51
opendevmeetMeeting ended Wed Apr 17 13:51:04 2024 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)13:51
opendevmeetMinutes:        https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-04-17-13.00.html13:51
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-04-17-13.00.txt13:51
opendevmeetLog:            https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-04-17-13.00.log.html13:51
fricklerthx mnasiadka 13:51
SvenKiesker-krcek: here are some debug tools being mentioned which can record execution time of certain stuff in ansible, maybe you already know them, they are quite popular I think: https://www.redhat.com/sysadmin/faster-ansible-playbook-execution13:52
PrzemekKHow prevent kolla-ansible create entries in /etc/hosts ? I want to use dns server instead14:03
mnasiadkaPrzemekK: https://github.com/openstack/ansible-collection-kolla/blob/877f91f500897640494b4e5672345da8a710d3d1/roles/etc_hosts/defaults/main.yml#L314:05
PrzemekKDuring upgrade from 2023.1 to 2023.2 i can see that there are some issues that repeats in every eviroment example https://bugs.launchpad.net/kolla-ansible/+bug/2060823 or prometheus salt is "too long" after merge old with new passwords . There is need to reset docker volume for  Rabbitmq queues needs to be manually upgraded to cluster . There is need to delete docker volume for grafana/gnocchi14:09
PrzemekKrabbitmq upgrade process trying to check enabled features but images are still 2023.1 and it fails14:10
PrzemekKthx mnasiadka14:12
opendevreviewMartin Hiner proposed openstack/kolla-ansible master: Add container engine migration scenario  https://review.opendev.org/c/openstack/kolla-ansible/+/83694114:43
opendevreviewMerged openstack/kolla stable/zed: Fix OpenSearch file ownership issues  https://review.opendev.org/c/openstack/kolla/+/91608314:46
PrzemekKcould You merge this to master/bobcat Refactor external ceph . Its adds one ceph cluster but with different pools/users/cluster names https://review.opendev.org/c/openstack/kolla-ansible/+/907166 14:46
opendevreviewIvan Halomi proposed openstack/kolla-ansible master: Refactor of kolla_container_facts  https://review.opendev.org/c/openstack/kolla-ansible/+/91141714:58
opendevreviewMerged openstack/kolla master: Revert "CI/Master only: pin opensearch{-dashboards}"  https://review.opendev.org/c/openstack/kolla/+/91595215:39
opendevreviewRoman Krček proposed openstack/kolla-ansible master: Refactor services' check-containers and optimise  https://review.opendev.org/c/openstack/kolla-ansible/+/77324316:26
opendevreviewMerged openstack/kolla stable/2023.1: Fix OpenSearch file ownership issues  https://review.opendev.org/c/openstack/kolla/+/91608216:32
opendevreviewMichal Nasiadka proposed openstack/kolla master: WIP: kolla-toolbox: Use py311 on rpm distros  https://review.opendev.org/c/openstack/kolla/+/91610217:43
opendevreviewMaksim Malchuk proposed openstack/kayobe master: Fix default place for Ubuntu Apt keyrings  https://review.opendev.org/c/openstack/kayobe/+/91613819:48
opendevreviewMaksim Malchuk proposed openstack/ansible-collection-kolla master: Use deb822 format for Apt repositories on Ubuntu and drop apt-key  https://review.opendev.org/c/openstack/ansible-collection-kolla/+/91614321:48

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!