| opendevreview | lixuehai proposed openstack/kolla-ansible master: Fix api layer become slow when shutdown one control node https://review.opendev.org/c/openstack/kolla-ansible/+/849603 | 03:03 |
|---|---|---|
| opendevreview | lixuehai proposed openstack/kolla-ansible master: Fix api layer become slow when shutdown one control node https://review.opendev.org/c/openstack/kolla-ansible/+/849487 | 03:07 |
| *** amoralej|off is now known as amoralej | 07:19 | |
| niknon | Hi! If I run "sudo ovs-vsctl show" on a Kolla all-in-one host, should I then see a list of Neutron bridges etc.? I don't see anything when I run that command. | 08:07 |
| niknon | Some context: I get a lot of errors like "neutron_lib.exceptions.BridgeDoesNotExist: Bridge br-int does not exist", both from neutron-dhcp, manila etc. It seems like something is not configured correctly. | 08:10 |
| niknon | I thought that neutron created these by itself on boot? | 08:11 |
| hrw | morning | 08:39 |
| yoctozepto | morning | 08:46 |
| hrw | added cs9 to meeting agenda in whiteboard | 09:07 |
| hrw | as clustercheck needs solution | 09:10 |
| yoctozepto | hrw: I think you can try installing systemd in that image and working from there; the Fedora systemd has been container-friendly for some time already so I believe the RHEL9's one might be as well (and thus why they removed xinetd because then it was simply redundant) | 09:18 |
| hrw | btw - why we fetch clustercheck in mariadb-base? | 09:30 |
| yoctozepto | hrw: because it's deployed near mariadb-server anyway and possibly because it relied on the client libs? not sure now | 09:33 |
| hrw | this is shell script | 09:33 |
| yoctozepto | yup, so it calls something, right? ;-) | 09:34 |
| opendevreview | Merged openstack/kolla-ansible master: Set the ironic notification level https://review.opendev.org/c/openstack/kolla-ansible/+/828962 | 10:42 |
| *** amoralej is now known as amoralej|lunch | 12:29 | |
| *** amoralej|lunch is now known as amoralej | 13:29 | |
| yoctozepto | meeting in 6 minutes mgoddard mnasiadka hrw yoctozepto bbezak parallax Fl1nt frickler adrian-a | 13:54 |
| yoctozepto | #startmeeting kolla | 14:00 |
| opendevmeet | Meeting started Wed Jul 13 14:00:10 2022 UTC and is due to finish in 60 minutes. The chair is yoctozepto. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 14:00 |
| yoctozepto | #topic Roll-call | 14:00 |
| yoctozepto | o/ | 14:00 |
| mmalchuk | hi | 14:00 |
| hrw | o\ | 14:01 |
| yoctozepto | crowds today | 14:02 |
| yoctozepto | #topic Agenda | 14:02 |
| yoctozepto | * Roll-call | 14:02 |
| yoctozepto | * Agenda | 14:02 |
| yoctozepto | * Announcements | 14:02 |
| yoctozepto | * Review action items from the last meeting | 14:02 |
| yoctozepto | * CI status | 14:02 |
| yoctozepto | * Release tasks | 14:02 |
| yoctozepto | * Regular stable releases (first meeting in a month) | 14:02 |
| yoctozepto | * Current cycle planning | 14:02 |
| yoctozepto | * Additional agenda (from whiteboard) | 14:02 |
| yoctozepto | * Open discussion | 14:02 |
| yoctozepto | in the additional agenda we have | 14:03 |
| yoctozepto | (hrw) CentOS Stream 9 | 14:03 |
| yoctozepto | (frickler) Virtual PTG planning | 14:03 |
| yoctozepto | (m.hiner) tmpfs comparison in kolla_docker | 14:03 |
| yoctozepto | (headphoneJames) please review let's encrypt patch: https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 14:03 |
| TimBeermann[m] | o/ | 14:03 |
| yoctozepto | #topic Announcements | 14:03 |
| yoctozepto | nothing to announce | 14:03 |
| yoctozepto | #topic Review action items from the last meeting | 14:04 |
| yoctozepto | mnasiadka Monasca - ensure it's marked as unbuildable where it should be and evaluate later in the cycle if situation fixed itself | 14:04 |
| yoctozepto | not handled, mnasiadka not around | 14:04 |
| yoctozepto | readding | 14:04 |
| yoctozepto | #action mnasiadka Monasca - ensure it's marked as unbuildable where it should be and evaluate later in the cycle if situation fixed itself | 14:04 |
| yoctozepto | #topic CI status | 14:04 |
| ra-rau | o/ | 14:05 |
| yoctozepto | the CI seems green unless anyone noticed differently | 14:05 |
| yoctozepto | unsure about kayobe | 14:05 |
| yoctozepto | ok, let's move on | 14:06 |
| yoctozepto | #topic Release tasks | 14:07 |
| yoctozepto | https://releases.openstack.org/zed/schedule.html | 14:07 |
| yoctozepto | R-12 | 14:07 |
| yoctozepto | Zed-2 milestone | 14:08 |
| yoctozepto | we don't have any related tasks in kolla projects | 14:08 |
| yoctozepto | #topic Regular stable releases (first meeting in a month) | 14:09 |
| yoctozepto | it's not the first meeting in a month, nothing to discuss; the patch is pending | 14:09 |
| yoctozepto | #topic Current cycle planning | 14:09 |
| yoctozepto | how do we look with the different priorities? | 14:10 |
| yoctozepto | I know some are in additional agenda | 14:10 |
| yoctozepto | so we can discuss then | 14:10 |
| yoctozepto | unless someone did not include theirs and wishes to discuss them now | 14:10 |
| hrw | ;D | 14:11 |
| yoctozepto | ok, so I can report that kevko's proxysql is very close now | 14:11 |
| kevko | hi | 14:11 |
| hrw | and is not provided for cs9 | 14:11 |
| yoctozepto | yeah, that's a good topic; cs9 being proxysql-less is sad | 14:11 |
| hrw | last time I checked | 14:11 |
| kevko | i already reported to authors .. | 14:12 |
| kevko | and they replied they don't have a plans to build for cs9 | 14:12 |
| yoctozepto | duh | 14:12 |
| mmalchuk | for Kayobe please review https://review.opendev.org/c/openstack/kayobe/+/845338 and https://review.opendev.org/c/openstack/kayobe/+/840033 | 14:12 |
| yoctozepto | well, we could default to using non-cs9 images for proxysql | 14:13 |
| hrw | mmalchuk: we are in a meeting now | 14:13 |
| kevko | From what I see, CentOS 9 Stream is a rolling distro. | 14:13 |
| kevko | It's difficult for 3rd parties to support, as any components of the system can change/update/upgrade to unexpected versions and have unforeseen impact on the product. | 14:13 |
| kevko | You can try to install the RPM packages to see if any are compatible, still, long term it is a gamble. | 14:13 |
| kevko | You can also compile from source yourself. | 14:13 |
| kevko | For production I would recommend one of the distros we support, and matching packages from our repository. | 14:13 |
| yoctozepto | some debian fallback or whatever | 14:13 |
| kevko | ^^ from author | 14:13 |
| hrw | kevko: then let them build for alma9 or rhel9 etc | 14:13 |
| yoctozepto | ^ | 14:14 |
| hrw | kevko: fluentd image uses td-agent built in alma9 container | 14:14 |
| kevko | I don't think we will support CentOS Stream 9, | 14:14 |
| kevko | but we will support Almalinux 9 which is compatible with RHEL 9 | 14:14 |
| kevko | ETA, next release in a few weeks. | 14:14 |
| yoctozepto | awesome | 14:14 |
| kevko | yeah, they replying fast :P | 14:15 |
| yoctozepto | ok, guessing nothing else to discuss here | 14:17 |
| yoctozepto | let's move on | 14:17 |
| yoctozepto | and thank you for handling the situation, hrw & kevko | 14:17 |
| yoctozepto | #topic (hrw) CentOS Stream 9 | 14:17 |
| yoctozepto | go | 14:18 |
| hrw | ;D | 14:18 |
| hrw | we can build images (no ELK, Grafana, proxysql) but default k-a fails due to lack of mariadb-clustercheck (due to lack of xinetd in cs9) | 14:18 |
| hrw | official way for rhel9 family is 'use systemd instead of xinetd' | 14:19 |
| hrw | but we have dumb-init ' | 14:19 |
| hrw | meh++ | 14:19 |
| kevko | hmm | 14:20 |
| yoctozepto | does anyone know of another solution? | 14:21 |
| yoctozepto | maybe we could go with xinetd from source? ;d | 14:21 |
| hrw | I do not even know what clustercheck does and why we need it ;D | 14:21 |
| yoctozepto | it checks whether mariadb sever is in sync with the galera cluster | 14:21 |
| yoctozepto | and haproxy uses it to avoid broken replicas | 14:22 |
| kevko | well, clustercheck is reading status of wsrep in mariadb and returning 200/500 so haproxy now which is healtthy | 14:22 |
| kevko | because output is HTTP reply | 14:22 |
| yoctozepto | kevko: does proxysql require clustercheck? | 14:22 |
| kevko | nope | 14:22 |
| yoctozepto | ok, so we could also "force" users wishing to use cs9 to use proxysql | 14:22 |
| yoctozepto | kevko would not mind | 14:22 |
| yoctozepto | nor me | 14:22 |
| hrw | I have a feeling that Z release will slip quite a while due to distro updates | 14:23 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 14:23 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 14:23 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 14:23 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger cells job https://review.opendev.org/c/openstack/kolla-ansible/+/838916 | 14:23 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Use Docker healthchecks for mariadb-server service https://review.opendev.org/c/openstack/kolla-ansible/+/805616 | 14:23 |
| yoctozepto | hrw: anything else to discuss? | 14:27 |
| hrw | nope | 14:27 |
| yoctozepto | let's move on then | 14:28 |
| yoctozepto | #topic (frickler) Virtual PTG planning | 14:28 |
| yoctozepto | frickler not around | 14:28 |
| yoctozepto | does anyone want to discuss this topic? | 14:29 |
| TimBeermann[m] | Not from my side | 14:30 |
| ra-rau | Not from my side. Frickler is on vacation | 14:31 |
| hrw | when vPTG takes place? October? | 14:31 |
| yoctozepto | not sure | 14:33 |
| yoctozepto | but that makes sense | 14:33 |
| yoctozepto | it was usually (if not always) mid-October | 14:34 |
| hrw | so maybe we did not dropped that entry from march? | 14:35 |
| hrw | s/from/since | 14:35 |
| yoctozepto | no, frickler wanted to discuss it | 14:37 |
| yoctozepto | you all can see the discussion from the last meeting | 14:37 |
| hrw | ok, just wanted to be usre ;d | 14:37 |
| * yoctozepto moving on | 14:37 | |
| yoctozepto | sure | 14:37 |
| yoctozepto | #topic (m.hiner) tmpfs comparison in kolla_docker | 14:38 |
| m_hiner | So I have this issue where in the JSON we get from container inspect, the tmpfs is represented as a dictionary with keys being the directory names and values are it's settings | 14:39 |
| m_hiner | then in the kolla_docker module we compare that dictionary by converting it to set which only takes the keys so we basically just compare the directory names and not also their settings | 14:40 |
| m_hiner | My question is, is this intententional and non-issue? | 14:40 |
| opendevreview | Sergei Raiskii proposed openstack/kolla-ansible master: Allow cinder-backup to be configured to use S3 backend. https://review.opendev.org/c/openstack/kolla-ansible/+/849711 | 14:41 |
| yoctozepto | m_hiner: that is certainly possible | 14:42 |
| yoctozepto | I don't remember by heart what tmpfs settings are | 14:43 |
| yoctozepto | if asked, I would probably say there are none | 14:43 |
| yoctozepto | maybe they are dummies? | 14:43 |
| yoctozepto | let's check | 14:43 |
| yoctozepto | it seems it originally did not support options but now it does https://docs.docker.com/storage/tmpfs/#specify-tmpfs-options | 14:44 |
| yoctozepto | that said, I am pretty sure kolla-ansible does not use them | 14:45 |
| yoctozepto | hence why they are ignored | 14:45 |
| yoctozepto | did I help you? | 14:45 |
| m_hiner | so what you are saying is that it is fine as is, right? | 14:45 |
| yoctozepto | right | 14:46 |
| m_hiner | Okay, that's all, thank you | 14:46 |
| yoctozepto | ok, let's move on | 14:46 |
| hrw | yoctozepto: does it makes sense?: enable_mariadb_clustercheck: "{{ enable_haproxy if kolla_base_distro != 'centos' else 'false' }}" | 14:46 |
| kevko | i don't think this is ok | 14:47 |
| yoctozepto | hrw: you need "false" without quote marks | 14:47 |
| yoctozepto | but you can also do | 14:47 |
| mgoddard | yoctozepto, m_hiner: we can use tmpfs, although not by default | 14:47 |
| yoctozepto | "{{ enable_haproxy | bool and kolla_base_distro != 'centos' }}" | 14:48 |
| mgoddard | for cinder volume conversion | 14:48 |
| yoctozepto | mgoddard: but not its options | 14:48 |
| hrw | yoctozepto: thx | 14:48 |
| mgoddard | perhaps not | 14:48 |
| yoctozepto | #topic (headphoneJames) please review let's encrypt patch: https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 14:48 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla-ansible master: [WIP] add CentOS Stream 9 support https://review.opendev.org/c/openstack/kolla-ansible/+/839715 | 14:49 |
| yoctozepto | I guess this one is pretty self-explanatory | 14:49 |
| headphoneJames | Would love to get patch in this release | 14:50 |
| -opendevstatus- NOTICE: Due to an incident in our hosting provider, the tarballs.opendev.org site (and possibly other sites served from static.opendev.org) is offline while we attempt recovery | 14:53 | |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 14:58 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 14:58 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger cells job https://review.opendev.org/c/openstack/kolla-ansible/+/838916 | 14:58 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Use Docker healthchecks for mariadb-server service https://review.opendev.org/c/openstack/kolla-ansible/+/805616 | 14:58 |
| hrw | #endmeeting? | 15:02 |
| opendevmeet | Meeting ended Wed Jul 13 15:02:35 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:02 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2022/kolla.2022-07-13-14.00.html | 15:02 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2022/kolla.2022-07-13-14.00.txt | 15:02 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2022/kolla.2022-07-13-14.00.log.html | 15:02 |
| hrw | shit. since when I have chair powers? | 15:03 |
| kevko | Btw, i had a question, is there a reason why neutron is not splitted to neutron-api and neutron-server (neutron-rpc-server) ? | 15:04 |
| yoctozepto | thanks hrw | 15:04 |
| kevko | understand that it is probably from historical reasons ... | 15:04 |
| yoctozepto | sorry that someone caught my attention ;-) | 15:04 |
| kevko | but this should be reworked, shouldn't be ? | 15:04 |
| yoctozepto | kevko: you mean in neutron itself? | 15:05 |
| kevko | neutron-server container -> neutron-api, neutron-server | 15:05 |
| kevko | i mean in kolla | 15:05 |
| yoctozepto | kevko: are they two separate daemons elsewhere? | 15:05 |
| kevko | well, neutron-api as server which is just listening for API requiests ..and then RPC server no ? | 15:06 |
| yoctozepto | kevko: so far I have only seen it in a single process tbh | 15:07 |
| kevko | let me check and paste here | 15:08 |
| kevko | yoctozepto: (neutron-server)[root@controller0 /]# ls -la /var/lib/kolla/venv/bin/ | egrep 'neutron-api|neutron-rpc' | 15:10 |
| kevko | -rwxr-xr-x 1 root root 1866 Jun 28 09:28 neutron-api | 15:10 |
| kevko | -rwxr-xr-x 1 root root 267 Jun 28 09:28 neutron-rpc-server | 15:10 |
| kevko | because, sometimes you want to have different count of workers for rpc daemon and different for neutron-api | 15:10 |
| kevko | for now neutron-server container is doing both | 15:11 |
| kevko | yoctozepto: it is also in official documentation for neutron -> https://docs.openstack.org/neutron/rocky/admin/config-wsgi.html | 15:12 |
| kevko | yoctozepto: i think it is from rocky | 15:12 |
| kevko | also, healthcheck is only checking listen port for API ...not if it is connected to rabbitmq | 15:13 |
| yoctozepto | well, we set api_workers and rpc_workers independently so it's not an issue | 15:14 |
| yoctozepto | but splitting the two would help with their placement optimisation or whatever | 15:14 |
| yoctozepto | you can propose to split | 15:14 |
| yoctozepto | but I would find it even more beneficial to split it per role further; as in splitting api/controller role from neutron agents role (like nova and nova-cell now) | 15:15 |
| * yoctozepto off | 15:18 | |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 16:17 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 16:17 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 16:17 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger cells job https://review.opendev.org/c/openstack/kolla-ansible/+/838916 | 16:17 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Use Docker healthchecks for mariadb-server service https://review.opendev.org/c/openstack/kolla-ansible/+/805616 | 16:17 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 16:35 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 16:35 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 16:35 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger cells job https://review.opendev.org/c/openstack/kolla-ansible/+/838916 | 16:35 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Use Docker healthchecks for mariadb-server service https://review.opendev.org/c/openstack/kolla-ansible/+/805616 | 16:35 |
| *** amoralej is now known as amoralej|off | 16:47 | |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 16:52 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 16:52 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 16:52 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger cells job https://review.opendev.org/c/openstack/kolla-ansible/+/838916 | 16:52 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Use Docker healthchecks for mariadb-server service https://review.opendev.org/c/openstack/kolla-ansible/+/805616 | 16:52 |
| opendevreview | Sergei Raiskii proposed openstack/kolla-ansible master: Allow cinder-backup to be configured to use S3 backend. https://review.opendev.org/c/openstack/kolla-ansible/+/849711 | 17:49 |
| opendevreview | Sergei Raiskii proposed openstack/kolla-ansible master: Allow cinder-backup to be configured to use S3 backend. https://review.opendev.org/c/openstack/kolla-ansible/+/849711 | 18:14 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 19:29 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 19:29 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 19:29 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger cells job https://review.opendev.org/c/openstack/kolla-ansible/+/838916 | 19:29 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Use Docker healthchecks for mariadb-server service https://review.opendev.org/c/openstack/kolla-ansible/+/805616 | 19:29 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!