| opendevreview | Roman Krček proposed openstack/kolla-ansible master: Rewrite kolla-ansible CLI to python https://review.opendev.org/c/openstack/kolla-ansible/+/923110 | 05:49 |
|---|---|---|
| opendevreview | Roman Krček proposed openstack/kolla-ansible master: Don't notify handlers during copy-cert https://review.opendev.org/c/openstack/kolla-ansible/+/924651 | 06:00 |
| opendevreview | Roman Krček proposed openstack/kolla-ansible master: Performance: Don't notify handlers during config https://review.opendev.org/c/openstack/kolla-ansible/+/745164 | 06:00 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: switch to use mariadb flavored commands https://review.opendev.org/c/openstack/kolla-ansible/+/927039 | 06:49 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: ironic: use ironic_tftp_listen_address in dnsmasq.conf https://review.opendev.org/c/openstack/kolla-ansible/+/927079 | 07:04 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add certificates for RabbitMQ internode TLS https://review.opendev.org/c/openstack/kolla-ansible/+/921380 | 07:28 |
| opendevreview | Grzegorz Bialas proposed openstack/kolla-ansible master: Fix links in docs for unmaintained releases https://review.opendev.org/c/openstack/kolla-ansible/+/927022 | 08:21 |
| opendevreview | Sven Kieske proposed openstack/kolla-ansible master: harden haproxy TLS configuration https://review.opendev.org/c/openstack/kolla-ansible/+/915403 | 08:30 |
| SvenKieske | can some core look at https://review.opendev.org/c/openstack/kolla-ansible/+/898307 ? :) | 08:39 |
| opendevreview | Sven Kieske proposed openstack/kolla-ansible stable/2024.1: CI: improve get_logs.sh https://review.opendev.org/c/openstack/kolla-ansible/+/927089 | 08:41 |
| opendevreview | Sven Kieske proposed openstack/kolla-ansible stable/2023.2: CI: improve get_logs.sh https://review.opendev.org/c/openstack/kolla-ansible/+/927090 | 08:41 |
| opendevreview | Sven Kieske proposed openstack/kolla-ansible stable/2023.1: CI: improve get_logs.sh https://review.opendev.org/c/openstack/kolla-ansible/+/927091 | 08:41 |
| opendevreview | Grzegorz Bialas proposed openstack/kolla-ansible master: Fix links in docs for unmaintained releases https://review.opendev.org/c/openstack/kolla-ansible/+/927022 | 08:49 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Use py3 as the default runtime for tox https://review.opendev.org/c/openstack/kolla-ansible/+/898307 | 09:12 |
| mnasiadka | I did have a look, my OCD won with that whitespace | 09:13 |
| SvenKieske | :D that's bothering me for weeks already | 09:16 |
| opendevreview | Grzegorz Bialas proposed openstack/kolla-ansible master: Fix links in docs for unmaintained releases https://review.opendev.org/c/openstack/kolla-ansible/+/927022 | 09:38 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: [WIP] Enable ML2/OVN and distributed FIP by default. https://review.opendev.org/c/openstack/kolla-ansible/+/904959 | 10:06 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: Disable wsrep_provider plugin https://review.opendev.org/c/openstack/kolla-ansible/+/927096 | 10:07 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: [DNM] Test seed-vm job with Ubuntu image without EFI changes https://review.opendev.org/c/openstack/kayobe/+/927100 | 10:39 |
| opendevreview | Merged openstack/kolla-ansible master: Use py3 as the default runtime for tox https://review.opendev.org/c/openstack/kolla-ansible/+/898307 | 10:46 |
| frickler | .oO(one small space for gerrit, a giant leap for kolla reviewers) :D | 11:06 |
| mnasiadka | haha | 11:07 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: [DNM] Test seed-vm job with Ubuntu image without EFI changes https://review.opendev.org/c/openstack/kayobe/+/927100 | 11:14 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Implement TLS for Redis https://review.opendev.org/c/openstack/kolla-ansible/+/909188 | 11:32 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add caching options https://review.opendev.org/c/openstack/kolla-ansible/+/927104 | 11:32 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add Redis as caching backend for Keystone https://review.opendev.org/c/openstack/kolla-ansible/+/909201 | 11:33 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add Redis as caching backend for Nova https://review.opendev.org/c/openstack/kolla-ansible/+/909203 | 11:34 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add Redis as caching backend for Placement https://review.opendev.org/c/openstack/kolla-ansible/+/909222 | 11:35 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add Redis as caching backend for Heat https://review.opendev.org/c/openstack/kolla-ansible/+/909224 | 11:35 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add Redis as caching backend for Ceilometer https://review.opendev.org/c/openstack/kolla-ansible/+/926721 | 11:36 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Fix upgrade jobs failing on adding new certificates https://review.opendev.org/c/openstack/kolla-ansible/+/926284 | 11:53 |
| SvenKieske | mnasiadka: wondering if the processes inside neutron_ovn_agent container run in the correct pid namespace when I look at the logs? https://review.opendev.org/c/openstack/kolla-ansible/+/904959 ps output is either empty or cut off | 11:54 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: [WIP] Enable ML2/OVN and distributed FIP by default. https://review.opendev.org/c/openstack/kolla-ansible/+/904959 | 11:55 |
| mnasiadka | SvenKieske: I just love debugging in CI ;) | 11:55 |
| SvenKieske | who doesn't? :) ah there was a stray pipe :D | 11:56 |
| SvenKieske | the ovn status checks below that seem also rather brittle tbh | 11:57 |
| mnasiadka | for now it's a WIP with debugging needed ;) | 11:58 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Fix upgrade jobs failing on adding new certificates https://review.opendev.org/c/openstack/kolla-ansible/+/926284 | 12:08 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Fix upgrade jobs failing on adding new certificates https://review.opendev.org/c/openstack/kolla-ansible/+/926284 | 12:09 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: CI: Fix upgrade jobs failing on adding new certificates https://review.opendev.org/c/openstack/kolla-ansible/+/926284 | 12:10 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add certificates for RabbitMQ internode TLS https://review.opendev.org/c/openstack/kolla-ansible/+/921380 | 12:11 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: CI: Fix upgrade jobs failing on adding new certificates https://review.opendev.org/c/openstack/kolla-ansible/+/926284 | 12:11 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add support for RabbitMQ internode tls https://review.opendev.org/c/openstack/kolla-ansible/+/921381 | 12:11 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: Bump to 11.4 https://review.opendev.org/c/openstack/kolla/+/926981 | 12:14 |
| opendevreview | Merged openstack/kolla-ansible stable/2024.1: Fix prometheus.yml templating https://review.opendev.org/c/openstack/kolla-ansible/+/926618 | 12:17 |
| mnasiadka | kevko: can you please have a look here: https://review.opendev.org/c/openstack/kolla-ansible/+/925971 ? | 12:21 |
| opendevreview | Merged openstack/kolla-ansible master: Add REQUESTS_CA_BUNDLE to kolla-toolbox container https://review.opendev.org/c/openstack/kolla-ansible/+/923105 | 12:24 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Enable internal/backend TLS in LE scenario https://review.opendev.org/c/openstack/kolla-ansible/+/926210 | 12:26 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Enable internal/backend TLS in LE scenario https://review.opendev.org/c/openstack/kolla-ansible/+/926210 | 12:27 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: CI: Fix upgrade jobs failing on adding new certificates https://review.opendev.org/c/openstack/kolla-ansible/+/926284 | 13:04 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Switch mariadb's loadbalancer from HAProxy to ProxySQL https://review.opendev.org/c/openstack/kolla-ansible/+/913724 | 13:04 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add backend TLS between MariaDB and ProxySQL https://review.opendev.org/c/openstack/kolla-ansible/+/909912 | 13:04 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add frontend TLS ability to ProxySQL https://review.opendev.org/c/openstack/kolla-ansible/+/925500 | 13:04 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add frontend database TLS for Keystone https://review.opendev.org/c/openstack/kolla-ansible/+/925507 | 13:04 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add caching options https://review.opendev.org/c/openstack/kolla-ansible/+/927104 | 13:12 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: Disable wsrep_provider plugin https://review.opendev.org/c/openstack/kolla-ansible/+/927096 | 14:04 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: DNM: Add some debug to healthcheck_port https://review.opendev.org/c/openstack/kolla/+/927149 | 14:46 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: Add label before upgrade to 11 https://review.opendev.org/c/openstack/kolla/+/927158 | 15:03 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: switch to use mariadb flavored commands https://review.opendev.org/c/openstack/kolla-ansible/+/927039 | 15:04 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: Disable wsrep_provider plugin https://review.opendev.org/c/openstack/kolla-ansible/+/927096 | 15:12 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: Disable wsrep_provider plugin https://review.opendev.org/c/openstack/kolla-ansible/+/927096 | 15:12 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: Bump to 11.4 https://review.opendev.org/c/openstack/kolla/+/926981 | 15:13 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: Bump to 11.4 https://review.opendev.org/c/openstack/kolla/+/926981 | 15:13 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: CI: Disable CentOS/Rocky/Ubuntu aarch64 jobs https://review.opendev.org/c/openstack/kolla/+/927163 | 15:17 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Enable internal/backend TLS in LE scenario https://review.opendev.org/c/openstack/kolla-ansible/+/926210 | 15:20 |
| mnasiadka | frickler: https://review.opendev.org/c/openstack/kolla/+/927163 - posted that to ease the strain on OSUOSL a bit - might be we'll need to disable the Debian jobs as well until situation stabilises... (cc: fungi) | 15:22 |
| frickler | mnasiadka: ack, +6 ;) this morning when I saw the arm pipeline was empty, I was hoping that things had recovered, sadly it doesn't look like that | 15:25 |
| mnasiadka | well, basically the debian build job finishes in 2hr 59minutes and then it's success - or we get over it and it gets timed out | 15:26 |
| mnasiadka | comparing to 30 minutes in the past - I don't think we should bump up the time limit... | 15:26 |
| frickler | ack | 15:28 |
| samcat116 | Hi all, does someone know the rationale about setting the rpc workers this low: https://github.com/openstack/kolla-ansible/blob/ab4a79d552e3d382bd31e8e08a5931c035db2c69/ansible/group_vars/all.yml#L689 | 16:33 |
| samcat116 | with modern servers I can't imagine a scenario where the number of cpus calculated there would be used rather than the hard coded values, resulting in a pretty small number of workers | 16:33 |
| frickler | samcat116: iiuc the idea is to have a default that works well for the limited resources in CI or poc deployments. for your production setup you can tune it as needed | 16:40 |
| samcat116 | Is there any docs or guidance on that tuning? | 16:40 |
| samcat116 | We’re seeing some intermittent neutron 504 timeouts and I’m wondering if this is partially responsible | 16:41 |
| samcat116 | On a cloud of around 1500vms and 3000-4000 networks | 16:41 |
| samcat116 | I get that’s likely on the neutron side and not the kills side obviously | 16:42 |
| opendevreview | Merged openstack/kolla master: CI: Disable CentOS/Rocky/Ubuntu aarch64 jobs https://review.opendev.org/c/openstack/kolla/+/927163 | 16:54 |
| frickler | samcat116: that's a pretty high network/vm ratio I'd say. I'm not aware of any specific tuning guide, but maybe https://docs.openstack.org/large-scale/index.html can be interesting for you | 17:26 |
| mnasiadka | samcat116: with ML2/OVS most probably RMQ is more problematic than the number of workers - not speaking about the storm that will happen when you restart l3-agent/openvswitch-agent | 17:43 |
| samcat116 | we are running OVN, so after reading more I guess these aren't even useful | 17:43 |
| mnasiadka | Well, then maybe it would be good to start a thread on #openstack-neutron to find what those 504s are really linked to | 17:45 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: [WIP] Enable ML2/OVN and distributed FIP by default. https://review.opendev.org/c/openstack/kolla-ansible/+/904959 | 17:49 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: Disable wsrep_provider plugin https://review.opendev.org/c/openstack/kolla-ansible/+/927096 | 17:52 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: Disable wsrep_provider plugin https://review.opendev.org/c/openstack/kolla-ansible/+/927096 | 17:52 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: DNM: Run fio tests https://review.opendev.org/c/openstack/kolla/+/927210 | 20:06 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: DNM: Run fio tests https://review.opendev.org/c/openstack/kolla/+/927210 | 20:09 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: DNM: Run fio tests https://review.opendev.org/c/openstack/kolla/+/927210 | 20:14 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: DNM: Run fio tests https://review.opendev.org/c/openstack/kolla/+/927210 | 20:18 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!