| opendevreview | Michal Arbet proposed openstack/kolla master: Fix permissions for ironic metrics https://review.opendev.org/c/openstack/kolla/+/940514 | 06:16 |
|---|---|---|
| opendevreview | Michal Arbet proposed openstack/kolla master: Fix permissions for ironic metrics https://review.opendev.org/c/openstack/kolla/+/940514 | 06:24 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Install pycadf from pypi package https://review.opendev.org/c/openstack/kolla/+/940605 | 06:31 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Fix permissions for ironic metrics https://review.opendev.org/c/openstack/kolla/+/940514 | 06:31 |
| kevko | frickler: replied to your comments | 06:44 |
| opendevreview | Roman Krcek proposed openstack/kolla-ansible master: Merge of container_facts modules https://review.opendev.org/c/openstack/kolla-ansible/+/912460 | 08:47 |
| opendevreview | Roman Krcek proposed openstack/kolla-ansible master: Add action for getting container names list https://review.opendev.org/c/openstack/kolla-ansible/+/924389 | 08:47 |
| opendevreview | Roman Krcek proposed openstack/kolla-ansible master: Add container engine migration scenario https://review.opendev.org/c/openstack/kolla-ansible/+/836941 | 08:47 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible stable/2024.2: Fix Grafana datasource update https://review.opendev.org/c/openstack/kolla-ansible/+/940856 | 08:52 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible stable/2024.1: Fix Grafana datasource update https://review.opendev.org/c/openstack/kolla-ansible/+/940857 | 08:52 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible stable/2023.2: Fix Grafana datasource update https://review.opendev.org/c/openstack/kolla-ansible/+/940858 | 08:52 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/2024.2: Support forcing time synchronisation https://review.opendev.org/c/openstack/kayobe/+/940869 | 10:05 |
| opendevreview | Matt Crees proposed openstack/kayobe master: Drop kolla-tags and kolla-limit https://review.opendev.org/c/openstack/kayobe/+/935669 | 11:06 |
| kevko | frickler: can u please review comments so we can continue reviewing/merging rabbitmq queue manager ? | 11:52 |
| opendevreview | Verification of a change to openstack/kayobe master failed: Replace pause with chronyc waitsync in ntp sync https://review.opendev.org/c/openstack/kayobe/+/940646 | 12:05 |
| opendevreview | Merged openstack/kolla-ansible master: Fix Grafana datasource update https://review.opendev.org/c/openstack/kolla-ansible/+/940133 | 12:06 |
| andreykurilin | Hi folks! Can someone suggest the direction to dig? One of CI jobs failed with `NoneType: None` message on `kolla-ansible reconfigure -i /etc/kolla/inventory -vvv` step. | 12:11 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible stable/2024.2: Fix Grafana datasource update https://review.opendev.org/c/openstack/kolla-ansible/+/940856 | 12:36 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible stable/2024.1: Fix Grafana datasource update https://review.opendev.org/c/openstack/kolla-ansible/+/940857 | 12:36 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible stable/2023.2: Fix Grafana datasource update https://review.opendev.org/c/openstack/kolla-ansible/+/940858 | 12:37 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Fix permissions for ironic metrics https://review.opendev.org/c/openstack/kolla/+/940514 | 12:42 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Install pycadf from pypi package https://review.opendev.org/c/openstack/kolla/+/940605 | 12:43 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Fix permissions for ironic metrics https://review.opendev.org/c/openstack/kolla/+/940514 | 12:43 |
| opendevreview | Jakub Darmach proposed openstack/kolla stable/2024.1: Add support for Ubuntu 24.04 LTS https://review.opendev.org/c/openstack/kolla/+/932386 | 14:35 |
| dcapone2004 | when upgrading rabbitmq between 2023.1 and 2023.2, the docs reference om_enable_rabbitmq_quorum_queues and om_enable_rabbitmq_high_availability ... neither of these options are in my current globals.yml file on 2023.1.... is this correct? | 15:11 |
| dcapone2004 | I am also not seeing that variable in etc_examples folder of the 2023.2 release, so I am tring to follow the docs but I am unsure where these variables are set and how to safely upgrade rabbitmq for a transition to 2023.2 | 15:22 |
| opendevreview | Andrey Kurilin proposed openstack/kolla-ansible master: Makes Grafana database ssl options configurable https://review.opendev.org/c/openstack/kolla-ansible/+/940886 | 15:23 |
| priteau | dcapone2004: The quorum queue option was added as a backport after release: https://review.opendev.org/c/openstack/kolla-ansible/+/902387 | 15:25 |
| priteau | You can probably diff the changes to bring your own globals.yml up to date. | 15:26 |
| dcapone2004 | priteau: I assume I need that variable in my globals.yml to safely upgrade? | 15:26 |
| priteau | There is a precheck that will complain if you are not using either om_enable_rabbitmq_quorum_queues or om_enable_rabbitmq_high_availability | 15:27 |
| priteau | You need to pick one | 15:27 |
| dcapone2004 | it is a bit confusing as the "upgrade" page of the 2023.2 docs do not make mention of separately upgrading rabbitmq unless you are performing a SLURP update, but if you did into the rabbitmq configuration page, it has a separate specific procedure for upgrading rabbitmq, so I am a little confused if the rabbitmq separate upgrade is needed | 15:28 |
| dcapone2004 | dig* | 15:28 |
| kevko | andreykurilin: link ? | 15:36 |
| andreykurilin | kevko: https://zuul.opendev.org/t/openstack/build/1616a7134fe84526a50fdf185329255c from https://review.opendev.org/c/openstack/kolla-ansible/+/940825 | 15:37 |
| dcapone2004 | priteau: also, fyi it appears that the inclusion of the variable itself into the globals.yml was left off the merge and updates as I reviewed the last version of globals.yml in the KA repo and it also does not include the variables, so diff I do not think would help here | 15:37 |
| dcapone2004 | it actually isn't included in any of the globals.yml files up to and including the current master repo .... so does that variable get defined in globals.yaml or elsewhere? | 15:41 |
| kevko | andreykurilin: non-related i think | 15:42 |
| kevko | andreykurilin: mariadb didn't create a cluster | 15:43 |
| kevko | andreykurilin: 2025-02-05 19:46:13 2 [ERROR] WSREP: ./gcs/src/gcs.cpp:s_join():990: Sending JOIN failed: -103 (Connection was closed). | 15:43 |
| kevko | 2025-02-05 19:46:13 2 [Warning] WSREP: Failed to JOIN the cluster after SST gcs_join(01e127e1-e3f9-11ef-9365-b799a7089f5d:21) failed | 15:43 |
| kevko | andreykurilin: https://ee4b9eb34e1fd5d099ea-4540e2fcfebb146227dc28f41e3112d0.ssl.cf5.rackcdn.com/940825/1/check/kolla-ansible-ubuntu-mariadb/1616a71/primary/logs/kolla/mariadb/mariadb.txt | 15:43 |
| kevko | andreykurilin: so monitor user was not created ..and playbook failed | 15:43 |
| kevko | andreykurilin: just recheck | 15:43 |
| bbezak | frickler: got issue with that change - https://review.opendev.org/c/openstack/kolla-ansible/+/910503. not sure why it can't workflow - as related changed got merged - however somehow on the different patchset | 15:44 |
| dcapone2004 | ok, in digging further, I think I understand....on 2023.1 by default transient queues are enabled.... ONLY IF you want to switch to durable queues, does the extra steps need to be followed | 15:48 |
| dcapone2004 | however, if upgrading to 2023.2, to maintain transient queues, I need to add om_enable_rabbitmq_quorum_queues : false and om_enable_rabbitmq_high_availability: true to the globals.yml file before upgrading | 15:49 |
| dcapone2004 | am I also correct that switching to durable queues requires downtime??? The procedure indicates a need to stop services that rely on rabbitmq to make the change which afaik includes neutron and also afaik if neutron containers are stopped, network traffic will stop | 15:50 |
| andreykurilin | kevko: thank you for looking! | 15:53 |
| priteau | dcapone2004: that's correct, it requires a full stop of all services using RMQ | 16:00 |
| dcapone2004 | priteau: thanks....am I correct that I am safely upgrade without downtime, by simply using KA upgrade IF I set om_enable_rabbitmq_quorum_queues : false and om_enable_rabbitmq_high_availability: true in globals.yml with no other special steps? | 16:02 |
| frickler | bbezak: depends-on isn't merged https://review.opendev.org/c/openstack/kolla-ansible/+/899615 | 16:05 |
| bbezak | Right. I didn't notice in this long commit msg. Thank you | 16:06 |
| -opendevstatus- NOTICE: nominations for the OpenStack PTL and TC positions are now open, for details see https://governance.openstack.org/election/ | 16:08 | |
| opendevreview | Massimiliano Favaro-Bedford proposed openstack/kayobe master: Add variable to allow image builds of different platforms. https://review.opendev.org/c/openstack/kayobe/+/940894 | 16:13 |
| kevko | dcapone2004: well, we can't stop services ..we will migrate it on the fly | 16:14 |
| kevko | dcapone2004: but we need to patch oslo.messaging i think ...and still not done | 16:14 |
| priteau | dcapone2004: If you are not already using ha queues, I think you will need full stop? | 16:15 |
| kevko | bbezak: https://review.opendev.org/c/openstack/kolla-ansible/+/940496 ? | 16:15 |
| kevko | can u please ? | 16:15 |
| dcapone2004 | kevko: the docs as of now say a full stop is needed to go from transient to durable queues....I think your comment is saying that you intend to change this requirement in the future, but it isn't completed yet | 16:16 |
| kevko | dcapone2004: well, switch to quorum queues can be fatal ...we have +/- 150 computes and thousands of k8s running on top .... if there is some service down ...it's really fatal for us ... | 16:17 |
| kevko | dcapone2004: so, we will probably patch oslo.messaging and provide migration path somehow ...from a code it can be done when I've checked firstly ... | 16:18 |
| kevko | dcapone2004: but yeah, official opendev or kolla way don't exist ...just stop ..reset ..restart yeagh | 16:18 |
| dcapone2004 | kevko: so safest way to upgrade from 2023.1 to 2023.2 and beyond for the time being is staying on transient queues | 16:19 |
| dcapone2004 | kevko: which I can do like any othe rupgrade in the past, clone 2023.2 repo, merge passwords / globals.yml variables | 16:19 |
| dcapone2004 | kevko: add om_enable_rabbitmq_quorum_queues : false and om_enable_rabbitmq_high_availability: true to the globals.yml file | 16:20 |
| kevko | dcapone2004: correct | 16:20 |
| dcapone2004 | kevko: run kolla-ansible upgrade... | 16:20 |
| kevko | dcapone2004: exactly | 16:20 |
| kevko | dcapone2004: you can run it until rabbitmq 4.0 will be here in kolla | 16:21 |
| dcapone2004 | kevko: awesome, thank you very much....all the rabbitmq noise was stressing me out lol | 16:21 |
| kevko | dcapone2004: I like this noise; after all, if everything worked as it should, many of us wouldn't have a job , right ? :D | 16:22 |
| dcapone2004 | kevko: valid point | 16:27 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!