| opendevreview | Roman Krcek proposed openstack/kolla-ansible master: Convert destroy scripts into playbooks https://review.opendev.org/c/openstack/kolla-ansible/+/920714 | 07:46 |
|---|---|---|
| opendevreview | Dr. Jens Harbott proposed openstack/kolla master: Add mtools package to ironic-conductor image https://review.opendev.org/c/openstack/kolla/+/942785 | 11:57 |
| opendevreview | Ivan Vnučko proposed openstack/kolla-ansible master: Add backend TLS encryption for Masakari https://review.opendev.org/c/openstack/kolla-ansible/+/927228 | 12:44 |
| opendevreview | Merged openstack/kayobe master: CI: Use libpod/registry as container registry https://review.opendev.org/c/openstack/kayobe/+/940713 | 13:36 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix Redis Sentinel authentication for octavia's jobboard HA https://review.opendev.org/c/openstack/kolla-ansible/+/942799 | 13:59 |
| bbezak_2 | #startmeeting kolla | 14:00 |
| opendevmeet | Meeting started Wed Feb 26 14:00:19 2025 UTC and is due to finish in 60 minutes. The chair is bbezak_2. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 14:00 |
| bbezak_2 | #topic rollcall | 14:00 |
| r-krcek | o/ | 14:00 |
| MattCrees[m] | o/ | 14:00 |
| mhiner | o/ | 14:00 |
| kevko | \o | 14:01 |
| kevko | (half here ..i am in waiting room in a hospital) | 14:01 |
| bbezak_2 | #topic agenda | 14:03 |
| bbezak_2 | * Roll-call * Agenda * Announcements * Review action items from the last meeting * CI status * Release tasks * Regular stable releases (first meeting in a month) * Current cycle planning * Additional agenda (from whiteboard) * Open discussion | 14:03 |
| yuval | 0/ | 14:03 |
| bbezak_2 | #topic CI status | 14:03 |
| yuval | kevko: feel well | 14:04 |
| bbezak_2 | yes, kevko take care | 14:04 |
| bbezak_2 | I didn't see much CI issues recently, except some Kayobe CI fixes | 14:04 |
| yuval | in my patch I saw a glitch | 14:05 |
| bbezak_2 | I need to look into ARM64 jobs wich have issues | 14:05 |
| yuval | kolla-ansible-rocky9-zun (failed and after re-run it passed) | 14:06 |
| bbezak_2 | yuval: ok, let us know if you will see that more often | 14:06 |
| bbezak_2 | #topic Release tasks | 14:07 |
| bbezak_2 | I think mr koperg did what he promised | 14:07 |
| bbezak_2 | we probably need to wait still for RDO epoxy | 14:07 |
| bbezak_2 | #topic Current cycle planning | 14:09 |
| MattCrees[m] | On the track towards RabbitMQ 4.0, kevko has done good work finishing up this patch. I think it's good to go: https://review.opendev.org/c/openstack/kolla-ansible/+/924623 | 14:10 |
| kevko | I would like to ask if anybody knows why sometime neutron+ovn internal dns not working ? | 14:10 |
| bbezak_2 | cool Matt. let's review it then | 14:11 |
| kevko | https://review.opendev.org/c/openstack/kolla-ansible/+/942743?tab=change-view-tab-header-zuul-results-summary | 14:11 |
| kevko | and it's always ubuntu | 14:11 |
| kevko | https://094b12513443a4ca772b-1fa4167271ed7a639c6f5f9054b6d37e.ssl.cf1.rackcdn.com/942743/1/check/kolla-ansible-ubuntu-upgrade-ovn/f2d9dae/primary/logs/ansible/test-core-openstack | 14:11 |
| kevko | MattCrees[m] +1 yeah, i think that one on the bottom of the relation chain is ready ... | 14:11 |
| bbezak_2 | more often broken than not? | 14:12 |
| kevko | lemme check | 14:12 |
| *** bbezak1 is now known as bbezak | 14:13 | |
| kevko | hmm, well, it's not easy to check just from overall if failure or success ...i need to click and open every test to check if it is exactly that problem | 14:13 |
| kevko | but i was wondering ...why neutron just don't fill the dns_name (there is None) if port is created with it ...with OVN | 14:14 |
| bbezak_2 | can't remember from the top of my head. but I think we're using internal dns in most of our clouds and it is working - within project | 14:15 |
| bbezak_2 | #topic Additional agenda | 14:16 |
| bbezak_2 | ok, it looks like we decided to push canonical to update python-docker in noble. let's see how it will work out | 14:17 |
| mhiner | Yes, so far no response from Canonical guys | 14:18 |
| mhiner | Additionally, I have been thinking about implementing this blueprint: https://blueprints.launchpad.net/kolla-ansible/+spec/capture-docker-logs | 14:18 |
| bbezak_2 | it is usually slow process | 14:18 |
| mhiner | First step could be just introduction of log driver and options to kolla_container | 14:19 |
| mhiner | What are your thoughts on this? | 14:19 |
| bbezak_2 | looks like PTG kind of discussion to me | 14:21 |
| bbezak_2 | but I think it would be good to do that if it is not too difficult to implement | 14:21 |
| bbezak_2 | but indeed - for instance keepalived is a tough one | 14:22 |
| bbezak_2 | and it would be good to capture those logs | 14:22 |
| bbezak_2 | I think some PoC would be good | 14:23 |
| mhiner | Alright, I will try to work on it and let's see what comes out of it | 14:23 |
| bbezak_2 | thank you | 14:24 |
| bbezak_2 | #topic Open discussion | 14:24 |
| bbezak_2 | me and mnasiadka will be away for couple of weeks - we're going to OpenInfra days US | 14:26 |
| bbezak_2 | question if somebody wants to chair those meetings, or we should cancel? | 14:26 |
| yuval | probably cancelling is not a good option :) | 14:27 |
| yuval | but I really dont know how to chair openstack meeting | 14:27 |
| bbezak_2 | it is not too difficult- https://docs.openstack.org/kolla/latest/contributor/meeting.html | 14:28 |
| bbezak_2 | anyway I'll send out a email to openstack-discuss with that question, for people that are not present today | 14:29 |
| yuval | Thanks | 14:29 |
| jovial | On the Ubuntu DNS issue. I've seen issues with resolved using eDNS. Could it be that? | 14:30 |
| jovial | its basically the limitation described in https://docs.openstack.org/neutron/latest/ovn/gaps.html | 14:31 |
| jovial | resolved likes to enable edns by default: https://github.com/systemd/systemd/blob/70516b026b7a83dbe7b8141c67f2dbe412ce24d9/src/resolve/resolv.conf#L18 | 14:32 |
| jovial | kevko, ^ just a thought | 14:32 |
| bbezak_2 | thx jovial | 14:34 |
| yuval | I want to bring up this patch: https://review.opendev.org/c/openstack/kolla-ansible/+/938649 | 14:34 |
| yuval | since we are very near the code freeze | 14:34 |
| kevko | Which version of docker python ? | 14:34 |
| kevko | I can help maybe | 14:34 |
| mhiner | Which one is problematic? | 14:34 |
| bbezak_2 | kevko https://bugs.launchpad.net/ubuntu/+source/python-docker/+bug/2098863 | 14:35 |
| kevko | Jovial don't think so | 14:35 |
| kevko | Because it is from time to time | 14:35 |
| bbezak_2 | I added RP1 to your change, that we won't miss it before release yuval | 14:36 |
| yuval | Thank you | 14:36 |
| bbezak_2 | ok, thank you all for joining! | 14:41 |
| bbezak_2 | #endmeeting | 14:41 |
| opendevmeet | Meeting ended Wed Feb 26 14:41:10 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:41 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2025/kolla.2025-02-26-14.00.html | 14:41 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2025/kolla.2025-02-26-14.00.txt | 14:41 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2025/kolla.2025-02-26-14.00.log.html | 14:41 |
| r-krcek | thank you | 14:41 |
| koperg[m] | !nick | 14:43 |
| opendevmeet | koperg[m]: Error: You don't have the admin capability. If you think that you should have this capability, be sure that you are identified before trying again. The 'whoami' command can tell you if you're identified. | 14:43 |
| kevko | I will build and upload python3-docker to debian sid and canonical will take it | 14:43 |
| mhiner | Thank you | 14:43 |
| yuval | thank you | 14:44 |
| mostafa | Hi guys. | 15:05 |
| mostafa | Is the meeting in progress? | 15:05 |
| bbezak | Hi - meeting is already finished, it is being held on 14:00 UTC | 15:09 |
| bbezak | https://etherpad.opendev.org/p/KollaWhiteBoard#L45 | 15:09 |
| mostafa-s | is the meeting ended? | 15:11 |
| mostafa-s | ??? | 15:13 |
| kevko | yep | 15:19 |
| kevko | but feel free to ask anything | 15:20 |
| shermanm | hey, I wanted to ask if anyone had thoughts on best practices for maintaining "downstream" forks/variants based off of kolla | 15:35 |
| shermanm | I know we maintain a number of project branches for our own consumption, and figure we can't be the only ones | 15:35 |
| shermanm | it generates a significant (and expected) workload to set up CI/CD, testing, etc. internally, and eventually I realized it seemed that we were redoing a lot of the work that had already gone into e.g. zuul upstream | 15:35 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible master: Bump ansible-core versions to 2.17 and 2.18 https://review.opendev.org/c/openstack/kolla-ansible/+/941787 | 15:41 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix Redis Sentinel authentication for octavia's jobboard HA https://review.opendev.org/c/openstack/kolla-ansible/+/942799 | 15:43 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible master: Bump ansible-core versions to 2.17 and 2.18 https://review.opendev.org/c/openstack/kolla-ansible/+/941787 | 16:14 |
| opendevreview | Grzegorz Koper proposed openstack/ansible-collection-kolla master: Installing packaging as a dependency for pip module in ansible-core 2.16+ https://review.opendev.org/c/openstack/ansible-collection-kolla/+/942828 | 16:24 |
| opendevreview | Grzegorz Koper proposed openstack/kolla-ansible master: Bump ansible-core versions to 2.17 and 2.18 https://review.opendev.org/c/openstack/kolla-ansible/+/941787 | 16:30 |
| opendevreview | Grzegorz Koper proposed openstack/ansible-collection-kolla master: Installing packaging as a dependency for pip module in ansible-core 2.16+ https://review.opendev.org/c/openstack/ansible-collection-kolla/+/942828 | 16:45 |
| opendevreview | Roman Krcek proposed openstack/ansible-collection-kolla master: Add uninstall tasks https://review.opendev.org/c/openstack/ansible-collection-kolla/+/925083 | 18:29 |
| tafkamax | Hi. I am using kolla-ansible to deploy openstack 2024.2. I accidentaly restarted docker service on my main controller and now I am getting weird stuff happening. Every few minutes some services state goes to down. Heat service and Designate service for example. | 19:27 |
| tafkamax | the heat service shows 2025-02-26 21:25:10.883 790 ERROR oslo_messaging._drivers.amqpdriver [None req-599b52eb-f8d0-451e-8da6-90aa62f096e4 admin - - - - -] The reply b5ca46e3ce584243aadd3aa6fd65c712 failed to send after 60 seconds due to a missing queue (reply_436718ccf84c41b89e3ba5ecd055f44d). Abandoning...: oslo_messaging.exceptions.MessageUndeliverable | 19:27 |
| tafkamax | This is the heat-engine.log | 19:28 |
| tafkamax | The main controller rabbitmq shows the following. | 19:29 |
| tafkamax | 2025-02-26 21:28:32.239189+02:00 [info] <0.1208.0> Asking leader {'%2F_engine_worker.06e5af4d-520c-490a-9531-831861009ac0','rabbit@stack-ctrl-02'} to resend from 7 | 19:29 |
| tafkamax | 2025-02-26 21:28:39.301273+02:00 [info] <0.992.0> queue 'cinder-backup' in vhost '/': term mismatch - follower had entry at 8 with term 1 but not with term 6 | 19:29 |
| tafkamax | 2025-02-26 21:28:39.301273+02:00 [info] <0.992.0> Asking leader {'%2F_cinder-backup','rabbit@stack-ctrl-02'} to resend from 10 | 19:29 |
| tafkamax | I don't know what to exactly look for. | 19:29 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix Redis Sentinel authentication for octavia's jobboard HA https://review.opendev.org/c/openstack/kolla-ansible/+/942799 | 19:45 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add oslo.messaging Queue Manager https://review.opendev.org/c/openstack/kolla-ansible/+/924623 | 22:30 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Allow ignoring missing containers in service-stop https://review.opendev.org/c/openstack/kolla-ansible/+/939657 | 22:30 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Rabbitmq: enable quorum for transient queues https://review.opendev.org/c/openstack/kolla-ansible/+/927497 | 22:30 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: rabbitmq: Add support for using stream queues for fanout https://review.opendev.org/c/openstack/kolla-ansible/+/916911 | 22:30 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Remove om_enable_rabbitmq_high_availability https://review.opendev.org/c/openstack/kolla-ansible/+/940690 | 22:30 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Don't allow quorum queues to be disabled https://review.opendev.org/c/openstack/kolla-ansible/+/940691 | 22:30 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Upgrade RabbitMQ docs for the Epoxy release https://review.opendev.org/c/openstack/kolla-ansible/+/940692 | 22:30 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Log DNS_ASSIGNMENT to see what is happening https://review.opendev.org/c/openstack/kolla-ansible/+/942867 | 22:30 |
| *** jhorstmann is now known as Guest10221 | 23:03 | |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Log DNS_ASSIGNMENT to see what is happening https://review.opendev.org/c/openstack/kolla-ansible/+/942867 | 23:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add oslo.messaging Queue Manager https://review.opendev.org/c/openstack/kolla-ansible/+/924623 | 23:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Allow ignoring missing containers in service-stop https://review.opendev.org/c/openstack/kolla-ansible/+/939657 | 23:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Rabbitmq: enable quorum for transient queues https://review.opendev.org/c/openstack/kolla-ansible/+/927497 | 23:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: rabbitmq: Add support for using stream queues for fanout https://review.opendev.org/c/openstack/kolla-ansible/+/916911 | 23:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Remove om_enable_rabbitmq_high_availability https://review.opendev.org/c/openstack/kolla-ansible/+/940690 | 23:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Don't allow quorum queues to be disabled https://review.opendev.org/c/openstack/kolla-ansible/+/940691 | 23:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Upgrade RabbitMQ docs for the Epoxy release https://review.opendev.org/c/openstack/kolla-ansible/+/940692 | 23:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Log output from extension list https://review.opendev.org/c/openstack/kolla-ansible/+/942867 | 23:10 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add oslo.messaging Queue Manager https://review.opendev.org/c/openstack/kolla-ansible/+/924623 | 23:10 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Allow ignoring missing containers in service-stop https://review.opendev.org/c/openstack/kolla-ansible/+/939657 | 23:10 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Rabbitmq: enable quorum for transient queues https://review.opendev.org/c/openstack/kolla-ansible/+/927497 | 23:10 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: rabbitmq: Add support for using stream queues for fanout https://review.opendev.org/c/openstack/kolla-ansible/+/916911 | 23:10 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Remove om_enable_rabbitmq_high_availability https://review.opendev.org/c/openstack/kolla-ansible/+/940690 | 23:10 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Don't allow quorum queues to be disabled https://review.opendev.org/c/openstack/kolla-ansible/+/940691 | 23:10 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Upgrade RabbitMQ docs for the Epoxy release https://review.opendev.org/c/openstack/kolla-ansible/+/940692 | 23:10 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Log output from extension list https://review.opendev.org/c/openstack/kolla-ansible/+/942867 | 23:18 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add oslo.messaging Queue Manager https://review.opendev.org/c/openstack/kolla-ansible/+/924623 | 23:18 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Allow ignoring missing containers in service-stop https://review.opendev.org/c/openstack/kolla-ansible/+/939657 | 23:18 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Rabbitmq: enable quorum for transient queues https://review.opendev.org/c/openstack/kolla-ansible/+/927497 | 23:18 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: rabbitmq: Add support for using stream queues for fanout https://review.opendev.org/c/openstack/kolla-ansible/+/916911 | 23:18 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Remove om_enable_rabbitmq_high_availability https://review.opendev.org/c/openstack/kolla-ansible/+/940690 | 23:18 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Don't allow quorum queues to be disabled https://review.opendev.org/c/openstack/kolla-ansible/+/940691 | 23:18 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Upgrade RabbitMQ docs for the Epoxy release https://review.opendev.org/c/openstack/kolla-ansible/+/940692 | 23:18 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!