| yusufgungor | Hi everyone, we are using zed version openstack with oslo.messaging version is 14.0.1 for all services. | 07:58 |
|---|---|---|
| yusufgungor | Our rabbitmq installed and configured via kolla-ansible. | 07:58 |
| yusufgungor | "amqp_durable_queues = true" is exist for all services under oslo_messaging_rabbit sections. | 07:58 |
| yusufgungor | Also there exist a ha-all policy with "ha-mode: all", "ha-promote-on-shutdown: always" and pattern ^(?!(amq\.)|(.*_fanout_)|(reply_)).* | 07:58 |
| yusufgungor | After rebooting the all controller nodes one by one, some reply queues is not recreated. So the not HA queues like reply is not recreated. | 07:58 |
| yusufgungor | Do you have any idea to fix? | 07:58 |
| yusufgungor | Logs like below: | 07:58 |
| yusufgungor | /var/log/kolla/nova/nova-conductor.log:2023-08-16 10:52:13.970 27 WARNING oslo_messaging._drivers.amqpdriver [None req-bda2bd6c-5c18-48ce-924c-6d9af2fb28b6 - - - - - -] reply_60d6e5b31df946a391621751856800ce doesn't exist, drop reply to 29951416f27546cdbbc70146e619439b: oslo_messaging.exceptions.MessageUndeliverable | 07:58 |
| yusufgungor | /var/log/kolla/nova/nova-conductor.log:2023-08-16 10:52:13.971 27 ERROR oslo_messaging._drivers.amqpdriver [None req-bda2bd6c-5c18-48ce-924c-6d9af2fb28b6 - - - - - -] The reply 29951416f27546cdbbc70146e619439b failed to send after 60 seconds due to a missing queue (reply_60d6e5b31df946a391621751856800ce). Abandoning...: oslo_messaging.exceptions.MessageUndeliverable | 07:58 |
| yusufgungor | /var/log/kolla/neutron/neutron-server.log:2023-08-15 17:36:24.333 40 ERROR oslo_messaging._drivers.amqpdriver [None req-0b7ed5e1-9488-4095-8d92-c9aeb333f011 - - - - - -] The reply d79b3d1485574cf9b0f418f0e920421c failed to send after 60 seconds due to a missing queue (reply_c8d743b4e7184a7ca0be09c43ed03608). Abandoning...: oslo_messaging.exceptions.MessageUndeliverable | 07:58 |
| yusufgungor | /var/log/kolla/neutron/neutron-server.log:2023-08-15 17:36:47.533 54 WARNING oslo_messaging._drivers.amqpdriver [None req-be715e9b-7139-4e92-9e78-a103ec828d9d - - - - - -] reply_5c0c508ae6a44d6c859b21b9eec8a828 doesn't exist, drop reply to 14f52a5591a24f0890b0c956c799b750: oslo_messaging.exceptions.MessageUndeliverable | 07:58 |
| opendevreview | Arnaud Morin proposed openstack/oslo.messaging master: Enable use of quorum queues for transient messages https://review.opendev.org/c/openstack/oslo.messaging/+/888479 | 08:45 |
| opendevreview | Arnaud Morin proposed openstack/oslo.messaging master: Auto-delete the failed quorum rabbit queues https://review.opendev.org/c/openstack/oslo.messaging/+/889313 | 08:45 |
| opendevreview | Arnaud Morin proposed openstack/oslo.messaging master: Allow creating transient queues with no expire https://review.opendev.org/c/openstack/oslo.messaging/+/889701 | 08:45 |
| opendevreview | Arnaud Morin proposed openstack/oslo.messaging master: Add some logs when sending RPC messages https://review.opendev.org/c/openstack/oslo.messaging/+/890824 | 08:45 |
| opendevreview | Arnaud Morin proposed openstack/oslo.messaging master: Add QManager to amqp driver https://review.opendev.org/c/openstack/oslo.messaging/+/889425 | 08:45 |
| opendevreview | Arnaud Morin proposed openstack/oslo.messaging master: Add an option to use rabbitmq stream for fanout queues https://review.opendev.org/c/openstack/oslo.messaging/+/890825 | 08:45 |
| opendevreview | Tobias Urdin proposed openstack/taskflow master: Update redis dep to >=3.0.0 https://review.opendev.org/c/openstack/taskflow/+/891597 | 13:58 |
| JayF | Is there anyone else who can review https://review.opendev.org/c/openstack/oslo.messaging/+/891096 other than stephenfin? We need another +2 from an oslo core to land this CVE fix | 14:53 |
| JayF | stephenfin: I'm working on your follow up request, and notice that https://github.com/openstack/oslo.context/blob/master/oslo_context/context.py#L357 exists | 15:30 |
| JayF | stephenfin: I kinda think there's value, for notifications, in doing something more hydated (with service_* info) | 15:30 |
| JayF | but I wanted to point out it exists, and give a chance for someone to scream before I finish implementing a new one lol | 15:30 |
| JayF | I've convinced myself a new method is better, because getting back a useful, redacted context is a different thing than returning a dict of values for logging | 15:34 |
| opendevreview | Jay Faulkner proposed openstack/oslo.context master: Add method for getting redacted copy of context https://review.opendev.org/c/openstack/oslo.context/+/891608 | 15:36 |
| opendevreview | Jay Faulkner proposed openstack/oslo.messaging master: If possible, use context method for sanitizing https://review.opendev.org/c/openstack/oslo.messaging/+/891610 | 15:46 |
| opendevreview | Jay Faulkner proposed openstack/oslo.messaging master: If possible, use context method for sanitizing https://review.opendev.org/c/openstack/oslo.messaging/+/891610 | 15:47 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!