| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-hardening master: Auto-fix yaml rules https://review.opendev.org/c/openstack/ansible-hardening/+/941501 | 06:46 |
|---|---|---|
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-hardening master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/ansible-hardening/+/941502 | 06:54 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-memcached_server master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-memcached_server/+/941503 | 06:59 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-memcached_server master: Auto-fix yaml rules https://review.opendev.org/c/openstack/openstack-ansible-memcached_server/+/941504 | 07:00 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_openrc master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-openstack_openrc/+/941505 | 07:04 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-memcached_server master: Auto-fix yaml rules https://review.opendev.org/c/openstack/openstack-ansible-memcached_server/+/941503 | 07:04 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-memcached_server master: Auto-fix yaml rules https://review.opendev.org/c/openstack/openstack-ansible-memcached_server/+/941504 | 07:04 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-memcached_server master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-memcached_server/+/941504 | 07:04 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server master: Auto-fix yaml rules https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/941507 | 07:42 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/941509 | 07:44 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-systemd_service master: Auto-fix yaml rules https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/941512 | 08:00 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-systemd_service master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/941513 | 08:02 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-systemd_mount master: Auto-fix yaml rules https://review.opendev.org/c/openstack/ansible-role-systemd_mount/+/941514 | 08:04 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-systemd_mount master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/ansible-role-systemd_mount/+/941515 | 08:06 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-systemd_service master: Update requirements.yml to be installable without force flag https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/941517 | 08:09 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-systemd_networkd master: Auto-fix yaml rules https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/941518 | 08:12 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-systemd_networkd master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/941520 | 08:16 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-python_venv_build master: Auto-fix yaml rules https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/941521 | 08:20 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-python_venv_build master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/941522 | 08:24 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: Auto-fix yaml rules https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/941523 | 08:27 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/941524 | 08:29 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun master: Auto-fix yaml rules https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/941438 | 08:32 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/941440 | 08:32 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_hosts master: Auto-fix yaml rules https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/941528 | 08:36 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_hosts master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/941530 | 08:41 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_container_create master: Auto-fix yaml rules https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/941532 | 08:44 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_container_create master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/941533 | 08:46 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Auto-fix yaml rules https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/941537 | 08:57 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/941538 | 08:59 |
| admin1 | how do i downgrade from 3 controllers to just 1? | 11:07 |
| admin1 | upgrade will be easy .. just to add the rest of the hosts and the playbooks take care of it | 11:07 |
| admin1 | i want to downgrade to 1, reformat and redo the rest of the contorllers as virtual and then add them back | 11:08 |
| mgariepy | probably easier to do one a the time like we do for OS upgrade, | 12:13 |
| mgariepy | https://docs.openstack.org/openstack-ansible/latest/admin/upgrades/distribution-upgrades.html | 12:14 |
| noonedeadpunk | admin1: I think most problematic would be rabbitmq and memcached, as they're defined as list of IPs in all services | 13:44 |
| noonedeadpunk | so you'd need to reconfigure pretty much everything to stop services using them. I think I've described the way of doing that once in some ML... | 13:44 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: Auto-fix yaml rules https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/941523 | 15:51 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/941524 | 15:51 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_container_create master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/941533 | 15:53 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-ceph_client master: Auto-fix usage of modules via FQCN https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/941538 | 15:54 |
| jrosser | noonedeadpunk: https://paste.opendev.org/show/bUphct61nP30MIinYdHa/ | 17:07 |
| jrosser | with latest ansible there seems to be some horrible performance problem where ansible-galaxy iterates every single file in the virtualenv and git dirs | 17:08 |
| birbilakos | hello | 18:46 |
| birbilakos | After some network changes in my 2023.2 deployment which broke network connectivity to one controller node (and reverting that back) I get bombarded by these type of errors: https://paste.opendev.org/show/b8QRG6bxWiKh9CLM79oR/ | 18:48 |
| birbilakos | i also see that several nova actions either timeout or are very slow to go through. any ideas how to recover this? | 18:48 |
| birbilakos | I was thinking rerunning the rabbitmq playbook to re-deploy the service on the controllers and potentially wipe any remnants. Is that a good idea? | 18:49 |
| birbilakos | did try restarting the rabbitmq containers - same behavior | 18:49 |
| noonedeadpunk | um, could be worth running rabbitmq-install -e rabbitmq_upgrade=true | 19:23 |
| noonedeadpunk | as I think it's not ha queus yet | 19:23 |
| birbilakos | this is in a 3 controller node setup | 19:28 |
| birbilakos | will this re-create the containers too? | 19:28 |
| jrosser | it’s worth checking through our docs for rabbitmq mainatainance as well so that you can check the cluster status | 19:33 |
| jrosser | and also setting up the rabbitmq management web interface so you can inspect what’s happening | 19:33 |
| birbilakos | did run the playbook but nothing changed :( | 19:38 |
| birbilakos | some messages are getting through it seems, some others not | 19:38 |
| jrosser | there is no substitute for understanding the root cause of | 19:44 |
| birbilakos | i deployed the management interface, any ideas what are the creds? | 19:49 |
| birbilakos | got in as monitoring :) | 19:51 |
| jrosser | it should all be in the user_secrets file on the deploy host | 19:55 |
| birbilakos | yeap :) | 19:55 |
| birbilakos | nothing seems off in the management interface | 19:55 |
| birbilakos | all green and whatnot | 19:56 |
| birbilakos | what can be causing these type of errors if not rabbitmq? | 19:56 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!