| opendevreview | James Denton proposed openstack/openstack-ansible master: [WIP] Update documentation for LXC/metal and LXB/OVS/OVN https://review.opendev.org/c/openstack/openstack-ansible/+/867577 | 01:28 |
|---|---|---|
| jrosser | morning | 08:25 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic master: Update ironic documentation https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/867547 | 08:39 |
| noonedeadpunk | o/ | 09:05 |
| jrosser | morning | 09:36 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump OpenStack-Ansible for Zed release https://review.opendev.org/c/openstack/openstack-ansible/+/867618 | 09:50 |
| noonedeadpunk | I guess it's time I've reviewed PKI patches again.... | 09:52 |
| noonedeadpunk | Ok, so https://review.opendev.org/c/openstack/ansible-role-pki/+/867542 seems quite good regrdless of others I would say | 09:58 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-pki master: Ensure CA privatekey permissions https://review.opendev.org/c/openstack/ansible-role-pki/+/867555 | 10:03 |
| noonedeadpunk | jrosser: your idea with file seems quite good as well regardless of everything else ^ | 10:04 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-role-pki master: Allow to define mode and ownership for CA private keys https://review.opendev.org/c/openstack/ansible-role-pki/+/867553 | 10:27 |
| noonedeadpunk | meh, everything looks good to me | 10:35 |
| noonedeadpunk | Only not sure about https://review.opendev.org/c/openstack/ansible-role-pki/+/867549/ indeed. But it's also quite fair I would say | 10:36 |
| noonedeadpunk | But can become too complicated in the future | 10:36 |
| noonedeadpunk | So maybe worth avoiding it indeed | 10:36 |
| *** dviroel|out is now known as dviroel|rover | 11:12 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic master: Update ironic documentation https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/867547 | 11:50 |
| kleini | Is it true, that ceph_client/tasks/ceph_auth_extra_compute.yml is only meant to be working, if OSA has access to mon hosts of Ceph? I defined keyrings with keyring_src and keyring_dest and that only works with Glance and Cinder but not Nova. Would it make sense to extend ceph_auth_extra_compute.yml to support hat? I don't have access to mon hosts of Ceph instances. | 12:22 |
| noonedeadpunk | Um, I'm not sure. I had setup without access to mon hosts only in multinode sandbox, but it worked nicely | 12:41 |
| opendevreview | Merged openstack/ansible-role-pki master: Ensure CA privatekey permissions https://review.opendev.org/c/openstack/ansible-role-pki/+/867555 | 12:42 |
| noonedeadpunk | also looking at https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/866974/1/tasks/ceph_auth_extra_compute.yml now - file: absent should not fail if file doesn't exist, should it? | 12:42 |
| noonedeadpunk | also I haven't seen failures in that sandbox.... | 12:43 |
| noonedeadpunk | hm.... Seems smth off with neutron on metal :( | 13:00 |
| noonedeadpunk | https://zuul.opendev.org/t/openstack/build/e56e5d93df0043089e47447039d966a7/log/logs/host/neutron-server.service.journal-11-08-38.log.txt#2453 | 13:00 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump OpenStack-Ansible for Zed release https://review.opendev.org/c/openstack/openstack-ansible/+/867618 | 13:03 |
| noonedeadpunk | damn it :( | 13:10 |
| noonedeadpunk | and how it happens that lxc works at the same time.... | 13:11 |
| opendevreview | Damian Dąbrowski proposed openstack/ansible-role-pki stable/zed: Ensure CA privatekey permissions https://review.opendev.org/c/openstack/ansible-role-pki/+/867631 | 13:12 |
| mgariepy | it's weird. | 13:12 |
| opendevreview | Damian Dąbrowski proposed openstack/ansible-role-pki stable/yoga: Ensure CA privatekey permissions https://review.opendev.org/c/openstack/ansible-role-pki/+/867632 | 13:13 |
| opendevreview | Damian Dąbrowski proposed openstack/ansible-role-pki stable/xena: Ensure CA privatekey permissions https://review.opendev.org/c/openstack/ansible-role-pki/+/867633 | 13:13 |
| opendevreview | Damian Dąbrowski proposed openstack/ansible-role-pki stable/wallaby: Ensure CA privatekey permissions https://review.opendev.org/c/openstack/ansible-role-pki/+/867634 | 13:14 |
| jamesdenton | there are some ovn-related errors in neutron-server log, and "connection dropped (Protocol error)" in the ovn logs | 13:42 |
| mgariepy | almost 10 minutes before ? | 13:59 |
| noonedeadpunk | and eventually SQL query is exactly what timeouts in role | 14:00 |
| jamesdenton | true | 14:00 |
| jamesdenton | i've got an AIO spinning up, might be able to replicate soon | 14:00 |
| mgariepy | why does the instances info seems to stops at 10:25 ? | 14:04 |
| mgariepy | https://6f6f67e9fba8f2d009b9-fd1d23de4951783ce4905cf96230b70e.ssl.cf2.rackcdn.com/867618/1/check/openstack-ansible-deploy-aio_metal-ubuntu-jammy/e56e5d9/logs/openstack/instance-info/index.html | 14:05 |
| mgariepy | we do have logs in the services for up to 11:08 at least | 14:05 |
| kleini | noonedeadpunk: https://paste.opendev.org/show/b6mTLzi6IfUaxtfj74hi/ <- this is my current configuration for two ceph instances without having access to mon hosts. everything needs to be migrated: images and volumes | 14:14 |
| kleini | therefore I have two extra confs, one for glance and one for cinder/Nova | 14:15 |
| kleini | the extra ceph conf for glance does not have a secret_uuid and therefore those two cleanup tasks of ceph_auth_extra_compute.yml fail | 14:16 |
| noonedeadpunk | well, I used ceph cluster_name to separate these things | 14:16 |
| kleini | so, you have an easier configuration for this scenario? | 14:16 |
| noonedeadpunk | So I set `ceph_cluster_name` for group_vars | 14:16 |
| noonedeadpunk | well, my scenario was different clusters per AZ. | 14:17 |
| noonedeadpunk | Though I had https://opendev.org/openstack/openstack-ansible-ceph_client/commit/b3e7560e8022384b8269d1b380aae9602cea1824 | 14:17 |
| kleini | I need two Ceph instances in the same AZ. We need to migrate all images and volumes from "old" Ceph to new one. | 14:17 |
| noonedeadpunk | It doesn't really matter I guess, as it's more about path of configs that are expected | 14:18 |
| noonedeadpunk | As I placed ceph_cluster_name per az groups, but you can in cinder_all and glance_all | 14:19 |
| kleini | I don't see, that there will be two secrets registered in libvirt. As said: some part of volumes is in Ceph A and some in Ceph B. There can be VMs on the same compute host using maybe even two volumes whereof one is one Ceph A and one on Ceph B. | 14:21 |
| kleini | I looked all the way through ceph_auth_extra_compute.yml (on W and master) where a secret is created from keyring_src or keyring_dest in ceph_extra_confs and that is what I need if I don't have access to mon hosts to fetch/create keyrings there | 14:24 |
| kleini | I need Ceph A and B completely working in Glance, Cinder and Nova as I need a longer time frame to be able to migrate all images and volumes in Ceph. This works fine for Glance and Cinder with the above configuration but I am currently missing the secret of Ceph B with ceph2-cinder user in libvirt on the compute node. | 14:35 |
| kleini | noonedeadpunk: any thoughs on this? If not, I would try to extend ceph_auth_extra_compute.yml to add the ceph2-cinder secret to libvirt according to this configuration: https://paste.opendev.org/show/b6mTLzi6IfUaxtfj74hi/ | 14:53 |
| noonedeadpunk | jamesdenton: that looks like neutron bug to me. as once I've roledback SHA CI is happy | 15:03 |
| jamesdenton | ahh, interesting | 15:04 |
| noonedeadpunk | kleini: ah, ok, now I see what you're doing | 15:05 |
| noonedeadpunk | yeah, I think now we assume that only one ceph cluster is used per service (not multiple ones for same service) | 15:05 |
| noonedeadpunk | and interesting indeed, that lxc jobs passed, to it don't like ovn with api on same place kind of | 15:13 |
| jamesdenton | noonedeadpunk FWIW - i performed a local aio_metal deployment with 867618 and it seemed to work OK | 15:14 |
| noonedeadpunk | huh | 15:15 |
| noonedeadpunk | but well - what SHA was there? | 15:15 |
| noonedeadpunk | As I've rolled back now to previous neutron version | 15:15 |
| jamesdenton | how long ago? | 15:15 |
| noonedeadpunk | quite a while... | 15:16 |
| noonedeadpunk | `neutron_git_install_branch: 6927dfbb3e52ba298928362da5cce32d49b1e1f8` is what's broken in CI | 15:16 |
| noonedeadpunk | `neutron_git_install_branch: add538d7ff26d843fd43ca85f7dca9385dc3ecc1` is passing | 15:16 |
| noonedeadpunk | Changed at 2pm UTC | 15:17 |
| jamesdenton | ahh gotcha, yeah i've got add538d7ff26d843fd43ca85f7dca9385dc3ecc1 | 15:17 |
| jamesdenton | well ,then nevermind :) | 15:17 |
| jamesdenton | i missed your rollback | 15:17 |
| *** dviroel|rover is now known as dviroel|rover|lunch | 15:52 | |
| noonedeadpunk | we need to merge https://review.opendev.org/c/openstack/openstack-ansible/+/867618/2 sooner better | 16:15 |
| jamesdenton | ahh another zuul ui change | 16:24 |
| mgariepy | why do they mode the Backport stuff lol | 16:30 |
| *** dviroel|rover|lunch is now known as dviroel|rover | 16:38 | |
| noonedeadpunk | I guess you meant gerrit :) | 16:40 |
| mgariepy | move** bacport candiate box is right were the +w was.. | 16:47 |
| jamesdenton | yes, gerrit, sorry | 16:55 |
| noonedeadpunk | I more wonder about switch of +W and +V as they're even not alphabetic now | 16:58 |
| noonedeadpunk | As for backport - since it's not required for merge, it's hidden at all until it has some vote | 16:58 |
| opendevreview | Merged openstack/ansible-role-pki master: Backup CA key and certs by default https://review.opendev.org/c/openstack/ansible-role-pki/+/867542 | 18:36 |
| noonedeadpunk | fwiw I don't see keystone failures anymore | 20:29 |
| noonedeadpunk | not sure it was haproxy or workers fix, but it's not an issue anymore from what I see | 20:29 |
| jrosser | oh so close on merging the release patch | 20:46 |
| *** tosky_ is now known as tosky | 21:12 | |
| *** dviroel|rover is now known as dviroel|out | 21:31 | |
| noonedeadpunk | yup( | 21:37 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!