| jrosser | good morning | 07:53 |
|---|---|---|
| admin1 | good morning | 08:17 |
| admin1 | my repo is completely broke :( | 08:18 |
| admin1 | is there a variable to disable this gluster thing and go back to the old lsyncd ? | 08:18 |
| jrosser | admin1: no there is not | 08:22 |
| jrosser | you need to do mainatianance on the gluster cluster using the gluster cli | 08:22 |
| jrosser | lsyncd was not maintained and not available on all the distros we support | 08:23 |
| jrosser | and also that approach could not support deployments with multiple architectures at all | 08:24 |
| jrosser | if you provide some other shared filesystem yourself then you can replace the glusterfs mounts with that instead | 08:25 |
| jrosser | you can set `openstack_repo_server_enable_glusterfs` to False in your user variables | 08:27 |
| jrosser | and then define this https://github.com/openstack/openstack-ansible-repo_server/blob/master/defaults/main.yml#L56-L61 | 08:28 |
| jrosser | but then imho you just move the problem of providing a high availability shared filesystem to somewhere else | 08:28 |
| admin1 | https://paste.openstack.org/show/bMg76EExbS5gXQktdCCt/ is where i am stuck at | 08:31 |
| admin1 | and i have no idea even where to begin .. | 08:31 |
| admin1 | the gluster is in the controllers ? | 08:31 |
| admin1 | jrosser can it also use nfs ? | 08:34 |
| admin1 | i can try nfs if i can get some info on how to set it to use nfs | 08:35 |
| admin1 | oh .. i can just use nfs manually, mount and provide the share ? | 08:37 |
| jrosser | admin1: did you try at all any of the steps here that i gave you? https://docs.rackspace.com/support/how-to/recover-from-a-failed-server-in-a-glusterfs-array/ | 09:17 |
| jrosser | simple things like this https://paste.opendev.org/show/b9zsAHsNGAs0FgZnx2DJ/ | 09:20 |
| derekokeeffe | Hi all, back again. If anyone has a spare min could they possibly take a look at this? https://paste.openstack.org/show/bKFLFmSmrOf2qu1WgQYr/ please and thanks in advance | 10:36 |
| jrosser | derekokeeffe: i'm not sure - i can visit /project on my test deployment | 10:56 |
| jrosser | but ultimately neutronclient is getting 503 | 10:56 |
| derekokeeffe | Hmm, ok. Thanks for taking a look. Strange that it works fine once I change the URL manually | 10:56 |
| jrosser | so i would start by looking at the status of the neutron API on haproxy | 10:57 |
| NeilHanlon | jrosser++ that last time I saw that, derekokeeffe, was when my neutron container was all messed up | 13:38 |
| admin1 | thanks jrosser .. finally fixed gluster | 13:48 |
| admin1 | know a lot more about it now :D | 13:48 |
| jrosser | admin1: excellent! it's not that bad really - just a bunch more stuff to learn :) | 13:49 |
| admin1 | NeilHanlon , from the util, you can do things like openstack server list, network list, loadbalancer list etc | 13:49 |
| admin1 | to test all the APIs | 13:49 |
| admin1 | if none give 503 or issues, then horizon will work just fine | 13:49 |
| admin1 | if any one of it breaks ( your services) , horizon will complain | 13:49 |
| admin1 | so i try to do a cli command with --verbose first to check if all the services api are responding properly | 13:49 |
| jrosser | admin1: the way we use gluster, the node UUID is stored outside the container on the host, and persisting that is what allows the repo container to be easily deleted/recreated without having to manually fix the gluster stuff | 13:50 |
| derekokeeffe | Thanks jrosser and NeilHanlon, I'll take a look there so | 13:50 |
| jrosser | but when you lose the host you also lose the UUID and that then needs manual intervention, as the playbooks don't have a way to understand what has happened | 13:50 |
| jrosser | admin1: here is how the gluster state ends up on the host, through a bind mount https://github.com/openstack/openstack-ansible/blob/master/inventory/group_vars/repo_all.yml#L22-L25 | 13:53 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Remove conditional code for ubuntu version earlier than 20.x https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882826 | 14:06 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Allow ceph client keyring files to have custom owner/group/mode https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882827 | 14:06 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Allow ceph clients to be enabled or disabled https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882828 | 14:06 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add ceph client and configuration for immutable object cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882829 | 14:06 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add immutible object cache documentation https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882830 | 14:06 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Add ceph client key for immutable object cache https://review.opendev.org/c/openstack/openstack-ansible/+/882831 | 14:09 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add immutible object cache documentation https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882830 | 14:12 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add immutable object cache documentation https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882830 | 14:12 |
| admin1 | i stopped glusterd in all , deleted all the files and redid the repo install | 14:16 |
| admin1 | it started blank | 14:16 |
| admin1 | running setup openstack to let it build all | 14:17 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add immutable object cache documentation https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882830 | 14:47 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add config and documentation for ceph perisistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882849 | 14:47 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add releasenote for ceph immutable object cache and persistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882850 | 14:52 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Remove redundant `repo_server_enable_sync_manager` variable https://review.opendev.org/c/openstack/openstack-ansible/+/882862 | 15:00 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add immutable object cache documentation https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882830 | 15:30 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add config and documentation for ceph perisistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882849 | 15:30 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add releasenote for ceph immutable object cache and persistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882850 | 15:30 |
| opendevreview | Alfredo Moralejo proposed openstack/openstack-ansible-openstack_hosts master: Add support to install distro packages from RDO CloudSIG repos https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/880473 | 15:57 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add ceph client and configuration for immutable object cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882829 | 16:49 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add immutable object cache documentation https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882830 | 16:49 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add config and documentation for ceph perisistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882849 | 16:49 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add config and documentation for ceph perisistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882849 | 16:50 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add releasenote for ceph immutable object cache and persistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882850 | 16:50 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add immutable object cache documentation https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882830 | 17:11 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add config and documentation for ceph perisistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882849 | 17:12 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add ceph client and configuration for immutable object cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882829 | 18:01 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add immutable object cache documentation https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882830 | 18:01 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add config and documentation for ceph perisistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882849 | 18:02 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add releasenote for ceph immutable object cache and persistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882850 | 18:02 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add config and documentation for ceph perisistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882849 | 18:03 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client master: Add releasenote for ceph immutable object cache and persistent write log cache https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882850 | 18:03 |
| noonedeadpunk | Just to have that said, I'm not sure sure about https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882826 that's the base for everything else. | 21:41 |
| NeilHanlon | noonedeadpunk: while you're here.. does https://review.opendev.org/c/openstack/openstack-ansible/+/881665/2 need backport? and is it cool if I +2 that? | 21:43 |
| noonedeadpunk | I'm not 100% sure about the backport, but yeah it looks like the right thing to do if you ask me :) | 21:44 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!