Wednesday, 2023-07-05

hyperbabaHi there,07:56
hyperbabaI have a deployment of openstack using kolla-ansible. I lost one of the controller nodes due to hardware failure and created a new server. Now i'm having issues with etcd container which is restarting compleaining that member xxxxxx is already bootstrapped. Can anyone assist me with the issue?08:00
yusufgungorHi @hyperbaba you may try to remove failed etcd member and join again using etcdctl. There exist k8s etcd docs,  i think they can adaptable for openstack too https://itnext.io/breaking-down-and-fixing-etcd-cluster-d81e35b9260d08:43
SvenKieskeI think this is mainly a etcd question for the etcd community, not really kolla-ansible. you can of course also discuss/ask here, but you might get more responses in the etcd community I guess?08:59
frickleralthough it wouldn't hurt if kolla could detect and maybe even resolve this situation. contributions welcomed ;)09:16
spatelmnasiadka Hey! are you there buddy? I have a kolla override question - https://paste.opendev.org/show/bztnBpmwRPxTh7h1PNev/14:55
spatelI have this tree and i want to push driver = messagingv2 to all nova service but it doesn't work even i have nova.conf file there14:56
spatelI must be doing something wrong14:56
rockeyspatel: /etc/kolla/config/nova.conf is your friend, not .../config/nova/nova.conf14:58
spatelWhen I move nova.conf to ../config/nova.conf then my ceph implementation break and compute start throwing error saying not able to find ceph key :(14:59
spatelI am also shocked with that.. 15:00
rockeyi have .../config/{nova,neutron}.conf and .../config/nova/ceph.conf + keyrings15:01
spatelwhat about nova-api.conf and nova-scheduler.conf files? 15:02
spatelthey should be inside ../config ?15:02
rockeyspatel: https://pastebin.com/SeHeiqMi15:06
rockeythat's my layout15:06
spatelyou don't have nova-api.conf etc.. assuming that config can go to nova.conf15:07
rockeyshould be in the same place as nova.conf, but untested theory15:08
rockeyie, .../config/nova-api.conf15:08
spatelOkie! let me run some test 15:09
spatelI have notice one more thing that i have /etc/kolla/config/nova/ceph.client.nova.keyring file but it not push out to any node 15:09
spatelall i can see on compute node /etc/ceph/ceph.client.cinder.keyring file 15:10
rockeyverify in globals.yml, but keyring for nova perhaps defaults to same as cinder15:12
rockeyhttps://opendev.org/openstack/kolla-ansible/src/branch/master/etc/kolla/globals.yml#L45515:13
rockeyoh sorry, L46515:13
spatelYes! I can see that nova = cinder15:14
spatelwhen I move nova.conf to ../config/nova.conf and run kolla-ansible reconfigure I started seeing this error on all compute nodes - https://paste.opendev.org/show/bNbXtfpPGZknmdRqPGrg/15:14
spateleven right now I can't see any /etc/ceph/ceph.client.nova.keyring file on compute and it works so trying to hunt it down what is going on15:15
spatelThis is yoga but i have same config in Zed and everything working fine there.. so I am little confuse. anyway let me go through in depth and see 15:16
spatelah! I know what is going on, its nova.conf rbd_user = cinder vs rbd_user = nova issue..15:28

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!