| *** gary_perkins has quit IRC | 01:28 | |
| *** gary_perkins has joined #openstack-ansible | 01:29 | |
| *** kberger_ has quit IRC | 01:51 | |
| *** KeithMnemonic has joined #openstack-ansible | 02:10 | |
| *** gyee has quit IRC | 02:47 | |
| *** rh-jelabarre has quit IRC | 03:12 | |
| *** evrardjp has quit IRC | 04:35 | |
| *** evrardjp has joined #openstack-ansible | 04:36 | |
| *** yolanda has quit IRC | 07:38 | |
| *** yolanda has joined #openstack-ansible | 07:51 | |
| *** pcaruana has quit IRC | 08:09 | |
| *** pcaruana has joined #openstack-ansible | 08:26 | |
| *** pcaruana has quit IRC | 08:44 | |
| *** pcaruana has joined #openstack-ansible | 08:44 | |
| *** gshippey has joined #openstack-ansible | 09:07 | |
| *** tosky has joined #openstack-ansible | 09:30 | |
| *** yolanda has quit IRC | 09:43 | |
| *** spatel has joined #openstack-ansible | 10:18 | |
| *** spatel has quit IRC | 10:23 | |
| *** andrelf has joined #openstack-ansible | 10:24 | |
| *** andrelf has quit IRC | 10:25 | |
| *** andrelf has joined #openstack-ansible | 10:27 | |
| *** andrelf has left #openstack-ansible | 10:27 | |
| *** andrelf has joined #openstack-ansible | 10:27 | |
| *** andrea15 has quit IRC | 10:46 | |
| andrelf | hello. im trying to install a test environment with openstack ansible with 1 infra + 1 compute + 1 storage but for some reason a few a days the playbook setup-hosts.yml was failing in lxc_hosts : Format the machines sparse file but now its falling in lxc_hosts : Ensure that the LXC cache has been prepared when it trys to chroot | 11:05 |
|---|---|---|
| andrelf | /var/lib/machines/debian-10-amd64 /opt/cache-prep-commands.sh because it cant find the apt-listchnages inside the chroot. how can i find where in the tasks should the apt-listchanges been installed in chroot and it wasn't? | 11:05 |
| jrosser | andrelf: the debian container image contains the config file for apt-listchanges but not the package itself, so really the correct answer is to remove the config file, see this https://review.opendev.org/#/c/718882 | 11:28 |
| andrelf | jrosser i was trying to research that problem and it seams that apt-listchanges is installed but it depends on python3 and not python2.7 when i prepared the target hosts like its reported here https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/targethosts.html | 11:36 |
| andrelf | and i will check your link thanks | 11:40 |
| *** rh-jelabarre has joined #openstack-ansible | 11:56 | |
| *** rh-jelabarre has quit IRC | 11:56 | |
| *** rh-jelabarre has joined #openstack-ansible | 11:56 | |
| andrelf | i don't know what what im doing wrong but in this doc https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/targethosts.html you say that if i have the vg lxc it will be used intead of the /var/lib/lxc but what im seasing is that my /var is growing in space and no new logical volumes being created in lxc volume group. what im | 12:18 |
| andrelf | doing wrong? | 12:18 |
| noonedeadpunk | andrelf: I think lxc_container_backing_store should be set to lvm in user_variables... | 12:25 |
| noonedeadpunk | Can you try this out? | 12:26 |
| noonedeadpunk | (Probably we need to update docs afterwards) | 12:26 |
| andrelf | atm i have my /var already at full 100% should i try to delete /var/lib/lxc folder and set that setting and try again? | 12:30 |
| andrelf | noonedeadpunk | 12:30 |
| noonedeadpunk | andrelf: do you have your openstack running now? Or you're jsut in process of deploying it? | 12:32 |
| andrelf | noonedeadpunk still in process of deploying it | 12:33 |
| noonedeadpunk | ok, than you can probably remove created containers which will free up space in /var | 12:34 |
| noonedeadpunk | so run `openstack-ansible playbooks/containers-lxc-destroy.yml` | 12:35 |
| andrelf | oh ok thanks | 12:35 |
| andrelf | noonedeadpunk now its creating the logical volumes thanks for the help. now lets wait and see whats the next task that fails hehhe | 12:40 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [doc] Adjust deploy-guide to install py3 https://review.opendev.org/724836 | 12:54 |
| noonedeadpunk | andrelf: does https://review.opendev.org/#/c/724836/ look like reasonable fix of the doc? | 12:54 |
| andrelf | noonedeadpunk regarding the docs yes, regarding the python3 i haven't tested regarding the apt-listchanges error above. what i have done was to add tthat specific task in https://review.opendev.org/#/c/718882 in the correspondent rule in my deployment host. | 13:14 |
| *** itandops has quit IRC | 13:20 | |
| *** itandops has joined #openstack-ansible | 13:35 | |
| *** Dwane has joined #openstack-ansible | 13:37 | |
| *** Dwane has quit IRC | 13:43 | |
| mgariepy | jrosser, are you using linux bridge ? or you are on a fancier tech ? | 13:45 |
| *** schwicht has quit IRC | 13:46 | |
| *** schwicht has joined #openstack-ansible | 13:47 | |
| jrosser | mgariepy: still linuxbridge here | 13:49 |
| mgariepy | ok | 13:49 |
| jrosser | I would like to spend time looking at ovs offload but always too much other stuff to do | 13:49 |
| mgariepy | in the past i had issue with ha routers so i disabled that, (recovery time was way too long to be useful) | 13:50 |
| jrosser | that would be my motivation to move to ovs, not specifically for ovs for the sake of it but to leverage hardware acceleration | 13:50 |
| mgariepy | i wonder if there is an easy way to be ""seamless"" while reinstalling the network node. moving the router take a few sec (which i'm ok with) but when the resolver gets down it can be noticiable quite a lot until the dhcp agent returns. | 13:51 |
| andrelf | when trying to install openstack-ansible in a similar faction to this example https://docs.openstack.org/openstack-ansible/latest/user/test/example.html the VIP for external and internal comunication should be configured in /etc/network/interfaces in infra host or its configured by ansible to keepalive by default (Task "install keepalived" has been | 14:40 |
| andrelf | omitted)? I'm asking because in the task TASK [python_venv_build : Upgrade pip/setuptools/wheel to the versions we want] it seams to not be able to access to it. | 14:40 |
| noonedeadpunk | andrelf: so VIP is managed with keepalived | 14:45 |
| noonedeadpunk | You should check that keepalived.conf has correct interfaces in it's conf | 14:46 |
| andrelf | in user_variables.yml settings? | 14:46 |
| noonedeadpunk | etc/keepalived/keepalived.conf on ctl hosts | 14:47 |
| noonedeadpunk | if they're not correct, you should adjust them in user_variables with haproxy_keepalived_external_interface and haproxy_keepalived_internal_interface | 14:49 |
| andrelf | because i want to test trying to add new infra host i suppose i should enable haproxy to use keepalvied also even it atm it only uses one ctl host right? | 14:49 |
| andrelf | ok i will check them | 14:49 |
| noonedeadpunk | actually no, then you shouldn't use keepalived | 14:49 |
| noonedeadpunk | But I gues then you need to manually add interfaces | 14:50 |
| noonedeadpunk | *manually add VIP | 14:50 |
| andrelf | so if i only have a ctl host i should many add the vip to interfaces host and only when trying to use multiple ctl hosts, enable keepalived and disable the manually add vip interface? | 14:51 |
| noonedeadpunk | Yeah, I think that's the way | 15:04 |
| noonedeadpunk | Or you can actually try to force isntall it if you want... | 15:05 |
| noonedeadpunk | not not sure this has much sense eventually | 15:05 |
| *** itandops has quit IRC | 15:12 | |
| *** itandops has joined #openstack-ansible | 15:13 | |
| andrelf | ok i have enabled the slave interfaces with internal and external vip and running openstack-ansible -vv setup-infrastructure.yml again | 15:18 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_hosts master: ensure 20listchanges is not in debian container https://review.opendev.org/718882 | 15:25 |
| andrelf | ok now it seams that infra lxc container can't comunicate with infra-host despiste being in same linux bridge | 15:26 |
| andrelf | now it works but still fails in TASK [python_venv_build : Upgrade pip/setuptools/wheel to the versions we want]... | 15:28 |
| andrelf | it seams haproxy is not runing... because i can access repo_all-back... lets see whats the problem with haproxy not starting then | 15:34 |
| noonedeadpunk | hm, actually not sure that haproxy is needed as well when having single node... | 15:36 |
| andrelf | its the default setting because i didn't change anything regarding haproxy. the problem in haproxy is because it cant bind socket to the internal and external vip (itried to configure 2 vip in the same network in br-mgmt). | 15:39 |
| andrelf | but in your https://docs.openstack.org/openstack-ansible/latest/user/test/example.html you define .10 as internal vip and .11 ans external (being the host ip also in br-mgmt) | 15:41 |
| jrosser | andrelf: have you built an all-in-one yet? | 15:42 |
| noonedeadpunk | ah, yeah, as you have lxc you eventually need haproxy | 15:43 |
| *** gyee has joined #openstack-ansible | 15:43 | |
| noonedeadpunk | So binding on the same network should work, as haproxy binds to the IP and it shoudn't care on what network this ip is | 15:45 |
| noonedeadpunk | except smth else listening on the same port on that IP | 15:45 |
| andrelf | jrosser: nope but i have read the documentation regarding it at time and it was decided to use the multi node setup because we wanted to add more resources | 15:49 |
| jrosser | andrelf: if you can make a virtual machine to run an AIO in then you will get an instant reference environment to look at when trying to debug your multinode deployment | 15:52 |
| andrelf | ok now the haproxy bind all the sockets but it can comunicate with each backend... thats strange | 15:52 |
| andrelf | jrosser i understand but the parts that im having problems are the nones that supossy are diferent mainly networking. im now trying to troubleshot each lxc container and trying to understand with the service is not runing in each container | 15:54 |
| *** schwicht has quit IRC | 15:58 | |
| *** lkoranda has joined #openstack-ansible | 16:07 | |
| *** evrardjp has quit IRC | 16:35 | |
| *** evrardjp has joined #openstack-ansible | 16:36 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: ensure 20listchanges is not in debian container https://review.opendev.org/718882 | 16:58 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [doc] Adjust deploy-guide to install py3 https://review.opendev.org/724836 | 17:02 |
| openstackgerrit | Duncan Martin Walker proposed openstack/openstack-ansible-ops master: Add support for experimental Metricbeat service module https://review.opendev.org/724896 | 17:05 |
| *** lkoranda has quit IRC | 17:09 | |
| *** gregwork has joined #openstack-ansible | 17:11 | |
| *** sri_ has joined #openstack-ansible | 17:44 | |
| sri_ | Hi team silly question , is it possible to deploy openstack without lxc containers using openstack-ansible ? | 17:46 |
| jrosser | sri_: you can, but you will need more nodes, specifically your loadbalancer (haproxy) will need to be on dedicated nodes | 17:47 |
| noonedeadpunk | jrosser: I think we do have haproxy just on ctl nodes on metal deployments | 17:49 |
| noonedeadpunk | sri_: it might be more hacky but generally it's possible I'd say | 17:49 |
| jrosser | ah - what is is that can't work then, becasue we have to take haproxy and horizon away for metal aio | 17:50 |
| jrosser | (i think?) | 17:50 |
| noonedeadpunk | but we did landed patch to make horizon listen on specific ip and alternate port (if provided)? | 17:51 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible-os_horizon/src/branch/master/defaults/main.yml#L407-L409 | 17:51 |
| sri_ | jrosser, noonedeadpunk, Ack, if that is case o-a supports both Centos (rpm's)and ubuntu (deb)? | 17:52 |
| jrosser | well thats a different question | 17:52 |
| jrosser | by default OSA installs openstack using source code | 17:52 |
| jrosser | there is an option to use UCA or RDO deb/rpm but then imho you very much miss what is special / cool about OSA if you do that | 17:53 |
| jrosser | but in terms of base OS you can use centos7, ubuntu 18.04, debian buster | 17:54 |
| sri_ | jrosser: yes you're right, we are trying to deploy openstack without any containers (lxc,docker) just using string rpm's | 17:56 |
| sri_ | jrosser: noonedeadpunk , Thanks's for the information :) I will try to find some doc's :) | 17:58 |
| jrosser | sri_: you are stuck with what you are given if you deploy from rpms | 17:58 |
| jrosser | for several years now i deploy openstack with OSA and all through that time i have had to apply patches to the upstream code | 17:59 |
| jrosser | to magnum / heat / keystone / horizon all at various times | 17:59 |
| jrosser | if you are investigating OSA as a deployment tool it is worth remebering that the thing it does best is deploy from source, becasue it puts in the hands of the deployer/operator *exactly* what code gets run | 18:00 |
| sri_ | jrosser: understood, so can we build rpm's for sources ? | 18:01 |
| jrosser | OSA builds python wheels for the openstack services for particular git SHA and installed them into venv | 18:02 |
| jrosser | they do not come from packages, normally | 18:02 |
| jrosser | the versions of what gets installed are precisely defined for any particular release of OSA like this https://github.com/openstack/openstack-ansible/blob/master/playbooks/defaults/repo_packages/openstack_services.yml | 18:05 |
| jrosser | and because all of those are ansible variables you are free to override any of those versions or SHA yourself to patch / fix as you need | 18:05 |
| sri_ | jrosser: awesome :) I will take look into it, again thank you so much for your time | 18:07 |
| jrosser | sri_: i can give you a precise example, my deployments need a fix for horizon at the moment, which we fix with a patch on a fork on github https://github.com/bbc/horizon/commit/b4e2b2d137ad9cfb4bfe1203245d4ca7a54f5a82 | 18:07 |
| jrosser | we override the variables for the horizon repo to point to that repo and branch, and re-run the horizon playbook | 18:08 |
| jrosser | now we are running our fixed code, no need to build packages or do anything complex | 18:08 |
| sri_ | cool, this looks very clean | 18:12 |
| mgariepy | how often do you rebase your code @jrosser ? | 18:13 |
| jrosser | we upgrade maybe once/twice a cycle | 18:14 |
| mgariepy | ok | 18:15 |
| jrosser | and have to keep an accurate log of what we’ve forked and when everything we’ve fixed merges | 18:15 |
| mgariepy | you are on train now ? or you lag behind a couple release ? | 18:15 |
| mgariepy | how do you do that the log keeping ? | 18:16 |
| jrosser | so far we pretty much manage to get the local patching down to pretty much zero at the end of each cycle | 18:16 |
| jrosser | it’s a confluence page, nothing fancy | 18:16 |
| mgariepy | that's nice. | 18:17 |
| andrelf | ok it seems that my previous errors were related to network issues in cables and linux bridge. im now recreating all containers and see if can already start galera service :D | 18:17 |
| jrosser | currently we are on train, and put in enough effort in OSA that everything we care about is fixed/merged in n+1 | 18:17 |
| jrosser | if we didn’t do that I could easily see getting stuck on an old release | 18:18 |
| mgariepy | do you patch openstack project also ? | 18:18 |
| mgariepy | or having to try to keep the patch on divergent code. which is a pain. | 18:18 |
| jrosser | very small but yes, heat, magnum, and currently a couple of small things in horizon | 18:19 |
| jrosser | 99% of our patches go into osa really | 18:19 |
| mgariepy | how many ppl in your team ? | 18:19 |
| jrosser | sort of 5 | 18:20 |
| jrosser | it’s a bit fluid though | 18:20 |
| mgariepy | ok nice. | 18:20 |
| jrosser | but we have to do everything, network, ceph, datacentre stuff | 18:20 |
| mgariepy | yeah | 18:21 |
| mgariepy | do you manage the ceph via osa ? or you are using something else ? | 18:22 |
| jrosser | we use ceph-ansible standalone | 18:31 |
| jrosser | that way we decouple upgrades of ceph from osa, and also the version of ansible ceph-ansible wants is contradicting the one osa wants quite often | 18:33 |
| jrosser | we also have a relatively large radosgw setup which I wouldn’t want to have on the same endpoint / loadbalancer as openstack | 18:36 |
| sri_ | jrosser: we are also do that same, it's always better run ceph as independent to openstack | 18:38 |
| sri_ | jrosser: this looks like good point to start with. https://github.com/openstack/openstack-ansible/blob/b74ce951ffc202cd78cd49d1c59de8a5d431f718/etc/openstack_deploy/env.d/aio_metal.yml.example | 18:39 |
| mgariepy | jrosser, ok nice, i also moved away ceph deployment outside of osa into ceph-ansible, | 18:39 |
| jrosser | seems like a common pattern, we also moved our elk stack deploy from the openstack-ansible-ops repo out to be standalone | 18:44 |
| jrosser | being able to specify -i <osa_inventory> -i <ceph_inventory> on modern ansibles is very powerful | 18:45 |
| *** gshippey has quit IRC | 19:29 | |
| *** KeithMnemonic has quit IRC | 19:39 | |
| andrelf | jrosser and noonedeadpunk thanks for your help. i was finally able to finish the deployment and access the dashboard :D | 19:44 |
| *** andrelf has quit IRC | 19:45 | |
| noonedeadpunk | cool, nice to hear! | 19:47 |
| redkrieg | I have recently changed from specifying my nova_nova_conf_overrides in a user_nova.yml file to using group_vars/[group]_hosts.yml. I have the same settings in all of my groups, but a few hosts in one group aren't rendering any of the values specified there. all the other hosts do. is there a way for me to query what ansible thinks the value of nova_nova_conf_overrides is for a specific host | 20:28 |
| redkrieg | and figure out where that came from? grep turns up nothing | 20:28 |
| *** itandops has quit IRC | 22:07 | |
| *** ansmith has quit IRC | 22:07 | |
| *** kopecmartin has quit IRC | 22:07 | |
| *** hamzy_ has quit IRC | 22:07 | |
| *** mmethot has quit IRC | 22:07 | |
| *** gouthamr has quit IRC | 22:07 | |
| *** evrardjp has quit IRC | 22:07 | |
| *** rh-jelabarre has quit IRC | 22:07 | |
| *** Jeffrey4l has quit IRC | 22:07 | |
| *** irclogbot_2 has quit IRC | 22:07 | |
| *** CeeMac has quit IRC | 22:07 | |
| *** admin0 has quit IRC | 22:07 | |
| *** waxfire has quit IRC | 22:07 | |
| *** Jeffrey4l has joined #openstack-ansible | 22:09 | |
| *** evrardjp has joined #openstack-ansible | 22:10 | |
| *** hamzy has joined #openstack-ansible | 22:10 | |
| *** mmethot has joined #openstack-ansible | 22:10 | |
| *** hamzy has quit IRC | 22:10 | |
| *** ansmith has joined #openstack-ansible | 22:10 | |
| *** irclogbot_3 has joined #openstack-ansible | 22:10 | |
| *** hamzy has joined #openstack-ansible | 22:11 | |
| *** gouthamr has joined #openstack-ansible | 22:11 | |
| *** CeeMac has joined #openstack-ansible | 22:13 | |
| *** tosky has quit IRC | 23:45 | |
| *** admin0 has joined #openstack-ansible | 23:59 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!