| *** schwicht has quit IRC | 00:12 | |
| *** rholloway has joined #openstack-ansible | 00:13 | |
| *** rholloway has quit IRC | 00:38 | |
| *** gyee has quit IRC | 00:51 | |
| df_sbr | TASK [os_ceilometer : Add keystone domain] ************************************************************************************************************************** | 01:50 |
|---|---|---|
| df_sbr | fatal: [compute004]: FAILED! => {"msg": "The conditional check '(groups[(ceilometer_services['ceilometer-agent-notification']['group'] | intersect(group_names))[0]] | intersect(ansible_play_hosts))[0] | length > 0' failed. The error was: error while evaluating conditional ((groups[(ceilometer_services['ceilometer-agent-notification']['group'] | intersect(group_names))[0]] | intersect(ansible_play_hosts))[0] | leng | 01:50 |
| df_sbr | th > 0): list object has no element 0\n\nThe error appears to be in '/etc/ansible/roles/os_ceilometer/tasks/service_setup.yml': line 34, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n block:\n - name: Add keystone domain\n ^ here\n"} | 01:50 |
| df_sbr | jrosser: Tried again with a fresh stable/train branch. Still getting the same errors here^ | 01:51 |
| *** dave-mccowan has quit IRC | 02:26 | |
| *** irclogbot_1 has quit IRC | 02:56 | |
| *** irclogbot_3 has joined #openstack-ansible | 03:08 | |
| *** irclogbot_3 has quit IRC | 03:09 | |
| *** irclogbot_2 has joined #openstack-ansible | 03:14 | |
| *** dave-mccowan has joined #openstack-ansible | 03:19 | |
| *** goldyfruit_ has quit IRC | 04:14 | |
| *** dave-mccowan has quit IRC | 05:39 | |
| *** pcaruana has joined #openstack-ansible | 05:43 | |
| *** raukadah is now known as chandankumar | 06:06 | |
| *** elenalindq has joined #openstack-ansible | 06:13 | |
| *** gokhani has quit IRC | 07:00 | |
| *** jawad_axd has joined #openstack-ansible | 07:17 | |
| *** jbadiapa has joined #openstack-ansible | 07:20 | |
| jrosser | df_sbr: everything fresh? including bootstrap-ansible.sh again? | 07:30 |
| *** irclogbot_2 has quit IRC | 07:44 | |
| *** irclogbot_2 has joined #openstack-ansible | 07:44 | |
| *** kopecmartin|off is now known as kopecmartin | 07:58 | |
| *** cshen has joined #openstack-ansible | 08:06 | |
| *** tosky has joined #openstack-ansible | 08:14 | |
| *** miloa has joined #openstack-ansible | 08:19 | |
| *** gokhani has joined #openstack-ansible | 08:24 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible master: Imported Translations from Zanata https://review.opendev.org/699060 | 08:30 |
| tacco | morning everyone | 08:40 |
| tacco | where are the /etc/hosts entrys directed? i would like to persistent add a custom /etc/host entry | 08:40 |
| tacco | what would be the best way | 08:40 |
| *** DanyC has joined #openstack-ansible | 08:57 | |
| *** DanyC has joined #openstack-ansible | 08:57 | |
| *** DanyC has quit IRC | 08:58 | |
| *** starborn has joined #openstack-ansible | 09:04 | |
| *** DanyC has joined #openstack-ansible | 09:06 | |
| *** jbadiapa has quit IRC | 09:13 | |
| *** jbadiapa has joined #openstack-ansible | 09:13 | |
| *** rpittau|afk is now known as rpittau | 09:18 | |
| jrosser | tacco: for the things that openstack-ansible knows about it's done here https://github.com/openstack/openstack-ansible-openstack_hosts/blob/master/templates/openstack-host-hostfile-setup.sh.j2 | 09:47 |
| jrosser | however for things that are outside the OSA inventory you might need to deal with that outside of OSA, during the whatever host preparation you do? | 09:47 |
| *** tosky_ has joined #openstack-ansible | 10:07 | |
| *** tosky is now known as Guest97727 | 10:07 | |
| *** tosky_ is now known as tosky | 10:07 | |
| *** Guest97727 has quit IRC | 10:08 | |
| *** gshippey has joined #openstack-ansible | 10:12 | |
| *** sshnaidm|afk is now known as sshnaidm | 11:21 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Imported Translations from Zanata https://review.opendev.org/699060 | 11:25 |
| openstackgerrit | Georgina Shippey proposed openstack/openstack-ansible-os_ceilometer master: Check host is within group https://review.opendev.org/699957 | 11:39 |
| tacco | jrosser: ok. thanks for pointing me there. | 11:45 |
| snadge | jrosser.. just wanted to say i finally got my vsphere multi vm openstack install to pass tempest and work properly | 11:56 |
| jrosser | \o/ excellent | 11:56 |
| snadge | it took me a while, but I got there eventually.. hehe | 11:56 |
| jrosser | what did you do with the tenant networking in the end? | 11:56 |
| snadge | it looks like you can fiddle with tempest and change parameters for the private network | 11:56 |
| snadge | but i ended up just using vxlan bridged to an interface which was common to all the vms (just added one) | 11:57 |
| snadge | and it seems to work | 11:57 |
| jrosser | cool, in a way thats the least config option becasue once that vxlan interface is working you can have a whole ton of tunnels over it | 11:57 |
| snadge | i thought i might have to enable something on vsphere to get vxlan to work.. but presumably thats if you want to switch it or route it to another vsphere box etc | 11:57 |
| jrosser | well, it's just UDP packets that the host is making that happen to be vxlan | 11:58 |
| snadge | since its all local to the one vsphere host and its only used internally between those vms specific to openstack.. then it would seem its not necessary | 11:58 |
| jrosser | if they can route to other hosts of yours then everything should be fine | 11:58 |
| snadge | right | 11:58 |
| jrosser | there is a bit of multicast involved though, so thats one to watch out for | 11:58 |
| snadge | promiscuous mode is enabled on that interface | 11:59 |
| jrosser | broadcast and arp and so on ends up on a multicast group | 11:59 |
| snadge | which is required for bridging support, and potentially that as well | 11:59 |
| snadge | in production we use vlans.. but im not sure why, since that is limited to 4000 tunnels or what have you | 12:00 |
| jrosser | it depends what you want to do really | 12:00 |
| *** DanyC has quit IRC | 12:00 | |
| snadge | my short term or immediate goal, is to replace their existing openstack setup, with a new version thats openstack-ansible deployed | 12:01 |
| jrosser | if you scale out a bit then vlans can be troublesome with lots of L2 spanned all over the place and STP blah blah, maybe hard to engineer resilience into that | 12:01 |
| snadge | it seems logical to mirror their existing setup as closely as possible | 12:01 |
| jrosser | absolutely | 12:01 |
| *** DanyC has joined #openstack-ansible | 12:02 | |
| snadge | and then discuss changing that to make it more resilient etc | 12:02 |
| snadge | for example they only have one infra node essentially, there is no clustering on that | 12:02 |
| jrosser | snadge: well it's not all excellent with the clustering either, seems rabbitmq is causing a lot of problems for people at upgrade time | 12:03 |
| snadge | but to add a second infra node, the support for that is included with OA | 12:04 |
| snadge | so their current strategy is to have two identical production setups.. one idle.. and they alternate between them | 12:04 |
| snadge | so "upgrade" is basically install new version onto idle cluster.. and move all the instances over to it when they're happy to do so | 12:04 |
| snadge | this is obviously a massive waste of hardware | 12:04 |
| snadge | but it avoids these upgrade pains somewhat | 12:05 |
| snadge | i guess you could also say thats a crude form of HA or redundancy.. if something goes super bad, you can quickly switch over to the "backup" system | 12:06 |
| snadge | when i say quickly.. within hours or a day | 12:06 |
| snadge | i see there is a release version of OA for train, but it looks like some people have ran into upgrade problems as you have mentioned | 12:08 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/ansible-hardening master: Fix ignoring of packages in 'latest' state https://review.opendev.org/699963 | 12:10 |
| noonedeadpunk | jrosser: I think that's the patch we need to fix chrony absence | 12:12 |
| noonedeadpunk | thanks for stopping me :p | 12:12 |
| jrosser | woah that is obtuse ansible used in that task | 12:18 |
| noonedeadpunk | actually things got broken after we merged this https://review.opendev.org/#/c/693614/ | 12:22 |
| df_sbr | jrosser: Yes. I rm -rf /etc/ansible/* ; rm -rf /opt/ansible-runtime ; rm -rf /etc/openstack-ansbile/ansible-facts; rm /etc/openstack-ansible/openstack_hostnames_ips.yml; rm /etc/openstack-ansible/openstack_inventory.json . Ran /opt/openstack-ansible/scripts/bootstrap-ansible.sh | 12:23 |
| df_sbr | ^ Sorry. The /etc path is /etc/openstack_deploy. | 12:24 |
| jrosser | noonedeadpunk: maybe now that can be simplified, if there are seperate tasks for add and remove | 12:26 |
| df_sbr | This is the line in openstack_user_config.yml. "metering-compute_hosts: *compute_hosts" | 12:26 |
| jrosser | then there can be two lists of packages | 12:26 |
| noonedeadpunk | but we still have for installed and latest so not sure | 12:26 |
| noonedeadpunk | ah, at least for absent we can probably simplify | 12:26 |
| jrosser | i can see how it got broken | 12:27 |
| jrosser | complexity++ | 12:27 |
| noonedeadpunk | yeah, indeed. | 12:27 |
| jrosser | df_sbr: we had a look at that issue earlier | 12:27 |
| jrosser | df_sbr: if possible could you try this https://review.opendev.org/#/c/699957/ | 12:28 |
| jrosser | that will be in /etc/ansible/roles/os_ceilometer/tasks/main.yml | 12:28 |
| noonedeadpunk | the best way to simplify this is to rework stig_packages_rhel7 and make a single package per item instead of included list | 12:29 |
| noonedeadpunk | it will extend variable size, but overall it should simplify logic | 12:29 |
| df_sbr | jrosser: checking now. | 12:29 |
| noonedeadpunk | But I'd probably leave as is as for now not to bring in new issues | 12:31 |
| jrosser | urgh yes that data structure is not nice | 12:31 |
| jrosser | df_sbr: if you have a gerrit account it would be great if you could leave some feedback on that patch | 12:33 |
| *** ansmith has quit IRC | 12:34 | |
| *** sum12 has quit IRC | 12:44 | |
| *** sum12 has joined #openstack-ansible | 12:48 | |
| ierdem | jrosser, i had face the same issue which df_sbr faced and i tried adding https://review.opendev.org/#/c/699957/. It solved, thank you | 13:02 |
| jrosser | ierdem: thanks! good to know it's addressed the issue | 13:03 |
| guilhermesp | jrosser: did you ever apply it in production? https://github.com/openstack/openstack-ansible-rabbitmq_server/commit/15e8837dd3972159f5d83fd95db5fc69f80f6139#diff-16ccf8c41f99a0867513224bbf0fd9b4R45 | 13:10 |
| jrosser | guilhermesp: thats merged in train? | 13:11 |
| * guilhermesp checks | 13:12 | |
| jrosser | if so we have it deployed now on a containers deployment - is there something you'd like me to check? | 13:12 |
| guilhermesp | yep it seems to be | 13:12 |
| guilhermesp | acutally my case is that i cherry pickes to stein, and when defining that line it seems to break with this http://paste.openstack.org/show/787787/ | 13:13 |
| guilhermesp | https://www.irccloud.com/pastebin/LHpTHop8/ | 13:13 |
| jrosser | there were other related things | 13:14 |
| jrosser | like this https://github.com/openstack/openstack-ansible-openstack_hosts/commit/a71523f7c361e2a2406cec38d572be23e826f664 | 13:14 |
| guilhermesp | huuuuuum | 13:15 |
| jrosser | that confused rabbitmq as the hostname it thought it had via /etc/hosts didnt match the nodename in the config | 13:16 |
| jrosser | and binding to the mgmt ip caused that to mismatch | 13:16 |
| guilhermesp | let me try that out | 13:16 |
| jrosser | i just have a hunch becasue of it saying "* suggestion: check if all server nodes and CLI tools use consistent hostnames when addressing each other" | 13:17 |
| jrosser | just commenting out those extra lines from /etc/hosts would find out quickly | 13:18 |
| guilhermesp | yep...I'm re running playbooks here to see if now works | 13:18 |
| guilhermesp | i think at the time we were working on making it work for rabbit this was already merged so | 13:18 |
| guilhermesp | I wasn't aware | 13:18 |
| jrosser | those patches are within 2 days of each other so i think this is all in the same area | 13:19 |
| openstackgerrit | Georgina Shippey proposed openstack/openstack-ansible-openstack_hosts master: Update Openstack code name to Ussuri https://review.opendev.org/699984 | 13:24 |
| guilhermesp | oh jrosser yeah it seems now that tasks are not breaking :P | 13:26 |
| guilhermesp | let's wait the final result | 13:26 |
| jrosser | guilhermesp: great :) that patch was in the bind-to-mgmt topic btw, maybe worth checking if theres anything else we need | 13:27 |
| jrosser | we/you | 13:27 |
| guilhermesp | yep. let's see after i finish the playbook if rabbitmgmt is functional for a non-containerized env | 13:28 |
| jrosser | did we ever mergen anything for galera...... | 13:28 |
| guilhermesp | hum dont remember tbh ... but well. i haven't had problems with galera in most of our non-containerized deployments | 13:29 |
| guilhermesp | hum weird, it seems it is still just listening to the vip | 13:29 |
| guilhermesp | https://www.irccloud.com/pastebin/821hRqCn/ | 13:29 |
| jrosser | hmm? thats odd, the VIP not the node mgmt address? | 13:31 |
| guilhermesp | yep it is | 13:32 |
| df_sbr | jrosser: Deployment is working now. I added a +1 on the patch in gerrit | 13:32 |
| jrosser | df_sbr: great, thanks for testing it | 13:32 |
| jrosser | df_sbr: that patch should hopefully merge today and we can backport to Train | 13:33 |
| guilhermesp | jrosser: take a look at this | 13:34 |
| guilhermesp | https://www.irccloud.com/pastebin/l8C4WLD4/ | 13:34 |
| df_sbr | jrosser: Thanks! | 13:34 |
| jrosser | in the meantime you can either patch locally or we have a means of overriding the SHA of the os_ceilometer role if you want something more robust | 13:34 |
| df_sbr | This is good for now. It's a dev environment. | 13:35 |
| guilhermesp | hum dont we enable it in the tasks? | 13:36 |
| jrosser | guilhermesp: wait its ok :) | 13:36 |
| jrosser | https://github.com/openstack/openstack-ansible/blob/master/inventory/group_vars/haproxy/haproxy.yml#L256-L266 | 13:36 |
| jrosser | if this is an infra node with all the services collapsed haproxy is accepting connnections to the rabbitmq mgmt interface on the vip | 13:37 |
| guilhermesp | yep that is right as we have the backend in place.... but it seems that in each controller i need to enable the plugin | 13:38 |
| guilhermesp | did it manually in one of the ctls... the other are still not listening to the process locally | 13:38 |
| jrosser | interesting https://opendev.org/openstack/openstack-ansible-rabbitmq_server/src/branch/master/defaults/main.yml#L93-L95 | 13:39 |
| guilhermesp | yeah and I've checked that before | 13:39 |
| jrosser | i wonder if we've updated rabbitmq to a point that the ansible module doesnt understand any more | 13:40 |
| noonedeadpunk | that might be actualy | 13:41 |
| noonedeadpunk | as 2.8 has absolutelly different output | 13:41 |
| noonedeadpunk | *3.8 | 13:41 |
| noonedeadpunk | I've placed a patch to over rabbitmq_policy related issue, but it seems that at this point nobody care about ansible modules anymore | 13:42 |
| noonedeadpunk | Like https://github.com/ansible/ansible/pull/64512 as placed month ago an no reviews up to this point | 13:43 |
| *** ansmith has joined #openstack-ansible | 13:45 | |
| *** KeithMnemonic has joined #openstack-ansible | 13:58 | |
| *** tosky_ has joined #openstack-ansible | 14:23 | |
| *** tosky is now known as Guest27890 | 14:23 | |
| *** tosky_ is now known as tosky | 14:23 | |
| *** Guest27890 has quit IRC | 14:26 | |
| *** KeithMnemonic has quit IRC | 14:31 | |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts stable/train: Update OpenStack code name to Train https://review.opendev.org/699446 | 14:47 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_magnum master: Fix magnum-api wsgi name https://review.opendev.org/699726 | 14:55 |
| *** theintern_ has joined #openstack-ansible | 15:03 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_magnum stable/train: Fix magnum-api wsgi name https://review.opendev.org/699738 | 15:05 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/stein: Make nova_novncproxy use BUMPs https://review.opendev.org/699113 | 15:08 |
| *** jawad_axd has quit IRC | 15:20 | |
| *** jawad_axd has joined #openstack-ansible | 15:21 | |
| *** jawad_ax_ has joined #openstack-ansible | 15:23 | |
| *** electrofelix has joined #openstack-ansible | 15:24 | |
| *** starborn has quit IRC | 15:25 | |
| *** jawad_axd has quit IRC | 15:26 | |
| *** jawad_ax_ has quit IRC | 15:28 | |
| *** cshen has quit IRC | 15:34 | |
| *** gyee has joined #openstack-ansible | 15:54 | |
| *** rpittau is now known as rpittau|afk | 16:02 | |
| *** cp- has quit IRC | 16:05 | |
| *** cp- has joined #openstack-ansible | 16:06 | |
| *** cp- has quit IRC | 16:11 | |
| *** cp- has joined #openstack-ansible | 16:12 | |
| openstackgerrit | Douglas Mendizábal proposed openstack/ansible-role-atos-hsm master: WIP: Add HA (load balancing) support https://review.opendev.org/700018 | 16:20 |
| *** jawad_axd has joined #openstack-ansible | 16:22 | |
| *** jawad_axd has quit IRC | 16:25 | |
| *** jawad_axd has joined #openstack-ansible | 16:26 | |
| *** jawad_axd has quit IRC | 16:27 | |
| *** miloa has quit IRC | 16:41 | |
| *** rholloway has joined #openstack-ansible | 17:07 | |
| *** trandles has joined #openstack-ansible | 17:19 | |
| *** chandankumar has quit IRC | 17:31 | |
| *** DanyC has quit IRC | 17:56 | |
| *** aedc has joined #openstack-ansible | 18:01 | |
| *** aedc has quit IRC | 18:02 | |
| *** udesale has joined #openstack-ansible | 18:03 | |
| *** gyee has quit IRC | 18:12 | |
| *** gyee has joined #openstack-ansible | 18:12 | |
| *** rholloway has quit IRC | 18:32 | |
| *** rholloway has joined #openstack-ansible | 18:33 | |
| noonedeadpunk | jrosser: ooooooops https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_5aa/699726/6/check/openstack-ansible-deploy-aio_metal-centos-7/5aa80c2/logs/ara-report/result/63c02a09-691c-462d-b5a4-62312f97be97/ | 18:35 |
| *** udesale has quit IRC | 18:37 | |
| noonedeadpunk | I really dunno what we can do about it except abandoning centos7 support as for now, or building python-libselinux from scratch... | 18:46 |
| *** rholloway has quit IRC | 19:11 | |
| *** rholloway has joined #openstack-ansible | 19:13 | |
| *** rholloway has quit IRC | 19:25 | |
| *** electrofelix has quit IRC | 19:28 | |
| *** dmsimard0 has joined #openstack-ansible | 19:41 | |
| *** dmsimard has quit IRC | 19:41 | |
| *** dmsimard0 is now known as dmsimard | 19:41 | |
| *** rodolof has quit IRC | 19:44 | |
| *** spotz has quit IRC | 19:44 | |
| *** rodolof has joined #openstack-ansible | 19:45 | |
| jrosser | I wonder how far off the centos 8 job is | 19:55 |
| *** sshnaidm is now known as sshnaidm|afk | 20:00 | |
| *** mattoliverau has quit IRC | 20:05 | |
| *** irclogbot_2 has quit IRC | 20:06 | |
| *** irclogbot_2 has joined #openstack-ansible | 20:09 | |
| *** elenalindq has quit IRC | 20:36 | |
| *** theintern_ has quit IRC | 20:46 | |
| *** pcaruana has quit IRC | 21:23 | |
| *** ansmith has quit IRC | 21:40 | |
| *** goldyfruit_ has joined #openstack-ansible | 21:57 | |
| *** goldyfruit_ has quit IRC | 22:06 | |
| *** jbadiapa has quit IRC | 23:13 | |
| *** ansmith has joined #openstack-ansible | 23:18 | |
| *** tosky has quit IRC | 23:22 | |
| *** admin0 has quit IRC | 23:26 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!