| *** macza has quit IRC | 00:05 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/rocky: Update systemd_service role for lock path fix https://review.openstack.org/632751 | 00:08 |
|---|---|---|
| *** strattao has quit IRC | 00:18 | |
| *** markvoelker has quit IRC | 00:20 | |
| *** macza has joined #openstack-ansible | 00:33 | |
| *** cmart has quit IRC | 00:36 | |
| prometheanfire | where are the rules for nat set up? | 00:36 |
| prometheanfire | for the containers | 00:36 |
| *** macza has joined #openstack-ansible | 00:37 | |
| logan- | https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/tasks/prepare_networking.yml#L159-L197 | 00:37 |
| logan- | oh sorry, that's for the gate | 00:37 |
| logan- | https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/templates/lxc-system-manage.j2#L76-L111 | 00:38 |
| prometheanfire | logan-: well, I am kinda running gate | 00:39 |
| prometheanfire | I ran run_tests.sh then tox -e functional from the tests repo | 00:39 |
| prometheanfire | off an infra image | 00:40 |
| logan- | the first is covering things like nova tempest vms reaching the internet. the second is what you see in the gate and prod handling masq for the eth0 interface in the containers | 00:40 |
| prometheanfire | ok, and dnsmasq is in charge of the reload | 00:42 |
| prometheanfire | had to restart the lxc-dnsmasq service, works now, will have to look into it if it occurs again | 00:43 |
| *** tstrul has joined #openstack-ansible | 00:47 | |
| *** tstrul has quit IRC | 00:48 | |
| *** bgmccollum has quit IRC | 00:48 | |
| *** tstrul has joined #openstack-ansible | 00:48 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-lxc_container_create master: Add gentoo support https://review.openstack.org/633092 | 00:48 |
| *** tstrul has quit IRC | 00:50 | |
| *** tstrul has joined #openstack-ansible | 00:50 | |
| *** tstrul has quit IRC | 00:51 | |
| *** tstrul has joined #openstack-ansible | 00:52 | |
| *** nurdie has joined #openstack-ansible | 00:54 | |
| *** TxGirlGeek has joined #openstack-ansible | 00:54 | |
| *** tstrul has quit IRC | 00:55 | |
| *** tstrul has joined #openstack-ansible | 00:55 | |
| *** tstrul has quit IRC | 00:58 | |
| *** tstrul has joined #openstack-ansible | 00:59 | |
| *** nurdie has quit IRC | 00:59 | |
| *** tstrul has quit IRC | 00:59 | |
| prometheanfire | every time I rerun the functional tests networking breaks | 01:07 |
| *** sawblade6 has quit IRC | 01:07 | |
| *** sawblade6 has joined #openstack-ansible | 01:08 | |
| *** gyee has quit IRC | 01:10 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-memcached_server master: add gentoo support to memcached role https://review.openstack.org/633093 | 01:11 |
| *** bgmccollum has joined #openstack-ansible | 01:17 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support https://review.openstack.org/608393 | 01:21 |
| *** TxGirlGeek has quit IRC | 01:30 | |
| *** cmart has joined #openstack-ansible | 01:37 | |
| openstackgerrit | Michael Vollman proposed openstack/openstack-ansible-os_manila master: Basic working os_manila role https://review.openstack.org/611930 | 01:42 |
| cloudnull | evenings all, sorry I was away most the day. | 02:10 |
| cloudnull | CeeMac did you make things go ? | 02:10 |
| cloudnull | is it working now ? | 02:11 |
| cloudnull | prometheanfire - we are using networkd in the containers, or it should be. | 02:11 |
| *** cmart has quit IRC | 02:14 | |
| *** vollman has quit IRC | 02:24 | |
| *** markvoelker has joined #openstack-ansible | 02:25 | |
| prometheanfire | cloudnull: my problem is that I have to restart lxc-dnsmasq every retest | 02:32 |
| cloudnull | why? | 02:33 |
| prometheanfire | dunno, focusing on rabbit now | 02:54 |
| prometheanfire | I'm in the phase where I just need to make minor changes (I hope) | 02:54 |
| *** markvoelker has quit IRC | 02:55 | |
| *** ebbex has quit IRC | 02:55 | |
| *** chandankumar has quit IRC | 02:57 | |
| *** chandankumar has joined #openstack-ansible | 02:59 | |
| prometheanfire | one thing I'll probably need ot do is set up a passed through dir for portage binpkgs | 03:00 |
| prometheanfire | cloudnull: question about rabbitmq upgrade check | 03:05 |
| prometheanfire | https://github.com/openstack/openstack-ansible-rabbitmq_server/blob/master/tasks/rabbitmq_upgrade_check.yml#L35 | 03:05 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova master: Fix default init config override for nova-compute https://review.openstack.org/633104 | 03:05 |
| prometheanfire | shell: "equery l net-misc/rabbitmq-server --format '$version' | tail -n 1" | 03:06 |
| *** udesale has joined #openstack-ansible | 03:06 | |
| prometheanfire | that's how I was trying to do it, but it returns nothing when not installed (correctly) | 03:08 |
| prometheanfire | but.... | 03:08 |
| prometheanfire | fatal: [infra1]: FAILED! => {"msg": "The conditional check 'not installed_rabbitmq.stdout | search(rabbitmq_package_version)' failed. The error was: error while evaluating conditional (not installed_rabbitmq.stdout | search(rabbitmq_package_version)): '_rabbitmq_package_version' is undefined\n\nThe error appears to have been in | 03:08 |
| prometheanfire | '/root/.ansible/roles/rabbitmq_server/tasks/rabbitmq_upgrade_check.yml': line 71, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Compare installed version of RabbitMQ with new version variable\n ^ here\n"} | 03:08 |
| prometheanfire | I think that this wouldn't work with zero output | 03:20 |
| prometheanfire | - not installed_rabbitmq.stdout | search(rabbitmq_package_version) | 03:20 |
| prometheanfire | so I think it's a bug? | 03:20 |
| prometheanfire | cloudnull: here you are for a little better context | 03:27 |
| prometheanfire | https://pasted.tech/pastes/0db21523579dde0ef26992716ed1f28210c13e25.raw | 03:27 |
| *** markvoelker has joined #openstack-ansible | 03:52 | |
| *** markvoelker has quit IRC | 04:25 | |
| *** shyamb has joined #openstack-ansible | 04:38 | |
| *** macza has quit IRC | 04:55 | |
| *** udesale has quit IRC | 05:01 | |
| *** udesale has joined #openstack-ansible | 05:04 | |
| *** shyamb has quit IRC | 05:07 | |
| *** shyamb has joined #openstack-ansible | 05:07 | |
| *** shyamb has quit IRC | 05:12 | |
| *** shyamb has joined #openstack-ansible | 05:13 | |
| *** markvoelker has joined #openstack-ansible | 05:22 | |
| *** shyamb has quit IRC | 05:32 | |
| *** shyamb has joined #openstack-ansible | 05:32 | |
| *** nurdie has joined #openstack-ansible | 05:39 | |
| *** macza has joined #openstack-ansible | 05:39 | |
| *** spsurya has joined #openstack-ansible | 05:40 | |
| *** udesale has quit IRC | 05:45 | |
| *** udesale has joined #openstack-ansible | 05:46 | |
| *** markvoelker has quit IRC | 05:54 | |
| *** shyamb has quit IRC | 05:55 | |
| *** shyamb has joined #openstack-ansible | 05:56 | |
| *** shyamb has quit IRC | 06:17 | |
| *** shyamb has joined #openstack-ansible | 06:21 | |
| *** macza_ has joined #openstack-ansible | 06:24 | |
| *** kopecmartin|off is now known as kopecmartin|devc | 06:27 | |
| *** macza has quit IRC | 06:28 | |
| *** macza_ has quit IRC | 06:28 | |
| *** shyamb has quit IRC | 06:34 | |
| *** shyamb has joined #openstack-ansible | 06:35 | |
| *** shyam89 has joined #openstack-ansible | 06:38 | |
| *** shyamb has quit IRC | 06:41 | |
| *** shyam89 has quit IRC | 06:42 | |
| *** shyamb has joined #openstack-ansible | 06:43 | |
| *** shyam89 has joined #openstack-ansible | 06:48 | |
| *** shyamb has quit IRC | 06:49 | |
| *** nurdie has quit IRC | 07:05 | |
| *** nurdie has joined #openstack-ansible | 07:05 | |
| *** nurdie has quit IRC | 07:09 | |
| *** sohny has joined #openstack-ansible | 07:13 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible master: Imported Translations from Zanata https://review.openstack.org/633150 | 07:21 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-rabbitmq_server master: add gentoo support to rabbitmq role https://review.openstack.org/633152 | 07:26 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-tests master: add Gentoo jobs as non-voting https://review.openstack.org/608102 | 07:27 |
| prometheanfire | stuck on Apply rabbitmq policies, but closer | 07:27 |
| *** ebbex has joined #openstack-ansible | 07:35 | |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Added dependencies of os_tempest role https://review.openstack.org/632726 | 07:43 |
| prometheanfire | strace showed in wait, killed and finished, on to galera (tomorrow) | 07:44 |
| *** shyam89 has quit IRC | 07:48 | |
| *** shyam89 has joined #openstack-ansible | 07:48 | |
| *** markvoelker has joined #openstack-ansible | 07:52 | |
| *** shyam89 has quit IRC | 08:20 | |
| *** markvoelker has quit IRC | 08:25 | |
| *** shardy has joined #openstack-ansible | 08:25 | |
| chandankumar | jrosser: Hello | 08:35 |
| chandankumar | jrosser: where we store netstat output in the osa jobs logs? | 08:35 |
| *** tosky has joined #openstack-ansible | 08:49 | |
| *** shardy has quit IRC | 08:50 | |
| *** shardy has joined #openstack-ansible | 08:51 | |
| chandankumar | I basically need to check whether port 22 is open or not in centos 7 tempest jobs | 08:52 |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Added dependencies of os_tempest role https://review.openstack.org/632726 | 08:55 |
| *** shyamb has joined #openstack-ansible | 08:59 | |
| *** hamzaachi has joined #openstack-ansible | 09:00 | |
| *** shyamb has quit IRC | 09:16 | |
| *** shyamb has joined #openstack-ansible | 09:16 | |
| *** shyamb has quit IRC | 09:20 | |
| *** shyamb has joined #openstack-ansible | 09:20 | |
| *** markvoelker has joined #openstack-ansible | 09:22 | |
| *** hamzaachi has quit IRC | 09:22 | |
| *** udesale has quit IRC | 09:29 | |
| *** udesale has joined #openstack-ansible | 09:30 | |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: [DNM] debug gate failure https://review.openstack.org/633173 | 09:36 |
| *** asettle has joined #openstack-ansible | 09:40 | |
| *** rgogunskiy has joined #openstack-ansible | 09:47 | |
| *** markvoelker has quit IRC | 09:55 | |
| *** shyamb has quit IRC | 10:08 | |
| *** aedc has joined #openstack-ansible | 10:10 | |
| odyssey4me | oh man, the Q->R upgrade periodic just passed within 30 secs of the limit: http://zuul.openstack.org/build/3880243188e64944a5e293c76fdb7b9a | 10:17 |
| chandankumar | odyssey4me: Hello | 10:18 |
| chandankumar | odyssey4me: where can i find in the logs whether port 22 is open or not? | 10:18 |
| chandankumar | odyssey4me: i mean netstat output in centos7 tempest failure logs? | 10:18 |
| odyssey4me | chandankumar for role tests, this is the script that collects the logs: https://github.com/openstack/openstack-ansible-tests/blob/master/test-log-collect.sh | 10:18 |
| *** shardy has quit IRC | 10:19 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Imported Translations from Zanata https://review.openstack.org/633150 | 10:19 |
| *** shardy has joined #openstack-ansible | 10:20 | |
| *** hamzaachi has joined #openstack-ansible | 10:21 | |
| *** shyamb has joined #openstack-ansible | 10:21 | |
| *** shyamb has quit IRC | 10:28 | |
| *** rgogunskiy has quit IRC | 10:31 | |
| *** shyamb has joined #openstack-ansible | 10:31 | |
| *** rgogunskiy has joined #openstack-ansible | 10:32 | |
| *** shyamb has quit IRC | 10:36 | |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-tests master: Gather different port status on different hosts https://review.openstack.org/633179 | 10:38 |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Always generate stackviz irrespective of tests pass or fail https://review.openstack.org/631967 | 10:42 |
| *** macza has joined #openstack-ansible | 10:47 | |
| *** macza_ has joined #openstack-ansible | 10:48 | |
| *** macza has quit IRC | 10:51 | |
| *** markvoelker has joined #openstack-ansible | 10:52 | |
| *** macza_ has quit IRC | 10:52 | |
| *** shyamb has joined #openstack-ansible | 10:55 | |
| *** shardy has quit IRC | 10:55 | |
| *** noonedeadpunk has joined #openstack-ansible | 11:01 | |
| *** shardy has joined #openstack-ansible | 11:10 | |
| *** shardy has quit IRC | 11:15 | |
| *** shardy has joined #openstack-ansible | 11:17 | |
| *** shardy has quit IRC | 11:22 | |
| *** shyam89 has joined #openstack-ansible | 11:23 | |
| *** markvoelker has quit IRC | 11:25 | |
| *** shyamb has quit IRC | 11:26 | |
| *** shyam89 has quit IRC | 11:34 | |
| *** shardy has joined #openstack-ansible | 11:35 | |
| *** shardy has quit IRC | 11:39 | |
| chandankumar | odyssey4me: need some help on this review https://review.openstack.org/#/c/633179/ | 11:46 |
| chandankumar | odyssey4me: http://logs.openstack.org/79/633179/1/check/openstack-ansible-functional-centos-7/460d822/logs/ara-report/result/49a48f9f-2710-4cf2-9882-3a9f1efa189b/ | 11:46 |
| odyssey4me | chandankumar hosts: all, but delegate to locahost? that makes absolutely no sense - why not just target localhost? | 11:47 |
| *** hamzaachi has quit IRC | 11:47 | |
| *** shyamb has joined #openstack-ansible | 11:47 | |
| odyssey4me | chandankumar oh, I see what you're trying to do - you're trying to make use of the fact cache to get the listening ports? | 11:48 |
| chandankumar | odyssey4me: yes | 11:49 |
| chandankumar | odyssey4me: I just copied it from openstack-ansible | 11:49 |
| odyssey4me | chandankumar but only for localhost? | 11:49 |
| jrosser | that is my ugly code :/ | 11:49 |
| chandankumar | odyssey4me: https://github.com/openstack/openstack-ansible/blob/master/playbooks/listening-port-report.yml | 11:49 |
| jrosser | it visits 'all', registers the netstat output and the writes a txt file of all of that on localhost | 11:50 |
| *** CeeMac has quit IRC | 11:50 | |
| chandankumar | will i add a single task to run only netstat command and gather the output in a file? | 11:50 |
| odyssey4me | jrosser ah ok, I see it now | 11:50 |
| odyssey4me | chandankumar the play needs user:root for that to work | 11:51 |
| jrosser | netstat on the host wont see in the containers | 11:51 |
| jrosser | so it ended up like that | 11:51 |
| chandankumar | odyssey4me: ok | 11:51 |
| odyssey4me | chandankumar also, netstat may need to also be installed on every host - I dunno if it's there by default | 11:51 |
| ioni | hello guys, i know it's not related to openstack ansible but mostly you guys run a cloud of different dimensions. have you guys had this problem with resources that werent deallocated from a compute node after a migration? here is a log: https://paste.xinu.at/ecVs8/ | 11:52 |
| jrosser | iirc i wrote this really for multinode deployments to go and grab from everywhere, and afaik netstat was available | 11:52 |
| jrosser | but that may be an ubuntu-ism | 11:52 |
| *** shardy has joined #openstack-ansible | 11:53 | |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-tests master: Gather different port status on different hosts https://review.openstack.org/633179 | 11:58 |
| *** CeeMac has joined #openstack-ansible | 11:59 | |
| *** CeeMac has quit IRC | 12:02 | |
| *** CeeMac has joined #openstack-ansible | 12:02 | |
| chandankumar | odyssey4me: I am trying to add the dependencies here https://review.openstack.org/#/c/632726/ but it is failing | 12:07 |
| odyssey4me | chandankumar that's odd | 12:09 |
| odyssey4me | I don't know why that's failing like that, given nothing else is changing. | 12:12 |
| odyssey4me | There will definitely be a clash between config_template in openstack-ansible-plugins and config_template. I don't know why the transition is taking so long - perhaps evrardjp can comment? | 12:13 |
| *** shyamb has quit IRC | 12:16 | |
| evrardjp | I didn't get the chance to start to work on that? | 12:16 |
| evrardjp | simply as that :) | 12:16 |
| evrardjp | we need to be coordinated on it | 12:16 |
| odyssey4me | evrardjp then perhaps we should schedule a day to get it done? | 12:16 |
| evrardjp | yeah that sounds fair | 12:16 |
| odyssey4me | or we put up all the patches needed, but -w them, then approve them all in the appropriate sequence? | 12:17 |
| evrardjp | if possible in european timezones would be easier for me and to get a faster ci | 12:17 |
| evrardjp | it depends on how far we want to go too | 12:19 |
| odyssey4me | I'm away at FOSDEM next Friday. I'll see you at FOSDEM, so perhaps we can put together a etherpad to plan what needs to be done, and can send it to the ML for comment - then set the date to get it done. | 12:19 |
| evrardjp | lgtm | 12:19 |
| chandankumar | odyssey4me: odyssey4me if we get it done, then we can also remove this copied stuff https://github.com/ceph/ceph-ansible/blob/master/library/config_template and script | 12:20 |
| chandankumar | and make them dependent on it | 12:20 |
| odyssey4me | chandankumar yes, although that may require that we have infra publish the config_template role to ansible-galaxy | 12:21 |
| *** nurdie has joined #openstack-ansible | 12:24 | |
| jamesdenton | mronin | 12:27 |
| jamesdenton | errrr... good morning | 12:27 |
| *** nurdie has quit IRC | 12:27 | |
| *** nurdie has joined #openstack-ansible | 12:28 | |
| CeeMac | morning jamesdenton | 12:31 |
| CeeMac | well, afternoon here :) | 12:31 |
| jamesdenton | :) | 12:31 |
| *** nurdie has quit IRC | 12:32 | |
| chandankumar | odyssey4me: port patch worked but this time http://logs.openstack.org/79/633179/2/check/openstack-ansible-linters/a0f260c/job-output.txt.gz#_2019-01-25_12_06_35_563613 | 12:55 |
| odyssey4me | chandankumar 'ANSIBLE0012 Commands should not change things if nothing needs doing' | 12:55 |
| odyssey4me | add 'changed_when: false' to the netstat command | 12:56 |
| chandankumar | ok | 12:56 |
| odyssey4me | its purpose is to gather a form of fact | 12:56 |
| odyssey4me | fact gathering should not result in a task being 'changed' | 12:56 |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-tests master: Gather different port status on different hosts https://review.openstack.org/633179 | 12:59 |
| chandankumar | odyssey4me: yes, thanks :-) | 12:59 |
| chandankumar | odyssey4me: jrosser here is the output from last run http://logs.openstack.org/79/633179/2/check/openstack-ansible-functional-centos-7/f3465eb/logs/host/listening_port_report.txt.txt.gz | 13:01 |
| *** eumel8 has joined #openstack-ansible | 13:05 | |
| *** hamzaachi has joined #openstack-ansible | 13:06 | |
| CeeMac | ok, fun question time | 13:14 |
| CeeMac | say, hypothetically, I'd been an idiot and forgot to exlude my LB VIP from the range allocatable for containers, and one of the galera containers picked up the address | 13:15 |
| CeeMac | if i removed that container (destory and remove from inventory) | 13:16 |
| CeeMac | and fixed the used_ips, then ran setup-hosts again, that would generate a new container with a new (non-conflicting) ip | 13:17 |
| CeeMac | if i then re-ran the os-galera playbook, would it just add another container to the same galera cluster and not try to recreate the whole cluster? | 13:17 |
| *** strattao has joined #openstack-ansible | 13:18 | |
| CeeMac | hypothetically >_> | 13:18 |
| jamesdenton | this is all starting to make sense now :D | 13:20 |
| CeeMac | also, is there some command i need to run to remove that galera container from the cluster first? | 13:20 |
| * CeeMac facepalms | 13:20 | |
| jamesdenton | I can't say for sure. There could be something useful in here: https://docs.openstack.org/openstack-ansible/latest/admin/maintenance-tasks/galera.html | 13:22 |
| *** nurdie has joined #openstack-ansible | 13:22 | |
| *** nurdie has quit IRC | 13:23 | |
| CeeMac | actually, would ammending the inventory file manually to change the IP cause any issues? | 13:23 |
| CeeMac | that would be more straightforward | 13:23 |
| *** nurdie has joined #openstack-ansible | 13:24 | |
| jamesdenton | You might be able to get away with destroying the container as described, then changing the IP in the inventory, then rebuilding it and rerunning respective playbooks. Caveat Emptor. | 13:24 |
| CeeMac | yeah, thats what I was thinking | 13:26 |
| CeeMac | i can just edit the json file directly right, or is there a dynamic-inventory command to change ip? | 13:28 |
| *** nurdie has quit IRC | 13:28 | |
| jamesdenton | I think you'd have to munge by hand | 13:29 |
| CeeMac | k | 13:29 |
| CeeMac | cool i'll give that a go, should hopefully make everything else just work! | 13:30 |
| CeeMac | or break it completely | 13:30 |
| * CeeMac shrugs | 13:30 | |
| jamesdenton | breaking things is part of the journey | 13:30 |
| CeeMac | thats how we learn right? | 13:30 |
| CeeMac | i'll need to re-run haproxy-install as well right? to update the new container ip? | 13:32 |
| CeeMac | or fudge the config manually? | 13:32 |
| jamesdenton | yes | 13:32 |
| jamesdenton | rerun | 13:32 |
| CeeMac | k. thanks | 13:32 |
| CeeMac | is it worth removing the ansible facts information for that container as well? | 13:37 |
| jamesdenton | sure | 13:42 |
| CeeMac | ok, i've changed the IP in the containers ansible_facts file, the openstack_inventroy.json and the openstack_hostnames_ips.yml file | 13:45 |
| CeeMac | think that covers everything? | 13:46 |
| jamesdenton | you can verify the changes by running inventory/dynamic_inventory.py | 13:47 |
| chandankumar | odyssey4me: https://bugs.launchpad.net/openstack-gate/+bug/1808010 tempest issues | 13:48 |
| openstack | Launchpad bug 1808010 in OpenStack Compute (nova) "Tempest cirros ssh setup fails due to lack of disk space causing config-drive setup to fail forcing fallback to metadata server which fails due to hitting 10 second timeout." [Medium,Confirmed] | 13:48 |
| jamesdenton | /opt/openstack-ansible/scripts/inventory-manage.py --list-host might even work | 13:48 |
| CeeMac | seems to look ok | 13:49 |
| jamesdenton | cool. | 13:49 |
| CeeMac | so, run setup-hosts --limit <host>,<container> first then os-galera --limit <host>,<container> ? | 13:50 |
| jamesdenton | more or less | 13:51 |
| jamesdenton | + haproxy | 13:51 |
| CeeMac | ah, yes | 13:51 |
| CeeMac | not forgetting haproxy | 13:51 |
| *** sohny has quit IRC | 13:51 | |
| odyssey4me | chandankumar so https://github.com/openstack/openstack-ansible-os_tempest/blob/master/defaults/main.yml#L242-L250 needs changing up then? | 13:52 |
| chandankumar | odyssey4me: yes | 13:52 |
| chandankumar | doing that | 13:52 |
| odyssey4me | fantastic, thanks | 13:52 |
| chandankumar | odyssey4me: updating image to 3.6 | 13:52 |
| chandankumar | odyssey4me: from where we get sha256 hash? | 13:53 |
| chandankumar | odyssey4me: there I am seeing pnly md5sums | 13:53 |
| odyssey4me | chandankumar just download it and run sha256sum against the file | 13:53 |
| chandankumar | odyssey4me: ok | 13:54 |
| odyssey4me | chandankumar you could also change it to use md5 instead | 13:57 |
| odyssey4me | checksum: "md5:..." | 13:57 |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Update cirros from 3.5 to 3.6 https://review.openstack.org/633208 | 13:57 |
| chandankumar | odyssey4me: ^^ | 13:58 |
| odyssey4me | chandankumar perhaps 'Related-Bug: xxxx' should be used? | 13:59 |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Update cirros from 3.5 to 3.6 https://review.openstack.org/633208 | 14:00 |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Update cirros from 3.5 to 3.6 https://review.openstack.org/633208 | 14:01 |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Update cirros from 3.5 to 3.6 https://review.openstack.org/633208 | 14:03 |
| odyssey4me | chandankumar ok, let's see how the gates respond | 14:04 |
| odyssey4me | do you think this resolves the gate issue we've had for a few days? | 14:05 |
| chandankumar | odyssey4me: I think so | 14:09 |
| chandankumar | odyssey4me: it started happening once we enabled config-drive i think | 14:10 |
| odyssey4me | chandankumar here's hoping that it works :) | 14:15 |
| chandankumar | odyssey4me: cloudnull jrosser https://review.openstack.org/633179 port scanning is good to go | 14:16 |
| CeeMac | jamesdenton, so that seems to have resolved that little issue | 14:23 |
| jamesdenton | glad to hear! | 14:23 |
| jamesdenton | is there a 'but' in there? | 14:24 |
| *** rgogunskiy has quit IRC | 14:45 | |
| *** dave-mccowan has joined #openstack-ansible | 14:46 | |
| *** hamzaachi has quit IRC | 14:49 | |
| *** strattao has quit IRC | 14:50 | |
| *** dave-mccowan has quit IRC | 14:51 | |
| chandankumar | odyssey4me: one of the job just failed while downloading image | 14:51 |
| *** Soopaman has joined #openstack-ansible | 14:55 | |
| *** strattao has joined #openstack-ansible | 15:01 | |
| *** spsurya has quit IRC | 15:07 | |
| CeeMac | jamesdenton, sorry, got distracted eleswhere. No 'but' so far! | 15:19 |
| CeeMac | although I've rediscovered another issue i never managed to resolve in previous runs | 15:19 |
| CeeMac | uploading images through the dashboard doesn't appear to work | 15:19 |
| CeeMac | just trying to work through logs | 15:20 |
| *** vollman has joined #openstack-ansible | 15:22 | |
| *** sohny has joined #openstack-ansible | 15:27 | |
| jamesdenton | cloudnull with the systemd role, is it possible to force a service to restart due to changes to a different service or task? | 15:28 |
| *** ostackz has joined #openstack-ansible | 15:28 | |
| *** spsurya has joined #openstack-ansible | 15:30 | |
| chandankumar | odyssey4me: http://logs.openstack.org/08/633208/4/check/openstack-ansible-functional-distro_install-centos-7/4525118/job-output.txt.gz#_2019-01-25_14_48_36_064953 | 15:32 |
| chandankumar | odyssey4me: it is showing in distro jobs also | 15:33 |
| chandankumar | odyssey4me: I am not sure what to do in this case? | 15:34 |
| ostackz | odyssey4me hi, Im trying to rerun all rocky playbooks(after not being able to add single compute node), but stuck with this https://pastebin.com/raw/j5mqc9Xr Could you point me how that can be solved? Need to clear caches, how? | 15:38 |
| *** pamsoo has joined #openstack-ansible | 15:39 | |
| cloudnull | mornings | 15:39 |
| cloudnull | jamesdenton yes | 15:39 |
| cloudnull | you can make a service PartOf another | 15:40 |
| cloudnull | https://www.freedesktop.org/software/systemd/man/systemd.unit.html#PartOf= | 15:40 |
| cloudnull | youd have to add that as a config_template override at this time however if we think it'd be useful as a general thing within the role we can add that | 15:40 |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Update cirros from 3.5 to 3.6 https://review.openstack.org/633208 | 15:42 |
| chandankumar | odyssey4me: ^^ libselinux fix in same patch | 15:42 |
| *** pamsoo has quit IRC | 15:45 | |
| broken_one | ostackz: it could be that the python package versions being asked to install either are outdated for your ubuntu version or they dont exist in pip's db | 15:45 |
| broken_one | my team ran into a similar issue on centos7 and we cheated by installing them in the base OS rather than letting OSA try to.. YMMV | 15:48 |
| chandankumar | broken_one: we were already installing it there | 15:49 |
| chandankumar | broken_one: but from ansible is runned and intreprter used needs to have libselinux python known | 15:49 |
| chandankumar | broken_one: https://github.com/openstack/openstack-ansible-os_tempest/blob/master/vars/redhat-7.yml#L21 | 15:50 |
| ThiagoCMC | Guys, I'm facing a hard time to deploy OSA (on Ubuntu 18) with "ceph-install.yml". I'm confused. The following doc: https://docs.openstack.org/openstack-ansible/rocky/user/ceph/full-deploy.html - looks incomplete. | 15:51 |
| ThiagoCMC | Then, I found this: http://bicofino.io/2017/09/05/openstack-ocata-deployment-with-openstack-ansible-and-ceph/ | 15:51 |
| ThiagoCMC | Which have more details on its openstack_user and user_vars | 15:51 |
| FrankZhang | chandankumar: hey man, I saw your commit gathering port status. Is this one related to centOS gating debugging? Thanks | 15:52 |
| ThiagoCMC | and also this: https://www.openstackfaq.com/openstack-ansible-ceph/ | 15:52 |
| ThiagoCMC | I have a friend with huge experience in ceph-ansible, the cluster is healthy! | 15:52 |
| ThiagoCMC | Also, Nova Hypervisors are seeing the Ceph storage space! And from ceph-mon containers, I can run `ceph` commands... | 15:53 |
| chandankumar | FrankZhang: https://review.openstack.org/#/c/633208/ will fix the issue | 15:53 |
| ThiagoCMC | However, Glance is failing to upload images and "os-cinder-install.yml" is failing to finish. | 15:53 |
| ostackz | broken_one trying to figure out how to work around this. You mean install pip with apt? Wondering if this is what breaks it "You are using pip version 18.1, however version 19.0.1 is available." | 15:53 |
| ThiagoCMC | Who can help me with this? :-P | 15:53 |
| FrankZhang | chandankumar: cool thanks! | 15:53 |
| ThiagoCMC | Sorry about sending too many messages... ^_^ | 15:54 |
| *** hamzaachi has joined #openstack-ansible | 15:54 | |
| ThiagoCMC | admin0, can you ping me when you're available? :-P | 15:55 |
| chandankumar | ThiagoCMC: is virtualenv requests issue? | 15:55 |
| ThiagoCMC | chandankumar, I don't think so... | 15:55 |
| ThiagoCMC | chandankumar, when I try to upload an image to glance, it stalls and fails. Even small raw cirros image. | 15:56 |
| *** udesale has quit IRC | 15:56 | |
| broken_one | ostackz: maybe try chandankumar suggestion first | 15:59 |
| broken_one | ostackz: the pip version was not an issue for us | 16:00 |
| broken_one | ostackz: using yum was the way we fixed some python libs and using pip for a couple of others | 16:00 |
| jamesdenton | thanks, cloudnull! | 16:01 |
| chandankumar | broken_one: if it is a import error request issue then install pip == 8.0.1 if not then I need ot take a look at the error | 16:01 |
| *** TxGirlGeek has joined #openstack-ansible | 16:02 | |
| broken_one | chandankumar: confirming we are still doing the "cheaty" way | 16:03 |
| ThiagoCMC | chandankumar, this is a screenshot where I can see that Nova is connected to Ceph: https://imgur.com/a/o15SPyi - so, it looks good but, not for Glance, neither Cinder. | 16:06 |
| odyssey4me | ostackz remember this chat? http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/%23openstack-ansible.2019-01-23.log.html#t2019-01-23T15:25:34 | 16:07 |
| odyssey4me | ostackz we got to the bottom of why that was happening: http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/%23openstack-ansible.2019-01-24.log.html#t2019-01-24T17:22:53 | 16:07 |
| *** macza has joined #openstack-ansible | 16:08 | |
| *** cmart has joined #openstack-ansible | 16:09 | |
| broken_one | ostackz chandankumar we aren't using the cheaty way in the kickstart files anymore so it appears OSA fixed the issue | 16:10 |
| broken_one | at least for centos7 | 16:11 |
| ostackz | odyssey4me sorry not following, could that be Im in wrong git branch? | 16:12 |
| *** macza has quit IRC | 16:13 | |
| *** aedc has quit IRC | 16:15 | |
| *** aedc has joined #openstack-ansible | 16:15 | |
| odyssey4me | ostackz yep, if you were getting error relating to the blazar role, then you were using OSA master, not rocky | 16:17 |
| ostackz | odyssey4me this pattern makes me think I somehow have managed to checkout stein, right? ...19.0.0.0b1... | 16:17 |
| odyssey4me | yep, exactly | 16:18 |
| ostackz | so, need to wipe out everything and start over I guess. There seems nothing fixable | 16:18 |
| broken_one | ostackz: you can see which branch you are working on by running $ git branch | 16:18 |
| odyssey4me | ostackz well, yeah - you can't roll openstack back :/ | 16:19 |
| CeeMac | ostackz, I hadn't changed much so i was able to checkout stable/rocky again | 16:19 |
| CeeMac | not sure if you are in the same boat | 16:19 |
| *** shyamb has joined #openstack-ansible | 16:20 | |
| ostackz | CeeMac my playbooks failed along the way asking to upgrade galera and rabbitmq like "-e rabbitmq_upgrade=true". I did that, but I guess that was bad for rocky and will have to start from scratch. | 16:21 |
| openstackgerrit | James Denton proposed openstack/ansible-role-systemd_service master: Allow PartOf to be defined in unit section https://review.openstack.org/633235 | 16:22 |
| jamesdenton | cloudnull >> https://review.openstack.org/#/c/633235/ | 16:22 |
| openstackgerrit | James Denton proposed openstack/openstack-ansible-os_neutron master: [WIP] Deploy Vector Packet Processing (VPP) Platform for Neutron https://review.openstack.org/631644 | 16:25 |
| CeeMac | ostackz, yeah that doesn't sound great | 16:35 |
| CeeMac | unless you complete the upgrade and just roll with it | 16:35 |
| prometheanfire | what's the 'current' desired version of rabbit in master? | 16:36 |
| broken_one | ostackz chandankumar: ok we fixed our python lib issue using this: https://bugs.launchpad.net/networking-midonet/+bug/1730314 | 16:37 |
| openstack | Launchpad bug 1636567 in openstack-ansible "duplicate for #1730314 devstack mitaka installation fails with error "Running setup.py bdist_wheel for libvirt-python: finished with status 'error'" in Ubuntu 16.10" [High,Fix released] - Assigned to Jesse Pretorius (jesse-pretorius) | 16:37 |
| broken_one | repo_build_upper_constraints_ovverides: | 16:38 |
| broken_one | - libvirt-python==4.10.0 | 16:38 |
| chandankumar | broken_one: cool! | 16:39 |
| *** spatel has joined #openstack-ansible | 16:43 | |
| spatel | cloudnull: ^^ | 16:43 |
| spatel | i have ELK question | 16:43 |
| cloudnull | jamesdenton one nit but otherwise nice! | 16:43 |
| cloudnull | spatel whats up? | 16:43 |
| spatel | I have 10 node ELK cluster running and its heavily used in production.. index rate is close to million/s | 16:44 |
| spatel | Do you know how do i create separate replication network in ELK? | 16:44 |
| *** tstrul has joined #openstack-ansible | 16:45 | |
| spatel | I want to set MTU9000 on replication network so it won't interfere with rest of the network.. | 16:45 |
| spatel | i didn't find any setting in ELK doc to set different replication network | 16:46 |
| jamesdenton | cloudnull nit away! | 16:46 |
| jamesdenton | a global option.. systemd_partof... would default to what, null? | 16:46 |
| *** macza has joined #openstack-ansible | 16:48 | |
| chandankumar | odyssey4me: http://logs.openstack.org/08/633208/5/check/openstack-ansible-functional-distro_install-centos-7/eabbeab/logs/openstack/tempest1/stestr_results.html still failing at WARN: failed: route add -net "0.0.0.0/0" gw "192.168.74.1" | 16:48 |
| chandankumar | odyssey4me: I need we can disable config drive option in tempest.conf | 16:49 |
| *** redrobot has quit IRC | 16:51 | |
| odyssey4me | broken_one oh dear, that's going to hurt badly at some point - we don't build the libvirt whell on purpose because it's so tied to the libvirt binary - instead we symlink it from the host into the venv | 16:55 |
| *** asettle has quit IRC | 16:57 | |
| broken_one | odyssey4me: it is what we had to do to make it work | 16:57 |
| broken_one | we are open to a better solution | 16:58 |
| cloudnull | jamesdenton just undefined . | 16:58 |
| cloudnull | then add a note in the defaults/main.yml how its used. | 16:58 |
| cloudnull | similar to the "systemd_after" option | 16:58 |
| odyssey4me | broken_one it's the end of my day, but I'd be happy to talk through it in more detail on monday | 16:58 |
| broken_one | sure thing. have a good weekend :) | 16:59 |
| cloudnull | spatel nothing we have at the moment | 17:00 |
| cloudnull | however we could certainly map that out. | 17:00 |
| jrosser | cloudnull: i wonder if thats packetbeat on * for little gain? | 17:00 |
| jrosser | spatel: do you know which elasticseaarch indexes are responsible for that? You can look in the management tabs in kibana to see which are the busy ones | 17:01 |
| *** stuartgr has quit IRC | 17:02 | |
| *** stuartgr has joined #openstack-ansible | 17:03 | |
| spatel | jrosser: looking.. | 17:05 |
| spatel | I have 32G memory on data node and i allocated 28G for JAVA HEAP | 17:06 |
| *** shyamb has quit IRC | 17:07 | |
| odyssey4me | OK, I’m out for the day & weekend. | 17:09 |
| odyssey4me | Have a great w/end folks! | 17:09 |
| chandankumar | odyssey4me: Have a great weekend, see ya on monday:-) | 17:09 |
| *** sohny has quit IRC | 17:10 | |
| jrosser | o/ enjoy | 17:10 |
| *** sohny has joined #openstack-ansible | 17:10 | |
| spatel | jrosser: check this out https://ibb.co/RyvnfLF | 17:11 |
| *** gyee has joined #openstack-ansible | 17:11 | |
| spatel | That is my index management tab | 17:11 |
| jrosser | iirc there is a view that you can order by index rate (i'm not in front of mine just now to check) | 17:12 |
| *** mrhillsman is now known as mrhillsman_lunch | 17:13 | |
| prometheanfire | https://github.com/ansible/ansible/pull/51340 may help, if it ever gets looked at | 17:14 |
| *** sohny has quit IRC | 17:15 | |
| *** hamzaachi has quit IRC | 17:17 | |
| cloudnull | spatel jrosser looks like we can set the publish host, which controls the flow of intra elasticsearch node traffic "https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#advanced-network-settings" | 17:27 |
| cloudnull | it be something worth exposing by default ? | 17:27 |
| prometheanfire | cloudnull: you ever use async and reties together? | 17:28 |
| cloudnull | i have not | 17:28 |
| cloudnull | though i think we do that in a couple places? | 17:28 |
| cloudnull | like, lxc-hosts when creating the cache? | 17:28 |
| prometheanfire | ok | 17:29 |
| spatel | cloudnull: as per your link network.publish_host should be my replication network right ? | 17:30 |
| cloudnull | yes. | 17:30 |
| spatel | network.bind_host should be data network where client will pump data | 17:30 |
| spatel | Let me see.. | 17:30 |
| cloudnull | if we exposed that, it'd need to do an inteface and address lookup | 17:30 |
| jrosser | spatel: what actual issue are you seeing - too much traffic on a particular interface? | 17:31 |
| spatel | I am seeing my data nodes are very slow and same time seeing lots of traffic between nodes... look like replication traffic.. | 17:32 |
| spatel | One thought was may be i don't have mtu 9000 set that may over killing my OS performance.. | 17:33 |
| cloudnull | https://pasted.tech/pastes/f17fef5c5a17ff46c13d6e66c863edd9b5a7e12f - maybe something like that to set the replication network | 17:34 |
| spatel | other thought was reduce JVM memory and give more memory to OS, this is my current setting -> ( 32G total = 28G (jvm) + 4G (os) ) | 17:34 |
| cloudnull | setting the MTU to 9000 might help however elk traffice shouldn't be fragementing very much, if at all. | 17:35 |
| jrosser | reducing the index rate can only help | 17:38 |
| jrosser | thats why i keep asking if you deployed packetbeat | 17:38 |
| cloudnull | ^ thats a good question | 17:38 |
| jrosser | and having ram available for filesystem cache is always a bonus | 17:39 |
| *** sohny has joined #openstack-ansible | 17:39 | |
| *** sohny has quit IRC | 17:39 | |
| *** sohny has joined #openstack-ansible | 17:39 | |
| *** aedc has quit IRC | 17:40 | |
| jrosser | going to mtu9000 will only make a marginal difference, and if you don't see a bunch of 'red' kernel time in htop and can attribute it to ip stack processing then it's likley not going to help | 17:40 |
| *** sohny has quit IRC | 17:43 | |
| *** sohny has joined #openstack-ansible | 17:43 | |
| *** sohny has quit IRC | 17:45 | |
| *** spatel has quit IRC | 17:57 | |
| ThiagoCMC | Guys, I just did a fresh install of OSA Rocky / Ubuntu, heat.log shows: | 18:02 |
| ThiagoCMC | The option "__file__" in conf is not known to auth_token | 18:02 |
| ThiagoCMC | The option "here" in conf is not known to auth_token | 18:02 |
| ThiagoCMC | Couple warnings... | 18:03 |
| ThiagoCMC | When I try to access: https://172.29.235.250/project/stacks/ - error. | 18:03 |
| ThiagoCMC | Error: Unable to retrieve stack list. | 18:05 |
| ThiagoCMC | Any idea? | 18:05 |
| *** shardy has quit IRC | 18:06 | |
| ThiagoCMC | "root@vuosctrl-3-heat-api-container-d6d7a3d4:/etc/heat# grep here * -r" shows nothing... lol | 18:06 |
| ThiagoCMC | weird | 18:06 |
| ThiagoCMC | I'm running: `openstack-ansible os-heat-install.yml` again | 18:10 |
| ThiagoCMC | "Playbook execution success" | 18:10 |
| ThiagoCMC | However, Heat still doesn't work. | 18:12 |
| ThiagoCMC | :-( | 18:12 |
| ThiagoCMC | There are two more warnings... | 18:12 |
| ThiagoCMC | Deprecated: Option "deferred_auth_method" from group "DEFAULT" is deprecated for removal (Stored password based deferred auth is broken when used with keystone v3 and is not supported.). Its value may be silently ignored in the future. | 18:12 |
| ThiagoCMC | keystonemiddleware.auth_token [-] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. | 18:12 |
| ThiagoCMC | Maybe that's why it's broken? | 18:13 |
| *** redrobot has joined #openstack-ansible | 18:14 | |
| *** mrhillsman_lunch is now known as mrhillsman | 18:15 | |
| *** nurdie has joined #openstack-ansible | 18:23 | |
| *** hamzaachi has joined #openstack-ansible | 18:25 | |
| *** nurdie has quit IRC | 18:28 | |
| *** Soopaman has quit IRC | 18:29 | |
| *** aedc has joined #openstack-ansible | 18:44 | |
| *** nurdie has joined #openstack-ansible | 18:44 | |
| *** nurdie has quit IRC | 18:48 | |
| *** kstev has joined #openstack-ansible | 18:59 | |
| *** tstrul has quit IRC | 19:08 | |
| *** kstev has quit IRC | 19:15 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-openstack_hosts master: add gentoo support https://review.openstack.org/608325 | 19:36 |
| *** electrofelix has quit IRC | 19:37 | |
| kaiokmo | ThiacoCMC: did you see if the heat services are running inside the container? | 19:39 |
| ThiagoCMC | kaiokmo, yes, heat api, cfn and engine are running. | 19:41 |
| ThiagoCMC | I can even restart them with systemctl | 19:41 |
| *** mgariepy has joined #openstack-ansible | 19:43 | |
| ThiagoCMC | kaiokmo, after restarting heat-api: https://paste.ubuntu.com/p/NyDNMdSkcH/ <- tail log | 19:47 |
| * prometheanfire wonders if rabbit needs wait_for so that policy isn't trying to be set before it's fully running | 19:48 | |
| *** hamzaachi has quit IRC | 19:48 | |
| prometheanfire | "stderr": "Error: this command requires the 'rabbit' app to be running on the target node. Start it with 'rabbitmqctl start_app'.\nArguments given:\n\t-q -n rabbit@infra1 list_policies -p / | 19:48 |
| ThiagoCMC | Also, my `openstack user list --os-cloud=default` works fine. | 19:50 |
| *** hamzaachi has joined #openstack-ansible | 19:52 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Read the path for the logstash queue path https://review.openstack.org/633265 | 19:55 |
| cloudnull | ThiagoCMC there was a heat issue we were working the other day with guilhermesp | 20:12 |
| cloudnull | something to do with the policy files being out of place? | 20:12 |
| cloudnull | i', not sure what the resolution of that was? | 20:12 |
| cloudnull | ThiagoCMC : maybe that's related to what you're seeing | 20:13 |
| *** strattao has quit IRC | 20:14 | |
| *** kstev has joined #openstack-ansible | 20:18 | |
| *** TxGirlGeek has quit IRC | 20:20 | |
| *** aedc has quit IRC | 20:22 | |
| ThiagoCMC | cloudnull, I tried to re-deploy it many times already, always starting fresh (from MaaS), same result, Heat isn't working with latest OSA/Rock stable branch... =/ | 20:24 |
| *** mgariepy has quit IRC | 20:25 | |
| *** nurdie has joined #openstack-ansible | 20:27 | |
| *** kstev has quit IRC | 20:29 | |
| cloudnull | does it say something about permission errors? | 20:29 |
| cloudnull | anything interesting in the heat logs ? | 20:30 |
| *** med_ has quit IRC | 20:30 | |
| *** nurdie has quit IRC | 20:31 | |
| jrosser | This looks related to those messages https://review.openstack.org/#/c/515291/ | 20:33 |
| ThiagoCMC | jrosser, yes, looks the same! https://paste.ubuntu.com/p/NyDNMdSkcH/ | 20:36 |
| jrosser | i think that is saying you should ignore those messages? | 20:36 |
| ThiagoCMC | Well, thing is that Heat isn't working. | 20:37 |
| ThiagoCMC | And, OSA Rocky uses Keystone v3, right? | 20:37 |
| *** spsurya has quit IRC | 20:37 | |
| ThiagoCMC | "Stored password based deferred auth is broken when used with keystone v3 and is not supported." | 20:37 |
| ThiagoCMC | Hard to ignore this one, if OSA/Rocky is set to keystone v3 only. I'm not sure. | 20:38 |
| ThiagoCMC | `openstack stack list` returns: "ERROR: Internal Error" | 20:39 |
| ThiagoCMC | From utility container | 20:40 |
| jrosser | the most useful info will be in the heat service log files i suspect | 20:44 |
| *** DanyC has joined #openstack-ansible | 20:45 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-rabbitmq_server master: fix rabbit slow starts causing errors https://review.openstack.org/633272 | 20:45 |
| ThiagoCMC | Heat service log? | 20:46 |
| ThiagoCMC | I can only see heat.log inside of its container... | 20:46 |
| *** nurdie has joined #openstack-ansible | 20:48 | |
| openstackgerrit | Michael Vollman proposed openstack/openstack-ansible-os_nova master: Avoid distro installing unused services https://review.openstack.org/633275 | 20:51 |
| *** nurdie has quit IRC | 20:52 | |
| openstackgerrit | Michael Vollman proposed openstack/openstack-ansible-os_cinder master: Avoid distro installing unused services https://review.openstack.org/633276 | 20:53 |
| *** hamzaachi has quit IRC | 20:59 | |
| openstackgerrit | Michael Vollman proposed openstack/openstack-ansible-os_neutron master: Avoid distro installing unused services https://review.openstack.org/633277 | 21:02 |
| *** errr has quit IRC | 21:04 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-rabbitmq_server master: add gentoo support to rabbitmq role https://review.openstack.org/633152 | 21:10 |
| *** TxGirlGeek has joined #openstack-ansible | 21:16 | |
| *** DanyC has quit IRC | 21:20 | |
| prometheanfire | osa needs galera/mariadb, is it specifically 'MariaDB Galera Cluster 10.0 Series' from https://downloads.mariadb.org/ or can it be mariadb from the links above that? | 21:21 |
| *** TxGirlGeek has quit IRC | 21:28 | |
| *** errr has joined #openstack-ansible | 21:28 | |
| ThiagoCMC | Also, `openstack stack list --debug" shows: "RESP BODY: {"explanation": "The server has either erred or is incapable of performing the requested operation.", "code": 500, "error": {"traceback": null, "type": "SSLError"}, "title": "Internal Server Error"}" | 21:33 |
| prometheanfire | for mariadb, are any of these features needed by it? | 21:33 |
| prometheanfire | backup bindist galera pam perl server systemd (-client-libs) -cracklib -debug -extraengine -innodb-lz4 -innodb-lzo -innodb-snappy -jdbc -jemalloc -kerberos -latin1 -libressl -mroonga -numa -odbc -oqgraph -profiling -rocksdb (-selinux) -sphinx -sst-mariabackup -sst-rsync -sst-xtrabackup -static -systemtap -tcmalloc -test -tokudb -xml -yassl | 21:33 |
| ThiagoCMC | "openstack user list --os-cloud=default" works! | 21:33 |
| prometheanfire | numa might be useful if that is passed into containers, not sure if that scheduling is numa aware outside or is passed through | 21:34 |
| *** demtwistas has joined #openstack-ansible | 21:38 | |
| *** demtwistas has quit IRC | 21:41 | |
| *** cmart has quit IRC | 21:41 | |
| ThiagoCMC | cloudnull, jrosser, found something! I guess... On haproxy after running `openstack stack list`: | 21:41 |
| ThiagoCMC | haproxy[22408]: 10.0.3.241:40892 [25/Jan/2019:21:40:54.123] keystone_service-front-1/1: SSL handshake failure | 21:41 |
| ThiagoCMC | Not sure if it's related | 21:42 |
| ThiagoCMC | This is actually, happening from time to time without my intervention... | 21:43 |
| ThiagoCMC | I'm using self-sign SSL cert for my lab, created like this: https://www.digitalocean.com/community/tutorials/how-to-create-a-self-signed-ssl-certificate-for-apache-in-ubuntu-18-04 | 21:52 |
| ThiagoCMC | I used this many times in the past without problems... | 21:52 |
| *** spatel has joined #openstack-ansible | 21:59 | |
| -spatel- total used free shared buff/cache available | 21:59 | |
| -spatel- Mem: 31G 30G 344M 15M 791M 479M | 21:59 | |
| -spatel- Swap: 4.0G 477M 3.5G | 21:59 | |
| spatel | My looks bad.. | 21:59 |
| spatel | i think i should reduce jvm memory from 28 to 24G | 22:00 |
| *** TxGirlGeek has joined #openstack-ansible | 22:07 | |
| jrosser | ThiagoCMC: dig around this a bit https://github.com/openstack/openstack-ansible-os_heat/commit/785fcfd33d29ddfee54f09cd6bf126990d64e4dd | 22:13 |
| jrosser | Perhaps something that previously would have hit the internal endpoint now hits the external one | 22:14 |
| jrosser | And then it doesn’t understand your self signed very, perhaps | 22:14 |
| jrosser | *cert | 22:14 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-galera_client master: add gentoo support to galera_client https://review.openstack.org/633289 | 22:18 |
| *** spatel has quit IRC | 22:32 | |
| *** cmart has joined #openstack-ansible | 22:47 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-openstack_hosts master: add gentoo support https://review.openstack.org/608325 | 22:49 |
| ThiagoCMC | jrosser, checking that, thanks! | 22:50 |
| *** KeithMnemonic has quit IRC | 22:51 | |
| ThiagoCMC | Is there a way to tell OSA that I'm using self-sign certs so it will just accept it as a "normal" cert? | 22:52 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-galera_client master: add gentoo support to galera_client https://review.openstack.org/633289 | 22:53 |
| *** eumel8 has quit IRC | 23:15 | |
| cloudnull | ThiagoCMC nothing in osa directly, however if you sync your certs out to your environment (containers, hosts, etc) that'd work | 23:18 |
| cloudnull | you could also do something like so https://learn.hashicorp.com/vault/secrets-management/sm-pki-engine | 23:20 |
| cloudnull | ah, reading back. | 23:20 |
| *** kopecmartin|devc is now known as kopecmartin|off | 23:20 | |
| cloudnull | if that SSL certificate on Haproxy is a public endpoint you could use something like letsencrypt to get a real certificate? | 23:21 |
| cloudnull | otherwise, --insecure on your client call should do the trick | 23:21 |
| *** cmart has quit IRC | 23:23 | |
| *** cmart has joined #openstack-ansible | 23:28 | |
| *** partlycloudy has joined #openstack-ansible | 23:48 | |
| ThiagoCMC | That's true! I'll try letsencrypt! I forgot how easy is to get valid certs these days... =P | 23:53 |
| ThiagoCMC | Maybe OSA should add support for letsencrypt! :-P | 23:53 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!