| *** osnaya has joined #openstack-ansible | 00:02 | |
| *** dxiri has quit IRC | 00:27 | |
| *** dxiri has joined #openstack-ansible | 00:28 | |
| *** dxiri has quit IRC | 00:33 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_neutron stable/queens: Trim whitespace in Neutron configuration file for LBaaS https://review.openstack.org/566765 | 00:38 |
|---|---|---|
| osnaya | @cloudnull @odyssey4me ... re-imaged target hosts (1 controller/1compute) with Ubuntu 16.04 and want to re-run. Target hosts Ubuntu by default doesn't install ansible and says Python can be found in minimal and python3. But our openstack-ansible to run it will need python 2.7.12 and Ansible, right? So is that pre-requisite manual step on target hosts? | 00:41 |
| osnaya | @cloudnull @odyssey4me all the steps in prepare target hosts are executed. Please clarify about python and ansible installation on target hosts? | 00:49 |
| rschulman | osnaya: FYI - I believe its after work hours for both of them, so don't be surprised if they don't answer tonight. | 00:53 |
| osnaya | @rschulman thanks. Just a generic question. Can you provide a pointer? I went over the docs for Pike and executed all steps on target hosts already. Didn't find anywhere about Python 2.7.12 or ansible install.... so just wondering. | 00:55 |
| osnaya | @rschulman from deployment host, I can't run ansible unless my target has python and ansible... so just wondering, if this should be part of pre-requisite steps | 00:57 |
| osnaya | or am I missing something... | 00:57 |
| rschulman | I'm not sure what you mean, osnaya , the instructions for preparing the nodes on ubuntu include apt-get install python | 01:00 |
| rschulman | and I believe (though I'm not sure) that the bootstrap-ansible.sh script installs ansible for you. | 01:00 |
| *** astellwag has quit IRC | 01:43 | |
| *** astellwag has joined #openstack-ansible | 01:47 | |
| *** dxiri has joined #openstack-ansible | 01:49 | |
| *** dxiri has quit IRC | 01:53 | |
| openstackgerrit | lidong proposed openstack/openstack-ansible-os_cloudkitty master: Update the outdated links https://review.openstack.org/566772 | 02:11 |
| *** dave-mccowan has quit IRC | 02:20 | |
| *** astellwag has quit IRC | 02:22 | |
| *** astellwag has joined #openstack-ansible | 02:26 | |
| *** dxiri has joined #openstack-ansible | 02:28 | |
| *** klamath has joined #openstack-ansible | 02:30 | |
| *** klamath_ has quit IRC | 02:33 | |
| *** esberglu has joined #openstack-ansible | 02:48 | |
| *** klamath has quit IRC | 03:18 | |
| *** mma has joined #openstack-ansible | 03:21 | |
| *** nicolasbock has quit IRC | 03:40 | |
| *** gyee has quit IRC | 03:48 | |
| *** udesale has joined #openstack-ansible | 03:54 | |
| *** mwarad has joined #openstack-ansible | 03:55 | |
| openstackgerrit | fupingxie proposed openstack/openstack-ansible-os_neutron master: Trivial: Fix the pep8 warning https://review.openstack.org/566780 | 03:59 |
| *** mwarad has quit IRC | 04:01 | |
| *** mwarad has joined #openstack-ansible | 04:04 | |
| *** mwarad has quit IRC | 04:17 | |
| *** gyee has joined #openstack-ansible | 04:17 | |
| *** gyee has quit IRC | 04:24 | |
| *** chhagarw has joined #openstack-ansible | 04:37 | |
| *** chhagarw has quit IRC | 04:41 | |
| *** dxiri has quit IRC | 04:42 | |
| *** chhagarw has joined #openstack-ansible | 04:44 | |
| *** klamath has joined #openstack-ansible | 05:18 | |
| *** radeks has joined #openstack-ansible | 05:24 | |
| osnaya | quit | 05:24 |
| *** osnaya has quit IRC | 05:24 | |
| *** osnaya has joined #openstack-ansible | 05:24 | |
| *** mwarad has joined #openstack-ansible | 05:30 | |
| *** klamath has quit IRC | 05:37 | |
| evrardjp | osnaya: yes python2 is required in the current stable branches. | 05:54 |
| openstackgerrit | fupingxie proposed openstack/openstack-ansible-os_tacker master: Trivial: Fix the pep8 warning https://review.openstack.org/566784 | 05:58 |
| *** mma has quit IRC | 06:01 | |
| *** markvoelker has quit IRC | 06:01 | |
| *** pcaruana has joined #openstack-ansible | 06:01 | |
| *** mma has joined #openstack-ansible | 06:01 | |
| *** mwarad has quit IRC | 06:04 | |
| *** mma has quit IRC | 06:06 | |
| *** armaan has joined #openstack-ansible | 06:07 | |
| openstackgerrit | fupingxie proposed openstack/openstack-ansible-os_zaqar master: Trivial: Fix the pep8 warning https://review.openstack.org/566786 | 06:08 |
| *** cmart has quit IRC | 06:10 | |
| *** mma has joined #openstack-ansible | 06:10 | |
| *** klamath has joined #openstack-ansible | 06:11 | |
| *** armaan has quit IRC | 06:11 | |
| *** mma has quit IRC | 06:11 | |
| *** armaan has joined #openstack-ansible | 06:12 | |
| *** klamath has quit IRC | 06:15 | |
| *** evin has joined #openstack-ansible | 06:17 | |
| *** mma has joined #openstack-ansible | 06:20 | |
| *** udesale has quit IRC | 06:23 | |
| *** udesale has joined #openstack-ansible | 06:23 | |
| *** armaan has quit IRC | 06:24 | |
| *** armaan has joined #openstack-ansible | 06:25 | |
| *** hamza21 has joined #openstack-ansible | 06:31 | |
| *** mwarad has joined #openstack-ansible | 06:36 | |
| *** bhujay has joined #openstack-ansible | 06:42 | |
| *** epalper has joined #openstack-ansible | 06:45 | |
| evrardjp | good morning | 06:52 |
| evrardjp | niraj_singh: do you want me to update the masakari base jobs patch? | 06:55 |
| evrardjp | or you're handling it? | 06:56 |
| niraj_singh | evrardjp: i am working on it. | 06:57 |
| evrardjp | ok cool | 06:58 |
| evrardjp | it's just whitespaces, but I don't have my easy trimmer with me today :p | 06:58 |
| evrardjp | so vim it would be | 06:58 |
| niraj_singh | There was so many white spaces. i removed them | 06:58 |
| niraj_singh | and currently linters is failing inmy env for openstack_hosts | 06:59 |
| niraj_singh | http://paste.openstack.org/show/720537/ | 07:00 |
| *** radeks_ has joined #openstack-ansible | 07:01 | |
| niraj_singh | And also there is some problem in my git. So working on it. | 07:01 |
| *** markvoelker has joined #openstack-ansible | 07:01 | |
| evrardjp | could you please push the patch with the whitespace removal? This way I can help too | 07:03 |
| evrardjp | it's a good first start :p | 07:03 |
| niraj_singh | ok. | 07:03 |
| *** mbuil has joined #openstack-ansible | 07:06 | |
| odyssey4me | o/ | 07:08 |
| *** armaan has quit IRC | 07:14 | |
| *** udesale has quit IRC | 07:15 | |
| *** udesale has joined #openstack-ansible | 07:15 | |
| evrardjp | odyssey4me: an opinion on https://review.openstack.org/#/c/566587/ vs removing the upgrade jobs? | 07:18 |
| *** bertys has joined #openstack-ansible | 07:18 | |
| odyssey4me | evrardjp yeah, let's do that - it's simple enough | 07:23 |
| odyssey4me | I think we should try and maintain the upgrade test for as long as reasonably possible | 07:23 |
| evrardjp | yup | 07:24 |
| *** klamath has joined #openstack-ansible | 07:25 | |
| evrardjp | I agree, I'd like to see a way to go from N to latest stable still tested. | 07:25 |
| evrardjp | hwoarang: opinion on https://review.openstack.org/#/c/566672/ ? | 07:25 |
| evrardjp | I like the idea of saying "ubuntu 16.04 ships by default with python2, so let's stick to python2 for it" | 07:26 |
| evrardjp | so if suse is using python3 by default, let's use python3 :) | 07:27 |
| evrardjp | ubuntu 18.04 will ship with python3 by default, so let's do python3 only | 07:27 |
| evrardjp | it's simple | 07:27 |
| *** klamath has quit IRC | 07:29 | |
| hwoarang | evrardjp: i have no opinion on that | 07:30 |
| evrardjp | ok | 07:30 |
| evrardjp | should we include that new package? I have seen that failure on ubuntu 16. and I remember you talked about it | 07:31 |
| openstackgerrit | Merged openstack/openstack-ansible-pip_install stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566653 | 07:31 |
| hwoarang | i have no opinion on that either. including this package alone doesn't fix the problem for me because it's missing from undex-url on pip.conf | 07:32 |
| openstackgerrit | Merged openstack/openstack-ansible-rsyslog_client stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566658 | 07:32 |
| evrardjp | ok | 07:32 |
| evrardjp | so maybe it's easier to not attempt to use python3 on 16.04 ? | 07:32 |
| evrardjp | this would unblock things | 07:32 |
| openstackgerrit | Merged openstack/openstack-ansible-ops stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566620 | 07:32 |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566611 | 07:33 |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566622 | 07:33 |
| openstackgerrit | Merged openstack/openstack-ansible-memcached_server stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566617 | 07:33 |
| openstackgerrit | Merged openstack/openstack-ansible-rsyslog_server stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566659 | 07:33 |
| openstackgerrit | Merged openstack/openstack-ansible-os_molteniron stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566636 | 07:33 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566637 | 07:33 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: [WIP] Move translations into metal https://review.openstack.org/566505 | 07:34 |
| hwoarang | evrardjp: what's the problem with 16.04? | 07:34 |
| *** udesale_ has joined #openstack-ansible | 07:34 | |
| hwoarang | i am behind my emails etc due to holidays so if something broke i am not aware of it | 07:35 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566612 | 07:35 |
| evrardjp | http://logs.openstack.org/05/566505/1/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/d722526/job-output.txt.gz#_2018-05-07_05_58_10_468433 | 07:36 |
| *** markvoelker has quit IRC | 07:36 | |
| evrardjp | hwoarang: it's not fully broken | 07:36 |
| evrardjp | I haven't seen that in all gates, weirdly | 07:36 |
| evrardjp | but that patch doesn't have any impact on this | 07:36 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566616 | 07:37 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566618 | 07:37 |
| hwoarang | very strange error | 07:37 |
| *** udesale has quit IRC | 07:37 | |
| *** tosky has joined #openstack-ansible | 07:38 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566615 | 07:46 |
| openstackgerrit | Merged openstack/openstack-ansible-os_searchlight stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566645 | 07:46 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_server stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566657 | 07:46 |
| openstackgerrit | Merged openstack/openstack-ansible-os_watcher stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566651 | 07:47 |
| openstackgerrit | Merged openstack/openstack-ansible-os_zaqar stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566652 | 07:48 |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566643 | 07:49 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566626 | 07:50 |
| openstackgerrit | Merged openstack/openstack-ansible-os_congress stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566627 | 07:50 |
| evrardjp | odyssey4me: do you know if I have to kill the newton jobs before EOL or will they get auto killed when the branch will die? | 07:52 |
| evrardjp | (thinking about periodics) | 07:52 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566648 | 07:56 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566624 | 07:57 |
| openstackgerrit | Merged openstack/openstack-ansible-rabbitmq_server stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566655 | 07:57 |
| *** sep__ has quit IRC | 07:58 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_almanach master: OSA0003: Password should not be logged https://review.openstack.org/566715 | 07:58 |
| *** lvdombrkr has joined #openstack-ansible | 07:58 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_almanach master: Updated from OpenStack Ansible Tests https://review.openstack.org/565104 | 07:58 |
| *** sep has joined #openstack-ansible | 08:02 | |
| evrardjp | if someone has a machine available, I'd be happy to know why the idempotency test fails in https://review.openstack.org/#/c/566610/1 | 08:03 |
| openstackgerrit | Merged openstack/openstack-ansible-os_panko stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566642 | 08:04 |
| odyssey4me | evrardjp I expect they will die with the branch - zuul doesn't ingest jobs from tags | 08:07 |
| evrardjp | ok | 08:07 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566629 | 08:09 |
| odyssey4me | hwoarang evrardjp yeah, perhaps we should just stick to python2 for now and plan for python3 next cycle when we can do it for all platforms in one go | 08:10 |
| odyssey4me | I mean, if any python3 testing and patches are implemented - that's fine... but I don't think we're in a place where we have enough resources to tackle it just yet | 08:11 |
| openstackgerrit | Merged openstack/openstack-ansible-os_gnocchi stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566630 | 08:11 |
| openstackgerrit | Merged openstack/openstack-ansible-os_horizon stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566632 | 08:11 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_almanach stable/queens: OSA0003: Password should not be logged https://review.openstack.org/566716 | 08:11 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566647 | 08:11 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_almanach stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566621 | 08:12 |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566623 | 08:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566650 | 08:13 |
| evrardjp | odyssey4me: agreed | 08:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566641 | 08:17 |
| openstackgerrit | Merged openstack/openstack-ansible-os_designate stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566628 | 08:18 |
| *** DanyC has joined #openstack-ansible | 08:19 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566639 | 08:19 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron stable/queens: Trim whitespace in Neutron configuration file for LBaaS https://review.openstack.org/566765 | 08:19 |
| hwoarang | evrardjp: odyssey4me re https://review.openstack.org/#/q/topic:remove_python3+(status:open+OR+status:merged) shouldn't we do that in rocky too? or are we going to introduce python3 in rocky? | 08:21 |
| osnaya | what is the current openstack-ansible version for Pike? Just checked on my setup and it shows "16.0.11" is that the latest code? The docs show "16.0.13"? | 08:21 |
| *** vnogin has joined #openstack-ansible | 08:26 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566635 | 08:27 |
| openstackgerrit | Merged openstack/openstack-ansible-os_zaqar master: Trivial: Fix the pep8 warning https://review.openstack.org/566786 | 08:28 |
| *** hamza21 has quit IRC | 08:33 | |
| *** markvoelker has joined #openstack-ansible | 08:33 | |
| hwoarang | odyssey4me: i have a question about https://github.com/openstack/openstack-ansible-repo_build/blob/master/tasks/repo_clone_git.yml#L34-L64 in opnfv we are seeing a lot of failures here because some git clones are failuing. the way the task is written does not seem to retry git-clone on failures. any suggestions on how to improve that? i read the commit message on why 'async' is used there but i dont quite understand it | 08:33 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker master: Trivial: Fix the pep8 warning https://review.openstack.org/566784 | 08:34 |
| *** mwarad has quit IRC | 08:37 | |
| *** nsingh has joined #openstack-ansible | 08:38 | |
| openstackgerrit | lidong proposed openstack/ansible-hardening master: Update the outdated links https://review.openstack.org/566815 | 08:38 |
| odyssey4me | hwoarang we learned later that async tasks don't do retries | 08:39 |
| odyssey4me | using async was to try and make the clones happen in parallel, rather than in serial | 08:39 |
| odyssey4me | but yeah, as it turns out when using async, the task doesn't retry - only the async retries... | 08:39 |
| odyssey4me | so the best there would be to remove the use of async - revert it to just using a standard task with retries | 08:40 |
| openstackgerrit | Merged openstack/openstack-ansible-plugins stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566654 | 08:40 |
| *** aspiers has quit IRC | 08:40 | |
| *** goldenfri has quit IRC | 08:41 | |
| *** dweaver has quit IRC | 08:41 | |
| *** tomcsi_ has quit IRC | 08:41 | |
| hwoarang | odyssey4me: ok thank you for the explanation | 08:41 |
| odyssey4me | hwoarang wrt python3 for rocky... I think the original intent was just remove python3 from queens... but given the current state and discussion earlier today, I think it might be good to do the same for rocky and tackle python3 properly next cycle | 08:41 |
| *** dweaver has joined #openstack-ansible | 08:41 | |
| *** niraj_singh has quit IRC | 08:41 | |
| *** tomcsi has joined #openstack-ansible | 08:41 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566633 | 08:43 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566649 | 08:44 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566640 | 08:44 |
| *** aspiers has joined #openstack-ansible | 08:45 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_congress master: Trivial: Fix the pep8 warning https://review.openstack.org/566499 | 08:46 |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566631 | 08:54 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Trivial: Fix the pep8 warning https://review.openstack.org/566780 | 08:56 |
| *** mwarad has joined #openstack-ansible | 08:58 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-repo_build master: tasks: repo_clone_git: Retry git-clone on failures https://review.openstack.org/566819 | 08:59 |
| *** armaan has joined #openstack-ansible | 09:01 | |
| *** markvoelker has quit IRC | 09:07 | |
| *** armaan has quit IRC | 09:08 | |
| evrardjp | odyssey4me: hwoarang I think it would be good to remove python3 from ubuntu16, and do what's best with ubuntu 18 | 09:08 |
| evrardjp | hwoarang: opensuse is python3 by default? | 09:09 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566634 | 09:10 |
| Tahvok | python3 is still not supported even in rocky? | 09:10 |
| evrardjp | Tahvok: I guess it will depend on people willing to make it work | 09:11 |
| evrardjp | last comment of hwoarang helps me understand the situation: some 16.04 images have python3 installed by default. | 09:11 |
| Tahvok | python3 is default in ubuntu 16.04 | 09:12 |
| Tahvok | But aren't we dealing with this by using venvs? | 09:12 |
| *** djdule has joined #openstack-ansible | 09:12 | |
| Tahvok | Wait, no, python3 is not default in 16.04.. | 09:13 |
| evrardjp | it is not :p | 09:13 |
| evrardjp | it is the default in 18.04 | 09:14 |
| Tahvok | Just booted my first instance of 18.04 yesterday.. | 09:14 |
| Tahvok | $ python --version | 09:15 |
| Tahvok | Python 2.7.15rc1 | 09:15 |
| evrardjp | I guess people change it with update-alternatives | 09:15 |
| Tahvok | Somehow it's not default on that instance | 09:15 |
| djdule | Hi, I need some help with installation of Ceph with openstack-ansible. I do have cloud that is already running, that was installed with openstack-ansible and it works fine. Now when I added Ceph nodes and re-run openstack-ansible setup-infrastructure.yml one of first error messages I get is fatal: [infra1_ceph-mon_container-7722acfe]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Error: container infra1_ce | 09:15 |
| evrardjp | Tahvok: python2 shouldn't be installed there? | 09:15 |
| djdule | any idea why it fails ? | 09:15 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566625 | 09:15 |
| *** mwarad has quit IRC | 09:15 | |
| Tahvok | evrardjp: let me try booting a clean instances | 09:16 |
| evrardjp | thanks | 09:16 |
| evrardjp | Tahvok: python --version doesn't give anything | 09:18 |
| evrardjp | by edfault | 09:18 |
| evrardjp | but python3 --version does :) | 09:18 |
| evrardjp | on 18.04 | 09:18 |
| evrardjp | so it means python doesn't symlink to anything for now | 09:18 |
| *** gkadam has joined #openstack-ansible | 09:19 | |
| Tahvok | Command 'python' not found, but can be installed with: | 09:19 |
| Tahvok | Well, there is no python at all by default | 09:19 |
| Tahvok | That is the same case actually for ubuntu 16.04 | 09:19 |
| Tahvok | I must say, this are cloud images of ubuntu: https://cloud-images.ubuntu.com/ | 09:20 |
| Tahvok | Ok, I was wrong, and I'm sorry XD | 09:20 |
| Tahvok | There is python3 installed | 09:20 |
| Tahvok | $ python3 --version | 09:21 |
| Tahvok | Python 3.6.5 | 09:21 |
| Tahvok | But why did they choose not to simplink to python? | 09:21 |
| Tahvok | symlink* | 09:21 |
| openstackgerrit | fupingxie proposed openstack/openstack-ansible-openstack_hosts master: Trivial: Fix the pep8 warning https://review.openstack.org/566830 | 09:23 |
| Tahvok | djdule: what's the rest of the error? Can you paste it somewhere? | 09:23 |
| djdule | @Thavok that is the only one I have. When execution finishes, everything else looks ok | 09:24 |
| Tahvok | djdule: also, are you trying to install ceph with openstack ansible, or do you already have ceph installed, and just want to use existing cluster? | 09:24 |
| djdule | here is play recap: | 09:24 |
| djdule | PLAY RECAP ************************************************************************************************************************************ compute1 : ok=1 changed=0 unreachable=0 failed=0 compute2 : ok=1 changed=0 unreachable=0 failed=0 compute3 : ok=1 changed=0 unreachable=0 failed=0 compute4 : ok=1 changed=0 unrea | 09:24 |
| Tahvok | djdule: not sure if you can see, but what you pasted here, is cutted at the end | 09:25 |
| djdule | No, ceph is not running, I want to install it with openstack-ansible | 09:25 |
| djdule | I do have one controller node and I want to put monitor there, and there are 2 nodes that I want to use as OSDs | 09:25 |
| evrardjp | Tahvok: that's a bad idea to symlink | 09:25 |
| Tahvok | evrardjp: why is that? | 09:25 |
| Miouge | I’m often hitting the max_connection=1600 in the Galera cluster. Most connections are in “Sleep” state. Is that normal? | 09:26 |
| evrardjp | because now all your scripts made for python2 will run on python3 by default | 09:26 |
| djdule | here is play recap https://justpaste.it/4qvz3 | 09:26 |
| Tahvok | Arch has been symlinked to python3 since... some years ago.. | 09:26 |
| *** klamath has joined #openstack-ansible | 09:26 | |
| evrardjp | a user should instead update-alternatives, to make sure he knows his stuff :) | 09:26 |
| evrardjp | haha | 09:26 |
| Tahvok | evrardjp: that's the problem of developers.. They should use python2 executable and not python3.. | 09:26 |
| evrardjp | userfriendliness | 09:26 |
| odyssey4me | Tahvok It's not really for us to try and assume that we know what's best for the distro - for us we just need to understand what they're doing and to use it the way it was designed to be used. | 09:27 |
| Tahvok | I agree | 09:28 |
| openstackgerrit | fupingxie proposed openstack/openstack-ansible-os_keystone master: Trivial: Fix the pep8 warning https://review.openstack.org/566833 | 09:28 |
| odyssey4me | If they aren't symlinking python -> python3, then as long as we know that we can work with it. | 09:28 |
| Tahvok | I was just trying to understand why that did that... | 09:28 |
| Tahvok | But do we need python3 or python2? | 09:28 |
| Tahvok | If we need python2, aren't we already using python2 by default everywhere? | 09:28 |
| Tahvok | If yes, why the missing symlink is a problem for us? | 09:29 |
| odyssey4me | we implemented python3 as a preference for xenial to see how it went... and it's not gone well - so we're discussing reverting that decision and sticking with python2 for this cycle | 09:29 |
| Tahvok | I see... Btw, what version of openstack will we support in ubuntu 18? | 09:30 |
| openstackgerrit | fupingxie proposed openstack/openstack-ansible-plugins master: Trivial: Fix the pep8 warning https://review.openstack.org/566834 | 09:31 |
| odyssey4me | unless someone does work to get bionic support in rocky, we'll only tackle that work in stein | 09:31 |
| *** klamath has quit IRC | 09:31 | |
| odyssey4me | if anyone wants bionic support for rocky, we'll need them to step up and do the patches | 09:31 |
| odyssey4me | I personally don't have time to tackle it. | 09:32 |
| Tahvok | but will we support stein in ubuntu 16? | 09:34 |
| Tahvok | For easier upgrade... | 09:34 |
| openstackgerrit | fupingxie proposed openstack/openstack-ansible-lxc_hosts master: Trivial: Fix the pep8 warning https://review.openstack.org/566836 | 09:35 |
| osnaya | @evrardjp How do I find out what is the latest version? 16.0.12 or .13? to do git clone? | 09:39 |
| evrardjp | git tag -l | 09:40 |
| odyssey4me | jamespage Is UCA xenial-updates going to get a Rocky dist soon - typically IIRC that happens at m1, but it's not happened yet | 09:40 |
| jamespage | odyssey4me: no it won't get rocky | 09:40 |
| jamespage | odyssey4me: queens was the last pocket for xenial | 09:40 |
| jamespage | we will do rocky for bionic only | 09:41 |
| evrardjp | Tahvok: no | 09:41 |
| evrardjp | Stein should probably be 18.04 | 09:41 |
| odyssey4me | jamespage hmm, ok thanks | 09:41 |
| odyssey4me | evrardjp Tahvok given that this is similar to the newton situation, we should probably ensure we do bionic support for Rocky so make Rocky the transitional release | 09:41 |
| evrardjp | exactly | 09:42 |
| odyssey4me | if we leave it to stein it's going to be too late - the binaries we rely on will be too far apart | 09:42 |
| djdule | @Tahvok I did re-run setup-infrastructure with -vvvv option and here is full output of that one failure https://justpaste.it/2u3xt | 09:42 |
| jamespage | bionic-rocky will open in the next week or so - we typically wait for the source ubuntu release to open first, then then open the UCA pockets | 09:42 |
| jamespage | obs queens was in xenial-queens and bionic, so its the one release where there is overlay | 09:43 |
| jamespage | overlap | 09:43 |
| Tahvok | odyssey4me: So perhaps we should do queens the transition release, and not rocky? ^ | 09:45 |
| Tahvok | Though I understand we do not rely on ubuntu packages, so it's not required | 09:46 |
| *** cmart has joined #openstack-ansible | 09:47 | |
| odyssey4me | Tahvok Queens is stable - we're not going to push a bunch of work in it to destabilise it. | 09:47 |
| Tahvok | Makes sense | 09:48 |
| odyssey4me | That said, it could make sense to do that given that's an officially supported overlap. | 09:48 |
| odyssey4me | Perhaps we should discuss that at the meeting this afternoon. | 09:48 |
| Tahvok | If you can move it one hour earlier, I'll be able to join :) | 09:49 |
| Tahvok | odyssey4me: btw, can you please answer me here: https://review.openstack.org/#/c/566269/ | 09:51 |
| odyssey4me | Tahvok done | 09:53 |
| Tahvok | Thanks | 09:53 |
| Tahvok | From what I see at the rabbitmq_stopped tasks, we already stopping the rest of rabbitmq servers, and only after that the first one | 09:55 |
| Tahvok | https://github.com/openstack/openstack-ansible-rabbitmq_server/blob/master/tasks/rabbitmq_stopped.yml | 09:55 |
| Tahvok | The question is, after all rabbitmq servers are stopped, does it matter which one is upgraded first? | 09:56 |
| odyssey4me | I have no idea. | 09:57 |
| odyssey4me | If you have the time to work out an approach that works and is reliable, then propose the changes to make that happen. | 09:57 |
| *** udesale_ has quit IRC | 09:59 | |
| Tahvok | I'm going to check this.. I also think there is a race condition, since this is running on all rabbitmq containers, and the stop/start is happening in both of them - then the stop/start will happeng on both of them.. | 09:59 |
| Tahvok | odyssey4me: ok, will work on this | 10:00 |
| odyssey4me | thanks Tahvok | 10:00 |
| odyssey4me | I think it may make sense to do the serialisation in the role in this case given that it has to happen at special places and the upgrade procedure requires the whole cluster (except the first node) to be shut down when upgrading. However, perhaps the whole upgrade process can be made simpler somehow... it'll take some investigation to figure out. | 10:01 |
| openstackgerrit | Albert Mikaelyan proposed openstack/openstack-ansible-rabbitmq_server master: Revert "Do not restart rabbitmq when no version is changed" https://review.openstack.org/566839 | 10:02 |
| openstackgerrit | Albert Mikaelyan proposed openstack/openstack-ansible-rabbitmq_server stable/queens: Revert "Do not restart rabbitmq when no version is changed" https://review.openstack.org/566840 | 10:02 |
| Tahvok | Please revert this for now ^ | 10:03 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Ensure that repo checks use the right values https://review.openstack.org/564422 | 10:03 |
| *** armaan has joined #openstack-ansible | 10:04 | |
| *** markvoelker has joined #openstack-ansible | 10:04 | |
| odyssey4me | Tahvok can you abandon https://review.openstack.org/566121 and https://review.openstack.org/566119 then? | 10:04 |
| evrardjp | fair point | 10:05 |
| hwoarang | odyssey4me: evrardjp could you help me get https://review.openstack.org/#/c/566819/ in so i can unblock opnfv? | 10:07 |
| odyssey4me | hwoarang reviewed | 10:07 |
| odyssey4me | and quite honestly I think we should backport that as far back as possible | 10:07 |
| hwoarang | i want it to pike too | 10:08 |
| Tahvok | odyssey4me: was exactly doing that :) | 10:08 |
| odyssey4me | Tahvok great, thank you for taking the time - the infrastructure roles tend to get less attention as they generally 'just work', so if you're able to spend some time on them it's greatly appreciated | 10:09 |
| odyssey4me | hwoarang if you have a gap, can you review https://review.openstack.org/#/c/557896/ to help reduce the tempest failures due to slow hosts | 10:17 |
| openstackgerrit | Niraj Singh proposed openstack/openstack-ansible-os_masakari master: Add Maskari base jobs https://review.openstack.org/566538 | 10:18 |
| hwoarang | odyssey4me: done | 10:20 |
| odyssey4me | thanks hwoarang - evrardjp I'm guessing you'll be porting that back once it merges? | 10:20 |
| evrardjp | which one? | 10:22 |
| evrardjp | the git cloning or the timeouts? | 10:22 |
| evrardjp | I've seen tempest failure with 600 seconds too | 10:23 |
| evrardjp | just fyi | 10:23 |
| evrardjp | it's just far less frequent | 10:23 |
| evrardjp | I guess we need to fix the elephant in the room too | 10:23 |
| hwoarang | what is that | 10:24 |
| odyssey4me | hwoarang FYI with https://review.openstack.org/562606 I think the integrated build may fail due to https://github.com/openstack/openstack-ansible/blob/master/playbooks/utility-install.yml#L55-L76 | 10:25 |
| evrardjp | hwoarang: the fact that cinder-backup was taking 100% cpu | 10:26 |
| evrardjp | but I don't have anything to prove it | 10:26 |
| evrardjp | anymore | 10:26 |
| *** udesale has joined #openstack-ansible | 10:27 | |
| hwoarang | odyssey4me: oh :( does this have to be there? | 10:27 |
| *** udesale has quit IRC | 10:27 | |
| *** udesale has joined #openstack-ansible | 10:28 | |
| hwoarang | i often forget that the integrated-build does a lot more than calling the roles | 10:28 |
| odyssey4me | hwoarang it's the same sort of legacy - I'd personally rather see that switched to a distro package list | 10:28 |
| hwoarang | odyssey4me: i mean, why not drop these two tasks and let the clients being pulled in by the roles who needs them ie tempest? | 10:28 |
| hwoarang | why do we have to put them there in advance | 10:29 |
| hwoarang | forget about the recent tempest changes | 10:29 |
| odyssey4me | does the tempest role always execute? I don't think it does | 10:29 |
| hwoarang | hmm i thought it was always part of the gates | 10:29 |
| hwoarang | but good point | 10:29 |
| odyssey4me | https://github.com/openstack/openstack-ansible/blob/master/playbooks/setup-openstack.yml#L48-L49 | 10:30 |
| odyssey4me | for an AIO it does, because we set that - but for a prod environment it doesn't... for good reason actually because the tempest role does fun things like setup a demo account with a static username/password | 10:30 |
| hwoarang | yep | 10:31 |
| hwoarang | have you seen any actual failure because of that? | 10:31 |
| odyssey4me | the role has been built for the purpose of our testing, not really for generic use... we could change that | 10:31 |
| hwoarang | i mean because of the recent tempest changes | 10:31 |
| nsingh | evrardjp: Sorry for he late update. I was facing issue with my git. removed all the wwhite space and some other errors | 10:33 |
| *** nicolasbock has joined #openstack-ansible | 10:36 | |
| *** markvoelker has quit IRC | 10:38 | |
| *** yolanda_ has joined #openstack-ansible | 10:40 | |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build master: tasks: repo_clone_git: Retry git-clone on failures https://review.openstack.org/566819 | 10:41 |
| *** yolanda has quit IRC | 10:44 | |
| openstackgerrit | lidong proposed openstack/ansible-hardening master: Update the outdated links https://review.openstack.org/566815 | 10:47 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-repo_build stable/queens: tasks: repo_clone_git: Retry git-clone on failures https://review.openstack.org/566850 | 10:47 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-repo_build stable/pike: tasks: repo_clone_git: Retry git-clone on failures https://review.openstack.org/566851 | 10:47 |
| *** mma has quit IRC | 10:52 | |
| *** mma has joined #openstack-ansible | 10:52 | |
| *** mma has quit IRC | 10:56 | |
| evrardjp | tempest needs refactor, we discussed it, so whatever is properly spec-ed is fine | 11:01 |
| openstackgerrit | Merged openstack/openstack-ansible-rabbitmq_server master: Revert "Do not restart rabbitmq when no version is changed" https://review.openstack.org/566839 | 11:13 |
| openstackgerrit | Merged openstack/openstack-ansible-rabbitmq_server stable/queens: Revert "Do not restart rabbitmq when no version is changed" https://review.openstack.org/566840 | 11:13 |
| *** kysse has quit IRC | 11:13 | |
| hwoarang | evrardjp: odyssey4me could you help me get the backport https://review.openstack.org/#/q/Iff0c3c589f5e0a935e0e01cd820083b76d43cc92 in please? | 11:21 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-openstack_hosts master: [DNM] - Testing openSUSE openstack hosts https://review.openstack.org/566860 | 11:29 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: Overriding default tempest timeouts https://review.openstack.org/557896 | 11:33 |
| *** markvoelker has joined #openstack-ansible | 11:34 | |
| *** kysse has joined #openstack-ansible | 11:39 | |
| *** mma has joined #openstack-ansible | 11:43 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_tempest stable/queens: Overriding default tempest timeouts https://review.openstack.org/566862 | 11:43 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_tempest stable/pike: Overriding default tempest timeouts https://review.openstack.org/566863 | 11:43 |
| hwoarang | odyssey4me: evrardjp looks like openstack infra removed mirror_info.sh from images https://github.com/openstack-infra/project-config/commit/70eb9ea584ef442005481d3b8f403b864d78a903 | 11:51 |
| hwoarang | as such we don't place the correct mirror on openstack_hosts anymore | 11:52 |
| hwoarang | on openstack-hosts: common-tasks/test-set-nodepool-vars.yml | 11:52 |
| hwoarang | do you know what's the alternative? | 11:52 |
| hwoarang | pabelanger: ^ | 11:52 |
| hwoarang | looks like some things are hardcoded to 'http://mirror.dfw.rax.openstack.org' | 11:53 |
| hwoarang | the removal of mirror_info also affects get-ansible-role-requirements.yml | 11:55 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build stable/pike: tasks: repo_clone_git: Retry git-clone on failures https://review.openstack.org/566851 | 11:56 |
| *** dxiri has joined #openstack-ansible | 11:58 | |
| evrardjp | hwoarang: that's bad news | 11:58 |
| evrardjp | pabelanger: that would be nice if you could give us feedback there :) | 11:58 |
| odyssey4me | hwoarang I thought that particular script wasn't used any more - but another one is that does the same thing? | 11:58 |
| Tahvok | .лчда | 11:58 |
| Tahvok | sorry | 11:58 |
| hwoarang | odyssey4me: i am trying to find out what happened but we still use mirror_info.sh in our tests which do nothing for a while now | 11:58 |
| evrardjp | that's cyrilic? | 11:58 |
| hwoarang | odyssey4me: the prob is that we check for /etc/mirror_info.sh existence to figure out if we are on CI or not | 11:59 |
| Tahvok | evrardjp: yep | 12:01 |
| evrardjp | It's not in my list of things to learn soon, but I am still fairly interested :) | 12:02 |
| nsingh | Getting ERROR! Ansible sub-program not implemented: openstack-ansible. while running playbook. Any clue? | 12:04 |
| odyssey4me | hwoarang it's /etc/ci/mirror_info.sh and then we use its outputs to set the mirrors | 12:04 |
| mgariepy | morning everyone | 12:07 |
| *** markvoelker has quit IRC | 12:08 | |
| Tahvok | Do we want the roles to completely work standalone including infra roles such as rabbitmq? | 12:08 |
| Tahvok | Because I can easily fix the stop/start problem with using a few tasks outside of the role in the play | 12:09 |
| *** vnogin_ has joined #openstack-ansible | 12:09 | |
| Tahvok | But then the role won't be standalone... | 12:09 |
| hwoarang | odyssey4me: yeah there is no /etc/ci/mirror_info.sh anymore | 12:09 |
| *** vnogin has quit IRC | 12:09 | |
| Tahvok | evrardjp: I'm Russian born, so I know 3 languages... My gf though is learning Russian in University, and is doing pretty well with it | 12:10 |
| hwoarang | odyssey4me: look at https://github.com/openstack-infra/project-config/commit/70eb9ea584ef442005481d3b8f403b864d78a903 . The whole thing that was setting these things up have been removed. and if you look at the actual job lobs you see that none of our zuul specific things are executed | 12:10 |
| Tahvok | Just so you know, I didn't realize how Russian is hard, until she started her studies.. I guess it's considered one of the most hard languages for a reason :) | 12:11 |
| odyssey4me | hwoarang yes, but this happened ages ago and we've definitely had it working since - I think that script has moved to somewhere else... it's not gone as far as I know | 12:11 |
| hwoarang | well dont know what to say. the logs say that nothing zuul specific is executed | 12:12 |
| odyssey4me | hwoarang this merged yesterday, and see the output from the script is there: http://logs.openstack.org/39/560839/25/gate/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/7bc78c9/logs/ara-report/result/8e4e082d-2156-4761-8cfe-80076b9e41bf/ | 12:13 |
| errr | odyssey4me: re 1769715 the federation issue I opened yesterday. Im testing your suggestion now and will update that bug with the results. Thanks for the quick response to it | 12:14 |
| evrardjp | maybe we should just wait for pabelanger to show up :) | 12:14 |
| odyssey4me | hwoarang perhaps http://logs.openstack.org/39/560839/25/gate/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/7bc78c9/logs/ara-report/result/ea01236c-8f03-4684-bea4-3c037d74e59b/ is better | 12:14 |
| odyssey4me | errr no worries, apologies for the oversight on that issue - hopefully we can find a solution and get it ported back pronto | 12:14 |
| errr | :D | 12:15 |
| evrardjp | isn't /etc/ci/nirror_info.sh what's used by Triple-O? | 12:15 |
| hwoarang | hmm very strainge | 12:15 |
| mnaser | hi team | 12:15 |
| evrardjp | yes | 12:15 |
| *** markvoelker has joined #openstack-ansible | 12:15 | |
| evrardjp | mirror* | 12:15 |
| evrardjp | hi mnaser | 12:15 |
| hwoarang | evrardjp: i also have a question for you | 12:17 |
| hwoarang | http://logs.openstack.org/19/566819/1/gate/openstack-ansible-functional-opensuse-423/6e3661a/job-output.txt.gz#_2018-05-08_10_16_59_508510 | 12:17 |
| evrardjp | shoot | 12:18 |
| hwoarang | do you see that the output says that it tried to manage the same repo 3 times? i believe it does the right thing on the background but the output is confusing | 12:18 |
| mnaser | hmm infra help? | 12:18 |
| hwoarang | it tries to manage 'repo-oss' but we actually feed it 3 repos in the dictionary | 12:18 |
| mnaser | i can chime in | 12:18 |
| evrardjp | hwoarang: let me check that real quick :) | 12:19 |
| evrardjp | I see | 12:19 |
| evrardjp | it edits it twice | 12:19 |
| evrardjp | which is not really idempotent | 12:19 |
| hwoarang | i only see repo-oss in the output but repo-update and the cloud are missing | 12:20 |
| hwoarang | to me it seems like it's editing 1 repo 3 times instead of 3 repos | 12:20 |
| hwoarang | so maywe we feed it the wrong thing | 12:20 |
| evrardjp | well I can see a problem already | 12:21 |
| evrardjp | yes | 12:21 |
| evrardjp | so we feed | 12:21 |
| evrardjp | https://github.com/openstack/openstack-ansible-openstack_hosts/blob/master/defaults/main.yml#L60 | 12:21 |
| hwoarang | yep | 12:22 |
| evrardjp | this could be fine (I have to check if the urls are fine) | 12:22 |
| evrardjp | https://github.com/openstack/openstack-ansible-openstack_hosts/blob/master/vars/suse-42.yml#L113-L125 | 12:22 |
| *** dave-mccowan has joined #openstack-ansible | 12:22 | |
| evrardjp | but as you can see we + the list | 12:22 |
| evrardjp | lists* | 12:22 |
| evrardjp | we don't union them | 12:22 |
| hwoarang | yep | 12:22 |
| hwoarang | not sure why that would cause problems though | 12:23 |
| evrardjp | well I am checking everything first :) | 12:23 |
| evrardjp | so user_external_repos_list doesn't seem to be populated | 12:23 |
| evrardjp | so that's a good thing, now your issue :) | 12:24 |
| *** dave-mcc_ has joined #openstack-ansible | 12:25 | |
| evrardjp | hwoarang: http://logs.openstack.org/19/566819/1/gate/openstack-ansible-functional-opensuse-423/6e3661a/logs/ara-report/result/e73c8125-6544-47b6-9947-1ba8b96c0b5a/ | 12:26 |
| evrardjp | so you can see with the callback that the data was well sent to ansible, but the callback lies to you | 12:26 |
| evrardjp | the callback used in the log* | 12:26 |
| *** dave-mccowan has quit IRC | 12:26 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-openstack_hosts master: Retry restarting systemd-journald https://review.openstack.org/566866 | 12:26 |
| mnaser | https://review.openstack.org/#/q/I33e84b312e24db07fe9e875ca1546bf9f917cca8 - 3 seconds of your time :-) | 12:27 |
| evrardjp | the only problem is the item label | 12:28 |
| hwoarang | yeah quite possibly | 12:28 |
| hwoarang | i can see the repos are present | 12:28 |
| evrardjp | hwoarang: want me to fix that? | 12:29 |
| hwoarang | well if you have time yes. i am trying to find out why opensuse gets bad mirror right now so i can't look at it right now :/ | 12:30 |
| evrardjp | ok | 12:34 |
| evrardjp | let me push a patch real quick then | 12:34 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-tests master: common-tasks: nodepool: Check if /etc/ci/mirror_info.sh exists on localhost https://review.openstack.org/566869 | 12:38 |
| *** stuartgr1 has quit IRC | 12:42 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: Add loop label https://review.openstack.org/566872 | 12:42 |
| cloudnull | Mornings | 12:44 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_almanach stable/queens: Be clear about python3: Remove its support https://review.openstack.org/566599 | 12:45 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-tests master: test-log-collect.sh: Fix log collection for host https://review.openstack.org/566875 | 12:47 |
| hwoarang | evrardjp: thanks for the fix | 12:48 |
| evrardjp | let's hope it works :) | 12:48 |
| evrardjp | let's see what's in the ARA + the log | 12:48 |
| evrardjp | but it should be clearer, as the _item placeholder wouldn't be used for the default output callback | 12:49 |
| *** openstackgerrit has quit IRC | 12:49 | |
| rschulman | In the example "test" interface file there is a stanza for binding the external VIP that doesn't exist in any of the other examples. I can't seem to find other information about it on the web either. Can someone ELI5 how to get haproxy/keepalived/my interface file working together with my external subnet? | 12:50 |
| rschulman | (Or point me to a good resource, of course!) | 12:50 |
| rschulman | Oh... And "please" should probably go in there somewhere too. :) | 12:50 |
| evrardjp | rschulman: could you show me what you mean ? | 12:50 |
| evrardjp | in which file are you referring to? | 12:50 |
| *** stuartgr has joined #openstack-ansible | 12:51 | |
| evrardjp | for prod you can see it there: https://docs.openstack.org/openstack-ansible/latest/user/prod/example.html#user-variables | 12:51 |
| evrardjp | the test environment has only one infra node, so it doesn't need fancy clustering :) | 12:51 |
| evrardjp | keepalived is used for resiliency in the case you have multiple haproxy nodes | 12:52 |
| evrardjp | it's a quite simple tech :) | 12:52 |
| rschulman | Sure, right here: https://github.com/openstack/openstack-ansible/blob/master/etc/network/interfaces.d/openstack_interface.cfg.test.example#L41 | 12:52 |
| evrardjp | ok | 12:53 |
| evrardjp | so that's a different topic | 12:53 |
| rschulman | Right. I have that, but when haproxy tries to start, it errors out saying it can't bind the given IP. | 12:53 |
| evrardjp | even if it's test, it's probably a good idea to split the traffic from external and internal | 12:53 |
| evrardjp | https://github.com/openstack/openstack-ansible/blob/master/etc/network/interfaces.d/openstack_interface.cfg.test.example#L35 is the internal ip | 12:53 |
| evrardjp | https://github.com/openstack/openstack-ansible/blob/master/etc/network/interfaces.d/openstack_interface.cfg.test.example#L43 is the external ip | 12:53 |
| evrardjp | that's because haproxy doesn't see this ip | 12:54 |
| rschulman | Right. I'm working on the external one. Internal I think I understand. | 12:54 |
| evrardjp | you have a misconfigured interface | 12:54 |
| rschulman | Yes, I think so, but not sure where I'm supposed to configure the external IP in the interface file. | 12:55 |
| evrardjp | haproxy will try to bind on that ip, and tells you he can't bind, because he doesn't have the address:proto:port available | 12:55 |
| evrardjp | you can do exactly like what you see there | 12:55 |
| evrardjp | the :0 is an alias | 12:55 |
| rschulman | OK, and set it to a publicly routable IP address right? | 12:56 |
| jrosser | these examples don't really talk about what to do if you have a real "external" network that isn't the mgmt net | 12:57 |
| jrosser | thats a different matter | 12:57 |
| jrosser | ^ so i guess i'm just checking if that is/isnt the case here | 12:57 |
| rschulman | OK. | 12:58 |
| rschulman | Is there another resource that does go over that? | 12:58 |
| rschulman | I have a real external subnet that I want to use to serve horizon/apis and also give out to instances as floating. | 12:58 |
| jrosser | for each of your nodes that runs haproxy you will need to create an extra interface with an ip on that external network | 12:59 |
| jrosser | and then set up the external vip to be a 3rd ip which keepalived will float between your haproxy instances | 13:01 |
| rschulman | Aha! OK. Now I'm getting it. | 13:03 |
| *** openstackgerrit has joined #openstack-ansible | 13:04 | |
| openstackgerrit | Andy McCrae proposed openstack/ansible-config_template master: Allow default section in ini to be specified https://review.openstack.org/566292 | 13:04 |
| rschulman | If haproxy is in a vm, does that interface need to be on the host or in a tap inside the haproxy vm? | 13:05 |
| *** armaan has quit IRC | 13:05 | |
| *** gkadam_ has joined #openstack-ansible | 13:05 | |
| *** armaan has joined #openstack-ansible | 13:06 | |
| jrosser | rschulman: so long as the external interfaces on your haproxy nodes have L2 adjacency and can contact each other over the external network using the external IP you assign, you are good | 13:07 |
| *** djdule has quit IRC | 13:07 | |
| *** gkadam has quit IRC | 13:07 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_almanach stable/queens: OSA0003: Password should not be logged https://review.openstack.org/566716 | 13:09 |
| *** kstev has joined #openstack-ansible | 13:14 | |
| *** MikeW has quit IRC | 13:14 | |
| dcdamien | hwoarang: please review https://review.openstack.org/#/c/561218/ | 13:15 |
| rschulman | jrosser: Ok, thanks very much. | 13:15 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: Add loop label https://review.openstack.org/566872 | 13:16 |
| *** armaan has quit IRC | 13:23 | |
| *** armaan has joined #openstack-ansible | 13:23 | |
| evrardjp | dcdamien: if you like python I have a few ideas on how to improve this | 13:24 |
| *** klamath has joined #openstack-ansible | 13:28 | |
| *** vnogin_ has quit IRC | 13:28 | |
| dcdamien | evrardjp: not really, I know that this can be solved other way, but my approach was to make it work (currently it doesn't, at all) using smallest amount of time and then calmly consider how to do it better | 13:31 |
| *** klamath has quit IRC | 13:32 | |
| *** jwitko_ has joined #openstack-ansible | 13:33 | |
| *** rmart04 has joined #openstack-ansible | 13:34 | |
| *** jwitko_ is now known as jwitko | 13:35 | |
| evrardjp | dcdamien: fair enough :) | 13:38 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build stable/queens: tasks: repo_clone_git: Retry git-clone on failures https://review.openstack.org/566850 | 13:38 |
| *** jwitko_ has joined #openstack-ansible | 13:38 | |
| *** klamath has joined #openstack-ansible | 13:39 | |
| Tahvok | Is there some default ansible strategy set somewhere? | 13:41 |
| *** jwitko has quit IRC | 13:41 | |
| Tahvok | Or we use the ansible default: linear? | 13:42 |
| evrardjp | we override the default strategy | 13:43 |
| evrardjp | and super the linear one | 13:43 |
| *** hamza21 has joined #openstack-ansible | 13:46 | |
| errr | odyssey4me: so it sort of works.. but when I get redirectred from my okta page back to horizon its not working | 13:46 |
| Tahvok | evrardjp: which means? How it runs? Is it still linear? | 13:46 |
| evrardjp | it behaves almost the same way as linear | 13:47 |
| evrardjp | Tahvok: why do you ask? | 13:47 |
| evrardjp | https://github.com/openstack/openstack-ansible-plugins/blob/master/strategy/linear.py#L24-L27 | 13:47 |
| Tahvok | I've been thinking of the rabbitmq role, and I don't see how we can ensure a good run, with running it in serial (how it's done now) | 13:48 |
| evrardjp | what would you like to do? | 13:49 |
| Tahvok | basically, all rabbitmq servers should be stopped, all of them upgraded, and then all started - which is completely linear strategy, and I don't see how serial can fit in it | 13:49 |
| d34dh0r53 | cloudnull: I've got a lot of problems with you | 13:49 |
| evrardjp | That sounds like it would break things | 13:49 |
| Tahvok | what would break things? | 13:50 |
| evrardjp | wouldn't that cause downtime? | 13:50 |
| *** throwsb1 has joined #openstack-ansible | 13:50 | |
| evrardjp | the idea was to roll the upgrade | 13:50 |
| evrardjp | it seems to have worked in the past just fine? | 13:50 |
| Tahvok | But that's exactly how it's being done now - and already does cause downtime | 13:50 |
| errr | odyssey4me: basically anyhting at https://myhost:5000/Shibboleth.sso/* is a 404 | 13:50 |
| evrardjp | but I might be wrong | 13:50 |
| *** jwitko_ is now known as jwitko | 13:50 | |
| *** vnogin has joined #openstack-ansible | 13:50 | |
| evrardjp | so you're proposing to change it, and still have downtime, how can I know your downtime is better than current downtime? | 13:51 |
| Tahvok | This is actually worse, it causes downtime when it should not cause downtime - when there is no upgrade to be done | 13:51 |
| evrardjp | so that's soemthing we can fix | 13:51 |
| Tahvok | evrardjp: I propose to not cause downtime when upgrade is not needed | 13:51 |
| evrardjp | ok | 13:51 |
| Tahvok | The problem is that the role in current state is unreliable with all the serial things, and still causes downtime | 13:52 |
| evrardjp | so why not checking the fact of the current rabbitmq version at the beginning of the role for all the inventory_hostnames, and enable or not the upgrade flag? | 13:52 |
| evrardjp | We are open to solutions :) | 13:52 |
| Tahvok | Because perhaps the user does not want a downtime, and he's not aware that rabbitmq upgrade is needed? | 13:53 |
| Tahvok | I actually agree with the upgrade flag | 13:53 |
| Tahvok | I disagree with running in serial - as this might cause problem | 13:53 |
| evrardjp | Upgrading in serial seems to bring more solutions than issues to me | 13:54 |
| evrardjp | but I can be convinced otherwise | 13:54 |
| Tahvok | And we still have downtime, so I want to remove the serial flag first, and let the role deal with the upgrade process | 13:54 |
| Tahvok | evrardjp: the rabbitmq role already is causing a downtime - it stops all rabbitmq servers already: https://github.com/openstack/openstack-ansible-rabbitmq_server/blob/master/tasks/rabbitmq_stopped.yml | 13:55 |
| evrardjp | I still don't know what the problem with the serial is | 13:55 |
| Tahvok | See above ^ | 13:55 |
| evrardjp | in a certain order | 13:55 |
| evrardjp | what you're proposing is doing the wild west | 13:56 |
| Tahvok | The issue is race condition - what happens after the first batch completes? Are we stoppin the whole cluster again for the second batch? | 13:56 |
| Tahvok | Because that's exactly what is happening right now | 13:56 |
| evrardjp | Tahvok: https://github.com/openstack/openstack-ansible-rabbitmq_server/blob/d44eb3b39a4262995d8f551cf88fe81424460ab1/tasks/rabbitmq_post_install.yml#L89 | 13:57 |
| evrardjp | that seems a valid reason to restart. | 13:57 |
| evrardjp | Tahvok: what I think you should change is this: | 13:58 |
| evrardjp | https://github.com/openstack/openstack-ansible-rabbitmq_server/blob/d44eb3b39a4262995d8f551cf88fe81424460ab1/tasks/rabbitmq_stopped.yml#L22 | 13:58 |
| evrardjp | and https://github.com/openstack/openstack-ansible-rabbitmq_server/blob/d44eb3b39a4262995d8f551cf88fe81424460ab1/tasks/rabbitmq_stopped.yml#L34 | 13:59 |
| evrardjp | to intersect with the play_batch | 13:59 |
| evrardjp | that's it | 13:59 |
| evrardjp | and not even | 13:59 |
| *** dxiri has quit IRC | 14:00 | |
| Tahvok | evrardjp: another question is, how do I remove this? https://github.com/openstack/openstack-ansible/blob/master/playbooks/rabbitmq-install.yml#L39 | 14:00 |
| *** esberglu has joined #openstack-ansible | 14:00 | |
| evrardjp | because in batch 2, how would that condition trigger: https://github.com/openstack/openstack-ansible-rabbitmq_server/blob/467e3709a3791dabba4b1ffae89857e2d8caa734/tasks/rabbitmq_stopped.yml#L34 ? | 14:01 |
| Tahvok | I want the role to deal with the upgrade process | 14:01 |
| Tahvok | Completely, without play being part of it | 14:01 |
| evrardjp | technically that's not how orchestration should be done, but I understand that, with the current orchestration being in the role, we should use that. | 14:02 |
| evrardjp | I think in this case you can simply remove the code and see if it works or not :) | 14:02 |
| evrardjp | I think I just said it, we are open to change. | 14:03 |
| evrardjp | it just needs to be explained | 14:03 |
| evrardjp | justified | 14:03 |
| Tahvok | evrardjp: that's why I'm asking questions here, to hear your opinion :) | 14:03 |
| evrardjp | haha | 14:04 |
| evrardjp | ok | 14:04 |
| evrardjp | so | 14:04 |
| evrardjp | I think the best for orchestration is playbooks. It's simple. It's clear. You can scope to whatever host you want, and adapt many things you can't do in a task. | 14:05 |
| *** lbragstad has quit IRC | 14:05 | |
| evrardjp | However I think that roles should be a place to group actions | 14:05 |
| evrardjp | vars | 14:05 |
| evrardjp | and convenience things | 14:05 |
| *** lbragstad has joined #openstack-ansible | 14:05 | |
| evrardjp | so I think some tasks files can be re-used as much as we can | 14:05 |
| Tahvok | I completely agree | 14:06 |
| evrardjp | if we manage to remove the orchestration from the role, and include it in plays only, that would simplify things. But for doing that, we need tasks files included in the role, that we can call | 14:06 |
| evrardjp | the problem is happening if something must happen during the role execution | 14:06 |
| *** armaan has quit IRC | 14:07 | |
| *** armaan has joined #openstack-ansible | 14:07 | |
| Tahvok | Are you saying we should call specific tasks with include_role for the play? | 14:07 |
| evrardjp | even if I don't like it, yes that's what I mean | 14:07 |
| evrardjp | tasks_from | 14:08 |
| Tahvok | That sounds like complicating things too much | 14:08 |
| evrardjp | please propose an alternative :) | 14:08 |
| evrardjp | let's talk about upgrades | 14:09 |
| Tahvok | First, what parts of the role are you refering as orchestration in rabbitmq case? | 14:09 |
| evrardjp | it's easy | 14:09 |
| evrardjp | mmm | 14:09 |
| evrardjp | let me check what the role does but basically I guess knowing the cluster state to know when to restart. | 14:09 |
| evrardjp | so killing the nodes [1:] keeping the 0 alive | 14:10 |
| evrardjp | killing 0 to have latest thing | 14:10 |
| evrardjp | upgrading | 14:10 |
| evrardjp | making nodes [1:] join 0 | 14:10 |
| evrardjp | isn't that the goal? | 14:10 |
| evrardjp | even if that's not the goal, that can be adapted | 14:10 |
| evrardjp | but that's orchestration | 14:10 |
| Tahvok | You mean killing [1:] and upgrading | 14:10 |
| evrardjp | implementation details :) | 14:11 |
| Tahvok | This can be also done with multiple plays I think | 14:11 |
| evrardjp | what I mean here is that you have to do things differently on node x or y, and have a different series of things | 14:11 |
| *** hamza21 has quit IRC | 14:11 | |
| evrardjp | so your role is not deploying a fully functioning cluster? | 14:12 |
| Tahvok | Does it really matter which node is first? Isn't rabbit in active-active state? | 14:12 |
| odyssey4me | errr that makes sense I think... did you switch it up to use mod_wsgi, or have you tried to reconfigure it so that it works with uwsgi? | 14:12 |
| Tahvok | When I think about it, it doesn't matter which role is first, as long as at least 1 node is live during the whole upgrade process | 14:12 |
| evrardjp | http://www.rabbitmq.com/upgrade.html#multiple-nodes-upgrade | 14:12 |
| Tahvok | Yeah, that's what I'm reading myself right now | 14:13 |
| evrardjp | I am fine with offering two modes in a playbook: rolling and non-rolling | 14:14 |
| evrardjp | when upgrading | 14:14 |
| evrardjp | as long as its tested | 14:14 |
| Tahvok | Are we expecting incompatible versions upgrade? | 14:15 |
| evrardjp | we try and seewhat happens :) | 14:15 |
| Tahvok | When upgrading from one major or minor version of RabbitMQ to another (i.e. from 3.0.x to 3.1.x, or from 2.x.x to 3.x.x), the whole cluster must be taken down for the upgrade. | 14:16 |
| Tahvok | evrardjp: you mean we don't trust the guide, and check it ourselves before deciding? | 14:17 |
| Tahvok | That's fine, just somewhat unexpected :) | 14:17 |
| evrardjp | nope that's not what I meant | 14:17 |
| evrardjp | our version of erlang/rabbit is frozen, we set it. | 14:18 |
| evrardjp | we'll adapt to whatever is required. | 14:18 |
| *** hamzy has quit IRC | 14:18 | |
| evrardjp | if you think there is an issue here, let's fix it. | 14:18 |
| *** hamzy has joined #openstack-ansible | 14:18 | |
| evrardjp | But right now I am confused, you seem to have seen a bug, but I don't seem to see the code causing it. | 14:18 |
| evrardjp | maybe a bug would be more clear | 14:19 |
| evrardjp | or a patch suggestion | 14:19 |
| Tahvok | evrardjp: we have 2 clusters of openstack, ocata right now, but it doesn't matter as the rabbit role hasn't changed much | 14:21 |
| *** dxiri has joined #openstack-ansible | 14:21 | |
| Tahvok | Each time we run ansible with rabbitmq_upgrade flag, rabbitmq is going down completely, and the openstack service receive a lot of queue errors | 14:22 |
| evrardjp | that makes sense | 14:22 |
| evrardjp | but rabbitmq_upgrade should not be done by default in the plays | 14:22 |
| Tahvok | We have 2 controller machine - a cluster of 2 rabbitmq instances | 14:23 |
| evrardjp | only in upgrades, right? | 14:23 |
| Tahvok | evrardjp: it didn't make sense, as it doesn't say this anywhere in the minor version upgrade guide | 14:23 |
| *** chhavi__ has joined #openstack-ansible | 14:23 | |
| Tahvok | What I don't understand now - if we do a rolling upgrade - why does the cluster go down? | 14:24 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-tests master: common-tasks: nodepool: Set openSUSE OBS mirror for OpenStack CI https://review.openstack.org/566908 | 14:25 |
| *** armaan has quit IRC | 14:25 | |
| evrardjp | That's what we ahve to figure out | 14:25 |
| *** chhavi__ has quit IRC | 14:26 | |
| *** armaan has joined #openstack-ansible | 14:26 | |
| *** chhagarw has quit IRC | 14:26 | |
| *** chhagarw has joined #openstack-ansible | 14:27 | |
| Tahvok | What I think | 14:27 |
| evrardjp | hwoarang: could you check my comment there? https://review.openstack.org/#/c/566872/2 | 14:28 |
| odyssey4me | It may be better to make the role not do an upgrade, even if one's available. Then if the flag is set, do the upgrade. That way you can plan for the down-time. | 14:28 |
| evrardjp | Tahvok: please file a bug, I ahve spent too much time on this | 14:28 |
| Tahvok | Already have | 14:28 |
| evrardjp | cool | 14:28 |
| evrardjp | odyssey4me: isn't that what rabbitmq_upgrade isv? | 14:28 |
| evrardjp | -v | 14:28 |
| evrardjp | we are trying to solve too many issues in one go | 14:28 |
| evrardjp | just break the problem in pieces :) | 14:28 |
| odyssey4me | evrardjp perhaps - although I think it fails the playbook if there is an upgrade available, and you don't set the flag | 14:29 |
| evrardjp | clarify each of those | 14:29 |
| odyssey4me | I don't think we should do that - we should perhaps just warn, but carry on | 14:29 |
| evrardjp | or prompt if a var is not set | 14:29 |
| Tahvok | I'll work on the flag first, and then look more into the rolling upgrade issue | 14:29 |
| evrardjp | so in gates we can do things automatically, but still request user intervention | 14:29 |
| hwoarang | evrardjp: i dont understand the comment | 14:30 |
| hwoarang | is it a reply to my comment? | 14:30 |
| evrardjp | the var must be updated | 14:30 |
| evrardjp | there must be something somewhere tricking that output | 14:30 |
| hwoarang | the item var? | 14:30 |
| hwoarang | yeah so i am not sure if it's just an output problem or the task actually modifies the same repo 3 times | 14:32 |
| Tahvok | odyssey4me: the problem is that the stop process is happening in the play - so we need to check the version in the play as well? | 14:33 |
| Tahvok | https://github.com/openstack/openstack-ansible/blob/master/playbooks/rabbitmq-install.yml#L40 | 14:33 |
| openstackgerrit | Andy McCrae proposed openstack/ansible-config_template master: Allow default section in ini to be specified https://review.openstack.org/566292 | 14:34 |
| *** weezS has joined #openstack-ansible | 14:34 | |
| *** chhagarw has quit IRC | 14:34 | |
| odyssey4me | Tahvok perhaps make it simpler - if you set the flag, then shut down the secondary nodes in the playbook - whether an actual upgrade will happen or not. Then the role doesn't need to care about the whole cluster and we can focus the role on the single host it's running on. | 14:35 |
| evrardjp | hwoarang: let me try real quick | 14:35 |
| odyssey4me | The cluster information can then be given to it from the playbook with group_vars or something like that, making sure that the role doesn't have to reach into hostvars - the playbook gives it the dict to iterate through. | 14:36 |
| evrardjp | rabbitmq_upgrade: True means stopping everyone. Point. | 14:36 |
| evrardjp | :) | 14:36 |
| Tahvok | odyssey4me: and eventually restart it and cause a downtime? | 14:36 |
| evrardjp | if we want we can do later a rabbitmq_rolling_upgrade: True | 14:36 |
| evrardjp | and by default we just run the install? | 14:37 |
| odyssey4me | Tahvok if rabbitmq requires the whole cluster to go down for an upgrade, then you have no other choice. If that situation has changed, then obviously the upgrade process can change. | 14:37 |
| evrardjp | and you put the cluster join in a play that includes tasks from the role | 14:37 |
| evrardjp | so the play can check the clustering status easily | 14:37 |
| evrardjp | I don't know | 14:38 |
| evrardjp | many ideas floating around | 14:38 |
| Tahvok | So if we are causing a downtime - why it matters if it runs in serial or not? If it won't be in serial, I can safely remove 'Stop RabbitMQ nodes that are not the upgrader' play | 14:38 |
| odyssey4me | Fair point. | 14:39 |
| odyssey4me | Although I think serialising helps if a config change makes the service restart. | 14:39 |
| Tahvok | That's the point I've trying to make this whole conversation :D | 14:39 |
| Tahvok | odyssey4me: so when config changes not the whole cluster will go down? | 14:40 |
| odyssey4me | Part of the trouble is that no-one you're talking to has reviewed whether rabbitmq upgrades can be done in a better way which doesn't cause down-time. | 14:40 |
| odyssey4me | Yes, serialising prevents a full cluster outage simultaneously for something simple like a config change. | 14:41 |
| Tahvok | Well, so I think first I should focus on the upgrade process and removing the unneeded stop play - and later think of the flag | 14:41 |
| Tahvok | Because currently I don't see how can I 'simply' fix the flag issue | 14:41 |
| errr | odyssey4me: all I did was remove the line you said to remove in the bug report, then hit this error, then walked away to go have breakfast | 14:43 |
| odyssey4me | errr sounds awesome | 14:43 |
| errr | now Im full and ready to get back to it | 14:43 |
| *** lvdombrkr has quit IRC | 14:43 | |
| *** udesale has quit IRC | 14:53 | |
| osnaya | quit | 14:54 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts master: Retry restarting systemd-journald https://review.openstack.org/566866 | 14:55 |
| *** openstacking_123 has joined #openstack-ansible | 14:59 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts stable/queens: Retry restarting systemd-journald https://review.openstack.org/565894 | 15:00 |
| *** rmart04 has quit IRC | 15:01 | |
| errr | odyssey4me: according this this: https://docs.openstack.org/keystone/pike/advanced-topics/federation/configure_federation.html it seems like I need to not use uwsgi and make sure keystone runs under apache. | 15:02 |
| odyssey4me | errr yeah, I thought that might be needed - at least that's probably an easier thing to figure out for now | 15:03 |
| odyssey4me | it might still be capable of working with uwsgi, but there's no upstream guidance | 15:03 |
| errr | so why did it get removed? | 15:04 |
| errr | like we spent so much time making this work in newton | 15:04 |
| errr | and it seems as though it was removed without any testing to make sure the federation stuff still worked | 15:05 |
| *** DanyC has quit IRC | 15:05 | |
| odyssey4me | errr we standardised all services to work with uwsgi in pike as was the upstream goal | 15:05 |
| errr | :( | 15:05 |
| odyssey4me | this was an oversight, given we have no way of doing federation testing at this time | 15:06 |
| errr | not even using testshib? | 15:06 |
| odyssey4me | federation usage is very niche in this community, and I'd like to take the time to setup a test, but haven't had the time | 15:06 |
| errr | who do we talk to in order to make it a priority? | 15:07 |
| errr | because sales keeps making lots of deals assuming it works :) | 15:07 |
| errr | Im doing like 4 contracts now | 15:07 |
| evrardjp | which sales errr ? | 15:08 |
| evrardjp | :p | 15:08 |
| errr | well not me alone but our team is splitting them | 15:08 |
| evrardjp | that company should invest in making sure it works then | 15:08 |
| errr | evrardjp: Im a racker ;) | 15:08 |
| odyssey4me | errr the community runs on those who have their priorities and provide resources to achieve them - there has been no commitment from any contributing org for federation since kilo | 15:08 |
| odyssey4me | most of my work in this community is done on personal time | 15:08 |
| evrardjp | errr: ask d34dh0r53 then | 15:09 |
| *** vnogin has quit IRC | 15:10 | |
| evrardjp | he is the one that should take care of this :) | 15:10 |
| d34dh0r53 | errr: we should move this conversation over to slack, ping me | 15:11 |
| errr | ok | 15:11 |
| d34dh0r53 | tys | 15:11 |
| *** bertys has quit IRC | 15:12 | |
| evrardjp | Dear cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae,odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT,claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes,mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory,alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum,darrenc, JRobinson__, asettle, | 15:17 |
| evrardjp | colinmcnamara, thorst, adreznec, eil397,qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, Tahvok | 15:17 |
| evrardjp | Here is our agenda for today: https://wiki.openstack.org/wiki/Meetings/openstack-ansible | 15:17 |
| evrardjp | We have meeting in 45 minutes. | 15:17 |
| evrardjp | Please review the bugs or add eventual week highlights. | 15:18 |
| hwoarang | i am going to miss that i have a doc appointment | 15:18 |
| evrardjp | no worries | 15:19 |
| evrardjp | d34dh0r53: you can ofc move this conversation to a private channel, but the community won't be informed about your decisions | 15:20 |
| evrardjp | don't expect the community to accept something not done in the open, as design is one part of the 4 opens. | 15:21 |
| *** bhujay has quit IRC | 15:21 | |
| d34dh0r53 | evrardjp: totally agree, I just suggested a private channel to discuss the prioritization wrt RAX product | 15:21 |
| d34dh0r53 | all work will be done here and upstream | 15:22 |
| evrardjp | cool | 15:22 |
| evrardjp | not forcing you in any way :) | 15:22 |
| mgariepy | errr, if you want federation to work always, i guess the best way is to gate it. | 15:23 |
| evrardjp | yup :) | 15:24 |
| errr | d34dh0r53: I did a whois on your nick and I am not seeing a name matching that in slack. Can you just message me? Michael Rice | 15:24 |
| pabelanger | hwoarang: evrardjp, you can use the /etc/ci/mirror-info.sh script on images, that will have the info | 15:32 |
| hwoarang | pabelanger: yep i noticed i was looking at the wrong thing. Thanks! | 15:33 |
| pabelanger | yah | 15:33 |
| evrardjp | cool | 15:33 |
| evrardjp | that clarifies everything | 15:33 |
| evrardjp | thanks! | 15:33 |
| *** exodusftw has quit IRC | 15:36 | |
| *** gkadam__ has joined #openstack-ansible | 15:43 | |
| *** gkadam_ has quit IRC | 15:46 | |
| evrardjp | meeting in 7 minutes! | 15:53 |
| evrardjp | could we get some votes on https://review.openstack.org/#/c/566538/3 ? | 15:54 |
| evrardjp | this way masakari gets its first commit :) | 15:55 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest stable/pike: Overriding default tempest timeouts https://review.openstack.org/566863 | 16:01 |
| evrardjp | woot | 16:01 |
| evrardjp | it's time! | 16:01 |
| evrardjp | Dear cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae,odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT,claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes,mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory,alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum,darrenc, JRobinson__, asettle, | 16:01 |
| evrardjp | colinmcnamara, thorst, adreznec, eil397,qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, Tahvok | 16:01 |
| evrardjp | Here is our agenda for today: https://wiki.openstack.org/wiki/Meetings/openstack-ansible | 16:01 |
| evrardjp | #startmeeting openstack_ansible_meeting | 16:01 |
| openstack | Meeting started Tue May 8 16:01:33 2018 UTC and is due to finish in 60 minutes. The chair is evrardjp. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:01 |
| *** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)" | 16:01 | |
| openstack | The meeting name has been set to 'openstack_ansible_meeting' | 16:01 |
| evrardjp | #topic last week highlights | 16:02 |
| *** openstack changes topic to "last week highlights (Meeting topic: openstack_ansible_meeting)" | 16:02 | |
| evrardjp | evrardjp has introduced workarounds in ocata branch to still test N to O upgrades | 16:02 |
| evrardjp | evrardjp has started removing support of python3 for Queens and will do all stable branches: https://review.openstack.org/#/q/topic:remove_python3+(status:open+OR+status:merged) | 16:02 |
| evrardjp | evrardjp reminds that most upgrades are broken and need work. | 16:02 |
| evrardjp | hwoarang has been working on the 'distribution packages' spec: https://review.openstack.org/#/q/topic:bp/openstack-distribution-packages . Reviews are highly appreciated! | 16:02 |
| evrardjp | leaving a few minutes for people to process this | 16:02 |
| * prometheanfire wasn't aware we didn't support python3 | 16:03 | |
| evrardjp | prometheanfire: nobody works on it. Let's discuss this after the meeting. It doesn't prevent you from using it, or bringing it in. | 16:04 |
| prometheanfire | ok, I'm sure you've seen the python3 thread on -dev then | 16:04 |
| evrardjp | I'd welcome a gentoo python3 only. | 16:04 |
| evrardjp | Yes I did. | 16:04 |
| *** gyee has joined #openstack-ansible | 16:04 | |
| evrardjp | also master and stable branches are different :) | 16:05 |
| prometheanfire | ya, now that I'm back I can start working on things again :D | 16:05 |
| evrardjp | perfect | 16:05 |
| *** bhujay has joined #openstack-ansible | 16:05 | |
| evrardjp | ok I think we waited enough time for the roll call, let's continue with next topic! | 16:05 |
| evrardjp | #topic bug triage | 16:05 |
| *** openstack changes topic to "bug triage (Meeting topic: openstack_ansible_meeting)" | 16:05 | |
| spotz | o/ | 16:06 |
| evrardjp | we've got a few bugs today, https://etherpad.openstack.org/p/osa-bugtriage, so let's tackle them right away | 16:06 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1769665 | 16:06 |
| openstack | Launchpad bug 1769665 in openstack-ansible "rsyslog_client uses incorrect owner/perms on Centos7" [Undecided,New] | 16:06 |
| evrardjp | cloudnull: does that still applies ? | 16:06 |
| evrardjp | now that we've moved to journal logging? | 16:07 |
| cloudnull | evrardjp it does still apply until rsyslog is gone | 16:07 |
| cloudnull | its now optional, likely something we could fix in the stable branches. | 16:08 |
| odyssey4me | it also still applies until rsyslog is no longer used in all stable branches we have ;) | 16:08 |
| evrardjp | it's just a permissions issue, so it should be simple to fix I guess, but I have the impression we had that dance in the past | 16:08 |
| odyssey4me | so it'll be around for at least another 12-18 months | 16:08 |
| evrardjp | no branch is mentioned in the bug. For me that's master by default :) | 16:08 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia stable/queens: Adds certificate generation https://review.openstack.org/565845 | 16:09 |
| evrardjp | I will ask where this happens, to see if that hasn't been fixed by any chance. | 16:09 |
| evrardjp | Let's mark this as incomplete, shall we? | 16:09 |
| evrardjp | ok next | 16:10 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1769567 | 16:10 |
| openstack | Launchpad bug 1769567 in openstack-ansible "OpenSUSE Ceph client Define libvirt nova secret issue" [Undecided,New] | 16:10 |
| evrardjp | I think we can mark it as confirmed critical, because it breaks gates. | 16:10 |
| evrardjp | ok everyone? | 16:11 |
| evrardjp | 5 | 16:12 |
| *** epalper has quit IRC | 16:12 | |
| evrardjp | 4 | 16:12 |
| evrardjp | 3 | 16:12 |
| evrardjp | 2 | 16:12 |
| evrardjp | 1 | 16:12 |
| evrardjp | ok next | 16:12 |
| *** DanyC has joined #openstack-ansible | 16:12 | |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1769195 | 16:12 |
| openstack | Launchpad bug 1769195 in openstack-ansible "Ocata Series Release Notes in openstack-ansible" [Undecided,New] | 16:12 |
| mnaser | (sorry i'm late) | 16:12 |
| evrardjp | that would be really nice if we could have someone confirming this | 16:13 |
| *** DanyC_ has joined #openstack-ansible | 16:13 | |
| evrardjp | the bug fix has been rushed, and now that we seem to have more time, it would be nice to know if the fix only works in gates. | 16:13 |
| mnaser | i think someone pushed a change about that no? | 16:13 |
| *** cjloader has joined #openstack-ansible | 16:14 | |
| evrardjp | I have a patch that tests the release note, in Ocata. | 16:14 |
| odyssey4me | evrardjp ok, I'll pick that up | 16:14 |
| evrardjp | merging this would be just enough | 16:14 |
| evrardjp | to at least triage the issue | 16:14 |
| evrardjp | https://review.openstack.org/#/c/566587/ | 16:14 |
| evrardjp | ok next | 16:15 |
| evrardjp | (I've assigned odyssey4me but not triaged it, we'll see the evolution next week) | 16:15 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1769100 | 16:15 |
| openstack | Launchpad bug 1769100 in openstack-ansible "Use json_query whenever possible" [Undecided,New] | 16:15 |
| evrardjp | that's definitely a confirmed wishlist | 16:16 |
| mnaser | ++ | 16:16 |
| evrardjp | low-hanging-fruit if I do a first commit | 16:16 |
| evrardjp | ok let's move to next one | 16:16 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1768725 | 16:17 |
| openstack | Launchpad bug 1768725 in openstack-ansible "Ansible hardening takes comments into account for some checks" [Undecided,New] | 16:17 |
| *** DanyC has quit IRC | 16:17 | |
| *** spine55 has quit IRC | 16:17 | |
| evrardjp | oh my god, are the bugs only my bugs? :) | 16:17 |
| mnaser | that should be a low hanging fruit, easily by changing it to grep ^nameserver instead? | 16:17 |
| evrardjp | indeed | 16:17 |
| evrardjp | confirmed low? | 16:18 |
| mnaser | patch coming up from my way too :) | 16:18 |
| odyssey4me | sgtml | 16:18 |
| evrardjp | ok | 16:18 |
| evrardjp | next | 16:18 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1768654 | 16:18 |
| openstack | Launchpad bug 1768654 in openstack-ansible "Dead Link in -- Deviations from the Security Technical Implementation Guide (STIG) in ansible-hardening" [Undecided,New] | 16:18 |
| openstackgerrit | Mohammed Naser proposed openstack/ansible-hardening master: Ensure that comments are not counted https://review.openstack.org/566936 | 16:19 |
| * mnaser looks | 16:19 | |
| evrardjp | confirmed high | 16:19 |
| evrardjp | or medium because the feature doesn't break, just the interface with the users. | 16:19 |
| mnaser | id say its pretty high, if you dont know whats gonna happen by using this role, it's pretty useless | 16:19 |
| evrardjp | that's fir | 16:19 |
| evrardjp | fair | 16:19 |
| odyssey4me | yeah, fair enough | 16:20 |
| evrardjp | confirmed high it is! | 16:20 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1768554 | 16:20 |
| openstack | Launchpad bug 1768554 in openstack-ansible "None images (architecture diagrams) loaded within the page "Storage architecture in openstack-ansible"" [Undecided,New] | 16:20 |
| evrardjp | my bad. | 16:20 |
| evrardjp | that must be me. | 16:20 |
| evrardjp | confirmed high | 16:20 |
| evrardjp | I will fix it. | 16:20 |
| evrardjp | next | 16:21 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1768231 | 16:21 |
| openstack | Launchpad bug 1768231 in openstack-ansible "Cinder filter scheduler not enabled with multiple storage back ends" [Undecided,New] | 16:21 |
| evrardjp | oh we could indeed have some kind of convenience thing there, based on cinder_backends|length | 16:21 |
| mnaser | i ran into this too in queens oddly enough | 16:22 |
| mnaser | restart did fix it and it disappeared never to appear agian | 16:22 |
| evrardjp | we'd need to clean that cinder_backends dict, to remove the limit_container_type | 16:22 |
| evrardjp | mnaser: huh? | 16:22 |
| mnaser | the same issue | 16:23 |
| mnaser | two backends, one working and one down, restart made them both go up and it was okay after that | 16:23 |
| *** dxiri has quit IRC | 16:23 | |
| evrardjp | so you didn't have to change the default section ? | 16:23 |
| evrardjp | adding scheduler_default_filters = DriverFilter | 16:23 |
| mnaser | didn't touch a thing | 16:23 |
| evrardjp | interesting | 16:24 |
| mnaser | and both are up | 16:24 |
| *** dxiri has joined #openstack-ansible | 16:24 | |
| mnaser | according to cinder service-list | 16:24 |
| evrardjp | worth investigating a little more? | 16:24 |
| jrosser | i think the question here is if there is a misconfiguration in cinder, as in that missing config line | 16:24 |
| jrosser | or that something wierd happens on deployment thatis fixed with a restart | 16:24 |
| evrardjp | that is indeed my question | 16:25 |
| * evrardjp is amazed that jrosser starts to translate evrardjp's words like odyssey4me :) | 16:25 | |
| evrardjp | let's mark this as confirmed and medium? It would be nice to add a test scenario for it | 16:26 |
| evrardjp | ok everyone? | 16:26 |
| jrosser | we had two deploys that exhibited that, so fairly confident it's repeatable | 16:27 |
| evrardjp | confirmed then | 16:27 |
| *** radeks_ has quit IRC | 16:28 | |
| mnaser | yeah it happened to me too do its confirmed but not sure root cause | 16:29 |
| evrardjp | I don't think that scheduler_default_filters = DriverFilter is required | 16:29 |
| mnaser | likewise | 16:29 |
| jrosser | mnaser: there could well be hints in the log becasue it moans hugely about the DB being incorrect when cinder is first started | 16:29 |
| evrardjp | what's wrong with the cinder restarts? It seems it's the only service causing us pain :) | 16:30 |
| mnaser | "The driver filter and weigher scheduling can help ensure that the scheduler chooses the best back end based on requested volume properties as well as various back-end specific properties." | 16:30 |
| odyssey4me | there's a fair chance that we're setting things that no longer need setting | 16:30 |
| evrardjp | mnaser: I think we are on the same page :) | 16:30 |
| evrardjp | for triage, do we consider that medium or high? | 16:31 |
| evrardjp | I think medium is fine | 16:31 |
| evrardjp | ok let's continue | 16:33 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1768129 | 16:33 |
| openstack | Launchpad bug 1768129 in openstack-ansible "Default self-signed cert does not include IP SAN" [Undecided,New] | 16:33 |
| evrardjp | that's something we could do indeed. | 16:33 |
| evrardjp | It's not a feature in itself we've promoted, but it would make life simpler for many | 16:34 |
| mnaser | could be aligned with some of the work that cloudnull has been doing | 16:34 |
| mnaser | about self signed cert generation with openssl module | 16:34 |
| evrardjp | we should clarify that we should use fqdn though | 16:34 |
| evrardjp | yes it could | 16:34 |
| evrardjp | confirmed wishlist? | 16:35 |
| evrardjp | I don't think we can consider this as a bug, as we never promoted the use of self-signed certificates with IPs. | 16:35 |
| evrardjp | I'd never do that myself :p | 16:35 |
| evrardjp | ok | 16:35 |
| evrardjp | next | 16:36 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1758144 | 16:36 |
| openstack | Launchpad bug 1758144 in openstack-ansible "resolv.conf in containers set too late" [Undecided,New] | 16:36 |
| evrardjp | anyone wants to take this? | 16:36 |
| evrardjp | 5 | 16:36 |
| evrardjp | 4 | 16:36 |
| evrardjp | 3 | 16:36 |
| evrardjp | 2 | 16:36 |
| evrardjp | 1 | 16:36 |
| evrardjp | ok next | 16:36 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1755821 | 16:36 |
| openstack | Launchpad bug 1755821 in openstack-ansible "config_template fails to parse template if it contains a comment with leading spaces" [High,New] | 16:36 |
| * mnaser hit this yesterday upgrading to queens with neutron.conf.j2 :( | 16:37 | |
| evrardjp | same question: Anyone interested by confirming it/working on it? | 16:37 |
| mnaser | i can confirm it 100% | 16:37 |
| mnaser | but dont know if i understand teh whole config_template plugin structure enough to fix it | 16:37 |
| evrardjp | yes but you are also the bug submitter :) | 16:37 |
| evrardjp | but that's fine, I trust you! | 16:38 |
| mnaser | aha :) | 16:38 |
| evrardjp | mmm I can help you the config_template part | 16:38 |
| mnaser | set neutron_lbaasv2: true and watch it crash and burn | 16:38 |
| evrardjp | it's not that hard, it's a tempate copy | 16:38 |
| evrardjp | mnaser: could you have this as a scenario then? | 16:38 |
| mnaser | the issue occurs when the python ini parser gets an ini file that starts with someting that is not a section | 16:38 |
| evrardjp | isn't that octavia scenario btw? | 16:39 |
| mnaser | because the iini file starts with ' # General, applies to all host groups' rather than '# General, applies to all host groups' | 16:39 |
| mnaser | if we literally dropped '# General, applies to all host groups' it'll be fine | 16:39 |
| evrardjp | haha | 16:39 |
| mnaser | the thing is like the extra spaces mess up the python ini parser | 16:40 |
| evrardjp | mnaser: so not failing on the ansible managed part? | 16:40 |
| mnaser | so the exception doesn't even happen in config_template world | 16:40 |
| mnaser | if you try to parse an ini file with leading spaces and a comment it'll crash too (outside of ansible etc) | 16:40 |
| evrardjp | mnaser: should we remove the whitespaces there: http://git.openstack.org/cgit/openstack/openstack-ansible-os_neutron/tree/templates/neutron.conf.j2#n2 ? | 16:40 |
| evrardjp | adding - everywhere? | 16:40 |
| mnaser | evrardjp: i think that is the way to go, but the - trickery confuses me a lot so :P | 16:41 |
| mnaser | thats beyond me :P | 16:41 |
| mnaser | but indeed that would solve it | 16:41 |
| evrardjp | I think it's not fair to assume that config_template should scramble the files | 16:41 |
| *** gkadam__ has quit IRC | 16:42 | |
| mnaser | yeah sanitize it somehow | 16:42 |
| evrardjp | I am curious and I will try to work on it | 16:45 |
| evrardjp | ok next | 16:45 |
| evrardjp | #link https://bugs.launchpad.net/openstack-ansible/+bug/1743032 | 16:45 |
| openstack | Launchpad bug 1743032 in openstack-ansible "Galera cluster maintenance in OpenStack-Ansible" [Undecided,New] | 16:45 |
| evrardjp | don't rush on the bugs guys :) | 16:48 |
| evrardjp | or ladies :) | 16:48 |
| evrardjp | let's say folks, for the ease of language! | 16:48 |
| odyssey4me | :p | 16:48 |
| *** EmilienM is now known as EmilienM_PTO | 16:49 | |
| evrardjp | ok let's wrap up | 16:49 |
| evrardjp | #topic open discussion | 16:50 |
| *** openstack changes topic to "open discussion (Meeting topic: openstack_ansible_meeting)" | 16:50 | |
| evrardjp | we have 10 minutes for open discussion if someone wants to have a quick chat about a painful issue or anything else. | 16:50 |
| evrardjp | It's good weather outside here, so I am not particularily willing to stay more behind a keyboard | 16:50 |
| evrardjp | :) | 16:50 |
| prometheanfire | :D | 16:50 |
| evrardjp | ok I am done for the day | 16:52 |
| evrardjp | thanks everyone! | 16:53 |
| evrardjp | #endmeeting | 16:53 |
| *** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: https://goo.gl/tTmdgs" | 16:53 | |
| openstack | Meeting ended Tue May 8 16:53:14 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:53 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-05-08-16.01.html | 16:53 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-05-08-16.01.txt | 16:53 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-05-08-16.01.log.html | 16:53 |
| evrardjp | now that the meeting is over | 16:53 |
| evrardjp | we have the random selection of next chair | 16:53 |
| evrardjp | the first one | 16:53 |
| evrardjp | please watch the code here, that I will quickly write, just a sec or two | 16:54 |
| odyssey4me | a suggestion: ask for a volunteer within the meeting so that it's captured in the minutes | 16:55 |
| evrardjp | https://gist.github.com/evrardjp/cc8cb51a38e72204041b5cea3411ec5f | 16:55 |
| odyssey4me | if there's no volunteer, then do the random selection in the minutes too | 16:55 |
| evrardjp | yeah let's record this | 16:55 |
| evrardjp | let's reopen the meeting for it | 16:55 |
| evrardjp | #startmeeting openstack_ansible_meeting | 16:57 |
| openstack | Meeting started Tue May 8 16:57:07 2018 UTC and is due to finish in 60 minutes. The chair is evrardjp. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:57 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:57 |
| *** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)" | 16:57 | |
| openstack | The meeting name has been set to 'openstack_ansible_meeting' | 16:57 |
| evrardjp | #topic chair selection | 16:57 |
| *** openstack changes topic to "chair selection (Meeting topic: openstack_ansible_meeting)" | 16:57 | |
| evrardjp | anyone wants to chair next meeting? | 16:57 |
| osnaya | quit | 16:58 |
| *** osnaya has quit IRC | 16:58 | |
| *** openstacking_123 has quit IRC | 16:58 | |
| evrardjp | that seems we are calling random name generator! | 16:58 |
| evrardjp | 5 | 16:59 |
| *** osnaya has joined #openstack-ansible | 16:59 | |
| evrardjp | 4 | 16:59 |
| evrardjp | 3 | 16:59 |
| evrardjp | 2 | 16:59 |
| evrardjp | 1 | 16:59 |
| evrardjp | ./osa-next-meeting-chair.py | 16:59 |
| evrardjp | Our next meeting chair will be asettle | 16:59 |
| evrardjp | that's it for today! | 16:59 |
| evrardjp | thanks everyone | 16:59 |
| evrardjp | #endmeeting | 16:59 |
| *** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: https://goo.gl/tTmdgs" | 16:59 | |
| openstack | Meeting ended Tue May 8 16:59:50 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:59 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-05-08-16.57.html | 16:59 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-05-08-16.57.txt | 16:59 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-05-08-16.57.log.html | 16:59 |
| evrardjp | asettle: please be ready to chair the meeting next week. If you can't do it, please tell me. | 17:07 |
| *** mhayden has quit IRC | 17:09 | |
| *** mhayden has joined #openstack-ansible | 17:14 | |
| *** mhayden has joined #openstack-ansible | 17:15 | |
| *** gyee has quit IRC | 17:17 | |
| *** openstacking_123 has joined #openstack-ansible | 17:20 | |
| *** hamzy has quit IRC | 17:28 | |
| *** bhujay has quit IRC | 17:39 | |
| *** openstacking_123 has quit IRC | 17:41 | |
| *** openstacking_123 has joined #openstack-ansible | 17:56 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-tests master: Add bionic jobs https://review.openstack.org/566959 | 18:03 |
| *** mhayden has quit IRC | 18:09 | |
| *** openstacking_123 has quit IRC | 18:10 | |
| *** radeks_ has joined #openstack-ansible | 18:11 | |
| *** mhayden has joined #openstack-ansible | 18:14 | |
| *** gyee has joined #openstack-ansible | 18:19 | |
| *** hamzy has joined #openstack-ansible | 18:20 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest stable/queens: Overriding default tempest timeouts https://review.openstack.org/566862 | 18:24 |
| *** geb has quit IRC | 18:28 | |
| *** hwoarang has quit IRC | 18:28 | |
| *** hwoarang has joined #openstack-ansible | 18:29 | |
| *** geb has joined #openstack-ansible | 18:31 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/newton: Update release note for resolvconf cache prep failure https://review.openstack.org/566968 | 18:34 |
| odyssey4me | shananigans ^ thanks for the testing and feedback in the bug - that patch updates the release note | 18:35 |
| odyssey4me | I'm verifying the fix against ocata, pike and queens now | 18:35 |
| mnaser | evrardjp: maybe good to send an email letting asettle know that they'll be chairing next week :) | 18:39 |
| *** mbuil has quit IRC | 18:41 | |
| evrardjp[m] | she is aware :) | 18:41 |
| evrardjp[m] | but i have to remove her from the pool, sadly :( | 18:41 |
| mnaser | evrardjp[m]: oh how so? :x | 18:42 |
| osnaya | @cloudnull @odyssey4me setup-hosts.yml run giving one error @http://paste.openstack.org/show/720609/ .... Log and debug info from target host is provided. Can you please check and provide any pointers? | 18:44 |
| *** openstackgerrit has quit IRC | 18:49 | |
| evrardjp[m] | mnaser I guess you understand... people stepping down... | 18:50 |
| evrardjp[m] | that happens. | 18:50 |
| mnaser | ah | 18:50 |
| mnaser | ack | 18:50 |
| mnaser | rerun the RNG again? lol | 18:50 |
| evrardjp[m] | haha | 18:53 |
| odyssey4me | cores, we have operator validation for the update in https://review.openstack.org/566968 - can we get some reviews please? | 18:53 |
| evrardjp[m] | no I got a volunteer apparently | 18:53 |
| odyssey4me | I'm busy doing validation for ocata, pike and queens and will cherry-pick that patch upwards as I confirm. | 18:53 |
| *** cmart has quit IRC | 18:57 | |
| *** dxiri has quit IRC | 19:06 | |
| *** dxiri has joined #openstack-ansible | 19:07 | |
| *** dave-mcc_ has quit IRC | 19:14 | |
| *** djdule has joined #openstack-ansible | 19:14 | |
| djdule | Hi, We installed Openstack cloud, Pike version using Openstack-ansible and it works fine. It has 1 controller, 4 computing nodes, OS is Ubuntu 16.04. We now want to add 2 Ceph nodes, with monitoring running on controller node. We added new configuration and after executing: openstack-ansible setup-infrastructure.yml I do have error message: fatal: [infra1_ceph-mon_container-7722acfe]: FAILED! => {"changed": false, "fail | 19:15 |
| djdule | Any idea where to start looking for problem ? Thanks ! | 19:15 |
| odyssey4me | djdule did you execute setuphosts to create the containers? | 19:16 |
| djdule | Not exactly, no. But openstack was running without any problems before, so I would assume everything is OK | 19:17 |
| djdule | what is setuphosts exactly, it is not mentioned in documentation ? | 19:18 |
| djdule | you mean openstack-ansible setup-hosts.yml? | 19:19 |
| *** dave-mccowan has joined #openstack-ansible | 19:19 | |
| odyssey4me | djdule yes, sorry - typo | 19:23 |
| odyssey4me | setup-hosts preps the hosts and includes the lxc-containers-create playbook | 19:23 |
| odyssey4me | if you've added new host groups, there are new containers... so they need to be created | 19:23 |
| djdule | as far as I remember yes, but I am re-running it now in case I missed it | 19:24 |
| djdule | let me finish with it and I will report back what is the situation | 19:24 |
| evrardjp | for bionic, I will be slowly adding it, role per role, when I have a few minutes, and doing it python3 only. So you all know | 19:24 |
| djdule | thanks ! | 19:24 |
| evrardjp | if it doesn't work, I will be fallbacking into python2 for the role itself | 19:24 |
| odyssey4me | evrardjp sounds interesting - doing py3 before the python builds are simplified is brave | 19:25 |
| evrardjp | odyssey4me: it would be nice if you could vote on https://review.openstack.org/#/c/566959/1 | 19:25 |
| evrardjp | odyssey4me: or foolish | 19:25 |
| evrardjp | :) | 19:25 |
| evrardjp | but you know me | 19:25 |
| evrardjp | nobody is working on it, and I'd like to learn what's wrong | 19:25 |
| evrardjp | best way is to do it | 19:26 |
| evrardjp | not think about it very hard :) | 19:26 |
| odyssey4me | evrardjp I'd suggest going py2 straight up, because we could consider porting the work back to Queens given that Ubuntu is officially supporting that as an option. | 19:26 |
| odyssey4me | it would probably save us a lot of headache if that could be done. | 19:26 |
| odyssey4me | also, doing py2 means you're only trying to solve one problem - not 2 | 19:27 |
| djdule | @odyssey4me it seems it was the problem. setup-hosts changed lot of things, now setup-infrastructure is running, but I do not see any errors, and it used to be somewhere near the beginning | 19:28 |
| odyssey4me | djdule glad that worked out :) | 19:28 |
| djdule | @odyssey4me don't say it twice, let it finish first :) but it looks promissing | 19:29 |
| evrardjp | odyssey4me: I didn't plan to backport it. | 19:30 |
| *** mwe1 has quit IRC | 19:30 | |
| evrardjp | I think we discussed this | 19:30 |
| evrardjp | but it seems reasonable to do it | 19:30 |
| evrardjp | in that case, python2 is the only option. | 19:30 |
| odyssey4me | yeah, I don't like it - but we got fresh news today which provides a compelling argument to do so | 19:30 |
| evrardjp | it's not fresh, it was discussed at the PTG :) | 19:30 |
| evrardjp | at least jamespage told us about it :) | 19:31 |
| odyssey4me | that xenial and bionic would both support queens officially? | 19:31 |
| odyssey4me | if that was mentioned, I completely missed it | 19:31 |
| evrardjp | yes | 19:31 |
| evrardjp | that's alright, we are a team :) | 19:31 |
| odyssey4me | silly me, I would probably have jumped on testing it sooner and getting patches up earlier | 19:32 |
| evrardjp | haha that's fine you were busy | 19:32 |
| evrardjp | :) | 19:32 |
| *** openstacking_123 has joined #openstack-ansible | 19:33 | |
| *** djdule has quit IRC | 19:33 | |
| *** openstackgerrit has joined #openstack-ansible | 19:43 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/ansible-config_template master: Add Bionic testing https://review.openstack.org/567014 | 19:43 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/ansible-role-systemd_mount master: Add Bionic testing https://review.openstack.org/567015 | 19:46 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/ansible-role-systemd_networkd master: Add Bionic testing https://review.openstack.org/567016 | 19:48 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/ansible-role-systemd_service master: Add Bionic testing https://review.openstack.org/567017 | 19:49 |
| evrardjp | I let you on the python_venv_build :) | 19:49 |
| odyssey4me | evrardjp yeah, I can do that - thanks | 19:50 |
| evrardjp | it seems there is no vars file there, so it simplifies also the first roles | 19:50 |
| odyssey4me | evrardjp well, it's not platform specific IIRC... but the tests might need tweaking | 19:51 |
| evrardjp | we just need the same topic | 19:51 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-apt_package_pinning master: Add Bionic testing https://review.openstack.org/567018 | 19:51 |
| odyssey4me | the tests in my patch | 19:51 |
| evrardjp | we'll see one by one | 19:51 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-ceph_client master: Add Bionic testing https://review.openstack.org/567024 | 19:58 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_client master: Add Bionic testing https://review.openstack.org/567029 | 20:03 |
| *** radeks__ has joined #openstack-ansible | 20:04 | |
| *** radeks_ has quit IRC | 20:05 | |
| *** radeks has quit IRC | 20:06 | |
| *** esberglu has quit IRC | 20:07 | |
| *** mma has quit IRC | 20:07 | |
| *** tosky has quit IRC | 20:08 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_server master: Add Bionic testing https://review.openstack.org/567030 | 20:08 |
| *** tosky has joined #openstack-ansible | 20:08 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-haproxy_server master: Add Bionic testing https://review.openstack.org/567031 | 20:10 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-lxc_container_create master: Add Bionic testing https://review.openstack.org/567032 | 20:13 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Add bionic jobs https://review.openstack.org/566959 | 20:15 |
| *** cmart has joined #openstack-ansible | 20:15 | |
| *** esberglu has joined #openstack-ansible | 20:19 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-lxc_hosts master: Add Bionic testing https://review.openstack.org/567034 | 20:20 |
| *** pcaruana has quit IRC | 20:21 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/ansible-config_template master: Add Bionic testing https://review.openstack.org/567014 | 20:21 |
| evrardjp | odyssey4me: in case you ask, I planned to add bionic in the meta: when it passes test :) | 20:24 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-memcached_server master: Add Bionic testing https://review.openstack.org/567035 | 20:25 |
| odyssey4me | evrardjp better to add it in your patch, before people approve it! | 20:25 |
| evrardjp | no I mean when I will put it voting | 20:25 |
| evrardjp | when it's official we support it | 20:26 |
| odyssey4me | oh I see - well, I guess that makes sense | 20:26 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_blazar master: Updated from OpenStack Ansible Tests https://review.openstack.org/567036 | 20:26 |
| evrardjp | else I am afraid that people will see and say "it's supported!" | 20:26 |
| evrardjp | also this was used for the role maturity generator, so I'd rather wait a little | 20:26 |
| odyssey4me | fair enough | 20:27 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: Add Bionic testing https://review.openstack.org/567038 | 20:29 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: Properly use "Rocky" in release filename https://review.openstack.org/567040 | 20:32 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_openrc master: Add Bionic testing https://review.openstack.org/567041 | 20:33 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-nspawn_hosts master: Add Bionic testing https://review.openstack.org/567042 | 20:36 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-nspawn_container_create master: Add Bionic testing https://review.openstack.org/567043 | 20:37 |
| evrardjp | I will stop at all the infra bits | 20:39 |
| odyssey4me | heh, those are the easy ones :p | 20:40 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-plugins master: Add Bionic testing https://review.openstack.org/567044 | 20:40 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-pip_install master: Add Bionic testing https://review.openstack.org/567045 | 20:42 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-rabbitmq_server master: Add Bionic testing https://review.openstack.org/567046 | 20:43 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-rsyslog_client master: Add Bionic testing https://review.openstack.org/567047 | 20:45 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-rsyslog_server master: Add Bionic testing https://review.openstack.org/567048 | 20:45 |
| *** openstacking_123 has quit IRC | 20:50 | |
| evrardjp | odyssey4me: I don't think so :p | 20:50 |
| evrardjp | well | 20:50 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Add Bionic testing https://review.openstack.org/567049 | 20:50 |
| evrardjp | when the networking bits will be settled | 20:50 |
| jrosser | on a fresh deploy, /root/.pip/pip.conf seems to be set up to have find-links pointing to the as yet non existant repo server | 20:50 |
| jrosser | is that normal? because it incurs a gigantic timeout | 20:51 |
| evrardjp | nope I don't think so | 20:51 |
| odyssey4me | yes, that's normal - and I have patches trying to change that which haven't managed to merge yet | 20:51 |
| evrardjp | oh | 20:51 |
| evrardjp | I thought they merged | 20:51 |
| evrardjp | my bad | 20:51 |
| odyssey4me | in master, yes - not in queens yet | 20:51 |
| odyssey4me | actually not even master yet: https://review.openstack.org/549012 | 20:52 |
| evrardjp | OMG sorry | 20:53 |
| evrardjp | I have rechecked now | 20:53 |
| evrardjp | I have starred it but not followed :( | 20:53 |
| jrosser | ah ok | 20:53 |
| odyssey4me | don't bother rechecking that one until https://review.openstack.org/#/c/563114/6 merges | 20:53 |
| *** radeks__ has quit IRC | 20:53 | |
| evrardjp | oh waiting for ceph? | 20:54 |
| evrardjp | THat's never gonna pass | 20:54 |
| odyssey4me | no, every run failed one of the voting gates - different one each time | 20:54 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/ocata: Update release note for resolvconf cache prep failure https://review.openstack.org/567050 | 20:55 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Update release note for resolvconf cache prep failure https://review.openstack.org/567051 | 20:55 |
| evrardjp | yes I was mistaken it's ceph opensuse that fails | 20:55 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Update release note for resolvconf cache prep failure https://review.openstack.org/567052 | 20:55 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-lxc_container_create master: Add Bionic testing https://review.openstack.org/567032 | 20:56 |
| *** cmart has quit IRC | 20:59 | |
| * jrosser sort of given up rechecking my reviews - bit of an outbreak of CI antipattern :( | 20:59 | |
| throwsb1 | Hey, anyone around that can help point me to resolve dhcp not giving out IP's on tenent network? This is a new deploy with 3 infra nodes running queens. | 21:00 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Add Bionic testing https://review.openstack.org/567049 | 21:01 |
| throwsb1 | I can see request coming through using tcpdump on physical interface across infra and compute, but on vxlan-72, I only see the same request on the computes and not infra | 21:02 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-plugins master: Add Bionic testing https://review.openstack.org/567044 | 21:03 |
| *** DanyC_ has quit IRC | 21:04 | |
| *** kstev has quit IRC | 21:15 | |
| *** ianychoi has quit IRC | 21:15 | |
| cloudnull | evrardjp odyssey4me: do we have a bulk patch prep'd for copying the ubuntu-16.04.yml files to 18.04 ? | 21:28 |
| *** jwitko_ has joined #openstack-ansible | 21:30 | |
| *** jwitko has quit IRC | 21:32 | |
| evrardjp | cloudnull: what do you mean? | 21:33 |
| evrardjp | if you are thinking about vars/ files, I am not doing that | 21:33 |
| evrardjp | I try to rename the vars files to ubuntu if it's common | 21:33 |
| *** jwitko__ has joined #openstack-ansible | 21:35 | |
| *** jwitko__ has quit IRC | 21:36 | |
| *** kstev has joined #openstack-ansible | 21:37 | |
| cloudnull | yes | 21:38 |
| cloudnull | https://github.com/openstack/openstack-ansible-os_nova/blob/master/vars/ubuntu-16.04.yml to https://github.com/openstack/openstack-ansible-os_nova/blob/master/vars/ubuntu-18.04.yml | 21:38 |
| cloudnull | as a first step we should see what we can get to pass with the same setup | 21:39 |
| *** jwitko_ has quit IRC | 21:39 | |
| cloudnull | otherwise we're going through every role one at a time | 21:39 |
| *** throwsb1 has quit IRC | 21:40 | |
| odyssey4me | the 16.04 -> 18.04 changes are likely fairly minimal, so yeah - switching to just ubuntu.yml where possible would be nice | 21:41 |
| *** markvoelker_ has joined #openstack-ansible | 21:41 | |
| *** markvoelker has quit IRC | 21:42 | |
| cloudnull | seems like the easiest change to make would be to copy the file and prune it later. | 21:45 |
| cloudnull | however I'll leave that plan of attack up for debate. I don't really know. | 21:46 |
| DimGR | hi | 21:52 |
| cloudnull | o/ DimGR | 21:57 |
| DimGR | hi ho | 21:57 |
| *** cjloader has quit IRC | 22:04 | |
| *** cjloader_ has joined #openstack-ansible | 22:04 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_almanach stable/queens: Updated from OpenStack Ansible Tests https://review.openstack.org/566621 | 22:07 |
| *** cjloader_ has quit IRC | 22:09 | |
| *** jwitko has joined #openstack-ansible | 22:09 | |
| *** vnogin has joined #openstack-ansible | 22:11 | |
| *** esberglu has quit IRC | 22:12 | |
| *** vnogin has quit IRC | 22:15 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_tempest master: Prefer distribution packages for Tempest dependencies https://review.openstack.org/562606 | 22:23 |
| *** kysse has quit IRC | 22:50 | |
| *** sawblade6 has quit IRC | 22:52 | |
| *** kstev has quit IRC | 23:02 | |
| *** pmannidi has joined #openstack-ansible | 23:03 | |
| *** cjloader has joined #openstack-ansible | 23:23 | |
| *** cjloader has quit IRC | 23:27 | |
| *** tosky has quit IRC | 23:27 | |
| *** weezS has quit IRC | 23:43 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!