| *** mmercer has joined #openstack-ansible | 00:21 | |
| *** tux_ has joined #openstack-ansible | 00:29 | |
| *** vollman has quit IRC | 00:37 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-tests master: Add missing services and collect journal logs https://review.openstack.org/587259 | 00:46 |
|---|---|---|
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-tests master: Add missing services and collect journal logs https://review.openstack.org/587259 | 00:50 |
| cloudnull | bgmccollum nice! | 00:50 |
| bgmccollum | cloudnull seen this before? -- http://paste.openstack.org/show/726890/ | 00:54 |
| cloudnull | normally that happens when the mirror is not listed in the pip.conf as a trusted host | 00:56 |
| cloudnull | i've not seen that in the gate for some time? | 00:57 |
| bgmccollum | new mirror? | 00:57 |
| cloudnull | could be . | 00:57 |
| bgmccollum | is there a gate config that needs adjusting? | 00:58 |
| cloudnull | I'm not sure. where are you seeing that ? | 01:00 |
| cloudnull | is it in the gate jobs? | 01:00 |
| cloudnull | if you have access to the system you can change it in ~/.pip/pip.conf | 01:00 |
| bgmccollum | Gate -- http://logs.openstack.org/29/587229/1/check/openstack-ansible-deploy-aio_metal-ubuntu-xenial/621a4e2/ara-report/ | 01:01 |
| bgmccollum | http://logs.openstack.org/29/587229/1/check/openstack-ansible-deploy-aio_metal-ubuntu-xenial/621a4e2/job-output.txt.gz#_2018-07-30_23_27_11_691007 | 01:05 |
| cloudnull | yea that's an odd one. that should get picked up by the pip_install role and added into the local config, but maybe that's just not happening? | 01:07 |
| cloudnull | odyssey4me and evrardjp might know better? | 01:07 |
| *** gregoryo has joined #openstack-ansible | 01:16 | |
| *** gyee has quit IRC | 01:40 | |
| *** threestrands has quit IRC | 02:10 | |
| *** tux_ has quit IRC | 02:11 | |
| *** tux_ has joined #openstack-ansible | 02:12 | |
| *** mmercer has quit IRC | 02:17 | |
| *** Bhujay has joined #openstack-ansible | 02:22 | |
| tux_ | cloudnull: ^^ | 02:34 |
| tux_ | need hint http://paste.openstack.org/show/726887/ | 02:34 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cinder master: Convert role to use a common systemd service role https://review.openstack.org/574817 | 02:56 |
| *** Bhujay has quit IRC | 02:57 | |
| *** coreywright has quit IRC | 02:57 | |
| *** prasen has joined #openstack-ansible | 02:59 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-tests master: Add missing services and collect journal logs https://review.openstack.org/587259 | 03:00 |
| cloudnull | anyone around this evening want to give https://review.openstack.org/#/c/587093 a push through ? | 03:00 |
| cloudnull | tux_ I'd suspect something is not right in the /usr/local/bin/openstack-ansible.rc file which is setup when the the script `bootstrap-ansible.sh` is run | 03:02 |
| cloudnull | My suspicion is from the output "/var/opt/openstack-ansible/playbooks/OSA_INVENTORY_PATH/dynamic_inventory.py" | 03:02 |
| tux_ | hmmm | 03:02 |
| cloudnull | OSA_INVENTORY_PATH should be replaced with the playbooks path | 03:02 |
| tux_ | I made this changes to fix my lvm issue https://github.com/openstack/openstack-ansible/commit/8c8e4beda733648946140a5a14d23a346d3b5292#diff-0f8043cab00abbe1ccddb1444b0a3043 | 03:03 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_designate master: Convert role to use a common systemd service role https://review.openstack.org/587196 | 03:05 |
| *** prasen has quit IRC | 03:05 | |
| tux_ | cloudnull: its the same file, not sure why OSA_INVENTORY_PATH not getting set https://github.com/openstack/openstack-ansible/blob/8c8e4beda733648946140a5a14d23a346d3b5292/scripts/openstack-ansible.rc | 03:08 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cloudkitty master: Convert role to use a common systemd service role https://review.openstack.org/587187 | 03:09 |
| cloudnull | tux_ after you pulled in that change into openstack-ansible did you rerun the bootstrap script ? | 03:10 |
| tux_ | yes i did but let me give it a try again and see | 03:11 |
| cloudnull | maybe you had an environment variable set ? | 03:11 |
| *** exodusftw has quit IRC | 03:11 | |
| cloudnull | that it read it? | 03:11 |
| tux_ | i will try to clear set -o and will see | 03:12 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_barbican master: Convert role to use a common systemd service role https://review.openstack.org/587101 | 03:13 |
| *** exodusftw has joined #openstack-ansible | 03:15 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/ansible-hardening master: Add option to skip sudoers NOPASSWD check https://review.openstack.org/565959 | 03:18 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/ansible-hardening master: Add option to skip sudoers NOPASSWD check https://review.openstack.org/565959 | 03:18 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cinder master: Convert role to use a common systemd service role https://review.openstack.org/574817 | 03:24 |
| *** threestrands has joined #openstack-ansible | 03:26 | |
| tux_ | cloudnull: one more question, I have upgraded PIKE -> QUEENS and Queens using ceph-ansible module instead of individual roles from previous version | 03:26 |
| cloudnull | nice! | 03:27 |
| tux_ | http://paste.openstack.org/show/726898/ | 03:27 |
| tux_ | If you look at this dir.. shouldn't i delete those ceph-mon, ceph-osd etc.. ? | 03:28 |
| tux_ | queens using ceph-ansible roles not ceph-mon & ceph-osa etc | 03:28 |
| tux_ | cloudnull: you are right here "My suspicion is from the output "/var/opt/openstack-ansible/playbooks/OSA_INVENTORY_PATH/dynamic_inventory.py"" | 03:29 |
| tux_ | I set OSA_INVENTORY_PATH manually and it works | 03:29 |
| cloudnull | the script should set that assuming there's no variable in your bash environment that is overriding it | 03:32 |
| cloudnull | as for the roles, if it's a clean installation, using the vanilla roles, i'd probably delete the ones you don't need | 03:33 |
| tux_ | cloudnull: thanks for super duper hint :) | 03:35 |
| cloudnull | ++ glad i could help :) | 03:35 |
| *** indistylo has joined #openstack-ansible | 03:38 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-tests master: Add missing services and collect journal logs https://review.openstack.org/587259 | 03:40 |
| tux_ | cloudnull: now got this error http://paste.openstack.org/show/726900/ | 03:41 |
| tux_ | but this is something else not broken playbook | 03:42 |
| cloudnull | seems like whatever is feeding that task needs "osd_pool_default_pg_num" defined | 03:43 |
| cloudnull | I'm not sure where that is though | 03:43 |
| tux_ | i will dig and see.. | 03:44 |
| tux_ | cloudnull: thanks man!! | 03:45 |
| cloudnull | anytime | 03:45 |
| tux_ | I think this is the solution https://github.com/ceph/ceph-ansible/issues/2453 | 03:48 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_designate master: Convert role to use a common systemd service role https://review.openstack.org/587196 | 03:48 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cloudkitty master: Convert role to use a common systemd service role https://review.openstack.org/587187 | 03:49 |
| tux_ | cloudnull: Now i realized why it didn't set "OSA_INVENTORY_PATH" variable correct, Because i was running bootstrap script on terminal-A and running playbook on terminal-B :( | 03:52 |
| *** poopcat has quit IRC | 03:56 | |
| *** gkadam has joined #openstack-ansible | 04:05 | |
| *** andreykurilin has quit IRC | 04:07 | |
| *** yolanda has quit IRC | 04:09 | |
| *** jmccrory has quit IRC | 04:09 | |
| *** chason has quit IRC | 04:09 | |
| *** jmccrory has joined #openstack-ansible | 04:13 | |
| *** andreykurilin has joined #openstack-ansible | 04:13 | |
| *** chason has joined #openstack-ansible | 04:15 | |
| *** yolanda has joined #openstack-ansible | 04:24 | |
| *** tux_ has quit IRC | 04:24 | |
| *** asettle has quit IRC | 04:50 | |
| *** udesale has joined #openstack-ansible | 04:50 | |
| *** asettle has joined #openstack-ansible | 04:50 | |
| *** asettle is now known as Guest43527 | 04:50 | |
| *** eumel8 has quit IRC | 04:51 | |
| *** eumel8 has joined #openstack-ansible | 04:51 | |
| *** coreywright has joined #openstack-ansible | 04:58 | |
| *** aruns has joined #openstack-ansible | 05:02 | |
| *** taseer1 has joined #openstack-ansible | 05:02 | |
| *** indistylo has quit IRC | 05:04 | |
| *** Taseer has quit IRC | 05:06 | |
| *** hwoarang has quit IRC | 05:07 | |
| *** taseer1 is now known as Taseer | 05:08 | |
| *** hwoarang has joined #openstack-ansible | 05:09 | |
| *** aruns has quit IRC | 05:13 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-tests master: Add missing services and collect journal logs https://review.openstack.org/587259 | 05:18 |
| cloudnull | tux_ it happens :) | 05:19 |
| *** cshen has joined #openstack-ansible | 05:20 | |
| *** armaan has joined #openstack-ansible | 05:21 | |
| *** armaan has quit IRC | 05:23 | |
| *** armaan has joined #openstack-ansible | 05:24 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add networkd prefix when creating networks https://review.openstack.org/571898 | 05:25 |
| *** cshen has quit IRC | 05:25 | |
| *** Bhujay has joined #openstack-ansible | 05:27 | |
| *** armaan has quit IRC | 05:28 | |
| *** indistylo has joined #openstack-ansible | 05:33 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_designate master: Convert role to use a common systemd service role https://review.openstack.org/587196 | 05:33 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cloudkitty master: Convert role to use a common systemd service role https://review.openstack.org/587187 | 05:34 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cinder master: Convert role to use a common systemd service role https://review.openstack.org/574817 | 05:35 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_barbican master: Convert role to use a common systemd service role https://review.openstack.org/587101 | 05:35 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_ceilometer master: Add systemd journal logging to the service config https://review.openstack.org/587170 | 05:36 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_aodh master: Add systemd journal logging to the service config https://review.openstack.org/587074 | 05:36 |
| openstackgerrit | Merged openstack/openstack-ansible-os_blazar master: Remove unnecessary RabbitMQ setup/vars https://review.openstack.org/586472 | 05:47 |
| openstackgerrit | Merged openstack/openstack-ansible-os_blazar master: Remove unnecessary inventory configuration https://review.openstack.org/586489 | 05:47 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_barbican master: Convert role to use a common systemd service role https://review.openstack.org/587101 | 05:57 |
| *** gvrangan has joined #openstack-ansible | 05:58 | |
| *** gkadam has quit IRC | 06:04 | |
| *** gkadam has joined #openstack-ansible | 06:05 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_blazar master: Add the project source code repository in README https://review.openstack.org/586785 | 06:08 |
| openstackgerrit | Merged openstack/openstack-ansible-os_blazar master: Optimize the link address in docs https://review.openstack.org/579811 | 06:08 |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Delete duplicate 'Release notes' in README https://review.openstack.org/586776 | 06:13 |
| *** madorn has quit IRC | 06:15 | |
| *** madorn has joined #openstack-ansible | 06:17 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Undefine existing VM configurations during rebuild https://review.openstack.org/585001 | 06:19 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_trove master: remove unnessary '=' https://review.openstack.org/587292 | 06:19 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_searchlight master: remove unnecessary '=' https://review.openstack.org/587294 | 06:21 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_panko master: remove unnecessary '=' https://review.openstack.org/587296 | 06:25 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_trove master: remove unnecessary '=' https://review.openstack.org/587292 | 06:27 |
| *** cshen has joined #openstack-ansible | 06:29 | |
| openstackgerrit | caoyuan proposed openstack/openstack-ansible-os_blazar master: Add the project source code repository in README https://review.openstack.org/584834 | 06:32 |
| openstackgerrit | Merged openstack/openstack-ansible-os_searchlight master: Clean up vars, tests and overrides https://review.openstack.org/586539 | 06:32 |
| *** pcaruana has joined #openstack-ansible | 06:37 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_designate master: Convert role to use a common systemd service role https://review.openstack.org/587196 | 06:44 |
| jrosser | morning o/ | 06:44 |
| *** indistylo has quit IRC | 06:48 | |
| *** vnogin has joined #openstack-ansible | 06:51 | |
| *** indistylo has joined #openstack-ansible | 06:51 | |
| *** vnogin has quit IRC | 06:52 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_designate master: Trivial: Fix the pep8 warning https://review.openstack.org/576777 | 06:59 |
| openstackgerrit | Merged openstack/openstack-ansible-os_designate master: Add the project source code repository in README https://review.openstack.org/584838 | 06:59 |
| *** ppetit has joined #openstack-ansible | 07:04 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Convert logstash groks to a multi-pipeline setup https://review.openstack.org/586419 | 07:15 |
| *** pmannidi has quit IRC | 07:18 | |
| *** epalper has joined #openstack-ansible | 07:19 | |
| *** Bhujay has quit IRC | 07:22 | |
| *** Bhujay has joined #openstack-ansible | 07:23 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cinder master: Convert role to use a common systemd service role https://review.openstack.org/574817 | 07:27 |
| odyssey4me | o/ | 07:28 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_barbican master: Convert role to use a common systemd service role https://review.openstack.org/587101 | 07:30 |
| *** aruns has joined #openstack-ansible | 07:36 | |
| *** indistylo has quit IRC | 07:38 | |
| *** electrofelix has joined #openstack-ansible | 07:40 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Remove glance v1 api and tests https://review.openstack.org/587093 | 07:40 |
| *** DanyC has joined #openstack-ansible | 07:42 | |
| *** threestrands has quit IRC | 07:45 | |
| *** suggestable has joined #openstack-ansible | 07:47 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cloudkitty master: Move MQ vhost/user creation into role https://review.openstack.org/586784 | 07:47 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cloudkitty master: Cleanup vars and overrides https://review.openstack.org/586878 | 07:47 |
| *** shardy has joined #openstack-ansible | 07:47 | |
| *** udesale has quit IRC | 07:49 | |
| *** udesale has joined #openstack-ansible | 07:49 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cloudkitty master: Update to use oslo.messaging service RPC and Notify https://review.openstack.org/586586 | 07:50 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cloudkitty master: Move MQ vhost/user creation into role https://review.openstack.org/586784 | 07:50 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cloudkitty master: Cleanup playbook, vars and overrides https://review.openstack.org/586878 | 07:50 |
| jrosser | odyssey4me: if you have a moment these would be the next things needed to unblock the bionic work https://review.openstack.org/#/q/topic:lxc3+(status:open) | 07:50 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ceilometer master: Add systemd journal logging to the service config https://review.openstack.org/587170 | 07:55 |
| *** tosky has joined #openstack-ansible | 07:55 | |
| *** threestrands has joined #openstack-ansible | 08:00 | |
| *** hachi has joined #openstack-ansible | 08:00 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_blazar master: Convert role to use a common systemd service role https://review.openstack.org/587114 | 08:01 |
| *** threestrands has quit IRC | 08:03 | |
| odyssey4me | jrosser: looking | 08:04 |
| odyssey4me | jrosser: https://review.openstack.org/#/c/586692/3 looks like it's an exact copy of the xenial vars file - could we rather just rename that file to 'ubuntu.yml' so that the same applies to both? | 08:06 |
| jrosser | odyssey4me: use yes, in fact ive done them all like that, so leave the "Add vars for bionic" and i'll redo them | 08:07 |
| jrosser | the only one that is different is nova, and i wanted to ask why the dosfstools-dbg was needed? | 08:07 |
| odyssey4me | ok thanks - yeah if the exact copy works then we may as well rename | 08:07 |
| jrosser | if that can go then then nova can use the same vars file too | 08:08 |
| odyssey4me | this way the next ubuntu release will possibly be very close to a no-op | 08:08 |
| jrosser | odyssey4me: this doesnt exist in bionic https://github.com/openstack/openstack-ansible-os_nova/blob/master/vars/ubuntu-16.04.yml#L93 | 08:09 |
| odyssey4me | I onder whether it's even needed | 08:10 |
| odyssey4me | *wonder | 08:10 |
| jrosser | the only thing in it is debug symbols, i'm really not ssure why those are required | 08:10 |
| odyssey4me | here's the thing that added it: https://github.com/openstack/openstack-ansible-os_nova/commit/e771983f2cb05f8b6912bcf02e661658d9970fc0 | 08:11 |
| odyssey4me | yeah, I suspect that they aren't required - perhaps do a patch to remove that package, then do the rename patch on top of that | 08:11 |
| jrosser | thats looks suspect imho | 08:11 |
| odyssey4me | that way it's easy to revert if needed | 08:12 |
| odyssey4me | none of the other distro package lists have the debug symbols | 08:12 |
| *** udesale has quit IRC | 08:13 | |
| *** udesale has joined #openstack-ansible | 08:16 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Move MQ vhost/user creation into role https://review.openstack.org/583388 | 08:17 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Clean up test override vars https://review.openstack.org/587046 | 08:17 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Clarify the octavia-neutron event stream config options https://review.openstack.org/587140 | 08:17 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.25 https://review.openstack.org/584858 | 08:19 |
| openstackgerrit | Merged openstack/openstack-ansible-os_almanach master: Convert role to use a common systemd service role https://review.openstack.org/587070 | 08:20 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582324 | 08:27 |
| *** gkadam has quit IRC | 08:31 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582354 | 08:31 |
| *** gkadam has joined #openstack-ansible | 08:37 | |
| *** hachi has quit IRC | 08:39 | |
| *** gvrangan has quit IRC | 08:41 | |
| *** gkadam has quit IRC | 08:41 | |
| *** hachi has joined #openstack-ansible | 08:42 | |
| *** aruns has quit IRC | 08:46 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Add lxc3 compatibility to tests https://review.openstack.org/586885 | 08:48 |
| eumel8 | odyssey4me: Hi , for https://review.openstack.org/#/c/586866/ I have the same question. Why is the config tag required for translation tasks? | 08:50 |
| *** aruns has joined #openstack-ansible | 08:50 | |
| *** gvrangan has joined #openstack-ansible | 08:51 | |
| *** gregoryo has quit IRC | 09:03 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Add lxc3 compatibility to tests https://review.openstack.org/586887 | 09:07 |
| *** vnogin has joined #openstack-ansible | 09:12 | |
| *** hachi has quit IRC | 09:12 | |
| mbuil | does anybody know how to scape a . symbol inside curly braces? | 09:13 |
| odyssey4me | eumel8: the standard tags are divided in two - {role}-config and {role}-install... the second is used for anything which is a non-runtime task and the first is for runtime tasks... that task seems to me like a runtime task, so it should have the -config tag | 09:13 |
| odyssey4me | mbuil: a var shouldn't have a . symbol, unless the vars is referring to a dict - what's the situation you're trying to solve for? | 09:14 |
| mbuil | odyssey4me: if mgmt_interface is eth0 and mgmt_vlan is 10, I'd like the variable name to be eth0.10. However, I need to check if mgmt_vlan is 'native', in that case I want name to be eth0. I was trying this ==> name: "{{ (mgmt_vlan == 'native') | ternary(mgmt_interface, mgmt_interface.mgmt_vlan) }}" | 09:16 |
| *** taseer1 has joined #openstack-ansible | 09:19 | |
| *** Taseer has quit IRC | 09:19 | |
| odyssey4me | mbuil: oh that should just work, no escaping required given you're referring to a dict | 09:24 |
| odyssey4me | mbuil: you can also express it as mgmt_interface['mgmt_vlan'] | 09:24 |
| mbuil | odyssey4me: no, I am not. mgmt_interface is a string and mgmt_vlan is another string, I just want them to be concatenated with a point in between | 09:25 |
| odyssey4me | ok, so mgmt_interface is a dict that you want to retrieve a var from? or is it a var which changes to the appropriate dict | 09:26 |
| *** hachi has joined #openstack-ansible | 09:27 | |
| odyssey4me | mbuil: maybe a test playbook with some data to show what you're trying to achieve would be useful to help | 09:27 |
| mbuil | odyssey4me: mgmt_interface is a string. I just want to build something like {{ mgmt_interface }}.{{ mgmt_vlan }}, which results in eth0.10 or eth2.30, etc. But when I use it inside the ternary, then it considers mgmt_interface as a dictionary because of the point | 09:29 |
| openstackgerrit | Frank Kloeker proposed openstack/openstack-ansible-os_horizon master: fix execution of horizon_translations_update.yml https://review.openstack.org/586866 | 09:30 |
| eumel8 | odyssey4me: ok, thx ^^ | 09:30 |
| mbuil | odyssey4me: A solution could be to build the variable mgmt_interface_with_vlan: {{ mgmt_interface }}.{{ mgmt_vlan }} in vars/main.yml and then inside the ternary ==> ternary(mgmt_interface, mgmt_interface_with_vlan) | 09:31 |
| odyssey4me | mbuil: ok, and is 'mgmt_interface' also a var - or is the intended output just a string? | 09:32 |
| odyssey4me | because if you want just a string then do: ternary('mgmt_interface', 'mgmt_interface.mgmt_vlan') | 09:32 |
| mbuil | odyssey4me: it is also a var. mgmt_interface and mgmt_vlan are vars | 09:32 |
| odyssey4me | ah ok | 09:33 |
| odyssey4me | in that case I think you might have to do it using script rather than ternary | 09:33 |
| *** hachi has quit IRC | 09:34 | |
| odyssey4me | lemme think a bit | 09:34 |
| *** hachi has joined #openstack-ansible | 09:34 | |
| odyssey4me | mbuil: can you give me a sample data structure - one with and one without the vlan | 09:38 |
| mbuil | odyssey4me: not sure if I understand you, sorry. You mean what the result should be? Let's say mgmt_interface: 'eth0' and mgmt_vlan could be 'native' or any integer | 09:41 |
| odyssey4me | mbuil: no, I mean if you can give me a yaml sample of what the source data looks like then it'd help me think it through | 09:42 |
| *** taseer1 is now known as Taseer | 09:43 | |
| odyssey4me | mbuil: including a sample of the reulting eth0 with or without vlan I mean | 09:44 |
| odyssey4me | I guess maybe I don't need that | 09:45 |
| mbuil | odyssey4me: one second, perhaps the problem I am facing had nothing to do with what I explained you :P. Unfortunately, the error message is quite cryptic ==> 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'public_vlan' | 09:46 |
| mbuil | I thought it meant: I am trying to fetch the attribute public_vlan from a string | 09:47 |
| odyssey4me | mbuil: so in your thing you want to access something like the 'address' in this structure? https://gist.github.com/odyssey4me/7def43fef79564edc7aa6d918f344135 | 09:47 |
| mbuil | odyssey4me: ok, I understand now what you wanted from me! Sorry. Not exactly what you wrote. This would be the structure ==> https://gist.github.com/manuelbuil/297cd506d76940e68a4b629e656dbebe | 09:51 |
| mbuil | odyssey4me: that information should be fed to ==> https://github.com/opnfv/releng-xci/blob/master/xci/playbooks/roles/bootstrap-host/tasks/network_debian.yml#L39-L53 changing ansible_local.xci.network.xci_interface by mgmt_interface and not hardcoding the vlan_id (e.g. 10) but reading it from mgmt_vlan | 09:54 |
| odyssey4me | mbuil: not sure if this helps at all? https://gist.github.com/odyssey4me/7def43fef79564edc7aa6d918f344135 | 09:58 |
| odyssey4me | you'd either have to use a jinja script to prep the var before using it, or you'd have to use a set_fact task ahead of time... or something like that | 09:59 |
| *** tux_ has joined #openstack-ansible | 10:01 | |
| mbuil | odyssey4me: I see, thanks :) | 10:02 |
| *** tux_ has quit IRC | 10:06 | |
| odyssey4me | mbuil: updated with another way to achieve the same thing in a single task | 10:06 |
| *** taseer1 has joined #openstack-ansible | 10:11 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_octavia master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582324 | 10:11 |
| mbuil | odyssey4me: ok, I was more thinking in this direction. THanks again! | 10:11 |
| odyssey4me | mbuil: no problem | 10:11 |
| *** Taseer has quit IRC | 10:12 | |
| *** taseer2 has joined #openstack-ansible | 10:12 | |
| *** electrofelix has quit IRC | 10:12 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: Use generic vars file for ubuntu https://review.openstack.org/586689 | 10:14 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: Use generic vars file for ubuntu https://review.openstack.org/586690 | 10:15 |
| *** taseer1 has quit IRC | 10:15 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_designate master: Use generic vars file for ubuntu https://review.openstack.org/586691 | 10:15 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_glance master: Use generic vars file for ubuntu https://review.openstack.org/586692 | 10:16 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_gnocchi master: Use generic vars file for ubuntu https://review.openstack.org/586693 | 10:17 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_heat master: Use generic vars file for ubuntu https://review.openstack.org/586696 | 10:17 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic master: Use generic vars file for ubuntu https://review.openstack.org/586698 | 10:18 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_molteniron master: Use generic vars file for ubuntu https://review.openstack.org/586701 | 10:18 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: Use generic vars file for ubuntu https://review.openstack.org/586702 | 10:18 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Use generic vars file for ubuntu https://review.openstack.org/586704 | 10:19 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Remove dosfstools-dbg package from ubuntu installs https://review.openstack.org/587374 | 10:19 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: Use generic vars file for ubuntu https://review.openstack.org/586705 | 10:20 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_sahara master: Use generic vars file for ubuntu https://review.openstack.org/586707 | 10:21 |
| *** Bhujay has quit IRC | 10:22 | |
| *** kedorlaomer has joined #openstack-ansible | 10:24 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_trove master: Use generic vars file for ubuntu https://review.openstack.org/586711 | 10:25 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Implement clouds.yaml for swift installs https://review.openstack.org/587376 | 10:25 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582354 | 10:25 |
| openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible-ops master: Add Prometheus node_exporter https://review.openstack.org/587090 | 10:28 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tacker master: Use generic vars file for ubuntu https://review.openstack.org/586708 | 10:29 |
| *** kedorlaomer has quit IRC | 10:29 | |
| *** kedorlaomer has joined #openstack-ansible | 10:37 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_trove master: Switch to using project-templates https://review.openstack.org/586757 | 10:38 |
| *** taseer3 has joined #openstack-ansible | 10:39 | |
| *** taseer2 has quit IRC | 10:42 | |
| kedorlaomer | Hi, folks. I am trying to deploy openstack-ansible (as plain vanilla as possible) on a few boxes. But sadly, I cannot get past `setup-hosts.yml`. During execution, on each LXC container, `yum update` is being executed (we are using CentOS 7). It does not run through since none of the containers is even able to ping anything besides its host. That is apparently a network issue, and I have been trying to find the problem for the | 10:47 |
| *** electrofelix has joined #openstack-ansible | 10:47 | |
| *** kedorlaomer has quit IRC | 10:50 | |
| *** savvas has quit IRC | 10:56 | |
| *** savvas has joined #openstack-ansible | 10:56 | |
| *** kedorlaomer has joined #openstack-ansible | 10:57 | |
| kedorlaomer | Hi, folks. I am trying to deploy openstack-ansible (as plain vanilla as possible) on a few boxes. But sadly, I cannot get past `setup-hosts.yml`. During execution, on each LXC container, `yum update` is being executed (we are using CentOS 7). It does not run through since none of the containers is even able to ping anything besides its host. That is apparently a network issue, and I have been trying to find the problem for the | 10:58 |
| hakonw | your text is getting cut off btw kedorlaomer | 10:59 |
| savvas | Hi guys | 11:01 |
| jrosser | anyone with centos foo before i start digging into this? http://logs.openstack.org/15/586815/8/check/openstack-ansible-functional-centos-7/6d5e6eb/job-output.txt.gz#_2018-07-30_13_11_08_227877 | 11:03 |
| *** kedorlaomer has quit IRC | 11:07 | |
| evrardjp | folks, I am sorry to announce that m3 will be requested without any freeze of the roles. We didn't manage to get things in on time, we tried hard to make things pass, but timeouts of infrastructure are beating us everytime for a few days. | 11:21 |
| evrardjp | My next steps would then be to refactor the testing to get rid of the tempest testing. People that still want to use it still can. | 11:22 |
| evrardjp | I will also work on refactoring the releases to make sure this will never happen again. | 11:22 |
| evrardjp | I hope you're all okay with this. | 11:22 |
| *** taseer4 has joined #openstack-ansible | 11:22 | |
| *** vnogin has quit IRC | 11:22 | |
| *** taseer4 is now known as Taseer | 11:23 | |
| *** vnogin has joined #openstack-ansible | 11:24 | |
| *** Taseer has quit IRC | 11:24 | |
| *** Taseer has joined #openstack-ansible | 11:24 | |
| *** taseer3 has quit IRC | 11:26 | |
| evrardjp | cloudnull: are you there? | 11:28 |
| *** ansmith has quit IRC | 11:29 | |
| evrardjp | cloudnull: I'd like to discuss nspawn testing | 11:32 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582354 | 11:44 |
| *** lvdombrkr has joined #openstack-ansible | 11:44 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_aodh master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/587395 | 11:54 |
| odyssey4me | evrardjp: I think it's fine, and yes I think it's time for a change in how releases are done to relieve the pressure. | 11:55 |
| *** vnogin has quit IRC | 11:59 | |
| odyssey4me | cores, I have a metric tonne of patches up for review which really need some eyes: https://goo.gl/k9cCfF | 12:00 |
| *** vnogin has joined #openstack-ansible | 12:05 | |
| *** tux_ has joined #openstack-ansible | 12:06 | |
| bgmccollum | evrardjp odyssey4me can y | 12:07 |
| bgmccollum | evrardjp odyssey4me can you tell whats going on here? lots of gates failing with this same error -- http://logs.openstack.org/29/587229/1/check/openstack-ansible-deploy-aio_metal-ubuntu-xenial/ee5864f/job-output.txt.gz#_2018-07-31_01_41_10_751430 | 12:07 |
| bgmccollum | The repository located at mirror.mtl01.inap.openstack.org is not a trusted or secure host and is being ignored. | 12:08 |
| odyssey4me | bgmccollum: that should be a warning, not a failure | 12:09 |
| odyssey4me | bgmccollum: the real fail is: "Could not find a version that satisfies the requirement pbr<=4.0.4" | 12:09 |
| odyssey4me | bgmccollum: what patch is that from? | 12:10 |
| odyssey4me | this one? https://review.openstack.org/#/c/587229/ | 12:10 |
| bgmccollum | Yes, I noticed it in mine first, but also noticed the same issues with other patches... | 12:11 |
| bgmccollum | http://mirror.mtl01.inap.openstack.org/wheel/ubuntu-16.04-x86_64/pbr/ has pbr-4.0.2-py2.py3-none-any.whl | 12:11 |
| odyssey4me | bgmccollum: oh, but that didn't cause the whole thing to fail - that's a try/rescue, and the rescue succeeded | 12:12 |
| bgmccollum | Ah OK...i need a big red HERE IS THE REAL FAILURE | 12:13 |
| odyssey4me | right here: http://logs.openstack.org/29/587229/1/check/openstack-ansible-deploy-aio_metal-ubuntu-xenial/ee5864f/job-output.txt.gz#_2018-07-31_02_20_50_371696 | 12:13 |
| odyssey4me | bgmccollum: go into 'logs' -> 'ara-report' and you'll get a much easier to grok report | 12:14 |
| odyssey4me | this is the task failing, but we're not sure why - most likely a missing secret or something | 12:15 |
| odyssey4me | https://github.com/openstack/openstack-ansible-os_swift/blob/master/tasks/main.yml#L118-L130 | 12:15 |
| *** aruns__ has joined #openstack-ansible | 12:17 | |
| odyssey4me | yep, looks like swift_oslomsg_notify_password is not correctly defined, will push a patch up now to fix | 12:17 |
| *** aruns has quit IRC | 12:19 | |
| *** ppetit has quit IRC | 12:22 | |
| bgmccollum | thanks | 12:22 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Unblock gate: Provide default rabbitmq password https://review.openstack.org/587401 | 12:25 |
| odyssey4me | evrardjp https://review.openstack.org/587401 should resolve the issue bgmccollum just mentioned - it's a hack, but given we can't merge stuff easily into the integrated build this is the fastest way to get it out the way | 12:25 |
| odyssey4me | I'll update another patch I have in flight to properly resolve the issue now. | 12:26 |
| evrardjp | odyssey4me: I expected this. | 12:26 |
| evrardjp | I was wondering if it would show up on smaller roles like masakari | 12:26 |
| evrardjp | but there is probably no wiring there. | 12:26 |
| evrardjp | swift being a different one that's fun too. | 12:26 |
| *** kedorlaomer has joined #openstack-ansible | 12:34 | |
| *** savvas has quit IRC | 12:34 | |
| odyssey4me | evrardjp: I'm working on a patch now to remove the wiring from the integrated build, because once they're all merged into the roles there's no need for it any more | 12:34 |
| *** savvas has joined #openstack-ansible | 12:35 | |
| evrardjp | odyssey4me: there is a need of this wiring for upgrades | 12:35 |
| *** ppetit has joined #openstack-ansible | 12:35 | |
| evrardjp | so if you remove it, please add in the upgrade tool | 12:35 |
| odyssey4me | yep, good call - will add that to the same patch | 12:35 |
| evrardjp | (renaming variable _rabbitmq_ to _oslo_rpc_ ) | 12:36 |
| *** savvas has quit IRC | 12:39 | |
| *** savvas has joined #openstack-ansible | 12:40 | |
| *** tux_ has quit IRC | 12:40 | |
| *** savvas has quit IRC | 12:41 | |
| *** savvas has joined #openstack-ansible | 12:41 | |
| kedorlaomer | ... apparently a network issue, and I have been trying to find the problem for the past few weeks --- without success. So I am asking the pros now :-) What additional infos do you need? I just jot down a few peculiarites I found along the way: There are interfaces eth0 (connected to lxcbr0) and eth1 (connected to br-mgmt); if I `ip link delete` eth1, I can reach everything via eth0. This does not hold if I only set it `ip link | 12:42 |
| kedorlaomer | ... of approximately five to seven seconds in which I can reach everything (that is, from eth0, not eth1). I am just using the default configuration (that comes with queens) with a few IP addresses changed to match our network setup, so this strange error is less than expected. | 12:43 |
| odyssey4me | kedorlaomer: not sure how or what you're trying to communicate through, but it always gets cut off and your client fails uot | 12:43 |
| odyssey4me | try smaller sentences | 12:43 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: [WIP] Remove all MQ vhost/user and DB create tasks https://review.openstack.org/585504 | 12:44 |
| *** aruns has joined #openstack-ansible | 12:49 | |
| jrosser | here's the next lot of bionic stuff ready for review https://review.openstack.org/#/q/topic:bionic-vars+(status:open) | 12:50 |
| *** aruns__ has quit IRC | 12:51 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Remove all MQ vhost/user and DB create tasks https://review.openstack.org/585504 | 12:52 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Add more systematic healthchecks https://review.openstack.org/587408 | 12:56 |
| *** gillesMo has joined #openstack-ansible | 12:57 | |
| odyssey4me | evrardjp: I think https://review.openstack.org/585504 will cover it. | 12:57 |
| evrardjp | odyssey4me: haha the healthcheck infrastructure still relies on https://review.openstack.org/#/c/585504/3/playbooks/common-tasks/rabbitmq-vhost-user.yml | 12:58 |
| *** savvas has quit IRC | 12:58 | |
| evrardjp | it's fine | 12:58 |
| odyssey4me | evrardjp: see my patch - not any more ;) | 12:58 |
| evrardjp | odyssey4me: haha | 12:58 |
| evrardjp | I am refactoring those plays | 12:59 |
| *** savvas has joined #openstack-ansible | 12:59 | |
| evrardjp | to be more inline with the current order of the plays | 12:59 |
| evrardjp | and I will increase coverage slowly | 12:59 |
| evrardjp | and when it's ready I will add them in place of tempest. | 12:59 |
| odyssey4me | evrardjp: the one thing I was thinking about with those is to delete anything created | 13:00 |
| odyssey4me | so if a new rabbitmq user is created, then delete it afterwards | 13:00 |
| odyssey4me | otherwise it leaves things behind which isn't good | 13:00 |
| jamesdenton | odyssey4me evrardjp when you guys have some time, mind giving this a look and some advice on implementation? https://goo.gl/cCA781 | 13:01 |
| evrardjp | odyssey4me: it does it | 13:01 |
| evrardjp | "Remove guest user for rabbitmq" | 13:01 |
| openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible-ops master: Add Prometheus node_exporter https://review.openstack.org/587090 | 13:01 |
| odyssey4me | ah ok, missed that | 13:01 |
| evrardjp | odyssey4me: it might have some gaps | 13:02 |
| *** savvas has quit IRC | 13:02 | |
| *** savvas_ has joined #openstack-ansible | 13:02 | |
| *** aruns__ has joined #openstack-ansible | 13:06 | |
| *** aruns has quit IRC | 13:07 | |
| *** ansmith has joined #openstack-ansible | 13:07 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Add more systematic healthchecks https://review.openstack.org/587408 | 13:07 |
| odyssey4me | jamesdenton: eish, I'll leave that review set to mbuil and evrardjp given they've the networking expertise. | 13:09 |
| jamesdenton | don't be scared | 13:09 |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican master: Move MQ vhost/user creation into role https://review.openstack.org/583363 | 13:09 |
| jamesdenton | but ok :) | 13:10 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_nova master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584450 | 13:13 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible master: [WIP] Add qdrouterd role for messaging backend deployment https://review.openstack.org/570261 | 13:14 |
| evrardjp | odyssey4me: could you please adapt with my comments of https://review.openstack.org/#/c/585504/3 ? | 13:18 |
| odyssey4me | evrardjp: sure | 13:18 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Switch to using project-templates https://review.openstack.org/586755 | 13:19 |
| *** chandankumar has left #openstack-ansible | 13:19 | |
| cloudnull | evrardjp ping - re: nspawn testing - I'm here now for a little bit. | 13:19 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts stable/pike: Do not install linux-image-extra by default https://review.openstack.org/587075 | 13:20 |
| evrardjp | cloudnull: great | 13:20 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Remove all MQ vhost/user and DB create tasks https://review.openstack.org/585504 | 13:20 |
| cloudnull | Mornings all btw | 13:20 |
| odyssey4me | evrardjp: done | 13:20 |
| odyssey4me | -974 +56 ... now that's a good code change | 13:20 |
| cloudnull | Ha! | 13:20 |
| evrardjp | odyssey4me: yeah | 13:21 |
| evrardjp | let's see if things pass. | 13:21 |
| evrardjp | cloudnull: so no need to recheck that m3 patch anymore -- I have said a little above what I plan to do | 13:21 |
| evrardjp | #tired | 13:21 |
| evrardjp | but thanks cloudnull and odyssey4me for helping me in my rechecking times :) | 13:21 |
| odyssey4me | mostly just a bunch of group_vars, but of course that means we eat less memory after that - happy days :) | 13:21 |
| evrardjp | odyssey4me: also: double tasks | 13:21 |
| evrardjp | no double tasks, faster runs! | 13:22 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts stable/queens: Do not install linux-image-extra by default https://review.openstack.org/587076 | 13:22 |
| odyssey4me | evrardjp: ok, so are you abandoning the M3 pin and revert patches then? | 13:22 |
| evrardjp | odyssey4me: yes. | 13:23 |
| odyssey4me | could do with some core eyes on https://review.openstack.org/586586 when anyone has a moment | 13:23 |
| evrardjp | I will just tag with what's in repo | 13:23 |
| evrardjp | too bad | 13:23 |
| odyssey4me | evrardjp: yep, fair enough | 13:23 |
| odyssey4me | hopefully we can unblock everything and get it moving again | 13:23 |
| odyssey4me | also, implementing a simpler release mechanism as we discussed will be really helpful to take the pressure off | 13:24 |
| evrardjp | odyssey4me: we'll see | 13:24 |
| evrardjp | yeah | 13:24 |
| evrardjp | I am working on both, I kinda dropped the ball on my other works, but it's fine I guess? :p | 13:24 |
| evrardjp | hahaha | 13:24 |
| jrosser | so bionic tests - should i add a temporary voting: true for bionic to the roles that i've addressed? | 13:25 |
| mbuil | odyssey4me, jamesdenton: I'll try to review it today. Sorry, I am quite busy :( | 13:25 |
| odyssey4me | jrosser: well, if the passing rate is good across most roles, then perhaps we should adjust the template to make it voting and then go to the roles which aren't and set it no-voting temporarily | 13:26 |
| jamesdenton | no worries mbuil - thanks for any time you can give it. | 13:26 |
| jrosser | odyssey4me: everything in a-r-r.txt should be good except octavia and tempest | 13:28 |
| odyssey4me | cloudnull: if you have a cup of coffee in hand, most of these are pretty straightforward: https://goo.gl/k9cCfF | 13:28 |
| odyssey4me | jrosser: sounds to me like we should make the template voting then and we can either get the last ones fixed, or make them non-voting until we get the chance to | 13:29 |
| *** tux_ has joined #openstack-ansible | 13:30 | |
| ebbex_ | Can I override a single requirement in the ansible-hardening? Say I don't care about 'chrony' being installed. | 13:31 |
| ebbex_ | I tried setting 'security_enable_chrony: false' in user_variables.yml, but that didn't seem to work. | 13:31 |
| *** savvas_ has quit IRC | 13:31 | |
| *** aruns__ has quit IRC | 13:33 | |
| *** jwitko_ has joined #openstack-ansible | 13:34 | |
| openstackgerrit | Merged openstack/openstack-ansible-galera_server master: Remove the unnecessary space https://review.openstack.org/587094 | 13:34 |
| *** jwitko_ has quit IRC | 13:35 | |
| *** jwitko_ has joined #openstack-ansible | 13:36 | |
| *** gvrangan has quit IRC | 13:36 | |
| *** aruns__ has joined #openstack-ansible | 13:39 | |
| *** savvas has joined #openstack-ansible | 13:41 | |
| lbragstad | cloudnull: yeah - i started removing some of the files in /etc/system/ and then restarted | 13:41 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Implement clouds.yaml for root user https://review.openstack.org/587376 | 13:41 |
| *** tux_ has quit IRC | 13:45 | |
| lbragstad | still seeing things like "Unit var-lib-nova-instances.mount has failed." because /var/lib/nova/instances doesn't exist | 13:47 |
| lbragstad | attempting to clean up some of those unit with systemd now | 13:51 |
| openstackgerrit | Maksim Malchuk proposed openstack/openstack-ansible stable/queens: [DNM] test https://review.openstack.org/577902 | 13:55 |
| odyssey4me | evrardjp: answered your Q in https://review.openstack.org/#/c/586878/4 | 13:56 |
| evrardjp | odyssey4me: this is a series of group vars we will be able to remove in the future too. | 13:58 |
| evrardjp | or we should at least aim to remove | 13:58 |
| odyssey4me | evrardjp: ye | 13:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca master: Upgrade kafka repository https://review.openstack.org/587173 | 14:00 |
| openstackgerrit | Merged openstack/openstack-ansible-os_designate stable/queens: Use designate_galera_database https://review.openstack.org/585722 | 14:03 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests master: Change bionic to be voting https://review.openstack.org/587478 | 14:04 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_aodh master: Add systemd journal logging to the service config https://review.openstack.org/587074 | 14:04 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cloudkitty master: Convert role to use a common systemd service role https://review.openstack.org/587187 | 14:06 |
| *** tux_ has joined #openstack-ansible | 14:13 | |
| *** prasen has joined #openstack-ansible | 14:13 | |
| Adri2000 | after a pike->queens upgrade, what could explain that neutron agents are not running on bare metal? I don't have anything in /etc/openstack_deploy/env.d/ | 14:17 |
| Adri2000 | neutron_agents_container group (from inventory-manage.py -G) only has the containers, not the bare metal hosts | 14:17 |
| *** vnogin has quit IRC | 14:18 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [TEST] bionic https://review.openstack.org/586607 | 14:20 |
| evrardjp | Adri2000: not all are on bare metal | 14:20 |
| evrardjp | iirc | 14:21 |
| odyssey4me | Adri2000: I think that's working as designed, but I'm not entirely sure. I think you have to move them yourself. cloudnull may be able to advise. | 14:21 |
| Adri2000 | evrardjp: I have none on bare metal actually. l3-agent, dhcp-agent and linuxbridge-agent are still running only inside the container | 14:22 |
| Adri2000 | odyssey4me: relnote says " | 14:22 |
| Adri2000 | When upgrading there is nothing a deployer must immediately do to run neutron agent services on hosts within the network_hosts group. Simply executing the playbooks will deploy the neutron servers on the baremetal machines and will leave all existing agents containers alone. | 14:22 |
| Adri2000 | " | 14:22 |
| odyssey4me | I would have thought that there'd at least be new agents on bare metal, and you'd have to manually remove your old ones. But (this was some time ago) when I tried I found the same as you. | 14:22 |
| cloudnull | Adri2000 post upgrade you should have neutron agents running in containers and baremetal | 14:22 |
| Adri2000 | that's what I expected from reading the relnote | 14:22 |
| cloudnull | with the upgrade script there a few post upgrade tasks that have to be run to clean up those extra containers. | 14:23 |
| *** tux__ has joined #openstack-ansible | 14:23 | |
| cloudnull | devx and I did an upgrade to queens just the other day and that's how it worked for us ? | 14:23 |
| odyssey4me | Adri2000: do you happen to have some old env.d files in /etc/openstack_deploy ? | 14:24 |
| Adri2000 | odyssey4me: cleaned that up after what you told me yesterday | 14:25 |
| Adri2000 | so, no | 14:25 |
| cloudnull | a lot of folks have the neutron agents running on the infra hosts, are you sure the agents aren't started installed there (maybe just not started)? | 14:25 |
| odyssey4me | cloudnull: as far as I know there's nothing in the upgrade stuff to clean up, because when I was building the ones to clean up cinder, ironic, trove I vaguely remember seeing not host agents and not being sure how to proceed | 14:25 |
| odyssey4me | then I ran out of time | 14:25 |
| ivve | yo guys | 14:25 |
| ivve | this variable still required or fix implemented? nova_spicehtml5_git_repo: https://gitlab.freedesktop.org/spice/spice-html5.git | 14:25 |
| odyssey4me | ivve: fix implemented some time ago | 14:26 |
| ivve | in latest pike/stable | 14:26 |
| *** armaan has joined #openstack-ansible | 14:26 | |
| ivve | odyssey4me: cool thanks! | 14:26 |
| Adri2000 | my guess is that the issue is at the inventory level, given that the bare metal hosts are not part of the neutron_agent and neutron_agents_container ansible groups | 14:26 |
| cloudnull | ++ a custom env.d file would change the default behavior. | 14:26 |
| odyssey4me | Adri2000: if that is the case, then we have a bug somewhere | 14:26 |
| *** tonyontopic has joined #openstack-ansible | 14:26 | |
| cloudnull | somthing in /etc/openstack_deploy/env.d | 14:26 |
| *** tux_ has quit IRC | 14:26 | |
| *** zenirc369 has joined #openstack-ansible | 14:27 | |
| ivve | cloudnull: btw that environment with that later keepalived worked better (the bionic repo one) | 14:27 |
| cloudnull | for the upgrade we just did, we simply ran the run_upgrade.sh script and all was well with the world | 14:27 |
| cloudnull | ivve nice! was taht installed through the UCA repo or by pulling the bionic packages directly ? | 14:28 |
| Adri2000 | to be fair, I still have one env.d/ file, which is cinder-volume.yml and puts cinder-volume inside a container. but absolutely nothing related to neutron :( | 14:28 |
| ivve | cloudnull: bionic directly for both hosts & neutron_agents container | 14:28 |
| ivve | no issues found after a good couple of weeks now during summer | 14:29 |
| cloudnull | Adri2000 if you run `openstack-ansible os-neutron-install.yml --list-hosts` do you see any of the hosts that were running the neutron agents containers before? | 14:29 |
| cloudnull | ivve that's actually great news | 14:30 |
| Adri2000 | cloudnull: that lists only neutron containers (server and agents) as well as compute nodes | 14:30 |
| Adri2000 | doesn't list bare metal network_hosts | 14:31 |
| spotz | greetings peeps | 14:31 |
| odyssey4me | need another core review for https://review.openstack.org/587401 if anyone has a moment, and also https://review.openstack.org/586586 | 14:31 |
| cloudnull | Adri2000 can you have a look at your local neutron env.d file within the osa playbooks, do you have https://github.com/openstack/openstack-ansible/blob/stable/queens/inventory/env.d/neutron.yml#L71 ? | 14:32 |
| ivve | cloudnull: basic instruction is install universe & main restricted repos (bionic) and then update keepalived and debsums | 14:33 |
| cloudnull | Adri2000 also in the file /usr/local/bin/openstack-ansible.rc is the inventory path pointed to the playbook location with the correct env.d files? | 14:33 |
| Adri2000 | cloudnull: in /opt/openstack-ansible/inventory/env.d/neutron.yml, I do yes, is_metal: true. I have tried copying that file in /etc/opentack_deploy/env.d/ just in case, but that doesn't change the generated inventory | 14:33 |
| Adri2000 | `export ANSIBLE_INVENTORY="${ANSIBLE_INVENTORY:-/opt/openstack-ansible/inventory/dynamic_inventory.py,/etc/openstack_deploy/inventory.ini}"` | 14:34 |
| Adri2000 | that dynamic_inventory.py is next to the env.d/ directory, looks good to me | 14:35 |
| cloudnull | ++ that all looks good to me | 14:35 |
| cloudnull | Adri2000 anything in the openstack_user_config or conf.d files that is setting is_metal? | 14:38 |
| cloudnull | under /etc/openstack_deploy | 14:38 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_designate master: Convert role to use a common systemd service role https://review.openstack.org/587196 | 14:40 |
| openstackgerrit | Merged openstack/openstack-ansible-os_watcher master: Move MQ vhost/user creation into role https://review.openstack.org/583397 | 14:40 |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally stable/queens: Implement pypi-based install and package pinning for non u-c packages https://review.openstack.org/585170 | 14:40 |
| Adri2000 | cloudnull: "container_skel: cinder_volumes_container: properties: is_metal: false" in env.d/*. then nothing apart from "is_metal"s in openstack_inventory.json | 14:40 |
| openstackgerrit | Merged openstack/openstack-ansible-os_congress master: Move MQ vhost/user creation into role https://review.openstack.org/583369 | 14:42 |
| spotz | hey odyssey4me - in https://review.openstack.org/#/c/586586/4/tests/test-install-cloudkitty.yml password is secret and in the readme it's different, I'm assuming that's fine but want to double check before voting | 14:42 |
| spotz | https://review.openstack.org/#/c/586586 for reference | 14:42 |
| Adri2000 | I've just started again the VM where I did a pike AIO deployment + queens upgrade last week: same issue there. neutron agents only in the lxc container | 14:43 |
| cloudnull | Adri2000 are the cinder volumes containers on the same host as the neutron agents? maybe the property is bleeding over ? | 14:43 |
| * cloudnull grasping at straws | 14:43 | |
| cloudnull | having just done this in a prod environment the other day, it was working as expected. | 14:44 |
| cloudnull | so im not quite sure what is going on | 14:44 |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587021 | 14:44 |
| openstackgerrit | David Wilde proposed openstack/openstack-ansible-ops master: Deploy ELK in MNAIO https://review.openstack.org/586612 | 14:44 |
| Adri2000 | cloudnull: could you confirm that the network_hosts bare metal machines are supposed to be in the neutron_agent ansible group? in the neutron_agents_container ansible group as well | 14:46 |
| Adri2000 | ? | 14:46 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Remove all MQ vhost/user creation tasks/vars https://review.openstack.org/586524 | 14:47 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/585990 | 14:48 |
| cloudnull | Adri2000 I dont have access to the environment at this time however I can get someone to take a look a little later today | 14:49 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587026 | 14:49 |
| Adri2000 | thanks | 14:49 |
| cloudnull | anyone around want to give these a review https://review.openstack.org/#/q/starredby:cloudnull+status:open,n,z | 14:50 |
| *** vnogin has joined #openstack-ansible | 14:51 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove master: Add missing domain_id argument to os_project task https://review.openstack.org/587057 | 14:51 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker master: Move MQ vhost/user creation into role https://review.openstack.org/583394 | 14:51 |
| *** vollman has joined #openstack-ansible | 14:52 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587036 | 14:52 |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587011 | 14:53 |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586529 | 14:53 |
| *** KeithMnemonic has joined #openstack-ansible | 14:54 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587022 | 14:55 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Set lxc_* vars in one place https://review.openstack.org/583241 | 14:55 |
| *** armaan has quit IRC | 14:58 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_zun master: Move MQ vhost/user creation into role https://review.openstack.org/583399 | 14:59 |
| *** weezhard has joined #openstack-ansible | 15:00 | |
| *** FrankZhang has joined #openstack-ansible | 15:01 | |
| jrosser | can i get some more eyes on these https://review.openstack.org/#/q/topic:bionic-vars+(status:open+OR+status:merged) | 15:04 |
| cloudnull | jrosser looking | 15:08 |
| *** pcaruana has quit IRC | 15:10 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587033 | 15:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587027 | 15:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586535 | 15:13 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Add openstack client to Ansible venv https://review.openstack.org/582494 | 15:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587017 | 15:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586530 | 15:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Convert role to use a common systemd mount role https://review.openstack.org/552185 | 15:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587029 | 15:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587023 | 15:13 |
| *** dabukalam has quit IRC | 15:14 | |
| jrosser | cloudnull: cheers! | 15:15 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/queens: Map instances to cell_v2 only once. https://review.openstack.org/585156 | 15:15 |
| odyssey4me | evrardjp: could you revisit your vote on https://review.openstack.org/585157 and https://review.openstack.org/585158 with queens now merged? | 15:16 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586534 | 15:17 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587031 | 15:18 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587038 | 15:18 |
| *** armaan has joined #openstack-ansible | 15:18 | |
| *** vinsh has joined #openstack-ansible | 15:19 | |
| jrosser | seeing quite a few centos7 "Failed to connect to copr-be.cloud.fedoraproject.org" which is sad | 15:22 |
| cloudnull | fairly typical, sadly | 15:22 |
| *** udesale has quit IRC | 15:23 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty master: Update to use oslo.messaging service RPC and Notify https://review.openstack.org/586586 | 15:24 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty master: Move MQ vhost/user creation into role https://review.openstack.org/586784 | 15:24 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty master: Cleanup playbook, vars and overrides https://review.openstack.org/586878 | 15:24 |
| cloudnull | given the flakiness of the that repo I'd like to transition all of the cent7 testing to nspawn and get rid of lxc for the distro but it'll have to be something for the next cycle. | 15:24 |
| cloudnull | btw: evrardjp still want to talk about nspawn testing ? | 15:24 |
| *** cshen has quit IRC | 15:25 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582354 | 15:25 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cloudkitty master: Convert role to use a common systemd service role https://review.openstack.org/587187 | 15:27 |
| mnaser | cloudnull: hopefully nspawn for centos nexy cycle | 15:27 |
| mnaser | next cycle** | 15:27 |
| cloudnull | ++ | 15:27 |
| cloudnull | hows it mnaser ? | 15:27 |
| odyssey4me | cloudnull: Well, I'm inclined to think that perhaps CentOS should just be distro installs and no containers. Given the low helper count on that front, and the complexity of keeping source-based builds going on there - I'm inclined to suggest we go for simplicity. | 15:28 |
| cloudnull | that might be a good way forward for cent, for the time being. | 15:29 |
| odyssey4me | That said, it seems that we've managed to get it to a reasonably stable state now. It'd kinda be a shame to go backwards. :p | 15:29 |
| openstackgerrit | Merged openstack/openstack-ansible-os_molteniron master: Use generic vars file for ubuntu https://review.openstack.org/586701 | 15:31 |
| odyssey4me | I'm honestly just getting to a place where I wonder what the point is of using containers at all in our builds. We're using slices, so we have process isolation. We now also have complete distribution package vs venv isolation, so we're not needing any form of binary isolation. | 15:31 |
| cloudnull | I think we're on a pretty good trajectory ATM but cent7 needs some more love for sure. | 15:31 |
| odyssey4me | Containers are adding a ton of networking complexity, and I'm not really understanding their value. | 15:32 |
| *** dabukalam has joined #openstack-ansible | 15:33 | |
| *** dabukalam has quit IRC | 15:33 | |
| *** dabukalam has joined #openstack-ansible | 15:33 | |
| evrardjp | I am trying to see what's the minimum testing required to be accurate -- so less things to run. | 15:33 |
| evrardjp | I realize we have tons of duplications | 15:33 |
| cloudnull | odyssey4me you're not wrong. with all of the service isolation we do in the default units its very container like without the extra filesystem. that said the service isolation is not giving any network isoaltion for a service without a container and that's been useful in a lot of production cases. | 15:34 |
| odyssey4me | cloudnull: I'm just kinda feeling that perhaps container usage should be opt-in, rather than opt-out. | 15:34 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift master: Unblock gate: Provide default rabbitmq password https://review.openstack.org/587401 | 15:35 |
| odyssey4me | With that approach, we could actually just ditch LXC and have nspawn - and nspawn would only need to be something people care about if they opt into using it. | 15:35 |
| Tahvok | Will there be a meeting today? | 15:35 |
| jrosser | i like being able to destroy/recreate a thing entirely as if its a machine | 15:35 |
| jrosser | and there is a lot to be said for nothing much mattering on the hosts themselves, i rarely worry i'm going to mess up container X by fiddling in container Y | 15:36 |
| cloudnull | jrosser ++ | 15:36 |
| cloudnull | There's a lot of benefits to the current architecture in that regard. | 15:37 |
| cloudnull | One of the things i'd like to figure out this cycle is how to call openstack_hosts/pip_install less. deploying queens/rocky to prod seems like its running that role (or parts of it) 100000000000000 times. | 15:37 |
| odyssey4me | jrosser agreed, although I was thinking that if we just used chroots we could achieve the same thing without the network complexity... and then we could actually commit to the same pattern for absolutely everything (compute service included) | 15:37 |
| odyssey4me | cloudnull: uh, pike was doing that - it got cut down dramatically in queens | 15:38 |
| cloudnull | its very much still there. | 15:38 |
| odyssey4me | pip_install gets run in repo_server/repo_build as a meta dep - and I think that can get cleaned up | 15:39 |
| cloudnull | maybe less so than pike? | 15:39 |
| odyssey4me | it also gets run in openstack_hosts, and in the utility play - that is it as far as I know | 15:39 |
| odyssey4me | openstack_hosts gets run once for hosts and once for containers, and that's it, as far as I know | 15:39 |
| odyssey4me | if it's running more often than that, I'd be surprised | 15:40 |
| cloudnull | we saw a lot of it over the weekend doing a queens upgrade. | 15:40 |
| *** priteau has joined #openstack-ansible | 15:40 | |
| cloudnull | it felt like we were spending a lot of time watching tasks fly by with no change. | 15:41 |
| odyssey4me | hmm, we should look into that - whatever the fix, it's probably back portable | 15:41 |
| cloudnull | ++ | 15:42 |
| cloudnull | we're still compiling notes on the upgrades. | 15:42 |
| cloudnull | hopefully we'll get some prs in soon. | 15:42 |
| cloudnull | newton>queens in an evening, much fun was had by all. | 15:43 |
| priteau | odyssey4me: Hello. I am a Blazar core dev, I have recently started following openstack-ansible-os_blazar. I just noticed that https://review.openstack.org/#/c/586472/ got merged. I don't think it's right, Blazar uses RabbitMQ to send RPC betwen API and Manager. | 15:44 |
| odyssey4me | priteau: ah, that got merged because there didn't appear to be anything in the role using it | 15:45 |
| priteau | Oops, I didn't realize you were in your weekly meeting. | 15:45 |
| Tahvok | Shouldn't it start in 15 minutes? | 15:46 |
| odyssey4me | priteau: if you could put together a new patch to implement the right config entries, similar to the pattern in the other roles, that'd be a great help | 15:46 |
| odyssey4me | oh, and we're not in our meeting yet | 15:46 |
| priteau | Right, UTC time. | 15:46 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [TEST] bionic https://review.openstack.org/586607 | 15:47 |
| openstackgerrit | James Denton proposed openstack/openstack-ansible-os_neutron master: Adds networking-ovn support https://review.openstack.org/584067 | 15:48 |
| priteau | odyssey4me: I can take a look, but I am not familiar with openstack-ansible yet. | 15:48 |
| *** tonyontopic has quit IRC | 15:50 | |
| odyssey4me | priteau: it should mostly be a copy/paste/sed affair from one of the other roles - but if you find you're not comfortable doing it then ping me and I'll pick it up for you | 15:50 |
| odyssey4me | I'm very happy to help you through the process, and you'll find as a community we're usually helpful if we can be. | 15:51 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_blazar master: Convert role to use a common systemd service role https://review.openstack.org/587114 | 15:53 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_blazar master: This patch ensures that the functional tests for Blazar pass. https://review.openstack.org/571388 | 15:53 |
| cloudnull | oh, priteau ^ was rebasing blazar due to merge conflicts from the other changes. sorry if that's getting in the wya. | 15:54 |
| cloudnull | **in the way of something you were working on | 15:54 |
| priteau | odyssey4me: Thanks for the offer, I'll give it a try. | 15:56 |
| *** epalper has quit IRC | 15:56 | |
| openstackgerrit | Pierre Riteau proposed openstack/openstack-ansible-tests master: Fix link to mq_setup.yml https://review.openstack.org/587508 | 15:56 |
| priteau | ^^^ My first patch ;-) | 15:56 |
| evrardjp | who is hosting today's meeting? | 15:57 |
| mnaser | o/ | 15:57 |
| evrardjp | chairing* | 15:57 |
| mnaser | just about to highlight $world | 15:57 |
| evrardjp | go ahead | 15:57 |
| mnaser | cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum, darrenc, JRobinson__, colinmcnamara, thorst, adreznec, eil397, | 15:57 |
| mnaser | qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, Tahvok, mnaser, nicolasbock, jrosser | 15:57 |
| spotz | boo | 15:57 |
| mnaser | 5 minute (or 3 minute warning) :) | 15:57 |
| mnaser | evrardjp: could you do me the wonderful favour you always do? :D | 15:57 |
| evrardjp | it's already done | 15:57 |
| cloudnull | anyone want to hit https://review.openstack.org/#/c/573509/ through ? | 15:57 |
| mnaser | you da bess | 15:57 |
| cloudnull | :) | 15:58 |
| evrardjp | Editing the agenda | 15:58 |
| mnaser | cloudnull: doneso | 15:58 |
| cloudnull | TYVM! | 15:58 |
| mnaser | meetings always happen at lunch time here | 15:58 |
| mnaser | :'( | 15:58 |
| evrardjp | mnaser: or dinner time. | 16:00 |
| evrardjp | :) | 16:00 |
| cloudnull | or brunch time! | 16:00 |
| mnaser | #startmeeting openstack_ansible_meeting | 16:00 |
| guilhermesp | my plate is always making me company :P | 16:00 |
| evrardjp | we can do it earlier. | 16:00 |
| openstack | Meeting started Tue Jul 31 16:00:35 2018 UTC and is due to finish in 60 minutes. The chair is mnaser. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| *** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)" | 16:00 | |
| mnaser | #topic roll call | 16:00 |
| openstack | The meeting name has been set to 'openstack_ansible_meeting' | 16:00 |
| evrardjp | o/ | 16:00 |
| *** openstack changes topic to "roll call (Meeting topic: openstack_ansible_meeting)" | 16:00 | |
| cloudnull | o/ | 16:00 |
| mnaser | o/ | 16:00 |
| guilhermesp | \m/ | 16:00 |
| Tahvok | o/ | 16:01 |
| odyssey4me | o/ | 16:01 |
| mnaser | how's everyone doing on this lovely day | 16:01 |
| jrosser | o/ | 16:01 |
| mnaser | (everyone is doing great apparently) | 16:02 |
| mnaser | cool so lets dive in | 16:02 |
| mnaser | #topic last week highlights | 16:02 |
| *** openstack changes topic to "last week highlights (Meeting topic: openstack_ansible_meeting)" | 16:02 | |
| mnaser | odyssey4me: Reminded we need more work done on Bionic. | 16:02 |
| odyssey4me | and jrosser has picked up the challenge! | 16:02 |
| mnaser | jrosser: has been putting in work, hopefully he's been getting the reviews he needs (i am getting back to OSA 'officially' today) | 16:03 |
| jrosser | yes i have been - thanks everyone | 16:03 |
| mnaser | great | 16:03 |
| jrosser | i can cover some detail there? | 16:03 |
| d34dh0r53 | o/ | 16:03 |
| mnaser | sure if there's something we need to be up to date on :) | 16:03 |
| jrosser | top line is most stuff looks ok | 16:04 |
| jrosser | rabbitmq verion needed a bump | 16:04 |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Use generic vars file for ubuntu https://review.openstack.org/586696 | 16:04 |
| jrosser | bionic uses lxc 3.x which has some config keys changed | 16:04 |
| mnaser | that'll be a tricky one | 16:04 |
| spotz | o/ | 16:04 |
| jrosser | vars/ubuntu-16.04.yml -> ubuntu.yml fixes most things | 16:04 |
| jrosser | and glance v1 api removed | 16:05 |
| jrosser | thats been it really | 16:05 |
| mnaser | yeah i think it's good, it would maybe be good to have per-distro and per-version files | 16:05 |
| mnaser | so we can override if any specific distros have weird things per version | 16:05 |
| mnaser | but yeah, thanks for your awesome work | 16:05 |
| * mnaser will be putting in some centos 7 works again soon hopefully | 16:06 | |
| mnaser | hwoarang: Leap 15 is blocked due to lxc, sudo and mariadb bugs. Will work on that after holidays (and once aio_distro_basekit is in). | 16:06 |
| evrardjp | for the key changes we can use their converter tool | 16:06 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer master: Use generic vars file for ubuntu https://review.openstack.org/586689 | 16:06 |
| odyssey4me | mnaser: well, I've asked to converge to one file where possible - so that new versions are more of a no-op if possible | 16:06 |
| odyssey4me | less code churn | 16:06 |
| jrosser | no worries :) - again thanks for the reviews everyone | 16:06 |
| mnaser | odyssey4me: ++ absolutely agree | 16:06 |
| evrardjp | jrosser: maybe we should make sure the converter tool is used for upgrades? | 16:06 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker master: Use generic vars file for ubuntu https://review.openstack.org/586708 | 16:06 |
| andymccr | yeah thats a good idea | 16:06 |
| andymccr | the converter tool looks pretty useful | 16:07 |
| odyssey4me | converter tool? | 16:07 |
| *** tonyontopic has joined #openstack-ansible | 16:07 | |
| andymccr | lxc 3.0 comes with a tool to convert old conf --> new | 16:07 |
| mnaser | we're going to have to be careful about still supporting older lxc | 16:07 |
| jrosser | so far there has only been one key thats been a problem | 16:08 |
| mnaser | for things like centos | 16:08 |
| *** shardy has quit IRC | 16:08 | |
| odyssey4me | oh, interesting - although I suspect our upgrades for in-place would be more complex - not just converting existing config | 16:08 |
| odyssey4me | in the past it's been to refresh the whole host | 16:08 |
| andymccr | i ran into 2 i think, there is one for unconfined. but it may be a good idea - e.g. deploy as is now and then run the tool against the conf (or something similar) | 16:08 |
| evrardjp | odyssey4me: well that's why I bring the idea here: before starting full heads on renewing things,maybe we should just incorporate the tool | 16:08 |
| jrosser | lxc.aa_profile -> lxc.apparmor.profile | 16:08 |
| evrardjp | andymccr: ++ | 16:09 |
| jrosser | hoever there is a massive list of aparrently deprecated keys and a bunch of them are used | 16:09 |
| jrosser | to there is follow up work to check those | 16:09 |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove master: Use generic vars file for ubuntu https://review.openstack.org/586711 | 16:09 |
| jrosser | to/so | 16:09 |
| *** vnogin has quit IRC | 16:09 | |
| andymccr | until a point where we only support lxc 3.0 + and then drop the tool? i dunno thats just an idea really | 16:09 |
| evrardjp | If the tool doesn't break the file with new keys, we can just use them both | 16:10 |
| evrardjp | I mean always use the tool | 16:11 |
| evrardjp | I don't know | 16:11 |
| mnaser | im assuming lxc3 vars break lxc2 | 16:11 |
| evrardjp | that's not what I meant | 16:11 |
| mnaser | can we actually carry *both* lxc2 and lxc3 vars? | 16:11 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Use generic vars file for ubuntu https://review.openstack.org/586707 | 16:11 |
| evrardjp | I meant, for greenfields: run the new templating and always use the tool, for upgrades: do not template, just use the tool | 16:11 |
| evrardjp | well | 16:11 |
| evrardjp | long story short, making conditionals on the version and running the tool? | 16:12 |
| evrardjp | I misexplained myself again | 16:12 |
| *** vnogin has joined #openstack-ansible | 16:12 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_gnocchi master: Use generic vars file for ubuntu https://review.openstack.org/586693 | 16:13 |
| mnaser | i dont want us to dive toooo much into this discussion, perhaps we can discus this outside meeting? | 16:13 |
| evrardjp | generate things for lxc2, run the tool to migrate to lxc3 always -- so upgrades will have no problems as things haven't changed -- but greenfield will be auto migrated | 16:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Use generic vars file for ubuntu https://review.openstack.org/586692 | 16:13 |
| evrardjp | ok | 16:13 |
| mnaser | it is a very important subject but we are a bit limited on time, got a few bugs | 16:13 |
| mnaser | hwoarang: Leap 15 is blocked due to lxc, sudo and mariadb bugs. Will work on that after holidays (and once aio_distro_basekit is in). | 16:13 |
| mnaser | i dont think hwoarang is around so that probably hasnt progressed much | 16:13 |
| mnaser | or maybe it has | 16:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic master: Use generic vars file for ubuntu https://review.openstack.org/586698 | 16:14 |
| mnaser | anyone know a bit more details? :> | 16:14 |
| mnaser | i gueeeess not :<, we'll follow up next week for this | 16:14 |
| mnaser | evrardjp: M3 was last Thursday but we were waiting for a patch that never merged: https://review.openstack.org/#/c/585720/ . AP: Refactor testing to be lighter, refactor releasing, and release without the pin. | 16:14 |
| mnaser | so looks like we're struggling to get m3 patch in | 16:15 |
| mnaser | looks like cinder timeouts :\ at least the most recent runn | 16:15 |
| mnaser | can we have more people do rechecks and pay attention to that change if possible | 16:16 |
| evrardjp | mnaser: I have taken action points, I will release without the role pins. And I am working on the refactor testing already | 16:16 |
| mnaser | ok cool | 16:16 |
| evrardjp | mnaser: I don't want any more rechecks | 16:16 |
| mnaser | that seems reasonable. | 16:16 |
| evrardjp | that's useless now | 16:16 |
| mnaser | do we want to abandon that change then? | 16:16 |
| evrardjp | more than 20 rechecks on infra timeouts | 16:16 |
| evrardjp | I am abandonning the chain of changes and will do things differently. | 16:16 |
| mnaser | ok, sounds good | 16:16 |
| evrardjp | too bad for the reproducibility of m3 | 16:16 |
| mnaser | doubt anyone is deploying m3 release :X | 16:17 |
| *** vnogin has quit IRC | 16:17 | |
| mnaser | evrardjp: Congratulations on our next PTL: mnaser! | 16:17 |
| mnaser | ^^ well thats technically not official yet :p | 16:17 |
| mnaser | but just an fyi as evrardjp put it up | 16:17 |
| mnaser | evrardjp: Congratulations on our new core: jrosser! | 16:17 |
| mnaser | congrats jrosser ! welcome and thanks for all the work you've done, look forward to bothering you for reviews :D | 16:17 |
| jrosser | :) | 16:17 |
| mnaser | #topic bug triage | 16:17 |
| *** openstack changes topic to "bug triage (Meeting topic: openstack_ansible_meeting)" | 16:18 | |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1784660 | 16:18 |
| openstack | Launchpad bug 1784660 in openstack-ansible "[WIP] Implement Neutron OVS-DPDK Support" [Undecided,New] | 16:18 |
| Tahvok | Congrats guys! | 16:18 |
| d34dh0r53 | grats! | 16:18 |
| mnaser | well thats a nice to have, do we wanna put this as like confirmed/wishlist? | 16:18 |
| mnaser | or maybe in progress and assign to james as it looks like they might be wanting to pick up the work | 16:19 |
| odyssey4me | oh jamesdenton already has patches up for it | 16:19 |
| mnaser | i assume that this is just a 'tracking' bug | 16:20 |
| odyssey4me | I expect so - wishlist/in progress | 16:20 |
| Tahvok | Is the open discussion after the bug triage? I was thinking it's the other way around.. | 16:20 |
| odyssey4me | and assign to jamesdenton | 16:20 |
| mnaser | Tahvok: it is after :> | 16:20 |
| Tahvok | mnaser: ok, thanks! | 16:20 |
| mnaser | so its in progress/wishlist/james | 16:20 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1784537 | 16:20 |
| openstack | Launchpad bug 1784537 in openstack-ansible "Resource Usage tab doesn't appear in Horizon after installing Ceilometer" [Critical,New] | 16:20 |
| mnaser | did ceilometer get its own dashboard | 16:21 |
| odyssey4me | well, it used to a long time ago - it was terrible | 16:21 |
| mnaser | yeah but it was part of horizon afaik | 16:22 |
| odyssey4me | no idea | 16:22 |
| mnaser | i cant find a ceilomter dashboard project | 16:22 |
| mnaser | but | 16:22 |
| mnaser | i think it would make sense "Resource Usage" is not visible because | 16:22 |
| mnaser | ceilometer has no api anymore | 16:22 |
| mnaser | so horizon would poll the keystone service catalog and find nothing | 16:22 |
| *** zenirc369 has quit IRC | 16:23 | |
| Tahvok | I could only find some screenshots of it on google.. | 16:23 |
| *** armaan has quit IRC | 16:23 | |
| mnaser | one sec i tihnk it was removed | 16:23 |
| d34dh0r53 | wayback, like icehouse-ish | 16:24 |
| odyssey4me | unless 'resource usage' is that pie chart thing which shows how many computes used, etc | 16:24 |
| mnaser | https://github.com/openstack/horizon/commit/20ea82b9efe78516286c4b35c5dc644b296b4313#diff-23a06074ccaed6243de4f9b416f50c5f | 16:24 |
| mnaser | https://github.com/openstack/horizon/commit/20ea82b9efe78516286c4b35c5dc644b296b4313 | 16:24 |
| mnaser | it was removed | 16:24 |
| *** suggestable has quit IRC | 16:25 | |
| mnaser | so invalid/undecided | 16:25 |
| mnaser | adding a commit with that | 16:25 |
| mnaser | err | 16:25 |
| mnaser | comment with that commit | 16:25 |
| mnaser | english is hard | 16:25 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1784369 | 16:25 |
| openstack | Launchpad bug 1784369 in openstack-ansible "Pike to queen failed to cleanup nova container" [Undecided,New] | 16:25 |
| odyssey4me | looks like it hit a transient error | 16:26 |
| odyssey4me | maybe could do with retry/until on that task | 16:27 |
| mnaser | yeah it looks like rerunning it localyl worked fine | 16:27 |
| *** hachi has quit IRC | 16:27 | |
| mnaser | confirmed/low because it's upgrade related and only rarely might happen? | 16:28 |
| *** kaiokmo has quit IRC | 16:28 | |
| odyssey4me | yeah | 16:28 |
| mnaser | cool, confirmed/low | 16:28 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1784264 | 16:28 |
| openstack | Launchpad bug 1784264 in openstack-ansible "Pike to queen upgrade failed at repo-use.yml" [Undecided,New] | 16:28 |
| odyssey4me | with a suggestion to add retries to the task, and perhaps add the low-hanging-fruit tag | 16:28 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Add missing services and collect journal logs https://review.openstack.org/587259 | 16:28 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Use generic vars file for ubuntu https://review.openstack.org/586702 | 16:28 |
| mnaser | odyssey4me: done! | 16:29 |
| mnaser | ok so for that | 16:29 |
| mnaser | i think i know what happened | 16:29 |
| odyssey4me | this one should be resolved at the head of stable/queens now - so it'd be good to know the SHA/tag they were using | 16:29 |
| mnaser | we ran into this | 16:29 |
| *** lvdombrkr has quit IRC | 16:29 | |
| mnaser | the 404 one? | 16:29 |
| mnaser | aka 1784264? | 16:29 |
| odyssey4me | yep | 16:30 |
| mnaser | we ran into it if operating systems dont match | 16:30 |
| mnaser | so like if your repo is running centos 7.4 | 16:30 |
| mnaser | but your compute is 7.5 | 16:30 |
| odyssey4me | oh, that's interesting - no idea about that | 16:30 |
| mnaser | compute tries to download centos-7.5 | 16:30 |
| mnaser | but that doesnt exist in the repo | 16:30 |
| * mnaser thinks we should use centos-7 only rather than centos-7 | 16:30 | |
| odyssey4me | that'd definitely do it | 16:30 |
| mnaser | like major release only | 16:30 |
| mnaser | the workaround is to upgrade your repo and all containers | 16:31 |
| odyssey4me | there has to be a single repo container for every distro/version/arch combo you want to serve | 16:31 |
| mnaser | right, but centos 7.4 and 7.5 is similar (i think?) | 16:31 |
| odyssey4me | so if you have a mix of target hosts, you need a mix of repo containers | 16:31 |
| mnaser | i mean the difference isnt like 18.04 and 16.04 | 16:31 |
| odyssey4me | yeah, I dunno - I'm just saying what it was designed to do | 16:31 |
| mnaser | odyssey4me: i think your build code might fix all of this too | 16:32 |
| mnaser | i think? | 16:32 |
| odyssey4me | all of that will hopefully go away soon, because that was implemented because of special packages like python-libvirt, which we no longer build | 16:32 |
| Tahvok | centos is supporting only the latest version.. I think even the deployers just want the latest... | 16:32 |
| odyssey4me | so given we no longer build them, we could revert to a simpler deployment | 16:32 |
| odyssey4me | anyway, that's a ton of work still to come in the next cycle.. so for now the reporter needs to know what you just said | 16:33 |
| *** sdake has quit IRC | 16:33 | |
| *** sdake has joined #openstack-ansible | 16:33 | |
| *** sdake has quit IRC | 16:33 | |
| *** sdake has joined #openstack-ansible | 16:33 | |
| mnaser | added a commenty | 16:33 |
| mnaser | confirmed/medium | 16:33 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1784253 | 16:34 |
| openstack | Launchpad bug 1784253 in openstack-ansible "Upgrade pike to queens failed " [Undecided,New] | 16:34 |
| mnaser | id say | 16:35 |
| mnaser | repo server is broken | 16:35 |
| mnaser | so haproxy is returning 503 | 16:35 |
| mnaser | setting to invalid and adding comment that the repo server is down so thats the root cause | 16:36 |
| evrardjp | I think there is a message coming: more ppl should work on upgrades since I don't work on those anymore :( | 16:36 |
| mnaser | upgrades worked fine for me :\ | 16:36 |
| mnaser | (all these bugs are teh same user) | 16:37 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1784230 | 16:37 |
| openstack | Launchpad bug 1784230 in openstack-ansible "Error while running setup-openstack.yml" [Undecided,New] | 16:37 |
| mnaser | i am going to guess setup-infrasttructure.yml isnt running or there is a firewall | 16:38 |
| * mnaser notices hostname of system is ceph-mon | 16:38 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Use generic vars file for ubuntu https://review.openstack.org/586690 | 16:38 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Remove dosfstools-dbg package from ubuntu installs https://review.openstack.org/587374 | 16:38 |
| mnaser | incomplete will ask for info if they ran it? | 16:38 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Use generic vars file for ubuntu https://review.openstack.org/586704 | 16:38 |
| evrardjp | just a sec | 16:38 |
| odyssey4me | well, that error tends to come back if the galera_server haproxy connecitivity is flaky/broken | 16:39 |
| evrardjp | yeah it's weirdly skipping hosts too | 16:39 |
| evrardjp | we don't know the branch, the cli call, nor if setup-infra ran fine | 16:39 |
| evrardjp | incomplete is the proper triage, but there is no message there | 16:40 |
| evrardjp | for the bug reporter | 16:40 |
| mnaser | i just added a message. | 16:40 |
| mnaser | i was waiting because of the 'just a sec' | 16:40 |
| mnaser | :) | 16:40 |
| evrardjp | ok I saw your message | 16:40 |
| evrardjp | ok | 16:40 |
| evrardjp | great | 16:40 |
| evrardjp | nex | 16:40 |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1784066 | 16:40 |
| openstack | Launchpad bug 1784066 in openstack-ansible "ceph-install.yml using older version of ceph-ansible 3.0.9" [Undecided,New] | 16:40 |
| evrardjp | next | 16:40 |
| *** kaiokmo has joined #openstack-ansible | 16:40 | |
| mnaser | are we running newer ceph-ansible in queens or | 16:41 |
| *** hakonw has quit IRC | 16:42 | |
| *** hakonw has joined #openstack-ansible | 16:42 | |
| mnaser | https://github.com/openstack/openstack-ansible/blob/stable/queens/ansible-role-requirements.yml#L185-L188 | 16:43 |
| evrardjp | well | 16:43 |
| evrardjp | if he removes the version of ceph ansible | 16:43 |
| evrardjp | sorry | 16:43 |
| evrardjp | if he removes the roles | 16:43 |
| odyssey4me | the upgrade should have something to remove the old roles | 16:43 |
| evrardjp | then "rebootstrap" ceph ansible | 16:43 |
| odyssey4me | if it doesn't, that's a bug | 16:43 |
| evrardjp | he is still with a version of ansible coming from OSA which may not be compatible with ceph-ansible | 16:44 |
| odyssey4me | 'ceph-ansible' and 'ceph_client' are correct - the others are old | 16:44 |
| evrardjp | well we never said we will support ppl randomly bumping ceph-ansible with the current code | 16:44 |
| mnaser | oh look at comments they rebootstrapped | 16:45 |
| evrardjp | exactly | 16:45 |
| evrardjp | first line is osa was 3.0.9 I needed 3.1.x so I used the roles of 3.1 and now it doesn't work ... | 16:45 |
| guilhermesp | so non-osa roles needs to be replaced in case of upgrades, right? | 16:45 |
| evrardjp | well... we can point to why it's fialing | 16:46 |
| mnaser | invalid/unsupported use case? | 16:46 |
| mnaser | bugs are being used as support :\ | 16:46 |
| Tahvok | Is there other official place for support? | 16:46 |
| Tahvok | except irc... | 16:46 |
| mnaser | mailing lists | 16:46 |
| guilhermesp | they are all from tux__ | 16:46 |
| logan- | i think it is a valid bug looking at the upgrade playbooks | 16:46 |
| logan- | https://github.com/openstack/openstack-ansible/blob/master/scripts/upgrade-utilities/playbooks/ceph-galaxy-removal.yml something like that should have been created when the role-requirements moved to use the ceph-ansible repo instead of the galaxy role breakoust for -mon, -osd, etc | 16:47 |
| evrardjp | logan-: oh | 16:47 |
| evrardjp | so https://github.com/openstack/openstack-ansible/blob/stable/queens/ansible-role-requirements.yml#L188 is not enough | 16:47 |
| logan- | right that just checks out the ceph-ansible repo, butthe old galaxy roles are still sitting higher in the roles path precedence | 16:48 |
| evrardjp | ok | 16:48 |
| evrardjp | that's the trick | 16:48 |
| evrardjp | thanks logan- ! | 16:48 |
| odyssey4me | yep, this is a typical feature implementation without considering the upgrade tooling | 16:49 |
| odyssey4me | something we should be more wary of | 16:49 |
| evrardjp | it's super hard and we should all think about it in the reviews | 16:49 |
| evrardjp | but we are all hoomans | 16:49 |
| mnaser | so status? | 16:49 |
| guilhermesp | I think that could be important for monasca role, that has at least 5 role requirements | 16:49 |
| mnaser | confirmed medium? | 16:50 |
| evrardjp | confirmed high | 16:51 |
| odyssey4me | yeah, sounds fine to me - it'd be good to reference the discussion via an eavesdrop link | 16:51 |
| mnaser | k done | 16:51 |
| evrardjp | logan-: can you push up a simple patch ? | 16:51 |
| mnaser | ill put a comment to meeting after | 16:51 |
| *** aruns__ has quit IRC | 16:51 | |
| mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1783886 | 16:51 |
| openstack | Launchpad bug 1783886 in openstack-ansible "haproxy_server: rsyslog unable to log haproxy locally" [Undecided,New] | 16:51 |
| mnaser | need to up the pace to have some open discussion time. | 16:51 |
| logan- | evrardjp ya | 16:51 |
| openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca-agent master: Add Ubuntu Bionic 18.04 support https://review.openstack.org/586933 | 16:51 |
| odyssey4me | perhaps better to just cut the bug triage off now | 16:51 |
| odyssey4me | there are always bugs, those we miss this week we can cover next | 16:52 |
| mnaser | odyssey4me: i agree | 16:52 |
| odyssey4me | we have to make time for discussion | 16:52 |
| mnaser | ++ | 16:52 |
| mnaser | lets cut it off here then | 16:52 |
| mnaser | #topic open discussion | 16:52 |
| evrardjp | ok | 16:52 |
| *** openstack changes topic to "open discussion (Meeting topic: openstack_ansible_meeting)" | 16:52 | |
| mnaser | Openvswitch configuration does not handle configuration properly on compute nodes. It should be configured with different interfaces on neutron agent container and compute hosts. | 16:52 |
| mnaser | Tahvok: ^ all yours | 16:52 |
| Tahvok | Yep | 16:52 |
| odyssey4me | well, queens/rocky have no neutron agent containers any more ;) | 16:53 |
| Tahvok | I've created some drawing for better understanding of the issue | 16:53 |
| Tahvok | odyssey4me: so... What do they have? | 16:53 |
| mnaser | they sit on baremetal afaik | 16:53 |
| odyssey4me | It goes onto the host instead. | 16:53 |
| mnaser | get deployed directly on the host | 16:53 |
| mnaser | so this only affects pike and older i assume | 16:53 |
| Tahvok | What was the reasoning to go bare metal on this? | 16:54 |
| odyssey4me | Tahvok: sorry - shouldn't have interrupted you, please continue | 16:54 |
| *** Darcidride has joined #openstack-ansible | 16:54 | |
| Tahvok | You didn't... If it's bare metal, then the interface name on controller and compute is the same - issue sovled :/ | 16:55 |
| mnaser | i think there is a way of setting variables *per host* i think? | 16:55 |
| odyssey4me | Tahvok: It moved to bare metal so that L3 doesn't go down during major upgrades. | 16:55 |
| mnaser | so you can configure vars for specific containers to work around this issue | 16:55 |
| odyssey4me | And just generally that container changes don't affect the networking layer. | 16:55 |
| evrardjp | (because we had many issues of container rebootings) | 16:55 |
| Tahvok | Unless it would be cherry picked to pike? | 16:55 |
| mnaser | i dont think we'd have such a major change get cherry picked | 16:56 |
| odyssey4me | no, that's not going to happen | 16:56 |
| mnaser | however you can run it on baremetal | 16:56 |
| mnaser | by using env.d and is_metal | 16:56 |
| evrardjp | agreed | 16:56 |
| Tahvok | Not the bare metal thing | 16:56 |
| Tahvok | The ovs configuration thing | 16:56 |
| odyssey4me | yes, if you want to you can grab the env.d file and put it into your user space | 16:56 |
| mnaser | context for ovs configuration? | 16:57 |
| odyssey4me | oh yeah - so ovs config has always been partial and incomplete, but I've never seen anyone push up a patch to fix that | 16:57 |
| Tahvok | Sec | 16:57 |
| mnaser | ovs works for us :X | 16:57 |
| mnaser | thats on queens tho | 16:57 |
| mnaser | cloudnull: we're picking up your topic shortly after if you are around / have the info :p | 16:57 |
| odyssey4me | mnaser: just became the ovs expert in OSA :p | 16:57 |
| mnaser | damnit | 16:57 |
| mnaser | :p | 16:57 |
| Tahvok | Down here: https://docs.openstack.org/openstack-ansible-os_neutron/latest/app-openvswitch.html | 16:57 |
| Tahvok | It says to configure network_interface var to set the interface | 16:58 |
| mnaser | oh but that uses openvswitch for the br-mgmt bridge, we don't use ovs for br-mgmt, only for neutron | 16:58 |
| mnaser | so i wouldn't be able to help out much as it's a different usecase :X | 16:58 |
| tux__ | guilhermesp: Yes i have created those bug issue | 16:58 |
| odyssey4me | I'd venture to say that using OVS for the container back-end is probably over complicating your life. | 16:59 |
| Tahvok | All I'm saying if the neutron agents are containerized, the interface name would differ between it and the compute hosts | 16:59 |
| odyssey4me | Use linuxbridge for OSA's things, and use OVS for the neutron things - and keep them seperate. | 16:59 |
| mnaser | and i think there is a way in osa to override variables per host/container | 16:59 |
| mnaser | that is the same issue that you might run into if you run different hardware | 16:59 |
| mnaser | your port might be eth0 somewhere and eth1 elsewhere | 17:00 |
| Tahvok | Anyway, as you have said, starting from queens, it's bare metal, so there is no interface name difference | 17:00 |
| mnaser | cloudnull: are you around? :> | 17:00 |
| Tahvok | Issue solved I guess :/ | 17:00 |
| mnaser | Tahvok: ++ sorry about that, you can run baremetal with OSA in queens | 17:00 |
| mnaser | okay, cool, well, we're over time and i think cloudnull might be afks | 17:00 |
| Tahvok | mnaser: no, there won't be such difference, as you need to create br-vlan bridge - doesn't matter what interface is under it | 17:00 |
| mnaser | Tahvok: true | 17:01 |
| mnaser | #endmeeting | 17:01 |
| *** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: https://goo.gl/tTmdgs" | 17:01 | |
| openstack | Meeting ended Tue Jul 31 17:01:08 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:01 |
| odyssey4me | well, I know a little about what cloudnull wanted to raise | 17:01 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-07-31-16.00.html | 17:01 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-07-31-16.00.txt | 17:01 |
| mnaser | oh | 17:01 |
| mnaser | oops | 17:01 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-07-31-16.00.log.html | 17:01 |
| odyssey4me | oh well | 17:01 |
| mnaser | we can still talk about it | 17:01 |
| *** vnogin has joined #openstack-ansible | 17:01 | |
| Tahvok | Thanks a lot guys! | 17:01 |
| mnaser | for the future, i will try to timebox things | 17:01 |
| mnaser | even if we cut bug triage short | 17:01 |
| mnaser | i kinda felt like we always had to finish it but yeah | 17:01 |
| odyssey4me | well, it sorta comes down to how we're doing the distro installs - basically some of the packages have different names for the services - so we end up with our unit file and their unit file competing | 17:02 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Add packages required for osprofiler https://review.openstack.org/573509 | 17:02 |
| odyssey4me | mnaser: well, possibly best to get non bug things out the way first - then do bugs for as long as people participate | 17:02 |
| odyssey4me | that way we can get commitments to chair the meeting and any other decisions out the way | 17:02 |
| mnaser | odyssey4me: yeah, i agree | 17:03 |
| mnaser | odyssey4me: we shouldn't install unit files if its source deployments? | 17:03 |
| mnaser | and service names can be variable-ized i guess? | 17:03 |
| Tahvok | Ok, gtg, was nice to talk with you all! | 17:03 |
| odyssey4me | mnaser: well, as far as I know - I haven't really looked - how hwoarang set it up is that we overwrite our unit file over the package one | 17:03 |
| mnaser | oh that's no good :\ | 17:04 |
| odyssey4me | and for centos I think there are unit file differences, and he catered to those here and there IIRC | 17:04 |
| jrosser | i have struggled with wanting different networking arrangements on my network and compute nodes | 17:04 |
| jamesdenton | odyssey4me that was/is an issue for me with OVN. Package installs in /etc/init.d | 17:04 |
| odyssey4me | I guess we need to decide whether we do package installs the way they do it and don't interfere - or we do the mix and match approach. | 17:04 |
| odyssey4me | The trouble with the distribution deployment doing things differently is that we end up having to deal with two totally different architectures. | 17:05 |
| odyssey4me | For example, to uwsgi or not to uwsgi. | 17:05 |
| mnaser | odyssey4me: yeah, i agree, each distro has their own 'way' | 17:05 |
| odyssey4me | We don't really have the time or people to deal with every distribution's style of doing things - which is why I like how hwoarang went ebout it. | 17:06 |
| *** vnogin has quit IRC | 17:06 | |
| mnaser | yeah i dont mind using uwsgi still etc | 17:06 |
| odyssey4me | This way we have a uniform deployment style regardless of whether it's source or not, and regardless of the distro. | 17:06 |
| mnaser | but things like unit files.. i feel like we should stick to upstream | 17:06 |
| mnaser | oh wait | 17:06 |
| mnaser | but the upstream one doesnt have uwsgi | 17:06 |
| mnaser | ok, i get it | 17:06 |
| odyssey4me | yep | 17:06 |
| odyssey4me | so, to solve the immediate problem we'd have to do an audit and implement something to cater for name differences | 17:07 |
| mnaser | yeah | 17:07 |
| *** gyee has joined #openstack-ansible | 17:07 | |
| mnaser | btw, eyes on https://review.openstack.org/#/c/583745/ and pathc above would be nice, fix to make magnum work | 17:07 |
| cloudnull | ohai, im back now. sorry. | 17:08 |
| odyssey4me | we'd have to get more detail from cloudnull about where he saw it | 17:08 |
| mnaser | odyssey4me: pastebin shows designate | 17:08 |
| odyssey4me | mnaser: eh? magnum needs all that stuff duplicated? | 17:09 |
| *** vnogin has joined #openstack-ansible | 17:09 | |
| cloudnull | what odyssey4me said, sums up the issue at hand | 17:10 |
| *** Darcidride has quit IRC | 17:10 | |
| mnaser | odyssey4me: yes, keystone_authtoken is used to authenticate incoming requests, keystone_auth is used to talk to the actual apis | 17:10 |
| * odyssey4me is getting lost in all this nonsense | 17:10 | |
| mnaser | yep | 17:10 |
| cloudnull | example https://review.openstack.org/#/c/587196/7/vars/ubuntu-16.04.yml@43 | 17:10 |
| mnaser | it's really confusing | 17:10 |
| odyssey4me | cloudnull: I vaguely recall some of hwoarang's patches catering to different service names | 17:10 |
| cloudnull | for designate I am masking conflicting units | 17:11 |
| odyssey4me | perhaps for centos especially, but I didn't hunt for long enough | 17:11 |
| cloudnull | some distros bundle services, some done, some rename them from what they are upstream, some are just doing <WhateverTheyWant> | 17:11 |
| cloudnull | a lot of the distro package installs will work on a greenfield deployment | 17:12 |
| cloudnull | but wont survive a reboot | 17:12 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_almanach master: Switch to using project-templates https://review.openstack.org/587531 | 17:12 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_congress master: Switch to using project-templates https://review.openstack.org/587532 | 17:12 |
| cloudnull | or a container restart for that matter. | 17:12 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_monasca master: Switch to using project-templates https://review.openstack.org/587533 | 17:12 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_watcher master: Switch to using project-templates https://review.openstack.org/587534 | 17:12 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_zaqar master: Switch to using project-templates https://review.openstack.org/587535 | 17:12 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_zun master: Switch to using project-templates https://review.openstack.org/587536 | 17:12 |
| odyssey4me | that's a really good find cloudnull | 17:13 |
| cloudnull | so we need to either map conflicting units and mask them, like i was trying on designate, or create lists of what the units are supposed to be for the given distro packages. | 17:13 |
| odyssey4me | I wonder if it may be worth a bug report to associate any patches to, and perhaps hwoarang can help sort it out when he returns. | 17:14 |
| cloudnull | masking the conflicts seems like the best solution IMHO. creating a service map for specific distros seems like it's going to be a nightmare | 17:14 |
| cloudnull | however IDK ? | 17:14 |
| odyssey4me | how would you mask the conflicts without effectively doing all the mappings? | 17:15 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_searchlight master: Switch to using project-templates https://review.openstack.org/587541 | 17:15 |
| cloudnull | we'd let our service units run as expected and mask anything outside of that | 17:15 |
| odyssey4me | masked services would work, but it'd be a bit confusing to have these arbitrary masked things left behind | 17:16 |
| mnaser | yeah i wouldn't feel great masking things | 17:16 |
| mnaser | i'd rather do mappings | 17:16 |
| odyssey4me | mapping them is nice but then we have the inconsistencies between distributions | 17:16 |
| mnaser | well i feel like that would be a one time pain | 17:16 |
| odyssey4me | I think mapping would be better, but more painful obviously. | 17:16 |
| mnaser | ++ | 17:17 |
| cloudnull | creating the distro map would effectively be creating https://github.com/openstack/openstack-ansible-os_cinder/blob/master/defaults/main.yml#L294-L323 for each distro + source | 17:17 |
| mnaser | i mean we already her per-os var files | 17:17 |
| mnaser | cloudnull: im thinking only service_name is the one that has to be changed | 17:17 |
| mnaser | so maybe like | 17:17 |
| mnaser | cinder_scheduler_service_name in each os var file, and then that's added in the defaults file | 17:17 |
| cloudnull | sadaly its not quite that easy, some distros dont deploy all of the service binaries . | 17:18 |
| cloudnull | like ubuntu has designate-agent, and that's not a real thing | 17:18 |
| mnaser | could we leverage `condition` in there? | 17:18 |
| cloudnull | we could. | 17:18 |
| mnaser | it's not ideal.. but yeah :\ | 17:18 |
| mnaser | i cant think of a better way | 17:18 |
| mnaser | we work on standarizing apis | 17:19 |
| mnaser | but distros make our lives painful | 17:19 |
| mnaser | :'( | 17:19 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_nova master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584450 | 17:19 |
| mnaser | cloudnull: https://review.openstack.org/#/c/583745/2 if you don't mind kick this out.. and i'll get hacking on os_cinder to fix for centos | 17:19 |
| * cloudnull links to my talk about how I hate distros and chef. :) | 17:19 | |
| cloudnull | done. | 17:20 |
| mnaser | thank you sir | 17:20 |
| mnaser | so in source based installs | 17:20 |
| mnaser | i am failing with "cinder-rtstool is not installed correctly: OSError: [Errno 2] No such file or directory" in my os_cinder change | 17:20 |
| mnaser | does anyone know much about something where stuff doesnt appear in $PATH | 17:20 |
| *** aruns__ has joined #openstack-ansible | 17:21 | |
| odyssey4me | mnaser: where's your review? | 17:21 |
| mnaser | https://review.openstack.org/#/c/584604/ | 17:21 |
| cloudnull | so re: distro - I don't really know what to do to resolve the issue. folks should think about it though. if we leave things the way they are, come rocky, users that want to leverage this capability are going to have a miserable time (in prod). | 17:21 |
| mnaser | cloudnull: 100%. i intend to try and play catch up with this before end of cycle .. or we can put distro installs as a 'preview' or so | 17:22 |
| odyssey4me | cloudnull: could you register a bug for it so that next week when hwoarang is back he can pick it up | 17:22 |
| mnaser | to set expectations | 17:22 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_nova master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584450 | 17:22 |
| cloudnull | ++ | 17:22 |
| cloudnull | I'm happy to help get things done and have an environment(s) to test with. I know folks need or want distro installs so I'm excited to see it being worked on, even if I will only ever install from source :) | 17:23 |
| *** ppetit has quit IRC | 17:23 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: [WIP] Do no use tempest https://review.openstack.org/587544 | 17:24 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_glance master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584496 | 17:25 |
| guilhermesp | ppl I'm gonna get this fruit https://bugs.launchpad.net/openstack-ansible/+bug/1784369 | 17:25 |
| openstack | Launchpad bug 1784369 in openstack-ansible "Pike to queen failed to cleanup nova container" [Low,Confirmed] | 17:25 |
| odyssey4me | mnaser: so, cinder-rtstool appears to be used via rootwrap, which gets its path from https://github.com/openstack/openstack-ansible-os_cinder/blob/master/templates/rootwrap.conf.j2#L13 | 17:26 |
| odyssey4me | guilhermesp: thanks for helping out! | 17:26 |
| guilhermesp | basically add a retry in the task. But I'm wondering, this script is only at pike | 17:26 |
| guilhermesp | so should I submitt a patch directly or we need to backport (I cant see how backport) | 17:26 |
| odyssey4me | guilhermesp: yes, so no master patch - straight to that one because it only applies to that upgrade | 17:26 |
| guilhermesp | right odyssey4me thanks | 17:27 |
| *** DanyC has quit IRC | 17:27 | |
| mnaser | odyssey4me: so i think it should be there.. i assume | 17:28 |
| *** DanyC has joined #openstack-ansible | 17:28 | |
| mnaser | https://github.com/openstack/cinder/blob/master/setup.cfg#L68 | 17:28 |
| mnaser | it should be installed the same way other things are installed there | 17:28 |
| odyssey4me | mnaser: http://logs.openstack.org/04/584604/1/check/openstack-ansible-functional-centos-7/6f4abed/logs/etc/host/cinder/rootwrap.conf.txt.gz | 17:28 |
| odyssey4me | it's configured right | 17:29 |
| mnaser | (im running it locally as we speak) | 17:29 |
| *** vnogin has quit IRC | 17:29 | |
| mnaser | http://logs.openstack.org/04/584604/1/check/openstack-ansible-functional-centos-7/6f4abed/logs/etc/host/cinder/rootwrap.d/volume.filters.txt.gz the filter is there too | 17:29 |
| odyssey4me | yeah, so it's almost like it just wasn't installed - which seems odd | 17:30 |
| mnaser | fwiw i think this is the last issue to get centos jobs to pass | 17:31 |
| mnaser | :> | 17:31 |
| odyssey4me | perhaps it's not the rtstool that's not found, but something it uses | 17:31 |
| mnaser | thats what im suspecting | 17:31 |
| mnaser | like maybe the OSError is actually raised inside the rtstool | 17:31 |
| mnaser | like thats actual stderr output | 17:31 |
| odyssey4me | yep | 17:31 |
| odyssey4me | you have a build going to figure it out, right? | 17:32 |
| mnaser | yeah im running it locally right now to see whats going on | 17:32 |
| *** DanyC has quit IRC | 17:32 | |
| odyssey4me | ok, then time for me to check whether I broke the world with my patches | 17:32 |
| mnaser | D: | 17:33 |
| *** armaan has joined #openstack-ansible | 17:34 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582354 | 17:37 |
| cloudnull | s/odyssey4me/mhayden/g ? | 17:38 |
| cloudnull | :P | 17:38 |
| *** cshen has joined #openstack-ansible | 17:39 | |
| odyssey4me | :D | 17:39 |
| mnaser | i really wish we could speed up ansible | 17:41 |
| mnaser | :< | 17:41 |
| bgmccollum | export SPEED_MULTIPLIER=10 | 17:42 |
| mnaser | right | 17:42 |
| mnaser | i forgot about that | 17:42 |
| *** vnogin has joined #openstack-ansible | 17:44 | |
| cloudnull | mnaser have you given mitogen a try ? | 17:44 |
| mnaser | nope | 17:44 |
| cloudnull | its worth a try :) | 17:44 |
| mnaser | https://mitogen.readthedocs.io/en/stable/ ? | 17:44 |
| cloudnull | that's the one. | 17:44 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible stable/queens: Add retries to collect ID's for nova services https://review.openstack.org/587563 | 17:45 |
| *** tonyontopic has quit IRC | 17:47 | |
| odyssey4me | could I get another set of eyes on this patch please, other than cloudnull: https://review.openstack.org/587395 | 17:48 |
| lbragstad | does osa have a set of scripts for fixing the networking on a new 16.04 install? | 17:49 |
| cloudnull | mnaser we can selectively add mitogen for host based playbooks and watch magic happen. also i experimented with it by subclassing the strategy plugin so that it worked with our container connection plugin. it wokred, ok, but would need some more love. | 17:49 |
| * cloudnull goes to see if i can find the things I was hacking on . | 17:50 | |
| *** priteau has quit IRC | 17:50 | |
| *** hachi has joined #openstack-ansible | 17:50 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_magnum master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582296 | 17:52 |
| cloudnull | mnaser http://paste.openstack.org/show/726967/ | 17:52 |
| cloudnull | it might be something worth playing with . | 17:52 |
| cloudnull | i had it +/- working, but there are some edge cases. | 17:53 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_magnum master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582296 | 17:53 |
| mnaser | i wonder how much faster it is though | 17:54 |
| cloudnull | I think adding somthing like mitogen to our available strategies and including it on playbooks where we know we only target hosts would do wonders. | 17:54 |
| cloudnull | It's a lot faster. | 17:55 |
| *** tonyontopic has joined #openstack-ansible | 17:55 | |
| *** vnogin has quit IRC | 17:55 | |
| mnaser | and if we can leverage it's whole concept of trees | 17:55 |
| mnaser | and have the host call out to containers | 17:55 |
| cloudnull | yea I was just using the basic setup | 17:55 |
| cloudnull | something worth exploring maybe. | 17:55 |
| cloudnull | if folks have time to tinker. | 17:55 |
| mnaser | "Expect a 1.25x - 7x speedup and a CPU usage reduction of at least 2x" | 17:56 |
| mnaser | ugh i really wish i had time for tihs | 17:56 |
| mnaser | haha | 17:56 |
| cloudnull | me too :) | 17:56 |
| mnaser | but i really gotta get centos passign | 17:56 |
| *** jwitko_ has quit IRC | 17:56 | |
| cloudnull | maybe something we can hack-on at the PTG | 17:56 |
| *** jwitko_ has joined #openstack-ansible | 17:57 | |
| mnaser | lol @ one of the testimonials | 17:57 |
| mnaser | “The runtime was reduced from 1.5 hours on 4 servers to just under 3 minutes. Thanks!” | 17:57 |
| *** DanyC has joined #openstack-ansible | 17:57 | |
| mnaser | 1.5 hour ansible run on 4 server :X | 17:57 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cinder master: Convert role to use a common systemd service role https://review.openstack.org/574817 | 17:58 |
| mnaser | https://mitogen.readthedocs.io/en/stable/ansible.html#connection-delegation | 17:58 |
| mnaser | !!! | 17:58 |
| openstack | mnaser: Error: "!!" is not a valid command. | 17:58 |
| mnaser | To enable connection delegation, set mitogen_via=<inventory name> on the command line, or as host and group variables. | 17:58 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_ceilometer master: Add systemd journal logging to the service config https://review.openstack.org/587170 | 17:59 |
| cloudnull | fixing merge conflicts ^ | 17:59 |
| mnaser | does that mean we literally can drop our connection plugin and use mitogen_via only | 17:59 |
| cloudnull | potentially, yes | 17:59 |
| *** mattinator has joined #openstack-ansible | 18:00 | |
| mnaser | https://mitogen.readthedocs.io/en/stable/ansible.html#method-lxc | 18:00 |
| cloudnull | we'd need to probably carry a patch for some of the file transfer bits, but it'd be close | 18:00 |
| mnaser | it literally already exists | 18:00 |
| mnaser | ok so we use the ssh connection plugin in our own stuff to manage containers | 18:01 |
| mnaser | and guess who's going to lose productivity right now implementing this | 18:01 |
| mnaser | <== | 18:01 |
| cloudnull | hahaha | 18:01 |
| mnaser | if this works well it might be *really* easy to push up a patch and change the way it works | 18:01 |
| mnaser | it would just be a matter of fixing up inventory. | 18:01 |
| cloudnull | sorry for putting you on a tangent :) | 18:02 |
| mnaser | physical_host becomes becomes mitogen_via | 18:02 |
| cloudnull | ++ | 18:02 |
| mnaser | connection becomes lxc | 18:02 |
| mnaser | and thats it?! | 18:02 |
| *** DanyC has quit IRC | 18:02 | |
| mnaser | i totally have time while i wait for cinder to go up | 18:02 |
| * cloudnull runs off to another meeting. | 18:02 | |
| cloudnull | bbl | 18:02 |
| *** cshen has quit IRC | 18:02 | |
| *** electrofelix has quit IRC | 18:07 | |
| *** mmercer has joined #openstack-ansible | 18:11 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_server master: Remove repo_requires_pip_packages and pip_install meta-dep https://review.openstack.org/587575 | 18:12 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_build master: Remove pip_install meta-dep https://review.openstack.org/587577 | 18:16 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_server master: Remove repo_requires_pip_packages and pip_install meta-dep https://review.openstack.org/587575 | 18:16 |
| *** poopcat has joined #openstack-ansible | 18:17 | |
| mnaser | cloudnull: weird, just ran int against a 100+ node inventory and i didnt see that much of a speed up :X | 18:19 |
| mnaser | maybe i should try delegating out of a remote host | 18:21 |
| mnaser | odyssey4me: /openstack/venvs/cinder-testing/bin/cinder-rtstool exists perfectly normally | 18:23 |
| mnaser | >:( | 18:23 |
| evrardjp | mnaser: cloudnull I have a poc for mitogen | 18:23 |
| mnaser | be fun to hack on that | 18:24 |
| evrardjp | just fyi | 18:24 |
| evrardjp | I think it's not the most urgent thing we have to do | 18:24 |
| mnaser | of course | 18:24 |
| mnaser | unless it speeds up deploys/ci | 18:24 |
| mnaser | that would be a nice thing to have | 18:24 |
| odyssey4me | given the smallness of CI, I don't think it'll make any difference | 18:24 |
| *** cshen has joined #openstack-ansible | 18:25 | |
| odyssey4me | but meh, would be nice to see | 18:25 |
| mnaser | so apparently its useful as in it helps jobs run faster rather than scale more, but idk | 18:25 |
| mnaser | i just ran it and didnt see a speed up so not sure | 18:25 |
| odyssey4me | folks, https://github.com/openstack/molteniron appears to be quite dormant - any objections to pulling it out of the integrated build? | 18:25 |
| odyssey4me | it's not an official project - we probably shouldn't have ever let it get into the integrated build in the first place | 18:26 |
| mnaser | yeah if its not an official project i dont think it should be part of integrated :X | 18:26 |
| odyssey4me | evrardjp: any effect on our releasing/deliverables/whatever? | 18:27 |
| mnaser | i think cinder-rtstool is not being executed with rootwrap. | 18:28 |
| *** armaan has quit IRC | 18:28 | |
| mnaser | http://paste.openstack.org/show/726970/ | 18:28 |
| *** armaan has joined #openstack-ansible | 18:28 | |
| mnaser | ugh truncated 1 sec | 18:28 |
| mnaser | http://paste.openstack.org/show/726971/ | 18:29 |
| evrardjp | mnaser: I started by working on finding out what would win from mitogen, basically the with_items loop | 18:29 |
| evrardjp | it's not that much | 18:29 |
| evrardjp | we are quite efficient by ourselves | 18:29 |
| evrardjp | I have an idea on how to be very fast though | 18:29 |
| evrardjp | but it means multi-processing, locks, and strategy plugins. | 18:30 |
| mnaser | sounds like something that should land upstream ansible | 18:30 |
| evrardjp | odyssey4me: what do you mean by removing it | 18:30 |
| evrardjp | I don't tag it | 18:30 |
| odyssey4me | evrardjp: take the play and the repo pin out of the integrated build | 18:30 |
| odyssey4me | and the a-r-r ref | 18:31 |
| evrardjp | fine for me. | 18:31 |
| evrardjp | send a message to the ML | 18:31 |
| *** cshen has quit IRC | 18:31 | |
| evrardjp | I think it would be a good idea to make it self-contained | 18:31 |
| evrardjp | and applicable | 18:31 |
| evrardjp | well | 18:31 |
| evrardjp | let's not talk about these | 18:31 |
| evrardjp | : ) | 18:31 |
| tux__ | Folks i have question, I have testing ceph cluster running, now i want to flip all OSD to lvm scenario, there is no data on cluster yet. what would be the best way to do this? | 18:32 |
| *** cshen has joined #openstack-ansible | 18:33 | |
| evrardjp | so... is there someone checking at http://logs.openstack.org/periodic/git.openstack.org/openstack/openstack-ansible/master/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/668db2d/job-output.txt.gz#_2018-07-31_07_43_23_285047 ? | 18:34 |
| evrardjp | else there is no use of tagging m3 on the latest merge commit | 18:34 |
| mnaser | odyssey4me: uncovered the issue, cinder-rtstool verify runs without root so it doesnt use rootwrap, and it looks like $PATH does not include /openstack/venvs/cinder-testing/bin and thats why it fails | 18:36 |
| mnaser | so i think we need to update the systemd to include the path | 18:36 |
| odyssey4me | mnaser: hmm, I think cloudnullhas a patch up to make it systemd all the things which may solve that | 18:36 |
| *** Bhujay has joined #openstack-ansible | 18:38 | |
| evrardjp | cloudnull: odyssey4me mnaser where is this patch? | 18:38 |
| mnaser | https://review.openstack.org/#/c/574817/ | 18:38 |
| mnaser | does the systemd stuff actually add the correct $PATH | 18:38 |
| odyssey4me | https://review.openstack.org/574817 | 18:38 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_cinder master: Add target_helper to backend_defaults https://review.openstack.org/584604 | 18:40 |
| savvas | hey guys , does anyone know how to force Ceph Ansible to rerun the OSD part as part of deployment? | 18:43 |
| evrardjp | it looks very much like the first patch who broke the integrated. No change in the commit message to explain why things are done, and new features included in the patch. | 18:43 |
| savvas | It wouldn't configure the OSD on one node on its first run so I figured I'd empty the partition table and remove the ceph files to try and see if the playbook would rerun it, but it doesn't | 18:43 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Remove the molteniron service from the integrated build https://review.openstack.org/587586 | 18:43 |
| kaiokmo | guys, I have a quick question: which behavior/function do openstack-ansible-role-jobs template encapsulate? | 18:45 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_molteniron master: Move playbooks/config files into role examples https://review.openstack.org/587587 | 18:45 |
| evrardjp | kaiokmo: you can see the definitions in test | 18:45 |
| evrardjp | https://github.com/openstack/openstack-ansible-tests/blob/master/zuul.d/project-templates.yaml | 18:45 |
| kaiokmo | thank you evrardjp | 18:46 |
| evrardjp | yw | 18:46 |
| kaiokmo | I did find some info at https://docs.openstack.org/infra/openstack-zuul-jobs/project-templates.html | 18:46 |
| kaiokmo | now I know that I was looking at the wrong place :) | 18:46 |
| *** Bhujay has quit IRC | 18:48 | |
| cloudnull | evrardjp what are you on about here https://review.openstack.org/#/c/574817 ? | 18:50 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_rally master: Remove rally_requires_pip_packages https://review.openstack.org/587588 | 18:50 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Use ansible module instead of command https://review.openstack.org/587589 | 18:50 |
| cloudnull | I already got the keystone patch merged as the canary this is implementing the accepted pattern ? | 18:50 |
| kaiokmo | odyssey4me: does it make sense to switch to project-templates here as well? https://review.openstack.org/#/c/586933/ | 18:51 |
| cloudnull | this patch is simply adding the parts we missed in the first pass, which should've included journal logging. | 18:51 |
| odyssey4me | kaiokmo: can do, or can do on top of that as a different patch | 18:52 |
| cloudnull | if you're saying i should submit a pr to first ensure we have no logging to correct it later, i'd say no. | 18:53 |
| evrardjp | cloudnull: yes it's a new feature. | 18:53 |
| cloudnull | no. | 18:53 |
| *** DanyC has joined #openstack-ansible | 18:53 | |
| evrardjp | all features should be documented | 18:53 |
| evrardjp | it's not a 1->1 patch | 18:53 |
| *** cshen has quit IRC | 18:53 | |
| evrardjp | it's not simply moving to an equivalent | 18:54 |
| evrardjp | nothing is documented there | 18:54 |
| evrardjp | you add packages with no explanations | 18:54 |
| evrardjp | if other patches were wrongly merged in the past... well, that's the past, I don't see why we shouldn't improve | 18:55 |
| *** vnogin has joined #openstack-ansible | 18:55 | |
| kaiokmo | odyssey4me: ok. I prefer to add this on another patch, since it's more related to jrosser's work on bionic | 18:55 |
| *** DanyC has quit IRC | 18:55 | |
| cloudnull | so then you want me to make sure all of the services have no ability to collect logs, then go back and fix it? | 18:55 |
| *** DanyC has joined #openstack-ansible | 18:56 | |
| *** weezhard has quit IRC | 18:57 | |
| mnaser | evrardjp: ^ | 18:58 |
| *** cshen has joined #openstack-ansible | 18:58 | |
| mnaser | also, i'm kinda torn between sticking to upstream nginx or epel nginx | 18:58 |
| mnaser | what is everyone's thoughts about this? | 18:58 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: Ensure virtualenv is installed on centos https://review.openstack.org/587598 | 18:59 |
| *** DanyC has quit IRC | 19:00 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: Ensure virtualenv is installed on centos https://review.openstack.org/587598 | 19:01 |
| evrardjp | cloudnull: you're being silly there | 19:01 |
| odyssey4me | mnaser: you have a centos box up and running, right? | 19:01 |
| evrardjp | cloudnull: just explain why | 19:01 |
| mnaser | odyssey4me: yeah | 19:01 |
| odyssey4me | could you check to see what the version of python-virtualenv and python-pip is that's available from your repositories? | 19:02 |
| mnaser | cause right now i think its failing to install in the containerized CI because it's trying to pull the nginx gpg files | 19:02 |
| evrardjp | cloudnull: right now I don't know where does this come from, so I can't imagine 1 year from now. | 19:02 |
| mnaser | odyssey4me: sure one moemnt | 19:02 |
| evrardjp | odyssey4me: python-pip is not easily available on centos | 19:02 |
| evrardjp | virtualenv is | 19:02 |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/587395 | 19:02 |
| mnaser | it is via rdo release | 19:02 |
| mnaser | which we deploy | 19:02 |
| evrardjp | mnaser: virtualenv is even from centos repos, right? | 19:02 |
| mnaser | i think rdo publishes updated versions for both | 19:03 |
| evrardjp | I mean non rdo | 19:03 |
| evrardjp | ok | 19:03 |
| evrardjp | that's great | 19:03 |
| odyssey4me | we're only concerned about what rdo provides afaik | 19:03 |
| evrardjp | I thought pip was not in RDO | 19:03 |
| evrardjp | but was in software collections | 19:03 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Use keystone_auth for credentials https://review.openstack.org/583745 | 19:04 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Disable sending metrics for clusters https://review.openstack.org/583751 | 19:04 |
| mnaser | python-pip-8.1.2-1.el7.noarch and looks like virtualenv is from upstream as evrardjp said | 19:04 |
| evrardjp | mnaser: that's not too bad 8.1.2 | 19:04 |
| evrardjp | I will install it on the nodes then | 19:05 |
| odyssey4me | ok, those versions look good evrardjp - I think we can add both packages and inally be done with get-pip.py :) | 19:05 |
| *** mattinator has quit IRC | 19:05 | |
| mnaser | they're also tested and integrated inside rdo | 19:05 |
| odyssey4me | mnaser: virtualenv is 15, right? | 19:05 |
| mnaser | which means they should work | 19:05 |
| mnaser | python-virtualenv.noarch 15.1.0-2.el7 base | 19:05 |
| odyssey4me | fantastic | 19:05 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: Ensure virtualenv/pip are installed on centos https://review.openstack.org/587598 | 19:06 |
| evrardjp | odyssey4me: mnaser ^ | 19:06 |
| mnaser | does this mean we get rid of pip_install | 19:06 |
| evrardjp | yes. | 19:06 |
| evrardjp | finally! | 19:06 |
| odyssey4me | well, not exactly | 19:06 |
| evrardjp | well | 19:06 |
| evrardjp | we are closer! | 19:06 |
| evrardjp | :) | 19:06 |
| odyssey4me | but we can now change it to only drop the config file | 19:06 |
| evrardjp | odyssey4me: that's until we won't need the config file anymore. | 19:07 |
| evrardjp | :D | 19:07 |
| odyssey4me | evrardjp: yep, that's next cycle | 19:07 |
| evrardjp | meh we are closer everyday | 19:07 |
| evrardjp | we can do it | 19:07 |
| *** aruns__ has quit IRC | 19:08 | |
| mnaser | so | 19:08 |
| mnaser | nginx from epel or upstream | 19:08 |
| mnaser | thoughts? | 19:08 |
| evrardjp | upstream | 19:08 |
| evrardjp | wait | 19:08 |
| evrardjp | no | 19:08 |
| *** hachi has quit IRC | 19:08 | |
| evrardjp | I don't care in fact. | 19:08 |
| odyssey4me | doesn't epel have a broader test group? | 19:08 |
| evrardjp | up to you | 19:08 |
| evrardjp | odyssey4me: yes but versions fluctuate | 19:08 |
| mnaser | it probably moves less but that means we cant ensure the same version across distros | 19:09 |
| odyssey4me | better or worse than from nginx themselves? | 19:09 |
| mnaser | if we go upstream we can just deploy the same version all across | 19:09 |
| odyssey4me | I am definitely against using upstream across all distributions. | 19:09 |
| odyssey4me | We have enough moving parts - I'd like to reduce them, not increase them. | 19:10 |
| mnaser | but you feel that having diffferent nginx versions across distros is an easier problem to solve than having an extra repo we pull things from? | 19:10 |
| mnaser | it's the tradeoff | 19:10 |
| odyssey4me | yeah, we only use very basic capabilities anyway - so we don't need the latest of all the things | 19:11 |
| mnaser | fair to me | 19:11 |
| odyssey4me | We could also probably just as easily just switch to using Apache across the board and stop this dichotomy. | 19:11 |
| mnaser | i mean that's totally what upstream does and tests afaik | 19:12 |
| odyssey4me | We switched to using nginx due to upstream recommendations. | 19:12 |
| mnaser | oh | 19:12 |
| odyssey4me | We were implored to switch to nginx/uwsgi because it was so much better. | 19:12 |
| odyssey4me | But given we're using nginx as a reverse proxy, it could just as well be Apache. | 19:13 |
| odyssey4me | nginx, I believe, is more lightweight | 19:13 |
| mnaser | it is | 19:13 |
| mnaser | odyssey4me: hell, why not use uwsgi built in http server :X | 19:13 |
| odyssey4me | we actually wanted to move to using the uwsgi fastrouter instead of nginx/apache anywhere we could | 19:14 |
| odyssey4me | there's a spec up for that, and it totally makes sense to do | 19:14 |
| odyssey4me | it's only keystone federation that keeps us using apache for keystone | 19:14 |
| odyssey4me | anywhere else we use nginx/apache is changable as far as I know | 19:15 |
| odyssey4me | unfortuately we've not had time to get that all done | 19:15 |
| mnaser | im kinda tempted to drop nginx and switch to uwsgi http server to pave the way for fastrouter eventually odyssey4me :X | 19:16 |
| mnaser | one less thing to run. | 19:16 |
| odyssey4me | a nice thing about nginx over apache is that it logs to the systemd journal directly... the apache versions we have access to do not support that yet | 19:16 |
| *** mattinator has joined #openstack-ansible | 19:16 | |
| odyssey4me | mnaser: well, we don't use nginx in very many places | 19:17 |
| mnaser | well for keystone at least | 19:17 |
| mnaser | clarification | 19:17 |
| mnaser | keystone *without* federation | 19:17 |
| odyssey4me | repo_server (requires a web server of some ind), os_keystone (requires a web server for federation) | 19:17 |
| odyssey4me | I think zaqar uses a web server too. | 19:17 |
| odyssey4me | Any other places are probably just legacy. | 19:18 |
| mnaser | https://uwsgi-docs.readthedocs.io/en/latest/StaticFiles.html | 19:18 |
| mnaser | :X | 19:18 |
| mnaser | so that can be reused for repo_server | 19:19 |
| mnaser | os_keystone will probably still need apache for federation | 19:19 |
| openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca master: Add Ubuntu Bionic 18.04 support https://review.openstack.org/586932 | 19:20 |
| odyssey4me | yeh, so I'd suggest that we discuss (at the PTG) standardising on apache then for any web server needs, and uwsgi fastrouter for any services | 19:20 |
| odyssey4me | that makes cross-distro work easier and cuts tech debt | 19:21 |
| *** mattinator has quit IRC | 19:21 | |
| mnaser | odyssey4me: to get centos to pass, i need to figure out what to do with nginx.. i will push up a patch that uses uwsgi as an http server (which will actually pave the way to fastrouter i guess) | 19:22 |
| odyssey4me | is this for keystone? | 19:22 |
| mnaser | yeah | 19:22 |
| mnaser | also | 19:23 |
| mnaser | lol | 19:23 |
| mnaser | "TODO (odyssey4me): Convert the SP implementation to use nginx instead so that we do not have to be concerned with multiple web servers." | 19:23 |
| mnaser | :P | 19:23 |
| mnaser | looks like you've been there :P | 19:23 |
| odyssey4me | yeah, I did the work to switch it to nginx - and it converted it from apache/mod_wsgi to nginx/uwsgi without dropping an auth request | 19:24 |
| mnaser | thats awesome | 19:24 |
| odyssey4me | well, I guess it could convert from nginx -> apache in the same way | 19:25 |
| odyssey4me | uwsgi's already done | 19:25 |
| odyssey4me | hmm, you know - you can set the web server with a var | 19:25 |
| odyssey4me | https://github.com/openstack/openstack-ansible-os_keystone/blob/master/defaults/main.yml#L188 | 19:26 |
| odyssey4me | just change that conditional to also set 'apache' if it's centos | 19:26 |
| odyssey4me | job done | 19:26 |
| *** hachi has joined #openstack-ansible | 19:26 | |
| mnaser | thats if you wanna run apache :P | 19:27 |
| savvas | he guys, just pasting this again hoping somebody knows the answer to this: does anyone know how to force Ceph Ansible to rerun the OSD part as part of deployment? It wouldn't configure the OSD on one node on its first run so I figured I'd empty the partition table and remove the ceph files to try and see if the playbook would rerun it, but it doesn't | 19:27 |
| odyssey4me | well, that's a simple way to get this done for rocky without creating too much work for ourselves | 19:27 |
| mnaser | try ceph-disk zap | 19:27 |
| *** canori01 has quit IRC | 19:28 | |
| odyssey4me | next cycle we'll have plenty of time to change things up in the role | 19:28 |
| mnaser | i just dont wanna have our infra switch from nginx and back to uwsgi back and forth | 19:28 |
| mnaser | so thats why i don't mind taking on the work *if* i can present it in a clean way | 19:28 |
| mnaser | so not a pain to review and very clear on what's going on | 19:29 |
| * mnaser has been trying to get centos up for pretty much the entire cycle :( | 19:29 | |
| odyssey4me | well, take a crack at it if you want to - I was just giving you an easy way out to get centos passing ;) | 19:29 |
| mnaser | aha :) | 19:29 |
| odyssey4me | no nginx, means no epel - happy days | 19:29 |
| odyssey4me | oh, I think we still have it for lsync/nginx in the repo server | 19:30 |
| odyssey4me | but that's a small price to pay | 19:30 |
| openstackgerrit | Merged openstack/openstack-ansible-os_horizon master: Reenable designate dashboard for openSUSE https://review.openstack.org/585763 | 19:31 |
| *** gillesMo has quit IRC | 19:38 | |
| *** KeithMnemonic1 has joined #openstack-ansible | 19:39 | |
| *** KeithMnemonic has quit IRC | 19:40 | |
| *** KeithMnemonic2 has joined #openstack-ansible | 19:42 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_congress master: Convert role to use a common systemd service role https://review.openstack.org/587191 | 19:43 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install master: Use distribution packages for pip https://review.openstack.org/587609 | 19:43 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_barbican master: Convert role to use a common systemd service role https://review.openstack.org/587101 | 19:44 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install master: Use distribution packages for pip https://review.openstack.org/587609 | 19:44 |
| *** KeithMnemonic1 has quit IRC | 19:44 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_aodh master: Add systemd journal logging to the service config https://review.openstack.org/587074 | 19:45 |
| *** epalper has joined #openstack-ansible | 19:45 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_designate master: Convert role to use a common systemd service role https://review.openstack.org/587196 | 19:47 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Update all SHAs for 16.0.17 https://review.openstack.org/584812 | 19:47 |
| tux__ | In Openstack can i put policy like only 2 VM will get created most on compute node ? | 19:48 |
| tux__ | I want to create only 2 instance per compute node | 19:48 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_blazar master: Convert role to use a common systemd service role https://review.openstack.org/587114 | 19:48 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_keystone master: Drop Admin API from Keystone https://review.openstack.org/587611 | 19:50 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cloudkitty master: Convert role to use a common systemd service role https://review.openstack.org/587187 | 19:50 |
| mnaser | ^ scary keystone patch but a good cleanup | 19:51 |
| odyssey4me | evrardjp: ok, confirmed that I did not break the integrated build - yay! | 19:51 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cinder master: Convert role to use a common systemd service role https://review.openstack.org/574817 | 19:52 |
| odyssey4me | the bad news is that the integrated build is still failing on tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario very, very frequently | 19:52 |
| cloudnull | evrardjp ^ there you go | 19:52 |
| *** KeithMnemonic2 has quit IRC | 19:53 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_designate master: Convert role to use a common systemd service role https://review.openstack.org/587196 | 19:55 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_keystone master: Import GPG key for NGINX via Ansible https://review.openstack.org/587612 | 19:55 |
| mnaser | cloudnull: odyssey4me ^ that's a workaround for centos for now | 19:56 |
| mnaser | cloudnull: does your os_cinder patch do things that allow for adding the venv into $PATH? | 19:56 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_blazar master: Convert role to use a common systemd service role https://review.openstack.org/587114 | 19:56 |
| cloudnull | mnaser no. | 19:57 |
| cloudnull | I could make it do that though ? | 19:57 |
| mnaser | is there a way using the role to append things to PATH? | 19:57 |
| mnaser | yeah so cinder-rtstool is a tool that cinder uses on centos but it calls it sometimes without rootwrap | 19:57 |
| cloudnull | does it have to be the global path or just the service path ? | 19:57 |
| *** KeithMnemonic has joined #openstack-ansible | 19:57 | |
| mnaser | and because it does it without rootwrap, /openstack/venvs/cinder-testing/bin is not in the path | 19:58 |
| mnaser | just the service | 19:58 |
| cloudnull | can we just add the path to the sudoers? | 19:58 |
| mnaser | it would work if it was sudoers but this is executed not as root | 19:58 |
| cloudnull | ah | 19:58 |
| mnaser | it runs a 'verify' on service start up that apparently doesnt need root | 19:58 |
| *** DanyC has joined #openstack-ansible | 19:58 | |
| mnaser | so thats why it doesnt pick up the path from there | 19:58 |
| evrardjp | odyssey4me: I am working on the skip of the tempest | 19:59 |
| evrardjp | cloudnull: thanks | 19:59 |
| evrardjp | I will review | 19:59 |
| odyssey4me | evrardjp: awesome, thanks | 19:59 |
| *** epalper has quit IRC | 20:03 | |
| evrardjp | cloudnull: what do you mean by the "service was written in python" ? That's far from clear to me | 20:03 |
| evrardjp | the thing was running before without it, so there is obviously something changing -- that's what I want to know | 20:04 |
| mnaser | evrardjp: we're at 13 patch sets for something that we've merged all over the place :\ | 20:04 |
| cloudnull | evrardjp ^ its in the commit message. and the docs are there. | 20:04 |
| mnaser | the docs do tell that you need python-systemd | 20:05 |
| *** v1a4 has joined #openstack-ansible | 20:07 | |
| *** v1a4 has quit IRC | 20:07 | |
| *** vnogin has quit IRC | 20:08 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cinder master: Convert role to use a common systemd service role https://review.openstack.org/574817 | 20:10 |
| cloudnull | mnaser ^ | 20:10 |
| cloudnull | I added a environemnt variable which will set the path for the service | 20:10 |
| cloudnull | https://review.openstack.org/#/c/574817/15/tasks/cinder_install.yml@63 | 20:11 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Switch to using project-templates https://review.openstack.org/586755 | 20:11 |
| evrardjp | mnaser: I don't think it's silly to ask if it's unclear. Good for you if you understand. | 20:12 |
| evrardjp | mnaser: The thing is that if you're only two to understand it, then you'll be only two to maintain it. | 20:12 |
| evrardjp | I don't think I am unreasonable. | 20:13 |
| mnaser | i think that as OSA has less contributors, we have to a tad bit more reasonable in terms of asks | 20:14 |
| mnaser | churning through a change many times taking up CI cycles and with every iteration taking up 1~2 hours more means it slows us doooown | 20:14 |
| *** vnogin has joined #openstack-ansible | 20:14 | |
| mnaser | i'm all for being strict on things that are important and are fundamentally not working | 20:14 |
| evrardjp | mnaser: we've got history of cruft that's there because we didn't know we could clean up | 20:16 |
| evrardjp | if we don't learn of our mistakes, we will just repeat them forever. | 20:17 |
| mnaser | it's easy to know what we can and can't clean up. we remove it and the tests either pass or fail. and hopefully if we can run integrated repos on role jobs | 20:17 |
| mnaser | then no one has to worry about puling the wrong thing | 20:17 |
| mnaser | we can document the heck out of it and still pull it out and it breaks some other dependency if we have poor test coverage | 20:17 |
| cloudnull | evrardjp: is there more you want changed with those reviews? | 20:17 |
| *** mattinator has joined #openstack-ansible | 20:17 | |
| jrosser | can anyonr see why this seemed to just stop? https://review.openstack.org/#/c/586607/ i feel im missing something obvious | 20:18 |
| mnaser | jrosser: looking | 20:18 |
| evrardjp | cloudnull: with this dicussion: I'd say no: I am beyond caring. I'd like to know, personally now, why this package is needed. | 20:18 |
| mnaser | i think log_instance_info is failing jrosser | 20:19 |
| cloudnull | have you read the OSLO log documentation: "In order to enable the support you must have Python bindings for systemd installed." ? | 20:20 |
| cloudnull | https://docs.openstack.org/oslo.log/3.28.1/journal.html | 20:20 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts master: Ensure virtualenv/pip are installed on centos https://review.openstack.org/587598 | 20:20 |
| jrosser | mnaser: is that a zuul thing - i chopped most of the tests out there and wondered if id broken it | 20:21 |
| mnaser | https://github.com/openstack/openstack-ansible/blob/master/scripts/scripts-library.sh | 20:21 |
| mnaser | i feel like get_instance_info is failing somewhere | 20:21 |
| cloudnull | mnaser jrosser based on http://logs.openstack.org/07/586607/3/check/openstack-ansible-deploy-aio_lxc-ubuntu-bionic/382ce0b/job-output.txt.gz#_2018-07-31_17_10_54_068501 seems like it tried to ssh to itself and failed? | 20:21 |
| cloudnull | like br-mgmt wasn't up at that time | 20:22 |
| mnaser | i might be wrong yeah cloudnull might be more accurate | 20:22 |
| *** mattinator has quit IRC | 20:22 | |
| mnaser | jrosser: maybe run it in a local env? | 20:23 |
| mnaser | i nkow thats a bit of a pain but.. | 20:23 |
| jrosser | yes i think so | 20:23 |
| mnaser | jrosser: OR you can ask infra for a hold on job openstack-ansible-deploy-aio_lxc-ubuntu-bionic | 20:23 |
| mnaser | and what will happen is when that job fails next (which is when you recheck) | 20:23 |
| jrosser | its very unclear what point it was at when that error happened | 20:23 |
| mnaser | you can give them your ssh keys and they'll add them and you can ssh to the machine | 20:23 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: [WIP] Do no use tempest https://review.openstack.org/587544 | 20:24 |
| *** hachi has quit IRC | 20:24 | |
| mnaser | do we run a full tempest run or smoke tests | 20:25 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: [WIP] Do no use tempest https://review.openstack.org/587544 | 20:25 |
| cloudnull | we used to run the ops tests. im not sure now though | 20:25 |
| mnaser | i think maybe we should run the smoke tests and blacklist any which are often failing | 20:25 |
| mnaser | instead of not using tempest, i'm a bit not comfortable with the idea of not using *the* framework to test openstack | 20:26 |
| mnaser | let me see what tests we're running | 20:26 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: [WIP] Do no use tempest https://review.openstack.org/587544 | 20:26 |
| cloudnull | mnaser do you have an environment w/ cinder to test that env change to see if it help the pathing situation ? | 20:26 |
| mnaser | i can spin up one | 20:27 |
| mnaser | or i can rebase on yours | 20:27 |
| cloudnull | not that you need that patch | 20:27 |
| cloudnull | you could just add Environment=... to the service uit | 20:27 |
| cloudnull | **unit | 20:27 |
| cloudnull | then restart it to see if it's happya | 20:27 |
| mnaser | yeah i kinda destroyed my env already :< | 20:28 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_cinder master: Add target_helper to backend_defaults https://review.openstack.org/584604 | 20:28 |
| mnaser | rebased ^ | 20:28 |
| mnaser | so i think once that + https://review.openstack.org/#/c/587612/ land, we should be getting much better for centos | 20:29 |
| odyssey4me | mnaser: we run a limited set of tests - always aiming for scenario tests before anything else | 20:32 |
| *** ansmith has quit IRC | 20:32 | |
| odyssey4me | the trouble is they're very inconsistent, and very unstable | 20:33 |
| admin0 | how do we handle regions the OSA way ? | 20:33 |
| mnaser | odyssey4me: i see, well, are there certain tests that are unstable, maybe we can blacklist those and continue to run smoke tests on tempest | 20:33 |
| odyssey4me | mnaser: https://github.com/openstack/openstack-ansible-os_tempest/blob/master/defaults/main.yml#L68-L79 is the test set from the integrated buid | 20:34 |
| odyssey4me | *build | 20:34 |
| odyssey4me | mnaser: bear in mind that running the 'smoke' tests is a total waste of time for us | 20:34 |
| odyssey4me | we're testing different things to the upstream projects | 20:34 |
| mnaser | https://github.com/openstack/puppet-openstack-integration/blob/master/run_tests.sh#L264-L328 | 20:35 |
| mnaser | is what we ran in puppet openstack and it pretty much never failed, ever | 20:35 |
| odyssey4me | they're testing api compatibilities and that sort of things - whereas we're testing function only | 20:35 |
| cloudnull | admin0 typically I've deployed regions as two seperate deployments that share keystone | 20:35 |
| mnaser | let me look at our average runtime | 20:35 |
| admin0 | cloudnull, how do you share the keystone ? | 20:35 |
| odyssey4me | heh, that's a tiny test set and totally skips the nova scenario test which is what's failing for us | 20:36 |
| admin0 | a single deploy server that has access to both regions ? ..or one keystone+horizon outside of the deployment | 20:36 |
| cloudnull | admin0 https://github.com/cloudnull/osad-regions | 20:36 |
| odyssey4me | admin0: several methods - share a database, or share an ldap store, or use federation | 20:36 |
| cloudnull | that's old but kinda has what weve done befor e | 20:36 |
| evrardjp | I will wait that periodics are green before doing a m3 release if you all don't mind. | 20:37 |
| cloudnull | admin0 effectively auth region, compute region 1/2. now a days folks seem to gravitate toward Okta/LDAP | 20:38 |
| cloudnull | which IMHO is a lot easier | 20:38 |
| mnaser | odyssey4me: smoke tests will do things like upload to swift, create and delete subnets.. create a server and list,create volumes and list, etc | 20:39 |
| mnaser | so they're not just a ping | 20:39 |
| mnaser | they're actual functional tests, just very short ones | 20:39 |
| cloudnull | evrardjp +1 soungs good | 20:39 |
| evrardjp | mnaser: feel free to push up a patch then changing the whitelist. | 20:40 |
| odyssey4me | mnaser: I defer to your better knowledge - please go right ahead and adjust that whitelist :) | 20:40 |
| admin0 | cloudnull, LDAP is already in 3 places .. so individually, i can setup a cluster per region and they are standalone in their own horizon ..so for example region1.domain.cloud region2.domain.cloud and region3.domain.cloud are all up and running .. what i want to do is create something like private.domain.cloud and if people login here, they can see the 3 as regions | 20:40 |
| evrardjp | maybe I should stop building this healthcheck thing then. | 20:40 |
| *** armaan has quit IRC | 20:40 | |
| *** armaan_ has joined #openstack-ansible | 20:40 | |
| mnaser | honestly we rarely have issues in puppet ci world so i want to try and bridge those things together and hopefully bring some of the stability we have there | 20:41 |
| odyssey4me | mnaser: most of us are totally tempest ignorant and have really just tried to keep things going based on what we received | 20:42 |
| mnaser | odyssey4me: i feel ya on that | 20:43 |
| odyssey4me | you've got a lot more experience with all that | 20:43 |
| cloudnull | admin0 hand-waving over all of the intra-region networking, I'd probably setup an "identity" region that has access to the compute regions. | 20:43 |
| mnaser | we can look over the list afterwards together to make sure the checks are sufficnet | 20:43 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_tempest master: Run smoke and dashboard tests by default https://review.openstack.org/587617 | 20:44 |
| cloudnull | admin0 the compute regions would have all of the OS services minus keystone. | 20:44 |
| odyssey4me | as far as I'm concerned, if that gets us more stable - that's fine | 20:45 |
| cloudnull | ^ ++ | 20:45 |
| mnaser | how do we feel about running 4 scenarios similar to these -- https://github.com/openstack/puppet-openstack-integration#description | 20:46 |
| evrardjp | mnaser: just propose the patch | 20:46 |
| evrardjp | and we'll see how it fares in gates. | 20:46 |
| mnaser | i did as step 1 here https://review.openstack.org/#/c/587617/ | 20:46 |
| odyssey4me | mnaser: well, it was cloudnull who was dead against random scenario names at the last PTG ;) | 20:47 |
| mnaser | haha :p | 20:47 |
| mnaser | well right now for example we have 'metal' jobs which are cool and fast but.. does anyone deploy metal? | 20:47 |
| mnaser | do we even list a fully metal deployment as something that is supported and possible | 20:48 |
| odyssey4me | yes, the point of the tests was to make it possible | 20:48 |
| evrardjp | mnaser: well yes it's in our docs for a while now | 20:48 |
| mnaser | ok so its documented, cool | 20:48 |
| odyssey4me | and, quite frankly, I think metal should be our default - with container opt-in | 20:48 |
| evrardjp | anyway, you can change it the way you like | 20:48 |
| evrardjp | it's your thing now!: p | 20:48 |
| mnaser | oh i was just asking | 20:48 |
| evrardjp | well | 20:48 |
| odyssey4me | doing that would take away the majority of the first user confusion | 20:48 |
| mnaser | its not my thing :) | 20:48 |
| cloudnull | odyssey4me I'm kinda against changing the default . | 20:49 |
| mnaser | ideally i would want to put nspawn jobs as nv and having voting lxc jobs, the | 20:49 |
| mnaser | maybe i'm wrong but i feel like we're not testing heavily our defaults | 20:50 |
| evrardjp | cloudnull: would you explain why? | 20:50 |
| evrardjp | mnaser: what do you mean, and why do you have this impression? | 20:50 |
| mnaser | i agree that the whole container thing is kinda been a staple for now and we might need to take sometime before we can make such a switch | 20:50 |
| mnaser | well, aio_nspawn jobs exist but our defaults are not nspawn, aio_lxc only exists for ubuntu (and not other os) | 20:51 |
| mnaser | i assume aio_basekit runs lxc | 20:51 |
| cloudnull | evrardjp: a container based deployment has been the default for the entirety of this projects existence. I think we have a fairly sensible approach to containerization. | 20:51 |
| odyssey4me | mnaser: aio_lxc only doesn't exist for the other platforms because the other all time out | 20:51 |
| admin0 | cloudnull, so step1, setup-hosts, run infra playbook and then only the keystone and horizon playbook to create this identity region , then in all other regions ( run all except # the os-keystone and #os-horizon playbook so that they are without keystone .. and somehow (user_variables) point to the identity keystone ? | 20:51 |
| evrardjp | odyssey4me: ++ | 20:52 |
| mnaser | odyssey4me: ah i see | 20:52 |
| mnaser | so what you're saying is we need more POWERRRRRR | 20:52 |
| evrardjp | mnaser: and basekit was introduced because they couldn't run it | 20:52 |
| evrardjp | but we didn't want to reduce scope for existing | 20:52 |
| admin0 | what this will do is make everything dependent on this one identity provider .. isn't it much easier to make them seperate that they work on thier own, but also provide one set of horizon via which they can access it as regions ? | 20:52 |
| odyssey4me | well, my general idea to get tests done would be to make 2 basic config bases - nothing and compute_kit | 20:53 |
| mnaser | so really that means the aio_nspawn jobs might just never pass | 20:53 |
| cloudnull | all metal is great for folks who want that. but a number of large deployers (myself included) greatly benefit from the mechanics we have, and making that opt-in seems like we're throwing away what makes out project quite unique. | 20:53 |
| odyssey4me | then all role use those, plus whatever else they need - with a list kinda like devstack | 20:53 |
| mnaser | for centos at laest | 20:53 |
| mnaser | i see tremenedous value in containers tbh | 20:53 |
| mnaser | especially because it allows you to maintain seperate releases of openstack | 20:53 |
| evrardjp | cloudnull: it has been containerized, but we've stepped towards putting more agents on metal recently -- and we've been doing great. I can't remember the amount of ppl asking me to run everything on metal. I just don't think these are opposite approaches, I just think they complement each other quite well | 20:53 |
| *** savvas has quit IRC | 20:53 | |
| mnaser | i can upgrade keystone today and then worry about nova in 3 weeks | 20:54 |
| evrardjp | cloudnull: I am in favor of keeping containerized systems, but I am also in favor of simplicity. | 20:54 |
| *** savvas has joined #openstack-ansible | 20:54 | |
| cloudnull | agreed, I think the agents on hosts is an example of us doing the right things with contianers. | 20:54 |
| odyssey4me | mnaser: you could be doing that regardless of the containers - all the services are isolated into venvs | 20:54 |
| odyssey4me | but meh | 20:54 |
| mnaser | odyssey4me: you know i gave that thought not long ago but then... libvirt reminded me | 20:54 |
| mnaser | :P | 20:54 |
| mnaser | things like python-libvirt and other possible binary based runtime dependencies | 20:55 |
| odyssey4me | quite honestly the missing gap for non-containering would be to isolate the binary deps properly | 20:55 |
| mnaser | ++ | 20:55 |
| mnaser | i would agree | 20:55 |
| evrardjp | cloudnull: for me I'd say everything on bare metal for "gate" testing to have light systems, but when we want to do things well, we should run containerized and multi-node | 20:55 |
| cloudnull | neutron agents in containers was a bad decision when we did all so long ago. but we kept with it for $REASONS. | 20:55 |
| odyssey4me | so using a chroot for non-containering would basically complete the whole thing quite nicely, but it'd have to be done across everything - compute, libvirt, etc | 20:55 |
| mnaser | i wonder how much faster or slower things would be if we ran multinode jobs in the gate | 20:56 |
| cloudnull | it was a dirty hack, along with us running cinder volumens and nova compute in containers. | 20:56 |
| evrardjp | I oppose tho things in this vision "production ready" and "developer ready" | 20:56 |
| evrardjp | but my vision might be different than others :) | 20:56 |
| odyssey4me | mnaser: multinode brings us nothing for speed - for RPC we do multi-node tests and they take 4+ hours on *very* good hardware | 20:56 |
| tux__ | cloudnull: I want to create subnet in neutron and want to set route but i don't want to set default route on that subnet, what is the option in neutron ? | 20:56 |
| mnaser | odyssey4me: boo | 20:56 |
| odyssey4me | mnaser: to speed things up, we have to fundamentally change how we deploy | 20:57 |
| tux__ | my instance has two nic and want to make 1st nic default route | 20:57 |
| mnaser | odyssey4me: a profiling session all together would seem like a good exercise perhaps | 20:57 |
| odyssey4me | that's what the python build simplification work has been about, and what staged deployments will be next | 20:57 |
| admin0 | pre-packaged docker ? kolla way ? | 20:57 |
| admin0 | :) | 20:57 |
| tux__ | 2nd nic specific route | 20:57 |
| admin0 | :P | 20:57 |
| odyssey4me | mnaser: it's a bit dated now, but this is what comes next: https://github.com/openstack/openstack-ansible-specs/blob/master/specs/queens/deployment-stages.rst | 20:58 |
| odyssey4me | it'll need updating with lessons learned | 20:58 |
| cloudnull | tux__ there's a route option, though idk if it can help with a multi-nic setup | 20:58 |
| *** savvas has quit IRC | 20:59 | |
| odyssey4me | trying to make fundamental changes, without breaking upgrades and keeping things operational is hard - so it's taken a *lot* more time than anticipated | 20:59 |
| mnaser | odyssey4me: absolutely. it is hard | 20:59 |
| odyssey4me | never mind the fact that it's been a part-time activity | 20:59 |
| tux__ | cloudnull: if i use "--host-route destination=1.1.1.0/24,nexthop=192.168.5.254" in that cause does it going to set default route also or just specific route? | 20:59 |
| mnaser | also, mordred totally pinged me on this | 20:59 |
| mnaser | or not me, but infra | 20:59 |
| mnaser | https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html?highlight=with_items#migrating-from-with-x-to-loop | 20:59 |
| evrardjp | yep known thing | 21:00 |
| odyssey4me | yep, ansible 2.6 has some new quirks | 21:00 |
| evrardjp | I don't like the with_dict | 21:00 |
| evrardjp | I mean the new with_dict :) | 21:00 |
| evrardjp | blabla | dict2items! | 21:00 |
| evrardjp | so cool | 21:00 |
| evrardjp | I am off for today | 21:01 |
| mnaser | take care | 21:01 |
| evrardjp | I will stop this crusade on tempest then and tomorrow will tag m3. | 21:01 |
| cloudnull | just to note: our deployment speed isn't really all that slow. I mean deploying a new cloud to roughly 500 nodes (we did this just last week) takes a couple hours (like 3 total), so what? its a 500 node cloud. When the deployment was done I had a 500 compute node cloud that was 100% functional. | 21:03 |
| cloudnull | I'm all for improving the gate speed | 21:03 |
| cloudnull | but not if it sacrifices capabilities | 21:03 |
| mnaser | cloudnull: i agree | 21:04 |
| mnaser | my focus here is focus on improvements that can help the gate so we can iterate faster | 21:04 |
| admin0 | i have deployed a lot using osa: i do not find the containers to be bad or slow . in fact, like them because i can install extra packages on a specific container based on scripting or other needs | 21:04 |
| cloudnull | like i ran the deployment, i got lunch. i checked on it, i got coffee, done. | 21:04 |
| admin0 | everything neatly separated | 21:05 |
| cloudnull | ++ | 21:05 |
| cloudnull | I like odyssey4me chroot idea for the on metal services | 21:05 |
| cloudnull | to give us that host separation still | 21:05 |
| cloudnull | I also think we can do things better in a lot ways. | 21:06 |
| *** armaan_ has quit IRC | 21:06 | |
| odyssey4me | cloudnull: yeah, so the issue with our speed is that 1 node takes as long as 5 - pretty much | 21:06 |
| cloudnull | pretty much . | 21:06 |
| odyssey4me | the only way to cut that time is to be able to seperate out the build part into a seperate process which isn't used for gates | 21:06 |
| cloudnull | the deployment is as fast as the slowest node in the bunch | 21:06 |
| odyssey4me | so basically build the chroot/packages/whatever in one process, and consume it in another | 21:07 |
| mnaser | so for me i think things like say | 21:07 |
| odyssey4me | of course the hilarious thing is that doing distro package builds is proving that whole idea wrong :p | 21:07 |
| cloudnull | there's always doing things like so https://review.openstack.org/#/c/497742 - which cut roughly 45 min out of the gate | 21:08 |
| cloudnull | :D | 21:08 |
| cloudnull | granted that was 8 months ago | 21:08 |
| odyssey4me | yeah, so if we take on building a cli - then we can take on stuff like that without all the bash | 21:09 |
| mnaser | and i mean technically | 21:09 |
| mnaser | we have a cli already | 21:09 |
| mnaser | :p | 21:09 |
| cloudnull | odyssey4me ++ the distro gates are just as long, if not longer than the source gates. which is kinda funny. | 21:09 |
| mnaser | cloudnull: you know that's what has been bothering me | 21:09 |
| cloudnull | mnaser you're not wrong :) | 21:09 |
| mnaser | i feel like there's something wrong with that | 21:09 |
| mnaser | everytime i say i'll take sometime to check it out beucase it really doesnt add up! | 21:10 |
| mnaser | openstack-ansible-functional-centos-7 and openstack-ansible-functional-distro_install-centos-7 are both exactly 38 minutes in some random change i picked | 21:10 |
| cloudnull | we should look into that. It something in source polluting distro, or viseversa | 21:10 |
| *** savvas has joined #openstack-ansible | 21:10 | |
| cloudnull | **it could be something... | 21:11 |
| odyssey4me | mnaser cloudnull evrardjp well, if we build a python cli then it'd be a lot easier for us to make the bootstrapping and configuration process more elegant, and streamlined - and we could make OSA more pluggable | 21:11 |
| odyssey4me | you only download and use what you need, rther than everything | 21:11 |
| mnaser | absolutely | 21:11 |
| cloudnull | this all could simply be proving my anti-distro-religion right :) | 21:11 |
| odyssey4me | it'd be easier to also then put a structure into the roles, including the playbooks and all that - then the integrated repo can be much more lightweight | 21:12 |
| cloudnull | that might be a good . | 21:13 |
| mnaser | 14 minutes until it reaches running os_keystone | 21:13 |
| mnaser | 2 minutes longer to run os_keystone with source | 21:14 |
| mnaser | but a lot of tasks can be skipped which are not being skipped | 21:15 |
| mnaser | ex: creating keystone folders and stuff | 21:15 |
| *** savvas has quit IRC | 21:15 | |
| cloudnull | w/ distro the time it takes to install is constant, with source the repo is built but the actual deployment time of the service should then be faster. | 21:15 |
| cloudnull | the time we see in the gates is distro install installing pckages, and source making packages. | 21:16 |
| mnaser | maybe we should prebuild the image and then ship them out... oh wait.. | 21:16 |
| mnaser | P | 21:16 |
| mnaser | :P | 21:16 |
| cloudnull | i mean we kinda have that with the repo server. | 21:17 |
| *** mattinator has joined #openstack-ansible | 21:17 | |
| cloudnull | maybe we can publish our repo server bits on tarballs.o.o | 21:17 |
| mnaser | i always thought it would be nice to have official OSA artifacts | 21:17 |
| mnaser | ^^ yes | 21:17 |
| cloudnull | then just consume that in the gates and rebuild them using a periodic job? | 21:17 |
| * cloudnull handwaves over getting that all done | 21:17 | |
| mnaser | or | 21:17 |
| tux__ | cloudnull: trying to set "--gateway none" but it doesn't like | 21:18 |
| mnaser | we do a periodic that actually proposes changes to openstack-ansible with pins/rebuilds | 21:18 |
| mnaser | so we only change the pin when things pass | 21:18 |
| cloudnull | ++ that might work too | 21:18 |
| mnaser | that way downstream can never break us in a way | 21:18 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587020 | 21:19 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586532 | 21:19 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Move handler flush to after MQ setup https://review.openstack.org/587097 | 21:19 |
| mnaser | hey i think you broke the world odyssey4me | 21:19 |
| mnaser | :p | 21:19 |
| mnaser | No package matching 'python-pip' found available, installed or updated | 21:19 |
| cloudnull | tux__ what's the error? | 21:19 |
| cloudnull | is it setting up the route at lease? | 21:19 |
| odyssey4me | cloudnull: well, our need to build wheels is going away, because services are now published to pypi | 21:19 |
| *** savvas has joined #openstack-ansible | 21:20 | |
| cloudnull | that's cool! | 21:20 |
| odyssey4me | so we can just stop building from git by default, and have that be an optional path instead | 21:20 |
| cloudnull | https://pypi.org/project/cinder/ ? | 21:21 |
| cloudnull | or is that coming soon | 21:21 |
| cloudnull | what's old is new again :) | 21:21 |
| odyssey4me | well, yeah - not all of them are done yet | 21:21 |
| odyssey4me | https://pypi.org/project/neutron/ | 21:21 |
| *** mattinator has quit IRC | 21:21 | |
| odyssey4me | of course it's not useful to us until they get that done | 21:21 |
| tux__ | cloudnull: "Invalid input for gateway_ip. Reason: 'none' is not a valid IP address." | 21:21 |
| tux__ | "neutron subnet-create net-odmz-vlan11 10.11.0.0/21 --name sub-odmz-vlan11 --allocation-pool start=10.11.1.1,end=10.11.7.254 --dns-nameservers 10.30.0.10 10.30.0.11 8.8.8.8 --host-route destination=10.0.0.0/8,nexthop=10.11.0.1 --gateway=none" | 21:22 |
| odyssey4me | mnaser: where'd you see that? | 21:22 |
| cloudnull | odyssey4me that's good news though. | 21:22 |
| mnaser | https://review.openstack.org/#/c/587612/ | 21:22 |
| mnaser | and im noticing it in other stuff | 21:22 |
| cloudnull | having that as an option should greatly speed up the integrated build. | 21:22 |
| odyssey4me | mnaser: *we* broke the world ;) https://review.openstack.org/587598 | 21:23 |
| odyssey4me | looks like maybe RDO isn't present at the right time? | 21:23 |
| mnaser | how the hell did it pass ci then | 21:23 |
| cloudnull | rutro shaggy | 21:23 |
| tux__ | cloudnull: in document they said you can set "none" but not working https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/subnet.html | 21:24 |
| *** jwitko_ has quit IRC | 21:24 | |
| odyssey4me | mnaser: the task right before is to add the RDO repo | 21:24 |
| *** savvas has quit IRC | 21:24 | |
| cloudnull | tux__ I think if you set a static route the gateway has to be defined. | 21:24 |
| cloudnull | or set to auto | 21:24 |
| tux__ | I want to set two nic on VM (public/Private) public will use default gateway and private will use host specific route | 21:25 |
| mnaser | where and how | 21:25 |
| mnaser | gr | 21:25 |
| cloudnull | mnaser odyssey4me yup its everywhere http://logs.openstack.org/96/587196/9/check/openstack-ansible-functional-centos-7/6a6849e/job-output.txt.gz#_2018-07-31_20_38_13_288530 | 21:25 |
| tux__ | if both subnet will use default gateway then it will create issue | 21:25 |
| *** yolanda has quit IRC | 21:25 | |
| mnaser | odyssey4me: i assume you're close to eod | 21:26 |
| mnaser | so wanna propose the revert for now? | 21:26 |
| odyssey4me | mnaser: ok, so perhaps we should revert that one | 21:26 |
| mnaser | ++ | 21:26 |
| odyssey4me | although it's weird - I see RDO everywhere here: http://logs.openstack.org/12/587612/1/check/openstack-ansible-functional-centos-7/1eabec6/logs/ara-report/result/24d898ce-936a-40b3-a514-08ed639555b1/ | 21:26 |
| cloudnull | tux__ when the networks were created , did you define a gateway for both? | 21:26 |
| tux__ | cloudnull: "--no-gateway" it works :) | 21:26 |
| mnaser | maybe it's another repo that contains the dependencies | 21:26 |
| cloudnull | tux__ cool | 21:27 |
| mnaser | lets revert till we figure it out | 21:27 |
| odyssey4me | you reverting, or me? | 21:27 |
| tux__ | cloudnull: http://paste.openstack.org/show/726983/ | 21:27 |
| cloudnull | tux__ you could set a route metric in the instance to force it, if that doesn't work | 21:27 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-openstack_hosts master: Revert "Ensure virtualenv/pip are installed on centos" https://review.openstack.org/587630 | 21:27 |
| mnaser | odyssey4me: cloudnull ^ | 21:27 |
| tux__ | cloudnull: thanks | 21:27 |
| cloudnull | tux__ nice! | 21:28 |
| * cloudnull had not remembered there was a --no-gateway flag | 21:28 | |
| *** savvas has joined #openstack-ansible | 21:29 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts master: Remove unused openstack_host_use_distro_pip var https://review.openstack.org/587635 | 21:30 |
| *** ansmith has joined #openstack-ansible | 21:30 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts master: Install virtualenv vor CentOS via distro packages https://review.openstack.org/587637 | 21:31 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Change command for shell https://review.openstack.org/587638 | 21:32 |
| odyssey4me | mnaser cloudnull so python-pip was the trouble package, so I've added two patches to restore the other bits: https://review.openstack.org/587637 & https://review.openstack.org/587635 | 21:32 |
| odyssey4me | mnaser cloudnull mind giving a review for https://review.openstack.org/582354 & https://review.openstack.org/#/c/587376/ | 21:38 |
| cloudnull | done and done | 21:40 |
| odyssey4me | tyvm | 21:44 |
| *** cshen has quit IRC | 21:48 | |
| *** tux__ has quit IRC | 21:51 | |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts master: Revert "Ensure virtualenv/pip are installed on centos" https://review.openstack.org/587630 | 21:59 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587020 | 22:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586532 | 22:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Move handler flush to after MQ setup https://review.openstack.org/587097 | 22:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Disable functional tests temporarily https://review.openstack.org/587645 | 22:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Revert "Disable functional tests temporarily" https://review.openstack.org/587646 | 22:02 |
| odyssey4me | mnaser cloudnull ^ that will get designate back to operating again | 22:02 |
| odyssey4me | that whole chain | 22:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Use generic vars file for ubuntu https://review.openstack.org/586691 | 22:05 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Convert role to use a common systemd service role https://review.openstack.org/587196 | 22:05 |
| *** DanyC has quit IRC | 22:05 | |
| *** jwitko_ has joined #openstack-ansible | 22:05 | |
| odyssey4me | That's me out for the night - cheers folks! | 22:07 |
| cloudnull | take care odyssey4me | 22:11 |
| *** cshen has joined #openstack-ansible | 22:13 | |
| *** mattinator has joined #openstack-ansible | 22:16 | |
| *** cshen has quit IRC | 22:18 | |
| *** FrankZhang has quit IRC | 22:20 | |
| *** mattinator has quit IRC | 22:21 | |
| *** kaiokmo has quit IRC | 22:22 | |
| *** DanyC has joined #openstack-ansible | 22:27 | |
| *** DanyC has quit IRC | 22:42 | |
| *** tonyontopic has quit IRC | 22:51 | |
| *** mattinator has joined #openstack-ansible | 22:53 | |
| *** mattinator has quit IRC | 22:57 | |
| *** vnogin has quit IRC | 23:04 | |
| *** ThomasWhite has quit IRC | 23:06 | |
| *** mattinator has joined #openstack-ansible | 23:16 | |
| *** mattinator has quit IRC | 23:20 | |
| *** pmannidi has joined #openstack-ansible | 23:21 | |
| *** threestrands has joined #openstack-ansible | 23:24 | |
| *** threestrands has quit IRC | 23:24 | |
| *** threestrands has joined #openstack-ansible | 23:24 | |
| *** tux_ has joined #openstack-ansible | 23:28 | |
| *** tux_ has quit IRC | 23:33 | |
| *** tosky has quit IRC | 23:35 | |
| *** kaiokmo has joined #openstack-ansible | 23:38 | |
| *** mattinator has joined #openstack-ansible | 23:54 | |
| *** mattinator has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!