Wednesday, 2018-03-07

*** fusmu has quit IRC00:01
openstackgerritMerged openstack/openstack-ansible-lxc_container_create master: Always create containers with fixed MAC addresses  https://review.openstack.org/55016000:07
*** masuberu has quit IRC00:35
*** masuberu has joined #openstack-ansible00:37
*** chyka has quit IRC00:38
logan-pabelanger: the connection plugin just looks for a few vars in a host's inventory to enable the container namespace stuff...00:41
logan-on the container's inventory host, a physical_host var should point to the inventory_hostname of the bare metal host where the container is located00:42
logan-and the container_name var should contain the name of the container on that host00:42
logan-so usually it is sufficient to set container_name: "{{ inventory_hostname }}"00:42
logan-actually I think that might be defaulted anyway00:42
logan-if you run the task with -vvvvv you'll see some output from the connection plugin like "container confirmed" .. etc showing that it has successfully determined from your inventory that it is working against a container00:44
*** kstev1 has quit IRC00:49
pabelangercool00:49
pabelangerlike I said, totally want to try it out00:50
*** lbragstad has quit IRC01:27
*** dave-mccowan has joined #openstack-ansible01:56
*** admin0 has joined #openstack-ansible02:08
*** admin0 has quit IRC02:13
*** markvoelker has quit IRC02:25
*** markvoelker has joined #openstack-ansible02:26
*** masuberu has quit IRC02:42
*** markvoelker has quit IRC02:53
*** markvoelker has joined #openstack-ansible02:53
*** masber has joined #openstack-ansible03:02
*** nurdie has quit IRC03:03
*** nurdie has joined #openstack-ansible03:03
yifeihwoarang: ping03:20
*** masber has quit IRC03:36
*** masber has joined #openstack-ansible03:47
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Flush entire interface rather than just routes  https://review.openstack.org/55026904:02
*** dave-mccowan has quit IRC04:08
*** masuberu has joined #openstack-ansible04:36
*** ANKITA has joined #openstack-ansible04:40
*** masber has quit IRC04:40
*** masber has joined #openstack-ansible04:40
*** masuberu has quit IRC04:42
*** radeks has joined #openstack-ansible04:47
openstackgerritMerged openstack/openstack-ansible-lxc_container_create master: Flush entire interface rather than just routes  https://review.openstack.org/55026904:48
nsinghwhile installing my playbook i am getting below error:05:05
nsinghfatal: [aio1]: FAILED! => {"changed": false, "connection": "close", "content": "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>nginx/1.10.3 (Ubuntu)</center>\r\n</body>\r\n</html>\r\n", "content_length": "178", "content_type": "text/html", "date": "Wed, 07 Mar 2018 05:04:15 GMT", "msg": "Status code was not [200]: HTTP Error 404: Not Found", "redirected":05:05
nsinghfalse, "server": "nginx/1.10.3 (Ubuntu)", "status": 404, "url": "http://172.29.236.100:8181/venvs/17.0.0/ubuntu-16.04-x86_64/masakari-17.0.0-x86_64.checksum"05:05
nsinghany help?05:06
*** poopcat has quit IRC05:06
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins master: Resolve nsenter fork exception  https://review.openstack.org/55033105:06
cloudnullnsingh: where are you getting that error ?05:06
cloudnullis this in the aio create while running tempest ?05:07
nsinghcloudnull: i am getting this for the TASK [os_masakari : Retrieve checksum for venv download]05:08
cloudnullnsingh: I don't see that role https://github.com/openstack/openstack-ansible-os_masakari something you're working on ?05:09
nsinghi am creating POC for this only.05:10
cloudnullcool05:10
*** udesale has joined #openstack-ansible05:10
cloudnullthe error in that context is likely trying to download a pre-built venv05:10
cloudnullassuming  you're using the pattern we have in other os_ roles.05:11
nsinghyes05:11
cloudnullif so, you should be able to enable developer_mode05:11
nsinghif my role is on local machine how should i deal with this.05:12
cloudnullshould look something like https://github.com/openstack/openstack-ansible-os_glance/blob/master/defaults/main.yml#L2605:12
cloudnullwhich will bypass the venv lookup05:12
cloudnulland build everything on the target05:12
nsinghok05:13
cloudnullyour local role should have something lke https://github.com/openstack/openstack-ansible-os_glance/blob/master/tasks/glance_install.yml#L27-L4705:15
cloudnullfor developer mode05:15
cloudnulland https://github.com/openstack/openstack-ansible-os_glance/blob/master/tasks/glance_install.yml#L87-L10405:15
cloudnullthose same bits are most, if not all, of our os_ roles05:16
nsinghANKITA: i am able to ping 172.29.236.10005:25
ANKITA?05:25
*** chyka has joined #openstack-ansible05:26
cloudnullok, im off. take care all.05:27
*** chyka has quit IRC05:30
*** masuberu has joined #openstack-ansible05:32
*** ThomasS has quit IRC05:33
*** masber has quit IRC05:35
*** dariko has joined #openstack-ansible05:45
*** dariko has quit IRC05:56
*** chyka has joined #openstack-ansible06:11
*** chyka has quit IRC06:11
*** chyka has joined #openstack-ansible06:12
*** chyka_ has joined #openstack-ansible06:15
*** chyka has quit IRC06:18
*** chyka_ has quit IRC06:19
*** chyka has joined #openstack-ansible06:19
*** fusmu has joined #openstack-ansible06:20
*** masuberu has quit IRC06:38
*** jamesdenton has quit IRC06:44
*** chyka has quit IRC06:46
*** ThomasS has joined #openstack-ansible06:47
*** Sha0000 has joined #openstack-ansible06:49
*** Sha0000 has quit IRC06:50
*** prometheanfire has quit IRC06:56
cloudnullit looks like somethings gone wrong here https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/tasks/install_packages_debian.yml#L23-L6406:57
*** gkadam has joined #openstack-ansible06:58
cloudnullseeing it generate odd urls http://logs.openstack.org/29/548929/2/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/3f5b670/job-output.txt.gz#_2018-03-07_05_28_14_18458606:58
cloudnullIE: 'deb :///ubuntu-cloud-archive xenial-updates/pike main'06:59
cloudnullthe commands are working locally07:01
cloudnullbut i suspect the infra repos are now different?07:01
cloudnullspecifically https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/tasks/install_packages_debian.yml#L4407:05
cloudnullwhich is setting https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/tasks/prepare_aio_config.yml#L6807:06
cloudnulland setting ":///ubuntu-cloud-archive"07:06
*** threestrands_ has joined #openstack-ansible07:09
*** threestrands_ has quit IRC07:09
*** threestrands_ has joined #openstack-ansible07:09
*** threestrands has quit IRC07:10
*** ivve has quit IRC07:12
openstackgerritRussell Tweed proposed openstack/ansible-hardening master: Add check to ClamAV task to prevent simeltaneous content updates  https://review.openstack.org/55026807:13
*** threestrands_ has quit IRC07:13
cloudnulllooks like the commands are parsing the file correctly07:23
cloudnullaccordingly to the repo file in the logs07:24
cloudnullhttp://logs.openstack.org/29/548929/2/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/3f5b670/logs/openstack/instance-info/host_repo_info_05-27-41.log.txt.gz07:24
openstackgerritRussell Tweed proposed openstack/ansible-hardening master: Add check to ClamAV task to prevent simeltaneous content updates  https://review.openstack.org/55026807:28
*** Sha0000 has joined #openstack-ansible07:32
*** Sha0000 has quit IRC07:33
Taseercloudnull: looks like something fishy is happening with zuul07:34
TaseerI got all my jobs fail in under 10 minutes07:35
*** admin0 has joined #openstack-ansible07:39
*** holser__ has joined #openstack-ansible07:44
*** armaan has joined #openstack-ansible07:51
*** pcaruana has joined #openstack-ansible07:54
*** pcaruana has quit IRC07:55
*** pcaruana has joined #openstack-ansible07:55
odyssey4meTaseer yep, that seems to be a thing - adding the UCA repo crashes it when trying to apt-get update08:00
*** epalper has joined #openstack-ansible08:02
*** gus has quit IRC08:02
PTOIs it correct understood, that line 81 in https://github.com/openstack/openstack-ansible-os_neutron/blob/master/templates/neutron.conf.j2#L81 will be populated with the first defined provider networkin in the openstack_user_variables.yml?08:04
*** gus has joined #openstack-ansible08:04
*** olivierbourdon3- has quit IRC08:05
*** chyka has joined #openstack-ansible08:08
*** olivierbourdon38 has joined #openstack-ansible08:09
*** chyka has quit IRC08:13
*** ivve has joined #openstack-ansible08:14
*** admin0 has quit IRC08:15
odyssey4meok, it looks like infra has moved from using a script to an ansible toolset, and broken our implementation of how we read the mirrors08:16
odyssey4meI'll work out some patches to get the gates going again later today08:16
*** threestrands has joined #openstack-ansible08:32
*** threestrands has quit IRC08:32
*** threestrands has joined #openstack-ansible08:32
*** shardy has joined #openstack-ansible08:34
*** threestrands has quit IRC08:36
*** ANKITA has quit IRC08:43
*** electrofelix has joined #openstack-ansible08:47
openstackgerritRussell Tweed proposed openstack/ansible-hardening master: Add check to ClamAV task to prevent simeltaneous content updates  https://review.openstack.org/55026808:49
*** mrch_ has joined #openstack-ansible09:10
*** mbuil has joined #openstack-ansible09:12
*** mrch_ has quit IRC09:15
openstackgerritRussell Tweed proposed openstack/ansible-hardening master: Add check to ClamAV task to prevent simeltaneous content updates  https://review.openstack.org/55026809:18
*** olivierbourdon38 has quit IRC09:20
*** armaan has quit IRC09:24
hwoaranggood morning09:24
*** armaan has joined #openstack-ansible09:25
*** stuartgr has joined #openstack-ansible09:26
ionihello guys09:28
ioniis there a setting that i need to enable in order to ansible to upgrade all packages inside the container?09:28
*** PTO has quit IRC09:31
*** stuartgr has left #openstack-ansible09:32
*** olivierbourdon38 has joined #openstack-ansible09:34
*** olivierbourdon38 has quit IRC09:35
openstackgerritRussell Tweed proposed openstack/ansible-hardening master: Add check to ClamAV task to prevent simeltaneous content updates  https://review.openstack.org/55026809:36
*** ANKITA has joined #openstack-ansible09:38
*** armaan has quit IRC09:38
*** armaan has joined #openstack-ansible09:38
*** olivierbourdon38 has joined #openstack-ansible09:47
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: [TEST] Check mirror info  https://review.openstack.org/55041609:57
hwoarangthe gates do not look good today09:59
odyssey4mehwoarang yep, something changed in the way the apt sources are set - I'm on it09:59
hwoarangi also see failures outside of our domain10:00
hwoarangbuild-openstack-sphinx-docsRETRY_LIMIT in 39s10:00
hwoarangetc etc10:00
*** holser___ has joined #openstack-ansible10:07
*** holser__ has quit IRC10:07
*** hw_wutianwei has quit IRC10:08
*** hw_wutianwei has joined #openstack-ansible10:08
*** kukacz has quit IRC10:17
*** hw_wutianwei has quit IRC10:23
*** lvdombrkr has joined #openstack-ansible10:25
*** hw_wutianwei has joined #openstack-ansible10:31
*** Sha0000 has joined #openstack-ansible10:32
openstackgerritRussell Tweed proposed openstack/openstack-ansible-os_keystone master: Add memcache setup to shibboleth2.xml  https://review.openstack.org/55042710:33
*** Sha0000 has quit IRC10:43
*** mbuil has quit IRC10:43
*** gunix has joined #openstack-ansible10:54
gunixhello guys10:54
gunixi installed OSA-AIO and after that downloaded the RC file from horizon but i'm getting an error when i try to use the CLI10:54
gunixwhen i do hypervisor list i get HTTPSConnectionPool(host='10.40.2.142', port=5000): Max retries exceeded with url: /v3/auth/tokens (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))10:54
gunixam i missing something ? do i need to add another parameter to the rc file for ssl ingore self cert ?10:55
*** ivve has quit IRC11:02
*** jwitko has quit IRC11:04
*** udesale_ has joined #openstack-ansible11:19
*** shardy has quit IRC11:20
*** ivve has joined #openstack-ansible11:21
*** shardy has joined #openstack-ansible11:21
*** udesale has quit IRC11:21
*** shardy has quit IRC11:21
*** udesale has joined #openstack-ansible11:21
*** shardy has joined #openstack-ansible11:22
*** udesale_ has quit IRC11:24
*** epalper has quit IRC11:25
odyssey4mehwoarang back from a meeting - and yeah, I see that all jobs are hitting a retry limit11:29
*** ANKITA has quit IRC11:30
*** udesale has quit IRC11:36
hwoarangok so we are forced to take a break today :) goodie11:44
*** holser___ has quit IRC11:48
*** holser__ has joined #openstack-ansible11:48
*** Sha0000 has joined #openstack-ansible11:50
*** ivve has quit IRC12:00
*** Sha0000 has quit IRC12:10
*** karthi has joined #openstack-ansible12:12
*** gameon has joined #openstack-ansible12:16
*** speedmann has quit IRC12:18
*** speedmann has joined #openstack-ansible12:19
*** yolanda has quit IRC12:20
*** yolanda has joined #openstack-ansible12:21
*** mattoliverau has quit IRC12:21
*** mattoliverau has joined #openstack-ansible12:22
*** jamesdenton has joined #openstack-ansible12:25
*** armaan has quit IRC12:26
*** armaan has joined #openstack-ansible12:26
*** dave-mccowan has joined #openstack-ansible12:32
*** udesale has joined #openstack-ansible12:39
*** lbragstad has joined #openstack-ansible12:39
*** gameon has quit IRC12:43
*** chyka has joined #openstack-ansible12:46
*** karthi has quit IRC12:48
*** chyka has quit IRC12:50
*** armaan has quit IRC12:55
*** armaan has joined #openstack-ansible12:56
*** admin0 has joined #openstack-ansible12:56
*** admin0 has quit IRC12:57
mhaydenmorning12:57
*** karthi has joined #openstack-ansible13:09
*** Matias has quit IRC13:13
*** prometheanfire has joined #openstack-ansible13:20
*** karthi has quit IRC13:24
mhaydenodyssey4me / hwoarang: are these gate shenanigans something we can fix? or are they infra problems?13:24
*** Matias has joined #openstack-ansible13:25
mhaydenoh nvm, i see in the jobs13:26
mhaydenlooks like the fix merged: https://review.openstack.org/#/c/550425/13:29
*** Matias has quit IRC13:29
*** Matias has joined #openstack-ansible13:30
openstackgerritMajor Hayden proposed openstack/ansible-hardening master: Replace Fedora 26 with 27  https://review.openstack.org/55010913:30
*** ivve has joined #openstack-ansible13:34
*** Matias has quit IRC13:35
openstackgerritAndy McCrae proposed openstack/openstack-ansible master: Fix BOOTSTRAP_OPTS  https://review.openstack.org/55047313:35
mhaydenyeah jobs seem to be running okay now13:38
mhaydenodyssey4me: if you get time, could you review my idempotency patch for os-tests? https://review.openstack.org/55013513:39
*** Matias has joined #openstack-ansible13:39
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts master: Remove kernel modules: scsi_dh + x_tables  https://review.openstack.org/54975413:39
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts master: Make role idempotent  https://review.openstack.org/55010213:39
openstackgerritAndy McCrae proposed openstack/openstack-ansible master: Avoid putting rgw conf on ALL hosts  https://review.openstack.org/54984013:40
*** woodard has quit IRC13:41
*** woodard has joined #openstack-ansible13:41
openstackgerritMajor Hayden proposed openstack/openstack-ansible master: Use RDO Queens repo  https://review.openstack.org/54864113:42
openstackgerritMajor Hayden proposed openstack/openstack-ansible master: Use ARA instead of profile_tasks callback  https://review.openstack.org/54627113:42
mhaydenhwoarang / xgerman_: if y'all get a chance to get https://review.openstack.org/549036 settled, that would be good :)13:43
*** Matias has quit IRC13:46
*** esberglu has joined #openstack-ansible13:48
*** Matias has joined #openstack-ansible13:48
*** Matias has quit IRC13:53
*** jwitko_ has joined #openstack-ansible13:55
*** Matias has joined #openstack-ansible14:00
*** karthi has joined #openstack-ansible14:03
*** SerenaFeng has joined #openstack-ansible14:04
*** Matias has quit IRC14:06
hwoarangmhayden: i am still wondering about the dib pin to 2.9.014:06
hwoarangdo you know the reason?14:07
hwoarang2.11.0 or any new release should work right?14:07
*** karthi has quit IRC14:08
*** ivve has quit IRC14:09
*** esberglu has quit IRC14:14
*** lbragstad has quit IRC14:15
*** armaan has quit IRC14:16
*** Matias has joined #openstack-ansible14:18
*** armaan has joined #openstack-ansible14:20
openstackgerritMerged openstack/ansible-hardening master: Replace Fedora 26 with 27  https://review.openstack.org/55010914:21
*** lbragstad has joined #openstack-ansible14:21
*** ivve has joined #openstack-ansible14:22
*** armaan has quit IRC14:22
*** sar has joined #openstack-ansible14:23
*** Matias has quit IRC14:24
*** pylot has quit IRC14:27
jrossersurprised at how complicated this is http://git.openstack.org/cgit/openstack/openstack-ansible-os_keystone/tree/tasks/keystone_db_setup.yml#n4914:27
jrossersomething like this can handle non existing services https://www.irccloud.com/pastebin/UVcGwTBh/14:28
*** throwsb1 has joined #openstack-ansible14:28
*** ivve has quit IRC14:29
*** esberglu has joined #openstack-ansible14:29
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts master: Make role idempotent  https://review.openstack.org/55010214:33
*** chyka has joined #openstack-ansible14:37
*** Matias has joined #openstack-ansible14:41
*** chyka has quit IRC14:41
mhaydenhwoarang: no idea on that one:/14:42
*** ivve has joined #openstack-ansible14:43
*** lvdombrkr has quit IRC14:45
*** Matias has quit IRC14:45
*** electrofelix has quit IRC14:51
*** SerenaFeng has quit IRC14:51
cloudnullmhayden: odyssey4me: evrardjp: any chance you've had a moment to take a look at http://logs.openstack.org/71/503971/34/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/9d48ed4/job-output.txt.gz#_2018-03-07_12_34_04_418557 ?14:58
*** mbuil has joined #openstack-ansible14:58
cloudnulllooks like this is still an issue in the gate which is not correctly setting the UCA repos.14:58
mhaydennot i14:58
*** mbuil has quit IRC15:00
cloudnullseems to be picking up something like http://logs.openstack.org/29/548929/2/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/3582610/job-output.txt.gz#_2018-03-07_03_15_03_849120 in the gate15:00
*** Matias has joined #openstack-ansible15:01
*** epalper has joined #openstack-ansible15:01
openstackgerritMerged openstack/openstack-ansible-tests master: Add ansible-lint rule for apt_repository  https://review.openstack.org/54749715:01
*** pabelanger has quit IRC15:04
ioniguys15:04
ionii have a problem with nova install15:04
ioniTASK [Gather software version list]15:05
ioniKeyError: 'ansible_local'15:05
ionifatal: [ctrl1usf_nova_console_container-610db168]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""}15:05
cloudnullioni: are you running with a limit or tags?15:05
ionihttps://paste.xinu.at/TMV/15:06
*** Matias has quit IRC15:06
cloudnullI've seen the local facts not get set when running with a tag / limits15:06
ioniopenstack-ansible os-nova-install.yml15:06
cloudnullok.15:06
ionii have two controllers that have the same problem15:06
*** karthi has joined #openstack-ansible15:06
ionii'm running newton latest from yesterday, trying to update to it15:06
*** Matias has joined #openstack-ansible15:07
cloudnullioni: is it failing anywhere else before it runs into this error?15:10
*** kstev has joined #openstack-ansible15:10
cloudnullthe local facts are created / stored on the remote host15:11
cloudnullI'm wondering if there's a pre-condition that's not being met15:11
ionicloudnull, hmm yes15:11
ionion a compute node15:11
ionisome dependency problem15:12
*** pabelanger has joined #openstack-ansible15:12
ionii'll fix it15:12
ionii think from there is the problem15:12
cloudnullwhen you checked-out the new release did re-bootstrap ansible running scripts/bootstrap-ansible.sh15:12
ioniyes15:12
*** karthi has quit IRC15:14
ionimariadb-client : Depends: mariadb-client-10.0 (= 10.0.34+maria-1~xenial) but 10.0.34-0ubuntu0.16.04.1 is installed15:14
ionithat's the problem i had on a lot of components15:14
d34dh0r53moin15:15
cloudnullioni: I thought we had a pin in for some versions of mariadb15:15
cloudnullmaybe the old repos need to be cleaned out in favor of the new pinned versions ?15:16
cloudnullioni: can you see if the apt preference pins are in place ?15:16
*** radeks has quit IRC15:16
*** radeks has joined #openstack-ansible15:17
ioniPackage: *15:17
ioniPin: release o=MariaDB15:17
ioniPin-Priority: 100115:17
*** epalper has quit IRC15:17
ioni?15:17
ioniytere is15:17
ionithere is15:17
ionibut keep in mind that i have an old version of newton running15:17
ioni14.2.2 or something like that15:17
*** woodard has quit IRC15:18
cloudnullI wonder if that pin needs to be removed ahead of this upgrade?15:19
cloudnullthere was a version pin15:19
cloudnullIm looking for it15:19
cloudnullhttps://github.com/openstack/openstack-ansible-galera_server/blob/1075fe411aa7b37e6c536e1c1ea5d4bfd509d2b0/defaults/main.yml#L142-L15615:20
cloudnullioni: you should see something similar in the pins15:20
ioninope, i only have the first entry15:21
ionion compute node15:21
*** epalper has joined #openstack-ansible15:21
ionialso on galera container15:21
ionihttps://paste.xinu.at/fjRqJ/15:22
*** Matias has quit IRC15:22
*** sar has quit IRC15:23
ionicloudnull, the issue is that i believe newton wants a specific version from a diferent repo rather than the one from xenial15:23
ioni10.0.34+maria-1~xenial vs 10.0.34-0ubuntu0.16.04.115:23
ioniso remove the ubuntu one  mariadb-client fixes the issue15:24
ionifor me15:24
cloudnullinteresting.15:24
cloudnullso maybe we need a step in our inline upgrades to clear out the pins15:24
ioniroot@ctrl1nl-galera-container-76204eb3:~# dpkg -l | grep mariadb-client15:24
ioniii  mariadb-client              10.0.34+maria-1~xenial                     all          MariaDB database client (metapackage depending on the latest version)15:24
ioniii  mariadb-client-10.0         10.0.34+maria-1~xenial                     amd64        MariaDB database client binaries15:24
ioniii  mariadb-client-core-10.0    10.0.34+maria-1~xenial                     amd64        MariaDB database core client binaries15:24
cloudnullI wonder, did the deployment started life out before the pins or our repos got put in-place?15:27
ionii don't know if the pins were there in 14.2.215:28
*** karthi has joined #openstack-ansible15:29
*** armaan has joined #openstack-ansible15:32
*** udesale has quit IRC15:45
mardimHello all15:52
mardimoops wronf channel sorry15:53
*** Matias has joined #openstack-ansible16:00
*** Matias has quit IRC16:04
*** Matias has joined #openstack-ansible16:07
*** pcaruana has quit IRC16:10
*** Matias has quit IRC16:12
openstackgerritMerged openstack/openstack-ansible-os_octavia stable/pike: Adds some skip linters to the tests  https://review.openstack.org/54903616:13
mnaserhow can i force a rebuild of a venv?  i updated the hash for nova and reran repo-install and it didnt rebuild it16:14
openstackgerritMerged openstack/openstack-ansible-os_octavia stable/pike: Zuul: Remove project name  https://review.openstack.org/54193216:14
openstackgerritMerged openstack/openstack-ansible-os_octavia stable/pike: Fix file path  https://review.openstack.org/54533616:14
*** Matias has joined #openstack-ansible16:16
cloudnullmnaser: you can run repo-build.yml16:16
cloudnullor you can enable developer mode on a given role16:16
mnaserhad i ran repo-install16:18
mnasermy bad16:18
mnasers/had/ah/16:18
cloudnullthere's also a flag to force rebuild everything if you need too16:19
mnasercloudnull: i just changed a hash so i'm not sure it would recognize that the hash changed and rebuilds that specific venv16:19
cloudnullit should see the difference and rebuild the one venv. however `-e 'repo_build_venv_rebuild=true'` will force the rebuild16:21
*** Matias has quit IRC16:31
*** Leo_m has joined #openstack-ansible16:32
*** woodard has joined #openstack-ansible16:32
*** woodard has quit IRC16:32
*** woodard has joined #openstack-ansible16:40
*** olivierbourdon38 has quit IRC16:40
mnasercloudnull: alright, i added "nova_git_install_branch: 0b7f8064df75d01370bb4405547f216b004d90da" and ran .. openstack-ansible repo-build.yml -e 'repo_build_venv_rebuild=true' .. and nova is not being rebuilt16:45
mnaseris there something else to use because it looks like the wheel is not being rebuilt16:45
*** epalper has quit IRC16:45
*** fusmu has quit IRC16:45
andymccrjmccrory: do you remember why we didn't adopt the upstream os_user/os_project modules instead of the keystone: one we keep in OSA?16:47
*** karthi has quit IRC16:47
logan-mnaser: delete /etc/ansible/facts.d/openstack_ansible.fact on the build container16:49
logan-or at least set need_wheel_build = True16:50
cloudnullanyone mind giving https://review.openstack.org/#/c/550331/ a review16:50
cloudnullshould resolve the fork issue we're seeing randomely16:51
* cloudnull still needs to look into what's causing the UCA breakage in the gate.16:51
cloudnulllocal testing shows is passing, but the gate in master seems is very unhappy.16:51
mnaserlogan-: i think that might do the trick, thank you16:51
mnasercloudnull: can i help by another pair of eyes on logs?16:52
cloudnullsure.16:52
cloudnullhttp://logs.openstack.org/31/550331/1/experimental/openstack-ansible-integrated-deploy-aio/ac17d83/16:52
cloudnullis a recent set I was looking at16:52
mnaserok: [aio1] => (item={'state': 'absent', 'repo': 'deb :///ubuntu-cloud-archive xenial-updates/queens main', 'filename': 'ubuntu_cloud_archive_canonical_com_ubuntu'})16:52
mnaserthat cant be right16:52
cloudnullhttp://logs.openstack.org/31/550331/1/experimental/openstack-ansible-integrated-deploy-aio/ac17d83/job-output.txt.gz#_2018-03-07_07_28_33_72010116:53
cloudnullyup16:53
cloudnullso we pull the base repo from the hsot16:53
cloudnull**host16:53
cloudnulland then pipe that into a fact for future use when setting the UCA repo.16:53
mnaserhttps://github.com/openstack/openstack-ansible-openstack_hosts/blob/master/tasks/openstack_hosts_configure_apt.yml#L44-L6016:53
cloudnullthis is an exchange that only exists in the gate16:53
*** Matias has joined #openstack-ansible16:53
mnaserwhich seems to be "_package_repos + (user_external_repos_list | default([]))"16:54
mnasermaybe that is not being set properly16:54
cloudnullfor some more context http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/latest.log.html#t2018-03-07T06:57:5516:55
cloudnullit seems to be reading the sources list and storing the stdout16:55
cloudnullhttps://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/tasks/prepare_aio_config.yml#L6816:55
mnaserhttps://github.com/openstack/openstack-ansible/blob/e95bdb3fd65ba8561821b22f2c7f773869da7860/tests/roles/bootstrap-host/tasks/install_packages_debian.yml#L23-L3016:56
cloudnullhowever a local test of the command found here https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/tasks/install_packages_debian.yml#L24 on the repos logged here http://logs.openstack.org/31/550331/1/experimental/openstack-ansible-integrated-deploy-aio/ac17d83/logs/openstack/instance-info/host_repo_info_07-28-00.log.txt.gz seems to work fine16:56
mnaserthats some serious regex16:56
mnaser:p16:56
cloudnullso something in between is either setting the repos wrong or munging the stdout16:57
mnaserhttp://logs.openstack.org/31/550331/1/experimental/openstack-ansible-integrated-deploy-aio/ac17d83/logs/ara/result/de09e872-438d-4e6f-9ec2-4ff03d541100/16:57
mnaseri remember seeing a change adding trusted=yes lately.16:57
cloudnullyea, I'm curious if that'16:57
cloudnullis the source of our problems16:58
cloudnulldue to this hack16:58
mnaserit's def that16:58
mnaserbecause if you look at the ara output16:58
mnaserits adding a weird bunch of strings before16:58
mnasercloudnull: why dont we use zuul facts or the stuff in configure_mirrors.sh16:58
cloudnullI'm tempted to drop those commands / interactions for nwo16:58
cloudnull**now16:58
cloudnullmnaser: I do not know16:58
cloudnullwhat16:58
cloudnull** what's in those files ?16:58
cloudnullmaybe we can do this better16:59
mnasercloudnull: all images in the openstack infra ship a file called configure_mirrors.sh -- when you source that file, it exports a set of env variables which include the URLs to ALL mirrors in openstack infra16:59
cloudnullIE https://github.com/openstack-infra/zuul-jobs/blob/master/roles/configure-mirrors/templates/etc/apt/sources.list.j216:59
mnasercloudnull: nope, beyond that, one moment17:00
*** Leo_m_ has joined #openstack-ansible17:00
*** chyka has joined #openstack-ansible17:00
*** Leo_m_ has quit IRC17:02
*** Leo_m has quit IRC17:02
*** Leo_m has joined #openstack-ansible17:02
mnasercloudnull: https://github.com/openstack-infra/openstack-zuul-jobs/tree/master/roles/mirror-info17:03
mnaserit templates this file https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/roles/mirror-info/templates/mirror_info.sh.j2 out to /etc/ci/mirror_info.sh17:04
mnaserwe can source that file and then do something like lookup('env', 'NODEPOOL_UCA_MIRROR') | default(uca_etc)17:05
mnaserlet me push up something quick to try out17:05
*** chyka has quit IRC17:05
*** karthi has joined #openstack-ansible17:08
*** chyka has joined #openstack-ansible17:11
*** gkadam has quit IRC17:11
*** karthi has quit IRC17:13
*** armaan has quit IRC17:15
*** armaan has joined #openstack-ansible17:15
openstackgerritShannon Mitchell proposed openstack/openstack-ansible-ops master: Fix broken linuxbridge-agent config for the flat network in osa-ops mnaio  https://review.openstack.org/55053917:17
*** SimAloo has joined #openstack-ansible17:17
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info  https://review.openstack.org/55054217:20
mnasercloudnull: ^ lets see how this will break terribly17:20
cloudnullthat all looks good17:21
cloudnullif it gets past the openstack-hosts then https://www.myinstants.com/media/instants_images/boratgs.jpg17:22
*** olivierbourdon38 has joined #openstack-ansible17:24
*** olivierbourdon38 has quit IRC17:24
*** fusmu has joined #openstack-ansible17:25
*** olivierbourdon38 has joined #openstack-ansible17:26
jmccroryandymccr i don't. looks like those two have been around for a long time too17:30
idlemindyum -y update on host indicates priorities protections are excluding packages ... is this to keep a host in lock step? how do i apply any updates?17:30
*** fusmu has quit IRC17:30
andymccryeah hmm17:31
andymccrok might look into it17:31
andymccrmay aswell ditch them if possible :D17:31
*** olivierbourdon3- has joined #openstack-ansible17:32
*** olivierbourdon3- has quit IRC17:32
*** olivierbourdon3- has joined #openstack-ansible17:32
cloudnullidlemind: is this something you're seeing in the playbooks ?17:33
cloudnullor just something you're wanting to run across the cloud ?17:33
idlemindnah if i log into a host built by osa and try to update it via yum -y update i see: 591 packages excluded due to repository priority protections17:33
*** olivierbourdon38 has quit IRC17:33
cloudnullhum. I'17:35
cloudnull**I'm not sure where those protections are coming from .17:35
* cloudnull looking17:36
cloudnullI wonder if this is something coming from our galera roles ?17:36
cloudnullmhayden: ^17:36
idlemindpriorities.conf in /etc/yum/pluginconf.d17:36
spotzevrardjp: How close are we to EOL?17:36
cloudnullhttp://paste.openstack.org/show/693689/17:37
cloudnullidlemind:  looks like we'17:37
cloudnullbah...17:37
cloudnullI can't type atm17:37
spotzhehe17:37
cloudnulllooks like we're only setting the fastest mirror file17:38
idlemindhmm wonder how i got it17:38
cloudnullis there more priorities being set ?17:38
cloudnullidlemind: what release ?17:38
idlemindstable/pike17:38
idlemindhttps://github.com/openstack/openstack-ansible-openstack_hosts/blob/59112cbd84d6a8cb3652d780851d109d104692d6/tasks/openstack_hosts_configure_yum.yml17:40
idlemindat the bottom something about repo priorities i'll have to digest17:40
*** armaan has quit IRC17:47
*** armaan has joined #openstack-ansible17:48
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts master: Make role idempotent  https://review.openstack.org/55010217:52
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts master: Fix openstack_host_module_file typo  https://review.openstack.org/55054917:52
*** armaan has quit IRC17:53
*** armaan has joined #openstack-ansible17:54
*** sar has joined #openstack-ansible17:54
idlemindcloudnull alternatively just having it present might be the problem https://github.com/openstack/openstack-ansible-openstack_hosts/blob/45b2e96ec7dcade1e161868857084f9c436d6799/vars/redhat-7.yml#L91-L9217:55
idlemindfurther down in there it sets priorities17:56
*** pcaruana has joined #openstack-ansible17:57
idlemindresults in http://paste.openstack.org/show/693699/17:57
idlemindend result seems to be that i don't get updates (although doing it from cli on a host may not be the osa way)17:58
idlemindi have to imagine i'm due some updates this cloud was built at least a month ago so something seems hinky18:04
cloudnullyea I think thats something is off18:04
cloudnullThat paste shows quite a few pins18:05
cloudnullbut i'm not 100% on why that's happening within cent18:05
cloudnullIMO I'18:05
cloudnull**I'd blow that preference file aways and run teh update18:05
idlemindbrb after lunch; i was tinkering w/the priorities in yum.repos.d and yum update and all i get is more packages excluded for updates ... i'll disable it after lunch.18:05
cloudnullallowing the system to replce as needed18:05
idlemindit seems to be getting written for valid reason in the links i pasted18:06
cloudnullthough idk if that'd cause more problems than solve18:06
idlemindit's set in the default variables and there is a task in the openstack-hosts18:06
idlemindplay18:06
idlemindso somebody thought priorities were a good thing but it seems to break updates at least for me18:06
cloudnullyea I suspect that if its breaking for you it'll break for everyone18:09
*** osnaya has joined #openstack-ansible18:12
osnaya@cloudnull Hi there... Got your reference from Victor P at meetup last evening.18:15
cloudnullo/18:15
osnaya@cloudnull looking into openstack-ansible to deploy pike in our lab...18:16
cloudnullcool18:17
osnaya@cloudnull have openstack-ansible downloaded and configuring for deployment... want to deploy on Ubuntu16.04 host (not using containers at this stage). deploy it directly on Ubuntu18:18
cloudnullosnaya: yes you can do that18:20
cloudnullevrardjp: has recently made the tests so that's running in the gate18:21
osnaya@cloudnull there is a mention in the doc... that it can be deployed on host (config file needs to be created in /etc.d)..... can you point to any examples on how to bypass the container deployment such as aio_metal.yml18:21
cloudnullosnaya: you need to setup your environment to run all on metal18:22
cloudnullhttps://github.com/openstack/openstack-ansible/blob/8a9bbaab1d01d05c1053db75658bb7c91ce70689/etc/openstack_deploy/env.d/aio_metal.yml.example18:22
cloudnullif you put that file into the /etc/openstack_deploy/env.d/ location the inventory will not build any containers18:23
mnaserhmm18:23
mnaserit looks like ceph roles broke ansible linter jobs?18:23
cloudnulla new linter was added18:23
cloudnullmaybe related ?18:23
mnaserexception: no action detected in task. This often indicates a misspelled module name, or incorrect module path.18:23
mnaser- name: configure crush hierarchy18:24
mnaserin ceph-mon/tasks/crush_rules.yml18:24
osnaya@cloudnull ...looked at that file... which has "container_skel:" section in the example18:24
mnaserlooks like 'ceph_crush' module is a thing?18:24
mnaserbut i guess it's not being loaded when the linter is running?18:24
mnaserok so other than the gate being broken, the patch fixed that issue cloudnull18:26
cloudnullwoot!18:26
cloudnullits the wheel of progress18:26
cloudnull:)18:27
osnaya@cloudnull .... I want to know for deployment on the host, do we keep  "component_skel: and physical_skel:" sections blank for that example or do we have entries in there?18:27
*** olivierbourdon3- has quit IRC18:27
*** olivierbourdon38 has joined #openstack-ansible18:28
cloudnullosnaya: you will need all of those sections18:28
cloudnullthe override file is what you'd need to simply instruct the dynamic inventory to not construct the container host entries18:29
cloudnullif you have a generated inventory already you will have to flush it out and recreate it18:30
*** olivierbourdon38 has quit IRC18:30
osnaya@cloudnull I have not generated inventory yet...18:30
cloudnullok18:31
*** olivierbourdon38 has joined #openstack-ansible18:31
*** shardy has quit IRC18:31
cloudnullso you should just need to put that aio_metal.yml file in place and the rest should "just work"-tm18:31
cloudnullafter that file is in place follow the regular deployment document18:31
osnaya@cloudnull question I have is, in my case can I use the "component_skel and physical_skel" section from extra_container_yml.example as it in aio_metal.yml or the entries in these section for deployment on host will be different?18:32
*** olivierbourdon38 has quit IRC18:32
*** dariko has joined #openstack-ansible18:32
*** olivierbourdon38 has joined #openstack-ansible18:32
cloudnullyou can have those sections however those shouldn't be needed18:32
cloudnullthe core environment files are housed with the openstack-ansible code base18:33
osnaya@cloudnull cool... will give it a try and find out...18:33
cloudnullso the override file will just tickle the core environments to make them all deploy on metal18:34
osnaya@cloudnull when you and docs refer to deploy on metal is actually deploy on Ubuntu host, right?18:34
cloudnullosnaya: let me know how it goes.18:34
cloudnullyes, its to deploy on the baremetal without any containerization18:35
cloudnullyou can use ubuntu, suse, or centos18:35
osnaya@cloudnull yes, definitely.... will share the findings :) thank you18:35
osnaya@cloudnull thanks for the pointers...18:35
osnayaexit18:44
*** Leo_m has quit IRC18:44
osnayaquit18:47
*** osnaya has quit IRC18:47
goldenfriIs openvswitch not starting on 17.0.0.0rc1 (AIO) a known bug?18:50
idlemindcloudnull w/priorities disabled i get updates and no more excluded messages not a lot and they all come from epel18:55
*** sdake has joined #openstack-ansible18:59
*** sdake has quit IRC18:59
*** sdake has joined #openstack-ansible18:59
cloudnullgoldenfri: not one that im aware of19:01
goldenfrioh ok, I'll create one then :)19:01
goldenfriwell a bug report, not an actual bug19:01
cloudnullyou could create a bug too if you wanted :)19:02
cloudnullidlemind: so disabling the priorities across the cluster was needed to get updates flowing again?19:03
idlemindcloudnull i'm not positive; it was a small amount of updates (6) and they all came from epel so it's possible they were being ignored intentionally because a preferred repo had them at an older version19:03
cloudnullinteresting19:04
idlemindcloudnull i'm wondering if updates are triggering automatically otherwise19:04
idlemindmaybe mhayden could chime in19:04
idlemindif his updates are working as expected in his centos land19:04
cloudnullwe do use the package state of "latest" for the deployment so when running the playbooks the packages across the cluster should be updated when the playbooks are executed19:04
*** Leo_m has joined #openstack-ansible19:05
cloudnullbut, that might now catch normal system upgradwes19:05
cloudnull**upgrades19:05
cloudnullmhayden: would be a good person to ask though19:05
cloudnullespecially as it pertains to cent19:05
idlemindya usually a yum "*" state: latest would do the upgrade (along w/update_cache: yes) I believe ... those could be nestled in the playbooks along the way too19:06
mhaydenso we have package_state=present for centos 7 in queens/master right now19:06
mhaydento speed things up19:07
mhaydenwhat's the question/problem?19:07
*** jra has joined #openstack-ansible19:07
*** poopcat has joined #openstack-ansible19:09
*** olivierbourdon38 has quit IRC19:09
*** olivierbourdon38 has joined #openstack-ansible19:10
jraHi all, I'm working on a new OSA install of Pike with the intention of using OpenvSwitch rather than Linux Birdges, and I'm encountering a few issues after a successful OSA deployment. Notably, it looks like /var/run/openvswitch/ doesn't get created, so ovsdb can't start up; and neutron-openvswitch-agent wants to talk to ovsdb on localhost:6604, but it's only configured to listen on its socket, not that TCP port. Are these k19:13
*** olivierbourdon38 has quit IRC19:13
*** Leo_m has quit IRC19:14
*** Leo_m has joined #openstack-ansible19:15
goldenfri@mhayden what is the proper way to set a custom login_banner (couldn't find it in the docs), should I just change /etc/ansible/roles/ansible-hardening/files/login_banner.txt?19:15
*** olivierbourdon38 has joined #openstack-ansible19:16
*** olivierbourdon38 has quit IRC19:18
cloudnulljra: is that temp path required for ovs ?19:18
cloudnullI would think that'19:18
cloudnull**that's something ovs would create on start19:18
cloudnullbut maybe it's something we need to add to the tempfiles creation unit files19:18
jracloudnull: it doesn't seem to create the path on its own. once the path exists, it happily creates the socket file there19:18
odyssey4meandymccr we used our modules because the upstream ones weren't capable of doing all the things we needed, but that's likely changed now - we should use as many as we can - at the PTG we also discussed using the openstack CLI to fill any gaps left so that we can remove the federaiton stuff too19:19
*** olivierbourdon38 has joined #openstack-ansible19:19
mhaydengoldenfri: https://github.com/openstack/ansible-hardening/blob/a10fae4fe13c2f056cf3bde058653978c86ce187/defaults/main.yml#L28119:19
mhaydengoldenfri: security_login_banner_text is the variable you want to change19:19
andymccrodyssey4me: sweet thanks! I knew there was a historic reason i just couldn't remember19:19
andymccrcos we removed a bunch of the other modules19:19
odyssey4meandymccr there was also an issue with shade not being in u-c in older releases, but that's solved now... so we're a full go now for any upstream modules that do what we need19:20
odyssey4memnaser thanks for https://review.openstack.org/#/c/550542/1 - mind if I tweak it so that it can still work outside of openstack CI19:20
odyssey4me?19:21
jracloudnull: even once the socket is created, though, the agent isn't actually configured to connect to it. It's expecting a listener on localhost:6604, but ovsdb is only configured to listen on the socket.19:21
cloudnulljra: mind pasting the files under /etc/tmpfiles.d/openstack-* ?19:21
cloudnullfrom one of the compute hosts19:21
mnaserodyssey4me: sure, my theory was bootstrap_host_ubuntu_repo and bootstrap_host_ubuntu_security_repo would be defined somehow externally and it'll still work19:21
andymccrsweet - i might give that a go19:21
andymccrthanks odyssey4me19:21
mnaserbecause of "bootstrap_host_ubuntu_repo is not defined" and "bootstrap_host_ubuntu_security_repo is not defined"19:21
jraThis issue is appearing on the neutron servers, actually19:21
cloudnullok from that node then .19:22
jrais there a preferred pastebin for this channel?19:22
cloudnullI use http://paste.openstack.org/19:23
cloudnullbut any will do19:23
goldenfri@mhayden perfect thank you!19:23
jrahttp://paste.openstack.org/show/693711/19:23
odyssey4memnaser yeah, the thing rewrites the sources to a known and optimised way - whether in openstack CI or not... obviously something's changed which has broken the grep... but you're right that some things in the bootstrap-host role need changing to better use mirror info19:23
odyssey4meI actually had it on my TODO with the hope of fixing this week, even before this break happened19:24
odyssey4mehave we isolated what changed that caused this?19:24
mnaserodyssey4me: yes, infra added trusted=yes which breaks things19:24
cloudnulljra: ok, looks like we need to add that path to the neutron role so that we ensure teh temp path is present19:24
mnaserodyssey4me: http://logs.openstack.org/31/550331/1/experimental/openstack-ansible-integrated-deploy-aio/ac17d83/logs/ara/result/de09e872-438d-4e6f-9ec2-4ff03d541100/19:24
jra@cloudnull: how can I help with that?19:25
cloudnullI dont run ovs so I cant say why that happened, maybe someone here might be able to chime in on their experiences however I'd be happy to throw a pr at it if needed.19:25
cloudnulljra: if you want to add something like https://github.com/openstack/openstack-ansible-os_neutron/blob/master/templates/neutron-systemd-tmpfiles.j2#L319:25
odyssey4memnaser ah ok, I think we can safely just remove the rewriting of the sources and leave the host as it is19:26
cloudnullto the role when ovs is in use it'd be super helpful19:26
odyssey4meit was necessary back with trusty where we required the backports, but we don't do trust any more so meh19:26
cloudnullalso if you could document your experience and what you did to solve the issue in a launchpad bug it'd be appreciated19:26
mnaserodyssey4me: your final call :)19:26
odyssey4memnaser happy for me to modify the patch?19:26
mnaserodyssey4me: its all yours!19:26
cloudnullgoing to get a coffee brb19:27
idlemindmhayden i was seeing nothing but excluded packages on my 2 hosts because of yum priorities plugin ... i disabled it and i only had 6 updates to do ... not certain the last time i did an update but wanted to make sure your hosts were updating correctly w/it .. maybe there's a state: latest "*" somewhere that's doing updates when i re-run openstack-ansible to apply the day's configuration tweaks as i play19:27
evrardjphey everyone19:27
odyssey4meapologies for not getting to it sooner - we had a team outing today19:27
evrardjpyeah same.19:27
jracloudnull: sure, I'll get a bug filed and see about submitting a patch for it19:29
*** olivierb_ has joined #openstack-ansible19:29
mhaydenidlemind: so the priorities are there to ensure you get the correct packages installed19:29
*** holser__ has quit IRC19:29
mhaydenthere's reasons why we prefer packages from RDO over EPEL19:30
mhayden:)19:30
idlemindmhayden k so i just have to figure out how / when my boxes have been updating themselves :)19:30
*** Leo_m has quit IRC19:30
mhaydenhmm, perhaps you have automated updates turned on? that's part of the ansible-hardening role19:31
*** Leo_m has joined #openstack-ansible19:31
*** Leo_m has quit IRC19:32
*** rromans has quit IRC19:33
*** RandomTech has joined #openstack-ansible19:36
*** olivierbourdon38 has quit IRC19:36
*** olivierbourdon38 has joined #openstack-ansible19:36
RandomTechHello. I am having an issue with my openstack-ansible cluster set-up.19:38
idlemindmhayden that's likely where it's coming from; i'll check that role to verify19:38
RandomTechI am encountering this issue when running the setup-infrastructure playbook:19:38
RandomTechfatal: [ctrl-02_galera_container-fbb8def4]: FAILED! => {"attempts": 3, "changed": false, "failed": true, "msg": "Unable to start service mysql: Job for mariadb.service failed because the control process exit ed with error code. See \"systemctl status mariadb.service\" and \"journalctl -xe\" for details.\n"}19:38
RandomTechAny ideas?19:38
*** olivierbourdon38 has quit IRC19:38
*** Leo_m has joined #openstack-ansible19:39
*** olivierbourdon38 has joined #openstack-ansible19:39
*** Leo_m has quit IRC19:39
*** Leo_m has joined #openstack-ansible19:42
jracloudnull: https://bugs.launchpad.net/openstack-ansible/+bug/175414419:42
openstackLaunchpad bug 1754144 in openstack-ansible "OpenvSwitch run path not created" [Undecided,New]19:42
*** olivierbourdon38 has quit IRC19:43
*** olivierbourdon38 has joined #openstack-ansible19:43
jraAny thoughts on the tcp-port-vs-unix-socket issue? It looks to me like the OVS portions of the deploy don't create a functioning environment out of the box - is this a rare deployment type?19:44
*** Leo_m_ has joined #openstack-ansible19:45
evrardjpyes jra19:45
evrardjppatches welcome.19:45
evrardjpWe are working on it with OPNFV crew to make it part of our integrated gates19:46
evrardjpproperly tested and all19:46
*** Leo_m_ has quit IRC19:46
*** Leo_m has quit IRC19:47
*** olivierbourdon38 has quit IRC19:47
*** Leo_m has joined #openstack-ansible19:47
*** olivierbourdon38 has joined #openstack-ansible19:48
jraevrardjp: thanks, that's great to know19:49
jrawe're trying to decide whether to use OVS for a fairly large deployment that needs to get out the door in the next few months; if OVS isn't production-ready from OSA at this point, we'd probably choose to stick to Linux Bridges. Do you have any advice on that decision?19:50
*** olivierb_ has quit IRC19:51
idlemindjra what's the reasoning for ovs?19:51
*** olivierbourdon38 has quit IRC19:51
*** olivierbourdon38 has joined #openstack-ansible19:51
*** poopcat1 has joined #openstack-ansible19:51
*** jafeha__ has joined #openstack-ansible19:52
jraidlemind: mostly future-proofing; LB doesn't give us the prospect of hardware offloading, so parallel HPC-type workloads might not be an option if we go that way19:52
*** woodard has quit IRC19:52
*** poopcat has quit IRC19:52
*** jafeha has quit IRC19:52
*** radeks has quit IRC19:52
*** radeks has joined #openstack-ansible19:52
*** olivierbourdon38 has quit IRC19:53
*** throwsb11 has joined #openstack-ansible19:53
*** olivierbourdon38 has joined #openstack-ansible19:53
*** throwsb1 has quit IRC19:53
*** jamesdenton has quit IRC19:54
*** woodard has joined #openstack-ansible19:55
*** rromans has joined #openstack-ansible19:55
evrardjpthanks for the bug :)19:56
evrardjpjra: sticking to linux bridge is wiser for ops.19:57
evrardjpjra: I think you can use ovs already, but you might hit some bugs19:57
evrardjpalso first queens release will be out of the door soon19:57
evrardjpso you might want to try this too :)19:57
evrardjpif you don't need ovs, just don't use it, linux bridge has been around for ages, and it works :)19:58
jraevrardjp: we are planning to move up to queens before doing our big deployment, I'm trying to establish a stable dev before then19:58
jrabut your advice helps a lot, and I'll discuss with the team whether we want to do the development work that might be necessary to get a functional OVS versus just getting a working LB setup out the door19:58
evrardjpyeah it all depends on what you need to have19:59
evrardjpand your workloads19:59
*** woodard_ has joined #openstack-ansible20:00
*** woodard has quit IRC20:02
*** ivve has quit IRC20:08
idlemindis it possible to integrate this patch (https://review.openstack.org/#/c/545783/) into my openstack-ansible (stable/pike) cleanly? or do i have to wait for it to merge and backported to stable/pike of neutron and picked up there w/a commit-id update20:15
idlemindand is there a link to old bug triage irc meeting notes?20:18
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/queens: Bump A-R-R before first queens release  https://review.openstack.org/55058920:18
*** RandomTech has quit IRC20:23
idlemindfound the irc meeting logs20:24
*** jra has quit IRC20:26
*** throwsb11 has quit IRC20:31
*** hw_wutianwei has quit IRC20:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Clean-up gate code to use mirror_info  https://review.openstack.org/55054220:32
*** hw_wutianwei has joined #openstack-ansible20:32
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Update all SHAs for 16.0.10  https://review.openstack.org/55059320:35
odyssey4memnaser ^ that's a start - I'll work on doing some more cleanup and simplifying of bootstrap-host later this week... it would seem it's time20:35
odyssey4meof course, if you spot some things that you think will help simplify, please go ahead and propose patches :)20:36
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/queens: Add ansible-lint rule for apt_repository  https://review.openstack.org/55059420:37
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/pike: Add ansible-lint rule for apt_repository  https://review.openstack.org/55059520:37
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/ocata: Add ansible-lint rule for apt_repository  https://review.openstack.org/55059620:37
mnaserodyssey4me: that cleaned up nicely, the shell/source/write file bothers me but i cant think of a better cleaner way :D20:39
odyssey4memnaser yep, it's for tests only so I'm trying not to think too hard about it20:40
mnaserodyssey4me: yeah, that's valid, resources better spend in other more important things :)20:40
odyssey4meif there's a better way, I'm all ears20:40
odyssey4meif this passes, we'll need to port this back all the way to ocata (newton is about to EOL)... but there are subtle differences that need to be made for ocata.20:41
mnaserodyssey4me: i was thinking source the file in the gate shell job and use lookup('env', 'NODEPOOL_FOO') to create the file with a template20:42
mnaserbut i don't have the time to do that right now, so this is good enough by me :)20:43
odyssey4memnaser we're trying to reduce the amount of bash ;)20:43
odyssey4methat would work, but then it's also another template20:43
odyssey4memeh - it's kinda the same20:43
mnaserodyssey4me: in some ideal world, zuul calls the playbook directly, zuul exposes the mirror info via variables20:43
mnaserand we do this with variables only, but yeah :D20:44
odyssey4memnaser the trouble with zuul native jobs is that we have no way to run them outside of openstack-ci20:45
mnaserodyssey4me: yeah, that's the one thing i struggled with, but my thought process was that you'd end up with a simple ./run_tests.sh wrapper which install ansible locally and then starts the zuul playbook20:46
odyssey4meso our ideal situation is to have a thin shim that gets us to a place where we can run the job the way zuul runs it - that shim should be usable for vagrant, etc - but can be ignored by zuul20:46
mnasermy thoughts exactly :>20:46
*** Guest52 has joined #openstack-ansible20:46
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.18  https://review.openstack.org/55059920:47
odyssey4meto do that we'd ideally need to install the same version of zuul, which is achievable - and I'd like to discuss that option and figure out an easy way to reference the pin20:47
odyssey4meeven better would be for zuul to publish the venv containing the right ansible version... so we just download, fix the shebangs, and execute20:48
odyssey4mego one better and provide a curl | bash script to do it all for you ;)20:48
evrardjpmnaser: glad you are in sync :D20:48
persiaIn another channel, almost right now, there is a discussion about how venvs are not relocatable, which might break that model20:48
odyssey4me:) persia we use venvs, and we know there are some limitations - but it works for us20:49
evrardjpthey can be relocatable , depending on some cases20:49
*** Guest52 has left #openstack-ansible20:50
persiaFair enough :)20:50
odyssey4methey're not universally relocatable - it depends on c bindings and all that20:50
* persia was mostly amused at the synchonicity, and doesn't actually understand all the details about venv relocation20:50
odyssey4mebut anyway - I need to go, time to !computer for the day20:50
odyssey4mepersia it is actually quite weird how that happens :)20:50
odyssey4menight all20:53
*** dariko has quit IRC20:53
*** dariko has joined #openstack-ansible20:55
*** dariko has quit IRC20:55
*** hvvclint has joined #openstack-ansible21:01
*** osnaya has joined #openstack-ansible21:07
osnaya@cloudnull a quick q about ansible deployment on Ubuntu Host .... in env.d folder now I have aio_metal.yml... Do I need to remove the default extra_container.yml file or can it stay there but will be ignored21:11
*** pcaruana has quit IRC21:12
cloudnullits just an example file ,21:13
cloudnullyou can remove it if you like21:13
osnaya@cloudnull.... ok, so even if it is present, it will be ignore because aio_metal.yml will be picked up for execution right?21:14
*** Leo_m has quit IRC21:17
cloudnullyes21:20
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-os_neutron master: Create ovs temp directories when using ovs  https://review.openstack.org/55060921:25
cloudnulljra: ^21:25
mnaserok.. i'll kindly ask for spare eyes... 'nova_git_install_branch: 0b7f8064df75d01370bb4405547f216b004d90da' in user_variables.yml, then openstack-ansible repo-build.yml ... but osa doesn't seem to build that new wheel/venv21:27
mnaserbut it looks like for some reason, the variable isn't even being passed?21:27
mnaserskipping: [controller-1-dev_repo_container-e6c3f913] => (item={'name': u'nova', 'project_group': u'nova_all', 'fragment': None, 'url': u'https://git.openstack.org/openstack/nova', 'version': u'9adf97c0132c3e0d34bbb2e33b59c6e5b00bec57', 'original': u'git+https://git.openstack.org/openstack/nova@9adf97c0132c3e0d34bbb2e33b59c6e5b00bec57#egg=nova&gitname=nova&projectgroup=nova_all', 'egg_name': u'nova'})21:27
*** osnaya has quit IRC21:28
*** radeks has quit IRC21:52
cloudnullmnaser: if the underlying version of OSA hasn't changed and the venv already exists it wont rebuild unless the option "repo_build_venv_rebuild=yes" is set22:00
mnaser=yes or =true? i tried to run with =true but i guess i can try =yes22:00
mnaseropenstack-ansible repo-build.yml -e 'repo_build_venv_rebuild=true'22:00
cloudnulleither will work22:00
mnaseri'll give it another shot, maybe i missed something22:00
*** Leo_m has joined #openstack-ansible22:01
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests master: Improve idempotency  https://review.openstack.org/55013522:02
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts master: Make role idempotent  https://review.openstack.org/55010222:08
mnasercloudnull: this is strange, i ran it with that and it didnt attempt to rebuild stuff...22:09
cloudnullhum... that should be all that22:09
cloudnull*that's needed.22:09
mnasercloudnull: what's weirder is.. its trying to clone the git commit that is in the original openstack_services.yml .. but not the one in my user_variables.yml22:09
mnaserskipping: [controller-1-dev_repo_container-e6c3f913] => (item={'name': u'nova', 'project_group': u'nova_all', 'fragment': None, 'url': u'https://git.openstack.org/openstack/nova', 'version': u'9adf97c0132c3e0d34bbb2e33b59c6e5b00bec57', 'original': u'git+https://git.openstack.org/openstack/nova@9adf97c0132c3e0d34bbb2e33b59c6e5b00bec57#egg=nova&gitname=nova&projectgroup=nova_all', 'egg_name': u'nova'})22:09
mnaserrepo_build : Build package requirements file <= this tasks seems to just say 'ok' right away.. so not saying that it's changing anything22:10
cloudnullcurious, running `openstack-ansible repo-build.yml -e 'repo_build_venv_rebuild=true'` ?22:10
mnaseri also checked22:10
mnaserthe requirements.txt file it builds .. "nova # 9adf97c0132c3e0d34bbb2e33b59c6e5b00bec57"22:11
mnaserso that file is not even changing22:11
mnaser# grep nova_git /etc/openstack_deploy/user_variables.yml => nova_git_install_branch: 0b7f8064df75d01370bb4405547f216b004d90da22:11
mnaseri'll give that a shot but it looks to me that it's pretty much ignoring that22:11
mnaserby that, ignoring my 'nova_git_install_branch'22:12
cloudnullwith "nova_git_install_branch: $SOMESHA" in your user_variables.yml file it should just go.22:12
cloudnullthough you could also specify that on the cli22:12
cloudnull`openstack-ansible repo-build.yml -e 'repo_build_venv_rebuild=true' -e 'nova_git_install_branch=$SOMESHA'`22:12
mnaseri'll try that as a second attempt22:13
mnaseronce this run finishes22:13
*** openstackstatus has quit IRC22:13
*** Leo_m has quit IRC22:13
*** openstack has joined #openstack-ansible22:17
*** ChanServ sets mode: +o openstack22:17
mnaserthe load local packages task has resulted with `git+https://git.openstack.org/openstack/nova@9adf97c0132c3e0d34bbb2e33b59c6e5b00bec57#egg=nova&gitname=nova&projectgroup=nova_all` in remote_packages22:17
cloudnullmnaser: maybe you also need https://github.com/openstack/openstack-ansible-repo_build/blob/master/defaults/main.yml#L6522:17
mnasercloudnull: does the 'load local packages' task load things up to pass to repo_build? if they don't have the updated hash, i think the repo_build would never be able to solve the issue in the first place?22:18
cloudnullis "nova_git_install_branch" set within any of your existing user variable files ?22:18
*** Leo_m has joined #openstack-ansible22:18
mnasercloudnull: only user_secrets and user_variables, and it's set in user_variables to tip of stable/pike (0b7f8064df75d01370bb4405547f216b004d90da) but the value is not being used22:18
cloudnullok.22:18
mnasercloudnull: http://paste.openstack.org/show/693938/ and that was with -e nova_git_install_branch=22:19
mnaseroh, it got truncated.22:20
mnaserhttp://paste.openstack.org/show/693941/22:21
mnaserthis got truncated too but it has the git hashes which dont match..22:21
cloudnullhum, that's not great. can recursively grep for nova_git_install_branch in the /etc/openstack_deploy directory22:23
cloudnullwhat you have on the cli should always win22:23
mnasergrep -ir nova_git_install_branch /etc/openstack_deploy => /etc/openstack_deploy/user_variables.yml:nova_git_install_branch: 0b7f8064df75d01370bb4405547f216b004d90da22:23
mnaseryeah thats what i know with ansible too22:23
mhaydenhwoarang / cloudnull: on opensuse, what's the filesystem permissions for stuff in /etc/cron.d/ ?22:24
mnaseri'm gonna add a debug msg= nova_git_install_branch22:25
cloudnullroot:root22:25
cloudnullmhayden: ^22:25
mhaydenthe perms22:25
mhaydentrying to fiure out why it shows the cron as changed22:25
cloudnulldrwxr-xr-x22:25
mhaydenhttp://logs.openstack.org/02/550102/10/check/openstack-ansible-functional-opensuse-423/c09e034/job-output.txt.gz#_2018-03-07_22_16_21_58237122:25
mhaydeni'm thinking opensuse might use different file perms for stuff in /etc/cron.d/22:25
mhayden75522:25
mhaydenokay22:25
mhaydenno, wait, the files inside /etc/cron.d/22:26
mhaydensorry22:26
mhayden:P22:26
cloudnullits drwxr-xr-x on my machine running opensuse tumbleweed22:26
mnaserok.. added a debug.. "msg": "0b7f8064df75d01370bb4405547f216b004d90da" .. so the var is making its way22:26
cloudnull0644 for the filles22:26
mhaydenah, 0644!22:26
mhaydenthat's it22:26
mhaydenthank you sir22:26
mhaydencentos, ubuntu and suse are all different :|22:26
cloudnulllovely22:26
cloudnullmaybe we should use systemd timners22:26
cloudnull**timers22:26
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts master: Make role idempotent  https://review.openstack.org/55010222:27
mhaydenhopefully this works ^^22:27
*** woodard has joined #openstack-ansible22:27
*** hvvclint has quit IRC22:29
*** woodard_ has quit IRC22:31
*** woodard has quit IRC22:32
mnasermhayden: https://github.com/openstack/openstack-ansible-plugins/blob/master/lookup/py_pkgs.py#L325-L34022:33
mnaseroops22:33
mnasercloudnull: ^22:33
mnaserit looks like when the files are filtered to be parsed22:33
mnaserthe order of defaults vs user variable files22:33
mnaserdoesnt matter?22:33
mnaserill add a print locally but i think thats whats happening..22:34
cloudnullall of the user_ files should be parsed last22:34
mnasercloudnull: but that code doesnt seem to do any ordering22:35
*** esberglu has quit IRC22:35
*** fusmu has joined #openstack-ansible22:37
cloudnullit should be ordered by the file_name_words however we could zip those lists to filter more strictly22:38
mnasercloudnull: but the loop seems to happen over the file names provided then appended if they're part of file_name_words22:39
mnaserso if i understand the code.. if the order of files comes incorrect (user before than role default), this won't fix it22:39
cloudnulli see .22:40
mnaseri added a print and i see this as the first line22:40
mnaser['/opt/openstack-ansible/etc/openstack_deploy/user_secrets.yml', '/opt/openstack-ansible/etc/openstack_deploy/user_variables.yml', '/opt/openstack-ansible/playbooks/defaults/repo_packages/gnocchi.yml', '/opt/openstack-ansible/playbooks/defaults/repo_packages/nova_consoles.yml', '/opt/openstack-ansible/playbooks/defaults/repo_packages/openstack_services.yml',22:40
mnaser'/opt/openstack-ansible/playbooks/defaults/repo_packages/openstack_testing.yml', '/opt/openstack-ansible/playbooks/roles/system_crontab_coordination/defaults/main.yml', '/opt/openstack-ansible/tests/roles/bootstrap-host/defaults/main.yml', '/opt/openstack-ansible/tests/roles/bootstrap-host/vars/redhat.yml', '/opt/openstack-ansible/tests/roles/bootstrap-host/vars/ubuntu.yml',22:40
mnaser'/opt/openstack-ansible/tests/roles/bootstrap-host/vars/suse.yml']22:40
mnaseroops that was a lot, but you see it took it first, then openstack_services was after22:40
*** fusmu has quit IRC22:42
*** openstackstatus has quit IRC22:42
*** openstack has joined #openstack-ansible22:46
*** ChanServ sets mode: +o openstack22:46
cloudnullwell that would work if the '/etc/openstack_deploy/' was last22:46
mnaseryeah. i'm doing a bit more troubleshooting now22:46
mnaserso when it parses the '/opt/openstack-ansible/playbooks/../' path .. it generates all the git requirements, but when it does the iteration around /etc/ansible/roles and /etc/openstack_deploy .. it gets nothing22:50
mnaseri understand why /etc/ansible/roles gets nothing, but /etc/openstack_deploy should create the nova one22:50
*** esberglu has joined #openstack-ansible22:50
cloudnullindeed it should22:51
mnasercloudnull: ha22:51
mnasergot it22:51
mnaserhttps://github.com/openstack/openstack-ansible-plugins/blob/master/lookup/py_pkgs.py#L554-L56122:51
mnaseronly if a git_repo is defined it will actually process git22:51
mnaserwhich means that i will need to point the git repo too (even though it's just upstream)22:51
cloudnullah ha!22:52
mnaseri guess i should add the nova_git_project_group too..22:52
cloudnullhum... that's kinda a bummer. so both variables need to be defined22:53
cloudnullI guess most folks setting the sha have also set the repo url ].22:53
mnaserall 3 i guess22:53
mnaseroh22:53
mnaserno, not necessarily, itll get the override22:54
mnasermaybe the check should be based on sha too22:54
mnaseras that's most likely to be overridden more often than repo22:54
cloudnull++22:54
cloudnulladd git_install_branch to the key check and it should be ok.22:55
mnaserand i guess extra vars dont matter22:55
mnaserbecause this thing parses them outside ansible22:55
cloudnullyea the plugin feeds ansible. my bad. I was misleading you22:56
cloudnullwith the -e bit s22:56
mnaserno worries22:56
mnaserit wasnt straightforward :D22:56
cloudnullWe've been using this lookup for a few years now but I believe its one of the things that odyssey4me was wanting to remove given the pip install simplification effort22:57
cloudnullso hopefully we can make these types of changes a whole simpler in the near future22:57
*** armaan has quit IRC22:58
*** armaan has joined #openstack-ansible22:59
*** pmannidi has joined #openstack-ansible22:59
openstackgerritMohammed Naser proposed openstack/openstack-ansible-plugins master: Check for keys ending with git_install_branch  https://review.openstack.org/55064023:02
mnasercloudnull: ^23:03
mnasereverything worked just fine after23:03
mnaserno need for any extra vars, ansible figured out itself that it needed to rebuild wheels and venvs23:03
*** esberglu has quit IRC23:09
*** threestrands has joined #openstack-ansible23:12
*** threestrands has quit IRC23:12
*** threestrands has joined #openstack-ansible23:12
*** kstev has quit IRC23:19
*** armaan has quit IRC23:33
*** armaan has joined #openstack-ansible23:34
*** threestrands_ has joined #openstack-ansible23:34
*** threestrands_ has quit IRC23:35
*** threestrands_ has joined #openstack-ansible23:36
*** threestrands has quit IRC23:36
*** Leo_m has quit IRC23:38
*** Leo_m_ has joined #openstack-ansible23:39
*** osnaya has joined #openstack-ansible23:53
osnayaopenstack-ansible setup-hosts.yml --syntax-check throws error "Invalid extra vars data supplied. '@/etc/openstack_deploy/user_secrets.yml' could not be made into a dictionary" - any ideas or pointers please?23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!