Monday, 2018-02-12

*** perniciouscaffei has joined #openstack-ansible00:30
Guest61853If I'd like to keep the default container network in the OSA examples (172.29.236.0/22), and plan on setting up a Linux software router at the default gateway of this container network (172.29.236.1) for routing to internet access, then for the HAproxy installs that will live on the infrastructure nodes inside of the container network, how does the external specified LB VIP get configured?  I'm confused.00:47
Guest61853For instance, would I configure the external LB VIP with an IP on the internal container network and do something on the router to put it into the DMZ?00:48
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add nspawn container driver  https://review.openstack.org/47701701:04
*** rama_y has joined #openstack-ansible01:17
*** acormier has joined #openstack-ansible01:48
*** acormier has quit IRC02:03
*** acormier has joined #openstack-ansible02:03
openstackgerritMerged openstack/openstack-ansible-os_gnocchi master: Install only the required storage driver extras  https://review.openstack.org/54289802:10
Guest61853Ok -- looks like I can bind the infras with HAproxy directly to the external network and keepalived will then alias bind a floating external IP to whichever infra host holds the HAproxy master.02:11
Guest61853Is it true that the OSA deployment playbooks need to be run as user root?  Assume yes -- don't see anything in the deployment guide that confirms this.02:40
*** rama_y_ has joined #openstack-ansible02:42
*** rama_y has quit IRC02:42
*** rama_y_ has quit IRC02:43
openstackgerritMerged openstack/openstack-ansible master: bootstrap-host: Create swap without using a failed task  https://review.openstack.org/54310403:17
*** acormier_ has joined #openstack-ansible03:29
*** acormier has quit IRC03:33
*** gkadam has joined #openstack-ansible03:40
*** acormier_ has quit IRC03:42
*** bhujay has joined #openstack-ansible04:05
*** gkadam is now known as gkadam-afk04:09
*** dave-mccowan has quit IRC04:10
*** dave-mccowan has joined #openstack-ansible04:12
*** udesale has joined #openstack-ansible04:13
*** dariko has joined #openstack-ansible04:14
*** dave-mccowan has quit IRC04:18
*** taseer2 is now known as Taseer04:24
Taseerevrardjp: I have addressed odyssey4me 's comments04:25
Taseerthe system-init part is not working04:26
Taseerhttp://logs.openstack.org/91/522491/99/check/openstack-ansible-functional-ubuntu-xenial-nv/4282bb2/job-output.txt.gz#_2018-02-11_17_39_39_33151204:26
Taseerso this results in no congress-server's service04:26
*** gkadam-afk is now known as gkadam04:38
logan-spotz: etcd is OSA is only used if calico is selected as the network plugin. nothing else uses etcd yet04:43
logan-cloudnull: thanks for that ansible PR link... seems like that might be a huge improvement04:51
logan-also note to everyone... it looks like mariadb mirrors only have 10.1.31 in the 16.04 repo now04:51
logan-the apt pin should keep old clusters from updating I guess, but new clusters are SOL04:54
*** brad[] has quit IRC05:03
*** brad[] has joined #openstack-ansible05:05
tescracloudnull am again facing cache prepare failure? have removed the previous images and reformatted the infra 3 still the same result on exex of setup host05:06
tescra openstack-ansible lxc-hosts-setup.yml -e lxc_image_cache_refresh=yes -vvv --limit infra305:06
tescracan anyone suggest where am i going wrong have tried to put all the info regarding LXC TASKfailure in the gist :https://gist.github.com/ankita-anil-verma/727a074b56d01c1055655cfb2b0cdfa0/edit05:13
bhujaytescra: it gets downloaded here /tmp/rootfs.tar.xz  , the the aria2 log file will show  the download status . In ur case disk quota or something is preventing the download. Spend some time to identify the reason, it will certainly go through.05:25
Guest61853Logan:  seems the maria 10.1.30 is still available for xenial per the pike OSA release note URL (https://downloads.mariadb.com/MariaDB/mariadb-10.1.30/repo/ubuntu/)05:34
tescraaria2 log file has OK05:34
tescrai.e downloded05:34
tescratmp has rootfs.tar file05:36
logan-Guest61853: ah yep thanks. I guess it is still available on their mirror but it is gone from secondary mirrors (https://mariadb.com/kb/en/library/mirror-sites-for-mariadb/#secondary-rsync-mirrors) that sync from rsync.osuosl.org::mariadb05:42
*** acormier has joined #openstack-ansible05:43
logan-looks like the role uses download.mariadb.com by default so most people should be fine, good catch :)05:43
*** acormier has quit IRC05:47
*** armaan has quit IRC05:50
*** armaan has joined #openstack-ansible05:50
Guest61853Thanks -- hopefully that one doesn't disappear too :)05:51
*** olivierbourdon38 has joined #openstack-ansible05:54
*** dariko has quit IRC05:57
*** olivierbourdon38 has quit IRC06:08
*** hybridpollo has quit IRC06:09
*** ANKITA has joined #openstack-ansible06:33
ANKITAHI06:33
hwoaranggood morning07:10
hwoarangodyssey4me: what do you mean here https://review.openstack.org/#/c/537379/ ?07:10
*** Guest61853 has quit IRC07:10
*** sar has joined #openstack-ansible07:14
*** eumel8 has joined #openstack-ansible07:24
openstackgerritMerged openstack/openstack-ansible-lxc_hosts stable/ocata: Zuul: Remove project name  https://review.openstack.org/54305207:27
*** olivierbourdon38 has joined #openstack-ansible07:27
*** perniciouscaffei has quit IRC07:30
openstackgerritMerged openstack/ansible-hardening stable/pike: Zuul: Remove project name  https://review.openstack.org/54305307:35
openstackgerritMerged openstack/ansible-hardening stable/ocata: Zuul: Remove project name  https://review.openstack.org/54305807:35
openstackgerritMaxime Guyot proposed openstack/openstack-ansible master: Ceph RadosGW integration  https://review.openstack.org/51785607:35
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Reinstate ubuntu ceph gating  https://review.openstack.org/54122207:37
*** ANKITA has quit IRC07:45
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: Test keystone with a memcached pool  https://review.openstack.org/54000107:48
openstackgerritMerged openstack/openstack-ansible-galera_server stable/pike: Set proper yum repo priority on CentOS 7  https://review.openstack.org/54311107:52
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: Test keystone with a memcached pool  https://review.openstack.org/54000107:52
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: Test keystone with a memcached pool  https://review.openstack.org/54000107:54
hwoarangevrardjp: what kind of test do you need here? https://review.openstack.org/#/c/541310/07:58
openstackgerritMerged openstack/openstack-ansible-os_swift stable/pike: Zuul: Remove project name  https://review.openstack.org/54303708:07
*** pcaruana has joined #openstack-ansible08:07
*** tescra has quit IRC08:09
*** mbuil has joined #openstack-ansible08:15
jafehagood morning08:18
*** sxc731 has joined #openstack-ansible08:20
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server master: Join cluster without using a failed task  https://review.openstack.org/54310608:21
*** peri has joined #openstack-ansible08:30
*** cshen has joined #openstack-ansible08:31
*** sc68cal has quit IRC08:34
*** TESCRA has joined #openstack-ansible08:38
TESCRAhi08:38
TESCRAhi08:38
*** SON has joined #openstack-ansible08:41
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Remove unnecessary memcache variables  https://review.openstack.org/54096408:42
SONJOIN08:42
SONcan i switch from stable pike to stable ocata?08:43
openstackgerritMerged openstack/openstack-ansible-os_nova stable/pike: Zuul: Remove project name  https://review.openstack.org/54304108:46
*** Johnny has joined #openstack-ansible08:48
*** Johnny is now known as Guest2755408:48
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Remove impossible tests  https://review.openstack.org/54338408:56
evrardjphwoarang: sorry I missed your ping again08:58
evrardjpwill check right now08:58
hwoarangdont worry :)08:58
evrardjpgood morning jafeha TESCRA SON and hwoarang08:58
evrardjphwoarang: oh I see08:58
evrardjpI don't NEED a test08:58
evrardjpI just have one08:58
hwoaranghmm08:58
evrardjpI mean we have one on the way08:59
evrardjphttps://review.openstack.org/#/c/534318/08:59
evrardjpso if you want to be sure it works, probbably better to integrate one before the other.08:59
evrardjpOr alternatively we fix it in your patch, and see if it goes green in the test patch08:59
hwoarangevrardjp: we know it works because we tested in the opnfv08:59
evrardjphwoarang: let's merge it then08:59
evrardjpI just wanted to have a gating of ovs in osa directly09:00
hwoaranginteresting that an ovs AIO is on its way in09:00
hwoarangthank you evrardjp. did you have any further news regarding the latest mariadb failures?09:02
evrardjpnope at some point I stopped digging, debugging massive shell scripts where I have no starting knowledge aren't in my best skillz09:04
evrardjpso we pinned repos in all branches, and included release notes09:04
evrardjpoh yeah, so in the next days, please don't merge any requirements changes.09:05
evrardjpI will try to issue a rc when https://review.openstack.org/#/c/543131/ is passing tests. rc + branching.09:06
TESCRAevrardjp GM09:07
evrardjp?09:07
evrardjpoh good morning :)09:08
evrardjpGM brings me so much souvenirs, I didn't know which one you were targetting :D09:08
*** pbandark has joined #openstack-ansible09:13
TESCRAlxc_hosts : Retrieve base image task fails with disk qouta exceed error09:17
TESCRAor if image is created it fails at module /opt/ansible-runtime/local/lib/python2.7/site-packages/ansible/modules/utilities/logic/async_status.py09:18
TESCRAits become a viscous circle....09:20
*** brad[] has quit IRC09:20
TESCRAvicious circle no way out...!!09:21
TESCRAhttps://gist.github.com/ankita-anil-verma/727a074b56d01c1055655cfb2b0cdfa009:24
jrosserthis looks like machinectl quota exceeded09:25
*** brad[] has joined #openstack-ansible09:32
TESCRAhad formatted the server it went back to the first issue which was failure of TASK Ensure lxc cache prepared @09:32
*** shardy has joined #openstack-ansible09:33
TESCRAfaiure @ /opt/ansible-runtime/local/lib/python2.7/site-packages/ansible/modules/utilities/logic/async_status.py module09:34
TESCRAjrosser09:34
TESCRAthe <10.40.100.23> (0, '\n{"started": 1, "invocation": {"module_args": {"jid": "955189836941.17604", "mode": "status"}}, "finished": 0, "changed": false, "ansible_job_id": "955189836941.17604"}\n', '')09:35
TESCRAinvacation fails09:35
TESCRAam no longer sure whether the falut is with server or  inventory.json file at deployment node09:37
*** shardy has quit IRC09:43
*** shardy has joined #openstack-ansible09:43
SON?09:43
evrardjpTESCRA: which branch are you using ?09:44
*** hamza21 has joined #openstack-ansible09:45
openstackgerritMerged openstack/openstack-ansible-os_glance stable/pike: Update glance NFS for systemd  https://review.openstack.org/53957709:45
evrardjpTESCRA: fyi, async failures can happen, as long as the task checking the failure doesn't fail. In your case, it looks like your quota is exceeded, so you should work that out. Probably a storage issue. I am not familiar of all of this though.09:45
evrardjpjrosser: and cloudnull are more experienced with it.09:45
evrardjpbut I guess cleaning things out should be better.09:46
TESCRAok shall format and retry09:47
evrardjpTESCRA: wait a sec09:48
evrardjphttps://bugs.launchpad.net/openstack-ansible/+bug/174536109:48
openstackLaunchpad bug 1745361 in openstack-ansible "Failed to create subvolume /var/lib/machines/ when run 'openstack-ansible setup-hosts.yml' multiple times" [Undecided,In progress] - Assigned to Kevin Carter (kevin-carter)09:48
evrardjpis that what you have?09:48
TESCRAlet me check it09:48
openstackgerritPeriyasamy Palanisamy proposed openstack/openstack-ansible-os_neutron master: add ml2 config for networking bgpvpn  https://review.openstack.org/52259809:49
*** xgerman_ has quit IRC09:49
*** xgerman_ has joined #openstack-ansible09:50
openstackgerritPeriyasamy Palanisamy proposed openstack/openstack-ansible-os_neutron master: add ml2 config for networking bgpvpn  https://review.openstack.org/52259809:50
*** armaan has quit IRC09:53
*** armaan has joined #openstack-ansible09:54
Taseerevrardjp: about my patch09:56
Taseerhttp://logs.openstack.org/91/522491/99/check/openstack-ansible-functional-ubuntu-xenial-nv/4282bb2/job-output.txt.gz#_2018-02-11_17_39_39_33151209:56
*** electrocucaracha has quit IRC09:56
Taseerthe systemd-init tasks are not working09:56
*** electrocucaracha has joined #openstack-ansible09:57
Taseershould I address them in this patch ?09:57
evrardjpnah we can do it in a later patch. Here it proves it's not working, the first patch is for inclusion in gerrit, if you fixed all the concerns of odyssey4me it's a good start.09:57
evrardjplet me check09:57
TaseerI got a +209:57
TaseerI don't want to lose that !09:58
*** SmearedBeard has joined #openstack-ansible09:59
jrosserevrardjp: surprises me if someone ends up with machinectl containers, unless the aio user_vars is used as a starting point which seems to default to that as far as i can see10:01
evrardjpTaseer: don't worry you can get +2 later :)10:05
evrardjpTaseer: I've updated my review.10:05
evrardjpjrosser: well, there are ppl using aio quickstart.10:05
openstackgerritMerged openstack/openstack-ansible-os_trove master: Remove Trove's support for Heat  https://review.openstack.org/53467710:05
Taseerevrardjp: thanks10:06
evrardjpshould they as starting point? Maybe -- should they rely on it to know how this works, probably not10:06
evrardjpjrosser: I think that if it's not reliable enough, we shouldn't put it as default in our aio gates then.10:06
jrosserevrardjp: here https://github.com/openstack/openstack-ansible/blob/7eeef0b9c09d403f868d7690cda509ec5d30958b/tests/roles/bootstrap-host/templates/user_variables.aio.yml.j2#L17710:07
*** armaan has quit IRC10:07
*** armaan has joined #openstack-ansible10:08
evrardjpjrosser: my point still stands: We shouldn't set it as a default AIO var if it's not reliable. But we don't get much issues, and I think there is already a fix in master10:09
TESCRAevrardjp if i format the machine the task Ensure Cache fails at module TASK [lxc_hosts : Ensure that the LXC cache has been prepared]10:10
TESCRAback to same pt10:10
evrardjphave you enough disk space?10:10
evrardjpTESCRA: ^10:10
TESCRAeven after following instructions10:10
evrardjpwe don't have that in our gates, so it's something on your machine10:11
evrardjpand which branch are you using?10:11
evrardjpis it master, or stable/pike ?10:11
TESCRAyes as /dev/loop0                    16G  578M   14G   5% /var/lib/machines10:12
*** SmearedBeard has quit IRC10:12
TESCRAis used ony 5%10:12
TESCRAstable/pike10:12
TESCRAnot master10:12
evrardjpmachinectl shouldn't be used for that10:13
evrardjpstable/pike is using dir or overlayfs by default10:13
evrardjpyou have another issue10:13
TESCRA* stable/pike10:13
TESCRA?10:13
evrardjpwhat are your user variables and openstack_user_config ? Could you paste that to a paste.openstack.org ?10:14
TESCRAsure10:14
evrardjpjrosser: You had no issue with machinectl, did you?10:14
evrardjpor are you using simply zfs backend without machinectl ?10:14
evrardjpwell I mean I guess you're using zfs for the machines mount point, whatever that mountpoint is10:15
evrardjpmachinectl is probably a dir/btrfs thing only10:15
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: bootstrap-host: Create swap without using a failed task  https://review.openstack.org/54341010:16
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/ocata: bootstrap-host: Create swap without using a failed task  https://review.openstack.org/54341110:16
jrosserevrardjp: i have no experince with machinectl, just slid zfs in behind the existing stuff10:17
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Join cluster without using a failed task  https://review.openstack.org/54341210:17
TESCRAPaste #66964410:20
evrardjpTESCRA: it's easier with a full link but ok :)10:22
TESCRAPaste #66964810:23
*** alanmeadows has quit IRC10:23
TESCRAok10:23
*** alanmeadows has joined #openstack-ansible10:23
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249110:23
TESCRAhttp://paste.openstack.org/show/669648/10:23
TESCRAhttp://paste.openstack.org/show/669644/10:24
evrardjpTESCRA: a few comments:10:24
TESCRAis that f9 evrardjp10:24
evrardjp1) You should use reserved ips. else, overlap can happen10:24
evrardjp2) don't do new lines after an -10:25
evrardjpjust append.10:25
evrardjplike this: https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.pod.example#L6010:25
TESCRApod?10:26
TESCRAam not using pods its bare metal10:27
openstackgerritMerged openstack/openstack-ansible-tests master: Update rabbitmq install for cluster and upgrade tests  https://review.openstack.org/54324510:27
evrardjpno, just giving an example of no line breaks10:28
evrardjp3)  for http://paste.openstack.org/show/669644/  L102, L113, L123, it can probably be a user_variable, or in a userspace  group_vars/glance_all.yml10:28
*** yifei has quit IRC10:29
evrardjpit doesn't need to be repeated.10:29
*** yifei1 has joined #openstack-ansible10:29
evrardjpon top of that you are setting container_vars twice into the file for the same nodes. You should probably avoid it10:29
openstackgerritMerged openstack/openstack-ansible-galera_client master: Set proper yum repo priority on CentOS 7  https://review.openstack.org/54276210:30
evrardjpon http://paste.openstack.org/show/669648/ is not a valid vip/cidr10:30
evrardjphaproxy_keepalived_internal_vip_cidr: "10.40.100.0/22"10:30
evrardjpit's a valid cidr, but not a valid vip. you should proably use 10.40.100.10/2210:31
*** olivierbourdon38 has quit IRC10:31
*** yifei1 is now known as yifei10:31
evrardjpwhen you have all of that changed, re-run all your playbooks.10:31
TESCRAok thanks10:32
evrardjpIn your case .0 is not an ip, it's a network :)10:32
evrardjpTESCRA: if I am not clear in any way, just ask for clarifications here, we can help you!10:35
*** electrofelix has joined #openstack-ansible10:36
TESCRAhave removed the spaces and changed vip cidr but could not follow the rest of ur instructions ...would u pls elaborate? @ userspace  group_vars/glance_all.yml10:37
TESCRA& you are setting container_vars twice10:38
TESCRAevrardjp?10:38
TESCRAalso am i to rerun the bootstrap script?10:39
TESCRAafer all these changes/10:39
TESCRAfor it refeclect in inventory file?10:40
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client stable/pike: Set proper yum repo priority on CentOS 7  https://review.openstack.org/54342110:42
*** jafeha has left #openstack-ansible10:43
*** jafeha has joined #openstack-ansible10:47
*** mrhillsman has quit IRC10:57
*** mrhillsman has joined #openstack-ansible10:58
*** udesale_ has joined #openstack-ansible10:58
evrardjpTESCRA: ok10:59
evrardjpso10:59
evrardjpcould you re-paste your file with your latest changes?10:59
*** udesale has quit IRC11:01
evrardjpyour files*11:04
*** udesale_ has quit IRC11:07
*** sxc731 has quit IRC11:08
TESCRAhttps://gist.github.com/ankita-anil-verma/856f783e75389f03863c7be7c565efd9 evrerjp11:12
TESCRAevrarjp https://gist.github.com/ankita-anil-verma/856f783e75389f03863c7be7c565efd911:12
openstackgerritMerged openstack/openstack-ansible-os_nova stable/ocata: Zuul: Remove project name  https://review.openstack.org/54285111:16
evrardjpTESCRA: did you configure NFS yourself?11:27
TESCRAevrardjp any more changes to be made ?11:28
TESCRAy something wrong in it?11:28
evrardjpTESCRA: yes, I am writing your file11:29
TESCRA?11:29
evrardjphttps://gist.github.com/evrardjp/8385b2a5ba0d160dc948f900e4d905f011:29
evrardjpTESCRA: ^11:33
evrardjpyou should probably cleanup your inventory and start from scratch again11:33
evrardjpin case you had an overlapping IP11:33
evrardjpso destroying all your containers, and restarting.11:34
evrardjpI mean restarting the process.11:34
TESCRAthanks evrardjp11:35
TESCRAshall i delete user secrets too have removed openstack_inventory.json & backup_openstack_inventory.tar11:37
TESCRAran lxc ontainerdelete.yml11:37
TESCRA11:38
TESCRAlxc-containers-destroy.yml11:39
TESCRAwill it suffice ?11:39
TESCRAor shall i format deploy host?11:40
TESCRAevrardjp?11:40
evrardjpto start from scratch you should start by removing the containers11:42
evrardjplxc-containers-destroy would do the trick11:42
*** sxc731 has joined #openstack-ansible11:42
*** stuartgr has joined #openstack-ansible11:42
evrardjpthen you should remove your openstack_inventory, hostname ips, and basically everything that's not user_*.yml and openstack_user_config.yml11:42
*** SON has quit IRC11:47
*** taseer1 has joined #openstack-ansible11:50
*** Taseer has quit IRC11:51
*** taseer2 has joined #openstack-ansible11:52
*** taseer1 has quit IRC11:55
armaanodyssey4me: hello!11:56
evrardjphello armaan11:57
evrardjp:)11:57
evrardjphow are things ?11:57
armaanevrardjp: hello :)11:57
armaanevrardjp: not bad, how about you?11:57
evrardjpguess what, today I was proactive, and it's now doing an aio upgrade for you.11:58
armaanevrardjp: wow, thanks much :)11:58
evrardjpmaybe I should have your extra config to test magnum :p11:58
armaanevrardjp: sure, let me paste it for ya!11:58
evrardjpwell it's just one screen window I'm not doing much, I just let the electrons flow.11:58
evrardjpwhat's up on your side armaan ?12:00
evrardjpI mean outside you waiting for us having stable upgrades and all :p12:00
*** hamza21 has quit IRC12:01
armaanevrardjp: Planning to install Magnum(Pike) in Ocata and some Ceph performance tuning :)12:01
evrardjpWOW, living the life I see!12:01
evrardjpinstalling magnum Pike on ocata seems a bad idea12:02
evrardjpfor requirements and all12:02
armaanAnd also, planning for upgrading two environments from Ocata to Pike as soon as i get the signal from you :)12:02
evrardjpI'd say I am more confident about the upgrade as I was two weeks ago, and two days ago with the mariadb issue12:02
armaanYep, ran in to some errors12:03
evrardjparmaan: remember to not update your apt sources unless you're on the latest code from O/P/master12:03
armaancool, you fixed the mariadb issue -- nice!12:03
evrardjpwell, we froze to 10.1.3012:03
evrardjpso you'll have two repos, might worth cleaning yours12:04
evrardjparmaan: https://docs.openstack.org/releasenotes/openstack-ansible/ocata.html#id112:04
armaanoh, thank you for documenting that info -- very useful for people like me12:05
armaanso these overrides should be set before starting the point upgrade otherwise mariadb will complain12:08
*** dave-mccowan has joined #openstack-ansible12:09
evrardjpafk lunch12:11
odyssey4meyep, I'm looking into how to properly configure the pike magnum on ocata for you - things that should work aren't, so I'm mucking with it a bit to see whether there's a bug or I'm being dumb12:11
armaanodyssey4me: roger that! thanks :)12:13
*** gkadam has quit IRC12:14
openstackgerritMerged openstack/openstack-ansible master: l3_agent should only appear in computes when dvr  https://review.openstack.org/54131012:17
evrardjpthx odyssey4me12:26
*** acormier has joined #openstack-ansible12:27
*** acormier has quit IRC12:28
*** acormier has joined #openstack-ansible12:28
*** olivierbourdon38 has joined #openstack-ansible12:34
*** acormier has quit IRC12:35
*** hw_wutianwei has joined #openstack-ansible12:41
*** hamza21 has joined #openstack-ansible12:45
evrardjpthe update sha patch doesn't seem to work on centos, but seems to work on ubuntu12:47
evrardjpI don't like this12:47
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: Test keystone with a memcached pool  https://review.openstack.org/54000112:57
spotzthanks logan-13:00
odyssey4mearmaan I haven't verified whether it makes a working system, but https://gist.github.com/odyssey4me/fc69b1eb68f250e37815246e37fd13f2 at least completes the repo build :)13:04
*** armaan_ has joined #openstack-ansible13:05
odyssey4mearmaan_ ^13:05
armaan_odyssey4me: O/13:05
armaan_odyssey4me: i was offline for a while, did i miss any message from you?13:06
odyssey4mearmaan_ I haven't verified whether it makes a working system, but https://gist.github.com/odyssey4me/fc69b1eb68f250e37815246e37fd13f2 at least completes the repo build :)13:06
armaan_odyssey4me: cool, let me try this...13:07
*** armaan has quit IRC13:08
TESCRAevrardjp shall i remove ansible_facts folder too as i assume it has mapping for containers and ips?13:14
evrardjpyes13:16
evrardjpTESCRA: like I said, everything except your user_*.yml and your openstack_user_config .13:16
*** kstev has joined #openstack-ansible13:24
*** kstev1 has joined #openstack-ansible13:26
armaan_odyssey4me: looks good :) http://paste.openstack.org/show/669891/ moving on to the next step i.e. installing magnum13:27
*** kstev has quit IRC13:28
TESCRAyup i've removed everthing recreated the folder thanks evrardjp13:29
mgariepymorning everyone13:32
mgariepyevrardjp, what patch on centos ?13:33
evrardjpthe sha bump13:34
evrardjphttps://review.openstack.org/#/c/543131/13:34
mnaserhm13:36
mnaserbunch of timeouts at gate yesterday13:36
mnaseranything documented or just throw a recheck?13:36
sarAm i correct in assuming that setup-openstack.yml just calls os-*-install.yml? I should be able to use the os-*-install playbooks separately, right?13:38
*** vnogin has joined #openstack-ansible13:43
odyssey4mesar yep13:43
*** acormier has joined #openstack-ansible13:45
sarGreat, thanks!13:46
*** woodard has quit IRC13:47
*** eumel8 has quit IRC13:48
*** woodard has joined #openstack-ansible13:48
*** acormier has quit IRC13:49
*** vnogin has quit IRC13:57
*** acormier has joined #openstack-ansible13:57
mhaydenhappy monday14:00
*** acormier has quit IRC14:02
mgariepyola mhayden14:03
*** hachi has joined #openstack-ansible14:06
TESCRAevrardjp cloudnull despite of restarting from scratch am still facing the same issue on setuphost.yml exe FAILED - RETRYING: Ensure that the LXC cache has been prepared (40 retries left). FAILED - RETRYING: Ensure that the LXC cache has been prepared (39 retries left). FAILED - RETRYING: Ensure that the LXC cache has been prepared (38 retries left).14:07
*** hamza21 has quit IRC14:09
spotzhey mhayden14:09
evrardjpTESCRA: wait for the task to finish before saying it has definitely failed14:09
odyssey4meTESCRA that's normal, it should eventually pass - if it doesn't, check the log file mentioned previously on the lxc host to see why the cache prep failed14:09
evrardjpdo not control -c14:09
odyssey4meit retries for 20 mins IIRC and only fails if it did not complete in that time14:10
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_magnum stable/pike: Fix cinder_service_region undefined  https://review.openstack.org/54346514:12
*** neith has quit IRC14:12
*** neith has joined #openstack-ansible14:12
mhaydenevrardjp: thanks for working on that upstream galera bug ;)14:13
openstackgerritMohammed Naser proposed openstack/openstack-ansible stable/pike: Fix cinder_service_region undefined  https://review.openstack.org/54346714:15
evrardjpmhayden: I am not really working on it14:16
evrardjpI just saw an update, thought it was wrong ... So I registered on their jira to explain what I see :p14:17
evrardjpregistering was the hardest part for me.14:17
evrardjpSpiritually intensive.14:17
evrardjp:D14:17
*** sxc731 has quit IRC14:19
*** udesale has joined #openstack-ansible14:20
odyssey4meif any cores have a bit of time for reviews, I've got a ton of them waiting due to some weekend fun ;) https://review.openstack.org/#/q/owner:jesse-pretorius+status:open+label:Verified%253E%253D1%252Czuul+NOT+label:Workflow%253C%253D-1+NOT+label:Code-Review%253C%253D-114:20
TESCRAevrardjp it had failed again !! odyssey4me the log file for lxc create has only 1 err "E: Can not write log (Is /dev/pts mounted?) - posix_openpt (2: No such file or directory) (Reading database ... 13031 files and directories currently installed.)" had used apt-get dist-update to fix but then i faced diskqouta issue !!14:22
TESCRAam i repeating the same mistake?14:22
evrardjpodyssey4me: wow.14:22
openstackgerritMerged openstack/openstack-ansible-pip_install master: Set pip to use the pypi-server when locked down  https://review.openstack.org/52688914:23
*** udesale_ has joined #openstack-ansible14:24
*** udesale has quit IRC14:25
idlemindany thought on how hard / crazy it would be to add ipv6 internal / external urls (or just use a hostname at that point?)14:37
odyssey4meuse a hostname for your public endpoints, and have the ipv6 addresses bound on the LB14:38
*** armaan has joined #openstack-ansible14:39
idlemindodyssey4me thx the loadbalancer (haproxy) would get configured w/the hostname in that case so 2 vips wouldn't need to be created a v4 and v6 one?14:39
odyssey4meI know we've looked into ipv6 end-to-end before, but various parts of openstack weren't able to do it yet... not sure of the current state, but I'm pretty sure that if the whole stack can do it now then we'd be happy to entertain patches to do it14:39
idlemindi'm an haproxy newb, f5 i'd need a vip for each address family14:39
*** lbragstad has joined #openstack-ansible14:39
idlemindcool cool14:40
idlemindya i'm an ipv6 crazy14:40
odyssey4mewe've discussed several times how we'd prefer to actually just ditch ipv4 and go ipv6 all the way14:41
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_hosts master: Clean-up old systemd prep and allow machinctl to grow  https://review.openstack.org/52759214:42
*** armaan_ has quit IRC14:42
idlemindya i suppose you could replace rfc1918 w/the ipv6 documentation prefix14:42
*** sxc731 has joined #openstack-ansible14:45
mnaserhas anyone done some horizon tweaking...? TTFB for api requests (pages running angular, so just plain json response) is 3.86 seconds :14:49
mnaser:\14:49
evrardjpidlemind: hey. I still advise for separation of the networks, so you should probably have two different networks for it in ipv6.14:49
evrardjpmnaser: which branch?14:49
evrardjpIs your keystone fine?14:49
idlemindevrardjp different networks as in w/in openstack over a single "ext-net" w/a v4 and v6 subnet?14:50
evrardjpcould you do api calls in the cli, see if it's better?14:50
idlemindor outside?14:50
evrardjpidlemind: internal management network different than public api access network14:50
mnaserevrardjp: stable/pike, `openstack token issue --timings` => 0.711695 total14:50
evrardjpmmm14:50
idlemindevrardjp oh ya that makes sense14:50
idlemindcan still be accomplished w/gua ipv6, just make sure routing and security rules are in place accordingly14:51
*** mislukking has joined #openstack-ansible14:51
evrardjpidlemind: I try to, generally :)14:51
evrardjprouting is not the same as l2+l3 isolation.14:51
odyssey4mecloudnull a minor tweak on https://review.openstack.org/527592, but also a question there :)14:51
openstackgerritMerged openstack/openstack-ansible-galera_client stable/pike: Set proper yum repo priority on CentOS 7  https://review.openstack.org/54342114:51
evrardjpremember they will have a same local link address :D14:51
evrardjpwell long story that I will avoid for today14:52
*** acormier has joined #openstack-ansible14:52
idlemindoh i suppose if your "external" ipv6 ip is also on the br-mgmt internal side14:52
idlemindi already created a separate vlan for my "external" access and my br-mgmt is internal w/a firewall between the "external" and "internal" legs14:53
idlemindin ipv4 land14:53
idlemindbut i suppose that's normal14:53
*** bhujay has quit IRC14:53
idlemindas i'm learning more about how the docs setup networking and what not14:53
matttthe gate timeouts, that just indicates that the run took greater than X mins right?14:55
*** woodard has quit IRC14:57
*** esberglu has joined #openstack-ansible14:59
*** acormier has quit IRC15:05
*** acormier has joined #openstack-ansible15:05
TESCRAodyssey4me will this cmd work mount /dev/pts -o remount,gid=5,mode=620 am not sure15:05
TESCRAalso would appreciate if anyone can tell me where am i going wrong https://gist.github.com/ankita-anil-verma/2f2c9118d55513c731dcc9cbb339026215:06
TESCRAits been a loop15:06
TESCRAnow am facing disk quota issue...15:07
*** eumel8 has joined #openstack-ansible15:11
*** mislukking has quit IRC15:15
mgariepyevrardjp,  replied your comment in https://review.openstack.org/#/c/542994/15:15
*** hachi has quit IRC15:17
*** sar has quit IRC15:18
*** jwitko has joined #openstack-ansible15:19
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_nova master: Stop running non-voting jobs in gate  https://review.openstack.org/54348615:20
armaanodyssey4me: It worked! You're awesome :D ... thanks a million.15:21
*** Leo_m has joined #openstack-ansible15:22
TESCRAodyssey4me Is threrea feasible solution have tried removing machine.raw frm /var/lib15:23
TESCRAits been three days n am still stuck at same issue recurrent loop!!15:24
*** mislukking has joined #openstack-ansible15:27
idlemindtescra if you just rerun does infra3 complete? are you out of disk space on infra3? maybe blow away infra3 and start it over, reinstall os, clear it out of the inventory (and it's containers)15:29
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_hosts master: Clean-up old systemd prep and allow machinctl to grow  https://review.openstack.org/52759215:30
xdfiltrying to understand why sometimes haproxy_ssl_cert is being ignored15:31
*** cloudnull has quit IRC15:32
TESCRAidlemind have formated infra3 4 times also there is 500GB HDDin it15:32
idlemindtescra interesting, same os as infra2 and infra1?15:33
TESCRAyup15:33
TESCRAno issue with those 2 nodes15:33
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server stable/pike: Join cluster without using a failed task  https://review.openstack.org/54341215:35
eumel8evrardjp: you have 5 minutes for 2 questions?15:36
*** cloudnull has joined #openstack-ansible15:36
TESCRAroot@infra2:/home/osadmin# uname -r 4.4.0-112-generic15:38
TESCRAcloudnull can u please suggest a way out am facing the same issues can tell me where am i going wrong https://gist.github.com/ankita-anil-verma/2f2c9118d55513c731dcc9cbb339026215:39
idlemindtescra and each time you get to that point and it fails on lxc cache prep?15:40
idleminddid you build infra2 and infra1 succesfully past that point and update the osa repo in anyway?15:40
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_nova master: Add SELinux contexts for nova log directory  https://review.openstack.org/54349915:40
TESCRAyes then i go into infra 3 do dist-upgrade fix missing15:40
TESCRAyes infra1&2 had no issue15:41
idlemindtescra, i meant the openstack-ansible git repo, did you pull in a commit or fix after infra1 and 2 were built past this lxc stage?15:41
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server master: Implement separate deploy and upgrade tests  https://review.openstack.org/54312915:41
idlemindi'm wondering if you introduced a change that subsequently broke that task15:41
TESCRAno not even once15:42
idlemindk then i'm stumped as to why 2 of your hosts would get through the lxc-cache prepare task and 1 wouldn't when all the variables are the same (os, disk)15:42
cloudnullTESCRA: is /var/lib/machines mounted?15:43
cloudnull`systemctl status var-lib-machines`15:43
cloudnull`systemctl status var-lib-machines.mount`15:43
TESCRAso am i n it's been 3 days ina row15:43
TESCRAcloud null used mount /dev/pts -o remount,mode=620 cmd15:44
TESCRAto do that not sure if its correct15:44
cloudnull`systemctl restart var-lib-machines.mount`15:44
cloudnullthe machines mount is managed by systemd15:44
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/pike: Update rabbitmq install for cluster and upgrade tests  https://review.openstack.org/54350015:44
*** woodard has joined #openstack-ansible15:46
TESCRAcloudnull had deleted so ConditionPathExists=/var/lib/machines.raw15:47
TESCRAfails15:47
TESCRAinactive15:47
TESCRAConditionPathExists=/var/lib/machines.raw was not met15:48
TESCRAhttps://bugs.launchpad.net/openstack-ansible/+bug/174536115:50
openstackLaunchpad bug 1745361 in openstack-ansible "Failed to create subvolume /var/lib/machines/ when run 'openstack-ansible setup-hosts.yml' multiple times" [Undecided,In progress] - Assigned to Kevin Carter (kevin-carter)15:50
TESCRAcloudnull as per its instruction had deleted machines.raw.....15:50
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Implement separate deploy and upgrade tests  https://review.openstack.org/54350615:52
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Implement separate deploy and upgrade tests  https://review.openstack.org/54350615:53
*** olivierbourdon38 has quit IRC15:53
*** udesale has joined #openstack-ansible15:53
*** udesale_ has quit IRC15:53
*** armaan has quit IRC15:54
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Implement separate deploy and upgrade tests  https://review.openstack.org/54350615:54
*** armaan has joined #openstack-ansible15:54
openstackgerritMajor Hayden proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Use systemd module instead of command  https://review.openstack.org/54350915:55
openstackgerritMajor Hayden proposed openstack/openstack-ansible-openstack_hosts stable/pike: Ensure openstack_hosts sets nodepool variables  https://review.openstack.org/54351115:57
*** kstev1 has quit IRC15:59
openstackgerritJames E. Blair proposed openstack/openstack-ansible-tests stable/ocata: Zuul: Remove project name  https://review.openstack.org/54351316:01
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_neutron master: Stop running non-voting jobs in gate  https://review.openstack.org/54351416:02
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_ironic master: Stop running non-voting jobs in gate  https://review.openstack.org/54351516:03
*** pcaruana has quit IRC16:04
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_aodh master: Stop running non-voting jobs in gate  https://review.openstack.org/54351616:04
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_cinder master: Stop running non-voting jobs in gate  https://review.openstack.org/54351716:05
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_designate master: Stop running non-voting jobs in gate  https://review.openstack.org/54351816:06
*** sxc731 has quit IRC16:07
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server master: tests: Remove duplicated container creation  https://review.openstack.org/54351916:07
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_glance master: Stop running non-voting jobs in gate  https://review.openstack.org/54352016:07
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_gnocchi master: Stop running non-voting jobs in gate  https://review.openstack.org/54352116:08
*** sxc731 has joined #openstack-ansible16:09
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_heat master: Stop running non-voting jobs in gate  https://review.openstack.org/54352216:09
*** mislukking_ has joined #openstack-ansible16:14
*** mislukking has quit IRC16:17
*** kmARC has quit IRC16:19
*** kmARC has joined #openstack-ansible16:20
evrardjpeumel8: ofc!16:20
*** mislukking_ has quit IRC16:20
evrardjphow can I help?16:21
eumel8evrardjp: argh, just closed the tab ;)16:24
evrardjpsorry if I was too slow!16:24
TESCRAcloudnull had reexe setup now the systemctl status var-lib-machines.mount is active16:24
eumel8evrardjp: first question was, if you want to talk about or doing setup project doc translation during PTG?16:25
eumel8evrardjp: already added to the etherpad, but there is no timetable for the topics: https://etherpad.openstack.org/p/osa-rocky-ptg16:25
TESCRAbut the setuphost failed with same err FAILED - RETRYING: Ensure that the LXC cache has been prepared (1 retries left). fatal: [infra3]: FAILED! => {"ansible_job_id": "374029639326.8072", "attempts": 60, "changed": false, "failed": true, "finished": 0, "started": 1}16:26
TESCRAcloudnull what shall i do next?16:27
odyssey4meTESCRA look at the prep log file to see why it failed16:28
evrardjpeumel8: yes, I am sorry it took so long. I said q1 2018, and we are almost out of time in q1.16:28
evrardjpI'd like odyssey4me to join us on that.16:29
odyssey4meevrardjp ?16:29
evrardjpodyssey4me: it's a new base job for translations that need to hook up into things.16:29
evrardjpAs an infra liaison, and because of the fact we'll need to hook up with infra jobs, I thought it would be good if we could work on this16:29
evrardjpnot aware of the full implications but AJaeger could help too IIRC.16:30
evrardjpeumel8: you did great proposing the session! The timetable isn't set, but I will make time for you. This is important.16:31
evrardjpat least to me16:31
odyssey4meoh sure - it probably makes sense to actually turn that into an all metal build16:31
evrardjp:D16:31
eumel8evrardjp: yeah, make the osa french ;-)16:32
evrardjplol no.16:32
evrardjpbut I am all for the accessibility.16:33
mhaydenif someone has a moment to help me get a ansible-hardening bugfix moving along... :) https://review.openstack.org/54139816:33
evrardjpreducing the barrier of entry is good.16:33
eumel8On Thursday of the PTG week I'm free, so no special appointments for me16:33
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_horizon master: Stop running non-voting jobs in gate  https://review.openstack.org/54353016:34
mhaydenmany thanks spotz  and evrardjp  :)16:34
evrardjpeasy to review16:34
evrardjpI thank you!16:34
mhaydenmore green in the centos jobs lately http://zuul.openstack.org/builds.html?job_name=openstack-ansible-functional-centos-716:34
evrardjpI will then have to complain about something, I don't know what yet.16:34
*** jamespage has quit IRC16:34
evrardjpmhayden: :D16:34
idlemindi'd say16:34
evrardjpI need help on something16:34
*** Guest27554 has quit IRC16:34
*** jamespage has joined #openstack-ansible16:35
mhaydeneven with getting goosed by selinux enforcing :P16:35
evrardjpI am a little stuck16:35
odyssey4meI hear that more +2's makes it merge faster16:35
spotzNP, caught me between videos16:35
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_barbican master: Stop running non-voting jobs in gate  https://review.openstack.org/54353116:35
evrardjpodyssey4me: lol.16:35
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_keystone master: Stop running non-voting jobs in gate  https://review.openstack.org/54353216:36
mhaydenevrardjp: for why are you stuck16:36
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249116:36
mhaydenevrardjp: do you need bail money?16:36
openstackgerritAndy McCrae proposed openstack/openstack-ansible master: Add base kit scenario  https://review.openstack.org/54353416:37
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_magnum master: Stop running non-voting jobs in gate  https://review.openstack.org/54353516:37
evrardjpmhayden: yes. In SSL tokens please.16:37
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_octavia master: Stop running non-voting jobs in gate  https://review.openstack.org/54353716:40
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_sahara master: Stop running non-voting jobs in gate  https://review.openstack.org/54353916:41
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_tacker master: Stop running non-voting jobs in gate  https://review.openstack.org/54354116:42
*** kstev has joined #openstack-ansible16:42
openstackgerritPaul Belanger proposed openstack/openstack-ansible-os_trove master: Stop running non-voting jobs in gate  https://review.openstack.org/54354216:42
TESCRAodyssey4me log file has nothing new status of systemctl is   Active: active (mounted) since Mon 2018-02-12 08:09:06 PST; 32min ago Condition: start condition failed at Mon 2018-02-12 07:22:37 PST; 1h 18min ago            ConditionPathExists=/var/lib/machines.raw was not met     Where: /var/lib/machines      What: /dev/loop0     Tasks: 0 (limit: 8192)    Memory: 0B       CPU: 016:43
*** dariko has joined #openstack-ansible16:43
evrardjpcloudnull: why has TESCRA all these problems with machinectl under stable/pike? That sounds very weird to me, because we have dir or overlayfs as default in stable/pike, right?16:44
evrardjpeumel8: we'll do it on Thursday. I don't have a planning yet for that -- if you prefer it to be set in stone before the PTG, then we can decied the timeframe now16:45
TESCRAshall i exe openstack-ansible lxc-hosts-setup.yml -e lxc_image_cache_refresh=yes -vvv --limit infra3?16:46
eumel8evrardjp: don't worry, as I already mentioned I'm free the whole day. I would like to join you in the morning to your retro session but I have no requirements when we do the doc thing.16:49
*** olivierbourdon38 has joined #openstack-ansible16:49
*** acormier has quit IRC16:52
*** acormier has joined #openstack-ansible16:52
*** sxc731 has quit IRC16:52
openstackgerritMerged openstack/ansible-hardening master: Move aide db when needed  https://review.openstack.org/54139817:00
TESCRAPlease help me resolve / understand the exact issue in gist https://gist.github.com/ankita-anil-verma/2f2c9118d55513c731dcc9cbb339026217:04
TESCRAam at stable/pikefaing issue with setuphost.yml17:05
cloudnullsorry just getting back from meetings this mornings17:06
cloudnullevrardjp: the machinectl backend is there for basic image store17:07
idlemindtescra did you try running that openstack-ansible lxc-hosts-setup.yml -e lxc_image_cache_refresh=yes -vvv --limit infra317:07
evrardjpwell lxc doesn't need it17:07
cloudnullits been there since newton?17:08
evrardjpwell we won't come back on this17:08
TESCRA?17:08
ivveanyone else had ha-tenant network dhcp agents die and never restart without any logtraces?17:08
evrardjpivve: no17:08
evrardjpmemory issue?17:08
evrardjpcloudnull: so it's normal :)17:08
ivve(also restarting neutron-l3/dhcp-agents didn't help17:08
ivve)17:09
*** SerenaFeng has joined #openstack-ansible17:09
cloudnullthe images are downloaded and stored at /var/lib/machines which has been there for image management. in pike we added the abiltity to run all of the containers from that location, along with using native zfs and btrfs17:09
TESCRAidlemind not yet cause when i did that earlier had to face disk qouta issue17:10
ivvenoticed that assigning flips worked but they never got to the qrouter namespace.. wierd added them now its all good17:10
ivveso confusing :D17:10
TESCRAivve :)17:10
cloudnullTESCRA: did this cloud start life out as an AIO17:10
cloudnull?17:10
TESCRAno17:11
ivvebtw i saw many of my reported pike/centos stuff got merged to stable, thanks all involved! (galera, lbaas and stuff)17:11
TESCRAit was always production appendix B17:11
cloudnullok17:11
cloudnullso is the issue fixed after you restarted the machiens mount?17:12
TESCRAcloudnull shall i exe openstack-ansible lxc-hosts-setup.yml -e lxc_image_cache_refresh=yes -vvv --limit infra3 ?17:12
TESCRAnot sure as i had perviously deleted machine.raw onreexe of scrpit got it back n have mounted machin17:14
cloudnullok17:14
TESCRAdonno what 2 do nxt17:14
cloudnullclean out the lxc download cache as well and then rerun that playbook.17:14
ivveoh btw. in new centos or ubuntu deployments (stable/pike), using the openstack client in the utility container: "alarm list" outputs and SSL error with a bad handshake... am i missing something?17:14
cloudnull`/var/cache/lxc/downloads/*`17:15
ivve(im fiddling with aodh,ceilometer,gnocchi and trying to learn metrics/alarms)17:15
cloudnullivve: maybe it's trying to use SSL on the internalURL ?17:15
odyssey4meevrardjp has been seeing and troubleshooting something similar17:16
ivvecloudnull: acutally no17:16
ivveits the external17:16
TESCRAopenstack-ansible lxc-hosts-setup.yml -e lxc_image_cache_refresh=yes -vvv --limit infra3 will clean it or shall i use rm -rf /var/cache/lxc/downloads/*17:16
ivveUnable to establish connection to https://x.x.x.x:8042/v2/alarms17:17
*** udesale has quit IRC17:17
evrardjpivve: what's the reason ?17:17
evrardjpoh yeah bad handshake17:17
ivvemetrics seems fine but querying for status gives http 50017:17
cloudnulldoes it need an insecure flag?17:17
evrardjpI am in it right now.17:17
evrardjpcloudnull: yea it's probably that17:17
evrardjpI don't know if we are talking about the same thing though17:18
evrardjpivve: oh yeah, stable/pike openstack client bad handshake17:18
evrardjpyeah that looks like the same thing17:18
ivveevrardjp: check!17:18
cloudnullTESCRA: yes.17:18
cloudnullthen rerun the playbook.17:18
cloudnull`openstack-ansible lxc-hosts-setup.yml -e lxc_image_cache_refresh=yes --limit $NODE`17:18
evrardjpivve: I am checking in the openstack sdk17:19
ivveevrardjp: reason for me asking is that i think heat is having issues17:19
evrardjpwill give feedback17:19
evrardjpmmm17:19
evrardjpYeah I am not surprised.17:19
ivveevrardjp: ok cool, deploying heat templates with autoscaling or something using metrics/aodh is borked :(17:19
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249117:19
ivveevrardjp: is there a bug submitted?17:20
evrardjpafk sorry17:21
ivvenp :)17:21
ivveevrardjp: hmm using "aodh alarm list" gives a http 50317:22
ivveno server avail17:22
cloudnullyou can disable that with `btrfs quota disable /var/lib/machines`17:22
cloudnullopps wrong window17:23
TESCRAbtrfs quota disable /var/lib/machines 4 me?17:23
ivveno matter --insecure or not17:23
cloudnullTESCRA: yes.17:24
cloudnullwas replying to your PM but we can talk here too :)17:24
ivvecontrollers are listening on the correct ports 8042 and haproxy/keepalived looks ok. not sure about haproxy stanzas for aodh though17:24
ivve(running ha with 3 controllers)17:25
ivvewho came up with the name for the alarming service lol :P17:26
*** acormier_ has joined #openstack-ansible17:27
*** acormier has quit IRC17:27
openstackgerritMerged openstack/openstack-ansible-galera_server stable/pike: Restore upgrade check & gate  https://review.openstack.org/54323517:32
openstackgerritMerged openstack/openstack-ansible-galera_server stable/pike: tests: test-galera-upgrades.sh: Remove the exit trap  https://review.openstack.org/54322217:32
*** sxc731 has joined #openstack-ansible17:39
*** hw_wutianwei_ has quit IRC17:39
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/pike: tests: test-keystone-upgrades.sh: Remove the exit trap  https://review.openstack.org/54146817:46
*** sxc731 has quit IRC17:47
TESCRAcloudnull u there?17:47
*** ullnk1 has joined #openstack-ansible17:49
*** ullnk1 has quit IRC17:52
evrardjpok back17:59
*** hamza21 has joined #openstack-ansible18:00
openstackgerritJames E. Blair proposed openstack/openstack-ansible-apt_package_pinning stable/newton: Zuul: Remove project name  https://review.openstack.org/54356518:02
*** Leo_m has quit IRC18:06
*** Leo_m has joined #openstack-ansible18:07
*** Leo_m has quit IRC18:11
*** vnogin has joined #openstack-ansible18:11
*** Leo_m has joined #openstack-ansible18:19
*** Leo_m has quit IRC18:24
*** hamza21 has quit IRC18:25
*** hamza21 has joined #openstack-ansible18:26
*** vnogin has quit IRC18:29
*** Leo_m has joined #openstack-ansible18:29
*** Leo_m has quit IRC18:29
*** Leo_m has joined #openstack-ansible18:31
*** stuartgr has quit IRC18:31
*** peri has quit IRC18:33
*** Leo_m has quit IRC18:34
*** electrofelix has quit IRC18:34
hwoaranghas anyone seen this kind of prob before "ERROR! Unexpected Exception, this is probably a bug: 'InventoryManager' object has no attribute 'get_group_dict'" ?18:36
hwoaranggetting that when running the tox tests in vagrant18:37
*** electrofelix has joined #openstack-ansible18:37
hwoaranghmm https://github.com/ansible/ansible/issues/3103218:37
hwoarang'custom strategy plugin...'18:38
openstackgerritMerged openstack/openstack-ansible stable/pike: Remove impossible tests  https://review.openstack.org/54338418:43
openstackgerritMerged openstack/openstack-ansible master: Reinstate ubuntu ceph gating  https://review.openstack.org/54122218:43
openstackgerritMerged openstack/openstack-ansible master: Move setup-utility before other plays  https://review.openstack.org/54170918:43
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Make connection plugin compatible with Ansible 2.5  https://review.openstack.org/54357618:44
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Make connection plugin compatible with Ansible 2.5  https://review.openstack.org/54357618:47
evrardjphwoarang: yes we have one!18:49
hwoarangevrardjp: seems like i had some old data in my repos18:50
*** sar has joined #openstack-ansible18:50
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Make connection plugin compatible with Ansible 2.5  https://review.openstack.org/54357618:50
evrardjpyeah I'd say you have not the right version of plugins for your ansible18:51
evrardjpjmccrory, always ahead :)18:51
evrardjpjmccrory: working on the magic variables thing?18:51
*** electrofelix has quit IRC18:51
evrardjphaha yes you are!18:51
evrardjpyou're the deal!18:52
evrardjp:D18:52
odyssey4meperhaps we should try to kill all the warnings from ansible 2.4 before we go there ;)18:52
evrardjphaha yeah, maybe.18:52
evrardjpI am still fixing Ocata to pike right now, don't tell me more things about the future!18:52
evrardjp:D18:52
jmccroryevrardjp : yep. that was able to run through the lxc container create play, but still need some role changes for 2.518:52
evrardjpoh interesting18:52
evrardjpwill you do an AnsibleLooseVersion in those plugins?18:53
evrardjpthis way we are ready for both?18:53
jmccrorystat's get_md5 default change to False for example18:53
evrardjpI have to go for dinner.18:53
jmccroryyeah, it's using LooseVersion. wanted to push it up to make sure 2.4 still works18:53
evrardjpDarn those changes!18:53
evrardjpgreat!18:53
evrardjpWill see in the review tomorrow18:53
jmccrorybest part is none of plugin changes are documented in changelog or porting guide18:54
evrardjpas always  :)18:54
evrardjpgrrr18:54
evrardjpsee you tomorrow guys!18:54
jmccrorysee ya18:55
evrardjpI meant folks ofc :)18:55
hwoarangcu!18:55
ivvebye :)18:55
openstackgerritMerged openstack/openstack-ansible stable/pike: Remove jinja2 delimiters in when  https://review.openstack.org/54139418:55
openstackgerritMerged openstack/openstack-ansible stable/ocata: bootstrap-host: Create swap without using a failed task  https://review.openstack.org/54341118:55
nurdiehttps://docs.openstack.org/opentack-ansible-os_glance/latest doesn't specify where those "Default variables" should be set18:55
nurdieor even where they "could" be set18:56
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249118:57
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249118:59
*** agrebennikov has joined #openstack-ansible19:00
nurdieso uhhh, can anybody tell me where those should be set? lol19:01
*** Leo_m has joined #openstack-ansible19:03
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249119:04
openstackgerritMerged openstack/openstack-ansible-os_barbican master: Stop running non-voting jobs in gate  https://review.openstack.org/54353119:06
*** sxc731 has joined #openstack-ansible19:14
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249119:14
*** SerenaFeng has quit IRC19:19
*** hamza21 has quit IRC19:20
*** hamza21 has joined #openstack-ansible19:24
openstackgerritMerged openstack/openstack-ansible-os_horizon master: Stop running non-voting jobs in gate  https://review.openstack.org/54353019:28
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_neutron master: Add SELinux contexts for neutron log directory  https://review.openstack.org/54358819:28
odyssey4menurdie all variables to override default values can be set in /etc/openstack_deploy/user_variables.yml (global override), or using /etc/openstack_deploy/{host_vars,group_vars} if you need more targeted group overrides19:31
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_cinder master: Add SELinux contexts for cinder log directory  https://review.openstack.org/54358919:32
odyssey4menurdie just be aware that there are already some behaviours overriding some defaults set in https://github.com/openstack/openstack-ansible/tree/stable/pike/group_vars, for example - where we try to make some things easier by using the inventory to decide what would be a good default19:33
*** lbragstad_ has joined #openstack-ansible19:41
*** Leo_m_ has joined #openstack-ansible19:42
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Make connection plugin compatible with Ansible 2.5  https://review.openstack.org/54357619:42
*** lbragstad_ has quit IRC19:43
*** Leo_m has quit IRC19:43
*** pramodrj07 has joined #openstack-ansible19:49
openstackgerritMerged openstack/openstack-ansible-os_tacker master: Stop running non-voting jobs in gate  https://review.openstack.org/54354119:50
openstackgerritMerged openstack/openstack-ansible-lxc_hosts master: Clean-up old systemd prep and allow machinctl to grow  https://review.openstack.org/52759220:01
*** hybridpollo has joined #openstack-ansible20:04
cloudnullanyone around want to give this a  push https://review.openstack.org/#/c/477017/20:15
*** mbuil has quit IRC20:15
cloudnull / review20:16
idlemindhmm ansible os_image doesn't seem to support making an image protected huh20:19
*** esberglu has quit IRC20:19
openstackgerritMerged openstack/openstack-ansible-os_octavia master: Stop running non-voting jobs in gate  https://review.openstack.org/54353720:21
*** hamza21 has quit IRC20:26
*** esberglu has joined #openstack-ansible20:30
openstackgerritMerged openstack/openstack-ansible-os_magnum master: Stop running non-voting jobs in gate  https://review.openstack.org/54353520:31
*** hw_wutianwei has quit IRC20:31
*** hw_wutianwei has joined #openstack-ansible20:32
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_nova master: Add SELinux contexts for nova log directory  https://review.openstack.org/54349920:39
openstackgerritJames E. Blair proposed openstack/openstack-ansible-pip_install stable/newton: Zuul: Remove project name  https://review.openstack.org/54361020:43
cloudnullidlemind: is that from the shade modules?20:47
*** eumel8 has quit IRC20:48
openstackgerritMerged openstack/openstack-ansible-os_sahara master: Stop running non-voting jobs in gate  https://review.openstack.org/54353920:49
mnaserwould anyone have a sample review of updating a role to using uwsgi?20:59
mnasermagnum running with werkzeug has awful performance, so i want to update the role to use uwsgi20:59
logan-I might be out of date but the last uwsgi implementation i recall was the nova placement api addition21:01
*** ChanServ has quit IRC21:02
mnaserlogan-: looking more of a service that was transformed rather than added with uwsgi21:03
openstackgerritJames E. Blair proposed openstack/openstack-ansible-openstack_openrc stable/newton: Zuul: Remove project name  https://review.openstack.org/54361621:03
*** adcpdk has quit IRC21:03
mnaseri doubt anyone is using magnum roles in role based on what im seeing (some things just plain don't work)21:03
mnaserhttps://github.com/openstack/openstack-ansible-os_nova/commit/823a80bd444ae0607519d42c15e969802459c78e21:04
mnaseri think this is what im looking for21:04
mnaserhttps://github.com/openstack/openstack-ansible-os_heat/commit/2b5fe2ef790c452f69f9fc2373944f49f0c1e95921:05
mnaserthis is simpler :>21:05
logan-:)21:06
*** sxc731 has quit IRC21:07
*** ChanServ has joined #openstack-ansible21:11
*** barjavel.freenode.net sets mode: +o ChanServ21:11
idlemindcloudnull yes it is21:19
openstackgerritLogan V proposed openstack/openstack-ansible master: [WIP] Implement iptables management  https://review.openstack.org/51209921:19
idlemindcloudnull sorry got absorbed into an ip routing issue21:19
idlemindi might have to go the way of some others and use direct api calls for some things in my automation over the shade based os_* modules as handy as they are21:20
openstackgerritMerged openstack/openstack-ansible-os_designate master: Stop running non-voting jobs in gate  https://review.openstack.org/54351821:21
openstackgerritJames E. Blair proposed openstack/openstack-ansible-apt_package_pinning stable/ocata: Zuul: Remove project name  https://review.openstack.org/54362121:23
mnaser(im sorry)21:32
openstackgerritMerged openstack/openstack-ansible-os_glance master: Stop running non-voting jobs in gate  https://review.openstack.org/54352021:34
openstackgerritMerged openstack/openstack-ansible-os_trove master: Stop running non-voting jobs in gate  https://review.openstack.org/54354221:34
openstackgerritMerged openstack/openstack-ansible-os_gnocchi master: Stop running non-voting jobs in gate  https://review.openstack.org/54352121:34
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_magnum master: Restart only when needed  https://review.openstack.org/54362621:35
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_magnum master: Fix variables in systemd unit file  https://review.openstack.org/54362721:35
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_magnum master: Refactor services to magnum_services  https://review.openstack.org/54362821:35
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_magnum master: Implement uWSGI for Magnum API  https://review.openstack.org/54362921:41
openstackgerritJames E. Blair proposed openstack/openstack-ansible-lxc_container_create stable/ocata: Zuul: Remove project name  https://review.openstack.org/54363021:43
bgmccollumI'm running an AIO / Newton environment, and seeing the metadata service stop, after not being able to communicate with Rabbit. Bouncing the metadata service after Rabbit has had enough time to come up seems to fix the problem. Anyone else seen this / known issue?21:45
bgmccollumCan reproduce with `systemctl stop lxc` then `systemctl start lxc`21:45
openstackgerritMerged openstack/openstack-ansible-apt_package_pinning stable/newton: Zuul: Remove project name  https://review.openstack.org/54356521:46
bgmccollumActually, might be conductor is not up...21:48
cloudnull bgmccollum I've not specifically seen that issue.21:54
bgmccollumcloudnull -- Yeah, so looks like metadata API starts before conductor, sends a messages, doesn't get a reply, and the WSGI server stops -- http://paste.openstack.org/show/670659/21:55
bgmccollummight need more refinement to the lxc.start.order21:55
matttbgmccollum: hey!  i'm hitting this problem too21:55
bgmccollummattt ^5 (then cries)21:56
openstackgerritMerged openstack/openstack-ansible-os_ironic master: Stop running non-voting jobs in gate  https://review.openstack.org/54351521:57
openstackgerritMerged openstack/openstack-ansible stable/ocata: Remove jinja from failed_when  https://review.openstack.org/54139221:57
*** vnogin has joined #openstack-ansible22:00
*** admin0 has joined #openstack-ansible22:03
openstackgerritJames E. Blair proposed openstack/openstack-ansible-galera_client stable/ocata: Zuul: Remove project name  https://review.openstack.org/54363722:03
admin0\o22:03
admin0long time no talk :)22:03
cloudnullo/ admin022:03
admin0hi cloudnull22:04
*** vnogin has quit IRC22:05
*** woodard has quit IRC22:06
openstackgerritMerged openstack/openstack-ansible-tests stable/pike: Use a venv with previous ansible version for upgrades  https://review.openstack.org/54323622:08
*** threestrands has joined #openstack-ansible22:12
*** kstev has quit IRC22:26
*** masber has quit IRC22:29
*** dariko has quit IRC22:31
*** sar has quit IRC22:34
*** acormier_ has quit IRC22:36
*** acormier has joined #openstack-ansible22:36
*** admin0 has quit IRC22:38
*** acormier has quit IRC22:40
*** acormier has joined #openstack-ansible22:41
*** acormier has quit IRC22:46
*** pmannidi has joined #openstack-ansible22:47
*** jwitko has quit IRC23:03
*** vadim_ has quit IRC23:05
*** woodard has joined #openstack-ansible23:13
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_designate stable/ocata: Zuul: Remove project name  https://review.openstack.org/54365323:24
*** pbandark has quit IRC23:46
*** acormier has joined #openstack-ansible23:48
*** Leo_m_ has quit IRC23:52
*** Leo_m has joined #openstack-ansible23:52
*** acormier has quit IRC23:52
*** abelur has quit IRC23:54

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!