Tuesday, 2017-05-16

inc0I'm going back home, later guys00:02
kfox1111k. have a good one.00:03
*** goldyfruit has quit IRC00:09
*** oanson has quit IRC00:10
*** oanson has joined #openstack-kolla00:10
*** scheuk has joined #openstack-kolla00:13
*** scheuk has joined #openstack-kolla00:14
*** open_box has joined #openstack-kolla00:19
open_boxall,00:19
open_boxi amgetting an error when trying to use kolla-ansible pull00:19
open_boxi get00:19
open_boxfatal: [localhost] ....... Unknown Error message: manifest unknown...00:20
open_boxhas anyone a quick tip on this ?00:20
spsuryamorning00:24
*** goldyfruit has joined #openstack-kolla00:25
*** gomarivera has quit IRC00:25
*** goldyfruit has quit IRC00:26
*** m1dev has quit IRC00:32
*** jascott1 has quit IRC00:33
*** jascott1 has joined #openstack-kolla00:38
*** stefan_bo has joined #openstack-kolla00:42
*** stefan_bo has quit IRC00:47
*** chas has joined #openstack-kolla00:47
*** harlowja has joined #openstack-kolla00:49
openstackgerrityunfeng zhou proposed openstack/kolla master: add libvirt-python for instance monitor.  https://review.openstack.org/46454200:50
*** chas has quit IRC00:53
*** dixiaoli has joined #openstack-kolla00:54
*** dixiaoli has quit IRC00:55
*** dixiaoli has joined #openstack-kolla00:55
*** cuongnv has joined #openstack-kolla00:56
*** chas has joined #openstack-kolla01:09
*** chas has quit IRC01:13
*** masuberu has joined #openstack-kolla01:17
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Fix vpnaas in service_plugins  https://review.openstack.org/46429201:25
masuberuhi, I was testing deploying openstack+ceph using kolla-ansible01:26
masuberuprechecks, pull and deployment went fine with no errors01:26
masuberuhowever I don't see any container running on the ceph nodes01:26
openstackgerritAnthony Lin proposed openstack/kolla-kubernetes master: Script to Deploy Kolla Kubernetes AIO  https://review.openstack.org/46431201:26
masuberuI am using centos binary images01:27
*** duonghq has joined #openstack-kolla01:27
masuberuand 3 physical nodes for ceph01:27
duonghqmorning guys01:27
masuberuI did labeled the disks as suggested here https://github.com/inc0/kolla-ansible-workshop/blob/master/chapter2.md01:27
eanylinbmaltais: Been busy day for me at office, just started reading the messages. Thanks for pointing that out, I agree with your views and have updated the script with host IP information01:28
eanylinyou can actually give -1 and put your comments too. Thanks01:29
*** chas has joined #openstack-kolla01:30
eanylinmorning duonghq01:31
*** unicell has quit IRC01:31
*** chas has quit IRC01:34
bmaltaiseanylin: Nice. I added comments and -1 as suggested.01:38
eanylinbmaltais: Ok, need to start and look at the VIP thingy I guess :)01:39
bmaltaiseanylin: Ran in new error while deploying cirros to volume: Error: Failed to perform requested operation on instance "test3", the instance has an error status: Please try again later [Error: Could not login to any iSCSI portal.].01:39
bmaltaiseanylin: Might be related to the fact that I run it on a 2 node k8t setup instead of a pure AIO01:40
eanylinbmaltais: Oh, ok01:40
eanylinI think we might need to use ceph for that one, I havent tested it01:40
bmaltaiseanylin: I will have to go back to a single node to retest. There might be an issue with the iscsi storage for volume with the AIO.01:41
eanylinbmaltais: Ok01:41
bmaltaiseanylin: POssible... but I thought cinder could be used for block also.01:42
*** harlowja has quit IRC01:42
bmaltaiseanylin: Might be wrong here.01:42
eanylinbmaltais: Yeah, it should be able to, I tested volume attach previously, havent test boot from volume01:43
bmaltaisGot 2 go01:45
bmaltaisNight01:45
eanylinbmaltais: Ok. Night01:46
*** zhubingbing has joined #openstack-kolla01:49
*** chas has joined #openstack-kolla01:50
duonghqeanylin, did you find problem with AIO setup?01:53
eanylinduonghq: For Kolla Ansible or Kolla Kubernetes?01:54
duonghqiirc, yesterday you got trouble with kolla-ansible?01:55
*** chas has quit IRC01:55
*** scheuk has quit IRC01:55
eanylinduonghq: Ah, ok. I havent had time to re-try again01:56
*** goldyfruit has joined #openstack-kolla01:56
*** harlowja has joined #openstack-kolla01:56
*** rwsu has joined #openstack-kolla01:56
eanylinbut I didnt try that on cluster, so wasnt sure and I had some problems with ceph when deploying the cluster, didnt had time to test again on that too.... somehow I had strange behavior with glance when trying to create image with init-runonce script even though the ceph_mon containers are running fine... so a few things to look at I guess...01:59
eanylinits probably environmental issues since you guys didnt see any of the problems that I encountered...01:59
duonghqokay,02:00
eanylinyeah.. am still trying to learn ceph :)02:01
eanylinseems like we need it for most things to work properly ;)02:01
duonghqsure, ceph is important component, I know a guys deployed ceph successfully02:04
duonghqbut I haven't get used to ceph by myself02:04
eanylinyeah, agree02:08
*** chas has joined #openstack-kolla02:11
*** eanylin has quit IRC02:15
*** chas has quit IRC02:16
*** zhubingbing has quit IRC02:18
*** harlowja has quit IRC02:20
*** rwellum has joined #openstack-kolla02:29
*** rhallisey has joined #openstack-kolla02:30
*** rhallisey has quit IRC02:30
*** rhallisey has joined #openstack-kolla02:31
*** chas has joined #openstack-kolla02:32
*** goldyfruit has quit IRC02:34
*** chas has quit IRC02:37
*** gomarivera has joined #openstack-kolla02:37
*** gomarivera has quit IRC02:42
*** yingjun has joined #openstack-kolla02:43
*** ducttape_ has joined #openstack-kolla02:43
*** eanylin has joined #openstack-kolla02:47
*** ducttape_ has quit IRC02:48
*** itlinux has joined #openstack-kolla02:49
*** manheim has joined #openstack-kolla02:50
*** yuanying has quit IRC02:51
itlinuxhi all I get a question for some of you.. since i have not done much on kolla.. when I run a prechecks it fails on the compute-node.. I have it set in the inventory as the following [compute]02:51
itlinuxkolla-computeansible_ssh_user=root ansible_become=True02:51
itlinuxany suggestions.. here is the error02:51
itlinux[kolla-compute]: FAILED! => {"changed": false, "cmd": ["/usr/bin/python", "-c", "import docker; print docker.__version__"], "delta": "0:00:00.012310", "end": "2017-05-13 23:32:31.859391", "failed": true, "failed_when_result": true, "rc": 1, "start": "2017-05-13 23:32:31.847081", "stderr": "Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\nImportError: No module named docker", "stderr_lines":02:52
duonghqitlinux, you haven't install python-docker02:53
duonghqor docker-python02:54
*** manheim has quit IRC02:54
duonghqcannot recall atm02:54
itlinuxok let me check on that..02:54
itlinuxsuper duonghq: that worked.. nice02:55
itlinuxnice to know putting notes there.. now02:55
*** krtaylor has joined #openstack-kolla02:56
*** zhubingbing_ has joined #openstack-kolla03:02
openstackgerrityunfeng zhou proposed openstack/kolla master: add libvirt-python for instance monitor.  https://review.openstack.org/46454203:05
itlinuxany tips on this issue -- "NNING HANDLER [openvswitch : Waiting for openvswitch_db service to be ready] ************************************************************************************************************************************************03:07
itlinuxok: [kollal-controller]03:07
itlinuxFAILED - RETRYING: Waiting for openvswitch_db service to be ready (30 retries left).03:07
itlinux"03:07
openstackgerrityunfeng zhou proposed openstack/kolla master: add libvirt-python for instance monitor.  https://review.openstack.org/46454203:09
*** rwsu has quit IRC03:13
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Add heat_stack_owner role to admin project  https://review.openstack.org/46491303:13
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Add heat_stack_owner role to admin project  https://review.openstack.org/46491303:14
spsuryaitlinux: yes i have seen that before03:15
spsuryaIIRC03:15
spsuryawill you try by resolving the hostname of controller node03:15
*** itlinux has quit IRC03:18
duonghqor maybe proxy-related issue :P03:18
*** fooliouno has joined #openstack-kolla03:21
*** itlinux has joined #openstack-kolla03:22
*** jamesPR has joined #openstack-kolla03:30
itlinuxany tips on this openvswitch error  cannot create directory '/run/openvswitch': Permission denied03:31
itlinuxI just used kolla-ansible deploy and have the compute on sep node.. that's pretty much it..03:31
*** salv-orlando has joined #openstack-kolla03:38
jamesPRsup itlinux03:40
itlinuxjamesPR: not sure I follow your suggestions :)03:42
*** jamesPR has quit IRC03:45
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' to necessary tasks (general roles)  https://review.openstack.org/39868203:46
masuberugood afternoon all, I am testing kolla-ansible with ceph and I am getting this error message https://pastebin.com/raw/e17gSHSK03:52
masuberuany idea?03:52
*** skramaja has joined #openstack-kolla03:57
*** yuanying has joined #openstack-kolla03:58
masuberuok, my bad, forgot to uncomment the cinder module from glbals.yml04:05
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only necessary tasks (default roles)  https://review.openstack.org/39868404:12
*** yuanying has quit IRC04:23
*** yuanying has joined #openstack-kolla04:23
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only neccesary tasks (all other roles)  https://review.openstack.org/39868504:31
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only neccesary tasks (all other roles)  https://review.openstack.org/39868504:32
*** chas has joined #openstack-kolla04:37
*** chas has quit IRC04:42
*** unicell has joined #openstack-kolla04:43
*** stefan_bo has joined #openstack-kolla04:43
*** stefan_bo has quit IRC04:48
*** zhubingbing_ has quit IRC04:49
*** zhubingbing_ has joined #openstack-kolla04:49
*** strigazi_ has joined #openstack-kolla04:51
*** strigazi has quit IRC04:54
openstackgerritMerged openstack/kolla-ansible master: Fix vpnaas in service_plugins  https://review.openstack.org/46429204:55
*** strigazi has joined #openstack-kolla04:56
*** strigazi_ has quit IRC04:58
*** chas has joined #openstack-kolla04:58
openstackgerritMerged openstack/kolla-ansible master: Fix designate prechecks  https://review.openstack.org/45218005:03
*** rwsu has joined #openstack-kolla05:03
*** chas has quit IRC05:03
openstackgerritMerged openstack/kolla-ansible master: Remove verbose log in memcached container  https://review.openstack.org/46413405:04
*** zhubingbing_ has quit IRC05:10
*** yuanying has quit IRC05:18
*** yuanying has joined #openstack-kolla05:19
*** yuanying has quit IRC05:19
*** yuanying has joined #openstack-kolla05:20
*** strigazi_ has joined #openstack-kolla05:25
*** rwellum has quit IRC05:26
*** janki has joined #openstack-kolla05:26
*** strigazi has quit IRC05:27
*** strigazi has joined #openstack-kolla05:30
*** strigazi_ has quit IRC05:32
*** csuttles has joined #openstack-kolla05:38
*** harlowja has joined #openstack-kolla05:47
*** zhubingbing has joined #openstack-kolla05:50
*** itlinux has quit IRC05:56
*** Serlex has joined #openstack-kolla05:58
*** itlinux has joined #openstack-kolla06:01
openstackgerritRemo Mattei proposed openstack/kolla-ansible master: CentOS yum adding the -y  https://review.openstack.org/46493906:03
*** ducttape_ has joined #openstack-kolla06:06
*** yuanying has quit IRC06:06
*** ducttape_ has quit IRC06:11
*** itlinux has quit IRC06:12
*** tovin07_ has joined #openstack-kolla06:17
*** mannamne has joined #openstack-kolla06:18
*** mannamne has quit IRC06:19
*** bjolo_ has joined #openstack-kolla06:20
*** bjolo has joined #openstack-kolla06:20
*** pcaruana has joined #openstack-kolla06:22
*** magicboiz has joined #openstack-kolla06:25
*** manheim has joined #openstack-kolla06:26
*** magicboiz has quit IRC06:29
*** manheim has quit IRC06:31
*** magicboiz has joined #openstack-kolla06:34
*** harlowja has quit IRC06:50
*** jaosorior has quit IRC06:53
*** chas has joined #openstack-kolla06:55
*** yuanying has joined #openstack-kolla06:59
*** matrohon has joined #openstack-kolla06:59
openstackgerritMerged openstack/kolla-ansible master: Updated from global requirements  https://review.openstack.org/46254807:10
openstackgerritMathieu Rohon proposed openstack/kolla master: Add skydive service  https://review.openstack.org/45999507:12
*** sbezverk has quit IRC07:16
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' to necessary tasks (general roles)  https://review.openstack.org/39868207:17
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla master: Fixing the running of keystone  https://review.openstack.org/46495307:18
*** manheim has joined #openstack-kolla07:35
*** chenn has joined #openstack-kolla07:38
*** jaosorior has joined #openstack-kolla07:43
*** egonzalez has joined #openstack-kolla07:51
*** qiliang27 has quit IRC07:56
*** qiliang27 has joined #openstack-kolla07:56
*** mgoddard__ has joined #openstack-kolla07:59
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible stable/ocata: Fix designate prechecks  https://review.openstack.org/46496207:59
manheimhello08:00
*** salv-orlando has quit IRC08:00
openstackgerritChen proposed openstack/kolla-ansible master: Avoid facts gathered twice when deploying with --limit  https://review.openstack.org/46496308:01
openstackgerritMerged openstack/kolla-ansible master: Fix permission on chrony.conf  https://review.openstack.org/46246208:02
*** qiliang27 has quit IRC08:02
*** dixiaoli has quit IRC08:04
*** ducttape_ has joined #openstack-kolla08:08
openstackgerritMartin Mágr proposed openstack/kolla master: Sensu images  https://review.openstack.org/46359308:14
*** ducttape_ has quit IRC08:15
*** jaosorior is now known as jaosorior_lunch08:20
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Add heat_stack_owner role to admin project  https://review.openstack.org/46491308:21
*** mannamne has joined #openstack-kolla08:25
*** jascott1 has quit IRC08:28
*** dixiaoli has joined #openstack-kolla08:28
*** athomas has joined #openstack-kolla08:33
*** chenn has quit IRC08:36
*** stefan_bo has joined #openstack-kolla08:45
*** Serlex has quit IRC08:47
*** stefan_bo has quit IRC08:49
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Use auth vars for admin user  https://review.openstack.org/46498708:49
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Use auth vars for admin user  https://review.openstack.org/46498708:54
*** chenn has joined #openstack-kolla08:55
*** YuYangWang has joined #openstack-kolla08:57
openstackgerritMerged openstack/kolla master: Fix the "tox -e docs" faild  https://review.openstack.org/46413509:12
*** jaosorior_lunch is now known as jaosorior09:15
*** dixiaoli has quit IRC09:15
*** janki has quit IRC09:16
*** dixiaoli has joined #openstack-kolla09:16
*** janki has joined #openstack-kolla09:16
openstackgerritLi Yingjun proposed openstack/kolla-ansible master: Removing ovs bridge before destroying container  https://review.openstack.org/46499809:22
*** Serlex has joined #openstack-kolla09:23
*** lpetrut has joined #openstack-kolla09:36
*** yingjun has quit IRC09:39
*** janki has quit IRC09:43
*** janki has joined #openstack-kolla09:43
*** yingjun has joined #openstack-kolla09:49
*** yingjun has quit IRC09:50
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Add possibility to configure tenant and provider VLANs  https://review.openstack.org/46467509:52
*** tovin07_ has quit IRC09:56
*** shardy has joined #openstack-kolla09:57
*** sambetts|afk is now known as sambetts09:58
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Re-enable bifrost test_build  https://review.openstack.org/46501210:03
*** tvignaud_ is now known as tvignaud10:09
*** cuongnv has quit IRC10:10
*** dixiaoli has quit IRC10:14
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Enable manila-data for rpm binary  https://review.openstack.org/46501810:23
*** magicboiz has joined #openstack-kolla10:40
*** mannamne has quit IRC10:45
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Add external_vip_interface precheck  https://review.openstack.org/46502910:53
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Add external_vip_interface precheck  https://review.openstack.org/46502910:54
*** open_box has quit IRC11:00
*** jchhatbar has joined #openstack-kolla11:06
*** janki has quit IRC11:08
*** shardy is now known as shardy_lunch11:10
*** jchhatbar has quit IRC11:10
*** janki has joined #openstack-kolla11:11
*** jchhatbar has joined #openstack-kolla11:12
*** janki has quit IRC11:15
*** hrw has joined #openstack-kolla11:16
hrwmorning11:16
manheimthere is a limit on how many dockerhub repos I can have on the a normal user?11:20
*** jascott1 has joined #openstack-kolla11:23
*** rwellum has joined #openstack-kolla11:24
*** jascott1 has quit IRC11:27
*** zhubingbing has quit IRC11:32
*** zhubingbing_ has joined #openstack-kolla11:35
*** rwallner has joined #openstack-kolla11:42
*** shardy_lunch is now known as shardy11:50
*** yingjun has joined #openstack-kolla12:03
*** zhubingbing_ has quit IRC12:04
*** bmaltais_ has joined #openstack-kolla12:06
*** ducttape_ has joined #openstack-kolla12:06
*** jamesPR has joined #openstack-kolla12:10
*** ducttape_ has quit IRC12:11
*** dave-mccowan has joined #openstack-kolla12:21
*** bjolo__ has joined #openstack-kolla12:27
bjolo__hello12:27
jamesPRsup12:33
*** ducttape_ has joined #openstack-kolla12:34
*** ducttape_ has quit IRC12:35
*** bjolo_ has quit IRC12:39
*** bjolo has quit IRC12:39
*** bjolo__ has quit IRC12:39
*** stefan_bo has joined #openstack-kolla12:46
*** japestinho has quit IRC12:47
*** rwallner has quit IRC12:48
*** salv-orlando has joined #openstack-kolla12:49
*** zhubingbing_ has joined #openstack-kolla12:49
*** stefan_bo has quit IRC12:51
egonzalezjamesPR, fixed the SSL issue?12:52
*** dciabrin has quit IRC12:52
*** rwallner has joined #openstack-kolla12:53
*** dciabrin has joined #openstack-kolla12:53
*** rwallner has quit IRC12:55
*** rwallner has joined #openstack-kolla12:56
*** rwallner_ has joined #openstack-kolla12:57
*** rwallner has quit IRC12:57
*** rwallner_ has quit IRC12:57
*** rwallner has joined #openstack-kolla12:57
*** goldyfruit has joined #openstack-kolla12:59
*** eanylin has quit IRC13:07
rwellumegonzalez: you use centos VM?13:08
egonzalezrwellum, yep13:08
rwellumHave you tried to pull kubernetes lately? For some reason the repo is broken for me.13:09
*** YuYangWang has quit IRC13:10
rwellumhttp://yum.kubernetes.io/repos/kubernetes-el7-x86_64/repodata/repomd.xml: [Errno 14] HTTPS Error 302 - Found13:11
egonzalezrwellum, i can access to the repoxml url13:22
*** jascott1 has joined #openstack-kolla13:24
*** eanylin has joined #openstack-kolla13:24
*** salv-orlando has quit IRC13:25
rwellumThanks egonzalez13:25
*** jascott1 has quit IRC13:29
rwellumegonzalez: if you do: 'curl -O http://yum.kubernetes.io/repos/kubernetes-el7-x86_64/repodata/repomd.xml' - what file do you get?13:29
egonzalezrwellum, seems something is wrong http://paste.openstack.org/show/60966713:31
*** yingjun has quit IRC13:32
rwellumegonzalez: that's what I get13:32
rwellumThat's the error I get when I try to yum install kubeadm with the kubernetes.repo13:33
egonzalezrwellum, missing an -L on curl13:33
*** yingjun has joined #openstack-kolla13:33
egonzalezhttp://paste.openstack.org/show/609668/13:33
*** vhosakot has joined #openstack-kolla13:34
rwellumAh!13:34
*** jtriley has joined #openstack-kolla13:34
*** fooliouno has quit IRC13:37
*** jrist has quit IRC13:43
rwellumegonzalez: so not sure why this one VM stopped working, even reverting it to a stable version had same issue. Moving on to a new VM...13:44
egonzalezrwellum, guess yes, but have you tried cleaning yum cache?13:45
rwellumYeah tried that13:45
egonzalezrwellum, if pass me the commands to install k8s will try in a vm13:46
rwellumegonzalez: thanks. I'm playing with my script to install kolla: https://github.com/RichWellum/k8s - welcome to clone and take a look. I'll probably abandon this when the ansible playbook is stable - but for now this meets my needs.13:50
vhosakotrwellum: cool scripts :)13:51
*** ducttape_ has joined #openstack-kolla13:52
rwellumAh the bash script is a fork of something that was posted yesterday - I was going to make some changes and submit a review. But I was already working on k8s.py13:53
vhosakotah I see13:54
rwellumwelcome to try vhosakot - k8s.py gets as far as installing mariadb then I was seeing an issue13:54
vhosakotrwellum: sure, will try and let you know.13:55
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Add possibility to configure tenant and provider VLANs  https://review.openstack.org/46467513:56
vhosakotlots of emails on openstack-dev about kolla images :)13:56
egonzalezrwellum, getting No package kubernetes-cni-1.6.2-0 available.13:57
*** jaosorior is now known as jaosorior_away14:00
egonzalezrwellum, kube* is installed14:02
*** gomarivera has joined #openstack-kolla14:02
inc0good morning14:03
vhosakotmorning inc014:06
hrwhi inc014:06
hrwvhosakot: that's good14:06
hrwinc0: on tomorrow's meeting gema would like to discuss aarch64 images hosting/generation etc. We (linaro) would like to mimic what kolla is doing with x86-6414:07
vhosakotmanheim: https://www.docker.com/subscription-pricing has info about the number of repos we get.  For DockerHub, I see 20 private repos, unlimited public repos.  I think the pulls/pushes/tags are also unlimited for public repos.14:07
*** athomas has quit IRC14:07
hrwsth like that14:07
vhosakothrw: what's good? :)14:07
hrwvhosakot: that it is discussed14:07
inc0hrw: I already hijacked this agenda item for general image hosting;)14:08
hrwinc0: ;D14:08
inc0as it's still being discussed14:08
vhosakothrw: yeah yeah, I like the discussion and how broad it is.  wanted to jump in with a reply few times, but did not :)14:08
inc0but arm images are going to be part of it14:08
sdakeinc0 making assumptions :)14:08
hrwinc0: I hope that ibm people (krtaylor?) will follow for ppc64le14:09
hrwsdake: someone has to ;d14:09
inc0well once we figure it one arch, we'll just follow the same process14:09
*** gomarivera has quit IRC14:09
hrwyuep14:09
*** cuongnv has joined #openstack-kolla14:10
*** gomarivera_ has joined #openstack-kolla14:10
*** jtriley has quit IRC14:11
vhosakotmanheim: we use quay enterprise in my company.  It is faster for me sometimes to pull from enterprise quay than from DockerHub.14:12
vhosakotmanheim: I'm in Boston, the nearest DockerHub I hit is in TX, and the nearest quay I hit is RTP (which is much closer to me).14:12
*** jtriley has joined #openstack-kolla14:17
manheimi have only publicreps14:17
manheimpublic repos*14:17
manheimyes, having a local registry is nice14:17
vhosakotah cool14:18
* krtaylor reads scrollback14:20
*** myoung|bbl has quit IRC14:21
zhubingbing_hey guys14:21
krtaylorinc0, gema - I'll be there14:21
egonzalezsup zhubingbing_14:21
dave-mccowani'm trying to insall kolla-k8s following the deployment guide.  docker daemon is not starting on my server.  any tips on debugging docker start up?14:22
zhubingbing_<dave-mccowan> you can look /vag/log/message14:22
*** myoung has joined #openstack-kolla14:22
*** fguillot has joined #openstack-kolla14:23
vhosakotdave-mccowan: sudo ;)14:23
zhubingbing_k8s 1.6.0 need add something14:23
gemakrtaylor: +114:24
dave-mccowanin /var/log/messages, from kubectl, "NetworkPlugin cni failed to teardown pod"14:25
*** jchhatbar is now known as janki14:26
hrwkrtaylor: cool!14:26
hrwkrtaylor: btw - have you tested kolla on ppc64le?14:26
dave-mccowanin /var/log/messages, from docker: " Failed to start Docker Storage Setup."14:27
*** dixiaoli has joined #openstack-kolla14:28
*** dixiaoli has quit IRC14:28
*** dixiaoli has joined #openstack-kolla14:29
*** zhubingbing_ has quit IRC14:32
*** srwilkers has joined #openstack-kolla14:33
vhosakotdave-mccowan: any error in "systemctl status -l docker.service" ?14:33
krtaylorhrw, this is timely since I am just now getting back to kolla after summit, I hope to test this afternoon14:34
hrwkrtaylor: cool. share results please14:34
krtaylorhrw, I had to stop for the past several weeks14:34
krtaylorhrw, will do14:35
*** yingjun has quit IRC14:35
hrwkrtaylor: noticed. took care of ppc64le in meantime14:35
krtayloryes, I have been watching14:35
*** dixiaoli has quit IRC14:36
dave-mccowanvhosakot yes, looks like some warnings there14:36
*** yingjun has joined #openstack-kolla14:37
dave-mccowanvhosakot http://paste.openstack.org/show/609680/14:37
vhosakotdave-mccowan: there is "Active: active (running)" which I think is good.  I'd build and run a test DockerFile/container to check docker daemon's sanity.14:38
dave-mccowanvhosakot "docker info" returns "Cannot connect to the Docker daemon. Is the docker daemon running on this host?"14:39
vhosakotdave-mccowan: are you root?  can you try with sudo?14:39
*** tovin07 has joined #openstack-kolla14:40
*** skramaja has quit IRC14:40
dave-mccowanvhosakot ack!  yep, that's it.  ok, docker is good. :-)14:40
*** zhubingbing_ has joined #openstack-kolla14:41
vhosakotdave-mccowan: well, you can setup the docker daemon to not use sudo.14:41
hrwdave-mccowan: which distro you are using?14:41
vhosakotdave-mccowan: I usually do something "docker run -it ubuntu:latest /bin/bash" to sanitize my docker daemon.14:41
dave-mccowanhrw centos14:42
vhosakotor centos:latest depending on your distro dave-mccowan14:42
hrwdave-mccowan: docker works for me as a normal user but I forgot which magic I had to do for it14:43
hrwdave-mccowan: but my user is in root and in wheel groups14:44
*** zhubingbing_ has quit IRC14:44
*** gomarivera_ has quit IRC14:45
dave-mccowanit looks like docker and k8s are up on my server, but on step 2 when i run "sudo kubectl run -i -t $(uuidgen) --image=busybox --restart=Never" I get:14:46
dave-mccowanThe connection to the server localhost:8080 was refused - did you specify the right host or port?14:47
*** zhubingbing_ has joined #openstack-kolla14:47
sdakesup dave-mccowan14:47
sdakesup zhenguo14:47
sdakesup zhubingbing_ :)14:47
zhubingbing_hi sdake14:48
dave-mccowanhi sdake.  i missed you last week.  hard to believe we never crossed paths. :-)14:48
sdakedave-mccowan i was ill with food poisoning or the like for roughly 40 hours14:48
sdakesadly i missed 2 days of the conference as a result14:48
dave-mccowansdake yuck.14:48
*** hongbin has joined #openstack-kolla14:48
*** jtriley_ has joined #openstack-kolla14:50
dave-mccowansdake i'm try to get kolla-k8s up and running to take a look at TLS stuff.  i'm stuck on step 2.  vhosakot and hrw have been helping me.14:50
sdakedave-mccowan currently kubernetes upstream is BDWNF iiuc14:50
vhosakotsdake: due to the new (broken ?) k8s version that was release recently?14:51
*** skramaja has joined #openstack-kolla14:51
vhosakotsdake: take care, did not know you were food poisoned... the Boston lobsters eh?14:51
sdakewho knows what food it was or if it was food at all14:52
vhosakotright14:52
sdakei am feeling better as of about thursday morning14:52
*** srwilkers has quit IRC14:52
*** jtriley has quit IRC14:52
vhosakotcool14:53
*** gomarivera_ has joined #openstack-kolla14:53
*** athomas has joined #openstack-kolla14:54
vhosakotdave-mccowan: haven't got a chance to deploy kolla-k8s after summit, will deploy this week and let you know.14:54
*** kbaegis has joined #openstack-kolla14:54
kbaegisHi guys. So there was a heavy emphasis on CI/CD at the summit. Is anyone working on an open toolchain with jenkins/kolla?14:55
*** jtriley_ has quit IRC14:55
*** srwilkers has joined #openstack-kolla14:55
*** zhubingbing_ has quit IRC14:56
sdakedave-mccowan all I can say is kubernetes 1.6.2 was working and kubernettes 1.6.3 is broken - and kubernetes 1.6.2 is no longer in the upstream repositories14:56
kbaegisIs Solum supposed to be the approach for that?14:57
*** chas has quit IRC15:01
hrwhm.15:02
*** chas has joined #openstack-kolla15:02
hrwrequirements.txt lacks requests15:02
*** srwilkers has quit IRC15:03
hrwwebsocket as well15:04
*** srwilkers has joined #openstack-kolla15:04
*** srwilkers_ has joined #openstack-kolla15:04
inc0vhosakot: ping15:05
*** srwilkers_ has left #openstack-kolla15:05
vhosakotinc0: pong15:05
inc0hey:) so let's talk about the thing;)15:05
*** YuYangWang has joined #openstack-kolla15:06
inc0guys, I'm coming out of closet, I need to tell y'all something15:06
inc0;)15:06
inc0I'm thinking of starting new sub-kolla project15:06
*** chas has quit IRC15:06
inc0besides published images, there is another request that keeps coming back to Kolla15:06
inc0and that is all these fancy network underlays15:06
inc0ODL, Calico15:07
vhosakotinc0: sure, I remember you mentioned about new kolla project for fast data and ODL at the summit.15:07
inc0yup15:07
inc0I'm thinking we could create new kolla deliverable15:07
egonzalezinc0, yeah, i was thinking something similar, a kolla for nfv envs15:07
inc0that will use kolla-ansible methodology of deployment, which is proven and correct now15:07
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Remove more dependencies on kolla-ansible  https://review.openstack.org/46508915:08
inc0to deploy networking underlays that will naturally fit with kolla-ansible15:08
inc0as in, same inventory, globals.yml15:08
hrwegonzalez: nfv and dpdk stuff15:08
inc0yup15:08
inc0in fact we already have roles for both odl and ovs-dpdk15:08
*** yingjun has quit IRC15:08
inc0they weren't merged to kolla-ansible as it's well...complex15:09
inc0calico was around 6 months ago15:09
*** itlinux has joined #openstack-kolla15:09
inc0I'm thinking we need new place for things like that15:09
vhosakotyeah, I think for all the next-gen goodness for NFV/SDN space - Fast Data (df.io), VPP, ODL, Calico, DPDK, SR-IOV, etc etc?15:09
vhosakotdf.io*15:09
*** jtriley has joined #openstack-kolla15:09
vhosakotfd.io*  sorry for multiple typos.. lol15:09
inc0they will have various levels of quality, at least from beggining15:09
inc0but that's fine15:09
vhosakotinc0: will this be a big-tent project like kolla-ansible and kolla-k8s?15:10
inc0since it's optional and perpendicular to kolla-ansible15:10
inc0vhosakot: since it'll be part of Kolla, yes15:10
inc0it's not a project15:10
hrwis it expected to get python docker stuff from distro when kolla is used?15:10
inc0a deliverable15:10
inc0hrw: I'm on the fence with this one15:10
inc0I'm good either way15:10
inc0but that's up to new community we'd need to kick off15:11
hrwI find requirements.txt to lack far too many things15:11
hrwrequests, websocket, docker, markupsafe, debtcollector15:11
vhosakotinc0: it will be active repo right? with commits, gates, contributors?  or just a repo with files (Dockerfiles+Ansible playbooks) needed for NFV?15:12
inc0hrw...fix it?:)15:12
inc0but fresh installs of kolla works15:12
hrwinc0: will have a patch15:12
inc0vhosakot: I think Dockerfiles will still land into Kolla15:12
hrwinc0: I do "git clone;cd kolla; ./tools/build.py"15:12
inc0at least until we figure out better way to span kolla image infra across different project15:13
hrwinc0: with virtualenv and pip install -r requirements.txt in meantime15:13
vhosakotinc0: right right, so the new repo will have the ansible playbooks?15:13
inc0yes15:13
inc0that was my tinking15:13
openstackgerritRemo Mattei proposed openstack/kolla-ansible master: CentOS yum adding the -y Ubuntu apt-get adding the -y  https://review.openstack.org/46493915:13
*** gema has quit IRC15:13
inc0maybe we could change kolla-build in a way that it could be reused by other projects, that's good conversation to have15:13
vhosakotinc0: why not keep them in kolla-ansible itself?  why have a new repo? aren't they just ways to deploy network underlay (L2 and L3) for kolla like OVS and LinuxBridge we have today in kolla-ansible?15:14
manheimguys, ubuntu binary 4.0.0 horizon working for you?15:15
inc0vhosakot: problem is, when something like that gets proposed15:15
inc0it's usually very complex and scary15:16
inc0we've already got few of these and they weren't merged because core team was slow with reviews, that's because core team don't like to review things they don't understand15:16
inc0myself included15:17
inc0and network underlays are science on it's own15:17
inc0so I'd rather have separate team from kolla-ansible15:17
inc0that will be more network-minded15:17
inc0and limit their scope to this15:17
hrw~curse None != "None"15:17
inc0:D15:17
*** itlinux has quit IRC15:18
*** gema has joined #openstack-kolla15:18
vhosakotinc0: agreed, they (next-gen L2/L3 underlay for SDN/NFV/fast data with OpenFlow goodness blah blah) are complex compared to plain OVS, LinuxBridge and neutron's L3 (NAT) router.15:18
*** srwilkers has quit IRC15:19
*** srwilkers has joined #openstack-kolla15:19
hrwpython suxx at true/false/none stuff15:20
inc0hrw: None shouldn't be "None"...types matter man;)15:20
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Remove more dependencies on kolla-ansible  https://review.openstack.org/46508915:20
hrwkolla/common/config.py in line 48 uses None. kolla/image/build.py in line 630 has 'elif config is None:' which works on CentOS 7 but not on Debian/jessie15:21
inc0ah15:21
gemaare you sure the problem is the OS and not the version of python?15:21
inc0yeah, that's not python issue, that's operators15:21
inc0and we misused these15:21
inc0but that's a good point15:21
vhosakotinc0: what will the new repo called?  kolla-NFV, kolla-SDN, kolla-for-telco ?15:21
inc0python -v plz15:21
hrw2.7.5 centos, 2.7.9 debian15:22
inc0let the bikeshed commence!15:22
egonzalezkolla-infra ?15:22
inc0kolla-infra might be misleading15:22
inc0I'd keep it networking'y15:22
inc0kolla-rich-network was my idea but I don't like this name particulary15:22
*** chas has joined #openstack-kolla15:22
inc0kolla-sdn is a bit better, but meh...15:22
vhosakotdoesn't that make kolla-ansible poor ? ;)15:23
hrwinc0: kolla-sdn is good name15:23
inc0kolla-containers-k8s-sdn, put as much hype as possible15:23
inc0kolla-ansible-sdn maybe?15:23
*** yingjun has joined #openstack-kolla15:23
inc0since it's ansible, it will be "under" kolla-ansible at least at begginging15:23
vhosakotinc0: so, this new repo for SDN/NFV/Telco/fast data, does it deploy kolla using ansible or kolla-k8s?15:24
inc0if we do it correctly, there is no reason why this couldn't be reused15:24
inc0vhosakot: my thinking is15:24
inc0it deploys odl+ovs+dpdk15:24
inc0that's it15:24
inc0and in kolla proper we say enable_openvswitch = False15:24
*** jascott1 has joined #openstack-kolla15:25
inc0so you deploy network underlay, it deploys ovs15:25
hrwinc0: idea for that none shit?15:25
inc0kolla-ansible just uses ovs15:25
inc0hrw: pick either is None or == None and move rest of build script to use same method?15:25
hrwinc0: kolla uses both forms already iirc15:26
*** zhubingbing_ has joined #openstack-kolla15:26
*** dciabrin has quit IRC15:26
egonzaleztools from new repo could be reused by k8s too, i dont think tie to ansible is good, just deploy odl+dpdk and k8s or ansible can make use of it15:26
*** dciabrin has joined #openstack-kolla15:26
gemayeah, why do you guys need a new repo, cannot this thing be made configurable?15:26
*** bmaltais_ has quit IRC15:26
inc0hrw: what I'm saying, let's pick one and replace all occurences of another15:27
*** chas has quit IRC15:27
inc0egonzalez: well, yes and no15:27
inc0it's gonna be ansible code15:27
inc0and binding ansible-deployed ovs with k8s is ugly, possible but ugly15:27
inc0we might very well end up with kolla-k8s-sdn at some point15:28
inc0or just figure out how to reuse kolla-ansible-sd15:28
inc0n15:28
*** skramaja has quit IRC15:28
inc0but I'd focus scope on kolla-ansible for now15:28
inc0but that's my opinion, we can go either way15:28
*** jascott1 has quit IRC15:29
inc0however, seeing that we already discuss naming15:29
inc0I assume we're doing this?15:29
*** zhubingbing_ has quit IRC15:29
egonzalezI'm in15:30
gemadoing what?15:30
rwellumegonzalez: regarding 1.6.2 - you saw sdake comment on that?15:30
gemafell and entered the channel again without context15:30
vhosakotinc0: so the new repo will have like enable_odl, enable_ovsdpdk, enable_vpp, enable_sriov in globals.yaml?15:30
*** zhubingbing_ has joined #openstack-kolla15:30
inc0something like that15:30
hrwsorted out in other way ;D15:31
hrws/None/"" and then 'elif not config" :D15:31
inc0and we'll figure out how to inject configs to kolla-ansible's neutron15:31
inc0I have few ideas already15:31
egonzalezrwellum, just read now, didnt look at the version15:31
vhosakotinc0: will there be a vote from the community for this.  I like the idea of separate repo and gates for these complex new networking technologies.15:31
*** zhubingbing_ has quit IRC15:32
inc0vhosakot: we agreed few months back that 2*+2 + PTL vote is enough15:32
*** salv-orlando has joined #openstack-kolla15:32
inc0so yes, I'll kick off voting with myself giving +2;)15:32
vhosakotah cool15:32
inc0I'll also in this thread start recruting core team15:33
inc0I think from kolla anyone who wants to be core just gets it automatically15:33
inc0from rest of the world, I guess we'll just pick few people who contributes?15:33
inc0we'll figure that one out, depending how many volunteers we'll gather across community15:33
*** cuongnv has quit IRC15:34
*** Serlex has quit IRC15:35
vhosakotI've deployed OpenStack with ODL, VPP, SR-IOV and can help.15:35
inc0well Vikram, I'd give you +2 powers any day there:)15:36
vhosakotI can advertise this new kolla repo in the "OpenStack with OpenDaylight" talk I'm giving at a conference in June - https://www.ciscolive.com/us/learn/sessions/session-catalog/?search=DEVNET-204115:36
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: fix handling of delorean on non-x86  https://review.openstack.org/46509815:36
inc0so if you want to be in this project and help shaping it up, I'll be extremally happy to have you:)15:37
hrwinc0: can you take a look at ^^ and comment?15:37
inc0I also have few collegues from Intel who wants to join forces there15:37
vhosakotinc0: sure, will let you know :)15:37
inc0specifically for OPNFV now15:37
egonzalezmaybe this deliverable makes more sense under OPNFV umbrella as tripleo and fuel do, they have resources for performance testing15:38
*** sbezverk has joined #openstack-kolla15:39
inc0egonzalez: I thought about that15:39
inc0thing is, OPNFV is a foundation over ODL15:39
inc0Calico doesn't really have foundation over15:39
inc0there are other less known networking solutions out there15:39
inc0I think that's going to resonate well with OpenStack mission to be used by other communities too15:40
inc0dims ^15:40
*** gfidente is now known as gfidente|afk15:40
*** lpetrut has quit IRC15:40
* dims reading scrollback15:41
inc0we're discussing new deliverable for sdn-kinda things15:41
inc0as underlay of kolla-ansible15:41
dimsack inc015:41
inc0thing is, way I envision it15:41
inc0it doesn't have to be kolla-ansible specific15:42
inc0it's going to be very well suited for kolla-ansible15:42
inc0but it's gonna stand on it's own ideally15:42
inc0do you think we can make this as a "bridge" between external communities too?15:42
*** chas has joined #openstack-kolla15:43
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Remove more dependencies on kolla-ansible  https://review.openstack.org/46508915:45
vhosakotOpenStack can totally be the virtualization layer on top dpdk/odl networking, and under the NFV apps doing service-chaining for telco/sp, and here is where the new kolla repo can help.15:46
*** srwilkers has quit IRC15:46
*** srwilkers has joined #openstack-kolla15:46
vhosakotdon't know if the new repo become inactive soon due to less contributors/users.15:47
*** chas has quit IRC15:48
*** jamesPR has quit IRC15:54
inc0well, I already have few willing bodies to kick this off15:54
inc0and from my few discussions in summit, I got lot of good feedbach15:54
inc0and not only from OpenStack people15:54
*** tonanhngo has joined #openstack-kolla15:54
rwellumegonzalez: if 1.6.2 is gone, and 1.6.3 is broken... How are the gates etc working?15:55
inc0rwellum: I think sbezverk pushed change that uses compiled 1.6.3 with fix15:56
inc0btw what the actual F### k8s...removing minor versions like that15:56
vhosakotinc0: cool, nice to know contributors are interested, yes, I can help too.15:56
sbezverkinc0: right, currently gate test runs all green15:57
egonzalezrwellum, the script installed me rpm -qa | grep -i kube15:57
egonzalezkubelet-1.6.2-0.x86_6415:57
egonzalezkubernetes-cni-0.5.1-0.x86_6415:57
egonzalezkubeadm-1.6.2-0.x86_6415:57
egonzalezkubectl-1.6.2-0.x86_6415:57
*** gardlt has joined #openstack-kolla15:57
*** janki has quit IRC15:57
rwellumegonzalez: I can change that to 1.6.3 - look at what sbezverk is doing in the gate15:58
sbezverkbasically I posted on my github a custom version of kubelet which is 1.6.3 + 1 fix after official binary of 1.6.3 is installed I getting that binary and overwrite official kubelet15:59
sbezverkwe need to do it until 1.6.4 makes into repos15:59
rwellumWhat's the bug with 1.6.3 - symptoms?16:00
sbezverkrwellum: we cannot mount file inside of a container16:00
*** fooliouno has joined #openstack-kolla16:01
sbezverkspecifically when we try to mount resolv.conf (workaround) it failes with syntax error16:01
rwellumAh. Can you point me to the gate code. I'm looking but haven't found it yet.16:02
sbezverkit is not a gate code, it is when container tries to mount a share16:02
*** srwilkers_ has joined #openstack-kolla16:03
kfox1111morning.16:04
*** chas has joined #openstack-kolla16:04
sbezverkrwellum: check this link http://logs.openstack.org/93/464393/1/check/gate-kolla-kubernetes-deploy-centos-binary-3-ceph-multi-nv/8e35c32/console.html#_2017-05-15_01_05_15_05435716:04
rwellumsbezverk:  https://www.irccloud.com/pastebin/RLfWuGwS/16:04
*** matrohon has quit IRC16:05
sbezverkrwellum: sorry I did not understand wha you meant..16:05
sbezverkrwellum: I thought you were asking about the actual issue with kubelet16:05
sbezverkkfox1111: morning16:06
rwellumBoth I  guess sbezverk : symptom and cure is nice :)16:06
*** egonzalez has quit IRC16:06
sbezverkrwellum: link above is the symptom16:06
manheimcinder is broken for ubuntu binary 4.0.0?16:07
manheimi get cinder-api command not found16:07
*** srwilkers_ has quit IRC16:08
inc0manheim: yes, yes it is16:08
inc0not our fault:(16:08
manheimah16:08
manheimok16:08
manheimi'll just ignore that for now16:08
manheimany workaround?16:08
inc0I've learned to use ubuntu source16:09
inc0ubuntu source works well16:09
manheimok16:09
*** chas has quit IRC16:09
manheimi'll redeploy source than16:09
manheimty16:09
inc0and in general it's much more stable than binary16:09
manheimgood to know16:09
kfox1111inc0: if its known broken, shuldn't we fix it?16:09
kfox1111inc0: or is it an upstream package broken? still, I would not expect ubuntu to leave a broken package for long.16:10
inc0kfox1111: afair it's upstream16:10
inc0kfox1111: do you follow ML thread regarding containers in dockerhub?16:11
kfox1111whats broken about their packages?16:11
kfox1111haven't fully cought up on my mail since the summit yet.16:11
kfox1111have the subject handy?16:11
*** manheim has quit IRC16:11
inc0[openstack-dev] [tc][infra][release][security][stable][kolla][loci][tripleo][docker][kubernetes] do we want to be publishing binary container images?16:11
inc0it's a wall of text out there, but I'd love to have your feedback16:12
*** jascott1 has joined #openstack-kolla16:13
kfox1111thanks.16:13
rwellumsbezverk: your version of kubectl - does it require a particular version of kubernetes?16:14
*** dmsimard has quit IRC16:14
kfox1111I'm kind of in the, I want it, I know others want it, so I'm going to build it frame of mind.16:14
kfox1111others can bike shed it.16:14
kfox1111rwellum: I've seen some slight incompatabilities between kubectl and kubernetes for stuff like alpha/beta features.16:15
sbezverkrwellum: should not, but gates deployes 1.6.316:15
kfox1111they only garantee compatability between stable features.16:15
*** jtriley has quit IRC16:16
*** dmsimard has joined #openstack-kolla16:17
hrwkolla uses 1.5.416:17
hrwin images16:17
inc0kfox1111: I'll try to rebuild set of images and docker save them16:17
inc0we'll see how much space it take16:18
kfox1111inc0: cool.16:18
* hrw off16:18
*** yingjun has quit IRC16:21
*** tovin07 has left #openstack-kolla16:23
*** ducttape_ has quit IRC16:24
*** ducttape_ has joined #openstack-kolla16:25
*** chas has joined #openstack-kolla16:25
kfox1111inc0: bleh. this conversation had to have come up when I was all drugged up for my back... bad timing. :/16:25
inc0yeah16:25
inc0while Doug make good points, I think we're entering uncharted wated with this16:26
inc0as we aren't distro, but we are kinda packagers16:26
kfox1111right.16:26
kfox1111we already are shipping containers. kind of bad ones. we just want to make them better.16:27
kfox1111distros are whom you pay for 'support'.16:27
kfox1111we're not a distro.16:27
inc0what we should do is to start The Kolla Company just for that;)16:27
*** salv-orlando has quit IRC16:28
kfox1111heh.16:29
*** chas has quit IRC16:29
kfox1111no reason therre couldn't be a non proffit if it needs to come to that.16:29
kfox1111technical solution to a political problem. ;)16:30
*** gomarivera_ has quit IRC16:30
inc0well thing is, way I envision it is to have set of images that pass our gates16:31
inc0add that to better gates, and you have recipe for good things16:31
jascott1wow i dont get this anti image argument at all16:31
inc0well I get that16:32
inc0it's about "openstack certified golden images"16:32
jascott1yeah thats bs16:32
*** ipsecguy has joined #openstack-kolla16:32
inc0which puts us on the line if images aren't golden16:32
jascott1nothing is secure16:32
*** ipsecguy_ has quit IRC16:32
jascott1total bs to me16:32
inc0so I agree with Doug, it's important issue16:32
jascott1whats worse is letting others pollute with bad images16:32
inc0that being said, I personally think we can handle it with proper messaging16:32
jascott1with nothing official to compare to16:33
kfox1111honestly, I think its the distro's pushing back.16:33
inc0not giving people access to containers because they might misunderstand quality standard (or lack of thereof) they are subject to16:33
kfox1111they think it will cut into sales.16:33
kfox1111but I think they are already salting the fields with their self intrests. :/16:34
kfox1111right now we already ship containers. we just want to ship fresher ones. that is an improvement to security.16:34
jascott1who gets a piece of software without any support agreement and things they magically have support?16:34
inc0well, I wouldn't accuse parties involved in this conversation for that. If anything all of them proven over and over to have community interest in mind16:34
kbaegisAnyone else have issues deploying solum with HAProxy? "solum-application-deployment" not configured in "haproxy.cfg.j2"16:34
jascott1things/thinks16:34
kfox1111jascott1: exactly.16:34
inc0we were flying under radar I guess16:34
kfox1111inc0: yeah, they may have never noticed.16:35
inc0what I'm saying is, rules might have changed16:35
inc0dockerhub is not apt16:35
jascott1chain of custody of the image is all we have right? building it yourself is not going to make it more secure16:37
jascott1so 'make them build it themeselves, not because its more secure but becuase then its "on them"'16:37
rwellumhrw: thanks 1.5.4 did the trick for me.16:37
inc0yeah, I think best case for stability is download container we built and run them through your CI16:37
*** salv-orlando has joined #openstack-kolla16:37
inc0because then you get openstack CI AND your CI16:38
inc0it's additive = better16:38
jascott1exactly!16:38
jascott1i dont think containers would have caught on if everyone had this policy :/16:38
kbaegisI need a primer on CI for openstack/kolla. Anyone have such a thing?16:38
inc0primer? what do you mean? (my english ends there)16:39
kfox1111inc0: +1. when you can afford it.16:39
inc0beggard can't be choosers;)16:39
kfox1111kbaegis: there isn't one yet. :/16:40
jascott1then the question is, why cant they be secure (as possible) and have checksum>16:40
inc0if but you get *some* testing even you don't have any CI16:40
kbaegiskfox1111: Thanks. Primer as in introduction for someone playing with it16:40
inc0kk16:40
inc0my english improved today16:40
jascott1kbaegis are you talking about the understanding the gates or setting up your own CI?16:40
inc0how glorious16:40
kbaegisEnabling solum breaks my deployment16:40
jascott1\o/16:41
kbaegisjascott1: setting up my own, not rebuilding the OSF infrastructuer16:41
kbaegis*infrastructure16:41
jascott1which CI system?16:41
kbaegisjascott1: Ideally? Jenkins16:41
jascott1dont the infra jobs kick off with jenkins?16:42
*** salv-orlando has quit IRC16:42
*** jamesPR has joined #openstack-kolla16:42
jamesPRhi all :D16:42
jascott1hi16:42
jamesPRsup jascott116:43
jascott1just living the dream16:43
jamesPRwe are all in the matrix man ;)16:44
sdakejascott1 me too dude, white picket fence and all16:44
jascott1how many times have they destroy zion that is the question ;)16:44
jamesPRlol16:44
kbaegis42 times16:45
jascott1lol16:45
*** shardy_ has quit IRC16:46
kbaegisYeah, there's definitely a bug with stable/ocata kolla-ansible with solum enabled16:46
*** chas has joined #openstack-kolla16:46
jascott1what up sdake!16:46
kbaegisIt will break haproxy, which will in turn cause weird failures in subsequent deploy commands (breaking cron, etc)16:46
sdakejascott1 working working you know how it goes :)16:46
kbaegisAll because of that /usr/local/share/kolla-ansible/ansible/roles/haproxy/templates/haproxy.cfg.j216:47
jascott1no doubt16:47
jascott1oh man I had my eyes dilated yesterday, it was pretty terrible16:47
*** salv-orlando has joined #openstack-kolla16:47
*** stefan_bo has joined #openstack-kolla16:48
*** unicell has quit IRC16:48
kbaegisWhat is 'solum-application-deployment' supposed to refer to?16:49
kbaegisIt's not defined16:49
*** jtriley has joined #openstack-kolla16:49
*** chas has quit IRC16:50
*** stefan_bo has quit IRC16:52
*** pcaruana has quit IRC16:53
*** StephenWang1991 has joined #openstack-kolla16:56
*** lpetrut has joined #openstack-kolla16:57
*** emccormick has joined #openstack-kolla16:58
*** bmaltais_ has joined #openstack-kolla17:00
*** StephenWang1991 has quit IRC17:00
emccormickhey guys. I'm trying to get started with Kolla (ansible) and set up a Newton-based cluster. I see that the kolla-ansible project is only tagged for Ocata while regular Kolla is tagged for everything *including* Ocata. What's the difference here?17:00
emccormickI assume for Newton I need just regular Kolla?17:00
emccormickor can Ocata kolla-ansible be used to install Newton Openstack?17:01
inc0emccormick: so at begingin of ocata we split ansible code from kolla to kolla-ansible17:02
inc0what today is kolla-ansible is, in reality, ansible dir from kolla in newton17:03
inc0so for N, Kolla project have all the things you need17:03
*** ducttap__ has joined #openstack-kolla17:04
*** shardy has quit IRC17:04
emccormickcool thanks!17:04
*** srwilkers_ has joined #openstack-kolla17:04
*** ducttape_ has quit IRC17:05
*** manheim has joined #openstack-kolla17:07
*** chas has joined #openstack-kolla17:07
*** gomarivera_ has joined #openstack-kolla17:07
*** chas has quit IRC17:11
*** manheim has quit IRC17:11
*** gomarivera_ has quit IRC17:11
*** ducttap__ has quit IRC17:16
inc0kfox1111: confirmed - you don't want tarballed images17:16
inc010gigs vs 2.7 gigs in tared registry17:16
*** pbourke has quit IRC17:22
kbaegisinc0: Razelle will be so disappointed.17:22
inc0Razelle?17:22
kbaegisinc0: nvm. Lady over at HP. Worked on TAR17:23
inc0ah17:23
inc0well17:23
*** pbourke has joined #openstack-kolla17:23
inc0question was17:23
inc0is docker save img1 img2 -o somefile better than tar.gz of /var/lib/registry17:24
kbaegisDoes anyone know the kolla SME on solum? Who is that?17:24
inc0spoiler alert, it isn't17:24
rwellumhttp://logs.openstack.org/63/461063/7/check/gate-kolla-kubernetes-deploy-centos-binary-2-helm-compute-kit-nv/c7f99db/console.html#_2017-05-15_21_24_40_703820 - these computekit time stamp errors - are they harmless?17:24
inc0kbaegis: I dobut we have anyone tbh, I'd suggest raising issue with Solum community17:24
kbaegisinc0: Thanks17:24
inc0I can help with general docker-fu/ansible-fu17:25
*** kbaegis has left #openstack-kolla17:25
*** gomarivera_ has joined #openstack-kolla17:25
*** kbaegis has joined #openstack-kolla17:26
kbaegisinc0: there isn't a openstack-solum irc. Guess it's mailer only :(17:26
*** srwilkers_ has quit IRC17:26
vhosakotemccormick: right, Newton has all the ansible playbooks out-of-the-box in https://github.com/openstack/kolla/tree/stable/newton/ansible pre-split.17:26
inc0kbaegis: try #solum17:27
*** chas has joined #openstack-kolla17:28
*** Manheim has joined #openstack-kolla17:29
*** athomas has quit IRC17:30
*** chas has quit IRC17:32
*** srwilkers has quit IRC17:34
*** srwilkers has joined #openstack-kolla17:34
*** zhubingbing_ has joined #openstack-kolla17:37
*** egonzalez has joined #openstack-kolla17:38
egonzalezkbaegis: there is a known issue with solum https://review.openstack.org/#/c/420407/17:39
*** zhubingbing_ has quit IRC17:40
rwellumIf a helm chart has a hard time initializing because of a missing config map (that I generated) - how would I go about debugging this?17:41
kbaegisegonzalez: Sounds like I need to kolla-build solum?17:42
*** sindhu has quit IRC17:43
egonzalezkbaegis: atm cannot remember, but was something like solum removed an useless service, let me verify17:44
*** egonzalez has quit IRC17:46
*** egonzalez has joined #openstack-kolla17:46
*** sindhu has joined #openstack-kolla17:47
*** egonzalez has quit IRC17:48
*** srwilkers has quit IRC17:48
*** egonzalez has joined #openstack-kolla17:48
*** srwilkers has joined #openstack-kolla17:48
*** harlowja has joined #openstack-kolla17:50
bmaltais_Just noticed that shutting down the AIO buils and the powering it up again result in a broken environment.17:54
bmaltais_nova-compute keep failing and the horizon dashboard eventually stop responding17:54
sbezverkrwellum: kubectl describe pod blan -n kolla and see what is not able to mount17:54
*** salv-orlando has quit IRC17:54
*** trozet has joined #openstack-kolla17:54
*** salv-orlando has joined #openstack-kolla17:54
bmaltais_Is there a "clean" way to stop a k8t environment? I know it is probably never supposed to happen...17:55
bmaltais_I mean, shutting totally a k8t cluster.17:56
*** ducttape_ has joined #openstack-kolla17:58
*** ducttap__ has joined #openstack-kolla17:59
*** lpetrut has quit IRC18:02
*** srwilkers has quit IRC18:02
*** ducttape_ has quit IRC18:03
*** srwilkers has joined #openstack-kolla18:03
* harlowja debating if i want to respond to that `do we want to be publishing binary container images` thread, lol18:06
harlowjainc0 i'm with u, i think there is some magical belief that people really have there own great CIs, lol18:07
harlowjaand i'd rather have those folks have something vs nothing, lol18:07
inc0exactly18:07
inc0and in case you actually have CI18:07
inc0this just makes things better18:07
* harlowja i can tell u , from trying to build out a CI since i got at godaddy, its not like simple18:07
*** srwilkers has quit IRC18:08
inc0it never is18:08
harlowjaso i'm pretty sure nobody really has it, lol (outside of major vendors/users)18:08
inc0doing kolla gates is still largery unfinished18:08
inc0and we did burn metric ton of hours into it already18:08
harlowjaya18:08
inc0and frankly? instead of build CI locally, build it in community18:08
inc0so everyone will benefit18:09
*** chas has joined #openstack-kolla18:09
harlowjaya, the part about ^ is that most companies have some set of existing CI stuff they already recommend18:09
harlowjaso person trying to build it in community would have to jump through hoops to do that18:10
inc0yeah, sure, and again, that's good18:10
inc0but in case you actually want to put it into opesntack ci, that's wonderful18:10
harlowjaagreed18:10
inc0and if we make it easier to add new scenerios18:10
inc0that's all the better18:10
*** mgoddard__ has quit IRC18:10
inc0thing is, we'll add another incentive for people to build gates18:11
harlowjaagreed18:12
*** srwilkers has joined #openstack-kolla18:14
*** chas has quit IRC18:14
*** bmace__ has quit IRC18:17
*** bmace__ has joined #openstack-kolla18:17
*** openstackgerrit has quit IRC18:17
*** bmaltais_ has quit IRC18:20
*** cinerama has joined #openstack-kolla18:21
kfox1111inc0: see my most recent email. :)18:22
kfox1111yeah. ci is hard to get right. takes years.18:23
rwellumsbezverk: that's what tells me the configmap is missing18:23
rwellumsbezverk https://www.irccloud.com/pastebin/cnBHWMUm/18:23
jamesPRegonzalez, I got something very strange, when we use RDO here we set the external IP on br-ex interface, with kolla we specify  neutron_external_interface: "eth0" and kolla_external_vip_interface: eth0 network_interface: "eth1" everything is fine minus horizon because I cannot access or ping kolla-external-vip-interface 192.168.25.254, but if I go to the node with this virtualIP and set this VIP to br-ex interface it works ? what's wrong in here?18:24
sbezverkrwellum: kubectl get configmap -n kolla | grep mariadb18:24
rwellumNothing sbezverk18:26
rwellumActually no configmaps at all18:26
rwellumI think I haven't recovered from the kolla-ansible split - but not sure what I am doing wrong.18:27
sbezverkrwellum: cool, that explains everything :) you need to create them18:27
sbezverkrwellum: I think we still need to use kollakube to create them18:27
rwellumDoing that but - at least now I have the command to see the config maps :)18:29
rwellumSo I can debug more.18:29
*** chas has joined #openstack-kolla18:30
*** chas has quit IRC18:35
*** srwilkers has quit IRC18:36
*** openstackgerrit has joined #openstack-kolla18:38
openstackgerritAndy Smith proposed openstack/kolla master: Add qdrouterd container  https://review.openstack.org/46514518:38
Manheimdeploying now with ubuntu source18:42
Manheimhope cinder and horizon will work18:43
Manheimthe better source images are in the case of centos also?18:43
kfox1111Manheim: the images on docker hub are not very well tested currently. :/18:45
kfox11112.x and 3.x are tested by kollla-kubernetes regularly.18:45
kfox1111I'm working on 4.x tests but not complete yet.18:45
egonzalezManheim source is fine in all distros, binary rpm distros are shown as more stable18:45
kfox1111its all about testing. nolt where the packages come from.18:46
*** lpetrut has joined #openstack-kolla18:48
*** chas has joined #openstack-kolla18:51
Manheimok18:52
Manheimwe'll i'll let you guys know when something breaks :D18:53
Manheimnice, cinder and horizon work18:54
*** chas has quit IRC18:56
sbezverkrwellum: you can create configmap manually19:05
jamesPRanyone know how to specify netmask in globals.yml ?19:06
*** manheim_ has joined #openstack-kolla19:07
jamesPRcan I use IPADDR/NETMASK ? is supported?19:07
inc0jamesPR: why would you specify netmask?19:08
inc0what for?19:08
jamesPRthe VIP is setting /3219:08
inc0yeah, I think it's keepalived thing19:08
jamesPRthen I cannot ping from the outside19:08
jamesPReven access to this VIP19:09
inc0well, it works normally19:09
inc0so problem is elsewhere19:09
jamesPRif I set it manually then it works19:09
jamesPRlol19:09
mnaserjamesPR is your vip sitting on the same network?19:09
mnaservips always get brought up on a /3219:09
mnaserthere should be two ips on the interfaces that run the vip, one with the "normal" subnet and one with the /3219:09
jamesPReth0 = 10.50.85.0/2419:09
jamesPReth1 = 192.168.25.0/2419:10
mnaserwhats your vip19:10
jamesPReth1 external network19:10
jamesPR192.168.25.25419:10
inc0uhh...isn't this broadcast of your network?19:10
jamesPRVIP19:10
jamesPRinc0, ?19:11
inc0ahh I missed my binary skills19:11
inc0so, this ip should reside on one of eth1's19:12
inc0alongside regular ip19:12
hrwinc0: do you have url to 'core dev mentorship program' page somewhere?19:12
*** chas has joined #openstack-kolla19:12
jamesPRegrep ^[^#] /etc/kolla/globals.yml19:12
jamesPR---19:13
jamesPRconfig_strategy: "COPY_ALWAYS"19:13
jamesPRkolla_base_distro: "centos"19:13
jamesPRkolla_install_type: "binary"19:13
jamesPRopenstack_release: "4.0.1"19:13
jamesPRnode_custom_config: "/etc/kolla/config"19:13
jamesPRkolla_internal_fqdn: "openstack-int.example.com"19:13
jamesPRkolla_internal_vip_address: "10.50.85.254"19:13
jamesPRkolla_external_fqdn: "openstack.example.com"19:13
jamesPRkolla_external_vip_address: "192.168.25.254"19:13
jamesPRdocker_registry: "10.50.85.1:5000"19:13
jamesPRnetwork_interface: "eth1"19:13
jamesPRneutron_external_interface: "eth0"19:13
jamesPRkolla_external_vip_interface: "br-ex"19:13
jamesPRapi_interface: "{{ network_interface }}"19:13
jamesPRneutron_plugin_agent: "openvswitch"19:13
jamesPRkolla_enable_tls_external: "yes"19:13
jamesPRkolla_external_fqdn_cert: "/etc/kolla/certificates/haproxy.pem"19:13
jamesPRnova_console: "novnc"19:13
jamesPRenable_central_logging: "yes"19:13
jamesPRenable_ceph: "yes"19:13
jamesPRenable_cinder: "yes"19:13
jamesPRenable_heat: "no"19:13
jamesPRenable_horizon: "yes"19:13
jamesPRenable_neutron_agent_ha: "yes"19:13
jamesPRnova_backend_ceph: "{{ enable_ceph }}"19:13
jamesPRthis is my globals19:13
hrwjamesPR: use pastebin for such paste!19:14
inc0man...please...nopaste..19:14
jamesPRmy bad :s19:14
inc0hrw: negative, we'll still doing this semi officially19:14
hrwinc0: ok19:15
hrwinc0: thinking about going for such in june ;D19:15
inc0yay19:15
inc0would be awesome19:15
inc0while I volunteer to mentor you, I'd suggest finding someone in Euro timezone19:15
jamesPRhttp://paste.openstack.org/show/609712/ <- ok there you go, if I specify kolla_external_vip_interface eth0 doesn't work19:16
*** chas has quit IRC19:16
jamesPRif I set this VIP manually to br-ex interface it works lol19:17
*** harlowja has quit IRC19:17
hrwinc0: yep19:17
hrwinc0: have to learn how to use images first ;D19:18
jamesPRis posible to disable the keepalived crap?19:19
openstackgerritDan Ardelean proposed openstack/kolla-ansible master: Add Hyper-V role  https://review.openstack.org/45568419:20
inc0jamesPR: sure19:20
inc0enable_haproxy: false19:20
jamesPRthen use haproxy to route everything to services endpoints in roundrobin fashion?19:20
inc0but you'll need to provide your own LB19:20
inc0well currently haproxy and keepalived are bundled together, and for good reason19:21
inc0if you have haproxy without keepalived19:21
inc0it's essentially like single node anyway19:21
jamesPRwell two haproxy servers19:21
*** manheim_ has quit IRC19:22
jamesPRwith keepalived19:22
jamesPRno keepalived on other nodes only on haproxy nodes19:22
*** YuYangWang has quit IRC19:24
inc0that's how it's deployed today19:25
*** salv-orlando has quit IRC19:28
kfox1111inc0: you talking kolla-ansible or kolla-kubernetes?19:31
inc0ansible19:31
kfox1111k.19:31
inc0yeah sorry, should've claryfy that19:32
kfox1111inc0: no worries. I've just heard some folks turning off haproxy in kolla-kubernetes and I think its a bad idea.19:32
*** chas has joined #openstack-kolla19:33
kfox1111they should only do that if very resource constrained.19:33
inc0well I turn off haproxy when I deploy on vms on neutron19:33
inc0because of allowed address pairs19:33
inc0but for prod...please don't19:33
kfox1111yeah. kolla-ansibles a different animal there too.19:34
kfox1111uses haproxy very differently.19:34
inc0I need to learn more about HA model in kolla-k8s19:34
kfox1111+119:35
kfox1111we need some reference architecture diagrams.19:35
inc0so many things so little time;)19:35
kfox1111story of my life. :)19:35
*** gfidente|afk has quit IRC19:36
*** chas has quit IRC19:37
jascott1+1 to all diagrams!19:37
sbezverkrwellum: try to use this command: kubectl create configmap mariadb --from-file=/etc/kolla/config.json  --from-file=/etc/kolla/galera.cnf  --from-file=/etc/kolla/wsrep-notify.sh -n kolla19:40
rwellumI think I found it sbezverk and it was a complete df moment...19:44
*** mannamne has joined #openstack-kolla19:44
rwellumAt least I see the configmap now and mariadb is coming up. But crashing *sigh*.19:45
*** blallau has joined #openstack-kolla19:45
sbezverkrwellum: you want too much ;)19:47
rwellumI know right...19:48
*** chas has joined #openstack-kolla19:54
*** chas has quit IRC19:59
*** blallau has quit IRC19:59
*** kbaegis has quit IRC20:02
*** harlowja has joined #openstack-kolla20:04
*** dave-mccowan has quit IRC20:10
fooliounoip a20:10
*** gomarivera_ has quit IRC20:12
rwellumsbezverk: do you normally change kolla_internal_vip_address: "10.10.10.254" - in globals.yml?20:12
rwellumor kfox1111 ?20:14
*** chas has joined #openstack-kolla20:15
kolla-slack1<kfox1111> Yeah. Address has to be available on your network20:15
rwellumI ask because it's not documented in the deployment guide, but when I read the globals.yml file is seems to indicate you should. But maybe it's overwritten by whatever you set in cloud.yml20:15
sbezverkrwellum: why do not you overwrite it in cloud.yaml?20:16
*** StephenWang1991 has joined #openstack-kolla20:17
kolla-slack1<kfox1111> I think until ansibe is fully gone, it might have to be in both.20:17
rwellumsbezverk: I don't get the question. I do overwrite it - pretty sure.20:18
*** chas has quit IRC20:19
*** fguillot has quit IRC20:19
rwellumkfox1111: I think ansible is fully gone no?20:21
*** StephenWang1991 has quit IRC20:22
sbezverkrwellum: sorry I was suggesting an alternative to changing it in globals.yml20:28
vhosakotrwellum: kolla_internal_vip_address must be an unused, available IP.  It is used as VIP  -  https://github.com/openstack/kolla-ansible/blob/master/doc/quickstart.rst.20:29
rwellumsbezverk: I forget the interactions between those two files tbh.20:31
rwellumvhosakot: I'm looking at kolla-kubernetes though20:31
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Support osprofile usage  https://review.openstack.org/45562820:32
*** jamesPR has quit IRC20:32
sbezverkrwellum: globals.yml is used when we run ansible to generate configs20:33
sbezverkso we are not 100% ansible free20:33
rwellumsbezverk: ok ansible - not kolla-ansible20:34
*** gomarivera_ has joined #openstack-kolla20:35
sbezverkrwellum: correct20:35
rwellumSo it has to be a free ip on the same subnet as set by external_vip in cloud.yml ?20:35
vhosakotrwellum: when I last deployed kolla-k8s using https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html, I did use kolla_internal_vip_address, just used external_vip in cloud.yml.  Haven't deployed in about 2 weeks now.20:35
vhosakotrwellum: I meant, I did _not_ use kolla_internal_vip_address.. sorry for typo20:36
*** chas has joined #openstack-kolla20:36
rwellumvhosakot: little confused. You didn't change it then?20:36
rwellumThe current default is: 10.10.10.25420:37
rwellumI guess that's my question, does external_vip in cloud.yml override kolla_internal_vip_address in globals.yml20:38
vhosakotrwellum: In https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html, I added the part under "Add required configuration to the end of /etc/kolla/globals.yml" and used kolla-ansible to generate default configs (kolla-ansible).  No, I did not use kolla_internal_vip_address.  There is no reference to kolla_internal_vip_address in https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.h20:38
vhosakottml.  again, this was 2.5 weeks ago :)20:38
rwellumSounds like the answer is it does not20:38
vhosakotrwellum: "does external_vip in cloud.yml override kolla_internal_vip_address in globals.yml"  <--  ah, I don't know.20:39
rwellumTerrible names btw :)20:40
*** chas has quit IRC20:40
*** salv-orlando has joined #openstack-kolla20:41
vhosakotin kolla-k8s, kolla-ansible is not used for deployment, it is used just to create the default configs.  So, I don't know the usage of kolla_internal_vip_address (used by kolla-ansible) in kolla-k8s.  Having said that, external_vip in cloud.yml is used by "helm install" to deploy k8s OpenStack pods.  sbezverk, is my understanding right?20:42
vhosakotrwellum: ^^^20:42
sbezverkvhosakot: kolla-ansible is not used at all20:42
vhosakotsbezverk: now, we dont have genconfig right?20:43
sbezverkvhosakot: ansible with playbook to generate config is used20:43
sbezverkvhosakot: scale down and k8s specific version is used20:43
vhosakotsbezverk: ah cool20:44
rwellumsbezverk: anything obliviously wrong stand out here?:  https://www.irccloud.com/pastebin/6jV8gXTh/20:46
sbezverkrwellum: init element is crashing, is it AIO?20:47
rwellumsbezverk: yes AIO20:47
sbezverkrwellum: which type os storage you use?20:47
vhosakotsbezverk: are you talking about this in gate to generate the configs form site.yml in gate --> http://logs.openstack.org/12/464312/4/check/gate-kolla-kubernetes-deploy-centos-binary-2-helm-compute-kit-nv/caf7b77/console.html#_2017-05-16_01_31_06_20572820:47
sbezverkvhosakot: exactamente ;)20:48
vhosakotsbezverk: cool, yeah, I learned about this in the summit.. cool, thanks for the clarificaiton :)20:48
vhosakotrwellum: so, the gate does use kolla_external_vip_address like this --> http://logs.openstack.org/12/464312/4/check/gate-kolla-kubernetes-deploy-centos-binary-2-helm-compute-kit-nv/caf7b77/console.html#_2017-05-16_01_30_17_01020620:48
rwellumvhosakot: yeah I am doing that now - so have cut the cord to kolla-ansible20:48
sbezverkrwellum: if you use host path for volumes, make sure no left overs in /var/lib/kolla/volumes20:49
vhosakotrwellum: cool20:49
*** stefan_bo has joined #openstack-kolla20:49
*** ducttap__ has quit IRC20:50
rwellumsbezverk: so this part (storage) is copied directly from the deployment guide. Not sure exactly - vhosakot ?20:50
rwellumI will clean that up now sbezverk - see if it makes a difference. Anything else in /var I should clean up?20:51
sbezverkrwellum: not that I can think of20:51
jascott1rwellum I ran into that, you need to del /var/lib/kolla/volumes as sbezverk said20:53
*** stefan_bo has quit IRC20:53
rwellumjascott1: thanks - trying his suggestion now20:54
*** kbaegis has joined #openstack-kolla20:55
vhosakotrwellum: which storage part are you talking about?  the losetup/pvcreate/vgcreate commands in https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html to loopback device?20:55
rwellumsbezverk: your 'grep -v Running' suggested at the summit works great. Very simple.20:55
rwellumvhosakot: Actually not doing that at all yet.20:55
sbezverkrwellum: :)20:55
fooliounosbezverk: trying baremeral config with controller on one node and compute on another. VMs are not getting DHCP addresses. Any idea what could be wrong? https://pastebin.com/dgr8S7z420:56
*** rhallisey has quit IRC20:56
vhosakotrwellum: me too, I didn't do it too as it is for iSCSI support with Cinder LVM (which is optional).20:56
*** chas has joined #openstack-kolla20:56
*** gardlt has quit IRC20:57
*** rhallisey has joined #openstack-kolla20:57
rwellumYeah20:57
*** gomarivera_ has quit IRC20:58
*** lpetrut has quit IRC20:58
sbezverkfooliouno: it is a bit hard. I would check each connection between VM and dhcp agent, btw where your dhcp is running to see if packets are getting out of VM and where they get dropped20:58
*** gomarivera_ has joined #openstack-kolla20:59
fooliounosbzverk: I did a tcpdump on the vxlan tunnel between the nodes. BOOT?DHCP message seems to get to the controller. assuming this should be passed to the dhcp agent container20:59
fooliounosbezverk: ^^21:00
*** chas has quit IRC21:01
sbezverkfooliouno: you need to do the same on dhcp agent interface on ovs bridge21:02
*** salv-orlando has quit IRC21:02
Manheimso I have some questions on tags in kolla21:02
Manheimi used tags for deploy or reconfigure but it never picked the up, just went with a full roles run21:03
sbezverkdue to shared name space issue it will be difficult to run tcpdump right on dhcp agent container interfaxe21:03
sbezverkbut if you see packets on br-int on interface facing dhcp agent then21:04
*** gomarivera_ has quit IRC21:04
*** inc0 has quit IRC21:04
sbezverkyou will need to focus just on agent21:04
sbezverkbtw neutron agent-list what does it show?21:04
*** inc0 has joined #openstack-kolla21:04
fooliounosbezverk: how do i determine the interface of the ovs bridge that dhcp agent is on?21:04
sbezverkfooliouno: kubectl exec {dhcp agent} -n kolla -- ip nets21:05
sbezverkfooliouno: kubectl exec {dhcp agent} -n kolla -- ip nets {dhcp namespace} ifconfig21:06
*** kbaegis has quit IRC21:06
fooliounosbezverk: everything is up on the agent-list21:06
sbezverkfooliouno: then you will need to find the same id on br-int21:07
*** rwallner has quit IRC21:10
*** jtriley has quit IRC21:10
fooliounosbezverk: operation not permitted for the ifconfig command21:10
*** rwallner has joined #openstack-kolla21:11
rwellumjascott1: sbezverk  - cleaning up /var/lib/kolla/volumes worked - thanks so much!21:11
jascott1prob needs to be in deployment guide21:11
sbezverkjascott1: good idea21:11
rwellumYeah def.21:12
jascott1not it!21:12
jascott1; )21:12
rwellumaha21:12
rwellumOk I'll do it :)21:12
*** manheim_ has joined #openstack-kolla21:13
*** rwallner_ has joined #openstack-kolla21:13
jascott1ha took 73mins to destroy 2813 openstack resources with terraform21:14
*** rwallner has quit IRC21:14
jascott1(cause they were all grouped into sets of 16)21:14
sbezverkjascott1: hmm, I thought terra forming is when you create things, and not destroying them ;)21:15
jascott1what you create must eventually be destroyed :)21:15
jascott1that was leftovers from inc0's kolla workshop21:16
*** chas has joined #openstack-kolla21:17
inc0terraforming is creating, marsforming is grinding it all to dust?21:17
*** rwallner_ has quit IRC21:17
*** rwallner has joined #openstack-kolla21:17
sbezverkinc0: lol21:18
vhosakotinc0: hahahaha21:19
*** rwallner has quit IRC21:20
*** rwallner has joined #openstack-kolla21:20
jascott1what is draining the swamp? Adding more alligators?21:21
Reepicheeplol,  I have used terraform to break my openstack instances .. just to see how it handles it21:21
*** rwallner has quit IRC21:21
*** rwallner has joined #openstack-kolla21:22
Reepicheepand jascott1 it took along time when it got to counting instances in the thousands21:22
*** chas has quit IRC21:22
inc0if you put enough aligators, at some point there will be no water left21:22
kolla-slack1<jascott1> lol that must be the strategy21:22
*** ducttape_ has joined #openstack-kolla21:23
Reepicheepand it actually was the "destroy" part which made terraform so nice.. anyone can create 1000 instances but deleting 1000 instances without custom scripting is not so much fun21:25
*** mannamne has quit IRC21:28
rwellumThere's this mystical 'helm nuke' plugin...21:28
rwellumvhosakot: sbezverk fully deployed kolla-k8s - thanks for help!21:28
vhosakotrwellum: fully meaning? horizon is up, nova vm boots and ping google with good L3?21:29
vhosakotrwellum: so, what did cleaning /var/lib/kolla/volumes exactly resolve?  the pod errors ?21:30
*** unicell has joined #openstack-kolla21:30
rwellumvhosakot: on the horizon step now. I meant all os services I deployed came up and stable.21:30
vhosakotrwellum: ah nice, cool.21:31
rwellumvhosakot: no cleaning the volumes allowed mariadb to come up21:31
rwellumvhosakot: yeah been trying since before the summit. The genconfig and move to 1.6.3 killed me.21:32
vhosakotrwellum: ah cool, mariadb21:32
vhosakotcool21:32
jascott1ya 1.6.3 got me, was working on playbook and then all of a sudden it exploded21:35
jascott1the init-mariadb job fails if volume exists21:36
rwellumjascott1: so you using sbezverk version of 1.6.3 then?21:37
rwellumDoes anyone know where the keystonerc_admin file should exist?21:37
jascott1sorry I mixed my statements there, 1.6.3 unrelated to volume thing21:38
sbezverkrwellum: you create them21:38
sbezverkby running a script21:38
*** chas has joined #openstack-kolla21:38
*** jascott1 has quit IRC21:38
sbezverkrwellum: build_local_admin_keystonerc.sh21:38
*** jascott1 has joined #openstack-kolla21:39
rwellumAh!21:39
rwellumty21:39
*** jascott1 has quit IRC21:41
*** jascott1_ has joined #openstack-kolla21:41
kolla-slack1<jascott1> haha just overheard random coworker complaining about having to build all containers all the time (prob not kolla)21:43
*** chas has quit IRC21:43
*** jascott1_ has quit IRC21:44
*** manheim_ has quit IRC21:45
*** manheim_ has joined #openstack-kolla21:46
*** manheim_ has quit IRC21:50
*** gomarivera_ has joined #openstack-kolla21:51
*** manheim_ has joined #openstack-kolla21:52
*** StephenWang1991 has joined #openstack-kolla21:52
*** manheim_ has quit IRC21:53
*** StephenWang1991 has quit IRC21:57
*** chas has joined #openstack-kolla21:59
*** chas has quit IRC22:03
*** salv-orlando has joined #openstack-kolla22:03
*** jascott1 has joined #openstack-kolla22:06
*** tonanhngo has quit IRC22:13
inc0speaking of which22:13
inc0with my glorious tool we can setup whole set of kolla images locally within 3min and single command;)22:13
jascott1inc0 can you spawn vm on sc cluster? Its not working for me but it was earlier (~3 hrs ago)22:15
jascott1not even working thru horizon22:15
jascott1aborted: Failed to allocate the network(s), not rescheduling.22:16
jascott1only added demo-net22:16
Manheimlet's say i have an existing deployment, if I want to add magnum to it I do: 1) enable magnum in globals 2) pull 3) deploy22:16
inc0hmm22:16
Manheimis this wirkflow correct?22:16
inc0openstack is broken, how fun, let me check what happened22:16
Manheimand most efficient22:17
jascott1maybe something got wrecked deleting all those resources earlier?22:17
jascott1Manheim sorry I dont know22:17
*** itlinux has joined #openstack-kolla22:19
rwellumIt's an important work-flow question. I added a 'what happens next' section to the deployment guide for just these type of questions.22:19
*** chas has joined #openstack-kolla22:20
jascott1well for kk8s you would just helm install magnum (if it existed ;)22:23
*** chas has quit IRC22:24
ReepicheepManheim It's not a bad idea to precheck before you deploy22:25
*** fooliouno has quit IRC22:25
Manheimoh, yes, prechecks and bootstrap are kinda in the deploy section22:25
Reepicheepuse basicly use "deploy" when your changes add new containers to your environment22:26
Reepicheepif the changes are configs only you can use a "reconfigure"22:26
Manheimaha22:27
Reepicheepand if the containers are being replaced you use "upgrade"22:27
*** tonanhngo has joined #openstack-kolla22:27
Manheimbuy why would I do reconfigure when, let's say i edit config for nova and I can just restart the container?22:27
Manheimcoz tags never worked for me, like kolla-ansible reconfigure -t nova22:28
Manheimjust an example22:28
Manheimreconfigure goes over all roles22:28
Reepicheepthe tags should do the base stuff then just the tagged roles, so there are core stuff that is run through each time22:29
ReepicheepI have always used "--tags nova" instead of "-t nova"22:30
jascott1inc0 i dont even know where to start troubleshooting22:30
inc0by reading logs jascott1 :P22:31
jascott1never!22:31
jascott1;)22:31
*** tonanhngo has quit IRC22:32
ReepicheepManheim you also should not have to do the bootstrap again unless you add an additional node22:32
Manheimused --tags nova i believe22:33
ReepicheepI'm not sure what all the tags are but that should work22:34
Manheimit's an ansible thing, to make the ops more granular22:35
Manheimone thing I love about ansible22:35
*** manheim_ has joined #openstack-kolla22:35
*** Manheim has quit IRC22:36
*** emccormick has quit IRC22:37
*** dave-mccowan has joined #openstack-kolla22:37
*** rwallner has quit IRC22:37
Reepicheepyeah.. I'm learning ansible,  I guess I was refering to that I don't know the extent of the list of "tags" that can be used and that I assume that "nova" was one of the tags22:37
*** manheim_ has quit IRC22:38
jascott1inc0 shouldnt there be a /var/log/kolla?22:39
inc0negative22:39
Reepicheeplol, half the fun of troubleshooting is finding the logs22:40
*** manheim has joined #openstack-kolla22:40
*** chas has joined #openstack-kolla22:41
*** rwallner_ has joined #openstack-kolla22:41
*** gomarivera_ has quit IRC22:42
jascott1apparently22:42
*** manheim has quit IRC22:43
*** salv-orlando has quit IRC22:44
ReepicheepI'm not yet sure if it's the same on kolla-k8s but on kolla-ansible there is a  docker volume created for the openstack logs22:44
*** rwallner_ has quit IRC22:45
*** rwallner has joined #openstack-kolla22:45
*** chas has quit IRC22:45
*** goldyfruit has quit IRC22:46
*** egonzalez has quit IRC22:46
*** manheim has joined #openstack-kolla22:46
*** rwallner has quit IRC22:49
jascott1oh this is openstack on top of ironic.. no k8s22:50
*** salv-orlando has joined #openstack-kolla22:50
*** emccormick has joined #openstack-kolla22:59
*** vhosakot has quit IRC23:00
*** chas has joined #openstack-kolla23:02
*** chas has quit IRC23:06
*** emccormick has quit IRC23:19
*** manheim has quit IRC23:21
*** chas has joined #openstack-kolla23:22
*** jamesPR has joined #openstack-kolla23:24
*** chas has quit IRC23:27
*** ducttape_ has quit IRC23:29
*** manheim has joined #openstack-kolla23:37
*** huzhengchuan has joined #openstack-kolla23:40
cineramahi folks...is the vagrant dev env in kolla-ansible the canonical location now?23:41
inc0cinerama: I'm pretty sure vagrant dev env is forgotten;)23:43
*** chas has joined #openstack-kolla23:43
cineramaactually deprecated or just kind of abandoned?23:44
*** chas has quit IRC23:48
jascott1cinerama hmm well it hasnt been touched in 6-7 months23:54
*** bmaltais has quit IRC23:55
jascott1we should probably add to kk8s deploy guide that it wont fit on vagrant and to not try23:56
*** yingjun has joined #openstack-kolla23:56
cineramayeah i checked the git logs but imo if it's not currently maintained might be better to note that in docs or just remove it23:57
jascott1im not sure the vagrant part changes much.. having an issue?23:58
*** manheim has quit IRC23:58
cineramayeah there's just a couple little things missing. i can submit a patch, just was wondering whether it's worthwhile23:58
jascott1and /contrib stuff is usually YMMV23:59
jascott1if you submit a patch then it will be maintained :)23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!