Monday, 2017-04-24

*** yangyapeng has quit IRC00:02
*** slagle has quit IRC00:20
*** spsurya has joined #openstack-kolla00:21
*** sbezverk has joined #openstack-kolla00:26
*** manheim has quit IRC00:32
*** jascott1 has joined #openstack-kolla00:32
*** jascott1 has quit IRC00:36
*** StephenWang1991 has joined #openstack-kolla00:39
*** chas has joined #openstack-kolla00:41
*** StephenWang1991 has quit IRC00:44
spsuryamorning all00:45
*** chas has quit IRC00:46
*** cuongnv has joined #openstack-kolla00:47
*** hieulq has joined #openstack-kolla00:47
*** tovin07 has joined #openstack-kolla00:47
*** trozet has quit IRC00:50
sdakesup spsurya00:56
openstackgerritSteven Dake proposed openstack/kolla-ansible master: DNM: Test for a deploy of kolla-kubernetes  https://review.openstack.org/45913700:57
spsuryasdake: wanted some info on your patch https://review.openstack.org/#/c/45285400:59
spsuryasame for this one too https://review.openstack.org/#/c/452855/00:59
spsuryaone small  item too https://review.openstack.org/#/c/457870/01:00
sdakespsurya ok - will get to it in a moment - eating shorlty01:04
spsuryasdake:  np01:05
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: DNM: Test out insecure setting  https://review.openstack.org/45913801:05
*** yangyapeng has joined #openstack-kolla01:11
spsuryaJeffrey4l: sup  https://review.openstack.org/#/c/45586601:12
*** duonghq has joined #openstack-kolla01:22
duonghqmorning guys01:22
Jeffrey4lspsurya, when not use https://docs.openstack.org/developer/kolla-ansible/vagrant-dev-env.html ?01:23
Jeffrey4li guess we should avoid to use github.com.01:23
spsuryaJeffrey4l:  yes we should, but that moment link was not working01:24
spsuryaI will the update the01:24
spsuryasame01:24
Jeffrey4lthe original link is wrong. try mine.01:24
*** zhubingbing has joined #openstack-kolla01:24
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-ansible master: Development Environment With Vagrant link not working  https://review.openstack.org/45586601:26
spsuryaJeffrey4l: corrected01:26
Jeffrey4lthanks. +2ed01:28
spsuryaJeffrey4l: thanks01:29
*** hrw has quit IRC01:30
*** hrw has joined #openstack-kolla01:32
*** iceyao has joined #openstack-kolla01:36
*** Seyong has joined #openstack-kolla01:36
openstackgerritMerged openstack/kolla-ansible master: Development Environment With Vagrant link not working  https://review.openstack.org/45586601:39
*** zhenguo has joined #openstack-kolla01:48
*** Pavo has quit IRC01:49
spsuryasdake not getting running        $kubectl get pods --all-namespaces | grep dns01:55
spsuryahttps://www.irccloud.com/pastebin/gzG0ohU8/01:55
*** Pavo has joined #openstack-kolla01:57
*** schwicht has quit IRC02:01
*** Pavo has quit IRC02:03
*** jascott1 has joined #openstack-kolla02:05
spsuryaApr 24 01:42:25 localhost journal: E0424 01:42:25.325310       1 leaderelection.go:229] error retrieving resource lock kube-system/kube-controller-manager: Get https://192.168.121.93:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 192.168.121.93:6443: getsockopt: connection refused02:10
spsuryathese are thge message log i can see the error02:11
*** jascott1 has quit IRC02:11
spsuryasbezverk: what can be the reason of failing this ?02:12
*** zhuzeyu has joined #openstack-kolla02:31
*** StephenWang1991 has joined #openstack-kolla02:38
*** chas has joined #openstack-kolla02:42
*** daidv has joined #openstack-kolla02:42
*** rwsu has joined #openstack-kolla02:45
sbezverkspsurya: netstat -tunlp, check if 6443 port is opened02:46
*** chas has quit IRC02:46
sbezverkif it is not then api-server is not running, if it is then make sure you copied /etc/kubernetes/admin.conf to $HOME/.kube/conifg02:47
zhuzeyuhi, sbezverk02:50
zhuzeyui have a question want to ask02:50
spsuryahttps://www.irccloud.com/pastebin/pnn7XYEr/02:51
zhuzeyuwhen i deploy kolla-k8s, how can i use local registry to pull images, I use the methods same as kolla-ansible used, but it seem that doesn't work02:52
zhuzeyuthanks02:52
daidvMorning, can anyone review for this backport: https://review.openstack.org/#/c/456513/02:56
sbezverkspsurya: port is opened which is good02:58
sbezverkthen copy file I mentioned before02:58
sbezverkchmod +r $HOME/.kube/conifg02:58
spsuryasbezverk: yes02:58
sbezverkand try again02:59
spsuryasbezverk: i see02:59
sbezverkzhuzeyu: you need to change your cloud yaml to point to your local registery02:59
sbezverkI am off for now, ttyl03:01
zhuzeyuthanks, let me have a try03:02
*** salv-orlando has joined #openstack-kolla03:07
*** salv-orlando has quit IRC03:12
*** ssurana has joined #openstack-kolla03:14
openstackgerritzhubingbing proposed openstack/kolla master: Fix tempest dockerfile  https://review.openstack.org/45915203:18
*** KImj has joined #openstack-kolla03:20
*** KImj has quit IRC03:25
*** MarginHu has joined #openstack-kolla03:26
*** tovin07_ has joined #openstack-kolla03:27
*** janonymous has joined #openstack-kolla03:29
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Add OS_REGION_NAME to admin-openrc.sh.j2  https://review.openstack.org/45915503:32
*** StephenWang1991 has quit IRC03:33
*** goldyfruit has quit IRC03:36
*** dave-mccowan has quit IRC03:37
*** gkadam has joined #openstack-kolla03:46
*** thomasem_ has quit IRC04:04
*** salv-orlando has joined #openstack-kolla04:08
*** jascott1 has joined #openstack-kolla04:08
*** salv-orlando has quit IRC04:13
*** jascott1 has quit IRC04:13
*** myoung has joined #openstack-kolla04:19
*** iceyao has quit IRC04:32
*** jayahn has joined #openstack-kolla04:36
*** tervancovan has joined #openstack-kolla04:38
tervancovanHi all04:38
tervancovanI found that reason why verification of k8s didn't working in my company04:40
tervancovanport 53 for these ips - >8.8.8.8, 8.8.4.4  was blocked04:40
*** StephenWang1991 has joined #openstack-kolla04:41
tervancovansdake so "nslookup kubernetes" did not run04:42
*** chas has joined #openstack-kolla04:43
openstackgerrithosup choi proposed openstack/kolla-kubernetes master: Update deployment-guide for ubuntu user  https://review.openstack.org/45916304:44
*** chas has quit IRC04:47
*** salv-orlando has joined #openstack-kolla04:57
*** thomasem_ has joined #openstack-kolla05:00
*** eanylin has quit IRC05:02
*** skramaja has joined #openstack-kolla05:16
*** Guest31033 has joined #openstack-kolla05:19
*** StephenW_ has joined #openstack-kolla05:29
*** StephenWang1991 has quit IRC05:29
*** daidv has quit IRC05:32
*** iceyao has joined #openstack-kolla05:46
*** iceyao has quit IRC05:50
*** MarginHu has quit IRC06:00
*** MarginHu has joined #openstack-kolla06:00
*** StephenW_ has quit IRC06:06
*** StephenWang1991 has joined #openstack-kolla06:07
*** jianyi has joined #openstack-kolla06:15
*** thomasem_ has quit IRC06:16
*** thomasem_ has joined #openstack-kolla06:20
*** jayahn has quit IRC06:21
*** leseb has quit IRC06:21
*** leseb has joined #openstack-kolla06:28
*** kolla-slack1 has joined #openstack-kolla06:36
*** yee379 has quit IRC06:36
*** kolla-slack has quit IRC06:36
*** yee379 has joined #openstack-kolla06:37
*** shardy has joined #openstack-kolla06:42
*** chas has joined #openstack-kolla06:43
zhuzeyuhi06:47
tervancovanhi06:48
*** chas has quit IRC06:48
zhuzeyui have a question want to ask, does the version of kubernetes-entrypoint iamges only use 4.0.006:49
*** salv-orlando has quit IRC06:50
zhuzeyui modify the images version in cloud.yaml , other service can pull images with 4.0.1 in local registry, but I find the in log that kubernetes-entrypoint still use 4.0.006:51
masbergood afternoon all, horizon ubuntu binaries images from docker hub looks like doesnt have css and images?06:52
zhuzeyubut there is no 4.0.0 kubernetes-entrypoint in my  local registry, so it get eror06:52
masberI can get to the login page but image and format is missing06:52
*** iceyao has joined #openstack-kolla06:53
*** athomas has joined #openstack-kolla06:54
masbersame issue after login06:55
masberlots of http 404 errors form the website06:55
*** iceyao has quit IRC06:57
*** satyar has joined #openstack-kolla06:58
*** numans has joined #openstack-kolla07:00
openstackgerritNuman Siddique proposed openstack/kolla master: Support openvswitch-ovn docker images  https://review.openstack.org/45917907:00
openstackgerritBertrand Lallau proposed openstack/kolla-ansible stable/ocata: Fix outdated InfluxDB configuration  https://review.openstack.org/45649107:02
*** chas has joined #openstack-kolla07:07
*** manheim has joined #openstack-kolla07:20
*** magicboiz has joined #openstack-kolla07:21
*** manheim has quit IRC07:21
*** manheim has joined #openstack-kolla07:22
*** pcaruana has joined #openstack-kolla07:24
*** ssurana has quit IRC07:26
*** lpetrut has joined #openstack-kolla07:26
hrwmorning07:26
*** ssurana has joined #openstack-kolla07:27
*** shardy has quit IRC07:28
*** matrohon_ has joined #openstack-kolla07:31
*** ssurana has quit IRC07:31
manheimhello07:33
*** jaosorior has joined #openstack-kolla07:33
*** rmart04 has joined #openstack-kolla07:34
tervancovanCan I use just one NIC (eth) to test of kolla-k8s?07:36
tervancovanI have a just one nic device (eth0)07:36
hrwmy patch of a day is https://review.openstack.org/#/c/430940 ;D07:37
tervancovanso Can I set  "neutron_external_interface=eth0"  like this?07:37
*** shardy has joined #openstack-kolla07:40
*** manheim has quit IRC07:42
*** lpetrut has quit IRC07:43
*** Serlex has joined #openstack-kolla07:44
*** iceyao has joined #openstack-kolla07:48
*** iceyao has quit IRC07:52
*** blallau has left #openstack-kolla07:55
*** blallau has joined #openstack-kolla07:55
*** salv-orlando has joined #openstack-kolla07:58
*** iceyao has joined #openstack-kolla08:01
*** gfidente has joined #openstack-kolla08:07
*** manheim has joined #openstack-kolla08:10
*** lpetrut has joined #openstack-kolla08:12
SerlexMorning mgkwill08:12
*** dixiaoli has joined #openstack-kolla08:16
*** oanson has quit IRC08:23
*** oanson has joined #openstack-kolla08:24
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Tempest: remove all oslo.policy related stuff  https://review.openstack.org/45919008:24
*** ssurana has joined #openstack-kolla08:29
*** magicboiz has quit IRC08:30
*** pbourke has quit IRC08:31
*** pbourke has joined #openstack-kolla08:31
zhuzeyutervancovan, need two nic device in doc08:33
*** ssurana has quit IRC08:35
*** zhubingbing has quit IRC08:42
*** magicboiz has joined #openstack-kolla08:42
*** zhubingbing has joined #openstack-kolla08:44
openstackgerritDou Rui Yuan proposed openstack/kolla-ansible master: Use cryptography instead of pycrypto  https://review.openstack.org/45919508:47
tervancovanzhuzeyu thank you for reminding it but I have only one device right now so I am looking for work around right now...08:54
*** mgoddard has joined #openstack-kolla08:58
zhuzeyuwhy not add virtual network interface to test09:15
*** zhubingbing has quit IRC09:16
*** zhubingbing_ has joined #openstack-kolla09:18
*** skramaja has quit IRC09:19
*** skramaja has joined #openstack-kolla09:19
*** rmart04_ has joined #openstack-kolla09:23
*** lpetrut has quit IRC09:24
openstackgerritMerged openstack/kolla-ansible stable/ocata: Fix outdated InfluxDB configuration  https://review.openstack.org/45649109:24
*** rmart04 has quit IRC09:24
*** rmart04_ is now known as rmart0409:24
*** MarginHu has quit IRC09:24
*** MarginHu has joined #openstack-kolla09:24
openstackgerritMerged openstack/kolla-ansible stable/ocata: neutron_vpnaas_agent: kolla_docker parameter 'volumes' is wrong  https://review.openstack.org/45787909:25
openstackgerritzhuangzejian proposed openstack/kolla-ansible master: Project service has none description in deployment  https://review.openstack.org/45921109:26
*** MarginHu has quit IRC09:28
*** rmart04 has quit IRC09:29
*** rmart04 has joined #openstack-kolla09:30
openstackgerritDou Rui Yuan proposed openstack/kolla-ansible master: Use cryptography instead of pycrypto  https://review.openstack.org/45919509:31
*** zhuzeyu has quit IRC09:38
*** zhuzeyu has joined #openstack-kolla09:39
*** StephenWang1991 has quit IRC09:39
*** StephenWang1991 has joined #openstack-kolla09:40
*** zhubingbing_ has quit IRC09:41
*** sambetts|afk is now known as sambetts09:47
*** jmccarthy has joined #openstack-kolla09:47
*** salv-orl_ has joined #openstack-kolla09:49
*** satyar has quit IRC09:52
*** salv-orlando has quit IRC09:52
*** masber has quit IRC09:54
*** dixiaoli_ has joined #openstack-kolla09:54
*** dixiaoli has quit IRC09:56
spsuryatervancovan: you can create dummy one09:58
spsuryawith ip link09:58
tervancovanspsurya thanks09:58
tervancovanspsurya could you explain or share link of document for it?09:59
spsuryatervancovan: sure09:59
*** zhubingbing has joined #openstack-kolla10:04
*** iceyao has quit IRC10:07
*** skramaja_ has joined #openstack-kolla10:09
*** skramaja has quit IRC10:09
*** cuongnv has quit IRC10:10
*** egonzalez has joined #openstack-kolla10:11
*** jascott1 has joined #openstack-kolla10:11
openstackgerrithosup choi proposed openstack/kolla-kubernetes master: Seperate kolla and kolla-ansible search paths Add method to find  kolla-ansible path Fix variable name from kolla_dir to kolla_ansible_dir  https://review.openstack.org/45922810:12
*** jascott1 has quit IRC10:17
*** tovin07_ has quit IRC10:18
jmccarthyHave any "4.0.1" images been pushed or ? I tried a deploy but I think it complained about missing toolbox (I was able to deploy "4.0.0")10:18
egonzalezjmccarthy no, images aren't pushed yet10:20
jmccarthyAh ok just checking it wasn't just me :)10:22
*** yangyapeng has quit IRC10:22
sdakespsurya the reason dns fails to enter 3/3 is because the CNI is not yet operational10:22
sdakemorning folks10:24
tervancovanmorning sdake10:24
sdakemorning tervancovan10:24
sdakejmccarthy i dont know if 4.0.1 images have been pushed or not10:25
spsuryatervancovan:   $ sudo ip link add eth1 type dummy | true10:26
spsuryasudo ip link set eth1 up10:26
spsuryasdake: morning10:26
*** jianyi has quit IRC10:26
tervancovanThaaaanks spsurya10:27
*** StephenWang1991 has quit IRC10:28
*** iceyao has joined #openstack-kolla10:30
*** ssurana has joined #openstack-kolla10:31
spsuryatervancovan: np10:34
spsuryasdake: what can be the possible solution10:35
spsurya?10:35
sdakespsurya run kubectl get pods -n kube-system -> paste10:35
*** ssurana has quit IRC10:36
*** oanson has quit IRC10:37
spsuryanot running atm10:37
spsuryawilll paste soon10:37
*** oanson has joined #openstack-kolla10:37
*** felixtsang has joined #openstack-kolla10:38
felixtsanghi every one, newly join the chat . i am studying kolla deployment.  A question : Can kolla deploy HA controller nodes?10:39
jmccarthyOk, I'll have a go at building them instead maybe in a while, for me I seem to be having trouble since openvswitch was broken out into it's own role (out from under neutron)10:39
spsuryasdake:10:41
spsuryahttps://www.irccloud.com/pastebin/5vBvqXRH/10:41
spsuryajust run. I will run this again.10:42
spsuryahttps://www.irccloud.com/pastebin/s18yoEu1/10:42
spsuryaall ran except dns one10:42
egonzalezjmccarthy ovs split should not have any behaviour in kolla images, images were already independent, but ansible role was merged into neutron10:45
egonzalezfelixtsang yep, just add more nodes into [control] group at inventory file10:45
jmccarthySorry I don't mean to confuse images/ansible - I mean get both by pip install kolla-ansible==<version>10:46
felixtsang++egonzalez10:46
jmccarthyI think it's the other way ? openvswitch is no longer under neutron now10:46
egonzalezjmccarthy yep, was merged and splited out now10:47
jmccarthyWhich is why I wanted to try 4.0.1, thinking that they are split in that, and see if it worked :)10:48
egonzalezjmccarthy nope, in 4.0.1 (stable) is not splited, only in current master branch10:49
jmccarthyOh right - ok 4.0.1 wouldn't help me then :)10:49
sdakespsurya was this not working for you previously?10:51
spsuryasdake: it was working for me10:51
sdakeand then stopped working?10:51
spsuryabut since two days it is not10:51
spsuryaIIRC10:51
*** felixtsang has quit IRC10:51
sdakei haven' tused the guide in about two days10:51
sdakeso - it could be that canal images have changed in some subtle way10:52
sdakeor it could be an environmental problem on your end10:52
spsuryaI am runing the same i used to10:52
*** iceyao has quit IRC10:54
spsuryasdake: ttyl10:56
sdakespsurya night - i'll try to look at that problem today10:56
sdakespsurya i just woke up - so getting warmped up10:56
openstackgerritMerged openstack/kolla-ansible stable/ocata: Mount /lib/modules folder for cinder-volume container  https://review.openstack.org/45843011:03
*** dave-mccowan has joined #openstack-kolla11:09
*** zhugaoxiao has quit IRC11:13
hrwsdake: can you look at https://review.openstack.org/430940 one? (main non-x86 support one)11:13
*** salv-orl_ has quit IRC11:15
*** satyar has joined #openstack-kolla11:16
*** zhurong has joined #openstack-kolla11:17
*** zhugaoxiao has joined #openstack-kolla11:17
*** janki has joined #openstack-kolla11:21
*** rhallisey has joined #openstack-kolla11:28
*** oanson has quit IRC11:28
openstackgerritMerged openstack/kolla-ansible master: Tempest: remove all oslo.policy related stuff  https://review.openstack.org/45919011:29
*** oanson has joined #openstack-kolla11:29
*** iniazi_ has joined #openstack-kolla11:29
egonzalezblallau ping11:30
*** iniazi has quit IRC11:30
blallauyep11:32
*** zhugaoxiao has quit IRC11:34
*** zhugaoxiao has joined #openstack-kolla11:35
*** lpetrut has joined #openstack-kolla11:38
*** Pavo has joined #openstack-kolla11:39
spsuryasdake: just came  from office11:42
openstackgerritMerged openstack/kolla-ansible master: Fix swift-ring-builder path for container  https://review.openstack.org/45905411:44
*** dixiaoli_ has quit IRC11:48
*** lpetrut_ has joined #openstack-kolla11:48
openstackgerritZeyu Zhu proposed openstack/kolla-kubernetes master: Modify the image_tag to 4.0.0 in all_values.yaml  https://review.openstack.org/45925011:48
openstackgerritMerged openstack/kolla-ansible master: Add kolla-ansible to operating-kolla.rst document  https://review.openstack.org/45611011:50
*** janki has quit IRC11:53
*** jchhatbar has joined #openstack-kolla11:53
*** rwallner has joined #openstack-kolla11:59
*** iceyao has joined #openstack-kolla11:59
*** slagle has joined #openstack-kolla12:00
*** salv-orlando has joined #openstack-kolla12:00
*** lpetrut_ has quit IRC12:01
*** lpetrut_ has joined #openstack-kolla12:03
*** lpetrut_ has quit IRC12:03
*** lpetrut_ has joined #openstack-kolla12:03
*** athomas has quit IRC12:06
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ceph: take care of ceph-fuse package availability  https://review.openstack.org/44063412:06
*** athomas has joined #openstack-kolla12:12
*** jascott1 has joined #openstack-kolla12:13
*** jchhatbar is now known as janki12:16
*** jascott1 has quit IRC12:18
*** lpetrut_ has quit IRC12:19
*** lpetrut_ has joined #openstack-kolla12:19
*** haplo37_ has quit IRC12:23
openstackgerritPavel Gluschak (scsnow) proposed openstack/kolla-ansible master: kolla_toolbox: allow docker api version to be specified  https://review.openstack.org/45926912:28
*** haplo37_ has joined #openstack-kolla12:34
*** ssurana has joined #openstack-kolla12:34
*** rwellum has joined #openstack-kolla12:35
openstackgerrithosup choi proposed openstack/kolla-kubernetes master: Seperate kolla and kolla-ansible search paths Add method to find  kolla-ansible path Add dictionary variable for ymls Add array variable for roles Fix variable name from kolla_dir to kolla_ansible_dir  https://review.openstack.org/45922812:36
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add vmware DVS support to kolla-ansible  https://review.openstack.org/45927012:38
*** ssurana has quit IRC12:38
*** zhurong has quit IRC12:39
openstackgerrithosup choi proposed openstack/kolla-kubernetes master: Update deployment-guide for ubuntu user Add title for installing ubunut package Change-Id: Ib67a90ac2b0504e9fe6b635b71612b87631507d1  https://review.openstack.org/45916312:39
openstackgerritjimmygc proposed openstack/kolla master: Support VMware DVS  https://review.openstack.org/45927312:45
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add vmware DVS support to kolla-ansible  https://review.openstack.org/45927012:46
*** salv-orlando has quit IRC12:47
openstackgerrithosup choi proposed openstack/kolla-kubernetes master: Seperate kolla and kolla-ansible search paths  https://review.openstack.org/45922812:48
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Add cli options to cleanup-images  https://review.openstack.org/43150312:53
*** manheim has quit IRC12:53
openstackgerritMerged openstack/kolla-ansible master: Fix the remove mariadb volume failed  https://review.openstack.org/44735912:53
*** lpetrut_ has quit IRC12:54
*** lpetrut has quit IRC12:57
*** goldyfruit has joined #openstack-kolla12:58
*** kencjohnston has quit IRC13:02
*** iceyao has left #openstack-kolla13:02
*** yangyapeng has joined #openstack-kolla13:05
*** gkadam has quit IRC13:07
*** schwicht has joined #openstack-kolla13:11
*** trozet has joined #openstack-kolla13:12
*** Guest31033 has quit IRC13:22
*** lpetrut has joined #openstack-kolla13:28
*** jtriley has joined #openstack-kolla13:31
*** srwilkers has joined #openstack-kolla13:33
Serlexmgkwill: it was inventory issue, ignore :)13:33
*** n0isyn0ise has left #openstack-kolla13:34
*** lpetrut has quit IRC13:34
*** eanylin has joined #openstack-kolla13:46
*** manheim has joined #openstack-kolla13:47
*** tovin07_ has joined #openstack-kolla13:52
*** manheim has quit IRC13:53
*** manheim has joined #openstack-kolla13:54
*** tovin07_ has left #openstack-kolla13:56
*** manheim_ has joined #openstack-kolla14:01
*** manheim has quit IRC14:04
*** apuimedo has joined #openstack-kolla14:15
apuimedoinc0: sdake: We need to push some images to dockerhub and I was wondering if you use openstack infra for the pushes to https://hub.docker.com/u/kolla/14:16
dmelladoegonzalez: ^^ ;)14:17
egonzalezapuimedo for now is manual work, but automating pushes is wip atm14:18
apuimedoegonzalez: define manual14:18
dmelladoegonzalez: oh, I thought that was automated already14:18
dmelladoto you have some documentation on it?14:19
egonzalezbuild local and push to dockerhub by someone14:19
dmellados/t/d14:19
apuimedoas in somebody builds them on their laptop/server and then massively pushes? With a script14:19
apuimedook, fair enough14:19
inc0it's nto automated14:19
inc0but that's top of my list14:19
dmelladois there any spec or plan to have that automated?14:19
dmelladoinc0: ?14:19
apuimedoinc0: egonzalez: what's the current work on automating it?14:19
hrwcan someone (other than egonzalez) check https://review.openstack.org/#/c/434810/ patch? it handles nova-libvirt on ppc64le14:20
inc0we had a lot of discussions regarging tag naming and stuff like tha14:20
inc0t14:20
inc0I'll work on script today14:20
openstackgerritMerged openstack/kolla master: handle rtslib(-fb) package names and dependencies  https://review.openstack.org/45080514:20
dmelladoinc0: I'll keep an eye on it, thanks!14:21
apuimedoinc0: and you'll run it on the infra, or externally for now?14:21
inc0I'll try to get it to infra asap14:22
dmelladoso the image building is done just in local laptop, got it14:22
inc0yeah14:23
inc0we want to run daily builds14:24
apuimedoinc0: we wanted to do per commit14:26
apuimedosince our build burden is lower14:26
inc0well per commit is a little overkill imho14:27
hrwrtslib merged! yay! :D14:28
*** yangyape_ has joined #openstack-kolla14:28
inc0do you think we need per-commit14:28
inc0?14:28
inc0(it's also going to be a bit of logistical problem)14:28
inc0daily - I'm going to use prebuild images from latest commit14:29
inc0we upload built images per commit in form of tarballed registry14:29
inc0to tarballs.openstack.org14:29
*** yangyapeng has quit IRC14:30
apuimedoinc0: nah... We don't really need it14:31
apuimedoit's because we were using dockerhub automated builds14:31
*** magicboiz has quit IRC14:31
Reepicheepout of curiosity.. and forgive me I haven't looked but are there any smoke / spec tests beyond docker build status for the images?14:33
*** magicboiz has joined #openstack-kolla14:33
inc0Reepicheep: we also run deploy gates on merhe14:33
inc0on-voting but we don't merge if they fail14:33
*** srwilkers has quit IRC14:33
Reepicheepok.. I see so that is pre-commit14:34
inc0so it works like this, you commit- gates run, you merge-images are uploaded, daily we take latest uploaded images and will push it to dockerhub14:35
*** lpetrut has joined #openstack-kolla14:35
openstackgerritMerged openstack/kolla master: Fix tempest dockerfile  https://review.openstack.org/45915214:36
Reepicheepthanks,  That makes sense.  I would agree per-commit pushs to docker hub would seem a little over kill but daily would be nice14:36
*** eaguilar has joined #openstack-kolla14:37
*** srwilkers has joined #openstack-kolla14:38
*** magicboiz has quit IRC14:38
hrwdown to 4 patches in queue14:43
hrwhttps://review.openstack.org/430940 adds non-x86 support. tested with debian, centos, ubuntu on aarch64 (150-200 images per combination), ppc64le (good but worse that aarch64 due to lack of packages in centos) and x86-64 (to know it works still).14:47
hrwhttps://review.openstack.org/434810 takes care of nova-libvirt dependencies on ppc64le (no Xen, no DMI)14:48
hrwhttps://review.openstack.org/440634 takes care of 'ceph-fuse' package availability (which is poor)14:48
hrwand https://review.openstack.org/444832 (mariadb repos under CentOS) is waiting for recheck results14:49
hrwif someone has a time then please take a look14:49
*** japestinho has quit IRC14:50
manheim_i can see that ansible 2.3 is in apt repos, did you have any issues with it?14:52
manheim_maybe something in TASK [mariadb : Creating haproxy mysql user]14:52
openstackgerritPavel Gluschak (scsnow) proposed openstack/kolla-ansible master: kolla_toolbox: allow docker api version to be specified  https://review.openstack.org/45926914:55
*** manheim_ has quit IRC14:57
*** lpetrut has quit IRC14:57
*** magicboiz has joined #openstack-kolla14:57
*** adrian_otto has joined #openstack-kolla15:01
*** salv-orlando has joined #openstack-kolla15:02
*** manheim has joined #openstack-kolla15:05
*** eaguilar_ has joined #openstack-kolla15:06
egonzalezhrw what you mean at comment in https://review.openstack.org/#/c/431503/6 ?15:09
*** eaguilar has quit IRC15:09
hrwegonzalez: you run 'docker images' for each argument15:09
hrwegonzalez: if I run that script with more than one option then you run 'docker images' per option15:10
egonzalezhrw is only run once, since each argument cannot be used with others15:10
hrwegonzalez: "cleanup-images --all --dangling" has sense still15:11
hrwegonzalez: --all will remove all Kolla ones, --dangling will remove orphans and base distro images stay15:12
*** skramaja_ has quit IRC15:12
hrwsure, it will remove just the second ones which will be another script bug15:13
*** janki has quit IRC15:13
openstackgerritMerged openstack/kolla stable/newton: Fix neutron-vpnaas for redhat derived bases  https://review.openstack.org/44995215:13
*** felixtsang has joined #openstack-kolla15:14
*** ssurana has joined #openstack-kolla15:15
hrwegonzalez: is my answer good?15:16
egonzalezhrw, --all remove also kolla dangling images (not all dangling for non kolla images)15:17
hrwI thought that --dangling is more for untagged ones like this:15:19
hrw<none>                                              <none>              4ad1f9535b94        2 weeks ago         522.3 MB15:19
*** dixiaoli has joined #openstack-kolla15:19
hrwI use docker-clean usually15:19
*** ssurana has quit IRC15:19
egonzalezhrw yep, filter dangling=true output all dangling images (kolla and not kolla)15:20
egonzalezhrw --all removes all images with label=kolla_version (this include orphaned images, since also have the label)15:21
hrwegonzalez: so --all --dangling still have sense?15:21
*** apuimedo is now known as apuimedo|away15:21
hrwon my aarch64/ssd machine 'docker images|wc -l' with 1184 images takes 21.69s15:21
*** dixiaoli has quit IRC15:22
hrwmy x86-64/ssd does same in 3.12s (1097 images)15:22
hrwin my 'lets see how many images we already built' I have 6 iterations over 'docker images' - moved to 'ask once, store in file' after 3rd run of script15:24
*** dixiaoli has joined #openstack-kolla15:24
*** eaguilar_ has quit IRC15:26
*** dixiaoli has quit IRC15:28
*** david-lyle has joined #openstack-kolla15:29
*** mgoddard has quit IRC15:30
*** trozet has quit IRC15:31
*** krtaylor has quit IRC15:34
*** eaguilar has joined #openstack-kolla15:34
kfox1111morning.15:37
hrwhi kfox111115:40
*** zhurong has joined #openstack-kolla15:42
*** apuimedo|away is now known as apuimedo15:44
*** felixtsang has quit IRC15:45
*** salv-orl_ has joined #openstack-kolla15:49
*** felixtsang has joined #openstack-kolla15:50
kfox1111hrw: hi. :)15:51
*** rmart04 has quit IRC15:51
*** salv-orlando has quit IRC15:52
*** zhurong has quit IRC15:57
*** blallau has quit IRC16:01
openstackgerritBertrand Lallau proposed openstack/kolla-kubernetes master: deployment-guide.rst enhancement and small fixes  https://review.openstack.org/45939216:02
*** zhurong has joined #openstack-kolla16:05
*** zhurong has quit IRC16:07
*** mdnadeem_home has joined #openstack-kolla16:08
*** mgoddard has joined #openstack-kolla16:11
*** manheim has quit IRC16:13
*** manheim has joined #openstack-kolla16:14
*** Serlex has quit IRC16:19
*** manheim has quit IRC16:20
*** jmccarthy has left #openstack-kolla16:21
*** salv-orl_ has quit IRC16:26
*** matrohon_ has quit IRC16:29
*** jascott1 has joined #openstack-kolla16:31
*** krtaylor has joined #openstack-kolla16:31
*** rmart04 has joined #openstack-kolla16:33
openstackgerritMerged openstack/kolla master: mariadb: handle lack of external repos on non x86  https://review.openstack.org/44483216:34
hrwYAY!16:35
*** rmart04 has quit IRC16:38
*** srwilkers has quit IRC16:39
*** athomas has quit IRC16:40
*** athomas has joined #openstack-kolla16:50
*** egonzalez has quit IRC16:51
*** felixtsang has quit IRC16:59
*** felixtsang has joined #openstack-kolla16:59
*** jaosorior is now known as jaosorior_away16:59
*** adrian_otto1 has joined #openstack-kolla17:06
*** zhugaoxiao has quit IRC17:06
*** zhugaoxiao has joined #openstack-kolla17:07
*** adrian_otto has quit IRC17:09
*** jemcevoy has joined #openstack-kolla17:12
*** srwilkers has joined #openstack-kolla17:14
*** imcsk8 has quit IRC17:23
*** imcsk8 has joined #openstack-kolla17:23
*** ssurana has joined #openstack-kolla17:23
*** adrian_otto1 has quit IRC17:23
*** srwilkers has quit IRC17:38
*** salv-orlando has joined #openstack-kolla17:38
*** gfidente is now known as gfidente|afk17:38
*** mdnadeem_home has quit IRC17:41
*** tonanhngo has joined #openstack-kolla17:45
*** felixtsang has quit IRC17:48
*** jascott1 has quit IRC17:51
*** srwilkers has joined #openstack-kolla17:54
*** chas has quit IRC17:56
*** felixtsang has joined #openstack-kolla17:58
*** manheim has joined #openstack-kolla18:16
inc0sleepy Monday I see18:27
*** manheim has quit IRC18:29
*** manheim has joined #openstack-kolla18:29
kfox1111indeed18:36
*** felixtsang has quit IRC18:38
*** felixtsang has joined #openstack-kolla18:39
kfox1111interesting.... https://review.openstack.org/#/c/456760/9/ansible/tune/roles/logging/templates/logging.conf.j218:40
*** manheim has quit IRC18:41
*** manheim has joined #openstack-kolla18:41
*** felixtsang has quit IRC18:43
*** salv-orlando has quit IRC18:45
*** mgoddard has quit IRC18:48
inc0never used this project18:48
*** manheim has quit IRC18:59
*** srwilkers has quit IRC18:59
*** schwicht has quit IRC18:59
*** jascott1 has joined #openstack-kolla19:08
openstackgerritBertrand Lallau proposed openstack/kolla-kubernetes master: deployment-guide.rst enhancement and small fixes  https://review.openstack.org/45939219:09
*** schwicht has joined #openstack-kolla19:13
*** satyar has quit IRC19:18
inc0centos people, for kolla-ansible "Aborting, target uses selinux but python bindings (libselinux-python) aren't installed!" <- how to handle this error19:19
inc0?19:19
inc0sdake sbezverk ^19:21
sbezverkinc0: never seen this error before..19:21
inc0thats kolla-ansible tho, but I expect it to be something non-ansible specifi19:22
inc0c19:22
*** dciabrin has quit IRC19:23
*** srwilkers has joined #openstack-kolla19:23
sbezverkinc0: if selinux get completely disabled, do you still see it?19:23
inc0disable selinux...that's it probably19:23
*** dciabrin has joined #openstack-kolla19:24
*** schwicht has quit IRC19:24
sbezverkinc0: setenforce 0 and try19:24
inc0y, it was enforcing19:24
*** Pavo has quit IRC19:26
*** jmccarthy has joined #openstack-kolla19:27
*** schwicht has joined #openstack-kolla19:27
jmccarthyIn the all-in-one inventory, are additional sections for openvswitch -db, and -vswitchd needed ? Just wondering, if sub components are not listed, then they all go according to the main openvswitch definition is it ?19:29
jmccarthy(or mulitnode for that matter)19:30
sean-k-mooneyinc0:  was that form a nova-compute log19:30
inc0sean-k-mooney: deployment19:30
inc0it fails on creating file on localhost by mariadb role19:31
sean-k-mooneyoh this wasnt spawning a vm19:31
sean-k-mooneyah i see19:31
inc0jmccarthy: there is little difference between aio and multinode19:31
inc0ehh I'm new to centos :(19:32
jmccarthyOk, what I mean is openvswitch has, network, compute, and manilla-share - so the openvswitch_vswitchd and openvswitch_db for example will end up on all of them or ?19:33
*** shardy has quit IRC19:33
sean-k-mooneyi would guess there has been a change to hte mysql binary that nolonger pulls in that dependecy19:33
inc0hmmm...good point19:33
sean-k-mooneyjmccarthy: it should19:34
inc0there is no need for 2 groups I think19:34
*** shardy has joined #openstack-kolla19:34
sean-k-mooneyopenvswitch is required to be deployed on nodes in the network, compute, and manilla-share groups19:34
sean-k-mooneyi think its only going to do it once though even if the host is listed in all of the above19:35
jmccarthysean, inc0 - ok I don't know it well enough offhand, I seem to be having issue with openvswitch since it is listed separately from neutron now - didn't know if it was related to that19:35
*** salv-orlando has joined #openstack-kolla19:35
sean-k-mooneyjmccarthy: it should be enabled by default so you should not have need to change anything if you are using the default inventory file19:36
inc0this task fails https://github.com/openstack/kolla-ansible/blob/5a4cb8a56fa5b5c37874d21f992d3ce77a11e49f/ansible/roles/mariadb/tasks/lookup_cluster.yml#L819:36
jmccarthyOk, I'll have to try it again and see where I'm going wrong :)19:37
sean-k-mooneyinc0: this is selinux on the host blocking you then as this is a local action19:37
sean-k-mooneywe used to have an entry in the quickstart a release or 2 ago telling you to disable selinux on the host19:38
sean-k-mooneyhave you priviouly had mariadb installed on the host?19:38
inc0dmsimard for the rescue;) https://dmsimard.com/2016/01/08/selinux-python-virtualenv-chroot-and-ansible-dont-play-nice/ thanks man!19:39
*** shardy has quit IRC19:39
*** felixtsang has joined #openstack-kolla19:40
sean-k-mooneyinc0: ah yes ansible_python_interpreter i have used that before19:41
inc0helped.19:41
sean-k-mooneyso why is this running in a python virtual enviornment anyway?19:42
sean-k-mooneyis this just a querk of your dev env?19:43
*** iniazi_ has quit IRC19:46
*** iniazi has joined #openstack-kolla19:46
*** schwicht has quit IRC19:48
*** chas has joined #openstack-kolla19:56
*** matrohon_ has joined #openstack-kolla19:56
dmsimardoh hai19:58
dmsimardYou can set ansible_python_interpreter at the task level if need be19:59
dmsimardi.e, https://github.com/rdo-infra/ci-config/blob/master/jenkins/jobs/scripts/weirdo-create-vm.sh#L99-L10419:59
* dmsimard goes back to his cave19:59
*** chas has quit IRC20:01
*** sambetts is now known as sambetts|afk20:01
*** pcaruana has quit IRC20:01
*** harlowja has quit IRC20:03
*** schwicht has joined #openstack-kolla20:07
*** MasterOfBugs has joined #openstack-kolla20:08
*** manheim has joined #openstack-kolla20:09
*** tonanhngo_ has joined #openstack-kolla20:12
*** manheim has quit IRC20:13
*** tonanhngo has quit IRC20:14
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: Docker Logging Support  https://review.openstack.org/45864920:16
*** tonanhngo has joined #openstack-kolla20:20
*** tonanhngo_ has quit IRC20:22
inc0jemcevoy: you had issues with ovs restarting due to guru meditation?20:23
inc0or sth like that right?20:23
inc0kfox1111: around?20:26
*** jascott1 has quit IRC20:26
*** jascott1 has joined #openstack-kolla20:26
inc0my ovs container restart with this "Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports."20:27
inc0seen that anyone?20:27
*** rhallisey has quit IRC20:29
*** eaguilar_ has joined #openstack-kolla20:37
*** eaguilar_ has quit IRC20:37
*** rwallner has quit IRC20:38
*** eaguilar has quit IRC20:38
*** rwallner has joined #openstack-kolla20:38
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: Docker Logging Support  https://review.openstack.org/45864920:43
*** srwilkers has quit IRC20:44
*** srwilkers has joined #openstack-kolla20:45
kfox1111inc0: ping.20:49
inc0hey20:49
kfox1111whats up?20:49
inc0I'm trying to debug strange issue, but I think I'm onto sth20:49
inc0and it might be k-ans related20:49
kfox1111k. whats up?20:50
kfox1111(or not, which ever the case may be. :)20:50
*** rwallner has quit IRC20:51
*** krtaylor has quit IRC20:51
*** rwallner has joined #openstack-kolla20:51
inc0for some reason my ovs doesn't seem to be deployed20:52
kfox1111ok.20:53
*** dixiaoli has joined #openstack-kolla20:58
*** matrohon_ has quit IRC20:58
*** harlowja has joined #openstack-kolla20:59
*** adrian_otto has joined #openstack-kolla21:02
jascott1kfox1111 I figured out the cause of my mysql problem, it couldnt bind port because apparently I am using wrong ip somewhere21:03
kfox1111hm.21:03
* inc0 facepalm21:03
inc0I had master in my kolla-ansible21:03
jascott1can anyone take a look at this and explain the mapping of k8s interfaces to kolla-k8s ? https://etherpad.openstack.org/p/knet21:04
kfox1111jascott1: I think it binds to 0.0.0.0. so not sure an ip. but having an already allocated mysql port might do it.21:04
jascott1hmm21:05
jascott1so pod and service CIDRs are not used for kolla specific settings correct?21:06
kfox1111jascott1: thats your config?21:06
jascott1no I just took all the network settings out the guide so I could reason about them21:06
kfox1111k. should I tweak in place or add notes?21:07
kfox1111maybe just add I guess...21:07
jascott1please21:07
jascott1either!21:07
jascott1after I understand it maybe I can write up something for others to explain21:07
kfox1111cool.21:09
*** Pavo has joined #openstack-kolla21:11
*** rwallner has quit IRC21:14
*** rwallner has joined #openstack-kolla21:15
*** vhosakot has joined #openstack-kolla21:18
*** adrian_otto has quit IRC21:19
*** Pavo has quit IRC21:20
*** adrian_otto has joined #openstack-kolla21:21
kfox1111jascott1: I think thats the traffic type use cases?21:21
kfox1111can you think of any more?21:21
jascott1i cannot21:21
jascott1thanks so much21:21
kfox1111does that kind of help?21:22
jascott1it does21:22
jascott1so  in  the end can someone get by with 2 nics plus flannel (and calicos) ?21:22
kfox1111hmm..... depends on how yo uare defining nics I guess.21:23
kfox1111flannel and calico's are virtual.21:23
jascott1right21:23
kfox1111you can get away with having just one physical nic if using vlans.21:23
kfox1111(I do that on one cloud)21:24
jascott1so everything kolla related seems to use mgmt interface21:24
kfox1111could probably get away with 1 network, but two's safer.21:24
jascott1well not external21:24
kfox1111and 3's much preffered if you have bmc's imo.21:24
jascott1in globals, api_interface and tunnel are left as 0.0.0.0 ?21:24
kfox1111yeah. its a carryover from when the config was in kolla-ansible.21:25
jascott1oh ah21:25
kfox1111I think we shoudl move those two config options directly into kolla-kube's config now.21:25
kfox1111so the user never has to touch it.21:25
kfox1111its bouod to 0.0.0.0, but the servers are in a pod's ip, so they only really have 1 ip.21:25
kfox1111so doesn't need to worry about it.21:26
kfox1111in fact, if you got a moment, and wanted to make a ps that went through kolla-kubernetes/ansible and s/api_interface_adddress/0.0.0.0/ that would be cool. :)21:26
openstackgerritRich Wellum proposed openstack/kolla-kubernetes master: Various cleanups and additions  https://review.openstack.org/45893921:27
*** rwallner has quit IRC21:28
jascott1ok thanks21:28
jascott1im going to try to get mine working and then can do21:28
kfox1111ok. cool. :)21:28
*** rwallner has joined #openstack-kolla21:28
rwellumHey jascott1 and kfox1111 can you take a look at: http://prnt.sc/f0bmuv - see if it makes any sense? :)21:29
jascott1rwellum i cannot tell you if its correct but it does make sense :)21:30
*** trozet has joined #openstack-kolla21:31
kfox1111rwellum: vibr0 is usually used to nat traffic out to the internet, but not traffic back.21:31
*** mgoddard has joined #openstack-kolla21:32
kfox1111so... are you trying to make it such that vm's running inside the vm get onto the external network and visa versa,21:32
*** jtriley has quit IRC21:32
kfox1111or make it such that vm's can reach out to the external netwokr, but cant be contacted back in?21:32
*** schwicht has quit IRC21:34
kfox1111I guess another questionn... is 10.240.43.234 network completely private, or is it your 'corperate private network'?21:34
kfox1111(or home use, or whichever?)21:34
*** manheim has joined #openstack-kolla21:35
*** schwicht has joined #openstack-kolla21:35
rwellumIn the deployment guide - sdake states that the neutron interface should be ip-less, for neutron to work. In my case, I just wanted two interfaces and this is usually how I achieve it - one is the physical through, and the other off vibr0. These questions are actually exactly why I wanted to draw this out.21:35
rwellum10.240.. is corporate private.21:35
kfox1111not strictly true. but if you want to use add_port, it will break things if it has an ip.21:36
kfox1111do you want fips on your vm's to show up on corperate private?21:36
kfox1111or just be able to reach them without corperate knowing?21:37
rwellumIt's handy because I can access them everywhere - but not sure it's what they want... Don't think they want openstack traffic running over their internal network...21:37
rwellumI guess if this was bare-metal, 'eth0' would be one of the physical ports with an assigned dhcp address, and eth1 - another port but don't use dhcp or assign a static ip?21:38
kfox1111it will happen either way, its just if they can see them or not.21:38
rwellumYeah true.21:38
*** krtaylor has joined #openstack-kolla21:39
kfox1111I guess a related question is, can you get a block of ip's on the 10.240.43.xxx network, or is that all just dhcp?21:39
*** jascott1 has quit IRC21:39
*** jascott1 has joined #openstack-kolla21:40
openstackgerritMarcus Williams proposed openstack/kolla-ansible master: Add OpenDaylight role  https://review.openstack.org/41636721:41
kfox1111so, in your drawing, I would change "physical interfaces" to be "bridges".21:41
rwellumit's just dhcp21:42
rwellumok good point!21:42
*** jascott1 has quit IRC21:42
kfox1111then on the right side of bridges, on the top, have eno1(10.240.43.234)21:42
kfox1111and on the right side of vibr0, have a nat box that points up to the eno1?21:42
kfox1111well... technically the ip's probably on br1 and shoudl point there...21:43
kfox1111ok. dhcp network you don't control....21:43
kfox1111so your only choice is to hide the openstack behind a nat.21:43
kfox1111the vibr0 should be good enough for that.21:43
kfox1111are there going to be multiple machines in this setup? or this kind of a testing laptop kind of thing?21:44
rwellumprntscr.com/f0epg021:44
rwellumRight now it's just one vm. But I'd like to add multiple VM's if possible - to test out multi-node21:45
kfox1111same or different boxes though?21:45
*** mgoddard has quit IRC21:45
rwellumSame box I think,21:45
kfox1111k.21:45
sbezverkkfox1111: I hit interesting issue.. I had to shut down my cluster for the weekend, when I brought it back up there was no connectivity. I suspect it happened because of flannel leases got expired21:45
kfox1111then I'd skip bringing the br1 bridge into the vm at all.21:46
kfox1111sbezverk: entirely possible. did you shut down evereything, or just some of the controllers?21:46
sbezverkso when compute node came up it got new range and some how was incapable to reconverge properly21:46
sbezverkkfox1111: everything21:46
sbezverkshutdown and powerdown21:47
rwellumThat's a good point kfox1111 - I'll think about the br1 a bit more.21:47
kfox1111sbezverk: hmm... I would have thought the init container would rerun and reip with the changes.21:47
sbezverkkfox1111: I had even some native centos containers I use to test connectivity, even these could not ping outside21:47
rwellumsbezverk: when you get a chance can you look at http://prntscr.com/f0epg0 and tell me if it's how or similar to how you set up your VM's and what you do differently?21:47
kfox1111rwellum: yeah. so vm (192.168.122.N) -> vibr0 (192.168.122.1) -> nat -> eno1 (10.240.43.234)21:48
*** Pavo has joined #openstack-kolla21:48
kfox1111sbezverk: its entirely possible we somehow didn't handle changing an ip somewhere. we don't test it anywhere.21:48
*** manheim has quit IRC21:49
*** salv-orl_ has joined #openstack-kolla21:49
kfox1111sbezverk: once the reboot gate is more done, that really should go in there. shutdown the node. change the flannel space, then bring it back up.21:49
sbezverkkfox1111: I am positive it not our problem, it is kube/networking problem21:49
sbezverkrwellum: there is a mistake in the slide21:49
sbezverkfor cinder volume21:49
rwellumgot it kfox1111  - thanks!21:49
kfox1111sbezverk: k8s does handle the flannel range changing during reboot. I have tested that for sure on another system.21:49
sbezverkwe need to specify IP address of the host where that volume group is hosted21:50
sbezverkkfox1111: I tested that too, but whith extended time offline, never before21:50
kfox1111rwellum: I'd aslo tweak the libvirt vibr0 default network to reserve half the addresses. then hand those to neutron.21:50
kfox1111sbezverk: yeah. I did run into that exact case and was able to recover from it. at least on kube 1.4.21:51
*** rwallner has quit IRC21:51
kfox1111what I think there is a problem with though, is if flannel is self hosted, the compute node doesn't go down, but flannel dies for a significant amount of time, then relaunches.21:51
*** rwallner has joined #openstack-kolla21:51
kfox1111which is maybe possible with an extended controller only situation.21:51
*** salv-orlando has quit IRC21:51
kfox1111havent tested that case.21:51
kfox1111I thin kthe node has to be rebooted to recover from that case, as the ip's might stickaround.21:52
*** manheim has joined #openstack-kolla21:52
*** goldyfruit has quit IRC21:52
kfox1111in your case though, not sure why it would be a problem. we would need to dig into it more.21:52
kfox1111I don't know of anything specifically that would cause a problem.21:52
*** gfidente|afk has quit IRC21:52
sbezverkkfox1111: possibly, but all these just shows how fragile k8s networking is atm..21:53
kfox1111hmm.... possibly.21:53
kfox1111one of the things I've been starting to do,21:53
kfox1111is switch flannel to get its ip ranges from k8s.21:54
sbezverkI was told to try contiv, some people are saying it is more robust.. I guess after the summit I will give it a try21:54
kfox1111it runs etcd'less then and I think is emune to that issue.21:54
kfox1111If k8s is handing out ranges and registering them with the node object, it shoudl be static for the nodes life.21:54
sbezverkkfox1111: right if we can eliminate a posibility of stale state, it should be more robust21:54
sbezverkkfox1111: right, this whole idea of switching subnets over time gives me chills21:55
kfox1111sbezverk: this is how that is done: https://github.com/kfox1111/charts/tree/flannel/stable/flannel21:56
kfox1111kubeadm init needs to be tweaked like: https://review.openstack.org/#/c/456964/8/tools/setup_kubernetes.sh21:57
sbezverkkfox1111: in your case you will not use etcd?21:57
kfox1111so it knows what range to hand out.21:57
kfox1111yeah. there already is an etcd cluster there, in k8s.21:57
kfox1111it uses that instead of launching a new one.21:57
*** chas has joined #openstack-kolla21:58
sbezverkkfox1111: how do you inform compute nodes which subnet to use for pods?21:58
kfox1111they ask the apiserver for a range.21:58
kfox1111the extra flag to kubeadm init tels the apiserver which range to use.21:58
sbezverkkfox1111: and api server always gives back the same range?21:59
kfox1111flannel just asks the kubelet then for it.21:59
kfox1111that, I don't know for sure.21:59
sbezverkto the same compute node?21:59
kfox1111I would expect so though.21:59
kfox1111no reason to reclaim a range when you think the node will come back.21:59
kfox1111as k8s registers nodes.21:59
sbezverkkfox1111: I am not sure but I think it is not always the case and potentially subnet could change22:00
kfox1111flannel with direct etcd, etcd has no idea when a flannel will come and go, so has to reclaim.22:00
kfox1111only way to know for sure would be to read code/test/talk to a dev.22:01
sbezverkkfox1111: right22:01
*** chas has quit IRC22:02
*** jmccarthy has left #openstack-kolla22:02
*** rwallner has quit IRC22:04
kfox1111infuriating...22:04
kfox1111some super minor change in logging is breaking the ability to put a fip on a vm...22:05
*** rwallner has joined #openstack-kolla22:05
sbezverkkfox1111: should not be any relation?!?22:05
sbezverkkfox1111: unless in the code they relay on some logging and since it is no longer there, breaks22:06
kfox1111yeah. I then went in and turned it back off on neutron/nova/libvirt.22:06
kfox1111still failing...22:06
kfox1111very weird.22:06
kfox1111trying to figure this one out almost a week. :/22:07
kfox1111gonna just have to bisect the ps.... :/22:08
*** Pavo has quit IRC22:09
sbezverkkfox1111: do you see the same issue locally?22:09
kfox1111haven't tested locally yet.22:10
sbezverkkfox1111: some issues are easier to debug locally, connected to the containers..22:11
kfox1111going to strip it down to the bare bones, then will add patch chunks in half at a time.22:12
kfox1111btw...22:12
kfox1111found this yesterday. very cool:22:12
kfox1111https://grafana.com/dashboards/31522:13
kfox1111I just launched a prometheus/grafana via helm charts, then pulled that in. :)22:13
kfox1111https://grafana.com/dashboards/917 works really well too. :)22:13
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: Docker Logging Support  https://review.openstack.org/45864922:15
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: WIP: test1  https://review.openstack.org/45946322:16
*** rwallner has quit IRC22:17
*** rwallner has joined #openstack-kolla22:17
*** trozet has quit IRC22:18
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: WIP: t2  https://review.openstack.org/45946422:19
sbezverkkfox1111: looks nice..22:20
*** trozet has joined #openstack-kolla22:20
kfox1111yeah. really digging prometheus.22:20
kfox1111they way it works makes it really easy to add extra random data too it, like power metring from random, old, unsupported metered pdu's, or networked thermostats. :)22:21
kfox1111all it takes a lot of times is a simple bash script to glue it together (or whatever else your scripting language of choice is at the time)22:24
*** rwallner has quit IRC22:30
*** rwallner has joined #openstack-kolla22:30
rwellumsbezverk: what's the mistake in the slide? Or did kfox1111 cover it?22:31
sbezverkrwellum: for cinder volume the syntax is - 'IP address': 'volume group name'22:32
sbezverkon the slide you have - 'interface': 'volume group name'22:33
sbezverkwhich is incorrect22:33
*** schwicht has quit IRC22:36
rwellumYou are correct - thanks!22:39
*** manheim has quit IRC22:39
*** manheim has joined #openstack-kolla22:40
*** trozet has quit IRC22:42
*** rwallner has quit IRC22:43
*** rwallner has joined #openstack-kolla22:43
*** goldyfruit has joined #openstack-kolla22:46
*** adrian_otto has quit IRC22:52
*** rwallner has quit IRC22:56
*** rwallner has joined #openstack-kolla22:56
rwellumOtherwise sbezverk - is the picture correct, especially the two NICS I am using?22:57
sbezverkI use two nics, in some special cases more, but normally 2 is sufficient22:58
*** manheim has quit IRC23:00
kfox1111rwellum: still think you only need the one.23:01
kfox1111hmm.. so t1 passed, and t2 failed... its there somehwere...23:02
rwellumThat would make it easier, but how - the guide is pretty clear that you need two?23:03
rwellumAs in neutron needs a dedicated interface, and then you need a mgmt23:04
inc0yay for novaclient23:04
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: WIP: test1  https://review.openstack.org/45946323:04
inc0fresh install of openstack client can't add rules to secgroup23:05
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: WIP: test2  https://review.openstack.org/45946423:07
kfox1111rwellum: depends on connectivity and if you want to run ovs in a container or not.23:09
kfox1111the public network shoudl be bridged to virbr0.23:10
*** salv-orl_ has quit IRC23:10
*** rwallner has quit IRC23:10
kfox1111that could be done with ovs if its running on the host, or via a linux bridge without.23:10
*** rwallner has joined #openstack-kolla23:11
rwellumSo ovs in a container - need the bridge.23:12
*** schwicht has joined #openstack-kolla23:12
kfox1111need an extra bridge.23:12
*** vhosakot has quit IRC23:13
rwellumHmm so same considerations bare-metal right?23:13
kfox1111well, not exactly.23:14
kfox1111this is under the constraints that: 1, you have a corperate network, that you don't want (or cant have) the vms directly on,23:14
*** jascott1 has joined #openstack-kolla23:14
kfox1111and 2, you will only have 1 physical machine involved.23:14
kfox1111so can be simpler then other setups.23:15
*** eanylin has quit IRC23:16
rwellumOk right - that makes sense - I see the point you made earlier about would multiple vm's be on the same server or not.23:16
jascott1rwellum did you see this as well? might be helpful idk https://etherpad.openstack.org/p/knet23:16
inc0ok I'm done with installation of openstack23:17
jascott1woot23:17
inc0took longer than I hoped23:17
inc0tomorrow - vpn server23:17
rwellumHmm jascott1 it's got some good info and some out of date :)23:18
jascott1feel free to add any points23:18
rwellumI'll check it's lined up with my slide, thanks for that.23:18
*** schwicht has quit IRC23:19
rwellumwill do23:19
*** schwicht has joined #openstack-kolla23:21
*** hyakuhei has quit IRC23:22
jascott1rwellum yeah I went thru the guide and took extracted all the references to network stuff so can grok it23:22
jascott1I think some example setups would go a long way23:22
jascott1like diagrams for AIO and also a simple cluster23:22
kfox1111+1.23:23
kfox1111been meaning to work on some reference architecture diagrams.23:23
kfox1111just havent had time. :/23:23
jascott1maybe between me and rwellum we can take care of it23:23
rwellumWell - I like the picture view - because I've made mistake after mistake using sed etc to try the grok.23:24
kfox1111cool.23:24
*** rwallner has quit IRC23:24
*** schwicht has quit IRC23:24
rwellumAgree I'll help jascott123:24
*** rwallner has joined #openstack-kolla23:24
jascott1awesome23:25
*** hyakuhei has joined #openstack-kolla23:28
*** dave-mccowan has quit IRC23:30
*** dixiaoli has quit IRC23:35
*** eaguilar has joined #openstack-kolla23:36
*** rwallner has quit IRC23:37
*** rwallner has joined #openstack-kolla23:38
kfox1111I don't have too much time before the summit, but do ping me if you would like some review.23:39
rwellumI'll be at the summit so no escape kfox1111 :)23:40
kfox1111hehe. fairly warned I be, says I. :)23:40
*** jrobinson has joined #openstack-kolla23:42
*** tonanhngo has quit IRC23:43
*** jrobinson has left #openstack-kolla23:44
srwilkerso/23:45
jascott1what up srwilkers!23:46
srwilkersnot much dude. how are you?23:46
jascott1good thanks23:47
jascott1yourself?23:47
srwilkersnot too shabby. just hacking away23:49
spsuryamorning alll23:50
jascott1yeah im trying to understand this thing called "networking"23:50
*** iniazi has quit IRC23:50
jascott1which I used to think I understood in the long long ago23:50
*** rwallner has quit IRC23:50
*** iniazi has joined #openstack-kolla23:51
*** rwallner has joined #openstack-kolla23:51
rwellumOverloaded nouns jascott1 - os seems to love that. And naming the same thing differently.23:51
spsuryasup jascott123:51
jascott1yeah then sprinkle in nomenclature from kubernetes for extra fun23:52
spsuryasup rwellum23:52
jascott1hey spsurya!23:52
rwellumexactly jascott123:52
rwellumGood evening spsurya :)23:52
*** felixtsang has quit IRC23:54
*** imcsk8 has quit IRC23:54
*** tonanhngo has joined #openstack-kolla23:55
jascott1rwellum what do you use for diagrams?23:56
rwellumOh no terrible question...23:56
jascott1haha23:56
rwellumpowerpoint :(23:56
* jascott1 averts eye contact23:57
rwellumSomeone told me about draw.io?23:57
jascott1:)23:57
rwellumI know right... Now no-one will talk to me.23:57
jascott1I use plant UML when I can23:57
jascott1but not suitable for everything23:58
*** chas has joined #openstack-kolla23:58
jascott1it has a new type 'deployment' that might work for a network diagram23:58
rwellumThat does look interesting - book marked.23:59
jascott1oh its definitely great for sequence diagrams23:59
jascott1others vary but it rules for those23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!