Thursday, 2016-05-05

*** barra204 has quit IRC00:13
*** MarMat has quit IRC00:15
*** sacharya has joined #openstack-kolla00:34
*** sacharya has quit IRC00:39
*** Bucanero has joined #openstack-kolla00:42
*** ccesario___ has joined #openstack-kolla00:48
*** banix has joined #openstack-kolla00:53
*** absubram has joined #openstack-kolla00:57
*** Jeffrey4l has joined #openstack-kolla01:07
*** ozialien10 has quit IRC01:12
*** absubram has quit IRC01:12
*** mummer has quit IRC01:15
*** daneyon has joined #openstack-kolla01:19
*** sdake has joined #openstack-kolla01:22
*** unicell1 has quit IRC01:25
*** daneyon_ has joined #openstack-kolla01:25
*** unicell has joined #openstack-kolla01:25
*** unicell1 has joined #openstack-kolla01:26
*** unicell has quit IRC01:26
*** daneyon has quit IRC01:28
*** dims has quit IRC01:30
*** ravig has joined #openstack-kolla01:31
*** dims has joined #openstack-kolla01:35
*** ravig has quit IRC01:36
*** daneyon_ has quit IRC01:36
*** vhosakot has joined #openstack-kolla01:40
*** ravig has joined #openstack-kolla01:44
*** ravig has quit IRC01:45
*** ravig has joined #openstack-kolla01:45
*** dolpher has joined #openstack-kolla01:59
*** unicell1 has quit IRC02:02
sdakefinally caught the logging talk02:05
sdakegood work folks!02:05
*** salv-orlando has joined #openstack-kolla02:08
*** sdake_ has joined #openstack-kolla02:08
*** sdake has quit IRC02:11
*** banix has quit IRC02:12
*** salv-orlando has quit IRC02:17
*** ccesario___ has quit IRC02:24
*** ccesario___ has joined #openstack-kolla02:24
*** sdake_ has quit IRC02:27
*** vhosakot has quit IRC02:29
*** salv-orlando has joined #openstack-kolla02:30
*** salv-orlando has quit IRC02:31
*** salv-orlando has joined #openstack-kolla02:31
*** vhosakot has joined #openstack-kolla02:34
*** sacharya has joined #openstack-kolla02:39
*** banix has joined #openstack-kolla02:41
*** salv-orlando has quit IRC02:45
*** banix has quit IRC02:52
*** rstarmer has joined #openstack-kolla03:00
*** Jeffrey4l has quit IRC03:05
*** Jeffrey4l has joined #openstack-kolla03:05
*** huikang has quit IRC03:15
*** fragatina has quit IRC03:17
*** salv-orlando has joined #openstack-kolla03:20
openstackgerritJeff Peeler proposed openstack/kolla: Generate image dependencies on CLI  https://review.openstack.org/31281603:36
*** jackning has joined #openstack-kolla03:39
jackningHi,Jeffrey Zhang,on https://review.openstack.org/#/c/306928/,do you accept our patch after you and your partners  meeting03:41
patchbotjackning: patch 306928 - kolla - Added webroot config to dashboard03:41
*** salv-orlando has quit IRC03:42
*** ravig has quit IRC03:57
*** ravig has joined #openstack-kolla03:59
openstackgerritJeff Peeler proposed openstack/kolla: Generate image dependencies on CLI  https://review.openstack.org/31281604:03
*** unicell has joined #openstack-kolla04:11
*** rstarmer has quit IRC04:15
*** sdake has joined #openstack-kolla04:18
*** unicell1 has joined #openstack-kolla04:24
*** unicell has quit IRC04:26
*** vhosakot has quit IRC04:30
*** rstarmer has joined #openstack-kolla04:30
*** fragatina has joined #openstack-kolla04:39
*** MarMat has joined #openstack-kolla04:41
*** dave-mccowan has quit IRC04:45
*** sdake has quit IRC04:47
*** sdake has joined #openstack-kolla04:48
*** reddydodda has joined #openstack-kolla04:48
*** fragatin_ has joined #openstack-kolla05:04
*** fragatina has quit IRC05:08
*** unicell has joined #openstack-kolla05:09
*** unicell1 has quit IRC05:09
*** ravig has quit IRC05:28
*** openstackgerrit has quit IRC06:03
*** openstackgerrit has joined #openstack-kolla06:03
*** fragatin_ has quit IRC06:07
*** MarMat has quit IRC06:08
*** daneyon has joined #openstack-kolla06:08
*** daneyon_ has joined #openstack-kolla06:09
*** daneyon has quit IRC06:13
*** Mr_Broken has joined #openstack-kolla06:17
*** unicell1 has joined #openstack-kolla06:23
*** unicell has quit IRC06:24
*** daneyon_ has quit IRC06:24
*** daneyon has joined #openstack-kolla06:24
*** SiRiuS has joined #openstack-kolla06:25
*** stvnoyes has joined #openstack-kolla06:27
*** sdake has quit IRC06:32
*** egonzalez has joined #openstack-kolla06:35
*** daneyon has quit IRC06:37
*** jmccarthy has joined #openstack-kolla06:37
*** jmccarthy has left #openstack-kolla06:37
*** SiRiuS has quit IRC07:00
*** reddydodda has quit IRC07:12
*** sacharya has quit IRC07:20
*** Serlex has joined #openstack-kolla07:20
*** athomas has joined #openstack-kolla07:21
*** reddydodda has joined #openstack-kolla07:29
*** sbezverk has quit IRC07:30
*** sbezverk has joined #openstack-kolla07:30
*** shardy has joined #openstack-kolla07:37
*** shardy has quit IRC07:38
*** shardy has joined #openstack-kolla07:38
*** mikelk has joined #openstack-kolla07:41
*** gfidente has joined #openstack-kolla07:49
*** gfidente has joined #openstack-kolla07:49
*** dmk0202 has joined #openstack-kolla07:57
*** rstarmer has quit IRC08:01
*** diga has joined #openstack-kolla08:01
*** dmk0202 has quit IRC08:03
*** mbound has joined #openstack-kolla08:10
*** dmk0202 has joined #openstack-kolla08:11
*** reddydodda has quit IRC08:12
*** rstarmer has joined #openstack-kolla08:19
*** sacharya has joined #openstack-kolla08:20
openstackgerritPrithiv proposed openstack/kolla: Deletes fake containers on cleanup  https://review.openstack.org/31262908:24
*** sacharya has quit IRC08:26
*** diga has quit IRC08:29
*** reddydodda has joined #openstack-kolla08:44
*** egonzalez90 has joined #openstack-kolla09:11
*** egonzalez has quit IRC09:11
*** reddydodda has quit IRC09:15
*** egonzalez90 has quit IRC09:16
openstackgerritMichal Rostecki proposed openstack/kolla: [WIP] Exclude the docker registry image from cleanup-images  https://review.openstack.org/31067709:16
*** prithiv has joined #openstack-kolla09:19
prithivHi sdake09:21
prithivany update about this ?09:21
prithivhttps://bugs.launchpad.net/kolla/+bug/155401809:21
openstackLaunchpad bug 1554018 in kolla "Encrypt passwords.yaml file with ansible vault and decrypt while deploying" [Wishlist,Triaged] - Assigned to Prithiv (prithiv)09:21
prithivi have a local code which works and it requires just few lines of changes.09:21
*** sacharya has joined #openstack-kolla09:22
*** rstarmer has quit IRC09:22
*** pbourke has joined #openstack-kolla09:23
*** sdake has joined #openstack-kolla09:24
*** ccesario___ has quit IRC09:26
*** ccesario___ has joined #openstack-kolla09:26
*** sacharya has quit IRC09:27
*** reddydodda has joined #openstack-kolla09:28
sdakemorning09:31
*** reddydodda has quit IRC09:33
sdakegoing to bed at 5pm is not too good for waking up at 2am :(09:44
*** reddydodda has joined #openstack-kolla09:45
*** egonzalez has joined #openstack-kolla09:46
nihiliferanyone else is struggling with seeing kolla stats on stackalytics? o.O09:47
nihilifersdake: morning09:48
openstackgerritSteven Dake proposed openstack/kolla-kubernetes: Test of kubernetes gate  https://review.openstack.org/31287609:48
sdakehey nihilifer09:48
openstackgerritMerged openstack/kolla: Generate image dependencies on CLI  https://review.openstack.org/31281609:48
sdakenihilifer ya it looks like kolla is not on stackalytics anymore09:48
sdakemaybe its just a temporary thing09:49
openstackgerritMerged openstack/kolla: Remove dependencies on kazoo and friends  https://review.openstack.org/31277709:49
openstackgerritMerged openstack/kolla: Add a CONTRIBUTING.rst for Kolla  https://review.openstack.org/30540209:49
*** dolpher has quit IRC09:50
sdakenihilifer typically when i seee this its just a transient bug09:50
*** jackning has quit IRC09:50
sdakenihilfer iirc someone at mirantis maintains stackalytics, perhaps yo ucould ask around to get it reapiared09:53
sdakei use stackalytics all the time to take a look at core reviewers and the like09:53
nihilifersure, will try to talk with Ilya Shakhat09:56
*** ccesario___ has quit IRC09:57
sdakenihilifer i have idea why10:01
sdake        - release:cycle-trailing10:01
sdakewe added this as our release model type10:01
sdakeperhaps the tool doesn't know how to translate that10:01
nihiliferoh, this is probably not a problem10:04
nihiliferit's a different thing - stackalytics is fetching data from scratch now10:05
nihiliferand synced ~100 projects so far :P10:05
nihiliferwe need to wait10:05
*** reddydodda has quit IRC10:05
sdakeya it happens often10:07
sdakelike once a week10:07
sdakeeveryone always gets bent out of shape about it :)10:07
sdakeI have learned to live with i t :)10:07
*** Jeffrey4l has quit IRC10:15
*** Jeffrey4l has joined #openstack-kolla10:17
*** SiRiuS has joined #openstack-kolla10:17
*** reddydodda has joined #openstack-kolla10:25
*** SiRiuS has quit IRC10:26
*** SiRiuS has joined #openstack-kolla10:29
*** sdake has quit IRC10:29
*** ccesario___ has joined #openstack-kolla10:33
*** ccesario___ has quit IRC10:38
*** reddydodda has quit IRC10:38
*** ccesario___ has joined #openstack-kolla10:38
ccesariomorning guys!10:40
*** prithiv has quit IRC10:49
*** reddydodda has joined #openstack-kolla10:58
*** prithiv has joined #openstack-kolla11:00
*** prithiv has quit IRC11:04
*** reddydodda has quit IRC11:06
*** prithiv has joined #openstack-kolla11:06
*** prithiv has quit IRC11:16
*** prithiv has joined #openstack-kolla11:21
*** rhallisey has joined #openstack-kolla11:22
*** prithiv has quit IRC11:22
*** mlima has joined #openstack-kolla11:25
openstackgerritDavanum Srinivas (dims) proposed openstack/kolla-kubernetes: [WIP] testing jobs  https://review.openstack.org/31290111:25
openstackgerritRyan Hallisey proposed openstack/kolla: Spec: Deploy Kolla images using Kubernetes  https://review.openstack.org/30418211:26
rhalliseywe need to merge the spec asap since the repo is done11:26
rhalliseyneed review :) ^11:27
*** diga has joined #openstack-kolla11:39
*** dave-mccowan has joined #openstack-kolla11:53
*** dave-mcc_ has joined #openstack-kolla11:56
*** diga has quit IRC11:56
*** dave-mccowan has quit IRC11:58
*** Bucanero_ has joined #openstack-kolla12:00
*** Bucanero has quit IRC12:00
*** prithiv has joined #openstack-kolla12:04
*** SiRiuS has quit IRC12:07
*** Mr_Broken has quit IRC12:10
*** Mr_Broken has joined #openstack-kolla12:11
*** Mr_Broken has quit IRC12:15
mlimarhallisey, the tests in kolla-k8s will be only the gate?12:18
rhalliseymlima, which tests? The gate should be tiny right now.12:21
rhalliseynothing to really gate on12:22
mlimaunit tests, functional tests rhallisey12:22
*** Mr_Broken has joined #openstack-kolla12:23
rhalliseymlima, I'm not super concerned about it.  Will leave it up to the cores at first12:24
rhalliseythought I added some unit tests.  Maybe I didn't12:26
*** prithiv has quit IRC12:27
*** prithiv has joined #openstack-kolla12:33
*** prithiv has quit IRC12:34
*** dougs has joined #openstack-kolla12:35
*** prithiv has joined #openstack-kolla12:36
*** prithiv has quit IRC12:37
*** prithiv has joined #openstack-kolla12:39
mlimarhallisey, ok :)12:40
*** SiRiuS__ has joined #openstack-kolla12:40
*** egonzalez has quit IRC12:41
*** Serlex has quit IRC12:42
sbezverkrhallisey I am building ansible playbook to build 5 node kubernetes cluster on 5 vms for development testing would be happy to share when it is completed12:48
rhalliseysbezverk, we need to start with the services12:48
*** vhosakot has joined #openstack-kolla12:49
rhalliseythen we can orchestrate12:49
rhalliseysbezverk, sounds awesome though12:49
*** reddydodda has joined #openstack-kolla12:50
*** banix has joined #openstack-kolla12:50
*** Serlex has joined #openstack-kolla12:52
*** dwalsh has joined #openstack-kolla12:54
*** prithiv has quit IRC12:57
*** mbound has quit IRC12:59
*** prithiv has joined #openstack-kolla12:59
*** rstarmer has joined #openstack-kolla13:00
*** rstarmer has quit IRC13:00
*** jtriley has joined #openstack-kolla13:03
*** Bucanero_ has quit IRC13:12
*** Bucanero has joined #openstack-kolla13:13
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Keystone Service and Pod  https://review.openstack.org/31294413:22
*** sacharya has joined #openstack-kolla13:23
rhallisey^ just ignore.  Need the spec to complete first13:24
*** mbound has joined #openstack-kolla13:28
*** sacharya has quit IRC13:28
*** egonzalez has joined #openstack-kolla13:29
mag009_morning guys13:34
rhalliseymag009_, morning13:36
*** Bucanero_ has joined #openstack-kolla13:37
*** Bucanero has quit IRC13:40
openstackgerritMauricio Lima proposed openstack/kolla: Deleting only images from kolla build  https://review.openstack.org/28682313:40
*** Bucanero has joined #openstack-kolla13:41
*** inc0 has joined #openstack-kolla13:41
inc0good morning13:42
rhalliseyinc0, yo13:43
rhalliseyneed review on the kube spec13:43
mag009_guys, (I'm working with lyncos btw) do you have a template of how I should integrate a plugin for neutron alias calico13:43
mag009_:)13:43
inc0rhallisey, qui pro quo, I'll review kube, you'll test liberty. Deal?;)13:44
inc0mag009_, soo13:44
rhalliseyinc0, sure13:44
*** Bucanero_ has quit IRC13:45
*** Bucaner86 has joined #openstack-kolla13:45
inc0we have plugin support high on our priority list, but right now we don't have fully functional thing, that being said with a bit of hacking you should be able to get anything you want13:45
rhalliseyinc0, I can do the first shot at plugins support13:45
inc0https://github.com/openstack/kolla/blob/master/doc/image-building.rst#plugin-functionality thats one13:45
*** fragatina has joined #openstack-kolla13:45
inc0rhallisey, sure, I told that I'll do it but if you want:)13:46
inc0I'll focus on ansible 2 then13:46
rhalliseyinc0, ya do ansible 213:46
inc0mag009_, link above13:46
rhalliseyI'll do plugins13:46
mag009_yep perfect13:46
mag009_thx13:46
inc0if that won't be sufficient you can add custom dockerfile stuff by passing kolla-build --include-footer custom_dockerfile13:47
inc0you can add few lines for calico support there and build container with it13:47
inc03rd thing is custom config13:47
*** Bucanero has quit IRC13:47
*** openstackgerrit has quit IRC13:47
mag009_yep well what I did is a new folder under docker/neutron/calico-dhcp-agent13:47
*** openstackgerrit has joined #openstack-kolla13:47
mag009_and put the package in it to install13:48
*** Bucanero has joined #openstack-kolla13:48
inc0that's one way, but methods above might allow you to do it without changing anything in kolla code13:48
inc0well, whichever works best13:49
openstackgerritMerged openstack/kolla: Remove dependencies on kazoo and friends  https://review.openstack.org/31277613:51
*** Bucanero_ has joined #openstack-kolla13:51
*** Bucaner86 has quit IRC13:51
*** vhosakot has quit IRC13:51
*** reddydodda has quit IRC13:53
*** athomas has quit IRC13:53
*** Bucanero has quit IRC13:55
*** athomas has joined #openstack-kolla14:00
mag009_you guys dont support multi plugin?14:07
mag009_like lets say I want openvswitch + clico14:07
mag009_calico*14:07
mag009_cause if I look at the cisco example it would replace the ml2_conf.ini14:08
mag009_with cisco driver only14:08
mag009_I want to be able to deploy ironic thats why I'm asking14:08
inc0mag009_, well14:11
inc0not really14:11
inc0we'll need to figure it out14:11
inc0you'd like to have different ml2_conf on set of nodes?14:11
*** prithiv has quit IRC14:13
mag009_well I'd like to have my neutron-server(s) to use the mechanism_drivers = openvswitch,l2population,calico14:13
mag009_:)14:13
mag009_for example14:13
mag009_I can make it work... I was wondering if I was missing anything ;)14:14
inc0we had this discussion on summit14:14
mag009_thank god it's in ansible not puppet!14:14
mag009_:)14:14
inc0tho it's still not there yet14:14
inc0haha14:14
inc0in any case, once you will make it work, do share what you did plz14:14
inc0so we can support that in the future14:15
mag009_always14:15
mag009_you have no idea how many time we've changed the way we deploy openstack14:15
inc0I used to deploy OS myself, I may have slighest14:15
mag009_so you guys are working for Redhat?14:17
*** absubram has joined #openstack-kolla14:18
*** MarMat has joined #openstack-kolla14:19
*** reddydodda has joined #openstack-kolla14:19
inc0nope, Intel14:20
*** ravig has joined #openstack-kolla14:20
inc0we  have few RH people14:20
*** prithiv has joined #openstack-kolla14:20
inc0btw where do you guys work?;)14:21
*** Bucanero_ has quit IRC14:23
mag009_gameloft14:23
*** Bucanero has joined #openstack-kolla14:23
openstackgerritMichal Rostecki proposed openstack/kolla: Exclude the docker registry image from cleanup-images  https://review.openstack.org/31067714:25
*** Bucanero has quit IRC14:28
*** Mr_Broken has quit IRC14:29
*** Mr_Broken has joined #openstack-kolla14:30
*** Bucanero has joined #openstack-kolla14:32
sbezverkrhallisey ping14:32
rhalliseysbezverk, hey14:33
sbezverkrhallisey listen, do you know by any chance where kubelet get enabled for host networking, I am trying to bring up your keystone pod and it is failing because kubelet does not allow host networking.. could not find anything usefull on the internet14:34
*** Mr_Broken has quit IRC14:34
openstackgerritMichal Rostecki proposed openstack/kolla: Exclude the docker registry image from cleanup-images  https://review.openstack.org/31067714:34
sbezverkrhallisey I have allow privileged key but it  does not seem to be enough14:34
rhalliseysbezverk, what version are you using?14:35
rhalliseythat's odd14:35
*** NachoDuck has joined #openstack-kolla14:36
sbezverkrhallisey check this out http://paste.openstack.org/show/496235/14:36
sbezverkrhallisey I am using 1.1 on workers, will try to move to 1.214:37
rhalliseyI didn't know you had to enable it. Weird14:37
rhalliseysbezverk, supposedly the default in 1.1 should allow this14:38
LyncosAbout kolla-k8s ... I got questions14:38
*** Bucanero has quit IRC14:38
LyncosAre you guys going to deploy the k8s cluster ? or we will need a working cluster in order to deploy kolla on k8s ?14:39
rhalliseysbezverk, there's a flag in the kublet to run with host-netorking14:39
rhallisey--host-network-sources="*"14:39
rhalliseyLyncos, it's likely there will be some orchestration, though it hasn't been specified yet14:40
Lyncosok14:40
Lyncosthanks for the info14:40
rhalliseyLyncos, sure14:41
rhalliseyLyncos, https://github.com/rthallisey/kolla-kubernetes-personal/blob/master/doc/kube-allinone-setup.rst14:41
sbezverkrhallisey ok I will add it and re-try14:41
rhalliseyI'll likely port that over14:41
rhalliseysbezverk, you can also try hypercube14:41
Lyncosrhallisey: you will adapt it to multi node I guess14:41
rhallisey^^^ in the doc there14:41
Lyncosinteresting14:42
rhalliseyLyncos, that's where the orchestration will come in14:42
Lyncosok14:42
*** Bucanero has joined #openstack-kolla14:45
inc0I'm thinking of writing ansible role for k8s to deploy it along with ceph14:46
inc0kolla-ansible-k8s should be simple enough;)14:47
Lyncosinc0 are you going to deploy the same ceph for k8s than for OS14:47
Lyncos?14:47
inc0likely14:47
inc0no reason really to have separated14:47
inc0you can have separated stor pools14:47
Lyncosyeah14:48
*** MarMat has quit IRC14:48
LyncosHere we tend to make micro ceph clusters (not my choice)14:48
inc0and k8s doesn't really handle storage stuff well14:48
*** MarMat has joined #openstack-kolla14:48
Lyncosand did you guys plan a migration workflow for people that deployed with kolla-ansible to go with kolla-k8s ? or we will have to rebuild everything ?14:49
rhalliseyLyncos, there's definitely some potential for that.  It will have to be a blueprint we bring up down the road14:50
*** sdake has joined #openstack-kolla14:50
LyncosI just wanted to know because right now we are deploying and we can't really wait for k8s to be ready14:50
inc0we don't want to stop development of k8s by putting requirement of migration, however there is always a way14:50
Lyncosand we may want to switch to k8s later14:50
inc0Lyncos, sure, we'll figure it out14:51
*** absubram has quit IRC14:51
Lyncosok np14:51
inc0worst case scenerio, manual labor + live migration14:51
Lyncosyeah14:51
inc0but I hope it won't be the case14:51
sbezverkrhallisey it is better now, with new version it does not require any config for host netwokring14:52
rhalliseysbezverk, gotcha14:52
rhalliseywhich version just to take note of it?14:52
sbezverkthe container started but flapping due to missing config files, so it is expected..14:52
rhalliseygotcha14:53
*** ayoung has quit IRC14:59
*** Bucanero has quit IRC15:00
nihilifermag009_: we don't support custom neutron plugins atm15:02
*** mbound has quit IRC15:03
*** prithiv has quit IRC15:03
*** SiRiuS__ has quit IRC15:04
*** absubram has joined #openstack-kolla15:05
*** prithiv has joined #openstack-kolla15:05
*** prithiv has quit IRC15:06
*** Bucanero has joined #openstack-kolla15:06
openstackgerritMerged openstack/kolla: Deletes fake containers on cleanup  https://review.openstack.org/31262915:07
*** daneyon has joined #openstack-kolla15:09
*** mbound has joined #openstack-kolla15:09
pbourkenihilifer: http://docs.openstack.org/developer/kolla/image-building.html#plugin-functionality15:09
*** Serlex has quit IRC15:11
*** SiRiuS has joined #openstack-kolla15:12
*** ravig has quit IRC15:12
*** vhosakot has joined #openstack-kolla15:14
*** geseib007 has joined #openstack-kolla15:22
*** Bucanero has quit IRC15:22
*** Bucanero has joined #openstack-kolla15:23
*** sacharya has joined #openstack-kolla15:24
*** mummer has joined #openstack-kolla15:24
*** geseib has quit IRC15:25
*** absubram has quit IRC15:26
*** Bucanero_ has joined #openstack-kolla15:27
*** Bucanero has quit IRC15:27
*** absubram has joined #openstack-kolla15:27
*** sacharya has quit IRC15:29
*** Bucanero_ has quit IRC15:31
*** Bucanero has joined #openstack-kolla15:32
*** ayoung has joined #openstack-kolla15:32
openstackgerritMauricio Lima proposed openstack/kolla: Diagnostics for Environment  https://review.openstack.org/30660815:33
LyncosIs tempest working right now ?15:34
*** dmk0202 has quit IRC15:38
*** Bucanero has quit IRC15:41
vhosakotLyncos: you mean tempest in kolla ?15:42
Lyncosyeah .. I just end up on launchpad... I think it will be implemented in newton15:42
*** ravig has joined #openstack-kolla15:43
openstackgerritMauricio Lima proposed openstack/kolla: Deleting only images from kolla build  https://review.openstack.org/28682315:44
vhosakotyes, we're adding tempest tests for each service in Newton... which should strengthen the gate15:44
*** mikelk has quit IRC15:46
*** ayoung has quit IRC15:46
*** neilus has quit IRC15:48
*** salv-orlando has joined #openstack-kolla15:49
*** egonzalez90 has joined #openstack-kolla15:51
*** egonzalez90 has quit IRC15:51
*** egonzalez has quit IRC15:51
*** egonzalez has joined #openstack-kolla15:52
*** egonzalez90 has joined #openstack-kolla15:53
pbourkedoes anyone have a working ceph running atm?15:54
*** signed8bit has joined #openstack-kolla15:54
mag009_yep we do15:54
Lyncosin fact we have 2 :-)15:54
mag009_the openstack one is deployed with kolla15:54
pbourkemag009_: would you mind pasting me the output of blkid on one of your storage nodes?15:54
pbourkemag009_: Im having trouble getting kolla to find my labelled disk to provision as an osd, it seemed to be working fine yesterday15:55
*** ravig has quit IRC15:55
*** egonzalez has quit IRC15:56
mag009_well you need to set your device name to : KOLLA_CEPH_OSD_BOOTSTRAP15:56
pbourkeyes I've done that15:57
pbourkebut find_disks.py isn't finding it15:57
pbourkei had this issue before in swift and can't remember what the issue was15:57
mag009_once it's deployed I'm getting this :15:57
mag009_/dev/sdb1: UUID="a73e41b6-ccdc-4985-bc72-75346e6fbbf2" TYPE="xfs" PARTLABEL="KOLLA_CEPH_DATA_0" PARTUUID="45ca107b-c260-4c63-8bb2-8aa75edb20cd"15:57
mag009_/dev/sdb2: PARTLABEL="KOLLA_CEPH_DATA_0_J" PARTUUID="88e21c8a-7825-4d3a-acb7-d9dfcd147710"15:57
*** ravig has joined #openstack-kolla15:58
mag009_here's what I'm doing before running kolla deploy15:58
inc0pbourke, how it's not working?;)15:58
inc0I have working ceph15:58
mag009_parted -s {{item.0}} mklabel gpt && parted -s {{item.0}} mkpart xfs 1 1200GB && parted -s {{item.0}} name 1 KOLLA_CEPH_OSD_BOOTSTRAP15:58
mag009_this is how I prepare my device15:58
pbourkeinc0: i had it working yesterday :/ for some reason pyudev is only finding the following attrs on the block device: [u'DEVNAME', u'DEVPATH', u'DEVTYPE', u'MAJOR', u'MINOR', u'SUBSYSTEM']15:59
pbourkeso ID_PART_ENTRY_NAME / ID_FS_LABEL come up empty15:59
inc0duh, let me test it15:59
inc0(rebuilding liberty now, will deploy in a moment15:59
inc0)15:59
pbourkeinc0: you most likely wont have the problem :) its something on my end, just not sure what16:00
inc0pbourke, remember that you need to rebuild partition table each time, so re-KOLLA_BOOTSTRAP devs16:01
sdakeakwasnie inc0 elemoine- saw your presentation on logs via video, you all did a bang up job16:01
sdakeepecially with 3 presenters which is difficult16:01
inc0good to hear that:)16:02
sdakepbourke you aslo need to rm -rf /etc/*ceph* each time16:02
inc0kudos to akwasnie, she did most of the job16:02
sdakeand remove ceph mountpoint from /etc/ftab and reboot16:02
pbourkesdake: ack16:02
inc0cleanup host does that16:03
gmmahainc0: vhosakot: have you used vagrant VMs to be your deployment host for kolla ever?16:04
vhosakotgmmaha: no..16:05
vhosakotgmmaha: I deploy manually16:05
gmmahai am trying to use that and stable/mitaka branch for building a VM and the build gets stuck half way..16:05
gmmahavhosakot: aah ok16:05
gmmahanothing in the dmesg, logs seems fine other than some spewing about [ 3445.429847] aufs au_opts_verify:1597:docker[10155]: dirperm1 breaks the protection by the permission bits on the lower branch16:05
inc0gmmaha, negative16:06
* gmmaha wonders if the build server needs to umph w.r.t RAM and CPU16:06
gmmahainc0: aah ok.. thanks16:06
pbourkeinc0: while you're there...16:11
inc0yeah?16:12
pbourkeinc0: is this correct? https://github.com/openstack/kolla/blob/master/ansible/inventory/multinode#L3716:12
pbourkeshould haproxy not be on the control node16:12
inc0not really16:12
inc0reason is network nodes has to be connected to external network16:12
pbourkehow would this be working for anyone?16:12
*** ravig has quit IRC16:12
inc0with haproxy there there is no reason for connecting controllers to external network16:13
inc0it's working becasue haproxy have endpoints for both internal and external vip16:13
inc0and all the ips for controllers16:13
pbourkeso keepalived still goes on the controllers16:13
inc0no16:14
inc0keepalived goes on hosts where haproxy is16:14
inc0which is network node16:14
pbourkethat's what I thought. but the network node is not running any services16:14
inc0there is no reason for controllers to have floating ip16:14
inc0well it is running neutron agents16:15
*** ravig has joined #openstack-kolla16:15
openstackgerritMauricio Lima proposed openstack/kolla: Deleting only images from kolla build  https://review.openstack.org/28682316:15
*** mark-casey has joined #openstack-kolla16:15
inc0keepalived gives ip, haproxy binds to this ip:port per service16:15
inc0and everyone hits haproxy, which then load balances it among controllers16:16
sbezverkinc0 I noticed that is haproxy is not running on controller node then mariadb fails to use VIP16:16
inc0sbezverk, that shouldn't be the case16:16
inc0as maria should ping haproxy and haproxy forward it back to maria16:16
inc0seems like a bug16:16
sbezverkinc0 well we hit bunch of issue before when people tried to use multinode16:16
pbourkeinc0: got you16:16
mark-caseyspeak of the devil guys I was just coming in to talk about haproxy<->maria16:17
inc0lol16:17
pbourkei came across some strangeness in this area last week, but then got distracted16:17
sbezverkinc0 the solution was to make sure keepalived and haproxy runs on controllers as well16:17
* inc0 is hiding behind whatever and braces for flying rotten tomatoes16:17
pbourkeha16:18
inc0sbezverk, we need to debug this, this shouldn't be the case16:18
pbourkesbezverk: was it you by any chance I was helping with swift partition labelling before16:18
sbezverkpbourke yep16:19
pbourkesbezverk: iirc the issue was find_disks.py could not find the labelled partition. can you remember what the fix was? im seeing it again in ceph16:19
*** unicell1 has quit IRC16:19
sbezverkpbourke right, in my case the issue was that I was using LVM for swift disks and you cannot name partition as required, so I had to use prefix instead of strict to match on file system label16:21
pbourkehmm I dont think that's it in this case16:22
*** ccesario___ has quit IRC16:23
*** salv-orlando has quit IRC16:23
*** prithiv has joined #openstack-kolla16:23
sbezverkpbourke when I tried the same on loopback devices it seemed to work as loopback allowed create and name partition..16:23
pbourkesbezverk: thanks16:28
*** sdake_ has joined #openstack-kolla16:28
sbezverkinc0 I am not sure it is a bug :-) I thought it was just a typo in inventory..16:29
pbourkesame16:29
pbourkebut when inc0 explains it, it *should* work16:29
pbourkei'll test again16:29
pbourkeanother issue Im having is where sometimes the monitors refuse to form a quorum. ceph status just hangs forever16:30
pbourkedunno if any of you have seen that16:30
inc0brb16:30
*** inc0 has quit IRC16:30
*** daneyon has quit IRC16:30
*** inc0 has joined #openstack-kolla16:30
*** sdake has quit IRC16:31
inc0pbourke, remember that you need to change ceph config for aio16:32
pbourkeinc0: im doing multinode16:32
*** jtriley_ has joined #openstack-kolla16:35
*** salv-orlando has joined #openstack-kolla16:35
*** jtriley has quit IRC16:37
*** mark-casey has quit IRC16:37
*** mark-casey has joined #openstack-kolla16:38
*** sacharya has joined #openstack-kolla16:39
openstackgerritSteven Dake proposed openstack/kolla-kubernetes: Test of kubernetes gate  https://review.openstack.org/31287616:39
openstackgerritMauricio Lima proposed openstack/kolla: Diagnostics for Environment  https://review.openstack.org/30660816:40
pbourkeok I found my partition label issue at least. was missing the /dev bind mount into the toolbox in our copy16:41
pbourkeyey for not keeping upstream :/16:41
*** SiRiuS has quit IRC16:42
*** stvnoyes has left #openstack-kolla16:47
*** rajathagasthya has joined #openstack-kolla16:47
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla-kubernetes: Update for successful pep8  https://review.openstack.org/31305416:50
*** daneyon has joined #openstack-kolla16:54
coolsvapsdake_, dims ^^ green gate :)16:57
*** jtriley_ has quit IRC16:57
dims+1 coolsvap16:57
*** SiRiuS has joined #openstack-kolla16:59
*** prithiv has quit IRC16:59
*** daneyon has quit IRC17:00
*** sdake_ has quit IRC17:00
*** unicell has joined #openstack-kolla17:01
*** mbound has quit IRC17:01
*** mbound has joined #openstack-kolla17:02
*** daneyon has joined #openstack-kolla17:06
*** fragatina has quit IRC17:07
*** rajathagasthya has quit IRC17:07
*** fragatina has joined #openstack-kolla17:07
*** shardy has quit IRC17:07
*** gfidente has quit IRC17:08
*** daneyon_ has joined #openstack-kolla17:09
mark-caseyinc0: ping17:10
*** signed8bit is now known as signed8bit_Zzz17:10
*** unicell has quit IRC17:12
*** unicell has joined #openstack-kolla17:12
*** daneyon has quit IRC17:13
*** rajathagasthya has joined #openstack-kolla17:14
*** fragatina has quit IRC17:18
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla: WIP: Changes needed to deploy liberty with mitaka code  https://review.openstack.org/30839017:21
inc0mark-casey, pong17:21
*** jtriley has joined #openstack-kolla17:21
inc0pbourke, just deployed stable liberty with ceph, no issues17:22
*** athomas has quit IRC17:23
*** betheynyx has quit IRC17:24
mark-caseyinc0: is there a current method for automatically restarting failed containers?17:26
inc0restart policy: always?17:26
mark-caseyahhh. k17:26
inc0that's what you need?;)17:26
mark-caseyforgot about that. I'm looking at the galera wait time we discussed yesterday17:27
inc0I think default is never17:27
inc0ok17:27
mark-caseyon fail/recover17:27
*** mbound has quit IRC17:27
inc0btw, one more thing k8s will do better17:27
mark-caseyso currently the container comes back up and retransfers the DB I guess?17:27
inc0lights out stops being an issue17:27
openstackgerritMauricio Lima proposed openstack/kolla: Make Heka collect Ceph logs  https://review.openstack.org/31026117:27
inc0thanks mlima !17:28
*** mbound has joined #openstack-kolla17:28
mlimainc0, ??17:28
*** mbound has quit IRC17:28
inc0for ceph logs;)17:28
mlimaoh17:28
*** aginwala has joined #openstack-kolla17:28
inc0mark-casey, that would be my understanding17:29
inc0although I'm not super good at galera17:29
openstackgerritMauricio Lima proposed openstack/kolla: Diagnostics for Environment  https://review.openstack.org/30660817:29
mlimainc0, if you can please test the ceph logs17:30
*** jtriley has quit IRC17:30
mlimai tested it here and works fine17:30
inc0will do17:30
mlima:)17:31
*** signed8bit_Zzz is now known as signed8bit17:32
*** ayoung has joined #openstack-kolla17:34
mag009_+1 for ceph logs !!! :)17:35
mag009_that'd be awesome17:36
*** rajathagasthya has quit IRC17:39
*** ravig has quit IRC17:42
openstackgerritDavanum Srinivas (dims) proposed openstack/kolla-kubernetes: Port some tests from Kolla repo  https://review.openstack.org/31308117:42
*** rajathagasthya has joined #openstack-kolla17:42
*** jtriley has joined #openstack-kolla17:42
dimscoolsvap : ^^ that's the method i use in nova-docker to make sure it's in sync with nova/master17:42
dimscoolsvap : it depends on your review17:43
mark-caseyinc0: this may be able to remove waiting/lost queries during galera recovery   https://bugs.launchpad.net/kolla/+bug/157875217:46
openstackLaunchpad bug 1578752 in kolla "HAproxy does not consider wsrep_local_state when considering which db nodes/containers can accept traffic" [Undecided,New]17:46
*** ravig has joined #openstack-kolla17:46
*** unicell has quit IRC17:49
coolsvapdims, looks good seems py34 is not happy17:51
*** prithiv has joined #openstack-kolla17:56
inc0mark-casey, I'll believe you on this one;)17:58
*** prithiv has quit IRC17:59
*** Jeffrey4l has quit IRC18:01
mark-caseyheh18:04
openstackgerritDavanum Srinivas (dims) proposed openstack/kolla-kubernetes: Port some tests from Kolla repo  https://review.openstack.org/31308118:05
openstackgerritDavanum Srinivas (dims) proposed openstack/kolla-kubernetes: Support for release notes  https://review.openstack.org/31309018:05
*** aginwala has quit IRC18:07
*** jasonsb has joined #openstack-kolla18:07
*** aginwala has joined #openstack-kolla18:07
*** coolsvap has quit IRC18:08
*** salv-orlando has quit IRC18:11
*** aginwala has quit IRC18:11
*** fragatina has joined #openstack-kolla18:12
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Document a kubernetes allinone setup  https://review.openstack.org/31309518:14
rhalliseydims, ^18:14
openstackgerritDavanum Srinivas (dims) proposed openstack/kolla-kubernetes: Port some tests from Kolla repo  https://review.openstack.org/31308118:22
openstackgerritDavanum Srinivas (dims) proposed openstack/kolla-kubernetes: Support for release notes  https://review.openstack.org/31309018:22
*** unicell has joined #openstack-kolla18:22
*** inc0 has quit IRC18:22
*** unicell has quit IRC18:23
*** unicell1 has joined #openstack-kolla18:23
*** reddydodda has quit IRC18:27
dimsrhallisey : yep threw that into a script for k8sclient job a couple of days ago - https://github.com/openstack/python-k8sclient/blob/master/tools/kube-init.sh18:31
rhalliseyoh sweet18:31
*** inc0 has joined #openstack-kolla18:32
inc0ccesario, I've pushed fixed patch18:37
inc0gates for ubuntu are green18:37
inc0and spawned vm successfully18:37
inc0guys can you test out liberty patch? centos source is red18:37
ccesarioinc0, sure.... let me test it18:38
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Keystone Service and Pod  https://review.openstack.org/31294418:41
rhalliseydims, ^ try that out18:41
rhalliseythat should be more clear18:41
inc0rhallisey, if you have moment, could you test out stable/liberty patch plz?18:42
rhalliseyI'm building the images18:42
inc0thanks18:42
rhalliseyno problem :)18:42
rhalliseymy machine sucks at building docker images so it takes some time18:43
dimsrhallisey : ack will try that18:46
*** rajathagasthya has quit IRC18:46
dimsrhallisey : inc0 : we are not merging changes till spec is approved?18:47
inc0dims, spec is high level, I guess we could merge stuff that are obviously needed18:48
rhalliseydims, I'd like to wait on non-gating things, so we can just focus on the spec18:48
inc0what Ryan said;)18:48
*** sdake has joined #openstack-kolla18:48
rhalliseyinc0, what do you want me to test with liberty besides deploying18:49
dimsok, i threw a couple of reviews behind coolsvap's18:49
inc0sdake, tested liberty on ubuntu source with ceph - works like charm now18:49
rhalliseycool thanks dims18:49
inc0rhallisey, spawn a vm and so on18:49
*** geseib007 has quit IRC18:50
*** sdake_ has joined #openstack-kolla18:55
*** aginwala has joined #openstack-kolla18:55
*** sdake has quit IRC18:57
sbezverkinc0 ping18:57
v1k0d3nhey guys...18:58
v1k0d3ncan someone help me understand the kolla_internal_vip_address a little better18:58
v1k0d3n?18:58
*** dwalsh has quit IRC18:58
inc0sbezverk, what's up?18:58
inc0v1k0d3n, we have 2 vips18:58
v1k0d3nin one deployment, i picked an unused address...seemed to work. another deployment, and it does seem to work.18:58
inc01 for external network second for internal18:58
sbezverkinc0 I think I know what i wrong with haproxy but need to discuss18:59
sbezverkhave time?18:59
v1k0d3ninc0: my prechecks are failing on my internal18:59
v1k0d3nfatal: [localhost] => error while evaluating conditional: '169.254.' not in kolla_internal_vip_address and 'False' in kolla_internal_vip_address | ipaddr(ip_addr_output.stdout.split()[3])18:59
*** aginwala has quit IRC18:59
inc0sbezverk, go ahead18:59
sdake_the prechecks should allow link local addresses, although it is possible they dont19:00
sdake_vhosakot any ideas?19:00
sbezverkinc0 kolla have haproxy and keepalived running on the smae host which according to you should be networking node, but running keepalived make much more sense on hosts which you are trying to protect, controllers19:00
inc0sbezverk, not really19:01
mlimaguys, i have this erro19:01
mlimaTASK: [ceph | Bootstrapping Ceph OSDs] ****************************************19:01
mlimafatal: [localhost] => One or more undefined variables: 'dict object' has no attribute 'external_journal'19:01
inc0keepalived doesn't protect anything19:01
*** ravig has quit IRC19:01
sbezverkwell it generate virtual ip address and if controller dies which owns VIP, VIP should move to another controller19:01
inc0yeah, but that's not really protection19:01
inc0it's just ensuring that ip is somewhere and pointing to something that works19:02
*** aginwala has joined #openstack-kolla19:02
sbezverkinc0 I do not care how you want to call it, keepalived must be logically atached to whatever it belongsd19:02
sbezverkthe controllers19:02
inc0so not really19:02
inc0only reason for keepalived is to ensure that floating IP is on working node with HAProxy o it19:03
inc0on it19:03
sbezverkinc0 ok, then could you describe how it would work and monitor aliveness of controllers?19:03
inc0it doesn't really19:03
inc0haproxy does19:03
v1k0d3ninc0: so is there anything i'm missing?19:04
*** haplo37 has joined #openstack-kolla19:04
v1k0d3ni can't tell why this is failing. i'm using an unused address.19:04
inc0v1k0d3n, is it in same network as rest of  nodes19:04
inc0?19:04
v1k0d3nwell, in this case it's an AIO for PoC19:06
v1k0d3ninc0:19:06
openstackgerritMauricio Lima proposed openstack/kolla: [WIP]Manila driver session is needed only in manila-share  https://review.openstack.org/31123219:06
v1k0d3neven when i try an address in a different range it still fails.19:06
inc0hmm, can you show me output of ip a19:07
inc0?19:07
*** aginwala has quit IRC19:08
vhosakotsdake_: v1k0d3n: yes, 169.254.0.0/16 link local addresses are allowed kolla_internal_vip_address19:08
v1k0d3nthing is, this the exact same config worked on another host19:09
sbezverkinc0 in the config you are suggesting to have real HA, in addition to have 3 controller you need to have 2 network nodes, so the statement if you have 3 controllers you will have HA is incorrect..19:09
v1k0d3nthis is why i'm confused.19:10
inc0sbezverk, you need at least 2 network nodes anyway for L3 agent ha19:10
sbezverkinc0 but if I do not want to have L3 HA but I do care about internal HA, why would I waste host ??19:11
*** SiRiuS has quit IRC19:11
inc0sbezverk, I can hardly imagine someone like that;)19:11
inc0and you always can change inventory19:11
inc0inventory is kinda config file, it's meant to be edited19:12
sbezverkinc0 sure, but in this case the previously discussed is not an issue19:12
sbezverkbecause folks did not have 2 network nodes19:12
*** ravig has joined #openstack-kolla19:12
sbezverkhence haproxy and keepalived did not work with single network node config19:13
inc0well, you need 2 network nodes for ha, if you don't have it well, you're on your own19:13
v1k0d3nvhosakot: my problem is that 169.254. is always being referred to, even though i have kolla_internal_vip_address: "172.29.248.103"19:13
inc0security is main reaosn we run haproxy on network nodes19:13
*** signed8bit is now known as signed8bit_Zzz19:13
inc0but again, it's meant to be edited19:13
sbezverkinc0 ok, sounds good..19:14
mlimaTASK: [ceph | Bootstrapping Ceph OSDs] ****************************************19:14
mlimafatal: [localhost] => One or more undefined variables: 'dict object' has no attribute 'external_journal'19:14
mlima:/19:14
inc0with this config you don't have to expose controllers to external network19:14
vhosakotv1k0d3n: well, it is just a precheck... we use link local in gate without any issue  -->  https://github.com/openstack/kolla/blob/master/tools/deploy_aio.sh#L6219:14
*** ravig has quit IRC19:14
inc0and we don't support internal TLS, so if you want to have TLS to every service from external network, network-node should be only node exposed to external19:15
inc0hence endpoints should be there, all tls19:15
inc0and those without tls will be in internal network19:15
ccesarioinc0, build done.. without error!19:15
openstackgerritMerged openstack/kolla-kubernetes: Update for successful pep8  https://review.openstack.org/31305419:16
inc0rhallisey, dims https://github.com/openstack/kolla-kubernates19:16
inc040419:16
inc0did we miss something19:16
inc0?19:16
rhalliseyinc0, you have an a19:17
dimsspelling :)19:17
dimskubernetes19:17
rhalliseys/kubernates/kubernetes19:17
inc0-.-19:17
*** SiRiuS has joined #openstack-kolla19:18
sdake_dims note i added you as +2 to kolla-kubernete19:18
sdake_s19:18
v1k0d3nrhallisey: this is awesome!19:18
sdake_our thinking at this point is to add people that actively reviewthe spec or show interest in the dev early on19:18
v1k0d3n:)19:18
inc0yay now we can bug dims for reviews!19:18
dimsdear god!19:18
rhalliseyv1k0d3n, :D19:18
v1k0d3nam really happy about this.19:18
dimssdake_ : thanks.19:18
rhalliseyv1k0d3n, try and review the kolla-kubernetes spec19:19
rhalliseyleave any comments you might have19:19
sdake_v1k0d3n if you have two kolla clusters on the same network bad things will happen19:20
sdake_keepalived uses the same identifier for communication19:20
sdake_we should probably parameterize taht value19:21
-openstackstatus- NOTICE: Gerrit is restarting to address performance issues related to a suspected memory leak19:21
v1k0d3nsdake_: well, they're not exactly on the same network. development within openstack. using separate routers.19:21
sdake_v1k0d3n i dont know what your router would do with the packets19:22
v1k0d3nthat's a good bit of knowledge to keep handy though...i was actually wondering about that.19:22
sdake_v1k0d3n we found it originally in the bluebox gate19:22
sdake_we just haven't done anything about it in the code yet19:22
sdake_oddly we fixxed the bluebox gate ;-)19:22
haplo37Hi guys. Is it possible to docker container to doesn't see for example the management network ?19:23
sdake_haplo37 our docker containers use net=host, which means they will always see the host networks network stack19:23
sdake_which means no, its not possible to do what you asked19:23
*** signed8bit_Zzz is now known as signed8bit19:23
v1k0d3nrhallisey: are you planning on having a different IRC channel as well?19:24
*** ravig has joined #openstack-kolla19:24
rhalliseyv1k0d3n, no19:24
haplo37sdake_:ok, because it can be a security matter.19:25
rhalliseyv1k0d3n, since container related stuff is done here, it makes sense19:25
v1k0d3nok, that makes sense...i agree.19:25
sdake_haplo37 that is a possible threat scenario we are analzing as part of the VMT19:25
sdake_wheneverr we get back to that19:25
*** jtriley has quit IRC19:25
sdake_i think early next week i'll be ready :)19:25
haplo37VMT?19:26
v1k0d3nrhallisey: and just from a scope perspective, (sdake_ this is for you too), do you guys think that kolla will be delivered any differently? because kolla-ansible won't really do the deploy anymore, right? wouldn't kubernetes do that?19:27
v1k0d3ni have so many questions...lol19:27
sdake_we will have two deployment engines19:27
rhalliseyv1k0d3n, the ansible bits in kolla will still be its own deployment tool19:27
v1k0d3nok19:27
sdake_one ansible one k8s19:27
sdake_whether we split ansible out is a separate discussion to be had at a later time19:27
v1k0d3nok. good to know. just wondering about it.19:28
sdake_the main reason we are not having it now is backporting of our work19:28
sdake_if we split the repos, we would have to manaully backport any anasible work19:28
sdake_which would kind of suck19:28
sdake_i'd like to see the team get good at backporting in general before throwing moneky wrenches at them :)19:28
inc0and here I am, planning for kolla-ansible-k8s19:29
sdake_inc0 huh?19:30
inc0idea I'm playing with19:30
inc0add k8s role to ansible19:30
inc0change start.yamls to run stuff on k8s19:30
sdake_rhallisey you want to learn a little bit about reno19:31
inc0and in effect you have ansible deploying ceph and k8s, then doing bootstrap and running stuff on k8s afterwars19:31
rhalliseyinc0, rainbows and unicorns19:31
rhalliseysdake_, what's reno? Reno, Nevada?19:32
sdake_reno is a tool19:32
sdake_it needs to be added to kolla-kubernetess19:32
sdake_i can do it or you can19:32
sdake_pick who ;)19:32
dimshey, i have added it, see review19:33
rhalliseyoh this is docs eh19:33
dimsrhallisey : sdake_ : https://review.openstack.org/#/c/313090/19:34
patchbotdims: patch 313090 - kolla-kubernetes - Support for release notes19:34
sdake_dims nice ;)19:34
sdake_we also need docs infrasturcture added19:34
sdake_wonder why this isn't in the cookie cutter repo19:34
dimssdake_ : y, adding things slowly19:34
*** ravig has quit IRC19:34
*** ravig has joined #openstack-kolla19:35
dimsrhallisey : sdake_ : here's another one that will let us use code from kolla in the k8s repo - https://review.openstack.org/#/c/313081/19:35
patchbotdims: patch 313081 - kolla-kubernetes - Port some tests from Kolla repo19:35
sdake_folk i have a 1 hr groomingmeeting groan19:36
sdake_bbiaf19:36
*** rajathagasthya has joined #openstack-kolla19:46
*** absubram has quit IRC19:47
*** absubram has joined #openstack-kolla19:52
*** jtriley has joined #openstack-kolla19:53
v1k0d3nrhallisey: have a question....not sure where or how to capture this.19:59
v1k0d3nso SDN...19:59
v1k0d3nthere's neutron, which would be separate from the Kubernetes SDN underneath.20:00
v1k0d3nkind of wondering what this would look like.20:00
v1k0d3ndidn't want to open the can of worms in spec though20:01
rhalliseywe're going to run net=host20:01
rhalliseywell at least at first20:01
v1k0d3nok20:03
v1k0d3nsimple20:03
rhalliseyso we would have the services use the hosts network stack20:04
rhalliseyv1k0d3n, there's room to investigate us not doing that20:04
rhalliseyso not completely ruled out, but to get the project rolling we will probably use net=host everywhere20:04
*** mark-casey has quit IRC20:06
inc0most of services that require net=host can't be migrated anyway20:06
inc0so we can use kube sdn for subset of services20:07
inc0afk for a bit20:07
Lyncoshmmm20:07
LyncosERROR:kolla.cmd.build:cinder-base:Error'd with the following message20:07
LyncosERROR:kolla.cmd.build:cinder-base:Untar re-exec error: exit status 1: output: unexpected EOF20:07
LyncosI'm I doing something wrong ?20:07
openstackgerritDavanum Srinivas (dims) proposed openstack/kolla-kubernetes: Switch on warnerrors and universal wheel  https://review.openstack.org/31313720:08
rhalliseyLyncos, what distro are you using?20:08
Lyncosubuntu source20:08
*** ayoung has quit IRC20:10
LyncosI can't find that tar in dockerfile....20:10
*** ravig has quit IRC20:11
*** rhallisey has quit IRC20:14
*** rhallisey has joined #openstack-kolla20:14
*** rhallisey has quit IRC20:14
*** rhallisey has joined #openstack-kolla20:15
rhalliseyLyncos, https://github.com/docker/docker/issues/1479220:16
rhalliseyLyncos, some possibilities there20:16
rhalliseylooks like docker blowing up20:17
vhosakotLyncos: cinder-base builds fine for me on ubuntu source20:17
*** ravig has joined #openstack-kolla20:17
rhalliseyLyncos, what version of ubuntu are you using as a base?20:17
Lyncosxenial20:17
rhalliseyalso do you have a ton of images laying around?20:17
Lyncosbut we build with DIND20:17
*** aginwala has joined #openstack-kolla20:19
rhalliseyvhosakot, are you using --base-tag=14.04 ?20:20
rhalliseyinc0, mentioned it was the latest we can build with at the moment20:20
rhalliseyI don't know if that has changed20:20
vhosakotrhallisey: no... it defaults to 14.04 in code... https://github.com/openstack/kolla/blob/master/kolla/cmd/build.py#L34920:21
*** ravig has quit IRC20:21
rhalliseygotcha20:21
vhosakotrhallisey: Jeffrey raised a very good point yesterday that we must not default to 14 in xenial20:21
vhosakotLyncos: can you use --base-tag=1620:22
*** ravig has joined #openstack-kolla20:22
Lyncoskolla is ready for 16 ?20:22
vhosakotLyncos: we'll know once we test :)20:22
Lyncosno it will not work20:22
Lyncosccesrio posted a patchset yesterday to make it work20:23
vhosakotLyncos: yep, saw that20:23
Lyncosso I think it will not work because the source list will still point to trusty20:23
vhosakotbasically, we need to filter unsupported distros in code.. like we must not build on Ubuntu 16 in Mitaka... 16 in newton is ok20:23
*** ravig has quit IRC20:24
*** ravig has joined #openstack-kolla20:25
*** mummer has quit IRC20:26
*** mummer has joined #openstack-kolla20:26
*** ravig has quit IRC20:27
*** inc0 has quit IRC20:27
*** inc0 has joined #openstack-kolla20:28
*** banix has quit IRC20:28
*** ravig has joined #openstack-kolla20:28
gmmahahave you guys ever seen an issue like this when building images and pushing to local registry?20:28
gmmahahttp://paste.openstack.org/show/496273/20:28
inc0Lyncos, rhallisey kolla is no bueno on xenial now20:28
Lyncosyeah I know but this was working before...20:28
Lyncoswe build kolla from a Xenial machine and it was working just fine20:29
Lyncossomething changed on our side20:29
inc0Lyncos, if you have older code, it will download xenia20:29
-openstackstatus- NOTICE: Gerrit is restarting to revert incorrect changes to test result displays20:29
Lyncoswe only have new code :-)20:29
inc0default build.py will force 14.04 instead of latests20:29
*** ravig has quit IRC20:29
*** dmk0202 has joined #openstack-kolla20:29
inc0do you build with our build.py?20:30
Lyncoskolla-build20:30
inc0then yean20:30
inc0it should download 14.0420:30
*** jtriley has quit IRC20:30
inc0btw if you build on xenial host, it's ok20:30
inc0it's docker OS that is no bueno with xenial20:31
inc0aka our images20:31
ccesarioinc0,20:31
LyncosI have to run20:31
ccesariodeploy it works20:31
inc0cool20:32
Lyncoswill look into this tomorrowe20:32
inc0have fun Lyncos20:32
Lyncosthank you... 1h traffic jam20:32
inc0you can have laptop around and deploy openstack while on it20:32
inc0twice20:32
ccesarioinc0, http://paste.openstack.org/show/496274/20:32
vhosakotgmmaha: is your registry up ? can you curl it ?20:33
gmmahavhosakot: yeah the registry seems to be up20:34
gmmahalet me try and curl20:34
vhosakotgmmaha: the error has "The Web Server may be down"20:34
ccesarioinc0, any specific test ?20:34
inc0ccesario, boot a vm:)20:34
gmmahavhosakot: yeah, registry is running..20:34
*** jtriley has joined #openstack-kolla20:34
gmmahacurl to http://172.16.2.241:4000 is stuck20:35
vhosakotgmmaha: i'd try just pushing one image20:35
inc0gmmaha, try flushing your iptables20:35
vhosakotgmmaha: curl is stuck..... means registry is down20:35
vhosakotcan you do curl -v20:35
*** sdake has joined #openstack-kolla20:35
inc0vhosakot, it might mean iptables is dirty20:36
gmmahainc0: let me flush them20:36
inc0which happends sometimes if you build a lot20:36
vhosakotah ok20:36
gmmahavhosakot: http://paste.openstack.org/show/496276/20:36
gmmahadamn proxy.. :(20:36
*** sdake_ has quit IRC20:36
inc0there you go20:37
vhosakotgmmaha: yeah, flush your iptables... also, I'd restart the registry20:37
rhalliseyyo inc0 -> http://blog.kubernetes.io/2016/01/simple-leader-election-with-Kubernetes.html20:37
inc0<3 proxy20:37
gmmahavhosakot: sure.. let me do that20:37
sdakejust had a look at bifrost20:37
sdakeseems easy enough to place it in a container20:37
inc0"simple leader election" are 3 words that together make me suspicious;)20:37
rhalliseyha20:38
sdakeya how about complex ;)20:38
vhosakotshould be "best leader election" :)20:38
rhalliseyinc0, sounds like it could be of some help20:39
rhalliseylikely going to be a bp down the road though XD20:39
gmmahavhosakot: inc0: thanks.. ti was the damn proxy20:39
rhalliseywill note of it in the spec20:39
gmmahaadding it to no_proxy makes it all work20:39
gmmaha:)20:39
gmmahadman proxy!!20:39
vhosakotgmmaha: why do you us eproxy ? ;)20:39
vhosakotuse*20:39
gmmahavhosakot: all my machines are behind intel proxy20:39
gmmahaIntel is awesome :)20:40
vhosakotproxy magic is operator-specific and not kolla-related imo20:40
gmmahavhosakot: very much..20:42
gmmahajsut that everytime i ever deploy anything inside the cmapus, i have to worry about it20:42
*** jtriley has quit IRC20:43
gmmahavhosakot: would be good to make kolla a little resilient to proxy20:43
gmmahavhosakot: thanks for the help again.. i assumed that i am doing something wrong with kolla and google search sorta made me think that20:47
vhosakotcool20:47
*** daneyon_ has quit IRC20:47
*** aginwala has quit IRC20:48
*** Mr_Broken has joined #openstack-kolla20:48
nihiliferanyone has the same problem with kolla-toolbox building? http://paste.openstack.org/show/496278/20:49
nihilifer(i'm trying with centos source)20:49
nihiliferlooks like even installing the newest pip is failing due to pytz dependency, kinda stragne20:50
nihiliferlike it's happening here https://github.com/openstack/kolla/blob/master/docker/kolla-toolbox/Dockerfile.j2#L3520:50
*** aginwala has joined #openstack-kolla20:51
sbezverknihilifer I hit that issue and there was a fixed committed into master20:51
nihilifersbezverk: i have up to date repo :(20:52
sbezverknihilifer hmm, the issue is back :-( somebody must have removed the fix we committed..20:52
*** ccesario_ has joined #openstack-kolla20:53
sbezverknihilifer check this out https://review.openstack.org/30560620:54
sbezverkhttps://bugs.launchpad.net/kolla/+bug/156963320:54
openstackLaunchpad bug 1569633 in kolla "building kolla_toolbox container fails" [Critical,Fix released] - Assigned to Serguei Bezverkhi (sbezverk)20:54
vhosakotsbezverk: that fix was abanoned for mitaka.. the fix is in master... do you know why it was notmerged into mitaka ?20:55
*** neophy has joined #openstack-kolla20:55
vhosakotnihilifer: are you on mitaka ?20:55
nihilifernvm guys, it works for me now, thanks for hints20:56
nihilifervhosakot: nope, master, but not up to date20:57
vhosakotnihilifer: cool..20:57
vhosakotsbezverk: nihilifer: see sdake's comment in https://review.openstack.org/#/c/305606/20:57
patchbotvhosakot: patch 305606 - kolla (stable/mitaka) - Fixes kolla_toolbox failure due to missing pytz mo... (ABANDONED)20:57
*** haplo37 has quit IRC20:57
nihilifer(i forgot that pip install -e doesn't update files in /usr/share, lol)20:57
*** mbound has joined #openstack-kolla21:01
sdake10 minutes to deploy 3 node HA21:02
sdakeon centos21:02
sdakeyo inc0 how long did your 20 node deploy take21:02
inc010 node*21:03
*** ravig has joined #openstack-kolla21:03
inc0hold on, about to do it again21:03
*** aginwala has quit IRC21:03
sdakeinc0 good news centos source deploys 3 vms on 3 nodes21:05
sdakewith heat21:05
sdakeso i'd say everything is in working order21:05
sdakei have two questions21:05
sdake1) did you search for 127.0.0.1 in logs21:05
sdake2) did you update the upstream versions for all versions of the software21:05
sdakeif not, those last two things need to be done21:05
*** Mr_Broken has quit IRC21:06
*** Mr_Broken has joined #openstack-kolla21:07
*** ravig has quit IRC21:09
*** ccesario___ has joined #openstack-kolla21:10
*** ccesario_ has quit IRC21:10
sbezverkrhallisey ping21:10
inc0sdake, 6min21:11
inc0with cep21:11
inc0h21:11
sdakemust have had images cached or  something21:11
*** Mr_Broken has quit IRC21:11
inc0yeah21:12
inc0most of them were already on nodes21:12
sdakeinc0 can you answer the two questions above21:12
inc0configuring kibana to see logs21:13
rhalliseysbezverk, hey21:13
*** aginwala has joined #openstack-kolla21:14
inc0sdake, no 127.0.0.1 in any of logs21:15
inc0I think you had something in your configuration;P21:15
inc0problably some host resolved ip to localhost21:15
sdakeno i think i fixed allo fthem21:15
sdakebut wasn't sure21:15
sdakeand wanted verification21:15
*** aginwala has quit IRC21:16
*** ravig has joined #openstack-kolla21:16
*** ccesario___ has quit IRC21:16
*** ccesario___ has joined #openstack-kolla21:16
inc0sdake, this should point to liberty right? https://github.com/openstack/kolla/blob/stable/liberty/kolla/common/config.py#L2121:17
*** ccesario___ has quit IRC21:18
sdakeit caan point to whatever - in one of my patches i took out its usage21:18
sdakethat thing is only needed for master21:18
sdakeoh i took it out in mitaak21:18
sdakelong ago21:18
inc0sdake, shouldn't it point to liberty in stable/liberty?21:20
*** ccesario___ has joined #openstack-kolla21:20
sdakeit doesnt do anything21:20
sdakeit can point to null.com21:20
sdakeand would still do ntohing :)21:20
inc0ok21:22
sbezverkrhallisey in specs you mentioned that docker 1.10 is required21:22
*** ravig has quit IRC21:22
*** ravig has joined #openstack-kolla21:22
sbezverkbut when I try t install kubernetes on centos it forces docker version 1.9.121:22
sbezverkhave you noticed it in your test bed??21:23
rhalliseysbezverk, we could run kube in a container21:23
rhalliseyI've been using hypercube21:23
rhalliseywhich is containerized kubernetes21:23
sbezverkrhallisey I see..21:23
sbezverkrhallisey Hopefully k8s will bum up supported docker21:24
*** ravig has quit IRC21:24
*** mbound has quit IRC21:24
rhalliseysbezverk, what version of kubernetes are you using?21:24
sbezverkkubernetes-1.2.0-0.9.alpha1.gitb57e8bd.el7.x86_6421:25
sbezverkwhen I installed it it installed docker 1.9.121:25
*** ravig has joined #openstack-kolla21:25
*** ravig has quit IRC21:27
rhalliseysbezverk, could just be centos packaging requiring it21:29
rhalliseysbezverk, can you install kubernetes without installing docker?21:29
sbezverkrhallisey nope, it is forced to be installed and if you have version higher, it does not install k8s21:30
*** esharao has joined #openstack-kolla21:31
sbezverkrhallisey maybe it is forced by centos packaging, do not know21:31
rhalliseysbezverk, may have to use kube from source then21:32
sbezverkrhallisey I guess will have to do it when it comes to running kolla..21:34
sbezverkrhallisey man this version does not even support configmap :-( damn...21:37
*** aginwala has joined #openstack-kolla21:37
*** rstarmer has joined #openstack-kolla21:38
rhalliseysbezverk, 1.2?21:44
rhalliseydoesn't support configmap?21:44
*** aginwala has quit IRC21:44
rhalliseythink I need to bump the kubeversion then21:44
*** aginwala has joined #openstack-kolla21:45
*** ccesario___ has quit IRC21:45
rhalliseysbezverk, ya I'm able to run docker 1.10.2 with kube 1.2.021:45
rhalliseywith hyperkube21:46
*** aginwala_ has joined #openstack-kolla21:47
*** unicell1 has quit IRC21:48
*** sdake has quit IRC21:50
*** aginwala has quit IRC21:50
*** sdake has joined #openstack-kolla21:51
sbezverkrhallisey right kubectl that comes with my version does not understand keyword create configmap21:52
*** jasonsb has quit IRC21:55
*** sdake has quit IRC21:56
*** aginwala has joined #openstack-kolla21:56
*** vhosakot has quit IRC21:57
*** jasonsb has joined #openstack-kolla21:58
*** aginwala_ has quit IRC21:59
*** Mr_Broken has joined #openstack-kolla22:03
*** ayoung has joined #openstack-kolla22:03
*** ccesario___ has joined #openstack-kolla22:06
*** aginwala has quit IRC22:07
sbezverkrhallisey just confirmed configmap comes with version 1.3.0+22:08
rhalliseysbezverk, ok sweeet22:09
rhalliseyi'll bump to 1.322:09
*** unicell has joined #openstack-kolla22:12
*** aginwala has joined #openstack-kolla22:13
*** aginwala has quit IRC22:17
*** Mr_Broken has quit IRC22:19
*** banix has joined #openstack-kolla22:20
*** absubram has quit IRC22:22
*** rstarmer has quit IRC22:26
*** rajathagasthya has quit IRC22:26
*** rajathagasthya has joined #openstack-kolla22:31
*** fragatin_ has joined #openstack-kolla22:37
*** aginwala has joined #openstack-kolla22:38
*** signed8bit is now known as signed8bit_Zzz22:39
*** aginwala has quit IRC22:40
*** daneyon has joined #openstack-kolla22:40
*** fragatina has quit IRC22:41
rhalliseybeekhof, http://blog.kubernetes.io/2016/01/simple-leader-election-with-Kubernetes.html22:42
*** ccesario___ has quit IRC22:42
*** aginwala has joined #openstack-kolla22:42
rhalliseywas a fascinating blog22:43
*** dwalsh has joined #openstack-kolla22:43
openstackgerritRyan Hallisey proposed openstack/kolla: Spec: Deploy Kolla images using Kubernetes  https://review.openstack.org/30418222:44
*** sacharya has quit IRC22:45
*** neophy has quit IRC22:50
*** rhallisey has quit IRC22:50
*** salv-orlando has joined #openstack-kolla22:53
*** salv-orlando has joined #openstack-kolla22:54
openstackgerritMerged openstack/kolla-kubernetes: Support for release notes  https://review.openstack.org/31309022:55
*** aginwala has quit IRC22:57
*** aginwala has joined #openstack-kolla22:59
*** SiRiuS has quit IRC23:05
*** ayoung has quit IRC23:07
*** ccesario___ has joined #openstack-kolla23:09
*** daneyon has quit IRC23:11
*** rajathagasthya has quit IRC23:20
*** esharao has quit IRC23:22
*** salv-orlando has quit IRC23:26
*** dmk0202 has quit IRC23:42
*** sacharya has joined #openstack-kolla23:45
*** sacharya has quit IRC23:50
*** ayoung has joined #openstack-kolla23:53
*** mummer has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!