Tuesday, 2017-04-18

sdakesup spsurya00:05
sdakespsurya there was a discussion earlier today that affects the work you are doing00:05
sdakespsurya moment00:05
spsuryasdake yes00:06
spsuryathat was related to00:06
sdakehttp://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-04-17.log.html#t2017-04-17T15:59:5500:06
spsuryadocker not starting00:06
sdakespsurya please read that log00:06
spsuryasure00:07
*** guoguo has joined #openstack-kolla00:12
*** guoguo has quit IRC00:13
*** guoguo has joined #openstack-kolla00:14
sdakekfox1111 sbezverk - this is ready for review plz: https://review.openstack.org/#/c/451495/800:22
sdakespsurya what did you parse out the log00:22
*** jascott1 has joined #openstack-kolla00:24
sbezverksdake: let's wait for the gate00:25
sdakesbezverk thanks :)00:26
sbezverksdake:  https://review.openstack.org/45690600:26
spsuryasdake: till now what i understood is we are in right direction . So we need to complete the things of gate and dev env i.e the current work00:26
spsuryaplease correct if i am wrong00:26
sdakespsurya the way i parsed that conversation is as follows00:27
sdakethe work in my original devtools patch - some variant of that, or a refactor of the existing tools that setup canal, setup kubernetes etc stay put - without error detection00:28
sdakespsurya everything gate related goes to the /gate directory00:28
sdakethe gate tools then call the simpler more generalized setup_* tools in tools dir00:28
sdakerather the gate scripts in /gate call them00:28
sdakethen we can make forward progress from there00:29
sdakespsurya the end result of this for this milestone is that the devtools are gated00:29
sdakeand we don't endelessly spin on rewriting gating and development tools nad documentation00:29
sdakespsurya that is probably a lot to digest, and you will have to become an expert in the gate00:30
sdakespsurya i'd recommend tackling one file at a time if your up for the mission00:30
sdakekfox1111 ca nyou validate plz00:31
sdakejascott1 ^^00:31
sdakejascott1 spsurya the good news of this is you will become experts in the gate00:32
sdakeand also be able to become expert level contributors00:32
sbezverksdake: why touch gate??? why do not you create a separate directory for dev env and work in it??00:32
sdakesbezverk that isn't what kfox1111 wanted00:33
sdakesbezverk if you read that log above he had a fit because i did as you suggested :)00:33
sdakehttp://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-04-17.log.html#t2017-04-17T15:59:5500:33
*** huzhengchuan has joined #openstack-kolla00:33
sbezverksdake: kfox1111 is not going to work on this, if it were him then I think it would be doable, anybody else including myself would not be able to do it quickly and painlessly00:34
sdakei could do it quickly and painlessly00:34
sbezverkso this will just burn cycles and efforts of a one or possible more developers00:34
sdakehowever the goal isn't quick and painless00:34
sdakesbezverk these devs don't know how to contirbute to kolla-kubernetes00:35
sbezverkinstead of actually contributing usefull code00:35
sbezverksdake: I do not beleive it00:35
sdakethe reaosn they dont know how to contribute is because the devtools don't exist and the gate is broken00:35
openstackgerritMerged openstack/kolla-kubernetes master: Converting keepalived into helm charts  https://review.openstack.org/45690600:35
sbezverkit is so damn easy to setup, if somebody really wanted, he/she would have completed it in one day max00:35
sdakewhat is so damn easy to setup?00:36
sbezverkdevelopment env00:36
sdakesbezverk it took me 2 months with the help of 15 contributors to write the deployment guide00:36
sdakesbezverk that was all about learning for me how the gate worked00:36
sdakethat is a long ramp up time00:36
sbezverksdake: because you wanted a magic script00:36
sbezverkthat is all00:36
sdakesbezverk incorrct - I wanted documentaiton people could understand00:36
sdakea magic script is what the gate is00:37
sdakeblack fucking magic00:37
sbezverkjust doing copy and paste from gates with minor modification would bring env up in no time00:37
sdakesbezverk either you trust my judgement or you dont - this is the best path forward00:37
sbezverksdake: I am sorry sdake but I see waste of valuable cycles of other developers on grand initiatives..00:38
sdakelearning is not wasting00:38
sdakei attempted to mrege a review that simplifies the devleopment experience00:39
sdakekfox1111 wants a grand initiative00:39
sdakehere i tis00:39
sdakeI also want spsurya and jascott1 to learn how the gate works if they are to be contributor00:40
sdakes00:40
sbezverksdake: well, how many weeks you think it will take? do we really have this time??00:40
sdakeMY POV: it is impossible to contribute to this project if you don't understand how the gate operates00:40
sdakesbezverk it is scheduled for 1 month00:40
sdakethe alternative is these devs spin cycles on doing nothing00:41
sbezverksdake: -2 I contributed iscsi to kolla without any idea of how gates work00:41
sdakeas in they are completely blocked now00:41
sdakeiscsi didn't have a gate00:41
sdakestill doesn't00:41
sdakekolla-ansible has much less stringent gating requirements00:41
sdakethat is why you were able to make such fast progress00:41
sbezverksdake: anything I contributed to kolla was done without understanding the gate00:41
sdakesbezverk every one of my patches includes gate chagnes00:42
sdakesbezverk or requires gate changes00:42
sdakesbezverk except docs of course00:42
*** zhurong has joined #openstack-kolla00:42
sbezverksdake: hmm, even 10k of deleting PS?00:42
sdakesbezverk yup even those00:42
jascott1sdake you said my name three times and I have been summoned00:42
sdakei had to understand the gate00:42
sdakejascott1 :)00:42
sdakejascott1 do you want to learn how the gate works?00:43
sdakejascott1 this is what leads to understanding how to contribute to kolla-kubernetes00:43
spsuryasdake: sbezverk following your conversation00:44
sdakesbezverk it is unfortunate that kolla-kubernetes requires a deep understanding of gating to contribute to - that is what has been built00:44
jascott1ive been thru the gate a few months ago, im sure its changed a lot. I used it to understand what the hell was going on with the kolla-kubernetes install when the docs were so outta whack00:44
jascott1but yes I do want to learn how the gate works00:44
sdakejascott1 here is your opportunity00:44
sdake;)00:44
sdakejascott1 then you can actually contribute code00:44
spsuryasame here sdake00:45
sbezverkthat is only sdake POV, I think to contribute to kolla-kube is not required gate00:45
sdakesince you will be able to make gate changes on code contributions00:45
sbezverkknowledge, it is nice to have but not required00:45
sdakesbezverk please find me one example of a patch that doesn't touch the gate that isn't some sort of cleanup (such as removal of dead code)00:45
sdakeoen is all I ask00:45
sbezverkactually if you use gate scripts to setup your dev env, you will get that knwoledge00:45
sdakesbezverk you need to experience working on the gate to understand it00:46
jascott1I did some of that and did learn but its not ideal00:46
sdakemy understanding of the gate came through working on it00:46
sdakesbezverk find me one example :)00:47
sbezverkhow do you exaplina kolla/kolla-ansible? I bet only 10-20% of contributors have knowledge about the gate00:47
sbezverkbut they are still ACTIVELY contributing00:47
*** tonanhngo has quit IRC00:47
sdakesbezverk i explain it because the gating requirements are not stringent00:47
sdakethe gate barely tests anything in kolla-ansible00:47
sdakethe kolla gate is braindead easy -either it builds or doesn't00:47
*** Margin has joined #openstack-kolla00:48
sdakesbezverk we have a couple options - reduce gating requirements - or teach people how to work on the gate00:48
sdakesbezverk and kfox1111 are of the opinion that every patch should pas the gate - so lets enable contributors to do that00:48
sdakesbezverk if you have a third option I'm listening00:49
sdakeand sitll waiting for that one patch00:49
sdakesbezverk think back to our ironic gating work - which was fantastic btw00:50
sbezverksdake: please recheck I am worried about ironic job failure00:50
sdakewhy did you do that gating work?00:50
openstackgerritjeck proposed openstack/kolla-ansible master: Removing swift references from fstab  https://review.openstack.org/45680500:50
sdakesbezverk will do00:50
sdakesbezverk recheck which00:50
masbergood morning all, according to this doc https://github.com/openstack/kolla-ansible/blob/master/doc/ironic-guide.rst ironic is in tech preview. Is there a roadmap about fully support ironic?00:50
sbezverksdake: https://review.openstack.org/#/c/451495/00:51
sbezverkironic failure after your modification, I want to make sure it is transient issue00:51
sdakelooks like stuck in gate00:51
sdakerather stuck in build00:51
sdakehttp://logs.openstack.org/95/451495/8/check/gate-kolla-kubernetes-deploy-centos-source-4-ironic-nv/3d333b7/console.html#_2017-04-18_00_15_03_28547300:51
sdakesbezverk ya the ansible change  I ntroduced also suffered this transient fialure00:51
*** newmember has quit IRC00:53
sdakejascott1 spsurya is this work you are interested in - minus the fact that there is clearlly disagreemnet ont he topic :)00:53
jascott1sure00:53
sdakejascott1 spsurya cool once you understand the gate, you can do anything with kolla-kubernetes00:53
*** tonanhngo has joined #openstack-kolla00:53
jascott1maybe someone can give a deep dive or at least overview of the finer points00:54
*** newmember has joined #openstack-kolla00:54
sdakejascott1 when would you like?00:54
sdakejascott1 avialalbe now00:54
sdakespsurya ?00:54
spsuryahere00:54
spsuryaplease start00:54
sdakejascott1 had asked for a deep dive on the gate, i think that would helpy ou too spsurya if your interested in contributing00:54
jascott1ive got a hard stop in about 3000:54
jascott1but yes00:55
sdakewebex?00:55
sdakehttp://cisco.webex.com/join/stdake00:55
jascott1sure00:55
jascott1one sec00:55
sdake30 minutes wont be much of a deep dive00:55
*** tonanhngo_ has joined #openstack-kolla00:56
jascott1idk part1 ?00:56
sdakeyup part 100:56
*** tonanhngo has quit IRC00:58
*** tonanhngo_ has quit IRC01:01
*** cuongnv has joined #openstack-kolla01:02
sbezverksdake: I need to learn from you your way of giving deep dives ;)01:02
*** tovin07_ has joined #openstack-kolla01:02
sdakedamnit webex crashed01:08
sdakepile o garb01:08
*** StephenWang1991 has joined #openstack-kolla01:09
*** StephenWang1991 has quit IRC01:13
*** tervancovan has joined #openstack-kolla01:13
*** MasterOfBugs has quit IRC01:16
*** zhubingbing has joined #openstack-kolla01:27
*** hrw has quit IRC01:30
*** hrw has joined #openstack-kolla01:32
sdakesbezverk you are correct - doing what kfox1111 wants is difficult01:35
sdakesbezverk we do have the time to do it01:35
sdakeeanylin spsurya jascott1 thanks for attending01:35
*** tonanhngo has joined #openstack-kolla01:36
spsuryasdake nice01:37
*** MasterOfBugs has joined #openstack-kolla01:40
*** negronjl_ has joined #openstack-kolla01:42
*** al498u_ has quit IRC01:42
sdakethanks for the ack sbezverk01:44
*** tonanhngo has quit IRC01:45
*** StephenWang1991 has joined #openstack-kolla01:47
*** eanylin has quit IRC01:47
*** britthouser has quit IRC01:49
openstackgerritshaofeng cheng proposed openstack/kolla master: Add oslo-vmware module  https://review.openstack.org/45716301:49
*** britthouser has joined #openstack-kolla01:49
sdakesup britthouser01:50
*** iceyao has joined #openstack-kolla01:51
openstackgerritshaofeng cheng proposed openstack/kolla master: Add oslo-vmware module  https://review.openstack.org/45716301:52
*** ssurana has joined #openstack-kolla01:54
*** iceyao has quit IRC02:03
*** tonanhngo has joined #openstack-kolla02:04
*** iceyao has joined #openstack-kolla02:04
*** Margin has quit IRC02:05
*** unicell has quit IRC02:08
*** tonanhngo has quit IRC02:11
*** super has joined #openstack-kolla02:12
*** tonanhngo has joined #openstack-kolla02:16
*** eanylin has joined #openstack-kolla02:20
openstackgerritshaofeng cheng proposed openstack/kolla master: Add oslo-vmware module in glance_base image  https://review.openstack.org/45716302:23
*** tonanhngo has quit IRC02:28
*** StephenW_ has joined #openstack-kolla02:30
*** strigazi has quit IRC02:30
*** strigazi has joined #openstack-kolla02:31
*** lujinluo has joined #openstack-kolla02:32
*** StephenWang1991 has quit IRC02:33
*** tonanhngo has joined #openstack-kolla02:35
*** mbruzek has joined #openstack-kolla02:36
*** japestinho has joined #openstack-kolla02:37
*** iceyao has quit IRC02:49
*** schwicht has quit IRC02:53
*** zhurong has quit IRC03:03
*** lamt has joined #openstack-kolla03:04
*** tonyb has quit IRC03:04
*** dave-mccowan has quit IRC03:05
*** iceyao has joined #openstack-kolla03:06
*** sudipto has joined #openstack-kolla03:08
*** sudipto_ has joined #openstack-kolla03:08
*** zhurong has joined #openstack-kolla03:12
*** tonanhngo has quit IRC03:15
*** gkadam has joined #openstack-kolla03:15
*** gkadam is now known as gkadam-brb03:15
*** lamt has quit IRC03:18
*** lamt has joined #openstack-kolla03:19
*** iceyao has quit IRC03:20
*** iceyao has joined #openstack-kolla03:21
*** mbruzek has quit IRC03:31
*** StephenWang1991 has joined #openstack-kolla03:35
*** jtriley has quit IRC03:35
*** iceyao has quit IRC03:35
*** iceyao has joined #openstack-kolla03:36
*** StephenW_ has quit IRC03:37
*** iceyao has quit IRC03:42
*** lamt has quit IRC03:45
*** gkadam-brb is now known as gkadam03:46
*** lamt has joined #openstack-kolla03:52
*** zhurong has quit IRC04:00
*** lamt has quit IRC04:00
openstackgerritMerged openstack/kolla-ansible master: Fix 'node_config_directory' variable bad usage  https://review.openstack.org/45658804:02
*** iceyao has joined #openstack-kolla04:03
*** lamt has joined #openstack-kolla04:07
*** iceyao has quit IRC04:08
duonghqmnaser, ping04:08
spsuryaduonghq: can you please set +w https://review.openstack.org/#/c/455866/04:09
*** lamt has quit IRC04:13
*** lamt has joined #openstack-kolla04:14
*** unicell has joined #openstack-kolla04:15
*** lamt has quit IRC04:15
*** unicell has quit IRC04:16
*** zhurong has joined #openstack-kolla04:18
*** unicell has joined #openstack-kolla04:19
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Add panko.conf file in ceilometer container  https://review.openstack.org/44471904:20
duonghqspsurya, we cannot link to neighbor file?04:21
*** zhurong has quit IRC04:25
spsuryaduonghq: you mean this neigbour file https://github.com/openstack/kolla-ansible/blob/master/doc/vagrant-dev-env.rst04:26
openstackgerritMerged openstack/kolla master: Updated from global requirements  https://review.openstack.org/45592804:26
spsuryaduonghq: ?04:26
spsuryabtw the link i provided also fine04:28
*** zhubingbing has quit IRC04:28
spsuryaIMO04:28
duonghqspsurya, mins, I'm reading rst manual04:29
duonghq:)04:29
duonghqsorry, need making sure about smt04:29
*** unicell has quit IRC04:34
*** unicell has joined #openstack-kolla04:35
openstackgerritMerged openstack/kolla-ansible master: Switch kolla_docker to rely on SHA256 for image changes  https://review.openstack.org/45426504:35
*** chas has joined #openstack-kolla04:53
*** chas has quit IRC04:53
openstackgerritshaofeng cheng proposed openstack/kolla master: Add oslo-vmware module in glance_base image  https://review.openstack.org/45716304:53
*** chas has joined #openstack-kolla04:54
openstackgerritMerged openstack/kolla master: Use ironic-inspector user for ironic-inspector  https://review.openstack.org/44160504:55
*** chas has quit IRC04:58
*** zhurong has joined #openstack-kolla05:00
*** bmace has quit IRC05:12
*** bmace has joined #openstack-kolla05:13
*** sudipto_ has quit IRC05:18
*** sudipto has quit IRC05:18
*** sudipto_ has joined #openstack-kolla05:19
*** sudipto has joined #openstack-kolla05:19
*** jrobinson has quit IRC05:19
*** sudipto_ has quit IRC05:19
*** sudipto has quit IRC05:19
*** zhubingbing_ has joined #openstack-kolla05:21
*** skramaja has joined #openstack-kolla05:26
portdirectjascott1: nice work re the 'helm init --net-host' cool stuff :)05:30
jascott1hey! whats up?05:31
jascott1i like that it was ~5 lines total iirc05:31
portdirectyeah - I played around with running outside the cluster to achive the same effect - but having it in is really nice05:32
SamYapleportdirect-- the man who never sleeps05:32
portdirectfor installing proper networking backends is super useful05:33
portdirectSamYaple: we meet again.05:33
jascott1what up SamYaple!05:33
jascott1yeah i need to make a proper PR, going to add selectors and tolerations looks like05:33
*** iceyao has joined #openstack-kolla05:47
*** zhugaoxiao has quit IRC05:56
SamYapleyo jascott1. hows it going. still being awesome?06:00
*** Serlex has joined #openstack-kolla06:01
*** cuongnv has quit IRC06:02
*** cuongnv has joined #openstack-kolla06:02
jascott1haha i try to for a few minutes everyday06:07
*** tonanhngo has joined #openstack-kolla06:07
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Add panko.conf file in ceilometer container  https://review.openstack.org/44471906:09
*** dvx76 has joined #openstack-kolla06:14
*** matrohon has joined #openstack-kolla06:19
*** iceyao has quit IRC06:22
*** sayantan_ has quit IRC06:23
*** sudipto_ has joined #openstack-kolla06:26
*** sudipto has joined #openstack-kolla06:26
*** imcsk8 has quit IRC06:31
*** tonanhngo has quit IRC06:32
*** iceyao has joined #openstack-kolla06:35
openstackgerritjeck proposed openstack/kolla-ansible master: Removing swift references from fstab  https://review.openstack.org/45680506:37
*** sayantan_ has joined #openstack-kolla06:43
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Grafana: automatically set InfuxDB datasource  https://review.openstack.org/45709406:45
tervancovanh all06:46
tervancovanhi all06:46
tervancovanls06:48
*** sayantan_ has quit IRC06:48
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Add VMware DataStore support to glance  https://review.openstack.org/45217606:49
openstackgerritBertrand Lallau proposed openstack/kolla-ansible stable/ocata: Fix 'node_config_directory' variable bad usage  https://review.openstack.org/45747706:50
*** bjolo has joined #openstack-kolla06:50
bjolomorning06:50
tervancovanI have configured kolla-kubernetes environment using kubeadm-aio06:52
tervancovanand I think that there is more settings for launching kolla-kubernetes  in kubeadm-aio06:54
*** manheim has joined #openstack-kolla06:54
tervancovanIs it right?06:55
*** bjolo has quit IRC06:56
*** bjolo has joined #openstack-kolla06:57
*** pcaruana has joined #openstack-kolla06:59
*** tonanhngo has joined #openstack-kolla07:01
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Simplify Openstack services configuration possibilities  https://review.openstack.org/45668807:02
*** tonanhngo has quit IRC07:02
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Cleanup /run/netns host directory volume mount  https://review.openstack.org/45417907:07
*** sayantan_ has joined #openstack-kolla07:09
*** shardy has joined #openstack-kolla07:09
*** Serlex has quit IRC07:13
openstackgerritDuong Ha-Quang proposed openstack/kolla-kubernetes master: Heat service deployment  https://review.openstack.org/43725207:15
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Generalize api_interface_address variable usage  https://review.openstack.org/45747907:15
*** chas has joined #openstack-kolla07:23
*** magicboiz has joined #openstack-kolla07:29
openstackgerritJavier Castillo proposed openstack/kolla-ansible master: Fix incorrect fluend destination address  https://review.openstack.org/45726207:31
*** egonzalez has joined #openstack-kolla07:38
*** haplo37- has quit IRC07:39
*** haplo37_ has joined #openstack-kolla07:41
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Fix zun images  https://review.openstack.org/45625107:44
openstackgerritzhubingbing proposed openstack/kolla-ansible stable/ocata: Multi-regions Support  https://review.openstack.org/45749407:49
egonzalezzhubingbing_, thanks for zun logs fix ;)07:50
zhubingbing_;)07:50
zhubingbing_https://review.openstack.org/#/c/457494/ <egonzalez07:51
zhubingbing_please review it07:51
*** jaosorior has joined #openstack-kolla08:03
*** ssurana has quit IRC08:05
*** ssurana has joined #openstack-kolla08:08
*** zhurong has quit IRC08:09
*** ssurana has quit IRC08:10
spsuryazhubingbing_: http://paste.openstack.org/show/606938/         \08:11
spsuryacan you please check08:11
spsuryahave faced this issue08:11
spsuryakk8s08:11
zhubingbing_hmm08:12
spsuryaso what is happening  ?08:12
spsuryawhat could be the problem in this ?08:12
zhubingbing_k8s 1.6.0 have some problems08:13
spsuryaregarding what ? will please elaborate08:13
*** iceyao has quit IRC08:13
spsuryawill you*08:13
zhubingbing_i thinks you should install 1.5.408:13
spsuryazhubingbing_: but we are on the rack of 1.6.108:14
openstackgerritzhangdongqi proposed openstack/kolla-ansible master: neutron_vpnaas_agent: kolla_docker parameter 'volumes' is neutron_vpnaas_agent.volumes not neutron_vpnaas_agent.image  https://review.openstack.org/45750508:14
spsuryayou already know, i think08:14
*** iceyao has joined #openstack-kolla08:15
zhubingbing_wait08:15
zhubingbing_i give you a link08:15
*** iceyao has quit IRC08:17
spsuryazhubingbing_: sure08:17
*** iceyao has joined #openstack-kolla08:18
*** jascott1 has quit IRC08:19
*** MasterOfBugs has quit IRC08:19
*** jascott1 has joined #openstack-kolla08:19
hrwmorning08:20
spsuryahrw: morning08:21
zhubingbing_https://github.com/kubernetes/kubernetes/issues/4381508:21
hrwtime to check patches and refresh what is needed08:21
zhubingbing_spsurya08:21
*** dmk0202 has joined #openstack-kolla08:22
spsuryazhubingbing_: seems issue has been closed08:22
spsurya;)08:23
*** jascott1 has quit IRC08:24
*** iceyao has quit IRC08:26
*** zhurong has joined #openstack-kolla08:26
openstackgerritChen proposed openstack/kolla master: Use LABEL instead of MAINTAINER (deprecated) in Dockerfile.j2  https://review.openstack.org/45751308:27
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Support non-x86 architectures (aarch64, ppc64le)  https://review.openstack.org/43094008:31
*** openstackgerrit has quit IRC08:33
*** openstackgerrit has joined #openstack-kolla08:33
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Add trustee_domain_id in magnum.conf  https://review.openstack.org/44843808:33
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Add trustee_domain_id in magnum.conf  https://review.openstack.org/44843808:35
magicboizgood morning08:36
egonzalezmagicboiz, sup08:36
magicboizgood thx!08:36
magicboizegonzalez, any experience with trove?08:36
egonzaleznope08:37
magicboiz:(08:37
openstackgerritzhangdongqi proposed openstack/kolla-ansible master: neutron_vpnaas_agent: kolla_docker parameter 'volumes' is wrong.  https://review.openstack.org/45750508:37
magicboizwould you happen to know someone here?? :)08:38
egonzalezmagicboiz, post the question and maybe someone can answer08:40
magicboizwell, I have deployed trove in (ocata/ubuntu/source/tls) and I'm facing a trove strange error: I cannot create a trove instance, I'm getting these kind of messages: Unable to establish connection to http://0.0.0.0:5000/v2.0/tokens: HTTPConnectionPool(host='0.0.0.0', port=5000): Max retries exceeded with url: /v2.0/tokens (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection ob...08:41
*** Serlex has joined #openstack-kolla08:42
magicboizdigging into logs, I see that trove is asking Nova for an instance-id "None" which is wrong...."GET call to compute for http://192.168.12.254:8774/v2/b89517e0f2bb4561b6e2cd2fb03b66cf/servers/None"08:42
magicboizERROR trove.instance.models [-] Could not find nova server_id(None).08:43
magicboizit's strange. I cannot even delete failed trove instances.....¿?08:43
egonzalezmagicboiz, may be trove bug https://bugs.launchpad.net/trove/+bug/167165508:47
openstackLaunchpad bug 1671655 in OpenStack DBaaS (Trove) "trove_auth_url is not populated properly everywhere" [Medium,In progress] - Assigned to Erik Olof Gunnar Andersson (eandersson)08:47
egonzalezOSA have the same issue and are pointing to trove bugs https://bugs.launchpad.net/openstack-ansible/+bug/164768108:47
openstackLaunchpad bug 1647681 in openstack-ansible "trove error during publish_exists_event" [Undecided,In progress] - Assigned to Donovan Francesco (donovan-francesco)08:47
magicboizegonzalez, like always, thanks!08:50
magicboizhttps://review.openstack.org/#/c/455463/08:50
openstackgerritzhangdongqi proposed openstack/kolla-ansible master: neutron_vpnaas_agent: kolla_docker parameter 'volumes' is wrong.  https://review.openstack.org/45750508:51
*** newmember has quit IRC08:53
*** mgoddard has joined #openstack-kolla08:53
*** newmember has joined #openstack-kolla08:54
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Support non-x86 architectures (aarch64, ppc64le)  https://review.openstack.org/43094008:56
*** sayantan_ has quit IRC09:00
*** sayantan_ has joined #openstack-kolla09:00
*** gfidente has joined #openstack-kolla09:01
*** gfidente has quit IRC09:01
*** gfidente has joined #openstack-kolla09:01
openstackgerritDuong Ha-Quang proposed openstack/kolla-kubernetes master: Heat service deployment  https://review.openstack.org/43725209:03
*** derekjhyang has quit IRC09:03
*** sayantan_ has quit IRC09:06
*** Margin has joined #openstack-kolla09:10
*** sambetts|afk is now known as sambetts09:19
*** guoguo has quit IRC09:26
sdaketervancovan need some help?09:27
openstackgerritMerged openstack/kolla-ansible master: Cleanup /run/netns host directory volume mount  https://review.openstack.org/45417909:32
tervancovanyes sdake09:34
tervancovani need some help~!09:34
sdaketervancovan tryign to deploy kolla-kubernetes?09:35
tervancovanyes right09:35
sdaketervancovan ran through https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html09:35
tervancovanSo I have checked deployment-guide.rst in kolla-kuberenetes09:36
sdakeok - i guess itwould help if the steps were numbered09:36
sdakewhich step are you on - step 1 or 2 or 3?09:36
*** chas has quit IRC09:37
tervancovanI am in step 209:37
sdakevalidating kubernetes?09:37
tervancovanyes right09:37
sdakecan you paste the output you see on your screen during that nslookup operation?09:37
*** chas has joined #openstack-kolla09:37
tervancovanhttps://thepasteb.in/p/Vmh03Q0vxqXh409:40
openstackgerritshaofeng cheng proposed openstack/kolla master: Add oslo-vmware module in glance_base image  https://review.openstack.org/45716309:41
*** chas has quit IRC09:42
spsuryasdake: for DNS configuration with CIDR what should i put in /etc/systemd/system/kubelet.service.d/10-kubeadm.conf if my centos 7 VM ip is 192.168.121.45/24 ?09:42
*** zhurong has quit IRC09:42
sdakespsurya in that case, the pod CIDR and service CIDRs need not be modified09:43
sdakethe DNS server comes from the service CIDR09:43
openstackgerritshaofeng cheng proposed openstack/kolla master: Add oslo-vmware module in glance_base image  https://review.openstack.org/45716309:43
sdakeWarning This documentation assumes a POD CIDR of 10.1.0.0/16 and a service CIDR of 10.3.3.0/24. Two rules must be followed when reading this guide.09:44
sdakeThe service and pod cidr cannot overlap09:44
sdakeThe address spaces cannot already be allocated by your organization09:44
sdakeIf the POD and CIDR addresses overlap in this documentation with your organizations’s IP address ranges, they may be changed. Simply subtitute anywhere these addresses are used with the custom cidrs you hae chosen.09:44
sdakein this case since your a vm, the pod and service cidrs never overlap and are not allocated by your org09:44
sdakeyour in a vm09:44
spsuryaroger09:45
sdaketervancovan you did not follow the instructions :)09:51
sdaketervancovan the kubeadm defaults are 10.9616 fro the service cidr09:51
sdake10.96/16 for th eservice cidr09:51
sdaketervancovan i suspect you didn't change the dns server09:51
sdaketervancovan please cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf09:52
sdake-> paste09:53
*** skramaja has quit IRC09:56
*** sayantan_ has joined #openstack-kolla10:02
*** shardy has quit IRC10:06
*** sayantan_ has quit IRC10:07
*** Margin has quit IRC10:10
*** cuongnv has quit IRC10:12
spsuryasdake: http://paste.openstack.org/show/606990/10:12
spsuryacleaned many times10:12
*** tovin07_ has quit IRC10:13
*** asmaa has joined #openstack-kolla10:13
sdakespsurya -> [root@localhost vagrant]# CGROUP_DRIVER=$(sudo docker info | grep "Cgroup Driver" | awk '{print $3}')10:14
sdakeyou see this command is failing10:14
sdakespsurya prior to that operation10:14
sdake(the docs may be wrong)10:15
sdakespsurya can you clean up10:15
sdakeand stop prior to that CGRUP_DRIVER operation plz10:15
sdakeand wait for further instruction10:15
spsuryaroger10:15
sdakeCannot connect to the Docker daemon. Is the docker daemon running on this host?10:15
sdakethis is how i know there is  an error10:15
sdakefirst error is what we need to debug10:15
sdakeI know exacty what the problem is10:15
spsurya sdake once i installed  docker with      `curl -sSL https://get.docker.io | bash`10:18
*** chas has joined #openstack-kolla10:18
sdakespsurya - not correcct10:18
sdakespsurya please undo that10:18
*** chas has quit IRC10:19
*** chas_ has joined #openstack-kolla10:19
sdakeupstream docker does not work with kubeadm10:19
spsuryayes10:19
sdakespsurya reset to clean environment, then stop prior to the CGRUOP_DRIVER operation10:20
spsuryasure10:21
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Allow use of any backup backend for any volume backend  https://review.openstack.org/45755410:21
sdakespsurya when you get here -> sudo yum install -y docker ebtables kubeadm kubectl kubelet kubernetes-cni10:22
sdakestop10:22
sdakespsurya inbetween 63 and 69, i don't see that you have run one command as specified the deployment guide10:22
sdakeactually two commands10:23
*** sayantan_ has joined #openstack-kolla10:23
spsuryamoment10:24
spsuryadestroying and vagrant up10:24
*** yangyapeng has joined #openstack-kolla10:24
sdakek10:24
spsuryanow trying from fresh10:24
sdakespsurya would you be kindenough to share our vagrantfile10:24
sdakeour/your10:24
spsuryasure10:25
sdakei'll try on my end10:25
*** skramaja has joined #openstack-kolla10:25
sdaketervancovan where u at dude ;)10:27
*** sayantan_ has quit IRC10:27
spsuryasdake:  vagrant file              http://paste.openstack.org/show/606991/10:28
*** shardy has joined #openstack-kolla10:29
spsuryasdake: now at --->>>  sudo cp -a kubernetes.repo /etc/yum.repos.d10:31
spsuryanow at * sudo yum install -y docker ebtables kubeadm kubectl kubelet kubernetes-cni10:32
tervancovanThanks sdake10:35
*** rmart04 has joined #openstack-kolla10:35
sdaketervancovan did you get running?10:35
sdakespsurya ok - now after that step - what is the enxt step :)10:35
*** chas has joined #openstack-kolla10:36
tervancovanNot yet I am just back from dinner10:36
*** gfidente has quit IRC10:36
tervancovanSo The thing that I have to do is fix configuration in docker container of kubeadm-aio10:37
sdaketervancovan ok - well looking at your output, I notice your dns is not configured correctly10:37
tervancovanFor kolla-kubernetes CIDR rule10:37
sdaketervancovan which would indicate a direction was skipped ;)10:37
tervancovanyes right10:37
*** duonghq has quit IRC10:38
spsuryasdake: after that this one --->>> CGROUP_DRIVER=$(sudo docker info | grep "Cgroup Driver" | awk '{print $3}')10:38
*** chas_ has quit IRC10:38
sdakespsurya no- prior to that step10:39
sdakespsurya click "RELOAD" on the web browser10:39
sdakespsurya it is possibel you ahve old docs cached10:39
spsuryasdake: you are right10:41
spsuryawas at old one ;)10:41
*** chas_ has joined #openstack-kolla10:41
sdakespsurya would you submit a docs change and add a warning tag at the top that says "these docs are under rpaid change.  Please use the reload button before running through these instructions"10:42
sdakeor something similar10:42
sdakespsurya there is  already a warning the docs dont'work10:42
sdakeI think that canbe removed as the docs DO work10:42
sdakeor altered10:42
spsuryasdake  sure10:43
spsuryai will10:43
sdaketervancovan this may apply toy ou as well10:43
*** chas_ has quit IRC10:43
*** chas has quit IRC10:44
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Cleanup /lib/modules volume mounts in containers  https://review.openstack.org/45421410:45
tervancovansdake I think this  command should be applyed --->"sudo kubeadm init --pod-network-cidr=10.1.0.0/16 --service-cidr=10.3.3.0/24"10:47
spsuryasdake: should we be ok with the warning Usage of loopback devices  ?10:47
sdakespsurya i dont follow :)10:48
sdaketervancovan ithat is what the documents recommend10:49
spsuryasdake i mean http://paste.openstack.org/show/606992/10:49
sdakespsurya ya - we can ignore that for now10:49
sdakespsurya that is mainly a warning not an error10:50
*** jaosorior has quit IRC10:55
tervancovanroot@hosoup-ubuntu-64:~#  kubeadm init --pod-network-cidr=10.1.0.0/16 --service-cidr=10.3.3.0/2410:55
tervancovan[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.10:55
tervancovan[init] Using Kubernetes version: v1.6.010:55
tervancovan[init] Using Authorization mode: RBAC10:55
tervancovan[preflight] Running pre-flight checks10:55
*** tervancovan has quit IRC10:55
spsuryasdake worked :)10:55
sdakespsurya good news :)10:55
sdakespsurya which step, step 2?10:56
spsuryaat least got of that waiting10:56
sdakeor step 110:56
openstackgerritjeck proposed openstack/kolla-ansible master: Remove /etc/fstab Swift references during cleanup-host  https://review.openstack.org/45680510:56
*** tervancovan has joined #openstack-kolla10:56
spsuryasdake: 110:56
sdakespsurya on my gige internet that step takes about 30 seconds10:57
sdakespsurya YMMV - would be interesteed to know what i ttakes for you10:57
sdakekubeadm is downloading a slew of containers10:57
spsuryasdake now onwards it will take 29secs for me ;)10:57
sdakespsurya cool10:58
sdakespsurya can you submit that warning about the "click refresh on browser" when you ahve an opportunity10:58
spsuryayes for sure10:58
spsuryasdake10:58
sdakespsurya thanks :)10:59
tervancovanThis is output of   this command in container -->" sudo kubeadm init --pod-network-cidr=10.1.0.0/16 --service-cidr=10.3.3.0/2410:59
tervancovan" https://thepasteb.in/p/JZhpZMQZZMXhg10:59
*** jaosorior has joined #openstack-kolla11:01
sdaketervancovan run kubeadm reset11:05
sdaketervancovan you can't kubeadm followed by kubeadm without a reset inbetween11:05
tervancovanthank you sdake11:05
sdaketervancovan your DNS server was not set proeprly in kubeadm11:05
sdaketervancovan so - if you could please paste the output of :11:05
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Updated the warning message of deployment guide  https://review.openstack.org/45757111:05
sdakeetc/systemd/system/kubelet.service.d/10-kubeadm.conf11:06
*** shardy is now known as shardy_lunch11:06
tervancovanthis is outout of 10-kubeadm.conf  https://thepasteb.in/p/98hRQ7YXBMEsk11:07
sdaketervancovan looks good11:07
sdaketervancovan did you restart kubeadm11:07
sdakesorry11:07
sdakedid you restartkubelet11:07
*** gfidente has joined #openstack-kolla11:08
*** gfidente has quit IRC11:08
*** gfidente has joined #openstack-kolla11:08
sdakesudo systemctl stop kubelet11:08
sdakesudo systemctl enable kubelet11:08
sdakesudo systemctl start kubelet11:08
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Update simple_cell_setup to manual creation  https://review.openstack.org/45692011:09
tervancovan"kubeadm reset" -> https://thepasteb.in/p/RghnEZgYAV7cz11:09
sdaketervancovan run the 3 above sudo commands please11:10
*** zhurong has joined #openstack-kolla11:14
tervancovanI already did it11:16
tervancovanbtw I use "--skip-preflight-checks" this options to kubeadm init11:16
sdaketervancovan don't use that optoin11:17
sdaketervancovan follow the guide exactly11:17
sdaketervancovan do not deviate plz11:17
spsuryasdake http://paste.openstack.org/show/606998/11:19
*** chas has joined #openstack-kolla11:20
*** jascott1 has joined #openstack-kolla11:20
*** jascott1 has quit IRC11:25
sdakespsurya this is what it should look like:11:26
sdake[vagrant@localhost ~]$ kubectl run -i -t $(uuidgen) --image=busybox --restart=Never11:26
sdakeIf you don't see a command prompt, try pressing enter.11:26
sdakeServer:    10.3.3.1011:26
sdakeAddress 1: 10.3.3.10 kube-dns.kube-system.svc.cluster.local11:26
sdakeName:      kubernetes11:26
sdakeAddress 1: 10.3.3.1 kubernetes.default.svc.cluster.local11:26
sdakenote I just run with your vagrant setup11:26
sdakeand it wfm11:26
sdakespsurya the error you ahve is indicative of not changing the dns server11:27
sdakespsurya run kubectl get nodes -> paste11:27
sdakeafter typing exit a few time to get out of that busybox image11:27
spsuryahttps://www.irccloud.com/pastebin/LnOD0u2I/11:28
spsuryasdake11:29
sdakespsurya ok kubeadm get pods -n kube-system -> paste11:29
sdakerathe kubectl get pods -n kube-system -> paste11:29
spsuryahttps://www.irccloud.com/pastebin/Emvvvwsh/11:30
sdakedns 3/311:30
sdakelooks good11:30
spsuryaseems fine too me11:30
sdakeok - cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf11:30
sdakeNote Kubernetes uses x.x.x.10 as the DNS server. The Kolla developers don’t know precisely11:30
spsuryaEnvironment="KUBELET_DNS_ARGS=--cluster-dns=10.3.3.10 --cluster-domain=cluster.local"11:32
spsuryacorrect11:32
sdakeok - that looks good11:32
sdakedid you restart kubelet?11:32
spsuryanot yet11:32
sdakeok11:32
sdakewell -11:32
sdakethese instructions need to be followed in the order specified in the docs:11:33
sdakeThen reload the hand-modified service files:11:33
sdakesudo systemctl daemon-reload11:33
sdakeThen stop kubelet if it is running:11:33
sdakesudo systemctl stop kubelet11:33
sdakeThen enable and start docker and kubelet:11:33
sdakesudo systemctl enable kubelet11:33
sdakesudo systemctl start kubelet11:33
spsuryado we need to update in doc too11:35
spsurya?11:35
sdakethat is what is in the odc11:35
sdakestraight cut and paste from it11:36
sdakehere is cut and past efrom the docs:11:36
sdakeSetup the DNS server with the service CIDR:11:36
sdakesudo sed -i 's/10.96.0.10/10.3.3.10/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf11:36
sdakeNote Kubernetes uses x.x.x.10 as the DNS server. The Kolla developers don’t know precisely why this is the case, however, current speculation is that that 1..9 are reserved for future expansion of Kubernetes infrastructure services.11:36
sdakeThen reload the hand-modified service files:11:36
sdakesudo systemctl daemon-reload11:36
sdakeThen stop kubelet if it is running:11:36
sdakesudo systemctl stop kubelet11:36
sdakeThen enable and start docker and kubelet:11:36
sdakesudo systemctl enable kubelet11:36
sdakesudo systemctl start kubelet11:36
sdakeyou did the sed operation11:36
sdakei am not convinced you did the daemon reload operation or stop kubelet or enable kubelet or start kubelet11:36
spsuryai did11:37
sdakeok11:38
sdakeit could be a timing issue11:38
sdakei just ran through steps 1 and step2 using your vagrantfile11:38
sdakeand it WFM11:38
spsuryanice11:38
sdakeso lets do this11:39
sdakedon't reset a clean environment11:39
sdakeinstead, run cleanup-k8s.sh11:39
spsuryamay be you can take a look on this http://paste.openstack.org/show/606999/11:40
sdakespsurya run a new busybox and try again11:42
sdakewith the nslookup11:42
sdakespsurya i think you are not waiting for kubernetes to come up before running busybox11:43
sdakespsurya although I am not certain11:43
sdakespsurya ok - new recommendation11:44
sdakedestroy the vagrant env11:44
sdakeand do vagrant up again11:44
sdakevagrant ssh11:44
sdakeand start the steps11:44
spsuryamoment11:44
sdakeprior to validation, stop11:44
sdake(e.g. stop at step 2)11:44
sdakei.e. rather11:44
spsuryanow doing destroy and up11:45
*** bjolo has quit IRC11:46
openstackgerritMartin André proposed openstack/kolla master: Ensure tests run in isolation  https://review.openstack.org/45721411:48
pbourkehi sdake, looks like I missed some msgs from you over the weekend?11:51
sdakepbourke just wondering how lifes treating ya :)11:52
pbourkesdake: pretty good :) just back from long easter weekend11:52
*** rwallner has joined #openstack-kolla11:53
*** rwallner has quit IRC11:55
tervancovanThanks for helping sdake11:58
tervancovanI have to go home right now , and I need more trying for it11:59
tervancovantomorrow11:59
*** lujinluo has quit IRC12:01
*** rwallner has joined #openstack-kolla12:01
sdaketervancovan enjoy12:06
*** dave-mccowan has joined #openstack-kolla12:08
spsuryasdake again tainted12:12
spsuryai.e step 112:12
spsuryanow holding12:12
sdakespsurya run command kubectl get pods -n kube-system12:13
spsuryaran12:13
sdakeinto a past e:)12:13
spsuryaall are good12:13
sdakeinto past eplz12:13
*** goldyfruit has quit IRC12:14
spsuryahttps://www.irccloud.com/pastebin/YYYgwapc/12:14
sdakethose are not all good12:14
sdakeyou need to wait for dns to be 3/3 RUNNING12:14
sdakethen you are all good :)12:14
sdakeyou notice dns is 0/3 pending12:14
spsuryayeah12:14
spsuryahow long it takes ;)12:15
sdakedownloading images12:15
spsuryaits 9:15 PM here12:15
sdakeon my gige network it takes 2 minutes to bring up kubernetes12:15
sdakehow fast is your network?12:16
*** shardy_lunch is now known as shardy12:16
spsuryayour ntw is fabulous12:16
sdakeya 99$/mo12:16
sdakenot a bad deal :)12:16
spsuryaits office one12:16
spsuryanow got the dns 3/312:16
*** Margin has joined #openstack-kolla12:17
sdakenow run busybox12:17
spsuryahttps://www.irccloud.com/pastebin/cS5Jf8E2/12:19
spsuryagotch :)12:19
*** salv-orlando has joined #openstack-kolla12:19
spsuryasdake12:20
sdakespsurya ok two actions12:23
sdakewhich are two patches12:23
*** tervancovan has quit IRC12:23
sdake1) add what you learned regarding the fact that you need to refresh the page to th edocument warning at the top12:23
sdake2) add what you learned regarding the fact that you need to wait for kubernetes to come up before proceeding12:23
sdakespsurya I'll provide fast reviews on these and get them through the review queue quickly12:24
sdakethanks :)12:24
*** sayantan_ has joined #openstack-kolla12:24
spsuryafor sure sdake12:24
spsurya1 done12:24
spsurya2 getting it done12:24
*** sayantan_ has quit IRC12:28
*** srwilkers has joined #openstack-kolla12:30
*** lamt has joined #openstack-kolla12:44
*** gkadam has quit IRC12:44
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Deployment guide updated with few more information  https://review.openstack.org/45761412:44
spsuryasdake  https://review.openstack.org/457571   https://review.openstack.org/45761412:45
*** zhurong has quit IRC12:45
spsuryasdake thanks for continuos support12:46
sdakespsurya thank you :)12:46
*** lamt has quit IRC12:46
spsuryawill start from 3rd step once reach home12:47
spsuryahope eyes will be open otherwise tomorrow12:47
*** mbruzek has joined #openstack-kolla12:52
*** StephenWang1991 has quit IRC12:53
*** zhubingbing_ has quit IRC12:58
*** rwallner has quit IRC13:02
*** tonanhngo has joined #openstack-kolla13:03
*** sayantan_ has joined #openstack-kolla13:04
*** sudipto has quit IRC13:04
*** sudipto_ has quit IRC13:04
*** huzhengchuan has quit IRC13:06
*** jaosorior has quit IRC13:06
*** rwallner has joined #openstack-kolla13:07
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: Pinning OS_VOLUME_API_VERSION=2  https://review.openstack.org/45762613:08
*** tonanhngo has quit IRC13:08
*** goldyfruit has joined #openstack-kolla13:11
*** Pavo has joined #openstack-kolla13:12
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: Adding kolla_kubernetes_external_subnet var  https://review.openstack.org/45762713:13
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Use ansible from within the kolla_toolbox virtualenv  https://review.openstack.org/43502413:13
sdakespsurya i left reviews on your patches13:14
sdakespsurya i think they could use a small bit of tidying13:14
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla master: Reparent kolla-toolbox from openstack-base  https://review.openstack.org/43502313:14
*** jaosorior has joined #openstack-kolla13:14
*** dmsimard|afk is now known as dmsimard13:15
*** esharao has joined #openstack-kolla13:17
*** mbruzek has quit IRC13:17
*** jtriley has joined #openstack-kolla13:17
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla master: Update oraclelinux tag to new slim base image  https://review.openstack.org/43732913:20
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Add fwaas extension in l3 agent when neutron fwaas is enabled  https://review.openstack.org/45763113:20
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Only start one glance_api when using file backend  https://review.openstack.org/44865413:20
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Fix qos rules are not applied on ocata with ovs  https://review.openstack.org/45763313:24
*** pcaruana has quit IRC13:27
openstackgerritzhangdongqi proposed openstack/kolla-ansible master: neutron_vpnaas_agent: kolla_docker parameter 'volumes' is wrong  https://review.openstack.org/45750513:35
*** eanylin has quit IRC13:39
*** mbruzek has joined #openstack-kolla13:41
*** iceyao has joined #openstack-kolla13:47
*** lamt has joined #openstack-kolla13:49
*** schwicht has joined #openstack-kolla13:50
*** MarginHu2017 has joined #openstack-kolla13:53
*** al498u has joined #openstack-kolla13:55
*** Margin has quit IRC13:56
*** al498u_ has joined #openstack-kolla13:57
sdake0.6.0 is done and in the process of being tagged: https://launchpad.net/kolla-kubernetes/+milestone/0.6.014:00
*** al498u has quit IRC14:01
*** lamt has quit IRC14:02
sdake0.7.0 in progress - deadline may 15th: https://launchpad.net/kolla-kubernetes/+milestone/0.7.014:03
*** lamt has joined #openstack-kolla14:05
*** sudipto_ has joined #openstack-kolla14:08
*** sudipto has joined #openstack-kolla14:08
*** cuongnv has joined #openstack-kolla14:08
*** pcaruana has joined #openstack-kolla14:10
*** StephenWang1991 has joined #openstack-kolla14:13
*** sayantan_ has quit IRC14:13
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Version 0.7.0  https://review.openstack.org/45765714:18
*** sayantan_ has joined #openstack-kolla14:20
*** sayantan_ has quit IRC14:21
openstackgerritzhangdongqi proposed openstack/kolla-ansible master: neutron_vpnaas_agent: kolla_docker parameter 'volumes' is wrong  https://review.openstack.org/45750514:23
*** dave-mccowan has quit IRC14:24
*** MarginHu2017 has quit IRC14:25
openstackgerritMerged openstack/kolla-ansible master: Use ironic-inspector user for ironic inspector  https://review.openstack.org/44558714:30
*** unicell1 has joined #openstack-kolla14:34
*** huzhengchuan has joined #openstack-kolla14:34
*** unicell has quit IRC14:34
*** sudipto_ has quit IRC14:38
*** sudipto has quit IRC14:38
*** tovin07 has joined #openstack-kolla14:44
*** dave-mccowan has joined #openstack-kolla14:44
*** tovin07 has left #openstack-kolla14:46
*** iceyao has quit IRC14:47
*** MarginHu2017 has joined #openstack-kolla14:49
*** MarginHu2017 has quit IRC14:50
spsuryasdake: ok14:50
*** zhubingbing_ has joined #openstack-kolla14:53
*** manheim has quit IRC14:56
openstackgerritMerged openstack/kolla-ansible master: Set fluentd output to elasticsearch to use elasticsearch_address  https://review.openstack.org/44765414:56
*** mbruzek has quit IRC14:58
*** StephenWang1991 has quit IRC15:03
*** srwilkers has quit IRC15:03
*** srwilkers has joined #openstack-kolla15:06
*** blallau has joined #openstack-kolla15:07
*** Serlex has quit IRC15:08
*** StephenWang1991 has joined #openstack-kolla15:13
*** iceyao has joined #openstack-kolla15:14
sdakekfox1111 portdirect srwilkers if any of you ahve a moment - this is good for merge: https://review.openstack.org/#/c/457048/15:15
*** skramaja has quit IRC15:16
*** StephenWang1991 has quit IRC15:17
sbezverkit is already merged?!?!?15:17
sbezverkdo you want double merge now?15:17
*** StephenWang1991 has joined #openstack-kolla15:17
sbezverkonly one merge counts against stackalitics15:17
sdakesbezverk moment15:17
sdakesbezverk oh wierd - it wasn't merged a bit ago15:17
sdakesbezverk perhaps gerrit returned me the wrong results - or perhaps i didn't look at the correct review15:18
sdakesbezverk ya - that is wrong link15:18
sbezverkkfox1111: please ping me when you have a chance15:19
*** iceyao has quit IRC15:19
sdakesbezverk ya - wrong review - my bad: the correct one is: https://review.openstack.org/#/c/451495/15:19
sdakesbezverk also - stackalytics can DIAF :)15:19
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Updated the warning message of deployment guide  https://review.openstack.org/45757115:20
*** jascott1 has joined #openstack-kolla15:21
*** rmart04 has quit IRC15:22
*** manheim has joined #openstack-kolla15:22
*** huzhengchuan has quit IRC15:23
*** jascott1 has quit IRC15:25
*** manheim has quit IRC15:27
openstackgerritMerged openstack/kolla-kubernetes master: Updated the warning message of deployment guide  https://review.openstack.org/45757115:27
*** tonanhngo has joined #openstack-kolla15:29
sbezverkkolla-kubernetes cores, please check a couple of PS, minor issues found and addressed based on mutlinode cluster deployment: https://review.openstack.org/457626  and https://review.openstack.org/45762715:31
openstackgerritMerged openstack/kolla-kubernetes master: Nova-control service chart dependencies  https://review.openstack.org/45149515:33
kfox1111morning.15:33
kfox1111interesting.... whats required to support cinder v3?15:34
*** chas has quit IRC15:34
*** chas has joined #openstack-kolla15:35
sdakesuper kfox111115:37
*** huzhengchuan has joined #openstack-kolla15:38
*** chas has quit IRC15:39
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla master: Reparent kolla-toolbox from openstack-base  https://review.openstack.org/43502315:39
sdakeportdirect thanks15:41
*** pcaruana has quit IRC15:41
sbezverkkfox1111: cinder client when not pinned to v2 tries to find end poit cinderv315:43
sbezverkand fails since we do not support it15:43
kfox1111its just an entry in keystone pointing back to the regular cinder server? or is it something more?15:44
sbezverkkfox1111: it is keystone end point, remember we have cinder and cinderv215:45
sbezverknow it appears we need cinderv315:45
kfox1111so, the real fix is cp -a cinderv2/cinderv3 | sed 's/v2/v3/' ?15:45
kfox1111fun.15:45
sbezverkvolumev215:45
sbezverkno, we need to add volumev2 and volumev315:46
kfox1111ah. that too.15:46
sbezverkI mean add only volumev315:46
openstackgerritMerged openstack/kolla-kubernetes master: Adding kolla_kubernetes_external_subnet var  https://review.openstack.org/45762715:46
sbezverkkfox1111: are you busy now? I came accross really wierd issue in mutlinode, need a second pair of eyes..15:46
kfox1111yeah, I've got a bit. whats up?15:47
sbezverkkfox1111: check this out: http://paste.openstack.org/show/607020/15:47
*** matrohon has quit IRC15:47
sbezverkkfox1111: remember we added a check to openvswitch-agent to wait for opensvwitch to be operations15:48
kfox1111hmm...15:48
kfox1111is this in gate, or on your own box?15:48
*** salv-orl_ has joined #openstack-kolla15:49
sbezverkkfox1111: it is my 5 nodes cluster I am building summit demo15:49
kfox1111ah.15:49
kfox1111did you remember to turn off cri on all nodes?15:49
kfox1111(we should document that somewhere... )15:49
sbezverkkfox1111: DAMN!!!!!!!15:50
*** magicboiz has quit IRC15:50
kfox1111yeah.... I'm disapointed they enabled the beta cri by default. :/15:51
sbezverkkfox1111: :) thanks a lot for pointing it out15:51
kfox1111though I totally get why.15:51
kfox1111np. :)15:51
*** salv-orlando has quit IRC15:51
kfox1111docker inc renaming docker has been a huge pain.15:52
*** magicboiz has joined #openstack-kolla15:52
kfox1111I've started to have to educate people that when they say docker, they really mean containerd now.15:52
kfox1111they don't want "docker" anymore.15:52
*** mgoddard has quit IRC15:54
*** iceyao has joined #openstack-kolla15:54
*** jascott1 has joined #openstack-kolla15:55
*** rwallner has quit IRC15:55
sbezverkkfox1111: :)15:55
sdakekfox1111 +10015:56
sbezverkkfox1111: appears nothing want to make our life easier15:56
*** magicboiz has quit IRC15:56
kfox1111sbezverk: thats why we make the big bucks. ;)15:56
kfox1111or something like that....15:56
kfox1111:)15:56
*** gfhellma has joined #openstack-kolla15:57
sbezverkkfox1111: that was it.. now openvswitch-agent is very happy15:59
*** iceyao has quit IRC15:59
kfox1111cool.15:59
*** jascott1 has quit IRC16:00
*** rwallner has joined #openstack-kolla16:01
*** jtriley_ has joined #openstack-kolla16:02
*** inc0 has joined #openstack-kolla16:02
kfox1111wow... this order is coming by snail. may 5-26. :)16:04
*** jtriley has quit IRC16:05
*** egonzalez has quit IRC16:05
*** blallau has quit IRC16:05
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Deployment guide updated with few more information  https://review.openstack.org/45761416:08
*** magicboiz has joined #openstack-kolla16:08
inc0good morning16:08
*** rwallner has quit IRC16:12
*** salv-orl_ has quit IRC16:14
spsuryainc0: morning16:15
*** inc0 has quit IRC16:16
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Add panko.conf file in ceilometer container  https://review.openstack.org/44471916:17
*** cuongnv has quit IRC16:17
*** sudipto_ has joined #openstack-kolla16:18
*** sudipto has joined #openstack-kolla16:18
*** StephenWang1991 has quit IRC16:21
sbezverkkfox1111: that pinning PS, are you ok to ack it? adding volumev3 atm does not seem to be an option due to time constraints16:22
kfox1111sbezverk: yeah, but can you file a blueprint so we dont forget?16:23
sdakeand link so I can approve it plz16:23
sdakemorning inc016:23
sbezverksdake: you cannot approve to times ;)16:24
*** gfidente has quit IRC16:24
*** inc0 has joined #openstack-kolla16:24
sbezverkkfox1111: techdebt or blue-print?16:24
kfox1111I guess its a blue-print as its a new feature kind of?16:25
kfox1111not really sure.16:26
sbezverkkfox1111: blueprint is done16:27
kfox1111sbezverk: thanks.16:27
*** jascott1 has joined #openstack-kolla16:29
openstackgerritMerged openstack/kolla-ansible master: Add fwaas extension in l3 agent when neutron fwaas is enabled  https://review.openstack.org/45763116:30
openstackgerritMerged openstack/kolla-ansible stable/ocata: Fix qos rules are not applied on ocata with ovs  https://review.openstack.org/45763316:31
*** StephenWang1991 has joined #openstack-kolla16:31
inc0kfox1111, so reading your mail about tags, I agree with you but what you're asking for is overhaul of build system really16:33
kfox1111possibly?16:34
inc01. develop rock-solid version manifesto system16:34
kfox1111I think initially, I can do it by just retagging before upload.16:34
inc02. check existing version manifesto16:34
*** mgoddard has joined #openstack-kolla16:34
inc03. determine differences16:35
kfox1111yeah.16:35
inc04. tag accordingly and upload16:35
openstackgerritMerged openstack/kolla-kubernetes master: Pinning OS_VOLUME_API_VERSION=2  https://review.openstack.org/45762616:35
kfox1111so, I'm doing a bit of it now:16:35
kfox1111http://tarballs.openstack.org/kolla-kubernetes/gate/containers/16:35
kfox1111the list of versioned containers is here: http://tarballs.openstack.org/kolla-kubernetes/gate/containers/centos-binary-2-ceph-containers.txt16:36
*** StephenWang1991 has quit IRC16:36
kfox1111and here: http://tarballs.openstack.org/kolla-kubernetes/gate/containers/kubernetes-containers.txt16:36
kfox1111if it ends up pulling differently versioned containers, it updates the tarballs.16:36
*** StephenWang1991 has joined #openstack-kolla16:36
kfox1111we would need a refined version that had revisions and a container fingerprint.16:37
kfox1111something like 'rpm -qa | sort | md5sum' in the container.16:37
kfox1111if a freshly built one had a different fingerprint, its a canditate for updating. it should be tested at that point, and if passes, released with revision+1.16:38
kfox1111bbiab16:39
*** strigazi is now known as strigazi_AFK16:39
*** strigazi_AFK is now known as strigazi16:41
*** dave-mccowan has quit IRC16:41
*** StephenWang1991 has quit IRC16:41
*** zhubingbing_ has quit IRC16:42
inc0kfox1111, but that will be the case every time16:42
inc0for source builds at least16:42
inc0and binary builds aren't voting in gates16:43
*** jaosorior has quit IRC16:43
*** mbruzek has joined #openstack-kolla16:45
*** dave-mccowan has joined #openstack-kolla16:45
*** ksumit has joined #openstack-kolla16:46
*** harlowja_ has joined #openstack-kolla16:50
*** harlowja has quit IRC16:52
*** dmk0202 has quit IRC16:52
*** ssurana has joined #openstack-kolla16:52
*** unicell1 has quit IRC16:55
*** rwallner has joined #openstack-kolla16:59
*** rwallner_ has joined #openstack-kolla17:01
*** rwallne__ has joined #openstack-kolla17:02
*** rwallne__ has quit IRC17:03
*** rwallne__ has joined #openstack-kolla17:03
*** rwallner has quit IRC17:04
*** rwallner_ has quit IRC17:05
*** mbruzek has quit IRC17:17
*** tonanhngo has quit IRC17:21
*** jascott1 has quit IRC17:21
*** gfhellma_ has joined #openstack-kolla17:26
*** gfhellma has quit IRC17:29
*** shardy has quit IRC17:30
openstackgerritMarcus Williams proposed openstack/kolla master: Add OpenDaylight Container to kolla  https://review.openstack.org/41636917:32
*** chas has joined #openstack-kolla17:36
*** jtriley_ has quit IRC17:36
openstackgerritMarcus Williams proposed openstack/kolla master: Add OpenDaylight Container to kolla  https://review.openstack.org/41636917:36
*** unicell has joined #openstack-kolla17:37
openstackgerritMarcus Williams proposed openstack/kolla master: Add OpenDaylight Container to kolla  https://review.openstack.org/41636917:39
*** chas has quit IRC17:40
*** jascott1 has joined #openstack-kolla17:43
*** inc0 has quit IRC17:43
*** jascott1 has quit IRC17:44
*** StephenWang1991 has joined #openstack-kolla17:46
*** rwallne__ has quit IRC17:49
*** japestinho has quit IRC17:50
*** StephenWang1991 has quit IRC17:51
*** jascott1 has joined #openstack-kolla17:52
*** rwellum has joined #openstack-kolla17:59
*** srwilkers has quit IRC18:01
*** rwallner has joined #openstack-kolla18:06
*** sudipto has quit IRC18:07
*** sudipto_ has quit IRC18:07
*** gfhellma_ has quit IRC18:07
*** sambetts is now known as sambetts|afk18:08
openstackgerritMark Goddard proposed openstack/kolla stable/ocata: Add ironic-ui in horizon binary install  https://review.openstack.org/45775518:09
*** eaguilar has joined #openstack-kolla18:17
*** jascott1 has quit IRC18:21
*** jascott1 has joined #openstack-kolla18:22
*** inc0 has joined #openstack-kolla18:23
inc0back18:23
*** jascott1 has quit IRC18:25
sdakeinc0 since your back - mind hitting the review queue18:29
sdakeinc0 we have a bunch of stuff that is pretty easy to review and needs just needs a w+18:30
inc0omw18:30
sdaketx :)18:30
*** eaguilar has quit IRC18:32
*** eaguilar_ has joined #openstack-kolla18:32
sdakekfox1111 sbezverk can we have that cells discussion now18:34
sdaketell me what you want and i'll make it happen :)18:35
*** mgoddard has quit IRC18:37
*** srwilkers has joined #openstack-kolla18:39
*** rwallner has quit IRC18:42
*** tonanhngo has joined #openstack-kolla18:43
*** tonanhngo has quit IRC18:43
*** matrohon has joined #openstack-kolla18:57
*** rwallner has joined #openstack-kolla18:57
*** gfhellma has joined #openstack-kolla19:00
*** Manheim has joined #openstack-kolla19:01
Manheimhello19:04
*** dave-mccowan has quit IRC19:06
*** rwallner has quit IRC19:08
*** ksumit has quit IRC19:09
*** ksumit has joined #openstack-kolla19:09
*** rwallner has joined #openstack-kolla19:10
*** rwallner has quit IRC19:12
*** inc0_ has joined #openstack-kolla19:13
*** rwallner has joined #openstack-kolla19:15
*** rwallner_ has joined #openstack-kolla19:16
*** mbruzek has joined #openstack-kolla19:16
*** rwallner has quit IRC19:19
*** jascott1 has joined #openstack-kolla19:25
kfox1111sdake: I think the idea is, we don't put any ordering of when cell setup is performed.19:28
kfox1111instead, we do a nova-api-cell-discover-host-job.19:29
kfox1111maybe explicit outside the service chart?19:30
kfox1111not sure there's a non racy way to do it inside the service chart.19:30
kfox1111cause you never know if you got all the nova compute's up.19:30
*** dave-mccowan has joined #openstack-kolla19:32
sdakekfox1111 how would one do computekit in that model19:33
sbezverkkfox1111: I wanted to build service chart with to parts19:33
sbezverkcell0 prep/create and discover19:33
kfox1111sdake: two part. helm install compute-kit.... make sure all computes are up/ready, helm install nova-api-cell-discover-host-job19:33
kfox1111the other way to do it requires something like an operator...19:34
sbezverkin this case it looks like service and almost independent from other things19:34
kfox1111you need to know what nodes are suposed to be running nova-compute,19:34
kfox1111and to know if its showing in nova service list.19:34
kfox1111when all are there, you can run the discover-host-job.19:34
kfox1111maybe that functionality could be added to discover-host-job itself conditionally?19:35
*** pbourke has quit IRC19:36
*** pbourke has joined #openstack-kolla19:36
*** chas has joined #openstack-kolla19:37
*** jascott1 has quit IRC19:37
kfox1111part of the problem is there are two sources of knowlege to deay running the discover-host-job.19:37
kfox1111one's k8s, and the other is nova.19:37
kfox1111its probably non trivial to get that logic right.19:39
sdake vt19:39
kfox11111. kubectl get nodes, check for compute=x. count them.19:40
kfox11112. run nova service-list | grep nova-compute | wc -l19:40
sdake55rf19:40
kfox1111if !=, wait a bit and go back to step 1.19:40
*** chas has quit IRC19:41
*** magicboiz has quit IRC19:41
sbezverkkfox1111: I do not get why we need this complexity?19:44
sbezverkdiscovery does not need specific nodes19:44
sbezverkit will discover themm..19:44
*** jascott1 has joined #openstack-kolla19:44
kfox1111sbezverk: it only can discover them, if they are up and running.19:45
sbezverkbasicallly whatever gets registered19:45
kfox1111right. but for helm install compute-kit,19:45
kfox1111folks would assume they can laucnh a vm once its done.19:45
kfox1111but they need to wait for computes to finish spawning, then run discover-host-job.19:46
sbezverkwhy not to run it as a separate step??19:46
sbezverkfrankly I do not think it is a problem, especially since nobody will run compute kit with 2.0.019:46
sbezverkimage19:47
sbezverkso prod people will start using from version 1.0.019:47
kfox1111thats ok too.. we could say, helm install compute kit.... then wait until nova service-list looks correct to you , then run helm nova-api-discover-host-job19:47
sbezverkand good chance it will be 4.0.0 images or higher19:47
sbezverkkfox1111: exactly19:47
sbezverkthat if how I thought when proposed separate19:48
sbezverkservice chart including discovery job19:48
kfox1111I don't htink it needs a seperate service chart though.19:48
kfox1111nova-controller should be enough?19:48
sbezverkhmm, it is closer to compute19:49
kfox1111maybe we're discussing idfferent things.19:49
sbezverksince it is kind deoends19:49
kfox1111I'm suggesting,19:49
*** iceyao has joined #openstack-kolla19:50
kfox1111cell0-simple-setup belongs in nova-controller, and nova-compute and nova-api-cell-discover-host-job are seperate things?19:50
kfox1111you launch nova-controller/nova-compute, then wait for thigns to settle, then run the third one.19:50
kfox1111or if you do computekit, it does the first 2 together.19:50
sbezverkone way or another it will work19:51
kfox1111I think sdake was hoping to get the discover-host-job into the compute-kit too thoguh. but like I was saying, its a fair amount of logic work.19:51
sbezverkwe just need chose it ;)19:51
kfox1111my vote would be to throw nova-cell0-create-db-job in nova-controler service package, and add no deps on nova-compute (don't think that works anyway)19:52
kfox1111then add a documentation step to do discover-host-job19:53
jascott1lol http://fabric8.io/presentations/jfall-2016-nov/images/say-microservices.jpg19:53
*** david-lyle has joined #openstack-kolla19:53
*** ksumit has quit IRC19:53
kfox1111jascott1: hehe.19:54
jascott1was looking for k8s diagrams and found that19:54
*** iceyao has quit IRC19:54
kfox1111hehe19:54
kfox1111meeting next.19:55
kfox1111jascott1: almost got flannel deploying with helm. :)19:55
jascott1nice19:55
kfox1111one more minor thing I think.19:55
kfox1111theres another change I think needed to helm, but I'll drop the info in the review.19:55
kfox1111gotta head to next meeting.19:55
jascott1ok19:55
kfox1111wanted to make sure I got the whole thing working end to end.19:55
kfox1111there was a feature regression in 1.6 as aposed to 1.5 where I tested the concept.19:56
kfox1111I found a workaround though.19:56
kfox1111bbiab19:56
*** chas has joined #openstack-kolla19:57
openstackgerritMarcus Williams proposed openstack/kolla master: Add OpenDaylight Container to kolla  https://review.openstack.org/41636919:58
*** chas has quit IRC20:02
*** ksumit has joined #openstack-kolla20:02
*** srwilkers has quit IRC20:04
*** ksumit has quit IRC20:04
*** Pavo has quit IRC20:09
*** inc0_ has quit IRC20:15
*** chas has joined #openstack-kolla20:18
*** chas has quit IRC20:23
*** rwallner_ has quit IRC20:26
inc0devenv is almost ready20:30
inc0anything anyone wants me to look at before I move on?20:31
*** iceyao has joined #openstack-kolla20:31
*** openstackgerrit has quit IRC20:33
*** gfhellma_ has joined #openstack-kolla20:35
*** iceyao has quit IRC20:36
*** gfhellma has quit IRC20:36
SamYapleinc0: yea20:37
SamYaplecheck this out http://i.imgur.com/IdFtpWC.jpg20:37
inc0SamYaple: saw it before20:38
inc0too much internet.20:38
SamYapleyou just asked if anyone had anything to look at20:39
SamYapleyou didnt specify anything about "new"20:39
*** chas has joined #openstack-kolla20:39
inc0I also didn't say that I won't criticize it after I looked at it20:39
*** matrohon has quit IRC20:42
*** chas has quit IRC20:44
*** salv-orlando has joined #openstack-kolla20:45
*** sean-k-mooney has quit IRC20:46
*** gfhellma__ has joined #openstack-kolla20:46
*** gfhellma_ has quit IRC20:47
*** shasha___ has joined #openstack-kolla20:48
*** david-lyle has quit IRC20:49
*** chas has joined #openstack-kolla21:00
*** schwicht has quit IRC21:03
*** chas has quit IRC21:05
SamYaplehaha21:06
*** david-lyle has joined #openstack-kolla21:10
*** dmk0202 has joined #openstack-kolla21:11
*** spsurya has quit IRC21:13
*** iceyao has joined #openstack-kolla21:14
*** iceyao has quit IRC21:19
*** tonanhngo has joined #openstack-kolla21:21
*** chas has joined #openstack-kolla21:21
*** chas has quit IRC21:25
*** newmember has quit IRC21:25
*** mbruzek has quit IRC21:27
*** ksumit has joined #openstack-kolla21:27
*** eanylin has joined #openstack-kolla21:28
*** shasha___ has quit IRC21:28
*** Manheim has quit IRC21:32
*** eaguilar_ has quit IRC21:37
*** chas has joined #openstack-kolla21:42
*** manheim has joined #openstack-kolla21:44
*** chas has quit IRC21:46
*** manheim has quit IRC21:48
*** salv-orl_ has joined #openstack-kolla21:48
*** manheim has joined #openstack-kolla21:49
*** salv-orlando has quit IRC21:50
*** gfhellma__ has quit IRC21:51
*** MasterOfBugs has joined #openstack-kolla21:55
*** david-lyle has quit IRC21:55
*** iceyao has joined #openstack-kolla21:56
*** gfhellma has joined #openstack-kolla21:57
*** dmk0202 has quit IRC21:59
*** esharao has quit IRC22:00
*** iceyao has quit IRC22:00
*** gfhellma has quit IRC22:01
*** chas has joined #openstack-kolla22:03
*** goldyfruit has quit IRC22:05
*** chas has quit IRC22:07
jascott1why does this work `kubectl taint nodes kube1 node-role.kubernetes.io/master=:NoSchedule`22:10
jascott1but this does not kubectl taint nodes kube1 node-role.kubernetes.io/master=:NoSchedule-22:10
*** Pavo has joined #openstack-kolla22:15
jascott1whered every koala go?22:18
sbezverkjascott1: because to remove label you just specify labe name22:18
sbezverkkubectl taint nodes kube1 node-role.kubernetes.io/master-22:18
sbezverkyou remove label this way22:18
jascott1sbezverk thanks!22:19
jascott1that worked but curios in that would remove all taints22:19
jascott1with that key22:19
jascott1docs say can do just the one22:19
jascott1but thanks, trudging on22:19
jascott1sbezverk docs say http://paste.openstack.org/show/607062/22:20
sbezverkjascott1: sorry I did not get your question?22:23
*** Pavo has quit IRC22:23
sbezverkdo you mean taints from all nodes or something else?22:23
*** chas has joined #openstack-kolla22:23
jascott1no the specific NoSchedule taint22:24
jascott1in my pastebin see the difference22:24
jascott1first is just removing NoSchedule22:24
jascott1second removes all taints with that key22:24
jascott1idk its not critical for me atm but confusing22:24
sbezverkjascott1: I see now what you meant.. frankly I have not played much with this other tham removing it22:26
jascott1ah that removed the taint but I have different taint problem22:27
jascott1No nodes are available that match all of the following predicates:: PodToleratesNodeTaints22:27
*** Pavo has joined #openstack-kolla22:28
*** chas has quit IRC22:28
jascott1so I removed all taints from master, waiting on tiller to retry to schedule and it failed with that error. Maybe it doesnt recheck nodes?22:28
sbezverkjascott1: tiller goes on master without remving taints if I am not mistaken22:31
jascott1im not sure, kfox1111 said something about it floating around the cluster22:32
jascott1ive never payed attention and usually use a multinode setup even if its just vagrant based k8s22:33
*** Pavo has quit IRC22:34
sbezverkjascott1: tiller lands in kube-system namespace22:34
jascott1true22:35
sbezverknot sure if it can actually float arround especially since multimaster model is not yet supported22:35
*** Pavo has joined #openstack-kolla22:35
jascott1sbezverk https://github.com/kubernetes/helm/issues/2222#issuecomment-29456758922:36
*** Pavo has quit IRC22:36
jascott1idk if everything in kube-system runs on master by default?22:37
*** schwicht has joined #openstack-kolla22:37
jascott1i think my untaint didnt work because of the =22:39
*** manheim has quit IRC22:40
sbezverkjascott1: right you do not need =, only taint's name22:42
jascott1mystery solved and I was correct in assuming i was doing something dumb :)22:43
*** chas has joined #openstack-kolla22:44
kfox1111back.22:46
jascott1we were just discussing the finer points of taint removal :)22:47
kfox1111tiller has nothing to pin it to any node, so it floats by default. but if theh master is tainted, it won't land there.22:47
kfox1111hehe. yeah.22:47
jascott1you are correct sir as I have just experienced22:47
kfox1111hence the taint feature request. :)22:48
jascott1i was thinking the existing net-host could go in by itself and I can add the rest in another PR. does that make sense?22:48
*** salv-orl_ has quit IRC22:48
*** chas has quit IRC22:49
kfox1111yeah. though I think the ohter bit may need to go in.22:49
kfox1111so, here's the ps I've been working on:22:49
kfox1111https://review.openstack.org/#/c/456964/22:49
sbezverkkfox1111: in multinode scenario if multiple controllers are exist in kolla, it seems I have to use distributed neutron l3 agent22:50
kfox1111the helm init --net-host would have worked on 1.5, but on 1.6, they have a new behavior.22:50
kfox1111the node is tainted if cni is not started yet.22:50
kfox1111and the kube-scheduler skips scheduling deployments on the node. :/22:51
kfox1111even if net=host.22:51
kfox1111(there is a bug in for that).22:51
kfox1111but if its a DaemonSet and net=host, it works fine.22:51
kfox1111see, https://review.openstack.org/#/c/456964/7/tools/setup_helm.sh22:51
kfox1111lines 17-19.22:51
kfox1111the issue is kind of two part. DaemonSet works, but tiller can only have one instance.22:52
kfox1111so that can be worked around in a DaemonSet by supporting labels, and only labeling one node.22:52
kfox1111sbezverk: you talking dvr or l3-ha?22:53
jascott1but if we just add taints and labels to init we are good?22:53
kfox1111(or both)22:53
*** david-lyle has joined #openstack-kolla22:53
sbezverkkfox1111: yep22:54
kfox1111jascott1: ideally, yes. and thats what they would like to do, but 1.6 doesn't fully support taints/tolerations for cni yet, and the scheculer has some hard coded logic still. :/22:54
jascott1ooooh22:54
jascott1bummer22:54
kfox1111sbezverk: ideally, all new deployments would use dvr + l3ha.22:54
kfox1111jascott1: yeah. :/22:54
sbezverkeven if I use legacy, it starts on both controllers and it does not work well22:54
kfox1111jascott1: thats part of the reason the 1.6.0 release went so badly. :/22:54
kfox1111sbezverk: it should still work. it should register them with seperate agent names, and l3 traffic should be assigned to just one or the other.22:55
kfox1111but failover should take a long time.22:55
*** Reepicheep has quit IRC22:56
*** Reepicheep has joined #openstack-kolla22:56
*** david-lyle has quit IRC22:56
kfox1111jascott1: so, I think to make the net=host thing work as intended, for 1.6 support at least, we need a --daemonset flag and a --label key=value support.22:57
kfox1111the former just sed 's/Deployment/DaemonSet/' and the --node-selector support I mean, not labels.22:57
kfox1111key value set of node labels to add in.22:58
jascott1hmmm im not sure they (helm) will dig the daemonset flag22:58
jascott1since it seems like a workaround22:58
kfox1111not sure. I'd kind of like to use it anyway for one of my production systems.22:58
kfox1111I'm pinning a tiller to a particular host and net=host to make it run within the hosts firewall,22:59
kfox1111preventing others from being able to access it and create/delete pods.22:59
kfox1111will keep needing to do that until k8s/helm authentication is fully implemented.22:59
*** huzhengchuan has quit IRC23:00
kfox1111the daemonset thing will be really nice too when master election is added.23:00
kfox1111just run daemonset and a nodeselector of controller=true, and your ha setup is complete. :)23:01
*** goldyfruit has joined #openstack-kolla23:01
*** StephenWang1991 has joined #openstack-kolla23:02
jascott1kfox1111 hmmm you might want to pitch that on the 2222 issue or open a new one, although I havent had much input from technosophos23:03
kfox1111k.23:03
kfox1111I wonder how much the buyout is sucking up his time.23:03
jascott1shopping for yachts?23:03
jascott1i hope they are all but doubt it23:03
kfox1111anyone have a deb box handy?23:04
jascott1just ubuntu23:04
kfox1111nm... I cna just run it in docker...23:05
kfox1111docker run -it --rm ubuntu /bin/bash :)23:05
*** chas has joined #openstack-kolla23:05
*** StephenWang1991 has quit IRC23:07
jascott1since Intel and Rackspace are no longer together we dont have dev environments. halycon kinda broke on 1.6 so I created a playbook and at least have a dev env now23:07
jascott1the kolla-kubernetes part needs work ;)23:07
kfox1111hows this look for a container fingerprinter:23:08
kfox1111 which deb 2>/dev/null && echo pip && pip freeze | sort; which dpkg 2>/dev/null && echo dpkg && dpkg -l | awk '{print $1, $2, $3, $4}' | sort -k 2; which rpm 2>/dev/null && echo rpm && rpm -qa | sort23:08
jascott1can you add a couple more lines so my eyes will fully bleed out23:08
kfox1111hehe.23:08
kfox1111 which pip 2>/dev/null && echo pip && pip freeze | sort23:09
kfox1111which dpkg 2>/dev/null && echo dpkg && dpkg -l | awk '{print $1, $2, $3, $4}' | sort -k 223:09
kfox1111which rpm 2>/dev/null && echo rpm && rpm -qa | sort23:09
kfox1111so if anything in that output changes between two containers, the container got updated.23:10
*** chas has quit IRC23:10
kfox1111sbezverk: so... what problem are you seeing with multiple l3 agents?23:13
kfox1111I'm running them on one of my clusters and its ok. so maybe there's a misconfiguration somewhere?23:14
jascott1i wonder if they will fix the cni loading thing soon since it was such a problem23:17
*** ksumit has quit IRC23:18
jascott1haha just noticed the github issue count just gives up at 5000+23:25
*** goldyfruit has quit IRC23:26
*** chas has joined #openstack-kolla23:26
*** tonanhngo has quit IRC23:29
*** tonanhngo has joined #openstack-kolla23:29
*** chas has quit IRC23:31
jascott1kfox1111 if you can link me to relevant kubernetes issues I would be grateful.. no rush23:33
*** tonanhngo has quit IRC23:34
*** lamt has quit IRC23:40
*** tonanhngo has joined #openstack-kolla23:40
*** goldyfruit has joined #openstack-kolla23:42
*** manheim has joined #openstack-kolla23:42
*** tonanhngo has quit IRC23:45
*** iniazi has joined #openstack-kolla23:46
*** manheim has quit IRC23:46
*** chas has joined #openstack-kolla23:47
*** chas has quit IRC23:52
*** eaguilar has joined #openstack-kolla23:52
kfox1111jascott1: https://github.com/kubernetes/kubernetes/issues/4381523:53
*** spsurya has joined #openstack-kolla23:53
inc0kfox1111: will you join me on kolla project update session?23:54
spsuryamorning all23:55
inc0I could also use help during hands on workshops for kolla-ansible23:55
*** yangyapeng has quit IRC23:55
*** eanylin has quit IRC23:56
*** al498u_ has quit IRC23:56
*** goldyfruit has quit IRC23:58
kfox1111spsurya: hi. :)23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!