Wednesday, 2016-11-09

openstackgerritWei Cao proposed openstack/kolla: Remove Fedora support from manilla-data  https://review.openstack.org/39407400:00
*** sdake has joined #openstack-kolla00:13
*** portdirect has quit IRC00:16
*** absubram has quit IRC00:18
*** portdirect has joined #openstack-kolla00:19
*** DTadrzak has quit IRC00:19
*** david-lyle has quit IRC00:20
*** yingjun has joined #openstack-kolla00:25
*** neilus has joined #openstack-kolla00:25
*** neilus has quit IRC00:30
*** schwicht has joined #openstack-kolla00:30
openstackgerritWei Cao proposed openstack/kolla: Add karbor container  https://review.openstack.org/39500600:31
Pavoanyone around/00:31
*** schwicht has quit IRC00:34
openstackgerritJoshua Harlow proposed openstack/kolla: Show the rendered docker file  https://review.openstack.org/39527200:35
*** srwilkers has joined #openstack-kolla00:37
sdakesup pavo00:38
PavoI was gonna ask about tls deployment think I figured it out00:38
Pavoif you are using a secure docker registry does the bootstrap-servers copy over the certs?00:40
*** chas_ has joined #openstack-kolla00:44
openstackgerritJoshua Harlow proposed openstack/kolla: Add ability to save the things built  https://review.openstack.org/39527300:45
openstackgerritJoshua Harlow proposed openstack/kolla: Add ability to save the things built  https://review.openstack.org/39527300:46
openstackgerritJoshua Harlow proposed openstack/kolla: Add ability to save the things built  https://review.openstack.org/39527300:47
*** chas_ has quit IRC00:49
*** tovin07 has quit IRC00:49
*** hieulq has quit IRC00:49
*** hieulq has joined #openstack-kolla00:52
*** diogogmt has quit IRC00:52
*** tovin07 has joined #openstack-kolla00:54
*** zhubingbing has joined #openstack-kolla01:19
openstackgerritWei Cao proposed openstack/kolla: Add karbor container  https://review.openstack.org/39500601:19
*** neilus has joined #openstack-kolla01:21
openstackgerritzhubingbing proposed openstack/kolla: add panko dockerfile  https://review.openstack.org/38723201:24
*** neilus has quit IRC01:25
*** zhurong has joined #openstack-kolla01:30
*** sdake has quit IRC01:37
*** diogogmt has joined #openstack-kolla01:39
*** chas_ has joined #openstack-kolla01:45
*** chas_ has quit IRC01:49
*** duonghq has joined #openstack-kolla01:50
duonghqmorning01:50
duonghqcoolsvap, are you free right now?01:53
*** ayoung has quit IRC01:54
*** schwicht has joined #openstack-kolla01:58
harlowjaanyone have an example of using https://github.com/openstack/kolla/blob/master/kolla/template/filters.py#L1902:05
harlowjado i just set some override in my jinja2 header?02:05
*** tovin07_ has joined #openstack-kolla02:06
harlowjanm, i see http://docs.openstack.org/developer/kolla/image-building.html#package-customisation02:06
*** unicell has quit IRC02:08
*** neilus has joined #openstack-kolla02:17
*** david-lyle has joined #openstack-kolla02:20
*** david-lyle has quit IRC02:20
*** neilus has quit IRC02:21
*** david-lyle has joined #openstack-kolla02:22
zhubingbingping Mauricio Lima02:34
*** chas_ has joined #openstack-kolla02:45
*** chas_ has quit IRC02:50
*** srwilkers has quit IRC03:06
*** zhubingbing has quit IRC03:14
*** zhubingbing has joined #openstack-kolla03:17
*** haplo37 has quit IRC03:22
*** haplo37 has joined #openstack-kolla03:30
*** dave-mccowan has quit IRC03:38
*** portdirect has quit IRC03:44
*** chas_ has joined #openstack-kolla03:46
*** chas_ has quit IRC03:51
openstackgerritJoshua Harlow proposed openstack/kolla: Add ability to save the things built  https://review.openstack.org/39527303:51
*** schwicht has quit IRC04:10
*** fragatin_ has joined #openstack-kolla04:26
*** fragatina has quit IRC04:29
*** fragatin_ has quit IRC04:31
duonghqanybody build image recently with base: centos:7; as I can see, there is no openvswitch in epel04:44
openstackgerritJoshua Harlow proposed openstack/kolla: Stop matching incorrect plugin prefixes  https://review.openstack.org/39539905:00
*** fragatina has joined #openstack-kolla05:03
*** fragatina has quit IRC05:08
*** saneax-_-|AFK is now known as saneax05:16
*** david-lyle has quit IRC05:22
*** fragatina has joined #openstack-kolla05:22
*** david-lyle has joined #openstack-kolla05:23
*** fragatina has quit IRC05:27
*** chas_ has joined #openstack-kolla05:48
*** chas_ has quit IRC05:52
*** haplo37_ has quit IRC06:06
*** g3ek has quit IRC06:13
*** haplo37 has quit IRC06:13
*** haplo37 has joined #openstack-kolla06:23
*** haplo37_ has joined #openstack-kolla06:25
*** g3ek has joined #openstack-kolla06:26
*** zhurong has quit IRC06:35
*** zhurong has joined #openstack-kolla06:37
*** nihilifer has joined #openstack-kolla06:40
*** chas_ has joined #openstack-kolla06:48
*** jtriley has joined #openstack-kolla06:51
*** chas_ has quit IRC06:53
*** msimonin has joined #openstack-kolla06:54
zhubingbinghi guys06:56
sp_zhubingbing: hi06:56
zhubingbinghi06:56
*** jtriley has quit IRC06:56
zhubingbingnice meet you06:56
sp_nice to meet you too06:57
sp_zhubingbing: do you have something to ask ?? though i am new06:58
zhubingbingnp06:59
zhubingbingno question06:59
*** unicell has joined #openstack-kolla07:08
*** Jeffrey4l has joined #openstack-kolla07:09
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix plugin parsing for base images  https://review.openstack.org/39426707:18
*** chas_ has joined #openstack-kolla07:19
saneaxHi folks07:25
saneaxwas looking to install and try kolla-kubernetes, and following the docs here - http://docs.openstack.org/developer/kolla-kubernetes/index.html07:29
saneaxis there a separate IRC for kolla-kubernetes?07:29
jascott1no07:30
jascott1(not that they have told me about :)07:30
jascott1saneax did you have a question bout kolla-k8s?07:33
openstackgerritJeffrey Zhang proposed openstack/kolla: NO_MERGE: test mitaka branch  https://review.openstack.org/39387707:33
saneaxjascott1, yes, thanks - I have a baremetal with 64G and 24 Cores - thinking can I create 2 to 3 nodes with KVM and install a multinode setup?07:34
jascott1sounds reasonable07:35
saneaxalso, I have no previous exposure to installing kolla07:35
saneaxso, I am skipping the kolla installation and folling the kolla-k8s quickstart07:35
saneaxunforunate the doc on the quickstart does not talk of multinode setup07:36
saneaxit seems like its a single node install?07:36
jascott1i would look at all the docs not just the quickstart07:37
jascott1also there is a multinode doc07:37
jascott1http://docs.openstack.org/developer/kolla-kubernetes/multi-node.html07:37
sp_jascott1: is it possible to deploy kolla-k8s on single node ??07:38
saneaxOk jascott107:39
jascott1sp_ yes07:40
jascott1i would check this http://docs.openstack.org/developer/kolla-kubernetes/index.html07:40
openstackgerritJeffrey Zhang proposed openstack/kolla: NO_MERGE: test mitaka branch  https://review.openstack.org/39387707:40
saneaxthanks jascott107:41
sp_thanks *jascott1*07:43
jascott1np07:43
*** zhubingbing has quit IRC07:43
*** msimonin has quit IRC07:47
*** neilus has joined #openstack-kolla07:47
*** zhubingbing has joined #openstack-kolla07:47
*** jtriley has joined #openstack-kolla07:51
*** neilus has quit IRC07:52
*** matrohon has joined #openstack-kolla07:53
*** jtriley has quit IRC07:57
openstackgerritJeffrey Zhang proposed openstack/kolla: NO_MERGE: test mitaka branch  https://review.openstack.org/39387708:01
*** zhubingbing has quit IRC08:02
*** dcwangmit01_ has joined #openstack-kolla08:03
*** dcwangmit01 has quit IRC08:05
*** caowei_ has joined #openstack-kolla08:08
*** shardy has joined #openstack-kolla08:09
openstackgerritJeffrey Zhang proposed openstack/kolla: NO_MERGE: test mitaka branch  https://review.openstack.org/39387708:09
*** b_bezak has joined #openstack-kolla08:10
*** rmart04 has joined #openstack-kolla08:16
*** b_bezak_ has joined #openstack-kolla08:17
openstackgerritJeffrey Zhang proposed openstack/kolla: NO_MERGE: test mitaka branch  https://review.openstack.org/39387708:17
*** b_bezak has quit IRC08:20
*** b_bezak has joined #openstack-kolla08:21
*** b_bezak_ has quit IRC08:24
*** f13o has joined #openstack-kolla08:29
*** b_bezak has quit IRC08:31
openstackgerritJeffrey Zhang proposed openstack/kolla: Use openstack_service_workers variable in gate  https://review.openstack.org/39414808:43
*** caowei_ has quit IRC08:47
*** openstackgerrit has quit IRC08:48
*** openstackgerrit has joined #openstack-kolla08:48
*** jtriley has joined #openstack-kolla08:53
*** gfidente has joined #openstack-kolla08:53
*** bjolo has joined #openstack-kolla08:53
bjolomorning08:53
duonghqping bjolo08:54
bjoloduonghq, sup08:55
duonghqsaneax, some more info: I can run 2 node deployment w/ KVM on machine w/ 16GB RAM :)08:55
duonghqbjolo, did you build image recently with centos:7 base/08:55
bjoloyes08:56
bjolo2 days ago08:56
duonghqdid you have trouble with openvswitch?08:56
bjolonope08:56
bjolonot that i know of08:56
duonghqI cannot get openvswitch installed08:56
bjolook08:56
duonghqit's not in epel7 repo08:56
bjololet me kick of a build08:56
bjoloepel repo08:57
bjolois that used by default or something that you have added?08:58
*** jtriley has quit IRC08:58
duonghqit's from epel-release which in turn is install by default,08:58
duonghqcorrect me if I wrong08:58
*** jrist has quit IRC08:59
openstackgerritWei Cao proposed openstack/kolla: [WIP] Add karbor ansible role  https://review.openstack.org/39502208:59
*** Pavo has quit IRC09:00
*** jrist has joined #openstack-kolla09:00
*** caowei_ has joined #openstack-kolla09:02
openstackgerritWei Cao proposed openstack/kolla: Add karbor container  https://review.openstack.org/39500609:03
*** Serlex has joined #openstack-kolla09:09
*** egonzalez90 has joined #openstack-kolla09:10
*** sdake has joined #openstack-kolla09:11
sdakehey peeps09:11
*** stvnoyes1 has joined #openstack-kolla09:19
*** stvnoyes has quit IRC09:19
*** neilus has joined #openstack-kolla09:26
*** neilus_ has joined #openstack-kolla09:30
*** msimonin has joined #openstack-kolla09:30
duonghqhi sdake09:30
*** zhurong has quit IRC09:32
*** neilus has quit IRC09:32
duonghqbjolo, do you got any trouble with openvswitch?09:33
bjolocentos binary built just fine09:34
*** zhurong has joined #openstack-kolla09:35
bjolocould be the mirror that you are using that has issues09:35
*** caowei_ has quit IRC09:35
bjoloor the great fw of china09:35
*** msimonin has quit IRC09:35
bjolohi sdake09:35
*** msimonin has joined #openstack-kolla09:36
sdakesup bjolo09:38
*** sdake has quit IRC09:38
bjoloup late09:38
duonghqbjolo, I'm in Vietnam09:40
bjoloduonghq, ok sorry thougth you were in china :)09:41
duonghqbjolo, which repo does this package come from?09:41
bjolohmm how do i check that_09:41
bjolo?09:41
duonghqyou can start any container and jump in, run yum info openvswitch09:42
bjolotrue :)09:42
*** yingjun has quit IRC09:42
openstackgerritzhubingbing proposed openstack/kolla: add panko dockerfile  https://review.openstack.org/38723209:42
*** zhubingbing has joined #openstack-kolla09:45
*** rmart04 has quit IRC09:45
*** egonzalez has joined #openstack-kolla09:48
*** egonzalez90 has quit IRC09:48
duonghqegonzalez, ping09:48
egonzalezduonghq, pong09:49
openstackgerritManikandan R proposed openstack/kolla: Corrects typo "requried" in kolla_docker.py  https://review.openstack.org/39549209:49
duonghqhow do you think about my policy patch,09:49
duonghqI should remove non-API service you commented09:49
duonghqor fix it in follow-up patch?09:49
duonghqI'm not sure09:49
egonzalezI think the best option is keep all services for now. At least until we are sure which one need only API and which not.09:50
*** b_bezak has joined #openstack-kolla09:51
egonzalezAny other voice on this? Thanks09:51
egonzalezCan you share the PS link?09:51
duonghqegonzalez, moment09:52
bjolomorning egonzalez09:52
duonghq#link https://review.openstack.org/#/c/394260/09:52
duonghqthank egonzalez09:53
bjololooking at your comments on this https://review.openstack.org/#/c/392094/5/ansible/group_vars/all.yml09:53
bjolohttps://review.openstack.org/#/c/392094/509:53
egonzalezIa not wrong, commented after that09:53
*** jtriley has joined #openstack-kolla09:54
bjoloare you ok with +2 on this one now? looks good to me. The all.yml tunnel_interface_address will be set for each host correctly09:54
egonzalezI think someone from Kubernetes need to validate09:54
egonzalezBecause that variable was introduced by them09:55
bjolook a few different things here09:55
bjolokolla-ansible needs to fix this as well09:55
bjoloare you saying that the code in all.yml would not work?09:56
*** neilus_ has quit IRC09:56
egonzalezWill work for Ansible but don't know if for Kubernetes will do09:56
bjolokube people can fix it in their repo ;)09:57
bjolosdake around?09:58
*** jtriley has quit IRC09:59
saneaxthanks duonghq10:01
saneaxI am trying out and will let you folks know how it goes10:01
*** zhurong has quit IRC10:01
egonzalezbjolo, were you able to make networking work?10:02
bjolonope10:02
bjolonot yet10:03
bjoloim backtracking and just did a minimal deploy10:03
bjolodid a git pull just to learn that latest PS for kibana to fix logpath is broken :(10:03
bjolono central logging atm10:03
bjolowe really need better gates10:04
egonzalezWelcome to the tcpdump around the 1000 devices neutron creates ;)10:04
bjolohaha i know10:04
bjoloneutron is horrible10:04
*** zhubingbing has quit IRC10:05
bjolothis is a really good walkthrough https://www.youtube.com/watch?v=7IXEtUEZslg10:05
egonzalezIs funny when fails in PoC10:05
egonzalezWhen fails on prod. Look for another company10:05
bjolohaha10:05
*** zhubingbing has joined #openstack-kolla10:06
bjoloi went beta testing friday10:06
bjoloswear to god we tested networking, but in beta did not10:06
bjolonow everyone is chasing me to get it fixed10:06
bjoloand im just hitting the fricking wall on this one10:06
bjoloso the fact that kibana is not working is kinda frustrating10:08
bjolowe really need better testing10:08
bjoloand i mean large, all services enabled, integration testing.10:09
bjolowith TLS10:09
bjolomultinode10:09
bjolomultinetwork10:09
bjolodeploy, tempest run; change config; upgrade; tempest run10:10
*** tovin07_ has quit IRC10:10
bjoloas the number of openstack project are added to kolla, this just gets more and more critical10:10
*** neilus has joined #openstack-kolla10:10
bjoloi fear we will implode and drown in "this is not working bugs"10:11
egonzalezTotally agree10:14
egonzalezCan you gather some info from your nodes, i will try to debug dvr at the evening10:14
egonzalezNamespaces present in neutron and compute nodes, also docker containers running on them10:15
openstackgerritzhubingbing proposed openstack/kolla: add panko dockerfile  https://review.openstack.org/38723210:15
*** DTadrzak has joined #openstack-kolla10:15
egonzalezAnd ovs vsctl-show in neutron ovs containers10:16
bjoloduonghq, epel: mirrors.telianet.dk10:17
bjoloyum info openvswitch10:17
duonghqthank bjolo10:17
bjolofresh build right now10:18
bjoloso it should be good10:18
duonghqcannot understand why I cannot install openvswitch10:18
bjolowhats you build setup?10:18
duonghqas default, and centos/source10:19
bjolowhats you build command?10:19
duonghqI use build in tool10:19
bjolome too10:20
duonghq./build.py -i .header .I .footer openvswitch-base10:20
bjoloand whats in header and footer?10:20
duonghqproxy10:21
bjoloi would add --nocache10:21
duonghqme too10:21
bjolowhats the output from docker images?10:21
bjolosometimes you can get weird stuff from docker10:21
bjolodocker images -q | xargs -I {} docker rmi -f {}10:22
bjolocleans them all out10:22
*** neilus has quit IRC10:22
duonghqit said that cannot found openvswitch10:22
duonghqit's happen even when I jump in centos:7 and run openvswitch10:23
duonghqyum -y install openvswitch10:23
*** neilus has joined #openstack-kolla10:23
bjoloyum update first10:24
bjoloyum repolist10:24
bjoloyum search openvswitch10:24
duonghqmoment10:25
bjoloduonghq, remove containers as well before removing images10:28
duonghqsure10:29
*** zhubingbing has quit IRC10:29
duonghqbut I think the default repo of Kolla should be update10:29
duonghqit Dockerfile10:29
duonghq*in10:29
bjoloisnt epel a dynamic repo?10:31
duonghqsure10:31
bjoloi.e. it tries out several ones to find the best one for your location10:31
duonghqbjolo, I create epel mirror locally10:31
*** tonanhngo has quit IRC10:31
bjolook10:31
bjolois the package there?10:32
bjolowhat tool did you use to mirror the repo?10:32
duonghqno luck10:32
duonghqI used reposync10:33
duonghqofficial tool of yum10:33
duonghqfrom yum-utils10:33
bjoloany errors or issues?10:34
egonzalezMay be the problem the openvswitch base is build from base instead of openstack-base10:34
duonghqno, everything just fine10:34
duonghqegonzalez, openvswitch is installed in openvswitch-base10:34
bjolois the openvswitch package in your local repo?10:34
duonghqbjolo, no10:34
bjolothen thats your issue right10:34
bjolonot the kolla build10:35
duonghqbjolo, https://n40lab.wordpress.com/2016/03/02/openvswitch-lts-in-centos-7/10:35
duonghqbjolo, yup10:35
duonghqthe openvswitch issue10:35
duonghqrepo issue10:35
bjolohmm so how come it works for me?10:36
bjolono issues what so ever10:36
duonghqI got this error just from this week10:37
bjoloi dont have any local repos or build template overrides10:37
duonghqI cannot understand where is my setup grab openvswitch till now10:38
duonghqcentos 6 is ok10:38
*** neilus has quit IRC10:38
bjolo ./build.py --config-file /etc/kolla/kolla-build.conf --registry dockreg.mydomain.net:4000 --namespace $NAMESPACE --push --nocache --base centos --type binary --tag $TAG --logs-dir $LOGS10:39
bjolothe essential part in my build script10:39
duonghqI just put your option in kolla-build file10:39
duonghqyour 's same as me basically10:39
bjoloin my script i build for centos source/binary and ubuntu source/binary10:40
bjolohence the long command line :)10:40
duonghqit's ok, just personal taste10:41
openstackgerritWei Cao proposed openstack/kolla: Add karbor container  https://review.openstack.org/39500610:43
duonghqbjolo, I go home right now10:43
duonghqsee you in the meeting10:43
*** duonghq has quit IRC10:43
bjolook10:47
*** ppalacios has joined #openstack-kolla10:54
*** jtriley has joined #openstack-kolla10:55
*** msimonin has quit IRC10:57
*** msimonin has joined #openstack-kolla10:59
*** jtriley has quit IRC11:00
*** neilus has joined #openstack-kolla11:01
*** DanyC has joined #openstack-kolla11:02
openstackgerritSurya Prakash Singh proposed openstack/kolla: Consistent home directory creation for all the services  https://review.openstack.org/39017911:06
*** msimonin has quit IRC11:08
*** neilus has quit IRC11:13
*** schwicht has joined #openstack-kolla11:23
*** saneax is now known as saneax-_-|AFK11:28
*** caowei_ has joined #openstack-kolla11:29
*** neilus has joined #openstack-kolla11:32
openstackgerritSurya Prakash Singh proposed openstack/kolla: Consistent home directory creation for all the services  https://review.openstack.org/39017911:33
*** portdirect has joined #openstack-kolla11:37
*** tonanhngo has joined #openstack-kolla11:38
*** tonanhngo has quit IRC11:40
*** mliima has joined #openstack-kolla11:41
*** neilus has quit IRC11:42
*** egonzalez90 has joined #openstack-kolla11:48
*** egonzalez has quit IRC11:48
*** DTadrzak has quit IRC11:52
*** caowei_ has quit IRC11:54
*** jtriley has joined #openstack-kolla11:55
*** jtriley has quit IRC12:01
*** jmccarthy has quit IRC12:02
*** jmccarthy has joined #openstack-kolla12:03
*** neilus has joined #openstack-kolla12:04
*** DanyC has left #openstack-kolla12:10
*** caowei has joined #openstack-kolla12:12
*** DTadrzak has joined #openstack-kolla12:14
*** mkoderer has joined #openstack-kolla12:26
*** msimonin has joined #openstack-kolla12:27
openstackgerritJeffrey Zhang proposed openstack/kolla: NO_MERGE: test mitaka branch  https://review.openstack.org/39387712:32
*** schwicht has quit IRC12:33
*** caowei has quit IRC12:33
bjoloegonzalez90, i think i have found something. mtu size on br-tun12:38
egonzalez90Interesting, what mtu size you had?12:39
bjoloi have set mtu 9000 in neutron12:40
pomacAll interfaces except some have 9000 - the problem is that some has 150012:40
openstackgerritJeffrey Zhang proposed openstack/kolla: NO_MERGE: test mitaka branch  https://review.openstack.org/39387712:40
bjolobut br-tun is still 150012:40
pomacbjolo: sorry =)12:40
*** DTadrzak has quit IRC12:40
bjolobr-tun is vxlan over UDP and UDP does not fragment right?12:41
bjolohi pomac12:41
bjolo(colleague of mine)12:41
pomacbjolo: UDP does not fragment unless done manually12:41
*** tyrola has joined #openstack-kolla12:42
*** DTadrzak has joined #openstack-kolla12:42
bjoloi cant find anywhere in kolla where i can change/set br-tun mtu12:43
* bjolo is investigating12:43
egonzalez90IIRC we only configure MTU at instances(dhcp agent)12:44
egonzalez90Nothing on the baremetal side12:44
tyrolaHey guys. Can anyone help me with my kolla issue? I am trying to setup kolla with tagged vlans for api_interface12:45
bjolotyrola, create a vlan interface on the host side12:45
tyrolayes I did. My network interface is called just "network". I've added network.140 (for vlan id 140)12:46
tyrolaafter setting it to api_interface: "network.140" it creates some errors while deploying12:46
tyrolafatal: [controller01] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'dict object' has no attribute 'ipv4'", 'failed': True}12:46
tyrolaBut this interface has a valid ipv4.12:46
bjolowhat does prechecks say?12:47
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix horizon dashboard plugin  https://review.openstack.org/39389812:47
bjolothere is an issue that rabbitmq requires name resolution and it does that on network_interface12:48
bjolobut it requires that hostname should be resolved to IP on api_interface12:48
*** sdake has joined #openstack-kolla12:48
bjolothere is a bugg on this12:48
*** nradojevic has joined #openstack-kolla12:48
bjoloi my setup i dont use a dedicated api_interface12:48
bjolotreat network_interface as api interface12:48
*** zhurong has joined #openstack-kolla12:49
*** zhubingbing has joined #openstack-kolla12:50
bjoloJeffrey4l, nice PS :)12:52
*** caowei_ has joined #openstack-kolla12:52
bjoloplease tell me we get this backported to newton :)12:52
nradojevicHi, I've deleted the neutron-server container on my controller hosts and tried to deploy them again. But now I'm getting the following error message: chmod: changing permissions of '/var/log/kolla/neutron': Operation not permitted12:52
nradojevicCan someone help me to solve this? I'm using stable/mitaka release.12:52
Jeffrey4lbjolo, dashboard? yep, i think it can be backport to newton ;)12:53
bjoloyes12:53
tyrolaprechecks is failing at: Checking if kolla_internal_vip_address is in the same network as network_interface on all nodes12:55
bjolotyrola, ok12:55
bjolohmm did not know that was actually a requirement12:55
bjoloi have that i my setup, but unknowingly12:56
bjoloegonzalez90, there are quite a few config (kolla config overide) options for neutron and MTU12:57
*** jtriley has joined #openstack-kolla12:57
*** caowei_ has quit IRC12:57
bjolotyrola, what distro is on your hosts?12:58
tyrolacent os 712:58
tyrolaand the containers too12:59
bjolohmm can you really have a network interface named "network"12:59
bjolothought there were some form of naming standard for this12:59
pomacnot really12:59
pomacask systemd ppl - ;)12:59
portdirectyou can name them whatever you like13:00
*** msimonin has quit IRC13:00
tyrolait has been renamed in persistent rules 70 :)13:00
tyrolaI have 2 interfaces named network and storage13:01
bjoloyes, thats one thing, but if you want all the tools/scripts that handles networking to work properly i dont think you can. I have read some where the docs dictating allowed names on interfaces13:01
*** zhurong has quit IRC13:01
*** jtriley has quit IRC13:02
*** dwalsh has joined #openstack-kolla13:02
pomacsure, there can be things that expect network as a parameter/argument13:02
tyrolaOh I think I know what you mean. Ansible generated a variable ansible_<network interface name> and maybe this variables "ansible_network" is being used already13:03
bjolotyrola, yes things like that13:03
bjolofor ex13:03
tyrolaso it can't detect anssible_network.ipv413:03
tyrolaI will rename the interfaces on all nodes, give me some minutes13:03
*** portdirect has quit IRC13:04
tyrolaDo you know where I can find the rules of allowed names? Would be great if I can see by the name if it's storage or network.13:04
*** jtriley has joined #openstack-kolla13:06
*** zhurong has joined #openstack-kolla13:06
openstackgerritWang Liming proposed openstack/kolla: source doc bifrost.rst charactor error  https://review.openstack.org/39558113:08
*** mgiles has joined #openstack-kolla13:09
*** jtriley has quit IRC13:10
*** neilus has quit IRC13:10
*** lamt has joined #openstack-kolla13:10
tyrolaOk seems like Ansible has a problem if the network interface contains a . (dot) for vlan tags.13:15
tyrolathe interface is now called "ifnetwork.140". And Ansible thinks .140 is a attribute.13:16
tyrolaOne or more undefined variables: 'dict object' has no attribute u'ansible_ifnetwork.140'", 'failed': True}13:16
openstackgerritWang Liming proposed openstack/kolla: source doc bifrost.rst charactor error  https://review.openstack.org/39558613:16
pomacwhy not net-vlan14013:17
*** neilus has joined #openstack-kolla13:18
tyrolaSorry, I had no coffee yet. Sure that would be the easiest solution. Thanks to ansible and the power of renaming network devices :D13:20
*** skramaja has quit IRC13:20
*** inc0 has joined #openstack-kolla13:20
inc0o/13:22
*** caowei_ has joined #openstack-kolla13:22
inc0interesting times coming13:22
bjolosup inc013:23
*** schwicht has joined #openstack-kolla13:23
inc0election results13:23
bjolohttps://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/sec-Naming_Scheme_for_VLAN_Interfaces.html13:24
inc0I'm imigrant in country that just elected president shouting to fight imigrants13:24
britthouserI laugh, and then I cry, and then I laugh, and then I cry.13:24
bjolotyrola, ^^13:24
bjolotakes a fool to remain sane13:24
*** duonghq has joined #openstack-kolla13:24
duonghqtoday we have meeting as usual?13:24
inc0yes13:24
inc0we're back in business13:24
zhubingbinghi inc013:25
zhubingbingnice meet you13:25
*** lamt has quit IRC13:25
inc0wassup zhubingbing ?13:25
*** jistr is now known as jistr|brb13:26
zhubingbinghttps://review.openstack.org/#/c/353261/13:26
zhubingbingcan u help me13:27
zhubingbing~~)13:27
zhubingbingreview it13:27
duonghqroger inc013:28
sdakemorning peeps13:29
duonghqpbourke, ping13:29
duonghqmoring sdake13:29
sdakeinc0 rhallisey's spec gaining steam13:29
sdakeinc0 have you had a look?13:29
sdakesup zhubingbing sup duonghq13:29
zhubingbinghi13:29
zhubingbingsdake13:29
sdakeinc0 i wont make it to meeting, at cncf13:29
inc0sdake, not today, I'm thinking about starting drinking early13:29
bjolohttp://quotespics.com/wp-content/quote-images/give-a-fuck.jpg13:29
sdakebjolo pretty much ;-)13:30
sdakesup bjolo13:30
*** lamt has joined #openstack-kolla13:31
*** schwicht has quit IRC13:31
*** dave-mccowan has joined #openstack-kolla13:34
*** jheroux has joined #openstack-kolla13:34
*** tonanhngo has joined #openstack-kolla13:35
*** tonanhngo has quit IRC13:36
tyrolabjolo, is there a easy way in CentOS to rename a vlan device? It seems like it must contain a . (dot).13:38
bjolochange the name in /etc/sysconfig/network-scripts/<device>13:39
*** tonanhngo has joined #openstack-kolla13:39
tyrolathats my ifcfg http://pastebin.com/LZcsbfV113:39
bjoloifcfg-xyz13:39
tyrola- is also okay? i've tried it just with ifnetwork140 and the network does not work after reboot13:39
*** schwicht has joined #openstack-kolla13:40
*** b_bezak has quit IRC13:40
*** neilus has quit IRC13:40
*** b_bezak has joined #openstack-kolla13:40
bjolowhats the name of the file?13:41
tyrolaI name it always like the device name13:41
bjolono, i meant the file name is usually ifcfg-<device name>13:41
bjoloin my case i have ifcfg-bond0 and ifcfg-bond0.802 as vlan device of bond013:41
tyrolayes the network interface itself uses ifcfg-ifnetwork. and the vlan interface is called ifcfg-ifnetwork.14013:42
*** b_bezak_ has joined #openstack-kolla13:42
bjolotyrola, that should work i quess13:42
tyrolaYes it works but kolla (ansible) does not like the . char in the name if the interface13:43
bjolobut im not sure about calling a nic "ifnetwork"13:43
duonghqiirc you should espace this character13:43
bjolokolla should work fine as long as you quote the name13:43
tyrolahow can i escape it in the yml file?13:43
*** lrensing has joined #openstack-kolla13:43
bjolotunnel_interface: "bond0.820"13:44
tyrolanetwork_interface: "ifnetwork.140"13:44
bjoloyes13:44
tyrolaBut it seems like ansible thinks .140 is a attribute of ifnetwork13:44
tyrolafatal: [controller01] => {'msg': 'One or more items failed.', 'failed': True, 'changed': False, 'results': [{'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'dict object' has no attribute u'ansible_ifnetwork.140'", 'failed': True}]}13:44
bjolomitaka you said?13:45
tyrolayes13:45
bjoloor newton13:45
bjolomitaka13:45
tyrolamitaka branch13:45
*** b_bezak has quit IRC13:45
bjolobuild your own images?13:45
tyrolayes with centos7 containers and rdo13:45
tyrolabuild just a few days ago13:45
bjoloand this is from deploy?13:46
tyrolayes13:46
bjolopip list | grep kolla13:46
*** jistr|brb is now known as jistr13:46
tyrolals13:47
bjoloor are you runing from git13:47
tyroladirectly from git13:47
tyrolanot installed with pip13:47
openstackgerritzhubingbing proposed openstack/kolla: add panko role  https://review.openstack.org/39429813:47
bjolook13:47
bjolodo you have the same network interface names on all nodes?13:48
*** egonzalez90 has quit IRC13:48
*** egonzalez has joined #openstack-kolla13:48
tyrolaDo you if it has been fixed in kolla in the last few days? If yes it maybe its missing in our fork.13:48
tyrolayes I have13:48
bjoloi run newton (or trying to) not really keeping track of mitaka branch13:48
bjologit log13:49
tyrolaI will check13:49
bjoloif you ask me, i would not run a dedicated network for api13:49
bjolowell depends on how you set things up at large13:49
sdakeinc0 i dont have my passwd for freenode, can you change the topic to something more appropriate13:49
sdake3.0.1 is in the past now :)13:50
*** DTadrzak has quit IRC13:50
tyrolaDue some company security limitations we can't run the api network in the same network like the external subnets on the compute nodes, we are using neutron with bridge and provider networks.13:51
tyrolaSo the ifnetwork in my case also contains vlans for external network access13:51
sdaketyrola are you using ansible 2.z?13:52
sdaketyrola you can find out via pip show kolla13:53
sdakerather pip show ansible13:53
sdakeor ansible --version13:53
tyrolaOkay I think the git fork is a little bit older :( It seems like we are running a older kolla version with ansible 1.9. I will check if it's possible to upgrade.13:54
tyrolaWe did a lot of modifications (added many storage roles) for our HP 3par storage. Not so easy to merge.13:55
tyrolaOh it seems the stable/mitaka branch still using ansible 1.913:59
*** schwicht has quit IRC13:59
*** portdirect has joined #openstack-kolla14:00
*** jtriley has joined #openstack-kolla14:02
*** schwicht has joined #openstack-kolla14:04
tyrolaOkay seems really like ansible can't use {{ hostvars[host]['ansible_hostname'] }} {{ hostvars[host]['ansible_' + hostvars[host]['api_interface']]['ipv4']['address'] }} correctly with a dot.14:04
*** schwicht has quit IRC14:07
*** fguillot has joined #openstack-kolla14:07
*** eaguilar has joined #openstack-kolla14:09
*** lrensing has quit IRC14:10
*** nradojevic has quit IRC14:13
openstackgerritSteven Dake proposed openstack/kolla-kubernetes: Add advisory language to specs directory for kolla-kubernetes  https://review.openstack.org/39563614:15
sdaketyrola mitaka uses ansible 2.014:16
sdakehrm wait its 1.914:16
* sdake brain waking up - wtb caffiene :)14:17
sdaketyrola can you ping Jeffrey4l he understands htis problem in detail14:17
*** jrist has quit IRC14:17
*** b_bezak_ has quit IRC14:19
*** b_bezak has joined #openstack-kolla14:19
*** jrist has joined #openstack-kolla14:20
*** schwicht has joined #openstack-kolla14:23
openstackgerritzhubingbing proposed openstack/kolla: Add trove role  https://review.openstack.org/35490114:24
*** b_bezak has quit IRC14:24
*** schwicht has quit IRC14:28
*** neilus has joined #openstack-kolla14:28
sdakekfox1111 around?14:29
tyrolasdake, alright. I contact him. thanks.14:29
sdaketyrola note he is in china14:29
sdakeusuually he is aroudn this hour14:29
*** lrensing has joined #openstack-kolla14:30
sdakebut if not, you can always try the ml14:30
*** yingjun has joined #openstack-kolla14:31
openstackgerritWang Liming proposed openstack/kolla: update source doc kuryr-guide.rst and manila-hnas-guide.rst character error  https://review.openstack.org/39563814:33
*** neilus has quit IRC14:33
*** f13o_ has joined #openstack-kolla14:36
*** schwicht has joined #openstack-kolla14:36
*** f13o has quit IRC14:41
*** schwicht has quit IRC14:41
openstackgerritWei Cao proposed openstack/kolla: Add karbor container  https://review.openstack.org/39500614:42
openstackgerritWang Liming proposed openstack/kolla: update production-architecture-guide.rst Some words that are wrong in grammar  https://review.openstack.org/39564514:43
*** schwicht has joined #openstack-kolla14:45
*** schwicht has quit IRC14:52
*** b_bezak has joined #openstack-kolla14:54
*** srwilkers has joined #openstack-kolla14:58
*** schwicht has joined #openstack-kolla15:00
*** saneax-_-|AFK is now known as saneax15:03
*** zhurong has quit IRC15:03
*** schwicht has quit IRC15:04
*** neilus has joined #openstack-kolla15:10
*** DTadrzak has joined #openstack-kolla15:12
*** schwicht has joined #openstack-kolla15:12
*** diogogmt has quit IRC15:13
*** saneax is now known as saneax-_-|AFK15:14
*** caowei_ has quit IRC15:14
*** saneax-_-|AFK is now known as saneax15:14
*** jtriley has quit IRC15:15
*** DTadrzak has quit IRC15:16
*** schwicht has quit IRC15:17
*** msimonin has joined #openstack-kolla15:23
*** rhallisey has joined #openstack-kolla15:24
*** inc0 has quit IRC15:26
*** inc0 has joined #openstack-kolla15:27
rhalliseymorning15:27
lrensingmorning15:27
*** msimonin has quit IRC15:29
srwilkersmorning rhallisey15:29
*** jtriley has joined #openstack-kolla15:31
*** saneax is now known as saneax-_-|AFK15:32
*** yingjun has quit IRC15:33
*** yingjun has joined #openstack-kolla15:33
*** saneax-_-|AFK is now known as saneax15:33
*** absubram has joined #openstack-kolla15:33
*** absubram_ has joined #openstack-kolla15:34
inc0reminder guys, meeting in 26min15:36
inc0daylight savings15:36
inc0had saved our daylight once more15:36
*** CBR09 has joined #openstack-kolla15:37
*** absubram has quit IRC15:38
*** absubram_ is now known as absubram15:38
*** yingjun has quit IRC15:38
*** schwicht has joined #openstack-kolla15:39
srwilkersdamn you DST15:40
openstackgerritPete Birley proposed openstack/kolla: Add Custom template path option to kolla-build  https://review.openstack.org/39567615:41
*** nradojevic has joined #openstack-kolla15:42
*** egonzalez90 has joined #openstack-kolla15:42
*** egonzalez has quit IRC15:43
*** TxGirlGeek has joined #openstack-kolla15:44
inc0portdirect, I had a bit different idea about this15:44
inc0how about "template append path"?15:44
inc0default kolla path plus tpls from specified dir15:45
inc0instead of full new dir15:45
*** schwicht has quit IRC15:45
*** sdake has quit IRC15:50
openstackgerritPete Birley proposed openstack/kolla: Add Custom template path option to kolla-build  https://review.openstack.org/39567615:51
*** tonanhngo_ has joined #openstack-kolla15:51
*** tonanhngo has quit IRC15:51
*** sdake has joined #openstack-kolla15:52
*** ayoung has joined #openstack-kolla15:55
*** msimonin has joined #openstack-kolla15:55
sdakeinc0 i will make meeting today if i can15:55
sdakepbourke you about15:55
pbourkesdake: yeah15:55
*** tonanhngo_ has quit IRC15:55
sdakepbourke is stvnoyes1 interested in the operator spec ryan has produced15:55
sdakeoh hes here ;)15:56
sdakestvnoyes1 are you interested in the operator spec ryan has produced15:56
sdakestvnoyes1 if so, plz add your name to the list of assignees :)15:56
zhubingbinglater we should start meeting15:57
stvnoyes1sdake: mgiles will be working on that. I've been given another set of work so unfortunately I won't be able to take this on atm. I'll talk to mark and have him added.15:57
inc02 more minutes:)15:57
openstackgerritJames McCarthy proposed openstack/kolla: Missing customization of tenant network  https://review.openstack.org/39498815:58
rhalliseysdake, can we go straight to Go?15:59
sdakerhallisey nope15:59
rhalliseyok15:59
rhalliseythought so, those comments made it seem like we could15:59
sdakerhallisey reason being we would need to enable the ecosystem15:59
*** schwicht has joined #openstack-kolla15:59
sdakefi we enabled the openstack ecosystem with go, we could do it in go15:59
rhalliseyok I will add that detail15:59
sdakethat would be longer term work15:59
portdirectinc0: sorry irc in background :(15:59
*** msimonin has quit IRC15:59
sdakestvnoyes1 roger15:59
sdakeoh is meeting in 2 minutes?16:00
rhalliseysdake, right now16:00
portdirectinc0: I'm ok with that idea: I'll update the pr after the meeting16:00
inc0thanks16:00
sdakeis the meeting now?16:00
sdakeor in 1 hr16:00
inc0now16:00
sdake(no idea, traveling)16:00
sdakeok thanks16:00
inc0our daylight got saved16:00
*** DTadrzak has joined #openstack-kolla16:01
*** ayoung has quit IRC16:01
*** eaguilar has quit IRC16:01
*** b_bezak has quit IRC16:01
*** saneax is now known as saneax-_-|AFK16:01
*** saneax-_-|AFK is now known as saneax16:01
*** saneax is now known as saneax-_-|AFK16:03
*** schwicht has quit IRC16:04
*** msimonin has joined #openstack-kolla16:05
pprokopHi sorry for being late16:06
egonzalez90pprokop: #openstack-meeting-416:06
pprokopthanks16:07
*** jtriley has quit IRC16:10
*** neilus has quit IRC16:11
sdakesay stvnoyes1 - we have our kolla irc meeting now in openstack-meeting-4 if your interested16:11
*** schwicht has joined #openstack-kolla16:11
*** saneax-_-|AFK is now known as saneax16:13
openstackgerritMerged openstack/kolla: add panko dockerfile  https://review.openstack.org/38723216:14
*** schwicht has quit IRC16:16
*** jtriley has joined #openstack-kolla16:17
*** absubram has quit IRC16:17
*** Marcellin__ has joined #openstack-kolla16:20
*** CBR09 has quit IRC16:20
*** saneax is now known as saneax-_-|AFK16:22
*** saneax-_-|AFK is now known as saneax16:22
*** saneax is now known as saneax-_-|AFK16:23
*** saneax-_-|AFK is now known as saneax16:23
*** schwicht has joined #openstack-kolla16:26
*** schwicht has quit IRC16:26
*** schwicht has joined #openstack-kolla16:27
*** nradojevic has quit IRC16:30
*** msimonin has quit IRC16:31
*** schwicht has quit IRC16:32
*** saneax is now known as saneax-_-|AFK16:33
*** jemcevoy has quit IRC16:33
*** saneax-_-|AFK is now known as saneax16:34
*** berendt has joined #openstack-kolla16:35
*** saneax is now known as saneax-_-|AFK16:37
*** saneax-_-|AFK is now known as saneax16:37
*** schwicht has joined #openstack-kolla16:38
*** sean-k-mooney has quit IRC16:39
*** msimonin has joined #openstack-kolla16:39
*** strigazi is now known as strigazi_AFK16:39
*** sean-k-mooney has joined #openstack-kolla16:40
*** saneax is now known as saneax-_-|AFK16:40
*** saneax-_-|AFK is now known as saneax16:41
*** AnswerGu1 has joined #openstack-kolla16:41
*** fragatina has joined #openstack-kolla16:42
*** schwicht has quit IRC16:43
*** f13o_ has quit IRC16:44
*** msimonin has quit IRC16:44
bjolopbourke, around?16:45
openstackgerritzhubingbing proposed openstack/kolla: Add trove role  https://review.openstack.org/35490116:45
*** jemcevoy has joined #openstack-kolla16:46
pbourkebjolo: hey bjorn!16:46
*** saneax is now known as saneax-_-|AFK16:46
*** saneax-_-|AFK is now known as saneax16:46
*** msimonin has joined #openstack-kolla16:49
AnswerGu1So, after a success build (--base ubuntu --type source --tag 3.0.0) and deploy ... after I source admin-openrc.sh ... when I try to run ./tools/init-runonce I get: Configuring neutron.16:49
AnswerGu1<html><body><h1>503 Service Unavailable</h1>16:49
AnswerGu1No server is available to handle this request.16:49
AnswerGu1.... 11 times16:49
AnswerGu1(once for each neutron invocation in the init-runonce script).16:50
*** saneax is now known as saneax-_-|AFK16:50
*** saneax-_-|AFK is now known as saneax16:50
AnswerGu1docker -ps | neutron shows the services running except for this:16:50
*** bmace has quit IRC16:50
*** n3v3rm0r3r has quit IRC16:50
*** bmace has joined #openstack-kolla16:50
AnswerGu143f27xxxxxxx        10.1.xx.xx:4000/kolla/ubuntu-source-neutron-server:3.0.0    \16:51
AnswerGu1"kolla_start"            2 hours ago         Restarting (1) 20 minutes ago16:51
bjolopbourke, weekly ongoing...16:51
bjolopbourke, i have some issues with multiple external networks16:51
bjolopbourke, thought i pick your brain on it16:51
bjolopbourke, singel external works, but multiple does not. traffic does not get out16:52
egonzalez90AnswerGu1: what does docker logs neutron_server return?16:52
*** saneax is now known as saneax-_-|AFK16:52
*** saneax-_-|AFK is now known as saneax16:53
pbourkebjolo: hmm ok so I remember you were having issues with this, just trying to remember exactly. I think you were missing the patch that configured the bridges in openvswitch?16:53
AnswerGu1I'm looking for that now.16:53
bjolopbourke, im past that point16:53
*** msimonin has quit IRC16:53
bjolothe deploy works, bridges created etc. the issue is that traffic does not find its way out to external network16:54
pbourkehmm ok16:54
*** saneax is now known as saneax-_-|AFK16:54
bjoloit seems packets are dropped by br-int openflow rules on network node16:54
AnswerGu1egonzalez90: the tail of my docker logs for the neutron-server container:16:55
AnswerGu1Running command: 'neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_lbaas.conf --config-file /etc/neutron/neutron_vpnaas.conf'16:55
AnswerGu1Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.16:55
bjololike it goes out the router but br-int cant find the external bridge16:55
AnswerGu1Wo we have a preferred pastebin?16:55
egonzalez90AnswerGu1: please paste it in paste.openstack.org16:55
AnswerGu1I'm looking to see if this bug applies to me: https://bugs.launchpad.net/kolla/+bug/159241816:56
openstackLaunchpad bug 1592418 in kolla "neutron-agent container not working with type source and neutron lbaas enabled" [High,Confirmed] - Assigned to Christian Berendt (berendt)16:56
pomacpbourke: it also looks like the interface does the wrong source route16:57
pomacpbourke: it doesnt answer with the local ip but with the external ip - which is the ignored by the vm16:58
*** schwicht has joined #openstack-kolla16:58
*** DTadrzak has quit IRC16:59
rhalliseyok continue with any discussion16:59
* rhallisey needs food16:59
inc0portdirect, https://www.youtube.com/watch?v=9VDvgL58h_Y16:59
portdirectawesome :)17:00
AnswerGu1egonzalez90: http://paste.openstack.org/show/588595/17:00
duonghqpbourke, ping17:00
srwilkersinc0: fitting with the atmosphere today17:01
pbourkepomac: bjolo: hmm ok I haven't seen this17:01
*** msimonin has joined #openstack-kolla17:01
pbourkepomac: bjolo: but if you both are it sounds like a problem17:01
*** eaguilar has joined #openstack-kolla17:01
jascott1inc0 morning! are we meeting?17:01
bjolowe are sitting together17:01
bjolopomac is my colleague17:01
inc0jascott1, daylight didnt save you17:01
duonghqpbourke, can you give some opinions about new comment from Jeffrey4l and egonzalez90 in https://review.openstack.org/#/c/394260/17:01
qwangpprokop: sorry I was late this mornig17:01
inc0we just finished17:02
jascott1i was up17:02
jascott1oh17:02
jascott1dang17:02
jascott1whatd i miss?17:02
qwangi joined the meeting for the second half but didn't say anything. it was too tense17:02
inc0flame war:)17:02
inc0read the logs17:02
portdirectI put together a rough PoC for kolla build that squashes image layers in a dockerfile into a single image (https://review.openstack.org/#/c/392115/). it saves about 30% of the base image size and potentially frees us up to make some further optimisations (like removing gcc and dev libs) for source containers.17:02
pprokopJust opensource17:02
*** neilus has joined #openstack-kolla17:03
duonghqrhallisey, ping17:03
pprokopFriday is an holiday in Poland17:03
jascott1aight thx17:03
* jascott1 adjusts his sundial17:03
pprokopso probably Monday is fine just ping me what time17:03
*** schwicht has quit IRC17:03
duonghqsdake, ping17:04
rhalliseyduonghq, hey17:04
duonghqsdake, you still in conference?17:04
rhalliseyduonghq, so you mentioned deps with ansible17:04
pbourkebjolo: aha. let me check with some others here and see if they're come across this17:04
portdirectdocker is putting similar functionallity upstream - but is squashes all layers into a single one (meaning massive pulls...) if anyone's interested I could develop this further, my experience is that it can knock quite a signifigant amount of space (~40%) off images.17:04
duonghqrhallisey, yep, I mean cross-projects dependency17:05
duonghqrhallisey, e.g. Magnum depends on Heat17:05
*** schwicht has joined #openstack-kolla17:05
*** tonanhngo has joined #openstack-kolla17:05
sdakeduonghq trying to be17:05
sdakeduonghq need to shower17:06
*** zhubingbing has quit IRC17:06
sdakespit it out17:06
pprokopand about choreography and orchestration (pretty good post) http://stackoverflow.com/questions/4127241/orchestration-vs-choreography17:06
duonghqsdake, wish that you can review ansible-become bp implementation: start with https://review.openstack.org/#/c/358539/ after the shower17:07
sdakepprokop thanks for link17:07
pprokop:) my pleasure17:07
sdakeduonghq i dont have time for reviews today sorry17:07
sdakewe have a large core review team, assume they will get to it eventually :)17:07
*** neilus has quit IRC17:07
rhalliseyduonghq, what about it specifcally?17:07
duonghqsdake, I asked you just because you approved my bp, sorry17:08
duonghqrhallisey, most basic example is every service depends on Keystone (in most case), we addressed it well.17:08
rhalliseyduonghq, right, ansible worked great for that17:09
duonghqbut for upcoming service, iirc we have not had any general mechanism for declare the deps graph17:09
*** vbel has joined #openstack-kolla17:09
sdakepprokop ok reading the first response which is pretty good and represents my understanding of entrypoint and operators17:09
*** tonanhngo has quit IRC17:09
*** schwicht has quit IRC17:09
sdake"This means that a choreography differs from an orchestration with respect to where the logic that controls the interactions between the services involved should reside."17:09
*** vbel has quit IRC17:10
egonzalez90agree with duonghq , as your previous example of magnum with heat17:10
*** tonanhngo has joined #openstack-kolla17:10
sdakeI am not keen on having the containers changed17:10
inc0sdake, we can introduce new container, brand new17:10
inc0which will act as init container17:10
sdakethat is what an action container i s!17:10
sdakesorry not action container17:10
inc0it's sole purpose will be to not run until deps are met17:11
rhalliseyinc0, what I'm worried about is can you solved all cases of that problem for OpenStack17:11
pprokopthat is why you can control it via envs17:11
inc0I can't think of case that couldn't be solved17:12
pprokopand only thing you provide17:12
*** jtriley has quit IRC17:12
pprokopis a name of k8s service,job, ds17:12
sdakeinc0 what you propose is exactly what an operator container is17:13
rhalliseyso the env is provided on the fly to list deps17:13
sdakeinc0 except operator contaienr does some bootstrapping as well17:13
pprokopeach deployment is different and Helm just have to template those envs17:13
inc0sdake, not exactly17:14
pprokopin entrypoint approach k8s jobs does bootstraping and you just append K8s_JOB env17:14
inc0as operator will run the logic itself17:14
inc0question, what happends if operator dies?17:14
sdakeinc0 no, operator runs a controller17:14
sdakeinc0 controller rusn the logic itself17:14
rhalliseyinc0, doesn't matter if it dies17:14
sdakeits the same thing exactly17:14
rhalliseyppalacios, what about upgrading though17:14
*** matrohon has quit IRC17:14
rhalliseyoops17:14
sdakei guess this raises a new question as we head down the rabbit hole17:14
rhalliseypprokop, what about upgrading though17:14
sdakeshoudl we use entrypoint in our operator containers17:15
rhalliseydep logic17:15
pprokopfollow k8s way update images perform rolling update and run job for db-sync17:15
rhalliseythat part I'm scared of17:16
inc0you don't really deal with deps during upgrade17:16
pprokopyes17:16
pprokopOVO works ?17:16
inc0only dep is "run db sync" -> "run upgrade"17:16
inc0ovo deal with different issue17:17
duonghqrolling upgrade and then 0-downtime upgrade for db and ovo is different service-to-service17:17
pprokopok17:17
sdakeright - which requires an operator17:17
rhalliseyit's a workflow, you'll need to to these things in order17:17
rhalliseyan op will not trust if you don't17:17
sdakeya we have to keep in mind here folks, we have built up a "16% interest" in kolla (the ansible part) in the operator community (data from last user survey)17:18
rhalliseyI mean k8s rolling upgrade is fantastic, but there operation is a huge pain point for ops17:18
rhalliseybecause it's so sensitive17:18
sdakeif the Operators (the people using kolla) don't trust kolla-kubernetes17:18
sdakeit could impact kolla-ansible17:18
sdakeand Operators wont trust containers that self-magically organize17:19
inc0if somebody wants to run kolla-k8s, they better know what k8s before17:19
rhalliseypprokop, idk just some thoughts.  I think kfox1111 will have some input on the matter17:19
sdakeya - so gotta jet folks17:19
sdakegoign to hit the floor with your questions17:19
duonghqI noticed that we have not implemented rolling upgrade properly in kolla-ansible for most services, if me if I wrong17:19
sdakeand see what people think of entrypoint17:19
inc0rhallisey,  I'm positive we can make it transparent enough for good debug17:19
rhalliseyI think k8s rolling update looks great and can be great for updates, but I want to be super careful17:19
inc0it's not matter of arch, it's matter of toolset17:20
portdirectsdake: lates :)17:20
portdirect*laters17:20
inc0k8s rolling upgrades is simple thing17:20
inc0upgrade pod one at the time17:20
inc0that's it17:20
rhalliseyit is and that I suppose is part of the probelm17:20
rhalliseybecause openstack is not a simple upgrade17:20
duonghqinc0, in this aspect, serial from ansible is nearly the same17:21
duonghqbut the hardest part is other17:21
inc0duonghq, exactly17:21
rhalliseyok I'll brb17:21
rhalliseypprokop, curious about the demo.  Also keep posting in the spec. Super appreciate it! :)17:21
rhalliseythe PoCs should help17:22
*** MarMat has joined #openstack-kolla17:22
ppalaciosrhallisey: that was not for me, right?17:23
pprokopOkay :D17:23
duonghqwe can rolling upgrade service binary/code very easily with out toolset, but if we only do that, maybe something'll break when major version upgrade is done17:23
duonghqexample for ovo: we need turn on and off version pinning for some service17:24
duonghqbefore and after rolling upgrade respectively17:24
duonghqseem that we have not implemented this?17:24
sdakepprokop if you are really keen to take a look at entrypoint for kolla, I think what would be feasible is something inc0 mentioned17:31
inc0sdake, this is what it is17:32
sdakewhich is to treat it as a seperate container17:32
inc0we discussed that with Piotrek yesterday, and I knew this arch before17:32
inc0we will have dedicated "lock" container17:32
sdakepprokop - in essence make an "operator" container using entrypoint17:33
*** n3v3rm0r3r has joined #openstack-kolla17:33
sdakepprokop to me, jamming entrypoint itno every one of our containers is a non-starter for a whole lot of reasons17:33
inc0so again, my understanding of entrypoint might fail17:33
sdakethe main one being IT DIDNT WORK IN THE PAST17:33
inc0but it seems like external pod that does magic17:33
pprokopit's pointless to put it into operators17:34
inc0sdake, it's different approach...fundamentally...17:34
pprokopbecause it was not designed to work this way17:34
bjolopbourke, when vm tries to ping external physical gw. arp request goes out and reply comes back, but br-int drops them it seems like17:34
bjolohttp://paste.openstack.org/show/588600/17:34
inc0trust me on that if you don't want to hear lengthy explanation17:34
sdakeinc0 ok, if its fundamentallly a different approach and being taken by stackenetes and fuel-ccp, why should we use it?17:34
sdakewouldn't that just make kolla-kubernetes fuel-ccp or stackanetes?17:34
inc0fundamentally different from what we had in compose17:34
inc0no, there are more differences17:35
rhalliseyppalacios, sorry that was for pprokop.17:35
inc0also, good ideas should be spread17:35
pprokopso you are just turning that this idea cause someone else used it ?17:35
inc0not being similar to stackanetes is not good reason for me to not use this tech17:35
pprokop:D17:35
sdakepprokop my question is why should we replicate someone elses implemetnation fo the same exact thing?17:35
inc0sdake, we will reuse this:)17:36
pprokopYou can just use this implementation.17:36
pprokopNot rewrite it.17:36
inc0maybe tailor it to our own ends17:36
inc0like put it into init container instead of other containers17:36
sdakepprokop i am more keen on operator containers17:36
sdakewith controllers inside17:36
inc0sdake, why?17:36
pprokopOkay so let's make POC's.17:36
inc0explain me what is real benefit17:36
pprokopAnd see what works best.17:37
sdakesimplicity is the benefit17:37
pprokopIt's pointless to argue.17:37
sdakepprokop if stackanetes has got it to work, i have no doubt it can work17:37
inc0ok, so another try at "my understanding what operator is"17:37
sdakereliable, not so much17:37
sdakeinc0 kfox1111 explains it well in the review17:37
inc0it's external pod that will query kubectl and run pods and whatnot based on what it sees in kubectl17:37
sdakeread it ;)17:37
sdakeright17:38
*** egonzalez90 has quit IRC17:38
pprokopThis is k8s-appcontroller17:38
inc0how appcontroller differs from operator?17:38
pprokopso don't write your own staff then.17:38
sdakean operator is a container that contains an implementation of code17:38
pprokopit is an operator in your sense.17:38
sdakethe code is called a controller17:38
pprokopBut it was created before operator name was invented.17:39
sdakeperhaps mirantis has written k8s-appcontroller17:39
sdakebut to be brutally frank, I don't trust mirantis17:39
*** schwicht has joined #openstack-kolla17:39
sdakei have no mistrust of coreos17:39
sdakebut i dont trust mirantis17:39
sdakeand nothing mirantis does is EVER going to change that17:39
inc0whatever17:39
pprokopSo this are different folks then fuel-ccp.17:39
inc0let's talk about ideas not companies17:39
srwilkersplaying devils advocate, but sometimes those two are intertwined inc017:40
sdakepprokop operator is a container that contains a controller17:40
srwilkersnot always, but sometimes17:40
sdakepprokop its possible k8s-appcontroller can solve our problems17:40
inc0srwilkers, I'm not saying use appcontroller17:40
pprokopI undeerstand that.17:40
inc0but if they reimplemtn similar approach17:40
*** dwalsh has quit IRC17:40
inc0let's talk about approach17:40
sdakethis falls under definiition of "planning paralysis"17:41
sdakeand "picking winners"17:41
*** TxGirlGeek has quit IRC17:41
jowiszhello, i`m trying to install ceilometer and cloudkitty with kolla-ansible and i`m getting this error17:42
sdakeso appcontroller is out in my book, simply because i am tired of being played by mirantis's management's strategies17:42
*** TxGirlGeek has joined #openstack-kolla17:42
jowiszfatal: [wcm2.stack]: FAILED! => {"changed": false, "failed": true, "msg": "The requested image does not exist: 172.20.0.250:4000/kolla/centos-binary-cloudkitty-processor:3.0.0"}17:42
sdakeentrypoint - none of the kolla-kubernetes core team is keen on for a variety of reasons17:42
pprokopand if it goes to incubator ?17:42
jowiszand [root@kolla-deploy BLUE/WRO ~]# docker images | grep cloudkitty17:42
jowisz[root@kolla-deploy BLUE/WRO ~]#17:42
inc0please please please don't tell me we'll have to write 500loc of golang per every service in openstack https://github.com/coreos/etcd-operator/blob/master/pkg/cluster/cluster.go17:42
jowiszhow can i update docker registry? or pull missing images ?17:42
sdakeok i gotta jet17:43
sdakebut thats a summary of where I stand on this17:43
inc0jowisz, https://github.com/openstack/kolla/blob/master/doc/image-building.rst proponuje budowac:)17:43
jowiszinc0: dzięki ;) zerknę17:44
jowiszale kolega mnie tu pisze, ze robił kolla-build17:44
jowisz;)17:44
inc0anyway, le'ts move to english;) soo, I'm not quite sure if we have cloudkitty17:45
sean-k-mooneyinc0: there is a docker container i think17:46
jowiszsure ;)17:46
jowiszroot@kolla-deploy BLUE/WRO ~]# kolla-build cloudkitty-base cloudkitty-api cloudkitty-processor17:46
sean-k-mooneyinc0: not sure if ansible code is there17:46
jowiszsomething is building :)17:46
sean-k-mooneyjowisz: you can do  "kolla-build cloudkitty"  to build anything with cloudkitty in the name for future reference17:47
jowiszsean-k-mooney: thanks :)17:48
*** duonghq has quit IRC17:53
openstackgerritJeffrey Zhang proposed openstack/kolla: NO_MERGE: test mitaka branch  https://review.openstack.org/39387717:59
*** msimonin has quit IRC17:59
*** pbourke has quit IRC17:59
inc0ok we do have cloudkitty there17:59
inc0I can't vouch for deployment code tho17:59
jowiszunfortunately:18:00
jowiszERROR:kolla.image.build.cloudkitty-base:Error'd with the following message18:00
jowiszERROR:kolla.image.build.cloudkitty-base:The command '/bin/sh -c echo 'binary not yet available for centos'     && /bin/false' returned a non-zero code: 118:00
inc0ahh18:00
inc0that would make sense18:00
jowisz;)18:00
inc0so if you want hacky way to do it18:01
inc0build it with source and retag this name to binary18:01
inc0so kolla-build cloutkitty -t source18:01
*** pbourke has joined #openstack-kolla18:01
inc0look what containers will be built and retag them from  172.20.0.250:4000/kolla/centos-source-cloudkitty-processor:3.0.0 to  172.20.0.250:4000/kolla/centos-binary-cloudkitty-processor:3.0.018:01
jowiszinc0: ok, i will try ;)18:02
*** mkoderer has quit IRC18:03
*** jtriley has joined #openstack-kolla18:03
*** fragatin_ has joined #openstack-kolla18:04
*** nhlfr_ has joined #openstack-kolla18:04
inc0jowisz, remember to do docker push afterwards18:05
jowiszsure18:05
*** Serlex has quit IRC18:06
*** nihilifer has quit IRC18:07
*** fragatina has quit IRC18:07
portdirectkfox1111: ping18:09
mliimasdake, https://review.openstack.org/#/c/393471/18:09
*** unicell has quit IRC18:09
*** nihilifer has joined #openstack-kolla18:10
portdirectinc0: ping18:12
inc0here18:12
portdirecthey  - just redoing that custom image template dir stuff - are we gonna add the custom dir to the tuple to search though?18:14
portdirectlost track of stuff during the meeting :)18:15
awiddersheimanyone able to review?18:15
awiddersheimhttps://review.openstack.org/#/c/279913/18:15
portdirectthink that was the plan - but do you have a peference for if the custom template dir takes presendence or is last to search?18:16
inc0portdirect, I think if you add it it will pick first found, not sure tho18:16
inc0we need to merge multiple dirs really18:16
inc0so mignt need more coding for that18:17
*** sdake has quit IRC18:19
portdirectthats what I was thinking - we would need to make it search on a container by container basis - up for doing it, kfox was also wanting to have the facility to have multiple additional dirs - which makes sense. though its evening here so it''l be the moring before i get round it it.18:20
*** portdirect is now known as portdirect_away18:22
*** ayoung has joined #openstack-kolla18:33
*** unicell has joined #openstack-kolla18:36
*** huikang has joined #openstack-kolla18:38
*** fragatin_ has quit IRC18:47
*** fragatina has joined #openstack-kolla18:48
*** f13o has joined #openstack-kolla18:55
*** harlowja has quit IRC18:59
*** mliima_ has joined #openstack-kolla19:00
*** jtriley has quit IRC19:02
*** jtriley has joined #openstack-kolla19:02
*** mliima has quit IRC19:03
*** diogogmt has joined #openstack-kolla19:03
openstackgerritLarry Rensing proposed openstack/kolla-kubernetes: Adding prechecks script  https://review.openstack.org/39456919:07
*** dcwangmit01 has joined #openstack-kolla19:08
openstackgerritsean mooney proposed openstack/kolla: [WIP] adds support for ovs with dpdk netdev datapath  https://review.openstack.org/34235419:08
*** dcwangmit01_ has quit IRC19:10
*** eaguilar has quit IRC19:10
*** eaguilar has joined #openstack-kolla19:13
*** coolsvap has quit IRC19:17
*** dave-mccowan has quit IRC19:25
*** dave-mccowan has joined #openstack-kolla19:26
*** eaguilar has quit IRC19:28
*** lrensing has quit IRC19:28
*** eaguilar has joined #openstack-kolla19:28
*** chas_ has quit IRC19:33
*** dave-mccowan has quit IRC19:36
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Spec - Kolla-Kubernetes Deployment Architecture  https://review.openstack.org/39225719:40
*** msimonin has joined #openstack-kolla19:41
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Fix new kubeadm precheck  https://review.openstack.org/39577919:47
openstackgerritMerged openstack/kolla: Remove Fedora support from manilla-data  https://review.openstack.org/39407419:51
*** stvnoyes has joined #openstack-kolla19:52
*** msimonin1 has joined #openstack-kolla19:54
*** stvnoyes1 has quit IRC19:55
*** msimonin has quit IRC19:58
*** neilus has joined #openstack-kolla20:01
*** papacz has quit IRC20:04
*** sdake has joined #openstack-kolla20:08
*** gfidente has quit IRC20:13
*** lrensing has joined #openstack-kolla20:13
openstackgerritMerged openstack/kolla: Update manila-data service inventory  https://review.openstack.org/39347120:14
*** n3v3rm0r3r has quit IRC20:16
*** ayoung has quit IRC20:17
*** eaguilar has quit IRC20:18
*** jtriley has quit IRC20:18
openstackgerritSteven Dake proposed openstack/kolla-kubernetes: Add advisory language to specs directory for kolla-kubernetes  https://review.openstack.org/39563620:19
*** ppalacios has quit IRC20:19
*** TxGirlGeek has quit IRC20:24
*** huikang has quit IRC20:25
*** huikang has joined #openstack-kolla20:26
*** TxGirlGeek has joined #openstack-kolla20:28
*** huikang has quit IRC20:30
*** huikang has joined #openstack-kolla20:33
*** chas_ has joined #openstack-kolla20:34
*** mliima_ has quit IRC20:36
*** dave-mccowan has joined #openstack-kolla20:38
*** v1k0d3n has joined #openstack-kolla20:38
*** chas_ has quit IRC20:39
v1k0d3ninc0: hey man20:41
v1k0d3nwhat's up dude.20:41
v1k0d3nsorry missed the meeting this morning.20:41
inc0np, you missed quite a flame war:_)20:41
v1k0d3nhad to attend some things at the conf this morning.20:41
inc0I'll need to ask you about operators thingies20:42
v1k0d3nyeah i hear. unfortunate.20:42
v1k0d3nok, cool. shoot.20:42
inc0nah, that's good, there is some disconnect somewhere20:42
inc0so I took a look at etcd operator20:42
inc0and to me it's a bunch of code in which we ourselves implement code to do stuff with etcd20:42
v1k0d3nright. it's the longer path, i agree.20:43
inc0so instead of just deploying stuff, we create tool that talks to another tool (k8s)20:43
inc0seems...complex20:43
inc0what's the short path?20:43
v1k0d3nbut entrypoint is a short-term iterative approach.20:43
v1k0d3ncoreos and intel (as you know).20:43
v1k0d3nhowever...20:43
inc0yeah, entrypoint will work20:43
v1k0d3nCoreOS is showing a new path.20:43
v1k0d3none that works better.20:44
inc0and I'm not sure if I like it20:44
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Fix new kubeadm precheck  https://review.openstack.org/39577920:44
v1k0d3nbetter logic20:44
inc0I mean, it's the same as putting ansible playbooks into container20:44
v1k0d3nwell, have you run it?20:44
v1k0d3nno it's not, not at all.20:44
v1k0d3nso have another question for you...20:44
inc0go ahead20:44
v1k0d3nbefore we go down this path.20:44
v1k0d3nare you concerned with the time to build this solution?20:44
inc0yeah, and complexity to maintain it20:45
v1k0d3nbecause of the debt to maintain with small group?20:45
v1k0d3nok20:45
v1k0d3ni feel you.20:45
inc0not small group20:45
v1k0d3ni have a meeting at 1:30 with alex polvi to discuss this.20:45
inc0ok20:45
v1k0d3nwhat if i can get their help. would you consider more?20:45
inc0well, I just need to understand why20:45
v1k0d3nhave to be honest, entrypoint was the best solution until operator architecture/model announcement last week.20:46
inc0what's the real value and what are issues with different approaches20:46
v1k0d3nok, here's one immediate.20:46
v1k0d3nansible stays the say.20:46
inc0so operator model to me looks like reimplemented ansible tbh, but I might be mistaken20:46
v1k0d3noperator logic can be plugged in with helm easily.20:46
v1k0d3nare you familiar with kubernetes federated API?20:46
inc0no20:46
v1k0d3nbasically, write your own API20:47
v1k0d3nand use with the kubernetes cluster.20:47
v1k0d3nso with federated API, a controller can have access to the etcd backend, so it knows what is deployed and health of system.20:47
v1k0d3nentrypoint does not.20:47
v1k0d3nit's a build/deploy and forget.20:47
inc0so consider this20:47
kfox1111I missed a good flame war? darn.20:47
kfox1111:)20:47
v1k0d3noperator has try operator logic.20:47
* kfox1111 wonders off to find a log....20:48
v1k0d3nyes kfox111120:48
inc0we create entrypoint as it will deal with deploy20:48
inc0and deps20:48
srwilkershey kfox1111, welcome to the party20:48
kfox1111:)20:48
v1k0d3nbtw... kfox1111 can you weigh in your thoughts on entrypoint vs operator logic (if you know about each).20:48
inc0and on the side we create operator with toolset required for debugging, maintainng and whatnot20:48
v1k0d3ninc0: i have to be honest...entrypoint is a bandaid quickpath.20:48
inc0so let's discuss this, why?20:49
v1k0d3ninc0: deals with deploy....and forget it.20:49
inc0not really20:49
v1k0d3n*sigh20:49
kfox1111which one is the entrypoint thing?20:49
inc0also will deal with all the restart, full downtime of cluster and whatnot20:49
v1k0d3ni just need to be clear....20:49
kfox1111that the process that slips in the container init to block it from starting until ready?20:49
v1k0d3nwe need logic of an operator.20:49
inc0kfox1111, entrypoint would be init-container that will wait for all the deps to be met20:49
v1k0d3nit's new, i agree.20:49
kfox1111ah.20:50
rhalliseykfox1111, lol yes :)20:50
v1k0d3nhowever...that is the future20:50
kfox1111actually, I think both have their place in a way.20:50
rhalliseyv1k0d3n, we're trying to outline this in the spec20:50
inc0operator smells to me like an abstraction layer over an abstraction layer (k8s)20:50
kfox1111i for sure believe the orchestration/workflow logic has no buisness what so ever being in the normal containers.20:50
v1k0d3nit is the theme that is reenforced at this conference i'm at right now.20:50
openstackgerritLarry Rensing proposed openstack/kolla-kubernetes: Adding prechecks script  https://review.openstack.org/39456920:50
rhalliseybut ultimatly I think the poc will tell the most20:50
v1k0d3none of the reason i wanted to go to this was to share back.20:50
v1k0d3nso i'm sharing :)20:51
kfox1111the endpoint thing is just a way to block until deps come up, for containers that don't just do the right thing already when their deps are not up.20:51
inc0god I wish I could have this discussion out there20:51
rhalliseykfox1111, flame war is a good read :)20:51
kfox1111ideally the daemon itself (nova-api) would just block until its deps, mariadb comes up. not crash.20:51
v1k0d3nok... inc0 i have an ask20:52
v1k0d3nplease....20:52
kfox1111hmm.. yup. its long. wil read. :)20:52
v1k0d3ngo out to the helm group. go out to kubernetes. start asking about operators.20:52
v1k0d3nthey are not the same thing.20:52
v1k0d3nand also...20:52
kfox1111I still lothe that term. :/20:52
v1k0d3nlogically20:52
v1k0d3nCOREOS is the team behind entrypoint20:52
kfox1111I'm an operator.20:52
v1k0d3nand NOW operator.20:52
kfox1111now an otomiton is stealing the term. :)20:53
v1k0d3nlol20:53
kfox1111but seriously,20:53
kfox1111it makes it hard to disucss our users.20:53
srwilkerstheyre sticking you in software kfox20:53
v1k0d3nshould be controller, but don't want to go there.20:53
srwilkersbeware20:53
v1k0d3nlo20:53
v1k0d3nlol20:53
inc0v1k0d3n, I'm not defending entrypoint as the best solution there is20:53
kfox1111"human operator does x" "operator does y"20:53
srwilkerswelcome to the matrix20:53
inc0I just took a look at etcd operator and saw couple k lines of golang20:53
kfox1111I don't think coreos developing both technologies is nessisarily bad.20:53
inc0I don't want to maintain multi-thousand loc files per service20:54
kfox1111I think they fill different gaps for the most part, with a bit of overlap.20:54
v1k0d3ni think some meetings need to be had20:54
v1k0d3nthis group (cores)  and coreos and helm20:54
inc0possibly, I might misunderstand stuff20:54
kfox1111the deal with etcd operator is, its a virtual human maintaining etcd with a bunch of logic. its all the worflow bits a real operator needs to do.20:54
kfox1111thats why its so much code. :/20:55
v1k0d3nlook, both coreos and helm are telling us (large company) operators are the way to go, with helm managing depends building (using abstracted config maps).20:55
kfox1111openstack's more complicated then etcd. it will probably need quite a bit of code to orchestrate as well.20:55
inc0and that's what ansible normally does..20:55
rhalliseyI mean it has to be20:55
kfox1111comparable to kolla/ansible/* :/20:55
rhalliseyopenstack lifecycle is challenging20:55
v1k0d3ni don't know why there is so much resistance to it, just to implement an older concept...that will force the group to re-write ansible portion of kolla.20:55
inc0yeah, operator seems to me like puting ansible bootstrapping into container and call it a proud name20:55
kfox1111inc0: thats what it is really.20:56
v1k0d3nand i'm saying that i can talk to alex about getting help in this repo to write the controllers.20:56
kfox1111it could even be done using ansible.20:56
*** psanchez has quit IRC20:56
v1k0d3nyou only need a few.20:56
kfox1111the nice thing is, it hides it in containers though. ;)20:56
inc0only worse because youy need to reimplement ansible where you're at it20:56
kfox1111not really.20:56
kfox1111k8s does all the deployment bits. you just need to use ansible to orchestrate k8s.20:56
inc0from within a container20:57
kfox1111yeah.20:57
v1k0d3nkfox1111: no20:57
kfox1111containers just packaging.20:57
inc0because obviously when we did that outside of container it was dirty and bad;)_20:57
v1k0d3nno ansible to orchestrate k8s20:57
*** slagle has quit IRC20:57
v1k0d3nthat's a horrible path forward20:57
kfox1111inc0: it was visible then. ;)20:57
*** stevebaker has quit IRC20:57
inc0v1k0d3n, golang instead of ansible..20:57
kfox1111v1k0d3n: short term and long term goals.20:57
rhalliseyI wouldn't say you re implement ansible, but only the workflow aspect of it20:57
rhalliseynone of the layering20:57
kfox1111short term is to get a poc of orchestration going at all.20:57
inc0ansible is a workflow;)20:57
v1k0d3nguys...helm is already written for this team20:57
rhalliseyit's really heat20:57
rhalliseyimo20:57
kfox1111the nice thing about operators is language doesn't matter.20:57
v1k0d3nthat's why SAP came out with the helm stuff20:58
v1k0d3ngo try it out20:58
kfox1111so it can be switched out by go at any time.20:58
v1k0d3nexactly with kfox1111 that lang doesnt matter20:58
v1k0d3nuse python20:58
kfox1111python would work too.20:58
v1k0d3nif there are scale issues...then use go, but i don't think there's going to be any issue20:58
v1k0d3nuse what the group knows. python. perfect.20:58
kfox1111yeah. the workflow bits will be pretty trivial. python would work pretty well.20:58
v1k0d3ninc0: ansible is a BAD workflow20:59
rhalliseykfox1111, python will be good at first.  bugroger mentioned it will be a much taller task than using go20:59
inc0and let's use this nifty workflow tool written in python called ansible;)20:59
v1k0d3nold20:59
v1k0d3nnot new or cloud native approach20:59
inc0I'm teasing now20:59
v1k0d3nah. hard to tell over IRC :)20:59
inc0but it does seem to be like putting workflow inside container20:59
kfox1111v1k0d3n: seperate ansible the engine from ansible the deployment tool. they are in the same tool but both don't have to be used.20:59
inc0and if you'd use ansible module for talking to k8s21:00
inc0that can be very well implemented as a playbook21:00
v1k0d3nomg...have you used the ansible mod for kubernets?21:00
v1k0d3ndidn't work well last i tried.21:00
rhalliseyyes21:00
v1k0d3nok.21:00
inc0not saying it's good, command: kubectl... will work too21:00
rhalliseyv1k0d3n, I fixed it, but my patches aren't being accepted21:00
v1k0d3nomg....21:00
rhalliseybeen sitting there for 6 months21:00
inc0ok, forget ansible21:00
inc0I just used is as a reference21:01
kfox1111rhallisey: yeah. thats a big problem.21:01
inc0you're putting << insert your favorite workflow mechanism >> insice a container and call it a proud name21:01
kfox1111I think python is prbably best tradeoff for now because of that.21:01
v1k0d3nkubectl and ansible are not good. why would we do this when we have an awesome API, and a kubernetes project supported kubernetes deployment, and depends tool?21:01
kfox1111inc0: no, theres an abstraction here...21:01
kfox1111thirdparty resource is the contract.21:01
inc0you mean k8s?21:02
kfox1111the oprator, whatever language, consuimes it and makes it work.21:02
v1k0d3nare you talking about federated API?21:02
kfox1111the language doesn't really matter to the user, only the contract does.21:02
v1k0d3nyou lost me kfox111121:02
kfox1111k. example.21:02
kfox1111nova-api is written in python right now.21:02
v1k0d3nwell...ok21:02
v1k0d3nso we're getting off track21:02
kfox1111does random user care its in python, or that the rest-api works consistantly?21:02
v1k0d3nso...kolla-k8s is going ansible and entrypoint?21:03
v1k0d3njust need to know.21:03
kfox1111it can be rewritten in go tmorrow and I wouldnt' care. but if /v2/... breaks, I would care.21:03
rhalliseyv1k0d3n, no21:03
v1k0d3nthis is absolutely huge for us.21:03
inc0kfox1111, but what I mean is, we're writing workflow and putting it into container right?21:03
rhalliseyv1k0d3n, at least not atm21:03
v1k0d3nit will be the disconnect.21:03
kfox1111with operators, the thirdparty entrypoint/yaml description is the contract. the api for the operator.21:03
inc0that's all operator is?21:03
kfox1111what consumes it, and makes it realize the request, is whats in the container, and what the user probably doesn't care what language its written in.21:03
kfox1111the interface they are using to implement operators is the k8s thirdpartyresource.21:05
rhalliseyv1k0d3n, I outlined it in the spec what's currently being considered: https://review.openstack.org/#/c/392257/21:05
kfox1111which is just a way to extend k8s with new object types.21:05
kfox1111so, say, assuming you had a single openstack operator for all of kolla.21:05
kfox1111we'd create an openstack resource type, that loks like the contents of globals.yml.21:05
*** dwalsh has joined #openstack-kolla21:06
kfox1111the operator would then be handed it, run genconfig, and launch containers with all the config bits generated from it.21:06
inc0which would basically be kolla-ansible that changes kolla_docker starting containers to kubectl21:06
kfox1111when the user wants to upgrade, they change the gloals to say, s/mitaka/ocata/21:06
kfox1111the operator would see the change, run through an upgrade procedure on each of the containers/configs.21:07
inc0only inside pod21:07
kfox1111yeah, and driving k8s.21:07
inc0that's not as bad as it sounds, but meh, just sayin', operators = I'm not amused21:07
kfox1111yeah. its just a different way of doing the workflow stuff we've talked about adnosium. :)21:08
inc0yet another wheel21:08
v1k0d3nso inc0 we just need a direction. sort of quickly from our pov. i think a lot of this is just unclear understanding of the operator and entrypoint differences.21:08
kfox1111it can be external to k8s with ansible/chef/whatever,21:08
rhalliseyinc0, heh :)21:08
kfox1111or can be done internal in k8s as an operator.21:08
v1k0d3nremember, i was a huge supporter of entrypoint a couple weeks ago :)21:08
v1k0d3nat the time, it was the fastest path.21:08
inc0v1k0d3n, so from my point of view....we're re-implementing workflow21:08
inc0inside a container21:08
kfox1111so, the really shortest path is to continue with the existing kolla-kube-ansible stuff.21:09
v1k0d3nmisunderstanding21:09
inc0we could do it outside of container21:09
*** fguillot has quit IRC21:09
v1k0d3nand that will break what we need21:09
v1k0d3nand NO21:09
v1k0d3ncontainer needs to do it.21:09
kfox1111it can be run outside k8s, or can be crammed into a contianer and wrapped in an operater very esily.21:09
v1k0d3nunderstanding of a cloud native app is huge21:09
kfox1111then both types of workflows can be evaluated.21:09
v1k0d3ni think education is really important here.21:09
v1k0d3neven evaluating each is going to stall what we need.21:09
kfox1111agreed. and this stuff is like a week old.21:09
kfox1111how do you know operators are the way to go now?21:10
v1k0d3n^^ i agree with that.21:10
v1k0d3nkfox1111: this was the whole reason for coming to kubecon21:10
v1k0d3nyes21:10
kfox1111we should figrure out an quick/easy way to evaluate several things.21:10
v1k0d3nthis is the theme21:10
kfox1111then pick the best. :)21:10
inc0so I will bet money on it that when somebody will start using etcd operator they'll find all the nice edge cases that happends in distributed systems21:10
v1k0d3ni have a better plan guys.21:10
kfox1111I do think operators have a very compelling story behind them.21:10
v1k0d3none common theme here.......21:10
inc0race conditions during operations21:10
kfox1111its just really really young.21:11
inc0and stuff like that21:11
*** aric49 has joined #openstack-kolla21:11
v1k0d3ninc0: race conditions?21:11
rhalliseyagreed, we don't know the best path. Code hasn't been written and tested21:11
v1k0d3nare we not limiting operators?21:11
v1k0d3ncgroup limit them21:11
inc0when operator thinks it's already deployed but it isnt21:11
rhalliseyalso the reference architecture for operators within openstack is non existent21:11
inc0you know what would be great?21:11
v1k0d3nthat's the point....logic is built in.21:11
rhalliseynot within21:11
v1k0d3nit's binary, it's there or not21:11
inc0write a DLS dedicated for operators21:11
rhalliseyusing operators for a complex app like openstack21:11
inc0DSL21:11
kfox1111inc0: I don't think its racy. there is one operator container running, getting the requested state.21:12
v1k0d3nlot of misunderstanding21:12
v1k0d3ni think we need to have meetings with coreos.21:12
v1k0d3nthis is pivitol21:12
inc0kfox1111, what of fail tolerance? you can store only so much state in k8s21:12
v1k0d3n*pivotal21:12
kfox1111inc0: yeah, the idea of thirdparty resources are to extend k8s using k8s.21:12
v1k0d3nif we deploy entrypoint because we "feel" it's better....21:12
v1k0d3nand rewrite ansible.21:12
v1k0d3nand learn that operators are the way to go....21:13
kfox1111so it stores its state in a k8s configmap or something if it needs to stave state, or rebuilds state by quering k8s on startup.21:13
inc0maybe what we need is cloud-native-container-native-k8s-running version of ansible21:13
rhalliseyI mean do you guy think operators are not it? Maybe we need to invent something else here21:13
kfox1111entrypoint is not an orchestration system.21:13
kfox1111itss just a way to block stuff.21:13
inc0nah, it's simple mechanism to avoid race conditions during deployment21:13
inc0really really easy stuff21:13
kfox1111I do think operators are likely a good path for implementing workflow.21:13
v1k0d3n maybe what we need is cloud-native-container-native-k8s-running version of ansible = total contradiction fyi21:13
kfox1111I really do.21:14
inc0v1k0d3n, ansible as in workflow engine21:14
rhalliseyv1k0d3n, yes but that's not on their roadmap21:14
kfox1111operators though are about api more then anything.21:14
inc0"do thing, wait till thing is done, do another thing"21:14
kfox1111so language really doesn't matter.21:14
rhalliseyanything of the sort21:14
kfox1111python/go/ansible, all would work I think.21:14
v1k0d3nkfox1111: exactly right21:14
v1k0d3nAPI driven21:14
v1k0d3ncloud native21:14
v1k0d3ndynamically scheduled.21:14
kfox1111its basically the same contract k8s provides.21:15
v1k0d3nansible is not dynamically scheduled.21:15
v1k0d3nthat is human operator scheduled.21:15
inc0it can be;)21:15
kfox1111"give me a declarative definition of what you want, and k8s will make it happen"21:15
inc0like puppet21:15
rhalliseycould you say OpenStack is not though?21:15
kfox1111for operators, k8s is extended by an container running in the same k8s.21:15
inc0same thing;)21:15
inc0maybe puppet-k8s21:15
v1k0d3ninc0: omg...smh21:15
inc0?21:15
v1k0d3nok21:15
inc0v1k0d3n, let's have a hangout21:15
v1k0d3nthere's misunderstanding here.21:15
inc0with coreos guys21:15
kfox1111puppet-k8s would work too. language really really doesn't matter.21:16
inc0I'm sure there is21:16
v1k0d3ni can't. meeting with CoreOS to talk about this very subject21:16
kfox1111whatevers the best tool for the job.21:16
v1k0d3ni thought this would be so much easier.21:16
v1k0d3nthere's too much religion21:16
inc0v1k0d3n, I'm not *against* it21:16
inc0no religion involved21:16
inc0I might just misunderstand the idea21:16
rhalliseyv1k0d3n, this is a tough problem b/c openstack and kube interaction21:16
kfox1111v1k0d3n: i'm not against it either. nor religious about it.21:16
v1k0d3ninc0: we're not moving forward.21:16
inc0totally possible21:16
v1k0d3ndays of discussion around this.21:16
inc0v1k0d3n, I want to know where we going before we move21:16
inc0days? it exists a week!21:17
rhalliseyI think we're moving forward with the spec and POCs happening21:17
inc0in total21:17
kfox1111v1k0d3n: no, I think its that most people really do't have a handle on operators yet.21:17
kfox1111so more of a push back from lack of understanding I think.21:17
kfox1111more education is needed.21:17
v1k0d3ninc0: ....we're not using etcd exactly as is.....21:17
inc0v1k0d3n, how about you guys alongside with sdake write a poc of operator-driven deploy of mariadb+keystone21:17
v1k0d3nthis is a MODEL21:17
v1k0d3na show of how it should be done.21:17
v1k0d3nthis is just an architecture logic.21:17
v1k0d3ndifferently explained.21:17
v1k0d3nit's not out a week....the concept was there all along, just someone showed us a better path.21:18
inc0v1k0d3n, can you do that for me?21:18
v1k0d3nit's an idea...not a technology.21:18
inc0regardless of language used21:18
kfox1111yeah. a phylosophy.21:18
inc0just deploy mariadb and keystone with it21:18
v1k0d3n^^ right21:18
kfox1111extend k8s with a container to allow the user to request a particular declarative object, and the operator drives k8s to make it realized.21:19
inc0and I get that, all I'm saying is workflow automation is hard problem21:19
inc0really hard21:19
v1k0d3nso a comment like "it's only been out a week" is like finding water underground. it's always been there.21:19
kfox1111inc0: +121:19
inc0especially if we want distributed workflow automation21:19
*** neilus has quit IRC21:19
kfox1111all around, I don't think this matters too much.21:19
kfox1111the workflow is the hard part.21:19
kfox1111whether its drivven by data fed through k8s thirdparty resources,21:19
v1k0d3ni agree.21:19
inc0yeah, whether we put it inside container or not21:19
kfox1111or by the user directly,21:19
v1k0d3ninc0: my whole point of coming in here is to get buyin....21:20
kfox1111the logic is the hard part.21:20
v1k0d3nlisten for a sec.21:20
kfox1111the input format is easy.21:20
v1k0d3nif you would agree to operator, i have more of a case with Alex at 1:3021:20
v1k0d3nto say, "we can explore this, can you help"21:20
v1k0d3nif we have no path, i have nothing to say to him21:20
inc0and I'm not too keen in implementing this logic on programming language level21:20
v1k0d3nzero leverate.21:20
v1k0d3nleverage21:20
kfox1111I'm all for giving operators a shoit. I think they are a good thing on the face of it.21:20
kfox1111shot21:20
inc0v1k0d3n, say that we agree21:20
rhalliseyv1k0d3n, that's what we're doing (me and sdake)21:20
inc0and we'll deal with issues later21:20
rhalliseyv1k0d3n, tell him to read the spec21:21
inc0I'm just worried that we underestimate complexity of this thing21:21
v1k0d3nlet me find out from alex what they are doing with stackanetes and the forward path.21:21
kfox1111yeah. the spec lis looking pretty good.21:21
kfox1111inc0: thats a problem if workflow, not really operators.21:21
v1k0d3nand if they say "we need operators" then what?21:21
kfox1111I think.21:21
v1k0d3nthey want to go back to kolla...this much i know21:21
kfox1111workflow itself is really hard/complicated.21:21
v1k0d3nbeen said several times21:21
v1k0d3ngotta run guys. close to time.21:21
inc0v1k0d3n, ok, if operators will get them back, greay21:21
kfox1111an operator is mostly just wrapping/syntactic sugar21:21
rhalliseyv1k0d3n, that's good. Let's start with the operator common ground and go form there21:21
inc0let's roll with this21:21
v1k0d3njust wanted to share.21:21
v1k0d3nlet me find out. be back in a bit21:22
inc0again I'm not opposed, just this thing is really complex21:22
*** v1k0d3n has quit IRC21:22
rhalliseysince we clearly don't know the final answer here21:22
rhalliseywe need to have some POCs21:22
inc0and I don't want to reinvent wheels21:22
inc0these are my only reservations21:22
rhalliseyinc0, I agree dude21:22
kfox1111inc0: +121:22
rhalliseywe need to find how this fits kube and OpenStack21:22
inc0I go to etcd controller and see golang code implementing workflow for deployment of single (complex) application21:23
rhalliseyI think we all believe it can, but the how is still WIP21:23
inc0I could very well implement it in python, but we didn't we used ansible already cuz ansible helps with certain stuff, like failure handling and whatnot21:23
rhalliseyinc0, ehh failure handling :), but poor error reporting21:24
inc0maybe what we need is clever way to implement stuff with operators without writing EVERYTHING from scratch21:24
kfox1111k8s itself helps alreayd with a lot of failure cases.21:24
inc0maybe that's a good idea for operator-centric project21:24
rhalliseyinc0, yes I agree, but I think the crux here is that operators are a good step21:24
srwilkersreally loving how active this chats been the past week21:24
rhalliseyimno21:24
inc0to write a DSL for it21:24
kfox1111mostly the operator is just driving k8s to do a few things. if we do it per service level,21:24
kfox1111its probably 5 or 6 steps max per operator.21:24
rhalliseysrwilkers, :)21:24
inc0ah, we lost Brandon21:25
inc0duh21:25
srwilkersyeah, he had to jet21:25
inc0ok, on the bright note, http://paste.openstack.org/show/588612/ <- put this into globals.yml and it will include contents of file specified into playbook21:26
inc0ansible customizations \o/21:26
inc0I got it almost working21:26
kfox1111nice.21:26
srwilkersso rhallisey, you and sdake are working on the operator poc yeah?21:27
kfox1111we need to figure out how to do similar things with kolla-kubernetes too.21:27
inc0kfox1111, won't helm help with that?21:27
kfox1111some.21:27
kfox1111I'm more talking about genconfig ini file overrides.21:27
kfox1111there isn't an easy way to override some values currently I think.21:27
kfox1111I think kolla-ansible supports that, but I think it was doing it on the host using ansible, which doesn't work for kolla-kubernetes.21:28
*** psanchez has joined #openstack-kolla21:28
sdakewow lacks of scrollback21:29
kfox1111indeed21:29
sdakelots21:29
sdakeautocorrect failure21:29
*** slagle has joined #openstack-kolla21:31
sdakehey slagle how goes it21:31
sdakeinc0 re clever way to implement stuff with controllers - thats with a base class that does everything\21:32
sdakei have thought about a dsl too, but that starts to infringe on ansible territory i think21:32
inc0what I'm saying is - we're reinventing wheel there, let's make sure it's nice and round21:33
kfox1111[A[B[B[B[B21:33
inc0tmux?21:33
kfox1111screen. :)21:33
kfox1111wifi partially died.21:33
*** Jeffrey4l has quit IRC21:34
*** stevebaker has joined #openstack-kolla21:34
kfox1111sdake: yeah. I think the vast majority of the services just "create mysql db, create keystone entries, start k8s deployment object"21:34
kfox1111so the code will mostly be the same for the operators.21:34
kfox1111the differences usualy are in the upgrade code.21:34
sdakekfox1111 a wrapper only installs a command which does orchestration21:34
rhalliseysrwilkers, ya21:35
*** chas_ has joined #openstack-kolla21:35
*** nihilifer has quit IRC21:36
kfox1111sdake: yeah. I'm saying I think if we write a python operator for most of the base services, they can all inherit from a base class that does those actions and is basially just a sub to allow us to add upgrade logic specific to the service later.21:37
*** fguillot has joined #openstack-kolla21:38
inc0ehh21:38
inc0I think this whole model is overhyped21:38
inc0this is just a fucking script that does stuff21:38
inc0no matter how you call it21:39
kfox1111inc0: overhyhed/under-understood, but still good.21:39
inc0as good as script written21:39
kfox1111it just makes it such that you can kubectl create -f mycloud.yaml21:39
kfox1111intead of calling pythonscript mycloud.yaml21:39
inc0yeah21:39
inc0pretty much21:39
inc0make shipping easier21:39
*** ayoung has joined #openstack-kolla21:40
kfox1111yeah.21:40
kfox1111as you can:21:40
kfox1111helm install kolla-operator21:40
kfox1111kubectl create -f mycloud.yaml21:40
kfox1111profit. :)21:40
*** chas_ has quit IRC21:40
kfox1111no non k8s stuff needed installed.21:40
inc0just this script in python will be fuckton of work to make it well21:40
kfox1111could be.21:40
pronemm i belive we have an error here http://docs.openstack.org/developer/kolla/multinode.html21:41
inc0with infinite resources every model is a good one21:41
kfox1111but in an operator or in a stand alone script, thats the same work either way.21:41
inc0whats that pron ?21:41
pronMountFlags=shared missing in ubuntu service file21:41
proni think at least , since i am trying to play with kolla atm21:42
*** aric49 has quit IRC21:42
inc0you're right21:42
*** aric49 has joined #openstack-kolla21:42
inc0wanna make a fix?:)P21:42
*** aric49 has quit IRC21:42
sdakekfox1111 know much about python?21:43
sdakethats the base class i was speaking of ;)21:44
*** nihilifer has joined #openstack-kolla21:44
* pron hides21:44
inc0still, fuckton of work to make it correctly21:44
srwilkersinc0, pron: i can fix it right quick if you dont want to21:45
pronlets make i deal , i dont do fixes but report stuff here :P most likely a lot since our plan is to use kolla :P21:45
sdakei'd admit makign upgrade work correctly will be work21:45
pronmake a deal*21:45
sdakedeploy will be easy21:45
sdakepron file bugs :)21:46
inc0pron, sure, any help is appreciated21:46
pronwell ;D i will find a ways to break stuff21:46
inc0tha'ts core reviewers issue to spot21:46
rhalliseysdake, ya doing the entire life cycle amplifies the problem21:46
inc0don't worry, it's not that easy to break stuff pron21:46
inc0but it is cool feeling when you contribute21:46
pronhehe lets see i alrdy spent some time with weird stuff that i wasnt avare of :P21:46
pronfirst of all i am tryign to do that in mirantis os deployment :D21:47
inc0anyway, I'd encourage you to try21:47
inc0like...deploy OS with nailgun or ...?21:47
inc0sdake, no, it won't be that easy21:47
inc0even deploy21:47
pronnono i have tht mirantis thing deployed for well the sake of having it before i vent to barceolna21:47
inc0remember compose?21:48
inc0think if this shellscript we used as operator21:48
inc0put it into container21:48
sdakeburned into my brian21:48
inc0and tha'ts it21:48
srwilkersim sure brian doesnt appreciate that sdake21:48
inc019 out of 20 times it works21:48
pronanyways i like that bugs are welcomed here :P so i will bring em if i spot something21:48
sdakesrwilkers nope21:48
inc0thanks pron21:48
sdakepron our bug triage is a little laggy21:48
sdakepron so if somethign critical comes up feel free to point it out21:49
*** harlowja has joined #openstack-kolla21:49
sdakepron however the common services rae in rock solid shape21:49
pronwell at least i am not turned away :P list most projects newdays do21:49
pronlike* ( sorry i type liek dislectic )21:49
inc0nah, bugs are greatly appreciated and we thank you for them21:49
*** huikang has quit IRC21:50
inc0just in certain cases making a patchset with fix takes even less time;)21:50
*** huikang has joined #openstack-kolla21:50
*** mgiles has quit IRC21:52
*** huikang has quit IRC21:54
*** jheroux has quit IRC22:04
kfox1111sry. back.22:05
*** msimonin has joined #openstack-kolla22:05
*** msimonin1 has quit IRC22:06
kfox1111inc0: compose is too single minded/static.22:06
kfox1111k8s behaves quite a bit differently.22:06
kfox1111so it will probably be a different set of issues. :)22:07
*** huikang has joined #openstack-kolla22:07
inc0I know, we actually have source to query state of things22:07
*** fguillot has quit IRC22:07
*** sdake has quit IRC22:09
kfox1111:)22:12
kfox1111stupid kubeadm preflight checks... :/22:13
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Fix new kubeadm precheck  https://review.openstack.org/39577922:14
*** lrensing has quit IRC22:15
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla: DO NOT MERGE!: PoC of ansible override  https://review.openstack.org/39585122:22
*** schwicht has quit IRC22:23
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla: DO NOT MERGE!: PoC of ansible override  https://review.openstack.org/39585122:24
*** neilus has joined #openstack-kolla22:25
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Fix new kubeadm precheck  https://review.openstack.org/39577922:29
*** n3v3rm0r3r has joined #openstack-kolla22:29
*** berendt has quit IRC22:36
*** chas_ has joined #openstack-kolla22:36
*** chas_ has quit IRC22:41
*** schwicht has joined #openstack-kolla22:45
*** rhallisey has quit IRC22:48
*** dwalsh has quit IRC22:51
*** dwalsh has joined #openstack-kolla22:52
*** dwalsh has quit IRC22:57
*** tonanhngo has quit IRC22:57
openstackgerritOpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements  https://review.openstack.org/39532422:57
*** schwicht has quit IRC22:58
*** dwalsh has joined #openstack-kolla22:58
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Fix new kubeadm precheck  https://review.openstack.org/39577922:59
*** Pavo has joined #openstack-kolla23:01
*** schwicht has joined #openstack-kolla23:02
*** neilus has quit IRC23:08
*** neilus has joined #openstack-kolla23:08
*** harlowja_ has joined #openstack-kolla23:08
openstackgerritSteve Wilkerson proposed openstack/kolla: Added MountFlags for Ubuntu in multinode guide  https://review.openstack.org/39586523:08
*** sdake has joined #openstack-kolla23:09
*** harlowja has quit IRC23:12
*** schwicht has quit IRC23:12
*** msimonin has quit IRC23:14
*** alteriks has quit IRC23:15
*** saneax-_-|AFK is now known as saneax23:15
*** f13o has quit IRC23:16
*** ayoung has quit IRC23:16
*** huikang has quit IRC23:16
*** huikang has joined #openstack-kolla23:17
*** tonanhngo has joined #openstack-kolla23:17
*** alteriks has joined #openstack-kolla23:19
*** TxGirlGeek has quit IRC23:21
*** huikang has quit IRC23:21
*** tonanhngo has quit IRC23:22
*** ayoung has joined #openstack-kolla23:28
kfox1111sdake: alive?23:31
sdakewound me kfox111123:31
*** Jeffrey4l has joined #openstack-kolla23:31
kfox1111what was the etherpad url you give me to put helm stuff in?23:31
kfox1111I came up with another one.23:31
Pavoevening23:31
*** Jeffrey4l has quit IRC23:32
Pavoquestion23:32
*** Jeffrey4l has joined #openstack-kolla23:32
Pavoif I am using Ceph on both Controller and Compute nodes, why would Horizon only show the Compute node amount and not both23:32
*** dwalsh has quit IRC23:33
*** Jeffrey4l has quit IRC23:33
sdakekfox1111 no idea23:34
*** msimonin has joined #openstack-kolla23:34
*** Jeffrey4l has joined #openstack-kolla23:34
kfox1111was one of the summit ones I think.23:36
*** jrich523 has quit IRC23:37
kfox1111found it. https://etherpad.openstack.org/p/kolla-ocata-summit-kolla-k8s-road-map23:37
inc0sdake, https://review.openstack.org/#/c/395851/223:39
kfox1111sdake: can you read hte last bullet there and see if it makes sense?23:40
*** tonanhngo has joined #openstack-kolla23:41
*** lamt has quit IRC23:42
*** jrich523 has joined #openstack-kolla23:44
jascott1 im using kolla keystone image with helm, (openstack-helm uses their own custom images) and I am just trying to get the keystone container up23:45
jascott1I see this env var KOLLA_BASE_DISTRO=ubuntu23:45
jascott1but when i let the container do its bootup, it fails trying to start httpd23:45
jascott1which should only occur on non-debian23:45
jascott1does it not use KOLLA_BASE_DISTRO=ubuntu env var?23:45
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Fix new kubeadm precheck  https://review.openstack.org/39577923:46
kfox1111I'm guessing that comes from /etc/kolla/horizon/config.json23:48
kfox1111or however your mapping that in.23:48
kfox1111not sure the env var does anything for that?23:49
jascott1hmm i will check there thx23:49
kfox1111actually,23:50
kfox1111there is code in /usr/local/bin/kolla_extend_start inside the container that seems to use it.23:50
*** tonanhngo has quit IRC23:50
jascott1ya config.json just says which config files to copy and what perms to set23:50
kfox1111plus what command to execute.23:51
kfox1111so has an httpd command.23:51
kfox1111not sure that varies on ubuntu/centos though.23:51
kfox1111I know debian renames their stuff apache223:51
*** inc0 has quit IRC23:52
jascott1im going to petition new pres to require all containers to include vim-tiny ;)23:52
jascott1ya that script looks like it just pulls from env... not sure whats going on23:53
srwilkersyes pls23:53
kfox1111yeah. I'm having a hell of a time transitioning to a vi world from vim.23:54
jascott1you can actually mount vim tiny as volume, used to do that w docker23:54
kfox1111I want vim when I type vi if its installed, but that half the time works and half not. :/23:54
kfox1111so just typing vi doesnt' always get me what I want. :/23:54
kfox1111I've asked k8s to provide something sligtly different.23:54
jascott1yeah I always type vi23:55
kfox1111what I want, is an extension of the sidecar idea.23:55
kfox1111a toolbox sidecar.23:55
jascott1not familiar tho I have seen that term around23:55
kfox1111then you attach the sidecar, get a shell from the sidecar, with vim, gdb, strace, tcpdump ,etc23:55
kfox1111and but the filesystem would be from the container your debugging. :)23:55
*** tonanhngo has joined #openstack-kolla23:55
jascott1ah cool23:55
kfox1111so you could keep your main containers really thin :)23:56
srwilkersthats actually pretty great23:56
jascott1without exposing the filesystem from main conainer to the sidecar manually I suppose23:56
kfox1111yeah.23:56
*** chas_ has joined #openstack-kolla23:56
kfox1111would be super useful.23:56
kfox1111each admin could even then have their own custom toolbox. and when detached, it just vanishes. :)23:56
kfox1111someone totally needs to implement that. :)23:57
jascott1sounds like everyone would like it23:57
kfox1111pile on: https://github.com/kubernetes/kubernetes/issues/27464 :)23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!