Friday, 2016-11-18

kfox1111well, I think the complication is going to be in the live upgrading workflows each of the openstack components come up with.00:00
alanmeadowsis that it would be better if it could provide the gap as k8s functionality is missing to make this seamless with its own upgrade/rollback workflows00:00
kfox1111the workflow's gotta be implementable in whatever workflow helm provides.00:00
alanmeadowsthat maybe naive00:00
kfox1111or needs a k8s extention to do that logic.00:00
alanmeadowsbut thats where my thinking is until I get deeper into it00:00
kfox1111yeah.00:00
alanmeadowsright so ideally one says00:01
kfox1111I've been coming from the, I want to put this into production as soon as I can. which means,00:01
alanmeadowsi want to go fully native for as much k8s work as I can do for each component00:01
kfox1111manual workflow. which I'm good with. :)00:01
kfox1111cause I'm already manualy deploying openstack with packages. so k8s openstack is a huge step up. :)00:01
alanmeadowsbut there will be gaps and something has to fill it, hoping helm can (probably not right now)00:01
* kfox1111 nods00:01
alanmeadowsand also whats going to help with a larger upgrade orchestration, again _maybe_ helm, or maybe thats asking too much00:01
alanmeadowsdon't know, being optimistic ;-)00:02
kfox1111yeah. k8s seems to really have a good toolset for deploying webapps. and openstack's pretty much a web app.00:02
kfox1111with the possible exeption of the sql db stuff.00:02
kfox1111its just a hugely complicated webapp. :)00:02
*** ayoung has quit IRC00:02
alanmeadowsright the devil is the orchestration because of the spaghetti ties00:02
alanmeadowsand for a super robust approach00:02
kfox1111I have no doubt helm will be able to orchestrate a web ap that complicated eventually.00:03
alanmeadowsenough state saving so you _can_ rollback00:03
kfox1111just not sure when. if it can now, or if it will take a while.00:03
kfox1111yeah.00:03
kfox1111having one superpackage for the whole deployment is cool for deployment.00:03
kfox1111pretty scary for day2 when you have to fix one little thing in one k8s object.00:03
alanmeadowsyeah based on the documentation i've absorbed one big superpackage for upgrades doesn't seem like something helm has really architected just yet00:04
kfox1111you could tweak the super pakcage and upgrade. but it carries the resk of breaking everything.00:04
*** msimonin1 has quit IRC00:04
alanmeadowsseems like you could stitch some magic together with the pre and post hooks too00:04
kfox1111yeah.00:04
alanmeadowswhich I had to discover with the ceph chart00:04
kfox1111jobs are really cool. :)00:04
alanmeadowsand jobs ;-)00:05
kfox1111I've yet to try out the cronjob type but it looks really nice too.00:05
kfox1111I'm planning on giving that a try for fernet token rotation at some point.00:05
alanmeadowsthe concept of hooks is there00:05
alanmeadowshttps://github.com/att-comdev/aic-helm/blob/master/ceph/templates/pre-install-secret.yaml00:05
alanmeadowsso that gives me hope00:05
alanmeadowsjust needs expansion00:06
jascott1if we were to ugprade sub chart and then upgrade whole release, is helm smart enough to just run package that has version change?00:06
kfox1111yeah. that was new since I looked at it last. that will go a long way.00:06
kfox1111jascott1: not sure....00:06
alanmeadowsmy adventures with upgrades is really thin00:06
kfox1111jascott1: I also asked what the behavior was for upgrading a daemonset.00:06
alanmeadowsbut I plan to go there soon00:06
kfox1111haven't heard back.00:06
kfox1111alanmeadows: it seems to have been added only since alpha3 so very very new. :/00:07
alanmeadowsyup ;-)00:07
kfox1111I'm worried it will delete daemonsets on upgrade.00:07
*** asalkeld has joined #openstack-kolla00:07
*** Pavo has quit IRC00:07
alanmeadowsyou'd think it'd be smarter with a state container00:07
alanmeadowsbut who knows :)00:07
kfox1111yeah. being so new, I'd guess they have only covered the common cases. but not sure.00:08
kfox1111which is why I've been playing it conservitive, and looking at packaging all the things individualy.00:08
alanmeadowsi look at this way00:08
kfox1111if I deploy it that way, I can upgrade precicely the things I want to, and leave the rest alone.00:08
alanmeadowsby the time we chart everything properly00:08
alanmeadowsin a way thats operationalized00:09
alanmeadowshopefully more superpackage work00:09
alanmeadowswill have made it in ;-)00:09
kfox1111and for those that want a super pakcage, we can include those ina an overarchign package with deps.00:09
kfox1111yeah. I'm hoping that will be the case. :)00:09
jascott1from a helm pov i could see an openstack install looking like several component releases00:09
kfox1111though converting template number 2 took about 30 min.00:09
alanmeadowstrue, once youve got a pattern ...00:10
kfox1111I think we could get all the existing kolla-kubernetes templates into helm in proably a few days.00:10
alanmeadowsi also think even the idea of a superpackage00:10
alanmeadowsis a bit of a hack in helm00:10
*** zhurong has joined #openstack-kolla00:10
kfox1111yeah.00:10
alanmeadowsit needs some more thought00:10
kfox1111I was thinking something like:00:10
kfox1111openstack package -> compute kit package -> keystone/nova/glance/cinder/neutron -> l3 package, openvswitch-agent, etc00:11
alanmeadowsthe fact that for e.g. openstack-helm has a middle layer of an uber openstack chart to keep the actual region overrides from having their requirements shifting is a bit kludgy00:11
alanmeadowsit works00:11
kfox1111coudl have a "advanced services package" with stuff like sahara,trove,magnum,00:11
kfox1111etc.00:11
srwilkerI was working on mariadb/keystone today to get familiar with Helm. I'm sure someone might beat me to the punch, but hoping to have it pushed this evening00:12
*** asalkeld has quit IRC00:12
kfox1111srwilker: have a look at this: https://review.openstack.org/#/c/396296/00:12
*** v1k0d3n has joined #openstack-kolla00:12
alanmeadowskfox1111: I think breaking it out that way may help upgrades later00:12
*** Pavo has joined #openstack-kolla00:12
kfox1111I finished retooling it today. should make it easier to add more.00:12
kfox1111alanmeadows: yeah.00:12
srwilkerkfox1111: I was going off that after awhile. It's proved helpfup00:13
srwilkerHelpful00:13
kfox1111srwilker: cool. :)00:13
rhalliseykfox1111, what pieces of the helm work are covered in your patch?00:13
rhalliseylooks like neutron00:13
kfox1111rhallisey: a copule of templates from neutron to prove out the code sharing, and build system will work.00:13
rhalliseyok00:13
rhalliseyI'll make bp's for each service so we can split that work up00:14
kfox1111it does successfully share the common-lib between multiple templates. so should be easy to convert the rest.00:14
rhalliseydoing the same for operators00:14
rhalliseyok00:14
kfox1111cool. :)00:14
srwilkerGoing to snooze for the openstack meetup. Be back online in a few hours00:16
kfox1111srwilker: cool. l8r.00:17
*** v1k0d3n has quit IRC00:17
*** hfu has joined #openstack-kolla00:17
*** masterdubs has joined #openstack-kolla00:22
*** masterdubs has quit IRC00:23
*** zhurong has quit IRC00:24
*** asalkeld has joined #openstack-kolla00:27
*** eaguilar has quit IRC00:27
*** asalkeld has quit IRC00:32
*** yingjun has joined #openstack-kolla00:32
jascott1will we try to ship pvc resources? I read somewhere it was recommended not to00:33
kfox1111unknown...00:33
kfox1111they kind of feel like configmaps to me.00:34
kfox1111I was kind of thinking of making pvc configurable for rabbit too.00:34
kfox1111for rpc rabbits, being stateless is ok. (maybe even prefereed)00:35
*** tonanhngo has quit IRC00:39
*** yingjun has quit IRC00:40
*** yingjun has joined #openstack-kolla00:40
*** sean-k-mooney has quit IRC00:46
*** sean-k-mooney has joined #openstack-kolla00:46
*** ayoung has joined #openstack-kolla00:50
kfox1111oh... I wonder if the dynamic pvc support for ceph landed in 1.5....00:52
*** tonanhngo has joined #openstack-kolla00:54
kfox1111https://github.com/kubernetes/kubernetes/issues/3647000:55
*** v1k0d3n has joined #openstack-kolla00:55
*** tonanhngo_ has joined #openstack-kolla00:56
*** severion has joined #openstack-kolla00:59
*** tonanhngo has quit IRC00:59
*** tonanhngo_ has quit IRC01:00
*** yingjun has quit IRC01:01
*** v1k0d3n has quit IRC01:02
*** jrich523 has quit IRC01:04
*** ayoung has quit IRC01:06
*** jrich523 has joined #openstack-kolla01:08
*** eaguilar has joined #openstack-kolla01:09
*** rhallisey has quit IRC01:24
*** jemcevoy has quit IRC01:33
*** zhurong has joined #openstack-kolla01:36
*** zhugaoxiao has quit IRC01:47
*** zhangyufei has joined #openstack-kolla01:48
*** newmember has joined #openstack-kolla01:48
*** inc0 has joined #openstack-kolla01:50
*** severion has quit IRC01:56
*** v1k0d3n has joined #openstack-kolla01:56
*** v1k0d3n has quit IRC01:59
*** f13o has joined #openstack-kolla02:00
*** v1k0d3n has joined #openstack-kolla02:00
*** f13o has quit IRC02:06
*** inc0 has quit IRC02:07
*** Pavo has quit IRC02:07
*** sbezverk_ has joined #openstack-kolla02:08
*** sbezverk has quit IRC02:10
*** Pavo has joined #openstack-kolla02:12
QuentinMkfox1111: you might be interested in https://github.com/kubernetes/kubernetes/pull/3394402:17
QuentinMkfox1111: direct usage of the Cinder/Keystone APIs to mount/dynamically provision Cinder volumes in Kubernetes without the need to use the OpenStack provider: credentials are provided by a secret.02:18
*** zhangyufei has quit IRC02:19
*** adrian_otto has quit IRC02:19
openstackgerritJeffrey Zhang proposed openstack/kolla: Use kolla-ansible for deployment  https://review.openstack.org/39850602:22
*** zhangyufei has joined #openstack-kolla02:27
*** v1k0d3n has quit IRC02:29
*** v1k0d3n has joined #openstack-kolla02:29
*** yingjun has joined #openstack-kolla02:42
*** severion has joined #openstack-kolla02:42
*** v1k0d3n has quit IRC02:46
*** dave-mccowan has joined #openstack-kolla02:46
*** yingjun_ has joined #openstack-kolla02:52
*** yingjun has quit IRC02:55
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850102:57
*** unicell has quit IRC02:57
*** fragatina has quit IRC03:03
openstackgerritJeffrey Zhang proposed openstack/kolla: Use kolla-ansible for deployment  https://review.openstack.org/39850603:03
*** fragatina has joined #openstack-kolla03:05
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Move glance precheck into its own role  https://review.openstack.org/39935403:09
*** fragatin_ has joined #openstack-kolla03:09
*** fragatina has quit IRC03:09
*** zhurong_ has joined #openstack-kolla03:10
*** zhurong has quit IRC03:12
*** fragatin_ has quit IRC03:14
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Add chrony ansible role  https://review.openstack.org/39935503:14
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Fix neutron.conf.j2 metadata_workers spelling error  https://review.openstack.org/39935603:18
*** zhangyufei has quit IRC03:18
*** yingjun has joined #openstack-kolla03:26
*** yingjun_ has quit IRC03:28
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850103:29
*** awiddersheim has joined #openstack-kolla03:31
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Remove build related env in tox.ini  https://review.openstack.org/39936003:31
*** tonanhngo has joined #openstack-kolla03:41
*** zhurong_ has quit IRC03:42
*** eaguilar has quit IRC03:42
*** tonanhngo has quit IRC03:42
*** zhurong has joined #openstack-kolla03:43
*** tovin07 has joined #openstack-kolla03:55
*** dave-mccowan has quit IRC03:56
*** g3ek has quit IRC03:59
*** haplo37_ has quit IRC04:00
*** tonanhngo has joined #openstack-kolla04:03
*** tonanhngo has quit IRC04:05
*** Pavo has quit IRC04:07
*** sp_ has joined #openstack-kolla04:11
*** Pavo has joined #openstack-kolla04:12
openstackgerritSurya Prakash Singh proposed openstack/kolla: Consistent home directory creation for all the services  https://review.openstack.org/39017904:12
*** imcsk8 has quit IRC04:17
*** imcsk8 has joined #openstack-kolla04:18
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850104:29
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible: Specify 'become' to neccesary tasks (general roles)  https://review.openstack.org/39868204:34
*** adrian_otto has joined #openstack-kolla04:35
*** coolsvap has joined #openstack-kolla04:37
*** ayoung has joined #openstack-kolla04:43
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850105:02
*** unicell has joined #openstack-kolla05:03
*** severion has quit IRC05:07
*** v1k0d3n has joined #openstack-kolla05:08
*** v1k0d3n has quit IRC05:11
*** v1k0d3n has joined #openstack-kolla05:12
*** v1k0d3n has quit IRC05:17
*** v1k0d3n has joined #openstack-kolla05:18
*** srwilker has quit IRC05:37
*** akscram has quit IRC05:46
*** akscram has joined #openstack-kolla05:47
*** haplo37 has quit IRC05:52
*** prameswar has joined #openstack-kolla05:55
coolsvapbmace: done05:55
bmacecoolsvap: done? :)05:57
coolsvapbmace: updated the overview on kolla-ansible lp05:57
bmacecoolsvap: aaah, ok, that message was really old.  lost context.05:58
coolsvapsorry I think i was disconnected in between so my session started with that as the last message on the channel05:59
*** zhangyufei has joined #openstack-kolla05:59
bmacei moved the ansible stop changes over into the kolla-ansible repo and made a note on how you can easily use it against a single or list of hosts:  https://review.openstack.org/#/c/399289/105:59
*** v1k0d3n has quit IRC06:00
*** haplo37 has joined #openstack-kolla06:04
*** g3ek has joined #openstack-kolla06:05
*** Pavo has quit IRC06:07
*** markmcclain has quit IRC06:09
openstackgerritCao Xuan Hoang proposed openstack/kolla-kubernetes: Remove white space between print ()  https://review.openstack.org/39938106:10
*** markmcclain has joined #openstack-kolla06:10
*** Pavo has joined #openstack-kolla06:11
*** haplo37 has quit IRC06:13
*** g3ek has quit IRC06:14
*** adrian_otto has quit IRC06:18
*** g3ek has joined #openstack-kolla06:26
*** haplo37 has joined #openstack-kolla06:26
*** msimonin has joined #openstack-kolla06:28
*** msimonin has quit IRC06:28
*** zhangyufei has quit IRC06:29
openstackgerritZeyu Zhu proposed openstack/kolla-ansible: Add blank space to deploy.yml file  https://review.openstack.org/39938906:36
*** pc_m has quit IRC06:38
*** zhangyufei has joined #openstack-kolla06:40
*** tonanhngo has joined #openstack-kolla06:58
*** tonanhngo has quit IRC07:00
*** haplo37_ has joined #openstack-kolla07:05
*** Satya_ has joined #openstack-kolla07:08
bjolomorning07:15
*** zhangyufei has quit IRC07:15
*** DTadrzak has joined #openstack-kolla07:18
*** zhangyufei has joined #openstack-kolla07:21
*** msimonin has joined #openstack-kolla07:22
*** msimonin has quit IRC07:22
openstackgerritSurya Prakash Singh proposed openstack/kolla: Closes-Bug: #1082248  https://review.openstack.org/39941107:29
openstackbug 1082248 in kolla "Use uuidutils instead of uuid.uuid4()" [Wishlist,In progress] https://launchpad.net/bugs/1082248 - Assigned to Surya Prakash Singh (confisurya)07:29
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Allow neutron-fwaas to be enabled in Neutron  https://review.openstack.org/39833607:29
*** sdake_ has joined #openstack-kolla07:32
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850107:34
*** hfu has quit IRC07:38
*** hfu has joined #openstack-kolla07:39
openstackgerritMerged openstack/kolla-ansible: Remove build related env in tox.ini  https://review.openstack.org/39936007:40
sdake_sup peeps07:49
*** lukl has quit IRC07:49
*** f13o has joined #openstack-kolla07:52
*** imcsk8 has quit IRC07:56
*** imcsk8 has joined #openstack-kolla07:56
*** duonghq has joined #openstack-kolla07:57
*** v1k0d3n has joined #openstack-kolla08:00
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850108:02
*** openstackgerrit has quit IRC08:03
*** openstackgerrit has joined #openstack-kolla08:03
sdake_duonghq ok dude, i'm back :)08:03
sdake_duonghq whats up08:03
duonghqsdake_, atm, gerrit link to bp is working, last few days, the link is broken08:05
*** v1k0d3n has quit IRC08:05
sdake_you mean when you subit something to gerrit it does't go to t he blueprint?08:06
sdake_for which repo/luanhcpad?08:06
duonghqwhen I clicked the blueprint link in commit message, it said that bp is not existed08:06
duonghqfor kolla-ansible08:06
duonghqbut atm, everything is ok08:06
sdake_ok08:07
sdake_either someone else fixed it08:07
sdake_or it fixed itself :)08:07
*** Pavo has quit IRC08:07
duonghqI have believe that it fixed has self-healing ability08:07
duonghq(j/k)08:07
coolsvapsdake_: duonghq kolla-ansible was not part of openstack earlier08:08
coolsvapso the link was not working08:08
duonghqcoolsvap, understood,08:08
*** pc_m has joined #openstack-kolla08:09
duonghqbut kolla-ansible gate still broken?08:09
duonghqjust to confirm08:09
sdake_coolsvap ya that would do it08:09
sdake_coolsvap thta was the first thingi was goin to check actually :)08:09
coolsvapi did that when i first encoutered the issue08:10
duonghqthank coolsvap08:10
*** matrohon has joined #openstack-kolla08:12
*** Pavo has joined #openstack-kolla08:12
sdake_duonghq unknown - Jeffrey4l has been working on it08:12
sdake_Jeffrey4l need any help to ge t the gate unblcoked?08:12
sdake_I just woke up for 12 hrs of hibernation08:12
sdake_here to help08:12
sdake_for/from08:13
Jeffrey4lsdake_, no. it almost be done.08:13
sdake_nice08:13
Jeffrey4lkolla gate is OK https://review.openstack.org/39850608:13
duonghqsdake_, cool08:13
sdake_Jeffrey4l yup i saw kolla gate is ok08:14
*** duonghq has quit IRC08:15
*** hogepodge has quit IRC08:15
sdake_disagree re globals and passwords going in kolla-ansible08:18
openstackgerritwangwei proposed openstack/kolla: Add the function of docker login before pushing to registry  https://review.openstack.org/39942608:18
sdake_or atleast ithink its open to discussion08:18
openstackgerritMerged openstack/kolla: Use kolla-ansible for deployment  https://review.openstack.org/39850608:18
sdake_Jeffrey4l ^08:19
Jeffrey4lthanks.08:21
sdake_no i mean see above re disagree Jeffrey4l08:21
Jeffrey4lre globals and passwords: why? any reason? it is only used by ansible.08:22
Jeffrey4lsdake_,08:22
openstackgerritMerged openstack/kolla: Fix missing libvirt python module in ceilometer-compute (v2)  https://review.openstack.org/39914508:22
sdake_globals and passwords re used by kubernetes as well08:22
sdake_the idea is to hvae one config file to rule them all08:22
sdake_that way operators dont hve to learn two ways to config things08:22
Jeffrey4lkubernets also use config file generated by kolla-ansible. this is the root cause.08:23
Jeffrey4lsdake_,08:23
*** zhubingbing has joined #openstack-kolla08:26
zhubingbinghello guys08:26
sdake_sup zhubingbing08:27
zhubingbinghi08:27
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850108:28
*** Satya_ has quit IRC08:28
*** Satya_ has joined #openstack-kolla08:29
coolsvapJeffrey4l: did you get the release script running?08:29
Jeffrey4lcoolsvap, yep. it works as expected ;)08:29
Jeffrey4lbtw, i am using arch, and run this script in a python virtual env.08:29
coolsvapJeffrey4l: cool i will try that once more maybe next time08:30
coolsvapi just saw a flurry of mails changing milestone, thought to ask you08:30
Satya_Hi Jeffrey08:31
Satya_regarding rabbitmq issue08:31
Jeffrey4lcoolsvap, you can try it now. like move all unfinished bug into o2.08:31
Satya_we have a solution now08:31
Jeffrey4leven though nothing will happen. but you can make sure the script runs successfully.08:31
Satya_which can have rabbit run in a mgmt interface rather than api08:32
Jeffrey4lSatya_, realy? how?08:32
*** zhangyufei has quit IRC08:32
*** msimonin has joined #openstack-kolla08:32
Satya_we changed the kolla code and added a new parameter as rabbit_interface08:32
Jeffrey4lcoolsvap, yep.08:32
Satya_can we push that to upstream?08:32
sdake_dsyqupdytrsm pld08:32
Jeffrey4li do not think so :(08:32
Satya_Hi Sdake08:32
sdake_damn08:32
sdake_Satya_ can ou run a git review on it plz08:33
sdake_Satya_ i really would like to see this happen08:33
sdake_rabbitmq should not be running on the api interface08:33
Satya_sure08:33
Jeffrey4lit is more a edge cause i think.08:33
*** STOP_nwo has joined #openstack-kolla08:33
STOP_nwohi08:33
Jeffrey4lsdake_, all port running on the api interface, like mysql, mongodb.08:33
sdake_Jeffrey4l i know08:34
STOP_nwowhere u from ppl08:34
sdake_the infra needs to not run on the api interface08:34
sdake_STOP_nwo huh?08:34
Jeffrey4lthere is no mgmt interface concept in kolla. but the api interface is.08:34
Jeffrey4lthe really api_interface is the extenral api interface.08:34
sdake_Jeffrey4l right - so lets add a mgmt interface08:34
Satya_we changed all other components to use the mgmt interface (rabbit_interface) for transport URL08:35
Satya_http://paste.openstack.org/show/589680/08:35
*** zhangyufei has joined #openstack-kolla08:35
Jeffrey4lthen what's the difference between api interface, mgmt interface and external vip interface?08:35
sdake_api interface = internal interface on which api runs08:36
*** egonzalez90 has joined #openstack-kolla08:36
sdake_mgmt interface = internal interface on which mgmt infrasturcture runs08:36
sdake_external vip interface is the VIP for mapping external connections to internal VIP08:36
sdake_is that right08:37
Jeffrey4lhow to define the mgmt infra? non-openstack service?08:37
sdake_just woke up and haven't had my dose of caffeine yet08:37
Jeffrey4l;)08:37
sdake_Jeffrey4l right, non-openstack service08:37
Jeffrey4lhmm, we set authencation on both openstack service and non-openstack service. i do not think we need separate this two kind of service.08:39
sdake_its not about auth08:39
sdake_its about network traffic08:39
Jeffrey4lok. let met think08:40
*** msimonin has left #openstack-kolla08:40
*** zhangyufei has quit IRC08:41
egonzalez90Jeffrey4l: morning, globals.yml is going to be at kolla-ansible repo. Is not going to be shared between ansible and k8s?08:45
openstackgerritwangwei proposed openstack/kolla: Add the function of docker login before pushing  https://review.openstack.org/39942608:45
*** tovin07_ has joined #openstack-kolla08:45
*** tovin07 has quit IRC08:45
Jeffrey4legonzalez90, the question is why kolla-k8s need such file. the answer is kolla-k8s need kolla-ansible's genconfig features.08:46
Jeffrey4lso it is nothing to do with where is the globals.yml and passwords.yml file.08:46
Satya_https://bugs.launchpad.net/kolla/+bug/164287808:46
openstackLaunchpad bug 1642878 in kolla "RabbitMQ should communicate through a different network rather than api_network" [Undecided,New]08:46
*** zhubingbing has quit IRC08:48
*** zhangyufei has joined #openstack-kolla08:50
sdake_https://www.youtube.com/watch?v=NGOvH1T-dsA08:52
sdake_Jeffrey4l so, where the file goes i am less interested in08:52
sdake_Jeffrey4l what i'm intereted in is that is is consistent between deliverables08:53
Jeffrey4lsdake_, what's that means? more specific?08:53
*** shardy has joined #openstack-kolla08:53
sdake_Jeffrey4l same config file for kolla-ansible as kolla-kubernetes08:54
sdake_how that happens, idc :)08:54
sdake_i'm ok with C&P in fact :)08:54
sdake_although that is not idela08:54
sdake_ideal08:55
Jeffrey4lhmm. hate C&P. we can not maintain two files in two repo.08:55
sdake_Jeffrey4l right, not ideal08:55
sdake_Jeffrey4l so then the question is where does th file go08:55
sdake_we are talking about putting the default config in the kolla repo08:56
Jeffrey4lnot the file. ( globals or passwords. ) but all the tempaltes files.08:56
sdake_(for each service)08:56
sdake_Jeffrey4l sorrry dude if I seem dense, i'm sort of exhausted08:56
sdake_Jeffrey4l but i didn't grok tha lst sensense08:56
sdake_sentence08:58
bmacefyi sdake_ i added you to a re-review of the stop changes.  i got them all moved over into the new kolla-ansible repo08:59
Jeffrey4lthe kolla-k8s and kolla-ansible hasn't the same configuration for certain service. there are lots if-else in the configuration file. if we put it in common place ( like kolla ), it will be full of if-else.08:59
Jeffrey4lbut i have no better idea now ;(09:00
*** narasimha_SV has joined #openstack-kolla09:00
portdirectgate for kolla good to go?09:00
Jeffrey4lportdirect, yep.09:00
*** zhangyufei has quit IRC09:00
portdirectwhoot! cheers Jeffrey4l :)09:00
sdake_Jeffrey4l we need to abstract all that if-then-else crap out into one place09:00
sdake_Jeffrey4l the reason ew ehae if else atm is because of things like ansible_* veriable09:00
narasimha_SVhi sorry to ask like this . I dont find ansible folder in master branch of kolla09:01
sdake_narasimha_SV git checkout http://github.com/openstack/kolla-ansible09:01
*** zhurong has quit IRC09:01
*** ChanServ sets mode: +o sdake_09:01
*** sdake_ is now known as sdake09:01
*** ChanServ sets mode: +o sdake09:01
narasimha_SVok09:01
narasimha_SVthanks sdake09:02
*** zhurong has joined #openstack-kolla09:02
*** sdake changes topic to "Looking for Ansible deployment? git checkout http://github.com/openstack/kolla-ansible; New to Kolla? Please read the documentation here: http://docs.openstack.org/developer/kolla/; Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA):"09:02
*** gfidente has joined #openstack-kolla09:03
*** gfidente has joined #openstack-kolla09:03
Jeffrey4lsdake, that's will be optimal.09:04
sdakeJeffrey4l right09:04
Jeffrey4lneed out. brb09:04
*** liyifeng has quit IRC09:06
*** rmart04 has joined #openstack-kolla09:12
openstackgerritnarasimha18sv proposed openstack/kolla-ansible: typo error of keyring spelling  https://review.openstack.org/39945509:12
*** zhangyufei has joined #openstack-kolla09:15
*** zhangyufei has joined #openstack-kolla09:15
*** strigazi_AFK is now known as strigazi09:15
narasimha_SVhi is there anyone who has worked on GNOCCHI here ?09:16
narasimha_SVI need some help09:16
*** liyifeng has joined #openstack-kolla09:18
Satya_hi09:22
openstackgerritwangwei proposed openstack/kolla: Add the function of docker login before pushing  https://review.openstack.org/39942609:24
*** zhangyufei has quit IRC09:29
*** papacz has joined #openstack-kolla09:31
*** shardy has quit IRC09:34
*** shardy has joined #openstack-kolla09:34
*** yingjun has quit IRC09:38
*** yingjun has joined #openstack-kolla09:38
*** yingjun has quit IRC09:38
*** yingjun has joined #openstack-kolla09:39
*** papacz has quit IRC09:39
*** yingjun has quit IRC09:39
openstackgerritMerged openstack/kolla-kubernetes: TrivialFix: Remove unused code  https://review.openstack.org/36856909:40
openstackgerritMerged openstack/kolla-kubernetes: Clean imports in code  https://review.openstack.org/36859209:40
*** STOP_nwo has quit IRC09:42
*** hfu has quit IRC09:43
openstackgerritMerged openstack/kolla-kubernetes: Remove white space between print ()  https://review.openstack.org/39938109:48
*** papacz has joined #openstack-kolla09:49
*** derekjhyang has quit IRC09:51
*** zhurong has quit IRC09:59
*** Pavo has quit IRC10:07
*** Pavo has joined #openstack-kolla10:11
*** tovin07_ has quit IRC10:26
*** Satya_ has quit IRC10:27
*** hieulq has quit IRC10:28
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible: Ignore dom0 qemu processes during destroy  https://review.openstack.org/39920310:28
*** fragatina has joined #openstack-kolla10:35
*** fragatina has quit IRC10:36
*** fragatina has joined #openstack-kolla10:37
*** ppalacios has joined #openstack-kolla10:40
*** senk has joined #openstack-kolla10:44
*** liyifeng has quit IRC10:45
*** sdake has quit IRC10:48
pbourkebjolo: morning, bad news for you :p10:55
pbourkebjolo: seems to work with centos/newton also. have dumped lots of info on the bug for you10:55
*** DTadrzak has quit IRC11:04
*** ppalacios1 has joined #openstack-kolla11:05
*** ppalacios1 has quit IRC11:06
*** ppalacios has quit IRC11:07
openstackgerritJavier Castillo Alcíbar proposed openstack/kolla-ansible: Fix missing heka/heka-ceilometer.toml in ceilometer_compute  https://review.openstack.org/39913411:09
openstackgerritJavier Castillo Alcíbar proposed openstack/kolla-ansible: Included new temmplates  https://review.openstack.org/39952511:09
*** sdake has joined #openstack-kolla11:16
*** prameswar has quit IRC11:24
sdakerbergeron around?11:30
bjolopbourke, wtf!11:30
bjoloyou are supposed to FAIL :)11:31
sdakerbergeron thanks for teacing me about this youtube feature: https://youtu.be/mqT66AVml48?t=3m10s11:31
bjolopbourke, tnx so much for taking the time to do this11:34
bjoloi will really read through your setup and see why we fail11:34
pbourkebjolo: np, sorry you're having so much trouble with it11:34
*** khamtamtun has joined #openstack-kolla11:35
bjolooff the top of my head you use real interfaces and i use vlan interfaces but that should not cause any problems11:35
bjoloaaaahhh maybe it does11:36
bjoloall vlan interfaces share the same mac address by default11:37
bjolo[root@eselde02u37 kolla]# ip l | grep -B1 bond0 | grep link | tail -n 311:40
bjolo    link/ether e4:1d:2d:bc:cc:c0 brd ff:ff:ff:ff:ff:ff11:40
bjolo    link/ether e4:1d:2d:bc:cc:c0 brd ff:ff:ff:ff:ff:ff11:40
bjolomaybe that throws off neutron agent11:40
openstackgerritPete Birley proposed openstack/kolla: Update Percona repo to current release  https://review.openstack.org/39954911:40
bjolosome of OF rules are based on mac addresses right11:41
bjolopbourke, you still have you env up?11:41
bjoloif so, can you please dump the flows on br-int and both external bridges11:42
pbourkebjolo: i've gone back to the oraclelinux/master setup but that should be ok11:42
bjoloyes11:42
pbourkebjolo: one sec11:42
bjolosince it works in both setups11:42
*** prameswar has joined #openstack-kolla11:50
*** eaguilar has joined #openstack-kolla11:52
sdakeJeffrey4l around11:52
*** hfu has joined #openstack-kolla11:59
*** yingjun has joined #openstack-kolla12:02
*** srwilkers has joined #openstack-kolla12:03
*** Pavo has quit IRC12:07
sdakeJeffrey4l if you appear (and havn'et gone to bed) could you let me know what state of gate is.  All reviews currently blocked in kolla-ansible, and want to know if there is anything I can do to help get that unblocked (including writing pc code) :)12:09
*** Pavo has joined #openstack-kolla12:12
*** khamtamtun has quit IRC12:12
sdakewow kolla #3 on reviews for ocata cycle atm12:13
sdakeright behind neutron and nova ;)12:13
sdake#4 on commits12:13
pbourkebjolo: those logs are there now12:16
*** sdake_ has joined #openstack-kolla12:16
openstackgerritJavier Castillo Alcíbar proposed openstack/kolla-ansible: updated heka-ceilometer template file  https://review.openstack.org/39956612:18
*** sdake has quit IRC12:20
mgoddardhi, has anyone come across issues when using the kolla-ansible reconfigure command where containers are reconfigured unnecessarily because of permissions errors?12:21
*** derekjhyang has joined #openstack-kolla12:21
mgoddardin */tasks/reconfigure.yaml the kolla_set_configs command is run without sudo, whereas sudo -E is used in kolla_start.12:22
mgoddard(we have a modified config.json that copies in a non-root unreadable file)12:23
openstackgerritSteven Dake proposed openstack/kolla: Clean up loc  https://review.openstack.org/39957012:25
openstackgerritSteven Dake proposed openstack/kolla: Remove init-runonce from docker repo  https://review.openstack.org/39957212:28
*** prameswar has quit IRC12:29
*** hogepodge has joined #openstack-kolla12:31
*** rhallisey has joined #openstack-kolla12:33
Jeffrey4lsdake_, https://review.openstack.org/398501  patch set 13 works  expect oracle related jobs. i am debugging the oracle jobs.12:36
* Jeffrey4l is taking care of son. 12:36
Jeffrey4lshould fix this today.12:36
pbourkeJeffrey4l: let me know if I can help12:38
pbourkeJeffrey4l: looking at PS 13 but all still seem to be failed12:38
pbourkeincluding centos12:38
Jeffrey4lhttps://review.openstack.org/#/c/398501/  ps 13 works except oracle.12:41
*** tonanhngo has joined #openstack-kolla12:41
Jeffrey4ldid u open the correct url?12:42
openstackgerritSteven Dake proposed openstack/kolla: Clean up README.rst in docker repo  https://review.openstack.org/39958212:42
Jeffrey4lit is weird now. oracle jobs say ' OSError: [Errno 2] No such file or directory' http://logs.openstack.org/01/398501/13/check/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/06cda4a/console.html#_2016-11-18_07_09_25_23692312:43
Jeffrey4li have no idea what file he required.12:43
*** tonanhngo has quit IRC12:43
Jeffrey4lif u have any idea. it will be very helpful.12:43
sdake_Jeffrey4l let me have a look at tthe review and see if anthing pops out12:44
sdake_Jeffrey4l ps13 is red on all deploy gates12:44
Jeffrey4lsdake_, patch 13 is not the latest ps.12:44
sdake_Jeffrey4l yup i know12:45
Jeffrey4lclick toggle ci button at the bottom.12:45
sdake_i toggle'd ci12:45
sdake_and i see it is passing some gates12:45
sdake_wierd how it is showing in main gate screen failures12:45
Jeffrey4lsdake_, because it only show the last PS's result. ( i added some test lines now. so the last/latest ps all failed. )12:46
sdake_http://logs.openstack.org/01/398501/13/check/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/06cda4a/console.html#_2016-11-18_07_09_25_23381012:46
sdake_Jeffrey4l i used /13 at end of url... so that shouldn't show latest patchset12:47
sdake_but ok got it :)12:47
*** Jeffrey4l has quit IRC12:47
sdake_pbourke I see r/home/jenkins12:47
sdake_that seems wrong?12:48
pbourkesdake_: -r12:48
sdake_ok well there needs to be a spce in there12:48
pbourkeits subprocess that's raising the exception so I think it may be trying to execute a binary that's not there12:48
sdake_subprocess finds the process12:49
sdake_or there would not be that huge backtrace12:49
sdake_what it isn't finding is the requirements.txt file and test-requirements.txt12:49
sdake_is -rrequirements.txt an acceptable usage?12:49
sdake_my best guess at this point is wrong version of pip12:50
sdake_lemme pull that repo and see what tox.ini looks like12:50
pbourkepython -c 'import subprocess; subprocess.check_output("foo")'12:51
pbourkesame stacktrace12:51
sdake_pbourke ok12:52
*** srwilkers has quit IRC12:52
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850112:52
sdake_next best guess is lack of pip in the system12:52
sdake_as in centos ships it, oracle linux doesn't12:53
sdake_lemme pull down jeffrey's patch now12:53
openstackgerritPete Birley proposed openstack/kolla: DO NOT MERGE!: Remove unneeded PythonMySQL libs from OpenStack Base Image  https://review.openstack.org/39904212:54
*** liyifeng has joined #openstack-kolla12:55
openstackgerritPete Birley proposed openstack/kolla: DO NOT MERGE!: Remove unneeded PythonMySQL libs from OpenStack Base Image  https://review.openstack.org/39904212:55
*** yingjun has quit IRC12:57
*** yingjun has joined #openstack-kolla12:57
*** yingjun has quit IRC13:01
openstackgerritSteven Dake proposed openstack/kolla: Clean up loc  https://review.openstack.org/39957013:02
*** magicboiz has quit IRC13:05
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Allow neutron-fwaas to be enabled in Neutron  https://review.openstack.org/39833613:06
*** senk has quit IRC13:07
*** Jeffrey4l has joined #openstack-kolla13:07
Jeffrey4lpbourke, about https://review.openstack.org/398336 the releasenote is parsed to {"Add support for neutron-fwaas. Set 'enable_neutron_fwaas": "yes' to enable."}13:08
sdake_Jeffrey4l quick q, what is current speculation on why oraclelinux is failing?13:10
sdake_ls13:11
Jeffrey4lsdake_, no idea.13:11
Jeffrey4li suspend the some software version is different. but oracle and centos are using the same base diso.13:12
sdake_right13:12
sdake_i just put that together ;)13:12
Jeffrey4lin the latest ps. i print all rpm version and pip freeze.13:12
sdake_looking at tox.ini from patchset 1313:13
sdake_-  testr run test_build.DeployTestCentosBinary13:13
sdake_oh nm thats there for oraclelinux as well13:14
sdake_brain still booting :)13:14
sdake_Jeffrey4l did you use pip to print out the software versions of the python stuff too?13:14
sdake_how about selinux?13:15
Jeffrey4lrpm will print the python version.13:15
sdake_rpm doesn't print python version if pip is used to do the install13:15
*** lamt has joined #openstack-kolla13:15
sdake_it prints the rpm version of the package not the pip version fof the package13:15
sdake_is audit log captured anywhere?13:16
sdake_(re selinux idea)13:16
Jeffrey4llatest logs http://logs.openstack.org/01/398501/17/check/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/fc6dcf9/console.html13:16
Jeffrey4lsdake_, no ;(13:17
sdake_can you turn off selinux so we can try that line of thinking13:17
Jeffrey4lok. i can try in next patch.13:17
sdake_and eliminate it as a cause13:17
sdake_cool13:17
Jeffrey4lanything else u want to catch?13:17
*** dave-mccowan has joined #openstack-kolla13:18
sdake_Jeffrey4l http://logs.openstack.org/01/398501/17/check/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/fc6dcf9/console.html#_2016-11-18_13_06_18_60507513:19
sdake_what is the $ about at the end of the tox command?13:19
sdake_wather in the middle13:19
sdake_rather in the middle13:19
sdake_looks super suspicious13:20
Jeffrey4lmay be.13:20
sdake_ /home/jenkins/workspace/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/.tox$13:20
Jeffrey4lit should the current workspace.13:21
*** zhubingbing has joined #openstack-kolla13:21
openstackgerritMauricio Lima proposed openstack/kolla-ansible: Remove docker from kolla-ansible  https://review.openstack.org/39832013:21
Jeffrey4lit created the deploy-oraclelinux-binary  virtualenv successfully.13:21
Jeffrey4lbut when calling pip command with failed.13:21
sdake_Jeffrey4l check this out13:22
sdake_http://logs.openstack.org/01/398501/17/check/gate-kolla-ansible-dsvm-deploy-centos-source-centos-7-nv/1b617ca/console.html#_2016-11-18_13_05_08_27287813:22
sdake_centos source13:23
sdake_is doing something with oraclelinux?13:23
Jeffrey4lsdake_, it is work.  in the patch, i write `tox -e oraclexxxx for all gate`13:23
Jeffrey4lit is ok.13:23
Jeffrey4lcheck the changed i made in the latest patch.13:24
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850113:25
*** msimonin has joined #openstack-kolla13:26
sdake_i have a better speculatiion now13:26
sdake_https://review.openstack.org/#/c/398501/17/tools/setup_gate.sh13:26
sdake_see line 3213:26
Jeffrey4linvocation failed (errno 2), args: ['/home/jenkins/workspace/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/.tox/deploy-oraclelinux-binary/bin/pip', 'install', '-U', '-r/home/jenkins/workspace/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/requirements.txt', '-r/home/jenkins/workspace/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/test-requirements.txt'], cwd: /home/jenkins/workspace/gate-kolla-ansib13:26
Jeffrey4lle-dsvm-deploy-oraclelinux-binary-centos-7-nv13:26
sdake_sudo only works in predefined paths13:27
sdake_Jeffrey4l throwing ideas at you :)13:28
Jeffrey4lbut line 32 works.13:28
sdake_https://review.openstack.org/#/c/398501/17/tox.ini13:30
sdake_see line 15613:31
sdake_compare to orcle linux implementation of tox.ini13:31
Jeffrey4lhmm isn't it the same?13:32
sdake_see line 16813:33
Jeffrey4lleft or right?13:33
sdake_left13:33
Jeffrey4lwe have sudo in whitelist_extennals13:33
sdake_in oraclelinux yes, in ubuntu no13:33
Jeffrey4lso?13:33
Jeffrey4lyes. but centos has sudo.13:34
Jeffrey4land centos and ubuntu works.13:34
*** sdake has joined #openstack-kolla13:36
sdakeJeffrey4l i'll take out sudo and try that in a different changeset with your work13:36
sdakecan you try selinux approach?13:37
Jeffrey4lok.13:37
sdake(ie diable)13:37
Jeffrey4lsorry?13:37
sdakedisable selinux on all centos13:37
Jeffrey4li am check the tox source code, and try to see when it may raise such exception.13:37
Jeffrey4lsdake, ok.13:37
sdakethat way we got two jobs going in the gates at once13:37
*** sdake_ has quit IRC13:38
Jeffrey4lyep.13:39
*** sdake_ has joined #openstack-kolla13:40
sdake_centos has sudo in the whilelist externals13:40
sdake_which is odd13:40
sdake_wonder why ubuntu even works ;)13:40
*** sdake has quit IRC13:43
*** srwilker has joined #openstack-kolla13:44
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850113:48
openstackgerritSteven Dake proposed openstack/kolla: Remove bandit references from tox.ini  https://review.openstack.org/39959713:50
openstackgerritSteven Dake proposed openstack/kolla: Remove Ansible references from tox.ini  https://review.openstack.org/39959713:50
*** senk has joined #openstack-kolla13:51
openstackgerritSteven Dake proposed openstack/kolla-ansible: Remove docker reference related to bandit from tox.ini  https://review.openstack.org/39960013:52
*** dave-mcc_ has joined #openstack-kolla13:53
sdake_dave-mcc_ around?13:54
Jeffrey4lsdake_, http://logs.openstack.org/01/398501/19/check/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/1818a09/console.html#_2016-11-18_13_52_31_89757113:55
dave-mcc_sdake_ good morning!13:55
sdake_dave-mcc_ sup dave13:55
Jeffrey4lneed more debugging.13:55
*** dave-mccowan has quit IRC13:55
sdake_dave-mcc_ jeffrey and i have a question13:55
sdake_dave-mcc_ in kolla, mariadb and rabbitmq bind to the api interface13:55
sdake_i would htink it makes more sense to bind to the management interface13:56
sdake_kolla has no management interface13:56
sdake_thoughts on that?13:56
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850113:56
dave-mcc_are you asking a question on nomenclature?13:56
dave-mcc_or on design?13:56
*** krtaylor has quit IRC13:57
Jeffrey4ldave-mcc_, design13:58
sdake_dave-mcc_ design14:00
sdake_our api interface is what the apis bind to14:00
dave-mcc_Jeffrey4l sdake_ kolla calls the internal interface "api_interface".  so, the design is as you expect: maria and rabbit are on the internal network.14:00
dave-mcc_sdake_ it's what the internal apis bind to14:01
sdake_dave-mcc_ so its secure? (main concern here)14:01
sdake_Jeffrey4l i see you turned off selinux and it looks like it still doesn't work properly14:01
Jeffrey4lsdake_, yep.14:01
sdake_from that log14:02
dave-mcc_sdake_ a prudent operator will isolate the internal network (and thus all the api_interfaces).  no one should talk to those except internal services.14:02
sdake_dave-mcc_ cool14:02
sdake_so api_interface should really be called internal_interface14:02
dave-mcc_sdake_ the external network is the one humans talk to.  those endpoints are protected by TLS.14:02
dave-mcc_sdake_ yea.  that would save the confusion.  it's really a question of nomenclature.14:03
sdake_ok well i guess thatship hsa sailed14:03
sdake_as long as its secure14:03
sdake_;)14:03
*** fguillot has joined #openstack-kolla14:03
Jeffrey4lsdake_, check this line http://logs.openstack.org/01/398501/20/check/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/fd95181/console.html#_2016-11-18_14_03_49_72737114:04
Jeffrey4lhmm interesting. the interpreter is cut.14:06
Jeffrey4lpip: /home/jenkins/workspace/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-cent: bad interpreter: No such file or directory14:06
sdake_Jeffrey4l did project config change?14:06
dave-mcc_sdake_ Jeffrey4l i was just reading haproxy.cfg.j2 to double check.  it's still secure, but i describe it wrong.14:06
Jeffrey4li do not think so. and centos works. centos and oraclelinux using the same base image, which is 'cento-714:07
sdake_if project-config was changed incorrectly (someone may have fatfingerered)14:07
*** Pavo has quit IRC14:07
sdake_antoher possibility is the length is too long  for either zuul or tox14:07
sdake_count the characters :)14:07
dave-mcc_sdake_ Jeffrey4l api_interface is internal only, but no one should send to it directly.  all API traffic goes to either the internal VIP or external VIP.  HAProxy unwraps TLS and send the external traffic to the internal network.14:08
sdake_dave-mcc_ that sounds correct14:08
sdake_thanks14:08
Jeffrey4lbut the weird things is centos works.14:08
sdake_oraclelinux > centos in terms of length14:09
pbourkehttp://stackoverflow.com/questions/10813538/shebang-line-limit-in-bash-and-linux-kernel14:09
pbourkehis example works on ubuntu but not on oraclelinux for me14:09
dave-mcc_sdake_ Jeffrey4l the only access an external user should have to an api_interface is through HAProxy, and there is no route for maria or rabbit to go from external_vip to api_interface.14:09
Jeffrey4lwhy test oraclelinux, it is using centos image.  Linux centos-7-rax-iad-5452306 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct 24 16:09:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux14:10
pbourkeJeffrey4l: its more to test the docker images14:10
Jeffrey4lpbourke, it is prepare test virtual env on host os.14:11
pbourkehmm14:11
pbourkewhat sdake said, len(oraclelinux) > len(centos)14:11
dave-mcc_sdake_ Jeffrey4l when most devs use a single network or all-in-one, this nuance might get lost.  but, the internal network should a non-routable on a different interface than the external network.14:12
Jeffrey4lok. anyway. need fix this at first. then we can try to find why centos works but oracle linux not.14:12
sdake_Jeffrey4l can you make some kind of hack to convert oraclelinux to "ol" in tox.ini14:12
sdake_ot test out the limit theory14:12
*** Pavo has joined #openstack-kolla14:12
Jeffrey4ldave-mcc_, if he only use one network, add `mgmt` network ( or something else ) won't help.14:13
sdake_it will probably require a change to tox.ini and setup_gate.sh14:13
sdake_Jeffrey4l right i think dave_mcc agrees :)14:13
Jeffrey4lfyi, the shebang is cut at 80 character ;(14:14
sdake_which shebang do you speak of14:14
Jeffrey4lpip.14:15
pbourkeJeffrey4l: create an alias or symlink14:15
*** senk has quit IRC14:15
pbourkethen sed that into setup.sh14:15
Jeffrey4lcheck this and the following two lines http://logs.openstack.org/01/398501/20/check/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/fd95181/console.html#_2016-11-18_14_03_49_72737114:15
*** v1k0d3n has joined #openstack-kolla14:16
*** zhurong has joined #openstack-kolla14:18
sdake_pbourke symlink for which?14:20
pbourkesdake_: for the tox pip14:20
dave-mcc_Jeffrey4l sdake_ if an operator puts the external VIP on the same network as the internal interfaces, he's going to have a bad day.  maybe he could add iptables commands to secure it?14:21
sdake_pbourke don't totally follow other then its a rift on my suggestoin of working around the limit via some kind of hacking :)14:21
pbourkepretty much :)14:21
Jeffrey4lpbourke, it is caused by the content of pip ( first line, shebang), not the location of pip14:21
pbourkeln -s <big-long-path/pip> /tmp/pip14:21
pbourkesed -i s#<big-long-path/pip>#/tmp/pip#g setup.sh14:22
sdake_pbourke pip produces a shell script it appears14:23
sdake_that pip produced shell script is defective14:23
*** msimonin has quit IRC14:23
pbourkethen sed that14:23
sdake_sed it where?14:23
sdake_rather not pip, tox14:24
Jeffrey4la possible solution is change the pip shebang to : #!/usr/bin/env python14:24
pbourkesame idea I just got the scripts wrong14:24
Jeffrey4lsince the venv path is in the first location.14:24
openstackgerritMauricio Lima proposed openstack/kolla-ansible: Remove docker from kolla-ansible  https://review.openstack.org/39832014:24
pbourkethat sounds good too Jeffrey4l14:24
Jeffrey4lit should works.14:24
sdake_see working together - found the root cause ;)14:25
Jeffrey4lbut now, i curiosity why it won't work in oracle ;(14:25
*** severion has joined #openstack-kolla14:25
sdake_oraclelinux > centos14:25
sdake_centos probably right at the very limit14:25
Jeffrey4li do not think we found the root cause.14:25
*** goldyfruit has joined #openstack-kolla14:25
Jeffrey4loraclelinux and centos are using the same base image (centos-7)14:25
sdake_yes but oracleinux passed in the schebang taks up more space14:26
sdake_then centos does14:26
Jeffrey4lsdake_, there is no oraclelinux thing during that issue happen.14:26
sdake_this causes kernel not to understand the command14:26
Jeffrey4lthe oraclelinux is only used in docker image/container.14:26
sdake_Jeffrey4l http://logs.openstack.org/01/398501/20/check/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/fd95181/console.html#_2016-11-18_14_03_49_72919814:26
sdake_oraclelinux is right ther ein that schebang14:26
Jeffrey4lit is the python virtualenv name.14:27
sdake_gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/14:27
Jeffrey4lgenerated by tox -edeploy-oraclelinux14:27
Jeffrey4lgate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv   this is generated by CI scripts. automatically.14:27
sdake_look at hte log that i posted14:28
*** v1k0d3n has quit IRC14:28
sdake_its got the schebang command from pip right in it14:28
*** jheroux has joined #openstack-kolla14:28
sdake_oraclelinux = 11 charts, centos = 614:28
sdake_each used twice14:28
Jeffrey4lyes. but this is in *centos-7* image, still. it is nothing to do with oraclelinux.14:28
sdake_charts/chars14:28
Jeffrey4lsorry?14:28
Jeffrey4l11 6?14:29
sdake_characaters14:29
sdake_ok...14:29
sdake_ #!/home/jenkins/workspace/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/.tox/deploy-oraclelinux-binary/bin/python14:29
sdake_do you se eabove how oracleinux is used in the schebang?14:29
Jeffrey4lhmm. let count. seem so.14:29
*** severion has quit IRC14:30
*** zhurong has quit IRC14:30
Jeffrey4l➤ echo -n ' #!/home/jenkins/workspace/gate-kolla-ansible-dsvm-deploy-centos-binary-centos-7-nv/.tox/deploy-centos-binary/bin/python' | wc14:30
Jeffrey4l      0       1     12014:30
sdake_right less then 128 chars14:30
Jeffrey4lthere are 120 char for in centos case.14:30
sdake_ole > 128 chars14:30
*** derekjhyang has quit IRC14:31
Jeffrey4lbut the interpreate is cut at 80 chart, not 128 chat14:31
*** duonghq has joined #openstack-kolla14:31
Jeffrey4lecho -n '#!/home/jenkins/workspace/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-cent' |wc      <---   8014:31
sdake_according to that stackoverflow link above its cut at 128 chars14:31
*** inc0 has joined #openstack-kolla14:32
inc0gmorning14:32
sdake_where are yo getting the schebang with 80 chars from jeffrey4l, which log?14:32
sbezverk_rhallisey: ping14:33
rhalliseyyo14:33
inc0how do we do with voting gates?:)14:33
Jeffrey4lsdake_, this line http://logs.openstack.org/01/398501/20/check/gate-kolla-ansible-dsvm-deploy-oraclelinux-binary-centos-7-nv/fd95181/console.html#_2016-11-18_14_03_49_72737114:33
sbezverk_rhallisey: building helm for openvswitch. Should I create bp for this or it is done under one common bp for all helm charts?14:34
rhalliseysbezverk_, create a bp per service14:34
rhalliseyits easier to track and distribute the work14:34
sdake_Jeffrey4l that is not the schebang in the ctual pip file14:34
inc0rhallisey, can you please help me with puppet?14:34
sbezverk_rhallise: cool, will do thanks14:34
sdake_that is the output14:34
sdake_of zuul14:34
rhalliseyif someone want to create bps for each helm piece, I'd super appreciate it14:34
sdake_the reason its cut at 80 chars, who knows ;)14:34
*** zhurong has joined #openstack-kolla14:35
rhalliseyinc0, what's the issue ur having?14:35
inc0https://review.openstack.org/#/c/399221/ <- we need to make this work14:35
Jeffrey4lsdake_, i test on centos-7 just now. it do cut at 80 char. ;(14:35
inc0to enable reasonable gates for both kolla-ansible and k8s14:35
portdirectegonzalez90: you about?14:37
sdake_inc0 lets get the gates working first to unblock reviews in general ;)14:37
egonzalez90portdirect: yes14:37
inc0so registry would make it a lot easier14:38
sdake_not really14:38
sdake_it would make it more reliable14:38
sdake_easier = incremental14:38
inc0easier too14:38
inc0because zuul cloner won't be requirement14:39
portdirectegonzalez90: trying to get desgniate up, and have a few qs (I'll message you to leave these cats alone :) )14:39
Jeffrey4linc0, how to trigger the deployment gate by using registry?14:39
egonzalez90portdirect: ok14:39
inc0Jeffrey4l, trigger?14:39
inc0what do you mean?14:39
*** liyifeng has quit IRC14:39
*** sdake has joined #openstack-kolla14:39
sdakeinc0 zuul cloner is a 1 liner and is already implemented14:40
Jeffrey4linc0, u do not want to use zuul cloner, right?14:40
inc0yeah, for cross-repo gates14:40
sdakeJeffrey4l lets just get  the ansible gets unblocked plz so revies can start rolling14:40
Jeffrey4lhow to test kolla-ansible when kolla is changed?14:40
sdakeJeffrey4l you do that by putting kolla-ansible gates in kolla itself...14:41
sdakethis is call cross repo gating14:41
Jeffrey4lhow to solve such case: dockerfile change and only when the kolla ansible is changed.14:41
sdakefungi mentioned this in a previous thread14:41
Jeffrey4lsdake, yep. but this is implemented by using zuul-cloner. if not, how we implement such by using registry? inc014:42
inc0Jeffrey4l, not this14:42
sdakeJeffrey4l inc0 is talking about going the toher way14:42
*** msimonin has joined #openstack-kolla14:42
inc0what we can do with registry is ansible gates voting and easier to manage14:42
sdakewhen a commit hits kolla-asnible, a registry makes things more reliable14:42
inc0as we won't need to build images in ansible14:42
*** hfu has quit IRC14:43
Jeffrey4linc0, sdake got. sorry for misunderstand ;(14:43
sdakeJeffrey4l no reason to apologieze14:43
Jeffrey4land yes. it will be helpful :)14:43
sdakeJeffrey4l still, lets get ansible unblocked ;)14:43
sdakePLZ14:43
sdakei can't review a damn thing atm14:43
*** sdake_ has quit IRC14:43
sdakeso back on topic14:43
sdakethis 80 char thing vs 128 char thing14:43
Jeffrey4lfyi sdake shebang issue in pip source code. https://github.com/pypa/pip/issues/177314:43
*** msimonin has quit IRC14:43
openstackgerritMerged openstack/kolla: Clean up loc  https://review.openstack.org/39957014:44
*** hfu has joined #openstack-kolla14:45
sdakeJeffrey4l its a kernel limit14:45
Jeffrey4lye.14:45
Jeffrey4lyes.14:45
*** lamt has quit IRC14:46
Jeffrey4lOK. i got the solution.14:46
Jeffrey4lwill push a patch to test.14:46
sdakenice14:46
sdakeoff to breakfast bbl14:47
Jeffrey4lbut still care about why centos works not oraclelinux not.14:47
*** msimonin has joined #openstack-kolla14:49
*** berendt has joined #openstack-kolla14:49
berendtfirst customer site deployed with kolla :)14:49
Jeffrey4lberendt, cool.14:49
inc0congrats:)14:50
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix deploy gate  https://review.openstack.org/39850114:51
*** lamt has joined #openstack-kolla14:51
openstackgerritMerged openstack/kolla: Remove init-runonce from docker repo  https://review.openstack.org/39957214:52
*** jtriley has joined #openstack-kolla14:53
pbourkeberendt: nice14:54
pbourkeberendt: how many nodes?14:54
berendtat the beginning about 20 nodes14:55
*** zhurong has quit IRC14:55
*** v1k0d3n has joined #openstack-kolla14:56
*** adrian_otto has joined #openstack-kolla14:57
*** Satya_ has joined #openstack-kolla14:58
Satya_Hi sdake_ you there?14:58
sdakeberendt sweet news14:59
sdakeberendt imo send a cut :)15:00
sdakeSatya_ sup15:00
Satya_Please triage "https://bugs.launchpad.net/kolla-ansible/+bug/1642878"15:01
openstackLaunchpad bug 1642878 in kolla-ansible "RabbitMQ should communicate through a different network rather than api_network" [Undecided,New] - Assigned to Satya Sanjibani Routray (satroutr)15:01
sdakeSatya_ i am out of the triaging business for awhile, sorry15:01
inc0Satya_, that's not a bug...15:01
Satya_ohh ok...15:01
sdakeSatya_ we have a whole slew of core revieweres and other folks that can triage ping them :)15:02
inc0and we don't have anything called "management network"15:02
inc0this is called api_network in Kolla15:02
sdakeSatya_ check logs of this discsussion with dave_mcc today15:03
Satya_ok i will edit for secondary ID15:03
sdakeSatya_ the problem is simple - api_network is a misnomer15:03
sdakeit should be called "internal_network"15:03
openstackgerritJeffrey Zhang proposed openstack/kolla: Remove tools/init-runonce in setup.cfg  https://review.openstack.org/39962915:03
sdakewhich is confusing to everyone15:03
Jeffrey4lsdake, inc0 pbourke check this please, it blocks the gate https://review.openstack.org/39962915:04
Satya_i added a piece as rabbitmq_interface15:04
sdakeJeffrey4l hrm i guess i messed that up15:04
sdakesorry about that - +2ed your change15:04
Satya_so it will be flexible for api_network or any other internal network...15:04
sdakeJeffrey4l the worst part is I knew I needed to remove it from setup.cfg too15:04
sdakeand then forgot about it 30 seconds later15:05
Jeffrey4lthanks ;) never mind15:05
*** narasimha_SV has quit IRC15:05
pbourkeJeffrey4l: will I approve?15:05
sdakepbourke i did15:05
sdakepbourke but ya it would hav ebeen fine for you to do so i think15:06
Jeffrey4lthe fastest patch :)15:06
sdakeobviously an error that i introduced about 30 mins ago15:06
sdakesorry guys tired from 36 hrs of bullshit15:06
sdakegot 12 hrs sleep last night tho :)15:06
sdakeyay15:06
inc0Satya_, what's the use case for rabbitmq on dedicated network?15:06
inc0rabbitmq traffic can't really saturate any relevant datacenter network, it won't add any security as every node has to have access to it anyway15:07
Satya_usually in production everyone wants the traffic segrigation15:07
inc0yes...15:07
sdakeya i get segration of traffic... for a reason :)15:07
inc0api_interface exists15:07
Satya_and the rabbitmq traffic should go with a different network rather than the api network15:08
inc0why?15:08
sdakethat is what api_interfae is for15:08
sdakeSatya_ if we did that, then things like heat-engine would have to bind to it15:08
Satya_yeah all communication is going through the api network15:08
*** TxGirlGeek has joined #openstack-kolla15:08
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix ci gate  https://review.openstack.org/39850115:08
sdakeso halfof openstack woudl have to communicate with some kind of wierd internal/external vip15:08
inc0Satya_, all openstack service-to-service communication15:08
sdakewhich might reduce security15:08
sdakeSatya_ security is ensured on kolla by the external_vip_network15:09
sdakeSatya_ please read the logs from this morning15:09
inc0yeah, if you're afraid that api_network has to be exposed externally - it doesn't15:09
sdakeyour questions asked and answered there :)15:09
Satya_yep reading...15:09
sdakeapi_network shoudl be called internal_network15:09
sdakewhy is it called api_network?  Unknown.15:09
rhalliseyinc0, https://review.openstack.org/39922115:09
sdakecan we call it internal_network now, not sure15:10
inc0w00t rhallisey15:10
inc0thanks!15:10
rhalliseythat should help get it going15:10
rhalliseythat's closer15:10
sdakeit sure would be alot simpler on everyone if it wasn't misnamed15:10
rhalliseyyou don't need too much15:10
sdakerhallisey knows the puppets?15:11
Satya_ok we should have flexibility to use any network for rabbit "https://bugs.launchpad.net/kolla/+bug/1583655"15:11
openstackLaunchpad bug 1583655 in kolla newton "rabbitmq setup from secondary IP" [Critical,Confirmed]15:11
rhalliseysdake, ye15:11
sdakeSatya_ what youa re talking about is creating two internal interfaces15:11
Satya_yes15:11
sdakeone for the api internal interface, one for the servers internal interface15:11
Satya_yes15:12
inc0Satya_, tell me reason why anyone would want rabbitmq on separate iface please15:12
Satya_where one is used for pxe booting the system and system gets the name from dns15:12
Satya_and other is the network where openstack communicates15:12
inc0well what you don't want to do is to put rabbitmq on pxe network15:12
inc0because pxe *can* saturate network easily15:13
Satya_yes15:13
sdakeSatya_ ok, the problem with that model you just described is the api servers communicate with the actual servers servers :)15:13
inc0and it will break your running openstack15:13
Jeffrey4lSatya_, did my solution work? enable the extend_interface.15:13
sdakeSatya_ so those two networks would HAVE to be connected15:13
Satya_yes15:13
Jeffrey4lyou can move all your api_interface into your mgmt network.15:13
Satya_jeffrey the solution worked after changing the hostname15:13
sdakeok cats - time for a smoke15:14
sdakebbl15:14
Jeffrey4lSatya_, hmm. my solution is move api interface, not changing hostname ;(15:14
Satya_with that the problem will be if i am going for a periodic monitoring over the mgmt network15:14
Satya_then api request will get slower15:14
Satya_that didnt worked :(15:15
Jeffrey4linc0, the root cause is rabbitmq hostname must be resolvable for the ip it listening.15:15
Satya_jerrrey15:15
Satya_yes...15:15
inc0and hostname is set up for pxe network...?15:16
Jeffrey4lSatya_, if you just move the rabbitmq traffic to mgmt network, doesn't it become slow?15:17
rhalliseyinc0, are you in #openstack-infra?15:17
Satya_nope15:17
inc0forgot to join15:17
rhalliseyk15:17
Satya_as it will be used only for pxe booting15:18
Satya_may get the monitoring also but monitoring can be on a different network also15:18
Jeffrey4l( so many limitation are show together in you env ;( )15:18
Satya_so rabbit network will be only dedicated for messaging and pxe boot15:19
Satya_DavidTurner can give us more details also :)15:20
Satya_DavidTurner_ there?15:20
Jeffrey4lSatya_, check this https://www.rabbitmq.com/configure.html15:20
Jeffrey4lsearch 'env hsotname'15:20
Jeffrey4lseems the rabbitmq can use a different hostname rather than the default one.15:21
Jeffrey4lif so, you can change the hostname to `*-api` without changing the real hostname.15:21
Satya_we already tried that.. :( nodename dont take any effect...15:22
Jeffrey4lit is HOSTNAME, not NODENAME Satya_15:22
Jeffrey4lRABBITMQ_NODENAME and HOSTNAME are two different variables. Satya_15:23
inc0Jeffrey4l, it will take ansible_hostname15:23
*** prameswar has joined #openstack-kolla15:23
inc0what you can do is to force ansible_hostname to x in inventory15:23
Satya_NODENAME=bunny@myhost15:23
inc0but let me say that again, rabbitmq on pxe network is asking for troubles15:23
Jeffrey4lNameDefaultDescription15:24
Jeffrey4lHOSTNAME15:24
Jeffrey4lUnix, Linux: `env hostname`15:24
Jeffrey4lMacOSX: `env hostname -s`15:24
Jeffrey4lThe name of the current machine15:24
Jeffrey4lsorry for multi line.15:24
Jeffrey4lbut could u check this here https://www.rabbitmq.com/configure.html15:24
Jeffrey4li think it should work. ( tbh, i never tried this )15:24
*** jtriley has quit IRC15:25
Satya_it says rabbitmq depends on hostname15:25
Satya_but not specifying we can have a entry in the config so it will change the hostname of our wish...15:26
Satya_am i right or reading that in a different way not sure :(15:27
*** adrian_otto has quit IRC15:27
inc0Satya_, so hold on rabbitmq doesn't use api_interface IP really15:28
Jeffrey4lit says rabbitmq depends on HOSTNAME, and its default value is `env hostname`.15:28
*** prameswar has quit IRC15:28
Satya_nope15:29
Jeffrey4lthen i think it can be change. ( if not, that is no *default* concept )15:29
Satya_it uses the hostname configured15:29
DaveTurnerSatya_ I'm here15:29
Satya_Hi Dave :)15:29
inc0it uses hostname15:30
inc0exactly, and it will use hostname15:30
Satya_yes15:30
inc0and your hostname points to different interface than api15:30
inc0so rabbitmq will work on it's own15:30
inc0as for openstack services15:31
inc0just use config override to point to rabbitmq you want15:31
inc0and that should just work15:31
Satya_and if a dns is configured and pxe boot will be done from that network the hostname will be a different than api if someone wants to use the api network different than the pxe network15:31
inc0this somone is openstack services15:31
Satya_nope15:32
inc0so you can change configuration of these to use pxe address15:32
Satya_yes15:32
Satya_Dave can give more details about the situation we are in...15:33
Satya_smoke time for me :) will be back in a bit...15:33
inc0http://docs.openstack.org/developer/kolla/advanced-configuration.html#openstack-service-configuration-in-kolla15:33
inc0Satya_, ^15:33
inc0use this override mechanism and override transport_url for rabbitmq to point to nodes over pxe15:34
inc0and it should do what you want15:34
DaveTurnerSatya_ This dependency on hostname is a concern for use, since we typically install our corporate monitoring tool on all hosts, which also has a dependency on host name- but we do not want the monitoring traffic running on same vlan as rabbit traffic.15:34
DaveTurnerfor *us15:34
inc0DaveTurner, that's issue of rabbitmq tho, unfortunately it doesn't work with IPs15:35
inc0nothing we can do about it15:35
Satya_We have a working code which can help us actually..15:35
Satya_just want to check if we can put that for upstream for review...15:36
Satya_that introduce a new interface which can be configured to the primary interface from which the hostname is configured15:38
portdirectI've worked round rabbit is issues similar to this by bind mounting the containers /etc/hosts to somewhere in /var/run and injectsing a custom config into the container - dont now if this may help you?15:39
inc0yeah, that would actually work, override in-container /etc/hosts15:41
*** hfu has quit IRC15:41
*** senk has joined #openstack-kolla15:41
Satya_That will actually help but that need a lot of manual intervention for changing the transport url for all the services...15:41
Satya_:(15:41
*** hfu has joined #openstack-kolla15:41
*** berendt has quit IRC15:41
*** hfu has quit IRC15:42
inc0yeah, best we can do atm15:42
*** hfu has joined #openstack-kolla15:42
*** hfu has quit IRC15:43
*** hfu has joined #openstack-kolla15:43
Satya_atm? i know that as all time money/all time medicine/ a time machine15:43
Satya_:P15:43
*** hfu has quit IRC15:43
*** sdake has quit IRC15:44
*** hfu has joined #openstack-kolla15:44
*** senk has quit IRC15:44
*** hfu has quit IRC15:44
inc0at this time15:44
inc0or at the moment15:44
inc0rather15:44
*** hfu has joined #openstack-kolla15:45
Satya_:)15:45
*** senk has joined #openstack-kolla15:45
*** hfu has quit IRC15:45
*** senk has quit IRC15:45
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Mariadb Kubernetes Operator  https://review.openstack.org/39926315:45
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Kubernetes Operator base  https://review.openstack.org/39926215:45
portdirectwith newer versions of docker it can get quite ugly, as the OCI runtime expects to be able to play with /etc/hosts - though you can work round this by bind mounting somewhere else and then cating the correct contents of the hosts file into place before starting rabbit - so yeah it's a horrible hack..15:46
portdirectbut works15:46
Satya_and changing the transport url for all components15:47
portdirectsorry can't help there - only just starting to use kolla containers - I'm not using kolla-ansible I'm afraid :(15:49
Satya_hmm15:49
inc0changing transport url would be your best bet15:51
inc0on the bright not it's not THAT much15:51
inc0just similar file for every openstack service you're running15:51
inc0so one for glance, one for neutron15:51
*** sdake has joined #openstack-kolla15:51
inc0there are only so many services available15:51
sdakei wonder if you could run the univerise in a docker container15:52
sdake;-)15:52
inc0you can run vm in docker container and universe in vm15:52
sdakehow did Jeffrey4l 's workaround go?15:53
*** adrian_otto has joined #openstack-kolla15:54
*** tonanhngo has joined #openstack-kolla15:54
*** msimonin has quit IRC15:54
*** tonanhngo has quit IRC15:54
*** msimonin has joined #openstack-kolla15:54
*** msimonin has quit IRC15:56
Satya_inc0 i see transport url is in lot many places15:57
*** srwilker has quit IRC15:57
Satya_http://paste.openstack.org/show/589726/15:57
*** zhubingbing has quit IRC15:57
inc0yeah..once per every service15:57
inc0but do you run every service?15:57
Satya_yes15:58
Satya_including trying over gnocchi :P15:58
*** jtriley has joined #openstack-kolla15:58
inc0ok, well, grep -ir transport_url | cut -d/ -f115:58
inc0and do cp override file based on output15:59
*** eaguilar has quit IRC15:59
*** msimonin has joined #openstack-kolla15:59
inc0every problem in the world can be solved by shell one-liner15:59
*** eaguilar has joined #openstack-kolla15:59
Satya_lol yeah15:59
Satya_but i have to do on all servers16:00
inc0ansible then16:00
inc0and no16:00
inc0you don't16:00
inc0just deployment node16:00
Satya_currently i have 8 nodes16:01
Satya_but i am scared if i say this solution to the ops guys they will get mad over us...16:01
inc0Satya_, you do it only for deployment node16:04
inc0so just tell them to copy this directory to etc and done:)16:04
*** Pavo has quit IRC16:07
Satya_what about add a new interface as rabbitmq_interface and give the flexibility for user to change it anyways and default it to api_interface16:09
*** Pavo has joined #openstack-kolla16:12
inc0it's not really bad change, but we'd need to do this across the board16:13
Jeffrey4lsdake, it should work. but wait for the init-once fix to move on. https://review.openstack.org/39850116:13
inc0still, to be honest I'd stick to what's available today16:13
macszdid anyone try to use gpu passthrough with a kolla-provisioned cloud? there is this bug -> https://bugs.launchpad.net/nova/+bug/1642419 and I am wandering if this is docker-related issue or sth is wrong with nova16:13
openstackLaunchpad bug 1642419 in OpenStack Compute (nova) "GPU Passthrough isn't working" [Undecided,New]16:13
sdakemacsz could be either16:14
sbezverk_kfox1111: ping16:14
sdakei'd speculate gpu passthrough requires some type of host bindmount16:14
sdakei'd speculate nova gpu passthrough is not all that solid16:14
sdakejust speculations :)16:14
Satya_i have the change i can push16:15
inc0I added that it also affects us16:15
inc0I bet nova-compute somehow doesn't see gpu16:16
macszsdake: that might be true :) i hoped to push the responsibility on other project (if, for instance you knew that gpu passthrough was not working in kolla deployments), but I guess it stays in  it's home town for now :D16:16
*** tonanhngo has joined #openstack-kolla16:16
openstackgerritMerged openstack/kolla: Remove tools/init-runonce in setup.cfg  https://review.openstack.org/39962916:16
sdakemacsz i dont know anyone has ever tried16:17
*** tonanhngo has quit IRC16:17
sdakemacsz what is your use case btw16:17
inc0I know Sam had some fun with PCI passthru16:17
inc0but he got it working16:18
*** msimonin has quit IRC16:18
macszsdake: i am just doing a bug triage16:18
sdakemacsz roger16:18
Satya_macsz please triage "https://bugs.launchpad.net/kolla-ansible/+bug/1642878"16:19
openstackLaunchpad bug 1642878 in kolla-ansible "RabbitMQ should communicate through a different network rather than api_network" [Undecided,New] - Assigned to Satya Sanjibani Routray (satroutr)16:19
kfox1111morning16:19
kfox1111sbezverk_: ping16:20
sbezverk_kfox1111: hey, any chance you could separate common macro in a separate ps and push it I need it to complete openvswitch16:21
*** msimonin has joined #openstack-kolla16:21
kfox1111sbezverk_: I think it would be better to do it as a dependency on my ps. that way, you can reuse the build scripts and test it with the gate script more easily.16:22
macszSatya_: i am working on nova right now, I would love to help with everything, but my time is limited and thus I am afraid I can't help with this bug :)16:23
kfox1111should be able to just add an entry in tools/build_packages.py to get it to build your stuff. and tweak tests/bin/ceph_workflow.sh to swithout the openvswitch kollakube res call with the helm install one.16:24
*** tonanhngo has joined #openstack-kolla16:24
Satya_NP macsz :)16:24
*** tonanhngo has quit IRC16:25
*** adrian_otto has quit IRC16:26
*** tonanhngo has joined #openstack-kolla16:26
sbezverk_kfox1111: it is not very convenient as you keep changing other things in your ps, also bunch of other folks we need to do the same and this approach does not look too good..16:27
kfox1111I think I'm done changing plumbing, unless others have objections to it.16:28
kfox1111I'm thinking its close to commitable actually.16:28
portdirectkfox1111: I'll have a look over - but so far LGTM16:29
sbezverk_kfox1111: second question, what happened to namespace in spec? it is gone now.16:30
kfox1111sbezverk_: just commented on that.16:30
*** eaguilar has quit IRC16:31
*** duonghq has quit IRC16:32
portdirectkfox1111/sbezverk: I think thats the right approach with helm (no ns in chart) but we should double check - I've gone through helm's source as the docs seem pretty thin.16:32
sbezverk_portdirect: I think having helm variable with namespace consistent accross kolla kubernetes look better imho16:35
*** rmart04 has quit IRC16:36
sbezverk_it is one point to change if you want using another namespace16:36
kfox1111lets hold the conversation in the spec for posterity.16:37
portdirect+!16:38
portdirect+1 (whoops)16:38
kfox1111(I think having the conversation though is a good idea. I'm not too tied to any particular solution. trying to figure it out myself)16:38
*** aric49 has joined #openstack-kolla16:39
sbezverk_kfox1111: sounds good.. but helm not having something global and common is bad :-( Before we had a common kubernetes.yaml file with all variables, now we are going to have bunch of per-microservice config/value files. it looks like a config nightmare to me..16:40
kfox1111sbezverk_: I see it as a building block. we probably still want the overarching common config, as thats super nice.16:40
kfox1111so we may want to do something like have kollakube continue to be a thing,16:40
aric49Hi Everyone ---  quick question about deploying Kolla.16:41
kfox1111and just have kollakube res create ... call helm install?16:41
aric49According to the docs, it looks like Ubuntu 14.04 is supported. Does anyone know if Ubuntu 16.04 Xenial will work?16:41
kfox1111at least until some other workflow bits pop up, such as operators/(helm/entrypoint orchestration)16:41
sbezverk_kfox1111: ok, hopefully it will be addressed somehow or by somebody ;-)16:43
aric49Or if there is anything to watch out for when deploying on Xenial?16:43
kfox1111sbezverk_: yeah. :)16:43
kfox1111I think the architecture spec has it covered in at least two different ways.16:43
kfox1111sbezverk_: ryan's already started working on operators too. so it may be covered there soon.16:44
*** TxGirlGeek has quit IRC16:45
*** severion has joined #openstack-kolla16:46
*** TxGirlGeek has joined #openstack-kolla16:46
*** sdake has quit IRC16:47
*** egonzalez90 has quit IRC16:48
*** v1k0d3n has quit IRC16:49
*** TxGirlGeek has quit IRC16:50
*** TxGirlGeek has joined #openstack-kolla16:51
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible: Ignore dom0 qemu processes during destroy  https://review.openstack.org/39920316:53
*** TxGirlGeek has quit IRC16:53
*** TxGirlGeek has joined #openstack-kolla16:54
*** TxGirlGeek has quit IRC17:00
*** Serlex has joined #openstack-kolla17:01
*** TxGirlGeek has joined #openstack-kolla17:01
portdirectthrowing it out there - but it would be very easy to chain helm by having a chart containing jobs that simply launched other helm charts17:03
*** TxGirlGeek has quit IRC17:03
*** adrian_otto has joined #openstack-kolla17:04
*** TxGirlGeek has joined #openstack-kolla17:04
*** msimonin has quit IRC17:05
openstackgerritMerged openstack/kolla-ansible: Fix neutron.conf.j2 metadata_workers spelling error  https://review.openstack.org/39935617:08
*** dasm is now known as ferros17:09
*** ferros is now known as dasm17:09
inc0can I get ack on this? https://review.openstack.org/#/c/399082/17:10
*** bmace has quit IRC17:10
*** bmace has joined #openstack-kolla17:11
kfox1111portdirect: True.17:13
*** matrohon has quit IRC17:13
kfox1111that may solve the issue I came up with where you may want one package, but multiple subpackage instantiations of the same package.17:13
mliimadone inc017:16
inc0thanks17:17
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible: Tacker NFV Ansible support  https://review.openstack.org/39917917:21
*** eaguilar has joined #openstack-kolla17:26
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible: Horizon custom policies  https://review.openstack.org/39970717:27
sbezverk_kfox1111: when I have two yaml in template and run helm install, helm picks up olny first file in template and ignores second, am I missing something or I need to dump all specs in the same yaml file?17:27
kfox1111so...17:29
kfox1111in regular helm build, it should grab everything in the templates directory.17:29
kfox1111for our stuff, I've been making it one object per package. that way you have nice fine grained upgradability.17:30
kfox1111we can then make upper level packages that combine the subpackages.17:30
*** severion has quit IRC17:34
sbezverk_kfox1111: that part I figured out, I am doing one chart for ovsdb and one chart for vswitchd17:34
kfox1111can I please get another kola-kube core for this: https://review.openstack.org/#/c/396812/17:34
*** v1k0d3n has joined #openstack-kolla17:34
sbezverk_but along with ovsdb I have config map17:35
sbezverk_helm pick up configmap but ignores yaml for daemonset17:35
sbezverk_strange :-(17:35
kfox1111I think we want to not put the configmaps in the the packages.17:35
kfox1111otherwise, the user will have to get into the buisness of customizing packages to do config updates.17:36
kfox1111or alternatively for some workflow's, we put configmaps in their own packages.17:37
sbezverk_kfox1111: configmap is temporary just to test ovs and not bother with generating config maps17:37
kfox1111ah.17:37
kfox1111there's the gate tests too.17:37
kfox1111which is why I was recommending basing on my ps.17:38
kfox1111I've been running helm install --dry-run --debug testpackage,17:38
kfox1111looking to see if it renders fine, then git review; check experimental17:38
openstackgerritMerged openstack/kolla-kubernetes: External openvswitch  https://review.openstack.org/39681217:38
kfox1111sbezverk_: thanks17:39
kfox1111meeting time. bbiab17:40
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible: Add custom policies in service.json  https://review.openstack.org/39971217:40
*** rmart04 has joined #openstack-kolla17:43
inc0pbourke, still around?17:44
pbourkeinc0: yeah17:44
inc0https://review.openstack.org/#/c/398501/ <- care to take a look17:45
inc0?17:45
inc0oracle gates are bad, maybe you'll spot what's wrong17:46
*** rmart04_ has joined #openstack-kolla17:46
*** schwicht has joined #openstack-kolla17:47
openstackgerritMerged openstack/kolla: Dependency init container  https://review.openstack.org/39908217:48
*** rmart04_ has quit IRC17:48
*** rmart04 has quit IRC17:48
inc0yay merge with voting gates \o/17:49
pbourkeinc0: myself Jeffrey4l  and sdake were looking at this earlier17:52
pbourkewe had an idea so I need to check if Jeffrey4l tried that yet or not17:53
inc0ok:)17:53
Jeffrey4lpbourke, inc0 still struggle on this issue ;(17:53
openstackgerritMauricio Lima proposed openstack/kolla: Add a section regarding share migration  https://review.openstack.org/39971517:53
pbourkeJeffrey4l: did you try hacking the shebang?17:53
Jeffrey4li am trying to remove the `tox -e xxxx` right now.17:53
Jeffrey4lit is useless after split.17:53
Jeffrey4lpbourke, hacking do not work. before run tox -e, the virtual env and the pip file need to be hack is not exist.17:54
pbourke:(17:55
pbourkeffs17:55
openstackgerritMauricio Lima proposed openstack/kolla: Add a section regarding share migration  https://review.openstack.org/39971517:56
*** adrian_otto has quit IRC17:58
pbourkeJeffrey4l: python -m also not working?18:05
Jeffrey4lpbourke, yes. tox call `pip freeze` there is no place to overwrite this.18:05
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix ci gate  https://review.openstack.org/39850118:06
*** unicell has quit IRC18:06
*** Pavo has quit IRC18:07
Jeffrey4lpbourke, ^& this make life easier. but it won't work until this https://review.openstack.org/#/c/399694/ is merged.18:07
mliimaJeffrey4l, we need merge it https://review.openstack.org/#/c/399694/ ?18:08
Jeffrey4lmliima, yep..18:08
Jeffrey4lmliima, https://review.openstack.org/398501 this need that change.18:09
mliimaoh god18:09
inc0Jeffrey4l, I'll ask infra cores to review it18:09
Jeffrey4lmliima, it make test easier. there is no need to call tox -e xxx to generated the images we need.18:09
Jeffrey4land it won't work too. ( we need other way solution )18:10
Jeffrey4lcurrent, i simple use `kolla-build -p gate || true ` :(18:10
Jeffrey4linc0, thanks.18:11
Jeffrey4lhrm, maybe  run `testr run test_build.DeployTestUbuntuSOurce` in kolla folder is better .18:11
*** adrian_otto has joined #openstack-kolla18:11
*** Pavo has joined #openstack-kolla18:12
*** pbourke has quit IRC18:12
*** pbourke has joined #openstack-kolla18:12
mliimaJeffrey4l, should have a better way to do `kolla-build -p gate || true `18:14
Jeffrey4lmaybe  run `testr run test_build.DeployTestUbuntuSOurce` in kolla folder is better .18:14
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Initial Helm support  https://review.openstack.org/39629618:14
Jeffrey4lmliima, more work should be done at this point.18:14
mliima:)18:14
Jeffrey4lor once the infra registry is done, we won't need this.18:14
jascott1checkout dude's handheld k8s cluster :) https://hackernoon.com/diy-kubernetes-cluster-with-x86-stick-pcs-b0b6b879f8a7#.o2mvf79u018:15
Jeffrey4ljust pull the image from infra'd registry.18:15
mliimathen this is temporary?18:15
mliima`kolla-build -p gate || true `18:15
kfox1111Jeffrey4l: wow. fun. :)18:15
Jeffrey4li hope so. i hope infra can set the docker registry up. inc0 is working on this.18:16
inc0yeah, this review https://review.openstack.org/#/c/399221/18:16
mliimaI do not know if it is worth spending time improving it if it will change after registration get ready Jeffrey4l18:16
mliimaregistry18:17
mliimainfra registry*** hehe18:17
Jeffrey4lmliima, wait for day. the registry PS seems will be merged soon,.18:17
mliimano problem Jeffrey4l, i trust in your work18:19
Jeffrey4lmliima, it is inc0's work. ;)18:19
inc0Jeffrey4l, we still need to write solid puppet18:19
*** senk has joined #openstack-kolla18:19
mliimahaha18:19
mliimaokay18:19
inc0but it will be super beneficial for out gates18:20
Jeffrey4lyep.18:20
inc0this will allow us voting deploy gates18:20
Jeffrey4lit will speed up the gate.18:20
mliimainc0, why not use 16.04? https://review.openstack.org/#/c/399221/218:22
mliimaany special reason?18:22
inc0infra runs Trusty mostly on mirror nodes afaik18:22
mliimaoh18:23
mliimamake sense18:23
*** Serlex has quit IRC18:29
*** jtriley has quit IRC18:33
*** unicell has joined #openstack-kolla18:36
*** krtaylor has joined #openstack-kolla18:46
*** Satya_ has quit IRC18:55
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: fix ci gate  https://review.openstack.org/39850118:58
*** lamt has quit IRC18:58
*** unicell has quit IRC19:00
*** sdake has joined #openstack-kolla19:02
*** senk has quit IRC19:09
*** schwicht has quit IRC19:10
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Initial Helm support  https://review.openstack.org/39629619:15
*** gfidente has quit IRC19:16
*** jtriley has joined #openstack-kolla19:17
sdakehi peeps19:22
kfox1111hi19:22
*** mliima has quit IRC19:23
*** unicell has joined #openstack-kolla19:32
*** unicell has quit IRC19:32
*** unicell has joined #openstack-kolla19:32
*** schwicht has joined #openstack-kolla19:38
*** jtriley has quit IRC19:51
kfox1111sbezverk_: thanks by the way, for reviewing and pushing/asking questions about the helm archetecture stuff.19:58
kfox1111its kind of weird the way I did it, and its good to figure out why I did what I did and if its really the right aproach.19:58
sbezverk_kfox1111: sorry Kevin, but it feels a bit uncomfortable using the process as it is now, well at least to me.20:06
sdakewhich process, the review process?20:07
sbezverk_kfox1111: I send emails to some helm experts asking for some input, will share if anything comes back from them..20:07
sbezverk_sdake: no, the way things rendered with helm20:07
*** jtriley has joined #openstack-kolla20:07
*** Pavo has quit IRC20:07
kfox1111sbezverk_: I think we should write down the issues we have discovered,20:07
kfox1111and run the set by them and see what they think.20:08
sdakesound fantastic20:08
sdakei'd like to know to o:)20:08
sbezverk_kfox1111: +220:09
kfox1111I really need to get some other work done today though, so will have to wait till at least tonight. :/20:10
sbezverk_kfox1111: no rush :-)20:12
*** schwicht has quit IRC20:12
*** Pavo has joined #openstack-kolla20:12
kfox1111semi rush. :/   holds up doing anything other then prototyping.20:12
*** schwicht has joined #openstack-kolla20:13
kfox1111sbezverk_: with helm and nesting, the vars are the api for the package. a lot of things we can decide upon later, but it will be painful to change an api. so we should invest the most time into figuring out what that part should be.20:15
sbezverk_kfox1111: ok, if you do not mind I want to try nesting approach, just to see if I hit some roadblock20:25
sbezverk_and if it works as helm people suggesting20:25
inc0the world ends, got 500 from gmail20:26
*** eaguilar has quit IRC20:28
sdakeinc0 internet not been woroking well last couple of days20:39
inc0yeah20:39
inc0on that note, I'm off20:39
inc0need to see Houston before I leave TX;)20:40
inc0have a good weekend folk20:40
inc0s20:40
*** inc0 has quit IRC20:40
sdakekfox1111 possibly a dumb question20:44
sdakere https://review.openstack.org/#/c/396296/27/helm/src/neutron/templates/l3_agent_daemonset.yaml20:44
sdakewhy are VolumeMounts json in a yaml file?20:44
*** ipsecguy has joined #openstack-kolla20:45
sbezverk_sdake: it is the way how init containers are implemented in kube these days20:48
sdakeinit containers not implemented in yaml because???20:48
sdakejust curious if you know20:48
jascott1sdake you are talking about init container?20:48
sdakejascott1 apparently :)20:49
sbezverk_ sdake: because it is still alpha feature, will be moved to yaml when stop being alpha20:49
sdakecool20:49
sdakethanks sbezverk_20:49
jascott1that was my guess too  :)20:49
sdakeso the alpha apis use json, the non-alpha apis use yaml?20:49
sbezverk_sdake: yep20:49
*** ipsecguy_ has quit IRC20:49
sdakewill upstream kubernetes definately move that to yaml later?20:50
sdakehelm serve is a rest API?20:51
jascott1helm serve creates local chart repo20:51
sdakeand exposes via which type of api jascott1 ?20:52
sdakeor it just on disk?20:52
jascott1i think it is just expected to have index file with all packages listed in json20:52
jascott1dont know that it has an api20:52
jascott1basically a simple webserver20:53
sdakewhen you get helm packages20:53
sdakeyou are doing  a GET operation on some rest endpoint ?20:53
jascott1ya so you can do 'helm package'20:53
jascott1on a local repo20:53
sdakejascott1 my understanding of helm after reading zero docs and just obsorbing the last couple of weeks20:54
sdakeis that it has a build process20:54
sdakewith the current implemenetaiton we got, can we build it once and forget about it?20:54
jascott1when you use helm to retrieve package from repo, it looks at that repo's index file, finds the chart/version you are lookin for and returns that tgz20:54
sdakemuch like we do with docker containers now20:54
jascott1pretty much20:55
sdakeok, but there is config in these things?20:55
sdakei know there is the --set operation20:55
sdakethats super clunky to me - for the same reason ENV variables are clunky20:55
sdakeyou gotta know em all ahead of time20:55
jascott1the repo is pretty dumb, they have examples of using github pages to serve one20:56
sdakethe build is via rest api?20:56
jascott1tiller has api for releases20:56
jascott1but the repo doesnt afaik20:56
kfox1111sdake: alpha api's are things k8s doesn't want to commit too 100% by puting them in their yaml spec.20:56
kfox1111so they cram them into key/value pairs in attributes until they stabalize.20:57
kfox1111serialized to a string, json's a little cleaner then yaml for doing that.20:57
sdakekfox1111 ahh, got it20:57
sdakekfox1111 thanks for the background20:57
sdakei'm after a specific peice of information jascott120:57
kfox1111init containers are now listed as beta in 1.420:58
sdakewhen we implement operators, they will use helm charts to deploy20:58
jascott1ya i forgot to mention its a beta annotation now20:58
sdakedoes that mean operators also need to build the helm charts?20:58
kfox1111sdake: no.20:58
sdakecool20:58
jascott1if the operator is crafting an update maybe20:58
sdakenot cool :)20:59
kfox1111I want to be able to ship binary prebuilt packages just like we ship binary prebuilt containers.20:59
jascott1imo depdends20:59
*** shardy has quit IRC20:59
sdakeok which one is it :)20:59
kfox1111so users can not have to worry about building packages at all, unless they are using them for orchestration.20:59
sdakeneed to know up front to implement operators20:59
jascott1idk kfox1111 seems to have designs on it so maybe he has thought it through :)20:59
kfox1111and then they only need to build the orchestrator packages.20:59
kfox1111so operators wouldn't use that.20:59
sdakewhat is an orchestor package21:00
sdakeyou mean the master package?21:00
kfox1111sdake: the entrypoint/helm thing.21:00
kfox1111making a "neutron" package with all the microservices packages in it, and using helm to orchestrate the deployment.21:00
sdakeso my assumption would be the operator would do a helm install neutron21:01
kfox1111no. the operator would helm install neutron-l3-agent, neutron-openvswitch-agent, etc.21:01
kfox1111as its orchestrating the deployment of the microservices.21:01
sdakeok21:01
kfox1111"one of these, two of these, etc.."21:01
sbezverk_kfox1111: it becomes tricky if you want to deploy just 1 microservice21:01
jascott1my understanding of the existing operator is that they watch TPRs for changes and then invoke some action (as well as doing other non TPR related stuff)21:01
sdakekfox1111 i like that alot21:01
kfox1111sbezverk_: can always just use the package directly in that case.21:01
sdakekfox1111 i said i wouldn't harass you for  a week :)21:02
kfox1111sdake: so, the neturon operator would do the logic in ceph_workflow.sh from the corisponding neutron stuff.21:02
kfox1111jascott1: right.21:02
kfox1111jascott1: so the tpr has the config description for "neutron", and the operator decides what microservices are needed, then instantiates the helm microservices packages related to the desired config.21:03
sdakekfox1111 cool - so the config does entirely from configmaps, is that accurate?21:03
sdakeI would think there would be other config options needed as well in the long term future21:03
kfox1111semitrue.21:03
kfox1111the microservices themselves have some config bits too.21:04
kfox1111there's config for openstack stuff. nova.conf kinds of things. container level.21:04
kfox1111and then theres a little bit of config for the micrservices themselves.21:04
sdakethat config is what?21:04
kfox1111enable_kube_logger=true #slide in a fluentd into the pod so it will work with elasticsearch... kinds of things.21:04
jascott1we might need a flowchart ;)21:05
kfox1111+1 for flowchart. :)21:05
sdakeso happy to make a diagram21:05
sdakedo we have a full list of the configs the microservices need?21:05
sdakeor is there a way to find that out21:05
sdakewhat i'm really after is - does helm have some specific config bits related to openstack itself21:06
kfox1111no. :/ we'd have to crawl the services subdirectory and find out whats there now.21:06
kfox1111no.21:06
kfox1111openstack configs themselves just come in through configmaps.21:06
sdakecool so its got a bunch of undefined config options21:06
sdakethat are part of the services directory21:06
sdakeservices directory contains what - k8s stuff or helm stuff?21:06
kfox1111microservices have some config stuff that tweak the k8s objects to change/add features. like enabling/disabling entrypoint, selecting log agent, that kind of thing.21:07
kfox1111those come through helm vars.21:07
kfox1111service discovery's an overloaded term. which definition are you using?21:07
sdakeare those standardized?21:07
sdakei didn't say service discovery21:07
kfox1111sdake: kind of.21:07
sdakei said services DIRectory21:07
kfox1111oh.21:08
kfox1111ok.21:08
kfox1111right.21:08
kfox1111services directory is the current jinja2 based k8s objects21:08
kfox1111those will be converted to helm golang based templates.21:08
sdakewill be converted where?21:08
sdakewhen and by who :)21:09
kfox1111the helm subdir21:09
kfox1111I've got 2 converted in the prototype here:21:09
kfox1111https://review.openstack.org/#/c/39629621:09
sdakei see, so we are dumping k8s objects entirely?21:09
kfox1111once we come to an agreement on how the build system should work, we can do the rest.21:09
sdakei am looking at that review right now21:09
sdakethats what triggered these qs :)21:09
kfox1111no. just switching out jinja2 templates for golang templates21:10
jascott1the k8s objects are just being transformed from jinja to gotpl21:10
kfox1111and reparenting them to the helm dir.21:10
jascott1sorry replacing is better choice than transforming21:10
jascott1(of words)21:10
*** jheroux has quit IRC21:11
*** fragatina has quit IRC21:11
jascott1if we have 'master' values file with everything in it, how would we be able to use that to install only specific charts?21:12
sdakeso sounds like helm as per spec of being optional is not optional21:12
kfox1111jascott1: we can't have master values in helm.21:12
kfox1111:/21:12
jascott1I could see parsing it for the 'neutron' key and then applying those values to neutron chart21:12
kfox1111sdake: I can't quite parse that.21:12
sdakeis helm no longer optional after a reparenting and reimplementation in gotpl?21:13
kfox1111sdake: oh. you mean, the templates?21:13
kfox1111yes/no...21:13
sdakemaybe?21:13
sdakewhich one of the 3 :)21:13
kfox1111the templates are pure gotpl. so could be loaded into a webserver or something to render.21:13
kfox1111rather hten helm.21:13
sdakeok so let me be more precise21:13
kfox1111so, yes, more generic then just helm.21:13
sbezverk_jascott1: has supports for global variables but it is kinbd of ugly21:13
kfox1111though not without a bit of code.21:13
sbezverk_I meant helm21:14
sdakeare we picking a winner with helm and ignoring the kubernetes api for things like creating pods and whatnot21:14
kfox1111sdake: for some things, yes.21:14
sdakewhich classification of things?21:14
sdakeare in vs out21:14
kfox1111ie, we're using helm to package up templates that render to k8s objects and helm passes to k8s.21:14
kfox1111for other things, like configmaps, secrets, etc,21:14
kfox1111no21:14
sdakeok so the yes = picking a winner21:15
sdakewhich is fine21:15
sdakei think helm is a natural choice here21:15
kfox1111yeah. mostly picking a winner.21:15
kfox1111agreed.21:15
sdakei am really keen to keep build separate from deploy21:16
kfox1111it wouldn't be hard to switch to a different pm though if things changed.21:16
kfox1111+121:16
sdakeand to do that, need to keep config separate from build21:16
kfox1111which is why the build_package.py thing is a thing.21:16
kfox1111that too. yeah.21:16
sdakeso, we have some config that gets built into the package21:17
sdakehere is where i'm stuck if build separate from deploy and deploy depends on config21:18
kfox1111not really.21:18
sdake(to configure the helm package)21:18
kfox1111we have default vars for templates built into the packages.21:18
kfox1111which anything is free to override.21:18
sdakehrm, going back on my word about not harassing you for  a week :)21:18
kfox1111:)21:18
*** dave-mcc_ has quit IRC21:18
kfox1111its important to get right. no worries.21:19
sdakewell this is a keen lesson kolla learned early on21:19
sdakeand i want to make sure we reuse it21:19
rhalliseyyo21:19
sdakeis to not have a slew of config variables for everything21:19
sdakebut instead provide a way to override custom stuff21:19
sdakewe learned this with docker's ENV pattern21:19
sdakewhich sucks from my experience21:20
kfox1111right.21:20
sdakeso keen not to repeat that21:20
rhalliseyeverything is layered like we said21:20
sdakegot it rhallisey21:20
kfox1111helm follows the same docker env mispattern21:20
sdakefocused on implementing operators :)21:20
kfox1111if you put configmaps in them.21:20
rhallisey:D21:20
rhalliseyoperators are an essentials piece21:20
sdakerhallisey not focused on rewritting the spec :)21:20
rhalliseyya I'm just reading bacl21:21
rhalliseysdake, did you see latest patch?21:21
rhalliseyI added the dockerfiles21:21
sdakerhallisey i have not21:21
rhalliseykk21:21
sdakerhallisey i just woke up about an hr ago21:21
rhalliseynp21:21
sdakeafter crashing again around 8am this morning21:21
jascott1ah the mad scientist schedule :)21:22
sdakejascott1 pretty much21:24
*** adrian_otto has quit IRC21:24
sdakekfox1111 mind pointing me at an example helm variable21:24
jascott1love it! I have to stick a modified maker schedule (late morning) since I have to show up to Intel quite a bit21:24
sdakeso i Ican figure out how to launch helm correctly in  the operator21:25
kfox1111yup. sec21:25
jascott1i miss being able to work as late as I want :/21:25
kfox1111https://review.openstack.org/#/c/396296/27/tests/bin/ceph_workflow.sh21:25
kfox1111the operator would be replacing the workflow in that script.21:26
kfox1111so would be calling helm in the same way.21:26
sdakeok thats cool21:26
sdakebut I'd like to understand a level deeper ;)21:26
sdakefor personal growth21:26
kfox1111grab the ps, do a: tools/build_packages.py /tmp/repo21:26
*** msimonin has joined #openstack-kolla21:26
kfox1111then:21:26
kfox1111helm install /tmp/repo/neutron-openvswitch-agent.* --set enable_kube_logger=false,type=compute,selector_key=kolla_compute,tunnel_interface=docker0 \ --namespace kolla --name neutron-openvswitch-agent-compute --dry-run --debug21:27
jascott1so we dont think a top level values file will work? wont it be unruly if they are all spread out?21:27
kfox1111then you canmess around with the --set flags.21:27
kfox1111jascott1: yup, and yup.21:28
kfox1111jascott1: thats one reason I'm building the common lib and build bits21:28
kfox1111jascott1: so the code is at least shared, and defaults are common.21:28
jascott1keys should be common too21:29
jascott1service_port over mariadb_service_port for instance21:29
jascott1much of it can be generic that way21:29
kfox1111services should too. yeah.21:30
kfox1111what do you mean by keys?21:30
jascott1in the yaml21:30
jascott1i still have master values file in my head21:30
kfox1111oh. "helm install kolla/neutron-openvswitch-agent --version 2.0.2-1 \ --set enable_kube_logger=false,type=compute,selector_key=kolla_compute,tunnel_interface=docker0 \ --namespace kolla --name neutron-openvswitch-agent-compute21:30
kfox1111bad paste....21:30
kfox1111so, "service_port over mariadb_service_port for instance"21:30
jascott1it has keys: global, nuetron, nova etc21:30
kfox1111got it. yeah.21:30
jascott1pass that subtree to respective chart21:30
sdakei'm fairly keen to have one set of global config options between platform tools21:31
kfox1111yeah. the operators will do some kind of passing those config bits through to the right packages.21:31
jascott1so we only need one or two common template for something like readinessProbes21:31
jascott1and the common template can just reference {{ service_port }}21:31
kfox1111jascott1: we have common-deployments with all that in it.21:31
sbezverk_kfox1111: hehe I got ovsdb running using helm chart :-)21:32
kfox1111sbezverk_: nice.21:32
sbezverk_kfox1111: but I had to use nested approach for now..21:32
kfox1111sbezverk_: yeah. shouln't be much different really from the fully nested, and the aproach I've been using.21:33
sbezverk_kfox1111: for vswitchd I will add entrypoint to wait for ovs socket21:33
*** Jeffrey4l has quit IRC21:33
kfox1111cool.21:33
kfox1111I think we should add a common lib thingy for that.21:34
kfox1111put a bare init-container in each object, then include the common_lib.init-containers in it.21:34
jascott1and push deps for init container through values?21:34
kfox1111yeah.21:34
jascott1thats what I have been assuming21:34
jascott1cool21:34
sbezverk_kfox1111: sounds good21:35
sdakeok so where in the code is an example of a config value for helm kfox1111 ?21:35
sdakekfox1111 so I know what to look for :)21:35
sdakenot the ceph workflow21:36
sdakebut the helm package21:36
kfox1111for the defaults, or the consumption of them?21:36
kfox1111and in default helm building, or the build system I came up with?21:36
jascott1there is nothing helm specific except the version of the packages and its deps21:36
sdakenot helm sepcific, package specific21:36
jascott1the values.yaml?21:37
kfox1111sdake: so, like heres an example of config values that are shared betwen all packages: https://review.openstack.org/#/c/396296/27/helm/src/common_values.yaml21:37
kfox1111heres some that are specific to l3agent:21:38
kfox1111https://review.openstack.org/#/c/396296/27/helm/src/neutron/values/l3_agent_daemonset.yaml21:38
sdakeis common_values.yaml a magic file?21:38
sdakefileame that is21:38
*** fguillot has quit IRC21:38
jascott1no21:38
jascott1values.yaml is tho21:38
sdakek21:38
jascott1magic meaning autmatically picked up?21:38
kfox1111build_packages.py combines the common and l3 agent ones together into the final values.yaml for the final l3agent package.21:38
kfox1111so helm just sees it as one unified set of values that a user can set.21:39
sdakeif I could make one request, it would be to see you one more time... :)21:39
kfox1111this was done so that we idn't have to manually maintain the common stuff in every packages values.yaml21:39
sdakeno really, what I'm sort of after is to keep the second example at a minimum21:39
*** fragatina has joined #openstack-kolla21:41
sdakefor example on the second example, line #1:21:42
sdaketype: network21:42
kfox1111yeah. what about it?21:42
sdakewhat else could type be there?21:42
kfox1111anything really.21:43
kfox1111the idea in that specific case,21:43
kfox1111is that you have different sets of hardware. such as network nodes, or compute nodes, or some other set of nodes.21:43
kfox1111you can override the selector_key/value variables to make that daemonset land on a subset of your nodes.21:44
kfox1111you then may need multiple helm package instances of that package to land multipe to cover your cluster with them.21:44
jascott1node selector then21:44
sdakeok i totaly didn't understand that21:45
sdakelet me use a different example :)21:45
sdakelogger_configmap_name: neutron-l3-agent-logger21:45
sdakewhy configmap name needs configuration?21:45
kfox1111cause you might want to do something different.21:45
kfox1111ok.. so...21:45
kfox1111back slightly to the previous exmapl, but more concrete..21:45
kfox1111multinode, we launch two instances of openvswitch-agent package.21:46
kfox1111with setteings type=network, and type=compute21:46
kfox1111by default, they share the same neutron-openvswitch-agent-logger configmap.21:46
kfox1111but. if you wanted, you could make a second configmap, neutron-openvswitch-agent-compute, and:21:47
kfox1111with setteings type=network, and type=compute,logger_configmap_name=neutron-openvswitch-agent-compute21:47
kfox1111and then they could be configurable seperately.21:47
sdakeok so sanity out of he box defaults21:47
sdakethis is for flexibility?21:48
kfox1111yeah. sane defaults, and customizability for those that need it.21:48
kfox1111yeah.21:48
sdakeok, well lets not abuse it plz :)21:48
kfox1111as little as possible. :)21:48
sdakebecause it makes life seriously hard to document config options21:48
kfox1111yeah.21:48
sdakethats another thign kolla decidedo n early on21:48
kfox1111+121:49
sdakevery few config options21:49
sdakealthough obviously things have bloated21:49
kfox1111thats the api I was taking about.21:49
sdakewhich I dont like but I understand21:49
jascott1I think its a clever solution but also worried about config complexity21:49
kfox1111packages have an api. which is the values they support.21:49
kfox1111the config complexity is mostly in the openstack config files themselves, and totally outside this piece of the architecture.21:49
sdakejascott1 that is enabled by placing configmaps outside the helm package itself21:50
jascott1i was talking about your 'type'21:51
kfox1111jascott1: ah. yeah. we need to minimize that sort of thing, but its critical to have that sort of thing too. :/21:51
jascott1its another thing to learn in order to understand system21:51
jascott1no doubt21:51
sdakeif it works out of the box, who cares :)21:51
kfox1111yeah. what we need maybe is a document or a few reference archetectures.21:51
jascott1true dat sir21:51
kfox1111or something.21:51
kfox1111like, "if you want to have a scaled out system, with one rabbit per service, heres an example of that..."21:52
sdakekfox1111 ya we went down that road during compose of documeitng all the config options21:52
sdakeit quickly gerw totally out of control21:52
jascott1would need a set of golden examples21:52
kfox1111helm install --name rabbit-neutron --set configname=neutron-rabbit,state=stateless21:52
jascott1but if it dosnt exist for a use case it might be hard on operator21:53
kfox1111I think there will probably be just a few common deployment types.21:53
kfox1111all in one. one controller + computes. ha controllers  2  compute host aggregates.21:53
kfox1111so shouldn't be too hard to document a few reference deployments.21:54
jascott1refresh me plz on current thoughts on upgrade? mostly operator based or are we thinking helm upgrade/rollback hooks etc will be involved?21:54
jascott1asking because we dont want to config ourselves into some corner before we understand that piece21:54
kfox1111jascott1: both. :/21:54
kfox1111some folks want one way, some another.21:55
kfox1111so parallel implementations of that part.21:55
kfox1111but not mixed.21:55
kfox1111so either operator based upgrades, or helm/entrypoint, but not both together.21:55
jascott1ok my real question was do we think helm upgrade can do the job21:55
jascott1ah21:55
sdakei struggle to see how helm can do upgrades correctly :)21:55
*** msimonin has quit IRC21:55
kfox1111I think helm upgrade can do minor, rolling upgrades of a subset of packages.21:55
kfox1111otherwise, I'm unconvinced.21:55
kfox1111I think there are 3 types of upgrades really.21:56
*** schwicht has quit IRC21:56
*** dave-mcc_ has joined #openstack-kolla21:56
kfox1111"zero downtime". ie, k8s deployment object types that can rolling upgrade minor version N to miner verion N.21:56
*** jtriley_ has joined #openstack-kolla21:56
kfox1111helm can do that.21:56
*** schwicht has joined #openstack-kolla21:56
sdakedo you mean x.y.z = upgrade z?21:57
sdaken is not semver ;)21:57
kfox1111"zero downtime major upgrade" this is the crown jewels... totally don't think helm can do it personally.21:57
kfox1111sdake: yeah. z upgrade I mean.21:57
kfox1111"downtime upgrade of just control plane". might be able to do with helm.21:57
kfox1111"downtime of control + data plane" (I lied, gues there are 4). helm can do it for sure. pretty undesirable though.21:58
kfox1111I was planning on writing up a spec or document with those spelled out and some thoughts on them.21:59
*** schwicht has quit IRC21:59
sdakeplease not another spec21:59
*** jtriley has quit IRC21:59
kfox1111doc then.21:59
sdakewfm21:59
*** dave-mcc_ has quit IRC21:59
jascott1helm has concept of release and subsets will either be in the single master release or there will be multiple releases at all times right?21:59
sbezverk_kfox1111: was there an agreement how to name entry point containers inside of pod?22:00
kfox1111jascott1: not sure... again, depends on if we're talking about just microservice pakages, or uberpackages for helm based deployjent/orchestration22:00
jascott1we were talking about this before but think we said we werent sure of granularity of upgrades22:00
kfox1111sbezverk_: the term inc0 came up with I think.22:01
sbezverk_kfox1111: vswitchd-entrypoint would do?22:01
*** jtriley_ has quit IRC22:01
kfox1111sbezverk_: "dependencies" ?22:01
sbezverk_oh that crap name that does not mean too much?!?!?!?22:01
kfox1111the "dependencies" init container.22:02
sbezverk_kfox1111: sounds great to me22:02
kfox1111then you can kubectl logs foo -c dependencies. :)22:02
jascott1so we have release 1.0.0 with everything. I want to jut uprade nuetron so I bump that sub chart and roll master release 1.0.1 and do upgrade <- this is how we are thinking?22:02
rhalliseydependency init container is the name22:02
kfox1111jascott1: for those wanting entrypoint. most of us are thinking operators though.22:02
kfox1111for operators, we just make a new openvswitch-agent package and upgrade only that.22:03
sbezverk_kfox1111: but even with operator you still need it to wait for depending service to come up no?22:03
kfox1111so no reason to involve all the rest of neutron if nothing changes there.22:03
kfox1111sbezverk_: yeah. in a few cases.22:03
sbezverk_like vswitchd waits for ovsdb socket22:03
jascott1makes sense for operator but even for entrypoint im concerned about what helm actually does with master release strategy22:03
kfox1111there's entrypoint for pre deps, and there's entrypoint for workflow.22:04
kfox1111s/pre/pure/22:04
kfox1111jascott1: agreed. honestly, I'm not convinced helm orchestration will wor kwell at all. but a bunch of folks do, so they can do it taht way if it works for them.22:04
sbezverk_I think entrypoint is excellent and lighweight for dependency check22:05
kfox1111I personally think operators are a better aproach to tackling the problem, at least for now.22:05
jascott1we are using entrypoint here to describe non-operator deployment22:05
kfox1111sbezverk_: it is. but it is used (abused?) by others to orchestrate things too, not just handing deps.22:05
kfox1111yeah.22:06
kfox1111I think it could work for a bunch of things and even full deployment,22:06
kfox1111if upgrades are not considered.22:06
sbezverk_kfox1111: it is theor problem, if somebody buys hammer and starts hammering his head, it is not a hammer problem22:06
jascott1lol22:06
kfox1111sbezverk_: right.22:06
kfox1111kubernetes-entrypoint is a great tool.22:07
jascott1sir id like to return this hammer, it is broken.22:07
kfox1111I'm not sure its a great tool for some of the jobs its currently being employed to tackle. :)22:07
*** Pavo has quit IRC22:07
jascott1agree22:07
kfox1111same with helm.22:07
kfox1111its a great tool.22:07
jascott1since its not a strategy per-se we sholuld prob call it non-operator22:08
kfox1111it might not be the best tool for solving every single openstack deployment task.22:08
jascott1casue operators will likely need to use entrypoint init conatiners (for jsut deps)22:08
kfox1111there are lots of non-operator strategies.22:08
kfox1111ansible/chef being others.22:08
kfox1111I'd be somewhat specific.22:08
kfox1111I've been calling it helm/entrypoint22:08
jascott1imean it sounds cool :)22:09
kfox1111as its orchetration by using just those two tools.22:09
*** Pavo has joined #openstack-kolla22:11
jascott1are we expecting people to abuse init conatiners in that model?22:12
kfox1111but some of what I've been trying to do with the helm build scripts, is to allow packages to be created for both workflow styles.22:12
kfox1111jascott1: absolutely. they don't see it as an abuse, but a prefered, good pattern.22:13
*** schwicht has joined #openstack-kolla22:13
kfox1111so, if it works for them, I'm ok with it.22:13
kfox1111I don't think it will work for me though.22:13
kfox1111but, they can prove me wrong. I'd be happy to be proven wrong. :)22:13
sbezverk_kfox1111: building packages is great for people who deploy, but I think for development, people should be able to deploy one specific service just using helm chart22:13
jascott1appreciate answering the hundred questions. trying to understand upgrade path and overlap between operators and helm/entrypoint22:13
kfox1111its simpler in a lot of ways.22:14
kfox1111sbezverk_: semi agree....22:14
kfox1111what you said is the helm/entrypoint orchestration strategy.22:14
kfox1111the other way is:22:14
kfox1111helm install kolla/neutron-operator22:15
kfox1111kubectl create -f neutron-config-for-service.yaml22:15
kfox1111then the neutron-operator code kicks in, and launches all the helm micropackages.22:15
jascott1is that create a TPR?22:16
kfox1111jascott1: yes.22:16
jascott1got it22:16
jascott1but to deliver works out of the box, wont we have to abuse init containers (or do someting else)?22:17
kfox1111there would be an overarching "compute-kit-operator" too.22:17
kfox1111helm install kolla/compute-kit-operator22:17
kfox1111kubectl create -f mycloud.yaml22:18
sbezverk_kfox1111: but that deploys all of nuetron things!! it is not what we tried to achive with jinja templates, we broke to microservice to achive the best granularity, I still do not get why we need to step back to inferior solution22:18
kfox1111which would trigger compute-kit operator to deploy neutron,glance,nova,etc.22:18
kfox1111sbezverk_: right. for microservices,22:18
kfox1111you can just deploy those directly with the microservice helm packages.22:18
kfox1111helm install kolla/neutron-l3-agent22:19
kfox1111that granularity isn't going away.22:19
sbezverk_kfox1111: but you need to build it, there is not way currently to deploy microservice from the source chart..22:19
kfox1111you don't deploy from source charts.22:20
kfox1111you deploy from built charts.22:20
kfox1111helm install only takes tgz's.22:20
kfox1111or entryies from a binary repo.22:20
kfox1111or am I wrong?22:21
sbezverk_kfox1111: hm, it is not what I see.. unless when I run helm install openvswitch it builds package under the hood22:21
kfox1111hmm... so helm can install from an uncompressed chart dir?22:21
kfox1111interesting....22:21
sbezverk_yep22:21
sbezverk_that is what I do22:21
kfox1111but I don't see a way for us to really do that. :/22:22
kfox1111we don't want to make a values.yaml and Chart.yaml for every microservice by hand, so its static22:22
kfox1111and keep it up to date.22:22
kfox1111so we have to build them.22:22
jascott1ya i have  been installing from local dir22:22
jascott1remove the tar if its there or it will use the tar over local tho22:22
jascott1brb22:23
kfox1111long term, my strategy is to have a build pipeline that builds kolla containers and helm microservice packages that point to them. bumping versions as minimally nessisary.22:24
kfox1111one binary released package update if any of the containers in the k8s object get updated.22:24
*** schwicht has quit IRC22:25
kfox1111that way, to minor upgrades forwards or backwoards, its just upgrading/downgrading helm packages at that level.22:26
*** schwicht has joined #openstack-kolla22:26
kfox1111and either a helm/entrypoint uberpackage or an orchestrator does the actual orchestration of that.22:26
*** schwicht has quit IRC22:29
*** schwicht has joined #openstack-kolla22:29
kfox1111so, I think we need to continue to build values.yaml and Chart.yaml for now. and then create a helm issue to see if they can make those files templatable.22:31
kfox1111then the build step can be skipped for those that want to try and skip it.22:31
*** adrian_otto has joined #openstack-kolla22:33
kfox1111I think this is kind of a helm infancy problem. 2.0 is really 1.0 in a way. very little in common with helm classic. so its bound to have some lacking features that it will get by 2.2.22:33
kfox1111adrian_otto: hi22:34
adrian_ottohi kfox111122:34
kfox1111adrian_otto: https://review.openstack.org/#/c/392257/22:35
adrian_ottocool, tx kfox111122:35
kfox1111I think we settled on the common bits being very granular, and supporting both operator or helm/entrypoint orchestration for those that want to go that route.22:35
kfox1111people who what operators can develop that code. those that want helm/entrypoint can there.22:36
kfox1111and a year or so from now, maybe there will be a clear winner, or both will show their own aproach is valid.22:36
kfox1111(or everyone's wrong and there will be anothe way. :)22:36
sbezverk_kfox1111: do you have an example handy how to pass dependency env parameter to entrypoint container?22:39
sbezverk_if I just do env/name/value do you think it should be the way they expect?22:40
kfox1111sbezverk_: just the examples here: https://github.com/stackanetes/kubernetes-entrypoint22:41
*** schwicht has quit IRC22:42
kfox1111DEPENDENCY_SERVICE=mariadb,keystone-api22:42
kfox1111DEPENDENCY_JOBS=nova-init,neutron-init22:42
kfox1111etc.22:42
sbezverk_kfox1111: I saw that one, I was not sure about syntax in init contianer..22:44
kfox1111what do you mean?22:44
kfox1111oh.22:44
kfox1111they are docker env vars.22:45
*** liyifeng has joined #openstack-kolla22:46
kfox1111sbezverk_: http://stackoverflow.com/questions/33478555/kubernetes-equivalent-of-env-file-in-docker22:49
kfox1111sbezverk_: the way I see it, there is not too much difference between requiring packages to be built for helm then for requring a build step for containers.22:52
kfox1111there isn't really a way to do a docker buildandrun . -it --rm . /bin/bash22:53
*** Pavo has quit IRC23:12
jascott1kfox1111 a missing piece for me wrt to helm/entrypoint strategy, what are we delivering for kolla-k8s?23:15
sdakejascott1 i dont follow23:22
*** Pavo has joined #openstack-kolla23:23
jascott1kfox1111 has been saying we have two strategies, operators and helm/entry-point23:23
jascott1Im trying to understand if/how we will abuse entry-point in kolla-k8s repo23:24
jascott1I like the idea of init containers being just being that and not abusing them23:24
jascott1abusing=making them do more than just wait for deps23:25
kfox1111jascott1: there are those that think kolla-kubernetes should not use operators but instead use helm/entrypoint.23:26
kfox1111if those folks want to contribute/maintain that code, I'm ok with it. :)23:27
kfox1111there are those that think thats not a good idea nad instead want to implement/use operators.23:27
kfox1111if those folks want to contribute/maintain that code, I'm ok with it. :)23:27
*** aric49 has quit IRC23:27
kfox1111my personal crystalball thinks the operator route is the more likely to produce success longterm. but thats my own personal opinion. I have been wrong before. :)23:28
*** Pavo has quit IRC23:29
kfox1111I won't push away those that think the helm/entrypoint way actually can be made to successfully fly. maybe it can.23:29
jascott1ah I guess I am looking for 'official goals' for ocata  for the kk8s release. is that the wrong way to think?23:29
kfox1111jascott1: its an opensource project. its powerd by those willing to write the code.23:29
kfox1111so, whatever they want to work on will be the stuff that gets done.23:30
kfox1111or whatever their managers do if they are paid to work on it.23:30
*** Jeffrey4l has joined #openstack-kolla23:31
jascott1for sure but thought we also had some collective agreed upon roadmap somewhere23:31
kfox1111I can only speak for my own motivations/managers goals. and they really want a solid base. so mostly the helm microservice package layer and below, and manual orchestration for now.23:32
kfox1111jascott1: the spec called for a layerd aproach so that everyone would have a place to contribute.23:32
kfox1111it layed out a plan for operators,23:33
kfox1111and had some details about entrypoint.23:33
sbezverk_Entrypoint WARNING: 2016/11/18 23:31:50 entrypoint.go:40: NAMESPACE env not set, using default23:34
sbezverk_Entrypoint INFO: 2016/11/18 23:31:50 entrypoint.go:61: Resolving /var/run/openvswitch/db.sock23:34
sbezverk_Entrypoint INFO: 2016/11/18 23:31:52 entrypoint.go:70: Dependency /var/run/openvswitch/db.sock is resolved23:34
sbezverk_it looks ok23:34
jascott1kfox1111 thanks23:34
kfox1111but the exact solution for helm/entrypoint will have to come from those that know how it works. I don't really understand how it could work at openstack scale.23:34
*** Pavo has joined #openstack-kolla23:34
kfox1111sbezverk_: probably need to set NAMESPACE to whatever helm is using. probably a var in the .Chart23:35
kfox1111sbezverk_: but your right, looks good. :)23:35
kfox1111sbezverk_: well, except maybe one thing...23:35
kfox1111does it check for the existance of the socket, or its operation?23:36
kfox1111it could still pass even if openvswitch wasnt running, depending on how it checks.23:36
kfox1111in which case, it may be better to keep the actual socket off of /dev/shm or something, so it disapears at reboot.23:36
sbezverk_kfox1111: good question, no idea, that is probably question to entrypoint developers23:37
kfox1111sbezverk_: i'm guessing its per service.23:38
sdakejascott1 ya - we dont have "official goals"  - people work on what they want to work on23:38
kfox1111nscd for example just ignores calls if the socket exists but the daemon isn't listening yet.23:38
kfox1111whre as most socket based things block until the server picks up.23:38
kfox1111so only the socket's existince is important.23:38
sdakedoes anyone know if Jeffrey4l fixed the ansible gate?23:39
sbezverk_kfox1111: who was the guy you submitted issue to for entrypoint?23:40
sbezverk_I think he could answer your concern.23:40
sdakeits not sufficient to check for the presence of the socket23:41
sdakethe socket must be inspected via its API to make sure things are rolling23:41
jascott1thats all k8s socket probes do afaik23:41
sdakeor badness ensues23:41
sdakewe suffered this with either mariadb or haproxy or rabbitmq23:41
sdakei forget which23:42
*** TxGirlGeek has quit IRC23:42
kfox1111sbezverk_: I just did to the issue tracker for it.23:42
sdakewe had to add alot of work to actually check the service was rolling before using it in bootstrap mode23:42
sdakebecause sockets get created before sockets provide service23:42
kfox1111sdake: yeah. depends on the service. if it opens the unix socket blocking or nonblocking.23:42
jascott1other projects like os-helm or stackanetes using either just socket or http for most probes and exec for the ones you just mentioned sdake23:42
kfox1111if blocking, all is well. it will wait for the server on the other end to launch.23:43
kfox1111nonblocking, things just break. :/23:43
kfox1111readyness probes are a good way to handle that too.23:43
sdakeprobably one workaround would be to change the socket from blocking to nonblocking after the service is up then23:43
sdakealthough that requires code changes in dependent software23:43
kfox1111make an exec readyness hook that calls mariadb client inside the mariadb server, and ensure its respoinsive.23:44
jascott1fyi a little survey of what others are doing https://gist.github.com/jascott1/a65b787af0750bafd161d47ea48b17dd23:44
kfox1111jascott1: that list is incomplete.23:44
jascott1for kk8s?23:45
jascott1yeah there is a common template right23:45
kfox1111yeah. most of our services use the following template: services/common/common-deployment.yml.j223:45
kfox1111yeah.23:45
sdakekfox1111 by incomplete do you mean the other 7 or 8 implementations of openstack on kubernetes? :)23:46
kfox1111sdake: no. just our list.23:46
kfox1111sdake: that just makes me sad. :/23:46
kfox1111thanks for that... :)23:46
sdakekfox1111 pretend like you didn't hear it23:47
sdakeI do that alot ;)23:47
sbezverk_do you guys can recommend any IDE for go lang for windows?23:47
kfox1111humans are really good at reinventing the wheel/not playing nice with each other.23:48
sdakesbezverk_ windows ide golang23:48
sdakeoxymoron? :)23:48
sdakekfox1111 there is no DSL for human beings :)23:48
sbezverk_sdake: go lang works on windows23:48
sbezverk_just in cli23:48
sdakesbezverk_ i was talking about all 3 together23:48
jascott1vim-go23:48
jascott1in fancy winbashhell23:48
kfox1111sdake: I know.. sucks. :/23:49
sdakekfox1111 actually it is what creates greatness in humans23:49
kfox1111sbezverk_: ubuntu for windows + vim? :)23:49
jascott1sbezverk_ if you like jetbrains stuff they have a nice golang plugin23:49
kfox1111sdake: yeah. mixed bag. :)23:49
sdakekfox1111 ya curse and blessing at same time23:49
kfox1111sdake: you read the dune series?23:49
sbezverk_jascott1: thanks it is much better option than others were suggesting ;-)23:50
sdakewhen i was a kid about 30 years ago23:50
sdakei dont remember it23:50
*** fragatin_ has joined #openstack-kolla23:50
kfox1111golden path man...23:50
kfox1111if you havent read the prequals, preprequals, and the final book, its worth it I think.23:50
kfox1111the stuff newer then that, not so much.23:51
sdakeya i remember that part23:51
kfox1111the final book explains the golden path finally.23:51
kfox1111but so much better when you have all the prequal material too.23:51
kfox1111ties it all together with a nice bow on top. :)23:51
jascott1sbezverk_ atom is not bad but kinda slow and alittle buggy23:53
*** fragatina has quit IRC23:53
jascott1tell me this, does Muadib take over or even live very long?23:55
kfox1111not very long.23:56
jascott1cool23:56
kfox1111his son though....23:56
*** schwicht has joined #openstack-kolla23:56
jascott1imean loved the character but was hoping it was epic scale story23:56
kfox1111yeah. it totally is epic.23:57
*** srwilker has joined #openstack-kolla23:57
*** sdake_ has joined #openstack-kolla23:57
jascott1i read one of Franks books in Dune universe about 25 years ago.. they were on a water planet.. no idea what its called now23:57
kfox1111the writer didn't get to totally finish it, but the son did, and did a fair job.23:57
kfox1111arakis23:57
kfox1111oh.23:57
jascott1i know that one from the movie :)23:57
rhalliseyno clue what dune is23:58
jascott1it may have been but they lived on these islands made of orgranic rock23:58
srwilkerOh man, Frank Herbert talk? What a conversation to join in23:58
kfox1111 I think. caladan23:58
kfox1111sdake: hehe.23:58
sbezverk_I still keep dvd version of old Dune movie :-)23:58
jascott1that they 'fed' waste to make them grow like coral23:58
kfox1111rhallisey: big scifi universe23:59
srwilkersbezverk_: I have it too23:59
jascott1I made myself wait to get 5.1 sound before watching it with the kids23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!