Sunday, 2017-01-08

*** sayantan_ has joined #openstack-kolla00:01
kfox1111sbezverk: oh. it looks like the default config.json copies the apache config's in place.00:01
kfox1111sbezverk: so you could probably just launch the keystone container with command /usr/sbin/httpd, and pvc at /var/www/html and it should serve out ok.00:02
sbezverkkfox1111: sorry not following00:03
kfox1111if you wanted to test a helm repo server in the gate right away,00:03
kfox1111instead of a helm-serv container, you could use the keystone container, override the container command to "/usr/sbin/httpd" and mount the helm repo pvc at /var/www/html.00:04
kfox1111you should get a working apache capable of sharing out helm packages.00:04
kfox1111should work with both the existing 2.0.2 container and 3.0.1 containers.00:04
kfox1111so you could get it workign in the gate today if you wanted to try that.00:04
sbezverkkfox1111: now I see what you mean..00:05
*** severion has joined #openstack-kolla00:05
kfox1111could even test it in the main gate by replacing the helm serv with it, and pointing the kolla repo to it.00:05
kfox1111would just need a bit of logic to copy the build packages into the pv.00:06
*** zhugaoxiao has quit IRC00:08
kfox1111sbezverk: did the project-config patch make sense to you?00:09
*** sdake_ has quit IRC00:10
sbezverkkfox1111: will check it in a sec, I am fixing your and sdake comments for the image PS00:11
kfox1111k00:11
sbezverkkfox1111: in projects.yaml helm-entrypoint does it any corresponding script or anything? or it is just a label for the job and we execute whatever we need?00:14
kfox1111projects.yaml?00:15
kfox1111oh.00:15
kfox1111it becomes an arg to the setup_gate script.00:15
kfox1111have a look in jenkins/jobs/kolla-kubernetes.yaml00:15
kfox1111you can see where it gets passed to the setup_gate script.00:16
kfox1111 tools/setup_gate.sh {action} {distro} {type} {config} {node} shell00:16
kfox1111arg 4.00:16
sbezverkI got the idea. can you paste me full url to jenkins/jobs/kolla-kubernetes.yaml00:17
kfox1111https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/kolla-kubernetes.yaml#L1700:18
sbezverkkfox1111: thank you I do not have this type of urls handy, they above my average ;-)00:18
kfox1111it took me a while on project-config.... kept forgetting it was in the openstack-infra namespace, not the openstack one like everything else.00:19
sbezverkkfox1111: so basically I just need to add new entry in this two files in opensatck-infra and then add a small PS branching got helm-operator parameter. right?00:21
kfox1111yup.00:21
kfox1111:)00:21
kfox1111pretty easy. :)00:21
sbezverkcool, let me do it then.. I like easy things for a change ;-)00:27
kfox1111hehe. I know, right? :)00:27
sbezverk:)00:29
portdirectsbezverk: just looked at helm container - cant see anything over and above kfox1111 and sdakes comments00:29
*** Pavo has quit IRC00:29
sbezverkportdirect: I have not pushed it yet I have now problem with right for /helm-repo folder :-(00:34
sbezverkdoing chmod and chown to see if it helps00:34
portdirectyeah - that's the downside of not being root :( will most likley need to set that up (chown) in an init container for pv00:36
portdirectI spent a few days running the docker daemon with user-namespaces turned on - but gave up in the end :(00:37
*** salv-orl_ has quit IRC00:38
sbezverkportdirect ahhh now I see why my chown is not working00:39
sbezverkit was run against local file system and real container uses external storage00:39
sbezverkso I will need sudo them to be able to prepare it properly00:40
portdirectif possible id try not to use sudo00:40
sbezverkportdirect: man what option then I have?!?!?!?00:41
portdirectand use a trick like this: https://github.com/openstack/kolla-kubernetes/blob/master/helm/kolla-common/templates/_common_create_keystone_service.yaml#L24-L25 in init container00:41
portdirectbut if thats not possible then sudo it is00:42
kfox1111sbezverk: init container with root.00:42
*** Pavo has joined #openstack-kolla00:42
sbezverkand you guys will be ok with this right?00:42
kfox1111then sudo isn't needed in the main container.00:42
sbezverkcool00:42
kfox1111the general idea is to keep things that are long running from running as root.00:43
kfox1111that way if a user breaks into the long running process they can't get to root.00:43
kfox1111in an init container though, its very transient, so much less risky.00:43
portdirectyeah ideally i'l like to get us to use:00:43
portdirecthttps://www.irccloud.com/pastebin/58bGtQ5K/00:44
portdirectbut know thats a long way off...00:44
kfox1111yeah. as much as possible.00:44
*** sdake has joined #openstack-kolla00:45
sbezverkportdirect: example you gave was for main container, do you have syntax for init container?00:54
portdirectyeah - just json of that 2sec and I'll knock one up00:54
sbezverkportdirect: it is freaking out on all my attempts :-)00:56
*** sdake has quit IRC00:56
sbezverkI thought this might work00:57
sbezverk              "securityContext": [00:57
sbezverk                  "runAsUser": "0"],00:57
sbezverkbut it does not like it00:57
portdirect{ not [00:57
portdirecthttps://www.irccloud.com/pastebin/HrqnFrQl/00:58
sbezverkah, ok let me try00:58
kfox1111theres somewhere in tree that's doing it.00:58
kfox1111helm/microservice/neutron-l3-agent/templates/l3_agent_daemonset.yaml does it.00:58
portdirectsbezverk: i find yaml much easier to parse so just just convert from that (using atom plugin) or http://codebeautify.org/yaml-to-json-xml-csv when thats not around, doing thats saved me so many headaches writing kube manifestss01:00
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Switch pv's to new global api  https://review.openstack.org/41763401:00
kfox1111portdirect: yeah. did that initally. then now that there's workign examples in tree, mostly just copy/paste from there.01:01
*** Pavo has quit IRC01:03
kfox1111really really want 1.6 to finish init containers so we can get away from json. :/01:03
sbezverkportdirect: I have to go with sudo, because when init container with root, then some helm related things are generated as root01:03
kfox1111well, json in yaml... which is worse.01:03
kfox1111sbezverk: just do two init containers.01:03
sbezverkand then main container gets into the same access issue01:03
kfox1111sbezverk: one to build the packages, then another to change it to owned as the right user?01:04
sbezverk%)01:04
sbezverkis not easier to run with root ;-) ??01:04
kfox1111security is not about easy. ;)01:04
kfox1111usually they are polar opposites. ;)01:04
kfox1111I guess this is brining up an interesting question... really should we be building packages in that container?01:05
kfox1111or just relying on the tarball transfer like thing?01:06
kfox1111do the build somewhere else?01:06
sbezverkkfox1111: I think adding packages into the container at baking is not great01:06
portdirectkfox1111: i dont think we should (off the top of my head) but not quite sure what the plan is01:07
kfox1111not sure its any worse then requireing a helm server in an init continer to build packages. :/01:07
*** msimonin1 has joined #openstack-kolla01:07
sbezverkkfox1111: we need to run helm package one way or another01:07
portdirectdeffo against adding them at container build01:07
sbezverkotherwise index will not get generated01:07
kfox1111portdirect: whys that?01:07
sbezverkI already hit it and confirmed with helm people01:07
kfox1111I think it would be benifitial in the same way as building a cache repo as well.01:08
kfox1111for containers.01:08
portdirectkfox1111: just speed of moving target - though perhaps I've not thought it through01:09
*** msimonin has quit IRC01:09
kfox1111portdirect: I'd like to make container updates much more common.01:10
portdirectyeah - getting master on a nightly is ther frist step on that path :)01:11
kfox1111I think we need to fix that pipeline.01:11
kfox1111yeah.01:11
kfox1111sbezverk: so, how about this for now,01:11
kfox1111run an init container as root,01:11
portdirectif you get me a script to do the other stuff we were talking about I may be able to devotte some bandwidth to that (again only gteed untill end of cycle)01:11
kfox1111git clone kolla-kubernetes,01:12
kfox1111add user build01:12
kfox1111switch to build and run prebuild/build microservices.01:12
kfox1111switch back to root to chown the data, then exit the init container.01:12
kfox1111then the apache container fires up and shares the data.01:12
kfox1111portdirect: yeah, thanks. was going to work on the script after we cut 0.4.001:13
sbezverkportdirect: which script01:13
sbezverk?01:13
sbezverkok it is working now with 2 init container01:14
kfox1111cool.01:14
openstackgerritSerguei Bezverkhi proposed openstack/kolla: Add helm-repository image  https://review.openstack.org/41742601:15
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Switch pv's to new global api  https://review.openstack.org/41763401:16
sbezverkkfox1111: portdirect: updated image with no root01:16
kfox1111looks good to me.01:18
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service  https://review.openstack.org/41747401:18
sbezverkkfox1111: now infra :-)01:19
kfox1111sbezverk: cool. it usually takes them a bout a day or so to merge things, unless you go begging. :)01:22
kfox1111I don't usually feel too bad asking for a quick review of a 2 line patch though. :)01:23
sbezverkkfox1111: I would think these guys must be the most paranoid since all opensatck infra is on them01:25
*** saneax is now known as saneax-_-|AFK01:26
kfox1111you'd think, but they are really good guys / usually very responsive.01:26
kfox1111they really want to be helpful.01:26
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Switch pv's to new global api  https://review.openstack.org/41763401:29
sbezverkkfox1111: Let's see ;-)01:30
sbezverkhttps://review.openstack.org/41763501:30
kfox1111+1 :)01:31
*** sdake has joined #openstack-kolla01:31
portdirectsbezverk: just saw your question: I'm pushing nightly mastersfor kolla images to dockehub with the stuff in this repo: https://github.com/portdirect/kolla-cron-builder, I'm happy to take prs to build z stream images there01:37
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Place holder for helm-operator gate job  https://review.openstack.org/41763601:37
sbezverkportdirect: got it01:38
kfox1111yeah. I'm going to try and get some kind of change detection process going so that it only uploads changed containers rather then always upload new ones.01:38
*** salv-orlando has joined #openstack-kolla01:38
*** lmasiero has joined #openstack-kolla01:41
*** salv-orlando has quit IRC01:43
*** sdake has quit IRC01:46
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Switch pv's to new global api  https://review.openstack.org/41763401:50
*** msimonin1 has quit IRC02:06
*** lmasiero has quit IRC02:07
*** lmasiero has joined #openstack-kolla02:08
*** lmasiero has quit IRC02:09
*** lmasiero has joined #openstack-kolla02:09
*** lmasiero is now known as lucasmasiero02:10
*** salv-orlando has joined #openstack-kolla02:40
*** salv-orlando has quit IRC02:44
openstackgerritSerguei Bezverkhi proposed openstack/kolla: Add helm-repository image  https://review.openstack.org/41742603:07
*** sacharya has joined #openstack-kolla03:08
*** sacharya has quit IRC03:12
*** sacharya has joined #openstack-kolla03:13
*** sacharya has quit IRC03:14
*** sacharya has joined #openstack-kolla03:15
*** zhubingbing has joined #openstack-kolla03:29
openstackgerritjianyi proposed openstack/kolla-ansible: Add zun ansible role  https://review.openstack.org/41617903:35
*** salv-orlando has joined #openstack-kolla03:41
*** salv-orlando has quit IRC03:45
*** bjolo has quit IRC04:13
*** zhubingbing has quit IRC04:35
*** salv-orlando has joined #openstack-kolla04:41
*** salv-orlando has quit IRC04:46
*** zhubingbing has joined #openstack-kolla05:01
*** prameswar has joined #openstack-kolla05:10
*** sdake has joined #openstack-kolla05:16
*** lucasmasiero has quit IRC05:18
*** salv-orlando has joined #openstack-kolla05:42
*** salv-orlando has quit IRC05:47
*** hieulq_ has joined #openstack-kolla05:53
*** sdake has quit IRC05:57
*** sayantan_ has quit IRC06:03
*** adrian_otto has joined #openstack-kolla06:08
*** sayantan_ has joined #openstack-kolla06:09
*** sdake has joined #openstack-kolla06:36
*** sacharya has quit IRC06:43
*** salv-orlando has joined #openstack-kolla06:43
*** sacharya has joined #openstack-kolla06:44
*** sacharya has quit IRC06:48
*** Jeffrey4l_ has joined #openstack-kolla06:49
*** salv-orlando has quit IRC06:50
*** adrian_otto has quit IRC06:52
*** adrian_otto has joined #openstack-kolla06:53
openstackgerritMerged openstack/kolla-ansible: all.yml should be in alphabetical order  https://review.openstack.org/41757307:12
*** hieulq_ has quit IRC07:17
*** hieulq_ has joined #openstack-kolla07:38
*** sacharya has joined #openstack-kolla07:44
*** salv-orlando has joined #openstack-kolla07:46
*** sacharya has quit IRC07:49
*** salv-orlando has quit IRC07:51
*** adrian_otto has quit IRC07:58
*** prameswar has quit IRC08:09
*** sayantan_ has quit IRC08:29
*** hieulq_ has quit IRC08:43
*** msimonin has joined #openstack-kolla08:47
*** salv-orlando has joined #openstack-kolla08:47
*** sdake has quit IRC08:48
*** salv-orlando has quit IRC08:51
*** severion has quit IRC09:03
*** msimonin has quit IRC09:10
*** zhubingbing has quit IRC09:10
*** msimonin has joined #openstack-kolla09:11
*** saneax-_-|AFK is now known as saneax09:30
*** msimonin has quit IRC09:32
*** msimonin has joined #openstack-kolla09:36
*** saneax is now known as saneax-_-|AFK09:41
*** sp__ has joined #openstack-kolla09:46
*** saneax-_-|AFK is now known as saneax09:49
*** sp__ has quit IRC10:01
*** msimonin1 has joined #openstack-kolla10:16
*** msimonin has quit IRC10:18
*** prameswar has joined #openstack-kolla10:22
*** msimonin1 has quit IRC10:25
*** msimonin has joined #openstack-kolla10:26
*** sp__ has joined #openstack-kolla10:27
*** pbourke has quit IRC10:43
*** pbourke has joined #openstack-kolla10:43
*** salv-orlando has joined #openstack-kolla10:49
*** salv-orlando has quit IRC10:53
*** saneax is now known as saneax-_-|AFK10:57
*** saneax-_-|AFK is now known as saneax11:08
*** prameswar has quit IRC11:38
openstackgerritSurya Prakash Singh proposed openstack/kolla: support py3 for kolla/image module  https://review.openstack.org/41481311:43
openstackgerritJeffrey Zhang proposed openstack/kolla: DO_NOT_MERGE: TEST MASTER BRANCH  https://review.openstack.org/39989711:54
*** zhubingbing has joined #openstack-kolla12:00
openstackgerritMerged openstack/kolla: Fix zaqar container  https://review.openstack.org/40775712:11
*** masuberu has joined #openstack-kolla12:29
*** zhubingbing has quit IRC12:32
*** zhubingbing has joined #openstack-kolla12:32
*** sp__ has quit IRC12:33
*** salv-orlando has joined #openstack-kolla12:50
*** salv-orlando has quit IRC12:55
openstackgerritSerguei Bezverkhi proposed openstack/kolla: Add helm-repository image  https://review.openstack.org/41742612:56
portdirectsbezverk: around?13:47
portdirectif so qq - you know the simplesw way fo doing string subsititution in goptl? need to escape some '$'s from a values file so i can load them into env-vars13:48
portdirectcould always pre-escape them in the values file but seems a bit ugly to force users to do that13:49
*** salv-orlando has joined #openstack-kolla13:52
*** salv-orlando has quit IRC13:56
*** masuberu has quit IRC13:56
*** sp__ has joined #openstack-kolla13:58
*** msimonin has quit IRC14:00
sbezverkportdirect: good morning14:01
portdirecto/14:01
*** mattmceuen has quit IRC14:01
sbezverkportdirect: looking at your question, nothing comes from the top of my head..14:02
openstackgerritzhubingbing proposed openstack/kolla-ansible: Add fluentd role  https://review.openstack.org/40739214:03
sbezverkportdirect: how about replace function?14:04
sbezverkJeffrey4l_: ping14:04
Jeffrey4l_sbezverk, pong14:04
sbezverkJeffrey4l_: appreciate if you could review my PS again, I need to get that image in so I use it at the gate14:06
Jeffrey4l_roger. np.14:06
sbezverkportdirect: something like that $name | replace "$" ""14:07
portdirectsbezverk: this is why i ask you questions :)14:07
portdirectof course! doh - totally forgot to check sprig docs14:07
*** saneax is now known as saneax-_-|AFK14:10
sbezverkportdirect: if you could ack this please : https://review.openstack.org/41763614:10
portdirectnp14:12
openstackgerritMerged openstack/kolla-kubernetes: Place holder for helm-operator gate job  https://review.openstack.org/41763614:15
Jeffrey4l_sbezverk, done. nit.14:21
*** sp__ has quit IRC14:33
openstackgerritSerguei Bezverkhi proposed openstack/kolla: Add helm-repository image  https://review.openstack.org/41742614:38
*** sp_ has quit IRC14:39
sbezverkJeffrey4l_: thanks for review, I fixed last comment as well.14:41
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: [wip] enable ceilometer related project in gate  https://review.openstack.org/41227814:42
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: [wip] enable ceilometer related project in gate  https://review.openstack.org/41227814:51
*** salv-orlando has joined #openstack-kolla14:52
*** sp_ has joined #openstack-kolla14:53
*** lucasmasiero has joined #openstack-kolla14:55
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service  https://review.openstack.org/41747414:56
*** salv-orlando has quit IRC14:57
Jeffrey4l_cool. +2ed15:00
openstackgerritzhubingbing proposed openstack/kolla-ansible: Add fluentd role  https://review.openstack.org/40739215:01
sbezverkJeffrey4l_: thank you, appreciated. Need to fine another core for wfl ;-)15:07
Jeffrey4l_yep ;)15:08
*** msimonin has joined #openstack-kolla15:09
*** sdake has joined #openstack-kolla15:14
*** sdake_ has joined #openstack-kolla15:16
*** sdake has quit IRC15:19
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: [wip] enable ceilometer related project in gate  https://review.openstack.org/41227815:23
*** dave-mccowan has joined #openstack-kolla15:25
kfox1111morning15:44
kfox1111Jeffrey4l_: any idea on 3.0.2?15:44
sbezverkkfox1111: good morning15:44
kfox1111we still have that pip install workaround that we'd like to get rid of and wednesday's quickly aproaching.15:44
*** sayantan_ has joined #openstack-kolla15:52
*** sdake has joined #openstack-kolla15:59
*** sdake_ has quit IRC16:00
sbezverkkfox1111: do you have a second?16:01
Jeffrey4l_kfox1111, it is pushed. please check https://review.openstack.org/41744216:05
*** lucasmasiero has quit IRC16:05
kfox1111sbezverk: yeah.16:05
sbezverkkfox1111: need to discuss with you one thing..16:06
kfox1111Jeffrey4l_: cool. when does it get to the hub? onnce it finishes review?16:06
kfox1111sbezverk: sure. whats up?16:08
sbezverkkfox1111: when operator creates service related object I want to assign to each object created by this specific operator unique label. It will helm in future to operator easily discover its objects without mixing up with other object created by other operators.. what do you think?16:08
Jeffrey4l_kfox1111, no. we need push this manually.16:09
kfox1111yeah.16:09
kfox1111I think sticking that in the release name would help?16:09
kfox1111Jeffrey4l_: when wil lthat happen?16:09
sbezverkif you are ok with this approach then I need lablel: operator_id assigned to every object16:09
Jeffrey4l_kfox1111, should be soon after it is merged.16:10
kfox1111I dont think helm has a way to label a release16:10
kfox1111Jeffrey4l_: ok. thanks. I'll help push on reviewers then. :)16:10
Jeffrey4l_thanks a lot.16:10
sbezverkkfox1111: I am going to find out and if it is not I will request it16:10
kfox1111if we could get it done by tuesday, then we can get kolla-kubernetes updated in time for release on wednesday.16:10
sbezverkmost of operators I study use this flexible model16:10
kfox1111sbezverk: yeah. I think we'll be the first going through helm rather then direct to k8s. so might be an oversite in helm.16:11
openstackgerritzhubingbing proposed openstack/kolla-ansible: Add fluentd role  https://review.openstack.org/40739216:11
kfox1111they've been very good to help us with missing features though, so hopefully they will be able to do something for us quickly.16:12
sbezverkkfox1111: cool if we are in agreement at least general, then I explore this idea with helm folks16:12
kfox1111+116:12
kfox1111I think their release names get mapped into a configmap.16:12
kfox1111so they may be able to easily just tack on a few labels onto it.16:12
sbezverkkfox1111: idealy if they can let me pass label_name:label_value, that would be great16:13
sbezverkI can then in operator, generate unique label and assign it to all objects instantiated by this operator16:14
kfox1111its a cli thing, right? so maybe follow the k8s cli of foo=bar?16:14
kfox1111yeah.16:14
sbezverkyep cli of helm client16:14
kfox1111helm list -v foo=bar or something liek that.16:15
sbezverkexactly16:15
kfox1111yeah. sounds good. :)16:15
kfox1111would be very useful outside of just operators too.16:16
sbezverkkfox1111Ь exactly, even in kubectl you can specify a label for selecting  objects16:16
*** lucasmasiero has joined #openstack-kolla16:19
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Convert statefulset to global api  https://review.openstack.org/41769816:19
*** lucasmasiero has quit IRC16:24
kfox1111sbezverk: all_values has kolla_compute for both iscsid and tgtd16:25
kfox1111doesn't one of those go on the controller by default?16:25
Mech422anyone know a good trick for flattening nested dicts to make ansible happy?16:25
Mech422the stupid thing really should be able to iterate nested dicts :-(16:26
*** zhubingbing has quit IRC16:26
sbezverkkfox1111: tgtd goes where vg group is hosted and cinder-volume runs16:27
sbezverkiscsid goes to the compute node16:27
kfox1111k. I'll chane that one to kolla_controller then.16:27
*** Pavo has joined #openstack-kolla16:27
*** Pavo has quit IRC16:31
*** Pavo has joined #openstack-kolla16:33
*** salv-orlando has joined #openstack-kolla16:35
sbezverkkfox1111: here is helm feature request, please leave your feedback https://github.com/kubernetes/helm/issues/180016:46
kfox1111sbezverk: thanks. commented.16:51
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Convert statefulset to global api  https://review.openstack.org/41769816:55
kfox1111about 80 down, about 20 to go.16:58
kfox1111almost there.16:58
sbezverkguys let's not overdesign things ;-) https://s-media-cache-ak0.pinimg.com/736x/71/8f/38/718f380d398f01074352315960c6e77e.jpg16:59
kfox1111hehe17:00
kfox1111I don't think the pug was ever 'intellegently designed' by anyone. ;)17:00
sbezverk:)17:00
kfox1111but I'm partial to other breeds. :)17:00
kfox1111I'd also call it more, artificial selection rather then intellegently designed.17:07
kfox1111intellegently designed woudl be purposfully tweaking dna to have the things desired.17:08
kfox1111chair dogs from dune perhaps. :)17:08
sbezverkkfox1111: selection is just one of methods of design ;-)17:08
kfox1111fair.17:08
kfox1111Jeffrey4l_: I didn't realize, that review's for the release team?17:10
kfox1111if so, we may have a problem. I think they release on wednesdays.17:10
kfox1111which would overlap with the kolla-kubernetes release.17:11
openstackgerritMerged openstack/kolla: Add helm-repository image  https://review.openstack.org/41742617:11
*** msimonin has quit IRC17:12
*** msimonin has joined #openstack-kolla17:12
*** msimonin has quit IRC17:12
SamYaplekfox1111: does this interest you? http://paste.openstack.org/show/594239/17:16
SamYaplespecial-neutron-base took "real    1m55.521s" to build17:17
SamYaplethats from base distro (debian:jessie) to image tagged17:17
kfox1111nice.17:17
kfox1111looks big though?17:18
kfox1111https://hub.docker.com/r/kolla/ubuntu-binary-neutron-base/tags/17:18
kfox1111or is that a source vs binary thing?17:18
SamYaplethats a source v binary problem17:18
kfox1111https://hub.docker.com/r/kolla/ubuntu-source-neutron-base/tags/17:18
kfox1111did trunk bloat up?17:19
SamYapleyou do see those say "compressed size" right?17:19
*** bmace has quit IRC17:19
kfox1111ah.17:19
kfox1111ok. nm then. :)17:19
*** bmace has joined #openstack-kolla17:19
SamYaplebinary is tinyer in general, but compressed it should be almost identical now17:19
kfox1111so, your two are uncompressed size?17:19
SamYapleor even source pulls ahead17:19
SamYapleyea17:19
kfox1111nice. :)17:19
SamYaplebest part is...... it can build on dockerhub! with triggers!17:20
kfox1111cool. :)17:20
*** msimonin has joined #openstack-kolla17:20
SamYaplei havent even started to optimize anything. I bet i could get it down another 10-20%17:22
kfox1111nice. :)17:23
SamYapleat this point though, that size is mostly needed binaries, no cruft17:23
kfox1111have you compared it to portdirect's alpine based stuff for size?17:24
kfox1111curious how much tighter alpine ends up being.17:25
portdirecthttps://microbadger.com/images/kolla/ubuntu-source-neutron-server17:25
SamYaplejsut from what i know about alpine, i think (assuming it even _has_ all the packages) we could get a 200MB neutron image17:25
portdirecthttps://microbadger.com/images/port/openstack-neutron-api17:26
portdirect(again those are both compressed)17:26
SamYapledang portdirect. i crush those sizes17:26
SamYapleoh nvm17:26
SamYaplethe first one was still kolla offical17:26
SamYaplehmmm portdirect. you seem to be missing some deps17:27
portdirect'docker.io/port/openstack-neutron-api                            latest              2c0b705b4bc5        7 weeks ago         263.5 MB'17:28
portdirectwhich ones?17:28
SamYaple"special-neutron-base                                       latest              52d16e93c5ec        2 minutes ago       337 MB17:28
SamYapleso not too far off17:29
SamYapleand i install a bunch of deps like openvswitch and keepalived17:29
*** haplo37_ has quit IRC17:29
SamYaplei guess you might not need those for neutron-api?17:29
portdirectyeah - I dont have/need them17:29
kfox1111api shouldn't need those, yeah. can be in a seperate package.17:29
*** salv-orlando has quit IRC17:29
SamYapleyou say that but im pretty sure the api still needs some stuff like iptables nad ipset17:30
SamYaplebut this is just my memory talking17:30
portdirectnot with ovn17:30
kfox1111hmm... sinse we started the helm conversion, the ubuntu gate's not been pure ubuntu. :/17:30
SamYapleeither way the debian build i have and the alpine build are very close in size17:30
SamYapleif you consider the unneeded deps17:31
kfox1111some of the packages aren't tagged with a distro, so defaulting to centos.17:31
portdirectyeah - you gonna share the source anytime soon? :D17:31
kfox1111SamYaple: interesting. :)17:31
SamYapleportdirect: momentairly17:31
SamYapleportdirect: im going to dazzle you with showing you a 100% dockerhub build at that size (and speed!) first17:32
portdirectI can get alpine down by about another 50-60mb - but not worth it for me17:32
portdirectand the stament you make above confirms that even more17:32
*** haplo37_ has joined #openstack-kolla17:32
SamYaplei think i can get ubuntu down to 200MB uncompressed without much ado17:33
SamYapleone line rm at the end of the dockerfile17:33
SamYapleill tell you what though, once python3 is solid, this drops to maybe 100-150MB uncompressed17:33
portdirectI have the feeling this is gonna be a one line neutron sevre install :)17:33
SamYapleportdirect: it is. but with a twist. its a 2 minute build time with no build-tools!17:34
portdirecti think I may have an idea whats going on - 2 stage build?17:34
* portdirect thinking furiously17:35
SamYaplei have a repo server build, it builds all openstack/requirements files. takes ~10m. Does not need to be run repeatedly (it can be reused). can trgger off changes to openstack/requirements17:35
portdirectthought about removing all the .py files but then realised that was a rabbit hole too far17:36
SamYaplespecial-neutron-base is running docker inside the build process and starting that container and purging all the data, using only what it needs17:36
SamYapleportdirect: youve got it backwards, you remove all the .pyc17:36
SamYaple.pyc arent super compatible17:36
SamYaplespecial-neutron-base-no-deps   latest              9d91ce6e7f34        About a minute ago   314.4 MB17:36
portdirecttrue - but i told you it was a rabbit hole to far - removing pyc is far to easy :)17:37
SamYaplelol17:37
SamYapleanyway. so im going to finish this out in dockerhub and point you too it. im very excited to do this all 100% in dockerhub. with triggers!17:37
*** igordcard has quit IRC17:38
portdirectdo you can talk to docker api in dockerhub? or should i just wait to see what you've got :)17:38
*** igordcard has joined #openstack-kolla17:38
SamYapleportdirect: dude. its better. this is an automated build in dockerhub. like i just point it at a repo!17:39
SamYaplei never build or push anything17:39
Pavowait what?17:39
*** bmace has quit IRC17:39
SamYapleyep17:39
Pavoyou can build dockerfiles in docker hub now?17:39
portdirect'special-neutron-base is running docker inside the build process' this is the bit I'm having a mental block at17:39
*** bmace has joined #openstack-kolla17:39
portdirectso docker hub let you dind?17:39
SamYaplePavo: for a long time17:40
SamYaplehttps://hub.docker.com/r/samyaple/rabbitmq-clusterer/17:40
SamYaplethats an auto build ^17:40
kfox1111the hub gives you acess to the underlying docker?17:40
kfox1111or are you nesting your own docker?17:40
SamYaplenah but you can dind17:40
SamYaplenesting17:40
kfox1111interesting.17:40
SamYapleim still working out a kink thats messign me up though17:40
kfox1111how fragile do you think that will be?17:40
SamYapleso its not 100% sure yet17:40
kfox1111cause it may work today but not tomorrow?17:40
SamYaplenot very fragile in my experince17:41
kfox1111cool. :)17:41
SamYaplethe hub does go down, but i would be more worried about github going down17:41
SamYapleno github, no build17:41
portdirectcrazy that dockerhub lets you do privileged builds - thats a new one on me :)17:41
SamYapleportdirect: you can do unpriv dind17:41
Pavoyeah thats very interesting17:41
kfox1111no, was more thinking that the nested docker might break at some point.17:41
portdirectoh - 2 new ones on me then :)17:41
Pavoseems like that could become very exploitable17:41
SamYapleyou guys know im only using docker to pull the image right?17:42
SamYapleeven if you cant do dind, i can still pull the image17:43
SamYapleim just trying to stay with the tooling17:43
portdirectlets lay off you untill its up :)17:43
SamYaplethat would be better :P17:44
*** lamt has quit IRC17:48
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Convert statefulset to global api & fix gate workflows  https://review.openstack.org/41769817:50
kfox1111portdirect: have two more reviews ready:17:57
kfox1111https://review.openstack.org/#/c/417234/17:57
kfox1111and https://review.openstack.org/#/c/417627/17:57
kfox1111when you get a chance please.17:57
*** sdake has quit IRC17:59
*** lamt has joined #openstack-kolla18:02
*** msimonin has quit IRC18:04
openstackgerritMerged openstack/kolla-kubernetes: Switch keystone jobs to global api  https://review.openstack.org/41723418:05
openstackgerritMerged openstack/kolla-kubernetes: Convert create-db/manage-db scripts to global api.  https://review.openstack.org/41762718:05
sbezverkkfox1111: I built small script to run it as a operator in a container running in the cluster and using relm-repo. It worked :-)18:05
sbezverkhttp://paste.openstack.org/show/594241/18:05
openstackgerritMerged openstack/kolla-kubernetes: Switch pv's to new global api  https://review.openstack.org/41763418:06
portdirectSamYaple: 'docker.io/port/openstack-neutron-alpine                         no-cinder-deps      d47d2298fc22        2 minutes ago       212.2 MB' this is will the OVN and LBaaS plugins installed, and no fancy tricks - love to see if we could get it down further18:07
kfox1111portdirect: thanks.18:08
kfox1111sbezverk: awesome. :)18:08
sbezverkkfox1111: I need python expert to code it properly when I told sdake that I want to do it in go, he said no go ;-)18:09
kfox1111yeah, the tc needs to make a decision to support go.18:10
kfox1111I don't see that happening any time soon. :/18:10
sbezverkkfox1111: anyway if I do not find anybody, I will have to brush up on python18:11
kfox1111yeah.18:11
kfox1111its a good language to know though. :)18:12
SamYapleportdirect: is the base image for that openstack-base or pure alpine?18:12
portdirectfrom my repo? openstack-base18:13
SamYapleits 212.2MB ontop of openstack-base?18:13
portdirectyes18:14
SamYapleoh boy. then i may be able to beat the total size. I am 'FROM debian:jessie'18:14
SamYapleonce you see what i do, it can be adapted to alpine if its decent18:15
portdirectalpine -> port/aplpine -> port/openstack-base -> port/openstack-base-common-deps -> port/openstack-neutron-alpine18:15
SamYapledebian:jessie -> special-neutron-base18:15
portdirectyeah - I could optimise a lot more18:16
SamYapleyou should give it a shot18:16
portdirecti think we should be able to get the delta from base pretty much identical - so in theroy alpins hould be smaller - but I'm not sure if its worth it at that point18:17
*** msimonin has joined #openstack-kolla18:17
portdirectif you are doing that on a distro with proper CVE reporting etc thats much more useful18:18
*** msimonin has left #openstack-kolla18:18
SamYapleits more useful if people are willing to eat the single layer approach18:19
SamYapleim going to push the repo build to show how i do it18:19
SamYaplefor that part18:19
portdirectyeah - they are i think, the main critisem i hear about kolla is the number of layers18:19
SamYaplewell then do you think they would like a 1-layer approach? because thats what I have!18:20
portdirectthe only reason layers are good is they make building easier & quicker if building all images18:20
portdirectyou are 2 layer i think? base and delta?18:21
portdirect(but either way yes)18:21
SamYaplemore or less, yea18:23
SamYapleif you add in ENV stuff its like 4 layers. but whose counting18:23
portdirect(I dont count them either)18:23
portdirecttarballs count - metadata is (almost) free18:24
SamYapleportdirect: https://github.com/SamYaple/openstack-requirements18:26
SamYapleso that gets built, and at the very end we make a layer containing _only_ the built tarballs18:26
SamYaplein the neutron image we pull that layer somehow (with docker official tooling, or direct with curl)18:27
SamYaplethen we don't need build tools _and_ everyone shares teh same package versions18:27
SamYaplesame builds as well18:27
*** sp_ has quit IRC18:29
*** salv-orlando has joined #openstack-kolla18:30
*** markmcclain has quit IRC18:30
*** markmcclain has joined #openstack-kolla18:31
portdirectnice - taht makes sense - it thought about a similar approach using venv - but thats much better18:31
*** sbezverk has quit IRC18:34
*** salv-orlando has quit IRC18:35
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Convert statefulset to global api & fix gate workflows  https://review.openstack.org/41769818:35
*** sbezverk has joined #openstack-kolla18:41
*** sp_ has joined #openstack-kolla18:42
*** dave-mccowan has quit IRC18:46
*** msimonin has joined #openstack-kolla18:53
*** msimonin has quit IRC18:54
*** msimonin has joined #openstack-kolla18:54
*** haplo37 has quit IRC18:54
*** g3ek has quit IRC18:54
*** msimonin has quit IRC18:59
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Convert statefulset to global api & fix gate workflows  https://review.openstack.org/41769819:00
*** pcaruana has quit IRC19:03
*** g3ek has joined #openstack-kolla19:04
*** haplo37 has joined #openstack-kolla19:04
*** matrohon has joined #openstack-kolla19:11
*** bmace has quit IRC19:13
*** msimonin has joined #openstack-kolla19:15
*** msimonin has quit IRC19:17
*** msimonin has joined #openstack-kolla19:20
*** matrohon has quit IRC19:30
*** salv-orlando has joined #openstack-kolla19:31
*** msimonin has quit IRC19:32
*** salv-orlando has quit IRC19:35
*** msimonin has joined #openstack-kolla19:39
*** sayantan_ has quit IRC20:16
*** sacharya has joined #openstack-kolla20:26
*** salv-orlando has joined #openstack-kolla20:32
*** salv-orlando has quit IRC20:36
*** sdake has joined #openstack-kolla20:40
*** sayantan_ has joined #openstack-kolla20:48
*** sayantan_ has quit IRC20:53
*** msimonin has quit IRC20:58
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service  https://review.openstack.org/41747421:13
*** msimonin has joined #openstack-kolla21:19
*** sp_ has quit IRC21:19
*** sp_ has joined #openstack-kolla21:33
*** salv-orlando has joined #openstack-kolla21:33
*** Jeffrey4l_ has quit IRC21:35
*** sacharya has quit IRC21:35
*** sacharya has joined #openstack-kolla21:36
*** sdake has quit IRC21:36
*** salv-orlando has quit IRC21:37
*** sacharya has quit IRC21:40
*** sdake has joined #openstack-kolla21:40
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service  https://review.openstack.org/41747421:46
*** salv-orlando has joined #openstack-kolla21:47
*** Jeffrey4l_ has joined #openstack-kolla21:47
*** sdake has quit IRC21:48
*** severion has joined #openstack-kolla21:52
*** v1k0d3n has quit IRC21:54
*** Jeffrey4l_ has quit IRC21:57
*** Jeffrey4l_ has joined #openstack-kolla21:58
*** sp_ has quit IRC22:01
*** msimonin has quit IRC22:02
*** msimonin has joined #openstack-kolla22:10
*** sp_ has joined #openstack-kolla22:15
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service  https://review.openstack.org/41747422:17
*** sacharya has joined #openstack-kolla22:18
*** salv-orl_ has joined #openstack-kolla22:22
*** salv-orlando has quit IRC22:22
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service  https://review.openstack.org/41747422:52
*** jascott1_ has joined #openstack-kolla22:58
*** jascott1_ has quit IRC23:01
*** jrobinson has joined #openstack-kolla23:11
*** msimonin has quit IRC23:24
*** msimonin has joined #openstack-kolla23:24
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Helm repository container and service  https://review.openstack.org/41747423:28
*** msimonin has quit IRC23:29
*** salv-orlando has joined #openstack-kolla23:48
*** salv-orl_ has quit IRC23:48

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!