Sunday, 2016-12-11

openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40826300:06
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm for nova-compute  https://review.openstack.org/40948200:11
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40826300:13
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934900:14
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40826300:16
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40826300:17
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm for nova-libvirt  https://review.openstack.org/40948100:24
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934900:27
*** sayantan_ has quit IRC00:29
*** Pavo has quit IRC00:29
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40826300:34
openstackgerritcaoyuan proposed openstack/kolla: Fix some typo  https://review.openstack.org/40788400:34
*** Pavo has joined #openstack-kolla00:34
openstackgerritcaoyuan proposed openstack/kolla: Fix some typo  https://review.openstack.org/40788400:36
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40826300:36
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40826300:39
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40826300:54
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934900:55
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40826300:56
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm for installing other nova services  https://review.openstack.org/40948600:56
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services  https://review.openstack.org/40948700:59
sdakehey peeps01:02
sdakemgiles are you unblocked for dev work?01:03
mgilesYep.  Had some weird errors on the gate for my ps so have been breaking it down into smaller chunks to isolate the problem01:05
mgilesbut not blocked01:05
*** sdake_ has joined #openstack-kolla01:06
sdake_mgiles sounds good01:06
sdake_mgiles note some of the gate providers (specificailly rax.org and rax.iad) are particularly flakey with our gate scripts01:07
sdake_mgiles the way you tell is at the top of the gate output01:07
sdake_mgiles if you see a gate problem there, you can always type recheck in the log and the infrastructure will automaticallly retest the code01:07
*** mgiles_ has joined #openstack-kolla01:08
sdake_mgiles_ did you get all that :)01:08
mgiles_yes, but not sure what you mean by "type recheck in the log" sdake01:08
mgiles_sdake In any case, I don't htink that was the problem here, but good to know for future reference01:09
*** sdake has quit IRC01:09
sdake_mgiles_ cool - so recheck you know how when you do a review01:11
sdake_mgiles_ and you type some commentary and press "post"01:11
sdake_instead type "recheck" and press post01:11
mgiles_sdake yep01:11
mgiles_sdake ok.  didn't know that.  Thanks01:11
sdake_it will rerrun the gate01:11
sdake_cool its handy01:11
sdake_because gate is a little flakey01:12
sdake_gate suffers from heisenfailures01:12
sdake_i think we could fix rax.iad and rax.org if we could cosnisstently get jobs to run on them os we could sort out what is wrong01:12
mgiles_sdake_  Yep, I have seen other weird errors that I can't attibute to anything, so think I may have hit flakiness in some cases too01:13
sdake_infra has no way of specifying where a job runs01:13
kfox1111finally think I've got a fix to the gate now.01:13
sdake_so we can't isolate - we have to get lucky01:13
sdake_and there are something like 9 cloud providers01:13
sdake_so getting lucky is hard :)01:13
kfox1111sdake_: I've not been seeing as much flakyness on the kubernetes side on rax01:15
kfox1111so some difference between the ansible stuff and kubernetes stuff.01:16
kfox1111I did have to debug a lot in the kubernetes gate though, so maybe fixed an issue that plangues the rax side. the networking there is a bit goofy.01:16
kfox1111well. different at least.01:16
sdake_kfox1111 that is good news :)01:16
kfox1111I kept tryingt o correlate the curent resolver problem to one cloud or another. couldn't do it.01:17
kfox1111seemed to show up on all of them.01:17
sdake_kfox1111 sweet if you ahve some ideas to pass of th jeffrey4l any ideas01:17
sdake_kfox1111 it would be sweet if we could get our gates rock solid on kolla-ansible too01:17
kfox1111yeah. might some day have time for that.01:18
kfox1111I've been trying to pass around as much gate knowlege as  Ican01:19
kfox1111so hopefully I'm not the only one soon either.01:19
srwilkersjust curious, where can i find more information on the gates kfox1111? other than digging through the console logs?01:20
sdake_Jeffrey4l_ you about?01:20
kfox1111the logs are most of it.01:20
kfox1111the zuul page helps a bit too.01:21
kfox1111http://status.openstack.org/zuul/01:21
kfox1111can see whats running, and watch jobs live.01:21
sdake_zuul is the new jenkins01:21
kfox1111sdake_: I think I understand how to artefact stuff in the gate now, aftger reading trove's gates.01:22
sdake_every co investing heavily in it instead of jenkins01:22
kfox1111so we might be able to cache all sorts of things.01:22
srwilkersthanks kfox111101:22
kfox1111I ned to try it.01:22
kfox1111srwilkers: anything else you want to know, just ask. :)01:22
kfox1111the more folks that understand it, the better off we'll all be. :)01:23
sdake_kfox1111 not a fan of caching wrt gating - but feel free to do what needs to be done :)01:23
sdake_kfox1111 ^^++01:23
kfox1111sdake_: the caching will be about making the tests more granular. so we can catch breakages much qicker.01:23
sdake_yup understanding how the gate works is a bit complicated, hwoever, it has alot of value01:23
kfox1111or catch them in one place without affecting progress on another piece.01:24
sdake_kfox1111 the key with the gates is they should always start from a empty state01:24
sdake_caching introduces a non-empty state01:24
kfox1111sdake_: no, not nessisarily. the other altiernitive is starting from a known good state.01:24
sdake_how do you prove known good state ;)01:24
kfox1111you test it. :)01:25
kfox1111you have tests that test each other. :)01:25
sdake_right in the gate, which means the testing in the gate needs to be absolutelly perfect01:25
kfox1111for example.01:25
kfox1111you test out kubernetes 1.4.8. and cache it.01:25
sdake_kolla-kubernetes too young to make the gating absolutely perfect01:25
kfox1111then you use that known good copy.01:26
sdake_5% battery power - if i drop off01:26
kfox1111you then test kolla-kubernetes trunk against taht.01:26
kfox1111once you have a working gate for kolla-kubernetes,01:26
kfox1111you test kubernetes 1.4.9 against it.01:26
kfox1111then cache 1.4.9.01:26
kfox1111then you know when your checking reviews for kolla-kubernetes, that random kubernetes upgrades wont interfier with your testing.01:27
sdake_cache 1.4.9 of kubernetes images?01:27
sdake_rather tarballs?01:27
kfox1111yeah. cause its not just a tarball.01:27
kfox1111its docker containers too.01:27
kfox1111that it downloads.01:27
kfox1111and rpms.01:27
kfox1111or debs.01:28
sdake_ok - well my suggestion would be not to be too rocket scientisty on the gate early on :)01:28
kfox1111we cache all the deps, so thjat when we review code, we know that the deps have been pretested.01:28
kfox1111yup.01:28
kfox1111baby steps.01:28
kfox1111thats why I haven't implemneted it yet.01:28
sdake_because I didn't understand half of what you just said - admiditaly up  against a lcock on my laptop power to explore your ideas01:28
kfox1111working on getting the current one completely stable first.01:28
sdake_indeed we do cache our requirements01:29
kfox1111we cache some of or requirements. like for pip.01:29
sdake_or most of them anyway01:29
sdake_right01:29
kfox1111bujt kolla-kubernetes has a lot of other deps.01:29
kfox1111the kolla containers. kubernetes. ceph. etc.01:29
kfox1111kolla-ansible.01:29
sdake_if it works for pip it should work for others01:29
sdake_and it does work for pip with the occasional blip that infra quickly fixes01:30
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934901:30
kfox1111it would be nice if everyone would agree on a same caching mechanism.01:30
kfox1111but deb/ubuntu/pip/docker all have to do things differently. :/01:30
sdake_yup - i struggle to get caching of deb/rpms into infra01:31
sdake_however inc0 has taken over that work01:31
kfox1111cool. if he needs help.01:31
kfox1111I cache rpms at least all the time.01:31
sdake_jeffrey4l is helping him i think01:31
sdake_Jeffrey4l_ that is01:31
kfox1111cool.01:31
sdake_kfox1111 we don't reallycall it cachcing in openstack its called mirroring01:32
sdake_but its the same thing imo :)01:32
kfox1111yeah. same thing. :)01:32
*** zhubingbing has joined #openstack-kolla01:33
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm creation of Nova services  https://review.openstack.org/40947301:34
*** sdake_ has quit IRC01:37
sbezverk_sdake: ping01:48
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: More dns gate fixes  https://review.openstack.org/40826301:48
sbezverk_kfox1111: I see you went really wild with gatr testing!!01:50
sbezverk_gate01:50
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934901:51
portdirectsbezverk_: I've asked on slack but though there's a chance you might know:01:55
portdirectis it possible to get the CIDR that the CNI network uses for pods via the API?01:56
mgiles_kfox1111 seems you were right about the services being the culprit for my issues.  That's the only ps that failed the gates.  It's getting late here so I'm not up for digging into this further now, but will look tomorrow (maybe) or Monday to see if I can figure out what's going on.01:57
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40948801:57
mgiles_kfox1111 Thanks for the help01:57
kfox1111mgiles_: sure. :)01:58
kfox1111sbezverk_: yup. finally got the gate to block until dns is ready properly. we should really get that merged asap.01:58
kfox1111sbezverk_: also found a case where ubuntu external dns never worked.01:58
kfox1111though I don't think we actually used it in the gate until the workaround in the neutron entrypoint patch.01:59
portdirectmgiles_: dumping the endpoints from kubectl in the gate may be worth a shot while debugging - its helped me there - as its very easy to get the labeling slightly off.01:59
kfox1111portdirect: yeah. had to do do that enough, I made the gate log it.02:00
mgiles_portdirect Can you give me more details about what you did to get that?02:00
portdirect'kubectl get --namespace kolla endpoints'02:00
kfox1111mgiles_: or if in the gate, look under logs for svc*02:01
portdirect^^ nice :)02:01
mgiles_kfox1111  Cool, thanks02:01
mgiles_and portdirect ^^^  thanks02:01
portdirectnp, catch you on the flipside mgiles_ :)02:02
mgiles_good night02:03
*** mgiles_ has quit IRC02:03
sbezverk_kfox1111: dso you have a link for that PS to ack?02:04
kfox1111sbezverk_: yeah. sec02:04
openstackgerritzhubingbing proposed openstack/kolla-ansible: add panko role  https://review.openstack.org/40012202:04
portdirectsbezverk_: kfox1111 https://review.openstack.org/#/c/408263/?02:05
kfox1111sbezverk_: https://review.openstack.org/#/c/408263/02:05
kfox1111hehe.02:05
portdirecti'm watching you :)02:05
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934902:06
*** n0isyn0i1e has joined #openstack-kolla02:06
kfox1111portdirect: :)02:06
sbezverk_kfox1111: will wait for jenkins and then ack. that is one hacking looking thing ;-)02:08
portdirectkfox1111: thats a big hammer with the dns0 bridge :)02:08
kfox1111+1 for waiting. :)02:08
kfox1111portdirect: yeah. worked with infra on that one. :/02:08
kfox1111portdirect: they did something special in their ubuntu image to add a dns cache. :/02:08
portdirectgreat /s02:09
sbezverk_kfox1111: while we are waiting want to ask you guys. What would be the best wait to kill the pod on configmap change? I mean I can do just using kubectl but maybe you can suggest a more elegant way?02:09
*** n0isyn0ise has quit IRC02:10
sbezverk_portdirect: mgiles_:    ^^^02:10
portdirectwhere are you killing it from?02:10
sbezverk_portdirect: from a side car container that just detected a configmap change02:10
kfox1111sbezverk_: I'm a bit biased there I think. I'm not convinced it should be done that way at all, as your circomventing kube's management.02:11
sbezverk_I have all pieces working, just need to select a weapon for suicide02:11
kfox1111but I can think of two ways.02:11
kfox1111either kubectl delete the pod.02:11
sbezverk_kfox1111: how about livelness? we already use it :-)02:11
kfox1111or kubernetes labels the pods uniquely.02:11
kfox1111you can use the corrisponding docker labels to kill it.02:11
kfox1111hmm...02:12
portdirectkfox1111: doesnt that mean bind mounting the docker socket?02:12
kfox1111I guess, lets aproach it a little differntly.02:12
kfox1111do you want to kill the pod, or kill the container?02:12
portdirectwould need to be the whole pod02:12
sbezverk_kfox1111: I thing killing pod would be better02:12
portdirecti think livleyness would be the only secure way02:12
portdirectbut would create lot of an overhead02:13
kfox1111hmm.. a liveleness with exec hook might do the trick.02:13
sbezverk_portdirect: I agree I just do not see yet how I could get my logic attach there02:13
portdirectkubectl would be nicer but would require an access token with the ability to kill pods02:13
kfox1111but yeah, it would be a polling thing.02:13
kfox1111so would either take up resources, or be slow to notice.02:13
portdirectsbezverk_: I'm not convinced that a sidecar is the way to go there, but would love to be proved wrong02:14
sbezverk_kfox1111: with rolling upgrades you are ready wait for days ;-) so what is wrong with 10 seconds delay02:15
kfox1111it kind of viloates the orchestration seperation I'm concerned about in the same way entrypoint does.02:15
kfox1111I'm ok with it optionally though. just like optional entrypoint.02:15
kfox1111sbezverk_: its an issue of who triggers things.02:15
kfox1111and what is affected when.02:15
portdirectwould this not actually be better handled by an update? if you need config change in 10 sec the manual intervention or (dare i say it) a k8s-operator may be best?02:16
kfox1111in a deployment way, its triggered by me upgrading a deployment.02:16
sbezverk_kfox1111: I look at it from different angle, kube dns uses something similar why not it can also be used in a bit different flavor02:16
kfox1111sbezverk_: how does kube dns do something similar?02:16
sbezverk_kfox1111: there is a side container health check that checks dns and if it is not up after a certain time it restarts it02:17
sbezverk_same thing does tiller02:17
kfox1111thats just a health check though. tis not used to trigger a reconfigure.02:17
sbezverk_it is reaction on Event02:17
sbezverk_it is just a different event ;-)02:18
portdirectyeah - but two different classes of events02:18
kfox1111I totally believe the side car apoaach is good for that kind of thing. just not sure about configure.02:18
kfox1111here's the main difference.02:18
kfox1111in docker, they have a phlosophy.02:18
*** bachp has joined #openstack-kolla02:18
kfox1111delete/replace so containers are atomic.02:18
kfox1111kubernetes extends that phylosophy to atomic replication controllers.02:18
kfox1111all the pods in an rc should be identicle.02:19
kfox1111deployment extends that to have two atomic sets, and old and a new.02:19
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934902:19
kfox1111but if you update the configmap under an rc, then you can get pods in one rc that are in two different states,02:19
kfox1111breaking the atomicness of the rc.02:19
portdirectI'm in total agreement with kfox1111 here02:20
kfox1111depending on when a pod was launched in the set, or crached/recovered, it may be a type A pod , or a type B pod.02:20
kfox1111temporally.02:20
kfox1111the sidecar triggering rebuild is based on a phylosophy that is different then atomic rc's.02:21
sbezverk_kfox1111: hmm, since pods referenced the same configmap I do not see how they could be differnet02:21
kfox1111that may be fine. but I'm kind of fond of the phylosophy and would like to stick to it as much as reasonable.02:21
kfox1111sbezverk_: thats a potential issue that needs fixing.02:21
kfox1111with the current implementation.02:21
kfox1111so when configmaps are loaded in, they get a version.02:22
kfox1111any time its changed, it gets bumped.02:22
sbezverk_kfox1111: right and I am not saying it has to stay forever, but there is no good solution for anything but the deployment atm..02:22
kfox1111microservices point directly at a specific version when instantiated, and get updated to trigger a pod rebuild on configmap change.02:22
sbezverk_so if you can guarantee that kube community will address it BEFORE we need reconfiugration then fine..02:23
kfox1111sbezverk_: yeah. for now.02:23
kfox1111I fully expect them to get a deployment like feature for deamonsets/petsets at some point.02:23
kfox1111but, even doing it suboptomally, at the k8s object level still,02:23
kfox1111its eqiv functionality to doing it on each node with a sidecar.02:23
portdirectsbezverk_: this is why i think an external soltion would be better - it would lower the hoverhead and allow us to perform a depolyment like rolling upgrade - distrubuting this sort of controll will be very difficult to manage02:23
kfox1111kubectl delete daemonset, kubectl add daemonset has the same behavior.02:24
kfox1111all pods get deleted, and get created again.02:25
portdirect(I'm making assuptions here)02:25
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm creation of Nova services  https://review.openstack.org/40947302:25
sbezverk_portdirect: well, I get your point, but relying on external solution yo upretty much kill dynamic nature of kubernetes I think.02:25
kfox1111its using k8s controlelrs to do the control.02:25
kfox1111which I think is native k8s.02:25
portdirectwould it not be possible to thake what you have, and change it to kill the pods it targetted - rather than the pod it was in?02:26
kfox1111rather then add a new controller on the nodes.02:26
portdirect(yeah by external i ment to the pod, not to k8s)02:26
kfox1111portdirect: it could be done with shared unix sockets, and some other logic. but woudl be really ugly. :/02:26
portdirectOMG no02:27
kfox1111sbezverk_: there's a intermediate option too...02:27
kfox1111not sidecar, but operator.02:27
kfox1111load up a watcher for configmap changes, then it deletes all pods that the configmap touches.02:27
kfox1111it doesn't have to be on the same host as the pod.02:27
kfox1111in fact, thats desirable in some ways.02:27
portdirect^^ this is waht i was trying to describe02:27
kfox1111less of an attach surface.02:27
sbezverk_then you need to discover those pods to kill, where in this case the biggest damage is you just kill yurself02:27
kfox1111the opoerator should be lableled differently then the daemonset, so shouldn't be generally possible for it to kil itself.02:28
kfox1111unless it has a very bad bug. :)02:28
portdirect+102:28
sbezverk_kfox1111: no I was refering to side car comparing to operator02:28
kfox1111sidecar can still kilil the wrong pod if buggy. :/02:29
kfox1111if it talks to k8s to kill it.02:29
*** Pavo has quit IRC02:29
kfox1111could still kill other pods by mistake, at least on the same host, if uses docker to kill.02:29
kfox1111thats why I like using k8s itself to do that orchestration.02:29
kfox1111the rc controlls all the pods. so having the rc update trigger the pod delete is the cleanest.02:30
kfox1111as the rc is reposible for them all. single source of truth.02:30
portdirectWe should not give pods access to the docker api under any curcumstances. apart from anything it would require us to have a solution for rkt or whatever else comes along.02:30
kfox1111portdirect: true.02:31
sbezverk_portdirect: I was more inclined to use just kubectl02:31
sbezverk_to kill, but I guess will have to check operators02:32
kfox1111kubectl would be relatively clean, but gives a sidecar the power to kil things that are not itself.02:32
kfox1111has to talk to a more central service adding potential scalability issues,02:32
kfox1111and distributes credentials around.02:32
sbezverk_kfox1111: its pod where it runs nothing more02:32
kfox1111how do you restrict a kubectl account to just the pod that owns it?02:33
portdirectsbezverk_: how do you limit it it that?02:33
kfox1111I don't think thats currently possible. :/02:33
portdirectyou can limit access to read only, and by namespace only afaik?02:33
*** Jeffrey4l_ has quit IRC02:33
kfox1111portdirect: you can go a bit finer grained then that.02:34
kfox1111but you'd hjave to make a service acoount per rc pod.02:34
*** Pavo has joined #openstack-kolla02:34
kfox1111I don't think an rc can create a service account, and an rback rule that binds ito the currently launching pod.02:34
sbezverk_ok, I will loot into liveness hooks, cause in this case there should not be any security concernss02:34
kfox1111yeah. liveliness hook sounds like maybe the safest aporach there.02:34
kfox1111woul still need a read only kube account to pull for new configmaps.02:35
sbezverk_it is in case of a operator02:36
sbezverk_but for the side car this issue does not exist02:36
kfox1111how would you detect configmap changes without a k8s account?02:36
portdirecthow - it needs to poll the api to see if a new map exisits?02:36
sbezverk_easy, version number02:37
kfox1111portdirect: oh. I think the configmap stuff has a symlink and periodically updates itself.02:37
sbezverk_when I replace configmap it get new version02:37
kfox1111thats the ..data thing.02:37
kfox1111you could watch the symlinks.02:37
portdirectand that propogates out to the pod?02:37
sbezverk_do you want me to post what I have so far?02:37
sbezverk_the only thing is missing if final shoot at the head02:38
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for Glance PV/PVC  https://review.openstack.org/40949102:38
portdirectyeah - it would give us something to acuta;lly discuss constrctivly :)02:38
sbezverk_cool02:38
kfox1111yeah. sounds good.02:39
kfox1111sbezverk_: https://review.openstack.org/#/c/408263/ green. :)02:40
sbezverk_I will add as a file to tests/ in kolla-kubernetes02:40
kfox1111sbezverk_: k. sounds good.02:40
kfox1111portdirect: heh. thanks for the review. :)02:42
portdirectfelt I had to comment02:42
* kfox1111 chuck.es02:42
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: DO_NOT_MERGE PoC Configmap update monitor  https://review.openstack.org/40949302:45
*** duonghq has joined #openstack-kolla02:45
sbezverk_kfox1111: done, hopefully the gate will behave itself better02:45
kfox1111sbezverk_: thanks.02:45
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40948802:46
openstackgerritMerged openstack/kolla-kubernetes: More dns gate fixes  https://review.openstack.org/40826302:47
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40948802:47
sbezverk_portdirect: thank you for constructive comment :-)02:51
sbezverk_need to get back to family life for now.. have a good evening/night or some other time of day ;-)02:52
portdirectlaters sbezverk_ :)02:52
*** duonghq has quit IRC02:56
kfox1111sbezverk_: have a good one. :)02:58
*** Jeffrey4l has joined #openstack-kolla03:01
srwilkersnoticed a problem with the common_manage_db_job template, particularly here: https://github.com/openstack/kolla-kubernetes/blob/master/helm/kolla-common/templates/_common_manage_db_job.yaml#L3103:04
portdirectsrwilkers: needs to be service name?03:05
srwilkersproblem im having is that .resourceName is used for the name in metadata and used again for the name of the service configmap03:05
srwilkersyep portdirect03:05
portdirecton it03:05
srwilkersgiving me problems with cinder, since the name should be cinder, but the configmap name needs to be cinder-api03:05
srwilkerswell, cinder-manage-db, but i think you know what i mean03:05
*** v1k0d3n has quit IRC03:06
*** v1k0d3n has joined #openstack-kolla03:06
*** Jeffrey4l has quit IRC03:09
openstackgerritPete Birley proposed openstack/kolla-kubernetes: Fix configmap mapping in _common_manage_db_job.yaml  https://review.openstack.org/40949503:09
*** v1k0d3n has quit IRC03:09
*** v1k0d3n has joined #openstack-kolla03:10
portdirectsry - my bad03:11
srwilkersnothing to be sorry for man, it happens :)03:13
srwilkerstrying to figure out why the cinder stuffs not making it through the gate. slowly figuring things out bit by bit03:13
*** v1k0d3n has quit IRC03:18
*** v1k0d3n has joined #openstack-kolla03:27
*** sdake has joined #openstack-kolla03:29
*** williamcaban has quit IRC03:33
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934903:44
*** fragatina has joined #openstack-kolla03:45
*** fragatina has quit IRC03:53
*** fragatina has joined #openstack-kolla03:54
*** v1k0d3n has quit IRC03:58
*** v1k0d3n has joined #openstack-kolla03:59
*** sayantan_ has joined #openstack-kolla04:02
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services  https://review.openstack.org/40948704:02
*** williamcaban has joined #openstack-kolla04:04
*** v1k0d3n has quit IRC04:06
*** dave-mccowan has joined #openstack-kolla04:09
*** dave-mcc_ has joined #openstack-kolla04:13
*** dave-mccowan has quit IRC04:15
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934904:26
*** Pavo has quit IRC04:29
*** Pavo has joined #openstack-kolla04:34
*** dave-mcc_ has quit IRC04:36
*** diogogmt has quit IRC04:38
*** mgiles has quit IRC04:40
*** sdake has quit IRC04:45
*** portdirect is now known as portdirect_zzzzz04:46
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services  https://review.openstack.org/40948704:49
sbezverk_srwilkers: I saw you started to work on glance as well, do you plan to do it all?04:51
srwilkersyeah, trying to break it into chunks to make it more manageable for reviews. have services in a commit, another commit for PV/PVC, etc04:52
srwilkershoping to tie the rest of it up in the morning -- wife is wanting to watch a movie :)04:52
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: DO_NOT_MERGE PoC Configmap update monitor  https://review.openstack.org/40949304:53
srwilkerssbezverk_, could you possibly take a look at https://review.openstack.org/#/c/409495/ ? its making things weird for cinder right now04:54
sbezverk_srwilkers: ok cool. let me know if you want to split it for optimization as I need glance for some experiments in very near future..04:54
sbezverk_srwilkers: sure04:54
srwilkerssbezverk_, please dont let me hold you back. if you need the other parts done quickly, im not going to tell you to not finish the others04:54
srwilkerslike i mentioned, my plan was to finish it tomorrow. if you want to get it done this evening, im not in a position to tell you no :)04:55
sbezverk_srwilkers: if you finish them tomorrow that would be super.. I am not goint to do anything tonight04:56
sbezverk_anyway04:56
sbezverk_srwilkers: so the cinder issue is with config map name?04:57
srwilkersyeah. if you look at the common job here https://github.com/openstack/kolla-kubernetes/blob/master/helm/kolla-common/templates/_common_manage_db_job.yaml#L3104:58
srwilkersthe job should be named "cinder-manage-db", but the configmaps name is "cinder-api"04:58
srwilkerswhen it uses .resourceName for both, it ends up with "cinder-api-manage-db"/"cinder-api" or "cinder-manage-db"/"cinder"04:59
srwilkersheres the old cinder bootstrap job to manage the db for reference: https://github.com/openstack/kolla-kubernetes/blob/master/services/cinder/cinder-bootstrap-job-manage-db.yml.j205:02
*** srwilkers is now known as srwilkers_Zzzz05:07
*** fragatina has quit IRC05:31
*** fragatina has joined #openstack-kolla05:32
*** sayantan_ has quit IRC05:38
sbezverk_srwilkers: yeah, it would be better if common_manage_db was just dealing with passed variable and then  you the tempalte where you call it, you can assign value you want. In this case it will be more flexible.05:43
*** fragatina has quit IRC06:20
*** sayantan_ has joined #openstack-kolla06:28
*** Pavo has quit IRC06:29
*** fragatina has joined #openstack-kolla06:32
*** Pavo has joined #openstack-kolla06:34
SamYaplew00t07:09
SamYaplethats AIO mariadb,rabbitmq,keystone,glance,memcached deployed.07:09
SamYaplestill working on endpoitn and service creation07:09
SamYaplethis kolla-salt thing is going much faster than i thought07:10
*** sayantan_ has quit IRC07:28
*** fragatina has quit IRC07:32
*** b_bezak has joined #openstack-kolla08:02
*** b_bezak has quit IRC08:11
*** Pavo has quit IRC08:29
*** Pavo has joined #openstack-kolla08:34
*** sdake has joined #openstack-kolla08:53
*** sdake has quit IRC08:59
*** sdake has joined #openstack-kolla09:00
*** msimonin has joined #openstack-kolla09:05
*** msimonin has quit IRC09:09
*** sdake_ has joined #openstack-kolla09:33
*** sdake has quit IRC09:34
*** sdake_ has quit IRC09:41
*** sdake has joined #openstack-kolla09:44
*** jascott1_ has quit IRC10:07
*** sdake has quit IRC10:09
*** duonghq has joined #openstack-kolla10:26
*** duonghq_ has joined #openstack-kolla10:26
*** duonghq_ has quit IRC10:28
*** Pavo has quit IRC10:29
*** msimonin has joined #openstack-kolla10:33
*** Pavo has joined #openstack-kolla10:34
*** saneax-_-|AFK is now known as saneax10:44
*** saneax is now known as saneax-_-|AFK10:54
*** esmiurium_ has joined #openstack-kolla10:59
*** esmiurium has quit IRC11:03
*** jascott1_ has joined #openstack-kolla11:08
*** msimonin has quit IRC11:12
*** jascott1_ has quit IRC11:12
openstackgerritJeffrey Zhang proposed openstack/kolla: Install qemu-kvm rather then qemu-system-x86  https://review.openstack.org/40951311:15
*** zhubingbing has quit IRC11:15
*** duonghq has quit IRC11:20
openstackgerritWei Cao proposed openstack/kolla-ansible: Add freezer ansible role  https://review.openstack.org/40222412:18
*** rsjethani has joined #openstack-kolla12:20
rsjethaniHi all, I am relatively new to openstack deployment and i am confused between kolla and openstack-ansible12:21
rsjethanican anyone guide me12:21
rsjethanithanks12:21
rsjethaniso far I have some experience with devstack12:22
SamYaplersjethani: what would you like to know? i have pretty extensive experince with both12:22
rsjethaniSamyaple: I want to understand the difference between the two. So they have same purpose?12:23
*** caowei_ has joined #openstack-kolla12:24
SamYaplebroadly speaking, yes12:24
SamYapledeploy openstack in containers12:24
rsjethaniSamYaple: then how to decide which one to choose12:25
SamYaplei would suggest trying them both if you are unsure, but we are clearly biased here on #openstack-kolla toward kolla12:25
SamYaplekolla is significantly faster than openstack-ansible12:25
rsjethaniBut using OSA I think we can have VM based deployments also?12:27
rsjethaninot just container based...12:27
SamYaplenot natively. no. and thats not tested either12:27
rsjethanioh. ok12:27
SamYapletechnically openstack-ansible can be deployed on "metal", on a baremetal host or a vm12:27
SamYaplebut it is only gated and really tested against their lxc implementation12:28
SamYapleif you need baremetal deployment, give it a try. please let me know if you can get it to work!12:28
*** Pavo has quit IRC12:29
SamYapleopenstack-ansible in containers is the only thing that we can compare with kolla12:29
rsjethaniI have some code contribution in various core projects here and there hence so far I have used devstack for single node and multi node test environments12:30
SamYapleif you plan to do development for openstack, and are already familiar with devstack, you might apprecaite openstack-ansible more than kolla12:30
SamYapleKolla is a better production deployment tool, it uses imaging which allows for much better testing before moving to production12:31
rsjethaniI am quite interested in understanding simple to complex deployment12:31
SamYaplei would suggest you give them both a try then12:31
rsjethaniI have some knowledge of ansible as well as docker12:31
SamYaplethey both support all in one deploys12:31
SamYapleand kolla-ansible uses ansible use ilke openstack-ansible12:31
*** jascott1_ has joined #openstack-kolla12:31
SamYaplethe openstack ansible channel is also quite active and very helpful12:32
rsjethaniI am actually looking for some replacement of devstack because many a time I have to shutdown devstack vm then upon restart I have to run stack.sh agin which takes quite sometime12:34
*** Pavo has joined #openstack-kolla12:34
SamYapleboth have thier advantages rsjethani, i think you should try them both12:35
SamYapleand ask opinions in the #openstack-ansible channel as well12:35
SamYapleobviously we prefer kolla, and they prefer openstack-ansible :)12:35
rsjethanihaha right12:35
*** jascott1_ has quit IRC12:37
rsjethaniOne last question SamYaple, right now I am looking for simple lab environments where I should also be able to make changes easily, add remove services..and I should able to reboot my deployment without thinking too much. Can kolla give me what I want. Thank a lot12:39
SamYaplersjethani: of course. though I think openstack-ansible might be able to as well12:40
SamYaplereally anything other than devstack :)12:40
rsjethanithen why the dev community is still using devstack...that much12:40
SamYaplersjethani: i honestly don't know. I just don't12:41
SamYaplei used it once and refused to use it again12:41
rsjethaniok buddy thanks for the info12:41
SamYapleanytime :) make sure to hit up #openstack-ansible12:41
rsjethaniya sure :)12:42
openstackgerritWei Cao proposed openstack/kolla-ansible: Change Dockerfile to ansible role in the release note.  https://review.openstack.org/40951712:46
*** caowei_ has quit IRC12:50
*** caowei_ has joined #openstack-kolla12:51
openstackgerritWei Cao proposed openstack/kolla-ansible: Delete add-panko-dockerfile release note.  https://review.openstack.org/40952112:56
*** n0isyn0i1e has quit IRC13:02
*** zhubingbing has joined #openstack-kolla13:10
*** msimonin has joined #openstack-kolla13:11
*** msimonin has quit IRC13:14
*** msimonin1 has joined #openstack-kolla13:14
*** msimonin1 has left #openstack-kolla13:14
*** timss has quit IRC13:17
*** jascott1_ has joined #openstack-kolla13:34
*** timss has joined #openstack-kolla13:34
*** jascott1_ has quit IRC13:38
*** caowei_ has quit IRC13:59
*** awiddersheim has quit IRC14:03
openstackgerritMerged openstack/kolla-ansible: Enable watcher dashboard when enable_watcher is yes  https://review.openstack.org/40660814:22
*** v1k0d3n has joined #openstack-kolla14:22
*** caowei_ has joined #openstack-kolla14:22
*** Pavo has quit IRC14:29
*** Pavo has joined #openstack-kolla14:34
*** williamcaban has quit IRC14:38
*** williamcaban has joined #openstack-kolla14:38
*** wanghongxu_ has joined #openstack-kolla14:47
*** srwilkers_Zzzz is now known as srwilkers14:52
srwilkersgood morning14:52
sbezverk_srwilkers: good morning14:57
srwilkershey sbezverk_ :)14:58
kfox1111morning.15:00
*** yingjun has joined #openstack-kolla15:02
SamYaplemorning15:03
srwilkersmorning kfox1111 SamYaple15:04
*** caowei_ has quit IRC15:05
*** duonghq has joined #openstack-kolla15:06
duonghqevening guys15:08
*** sdake has joined #openstack-kolla15:09
kfox1111duonghq: evening. :)15:10
duonghqanybody try halcyon w/ Arch linux?15:10
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934915:11
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for Glance PV/PVC  https://review.openstack.org/40949115:13
*** portdirect_ has joined #openstack-kolla15:16
*** portdirect_ is now known as portdirect15:16
portdirecto/15:16
duonghqsup portdirect15:17
*** wanghongxu_ has quit IRC15:17
portdirectduonghq: dont think halcyon-kubernetes will work with arch at the moment as kubeadm is not avaible for that platform?15:18
duonghqportdirect, I'm trying15:18
portdirectduonghq: but halcyon-vagrant-kubernetes would be fine15:18
duonghqkubeadm shouldn't problem,15:18
*** wanghongxu_ has joined #openstack-kolla15:19
portdirectduonghq: oh (https://aur.archlinux.org/packages/kubeadm-git/) nice!, then with some tweaks to the ansible it should be fine :)15:20
*** awiddersheim has joined #openstack-kolla15:20
duonghqportdirect, I think arch Linux as host will be tricky to setup but I can15:21
openstackgerritprameswar proposed openstack/kolla: bifrost guide improved  https://review.openstack.org/40617715:21
sbezverk_kfox1111: I think we need to change common_svc a bit..  for the case when service's name and the name of svc object do not match.15:22
portdirectbeen fighting getting kubeadm running on fedora/centos atomic - as it expects to be able to write to some areas that we have mounted read only, but there are some patches coming in that I hope will fix that, getting it to support as many platforms as feasibly possible would be great, so we can ensure maxium portability :)15:23
kfox1111sbezverk_: I'm good with that. maybe we just make it another .var  and then the calling template can map it to whatever it needs to be.15:23
sbezverk_we can add suffix var and then service name will always be = element_name, but in metadata's name can be element_name + suffix.15:23
kfox1111portdirect: +1 for more support. does atomic have its own way of deploying k8s though?15:24
*** wanghongxu_ has quit IRC15:24
kfox1111cause kubeadm is cool and all, but if they have a more native way, we should support that too.15:24
*** wanghongxu_ has joined #openstack-kolla15:24
portdirectin the context of halycon i think that that should stick to kubeadm (though it not my project)15:25
kfox1111sbezverk_: or we add something like .svc_name or something, and then in the parent, it can define it to be .element_name, or printf "%s-svc" .element_name or whatever makes sense to the parent?15:25
kfox1111portdirect: oh. true. was speaking more generally.15:26
portdirectI've actually been doing a wee bit of work on the atmic kube deplyment tool (though I've not had time recently): https://github.com/projectatomic/commissaire15:26
portdirectIt also supports openshift, and I plan to add kolla-k8s deplyment as well15:26
kfox1111cool.15:27
srwilkerssbezverk_, i actually just threw a PS together for that very reason15:27
portdirectthough thats much further down the line :)15:27
sbezverk_srwilkers: have a link?15:27
srwilkersyep, one second15:28
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Change _common_svc.yaml to support differing service names  https://review.openstack.org/40953115:30
*** yingjun has quit IRC15:30
sbezverk_srwilkers: ok yep15:33
portdirectkfox1111: when you get a moment if you could give some feedback on my comment here it would be great: https://review.openstack.org/#/c/409479/2/helm/microservice/nova-api/templates/nova_api.yaml15:34
sdakemorning peeps15:34
sdakeI see irc channel is busy15:34
portdirecthey sdake15:34
v1k0d3nmorning15:34
sdakedo people ever sleep? :)15:34
sdakesup portdirect v1k0d3n15:34
v1k0d3n:)15:34
srwilkersyo sdake15:34
srwilkersoh hey v1k0d3n15:35
sdakesup srs15:35
sdakesup srwilkers15:35
v1k0d3nhey srwilkers15:35
srwilkerssdake, sleep is overrated15:35
sdakesrwilkers sometimes it is :)15:35
portdirectsdake: i think we just cnage the Iridium Power Cell's every now and again :)15:35
portdirect*change15:35
*** jascott1_ has joined #openstack-kolla15:35
duonghqmorning sdake15:38
sdakesup duonghq15:38
*** rsjethani has quit IRC15:40
*** jascott1_ has quit IRC15:40
kfox1111portdirect: sure. sec15:43
Pavomorning15:45
*** yingjun has joined #openstack-kolla15:45
sdakesup pavo15:46
Pavonotta, getting my openstack env ready to do some k8s testing15:46
Pavofound a pretty good walk through15:46
Pavohttps://keithtenzer.com/2015/06/01/application-containers-kubernetes-and-docker-from-scratch/15:47
Pavoand https://keithtenzer.com/2015/04/15/containers-at-scale-with-kubernetes-on-openstack/15:47
Pavogonna use CentOS Atomic15:48
portdirectPavo: I'd also check out halycon as it supports openstack :)15:48
v1k0d3n:)15:49
Pavobeen fooling around with Cisco CSR1000v as routers in between tenant networks also, working pretty good so far15:49
SamYapleso for kolla-k8s is ansible being used to orchastrate?15:49
SamYaplehow are you setuping up, say, multinode galera/15:49
Pavoportdirect got a link for them?15:49
portdirectPavo: https://github.com/att-comdev/halcyon-kubernetes15:50
Pavothanks portdirect15:50
portdirectPavo, v1k0d3n would be the best person to quiz about running it on openstack15:50
srwilkers;)15:51
sbezverk_kfox1111: might be mistaken but I think there is a bug here: https://github.com/openstack/kolla-kubernetes/blob/master/helm/kolla-common/templates/_common_create_db_job.yaml#L5015:51
Pavodoesn't seem to hard, but will hit v1k0d3n up if I run into trouble15:51
v1k0d3nPavo: been a little bit since i've had to do that. worked well before, but if you run into issues...please let me or srwilkers know.15:51
v1k0d3nthere may be some things about nat and interfaces in there to consider just fyi.15:52
kfox1111SamYaple: no orchestration yet. no galera.15:52
Pavoyeah thats if I use openstack native routers right?15:52
sbezverk_we call for common_volumes but we do not pass .Variable so in some cases logging config map gets attached to bootstrap jobs15:52
Pavobecause I am gonna use csr1000v instead15:52
v1k0d3nand currently works with the supported systems listed in the project. need to do some things to get it to work with others (like CoreOS...which is coming...someday when i have time).15:52
kfox1111SamYaple: there's a concept in k8s called an operator that I think will be a good fit for galera clusters15:52
v1k0d3nPavo: yes, that's definitely part of it; correct.15:53
srwilkersv1k0d3n, i can start looking at some of the other systems. i know youre strapped for time these days15:53
kfox1111SamYaple: it extends k8s using k8s to give k8s knowlege of how galera works. so you declaratively state you want a galera cluster, and it takes care of membership/recovery/etc for you.15:53
v1k0d3nmy first priority was making halcyon work for local devs...so it would unblock development support for kolla-kubernetes.15:53
v1k0d3nsrwilkers: you don't have to. i think it currently works for the 80% out there.15:54
v1k0d3nthe coreos is a bit tricky for kubeadm since last i checked it's not officially supported and had some wackiness.15:54
kfox1111sbezverk_: what do you thin kthe bug is. I'm not spotting it.15:55
kfox1111sbezverk_: I don't see a 'with' in that file, so its passing all of . streight through.15:55
sbezverk_I checked recent failure of srwilkers cinder, create db job failed because logging config map was not there15:55
kfox1111sbezverk_: which I think should have .Values in it.15:56
*** rsjethani has joined #openstack-kolla15:56
sbezverk_kfox1111: he was bootstrap pod set to true and it is supposed to turm off attaching configmap for looging but it did not15:56
kfox1111hmm...15:57
portdirectsbezverk_: got link?15:58
*** mattmceuen_away has quit IRC15:58
kfox1111it tried to add the logger container?15:58
sbezverk_http://logs.openstack.org/49/409349/17/check/gate-kolla-kubernetes-deploy-centos-binary-ceph-multi-nv/bc59741/15:58
sbezverk_nope. just logging configmap15:59
kfox1111so, volume, not volume_mount?15:59
sbezverk_http://logs.openstack.org/49/409349/17/check/gate-kolla-kubernetes-deploy-centos-binary-ceph-multi-nv/bc59741/logs/pods/kolla-cinder-create-db-3l176.txt15:59
sbezverk_last line in the last paste16:00
kfox1111right.16:00
kfox1111hmm...16:00
kfox1111so the code is this:16:00
kfox1111{{- $podTypeNotBootstrap := not .podTypeBootstrap }}16:00
kfox1111{{- if and .Values.enable_kube_logger $podTypeNotBootstrap }}16:00
sbezverk_the only reason this would not work if variables are not getting there16:01
portdirectbug is here? https://review.openstack.org/#/c/409349/17/helm/microservice/cinder-create-db/templates/cinder_create_db.yaml16:01
sbezverk_kfox1111: he does not explicitely disables logger so one condition is truee16:01
kfox1111sbezverk_: havent seen this in any of my jobs though...16:01
portdirectline 4, should be similar to: https://github.com/openstack/kolla-kubernetes/blob/master/helm/microservice/keystone-create-db/templates/keystone_create_db.yaml#L416:02
kfox1111oh.16:02
sbezverk_kfox1111: I agree it is really strange16:02
kfox1111yeah. I've been explicitly disabling it.16:02
kfox1111so, its not an issue with .Values then.16:02
portdirectits not passing the podTypeBootstrap, through16:02
kfox1111its an issue with podTypeBootstrap.16:02
kfox1111yeah.16:02
srwilkersshit yeah, good catch portdirect. completely passed that over16:02
kfox1111do you have a link to the review?16:02
*** rsjethani has quit IRC16:03
portdirectkfox1111: https://review.openstack.org/#/c/40934916:04
kfox1111portdirect: good catch. that was what I was going to look at next. you beet me to it. :)16:04
portdirectJust trying to catch up with srwilkers, thats guys a reviewing machine :)16:05
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934916:05
*** zhubingbing has quit IRC16:05
Pavoif I am using ceph as storage do I have to enable cinder, glance and nova to use ceph as a backend?16:06
kfox1111I usually enable cinder and glance and cinder support in nova, but don't back it with ceph.16:07
*** sdake has quit IRC16:07
*** msimonin has joined #openstack-kolla16:08
*** yingjun has quit IRC16:08
Pavook so in globals I only need to have cinder, glance enabled but do not have to enable glance_backend_ceph: or cinder_backend_ceph:16:08
*** yingjun has joined #openstack-kolla16:08
Pavothe reason I am asking I am not able to upload images in horizon after about 4 images16:09
kfox1111I don't remember the exact vars to set. see kolla-kubernetes/tests/conf/*16:09
Pavothis is kolla-ansible16:09
Pavono kolla-k8s16:09
Pavonot16:09
kfox1111Pavo: kolla-kubernetes uses kolla-ansible's config at the moment.16:10
kfox1111so same thing.16:10
Pavoah16:10
*** yingjun has quit IRC16:12
Pavomy ceph partitions have 30Tb so I am sure its not a limit I am running into16:12
kfox1111yeah. sounds like its not using ceph.16:15
kfox1111the config bits can be a little tricky to figure out.16:15
kfox1111Pavo: https://github.com/openstack/kolla-kubernetes/blob/master/tests/conf/ceph-all-in-one/kolla_config16:16
kfox1111look for the ceph or rbd optionsn.16:16
srwilkerskfox1111, could you take a gander at https://review.openstack.org/#/c/409531/ and possibly workflow it? its re: the _common_svc discussion earlier16:16
Pavowell only thing I am doing before a deployment is enabling ceph in gloabls, then using parted $DISK -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1 to label the drives I want to use with Ceph16:16
kfox1111srwilkers: looks good to me.16:19
kfox1111workflowed16:19
*** sdake has joined #openstack-kolla16:20
kfox1111Pavo: yeah. I think you need the other vars to get the services to use it.16:20
Pavook will try16:21
srwilkersthanks kfox1111 :)16:21
openstackgerritMerged openstack/kolla-kubernetes: Change _common_svc.yaml to support differing service names  https://review.openstack.org/40953116:23
kfox1111srwilkers: thanks for working on it. :)16:23
*** duonghq has quit IRC16:24
kfox1111you did all the work. I just pulled the trigger. :)16:24
openstackgerritMerged openstack/kolla-kubernetes: Helm for nova-compute  https://review.openstack.org/40948216:28
*** Pavo has quit IRC16:29
*** Pavo has joined #openstack-kolla16:30
sbezverk_kfox1111: did you know that pod restart does not call for pod init container?16:31
sbezverk_and what is more strange, pod restart does not clean emptydir volumes16:32
kfox1111through what restart mechanism?16:32
kfox1111liveness hooks?16:32
kfox1111I would expect those two behaviors as a set. so at least it is consistent.16:33
sbezverk_liveness16:33
sbezverk_hook16:33
portdirectsbezverk_: is the pod restarted or just the container?16:33
kfox1111it would be really bad if it didn't rerun the inits but cleared out the volumes.16:33
sbezverk_portdirect: pod restart16:33
kfox1111sbezverk_: does it rebuild the containers or restart them?16:34
kfox1111(do their docker id's change)16:34
sbezverk_kfox1111: I think so but will double check16:34
kfox1111I've never dug into the details. thanks for sharing. :)16:35
portdirectthe /pause container wont change afaik? I think its just going to restart the containers in the pod, but not acutally rebuild it.16:35
kfox1111I wouldn't quite have expected that behavior.16:35
kfox1111portdirect: yeah, I'd think pause would stick around so the ip doesn't change.16:36
kfox1111if its not rerunning ip/volume chnages.16:36
kfox1111rerunning init16:36
*** jascott1_ has joined #openstack-kolla16:37
kfox1111jascott1_: hi16:37
*** kamtamtun has joined #openstack-kolla16:38
kfox1111portdirect: I strted another pass at reviewing your stuff. but most if it is in merge conflicts. :/16:38
sbezverk_portdirect: hmmmm kubectl shows pod restart, but after checking container ID it seems it restarts only container where liveness check exists :-(16:38
kfox1111hmm... so its a container liveness check, not a pod livenes check. interesting.16:39
portdirectkfox1111:  yeah - I'm gonna have to head out in 20 to move some suff at a freinds house but will get them resolved as soon as i get back16:39
kfox1111thats at least consistent.16:39
sbezverk_kfox1111: exactly, this is big surprise for me16:39
kfox1111portdirect: ok cool. I'll try to review as soon as I can.16:39
portdirectnp - will need to toally redo the keystone api (add apache_common)16:40
kfox1111portdirect: yeah. the apache common one should be pretty simple I think. just the other common one stripped of all the haproxy/second container stuff.16:41
portdirect+116:41
kfox1111and the prestop hoook changed to apachectl shutdown or something like that.16:41
*** jascott1_ has quit IRC16:41
portdirectkfox1111: good point - though thats differnt in ubuntu/cent afaik - fun times :)16:42
kfox1111hehe. yeah.16:42
kfox1111ideally, kolla should provide that api in the container api.16:43
kfox1111"prestop hook"16:43
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services  https://review.openstack.org/40948716:43
kfox1111but for now, yeah, we can just do a bit of logic to determine what its named and call that.16:43
portdirect+1 to that - is there a convention on how we doo OS detection? I assum just checking /etc/os-release?16:43
kfox1111I'd maybe look for the binary names in question?16:44
kfox1111apachectl or whatever it gets renamed to?16:44
portdirectdoh - much better :)16:44
openstackgerritPete Birley proposed openstack/kolla-kubernetes: Fix configmap mapping in _common_manage_db_job.yaml  https://review.openstack.org/40949516:45
portdirect^^ can that get workflowed? think it's holding up srwilkers.16:46
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for Glance PV/PVC  https://review.openstack.org/40949116:46
*** kamtamtun has quit IRC16:46
kfox1111portdirect: as soon as it passes the gate, I'l wf it.16:47
portdirectnice one - catch you cats in a bit - I'm off to play moving houses16:47
*** portdirect is now known as portdirect_away16:47
*** sayantan_ has joined #openstack-kolla16:49
*** adrian_otto has joined #openstack-kolla16:57
sdakesounds painful portdirect_away16:57
sdakeoff to do more tech support for my family - bbl17:00
*** srwilkers is now known as srwilkers_away17:13
openstackgerritMerged openstack/kolla-kubernetes: Fix configmap mapping in _common_manage_db_job.yaml  https://review.openstack.org/40949517:15
SamYaplekfox1111: sounds interesting17:26
*** adrian_otto has quit IRC17:30
*** rsjethani has joined #openstack-kolla17:36
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database  https://review.openstack.org/40953817:36
*** wanghongxu_ has quit IRC17:38
*** sdake has quit IRC17:41
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934917:44
*** hogepodge has joined #openstack-kolla17:50
*** msimonin has quit IRC17:51
*** msimonin has joined #openstack-kolla18:00
*** pbourke has quit IRC18:03
*** pbourke has joined #openstack-kolla18:04
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934918:10
Pavoanyone have a clue why I keep getting this error?18:13
Pavohttp://paste.openstack.org/show/592051/18:13
Pavoseems like a python issue to me18:13
sbezverk_srwilkers: ping18:18
*** sdake has joined #openstack-kolla18:21
*** Pavo has quit IRC18:29
*** Pavo has joined #openstack-kolla18:30
SamYaplePavo: is your disk full?18:33
*** John__ has joined #openstack-kolla18:40
kfox1111SamYaple: do you release stuff to the hub?18:41
kfox1111there's two containers now on the hub that need updating.18:42
SamYaplekfox1111: no. kolla has never done a good job of that18:42
SamYaplethat is one thing ive always wanted the team to focus on for stability...18:43
kfox1111alternately know when 3.0.2 will be cut?18:43
SamYapleproduction and testing18:43
kfox1111SamYaple: +118:43
SamYaplei know nothing kfox1111. im just a visitor here18:43
sdakekfox1111 z streams are supposed to be cut on a 45 day schedule18:43
sdakewhether that is happening or not, i don't know18:44
sdakethe 45 day schedule lines up with the previous 6 month schedule of openstack18:44
sdakeunfortunately the schedule has chnaged to 4 months for ocata18:44
sdakeso that has thrown everything off related to zstream backport releases18:44
kfox1111the hub just says "1 month ago". :/18:44
kfox1111so don't know.18:44
kfox1111yeah.18:44
kfox1111I've got a review that works around it, but does a pip install in the startup which is really ugly.18:45
sdakestartup of which?18:45
kfox1111sdake: https://review.openstack.org/#/c/409488/3/helm/kolla-common/templates/_common_create_keystone_endpoint.yaml line 25.18:46
kfox1111I really would rther not do that.18:46
kfox1111but we should cut an updated hub container then.18:46
sdakefirst we have to cut a zstream tag ;)18:46
sdakejeffrey4l is the relese liasion for ocata18:47
kfox1111sdake: thats what bugs me for this whole z stream thing.18:47
kfox1111it doesn't change any code.18:47
SamYaplei seem to recall a discussion about the z stream being different from docker image versions18:47
kfox1111if you do the same thing a second time, you get a newer pip package and it all just starts working. :/18:47
SamYapleas in, we related a 3.0.2-1 and a 3.0.2-218:47
kfox1111its a revision, not a new version.18:47
kfox1111SamYaple: yeah. exactly.18:48
SamYaplethat was a plan at one point kfox111118:48
sdakeSamYaple yup someone wnated to do that18:48
sdakebut nobody ever did18:48
SamYaplewell we cant until we make image building better18:48
kfox1111I'd be willing to do the work.18:48
SamYaplekfox1111: im sure you can get repo access18:48
kfox1111on getting the scripts kergiggered around to allow it.18:48
SamYaplenot alot of people fighting for it18:48
sdakekfox1111 you already have docker hub push access18:48
kfox1111SamYaple: I have access. but been reluctant to use it, as I'm not a kolla core.18:48
kfox1111and I'm not too familior with the build proccess yet.18:49
SamYaplejust discuss it in the next meeting so everyone is aware18:49
SamYapleoh18:49
sdakeSamYaple agree - seems like a good agenda item18:49
SamYaplewell, get familiar first :P18:49
kfox1111+1 for more discussion.18:49
kfox1111yeah. was wondering if someone more familior with the hub could do this one, and then it gives me time to spin up for the next.18:49
SamYapleafaik, they still ahvent fixed the primary issue18:50
kfox1111whats that?18:50
SamYapleso it takes like 6 hours to upload even on gigabit connections18:50
SamYapleyou have to push every layer for every image when pushing to dockerhub18:50
SamYapleits like 100GB for all images18:50
kfox1111I thought the sha stuff they did fixed that?18:50
sdakekfox1111 here is the agenda that jeffrey4l is running from until inc0 gets back: https://wiki.openstack.org/wiki/Meetings/Kolla18:50
PavoSamYaple not at all, have 30Tb free18:50
Pavobut I think it might be my images18:51
SamYaplekfox1111: it was fixed for private registry18:51
Pavoso I am gonna just pull the offical kolla images18:51
*** sayantan_ has quit IRC18:51
Pavoonce I pull the offical kolla images how do I push them to my private registry?18:51
kfox1111SamYaple: hmm.. k.18:51
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Adds type label to service selector  https://review.openstack.org/40954318:51
kfox1111SamYaple: would one of the other pubilc repo's be a better fit then?18:51
sbezverk_kfox1111: to match with old appraoch of service selection, I added type label.18:52
SamYaplekfox1111: supposedly openstack infra was goign to get a registry18:52
sbezverk_kfox1111: see ps ^^^18:52
SamYaplekfox1111: but idk where monty is at with that. he kinda overreaches sometimes18:53
sdakenah infra isn't setting up a registry tmk18:53
kfox1111SamYaple: I think thatas just for gating, not for releasing?18:53
SamYaplesdake: monty started at one point, but didnt keep on it i dont think18:53
SamYaplekfox1111: nope, that would have ebeen for releasing18:53
SamYaplekfox1111: well for both18:53
kfox1111SamYaple: ah. ok. I'm down with that then.18:54
kfox1111if its stalled, maybe we can help.18:54
PavoSamYaple if I pull offical kolla images instead of building my own how do I push them into my private registry?18:54
SamYaplePavo: tag them as your repo18:54
kfox1111but I d on't really have much preference where it lands, so long as its scalable so lots of people can pull from it.18:54
kfox1111so like, the hub, google's or quay18:55
SamYaplekfox1111: thats probably dockerhub or quay18:55
SamYaplegive dockerhub a try, see if they fixed the upload-same-layer thing18:55
SamYapleif not, its alot of data to push up18:55
kfox1111yeah.18:56
kfox1111k.18:56
*** srwilkers_away is now known as srwilkers18:56
SamYaplek sleep time18:56
kfox1111nite18:56
srwilkerssbezverk_, just noticed your ping. was away19:00
Pavoif I did a kolla-ansible pull only why does my images look like this? http://paste.openstack.org/show/592052/19:01
Pavoshould they not have my registry info in them?19:01
kfox1111sbezverk_: looks good to me.19:01
sbezverk_srwilkers: np I just pushed type19:01
sbezverk_addition to svc19:02
srwilkersokay yeah19:03
srwilkersthat makes sense. thats what i meant when i replied about having a type. meant a type in the service selector19:04
srwilkersguess i shouldve been more clear19:04
sbezverk_srwilkers: that was not you, it was me..19:06
sbezverk_srwilkers: needed to confirm wit halready running test bed..19:06
srwilkers:)19:08
*** v1k0d3n has quit IRC19:13
*** larsks has joined #openstack-kolla19:16
sbezverk_kfox1111: please ack when you have a chance: https://review.openstack.org/40954319:21
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database  https://review.openstack.org/40953819:22
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934919:24
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database  https://review.openstack.org/40953819:27
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934919:28
*** diogogmt has joined #openstack-kolla19:32
Pavois this the correct way to tag the official kolla images to be able to push them into my private registry? docker tag b46dd1990bc1 deployer.local:4000/kolla/centos-binary-mariadb19:35
*** jmccarthy has quit IRC19:37
*** jmccarthy has joined #openstack-kolla19:38
*** mgiles has joined #openstack-kolla19:39
*** jascott1_ has joined #openstack-kolla19:39
*** jascott1_ has quit IRC19:44
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm for nova-libvirt  https://review.openstack.org/40948119:45
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm database creation for Nova  https://review.openstack.org/40947419:49
*** rsjethani has quit IRC19:51
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database  https://review.openstack.org/40953819:55
*** portdirect_away is now known as portdirect19:58
portdirecto/19:58
portdirecthey just been going trhough a few pr's and realised somthing, what release are the 2.0.2 images?19:59
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm for nova-api  https://review.openstack.org/40947919:59
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database  https://review.openstack.org/40953820:00
srwilkershey portdirect o/ -- im honestly not sure20:01
*** jascott1_ has joined #openstack-kolla20:02
portdirecthey srwilkers, just checked, mitaka i think, should we not be using newton at least?20:04
*** diogogmt has quit IRC20:05
*** jascott1_ has quit IRC20:07
*** msimonin1 has joined #openstack-kolla20:11
*** msimonin has quit IRC20:12
*** portdirect has quit IRC20:15
*** John__ has quit IRC20:26
*** Pavo has quit IRC20:29
*** Pavo has joined #openstack-kolla20:30
*** portdirect has joined #openstack-kolla20:35
openstackgerritMerged openstack/kolla-kubernetes: Helm for nova-libvirt  https://review.openstack.org/40948120:41
sbezverk_portdirect: it is mitaka, newton is 3.0.X20:44
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database  https://review.openstack.org/40953820:54
portdirectsbezverk_: I dont think it should make too much diference at the moment (it was mgiles's work nova db migration that made me think - as i think the api one came in with mitaka.) but we should prob make it a prority to shift over soon as all the microservices are done?20:54
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for Glance PV/PVC  https://review.openstack.org/40949120:55
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm creation of Nova services  https://review.openstack.org/40947320:56
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934920:57
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services  https://review.openstack.org/40948720:58
sbezverk_portdirect: if it were up to me, I would use master ;-) but Kevin suggested to complete all microservices on 2.0.2 and then switch all to 3.0.1 or 3.0.2.20:58
portdirectsbezverk_: master would be what I'd want too20:58
srwilkerssbezverk_, ah. should we move the image_tags to 2.0.2 for consistency's sake there, or keep running them with 3.0.1?20:59
sbezverk_portdirect: where is fun in life if you cannot fight with bleeding edge bugs ;-)20:59
portdirectsrwilkers: I would move backward if you've made the shift already20:59
srwilkersportdirect, yeah, ive been using 3.0.1 for the image_tag21:00
sbezverk_srwilkers: I do not think it is needed to change alreafy working services back to 2.0.221:00
Pavook I need help, I have redeployed evening using the official kolla images and keep getting this error when trying to upload a image in horizon http://paste.openstack.org/show/592055/21:00
srwilkerssbezverk_, none of my have been merged in yet, so doing 2.0.2 isnt a big deal21:00
Pavothat is from horizon.log21:00
portdirectsbezverk_: why else do you think I'd run fedora rawhide?21:00
sbezverk_srwilkers: in this case why not ;-)21:00
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database  https://review.openstack.org/40953821:02
Pavoanyone have a clue why that error is happening?21:03
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934921:04
portdirectPavo: what version of horizon?21:04
Pavowhat ever version is in the official kolla 3.0.1 image21:04
PavoI pulled all official kolla images21:05
Pavoany clue portdirect ?21:08
portdirectoff the top of my head that looks like a horizon, not kolla bug21:11
Pavolooks like python issues to me21:11
portdirecthow big was the image?21:12
kfox1111sbezverk_: done21:13
*** jascott1_ has joined #openstack-kolla21:13
sbezverk_kfox1111: thanks21:13
kfox1111np21:14
kfox1111one of the things still todo is to get the versioning stuff streightened out, and add a 3.0.x set to the gate too.21:16
kfox1111so we can support both sets.21:16
openstackgerritMerged openstack/kolla-kubernetes: Adds type label to service selector  https://review.openstack.org/40954321:16
portdirectcant we just dump 2?21:16
kfox1111that will make it easier to test upgrades when we're ready.21:16
kfox1111I'm kind of hoping to support 2 versions at a time to make upgrades easier?21:17
kfox1111so when ocata cuts, we drop 2.0.x?21:17
portdirector get a master gate?21:17
kfox1111yeah. a master woudl be nice.21:18
kfox1111more work then just switching out 2.0.2 for 3.0.1 though from the ub.21:18
kfox1111hub21:18
kfox1111master needs to get done at some point too though. for sure.21:18
portdirectthen we would be n and n+1, I't would be great if we could have a master in the hub as well?21:19
kfox1111on the fence on that one. I think it may change faster then might be a good fit on the hub?21:19
kfox1111I think with your docker save trick though, we can artefact trunk builds for the gate on tarballs.openstack.org21:19
kfox1111at least.21:19
kfox1111trove does a similar thing here: https://tarballs.openstack.org/trove/images/ubuntu/21:20
portdirectyeah - I could donate bandwidth for pushing to the hub if that was an issue  - but lets get the thing working - then work out how to upgrade :)21:21
sbezverk_I agree about master. it helps to be prepared and move forward21:21
kfox1111so they have a periodic job in zuul that just builds the images, does some sanity checks and pushes to that location.21:21
kfox1111then thier othr jobs that test againt master pull from there.21:21
kfox1111how about n and n-1 until we get master, then we swithc to n, n+1 after we get master?21:22
portdirectsweet - if we could do that that would be great - would it be easy to trigger a daily build ?21:22
kfox1111portdirect: I think so. trove seems to be agood template to copy from.21:22
kfox1111I just havent' had time to fully experiment with the idea yet.21:22
kfox1111I did talk with infra a bit about it though.21:22
*** kollabot5 has joined #openstack-kolla21:23
*** kollabot has quit IRC21:23
*** yee379 has quit IRC21:23
*** yee379 has joined #openstack-kolla21:23
kfox1111they were cool with the idea, as other s are already doing it.21:24
kfox1111see project-config//jenkins/jobs/trove.yaml, job template 'gate-trove-integration-buildimage-{datastore}{job-suffix}'21:24
kfox1111if you are curious.21:24
portdirectnice - lets chat about this later - just a bit concerned that we are developing for the last version - but not that concerned yet, just means that we are lagging behind other projects for both release, and k8s version21:24
kfox1111+1.21:25
*** msimonin1 has quit IRC21:25
kfox1111once I get neutron done, versions were kind of next on my todo list.21:25
kfox1111it would be easier to go to 3.0.1 only too though if we get fully to helm.21:25
kfox1111so we ca nget to 3.0.1 as ssoon as either one of those happens.21:25
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services  https://review.openstack.org/40948721:27
*** pomac has quit IRC21:28
kfox1111so, what do you guys think about doing something similar to the values.yaml for Charts?21:32
kfox1111we could make an all_charts.yaml.21:33
portdirectonly realy think the version would need to be changed/templated in charts, unless I'm missing somthing?21:34
kfox1111or, we could do a _Chart.yaml instead of Chart.yaml, in each dir, and then have prebuild use it to create Chart.yaml?21:34
kfox1111portdirect: yah. thats my thinking. the version should be easily changable.21:34
sbezverk_kfox1111: ideally helm should take care of that21:34
kfox1111sbezverk_: ideally yeah.21:35
kfox1111sbezverk_: no way to do that currently. :/21:35
portdirectseems a bit heavy duty for that? im kinda -1 at this stage, but would need to have a think21:35
kfox1111portdirect: ok, let me ask a little differnt way.21:35
kfox1111if we were to support 2.0.2 packages, and 3.0.1 packages,21:35
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm for installing other nova services  https://review.openstack.org/40948621:36
kfox1111I think it would be best to have the version of the helm packages reflect that. so neutron-server-2.0.2-1 and neutron-server-3.0.1-1 ?21:36
portdirectit would pretty easy to make prebuild.py parse and modify the value as required wouldnt it? I'd kinda perfer that off the top of my head - then each chart would use 'master' as the version untill we cut a release?21:36
kfox1111portdirect: yeah. but if it edits Chart.yaml and its checked in, then it will be kind of a pain for development I think.21:37
*** dave-mccowan has joined #openstack-kolla21:37
portdirectthats why it would be a release only option i think?21:37
portdirectmy concern is beasue it takes us further away from helm 'native'21:38
kfox1111oh. like, helm prebuild doesn't touch charts.yaml unless you need to change versions?21:38
portdirectyeah21:38
kfox1111some flag to helm_prebuild?21:38
kfox1111hmm..21:38
kfox1111yeah. that would work. good idea. :)21:38
portdirectsweet21:39
openstackgerritMerged openstack/kolla-kubernetes: Helm for nova-api  https://review.openstack.org/40947921:39
sbezverk_kfox1111: so we checkin version independent bits, and then what will make them prepped for a specific version?21:40
kfox1111I guess we check in only one specific verson and then allow it to be easily overriden.21:41
sbezverk_prepped I mean Chart.yaml get overridden for a requested version21:41
kfox1111with that model.21:41
kfox1111yeah.21:42
sbezverk_getting helm to support vars in Charts.yaml  would solve so much of headache21:45
kfox1111sbezverk_: +121:46
kfox1111values too.21:46
kfox1111I'm sure they will get there.21:46
kfox1111and if not, we can help. :)21:47
kfox1111does anyone test the containers pushed to the hub before they are pushed?21:49
*** dave-mccowan has quit IRC21:49
*** v1k0d3n has joined #openstack-kolla21:49
openstackgerritMerged openstack/kolla-kubernetes: Helm charts for Glance PV/PVC  https://review.openstack.org/40949121:50
openstackgerritMark Giles proposed openstack/kolla-kubernetes: Helm database creation for Nova  https://review.openstack.org/40947421:50
sbezverk_kfox1111: do you mean from kolla side right?21:50
*** williamcaban has quit IRC21:51
kfox1111sbezverk_: yeah.21:51
portdirecti hope they are - one of the major issues I've heard about kolla in the past is the lack of good images in the hub21:51
kfox1111portdirect: kolla thus far has considered them as a convenience, not something to be used for anything other then testing with.21:52
kfox1111thers probably a bunch more infra needed to be put in place to make it production ready.21:52
portdirectkfox1111: yeah - but users dont expect that :)21:52
kfox1111agreed.21:52
kfox1111just wondering if I cut a 3.0.1-2 release could we just upload it, or should more testing be done first?21:53
kfox1111(for the toolbox)21:53
*** rstarmer has quit IRC21:53
kfox1111wana get the neutron entrypoint stuff up but really fell uneasy about the pip thing in it.21:53
portdirectperhaps we should get a kolla-dev or similar namespace on the hub?21:53
kfox1111hmm. that may be possible.21:54
kfox1111but without some mechanism to keep it up to date, may und up in the same state the kolla/ namespace has.21:54
portdirectthought these debates all become moot if we get a registry via infra21:55
portdirectwas more thinking that it would allow you to push at will - if it was clear that it would be davelopment images21:55
portdirect(ie no qa)21:56
kfox1111yeha. true. solvable with infra registry too.21:56
kfox1111just wondering how to fix the existing release, if thats a good thing to do.21:57
*** pomac has joined #openstack-kolla21:57
sbezverk_kfox1111: I think it is a good thing, but I doubt it will be accepted as it brings inconsistency in versioning21:59
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934922:00
kfox1111yeah. I'm just not sure. the existing versioning has always thus far been about the code that uses to build the thing.22:00
kfox1111and none of that changes. so its still 3.0.1, just a dep gets updated.22:00
kfox1111but I do see your point.22:01
kfox1111which is why I was asking.22:01
kfox1111others may feel the same.22:01
sbezverk_kfox1111: some fixes can be backported and get published in z stream22:01
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services  https://review.openstack.org/40948722:02
kfox1111yeah. asked when that may be too. no one's sure. :/22:02
kfox1111wiht ocata being a short cycle, it may never happen.22:02
kfox1111the other alternative is making a new package.22:05
kfox1111we could make an openstack-cli container.22:05
kfox1111that may have other benifits.22:06
sbezverk_kfox1111: idk too many toolboxes ;-)22:09
kfox1111hehe. true.22:10
kfox1111I've wanted a general openstack-cli container for even my non containerd clouds though.22:10
kfox1111makes it easy to for users to get cli tools without doing a bunch of rpms or debs or whatever.22:10
kfox1111the opnestack cli has a lot of deps.22:11
sbezverk_kfox1111: right I remember we talked about that before and I think kolla-toolbox was supposed to address that22:11
kfox1111yeah. not sure what the kolla-toolbox was exactlyu intended for. seems like a catch all container.22:12
sbezverk_what if we open PS on kolla side and then ask to get it backported to stable newton22:12
kfox1111has ansible in it too.22:12
kfox1111could try. if it was prurely a new container, it might be easier to convince folks. but could be rejected as a feature too.22:13
kfox1111that kind of why I liked  theidea of just fixing the hub container, as it doesnt' requrie any changes at all to fix.22:13
sbezverk_kfox1111: the argument will be how people who downloaded this container before the change, know that there was a change?22:15
*** rstarmer has joined #openstack-kolla22:20
*** bmace has quit IRC22:20
*** sdake_ has joined #openstack-kolla22:20
*** bmace has joined #openstack-kolla22:20
*** rstarmer has quit IRC22:21
*** rstarmer has joined #openstack-kolla22:21
*** rstarmer has quit IRC22:22
*** rstarmer has joined #openstack-kolla22:22
*** rstarmer has joined #openstack-kolla22:23
*** rstarmer has quit IRC22:23
*** rstarmer has joined #openstack-kolla22:24
*** rstarmer has quit IRC22:24
*** sdake has quit IRC22:24
*** rstarmer has joined #openstack-kolla22:24
*** rstarmer has quit IRC22:25
*** rstarmer has joined #openstack-kolla22:25
*** williamcaban has joined #openstack-kolla22:25
*** rstarmer has quit IRC22:26
*** Pavo has quit IRC22:29
*** Pavo has joined #openstack-kolla22:29
*** larsks has quit IRC22:30
openstackgerritSteve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder  https://review.openstack.org/40934922:32
*** JRobinson__ has joined #openstack-kolla22:32
*** dave-mccowan has joined #openstack-kolla22:32
*** srwilkers is now known as srwilkers_away22:33
kfox1111sbezverk_: not sure there's a garentee that containers are stable on the hub.22:35
kfox1111oh. wait. are you saying, how will they know they may need to pull a newer one so that they can use kolla-kubernetes?22:37
sbezverk_kfox1111: I thought in opensource there is no such thing like guarantee ;-)22:37
kfox1111heh. in some ways yea.22:37
sbezverk_kfox1111: right that was my point22:37
kfox1111gotcha. yeah. that is a problem...22:37
sbezverk_they need to check build date22:37
kfox1111yeah. or we make it explicit as a different release tag. like 3.0.1-222:38
*** larsks has joined #openstack-kolla22:38
kfox1111then 3.0.1 still exists.22:38
kfox1111and we update kolla-kubnetes all_values to point to 3.0.1-222:38
kfox1111its not a different updated zstream though, so may not need to wait for a new release.22:39
*** dave-mccowan has quit IRC22:39
sbezverk_kfox1111: right but these are kolla images, you will need to sell them this idea..22:39
kfox1111+222:40
kfox1111+122:40
sbezverk_I am not sure if we can just at will go and build in kolla repo22:40
sbezverk_tags we think make sense..22:40
kfox1111yeah. I don't think we should either. shoudl go through kolla.22:40
kfox1111I'm in a weird place though, as  Ihave commit rights to the hub, and not many people want to say they can do anything with the hub.22:40
kfox1111probably should bring it up at the next meeting.22:41
kfox1111shoudl we then wait to merge any of the endpoint creation stuff, or merge the hack,22:41
kfox1111and undo the hack asap?22:41
sbezverk_+1 for hack22:42
kfox1111k.22:42
sbezverk_it will be a minor change to undo it22:42
kfox1111its ready now then if you want to merge it: https://review.openstack.org/#/c/409488/22:42
kfox1111yeah. just gota drop the user: 0 and the pip install from 2 files.22:42
sbezverk_kfox1111:done22:45
kfox1111k. thx.22:45
openstackgerritMerged openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs  https://review.openstack.org/40948822:45
*** msimonin has joined #openstack-kolla22:51
*** JRobinson__ has quit IRC22:53
*** msimonin1 has joined #openstack-kolla22:53
*** JRobinson__ has joined #openstack-kolla22:56
*** msimonin has quit IRC22:56
*** goldyfruit has joined #openstack-kolla22:57
*** goldyfruit has quit IRC23:12
portdirecthttps://github.com/portdirect/harbor23:22
*** JRobinson__ has quit IRC23:24
*** JRobinson__ has joined #openstack-kolla23:27
sbezverk_portdirect: so far I could not find a gracefull way to kill POD from wihtin itslef, other than kubectl :(23:37
*** msimonin1 has quit IRC23:39
*** msimonin has joined #openstack-kolla23:39
*** msimonin has quit IRC23:39
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: DO_NOT_MERGE PoC Configmap update monitor  https://review.openstack.org/40949323:44
portdirectsbezverk_: lol so that link was posted in the wrong place :( I'll have another look at the PoC you've done and upload my take if you like?23:52
kfox1111sbezverk_: still thinking the cleanest thing really is to update the k8s object as appropriate and have k8s deal with killing the pods.23:55
kfox1111:/23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!