openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/408263 | 00:06 |
---|---|---|
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm for nova-compute https://review.openstack.org/409482 | 00:11 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/408263 | 00:13 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 00:14 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/408263 | 00:16 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/408263 | 00:17 |
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm for nova-libvirt https://review.openstack.org/409481 | 00:24 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 00:27 |
*** sayantan_ has quit IRC | 00:29 | |
*** Pavo has quit IRC | 00:29 | |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/408263 | 00:34 |
openstackgerrit | caoyuan proposed openstack/kolla: Fix some typo https://review.openstack.org/407884 | 00:34 |
*** Pavo has joined #openstack-kolla | 00:34 | |
openstackgerrit | caoyuan proposed openstack/kolla: Fix some typo https://review.openstack.org/407884 | 00:36 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/408263 | 00:36 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/408263 | 00:39 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/408263 | 00:54 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 00:55 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/408263 | 00:56 |
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm for installing other nova services https://review.openstack.org/409486 | 00:56 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services https://review.openstack.org/409487 | 00:59 |
sdake | hey peeps | 01:02 |
sdake | mgiles are you unblocked for dev work? | 01:03 |
mgiles | Yep. Had some weird errors on the gate for my ps so have been breaking it down into smaller chunks to isolate the problem | 01:05 |
mgiles | but not blocked | 01:05 |
*** sdake_ has joined #openstack-kolla | 01:06 | |
sdake_ | mgiles sounds good | 01:06 |
sdake_ | mgiles note some of the gate providers (specificailly rax.org and rax.iad) are particularly flakey with our gate scripts | 01:07 |
sdake_ | mgiles the way you tell is at the top of the gate output | 01:07 |
sdake_ | mgiles if you see a gate problem there, you can always type recheck in the log and the infrastructure will automaticallly retest the code | 01:07 |
*** mgiles_ has joined #openstack-kolla | 01:08 | |
sdake_ | mgiles_ did you get all that :) | 01:08 |
mgiles_ | yes, but not sure what you mean by "type recheck in the log" sdake | 01:08 |
mgiles_ | sdake In any case, I don't htink that was the problem here, but good to know for future reference | 01:09 |
*** sdake has quit IRC | 01:09 | |
sdake_ | mgiles_ cool - so recheck you know how when you do a review | 01:11 |
sdake_ | mgiles_ and you type some commentary and press "post" | 01:11 |
sdake_ | instead type "recheck" and press post | 01:11 |
mgiles_ | sdake yep | 01:11 |
mgiles_ | sdake ok. didn't know that. Thanks | 01:11 |
sdake_ | it will rerrun the gate | 01:11 |
sdake_ | cool its handy | 01:11 |
sdake_ | because gate is a little flakey | 01:12 |
sdake_ | gate suffers from heisenfailures | 01:12 |
sdake_ | i think we could fix rax.iad and rax.org if we could cosnisstently get jobs to run on them os we could sort out what is wrong | 01:12 |
mgiles_ | sdake_ Yep, I have seen other weird errors that I can't attibute to anything, so think I may have hit flakiness in some cases too | 01:13 |
sdake_ | infra has no way of specifying where a job runs | 01:13 |
kfox1111 | finally think I've got a fix to the gate now. | 01:13 |
sdake_ | so we can't isolate - we have to get lucky | 01:13 |
sdake_ | and there are something like 9 cloud providers | 01:13 |
sdake_ | so getting lucky is hard :) | 01:13 |
kfox1111 | sdake_: I've not been seeing as much flakyness on the kubernetes side on rax | 01:15 |
kfox1111 | so some difference between the ansible stuff and kubernetes stuff. | 01:16 |
kfox1111 | I did have to debug a lot in the kubernetes gate though, so maybe fixed an issue that plangues the rax side. the networking there is a bit goofy. | 01:16 |
kfox1111 | well. different at least. | 01:16 |
sdake_ | kfox1111 that is good news :) | 01:16 |
kfox1111 | I kept tryingt o correlate the curent resolver problem to one cloud or another. couldn't do it. | 01:17 |
kfox1111 | seemed to show up on all of them. | 01:17 |
sdake_ | kfox1111 sweet if you ahve some ideas to pass of th jeffrey4l any ideas | 01:17 |
sdake_ | kfox1111 it would be sweet if we could get our gates rock solid on kolla-ansible too | 01:17 |
kfox1111 | yeah. might some day have time for that. | 01:18 |
kfox1111 | I've been trying to pass around as much gate knowlege as Ican | 01:19 |
kfox1111 | so hopefully I'm not the only one soon either. | 01:19 |
srwilkers | just curious, where can i find more information on the gates kfox1111? other than digging through the console logs? | 01:20 |
sdake_ | Jeffrey4l_ you about? | 01:20 |
kfox1111 | the logs are most of it. | 01:20 |
kfox1111 | the zuul page helps a bit too. | 01:21 |
kfox1111 | http://status.openstack.org/zuul/ | 01:21 |
kfox1111 | can see whats running, and watch jobs live. | 01:21 |
sdake_ | zuul is the new jenkins | 01:21 |
kfox1111 | sdake_: I think I understand how to artefact stuff in the gate now, aftger reading trove's gates. | 01:22 |
sdake_ | every co investing heavily in it instead of jenkins | 01:22 |
kfox1111 | so we might be able to cache all sorts of things. | 01:22 |
srwilkers | thanks kfox1111 | 01:22 |
kfox1111 | I ned to try it. | 01:22 |
kfox1111 | srwilkers: anything else you want to know, just ask. :) | 01:22 |
kfox1111 | the more folks that understand it, the better off we'll all be. :) | 01:23 |
sdake_ | kfox1111 not a fan of caching wrt gating - but feel free to do what needs to be done :) | 01:23 |
sdake_ | kfox1111 ^^++ | 01:23 |
kfox1111 | sdake_: the caching will be about making the tests more granular. so we can catch breakages much qicker. | 01:23 |
sdake_ | yup understanding how the gate works is a bit complicated, hwoever, it has alot of value | 01:23 |
kfox1111 | or catch them in one place without affecting progress on another piece. | 01:24 |
sdake_ | kfox1111 the key with the gates is they should always start from a empty state | 01:24 |
sdake_ | caching introduces a non-empty state | 01:24 |
kfox1111 | sdake_: no, not nessisarily. the other altiernitive is starting from a known good state. | 01:24 |
sdake_ | how do you prove known good state ;) | 01:24 |
kfox1111 | you test it. :) | 01:25 |
kfox1111 | you have tests that test each other. :) | 01:25 |
sdake_ | right in the gate, which means the testing in the gate needs to be absolutelly perfect | 01:25 |
kfox1111 | for example. | 01:25 |
kfox1111 | you test out kubernetes 1.4.8. and cache it. | 01:25 |
sdake_ | kolla-kubernetes too young to make the gating absolutely perfect | 01:25 |
kfox1111 | then you use that known good copy. | 01:26 |
sdake_ | 5% battery power - if i drop off | 01:26 |
kfox1111 | you then test kolla-kubernetes trunk against taht. | 01:26 |
kfox1111 | once you have a working gate for kolla-kubernetes, | 01:26 |
kfox1111 | you test kubernetes 1.4.9 against it. | 01:26 |
kfox1111 | then cache 1.4.9. | 01:26 |
kfox1111 | then you know when your checking reviews for kolla-kubernetes, that random kubernetes upgrades wont interfier with your testing. | 01:27 |
sdake_ | cache 1.4.9 of kubernetes images? | 01:27 |
sdake_ | rather tarballs? | 01:27 |
kfox1111 | yeah. cause its not just a tarball. | 01:27 |
kfox1111 | its docker containers too. | 01:27 |
kfox1111 | that it downloads. | 01:27 |
kfox1111 | and rpms. | 01:27 |
kfox1111 | or debs. | 01:28 |
sdake_ | ok - well my suggestion would be not to be too rocket scientisty on the gate early on :) | 01:28 |
kfox1111 | we cache all the deps, so thjat when we review code, we know that the deps have been pretested. | 01:28 |
kfox1111 | yup. | 01:28 |
kfox1111 | baby steps. | 01:28 |
kfox1111 | thats why I haven't implemneted it yet. | 01:28 |
sdake_ | because I didn't understand half of what you just said - admiditaly up against a lcock on my laptop power to explore your ideas | 01:28 |
kfox1111 | working on getting the current one completely stable first. | 01:28 |
sdake_ | indeed we do cache our requirements | 01:29 |
kfox1111 | we cache some of or requirements. like for pip. | 01:29 |
sdake_ | or most of them anyway | 01:29 |
sdake_ | right | 01:29 |
kfox1111 | bujt kolla-kubernetes has a lot of other deps. | 01:29 |
kfox1111 | the kolla containers. kubernetes. ceph. etc. | 01:29 |
kfox1111 | kolla-ansible. | 01:29 |
sdake_ | if it works for pip it should work for others | 01:29 |
sdake_ | and it does work for pip with the occasional blip that infra quickly fixes | 01:30 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 01:30 |
kfox1111 | it would be nice if everyone would agree on a same caching mechanism. | 01:30 |
kfox1111 | but deb/ubuntu/pip/docker all have to do things differently. :/ | 01:30 |
sdake_ | yup - i struggle to get caching of deb/rpms into infra | 01:31 |
sdake_ | however inc0 has taken over that work | 01:31 |
kfox1111 | cool. if he needs help. | 01:31 |
kfox1111 | I cache rpms at least all the time. | 01:31 |
sdake_ | jeffrey4l is helping him i think | 01:31 |
sdake_ | Jeffrey4l_ that is | 01:31 |
kfox1111 | cool. | 01:31 |
sdake_ | kfox1111 we don't reallycall it cachcing in openstack its called mirroring | 01:32 |
sdake_ | but its the same thing imo :) | 01:32 |
kfox1111 | yeah. same thing. :) | 01:32 |
*** zhubingbing has joined #openstack-kolla | 01:33 | |
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm creation of Nova services https://review.openstack.org/409473 | 01:34 |
*** sdake_ has quit IRC | 01:37 | |
sbezverk_ | sdake: ping | 01:48 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: More dns gate fixes https://review.openstack.org/408263 | 01:48 |
sbezverk_ | kfox1111: I see you went really wild with gatr testing!! | 01:50 |
sbezverk_ | gate | 01:50 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 01:51 |
portdirect | sbezverk_: I've asked on slack but though there's a chance you might know: | 01:55 |
portdirect | is it possible to get the CIDR that the CNI network uses for pods via the API? | 01:56 |
mgiles_ | kfox1111 seems you were right about the services being the culprit for my issues. That's the only ps that failed the gates. It's getting late here so I'm not up for digging into this further now, but will look tomorrow (maybe) or Monday to see if I can figure out what's going on. | 01:57 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/409488 | 01:57 |
mgiles_ | kfox1111 Thanks for the help | 01:57 |
kfox1111 | mgiles_: sure. :) | 01:58 |
kfox1111 | sbezverk_: yup. finally got the gate to block until dns is ready properly. we should really get that merged asap. | 01:58 |
kfox1111 | sbezverk_: also found a case where ubuntu external dns never worked. | 01:58 |
kfox1111 | though I don't think we actually used it in the gate until the workaround in the neutron entrypoint patch. | 01:59 |
portdirect | mgiles_: dumping the endpoints from kubectl in the gate may be worth a shot while debugging - its helped me there - as its very easy to get the labeling slightly off. | 01:59 |
kfox1111 | portdirect: yeah. had to do do that enough, I made the gate log it. | 02:00 |
mgiles_ | portdirect Can you give me more details about what you did to get that? | 02:00 |
portdirect | 'kubectl get --namespace kolla endpoints' | 02:00 |
kfox1111 | mgiles_: or if in the gate, look under logs for svc* | 02:01 |
portdirect | ^^ nice :) | 02:01 |
mgiles_ | kfox1111 Cool, thanks | 02:01 |
mgiles_ | and portdirect ^^^ thanks | 02:01 |
portdirect | np, catch you on the flipside mgiles_ :) | 02:02 |
mgiles_ | good night | 02:03 |
*** mgiles_ has quit IRC | 02:03 | |
sbezverk_ | kfox1111: dso you have a link for that PS to ack? | 02:04 |
kfox1111 | sbezverk_: yeah. sec | 02:04 |
openstackgerrit | zhubingbing proposed openstack/kolla-ansible: add panko role https://review.openstack.org/400122 | 02:04 |
portdirect | sbezverk_: kfox1111 https://review.openstack.org/#/c/408263/? | 02:05 |
kfox1111 | sbezverk_: https://review.openstack.org/#/c/408263/ | 02:05 |
kfox1111 | hehe. | 02:05 |
portdirect | i'm watching you :) | 02:05 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 02:06 |
*** n0isyn0i1e has joined #openstack-kolla | 02:06 | |
kfox1111 | portdirect: :) | 02:06 |
sbezverk_ | kfox1111: will wait for jenkins and then ack. that is one hacking looking thing ;-) | 02:08 |
portdirect | kfox1111: thats a big hammer with the dns0 bridge :) | 02:08 |
kfox1111 | +1 for waiting. :) | 02:08 |
kfox1111 | portdirect: yeah. worked with infra on that one. :/ | 02:08 |
kfox1111 | portdirect: they did something special in their ubuntu image to add a dns cache. :/ | 02:08 |
portdirect | great /s | 02:09 |
sbezverk_ | kfox1111: while we are waiting want to ask you guys. What would be the best wait to kill the pod on configmap change? I mean I can do just using kubectl but maybe you can suggest a more elegant way? | 02:09 |
*** n0isyn0ise has quit IRC | 02:10 | |
sbezverk_ | portdirect: mgiles_: ^^^ | 02:10 |
portdirect | where are you killing it from? | 02:10 |
sbezverk_ | portdirect: from a side car container that just detected a configmap change | 02:10 |
kfox1111 | sbezverk_: I'm a bit biased there I think. I'm not convinced it should be done that way at all, as your circomventing kube's management. | 02:11 |
sbezverk_ | I have all pieces working, just need to select a weapon for suicide | 02:11 |
kfox1111 | but I can think of two ways. | 02:11 |
kfox1111 | either kubectl delete the pod. | 02:11 |
sbezverk_ | kfox1111: how about livelness? we already use it :-) | 02:11 |
kfox1111 | or kubernetes labels the pods uniquely. | 02:11 |
kfox1111 | you can use the corrisponding docker labels to kill it. | 02:11 |
kfox1111 | hmm... | 02:12 |
portdirect | kfox1111: doesnt that mean bind mounting the docker socket? | 02:12 |
kfox1111 | I guess, lets aproach it a little differntly. | 02:12 |
kfox1111 | do you want to kill the pod, or kill the container? | 02:12 |
portdirect | would need to be the whole pod | 02:12 |
sbezverk_ | kfox1111: I thing killing pod would be better | 02:12 |
portdirect | i think livleyness would be the only secure way | 02:12 |
portdirect | but would create lot of an overhead | 02:13 |
kfox1111 | hmm.. a liveleness with exec hook might do the trick. | 02:13 |
sbezverk_ | portdirect: I agree I just do not see yet how I could get my logic attach there | 02:13 |
portdirect | kubectl would be nicer but would require an access token with the ability to kill pods | 02:13 |
kfox1111 | but yeah, it would be a polling thing. | 02:13 |
kfox1111 | so would either take up resources, or be slow to notice. | 02:13 |
portdirect | sbezverk_: I'm not convinced that a sidecar is the way to go there, but would love to be proved wrong | 02:14 |
sbezverk_ | kfox1111: with rolling upgrades you are ready wait for days ;-) so what is wrong with 10 seconds delay | 02:15 |
kfox1111 | it kind of viloates the orchestration seperation I'm concerned about in the same way entrypoint does. | 02:15 |
kfox1111 | I'm ok with it optionally though. just like optional entrypoint. | 02:15 |
kfox1111 | sbezverk_: its an issue of who triggers things. | 02:15 |
kfox1111 | and what is affected when. | 02:15 |
portdirect | would this not actually be better handled by an update? if you need config change in 10 sec the manual intervention or (dare i say it) a k8s-operator may be best? | 02:16 |
kfox1111 | in a deployment way, its triggered by me upgrading a deployment. | 02:16 |
sbezverk_ | kfox1111: I look at it from different angle, kube dns uses something similar why not it can also be used in a bit different flavor | 02:16 |
kfox1111 | sbezverk_: how does kube dns do something similar? | 02:16 |
sbezverk_ | kfox1111: there is a side container health check that checks dns and if it is not up after a certain time it restarts it | 02:17 |
sbezverk_ | same thing does tiller | 02:17 |
kfox1111 | thats just a health check though. tis not used to trigger a reconfigure. | 02:17 |
sbezverk_ | it is reaction on Event | 02:17 |
sbezverk_ | it is just a different event ;-) | 02:18 |
portdirect | yeah - but two different classes of events | 02:18 |
kfox1111 | I totally believe the side car apoaach is good for that kind of thing. just not sure about configure. | 02:18 |
kfox1111 | here's the main difference. | 02:18 |
kfox1111 | in docker, they have a phlosophy. | 02:18 |
*** bachp has joined #openstack-kolla | 02:18 | |
kfox1111 | delete/replace so containers are atomic. | 02:18 |
kfox1111 | kubernetes extends that phylosophy to atomic replication controllers. | 02:18 |
kfox1111 | all the pods in an rc should be identicle. | 02:19 |
kfox1111 | deployment extends that to have two atomic sets, and old and a new. | 02:19 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 02:19 |
kfox1111 | but if you update the configmap under an rc, then you can get pods in one rc that are in two different states, | 02:19 |
kfox1111 | breaking the atomicness of the rc. | 02:19 |
portdirect | I'm in total agreement with kfox1111 here | 02:20 |
kfox1111 | depending on when a pod was launched in the set, or crached/recovered, it may be a type A pod , or a type B pod. | 02:20 |
kfox1111 | temporally. | 02:20 |
kfox1111 | the sidecar triggering rebuild is based on a phylosophy that is different then atomic rc's. | 02:21 |
sbezverk_ | kfox1111: hmm, since pods referenced the same configmap I do not see how they could be differnet | 02:21 |
kfox1111 | that may be fine. but I'm kind of fond of the phylosophy and would like to stick to it as much as reasonable. | 02:21 |
kfox1111 | sbezverk_: thats a potential issue that needs fixing. | 02:21 |
kfox1111 | with the current implementation. | 02:21 |
kfox1111 | so when configmaps are loaded in, they get a version. | 02:22 |
kfox1111 | any time its changed, it gets bumped. | 02:22 |
sbezverk_ | kfox1111: right and I am not saying it has to stay forever, but there is no good solution for anything but the deployment atm.. | 02:22 |
kfox1111 | microservices point directly at a specific version when instantiated, and get updated to trigger a pod rebuild on configmap change. | 02:22 |
sbezverk_ | so if you can guarantee that kube community will address it BEFORE we need reconfiugration then fine.. | 02:23 |
kfox1111 | sbezverk_: yeah. for now. | 02:23 |
kfox1111 | I fully expect them to get a deployment like feature for deamonsets/petsets at some point. | 02:23 |
kfox1111 | but, even doing it suboptomally, at the k8s object level still, | 02:23 |
kfox1111 | its eqiv functionality to doing it on each node with a sidecar. | 02:23 |
portdirect | sbezverk_: this is why i think an external soltion would be better - it would lower the hoverhead and allow us to perform a depolyment like rolling upgrade - distrubuting this sort of controll will be very difficult to manage | 02:23 |
kfox1111 | kubectl delete daemonset, kubectl add daemonset has the same behavior. | 02:24 |
kfox1111 | all pods get deleted, and get created again. | 02:25 |
portdirect | (I'm making assuptions here) | 02:25 |
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm creation of Nova services https://review.openstack.org/409473 | 02:25 |
sbezverk_ | portdirect: well, I get your point, but relying on external solution yo upretty much kill dynamic nature of kubernetes I think. | 02:25 |
kfox1111 | its using k8s controlelrs to do the control. | 02:25 |
kfox1111 | which I think is native k8s. | 02:25 |
portdirect | would it not be possible to thake what you have, and change it to kill the pods it targetted - rather than the pod it was in? | 02:26 |
kfox1111 | rather then add a new controller on the nodes. | 02:26 |
portdirect | (yeah by external i ment to the pod, not to k8s) | 02:26 |
kfox1111 | portdirect: it could be done with shared unix sockets, and some other logic. but woudl be really ugly. :/ | 02:26 |
portdirect | OMG no | 02:27 |
kfox1111 | sbezverk_: there's a intermediate option too... | 02:27 |
kfox1111 | not sidecar, but operator. | 02:27 |
kfox1111 | load up a watcher for configmap changes, then it deletes all pods that the configmap touches. | 02:27 |
kfox1111 | it doesn't have to be on the same host as the pod. | 02:27 |
kfox1111 | in fact, thats desirable in some ways. | 02:27 |
portdirect | ^^ this is waht i was trying to describe | 02:27 |
kfox1111 | less of an attach surface. | 02:27 |
sbezverk_ | then you need to discover those pods to kill, where in this case the biggest damage is you just kill yurself | 02:27 |
kfox1111 | the opoerator should be lableled differently then the daemonset, so shouldn't be generally possible for it to kil itself. | 02:28 |
kfox1111 | unless it has a very bad bug. :) | 02:28 |
portdirect | +1 | 02:28 |
sbezverk_ | kfox1111: no I was refering to side car comparing to operator | 02:28 |
kfox1111 | sidecar can still kilil the wrong pod if buggy. :/ | 02:29 |
kfox1111 | if it talks to k8s to kill it. | 02:29 |
*** Pavo has quit IRC | 02:29 | |
kfox1111 | could still kill other pods by mistake, at least on the same host, if uses docker to kill. | 02:29 |
kfox1111 | thats why I like using k8s itself to do that orchestration. | 02:29 |
kfox1111 | the rc controlls all the pods. so having the rc update trigger the pod delete is the cleanest. | 02:30 |
kfox1111 | as the rc is reposible for them all. single source of truth. | 02:30 |
portdirect | We should not give pods access to the docker api under any curcumstances. apart from anything it would require us to have a solution for rkt or whatever else comes along. | 02:30 |
kfox1111 | portdirect: true. | 02:31 |
sbezverk_ | portdirect: I was more inclined to use just kubectl | 02:31 |
sbezverk_ | to kill, but I guess will have to check operators | 02:32 |
kfox1111 | kubectl would be relatively clean, but gives a sidecar the power to kil things that are not itself. | 02:32 |
kfox1111 | has to talk to a more central service adding potential scalability issues, | 02:32 |
kfox1111 | and distributes credentials around. | 02:32 |
sbezverk_ | kfox1111: its pod where it runs nothing more | 02:32 |
kfox1111 | how do you restrict a kubectl account to just the pod that owns it? | 02:33 |
portdirect | sbezverk_: how do you limit it it that? | 02:33 |
kfox1111 | I don't think thats currently possible. :/ | 02:33 |
portdirect | you can limit access to read only, and by namespace only afaik? | 02:33 |
*** Jeffrey4l_ has quit IRC | 02:33 | |
kfox1111 | portdirect: you can go a bit finer grained then that. | 02:34 |
kfox1111 | but you'd hjave to make a service acoount per rc pod. | 02:34 |
*** Pavo has joined #openstack-kolla | 02:34 | |
kfox1111 | I don't think an rc can create a service account, and an rback rule that binds ito the currently launching pod. | 02:34 |
sbezverk_ | ok, I will loot into liveness hooks, cause in this case there should not be any security concernss | 02:34 |
kfox1111 | yeah. liveliness hook sounds like maybe the safest aporach there. | 02:34 |
kfox1111 | woul still need a read only kube account to pull for new configmaps. | 02:35 |
sbezverk_ | it is in case of a operator | 02:36 |
sbezverk_ | but for the side car this issue does not exist | 02:36 |
kfox1111 | how would you detect configmap changes without a k8s account? | 02:36 |
portdirect | how - it needs to poll the api to see if a new map exisits? | 02:36 |
sbezverk_ | easy, version number | 02:37 |
kfox1111 | portdirect: oh. I think the configmap stuff has a symlink and periodically updates itself. | 02:37 |
sbezverk_ | when I replace configmap it get new version | 02:37 |
kfox1111 | thats the ..data thing. | 02:37 |
kfox1111 | you could watch the symlinks. | 02:37 |
portdirect | and that propogates out to the pod? | 02:37 |
sbezverk_ | do you want me to post what I have so far? | 02:37 |
sbezverk_ | the only thing is missing if final shoot at the head | 02:38 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for Glance PV/PVC https://review.openstack.org/409491 | 02:38 |
portdirect | yeah - it would give us something to acuta;lly discuss constrctivly :) | 02:38 |
sbezverk_ | cool | 02:38 |
kfox1111 | yeah. sounds good. | 02:39 |
kfox1111 | sbezverk_: https://review.openstack.org/#/c/408263/ green. :) | 02:40 |
sbezverk_ | I will add as a file to tests/ in kolla-kubernetes | 02:40 |
kfox1111 | sbezverk_: k. sounds good. | 02:40 |
kfox1111 | portdirect: heh. thanks for the review. :) | 02:42 |
portdirect | felt I had to comment | 02:42 |
* kfox1111 chuck.es | 02:42 | |
openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: DO_NOT_MERGE PoC Configmap update monitor https://review.openstack.org/409493 | 02:45 |
*** duonghq has joined #openstack-kolla | 02:45 | |
sbezverk_ | kfox1111: done, hopefully the gate will behave itself better | 02:45 |
kfox1111 | sbezverk_: thanks. | 02:45 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/409488 | 02:46 |
openstackgerrit | Merged openstack/kolla-kubernetes: More dns gate fixes https://review.openstack.org/408263 | 02:47 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/409488 | 02:47 |
sbezverk_ | portdirect: thank you for constructive comment :-) | 02:51 |
sbezverk_ | need to get back to family life for now.. have a good evening/night or some other time of day ;-) | 02:52 |
portdirect | laters sbezverk_ :) | 02:52 |
*** duonghq has quit IRC | 02:56 | |
kfox1111 | sbezverk_: have a good one. :) | 02:58 |
*** Jeffrey4l has joined #openstack-kolla | 03:01 | |
srwilkers | noticed a problem with the common_manage_db_job template, particularly here: https://github.com/openstack/kolla-kubernetes/blob/master/helm/kolla-common/templates/_common_manage_db_job.yaml#L31 | 03:04 |
portdirect | srwilkers: needs to be service name? | 03:05 |
srwilkers | problem im having is that .resourceName is used for the name in metadata and used again for the name of the service configmap | 03:05 |
srwilkers | yep portdirect | 03:05 |
portdirect | on it | 03:05 |
srwilkers | giving me problems with cinder, since the name should be cinder, but the configmap name needs to be cinder-api | 03:05 |
srwilkers | well, cinder-manage-db, but i think you know what i mean | 03:05 |
*** v1k0d3n has quit IRC | 03:06 | |
*** v1k0d3n has joined #openstack-kolla | 03:06 | |
*** Jeffrey4l has quit IRC | 03:09 | |
openstackgerrit | Pete Birley proposed openstack/kolla-kubernetes: Fix configmap mapping in _common_manage_db_job.yaml https://review.openstack.org/409495 | 03:09 |
*** v1k0d3n has quit IRC | 03:09 | |
*** v1k0d3n has joined #openstack-kolla | 03:10 | |
portdirect | sry - my bad | 03:11 |
srwilkers | nothing to be sorry for man, it happens :) | 03:13 |
srwilkers | trying to figure out why the cinder stuffs not making it through the gate. slowly figuring things out bit by bit | 03:13 |
*** v1k0d3n has quit IRC | 03:18 | |
*** v1k0d3n has joined #openstack-kolla | 03:27 | |
*** sdake has joined #openstack-kolla | 03:29 | |
*** williamcaban has quit IRC | 03:33 | |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 03:44 |
*** fragatina has joined #openstack-kolla | 03:45 | |
*** fragatina has quit IRC | 03:53 | |
*** fragatina has joined #openstack-kolla | 03:54 | |
*** v1k0d3n has quit IRC | 03:58 | |
*** v1k0d3n has joined #openstack-kolla | 03:59 | |
*** sayantan_ has joined #openstack-kolla | 04:02 | |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services https://review.openstack.org/409487 | 04:02 |
*** williamcaban has joined #openstack-kolla | 04:04 | |
*** v1k0d3n has quit IRC | 04:06 | |
*** dave-mccowan has joined #openstack-kolla | 04:09 | |
*** dave-mcc_ has joined #openstack-kolla | 04:13 | |
*** dave-mccowan has quit IRC | 04:15 | |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 04:26 |
*** Pavo has quit IRC | 04:29 | |
*** Pavo has joined #openstack-kolla | 04:34 | |
*** dave-mcc_ has quit IRC | 04:36 | |
*** diogogmt has quit IRC | 04:38 | |
*** mgiles has quit IRC | 04:40 | |
*** sdake has quit IRC | 04:45 | |
*** portdirect is now known as portdirect_zzzzz | 04:46 | |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services https://review.openstack.org/409487 | 04:49 |
sbezverk_ | srwilkers: I saw you started to work on glance as well, do you plan to do it all? | 04:51 |
srwilkers | yeah, trying to break it into chunks to make it more manageable for reviews. have services in a commit, another commit for PV/PVC, etc | 04:52 |
srwilkers | hoping to tie the rest of it up in the morning -- wife is wanting to watch a movie :) | 04:52 |
openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: DO_NOT_MERGE PoC Configmap update monitor https://review.openstack.org/409493 | 04:53 |
srwilkers | sbezverk_, could you possibly take a look at https://review.openstack.org/#/c/409495/ ? its making things weird for cinder right now | 04:54 |
sbezverk_ | srwilkers: ok cool. let me know if you want to split it for optimization as I need glance for some experiments in very near future.. | 04:54 |
sbezverk_ | srwilkers: sure | 04:54 |
srwilkers | sbezverk_, please dont let me hold you back. if you need the other parts done quickly, im not going to tell you to not finish the others | 04:54 |
srwilkers | like i mentioned, my plan was to finish it tomorrow. if you want to get it done this evening, im not in a position to tell you no :) | 04:55 |
sbezverk_ | srwilkers: if you finish them tomorrow that would be super.. I am not goint to do anything tonight | 04:56 |
sbezverk_ | anyway | 04:56 |
sbezverk_ | srwilkers: so the cinder issue is with config map name? | 04:57 |
srwilkers | yeah. if you look at the common job here https://github.com/openstack/kolla-kubernetes/blob/master/helm/kolla-common/templates/_common_manage_db_job.yaml#L31 | 04:58 |
srwilkers | the job should be named "cinder-manage-db", but the configmaps name is "cinder-api" | 04:58 |
srwilkers | when it uses .resourceName for both, it ends up with "cinder-api-manage-db"/"cinder-api" or "cinder-manage-db"/"cinder" | 04:59 |
srwilkers | heres the old cinder bootstrap job to manage the db for reference: https://github.com/openstack/kolla-kubernetes/blob/master/services/cinder/cinder-bootstrap-job-manage-db.yml.j2 | 05:02 |
*** srwilkers is now known as srwilkers_Zzzz | 05:07 | |
*** fragatina has quit IRC | 05:31 | |
*** fragatina has joined #openstack-kolla | 05:32 | |
*** sayantan_ has quit IRC | 05:38 | |
sbezverk_ | srwilkers: yeah, it would be better if common_manage_db was just dealing with passed variable and then you the tempalte where you call it, you can assign value you want. In this case it will be more flexible. | 05:43 |
*** fragatina has quit IRC | 06:20 | |
*** sayantan_ has joined #openstack-kolla | 06:28 | |
*** Pavo has quit IRC | 06:29 | |
*** fragatina has joined #openstack-kolla | 06:32 | |
*** Pavo has joined #openstack-kolla | 06:34 | |
SamYaple | w00t | 07:09 |
SamYaple | thats AIO mariadb,rabbitmq,keystone,glance,memcached deployed. | 07:09 |
SamYaple | still working on endpoitn and service creation | 07:09 |
SamYaple | this kolla-salt thing is going much faster than i thought | 07:10 |
*** sayantan_ has quit IRC | 07:28 | |
*** fragatina has quit IRC | 07:32 | |
*** b_bezak has joined #openstack-kolla | 08:02 | |
*** b_bezak has quit IRC | 08:11 | |
*** Pavo has quit IRC | 08:29 | |
*** Pavo has joined #openstack-kolla | 08:34 | |
*** sdake has joined #openstack-kolla | 08:53 | |
*** sdake has quit IRC | 08:59 | |
*** sdake has joined #openstack-kolla | 09:00 | |
*** msimonin has joined #openstack-kolla | 09:05 | |
*** msimonin has quit IRC | 09:09 | |
*** sdake_ has joined #openstack-kolla | 09:33 | |
*** sdake has quit IRC | 09:34 | |
*** sdake_ has quit IRC | 09:41 | |
*** sdake has joined #openstack-kolla | 09:44 | |
*** jascott1_ has quit IRC | 10:07 | |
*** sdake has quit IRC | 10:09 | |
*** duonghq has joined #openstack-kolla | 10:26 | |
*** duonghq_ has joined #openstack-kolla | 10:26 | |
*** duonghq_ has quit IRC | 10:28 | |
*** Pavo has quit IRC | 10:29 | |
*** msimonin has joined #openstack-kolla | 10:33 | |
*** Pavo has joined #openstack-kolla | 10:34 | |
*** saneax-_-|AFK is now known as saneax | 10:44 | |
*** saneax is now known as saneax-_-|AFK | 10:54 | |
*** esmiurium_ has joined #openstack-kolla | 10:59 | |
*** esmiurium has quit IRC | 11:03 | |
*** jascott1_ has joined #openstack-kolla | 11:08 | |
*** msimonin has quit IRC | 11:12 | |
*** jascott1_ has quit IRC | 11:12 | |
openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Install qemu-kvm rather then qemu-system-x86 https://review.openstack.org/409513 | 11:15 |
*** zhubingbing has quit IRC | 11:15 | |
*** duonghq has quit IRC | 11:20 | |
openstackgerrit | Wei Cao proposed openstack/kolla-ansible: Add freezer ansible role https://review.openstack.org/402224 | 12:18 |
*** rsjethani has joined #openstack-kolla | 12:20 | |
rsjethani | Hi all, I am relatively new to openstack deployment and i am confused between kolla and openstack-ansible | 12:21 |
rsjethani | can anyone guide me | 12:21 |
rsjethani | thanks | 12:21 |
rsjethani | so far I have some experience with devstack | 12:22 |
SamYaple | rsjethani: what would you like to know? i have pretty extensive experince with both | 12:22 |
rsjethani | Samyaple: I want to understand the difference between the two. So they have same purpose? | 12:23 |
*** caowei_ has joined #openstack-kolla | 12:24 | |
SamYaple | broadly speaking, yes | 12:24 |
SamYaple | deploy openstack in containers | 12:24 |
rsjethani | SamYaple: then how to decide which one to choose | 12:25 |
SamYaple | i would suggest trying them both if you are unsure, but we are clearly biased here on #openstack-kolla toward kolla | 12:25 |
SamYaple | kolla is significantly faster than openstack-ansible | 12:25 |
rsjethani | But using OSA I think we can have VM based deployments also? | 12:27 |
rsjethani | not just container based... | 12:27 |
SamYaple | not natively. no. and thats not tested either | 12:27 |
rsjethani | oh. ok | 12:27 |
SamYaple | technically openstack-ansible can be deployed on "metal", on a baremetal host or a vm | 12:27 |
SamYaple | but it is only gated and really tested against their lxc implementation | 12:28 |
SamYaple | if you need baremetal deployment, give it a try. please let me know if you can get it to work! | 12:28 |
*** Pavo has quit IRC | 12:29 | |
SamYaple | openstack-ansible in containers is the only thing that we can compare with kolla | 12:29 |
rsjethani | I have some code contribution in various core projects here and there hence so far I have used devstack for single node and multi node test environments | 12:30 |
SamYaple | if you plan to do development for openstack, and are already familiar with devstack, you might apprecaite openstack-ansible more than kolla | 12:30 |
SamYaple | Kolla is a better production deployment tool, it uses imaging which allows for much better testing before moving to production | 12:31 |
rsjethani | I am quite interested in understanding simple to complex deployment | 12:31 |
SamYaple | i would suggest you give them both a try then | 12:31 |
rsjethani | I have some knowledge of ansible as well as docker | 12:31 |
SamYaple | they both support all in one deploys | 12:31 |
SamYaple | and kolla-ansible uses ansible use ilke openstack-ansible | 12:31 |
*** jascott1_ has joined #openstack-kolla | 12:31 | |
SamYaple | the openstack ansible channel is also quite active and very helpful | 12:32 |
rsjethani | I am actually looking for some replacement of devstack because many a time I have to shutdown devstack vm then upon restart I have to run stack.sh agin which takes quite sometime | 12:34 |
*** Pavo has joined #openstack-kolla | 12:34 | |
SamYaple | both have thier advantages rsjethani, i think you should try them both | 12:35 |
SamYaple | and ask opinions in the #openstack-ansible channel as well | 12:35 |
SamYaple | obviously we prefer kolla, and they prefer openstack-ansible :) | 12:35 |
rsjethani | haha right | 12:35 |
*** jascott1_ has quit IRC | 12:37 | |
rsjethani | One last question SamYaple, right now I am looking for simple lab environments where I should also be able to make changes easily, add remove services..and I should able to reboot my deployment without thinking too much. Can kolla give me what I want. Thank a lot | 12:39 |
SamYaple | rsjethani: of course. though I think openstack-ansible might be able to as well | 12:40 |
SamYaple | really anything other than devstack :) | 12:40 |
rsjethani | then why the dev community is still using devstack...that much | 12:40 |
SamYaple | rsjethani: i honestly don't know. I just don't | 12:41 |
SamYaple | i used it once and refused to use it again | 12:41 |
rsjethani | ok buddy thanks for the info | 12:41 |
SamYaple | anytime :) make sure to hit up #openstack-ansible | 12:41 |
rsjethani | ya sure :) | 12:42 |
openstackgerrit | Wei Cao proposed openstack/kolla-ansible: Change Dockerfile to ansible role in the release note. https://review.openstack.org/409517 | 12:46 |
*** caowei_ has quit IRC | 12:50 | |
*** caowei_ has joined #openstack-kolla | 12:51 | |
openstackgerrit | Wei Cao proposed openstack/kolla-ansible: Delete add-panko-dockerfile release note. https://review.openstack.org/409521 | 12:56 |
*** n0isyn0i1e has quit IRC | 13:02 | |
*** zhubingbing has joined #openstack-kolla | 13:10 | |
*** msimonin has joined #openstack-kolla | 13:11 | |
*** msimonin has quit IRC | 13:14 | |
*** msimonin1 has joined #openstack-kolla | 13:14 | |
*** msimonin1 has left #openstack-kolla | 13:14 | |
*** timss has quit IRC | 13:17 | |
*** jascott1_ has joined #openstack-kolla | 13:34 | |
*** timss has joined #openstack-kolla | 13:34 | |
*** jascott1_ has quit IRC | 13:38 | |
*** caowei_ has quit IRC | 13:59 | |
*** awiddersheim has quit IRC | 14:03 | |
openstackgerrit | Merged openstack/kolla-ansible: Enable watcher dashboard when enable_watcher is yes https://review.openstack.org/406608 | 14:22 |
*** v1k0d3n has joined #openstack-kolla | 14:22 | |
*** caowei_ has joined #openstack-kolla | 14:22 | |
*** Pavo has quit IRC | 14:29 | |
*** Pavo has joined #openstack-kolla | 14:34 | |
*** williamcaban has quit IRC | 14:38 | |
*** williamcaban has joined #openstack-kolla | 14:38 | |
*** wanghongxu_ has joined #openstack-kolla | 14:47 | |
*** srwilkers_Zzzz is now known as srwilkers | 14:52 | |
srwilkers | good morning | 14:52 |
sbezverk_ | srwilkers: good morning | 14:57 |
srwilkers | hey sbezverk_ :) | 14:58 |
kfox1111 | morning. | 15:00 |
*** yingjun has joined #openstack-kolla | 15:02 | |
SamYaple | morning | 15:03 |
srwilkers | morning kfox1111 SamYaple | 15:04 |
*** caowei_ has quit IRC | 15:05 | |
*** duonghq has joined #openstack-kolla | 15:06 | |
duonghq | evening guys | 15:08 |
*** sdake has joined #openstack-kolla | 15:09 | |
kfox1111 | duonghq: evening. :) | 15:10 |
duonghq | anybody try halcyon w/ Arch linux? | 15:10 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 15:11 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for Glance PV/PVC https://review.openstack.org/409491 | 15:13 |
*** portdirect_ has joined #openstack-kolla | 15:16 | |
*** portdirect_ is now known as portdirect | 15:16 | |
portdirect | o/ | 15:16 |
duonghq | sup portdirect | 15:17 |
*** wanghongxu_ has quit IRC | 15:17 | |
portdirect | duonghq: dont think halcyon-kubernetes will work with arch at the moment as kubeadm is not avaible for that platform? | 15:18 |
duonghq | portdirect, I'm trying | 15:18 |
portdirect | duonghq: but halcyon-vagrant-kubernetes would be fine | 15:18 |
duonghq | kubeadm shouldn't problem, | 15:18 |
*** wanghongxu_ has joined #openstack-kolla | 15:19 | |
portdirect | duonghq: oh (https://aur.archlinux.org/packages/kubeadm-git/) nice!, then with some tweaks to the ansible it should be fine :) | 15:20 |
*** awiddersheim has joined #openstack-kolla | 15:20 | |
duonghq | portdirect, I think arch Linux as host will be tricky to setup but I can | 15:21 |
openstackgerrit | prameswar proposed openstack/kolla: bifrost guide improved https://review.openstack.org/406177 | 15:21 |
sbezverk_ | kfox1111: I think we need to change common_svc a bit.. for the case when service's name and the name of svc object do not match. | 15:22 |
portdirect | been fighting getting kubeadm running on fedora/centos atomic - as it expects to be able to write to some areas that we have mounted read only, but there are some patches coming in that I hope will fix that, getting it to support as many platforms as feasibly possible would be great, so we can ensure maxium portability :) | 15:23 |
kfox1111 | sbezverk_: I'm good with that. maybe we just make it another .var and then the calling template can map it to whatever it needs to be. | 15:23 |
sbezverk_ | we can add suffix var and then service name will always be = element_name, but in metadata's name can be element_name + suffix. | 15:23 |
kfox1111 | portdirect: +1 for more support. does atomic have its own way of deploying k8s though? | 15:24 |
*** wanghongxu_ has quit IRC | 15:24 | |
kfox1111 | cause kubeadm is cool and all, but if they have a more native way, we should support that too. | 15:24 |
*** wanghongxu_ has joined #openstack-kolla | 15:24 | |
portdirect | in the context of halycon i think that that should stick to kubeadm (though it not my project) | 15:25 |
kfox1111 | sbezverk_: or we add something like .svc_name or something, and then in the parent, it can define it to be .element_name, or printf "%s-svc" .element_name or whatever makes sense to the parent? | 15:25 |
kfox1111 | portdirect: oh. true. was speaking more generally. | 15:26 |
portdirect | I've actually been doing a wee bit of work on the atmic kube deplyment tool (though I've not had time recently): https://github.com/projectatomic/commissaire | 15:26 |
portdirect | It also supports openshift, and I plan to add kolla-k8s deplyment as well | 15:26 |
kfox1111 | cool. | 15:27 |
srwilkers | sbezverk_, i actually just threw a PS together for that very reason | 15:27 |
portdirect | though thats much further down the line :) | 15:27 |
sbezverk_ | srwilkers: have a link? | 15:27 |
srwilkers | yep, one second | 15:28 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Change _common_svc.yaml to support differing service names https://review.openstack.org/409531 | 15:30 |
*** yingjun has quit IRC | 15:30 | |
sbezverk_ | srwilkers: ok yep | 15:33 |
portdirect | kfox1111: when you get a moment if you could give some feedback on my comment here it would be great: https://review.openstack.org/#/c/409479/2/helm/microservice/nova-api/templates/nova_api.yaml | 15:34 |
sdake | morning peeps | 15:34 |
sdake | I see irc channel is busy | 15:34 |
portdirect | hey sdake | 15:34 |
v1k0d3n | morning | 15:34 |
sdake | do people ever sleep? :) | 15:34 |
sdake | sup portdirect v1k0d3n | 15:34 |
v1k0d3n | :) | 15:34 |
srwilkers | yo sdake | 15:34 |
srwilkers | oh hey v1k0d3n | 15:35 |
sdake | sup srs | 15:35 |
sdake | sup srwilkers | 15:35 |
v1k0d3n | hey srwilkers | 15:35 |
srwilkers | sdake, sleep is overrated | 15:35 |
sdake | srwilkers sometimes it is :) | 15:35 |
portdirect | sdake: i think we just cnage the Iridium Power Cell's every now and again :) | 15:35 |
portdirect | *change | 15:35 |
*** jascott1_ has joined #openstack-kolla | 15:35 | |
duonghq | morning sdake | 15:38 |
sdake | sup duonghq | 15:38 |
*** rsjethani has quit IRC | 15:40 | |
*** jascott1_ has quit IRC | 15:40 | |
kfox1111 | portdirect: sure. sec | 15:43 |
Pavo | morning | 15:45 |
*** yingjun has joined #openstack-kolla | 15:45 | |
sdake | sup pavo | 15:46 |
Pavo | notta, getting my openstack env ready to do some k8s testing | 15:46 |
Pavo | found a pretty good walk through | 15:46 |
Pavo | https://keithtenzer.com/2015/06/01/application-containers-kubernetes-and-docker-from-scratch/ | 15:47 |
Pavo | and https://keithtenzer.com/2015/04/15/containers-at-scale-with-kubernetes-on-openstack/ | 15:47 |
Pavo | gonna use CentOS Atomic | 15:48 |
portdirect | Pavo: I'd also check out halycon as it supports openstack :) | 15:48 |
v1k0d3n | :) | 15:49 |
Pavo | been fooling around with Cisco CSR1000v as routers in between tenant networks also, working pretty good so far | 15:49 |
SamYaple | so for kolla-k8s is ansible being used to orchastrate? | 15:49 |
SamYaple | how are you setuping up, say, multinode galera/ | 15:49 |
Pavo | portdirect got a link for them? | 15:49 |
portdirect | Pavo: https://github.com/att-comdev/halcyon-kubernetes | 15:50 |
Pavo | thanks portdirect | 15:50 |
portdirect | Pavo, v1k0d3n would be the best person to quiz about running it on openstack | 15:50 |
srwilkers | ;) | 15:51 |
sbezverk_ | kfox1111: might be mistaken but I think there is a bug here: https://github.com/openstack/kolla-kubernetes/blob/master/helm/kolla-common/templates/_common_create_db_job.yaml#L50 | 15:51 |
Pavo | doesn't seem to hard, but will hit v1k0d3n up if I run into trouble | 15:51 |
v1k0d3n | Pavo: been a little bit since i've had to do that. worked well before, but if you run into issues...please let me or srwilkers know. | 15:51 |
v1k0d3n | there may be some things about nat and interfaces in there to consider just fyi. | 15:52 |
kfox1111 | SamYaple: no orchestration yet. no galera. | 15:52 |
Pavo | yeah thats if I use openstack native routers right? | 15:52 |
sbezverk_ | we call for common_volumes but we do not pass .Variable so in some cases logging config map gets attached to bootstrap jobs | 15:52 |
Pavo | because I am gonna use csr1000v instead | 15:52 |
v1k0d3n | and currently works with the supported systems listed in the project. need to do some things to get it to work with others (like CoreOS...which is coming...someday when i have time). | 15:52 |
kfox1111 | SamYaple: there's a concept in k8s called an operator that I think will be a good fit for galera clusters | 15:52 |
v1k0d3n | Pavo: yes, that's definitely part of it; correct. | 15:53 |
srwilkers | v1k0d3n, i can start looking at some of the other systems. i know youre strapped for time these days | 15:53 |
kfox1111 | SamYaple: it extends k8s using k8s to give k8s knowlege of how galera works. so you declaratively state you want a galera cluster, and it takes care of membership/recovery/etc for you. | 15:53 |
v1k0d3n | my first priority was making halcyon work for local devs...so it would unblock development support for kolla-kubernetes. | 15:53 |
v1k0d3n | srwilkers: you don't have to. i think it currently works for the 80% out there. | 15:54 |
v1k0d3n | the coreos is a bit tricky for kubeadm since last i checked it's not officially supported and had some wackiness. | 15:54 |
kfox1111 | sbezverk_: what do you thin kthe bug is. I'm not spotting it. | 15:55 |
kfox1111 | sbezverk_: I don't see a 'with' in that file, so its passing all of . streight through. | 15:55 |
sbezverk_ | I checked recent failure of srwilkers cinder, create db job failed because logging config map was not there | 15:55 |
kfox1111 | sbezverk_: which I think should have .Values in it. | 15:56 |
*** rsjethani has joined #openstack-kolla | 15:56 | |
sbezverk_ | kfox1111: he was bootstrap pod set to true and it is supposed to turm off attaching configmap for looging but it did not | 15:56 |
kfox1111 | hmm... | 15:57 |
portdirect | sbezverk_: got link? | 15:58 |
*** mattmceuen_away has quit IRC | 15:58 | |
kfox1111 | it tried to add the logger container? | 15:58 |
sbezverk_ | http://logs.openstack.org/49/409349/17/check/gate-kolla-kubernetes-deploy-centos-binary-ceph-multi-nv/bc59741/ | 15:58 |
sbezverk_ | nope. just logging configmap | 15:59 |
kfox1111 | so, volume, not volume_mount? | 15:59 |
sbezverk_ | http://logs.openstack.org/49/409349/17/check/gate-kolla-kubernetes-deploy-centos-binary-ceph-multi-nv/bc59741/logs/pods/kolla-cinder-create-db-3l176.txt | 15:59 |
sbezverk_ | last line in the last paste | 16:00 |
kfox1111 | right. | 16:00 |
kfox1111 | hmm... | 16:00 |
kfox1111 | so the code is this: | 16:00 |
kfox1111 | {{- $podTypeNotBootstrap := not .podTypeBootstrap }} | 16:00 |
kfox1111 | {{- if and .Values.enable_kube_logger $podTypeNotBootstrap }} | 16:00 |
sbezverk_ | the only reason this would not work if variables are not getting there | 16:01 |
portdirect | bug is here? https://review.openstack.org/#/c/409349/17/helm/microservice/cinder-create-db/templates/cinder_create_db.yaml | 16:01 |
sbezverk_ | kfox1111: he does not explicitely disables logger so one condition is truee | 16:01 |
kfox1111 | sbezverk_: havent seen this in any of my jobs though... | 16:01 |
portdirect | line 4, should be similar to: https://github.com/openstack/kolla-kubernetes/blob/master/helm/microservice/keystone-create-db/templates/keystone_create_db.yaml#L4 | 16:02 |
kfox1111 | oh. | 16:02 |
sbezverk_ | kfox1111: I agree it is really strange | 16:02 |
kfox1111 | yeah. I've been explicitly disabling it. | 16:02 |
kfox1111 | so, its not an issue with .Values then. | 16:02 |
portdirect | its not passing the podTypeBootstrap, through | 16:02 |
kfox1111 | its an issue with podTypeBootstrap. | 16:02 |
kfox1111 | yeah. | 16:02 |
srwilkers | shit yeah, good catch portdirect. completely passed that over | 16:02 |
kfox1111 | do you have a link to the review? | 16:02 |
*** rsjethani has quit IRC | 16:03 | |
portdirect | kfox1111: https://review.openstack.org/#/c/409349 | 16:04 |
kfox1111 | portdirect: good catch. that was what I was going to look at next. you beet me to it. :) | 16:04 |
portdirect | Just trying to catch up with srwilkers, thats guys a reviewing machine :) | 16:05 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 16:05 |
*** zhubingbing has quit IRC | 16:05 | |
Pavo | if I am using ceph as storage do I have to enable cinder, glance and nova to use ceph as a backend? | 16:06 |
kfox1111 | I usually enable cinder and glance and cinder support in nova, but don't back it with ceph. | 16:07 |
*** sdake has quit IRC | 16:07 | |
*** msimonin has joined #openstack-kolla | 16:08 | |
*** yingjun has quit IRC | 16:08 | |
Pavo | ok so in globals I only need to have cinder, glance enabled but do not have to enable glance_backend_ceph: or cinder_backend_ceph: | 16:08 |
*** yingjun has joined #openstack-kolla | 16:08 | |
Pavo | the reason I am asking I am not able to upload images in horizon after about 4 images | 16:09 |
kfox1111 | I don't remember the exact vars to set. see kolla-kubernetes/tests/conf/* | 16:09 |
Pavo | this is kolla-ansible | 16:09 |
Pavo | no kolla-k8s | 16:09 |
Pavo | not | 16:09 |
kfox1111 | Pavo: kolla-kubernetes uses kolla-ansible's config at the moment. | 16:10 |
kfox1111 | so same thing. | 16:10 |
Pavo | ah | 16:10 |
*** yingjun has quit IRC | 16:12 | |
Pavo | my ceph partitions have 30Tb so I am sure its not a limit I am running into | 16:12 |
kfox1111 | yeah. sounds like its not using ceph. | 16:15 |
kfox1111 | the config bits can be a little tricky to figure out. | 16:15 |
kfox1111 | Pavo: https://github.com/openstack/kolla-kubernetes/blob/master/tests/conf/ceph-all-in-one/kolla_config | 16:16 |
kfox1111 | look for the ceph or rbd optionsn. | 16:16 |
srwilkers | kfox1111, could you take a gander at https://review.openstack.org/#/c/409531/ and possibly workflow it? its re: the _common_svc discussion earlier | 16:16 |
Pavo | well only thing I am doing before a deployment is enabling ceph in gloabls, then using parted $DISK -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1 to label the drives I want to use with Ceph | 16:16 |
kfox1111 | srwilkers: looks good to me. | 16:19 |
kfox1111 | workflowed | 16:19 |
*** sdake has joined #openstack-kolla | 16:20 | |
kfox1111 | Pavo: yeah. I think you need the other vars to get the services to use it. | 16:20 |
Pavo | ok will try | 16:21 |
srwilkers | thanks kfox1111 :) | 16:21 |
openstackgerrit | Merged openstack/kolla-kubernetes: Change _common_svc.yaml to support differing service names https://review.openstack.org/409531 | 16:23 |
kfox1111 | srwilkers: thanks for working on it. :) | 16:23 |
*** duonghq has quit IRC | 16:24 | |
kfox1111 | you did all the work. I just pulled the trigger. :) | 16:24 |
openstackgerrit | Merged openstack/kolla-kubernetes: Helm for nova-compute https://review.openstack.org/409482 | 16:28 |
*** Pavo has quit IRC | 16:29 | |
*** Pavo has joined #openstack-kolla | 16:30 | |
sbezverk_ | kfox1111: did you know that pod restart does not call for pod init container? | 16:31 |
sbezverk_ | and what is more strange, pod restart does not clean emptydir volumes | 16:32 |
kfox1111 | through what restart mechanism? | 16:32 |
kfox1111 | liveness hooks? | 16:32 |
kfox1111 | I would expect those two behaviors as a set. so at least it is consistent. | 16:33 |
sbezverk_ | liveness | 16:33 |
sbezverk_ | hook | 16:33 |
portdirect | sbezverk_: is the pod restarted or just the container? | 16:33 |
kfox1111 | it would be really bad if it didn't rerun the inits but cleared out the volumes. | 16:33 |
sbezverk_ | portdirect: pod restart | 16:33 |
kfox1111 | sbezverk_: does it rebuild the containers or restart them? | 16:34 |
kfox1111 | (do their docker id's change) | 16:34 |
sbezverk_ | kfox1111: I think so but will double check | 16:34 |
kfox1111 | I've never dug into the details. thanks for sharing. :) | 16:35 |
portdirect | the /pause container wont change afaik? I think its just going to restart the containers in the pod, but not acutally rebuild it. | 16:35 |
kfox1111 | I wouldn't quite have expected that behavior. | 16:35 |
kfox1111 | portdirect: yeah, I'd think pause would stick around so the ip doesn't change. | 16:36 |
kfox1111 | if its not rerunning ip/volume chnages. | 16:36 |
kfox1111 | rerunning init | 16:36 |
*** jascott1_ has joined #openstack-kolla | 16:37 | |
kfox1111 | jascott1_: hi | 16:37 |
*** kamtamtun has joined #openstack-kolla | 16:38 | |
kfox1111 | portdirect: I strted another pass at reviewing your stuff. but most if it is in merge conflicts. :/ | 16:38 |
sbezverk_ | portdirect: hmmmm kubectl shows pod restart, but after checking container ID it seems it restarts only container where liveness check exists :-( | 16:38 |
kfox1111 | hmm... so its a container liveness check, not a pod livenes check. interesting. | 16:39 |
portdirect | kfox1111: yeah - I'm gonna have to head out in 20 to move some suff at a freinds house but will get them resolved as soon as i get back | 16:39 |
kfox1111 | thats at least consistent. | 16:39 |
sbezverk_ | kfox1111: exactly, this is big surprise for me | 16:39 |
kfox1111 | portdirect: ok cool. I'll try to review as soon as I can. | 16:39 |
portdirect | np - will need to toally redo the keystone api (add apache_common) | 16:40 |
kfox1111 | portdirect: yeah. the apache common one should be pretty simple I think. just the other common one stripped of all the haproxy/second container stuff. | 16:41 |
portdirect | +1 | 16:41 |
kfox1111 | and the prestop hoook changed to apachectl shutdown or something like that. | 16:41 |
*** jascott1_ has quit IRC | 16:41 | |
portdirect | kfox1111: good point - though thats differnt in ubuntu/cent afaik - fun times :) | 16:42 |
kfox1111 | hehe. yeah. | 16:42 |
kfox1111 | ideally, kolla should provide that api in the container api. | 16:43 |
kfox1111 | "prestop hook" | 16:43 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services https://review.openstack.org/409487 | 16:43 |
kfox1111 | but for now, yeah, we can just do a bit of logic to determine what its named and call that. | 16:43 |
portdirect | +1 to that - is there a convention on how we doo OS detection? I assum just checking /etc/os-release? | 16:43 |
kfox1111 | I'd maybe look for the binary names in question? | 16:44 |
kfox1111 | apachectl or whatever it gets renamed to? | 16:44 |
portdirect | doh - much better :) | 16:44 |
openstackgerrit | Pete Birley proposed openstack/kolla-kubernetes: Fix configmap mapping in _common_manage_db_job.yaml https://review.openstack.org/409495 | 16:45 |
portdirect | ^^ can that get workflowed? think it's holding up srwilkers. | 16:46 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for Glance PV/PVC https://review.openstack.org/409491 | 16:46 |
*** kamtamtun has quit IRC | 16:46 | |
kfox1111 | portdirect: as soon as it passes the gate, I'l wf it. | 16:47 |
portdirect | nice one - catch you cats in a bit - I'm off to play moving houses | 16:47 |
*** portdirect is now known as portdirect_away | 16:47 | |
*** sayantan_ has joined #openstack-kolla | 16:49 | |
*** adrian_otto has joined #openstack-kolla | 16:57 | |
sdake | sounds painful portdirect_away | 16:57 |
sdake | off to do more tech support for my family - bbl | 17:00 |
*** srwilkers is now known as srwilkers_away | 17:13 | |
openstackgerrit | Merged openstack/kolla-kubernetes: Fix configmap mapping in _common_manage_db_job.yaml https://review.openstack.org/409495 | 17:15 |
SamYaple | kfox1111: sounds interesting | 17:26 |
*** adrian_otto has quit IRC | 17:30 | |
*** rsjethani has joined #openstack-kolla | 17:36 | |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database https://review.openstack.org/409538 | 17:36 |
*** wanghongxu_ has quit IRC | 17:38 | |
*** sdake has quit IRC | 17:41 | |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 17:44 |
*** hogepodge has joined #openstack-kolla | 17:50 | |
*** msimonin has quit IRC | 17:51 | |
*** msimonin has joined #openstack-kolla | 18:00 | |
*** pbourke has quit IRC | 18:03 | |
*** pbourke has joined #openstack-kolla | 18:04 | |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 18:10 |
Pavo | anyone have a clue why I keep getting this error? | 18:13 |
Pavo | http://paste.openstack.org/show/592051/ | 18:13 |
Pavo | seems like a python issue to me | 18:13 |
sbezverk_ | srwilkers: ping | 18:18 |
*** sdake has joined #openstack-kolla | 18:21 | |
*** Pavo has quit IRC | 18:29 | |
*** Pavo has joined #openstack-kolla | 18:30 | |
SamYaple | Pavo: is your disk full? | 18:33 |
*** John__ has joined #openstack-kolla | 18:40 | |
kfox1111 | SamYaple: do you release stuff to the hub? | 18:41 |
kfox1111 | there's two containers now on the hub that need updating. | 18:42 |
SamYaple | kfox1111: no. kolla has never done a good job of that | 18:42 |
SamYaple | that is one thing ive always wanted the team to focus on for stability... | 18:43 |
kfox1111 | alternately know when 3.0.2 will be cut? | 18:43 |
SamYaple | production and testing | 18:43 |
kfox1111 | SamYaple: +1 | 18:43 |
SamYaple | i know nothing kfox1111. im just a visitor here | 18:43 |
sdake | kfox1111 z streams are supposed to be cut on a 45 day schedule | 18:43 |
sdake | whether that is happening or not, i don't know | 18:44 |
sdake | the 45 day schedule lines up with the previous 6 month schedule of openstack | 18:44 |
sdake | unfortunately the schedule has chnaged to 4 months for ocata | 18:44 |
sdake | so that has thrown everything off related to zstream backport releases | 18:44 |
kfox1111 | the hub just says "1 month ago". :/ | 18:44 |
kfox1111 | so don't know. | 18:44 |
kfox1111 | yeah. | 18:44 |
kfox1111 | I've got a review that works around it, but does a pip install in the startup which is really ugly. | 18:45 |
sdake | startup of which? | 18:45 |
kfox1111 | sdake: https://review.openstack.org/#/c/409488/3/helm/kolla-common/templates/_common_create_keystone_endpoint.yaml line 25. | 18:46 |
kfox1111 | I really would rther not do that. | 18:46 |
kfox1111 | but we should cut an updated hub container then. | 18:46 |
sdake | first we have to cut a zstream tag ;) | 18:46 |
sdake | jeffrey4l is the relese liasion for ocata | 18:47 |
kfox1111 | sdake: thats what bugs me for this whole z stream thing. | 18:47 |
kfox1111 | it doesn't change any code. | 18:47 |
SamYaple | i seem to recall a discussion about the z stream being different from docker image versions | 18:47 |
kfox1111 | if you do the same thing a second time, you get a newer pip package and it all just starts working. :/ | 18:47 |
SamYaple | as in, we related a 3.0.2-1 and a 3.0.2-2 | 18:47 |
kfox1111 | its a revision, not a new version. | 18:47 |
kfox1111 | SamYaple: yeah. exactly. | 18:48 |
SamYaple | that was a plan at one point kfox1111 | 18:48 |
sdake | SamYaple yup someone wnated to do that | 18:48 |
sdake | but nobody ever did | 18:48 |
SamYaple | well we cant until we make image building better | 18:48 |
kfox1111 | I'd be willing to do the work. | 18:48 |
SamYaple | kfox1111: im sure you can get repo access | 18:48 |
kfox1111 | on getting the scripts kergiggered around to allow it. | 18:48 |
SamYaple | not alot of people fighting for it | 18:48 |
sdake | kfox1111 you already have docker hub push access | 18:48 |
kfox1111 | SamYaple: I have access. but been reluctant to use it, as I'm not a kolla core. | 18:48 |
kfox1111 | and I'm not too familior with the build proccess yet. | 18:49 |
SamYaple | just discuss it in the next meeting so everyone is aware | 18:49 |
SamYaple | oh | 18:49 |
sdake | SamYaple agree - seems like a good agenda item | 18:49 |
SamYaple | well, get familiar first :P | 18:49 |
kfox1111 | +1 for more discussion. | 18:49 |
kfox1111 | yeah. was wondering if someone more familior with the hub could do this one, and then it gives me time to spin up for the next. | 18:49 |
SamYaple | afaik, they still ahvent fixed the primary issue | 18:50 |
kfox1111 | whats that? | 18:50 |
SamYaple | so it takes like 6 hours to upload even on gigabit connections | 18:50 |
SamYaple | you have to push every layer for every image when pushing to dockerhub | 18:50 |
SamYaple | its like 100GB for all images | 18:50 |
kfox1111 | I thought the sha stuff they did fixed that? | 18:50 |
sdake | kfox1111 here is the agenda that jeffrey4l is running from until inc0 gets back: https://wiki.openstack.org/wiki/Meetings/Kolla | 18:50 |
Pavo | SamYaple not at all, have 30Tb free | 18:50 |
Pavo | but I think it might be my images | 18:51 |
SamYaple | kfox1111: it was fixed for private registry | 18:51 |
Pavo | so I am gonna just pull the offical kolla images | 18:51 |
*** sayantan_ has quit IRC | 18:51 | |
Pavo | once I pull the offical kolla images how do I push them to my private registry? | 18:51 |
kfox1111 | SamYaple: hmm.. k. | 18:51 |
openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Adds type label to service selector https://review.openstack.org/409543 | 18:51 |
kfox1111 | SamYaple: would one of the other pubilc repo's be a better fit then? | 18:51 |
sbezverk_ | kfox1111: to match with old appraoch of service selection, I added type label. | 18:52 |
SamYaple | kfox1111: supposedly openstack infra was goign to get a registry | 18:52 |
sbezverk_ | kfox1111: see ps ^^^ | 18:52 |
SamYaple | kfox1111: but idk where monty is at with that. he kinda overreaches sometimes | 18:53 |
sdake | nah infra isn't setting up a registry tmk | 18:53 |
kfox1111 | SamYaple: I think thatas just for gating, not for releasing? | 18:53 |
SamYaple | sdake: monty started at one point, but didnt keep on it i dont think | 18:53 |
SamYaple | kfox1111: nope, that would have ebeen for releasing | 18:53 |
SamYaple | kfox1111: well for both | 18:53 |
kfox1111 | SamYaple: ah. ok. I'm down with that then. | 18:54 |
kfox1111 | if its stalled, maybe we can help. | 18:54 |
Pavo | SamYaple if I pull offical kolla images instead of building my own how do I push them into my private registry? | 18:54 |
SamYaple | Pavo: tag them as your repo | 18:54 |
kfox1111 | but I d on't really have much preference where it lands, so long as its scalable so lots of people can pull from it. | 18:54 |
kfox1111 | so like, the hub, google's or quay | 18:55 |
SamYaple | kfox1111: thats probably dockerhub or quay | 18:55 |
SamYaple | give dockerhub a try, see if they fixed the upload-same-layer thing | 18:55 |
SamYaple | if not, its alot of data to push up | 18:55 |
kfox1111 | yeah. | 18:56 |
kfox1111 | k. | 18:56 |
*** srwilkers_away is now known as srwilkers | 18:56 | |
SamYaple | k sleep time | 18:56 |
kfox1111 | nite | 18:56 |
srwilkers | sbezverk_, just noticed your ping. was away | 19:00 |
Pavo | if I did a kolla-ansible pull only why does my images look like this? http://paste.openstack.org/show/592052/ | 19:01 |
Pavo | should they not have my registry info in them? | 19:01 |
kfox1111 | sbezverk_: looks good to me. | 19:01 |
sbezverk_ | srwilkers: np I just pushed type | 19:01 |
sbezverk_ | addition to svc | 19:02 |
srwilkers | okay yeah | 19:03 |
srwilkers | that makes sense. thats what i meant when i replied about having a type. meant a type in the service selector | 19:04 |
srwilkers | guess i shouldve been more clear | 19:04 |
sbezverk_ | srwilkers: that was not you, it was me.. | 19:06 |
sbezverk_ | srwilkers: needed to confirm wit halready running test bed.. | 19:06 |
srwilkers | :) | 19:08 |
*** v1k0d3n has quit IRC | 19:13 | |
*** larsks has joined #openstack-kolla | 19:16 | |
sbezverk_ | kfox1111: please ack when you have a chance: https://review.openstack.org/409543 | 19:21 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database https://review.openstack.org/409538 | 19:22 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 19:24 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database https://review.openstack.org/409538 | 19:27 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 19:28 |
*** diogogmt has joined #openstack-kolla | 19:32 | |
Pavo | is this the correct way to tag the official kolla images to be able to push them into my private registry? docker tag b46dd1990bc1 deployer.local:4000/kolla/centos-binary-mariadb | 19:35 |
*** jmccarthy has quit IRC | 19:37 | |
*** jmccarthy has joined #openstack-kolla | 19:38 | |
*** mgiles has joined #openstack-kolla | 19:39 | |
*** jascott1_ has joined #openstack-kolla | 19:39 | |
*** jascott1_ has quit IRC | 19:44 | |
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm for nova-libvirt https://review.openstack.org/409481 | 19:45 |
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm database creation for Nova https://review.openstack.org/409474 | 19:49 |
*** rsjethani has quit IRC | 19:51 | |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database https://review.openstack.org/409538 | 19:55 |
*** portdirect_away is now known as portdirect | 19:58 | |
portdirect | o/ | 19:58 |
portdirect | hey just been going trhough a few pr's and realised somthing, what release are the 2.0.2 images? | 19:59 |
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm for nova-api https://review.openstack.org/409479 | 19:59 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database https://review.openstack.org/409538 | 20:00 |
srwilkers | hey portdirect o/ -- im honestly not sure | 20:01 |
*** jascott1_ has joined #openstack-kolla | 20:02 | |
portdirect | hey srwilkers, just checked, mitaka i think, should we not be using newton at least? | 20:04 |
*** diogogmt has quit IRC | 20:05 | |
*** jascott1_ has quit IRC | 20:07 | |
*** msimonin1 has joined #openstack-kolla | 20:11 | |
*** msimonin has quit IRC | 20:12 | |
*** portdirect has quit IRC | 20:15 | |
*** John__ has quit IRC | 20:26 | |
*** Pavo has quit IRC | 20:29 | |
*** Pavo has joined #openstack-kolla | 20:30 | |
*** portdirect has joined #openstack-kolla | 20:35 | |
openstackgerrit | Merged openstack/kolla-kubernetes: Helm for nova-libvirt https://review.openstack.org/409481 | 20:41 |
sbezverk_ | portdirect: it is mitaka, newton is 3.0.X | 20:44 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database https://review.openstack.org/409538 | 20:54 |
portdirect | sbezverk_: I dont think it should make too much diference at the moment (it was mgiles's work nova db migration that made me think - as i think the api one came in with mitaka.) but we should prob make it a prority to shift over soon as all the microservices are done? | 20:54 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for Glance PV/PVC https://review.openstack.org/409491 | 20:55 |
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm creation of Nova services https://review.openstack.org/409473 | 20:56 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 20:57 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services https://review.openstack.org/409487 | 20:58 |
sbezverk_ | portdirect: if it were up to me, I would use master ;-) but Kevin suggested to complete all microservices on 2.0.2 and then switch all to 3.0.1 or 3.0.2. | 20:58 |
portdirect | sbezverk_: master would be what I'd want too | 20:58 |
srwilkers | sbezverk_, ah. should we move the image_tags to 2.0.2 for consistency's sake there, or keep running them with 3.0.1? | 20:59 |
sbezverk_ | portdirect: where is fun in life if you cannot fight with bleeding edge bugs ;-) | 20:59 |
portdirect | srwilkers: I would move backward if you've made the shift already | 20:59 |
srwilkers | portdirect, yeah, ive been using 3.0.1 for the image_tag | 21:00 |
sbezverk_ | srwilkers: I do not think it is needed to change alreafy working services back to 2.0.2 | 21:00 |
Pavo | ok I need help, I have redeployed evening using the official kolla images and keep getting this error when trying to upload a image in horizon http://paste.openstack.org/show/592055/ | 21:00 |
srwilkers | sbezverk_, none of my have been merged in yet, so doing 2.0.2 isnt a big deal | 21:00 |
Pavo | that is from horizon.log | 21:00 |
portdirect | sbezverk_: why else do you think I'd run fedora rawhide? | 21:00 |
sbezverk_ | srwilkers: in this case why not ;-) | 21:00 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for creating and managing the Glance database https://review.openstack.org/409538 | 21:02 |
Pavo | anyone have a clue why that error is happening? | 21:03 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 21:04 |
portdirect | Pavo: what version of horizon? | 21:04 |
Pavo | what ever version is in the official kolla 3.0.1 image | 21:04 |
Pavo | I pulled all official kolla images | 21:05 |
Pavo | any clue portdirect ? | 21:08 |
portdirect | off the top of my head that looks like a horizon, not kolla bug | 21:11 |
Pavo | looks like python issues to me | 21:11 |
portdirect | how big was the image? | 21:12 |
kfox1111 | sbezverk_: done | 21:13 |
*** jascott1_ has joined #openstack-kolla | 21:13 | |
sbezverk_ | kfox1111: thanks | 21:13 |
kfox1111 | np | 21:14 |
kfox1111 | one of the things still todo is to get the versioning stuff streightened out, and add a 3.0.x set to the gate too. | 21:16 |
kfox1111 | so we can support both sets. | 21:16 |
openstackgerrit | Merged openstack/kolla-kubernetes: Adds type label to service selector https://review.openstack.org/409543 | 21:16 |
portdirect | cant we just dump 2? | 21:16 |
kfox1111 | that will make it easier to test upgrades when we're ready. | 21:16 |
kfox1111 | I'm kind of hoping to support 2 versions at a time to make upgrades easier? | 21:17 |
kfox1111 | so when ocata cuts, we drop 2.0.x? | 21:17 |
portdirect | or get a master gate? | 21:17 |
kfox1111 | yeah. a master woudl be nice. | 21:18 |
kfox1111 | more work then just switching out 2.0.2 for 3.0.1 though from the ub. | 21:18 |
kfox1111 | hub | 21:18 |
kfox1111 | master needs to get done at some point too though. for sure. | 21:18 |
portdirect | then we would be n and n+1, I't would be great if we could have a master in the hub as well? | 21:19 |
kfox1111 | on the fence on that one. I think it may change faster then might be a good fit on the hub? | 21:19 |
kfox1111 | I think with your docker save trick though, we can artefact trunk builds for the gate on tarballs.openstack.org | 21:19 |
kfox1111 | at least. | 21:19 |
kfox1111 | trove does a similar thing here: https://tarballs.openstack.org/trove/images/ubuntu/ | 21:20 |
portdirect | yeah - I could donate bandwidth for pushing to the hub if that was an issue - but lets get the thing working - then work out how to upgrade :) | 21:21 |
sbezverk_ | I agree about master. it helps to be prepared and move forward | 21:21 |
kfox1111 | so they have a periodic job in zuul that just builds the images, does some sanity checks and pushes to that location. | 21:21 |
kfox1111 | then thier othr jobs that test againt master pull from there. | 21:21 |
kfox1111 | how about n and n-1 until we get master, then we swithc to n, n+1 after we get master? | 21:22 |
portdirect | sweet - if we could do that that would be great - would it be easy to trigger a daily build ? | 21:22 |
kfox1111 | portdirect: I think so. trove seems to be agood template to copy from. | 21:22 |
kfox1111 | I just havent' had time to fully experiment with the idea yet. | 21:22 |
kfox1111 | I did talk with infra a bit about it though. | 21:22 |
*** kollabot5 has joined #openstack-kolla | 21:23 | |
*** kollabot has quit IRC | 21:23 | |
*** yee379 has quit IRC | 21:23 | |
*** yee379 has joined #openstack-kolla | 21:23 | |
kfox1111 | they were cool with the idea, as other s are already doing it. | 21:24 |
kfox1111 | see project-config//jenkins/jobs/trove.yaml, job template 'gate-trove-integration-buildimage-{datastore}{job-suffix}' | 21:24 |
kfox1111 | if you are curious. | 21:24 |
portdirect | nice - lets chat about this later - just a bit concerned that we are developing for the last version - but not that concerned yet, just means that we are lagging behind other projects for both release, and k8s version | 21:24 |
kfox1111 | +1. | 21:25 |
*** msimonin1 has quit IRC | 21:25 | |
kfox1111 | once I get neutron done, versions were kind of next on my todo list. | 21:25 |
kfox1111 | it would be easier to go to 3.0.1 only too though if we get fully to helm. | 21:25 |
kfox1111 | so we ca nget to 3.0.1 as ssoon as either one of those happens. | 21:25 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services https://review.openstack.org/409487 | 21:27 |
*** pomac has quit IRC | 21:28 | |
kfox1111 | so, what do you guys think about doing something similar to the values.yaml for Charts? | 21:32 |
kfox1111 | we could make an all_charts.yaml. | 21:33 |
portdirect | only realy think the version would need to be changed/templated in charts, unless I'm missing somthing? | 21:34 |
kfox1111 | or, we could do a _Chart.yaml instead of Chart.yaml, in each dir, and then have prebuild use it to create Chart.yaml? | 21:34 |
kfox1111 | portdirect: yah. thats my thinking. the version should be easily changable. | 21:34 |
sbezverk_ | kfox1111: ideally helm should take care of that | 21:34 |
kfox1111 | sbezverk_: ideally yeah. | 21:35 |
kfox1111 | sbezverk_: no way to do that currently. :/ | 21:35 |
portdirect | seems a bit heavy duty for that? im kinda -1 at this stage, but would need to have a think | 21:35 |
kfox1111 | portdirect: ok, let me ask a little differnt way. | 21:35 |
kfox1111 | if we were to support 2.0.2 packages, and 3.0.1 packages, | 21:35 |
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm for installing other nova services https://review.openstack.org/409486 | 21:36 |
kfox1111 | I think it would be best to have the version of the helm packages reflect that. so neutron-server-2.0.2-1 and neutron-server-3.0.1-1 ? | 21:36 |
portdirect | it would pretty easy to make prebuild.py parse and modify the value as required wouldnt it? I'd kinda perfer that off the top of my head - then each chart would use 'master' as the version untill we cut a release? | 21:36 |
kfox1111 | portdirect: yeah. but if it edits Chart.yaml and its checked in, then it will be kind of a pain for development I think. | 21:37 |
*** dave-mccowan has joined #openstack-kolla | 21:37 | |
portdirect | thats why it would be a release only option i think? | 21:37 |
portdirect | my concern is beasue it takes us further away from helm 'native' | 21:38 |
kfox1111 | oh. like, helm prebuild doesn't touch charts.yaml unless you need to change versions? | 21:38 |
portdirect | yeah | 21:38 |
kfox1111 | some flag to helm_prebuild? | 21:38 |
kfox1111 | hmm.. | 21:38 |
kfox1111 | yeah. that would work. good idea. :) | 21:38 |
portdirect | sweet | 21:39 |
openstackgerrit | Merged openstack/kolla-kubernetes: Helm for nova-api https://review.openstack.org/409479 | 21:39 |
sbezverk_ | kfox1111: so we checkin version independent bits, and then what will make them prepped for a specific version? | 21:40 |
kfox1111 | I guess we check in only one specific verson and then allow it to be easily overriden. | 21:41 |
sbezverk_ | prepped I mean Chart.yaml get overridden for a requested version | 21:41 |
kfox1111 | with that model. | 21:41 |
kfox1111 | yeah. | 21:42 |
sbezverk_ | getting helm to support vars in Charts.yaml would solve so much of headache | 21:45 |
kfox1111 | sbezverk_: +1 | 21:46 |
kfox1111 | values too. | 21:46 |
kfox1111 | I'm sure they will get there. | 21:46 |
kfox1111 | and if not, we can help. :) | 21:47 |
kfox1111 | does anyone test the containers pushed to the hub before they are pushed? | 21:49 |
*** dave-mccowan has quit IRC | 21:49 | |
*** v1k0d3n has joined #openstack-kolla | 21:49 | |
openstackgerrit | Merged openstack/kolla-kubernetes: Helm charts for Glance PV/PVC https://review.openstack.org/409491 | 21:50 |
openstackgerrit | Mark Giles proposed openstack/kolla-kubernetes: Helm database creation for Nova https://review.openstack.org/409474 | 21:50 |
sbezverk_ | kfox1111: do you mean from kolla side right? | 21:50 |
*** williamcaban has quit IRC | 21:51 | |
kfox1111 | sbezverk_: yeah. | 21:51 |
portdirect | i hope they are - one of the major issues I've heard about kolla in the past is the lack of good images in the hub | 21:51 |
kfox1111 | portdirect: kolla thus far has considered them as a convenience, not something to be used for anything other then testing with. | 21:52 |
kfox1111 | thers probably a bunch more infra needed to be put in place to make it production ready. | 21:52 |
portdirect | kfox1111: yeah - but users dont expect that :) | 21:52 |
kfox1111 | agreed. | 21:52 |
kfox1111 | just wondering if I cut a 3.0.1-2 release could we just upload it, or should more testing be done first? | 21:53 |
kfox1111 | (for the toolbox) | 21:53 |
*** rstarmer has quit IRC | 21:53 | |
kfox1111 | wana get the neutron entrypoint stuff up but really fell uneasy about the pip thing in it. | 21:53 |
portdirect | perhaps we should get a kolla-dev or similar namespace on the hub? | 21:53 |
kfox1111 | hmm. that may be possible. | 21:54 |
kfox1111 | but without some mechanism to keep it up to date, may und up in the same state the kolla/ namespace has. | 21:54 |
portdirect | thought these debates all become moot if we get a registry via infra | 21:55 |
portdirect | was more thinking that it would allow you to push at will - if it was clear that it would be davelopment images | 21:55 |
portdirect | (ie no qa) | 21:56 |
kfox1111 | yeha. true. solvable with infra registry too. | 21:56 |
kfox1111 | just wondering how to fix the existing release, if thats a good thing to do. | 21:57 |
*** pomac has joined #openstack-kolla | 21:57 | |
sbezverk_ | kfox1111: I think it is a good thing, but I doubt it will be accepted as it brings inconsistency in versioning | 21:59 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 22:00 |
kfox1111 | yeah. I'm just not sure. the existing versioning has always thus far been about the code that uses to build the thing. | 22:00 |
kfox1111 | and none of that changes. so its still 3.0.1, just a dep gets updated. | 22:00 |
kfox1111 | but I do see your point. | 22:01 |
kfox1111 | which is why I was asking. | 22:01 |
kfox1111 | others may feel the same. | 22:01 |
sbezverk_ | kfox1111: some fixes can be backported and get published in z stream | 22:01 |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for glance services https://review.openstack.org/409487 | 22:02 |
kfox1111 | yeah. asked when that may be too. no one's sure. :/ | 22:02 |
kfox1111 | wiht ocata being a short cycle, it may never happen. | 22:02 |
kfox1111 | the other alternative is making a new package. | 22:05 |
kfox1111 | we could make an openstack-cli container. | 22:05 |
kfox1111 | that may have other benifits. | 22:06 |
sbezverk_ | kfox1111: idk too many toolboxes ;-) | 22:09 |
kfox1111 | hehe. true. | 22:10 |
kfox1111 | I've wanted a general openstack-cli container for even my non containerd clouds though. | 22:10 |
kfox1111 | makes it easy to for users to get cli tools without doing a bunch of rpms or debs or whatever. | 22:10 |
kfox1111 | the opnestack cli has a lot of deps. | 22:11 |
sbezverk_ | kfox1111: right I remember we talked about that before and I think kolla-toolbox was supposed to address that | 22:11 |
kfox1111 | yeah. not sure what the kolla-toolbox was exactlyu intended for. seems like a catch all container. | 22:12 |
sbezverk_ | what if we open PS on kolla side and then ask to get it backported to stable newton | 22:12 |
kfox1111 | has ansible in it too. | 22:12 |
kfox1111 | could try. if it was prurely a new container, it might be easier to convince folks. but could be rejected as a feature too. | 22:13 |
kfox1111 | that kind of why I liked theidea of just fixing the hub container, as it doesnt' requrie any changes at all to fix. | 22:13 |
sbezverk_ | kfox1111: the argument will be how people who downloaded this container before the change, know that there was a change? | 22:15 |
*** rstarmer has joined #openstack-kolla | 22:20 | |
*** bmace has quit IRC | 22:20 | |
*** sdake_ has joined #openstack-kolla | 22:20 | |
*** bmace has joined #openstack-kolla | 22:20 | |
*** rstarmer has quit IRC | 22:21 | |
*** rstarmer has joined #openstack-kolla | 22:21 | |
*** rstarmer has quit IRC | 22:22 | |
*** rstarmer has joined #openstack-kolla | 22:22 | |
*** rstarmer has joined #openstack-kolla | 22:23 | |
*** rstarmer has quit IRC | 22:23 | |
*** rstarmer has joined #openstack-kolla | 22:24 | |
*** rstarmer has quit IRC | 22:24 | |
*** sdake has quit IRC | 22:24 | |
*** rstarmer has joined #openstack-kolla | 22:24 | |
*** rstarmer has quit IRC | 22:25 | |
*** rstarmer has joined #openstack-kolla | 22:25 | |
*** williamcaban has joined #openstack-kolla | 22:25 | |
*** rstarmer has quit IRC | 22:26 | |
*** Pavo has quit IRC | 22:29 | |
*** Pavo has joined #openstack-kolla | 22:29 | |
*** larsks has quit IRC | 22:30 | |
openstackgerrit | Steve Wilkerson proposed openstack/kolla-kubernetes: Helm charts for adding Cinder https://review.openstack.org/409349 | 22:32 |
*** JRobinson__ has joined #openstack-kolla | 22:32 | |
*** dave-mccowan has joined #openstack-kolla | 22:32 | |
*** srwilkers is now known as srwilkers_away | 22:33 | |
kfox1111 | sbezverk_: not sure there's a garentee that containers are stable on the hub. | 22:35 |
kfox1111 | oh. wait. are you saying, how will they know they may need to pull a newer one so that they can use kolla-kubernetes? | 22:37 |
sbezverk_ | kfox1111: I thought in opensource there is no such thing like guarantee ;-) | 22:37 |
kfox1111 | heh. in some ways yea. | 22:37 |
sbezverk_ | kfox1111: right that was my point | 22:37 |
kfox1111 | gotcha. yeah. that is a problem... | 22:37 |
sbezverk_ | they need to check build date | 22:37 |
kfox1111 | yeah. or we make it explicit as a different release tag. like 3.0.1-2 | 22:38 |
*** larsks has joined #openstack-kolla | 22:38 | |
kfox1111 | then 3.0.1 still exists. | 22:38 |
kfox1111 | and we update kolla-kubnetes all_values to point to 3.0.1-2 | 22:38 |
kfox1111 | its not a different updated zstream though, so may not need to wait for a new release. | 22:39 |
*** dave-mccowan has quit IRC | 22:39 | |
sbezverk_ | kfox1111: right but these are kolla images, you will need to sell them this idea.. | 22:39 |
kfox1111 | +2 | 22:40 |
kfox1111 | +1 | 22:40 |
sbezverk_ | I am not sure if we can just at will go and build in kolla repo | 22:40 |
sbezverk_ | tags we think make sense.. | 22:40 |
kfox1111 | yeah. I don't think we should either. shoudl go through kolla. | 22:40 |
kfox1111 | I'm in a weird place though, as Ihave commit rights to the hub, and not many people want to say they can do anything with the hub. | 22:40 |
kfox1111 | probably should bring it up at the next meeting. | 22:41 |
kfox1111 | shoudl we then wait to merge any of the endpoint creation stuff, or merge the hack, | 22:41 |
kfox1111 | and undo the hack asap? | 22:41 |
sbezverk_ | +1 for hack | 22:42 |
kfox1111 | k. | 22:42 |
sbezverk_ | it will be a minor change to undo it | 22:42 |
kfox1111 | its ready now then if you want to merge it: https://review.openstack.org/#/c/409488/ | 22:42 |
kfox1111 | yeah. just gota drop the user: 0 and the pip install from 2 files. | 22:42 |
sbezverk_ | kfox1111:done | 22:45 |
kfox1111 | k. thx. | 22:45 |
openstackgerrit | Merged openstack/kolla-kubernetes: Add neutron keystone service/endpoint creation jobs https://review.openstack.org/409488 | 22:45 |
*** msimonin has joined #openstack-kolla | 22:51 | |
*** JRobinson__ has quit IRC | 22:53 | |
*** msimonin1 has joined #openstack-kolla | 22:53 | |
*** JRobinson__ has joined #openstack-kolla | 22:56 | |
*** msimonin has quit IRC | 22:56 | |
*** goldyfruit has joined #openstack-kolla | 22:57 | |
*** goldyfruit has quit IRC | 23:12 | |
portdirect | https://github.com/portdirect/harbor | 23:22 |
*** JRobinson__ has quit IRC | 23:24 | |
*** JRobinson__ has joined #openstack-kolla | 23:27 | |
sbezverk_ | portdirect: so far I could not find a gracefull way to kill POD from wihtin itslef, other than kubectl :( | 23:37 |
*** msimonin1 has quit IRC | 23:39 | |
*** msimonin has joined #openstack-kolla | 23:39 | |
*** msimonin has quit IRC | 23:39 | |
openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: DO_NOT_MERGE PoC Configmap update monitor https://review.openstack.org/409493 | 23:44 |
portdirect | sbezverk_: lol so that link was posted in the wrong place :( I'll have another look at the PoC you've done and upload my take if you like? | 23:52 |
kfox1111 | sbezverk_: still thinking the cleanest thing really is to update the k8s object as appropriate and have k8s deal with killing the pods. | 23:55 |
kfox1111 | :/ | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!