Friday, 2017-02-03

sbezverkinstead after a while, I suspect when nova-api service gets created00:00
sbezverkit starts crashing becaose real provider of nova-api service nova-api pod is not running00:00
jascott1svc is created but backend is not serving yet00:01
kfox1111maybe its the lack of readyness checks?00:01
kfox1111do we have enough currently?00:01
sbezverkas a learning go exercise, I am planning to do a code walk of entrypoint and build a debug images with bunch of debugs enabled00:01
*** Pavo has joined #openstack-kolla00:01
kfox1111I guess we do for the deployments.00:01
sbezverkjascott1: exactly00:01
kfox1111svc is created but not serving should be covered by endpoint. it says in the docs. if not its probably a bug.00:02
*** lamt has quit IRC00:02
sbezverkright, it sounds very much like a bug00:02
kfox1111thus far though, every time I thought I hit a bug in entrypoint its been a slightly wrong dep specification.00:02
sbezverkkfox1111: the problem is nobody from entrypoint wants to check provided information and confirm00:03
kfox1111got a link?00:03
sbezverkhttps://github.com/stackanetes/kubernetes-entrypoint/issues/1900:03
kfox1111thx00:04
jascott1it would be great to have a map of the dependencies00:05
jascott1from k8s perspective00:05
kfox1111sbezverk: k. so cinder-api is waiting to init...00:08
kfox1111sbezverk: but what is the thing that is depending on cinder-api but is starting too early?00:08
sbezverkscheduler00:09
kfox1111k. looking...00:10
sbezverkin the last version of cinder, I actually end up listing all bootstrap jobs00:10
sbezverkfor shceduler as well, so it would not try to get into running state00:10
kfox1111[{"name":"dependencies","image":"docker.io/kolla/centos-binary-kubernetes-entrypoint:4.0.0","env":[{"name":"DEPENDENCY_SERVICE","value":"cinder-api"},{"name":"NAMESPACE","value":"kolla"}]00:10
kfox1111hmmm... sure enough.00:11
kfox1111the cinder api never incremented its restart count, so its not that it crashed and restarted.00:11
kfox1111http://logs.openstack.org/97/424697/2/check/gate-kolla-kubernetes-deploy-centos-binary-2-helm-entrypoint-nv/54871ee/logs/pods/kolla-cinder-scheduler-0-dependencies.txt.gz00:12
kfox1111looks like it thinkgs cinder-api svc gained an endpoint.00:12
*** hfu has joined #openstack-kolla00:12
portdirectkfox1111: thats what I pointed out above00:12
kfox1111http://logs.openstack.org/97/424697/2/check/gate-kolla-kubernetes-deploy-centos-binary-2-helm-entrypoint-nv/54871ee/logs/svc/kolla-cinder-api.txt.gz shows no endpoints...00:12
portdirectis it possible to get lofs for endpoints?00:12
portdirect*logs00:12
kfox1111yeah. that dependencies link from above.00:13
kfox1111should be the endpoints log for the cinder-scheduler init container.00:13
portdirectno - we need the logs for the actualy endpoints themselves00:13
kfox1111glance-api?00:14
portdirectthat way we can see if its a but in k8s-entrypoint or somewhere else00:14
portdirect*bug00:14
kfox1111yeah. this could be a k8s-entrypoing bug, or a bug where k8s makes an entrypoint 'ready' for half a second anddrops it back out.00:14
portdirectyep00:14
sbezverkportdirect: just dump bunch of services one after another without wating, you will see it right away00:15
portdirectk8s-entrypoint marks the service as up when the endpoint list >000:15
kfox1111portdirect: here's the event log for cinder-api00:15
kfox1111http://logs.openstack.org/97/424697/2/check/gate-kolla-kubernetes-deploy-centos-binary-2-helm-entrypoint-nv/54871ee/logs/pods/kolla-cinder-api-1556585568-c1kq5.txt.gz00:15
kfox1111looks like it only ever started the deps container.00:15
kfox1111so it never shoudl have been ready.00:15
portdirectyeah - so somthing must have cuased the ep list to get populated - this could be a k8s bug i wonder?00:16
kfox1111yeah, possible.00:16
kfox1111are we sure that k8s-entrypoint just looks at the svc?00:16
kfox1111or does it check pods too?00:16
portdirectyeah - i can give you a link to the code if you like?00:17
portdirect2 sec00:17
portdirectthis is all it does: https://github.com/stackanetes/kubernetes-entrypoint/blob/master/dependencies/service/service.go#L27-L3600:17
portdirectget a list of ep's for a service, if the list >0 then assumes its good to go00:18
*** goldyfruit has quit IRC00:18
kfox1111looks fairly simple... but                               wonder what Subsets is.00:18
*** adrian_otto has joined #openstack-kolla00:19
portdirecti think ip/port mappings, need to double check the api docs though to be sure00:19
kfox1111https://kubernetes.io/docs/user-guide/services/00:20
sbezverkguys what is pod gets created and even it sits in init state, the service gets backend already assigned00:20
sbezverkso the service might have backend but pod still not operational as it waits for bootstrap jobs to complete00:21
kfox1111at the end of the run, the service had no endpoints.00:21
kfox1111so I think it added the pod prematurely to the endpoint list, then canceled it.00:21
kfox1111this does look to be a k8s problem, but only noticed by entrypoint.00:21
kfox1111it hsould only enter svc when it hits "ready" state. which it shouldn't until the readyness hook passes.00:22
portdirectright so, lets run with that theory00:22
kfox1111but it was only ever in the init state.00:22
portdirect2 solutions:00:23
portdirect1) get k8s-entrypint to test x times before reporting ok00:23
portdirect2) fix k8s00:23
kfox11111 might work, but if you have enough replica's, it could fail several times in a row. :/00:24
kfox11112 would be good, but might take a while.00:24
kfox1111but I can think of a 3rd.00:24
portdirectagree with your assement00:24
kfox1111curl the endpoint and block until it works?00:24
portdirectthat works too00:24
kfox1111that might be a more reliable test anyway then k8s-entrypoint.00:25
kfox1111just out of curiosity, does cinder-schedular actually call the cinder-api? I would have thoguht not.00:26
portdirecti shouldnt need to be dependant on it no00:26
sbezverkkfox1111L cinder api dependency for sheduler was put as a guarantee that bootstrap process is completed00:27
portdirectthe only way they interact if via the message bus afaik?00:27
kfox1111sbezverk: ah. but the same could be done by just looking at the boot jobs?00:27
kfox1111portdirect: yeah, thats what I thought too.00:28
kfox1111sbezverk: would start more in parallel then.00:28
sbezverkit is like chain of dependencies, api depeneds on bootstrap and scheduler depends on api as a result sheduler00:28
sbezverkdepends on everything00:28
kfox1111yeah, but the chain is not strigly needed. it could be started more in parallel.00:28
kfox1111boostrap things, then start cinder-api / cinder-scheduler at the same time.00:28
sbezverkfor cinder it is not that bad because it does not really need api00:29
sbezverkso by listing bootstrap jobs it got fixed00:29
portdirectthis is the same approach we took with os-helm with k8s-entrypoint (job dependencys)00:29
sbezverkbut other pods might really rely on api being up and running00:30
kfox1111sbezverk: yeah. I think there may need to be some deps between nova/neutron that way? though, we probably don't need too many deps, as openstack itself does deal ok with having services go away and come back.00:30
kfox1111in some cases at least.00:31
sbezverkkfox1111: I guess the first step would be where possible to start using jobs as dependecy00:33
kfox1111yeah. :/00:33
kfox1111really don't like that. :/00:33
*** lrensing has joined #openstack-kolla00:33
kfox1111if  version A of kolla-kube has boostrap jobs (a, b, c)00:33
kfox1111and version B of kolla-kube has boostrap jobs (b, c, d)00:34
kfox1111upgrading can get really ugly then. :/00:34
kfox1111while kind of ugly, I think maybe if we really wanted to depend on jobs,00:34
kfox1111as a signaling mechanism,00:35
kfox1111we make a sential job.00:35
kfox1111say, a cinder-is-bootstrapped-job.00:35
kfox1111make it depend on all the other boostrap jobs for cinder.00:35
kfox1111and only make the deps on jobs for other container depend on that one sentinal job.00:35
kfox1111then all kolla-kubernetes versions would only ever look for taht sentinal.00:36
kfox1111so upgrades could go smoother.00:36
kfox1111sbezverk: portdirect what do you think of that idea?00:39
sbezverkkfox1111: it add another level of complexety00:42
kfox1111yeah. but it removes a lot of complexity from upgrades?00:42
sbezverkcan you think of any use case where bootstrap jobs might be different?00:43
kfox1111we've already renamed boostrap jobs before, and split/added some.00:43
kfox1111I don't think we nessisarily need to do anything else, but "forever" is a long time to not change something.00:44
kfox1111the startup may perform better too, if it doesn't have to pull multiple jobs. though that may or may be a small amount of load.00:44
sbezverkkfox1111: I would try to push for the fix00:45
*** rwallner has joined #openstack-kolla00:45
kfox1111yeah. for sure we should do that.00:46
kfox1111but with as long as k8s fixes have been taking, I'm not holding my breath it would be usable any time soon. :/00:46
*** rwallner has quit IRC00:46
kfox1111that resolv.conf thing kills me. :/00:46
*** rwallner has joined #openstack-kolla00:47
sbezverkkfox1111: it is getting merged00:48
sbezverkI heard00:48
*** salv-orlando has joined #openstack-kolla00:49
kfox1111nice. :)00:49
*** lamt has joined #openstack-kolla00:50
kfox1111the shared namespace one is looking like it maybe 1.7 at very least. :/00:50
kfox1111maybe 1.8. they are a bit panic'ed over its ramifications.00:51
*** salv-orlando has quit IRC00:54
*** dave-mccowan has joined #openstack-kolla00:56
*** zhubingbing_ has joined #openstack-kolla00:58
zhubingbing_hello guys00:58
kfox1111hey. :)00:59
clileshi01:00
*** salv-orlando has joined #openstack-kolla01:00
*** salv-orlando has quit IRC01:06
*** skramaja_ has joined #openstack-kolla01:16
*** skramaja has quit IRC01:19
*** crushil has joined #openstack-kolla01:20
*** lamt has quit IRC01:21
*** lamt has joined #openstack-kolla01:26
*** sdake has joined #openstack-kolla01:28
*** lamt has quit IRC01:28
*** portdirect is now known as portdirect_away01:30
*** lamt has joined #openstack-kolla01:31
*** sbezverk has quit IRC01:34
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: [TEST][MASTER][KOLLA-ANSIBLE]  https://review.openstack.org/42278601:41
sdakebmace_ was busy all day - not sure ify ou got my mesage ealier about getting a n env working01:45
sdakebmace_ i think one thing you may or may not be aware of is cloud.yaml01:46
*** rwallner has quit IRC01:47
sdakekfox1111 what is setup-resolv-conf.sh plz?01:48
openstackgerritZeyu Zhu proposed openstack/kolla-ansible master: Remove unnecessary import in kolla_toolbox.py  https://review.openstack.org/42853501:48
*** sbezverk has joined #openstack-kolla01:52
*** zhurong has joined #openstack-kolla01:52
sdakesbezverk quick q, kolla-kubernetes vms ssh seems tow ork like a champ01:53
*** unicell has quit IRC01:53
sdakeping to google's dns works01:53
sdakehwoever, dns inside the vm is non-functional01:53
sdakei could hack resolve.conf inside the vm and get things going01:53
sdakecurious if you have a better way01:53
openstackgerritZeyu Zhu proposed openstack/kolla-ansible master: Remove unnecessary import in kolla_toolbox.py  https://review.openstack.org/42853502:01
*** Pavo has quit IRC02:01
*** salv-orlando has joined #openstack-kolla02:02
sdakeinc0 can you provide the link for the organized version of the agenda for the ptg?02:03
*** dave-mccowan has quit IRC02:03
*** Pavo has joined #openstack-kolla02:05
*** adrian_otto has quit IRC02:06
*** salv-orlando has quit IRC02:06
inc0https://etherpad.openstack.org/p/kolla-pike-ptg-schedule sdake02:07
sdaketx inc002:07
inc0kfox1111, sbezverk others, fill up k8s sessions as you guys see fit02:08
sdakewhat is day 1 #2?02:08
*** lrensing has quit IRC02:09
sdakeinc0 do you hve  the original link to the sessions we started from?02:10
* sdake has link overload02:12
*** mattmceuen has joined #openstack-kolla02:12
sdake9-5 = 9 hours02:12
*** lamt has quit IRC02:13
sdake1 hour lunch break = 9-12, 1-5 = 8 hours02:13
sdakerather subract one02:13
sdake9-5=8 hours02:13
sdake9-12, 1-5=7 hours02:13
*** lamt has joined #openstack-kolla02:13
sdakeinc0 were you able to sort out a wednesday collab time with the peeps in tripleo?02:14
sdakeI am leaving wed at 11am i htink02:14
sdakeif we coudl somehow get that in from 9-10, i think that would work best  for many people as many people may leave wed02:15
inc0I'll check it out, worst case scenerio I'll just meet with them, I leave Thur02:16
inc0kfox1111 will be around too02:17
sdakeinc0 cool thanks02:17
sdakeinc0 any idea of headcount?02:18
*** lamt has quit IRC02:18
sdakeinc0 woudl be nice to have dinner as a team mon and tue dutch style - although difficult to find a place last minute - so need to book asap02:19
inc0no idea really, we'll have to improvise02:19
inc0worked out in BCN:)02:19
inc0I don't think it will be super crowded tho02:19
*** mdnadeem has joined #openstack-kolla02:20
sdakebasic math = 500 people / (2 (since the ptg is sort of broken into two parts) + number of teams)02:21
sdake18 teams monday and tuesday02:23
*** Pavo_ has joined #openstack-kolla02:24
sdakethat gives us 15 people using my half-baked mathmatics :)02:24
sdakei'll see if I can hunt down some reservations for that headcount02:24
*** skramaja_ has quit IRC02:28
sdakesbezverk your color hurts my eyes - can you change it plz :)02:33
*** prameswar has joined #openstack-kolla02:39
*** unicell has joined #openstack-kolla02:40
*** MasterOfBugs has quit IRC02:47
*** kbyrne has quit IRC02:47
*** kbyrne has joined #openstack-kolla02:50
*** unicell has quit IRC02:53
*** dave-mccowan has joined #openstack-kolla02:53
openstackgerritjangpro2 proposed openstack/kolla-ansible master: This feature is supporting for collecting log from IPA.  https://review.openstack.org/42855303:01
*** salv-orlando has joined #openstack-kolla03:03
*** crushil has quit IRC03:05
*** salv-orlando has quit IRC03:07
*** lrensing has joined #openstack-kolla03:10
inc0sooo Expanse s2 doesn't disappoint.03:11
*** sayantan_ has quit IRC03:11
*** Pavo_ has quit IRC03:13
clileshow does the kolla_docker ansible module create a privlidged container?03:14
clilesor maybe I should just ask...03:15
*** dave-mccowan has quit IRC03:15
openstackgerritJeffrey Zhang proposed openstack/kolla master: [TEST][MASTER][KOLLA]  https://review.openstack.org/42662503:16
clilesshouldn't the cinder_volume container start with CAP_SYS_ADMIN or privlidged so it can call mount?03:16
clilesin a fresh install I'm getting errors unable to do nfs mount03:17
Jeffrey4lcliles, yes. this patch is fixing this https://review.openstack.org/42726803:18
clilesJeffrey4l: ty!!!!!!!!!!!!!!!1103:20
*** sayantani01 has joined #openstack-kolla03:20
openstackgerritMerged openstack/kolla-ansible master: Implement nova placement service  https://review.openstack.org/42771503:20
openstackgerritMerged openstack/kolla-ansible master: Fix typo in cloudkitty register.yml file  https://review.openstack.org/42812103:21
*** mattmceuen has quit IRC03:25
*** crushil has joined #openstack-kolla03:25
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Fix gnocchi auth issue  https://review.openstack.org/42812403:28
*** zhurong_ has joined #openstack-kolla03:36
*** zhurong has quit IRC03:38
*** sdake has quit IRC03:41
*** crushil has quit IRC03:41
*** mdnadeem has quit IRC03:42
*** l4yerffeJ_ has quit IRC03:52
*** l4yerffeJ_ has joined #openstack-kolla03:53
*** hfu has quit IRC04:00
*** Pavo has quit IRC04:01
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: [TEST][MASTER][KOLLA-ANSIBLE]  https://review.openstack.org/42278604:02
*** hfu has joined #openstack-kolla04:02
*** Pavo has joined #openstack-kolla04:06
*** crushil has joined #openstack-kolla04:07
*** sdake has joined #openstack-kolla04:08
*** zhurong_ has quit IRC04:11
*** mdnadeem has joined #openstack-kolla04:11
*** sdake_ has joined #openstack-kolla04:11
*** sdake has quit IRC04:12
openstackgerritJeffrey Zhang proposed openstack/kolla master: Using CI mirror host  https://review.openstack.org/36572104:14
clilesok, so anything hanging around w/ libvirt?04:18
openstackgerritMerged openstack/kolla-ansible master: Use required_if in kolla_docker.py  https://review.openstack.org/42378804:18
clilesI'm getting "libvirtError: Did not receive a reply." when launching anything on a fresh install04:19
*** mattmceuen has joined #openstack-kolla04:21
*** lrensing has quit IRC04:21
*** sayantani01 has quit IRC04:26
openstackgerritMerged openstack/kolla-ansible master: External_fqdn for host_href in barbican.conf  https://review.openstack.org/42669704:31
*** crushil has quit IRC04:33
*** crushil has joined #openstack-kolla04:33
clileswell if anyone can take a look at my my issue with nova/libvir on fresh install from master I'd appreciate it http://paste.openstack.org/show/khfSMD0r6kP6KB7UMNDD/04:36
*** crushil has quit IRC04:39
*** crushil has joined #openstack-kolla04:41
Jeffrey4lcliles, selinux?04:45
Jeffrey4lcould u try to disable selinux?04:45
openstackgerritJeffrey Zhang proposed openstack/kolla master: Using CI mirror host  https://review.openstack.org/36572104:52
*** sudipto has joined #openstack-kolla04:58
*** sudipto_ has joined #openstack-kolla04:59
*** mattmceuen has quit IRC05:00
*** salv-orlando has joined #openstack-kolla05:04
*** salv-orlando has quit IRC05:09
*** sudipto_ has quit IRC05:13
*** sudipto has quit IRC05:13
*** adrian_otto has joined #openstack-kolla05:14
*** crushil has quit IRC05:16
*** crushil has joined #openstack-kolla05:16
clilesJeffrey4l: success!05:20
clilesI thought selinux was supported?05:20
Jeffrey4lcliles, cool. which docker version are u using?05:20
cliles1.13.005:20
Jeffrey4lthere are some unknown issue. i tried deploy kolla w/ selinux, it works.05:21
Jeffrey4lno idea why it failed recently.05:21
clilesis there already a bug for selinux open?05:22
Jeffrey4lcliles, no could u file one?05:22
Jeffrey4lmoreover, could u attache you audit.log file in the bug description?05:22
clilesin kolla or kolla-ansible?05:22
Jeffrey4lkolla.05:22
Jeffrey4lwait. kolla-ansible is better.05:23
clilesok05:23
Jeffrey4lthanks.05:23
*** skramaja has joined #openstack-kolla05:30
openstackgerritJeffrey Zhang proposed openstack/kolla master: [TEST][MASTER][KOLLA] test docker engine 1.13  https://review.openstack.org/42857705:31
*** sayantani01 has joined #openstack-kolla05:33
clilesJeffrey4l: I opened it and attached05:39
clilesWould you be able to help me get some eyes on an older bug of mine please? https://bugs.launchpad.net/kolla/+bug/165413305:39
openstackLaunchpad bug 1654133 in kolla "horizon dashboard broken in master with centos binaries" [Undecided,New]05:39
Jeffrey4lcliles, selinux issue bug link?05:40
inc0Jeffrey4l, https://review.openstack.org/#/c/427931/ thoughts about this idea?05:40
openstackgerritJeffrey Zhang proposed openstack/kolla master: [TEST][MASTER][KOLLA] test docker engine 1.13  https://review.openstack.org/42857705:40
Jeffrey4lhave u ever tried the horizon dashboard, recently?05:40
clilesI just build and deployed today, so yeah05:41
Jeffrey4linc0, any benefit for this? why we do not setup our own?05:41
Jeffrey4lit will cause the size increase.05:41
inc0Jeffrey4l, ubuntu was broken for 2 months because we failed to notice old version of xtrabackup05:41
Jeffrey4land do not support oraclelinux/ again..05:41
inc0we will never be as good as mariadb in supporting mariadb05:41
clilesjeffrey4l: selinux bug is at https://bugs.launchpad.net/kolla-ansible/+bug/166150005:42
openstackLaunchpad bug 1661500 in kolla-ansible "selinux blocks libvirt from spawning instances" [Undecided,New]05:42
Jeffrey4ldo not check the details. but mariadb official image support gelara?05:42
inc0yes05:42
inc0as of version 10.1 official mariadb has galera05:42
Jeffrey4lhrm.. add more test will find the issue ASAP.05:43
Jeffrey4lwhat about the image total size and oraclelinux/centos image support for this case?05:43
inc0well we take distro that's given to us05:44
inc0I'm not sure if it's bad as long as it's well maintained05:44
inc0image total size isn't an issue imho05:44
inc0this thing builds in couple seconds05:44
inc0thing is, versions are going to be well maintained, all the packages will be present05:45
inc0and for example (according to SamYaple) rabbimq uses better version of erlang which for example doesn't have this IP parsing issue05:46
Jeffrey4l"we take distro that's given to us" means do not care about centos/oracle, or maintain our edition?05:46
inc0so I would like to hear about why should we care about this distro?05:47
inc0I mean, it's about who maintain packages05:47
inc0some like rdo, some like debian05:47
Jeffrey4lhigher version is better, in most of case. we can install higher infra service, with some extra repo.05:47
inc0but in case of for example mariadb, packages comes from mariadb itself05:47
inc0yeah, but we need to maintain this repo list and so on05:48
inc0like remember when percona messed up their ssl key?05:48
inc0with my approach we wouldn't even feel it05:48
inc0also our gates becomes much more stable as we will almost stop using any external repos besides dockerhub (we'll pull built images)05:49
Jeffrey4lre distro, some one really care. for example, some guys works on power support, who care debian only.05:50
inc0then they could host their own images in upstream docker05:50
inc0and use it as base for us05:50
inc0like, this FROM mariadb can be editable05:50
Jeffrey4linc0, re " using any external repos besides dockerhub" it is impossible. we depends on rdo, cloud-archive, at least.05:50
inc0and you can put your own image there05:50
inc0why? for source builds we don't need it05:51
Jeffrey4linc0, then how to setup those image? what the specification to build thoese image, which works with kolla?05:51
inc0only one real spec - they have to have all the packages installed05:51
inc0and we will mainain code to "kolla'ize" them05:52
inc0much like in patchset05:52
Jeffrey4linc0,  1. that's another topic, do u really wanna remove binary install type?   2. except that, there are some infra service which do not provide base image, or at least a stable image.05:52
*** sp__ has joined #openstack-kolla05:52
inc0btw Jeffrey4l are you coming to PTG?05:52
Jeffrey4linc0, no ;(05:52
Jeffrey4linc0, how do defined "all packages"?05:53
inc0Jeffrey4l, for those sevices without base image - we do what we do today05:53
inc0like in mariadb - it's all the mariadb packages, it's deps, xtrabackup stuff and so on05:53
Jeffrey4lwhen define "all packages", we have to sync to mariadb's official dockerfile.05:53
inc0that's what I did05:54
inc0and if someone wants some custom stuff05:54
inc0they can do the same05:54
openstackgerritMerged openstack/kolla-ansible master: Revert "corrects invalid logrotate option maxsize"  https://review.openstack.org/42764205:54
inc0for example, power guys could fork official mariadb image and make it power05:54
inc0or for example if something doesn't have debian base, they could build their debian base for this thing05:55
Jeffrey4lhrm, it will cause more issue in the future. for example, some mariadb guys make some change, and we do not know, and case cause kolla/kolla-ansible failed.05:55
inc0this way we would only need to care about packages required for openstack services05:55
inc0so far we made kolla-ansible failed05:55
inc0because we couldn't follow the versions of xtrabackup05:55
Jeffrey4lsorry? how kolla-ansible failed?05:55
inc0ubuntu was broken05:56
Jeffrey4loh right.05:56
Jeffrey4lif we have test for this, we can found it ASAP.05:56
inc0well, but we didn't05:56
Jeffrey4lthe really issue is: we haven't such test.05:56
Jeffrey4ldeploy external issue do not guarantee this, too.05:56
inc0also, can you say with 100% certenty that for example rabbimq (we pin version remember?) didnt have critical security patch?05:56
Jeffrey4lmaybe it guarantee the version number of the packages.05:57
Jeffrey4lbut it do not guarantee whether kolla-ansible configure it in right way.05:57
Jeffrey4lwe still need better test.05:57
inc0what I'm saying is to not own every service we have05:57
inc0because we don't have capacity to maintain all of them properly05:57
Jeffrey4linc0, re rabbitmq,  absolutely no05:58
Jeffrey4linc0, so rdo team and ubuntu cloud archive team maintain this :)05:58
inc0I'm sure rabbitmq community will know how to handle that stuff05:58
Jeffrey4lwe do not care about this, imo.05:58
inc0ubuntu didn't maintain mariadb:/05:58
inc0well, anyway, think about it05:58
inc0it's small revolution, but I think we need to lower scope of our project05:59
Jeffrey4lon other hand, why from mariadb:10.1, when we bump the number?05:59
Jeffrey4lit is almost the same issue?05:59
inc0when we feel comfortable05:59
Jeffrey4ls/?/./05:59
inc0not really05:59
inc0because security patches and so on will land into container anyway05:59
Jeffrey4lfrom official image only guarantee the package version is correct.06:00
*** Pavo has quit IRC06:01
Jeffrey4lthe official image build from some repo too. if we use that repo. then the security patches is landed.06:01
inc0yeah, but if we use same repo...why not use whole image?06:01
*** MasterOfBugs has joined #openstack-kolla06:01
inc0I hear more and more issues regarding our images06:01
inc0and I agree with some of them06:01
inc0what was good idea 2 years ago, might not be good any more06:02
inc0anyway, think about this a bit06:02
inc0we'll have session in PTG about images too06:02
Jeffrey4lhttps://github.com/docker-library/mariadb/blob/b558f64b736650b94df9a90e68ff9e3bc03d4bdb/10.1/Dockerfile  it come from deb https://repo.percona.com/apt jessie main06:02
openstackgerritqiankun.zhang proposed openstack/kolla master: Install tacker-horizon into horizon image  https://review.openstack.org/40781006:03
Jeffrey4lok.06:03
Jeffrey4lcheck the official dockerfile, there is nothing special. except that he use two extra mariadb repo.06:04
inc0yeah, I know06:04
inc0just, you'll never know06:04
inc0and there might be something special06:04
inc0instead of copying06:04
*** pramodrj07 has joined #openstack-kolla06:04
inc0I'd rather trust them06:04
openstackgerritqiankun.zhang proposed openstack/kolla master: Install tacker-horizon into horizon image  https://review.openstack.org/40781006:05
*** MasterOfBugs has quit IRC06:05
*** Pavo has joined #openstack-kolla06:05
*** salv-orlando has joined #openstack-kolla06:05
Jeffrey4ldepends on official repo, at first ;)06:05
inc0https://github.com/docker-library/rabbitmq/blob/master/debian/Dockerfile06:06
inc0rabbitmq is slightly more complex06:06
inc0they use newer erlang06:06
Jeffrey4li think we can.06:06
inc0yeah ofc we can copy their approach06:07
inc0I just think if we should06:07
inc0or rather focus on keeping openstack packages top notch06:07
inc0number of infra images will only grow06:07
inc0I would like to focus on monitoring next release06:08
inc0at some point we'll just have too much to handle06:08
Jeffrey4lon the other hand, i think another approach. kolla should build the image separately. for example, mariadb is no need to release every 6 month.06:08
inc0security updates06:09
inc0you never know when you need new mariadb06:09
*** salv-orlando has quit IRC06:10
Jeffrey4li think mariadb offical image do not know too.  they depends on repo. and maybe build the image periodic.06:10
Jeffrey4lor build the image when dockerfile changed.06:10
Jeffrey4lno one knows when the repo is updated.06:12
inc0well...mariadb people knows;)06:12
inc0but well, it's just an idea06:12
Jeffrey4land mariadb offical image do not mean mariadb developer team/mariadb package guys.06:13
Jeffrey4lthe contributors are https://github.com/docker-library/mariadb/graphs/contributors06:13
*** crushil has quit IRC06:13
Jeffrey4li guess most of them are work in docker company.06:13
*** saneax-_-|AFK is now known as saneax06:18
*** inc0 has quit IRC06:19
openstackgerritJeffrey Zhang proposed openstack/kolla master: [TEST][MASTER][KOLLA] test docker engine 1.13  https://review.openstack.org/42857706:21
openstackgerritVladislav Belogrudov proposed openstack/kolla master: Containerized sshd does not work with older PAM  https://review.openstack.org/41303006:29
openstackgerritlyanchih proposed openstack/kolla-ansible master: Secret generate fail with hostvars is undefined  https://review.openstack.org/41733106:33
*** prameswar has quit IRC06:36
*** unicell has joined #openstack-kolla06:41
*** zhurong has joined #openstack-kolla06:45
openstackgerritt.goto proposed openstack/kolla-ansible master: Fix typo in cinder role's handler task.  https://review.openstack.org/42859906:48
sdake_mariadb for centos is best provided by the distribution, not mariadb06:52
sdake_and on that note, I am off to bed06:52
*** sdake_ has quit IRC06:52
*** satyar has joined #openstack-kolla06:52
*** salv-orlando has joined #openstack-kolla06:57
*** unicell1 has joined #openstack-kolla06:57
*** unicell has quit IRC06:59
*** hieulq has joined #openstack-kolla07:02
*** sayantani01 has quit IRC07:07
*** sayantani01 has joined #openstack-kolla07:07
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Fix the pid_mode and ipc_mode value check for kolla_docker module  https://review.openstack.org/42860607:09
*** jrobinson has quit IRC07:13
*** adrian_otto has quit IRC07:20
*** bmace_ has quit IRC07:21
*** bmace_ has joined #openstack-kolla07:21
*** adrian_otto has joined #openstack-kolla07:26
*** athomas has joined #openstack-kolla07:28
*** adrian_otto has quit IRC07:33
*** sayantani01 has quit IRC07:48
*** sayantani01 has joined #openstack-kolla07:48
*** salv-orl_ has joined #openstack-kolla07:49
*** salv-orlando has quit IRC07:51
*** hieulq has quit IRC07:54
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Fix the pid_mode and ipc_mode value check for kolla_docker module  https://review.openstack.org/42860607:55
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Fix the pid_mode and ipc_mode value check for kolla_docker module  https://review.openstack.org/42860607:56
*** unicell1 has quit IRC07:57
*** prameswar has joined #openstack-kolla07:59
*** matrohon has joined #openstack-kolla08:00
*** Pavo has quit IRC08:01
*** hieulq has joined #openstack-kolla08:03
*** salv-orl_ has quit IRC08:04
*** salv-orlando has joined #openstack-kolla08:04
*** Pavo has joined #openstack-kolla08:05
*** salv-orlando has quit IRC08:08
*** hieulq has quit IRC08:13
*** magicboiz has joined #openstack-kolla08:14
*** pramodrj07 has quit IRC08:18
*** hieulq has joined #openstack-kolla08:21
*** Serlex has joined #openstack-kolla08:31
*** sayantani01 has quit IRC08:33
*** egonzalez has joined #openstack-kolla08:38
*** mannamne has joined #openstack-kolla08:38
openstackgerritJeffrey Zhang proposed openstack/kolla master: Using CI mirror host  https://review.openstack.org/36572108:43
*** hieulq has quit IRC08:45
Jeffrey4legonzalez, could u review this https://review.openstack.org/#/c/428606/ first?08:45
Jeffrey4lubuntu + binary for the nova package issue.   other failed cause by kolla images ( with placment api ) is not updated, with unknown reason.08:45
Jeffrey4lwhen another kolla patch is merged. the kolla images in tallballs.o.o site will be updated.08:46
*** sudipto has joined #openstack-kolla08:48
*** sudipto_ has joined #openstack-kolla08:48
openstackgerritVladislav Belogrudov proposed openstack/kolla master: Containerized sshd does not work with older PAM  https://review.openstack.org/41303008:49
Jeffrey4legonzalez, do u have any idea to explain this ? https://review.openstack.org/42857708:50
Jeffrey4lafter move the /docker file to /tmp/docker, docker-engine 1.13.0 works ;(08:51
openstackgerritMerged openstack/kolla-ansible master: Fix the pid_mode and ipc_mode value check for kolla_docker module  https://review.openstack.org/42860608:51
*** mannamne has quit IRC08:52
openstackgerritZeyu Zhu proposed openstack/kolla-ansible master: Remove unnecessary import in kolla_toolbox.py  https://review.openstack.org/42853508:54
*** shardy has joined #openstack-kolla08:54
openstackgerritJeffrey Zhang proposed openstack/kolla master: [TEST][MASTER][KOLLA]  https://review.openstack.org/42662508:54
*** hieulq has joined #openstack-kolla08:56
openstackgerritJeffrey Zhang proposed openstack/kolla master: Remove include_header and include_footer in all Dockerfiles  https://review.openstack.org/42801708:59
egonzalezJeffrey4l: https://github.com/docker/docker/issues/3058609:24
Jeffrey4legonzalez, aha, thanks. docker really create /docker. hrm09:25
openstackgerritMerged openstack/kolla-ansible master: Remove hardcode /etc/kolla path  https://review.openstack.org/42682709:26
egonzalezJeffrey4l: btw, docker ps, docker images and "old" docker commands are deprecated in 1.13 and removed in the next version09:31
egonzalezthe next version will need to pin point to a 1.13 as minimal09:32
Jeffrey4legonzalez, why? just docker cli change, the api should be stable.09:32
Jeffrey4lkolla-ansible is using docker api rather than docker cli.09:33
egonzalezthe API yes, more for scripts09:33
egonzalezand some ansible task run docker exec which is also in deprecation09:34
*** skuda has joined #openstack-kolla09:34
Jeffrey4lyep. we need find a more reasonable way to support docker since 1.10.09:34
*** athomas has quit IRC09:34
Jeffrey4lbtw, the kolla_toolbox.py module is using api rather than docker exec now.09:35
Jeffrey4lso there are few task is just docker exec directly, which we can move to kolla_toolbox module. :)09:35
*** mgoddard_ has joined #openstack-kolla09:37
*** hrw has quit IRC09:37
*** haplo37 has quit IRC09:40
*** haplo37 has joined #openstack-kolla09:42
*** hrw has joined #openstack-kolla09:43
openstackgerritJeffrey Zhang proposed openstack/kolla master: support docker engine 1.13 in gate  https://review.openstack.org/42857709:44
*** prameswar has quit IRC09:44
*** hfu has quit IRC09:47
openstackgerritJeffrey Zhang proposed openstack/kolla master: support docker engine 1.13 in gate  https://review.openstack.org/42857709:54
*** zhurong has quit IRC09:55
*** erlon has joined #openstack-kolla09:57
*** haplo37 has quit IRC09:58
*** gfidente has joined #openstack-kolla09:58
*** gfidente has quit IRC09:58
*** gfidente has joined #openstack-kolla09:58
*** Pavo has quit IRC10:01
*** satyar has quit IRC10:01
*** magicboiz has quit IRC10:05
openstackgerritMerged openstack/kolla master: Install tacker-horizon into horizon image  https://review.openstack.org/40781010:05
*** Pavo has joined #openstack-kolla10:05
*** salv-orlando has joined #openstack-kolla10:06
*** haplo37 has joined #openstack-kolla10:07
*** magicboiz has joined #openstack-kolla10:07
*** sp__ has quit IRC10:11
*** salv-orlando has quit IRC10:11
*** sambetts|afk is now known as sambetts10:27
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Fix gnocchi auth issue  https://review.openstack.org/42812410:27
*** pbourke has quit IRC10:34
*** mannamne has joined #openstack-kolla10:34
*** zhubingbing_ has quit IRC10:36
*** hieulq has quit IRC10:39
*** jascott1 has quit IRC10:54
*** jascott1 has joined #openstack-kolla10:55
*** jascott1 has quit IRC10:59
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Integrate designate with neutron  https://review.openstack.org/42467911:07
*** salv-orlando has joined #openstack-kolla11:08
openstackgerritJeffrey Zhang proposed openstack/kolla master: support docker engine 1.13 in gate  https://review.openstack.org/42857711:10
*** salv-orlando has quit IRC11:12
*** hfu has joined #openstack-kolla11:13
*** hfu has quit IRC11:16
*** sdake has joined #openstack-kolla11:18
*** athomas has joined #openstack-kolla11:21
*** salv-orlando has joined #openstack-kolla11:25
*** rmart04 has joined #openstack-kolla11:28
*** timss has quit IRC11:30
*** hfu has joined #openstack-kolla11:30
*** timss has joined #openstack-kolla11:30
openstackgerritMerged openstack/kolla master: Containerized sshd does not work with older PAM  https://review.openstack.org/41303011:35
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: [wip] enable ceilometer related project in gate  https://review.openstack.org/41227811:40
*** narasimha_SV has joined #openstack-kolla11:40
*** sdake_ has joined #openstack-kolla11:42
*** sdake has quit IRC11:46
*** magicboiz has quit IRC11:51
*** zhubingbing has joined #openstack-kolla11:53
*** hfu has quit IRC11:53
narasimha_SVas per this code http://paste.openstack.org/show/597540/11:54
narasimha_SVin ceilometer.conf there will be 2 mongodb ips in the connection variables11:54
narasimha_SVwill this work ???11:55
narasimha_SVi have 2 controllers11:55
*** crushil has joined #openstack-kolla11:56
*** Pavo has quit IRC12:01
egonzaleznarasimha_SV: yep, is a list of mongo hosts12:02
*** pbourke has joined #openstack-kolla12:03
*** kristian__ has joined #openstack-kolla12:03
egonzalezwhat at the end is mongo://ceilometer:password@host1,host2/ceilometer12:03
narasimha_SVyes12:04
narasimha_SVit is like that only12:04
narasimha_SVwill that work ??12:04
egonzalezyes, is how mongodb work12:04
narasimha_SVok12:04
narasimha_SVif we enable aodh will ceilometer commands work ??12:04
*** Pavo has joined #openstack-kolla12:05
*** rwallner has joined #openstack-kolla12:06
*** sudipto has quit IRC12:07
*** sudipto_ has quit IRC12:07
*** rwallner has quit IRC12:08
*** rwallner has joined #openstack-kolla12:09
*** dave-mccowan has joined #openstack-kolla12:10
*** rhallisey has joined #openstack-kolla12:10
*** rwallner has quit IRC12:12
*** rwallner has joined #openstack-kolla12:12
*** mliima has joined #openstack-kolla12:14
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Remove hardcoded usernames in register.yml  https://review.openstack.org/42871312:22
*** sdake has joined #openstack-kolla12:28
*** eaguilar has joined #openstack-kolla12:28
*** sdake_ has quit IRC12:30
*** satyar has joined #openstack-kolla12:31
*** eaguilar has quit IRC12:33
*** eaguilar has joined #openstack-kolla12:36
*** schwicht has joined #openstack-kolla12:40
*** skramaja has quit IRC12:43
*** eaguilar_ has joined #openstack-kolla12:44
*** eaguilar has quit IRC12:47
*** rwallner has quit IRC12:49
*** mkoderer has joined #openstack-kolla12:49
openstackgerritJeffrey Zhang proposed openstack/kolla master: support docker engine 1.13 in gate  https://review.openstack.org/42857712:51
*** athomas has quit IRC12:52
*** athomas has joined #openstack-kolla12:52
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Apply new register method  https://review.openstack.org/42871712:52
*** rwallner has joined #openstack-kolla12:53
*** rhallisey has quit IRC12:56
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Remove unnecessary until in register.yml file  https://review.openstack.org/42871913:04
*** zhurong has joined #openstack-kolla13:09
Jeffrey4legonzalez, whether did you read the info about docker exec deprecated?13:11
*** eaguilar_ has quit IRC13:11
egonzalezJeffrey4l: yesterday I was in a local docker meetup about whats new in docker 1.1313:12
Jeffrey4legonzalez, i do not found anything about the deprecated in the release note and this link https://docs.docker.com/engine/deprecated/13:12
Jeffrey4li doubt that now..13:12
*** eaguilar has joined #openstack-kolla13:16
*** sudipto has joined #openstack-kolla13:20
*** sudipto_ has joined #openstack-kolla13:20
*** salv-orlando has quit IRC13:24
*** jheroux has joined #openstack-kolla13:26
*** g3ek has quit IRC13:28
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding compute-kit chart  https://review.openstack.org/42815013:31
*** crushil has quit IRC13:32
alanmeadowsqwang: updated https://etherpad.openstack.org/p/openstack-on-helm13:32
*** rhallisey has joined #openstack-kolla13:32
egonzalezJeffrey4l: yeah, there is no deprecation message, guess the guy mean that are replaced and will deprecate in next releases13:32
*** eaguilar has quit IRC13:33
*** shardy is now known as shardy_mtg13:34
*** g3ek has joined #openstack-kolla13:37
Jeffrey4lmaybe13:37
Jeffrey4lor he may mean "docker container xxx" command? https://docs.docker.com/engine/reference/commandline/container/13:38
*** eaguilar has joined #openstack-kolla13:41
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/newton: Make sure ceilometer/service_credentials wotks with internalURL  https://review.openstack.org/42872813:45
*** crushil has joined #openstack-kolla13:53
*** eaguilar has quit IRC13:56
*** lamt has joined #openstack-kolla13:56
*** goldyfruit has joined #openstack-kolla13:57
*** crushil has quit IRC14:00
*** Pavo has quit IRC14:01
*** rwallner has quit IRC14:02
zhubingbinghello guys14:03
*** Pavo has joined #openstack-kolla14:05
pbourkeegonzalez: hey thanks for looking at the skip-parents patch. Any chance you could try running the tests locally (tox -epy27)? I can't reproduce the failures the gate is seeing...14:10
*** shardy_mtg is now known as shardy14:11
*** rwallner has joined #openstack-kolla14:12
*** crushil has joined #openstack-kolla14:12
*** lamt has quit IRC14:13
*** lrensing has joined #openstack-kolla14:19
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding compute-kit chart  https://review.openstack.org/42815014:19
*** athomas has quit IRC14:20
*** TxGirlGeek has joined #openstack-kolla14:22
*** jtriley has joined #openstack-kolla14:24
*** zioproto has joined #openstack-kolla14:26
egonzalezpbourke: works for me too14:26
pbourkeegonzalez: :/14:26
*** athomas has joined #openstack-kolla14:26
pbourkestrange thing is the first time it failed I could reproduce it in a xenial container, but now cant reproduce anywhere14:27
pbourkewill play around with it some more14:27
*** andyjones2001 has quit IRC14:28
*** satyar has quit IRC14:31
*** lrensing has quit IRC14:32
*** rmart04 has quit IRC14:35
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: [TEST][MASTER][KOLLA-ANSIBLE]  https://review.openstack.org/42278614:35
*** lrensing has joined #openstack-kolla14:40
*** lamt has joined #openstack-kolla14:41
*** srwilkers has joined #openstack-kolla14:41
*** Pavo_ has joined #openstack-kolla14:41
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding compute-kit chart  https://review.openstack.org/42815014:43
*** eaguilar has joined #openstack-kolla14:45
*** eaguilar has quit IRC14:46
*** srwilkers has quit IRC14:47
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: [wip] test scenario2 in gate  https://review.openstack.org/41227914:50
*** eaguilar has joined #openstack-kolla14:53
*** Pavo_ has quit IRC14:54
*** prameswar has joined #openstack-kolla14:57
*** lamt has quit IRC15:02
*** haplo37_ has quit IRC15:09
*** pcaruana has quit IRC15:11
openstackgerritMerged openstack/kolla-ansible master: Remove hardcoded usernames in register.yml  https://review.openstack.org/42871315:12
*** haplo37_ has joined #openstack-kolla15:12
prameswarhello everyone , i am configuring koll-kubernetes and my development environment is ready. but how to install all openstack components ?15:13
prameswareither manually one by one using "helm install stable/servicename" or is there any other method that will install default/(enable in global config) components ? or i have to use kolla-kubernetes binary ?15:13
*** rwallner has quit IRC15:14
prameswarwhat method you guys use for development environment ?15:14
*** pcaruana has joined #openstack-kolla15:17
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding compute-kit chart  https://review.openstack.org/42815015:18
*** saneax is now known as saneax-_-|AFK15:19
*** adrian_otto has joined #openstack-kolla15:20
sbezverkprameswar: the easiest way is to install by service15:20
sbezverkhere is an example to install mariadb: helm install  --debug helm/service/mariadb --namespace kolla --name mariadb --values ../kube-1_cloud.yaml15:21
sbezverkbut before you do that, you need to create a few things, like namespace, generate password, configmaps etc.15:22
sbezverkplease ping sdake, he has the latest steps to do it15:22
prameswarok then i think rest of think i need to follow from quick start guide and while installation of service i need to use helm instead of koll-kubernetes ?15:24
*** adrian_otto has quit IRC15:25
sbezverkprameswar: yes15:25
*** rwallner has joined #openstack-kolla15:25
prameswarok thanks sbezverk :) i am trying that will ping here if i face any problem15:26
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Apply new register method  https://review.openstack.org/42871715:27
sbezverkprameswar: sure thing and good luck15:28
*** matrohon has quit IRC15:29
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: [TEST][MASTER][KOLLA-ANSIBLE]  https://review.openstack.org/42278615:35
*** zhubingbing has quit IRC15:36
*** dave-mccowan has quit IRC15:40
*** eaguilar has quit IRC15:40
*** salv-orlando has joined #openstack-kolla15:44
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla master: Allow the use of previously built parent images  https://review.openstack.org/42632215:46
*** lamt has joined #openstack-kolla15:48
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: [wip] test scenario2 in gate  https://review.openstack.org/41227915:50
*** l4yerffeJ_ has quit IRC15:51
sdakeprameswar i will have a review up this weekend of a full script that works atleast with 1 master and 1 minion using the halcyon dev env - promise15:54
*** dave-mccowan has joined #openstack-kolla15:54
sdakeprameswar for extra credit, i'll have documentation to go along with it :)15:54
openstackgerritMerged openstack/kolla stable/newton: Make sure ceilometer/service_credentials wotks with internalURL  https://review.openstack.org/42872815:54
prameswarsdake : i am also trying ,if successful thn will help you in that :)15:57
pbourkesdake: egonzalez: mandre: rhallisey: berendt: Daviey: you guys ok to merge this? https://review.openstack.org/#/c/425749/15:59
berendtpbourke: lgtm16:00
rhalliseyya looks good16:00
*** Pavo has quit IRC16:00
*** Pavo has joined #openstack-kolla16:05
*** crushil has quit IRC16:06
*** crushil has joined #openstack-kolla16:06
*** sdake_ has joined #openstack-kolla16:07
*** rhallisey has quit IRC16:08
*** rhallisey has joined #openstack-kolla16:08
*** sdake has quit IRC16:08
*** sdake has joined #openstack-kolla16:09
sdakepbourke if its rendered on docs.oo in kolla-ansible i'm good with merging it16:10
sdakepbourke i hve to go tho - sorry can't look right htis minute16:10
pbourkesdake: does it render pre merge though?16:10
sdakei will be availble in 2 hours16:10
sdakeit does indeed render per merge16:11
sdakeoh pre16:11
sdakeno not pre16:11
sdakehowever - i think sant* had pushed her changese to kolla-ansible and it was approved - but can't confirm16:11
sdakei really gotta go - dropping off irc- will be back16:11
pbourke*pre*16:11
*** sdake has quit IRC16:12
*** sdake_ has quit IRC16:12
*** eaguilar has joined #openstack-kolla16:13
pbourkeJeffrey4l: any idea why the ansible gates aren't finding the placement image?16:14
Jeffrey4lpbourke, yes. ubuntu + binary tarball is not uploaded.16:14
Jeffrey4lsorry. centos + binary16:14
Jeffrey4lthe root cause it the timeout in zuul's post action is only 10 min, which is a little lower for kolla centos+binary registry tarball.16:15
Jeffrey4lpbourke, this patch is increasing the timeout https://review.openstack.org/#/c/428740/16:16
pbourkeJeffrey4l: nice one thanks16:16
Jeffrey4lnp ;)16:16
*** kristian__ has quit IRC16:17
*** egonzalez has quit IRC16:21
*** prameswar has quit IRC16:23
*** kristian__ has joined #openstack-kolla16:23
*** kristian__ has quit IRC16:24
*** kristian__ has joined #openstack-kolla16:28
*** adrian_otto has joined #openstack-kolla16:28
*** rwallner has quit IRC16:30
*** zhurong has quit IRC16:34
*** jrich523 has quit IRC16:35
*** jrich523 has joined #openstack-kolla16:35
*** eaguilar_ has joined #openstack-kolla16:36
*** eaguilar has quit IRC16:38
*** eaguila__ has joined #openstack-kolla16:38
*** eaguilar_ has quit IRC16:41
*** rwallner has joined #openstack-kolla16:43
openstackgerritMerged openstack/kolla master: Remove skip-event-database parameter in ceilometer  https://review.openstack.org/42754816:44
kfox1111morning.16:45
*** mdnadeem has quit IRC16:49
*** kristia__ has joined #openstack-kolla16:51
*** adrian_otto has quit IRC16:51
*** kristian__ has quit IRC16:53
*** zioproto has quit IRC16:53
*** satyar has joined #openstack-kolla16:55
satyarhi jeffrey4l16:55
satyarjust want to check if there is multiregional deployment support with kolla16:55
satyarSamYaple16:57
kfox1111I don't think kolla-ansible currently natively supports it, but could probably still make it work with a bit of effort.16:58
satyarkfox1111 any pointer will help17:00
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding compute-kit chart  https://review.openstack.org/42815017:01
kfox1111I have little experience with the kolla-ansible stuff, but have looked through the code.17:01
sbezverkkfox1111: good morning17:01
kfox1111one kolla-ansible install seems to only want to deal with a signle region.17:01
*** adrian_otto has joined #openstack-kolla17:02
kfox1111so I would guess the easiest way might be install two parallel kolla-ansible's. in vm's or containers. then I think the rest might work.17:02
kfox1111sbezverk: hey17:02
*** athomas has quit IRC17:04
des_consoladoguys what docker version are you using for kolla-kubernetes?17:04
sbezverkdes_consolado: 1.12.5 on my kube cluster17:05
openstackgerritIan Kumlien proposed openstack/kolla-ansible master: [RFC] Push libvirt-guest settings to the container  https://review.openstack.org/42881517:07
SamYaplesatyar: ?17:09
*** g3ek has quit IRC17:15
*** lamt has quit IRC17:19
*** sambetts is now known as sambetts|afk17:20
*** lamt has joined #openstack-kolla17:24
*** g3ek has joined #openstack-kolla17:26
*** eaguila__ has quit IRC17:32
*** lamt has quit IRC17:35
*** lamt has joined #openstack-kolla17:36
*** dave-mccowan has quit IRC17:38
*** shardy has quit IRC17:38
qwangalanmeadows: thank you very much. It looks much better now. I'll submit the abstract later today.17:40
*** inc0 has joined #openstack-kolla17:40
inc0good morning17:41
*** kristian__ has joined #openstack-kolla17:41
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding compute-kit chart  https://review.openstack.org/42815017:41
sbezverkinc0: good morning17:41
*** kristia__ has quit IRC17:44
*** rwallner has quit IRC17:44
*** rwallner has joined #openstack-kolla17:45
*** rwallner has quit IRC17:47
*** rwallner has joined #openstack-kolla17:47
*** adrian_otto1 has joined #openstack-kolla17:48
*** adrian_otto has quit IRC17:50
*** mgoddard_ has quit IRC17:52
*** kristian__ has quit IRC17:53
*** Kimmo_ has quit IRC17:56
*** tonyb has quit IRC17:56
*** tonyb has joined #openstack-kolla17:56
*** Kimmo_ has joined #openstack-kolla17:56
*** SamYaple has quit IRC17:56
*** devananda has quit IRC17:56
*** jowisz_ has quit IRC17:56
*** cinerama has quit IRC17:56
*** kwazar has quit IRC17:56
*** jgriffith has quit IRC17:56
*** dmsimard has quit IRC17:56
*** aimeeu has quit IRC17:56
*** PyroMani has quit IRC17:56
*** SamYaple has joined #openstack-kolla17:56
*** jowisz has joined #openstack-kolla17:56
*** kwazar has joined #openstack-kolla17:56
*** dmsimard has joined #openstack-kolla17:56
*** cinerama has joined #openstack-kolla17:56
*** devananda has joined #openstack-kolla17:56
*** dave-mccowan has joined #openstack-kolla17:57
inc0ahh the day of splits17:57
inc0it's been a while17:57
*** PyroMani has joined #openstack-kolla17:57
*** jgriffith has joined #openstack-kolla17:57
*** DuncanT has quit IRC17:59
*** DuncanT has joined #openstack-kolla18:00
*** Pavo has quit IRC18:00
*** jascott1 has joined #openstack-kolla18:02
*** lamt has quit IRC18:03
*** aimeeu has joined #openstack-kolla18:03
*** jascott1 has quit IRC18:04
*** Pavo has joined #openstack-kolla18:05
*** sudipto has quit IRC18:07
*** sudipto_ has quit IRC18:07
*** penfold has joined #openstack-kolla18:10
*** jascott1 has joined #openstack-kolla18:11
*** lamt has joined #openstack-kolla18:12
*** pc_m has quit IRC18:18
*** rwallner_ has joined #openstack-kolla18:19
*** rwallner has quit IRC18:22
*** pc_m has joined #openstack-kolla18:22
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding compute-kit chart  https://review.openstack.org/42815018:24
*** Serlex has quit IRC18:30
*** kristian__ has joined #openstack-kolla18:31
*** imcsk8 has quit IRC18:32
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: Pull container tallballs into the gate  https://review.openstack.org/42659818:33
*** kristian__ has quit IRC18:36
*** mannamne has quit IRC18:39
*** mgoddard_ has joined #openstack-kolla18:45
*** rwallner_ has quit IRC18:48
*** rwallner has joined #openstack-kolla18:48
*** rwallner has quit IRC18:53
*** rwallner has joined #openstack-kolla18:55
*** gfidente is now known as gfidente|afk18:56
*** sdake has joined #openstack-kolla18:56
*** TxGirlGeek has quit IRC18:57
*** TxGirlGeek has joined #openstack-kolla18:57
*** salv-orlando has quit IRC19:05
*** TxGirlGeek has quit IRC19:06
*** TxGirlGeek has joined #openstack-kolla19:07
*** salv-orlando has joined #openstack-kolla19:08
*** unicell has joined #openstack-kolla19:10
*** inc0 has quit IRC19:11
*** crushil has quit IRC19:15
*** TxGirlGeek has quit IRC19:20
*** TxGirlGeek has joined #openstack-kolla19:21
*** lamt has quit IRC19:25
*** eaguilar has joined #openstack-kolla19:25
*** lamt has joined #openstack-kolla19:26
*** lrensing has quit IRC19:29
kfox1111sbezverk: if you are starting up the compute kit after the individual services after delete,19:32
kfox1111you probably still need to delete all the statefulsets.19:32
kfox1111I don't think they go away when you delete the k8s objects.19:32
*** mliima has quit IRC19:37
satyarkfox1111 i am getting gateway timeout while attaching volume19:44
satyaris there anything i am missing in the nova.conf or cinder.conf19:45
kfox1111what kind of cinder backend?19:45
satyarexternal iscsi19:45
kfox1111hmm... never tried that, so not really sure.19:48
kfox1111anything in the nova-compute logs?19:49
*** salv-orl_ has joined #openstack-kolla19:49
satyarhttp://paste.openstack.org/show/597615/19:49
kfox1111hmm...19:51
kfox1111how about the cinder-api logs?19:51
*** narasimha_SV has quit IRC19:52
*** salv-orlando has quit IRC19:52
*** Marcellin__ has joined #openstack-kolla19:52
*** skuda has quit IRC19:53
*** inc0 has joined #openstack-kolla19:54
*** adrian_otto1 has quit IRC19:55
satyar1 sec19:56
*** adrian_otto has joined #openstack-kolla19:57
satyarkfox1111 there is no errors19:58
kfox1111how about cinder-volume?19:59
satyareven no errors in volume.log19:59
kfox1111debugging turned on?19:59
satyarnope19:59
satyarenabling it19:59
kfox1111openstack's logs are not terribly useful without it. :/19:59
*** Pavo has quit IRC20:01
inc0hmm..I'm wondering...we don't really need fluentd without central logging enabled right?20:03
kfox1111yeah. without elasticsearch, no where for fluentd to push things.20:04
*** Pavo has joined #openstack-kolla20:05
*** salv-orl_ has quit IRC20:05
*** salv-orlando has joined #openstack-kolla20:06
jascott1_o/20:08
jascott1\/\20:08
kfox1111hey20:09
jascott1ahoy?20:13
*** gfidente|afk has quit IRC20:14
inc0that's...an productive conversation20:15
*** mannamne has joined #openstack-kolla20:16
inc0I love our new mascot20:19
*** unicell has quit IRC20:20
kfox1111ar the final mascots out?20:22
*** jtriley_ has joined #openstack-kolla20:22
*** pramodrj07 has joined #openstack-kolla20:24
openstackgerritMerged openstack/kolla-ansible master: Update links in documents  https://review.openstack.org/42839220:24
*** pramodrj07 has quit IRC20:24
*** srwilkers has joined #openstack-kolla20:24
srwilkersafternoon o/20:24
*** jtriley has quit IRC20:25
*** MasterOfBugs has joined #openstack-kolla20:25
*** jheroux has quit IRC20:26
*** esmiurium has quit IRC20:26
*** imcsk8 has joined #openstack-kolla20:28
kfox1111srwilkers: hey.20:29
*** lrensing has joined #openstack-kolla20:30
sbezverksatyar: external iscsi for cinder will not work. currently cinder-volume uses tgtdadm command for colume management20:38
sbezverkand these commands do not have abbility to attach and manage remote iscsi targets20:39
*** adrian_otto has quit IRC20:40
kfox1111sbezverk: I'm guessing with kolla-ansible?20:40
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding compute-kit chart  https://review.openstack.org/42815020:41
sbezverkkfox1111: both20:41
sbezverkbasically cinder-volume container MUST be running on the same node hosting volume group and tgtd process running20:42
satyarsbezverk so if i have tgtd container running then i guess that will only manage local iscsi20:45
sbezverkcorrect20:45
satyaris it required for external also?20:45
*** lrensing has quit IRC20:45
sbezverksatyar: I have never done external iscsi, maybe there are verndor specific drivers allowing to do it. I havenever seen them though20:46
satyari know we have external iscsci running and able to attach volume from icehouse20:46
satyaris there any particular change gone in which blocked that feature20:47
*** lrensing has joined #openstack-kolla20:47
*** skuda has joined #openstack-kolla20:47
satyari am using zfs as iscsi storage20:48
kfox1111satyar: for kolla-kubernetes, we really haven't tested taht use case. if the config file is right though, there is a chance it would work.20:49
kfox1111and if the plugin was loaded into the container already.20:49
satyari am using kolla-ansivle20:49
satyaransible20:49
kfox1111I would guess kolla-ansible's support is more fleshed out there.20:49
kfox1111but have little experience with it.20:49
jascott1kfox1111 dont know they are out but snagged the link outta here the other day: https://drive.google.com/file/d/0B9SxQBJsT7y2Nnk0QjVnSTJ5Skg0NUxpcllvXzI2SXJYZHRr/view20:49
kfox1111jascott1: nice. :)20:50
jascott1better!20:50
kfox1111was just wondering, cause helm has an icon section, that would be nice to point at the official icons. :)20:50
*** kristian__ has joined #openstack-kolla20:50
jascott1no doubt!20:51
jascott1someday it might be in a dashboard even20:51
*** kristian__ has quit IRC20:54
kfox1111+120:55
sbezverkjascott1: really nice!! btw thank you very much for the link you sent, the bootcamp recording is great!20:55
jascott1sbezverk glad to hear it! I enjoyed it also20:56
inc0I need this image21:01
inc0with a caption21:01
inc0U wot m8?21:01
inc0idk, this koala feels....badass21:01
kfox1111indeed. :)21:02
inc0I need some australian to write it in Aussie21:02
inc0you know this meme "u wot m8"?21:06
*** sdake has quit IRC21:06
*** kristian__ has joined #openstack-kolla21:14
*** satyar has quit IRC21:14
*** kristian__ has quit IRC21:19
kfox1111fascinating... https://hub.docker.com/r/libreoffice/online/21:20
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding compute-kit chart  https://review.openstack.org/42815021:21
*** mgoddard_ has quit IRC21:22
*** crushil has joined #openstack-kolla21:26
*** skuda has quit IRC21:26
*** lrensing has quit IRC21:32
*** mgoddard_ has joined #openstack-kolla21:35
*** Jeffrey4l has quit IRC21:35
*** crushil has quit IRC21:35
*** Jeffrey4l has joined #openstack-kolla21:35
*** rwallner has quit IRC21:41
jascott1kfox1111 neato21:42
* jascott1 ponders dwarf fortress as a service...21:42
kfox1111with a helm package. ;)21:43
*** rwallner has joined #openstack-kolla21:45
*** salv-orlando has quit IRC21:45
*** jtriley_ has quit IRC21:45
*** rwallner has quit IRC21:49
jascott1but of course :)21:50
*** kristian__ has joined #openstack-kolla21:52
*** kristian__ has quit IRC21:56
*** adrian_otto has joined #openstack-kolla21:57
*** Pavo has quit IRC22:00
*** Pavo has joined #openstack-kolla22:05
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding compute-kit chart  https://review.openstack.org/42815022:09
*** emispur has joined #openstack-kolla22:09
openstackgerritIan Kumlien proposed openstack/kolla-ansible master: [RFC] Push libvirt-guest settings to the container  https://review.openstack.org/42881522:10
*** skuda has joined #openstack-kolla22:16
*** sp_ has quit IRC22:19
*** sp_ has joined #openstack-kolla22:20
*** srwilkers has quit IRC22:23
*** breitz has quit IRC22:26
*** breitz has joined #openstack-kolla22:27
*** Marcellin__ has quit IRC22:29
*** salv-orlando has joined #openstack-kolla22:29
*** sdake has joined #openstack-kolla22:31
*** sdake has quit IRC22:32
*** sdake has joined #openstack-kolla22:33
*** mannamne has quit IRC22:56
*** mgoddard_ has quit IRC23:00
*** rwallner has joined #openstack-kolla23:02
*** Pavo_ has joined #openstack-kolla23:05
*** adrian_otto has quit IRC23:05
*** Pavo is now known as Guest2970923:06
*** rwallner has quit IRC23:06
*** Pavo_ has quit IRC23:08
*** Pavo has joined #openstack-kolla23:09
*** mgoddard_ has joined #openstack-kolla23:10
*** Pavo has quit IRC23:10
*** Guest29709 has quit IRC23:11
*** Pavo has joined #openstack-kolla23:11
*** sdake has quit IRC23:11
*** Pavo has quit IRC23:17
*** Pavo has joined #openstack-kolla23:17
klindgrenAfternoon23:21
klindgrenI remember seeing a mailing list post about removing COPY_ONCE from kolla?23:21
klindgrenis that still the plan?23:21
kfox1111remove?23:22
kfox1111I haven't heard .23:22
klindgrenhttp://git.net/ml/openstack-dev/2016-12/msg00007.html23:22
kfox1111ah.23:23
kfox1111missed that somehow.23:23
kfox1111kolla-kubernetes is defaulting to copy once.23:23
*** unicell has joined #openstack-kolla23:24
klindgrenI think we are also going to be using copy_once23:24
klindgrenmakes roll back much easier23:24
kfox1111+123:24
kfox1111once you have a container orchestration engine such as k8s, then copy_once makes more sence I think.23:24
klindgrenI managed to answer my own question - I couldn't find the email in my inbox23:25
*** TxGirlGeek has quit IRC23:28
*** adrian_otto has joined #openstack-kolla23:30
*** kbyrne has quit IRC23:32
*** kbyrne has joined #openstack-kolla23:34
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix grafana database creation  https://review.openstack.org/42910923:34
*** saneax-_-|AFK is now known as saneax23:35
*** skuda has quit IRC23:35
*** penfold has quit IRC23:35
*** mgoddard_ has quit IRC23:36
inc0klindgren, I think we won't remove it after all23:37
inc0but really? you're thinking of copy_once? what's your use case?23:37
klindgrenDeployment with new code and deployment with existing code but config update23:40
klindgrencan be handled the exact same way for roll back if needed23:40
inc0yeah I guess, just re-create container23:40
*** kristian__ has joined #openstack-kolla23:42
*** kristian__ has quit IRC23:46
kfox1111inc0: the same use case as before. atomic deployments.23:46
kfox1111k8s provides machinery to allow easy rolling upgrades.23:46
kfox1111and rollbacks.23:47
kfox1111so better to be atomic within the replicasets so rollforward/back is atomic and works the way intended,23:47
kfox1111rather then having random intermediate states.23:47
*** salv-orlando has quit IRC23:48
*** salv-orlando has joined #openstack-kolla23:49
*** salv-orlando has quit IRC23:54
*** salv-orlando has joined #openstack-kolla23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!