Thursday, 2017-10-26

*** kevzha01 has joined #openstack-kolla00:00
*** Pavo has joined #openstack-kolla00:00
*** newmember has quit IRC00:00
*** kevzha01_ has joined #openstack-kolla00:01
*** kevzha01 has quit IRC00:04
*** kevzha01_ has quit IRC00:09
*** yangyapeng has quit IRC00:15
*** yangyapeng has joined #openstack-kolla00:15
*** danardelean has quit IRC00:15
*** glycerine has quit IRC00:19
*** yangyapeng has quit IRC00:20
*** Pavo has quit IRC00:22
*** Pavo has joined #openstack-kolla00:25
*** mummer has quit IRC00:31
*** danardelean has joined #openstack-kolla00:31
*** danardelean has quit IRC00:35
*** kbaegis has joined #openstack-kolla00:41
*** trozet has quit IRC00:45
*** tovin07_ has joined #openstack-kolla00:51
*** heigao has joined #openstack-kolla00:54
*** kiennt26 has joined #openstack-kolla00:55
*** glycerine has joined #openstack-kolla00:56
*** Pavo has quit IRC00:56
*** Pavo has joined #openstack-kolla00:56
*** Pavo has quit IRC00:58
*** rwsu has joined #openstack-kolla01:02
*** dave-mccowan has quit IRC01:05
*** heigao has quit IRC01:09
*** dave-mccowan has joined #openstack-kolla01:12
*** lukepatrick has joined #openstack-kolla01:14
*** lxdong has quit IRC01:15
*** tovin07_ has quit IRC01:16
openstackgerritHeetae Ahn proposed openstack/kolla master: Add RHEL subscription registration  https://review.openstack.org/49514801:18
*** salv-orl_ has joined #openstack-kolla01:18
*** salv-orlando has quit IRC01:21
*** kiennt26 has quit IRC01:23
*** huanxie has joined #openstack-kolla01:24
*** yangyapeng has joined #openstack-kolla01:26
*** yangyape_ has joined #openstack-kolla01:27
*** yangyapeng has quit IRC01:27
*** daidv has joined #openstack-kolla01:27
*** unicell has joined #openstack-kolla01:30
*** unicell has quit IRC01:34
*** kiennt26 has joined #openstack-kolla01:34
*** tovin07_ has joined #openstack-kolla01:41
*** itlinux has joined #openstack-kolla01:51
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Optimize zuul v3 jobs  https://review.openstack.org/51361201:58
*** lxdong has joined #openstack-kolla02:03
lxdongWill session client reduce tokens growing in the keystone database?02:03
korean101Hi guys02:08
korean101http://paste.openstack.org/show/624651/02:08
korean101anyone knows MTU set?02:08
korean101i use nested KVM and Docker02:08
korean101i login to VM but can't reach outside02:09
*** zhubingbing_ has joined #openstack-kolla02:10
*** rwsu has quit IRC02:11
*** lukepatrick has quit IRC02:13
*** unicell has joined #openstack-kolla02:13
*** rwsu has joined #openstack-kolla02:14
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Remove test variable for service_name in panko.conf file  https://review.openstack.org/51364002:15
*** unicell has quit IRC02:17
*** jamesbenson has joined #openstack-kolla02:19
*** vhosako__ has quit IRC02:22
*** vhosakot has joined #openstack-kolla02:22
openstackgerritMerged openstack/kolla stable/pike: Use upgrade rather than create_schema for wather database  https://review.openstack.org/51360902:23
*** unicell has joined #openstack-kolla02:26
*** zhubingbing_ has quit IRC02:27
*** zhubingbing_ has joined #openstack-kolla02:27
*** itlinux has quit IRC02:29
*** unicell has quit IRC02:30
*** masuberu has quit IRC02:34
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/pike: Fix prechecks fail with docker not in deployment host  https://review.openstack.org/51526702:34
*** unicell has joined #openstack-kolla02:38
*** pratapagoutham has quit IRC02:38
*** unicell has quit IRC02:42
*** larsks has quit IRC02:42
*** larsks has joined #openstack-kolla02:44
*** itlinux has joined #openstack-kolla02:45
*** vhosakot_ has joined #openstack-kolla02:47
*** itlinux has quit IRC02:48
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Revert "Remove show_multiple_locations in glance-api"  https://review.openstack.org/50979202:48
*** vhosakot has quit IRC02:48
*** zhubingbing has joined #openstack-kolla02:50
*** unicell has joined #openstack-kolla02:50
*** zhubingbing_ has quit IRC02:51
*** lhx__ has joined #openstack-kolla02:53
*** itlinux has joined #openstack-kolla02:53
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes master: Updated from global requirements  https://review.openstack.org/50944302:54
*** rwsu has quit IRC02:54
*** unicell has quit IRC02:55
*** masber has joined #openstack-kolla02:55
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes master: Updated from global requirements  https://review.openstack.org/50944302:55
*** masber has quit IRC02:57
*** lukepatrick has joined #openstack-kolla02:57
*** masber has joined #openstack-kolla02:57
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes master: Updated from global requirements  https://review.openstack.org/50944302:58
*** dave-mccowan has quit IRC03:10
*** kbaegis has quit IRC03:12
*** caowei has joined #openstack-kolla03:16
*** masber has quit IRC03:19
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Optimise the number of workers for each project  https://review.openstack.org/51527503:22
*** itlinux has quit IRC03:22
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Optimise the "openstack_freezer_auth"  https://review.openstack.org/51527603:28
openstackgerritSongHao proposed openstack/kolla-ansible master: Modify the instructions for setting up vip.  https://review.openstack.org/51208703:38
*** zhubingbing_ has joined #openstack-kolla03:40
*** zhubingbing has quit IRC03:41
*** gkadam has joined #openstack-kolla03:42
*** mdnadeem has joined #openstack-kolla03:54
*** jamesbenson has quit IRC04:00
*** masber has joined #openstack-kolla04:02
openstackgerritMerged openstack/kolla master: Add rsyslog image  https://review.openstack.org/51463204:02
openstackgerritMerged openstack/kolla-ansible master: Update the manila shares link  https://review.openstack.org/51300404:02
*** yangyape_ has quit IRC04:03
*** masber has quit IRC04:04
*** lukepatrick has quit IRC04:04
*** masber has joined #openstack-kolla04:04
masbergood afternoon, my cinder_api container is crashing (centos binary 4.0.0) this is the docker logs output http://paste.openstack.org/raw/624659/. Any thoughts?04:08
*** jamesbenson has joined #openstack-kolla04:16
*** huanxie has quit IRC04:20
*** jamesbenson has quit IRC04:20
masberquestion regarding the documentation page https://docs.openstack.org/kolla-ansible/latest/user/multinode.html should the docker service configuration be "/etc/systemd/system/docker.service" or "/etc/systemd/system/docker.service.d/kolla.conf" or doesn't matter?04:26
*** danardelean has joined #openstack-kolla04:32
*** skramaja has joined #openstack-kolla04:33
*** heigao has joined #openstack-kolla04:34
*** genek has quit IRC04:34
*** danardelean has quit IRC04:36
*** kiennt26 has quit IRC04:36
*** classicsnail has quit IRC04:41
*** classicsnail has joined #openstack-kolla04:41
inc0masber: doesn't matter afair04:42
spsuryamorning guys04:43
inc0newer dockers also works with /etc/docker/daemon.json04:43
inc0to confirm if your config works, you can call docker info04:43
masbertrue04:43
masberso when I do kolla-build --registry XXXXX will it replace the images already exists with the new ones?04:50
*** genek has joined #openstack-kolla04:50
*** huanxie has joined #openstack-kolla04:59
*** vhosakot_ has quit IRC05:04
*** vhosakot has joined #openstack-kolla05:05
*** vhosakot_ has joined #openstack-kolla05:12
*** vhosakot has quit IRC05:14
*** yangyapeng has joined #openstack-kolla05:18
kolla-slack<egonzalez> masber, need to add --push to upload them, --registry only set the registry ip in the image name05:20
*** yangyapeng has quit IRC05:23
*** rwsu has joined #openstack-kolla05:24
*** lrensing has joined #openstack-kolla05:24
*** genek has quit IRC05:26
*** dasTor_ has joined #openstack-kolla05:27
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes master: Updated from global requirements  https://review.openstack.org/50944305:27
openstackgerritwangwei proposed openstack/kolla-ansible master: Optimize the configuration of ceph  https://review.openstack.org/49660805:28
*** dasTor has quit IRC05:30
*** dasTor has joined #openstack-kolla05:31
*** dasTor_ has quit IRC05:31
*** jbadiapa_ has quit IRC05:32
*** genek has joined #openstack-kolla05:39
*** genek has quit IRC05:44
*** dasTor_ has joined #openstack-kolla05:46
*** lrensing has quit IRC05:48
*** dasTor has quit IRC05:49
*** egonzalez has joined #openstack-kolla06:00
*** salv-orlando has joined #openstack-kolla06:03
*** salv-orl_ has quit IRC06:04
masbercan I use /kolla/tools/cleanup-images to delete images from registry?06:05
masberif not how can I delete all images in local registry? I have too many rubish from previous attempts06:06
*** genek has joined #openstack-kolla06:06
*** lrensing has joined #openstack-kolla06:09
*** janki has joined #openstack-kolla06:31
egonzalezmasber, images cannot be removed from registry, only method afik, rm -rf <path_to_registry_volume>/*06:32
*** lrensing has quit IRC06:33
*** lpetrut has joined #openstack-kolla06:33
*** lpetrut has quit IRC06:34
kolla-slack<jascott1> might check to see if `docker prune` could work there06:40
*** kiennt26 has joined #openstack-kolla06:40
*** Zophar has joined #openstack-kolla06:40
*** danardelean has joined #openstack-kolla06:47
masberegonzalez, there is a docker registry API but I could not find the way of using it to delete images, I can query tags and repositories tho06:47
masberegonzalez, how can I find the path to volumes?06:47
masberalso fluentd container is failing, i am using kolla stable/pike branch. I am trying to find the error... could be this one? INFO:kolla.image.build.fluentd:/bin/sh: line 0: ulimit: open files: cannot modify limit: Operation not permitted06:49
masbercentos binary06:50
*** danardelean has quit IRC06:51
openstackgerritJuan Badia Payno proposed openstack/kolla master: WIP - fluentd rpm candidate  https://review.openstack.org/51529206:55
*** jbadiapa_ has joined #openstack-kolla06:57
*** jbadiapa_ is now known as jbadiapa06:57
*** yangyapeng has joined #openstack-kolla07:05
*** salv-orlando has quit IRC07:12
*** salv-orlando has joined #openstack-kolla07:12
*** salv-orlando has quit IRC07:16
*** dciabrin has joined #openstack-kolla07:18
openstackgerritMartin André proposed openstack/kolla master: Add Apache packages to mistral-api container  https://review.openstack.org/51308907:21
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/pike: Move to zuul v3 in project jobs  https://review.openstack.org/51360807:22
*** hachi_ has joined #openstack-kolla07:23
*** hrw has quit IRC07:26
egonzalezzhubingbing_, ping07:28
zhubingbing_hi07:28
openstackgerritMerged openstack/kolla-ansible master: Optimize zuul v3 jobs  https://review.openstack.org/51361207:30
*** huanxie has quit IRC07:31
*** hrw has joined #openstack-kolla07:31
*** danardelean has joined #openstack-kolla07:32
*** huanxie has joined #openstack-kolla07:39
*** salv-orlando has joined #openstack-kolla07:42
*** egonzalez has quit IRC07:44
*** kiennt26 has quit IRC07:48
*** serlex has joined #openstack-kolla07:49
*** paramite has joined #openstack-kolla07:49
*** hachi__ has joined #openstack-kolla07:50
*** hachi_ has quit IRC07:50
*** kiennt26 has joined #openstack-kolla07:51
*** danardelean has quit IRC07:52
*** danardelean has joined #openstack-kolla07:53
*** dangtrinhnt has joined #openstack-kolla07:54
*** danardelean has quit IRC07:57
*** dardelean has joined #openstack-kolla07:59
*** dougsz has joined #openstack-kolla08:02
*** hachi__ has quit IRC08:07
*** hachi__ has joined #openstack-kolla08:07
*** ktibi has joined #openstack-kolla08:18
*** klindgren__ has joined #openstack-kolla08:22
*** danardelean has joined #openstack-kolla08:23
*** david-lyle has quit IRC08:24
*** klindgren_ has quit IRC08:24
*** david-lyle has joined #openstack-kolla08:25
*** danardelean has quit IRC08:27
*** huanxie has quit IRC08:29
*** shardy has joined #openstack-kolla08:29
*** huanxie has joined #openstack-kolla08:35
*** lpetrut has joined #openstack-kolla08:36
*** jiriprox has quit IRC08:39
openstackgerritJuan Badia Payno proposed openstack/kolla master: WIP - fluentd rpm candidate  https://review.openstack.org/51529208:41
openstackgerritzhongshengping proposed openstack/kolla-ansible master: Add binary related jobs to check  https://review.openstack.org/51532508:47
*** lhx__ has quit IRC08:49
*** lhx__ has joined #openstack-kolla08:49
*** heigao has quit IRC08:56
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Optimise the "openstack_freezer_auth"  https://review.openstack.org/51527608:56
masberhi, could someone please give me a hand? cinder_api keeps crashing, these are the docker logs http://paste.openstack.org/raw/624681/08:59
masberI am using 4.0.0 centos binary08:59
*** danardelean has joined #openstack-kolla09:01
*** huanxie has quit IRC09:05
*** danardelean has quit IRC09:06
openstackgerritMartin André proposed openstack/kolla master: Add Apache packages to mistral-api container  https://review.openstack.org/51308909:08
*** danardelean has joined #openstack-kolla09:12
hrwmasber: add setting permissions for /run/httpd09:19
*** david-lyle has quit IRC09:26
*** danardelean has quit IRC09:26
*** dklyle has joined #openstack-kolla09:26
*** huanxie has joined #openstack-kolla09:27
openstackgerritHeetae Ahn proposed openstack/kolla master: Add RHEL subscription registration  https://review.openstack.org/49514809:35
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Add missing hostnames to fluentd log messages  https://review.openstack.org/51459309:38
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Optimise the "openstack_auth" for freezer and horizon  https://review.openstack.org/51527609:40
*** nrado has joined #openstack-kolla09:40
masberhrw, how could I do that is the container is constantly restarting?09:42
*** nrado has quit IRC09:44
openstackgerritDuong Ha-Quang proposed openstack/kolla master: XenServer: Add packages for initial support  https://review.openstack.org/51026609:45
*** danardelean has joined #openstack-kolla09:47
*** danardelean has quit IRC09:48
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Remove the unnecessary horizon_auth  https://review.openstack.org/51534009:52
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Remove the unnecessary horizon_auth  https://review.openstack.org/51534009:55
*** tovin07_ has quit IRC10:04
*** sambetts|afk is now known as sambetts10:09
*** masber has quit IRC10:09
hrwhttps://review.openstack.org/#/c/514614/ - can someone approve new repo for stable/pike? without it pike is useless on aarch64/debian10:09
openstackgerritzhangfei gao proposed openstack/kolla master: ironic-pxe: generate aarch64 bootfile grubaa64.efi  https://review.openstack.org/51254010:10
*** caowei has quit IRC10:15
*** kiennt26 has quit IRC10:16
*** danardelean has joined #openstack-kolla10:25
*** danardelean has quit IRC10:25
*** gfidente|afk is now known as gfidente10:31
openstackgerritMerged openstack/kolla-ansible stable/pike: Fix prechecks fail with docker not in deployment host  https://review.openstack.org/51526710:34
*** vhosakot_ has quit IRC10:34
zhubingbing_hi guys10:35
*** serlex has quit IRC10:38
*** serlex has joined #openstack-kolla10:39
openstackgerritJeffrey Zhang proposed openstack/kolla master: implement build jobs in kolla  https://review.openstack.org/51397110:40
*** serlex has left #openstack-kolla10:41
*** pbourke has quit IRC10:42
*** dasTor_ has quit IRC10:42
*** pbourke has joined #openstack-kolla10:44
gemaguys, are we publishing the x86 images in docker hub?10:44
gemaJeffrey4l: ^10:44
zhubingbing_ocata ?10:46
gemaany images10:46
gemawe are trying to figure out where to put the arm64 ones10:46
zhubingbing_yeah10:46
hrwocata are present iirc10:46
gemahrw: where?10:46
hrwhttps://hub.docker.com/r/kolla/ubuntu-source-collectd/tags/ shows 4.0.0 ones as latest10:47
gemahrw: cannot we put ours there as well?10:47
*** hachi__ has quit IRC10:48
hrwgema: if we are kolla upstream then yes. if we as linaro then we lack access10:48
*** hachi__ has joined #openstack-kolla10:48
gemahrw: let's add the relevant bits to CI in openstack so that our images end up there then10:48
gemabut it loks to me as if the publishing is rather sporadic10:48
gemaI wonder if it was done manually or how10:48
gemaor if that namespace is just abandoned10:49
zhubingbing_ Jeffrey4l know it10:49
gemaok, we'll wait for his answer10:49
zhubingbing_I'll call him for you waiting10:50
gemathanks, it's ok though10:53
gemahe'll come at some point :D10:53
hrwand then I have a question to him too ;D10:53
hrwwe lack 6.0.0 and 5.0.1 tags on kolla ;D10:53
Jeffrey4lgema, before we pushed images to docker hub manually. now we are trying to push the images automatically , based on zuul v3.10:54
Jeffrey4l5.0.0 images is never be pushed. and 5.0 for centos and oraclelinux should be dead duo to some bugs in centos 7.410:54
hrwJeffrey4l: hi10:57
Jeffrey4l5.0.1 tags should be tag soon and push the images to docker hub.11:02
Jeffrey4lbased on the policy, 6.0.0 images won't be pushed until kolla release 6.0.0 image.11:02
hrwJeffrey4l: asking about 5.0.1 cause recent images have this tag but no tag on git repo11:02
hrwhttp://git.openstack.org/cgit/openstack/kolla/refs/tags11:03
*** huanxie has quit IRC11:03
hrwJeffrey4l: and once 6.0.0 tag appear I can go and build queens images for tests without using "--tag queens" ;D11:03
Jeffrey4lsup hrw11:06
* hrw wishes for --extra-image-prefix argument11:09
hrwhaerwu/kolla-is-omg-so-cool-what-it-deserves-extras-debian-source-base ;d11:09
gemaJeffrey4l: the reason is because I was asking to put images on linaro docker as linar/kolla-openstack-imagename11:11
gemalinaro/kolla-openstack-imagename11:11
gemasince we have other images at linaro as well, I wanted to distinguish between then11:12
gemathem11:12
Jeffrey4lgema, i think that is cool.11:16
Jeffrey4lpush arm images is a good idea.11:16
Jeffrey4lit will be helpful for users.11:17
gemaJeffrey4l: the question is where we push them11:20
gemaofficial kolla repos11:20
bjololastlog mtu11:20
gemaor linaro11:20
*** masber has joined #openstack-kolla11:22
masber?msg NickServ identify masuberu11:23
Jeffrey4ldoes docker support multi arch now?11:23
gemaJeffrey4l: it does, you can have labels with the arch, but hrw can explain better11:24
hrwnever built multiarch image yet. All main distros used by Kolla (centos, debian, ubuntu) provide now mutliarch images11:25
Jeffrey4li think it is OK to  push the arm64 images to kolla namespace11:26
gemaJeffrey4l: ok, so we'll need to figure out how to do that and help with the zuul v3 automation11:27
hrwgema: to kolla/ namespace we can push 5.0.1 ones. 6.0.0 for tests will need place anyway11:27
gemaJeffrey4l: I'd like to understand the images lifecycle, when do you push images to docker hub11:28
*** jamesbenson has joined #openstack-kolla11:29
*** janki has quit IRC11:31
hrwgema: I see kolla/ on docker hub as space for stable tested images. so up to pike ones for now11:33
*** jamesbenson has quit IRC11:33
gemahrw: ok11:35
gemahrw: maybe we should add the option to have an image prefix11:35
gemabut before we go for that I'd like to know how that would affect kolla-ansible and kolla-k8s11:35
gemaand make it superoptional11:36
*** danardelean has joined #openstack-kolla11:41
*** danardelean has quit IRC11:41
*** danardelean has joined #openstack-kolla11:41
*** egonzalez has joined #openstack-kolla11:42
*** danardelean has quit IRC11:45
*** ansmith has quit IRC11:47
*** egonzalez has quit IRC11:51
*** dave-mccowan has joined #openstack-kolla11:51
*** jmccarthy has joined #openstack-kolla11:54
hrwgema: good part is that image-prefix is set in one place so patch would not complicate much11:54
*** magicboiz has quit IRC11:57
hrweven more fun.12:02
hrwwith local registry you can do whatever you want cause kolla_base_distro is set in ansible/group_vars/all.yml so namespace/whatever-you-want-distro-type-imagename is possible12:03
hrwextra note in documentation would solve that12:04
*** salv-orl_ has joined #openstack-kolla12:06
hrwnope. checks for kolla_base_distro exist12:06
lhx__hi guys, I'm very appreciate it if you spare time to have a look at this: https://bugs.launchpad.net/kolla-kubernetes/+bug/172678412:07
openstackLaunchpad bug 1726784 in kolla-kubernetes "can't get mirror with default repo url" [Undecided,New] - Assigned to Hanxi Liu (hanxi-liu)12:07
kolla-slack<egonzalez> As far i remember images are not pushed to dockerhub for various reasons12:07
kolla-slack<egonzalez> Extra work to do it manually for all distros/types12:07
kolla-slack<egonzalez> Not tested images12:07
kolla-slack<egonzalez> A decision in governance to not do it until we have an automatic method12:07
*** salv-orlando has quit IRC12:09
kolla-slack<masber> @egonzalez from where is docker-build getting the docker files? tarball?12:19
*** shardy has quit IRC12:20
rwellumlhx__: What is this vagrant environment you are using?12:21
lhx__rwellum, thanks for your reply, I use Vagrant 1.8.112:23
rwellumgema: there's also the tarballs location - that's where I am getting pike and master images from now. While wait on zuulv312:24
gemarwellum: where are they?12:25
rwellumlhx__: I mean this halcyon dev environment has been abandoned for a long time AFAIK.12:26
rwellumgema: http://tarballs.openstack.org/kolla/images/12:26
lhx__rwellum, sorry, I'm the new for kolla, I just built the environment according to the guide: https://docs.openstack.org/kolla-kubernetes/latest/development-environment.html12:28
rwellumlhx__: there's a big warning at the top of that page :)12:28
rwellumlhx__: You should start here: https://docs.openstack.org/kolla-kubernetes/latest/deployment-guide.html12:29
rwellumlhx__: Also please try: https://github.com/RichWellum/k8s12:29
rwellumI'm committing this to kolla-kubernetes - it's in review.12:30
lhx__rwellum, Oh, damn it12:30
bjolohey is the newton branch removed from kolla repo?12:30
gemahrw: ^12:30
rwellumlhx__: sorry!12:30
kolla-slack<egonzalez> masber, kolla-build renders dockerfiles from <repo>/docker folder and build from them12:30
lhx__rwellum, It's my negligence, I will try the new guide12:32
lhx__lol12:32
gemarwellum: is there a patch for your k8s new guide?12:32
rwellumnp - I started there 1 year ago - when it was still relevant and it still didn't work. That's why we built the other guide.12:33
gemarwellum: I can put kevinz on the right path if there is12:33
lhx__rwellum, btw, there is a trivial fix: https://review.openstack.org/#/c/514511/12:33
gemarwellum: I did ask him to fix the docs for the bug he found in the guide12:33
bjolohttp://paste.openstack.org/show/624703/12:33
bjolofresh git clone12:34
rwellumgema: https://review.openstack.org/#/c/487972/ - but I have like a 900 line change to it - haven't re-committed because I wanted to get Pike up and running.12:34
gemarwellum: ack12:34
gemawill give this info to him anyway12:34
gemabetter to put effort on the new thing12:34
gemathan on the obsolete one12:34
rwellumHe can use my github version for now gema  - it brings up Ocata like a champ.12:35
gemarwellum: ack12:35
lhx__rwellum, thanks guy :)12:35
rwellumlhx__: I just gave it +W so it should merge soon.12:35
lhx__I saw12:36
lhx__:)12:36
openstackgerritJuan Badia Payno proposed openstack/kolla-ansible master: WIP - fluentd rpm candidate test  https://review.openstack.org/51538912:36
rwellumCool - and lhx__ glad to have you onboard and trying this out.12:36
rwellumgema: I got 'added' to a hangouts group for the Upstream Institute for Sydney, apparently I am supposed to talk about containers. And they woke me up 4:15am this morning - texting to the group....12:37
*** jinke_ has quit IRC12:37
gemarwellum: omg, don't sleep next to your phone!12:38
lhx__rwellum, sure, I will try it , hope everything goes well  :-D12:38
rwellumI know right... :(12:38
gemarwellum: I have disabled most notifications when I was travelling and forgot to enable them back12:38
gemaso I am having a good relaxed time xD12:38
gemaphone-wise12:38
rwellumYeah much better - until I ignore all the texts from my wife and kids...12:38
*** jinke_ has joined #openstack-kolla12:38
rwellumFor too long12:38
gemahaha12:39
openstackgerritMerged openstack/kolla-kubernetes master: Remove extra characters  https://review.openstack.org/51451112:39
*** ansmith has joined #openstack-kolla12:41
rwellumgema: be cool if kevenz sees the same pike issues I am - there's definitely something messed up with libvirt.12:41
gemarwellum: yep12:46
gemarwellum: you'll have to talk to him on your evening time12:46
gemarwellum: he is off today though12:46
rwellumNot if I got up at 4:15am I won't.12:46
gemarwellum: I don't mean today, I mean in general x D12:48
bjolohrw: around12:49
bjolodo you have info about newton branch and pip packages?12:50
bjolothey all seem to be removed12:50
bjolothis is not cool. we have production clouds runing newton12:50
rwellumYeah I know... 😃12:50
*** goldyfruit has joined #openstack-kolla12:53
bjolopbourke: ^^12:53
*** skramaja has quit IRC12:53
genekHi, have anyone tried building images for arm64 architecture?12:56
*** shardy has joined #openstack-kolla12:57
rwellumbjolo: images or kolla itself?12:59
rwellumhttps://releases.openstack.org/teams/kolla.html - I still see Newton12:59
*** rhallisey has joined #openstack-kolla13:00
*** lhx__ has quit IRC13:03
ktibiHi, I try to use ceph but libvirt have error : libvirtError: Secret not found: no secret with matching uuid 'XXXXXX'13:06
ktibikolla configure secret for libvirt ?13:06
bjolokolla itself13:08
bjolohttp://paste.openstack.org/show/62470313:09
bjolopip install kolla-ansible also does not have any newton version13:10
bjolobut maybe thats in pip install kolla13:10
bjolook pip kolla newton is still there13:11
ktibiok need to restart libvirt container ><13:12
bjolobut why is the branch removed?13:12
rwellumbjolo - I don't see it as a branch either https://www.irccloud.com/pastebin/7nMlSfah/13:15
*** jaosorior has quit IRC13:16
*** mdnadeem has quit IRC13:17
genekGot error creating aufs mount to /var/lib/docker/aufs/mnt/ : invalid argument when building neutron-server on arm6413:17
genekAny ideas of possible causes to this error?13:18
genekother images build fine13:18
bjolorwellum: nope its gone afaict13:18
bjolorwellum: it was there just few weeks ago13:19
bjoloi did a build on the latest newton tarballs13:19
bjoloanyone knows when and who? most importantly why?13:20
*** sbezverk has joined #openstack-kolla13:20
rwellumIt's a question to inc0 or some other cores - I don't know.13:20
bjoloits one thing to not have support for an old version, but removing the branch is straight up a bad idea13:20
rwellumYeah there's no point to it either - not like it takes up much space etc.13:21
rwellummark it as unsupported maybe.13:21
bjoloinc0: should be online in a few13:22
bjolosdake: are you online yet. Usually up early :)13:23
sdakesup bjolo13:24
bjolohey sdake13:24
bjoloi knew you where up :)13:24
bjoloit seems like the newton and mitak branch is removed from kolla repo13:24
bjoloyou know why?13:25
sdakeopenstack supports upstream n and n-113:25
bjolohttps://www.irccloud.com/pastebin/7nMlSfah13:25
openstackgerritMark Duggan proposed openstack/kolla-ansible master: Fixing missing copy of iscsi container config files  https://review.openstack.org/51540613:25
sdakethat is pike and ocata13:25
sdakei am not sure why newton was removed13:25
sdakeas pike hasn't been released yet afaik :)13:25
bjoloyes support is one thing but removing the branches?13:25
*** gkadam_ has joined #openstack-kolla13:26
sdakeyes the branches  are removed because the stable team thinks that having them results in a situation where the branch must be maintained13:26
sdakepersonally I see no harm in keeping branched around and labeling them as unsupported13:26
sdakehowever the fundamental purpose of a branch is to create something supported for backports13:26
bjolois this true for all openstack projects or just kolla*13:26
*** gkadam has quit IRC13:27
sdakeif you need an older version hyou could try the tarballs13:27
sdakebjolo yes true for all os projects13:27
bjolook13:27
sdakethis is the value of buying openstack from a vendor13:27
rwellumsdake: but this is the kolla repo - not the images13:27
bjoloi know13:27
bjolobut kolla just had one repo for newton13:27
*** rmart04 has joined #openstack-kolla13:27
bjolokolla, kolla-ansible, kolla-kub shared it13:28
sdakei dont agree with it but understand why it is there13:28
bjoloi dont agree either13:28
sdakeso if you are asking me to defend the decision, your asking the wrong dude13:28
sdakeif your asking me to fight it, your asking the wrong dude :)13:28
bjolono not asking you13:28
rwellumBut anyway - until Pike is released we should get Newton branch back.  Who does this?13:28
bjoloi did not know this is how it was. just noticed that kolla newton branch was gone and we run a prod cloud on that13:29
bjolodont feel comfortable not have  the source tree for something i have in prod13:29
sdakeright13:30
bjoloin my world you never delete your history13:30
sdakei agree - deleting anything from git is terrible :)13:30
sdakethe original model of openstack was to be an upstream for downstream software distributors13:30
sdakesome people in the community (sounds like you) want openstack to be a community supported LTS type producvt13:31
ktibiHi all I found bug with glance + ceph with snapshot nova. Need to change show_image_direct_url to show_multiple_locations https://review.openstack.org/#/c/279630/13:31
sdakeopenstack doesn't have LTS13:31
sdakeI don't knwo why - I think mainy because the cost is very high13:31
ktibithe change is here https://github.com/openstack/kolla-ansible/blob/2e6fa4fe15025eaa1d4110b882e7522cd814491b/ansible/roles/glance/templates/glance-api.conf.j2#L1513:31
ktibiI test on fresh install and it works13:31
sdakektibi can you submit a patch via gerrit plz13:32
ktibiyes13:32
ktibisdake do you have tuto ?13:32
sdakehttps://docs.openstack.org/infra/manual/developers.html13:33
sdakebjolo i'd recommend upgrading from newton prior to the removal of the newton branch :)13:34
bjolosdake: we are working on that13:34
sdakecool13:34
bjoloi needed the newton branch in the lab exactly for that reason. testing the upgrade13:35
sdaketarballs?13:35
bjoloknowing this we need to setup our own git mirrors13:35
sdakewe don't recommend working from branches anyway - only tarballs13:36
*** gkadam_ has quit IRC13:37
bjolowell... old habit. cherrypick options, etc13:37
*** danardelean has joined #openstack-kolla13:38
bjolobut OS internally must have a golden version of the git tree right?13:39
bjoloi mean, what if tarballs.openstack got hacked. how are they able to recreate/verify the tarballs otherwise13:40
*** pratapagoutham has joined #openstack-kolla13:45
masberhi all, I used pike images from tarbal as I could not make the kolla-build to work, deployment is fine, I can create an instance and ssh through the router namespace but connection to outside world does not work, any thoughts?13:47
*** magicboiz has joined #openstack-kolla13:48
*** dosaboy has quit IRC13:48
ktibisdake ok ubuntu.com down gerrit next time ^^13:49
*** jaosorior has joined #openstack-kolla13:53
*** goldyfruit1 has joined #openstack-kolla13:54
*** egonzalez has joined #openstack-kolla13:55
*** goldyfruit has quit IRC13:55
*** hongbin has joined #openstack-kolla13:56
*** jinke_ has quit IRC13:56
*** jinke_ has joined #openstack-kolla13:57
egonzalezktibi, what release deploying? In master this is the revert for the issue which will need to be backported to stable branch14:00
egonzalezhttps://review.openstack.org/#/c/509792/14:00
*** dosaboy has joined #openstack-kolla14:00
ktibi5.0.014:01
ktibiok patch already exist14:01
*** jtriley has joined #openstack-kolla14:03
egonzalezPike is released already, newton is being removed now14:03
egonzalezbjolo, you can download latest change in git in newton-eol branch14:03
*** goldyfruit1 has quit IRC14:03
ktibido you have plan for a 5.1.0 or 5.0.1 ? with all minor fix like the qemu.conf or glance-api.conf ?14:03
egonzalezktibi, yes, is in release calendar14:05
egonzalezbut looks like we missed the date Oct 16 - Oct 2014:06
ktibiok ^^, do you have link for calendar ?14:06
egonzalezktibi, queens https://releases.openstack.org/queens/schedule.html , all https://releases.openstack.org/index.html14:06
ktibihum ok this is the official calender for openstack14:07
*** jbadiapa has quit IRC14:07
egonzalezktibi, yep, there is a schedule ics in "combined release calendar" link to apply to your calendar and be always updated14:08
*** dklyle has quit IRC14:15
masberegonzalez, did suggested me to add kolla/centos-source-manila-data in order to make manila to work?14:18
egonzalezmasber, not remember that but maybe I did, who knows :)14:22
masbermmm just wondering because it is not in the documentation so someone may have told me to put it14:23
*** newmember has joined #openstack-kolla14:25
egonzalezmasber, not sure if manila-data is needed in ocata, in pike binary package exists for sure, but in ocata rpm package was missing in the image14:25
masberthe thing is that I have this in my globals for ocata manila_data_image_full: kolla/centos-source-manila-data:4.0.0, now I wanted to do same for pike, like manila_data_image_full: kolla/centos-source-manila-data:5.0.1 but ansible complains about NotFound: 404 Client Error: Not Found (\"manifest for kolla/centos-source-manila-data:5.0.1 not found14:26
masberegonzalez, ^^14:26
egonzalezmore than missing, installed along manila-common in rpm packages14:26
egonzalezmasber, in pike not need to use source image anymore14:26
egonzalezmasber, images with tag 5.0.1 are not pushed to dockerhub14:27
masberyes, kolla is my local repo14:27
masberI got the images from tarbal14:28
egonzalezmasber, did you build also source images?14:28
masberonly binaries14:28
egonzalezmasber, in pike, remove the  manila_data_image_full from globals and try again14:28
egonzalezimage exists in binary14:28
masberok14:28
sdakebjolo i dont have the answer to your question about doomsday scenarios with tarballs.oo14:29
masberjust enable_manila: "yes"14:29
masber and enable_manila_backend_generic: "yes"14:29
masber right?14:29
sdakebjolo I do assume it is properly archived in multipel places :)14:29
sdakebad to make assumptions :)14:29
sdakeII can find out if you like14:30
ZopharQuick question: Could we re-trigger CI on review (with our "poor" permissions") or i should ask someone to do that?14:30
masberalso manila client is not installed by default14:30
masberneeds to be installed manually14:30
Zopharhttps://review.openstack.org/#/c/514676/14:30
egonzalezmasber, you can cherry-pick this change to stable/ocata, forgot to do that https://review.openstack.org/#/c/465018/14:30
kolla-slack<inc0> Bjolo you can access old code via git tags14:31
egonzalezmasber, client is user task not deployers14:31
masberunderstood14:31
kolla-slack<inc0> There is also discussion on summit about lts14:31
egonzalezbjolo, sdake, git repo has a tag with newton-eol with latest state of the newton branch14:31
egonzalezhttps://github.com/openstack/kolla/tree/newton-eol14:32
*** genek has quit IRC14:32
egonzalezZophar, just put recheck in comment box14:32
Zopharok i did... so i wait ;)14:33
Zopharthanks14:33
*** lrensing has joined #openstack-kolla14:46
*** zhubingbing__ has joined #openstack-kolla14:46
*** david-lyle has joined #openstack-kolla14:48
hrwbjolo: look at newton-eol branch14:49
*** lrensing has quit IRC14:50
*** kbaegis has joined #openstack-kolla14:55
*** kbaegis has joined #openstack-kolla14:55
*** kbaegis has joined #openstack-kolla14:55
*** kbaegis has quit IRC14:55
*** kbaegis has joined #openstack-kolla14:56
*** kbaegis has quit IRC14:56
*** kbaegis has joined #openstack-kolla14:56
*** kbaegis has joined #openstack-kolla14:57
*** jamesbenson has joined #openstack-kolla14:57
*** rmart04 has quit IRC14:58
*** zhubingbing__ has quit IRC14:58
*** zhubingbing has joined #openstack-kolla14:58
*** rich1iu has joined #openstack-kolla14:59
*** genek has joined #openstack-kolla14:59
*** genek has quit IRC14:59
*** genek has joined #openstack-kolla15:00
*** trozet has joined #openstack-kolla15:00
*** gkadam_ has joined #openstack-kolla15:02
*** jaosorior has quit IRC15:02
*** jbrooks has quit IRC15:04
hrwgema: ?15:06
*** egonzalez has quit IRC15:09
masberegonzalez, so to have ephemeral in local storage I need to have this in globals15:09
masbernova_backend_ceph: "no"15:09
masbernova_backend: "{{ 'rbd' if nova_backend_ceph | bool else 'default' }}"15:09
masberis that right?15:09
gemahrw: sorry on a meeting now15:09
hrwgema: np15:09
gemahrw: what was the question?15:10
hrwgema: you pinged me 3h ago15:10
gemahrw: irrelevant, I cannot remember anymore :D15:10
hrw;)15:10
gemasorry!15:11
*** kbaegis has quit IRC15:11
*** genek has quit IRC15:12
*** kbaegis has joined #openstack-kolla15:13
*** kbaegis has quit IRC15:13
*** kbaegis has joined #openstack-kolla15:13
*** jbrooks has joined #openstack-kolla15:14
*** lukepatrick has joined #openstack-kolla15:15
kolla-slack<egonzalez> masber, yep15:16
*** lrensing has joined #openstack-kolla15:18
* hrw off15:19
*** newmember has quit IRC15:19
*** newmember has joined #openstack-kolla15:19
*** hachi__ has quit IRC15:23
kfox1111morning.15:27
ktibiWhen I try to re-deploy ceph, Checking free port for RadosGW ==>  FAILED! => {"changed": false, "elapsed": 1, "failed": true, "msg": "Timeout when waiting for 172.16.7.2:6780 to stop."}15:31
ktibiI guess kolla doesn't try to stop container before no ?15:32
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode scenarios in zuulv3  https://review.openstack.org/51277915:39
*** itlinux has joined #openstack-kolla15:39
*** trozet has quit IRC15:45
*** klindgren_ has joined #openstack-kolla15:48
*** klindgren__ has quit IRC15:50
rich1iuHi, one question : Can kolla run on ARM64 platform ?15:51
*** iniaziaway is now known as iniazi15:55
*** athomas has quit IRC15:57
kolla-slack<masber> @egonzalez ok, I think I got the ephemeral on local disks setup, I have created a flavor and an instance... however the ephemeral disk should live in the /var/lib/nova/instances/<instance_id> on the same compute node hosting the instance but the folder is missing...16:01
*** danardelean has quit IRC16:05
ktibimasber with local storage you can't use live migration, ok for you ?16:07
openstackgerritJeffrey Zhang proposed openstack/kolla master: implement build jobs in kolla  https://review.openstack.org/51397116:07
masberktibi, yes no problem, this is for ephemeral disks only, I can always create volumes on ceph and attach it to the instance16:08
masberktibi, and live migration is not a problem for me. I don't run 24h services but more of a batch jobs16:09
masberso if the node dies it can be recreated16:09
ktibiok ;)16:09
masberephemeral is just for dirty IOps operations16:09
masbermeans scratch data16:09
ktibifor your pb, with /var/lib/nova/instances/, you check in the compute container ?16:09
masbermmm not on the container16:10
masberwhich container?16:10
masbernova_compute?16:10
ktibinova compute16:10
ktibior maybe libvirt16:10
masberktibi, are ephemeral disks supposed to be thin provisionning?16:12
masberI think I found it but the size doesn't match16:13
masber46M     /var/lib/docker/volumes/nova_compute/_data/instances/98c0618a-b3bc-476d-9d35-93b6581d793e16:13
masberI specified 512Gigs of ephemeral but the instance folder only has 46M?16:14
ktibiyes thin16:14
masberok16:14
masberthanks!16:14
ktibicheck Id of instance16:15
ktibimasber Maybe libvirt use link clone, 46M is just the delta between image and VM16:17
masberktibi, yes the folder name matches the instance id16:17
*** zhubingbing has quit IRC16:17
*** zhubingbing has joined #openstack-kolla16:18
masberktibi, what does it means --? "libvirt use link clone, 46M is just the delta between image and VM"?16:18
masberyou mean that delta is the boot drive?16:18
bjoloif i do a pip install kolla, how do i generate kolla-build.conf?16:18
masberbjolo, pip install tox then tox -e genconfig16:19
bjoloyes16:19
bjolobut in what dir?16:19
masberkolla dir16:20
rwellumhttps://docs.openstack.org/kolla/latest/admin/image-building.html bjolo16:20
masberroot kolla dir16:20
bjoloyes but pip install kolla is all over the place16:20
masberI don't know where pip installs kolla I clone from github16:20
bjoloi usually do to16:20
bjolobut since newton branch is removed now... and i need to build newton16:21
masberbjolo, look for a tox.ini?16:21
*** mummer has joined #openstack-kolla16:23
bjolonot found16:23
bjoloupdatedb; locate tox.ini16:23
rwellumhttps://www.irccloud.com/pastebin/SW6OqBGR/16:24
bjolothe docs are missing some steps16:24
bjolorwellum: i dont have the git source16:24
bjolono newton branch right16:24
bjoloi have done a pip install kolla16:24
bjolono tox.ini afaict16:25
rwellumbjolo: what about the the git tags mentioned above?16:25
bjoloah ok saw that now16:25
bjolokewl16:25
bjolobut anyhow, the docs ....16:26
rwellumThe docs need some work - I agree. But they are expecting the source16:26
ktibimasber I'm not sure but I guess if you deploy 50VM with one image, nova don't provide 50 disk with image size16:26
bjolowhy do we even have pip install kolla then for ocata and pike. newton i understand since kolla-ansible is in there16:27
ktibiI add designate to my kolla and I kill horizon T_T16:27
ktibicentos-binary-horizon:5.0.0                     "kolla_start"            6 minutes ago       Restarting (1) Less than a second ago                          horizon16:27
masberktibi, I am testing that now, I am going to put some data into the ephemeral disks and see if the instance folder grows16:27
ktibiNeed to rebuild horizon image for add designate plugin ?16:27
rwellumbjolo - what I do:  https://www.irccloud.com/pastebin/ASdyUWS9/16:28
ktibiok horizon fail.. cp: cannot stat '/usr/lib/python2.7/site-packages/designatedashboard/enabled/_*[^__].py': No such file or directory16:29
*** lukepatrick has quit IRC16:31
bjolowhat branch should i checkout the newton-eol tag on?16:31
bjologit checkout tags/newton-eol -b <branch>16:32
bjoloshould i be on master branch?16:32
*** zhubingbing has quit IRC16:32
ktibiHow Can I add designatedashboard in horizon container ???16:36
*** magicboiz has quit IRC16:37
masberktibi, yes the disk grows so it is thin provisioning16:37
masberthanks for your help!16:37
masber:)16:37
ktibifound https://review.openstack.org/#/c/480325/1/docker/horizon/Dockerfile.j2 Thx for centos ;)16:40
rwellumbjolo: ni actually - what options do you have?16:42
*** lpetrut has quit IRC16:43
*** mwynne has joined #openstack-kolla16:50
*** lukepatrick has joined #openstack-kolla16:52
openstackgerritMark Duggan proposed openstack/kolla-ansible master: Fixing missing copy of iscsi container config files  https://review.openstack.org/51540616:52
*** jmccarthy has left #openstack-kolla16:53
*** ntpttr_laptop has joined #openstack-kolla16:53
*** sambetts is now known as sambetts|afk16:58
*** danardelean has joined #openstack-kolla16:58
*** danardelean has quit IRC16:59
openstackgerritMartin André proposed openstack/kolla-ansible master: [WIP] Use local cirros image if available  https://review.openstack.org/49844917:00
*** rwsu has quit IRC17:13
*** ntpttr_laptop has quit IRC17:18
*** ntpttr_laptop has joined #openstack-kolla17:21
glycerineis there any sort of workaround for using ceph with ubuntu pike images? I get "no active mgr" and 0 pools using binary or source.17:22
SamYapleglycerine: kolla-ansible doesnt have support for luminous, and jewel doesnt work with the Pike images to my understanding17:23
SamYapleglycerine: so you have to go external ceph on the situation17:23
hrwglycerine: you need ceph luminous support. it is not present even in kolla/master yet17:24
*** rwsu has joined #openstack-kolla17:26
glycerinealright thanks for the help17:26
hrwrwellum: can you take a look at https://review.openstack.org/#/c/514614/ one? It adds repo for pike/debian which we need to be able to use pike on aarch64 at all. packages for x86-64 are provided as well.17:27
kolla-slack<carlp> Greetings! I'm trying to build and push the kolla images for pike based on centos to a local registry and it's not working. Many images do not build, and the ones that do build give an error like: ```INFO:kolla.image.build.opendaylight:Trying to push the image ERROR:kolla.image.build.opendaylight:Unknown error when pushing Traceback (most recent call last):   File17:29
kolla-slack"/home/carperry/.virtualenvs/py3-kolla/lib/python3.5/site-packages/kolla/image/build.py", line 217, in run     self.push_image(image)   File "/home/carperry/.virtualenvs/py3-kolla/lib/python3.5/site-packages/kolla/image/build.py", line 238, in push_image     stream = json.loads(response)   File "/usr/lib/python3.5/json/__init__.py", line 312, in loads     s.__class__.__name__)) TypeError: the JSON object must be str, not 'bytes'17:29
kolla-slackINFO:kolla.image.build:Attempt number: 2 to run task: PushTask(opendaylight) ```  any ideas?17:29
*** dougsz has quit IRC17:29
*** newmember has quit IRC17:33
hrwhttps://bugs.launchpad.net/kolla-ansible/+bug/1727795 - low priority for now but will have to be sorted out17:34
openstackLaunchpad bug 1727795 in kolla-ansible "Cirros image is hardcoded to x86-64" [Undecided,New]17:34
*** gkadam_ has quit IRC17:40
*** newmember has joined #openstack-kolla17:50
pratapagouthamanybody knows how to fix this error??    "pip install -U pip when i do this why is it throwing some locale error i tried google and tried almost all nothin worked "17:51
kolla-slack<carlp> Fixed my problem, I installed the registry cert but forgot to restart the docker service. Sorry!18:05
*** kbaegis has quit IRC18:11
*** lpetrut has joined #openstack-kolla18:18
*** danardelean has joined #openstack-kolla18:19
SamYaplehrw: deploying cirros is not a requirement for kolla-ansible. if i remember, it runs in init-runonce, which ive never used on my production clusters18:20
*** hachi__ has joined #openstack-kolla18:20
SamYaplehrw: but just do a bit of an ENV update there and youll be fine18:20
*** paramite has quit IRC18:20
*** hachi__ has quit IRC18:25
*** jamesbenson has quit IRC18:33
*** lrensing has quit IRC18:36
*** jamesbenson has joined #openstack-kolla18:39
rwellumhrw: done18:48
sdakeSamYaple hrw - luminous is  new vrsion of ceph?18:49
*** ntpttr_laptop has quit IRC18:51
SamYapleya. latest lts18:51
jamesbensonsdake: 12.2.1 L.18:58
SamYaplejamesbenson: 12.2.2 youre behind a version!18:59
SamYaplejamesbenson: oh nevermind. your right, i forgot i was running devel18:59
SamYapleheh18:59
jamesbensonsamyaple: ha! ;-)19:00
*** hachi__ has joined #openstack-kolla19:00
jamesbensonsamyaple: have you gotten this message before on the gui: POOL_APP_NOT_ENABLED: application not enabled on 2 pool(s)19:02
*** lpetrut has quit IRC19:04
*** lpetrut has joined #openstack-kolla19:04
SamYaplejamesbenson: yup19:05
*** danardelean has quit IRC19:05
SamYaplein luminous you should set teh application for the pools19:05
SamYapleso `ceph osd pool set application glance rbd19:05
SamYaplesomethign like that19:05
SamYapledont have t odo it for cache layers and stuff19:05
*** hachi__ has quit IRC19:14
*** danardelean has joined #openstack-kolla19:15
*** lukepatrick has quit IRC19:15
*** danardelean has quit IRC19:15
*** hachi__ has joined #openstack-kolla19:19
rwellumjamesbenson: do you use your single NIC concept much?19:20
*** gfidente has quit IRC19:20
jamesbensonWe primarily have 2 nics here... one public, one private with no internet access19:22
jamesbensonrwellum^19:22
rwellumOk - just curious because I have a case where I might need to deploy without that second nic19:23
rwellumhttps://review.openstack.org/#/c/484006/5/doc/network-interface.rst - will give this a go19:24
jamesbensonyeah, I think I tested that on our dev cluster, so it should work...19:26
*** kbyrne has quit IRC19:27
*** ntpttr_laptop has joined #openstack-kolla19:28
rwellumYou use/setup br0 - but br-ex would be a bit better for the documentation because that's the default in the kolla documentation.19:28
rwellumimo19:29
*** kbyrne has joined #openstack-kolla19:30
jamesbensonsamyaple, should gnocchi be rbd?19:31
jamesbensontrue, good point.19:31
jamesbensonsamyaple: FYI: sudo ceph osd pool application enable volumes rbd19:32
SamYaplejamesbenson: yea its backed into my provisioning so i dont remember the exact syntax :)19:33
SamYaplewhat does gnocchi store its data as?19:33
jamesbensonsamyaple: I guess the automatic tagging doesn't work right off the bat... http://ceph.com/community/new-luminous-pool-tags/#Automatic_Tagging19:33
jamesbensonthe command: sudo ceph osd pool application set images rbd.  Does work, but I needed to enable them first, so that other command does it all for me.19:34
SamYaplejamesbenson: automatic-tagging is for upgraded clusters, not newly create pools to my understanding19:34
jamesbensonah, okay, makes sense...19:34
jamesbensonI was going to say, all of those volumes exist here and none are tagged...19:34
SamYaplejamesbenson: anyway if cephfs, rbd, or rgw doesnt fit your application, you can name it something free-form (for example if gnocchi is just injecting raw objects into librados, it wouldnt fit any of that)19:35
SamYapleit needs *a* name, but the only builtin optimizations exist for those 3 above19:36
*** lukepatrick has joined #openstack-kolla19:38
SamYaplejamesbenson: ah even better. as of now the application tags do nothing. its for future use. so you can name them whatever without affecting anything19:38
jamesbensonsamyaple:  I don't personally used gnocchi at all...19:38
*** hrw has quit IRC19:38
jamesbenson:-\19:39
*** hrw has joined #openstack-kolla19:41
hrwSamYaple: (cirros) it may not be a requirement but I opened bug as kind of reminder for me/linaro to make it handled19:42
hrwsdake: luminous adds new service: ceph-mgr and so far no one wrote image for it19:43
jamesbensonSamYaple hrw I made my own little script to go and deploy public/private network/router/cirros and SSH into it to verify basics work...19:43
hrwjamesbenson: init-runonce does that19:43
hrwor kind of it19:44
jamesbensonyeah I found that out after the fact... but at least this is tailored for our env.19:44
jamesbensonhrw are you trying to get Luminous on kolla-ansible or k8s?19:44
hrwjamesbenson: I am more just kolla user. other people in team do -ansible -k8s ones19:45
rwelluminit-runonce is very tied to the gate architecture though19:45
*** harlowja has quit IRC19:46
SamYapleyea i wouldnt run init-runonce directly. its from like the initial kolla days. honestly we shouldnt even be using it in the gates anymore19:46
SamYaplei didnt realize it still existed19:46
*** lrensing has joined #openstack-kolla19:52
rwellumWhy's that SamYaple ?19:56
SamYapleits a non-idempotent script that is hardcoded to do legacy things still19:57
* hrw off19:57
rwellumRight it does call nova, neutron etc - instead of openstack api19:57
SamYaplerwellum: and if you run it more than once you break stuff :)19:58
rwellumYeah when I was first playing with stuff I was told 'just run init-runonce' - not realizing it was somewhat dangerous and very gate-proprietary.19:59
SamYaplei used to tell people "look at what init-runonce does" but yea20:00
*** lpetrut has quit IRC20:02
*** Pavo has joined #openstack-kolla20:10
*** kbaegis has joined #openstack-kolla20:14
*** kbaegis has joined #openstack-kolla20:14
*** kbaegis has quit IRC20:18
*** danardelean has joined #openstack-kolla20:19
*** danardelean has quit IRC20:20
*** danardelean has joined #openstack-kolla20:25
inc0now I think we have lock in place that won't allow you to run it more than once20:26
*** lukepatrick has quit IRC20:26
kolla-slack<jascott1> neat https://github.com/nghialv/promviz20:27
*** lukepatrick has joined #openstack-kolla20:28
*** newmember has quit IRC20:28
*** manjeets has joined #openstack-kolla20:30
*** harlowja has joined #openstack-kolla20:33
*** ansmith has quit IRC20:38
kfox1111jascott1: interesting.20:39
kfox1111looks similar to the ui weave does. but potentialy much more indepth.20:39
*** trozet has joined #openstack-kolla20:40
*** danardelean has quit IRC20:43
*** danardelean has joined #openstack-kolla20:43
*** danardelean has quit IRC20:48
*** hachi__ has quit IRC20:51
*** jtriley has quit IRC20:51
kolla-slack<jascott1> ah… write up by our friend Matt “The” Butcher https://open.microsoft.com/2017/10/23/announcing-brigade-event-driven-scripting-kubernetes/20:51
kfox1111jascott1: is there a unique set in golang?20:52
kolla-slack<jascott1> not in stdlib afaik20:52
kfox1111:(20:52
kolla-slack<jascott1> lemme check20:52
kolla-slack<jascott1> ya dont think so20:53
kfox1111ok. I'll just use a map[string]bool then.20:53
kfox1111thanks.20:54
*** ktibi has quit IRC20:54
kolla-slack<jascott1> btw I got the ceph  multinode deploy working and into playbook20:58
kolla-slack<jascott1> felt like this at 1:15am20:58
kolla-slack<jascott1> \o/20:58
kfox1111cool.20:58
*** mummer has quit IRC21:10
*** trozet has quit IRC21:11
SamYaplei know this is salt code, but this is makes all ceph-mgr ceph-osd and ceph-mon come up at the same time, even on a fresh cluster, and it works https://github.com/SamYaple/home-salt/tree/master/salt/ceph21:11
SamYaplemight be somethign useful to kolla-k8s21:11
kfox1111SamYaple: I think long term, rook is a much more natural fit.21:13
kfox1111as it follows the k8s operator pattern.21:13
kfox1111you helm install rook-operator21:13
kfox1111and kubectl create -f mycephcluster.yaml21:13
*** ntpttr_laptop__ has joined #openstack-kolla21:13
kfox1111kubectl create -f cephfs-foo.yaml, etc.21:13
kfox1111its a bit green yet. but developing fast.21:14
kfox1111see rook.io for details.21:14
*** ntpttr_laptop has quit IRC21:14
SamYaplekfox1111: what is rook replacing in the scenario i described?21:16
kolla-slack<carlp> After fixing my registry cert issue, I am still unable to build a complete set of images (which I don't really need but a core set would be nice). Lots of the same JSON errors I posted earlier. Any ideas?21:16
SamYaplekfox1111: `kubectl create -f mycephcluster.yaml` should bring up the cluster. period21:16
SamYaplewhy bring in anything from the outside?21:16
kfox1111SamYaple: it deals with ceph deployment/management in a k8s way.21:16
kfox1111mycephcluster is an abstract declariative request for a ceph cluster.21:17
SamYapleyes. but why bring it in when ceph can deal with it itself?21:17
SamYapleim confused here21:17
kfox1111but ceph cant.21:17
SamYapleit can21:17
SamYaplei just posted code showing that21:17
*** Pavo has quit IRC21:18
kfox1111SamYaple: who runs that salt stack? where does it run?21:19
kfox1111rook follows the operator pattern.21:19
SamYaplekfox1111: who runs kubectl, where does that run?21:19
SamYapleit doesnt need to be salt21:19
kfox1111so, it runs in k8s. it constantly looks for declarative k8s objects to inform it what to do.21:19
SamYapleits what the code is doing21:19
kfox1111operators are the idea of using k8s to extend k8s to orchestrate more complex things.21:20
kfox1111so deploying a whole ceph cluster works like deploying a pod.21:20
SamYaplemy point is you can remove the need for orchestration for ceph specifically... im making no comment on rook in genreal21:20
kfox1111yeah, no. what I'm sayuing though, is in a k8s system, doing things with operators are much preffered to doing it manually.21:21
SamYapleim confused as to what manual action you are talking about21:21
SamYapleat some point an operator has to say "run ceph", but thats it21:21
kfox1111maybe I'm miswording things...21:21
kfox1111let me try a different angle.21:22
kfox1111in k8s, using k8s is the preffered way of interacting with k8s then running an external tool that runs k8s for you.21:22
kfox1111ie, kubectl create ceph is better then salt deploy ceph in k8s.21:22
SamYapleyes i feel you are definitely not understanding what im saying21:22
kfox1111entirely possible.21:23
SamYapleim saying the logic in salt can be turned into mycephcluster.yaml which is just k8s pods and secrets and stuff and ceph will form on its own21:23
SamYapleim not saying you should use salt21:23
kfox1111no. thats not quite right.21:23
SamYapleim saying look at that logic to make your ceph not need outside orchestration21:23
kfox1111mycephcluster.yaml would be the values you feed your salt logic.21:23
SamYapleor rook21:23
* inc0 grabs popcorn21:23
SamYapleoh jesus. listen im nto talking about salt21:24
kfox1111you have a few things:21:24
kfox11111. declarative description of request.21:24
kfox11112. logic to take in declarative request and realize it.21:24
SamYapleim saying i coded some logic in salt that orchestrationlessly starts a ceph cluster.21:24
kfox11113. engine that performs #2.21:24
SamYaplewe shoudl put that logic in k8s21:24
kfox11114. env on which 3 executes.21:24
SamYaplekfox1111: do you think we need rook to start memcached containers?21:25
kfox1111rook doesn't deal with memcached.21:25
kfox1111it deals with deploying ceph using k8s as a k8s extention.21:26
kfox1111rook covers 2 and 3. using k8s as #4,21:26
kfox1111and #1 comes from the user via k8s crds.21:26
SamYapleok. let me rephrase because you are going down the wrong path here21:26
SamYapledo you think we need an operator to deploy memcached21:27
*** ansmith has joined #openstack-kolla21:27
kfox1111unlikely. memcached's so stateless and its orchestration is very primitive. k8s shoudl be able to handle it with its built in primatives.21:27
kfox1111galera though, probably needs an operator.21:27
SamYapleyay. here we are. we dont need an operator for ceph then21:28
SamYapletwas my point21:28
SamYaplejust like we dont need one for rabbitmq with autocluster+etcd21:28
* sdake grabs inc0's popccorn21:28
kfox1111we do need an operator for ceph. as its a non trivial orchestration that k8s itself will have a hard time handling.21:28
SamYaplekfox1111: and weve finally drilled down to the issue. that is the part im saying ive solved. it is trival to "orchestrate" now21:29
kfox1111you gotta bring up mons in the right order, generate creds, pass them around, then register osd's, etc.21:29
*** danardelean has joined #openstack-kolla21:29
SamYapleno you dont21:29
sdakekfox1111 there is always openshift-ansible21:29
SamYaplethats.. my.. point..21:29
sdakeor openshift-service-broker I mean21:29
kfox1111SamYaple: your missing my point I think. rook is solving this use caase already. no need to reinvent that wheel.21:29
SamYaplekfox1111: see this isnt reinvention, this is removing that layer21:29
SamYapleits simplifying because thats not needed21:29
kfox1111I don't think you understand operators.21:30
SamYaplebut man this was a pain to talk about. im just going to stay away from any k8s talks in this channel. they are never fun21:30
inc0kfox1111: I think what Sam is saying is, besides secret generation, key distribution and stuff like that, there is little orchestration21:30
kfox1111hah.21:30
kfox1111"besides all the hard stuff, there is little orchestration"21:30
SamYaplethis wasnt even about k8s lol. this was about ceph21:31
inc0so if you can pre-generate ceph secrets and keyrings, you can just load them and start mons21:31
*** danardelean has quit IRC21:31
kfox1111yeah, bbut you said for kolla-kubernetes, we should deploy ceph like X.21:31
kfox1111I'm saying, for kolla-kubernetes we shoudl suggesst deploying ceph like Y.21:31
kfox1111where I am stating Y is more k8s natural.21:31
kfox1111just my $0.0221:31
SamYapleim saying dont complicate somethign that doesnt need complication21:32
SamYaplelike dont use operaters to deploy memcached21:32
inc0I'd say we shouldn't deploy ceph at all, but I think what's Sam is saying is that rook is essentialy redundant if you can pre-generate keyrings21:32
kfox1111I never suggested we shoud deploy memcached with operators.21:32
kfox1111but deploying system services using k8s and not k8s is more complicated then just deploying everythign with k8s.21:33
SamYapleim all about code reuse, but im more about simplifying to get rid of the need for more code21:33
kfox1111thats I think where we differ.21:33
SamYaplebut see, in this case it isnt21:33
SamYaplefor ceph specifically21:33
SamYaplefire off all the containers at once and it can work, as ive done21:33
inc0SamYaple: but you did pre-load secrets right21:34
inc0?21:34
SamYapleinc0: if you look at secrets like passwords, then yes i did put in the passwords21:35
kfox1111sounds like the root of discussion is a philisophical difference of opinion. thats ok. but I think we are at an impass. thats ok.21:35
SamYapledo you autogenerate keystone passwords?21:35
inc0and you did put monmap, initial at least21:35
SamYapleinc0: nope21:35
SamYapleinc0: just the secrets21:35
kfox1111kolla-kubernetes won't provide a ceph solution. so everyone can feel free to do whats best for them.21:35
kfox1111ceph-deploy, salt, rook, all should work.21:36
SamYapleyea im just talking about ceph. im not going to talk about k8s anymore21:37
kfox1111yeah. which orchestration tool you use is orthoganal to the ceph software.21:46
SamYapleagain, no orchestration tool needed to deploy ceph. this is what ive done21:50
openstackgerritChristian Berendt proposed openstack/kolla-ansible master: Remove trailing whitespace in tools/init-runonce  https://review.openstack.org/51553221:50
SamYaplethat was the message i was bring the channel21:50
kfox1111thats where I'm confused. salt is an orchestration tool?21:51
SamYapleremove salt from the equation here. i just-so-happened to be using salt. thats where implemented the stuff21:52
openstackgerritChristian Berendt proposed openstack/kolla-ansible master: Remove trailing whitespaces  https://review.openstack.org/51553221:52
kfox1111ah. so you are trying to discuss the algorithm in the salt logic more then the actual implementation?21:52
SamYapleyup21:53
kfox1111ah. ok.21:53
SamYapleperhaps were mixing terms here. but my point is the same way you would deploy memcached or rabbitmq+autocluster is just start all the containers, and they work themselves out21:53
SamYaplenow you can do that with ceph21:53
*** dardelean has quit IRC21:53
kfox1111conceptually yes, but you still need to actually realize the implmentation, like you did in salt. its code to maintain.21:54
SamYaplei mean sure, so is telling k8s to start memcached containers21:54
kfox1111for memcache, the procedure is something like this:21:55
kfox1111user provides values and a chart to helm. helm uses templates and values to generate k8s objects.21:55
*** jascott1 has joined #openstack-kolla21:55
kfox1111k8s objects are uploaded to k8s which uses its built in orchestration to deploy memcached in pods.21:55
*** danardelean has joined #openstack-kolla21:56
kfox1111k8s's level of orchestration is very simplistic. it knows how to deploy pods basically in parallel but with no interconnectivity. its up to the pods themselves to orchestrate things, or something driving k8s to orchestrate.21:57
kfox1111if k8s's built in primatives are not up to the task.21:57
jascott1SamYapple where is this ceph stuff you made? Im working with ceph atm...21:57
SamYapleright thats what i mean by mixing terms. all we need are k8s primatives to deploy ceph now. no outside or advanced orchestration needed21:58
SamYaplejascott1: nah i just worked on a new flow for ceph deploying. happened to be using salt. can deploy ceph-mon ceph-osd ceph-mgr etc all in parallel without the orcehstration dance https://github.com/SamYaple/home-salt/tree/master/salt/ceph21:58
jascott1thanks21:59
*** dciabrin has quit IRC22:00
*** jascott1 has quit IRC22:01
kfox1111SamYaple: thats where i'm still confused. are you saying we should be using your salt logic, or implementing it in <what>?22:02
kfox1111ie, what is the action you are proposing. it doesn't 'just work' todayu, without some change. what is the change?22:03
SamYaplekfox1111: im saying there is a massive reduction in "logic" to the point that there isnt a need to coordinate between containers. every container has all the info it needs to do its job22:03
kfox1111ok. your arguing for 'internal orchestration'22:03
kfox1111similar to the kubernetes-entrypoint idea.22:03
kfox1111make the pods aware of k8s and each other enough to self orchestrate.22:04
SamYaplepossibly? if init containers count as internal orechstartions22:04
kfox1111yeah. its internal to the orchestration that k8s provides.22:04
kfox1111external orchestration is driving k8s's orchestration.22:04
kfox1111both are also possible.22:04
kfox1111external orchestrator -> k8s orchestration -> internal pod orchestration.22:04
kfox1111though you usually do one or the other.22:05
*** jamesbenson has quit IRC22:05
kfox1111I am not against deploying ceph that way. but, I'm more against kolla-kubernetes implementing ceph at all.22:05
kfox1111helm-ceph or something should.22:05
kfox1111ie, helm-ceph would be a competitor of rook.22:06
kfox1111I don't think kolla-kubernetes should compete with rook.22:06
kfox1111the scope of just running openstack on k8s is huge enough. :/22:06
kfox1111thats not to say kolla-kubernetes shouldn't by default require a ceph. I think that is a good thing.22:07
kfox1111openstack works best with ceph IMO.22:07
kfox1111but more like, ceph & k8s are base requirements of kolla-kubernetes. you bring those two things before starting on kolla-kubernetes deployment.22:08
*** genek has joined #openstack-kolla22:09
kfox1111does that kind of help explain my push back?22:12
*** genek has quit IRC22:13
*** dciabrin has joined #openstack-kolla22:15
SamYaplei was unaware that 1: kolla-k8s didnt deploy ceph currently and 2: kolla-k8s didnt want to deploy ceph22:15
SamYapleeither of those facts would have helped, yes22:15
kfox1111ah. ok. sorry for the confusion.22:16
SamYapleits cool. as always kfox1111, we leave on the same page agreeing with each other (mostly). we just take the long way to get tehre22:16
kfox1111heh. I know, right? :)22:17
SamYapleits cool, i still like talking to you22:17
kfox1111me too. :)22:17
SamYapleyou talk to yourself?22:17
SamYaple;)22:18
kfox1111usually the voices do all the talking. ;)22:18
*** spiette has quit IRC22:25
*** itlinux has quit IRC22:39
rwellumkfox1111: the issue I think is that kolla-ansible is a deployer or orchestrator and kolla-kubernetes is not?22:42
rwellumBut they look like a 1-1 comparison but they are not.22:42
kfox1111yeah. thats definitely part of it I think.22:42
kfox1111there is definately overlap between say ansible and kolla,22:43
kfox1111but there is also some fundimentally differnet things each.22:43
kfox1111ie, k8s does self healing. ansible doesnt so much.22:43
rwellumIf anything helm is closer to ansible22:44
kfox1111ansible provides primitives more at a node level.22:44
kfox1111k8s is more cluster level primiatives.22:44
kfox1111yeah. ansible does templating. k8s doesn't.22:44
kfox1111helm + k8s is maybe closer to ansible. but still a bit disjoint.22:45
kfox1111I tend to think of k8s in a much different way though. from this angle....22:45
kfox1111k8s is closer to a cluster os.22:45
kfox1111a primative one, but on which more complex things can be built.22:45
kfox1111ansible is more of a node management tool with a bit of workflow and resource management in it.22:46
kfox1111good for different things.22:46
kfox1111when you think of k8s as an os, then you deploy stuff in the os, rather then going under the hood and deploying things.22:47
kfox1111(this is very similar to coreos's phylosophy)22:47
kfox1111sshing into a node is kind of like attaching a debugger to a kernel and poking around. should be needed almost never.22:50
rwellumGood description/analogy ty22:53
*** salv-orl_ has quit IRC22:53
*** salv-orlando has joined #openstack-kolla22:54
rwellumso based on that kfox1111 - how SHOULD one orchestrate kolla-kubernetes?22:55
kfox1111the jury's still out on that one. but I still kind of think we should be doing an operator similar to rook.22:55
kfox1111openstacks orchestration is fairly complex. especially during upgrades.22:56
kfox1111doing it as an operator allows for the greatest flexability and the easiest use I think.22:56
rwellumCool. Let me know when the jury gets called in with the decision. :)22:56
kfox1111part of that is its the communities role to decide, or implement both.22:56
kfox1111I don't really have the cycles unfortunately to do too much with implementing orchestration.22:57
*** salv-orlando has quit IRC22:59
kfox1111I guess what I'm saying is, those who write the code get to have the most say in that?23:00
rwellumI guess yeah.23:00
*** trozet has joined #openstack-kolla23:04
*** lrensing has quit IRC23:08
openstackgerritMerged openstack/kolla master: XenServer: Add packages for initial support  https://review.openstack.org/51026623:10
*** lukepatrick has quit IRC23:14
*** trozet has quit IRC23:17
*** danardelean has quit IRC23:19
*** jascott1 has joined #openstack-kolla23:28
*** danardelean has joined #openstack-kolla23:35
*** danardelean has quit IRC23:39
*** salv-orlando has joined #openstack-kolla23:54
*** gkadam_ has joined #openstack-kolla23:56
*** salv-orlando has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!