Monday, 2018-01-08

*** retr0h has quit IRC00:05
*** mkrai has quit IRC00:05
*** bachp has quit IRC00:05
*** mkrai has joined #openstack-kolla00:13
*** cduchesne has quit IRC00:15
*** cduchesne has joined #openstack-kolla00:19
*** bachp has joined #openstack-kolla00:20
*** retr0h has joined #openstack-kolla00:20
*** masber has joined #openstack-kolla00:21
*** masber has quit IRC00:29
*** dardelean has joined #openstack-kolla00:40
*** dardelean has quit IRC00:44
*** kbaegis has joined #openstack-kolla00:45
*** caowei has joined #openstack-kolla00:57
*** ljjjustin has joined #openstack-kolla01:01
*** dave-mccowan has joined #openstack-kolla01:06
*** yangyapeng has joined #openstack-kolla01:10
*** masber has joined #openstack-kolla01:10
*** tovin07_ has joined #openstack-kolla01:12
*** unicell has joined #openstack-kolla01:19
*** daidv has joined #openstack-kolla01:20
*** daidv_ has joined #openstack-kolla01:20
*** unicell1 has quit IRC01:21
*** daidv_ has quit IRC01:24
*** kiennt26 has joined #openstack-kolla01:36
*** dardelean has joined #openstack-kolla01:42
*** kbaegis has quit IRC01:53
*** pengdake has joined #openstack-kolla01:58
*** dardelean has quit IRC02:26
*** dardelean has joined #openstack-kolla02:27
*** dardelean has quit IRC02:31
*** zhurong has joined #openstack-kolla02:46
spsuryamorning all02:56
*** dave-mccowan has quit IRC03:09
*** duonghq has joined #openstack-kolla03:09
xinliangmorning spsurya03:11
duonghqmorning guys03:11
xinlianghi all,  there is a building error on my patch: kolla-build-centos-source finger://ze06.openstack.org/dc0e1c05c0044d06ba9332c44040692d : RETRY_LIMIT in 1m 00s03:12
xinlianganyone knows what's the problem?03:12
xinliangThe ci tells: kolla-build-centos-source finger://ze06.openstack.org/dc0e1c05c0044d06ba9332c44040692d : RETRY_LIMIT in 1m 00s03:13
xinliangAnd can't see any log for that.03:13
*** kong has quit IRC03:17
*** kong has joined #openstack-kolla03:17
*** zshi has joined #openstack-kolla03:33
*** ddurst has quit IRC03:46
*** pengdake has quit IRC04:01
*** janonymous has joined #openstack-kolla04:01
openstackgerritJianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support neutron configuration  https://review.openstack.org/52599904:02
openstackgerritJianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support compute node bootstrap  https://review.openstack.org/53164004:02
openstackgerritJianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support compute node bootstrap  https://review.openstack.org/53164004:05
openstackgerritJianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support neutron configuration  https://review.openstack.org/52599904:05
*** ddurst has joined #openstack-kolla04:11
*** zshi is now known as zshi_afk04:21
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2]  https://review.openstack.org/53164104:22
*** pengdake has joined #openstack-kolla04:27
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Remove httplog in defaults section for haproxy.cfg  https://review.openstack.org/53164404:32
*** genek_ has quit IRC04:40
*** genek has joined #openstack-kolla04:43
openstackgerritMerged openstack/kolla-ansible stable/ocata: Remove extra configuration item in solum.conf.j2  https://review.openstack.org/47264704:45
openstackgerritMerged openstack/kolla-ansible stable/pike: Remove the unnecessary register.yml from upgrade.yml for Nova  https://review.openstack.org/52879904:45
openstackgerritMerged openstack/kolla-ansible master: change kuryr default driver option name  https://review.openstack.org/52904204:45
*** zhurong has quit IRC04:52
*** caowei has quit IRC04:52
*** ArminderSingh has joined #openstack-kolla05:12
*** yangyapeng has quit IRC05:16
*** yangyapeng has joined #openstack-kolla05:17
*** janki has joined #openstack-kolla05:28
*** skramaja has joined #openstack-kolla05:30
*** caowei has joined #openstack-kolla05:30
*** mikal has quit IRC05:33
*** zhurong has joined #openstack-kolla05:52
openstackgerritJianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support neutron configuration  https://review.openstack.org/52599906:02
*** janki has quit IRC06:17
*** janki has joined #openstack-kolla06:18
openstackgerritXinliang Liu proposed openstack/kolla-ansible master: Change debian mariadb backup tool to mariabackup  https://review.openstack.org/53165606:22
*** strgout has quit IRC06:31
*** tvignaud has joined #openstack-kolla06:37
*** jbadiapa has joined #openstack-kolla06:47
*** zhangfei has joined #openstack-kolla06:47
*** zshi_afk is now known as zshi06:49
*** coolsvap has joined #openstack-kolla06:51
openstackgerritChason Chan proposed openstack/kolla-ansible master: Enable vitrage dashboard  https://review.openstack.org/53045506:53
*** genek has quit IRC06:58
*** pcaruana has joined #openstack-kolla07:04
openstackgerritMerged openstack/kolla master: Add vitrage dashbord into horizon image  https://review.openstack.org/53045307:05
*** genek has joined #openstack-kolla07:07
*** egonzalez has joined #openstack-kolla07:11
*** bachp has quit IRC07:13
*** mgkwill has quit IRC07:13
*** mgkwill has joined #openstack-kolla07:13
*** bachp has joined #openstack-kolla07:13
*** brad[] has quit IRC07:15
*** mrunge has quit IRC07:31
*** mrunge has joined #openstack-kolla07:32
*** Arminder has joined #openstack-kolla07:33
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2]  https://review.openstack.org/53164107:34
*** dciabrin has joined #openstack-kolla07:36
openstackgerritMerged openstack/kolla-ansible master: Add rsync env for account & container replicators  https://review.openstack.org/52740207:37
openstackgerritMerged openstack/kolla-ansible stable/pike: Fix the cinder backup action failed when use external ceph  https://review.openstack.org/53022207:40
*** mrunge_ has joined #openstack-kolla07:41
*** mrunge has quit IRC07:44
*** zhangfei has quit IRC07:44
*** mrunge_ has quit IRC07:46
hrwmorning07:51
*** zhangfei has joined #openstack-kolla08:00
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2]  https://review.openstack.org/53164108:01
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2]  https://review.openstack.org/53164108:01
*** wojdec has joined #openstack-kolla08:06
*** b_bezak has joined #openstack-kolla08:07
*** mrostecki1 is now known as mrostecki08:11
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 3]  https://review.openstack.org/53168108:13
*** shardy has joined #openstack-kolla08:18
*** trozet has quit IRC08:19
*** wojdec1 has joined #openstack-kolla08:19
*** honza has quit IRC08:20
*** honza has joined #openstack-kolla08:20
*** honza is now known as Guest1450008:20
*** trozet has joined #openstack-kolla08:20
*** wojdec has quit IRC08:21
*** wojdec1 has quit IRC08:22
*** wojdec has joined #openstack-kolla08:23
*** lvdombrkr has joined #openstack-kolla08:28
*** kevzha01 has joined #openstack-kolla08:30
*** kevzha01 is now known as kevinz08:30
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Ensure docker running before execute the destroy action  https://review.openstack.org/53168808:34
*** ktibi has joined #openstack-kolla08:34
*** wojdec1 has joined #openstack-kolla08:35
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Ensure docker running before execute the destroy action  https://review.openstack.org/53168808:36
*** wojdec has quit IRC08:37
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 3]  https://review.openstack.org/53168108:37
*** nathharp has joined #openstack-kolla08:40
*** athomas has joined #openstack-kolla08:41
openstackgerritMerged openstack/kolla-ansible master: Mount kolla_logs volume in bifrost_deploy  https://review.openstack.org/53054508:44
egonzalezwho's going to Dublin PTG?08:45
* hrw plans to08:45
egonzalezlooking to ask for holidays, but wants to know if people will be there :)08:46
*** dalvarez has joined #openstack-kolla08:47
*** gfidente has joined #openstack-kolla08:51
*** gfidente has quit IRC08:51
*** gfidente has joined #openstack-kolla08:51
openstackgerritMerged openstack/kolla-ansible master: Ensure inspector uses internal ironic interface  https://review.openstack.org/53054408:52
openstackgerritMerged openstack/kolla-ansible master: Add ironic inspector to fluentd log collection  https://review.openstack.org/53050608:52
openstackgerritPierre Hanselmann proposed openstack/kolla-ansible stable/pike: Fix dhcp_agent configuration for vmware DVS  https://review.openstack.org/53169108:54
*** skramaja_ has joined #openstack-kolla08:54
*** skramaja has quit IRC08:54
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 3]  https://review.openstack.org/53168108:55
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2]  https://review.openstack.org/53164108:55
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 2]  https://review.openstack.org/53164108:55
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 3]  https://review.openstack.org/53168108:55
*** skramaja has joined #openstack-kolla08:59
*** skramaja_ has quit IRC08:59
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 4]  https://review.openstack.org/53169508:59
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 4]  https://review.openstack.org/53169509:00
*** jascott1 has quit IRC09:02
*** sbezverk has quit IRC09:11
*** Guest14500 is now known as honza09:14
*** owalsh_ is now known as owalsh09:15
*** mrunge has joined #openstack-kolla09:19
openstackgerritJeffrey Zhang proposed openstack/kolla master: Support squash docker images layers  https://review.openstack.org/52587109:21
hrwcan someone look at https://review.openstack.org/#/c/522712/ and get that merged?09:24
openstackgerritJianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support compute node bootstrap  https://review.openstack.org/53164009:26
openstackgerritJianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support neutron configuration  https://review.openstack.org/52599909:26
*** shardy has quit IRC09:27
*** dardelean has joined #openstack-kolla09:30
*** egonzalez has quit IRC09:32
*** shardy has joined #openstack-kolla09:32
*** dardelean has quit IRC09:36
*** dardelean has joined #openstack-kolla09:36
*** dardelean has quit IRC09:40
*** egonzalez has joined #openstack-kolla09:43
openstackgerritChason Chan proposed openstack/kolla-ansible master: Restructure the vpnaas roles  https://review.openstack.org/53002209:44
*** dardelean has joined #openstack-kolla09:45
egonzalezsdake, good luck with individual board election :)09:49
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 5]  https://review.openstack.org/53170209:51
spsuryaegonzalez: i am in plan.....but not 100% sure09:52
hrwegonzalez, spsurya: https://review.openstack.org/#/c/522712/ - can you? it is now the most important aarch64 blocker in kolla-ansible.09:53
egonzalezhrw, why not adding a docker_package_version: {% '..ce' if  aarch64 else '..1.12' %} and set {{ docker_package_version }} in install package list? this avoid a duplicated file just for a package name change09:55
*** bootsha has joined #openstack-kolla09:56
*** wojdec1 has quit IRC09:57
hrwegonzalez: can you write that in comment? I will look at doing that later today09:57
egonzalezon ti09:57
egonzalezit09:57
hrwthx09:57
*** wojdec has joined #openstack-kolla09:58
*** duonghq has quit IRC09:58
*** wojdec1 has joined #openstack-kolla10:02
*** ljjjustin has quit IRC10:02
*** zhurong has quit IRC10:02
*** jascott1 has joined #openstack-kolla10:03
*** wojdec has quit IRC10:04
spsuryahrw: ok10:04
*** kiennt26 has quit IRC10:07
*** dardelean has quit IRC10:08
*** dardelean has joined #openstack-kolla10:09
*** pbourke has quit IRC10:12
*** pbourke has joined #openstack-kolla10:12
*** dardelean has quit IRC10:13
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 6]  https://review.openstack.org/53170710:20
*** tovin07_ has quit IRC10:21
-openstackstatus- NOTICE: zuul has been restarted, all queues have been reset. please recheck your patches when appropriate10:25
*** bootsha_ has joined #openstack-kolla10:31
*** pengdake has quit IRC10:31
*** bootsha has quit IRC10:33
*** bootsha_ has quit IRC10:36
*** chandankumar has joined #openstack-kolla10:36
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 7]  https://review.openstack.org/53171110:36
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [Final]  https://review.openstack.org/53171210:48
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 4]  https://review.openstack.org/53169510:50
*** sambetts|afk is now known as sambetts10:50
*** bootsha has joined #openstack-kolla10:50
*** nanliu_ has quit IRC10:51
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support policy.yaml file [part 5]  https://review.openstack.org/53170210:51
*** nanliu_ has joined #openstack-kolla10:52
openstackgerritChandan Kumar proposed openstack/kolla master: Include building python-tempestconf from source in Tempest  https://review.openstack.org/53171410:53
hrwegonzalez: thanks for suggestion. rewrote. testing now10:58
*** caowei has quit IRC11:00
*** numans has quit IRC11:06
*** numans has joined #openstack-kolla11:08
*** bootsha has quit IRC11:09
hrwhm. from +67, -10 to +28, -3 ;D11:19
hrw+27, -311:23
*** bootsha has joined #openstack-kolla11:30
*** bootsha has quit IRC11:35
*** zhurong has joined #openstack-kolla11:35
openstackgerritMerged openstack/kolla master: Fix MariaDB bootstrap for 10.1 version  https://review.openstack.org/52919911:36
hrwyay!11:43
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: Handle newer Docker and support AArch64  https://review.openstack.org/52271211:50
hrwegonzalez: you should be happier with this version ;)11:50
*** egonzalez has quit IRC11:56
*** jistr is now known as jistr|mtg11:58
*** dave-mccowan has joined #openstack-kolla12:09
*** bootsha has joined #openstack-kolla12:13
*** rhallisey has joined #openstack-kolla12:17
*** bootsha has quit IRC12:19
*** bootsha has joined #openstack-kolla12:20
*** janki has quit IRC12:24
*** bootsha has quit IRC12:34
*** bootsha has joined #openstack-kolla12:37
*** bootsha has quit IRC12:43
*** bootsha has joined #openstack-kolla12:44
*** bootsha_ has joined #openstack-kolla12:47
*** bootsha_ has quit IRC12:47
*** dave-mccowan has quit IRC12:47
*** bootsha_ has joined #openstack-kolla12:48
*** dave-mccowan has joined #openstack-kolla12:49
*** bootsha has quit IRC12:50
*** zhurong has quit IRC13:01
*** hieulq has quit IRC13:04
*** skramaja has quit IRC13:05
*** hieulq has joined #openstack-kolla13:05
*** dardelean has joined #openstack-kolla13:10
*** dardelean has quit IRC13:14
*** bootsha_ has quit IRC13:14
*** dardelean has joined #openstack-kolla13:14
*** janonymous has quit IRC13:22
*** pcaruana has quit IRC13:25
*** pcaruana has joined #openstack-kolla13:25
rwellumjascott1: quick start looks interesting13:26
*** egonzalez has joined #openstack-kolla13:32
*** gkadam has joined #openstack-kolla13:33
*** jistr|mtg is now known as jistr13:35
*** coolsvap has quit IRC13:36
*** mchlumsky has joined #openstack-kolla13:39
hrwrwellum: hi13:44
rwellumHi hrw13:45
hrwrwellum: have a moment to look at https://review.openstack.org/522712 patch? it adds support for newer docker so we can deploy on aarch6413:45
rwellum:wave hrw13:45
rwellumHaha13:45
rwellumYes will look13:46
hrwthanks13:46
hrwI rewrote Tone's patch so it is now much shorter and easier to review13:46
rwellumhrw: do we have docker-ce official support for non-AArch64?13:52
hrwrwellum: docker-ce can be installed on x86-64 for sure. Just it was discussed that most of people here prefer to stay at 1.12 so I kept that as it was13:53
rwellumI'm not sure if that's a preference or just it was stable and worked?13:56
hrwrwellum: in other words: I kept 1.12 as it was and use docker-ce on aarch64 because this is version we (as Linaro) provide for debian/aarch64 ;d13:57
rwellumhrw: so it works for kolla-k8s on debian/aarch64 ?13:58
hrwrwellum: kolla-ansible14:00
hrwrwellum: from what I know there are some guys working on k8s on aarch64. so same docker-ce for them14:00
rwellumOk cool14:00
openstackgerritChandan Kumar proposed openstack/kolla master: Include building python-tempestconf from source in Tempest  https://review.openstack.org/53171414:02
hrwrwellum: this patch is our main blocker for deploy now14:02
chandankumarmandre: i think i have fixed all the comments please verify once you have time :-)14:03
rwellumhrw: just a couple of nits14:09
sdakeegonzalez thx :)14:10
hrwrwellum: thx14:11
*** yangyapeng has quit IRC14:11
hrwrwellum: that line in commit has 73 chars ;D14:11
mandrechandankumar: I reviewed again, back to you :)14:12
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: Handle newer Docker and support AArch64  https://review.openstack.org/52271214:13
hrwrwellum, mandre: please take a look and vote14:14
*** zhangfei has quit IRC14:25
*** lpetrut has joined #openstack-kolla14:31
*** dardelean has quit IRC14:34
*** dardelean has joined #openstack-kolla14:34
*** hw_wutianwei has joined #openstack-kolla14:36
*** kbaegis has joined #openstack-kolla14:42
openstackgerritChandan Kumar proposed openstack/kolla master: Include building python-tempestconf from source in Tempest  https://review.openstack.org/53171414:50
chandankumarmandre: ^^ i think this time i have done that, please have a look :-)14:50
mandrechandankumar: looks about right, let's wait for CI14:52
chandankumarmandre: sweet, Thank you :-)14:53
*** yangyapeng has joined #openstack-kolla14:59
*** sbezverk has joined #openstack-kolla15:09
*** ktibi has quit IRC15:20
openstackgerritJianghua Wang proposed openstack/kolla-ansible master: XenAPI: Support neutron configuration  https://review.openstack.org/52599915:27
openstackgerritJeffrey Zhang proposed openstack/kolla master: Support squash docker images layers  https://review.openstack.org/52587115:27
*** SamYaple_ has quit IRC15:30
*** SamYaple has joined #openstack-kolla15:30
*** rmart04 has joined #openstack-kolla15:31
*** b_bezak has quit IRC15:31
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Remove httplog and forwardfor in defaults section for haproxy.cfg  https://review.openstack.org/53164415:33
*** kbaegis has quit IRC15:40
*** ktibi has joined #openstack-kolla15:41
ktibianyone can check this issue ? https://bugs.launchpad.net/kolla-ansible/+bug/174039915:49
openstackLaunchpad bug 1740399 in kolla-ansible "Deployment fails with 3 Ceph mon" [Undecided,New]15:49
kolla-slack<aslancimbom100> ktibi how many ceph monitors do you have?15:54
sdaketime to delete ceph from kolla ;)15:55
kolla-slack<aslancimbom100> :+1:15:55
kolla-slack<aslancimbom100> ceph-ansible is very good option for anyone who wants it15:56
ktibiceph-ansible can deploy in docker ?15:57
ktibiaslancimbom100 I have 3 mon15:57
kolla-slack<aslancimbom100> yes with or without docker15:57
*** SamYaple has quit IRC15:57
*** SamYaple has joined #openstack-kolla15:57
*** SamYaple has quit IRC15:58
*** SamYaple has joined #openstack-kolla15:58
*** TxGirlGeek has joined #openstack-kolla15:59
ktibiaslancimbom100 no doc about docker in http://docs.ceph.com/ceph-ansible15:59
sdakeaslancimbom100 the problem is upgrades..16:00
ktibimaybe kolla can use ceph-ansible ?16:01
*** hw_wutianwei has quit IRC16:02
*** rmart04 has quit IRC16:03
sdakektibi we have talked long and hard about it - and the issue always comes back to upgrades16:06
*** gfidente has quit IRC16:09
*** gfidente has joined #openstack-kolla16:10
*** gfidente has quit IRC16:10
*** gfidente has joined #openstack-kolla16:10
ktibisdake ok so If I talk ceph luminous in kolla ...16:11
sdakektibi i'm the wrong fella to ask - try inc0 :)16:11
kolla-slack<aslancimbom100> ktibi yes documentation doesn't mention it, however if you take a look at github files docker options are obvious16:13
kolla-slack<aslancimbom100> ie it says `site.yml.sample`...that would be `site-docker.yml.sample`16:14
ktibiaslancimbom100 yes I can see,16:15
*** brad[] has joined #openstack-kolla16:18
hrwaslancimbom100: first I need to check how to build ceph-ansible images ;D16:20
ktibihrw so easy to build, it's same image for all16:23
ktibihttps://github.com/ceph/ceph-ansible/blob/4f7223d5fb5317e7fde9202d8b4f904a534420bf/docker/Linode.dockerfile16:23
hrwktibi: build for arm64 I mean ;D16:23
ktibiaslancimbom100 I saw playbook and how ceph-ansible works, a lot of command like shell: cat .... , if docker ... i don't know if the project is very mature :/16:25
kolla-slack<aslancimbom100> its opensource project, it only goes so far16:26
kolla-slack<aslancimbom100> or you could pay some company to install it for you ;)16:26
kolla-slack<aslancimbom100> Trial and error, i installed it within few hours16:26
*** krtaylor has quit IRC16:27
*** cah_link has joined #openstack-kolla16:27
*** krtaylor_ has joined #openstack-kolla16:27
ktibiaslancimbom100 yes it's opensource but the project is on ceph official repo, so redhat behind. And I'm not fan about deploy one image and play exec command in the container :/16:28
ktibiI prefer kolla method16:29
*** jistr is now known as jistr|off|mtg16:33
kfox1111morning.16:40
hrwhi kfox111116:41
kfox1111hi. :)16:42
hrwhttps://review.openstack.org/#/c/522712/ (handle newer docker) patch still waits for brave reviewers! sdake? egonzalez? mandre?16:42
*** cah_link has quit IRC16:42
hrwmuch shorter than earlier version, just 5 files changed, no new ones16:42
*** david-lyle has quit IRC16:47
*** david-lyle has joined #openstack-kolla16:48
*** pcaruana has quit IRC16:48
*** kbaegis has joined #openstack-kolla16:51
*** kbaegis has joined #openstack-kolla16:51
*** kbaegis has joined #openstack-kolla16:52
*** kbaegis has quit IRC16:52
*** david-lyle has quit IRC16:52
*** kbaegis has joined #openstack-kolla16:52
kbaegisHi all. I'm seeing this: /var/lib/ansible/.my.cnf on my kolla-ansible deploy.16:52
sbezverkkfox1111: if you want, I have steps to test csi hostpath driver, it is for a single node cluster though..16:52
kfox1111sbezverk: interesting. yeah, I'd like to see that, as it will show what is to come. :)16:53
kbaegisAnyone had that issue before? Looking for a quick resolution.16:56
sbezverkkfox1111: Here is general doc https://github.com/kubernetes-csi/docs..  here is my yaml. You only need to change image location. http://paste.openstack.org/show/640979/17:00
kfox1111cool. thanks. :)17:01
kfox1111sbezverk: just confirmed mount propigation works with crio. :)17:01
sbezverkkfox1111: btw it also required for csi17:02
kfox1111yeah. kind of figured it would be. :) I'm glad they pushed for it being a requirement.17:02
kfox1111that should keep the door wide open. :)17:02
kfox1111sbezverk: interesting pastebin. plan on making a chart out of that?17:03
sbezverkkfox1111: sorry, probably later, the main focus now is csi rbd driver17:04
kfox1111yeah, that would be very useful.17:05
kfox1111they have been stonewalling userspace rbd and cephfs for a while.17:06
kfox1111the user space versions often have more features then the kernel versions.17:06
*** egonzalez has quit IRC17:08
SamYaplektibi: kolla method was first!17:08
SamYaplektibi: me and leseb were writting the kolla-ceph and ceph-asnible stuff at the same time. i maintain we got docker imeplemntation done first.... (but we both had working POC within a few hours of each other :P )17:09
SamYaplethe ceph-docker stuff was actually pulled in from an external repo. and i agree, the layout is bad with all the shell/cat/if docker stuff17:10
*** TxGirlGeek has quit IRC17:11
kfox1111sbezverk: how do you feel about https://review.openstack.org/#/c/458649/10 ?17:15
kfox1111rwellum has been using it for a bit and likes it.17:15
kfox1111I just got fluent-bit to work with an elasticsearch-operator managed cluster too. so I think that may be a very easy way to go.17:15
*** jascott1 has quit IRC17:17
*** TxGirlGeek has joined #openstack-kolla17:17
*** jascott1 has joined #openstack-kolla17:17
sbezverkit is long overdue to have some decent logging in kolla-k8s ;)17:18
kfox1111yeah. been waiting for something to manage es to come about.17:18
kfox1111finally got elasticsearch-operator to work this week.17:18
kfox1111well, this last week.17:18
kfox1111without some place to catch the log messages, it was kind of not worth focusing on.17:19
*** unicell has quit IRC17:20
kfox1111I think now, you could probably use elasticserach-operator and rook and have everything inside k8s.17:20
kfox1111though I don't think they have fully fleshed out upgrades yet.17:20
kfox1111but would be great for a dev env I think.17:20
kfox1111deploy kube, use kube to deploy everything else.17:21
*** nathharp has quit IRC17:22
*** itlinux_ has joined #openstack-kolla17:27
inc0good mornign17:27
kfox1111inc0: good morning.17:28
inc0ktibi: so we talked with leseb in Denver about ceph-ansible - we've decided that for now we will keep kolla way in kolla - upgrade to L17:28
inc0but probably we'll want to move to ceph-ansible for next LTS, all depends on migration path17:28
*** itlinux_ has quit IRC17:30
*** itlinux_ has joined #openstack-kolla17:30
*** itlinux_ has quit IRC17:31
*** itlinux_ has joined #openstack-kolla17:31
*** david-lyle has joined #openstack-kolla17:34
*** itlinux_ has quit IRC17:34
*** jascott1 has quit IRC17:34
*** itlinux_ has joined #openstack-kolla17:36
*** itlinux_ has quit IRC17:37
*** itlinux_ has joined #openstack-kolla17:38
*** itlinux_ has quit IRC17:42
*** itlinux_ has joined #openstack-kolla17:42
*** itlinux_ has quit IRC17:43
*** david-lyle has quit IRC17:47
*** jascott1 has joined #openstack-kolla17:48
*** itlinux_ has joined #openstack-kolla17:49
*** itlinux_ has quit IRC17:49
*** itlinux has joined #openstack-kolla17:50
*** b_bezak has joined #openstack-kolla17:52
rwellumkfox1111: the patch is also great for folks using the infra but who aren't knowledgeable on containers etc. So yeah I like it. :)17:54
*** itlinux has quit IRC17:55
*** kbaegis has quit IRC17:55
*** itlinux has joined #openstack-kolla17:55
*** b_bezak has quit IRC17:56
*** bmace has joined #openstack-kolla18:01
*** mdorman has joined #openstack-kolla18:06
*** kristaps_ has joined #openstack-kolla18:06
*** lvdombrkr has quit IRC18:09
*** strgout has joined #openstack-kolla18:16
strgouthowdy all, looking for some input for ceph. I have a 2 node setup with control/network/storage on one box and compute on a different one.18:17
strgoutStorage is currently a md raid10 with 12 JBOD disks. I'm going to nuke the raid 10 and start over so i can use Ceph. Any pointers on how to setup the disks? I was thinking 2 - 4 for OS just to keep the mysql database snappy as possible but that might be over kill.18:18
strgoutdisks are 2TB 7200 SAS drives.18:19
strgoutBTW my compute node does have 4 900gb SAS drives also. Currently in raid 10 as well. Both boxes have 10 gig ports as well.18:20
jascott1strgout for kolla-ansible?18:22
strgoutyup18:22
*** Rodrigo_BR has joined #openstack-kolla18:23
strgoutwell my deploy host is different but ill be attempting this with kolla-ansible.18:23
strgoutI read the ceph dock i'm just not sure about journal drive(s?). I'm assuming cache teir is no go since i don't have SSD.18:24
kfox1111probably fine just using one disk per osd if you just have spinners.18:25
kfox1111ignore cache tier if you don't have ssd's. no real benifit in them, unless oou are playing with EC teirs or ssd's.18:25
strgoutok what about a journal drive? and should there be one then one?18:26
strgouterr more then one18:27
*** shardy is now known as shardy_afk18:27
strgoutand to be clear i want to use for for disk volumes for VMs as well.18:30
*** shardy_afk has quit IRC18:35
*** shardy_afk has joined #openstack-kolla18:36
inc0it would be highly beneficial to put journals on ssd18:38
inc0but all spindle doesn't really matter18:39
inc0in any case, you can put OSD on every drive with no additional confs imho18:39
inc0one thing to do is after deployment check your ceph18:40
inc0docker exec -it ceph_mon ceph -s18:40
inc0you'll probably see that number of pg/pgp is incorrect, fix it accordingly to size18:40
inc0of env18:40
inc0there are ceph docs about that18:41
*** lpetrut has quit IRC18:41
*** gfidente is now known as gfidente|afk18:53
*** TxGirlGeek has quit IRC18:55
kfox1111back.18:59
kfox1111just be careful to have some redundancy if you use ssd's.18:59
*** TxGirlGeek has joined #openstack-kolla19:00
kfox1111ceph is great because it lets you pull things out and not take out the whole system.19:00
kfox1111if you only put one ssd in or that kind of thing, then you break the model.19:00
strgouthmm ok so should be looking at 2 SSD if i was going to go that way?19:00
strgouti don't think i'll be doing SSD for a while FYI.19:01
strgoutCC is already on fire.19:01
strgoutbought a sound proof cabinet. :D19:02
inc0so nobody can hear you scream?19:04
*** TxGirlGeek has quit IRC19:04
strgoutif i don't get this openstack setup up and running yes very much so.19:04
kfox1111yeah. make sure you have at least two ssd's or 0. you don't want one drive blowing out all your storage.19:07
strgoutwell i have 0 covered at this point.19:07
kfox1111:)19:08
*** gfidente|afk has quit IRC19:11
*** gfidente has joined #openstack-kolla19:12
*** gfidente has quit IRC19:12
*** gfidente has joined #openstack-kolla19:12
*** ManoX has joined #openstack-kolla19:13
*** unicell has joined #openstack-kolla19:15
*** harlowja has joined #openstack-kolla19:19
strgouti don't know.. i may just stick with the iscis. I deployed with that but i'm seeing the compute isn't trying to open the tcp connection to the control box. Like nothing showing in netstat -anp | grep 326019:20
*** TxGirlGeek has joined #openstack-kolla19:21
*** pramodrj07 has joined #openstack-kolla19:22
*** pramodrj07 has quit IRC19:22
*** Rodrigo_BR has quit IRC19:30
*** lpetrut has joined #openstack-kolla19:41
mwynneHi guys. What's the best process to restart containers after a host reboots?19:43
*** david-lyle has joined #openstack-kolla19:43
SamYaplemwynne: probably running the playbooks to ensure proper starting order19:45
openstackgerritBrian Haley proposed openstack/kolla-kubernetes master: Start using nova_metadata_host  https://review.openstack.org/53189219:46
mwynneSamYaple: Which playbooks?19:51
mwynneI also don't seem to have a '/var/log/kolla/` directory, which I think should exist, right?19:52
*** jgriffith_ is now known as jgriffith20:03
*** strgout has quit IRC20:10
*** ManoX has quit IRC20:11
rwellummwynne: kolla-ansible or kolla-kubernetes?20:12
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: Docker Logging Support  https://review.openstack.org/45864920:13
mwynnerwellum: Ansible. (I found the logs, just need to get my services and containers going again)20:13
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: Docker Logging Support  https://review.openstack.org/45864920:14
kfox1111rwellum: I updated the ps. I think it should be mergable.20:15
*** itlinux has quit IRC20:19
*** itlinux has joined #openstack-kolla20:29
*** openstack has quit IRC20:38
*** openstack has joined #openstack-kolla20:41
*** ChanServ sets mode: +o openstack20:41
*** TxGirlGeek has quit IRC20:45
rwellumkfox1111: one basic qn about the PS: std_logger: true - so you're setting this to default - but it's over-writeable correct?20:52
*** TxGirlGeek has joined #openstack-kolla20:54
*** TxGirlGeek has quit IRC20:55
*** TxGirlGeek has joined #openstack-kolla20:56
*** Angeh has joined #openstack-kolla20:56
*** ktibi has quit IRC20:58
*** mdorman has quit IRC21:01
*** openstackgerrit has quit IRC21:03
hrwfsck. zuul died again21:11
kfox1111rwellum: yeah.21:12
kfox1111setting it to false sets it back to the old behavior.21:12
*** rhallisey has quit IRC21:14
rwellumkfox1111: ack - looking at it21:20
*** dardelean has quit IRC21:20
*** dardelean has joined #openstack-kolla21:21
mwynnerwellum: Did you have suggestions for getting the containers started in the correct order?21:23
*** dardelean has quit IRC21:26
mwynnerwellum: I'm having issues getting a compute node back up and running. the libvirt and nova compute containers aren't cooperating.21:28
*** david-lyle has quit IRC21:28
*** gfidente has quit IRC21:29
mwynneI haven't been able to find anything in the docs regarding this, but I'm probably just missing it, right?21:31
rwellummwynne: in general the docs aren't great regarding what to when things go wrong. But inc0 is better qualified to answer.21:41
inc0mwynne: how "not cooperating"?21:42
mwynneinc0: I just rebooted my compute node and all the containers came up except for the nova-compute one. I started it with `docker start`, and the VM came up shortly after that. However, it couldn't get a DHCP address for some reason now.21:43
mwynneinc0: What's the process like for restarting processes/containers in the event that things go wrong?21:44
inc0mwynne: kolla-ansible?21:44
mwynneThis was running just fine for me a few days ago, and now nothing is working. I haven't changed anything :S21:44
mwynneinc0: Yes.21:44
inc0hmm that's strange, well dhcp issue might be due to neutron agent21:45
inc0if you're running multinode tho21:45
inc0mariadb hates being restarted21:45
inc0you need to perform manual recovery21:46
mwynneinc0: I am running multinode21:46
inc0right21:46
inc0so probably mariab cries21:46
mwynneinc0: How can I verify that?21:46
inc0check logs if nodes can handle mariadb21:47
inc0most of services should raise errors21:47
inc0http://galeracluster.com/documentation-webpages/restartingcluster.html this is doc that descrbes what needs to happen21:48
mwynneinc0: Looks ok21:48
inc0soo...no errors anywhere?21:49
mwynneNot that I've seen21:49
inc0and vms still doesn't worj?21:49
mwynneneutron-server logs should be full of errors, right?21:49
mwynneWell, the VM is running21:49
inc0yeha I guess21:49
mwynnebut it's not getting an IP from DHCP21:49
inc0is dhcp agent healthy?21:49
mwynneSeems to be21:50
mwynnelogs look ok21:50
inc0soo I guess it's tcpdump time..21:50
mwynneOh, maybe not. neutron.agent.ovsdb.native.vlog [-] 10000-ms timeout __log_wakeup /usr/lib/python2.7/dist-packages/ovs/poller.py:18721:50
inc0you need to pinpoint where dhcp request/response is dropped21:50
*** TxGirlGeek has quit IRC21:50
mwynneinc0: is there a way I can just restart all containers, or similar?21:51
inc0also check docker ps -a if there are any failed containers that don't restart21:51
inc0restarting all of them is probably bad idea21:51
mwynneinc0: Well, what happens if/when a server is rebooted?21:52
inc0it should just work tbh21:52
*** openstackgerrit has joined #openstack-kolla21:52
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: Zuulv3 iptables  https://review.openstack.org/51672721:52
inc0so if it doesn't in your case, something is off21:52
kfox1111sbezverk: inc0: https://review.openstack.org/#/c/516727/21:52
*** itlinux has quit IRC21:52
mwynneSo, after a reboot everything should just come back up?21:53
inc0well again, if you reboot ALL of mariadb nodes at once21:54
inc0it's going to need restoration21:54
mwynneIf there's only 1 node?21:55
inc0should just work21:55
mwynneok21:55
inc0only one node of mariadb?21:55
mwynneyeah21:55
mwynneI have a controller and 2 compute nodes21:55
inc0still should just work, but that's not multinode;) I guess it is in a way you have compute node and whatnot21:55
mwynnecontroller has infra + os services and that's it21:56
inc0ok21:56
mwynnethen compute nodes have neutron agent and compute21:56
inc0yeah that's how it's supposed to work21:56
mwynneit's pseudo-multinode ;)21:56
inc0I assumed you have more than 2 nodes of control21:56
inc0that's where mariadb cries when you restart cluster21:56
mwynneAh, no. It's not AIO, but it's not "multinode" by your definition.21:56
inc0right, hence misunderstanding21:56
inc0you're right ofc, it's multinode21:57
inc0just not HA;)21:57
mwynneyes21:57
*** TxGirlGeek has joined #openstack-kolla21:58
*** TxGirlGeek has quit IRC21:59
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: Docker Logging Support  https://review.openstack.org/45864922:00
kfox1111rwellum: let me know what you think plz.22:00
mwynneinc0: I just needed to reboot all my nodes in order I guess...22:08
mwynneSeems to have brought everything back up :S22:08
inc0good to hear22:10
rwellumThis is where k8s is far superior at keeping things alive, reboot etc. At least in theory.22:14
*** TxGirlGeek has joined #openstack-kolla22:14
mwynnerwellum: I might have to give kolla-kubernetes a shot.22:15
rwellummwynne: +122:17
rwellumkfox1111: couple of reviews in.22:19
rwellumLove the logging. Think I've said that a few times now!22:19
kfox1111glad its working for you. :)22:25
kfox1111I'm really hoping the recently added json log output from oslo will work well.22:25
kfox1111between the two, would make for some really powerful efk.22:25
*** lpetrut has quit IRC22:27
*** dave-mccowan has quit IRC22:37
*** TxGirlGeek has quit IRC22:41
*** TxGirlGeek has joined #openstack-kolla22:46
*** itlinux has joined #openstack-kolla22:51
*** TxGirlGeek has quit IRC22:53
openstackgerritMerged openstack/kolla-kubernetes master: Zuulv3 iptables  https://review.openstack.org/51672722:54
*** david-lyle has joined #openstack-kolla22:55
*** kristaps_ has quit IRC22:59
*** kristaps_ has joined #openstack-kolla22:59
*** mchlumsky has quit IRC23:07
*** david-lyle has quit IRC23:12
*** david-lyle has joined #openstack-kolla23:13
*** TxGirlGeek has joined #openstack-kolla23:14
*** sambetts is now known as sambetts|afk23:22
*** Angeh has quit IRC23:26
*** dardelean has joined #openstack-kolla23:47
*** dardelean has quit IRC23:52
*** k_mouza has joined #openstack-kolla23:53
*** itlinux has quit IRC23:54

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!