Thursday, 2017-12-21

*** mnasiadka has quit IRC00:04
*** yingjun has joined #openstack-kolla00:12
*** mnasiadka has joined #openstack-kolla00:14
*** itlinux_ has quit IRC00:15
*** itlinux__ has quit IRC00:15
SamYapledocker-py or docker?00:17
*** salv-orlando has joined #openstack-kolla00:18
*** mnasiadka has quit IRC00:19
*** itlinux_ has joined #openstack-kolla00:19
*** itlinux__ has joined #openstack-kolla00:19
*** yangyape_ has quit IRC00:27
*** yangyapeng has joined #openstack-kolla00:27
*** mnasiadka has joined #openstack-kolla00:29
*** yangyapeng has quit IRC00:32
*** tovin07_ has joined #openstack-kolla00:32
*** mnasiadka has quit IRC00:34
*** duonghq has joined #openstack-kolla00:37
*** mnasiadka has joined #openstack-kolla00:44
*** caowei has joined #openstack-kolla00:44
*** mnasiadka has quit IRC00:49
masberinc0, I got it working after rebuilding the controll node00:53
*** itlinux__ has quit IRC00:54
*** itlinux_ has quit IRC00:55
*** hw_wutianwei has joined #openstack-kolla00:57
*** mnasiadka has joined #openstack-kolla00:59
*** goldyfruit has joined #openstack-kolla00:59
*** salv-orlando has quit IRC01:03
*** salv-orlando has joined #openstack-kolla01:04
*** mnasiadka has quit IRC01:04
*** zhangfei has joined #openstack-kolla01:04
*** yingjun has quit IRC01:12
*** yangyapeng has joined #openstack-kolla01:12
*** mnasiadka has joined #openstack-kolla01:14
*** mnasiadka has quit IRC01:18
*** mnasiadka has joined #openstack-kolla01:29
*** mnasiadka has quit IRC01:34
*** mnasiadka has joined #openstack-kolla01:44
*** salv-orlando has quit IRC01:46
*** mnasiadka has quit IRC01:48
*** threestrands has joined #openstack-kolla01:51
*** threestrands has quit IRC01:51
*** threestrands has joined #openstack-kolla01:51
*** jappleii__ has quit IRC01:52
*** dardelean_ has quit IRC01:59
*** mnasiadka has joined #openstack-kolla01:59
*** hw_wutianwei1 has joined #openstack-kolla02:02
*** mnasiadka has quit IRC02:04
*** kristaps_ has joined #openstack-kolla02:05
rwelluminc0: yeah by running the pip upgrade directly in the container. :)02:05
*** pbourke_ has joined #openstack-kolla02:05
*** gema_ has joined #openstack-kolla02:06
*** goldyfruit has quit IRC02:07
*** hw_wutianwei has quit IRC02:07
*** pbourke has quit IRC02:07
*** harlowja has quit IRC02:07
*** lvdombrkr has quit IRC02:07
*** obre has quit IRC02:07
*** bkero has quit IRC02:07
*** wirehead_ has quit IRC02:07
*** gema has quit IRC02:07
*** obre_ has joined #openstack-kolla02:07
*** wirehead_ has joined #openstack-kolla02:07
*** hw_wutianwei1 is now known as hw_wutianwei02:07
*** goldyfruit has joined #openstack-kolla02:09
*** bkero has joined #openstack-kolla02:09
*** dave-mccowan has joined #openstack-kolla02:09
*** sambetts|afk has quit IRC02:10
*** sambetts_ has joined #openstack-kolla02:13
*** mnasiadka has joined #openstack-kolla02:14
*** yangyapeng has quit IRC02:14
*** dardelean_ has joined #openstack-kolla02:14
*** yangyapeng has joined #openstack-kolla02:15
*** semoac has joined #openstack-kolla02:18
*** dardelean_ has quit IRC02:18
*** mnasiadka has quit IRC02:19
*** Pavo has quit IRC02:23
*** Pavo has joined #openstack-kolla02:25
*** Pavo has quit IRC02:27
*** mnasiadka has joined #openstack-kolla02:27
*** unicell has quit IRC02:29
*** jtriley has quit IRC02:35
*** mnasiadka has quit IRC02:35
*** reddreamster has joined #openstack-kolla02:39
*** salv-orlando has joined #openstack-kolla02:47
*** goldyfruit has quit IRC02:51
*** goldyfruit has joined #openstack-kolla02:55
*** masuberu has joined #openstack-kolla03:03
openstackgerritMerged openstack/kolla master: Add python-vitrageclient to openstack-base container  https://review.openstack.org/52867003:03
*** masber has quit IRC03:04
*** jtriley has joined #openstack-kolla03:06
*** ManoX has quit IRC03:06
*** goldyfruit has quit IRC03:07
*** goldyfruit has joined #openstack-kolla03:10
*** jtriley has quit IRC03:11
*** zhangfei has quit IRC03:12
*** goldyfruit has quit IRC03:15
*** masuberu has quit IRC03:17
*** dave-mccowan has quit IRC03:20
*** zhangfei has joined #openstack-kolla03:24
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Update the default value for enable_openvswitch  https://review.openstack.org/52936803:25
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Update the default value for enable_openvswitch  https://review.openstack.org/52936803:29
*** jtriley has joined #openstack-kolla03:31
*** janonymous has joined #openstack-kolla03:35
*** jtriley has quit IRC03:35
*** threestrands_ has joined #openstack-kolla03:39
*** threestrands_ has quit IRC03:39
*** threestrands_ has joined #openstack-kolla03:39
*** jtriley has joined #openstack-kolla03:39
*** unicell has joined #openstack-kolla03:40
*** unicell has quit IRC03:40
*** threestrands has quit IRC03:41
*** coolsvap has joined #openstack-kolla03:54
*** reddreamster has quit IRC04:12
*** salv-orlando has joined #openstack-kolla04:15
*** masber has joined #openstack-kolla04:17
*** salv-orlando has quit IRC04:20
spsuryainc0:  around ?04:27
*** lebauce has quit IRC05:06
*** lebauce has joined #openstack-kolla05:06
*** ruhe2 has joined #openstack-kolla05:12
*** ruhe has quit IRC05:13
*** ruhe2 is now known as ruhe05:13
*** lebauce has quit IRC05:15
*** lebauce has joined #openstack-kolla05:15
*** salv-orlando has joined #openstack-kolla05:16
*** salv-orlando has quit IRC05:21
*** jtriley has quit IRC05:31
*** Arminder has quit IRC05:47
*** Arminder has joined #openstack-kolla05:47
*** ArminderSingh has joined #openstack-kolla05:47
*** zhangfei has quit IRC05:56
*** jtriley has joined #openstack-kolla05:57
*** janki has joined #openstack-kolla06:03
*** salv-orlando has joined #openstack-kolla06:14
*** dardelean_ has joined #openstack-kolla06:15
*** salv-orl_ has joined #openstack-kolla06:17
*** mwynne_ has joined #openstack-kolla06:19
*** dardelean_ has quit IRC06:19
*** threestrands_ has quit IRC06:21
*** ArminderSingh- has joined #openstack-kolla06:24
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Fix wrong ceph mgr keyring name during creating  https://review.openstack.org/52949106:24
*** mandre_ has joined #openstack-kolla06:25
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Enable multinode scenarios in zuulv3  https://review.openstack.org/51277906:25
*** salv-orlando has quit IRC06:26
*** ArminderSingh has quit IRC06:26
*** krtaylor has quit IRC06:26
*** itlinux has quit IRC06:26
*** alanmeadows has quit IRC06:26
*** vkmc has quit IRC06:26
*** mwynne has quit IRC06:26
*** robcresswell has quit IRC06:26
*** TheJulia has quit IRC06:26
*** llua has quit IRC06:26
*** kencjohnston has quit IRC06:26
*** harmw has quit IRC06:26
*** mandre has quit IRC06:26
*** mgkwill has quit IRC06:26
*** rwellum has quit IRC06:26
*** mandre_ is now known as mandre06:26
*** llua has joined #openstack-kolla06:28
*** itlinux has joined #openstack-kolla06:30
*** alanmeadows has joined #openstack-kolla06:31
*** vkmc has joined #openstack-kolla06:31
*** robcresswell has joined #openstack-kolla06:31
*** TheJulia has joined #openstack-kolla06:31
*** kencjohnston has joined #openstack-kolla06:31
*** harmw has joined #openstack-kolla06:31
*** mgkwill has joined #openstack-kolla06:31
*** rwellum has joined #openstack-kolla06:31
*** krtaylor has joined #openstack-kolla06:33
*** livelace-link has quit IRC06:33
*** livelace-link has joined #openstack-kolla06:35
*** livelace-link has quit IRC06:36
*** heigao has joined #openstack-kolla06:37
*** alanmeadows has quit IRC06:48
*** vkmc has quit IRC06:48
*** robcresswell has quit IRC06:48
*** TheJulia has quit IRC06:48
*** kencjohnston has quit IRC06:48
*** harmw has quit IRC06:48
*** mgkwill has quit IRC06:48
*** rwellum has quit IRC06:48
*** kiennt26 has joined #openstack-kolla06:49
*** unicell has joined #openstack-kolla06:51
*** gema_ is now known as gema06:51
*** gema has quit IRC06:52
*** gema has joined #openstack-kolla06:52
*** zhenguo has quit IRC07:04
*** unicell has quit IRC07:10
*** cah_link has joined #openstack-kolla07:10
*** cah_link has quit IRC07:13
*** unicell has joined #openstack-kolla07:13
*** cah_link has joined #openstack-kolla07:13
*** harmw has joined #openstack-kolla07:24
*** TheJulia has joined #openstack-kolla07:24
*** mgkwill has joined #openstack-kolla07:24
*** robcresswell has joined #openstack-kolla07:24
*** kencjohnston has joined #openstack-kolla07:25
*** rwellum has joined #openstack-kolla07:25
*** alanmeadows has joined #openstack-kolla07:25
*** vkmc has joined #openstack-kolla07:30
*** vkmc has quit IRC07:30
*** vkmc has joined #openstack-kolla07:30
*** mcnanci has quit IRC07:37
hrwelo07:38
*** egonzalez has joined #openstack-kolla07:39
*** ianw_pto is now known as ianw07:43
*** ianw is now known as ianw_pto07:43
Jeffrey4linc0, multi node jobs are green now. https://review.openstack.org/512779 . it is a bug in ceph-mgr implementation.07:45
hrwegonzalez: http://mirror.centos.org/centos/7/cloud/x86_64/openstack-pike/common/ has mariadb 10.1.20 so centos can be switched quite easy - just abandon 3rdparty repo07:52
egonzalezhrw, not that easy, will need to change how to deploy mariadb07:53
egonzalezhas some major changes in that release07:53
hrwegonzalez: that's too07:53
hrwright07:53
hrwubuntu/x86-64 and ubuntu/ppc64le is 3rdparty repo change. ubuntu/aarch64 will be on whatever is in repo but that's not tested combo07:55
Jeffrey4lhrw, egonzalez what's wrong with the mariadb version? any difference between 10.1 and 10.0?07:56
egonzalezJeffrey4l, yep, clustering changed a bit, galera stuff is included in main mariadb packages07:56
hrwJeffrey4l: galera?07:56
egonzalezand is disabled by default07:56
hrwJeffrey4l: and Debian is at 10.1 so changes to support it need to be done anyway07:57
Jeffrey4lso should enable it through configuration explicit? right?07:57
* hrw has no idea how galera stuff works07:58
*** pcaruana has joined #openstack-kolla07:58
Jeffrey4linteresting. if mariadb version in pike repo is 10.1, why my latest build image is using 10.0 still ?07:59
gemaxinliang: ^07:59
* Jeffrey4l is digging more..07:59
xinlianggema, Jeffrey4l: thanks, will help to test the patch08:00
hrwJeffrey4l: because centos and ubuntu use 3rdparty repo for MariaDB packages08:00
egonzalez we're using packages from mariadb repos in this case08:00
Jeffrey4lbut pike repo has higher version. it should it the higher version i think.08:01
hrwJeffrey4l: not quite08:01
hrwJeffrey4l: rdo/centos pike repo has mariadb packages. 3rdparty one has MariaDB ones08:01
hrwnames matter in yum world08:02
hrwcentos/x86 has THREE repos with mariadb packages08:02
hrwtwo external ones08:02
hrwcheck my comments in docker/mariadb/Dockerfile.j208:03
Jeffrey4lok. here is why http://paste.openstack.org/show/629550/08:03
hrwyep08:04
Jeffrey4lthe mariadb server name in mariadb and centos-pike repo is different.08:04
*** jtriley has quit IRC08:09
*** amoralej|off is now known as amoralej08:12
Jeffrey4li am thinking we should remove mariadb.repo ;/08:13
hrw+108:15
hrwand simplify mariadb/Dockerfile.j208:15
*** xinliang has quit IRC08:17
hrwJeffrey4l: speaking of mariadb/galera... https://review.openstack.org/#/c/529371/ is one step closer to get galera in debian working (simple s/ubuntu/debian+ubuntu)08:17
hrwJeffrey4l: egonzalez just gave +2 so you can finish it ;D08:17
Jeffrey4lhrw, LGTM. +w08:17
hrw;)08:18
egonzalezyeah, removing mariadb repo will ease a lot removing epel from base08:18
Jeffrey4llet me try to push a patch and see what we will got.08:18
Jeffrey4lfound centos-ocata is using mariadb 10.1 too.08:19
hrwJeffrey4l: rdo went for 10.1 long time ago iirc08:19
openstackgerritMerged openstack/kolla-ansible master: Fix wrong ceph mgr keyring name during creating  https://review.openstack.org/52949108:19
Jeffrey4lyes. newton is 10.1 too.08:19
hrwWed, 17 Feb 2016 00:11:59 UTC08:19
hrwwas first 10.1 build in centos08:20
hrwfor rdo08:20
Jeffrey4lhrw, how do you find the exact date?08:20
hrwhttp://cbs.centos.org/koji/buildinfo?buildID=877008:20
hrwjust went to cbs and searched for mariadb ;d08:20
hrwand hguemar is one of rdo devs08:21
Jeffrey4lcool.08:21
*** dardelean_ has joined #openstack-kolla08:25
openstackgerritMerged openstack/kolla-ansible master: mariadb/galera: handle Debian as well  https://review.openstack.org/52937108:26
hrwyay ;)08:26
hrwnow in my queue there are just two patches ;) one harmless revert https://review.openstack.org/528100 (removes using of ceph repo key for debian) and one which needs debugging (cpu_mode one)08:27
*** mnasiadka has joined #openstack-kolla08:28
*** xinliang has joined #openstack-kolla08:29
*** xinliang has joined #openstack-kolla08:29
hrwxinliang: welcome back08:32
*** gkadam has joined #openstack-kolla08:32
xinlianghrw: morning08:33
*** heigao has quit IRC08:33
hrwegonzalez, Jeffrey4l: I will push patch to k-a to enable wsrep_on by default. It can stay in queue for as long as it needs.08:33
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: mariadb: enable galera clustering by default  https://review.openstack.org/52950508:34
hrwxinliang: check this patch instead of your kolla one.08:34
*** salv-orl_ has quit IRC08:35
*** jtriley has joined #openstack-kolla08:35
hrwiirc it should make no change on all 10.0 targets08:35
xinlianghrw: ok08:35
*** salv-orlando has joined #openstack-kolla08:35
*** dardelean_ has quit IRC08:38
*** jtriley has quit IRC08:39
*** salv-orlando has quit IRC08:40
*** salv-orlando has joined #openstack-kolla08:44
*** mdnadeem has joined #openstack-kolla08:45
*** ktibi has joined #openstack-kolla08:49
hrwxinliang: k-a failed on deploy and went further on second attempt08:53
*** zhangfei has joined #openstack-kolla08:56
*** dardelean_ has joined #openstack-kolla08:57
xinlianghrw:  test failed on my side too.08:57
xinlianghrw: but with these two patches it works : https://review.openstack.org/#/c/529371/08:58
xinlianghttps://review.openstack.org/#/c/468632/12/docker/mariadb/extend_start.sh08:58
*** jtriley has joined #openstack-kolla08:58
hrwcool08:59
*** mgoddard has joined #openstack-kolla09:01
*** dardelean_ has quit IRC09:02
*** dardelean has joined #openstack-kolla09:02
*** jtriley has quit IRC09:05
*** shardy has joined #openstack-kolla09:05
*** jtriley has joined #openstack-kolla09:07
*** zhangfei has quit IRC09:07
xinlianghrw: It seems this " mysqld_safe --wsrep-new-cluster --skip-networking --wsrep-on=OFF" will overwrite wsrep_on = ON in galera.cnf.j209:09
hrwxinliang: isn't  =OFF called only on bootstrapping'09:10
hrw?09:10
xinlianghrw: yes09:11
hrwxinliang: so (imho) bootstrap is done on master mariadb host and then clustering stuff gets enabled and db is synced to slaves09:12
xinlianghrw: I'm not sure about this.09:14
openstackgerritJustin Scott proposed openstack/kolla-kubernetes master: WIP deploy with ansible inside orchestration container  https://review.openstack.org/52084709:17
*** athomas has joined #openstack-kolla09:19
*** zhangfei has joined #openstack-kolla09:20
*** robbbe has joined #openstack-kolla09:20
*** rmart04 has joined #openstack-kolla09:24
*** b_bezak has joined #openstack-kolla09:27
*** jascott1 has quit IRC09:39
*** jascott1 has joined #openstack-kolla09:39
*** dardelean_ has joined #openstack-kolla09:40
*** gfidente has joined #openstack-kolla09:42
*** gfidente has quit IRC09:42
*** gfidente has joined #openstack-kolla09:42
*** jascott1 has quit IRC09:44
*** salv-orlando has quit IRC09:44
*** salv-orlando has joined #openstack-kolla09:45
*** mnasiadk1 has joined #openstack-kolla09:45
*** mnasiadk1 has quit IRC09:45
*** salv-orlando has quit IRC09:49
*** jtriley has quit IRC09:53
xinlianghrw: you are write mariad bootstrap only running in master node : https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/mariadb/tasks/bootstrap.yml#L1009:55
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Deploy ceph mgr during upgrade  https://review.openstack.org/52952910:00
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable Docker container inter-host communication  https://review.openstack.org/52930810:02
*** caowei has quit IRC10:03
*** tovin07_ has quit IRC10:16
*** lpetrut has joined #openstack-kolla10:18
*** jtriley has joined #openstack-kolla10:19
*** kiennt26 has quit IRC10:22
*** jtriley has quit IRC10:24
*** jtriley has joined #openstack-kolla10:29
*** sambetts_ is now known as sambetts10:31
ktibiHi all, I have little issue with bifrost in docker => https://github.com/openstack/bifrost/blob/stable/pike/playbooks/roles/bifrost-ironic-install/tasks/bootstrap.yml#L4010:32
ktibithe playbook can't change /etc/hosts in my bifrost container10:33
ktibisomeone has already encountered this problem10:33
*** pcaruana has quit IRC10:43
hrwxinliang: can you refresh https://review.openstack.org/#/c/468632 one and send for review if you use it?10:43
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Workaround for Bifrost being unable to write to /etc/hosts  https://review.openstack.org/52954310:43
*** salv-orlando has joined #openstack-kolla10:45
*** salv-orlando has quit IRC10:50
*** ktibi has quit IRC10:56
*** ktibi has joined #openstack-kolla11:06
openstackgerritDuong Ha-Quang proposed openstack/kolla master: [WIP] Apply Cinder rolling upgrade  https://review.openstack.org/52954411:11
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: [WIP] Apply Cinder rolling upgrade  https://review.openstack.org/52954611:13
*** jtriley has quit IRC11:14
*** duonghq has quit IRC11:15
openstackgerritJuan Badia Payno proposed openstack/kolla master: WIP - fluentd rpm candidate  https://review.openstack.org/51529211:22
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable Docker container inter-host communication  https://review.openstack.org/52930811:23
*** salv-orlando has joined #openstack-kolla11:24
*** ArminderSingh- has quit IRC11:24
*** Arminder has quit IRC11:24
*** jtriley has joined #openstack-kolla11:39
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Updated the doc  https://review.openstack.org/52955411:43
*** jtriley has quit IRC11:45
*** jtriley has joined #openstack-kolla11:46
egonzalezhrw, xinliang in previous attemps i made to use 10.1, allinone worked but then failed to add slaves, ensure it works just with those changes in multinode11:50
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable Docker container inter-host communication  https://review.openstack.org/52930811:52
openstackgerritChason Chan proposed openstack/kolla master: Remove the fwaas-dashboard for binary install type  https://review.openstack.org/52956111:57
*** mcnanci has joined #openstack-kolla12:03
*** ArminderSingh has joined #openstack-kolla12:06
*** Arminder has joined #openstack-kolla12:06
*** dave-mccowan has joined #openstack-kolla12:10
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable Docker container inter-host communication  https://review.openstack.org/52930812:10
*** vbel has joined #openstack-kolla12:11
vbelhello, I try to understand gate failures in https://review.openstack.org/#/c/529308/ - checked all logs and currently out of further ideas. Any hints are welcome :)12:12
vbelsystemd erroneously concatenates ExecStartPost with something else and fails. At the same time it works in my servers (rh/c/ol 7.4)12:14
vbelhow can I see output of e.g. tools/setup_RedHat.sh?12:16
egonzalezvbel, maybe need to set execstartpost to emty and then set the value12:22
egonzalezempty, same as execstart12:22
vbelwill try that now, thanks!12:23
*** jtriley has quit IRC12:23
egonzalezvbel, i've not seen such networking issue12:23
egonzalezbeen using 1.12, 1.13 and 1.1712:24
egonzalez17-ce12:24
*** dave-mccowan has quit IRC12:24
vbelwhen I upgrade it works, but after real clean up of /var/lib/docker /var/run/docker ... fresh install shows DROP in 'iptables -L | grep FORWARD'12:25
vbelon docker 17 start up12:25
vbelthis is a change in behavior12:25
vbelmany of my colleagues did not notice that too :)12:26
*** shardy is now known as shardy_afk12:26
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable Docker container inter-host communication  https://review.openstack.org/52930812:27
*** Guest36 has joined #openstack-kolla12:35
vbelegonzalez, ec 21 12:33:07 localhost systemd: Starting Docker Application Container Engine...12:38
vbelDec 21 12:33:07 localhost docker: docker: 'daemonExecStartPost=/sbin/iptables' is not a docker command.12:38
vbelthe same nonsense in http://logs.openstack.org/08/529308/10/check/kolla-ansible-centos-source/3bac8a3/primary/system_logs/messages12:38
vbelreally wonder how it manages to concatenate what cannot be concatenated :)12:39
vbelmay be it does not run real systemd?12:40
*** jascott1 has joined #openstack-kolla12:41
*** jascott1 has quit IRC12:46
*** hw_wutianwei has quit IRC12:47
*** jtriley has joined #openstack-kolla12:50
*** janki has quit IRC12:50
*** b_bezak_ has joined #openstack-kolla12:54
*** jtriley has quit IRC12:55
*** b_bezak has quit IRC12:57
*** dave-mccowan has joined #openstack-kolla12:58
*** jtriley has joined #openstack-kolla13:00
*** dave-mcc_ has joined #openstack-kolla13:07
*** dave-mccowan has quit IRC13:08
*** goldyfruit has joined #openstack-kolla13:10
*** yangyapeng has quit IRC13:10
*** goldyfruit has quit IRC13:10
*** yangyapeng has joined #openstack-kolla13:15
*** yangyapeng has quit IRC13:19
*** salv-orl_ has joined #openstack-kolla13:20
*** salv-orlando has quit IRC13:22
*** shardy_afk is now known as shardy13:29
zhangfeiegonzalez, could you help take a look https://review.openstack.org/#/c/513353/13:31
*** aagate has joined #openstack-kolla13:33
*** jtriley has quit IRC13:35
*** salv-orl_ has quit IRC13:39
*** salv-orlando has joined #openstack-kolla13:39
*** salv-orlando has quit IRC13:44
*** coolsvap has quit IRC13:44
*** dardelean_ has quit IRC13:44
*** dardelean_ has joined #openstack-kolla13:46
*** dardelean_ has quit IRC13:46
*** dardelean_ has joined #openstack-kolla13:47
*** dardelean_ has quit IRC13:47
*** dardelean_ has joined #openstack-kolla13:47
*** bjolo has quit IRC13:50
*** caowei has joined #openstack-kolla13:53
*** jtriley has joined #openstack-kolla13:58
*** bjolo has joined #openstack-kolla13:58
*** mchlumsky has joined #openstack-kolla14:04
*** dardelean_ has quit IRC14:11
*** jtriley has quit IRC14:13
*** dardelean_ has joined #openstack-kolla14:15
*** goldyfruit has joined #openstack-kolla14:15
*** cah_link has quit IRC14:16
*** jtriley has joined #openstack-kolla14:21
rwellumsbezverk: quick question when you have time?14:21
sbezverkrwellum hey shoot14:21
rwellumI seem to recall you had a shortcut to speed up the process of bringing up kolla-k8s, when the helm charts were already generated? Do you remember what you did?14:22
rwellumkolla-kubernetes/tools/helm_build_all.sh - can take 15-20m, and doesn't seem worth if if it's the same helm charts14:24
rwellumI've been picking it apart - but no luck yet.14:25
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable Docker container inter-host communication  https://review.openstack.org/52930814:25
sbezverkrwellum hmm, are you sure it was me? frankly I do not remember.14:27
rwellumsbezverk: I thought so - so worth a shot. No problem.14:27
sbezverkthe only thing I could think if to use tar archive14:27
sbezverkwhere all charts are already packaged14:27
rwellumYeah that's what I am trying.14:28
rwellum'Error: found in requirements.yaml, but missing in charts/ directory: cinder-volume-lvm-daemonset, iscsid-daemonset, tgtd-daemonset'14:28
rwellumSo I am missing a step (or 2). It's a fun area so I will keep digging at it.14:28
*** mgoddard has quit IRC14:30
sbezverkyeah you need to make sure all dependent charts are in the tar file14:30
sbezverkcheck Kevin's code for building service level charts14:31
sbezverkif I remember right he was packaging all child charts into a service level bundle14:31
rwellumOh right I recall that too. ty.14:32
*** mgoddard has joined #openstack-kolla14:32
*** salv-orlando has joined #openstack-kolla14:40
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable Docker container inter-host communication  https://review.openstack.org/52930814:40
*** dardelean_ has quit IRC14:42
*** robbbe has quit IRC14:42
*** jascott1 has joined #openstack-kolla14:43
*** dardelean_ has joined #openstack-kolla14:44
*** robbbe has joined #openstack-kolla14:44
*** salv-orlando has quit IRC14:44
*** yangyapeng has joined #openstack-kolla14:44
*** yangyapeng has quit IRC14:47
*** jascott1 has quit IRC14:47
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Update the default value for enable_openvswitch  https://review.openstack.org/52936814:49
*** b_bezak_ has quit IRC14:49
*** gkadam has quit IRC14:50
*** yangyapeng has joined #openstack-kolla14:51
*** zhangfei has quit IRC14:53
*** jtriley has quit IRC15:01
*** dardelean_ has quit IRC15:02
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable Docker container inter-host communication  https://review.openstack.org/52930815:03
sdakekfox1111 quick shell question for a shell master :)15:06
*** dave-mcc_ has quit IRC15:09
sdakekfox1111 sed inplace editing is not portable across unix platforms; I've written something that makes sed portable: https://github.com/istio/istio/pull/2303/commits/958f93af4b07c54f3c46e86a12bc85fb96d90fb4#diff-c864be78fb30a2c170271b1b64576a9dR13515:09
*** mnasiadka has quit IRC15:10
sdakekfox1111 a question arose in the review - how to maintain the the file properties that sed inplace edit offers15:11
*** dave-mcc_ has joined #openstack-kolla15:12
*** jtriley has joined #openstack-kolla15:14
*** mnasiadka has joined #openstack-kolla15:18
larskssdake: if instead of 'mv $2.new $2' you do 'cat $2.new > $2', you will preserve the properties (permissions, ownership, etc) of the original file.15:19
larskssdake: (and of course rm -f $2.new afterwards)15:19
*** mnasiadka has quit IRC15:26
*** Guest36 has quit IRC15:27
*** itlinux has quit IRC15:27
*** caowei has quit IRC15:28
*** mnasiadka has joined #openstack-kolla15:31
*** magicboiz has joined #openstack-kolla15:31
*** caowei has joined #openstack-kolla15:33
*** egonzalez has quit IRC15:33
*** calbers has quit IRC15:33
*** mnasiadka has quit IRC15:35
*** magicboiz has quit IRC15:36
*** magicboiz has joined #openstack-kolla15:36
*** jamespd_ has quit IRC15:36
*** calbers has joined #openstack-kolla15:39
*** magicboiz has quit IRC15:42
*** mnasiadka has joined #openstack-kolla15:43
*** jamespd has joined #openstack-kolla15:44
*** mnasiadka has quit IRC15:47
*** mnasiadka has joined #openstack-kolla15:49
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable Docker container inter-host communication  https://review.openstack.org/52930815:57
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable Docker container inter-host communication  https://review.openstack.org/52930815:58
*** lpetrut has quit IRC16:09
*** itlinux has joined #openstack-kolla16:14
*** lpetrut has joined #openstack-kolla16:17
*** edolnx has joined #openstack-kolla16:18
*** ManoX has joined #openstack-kolla16:21
*** lpetrut has quit IRC16:23
*** rmart04 has quit IRC16:26
*** ktibi has quit IRC16:27
*** dardelean_ has joined #openstack-kolla16:31
*** dardelean_ has quit IRC16:31
*** dciabrin has quit IRC16:31
*** dardelean_ has joined #openstack-kolla16:31
*** dciabrin has joined #openstack-kolla16:32
*** dardelean_ has quit IRC16:36
*** salv-orlando has joined #openstack-kolla16:40
*** jascott1 has joined #openstack-kolla16:45
*** mnasiadka has quit IRC16:48
*** jascott1 has quit IRC16:49
*** mnasiadka has joined #openstack-kolla16:50
*** bmace has quit IRC16:55
*** bmace has joined #openstack-kolla16:55
*** mnasiadka has quit IRC16:56
*** mnasiadka has joined #openstack-kolla16:59
sdakelarsks that seems counterintuitive but I'll give it a go - thanks!17:01
larskssdake: actually, it's fairly intuitive when you think about it.  Using '>', you're not *creating a new file*, you're just truncating and re-writing an existing one.17:02
larsksUse mv, you are creating a new file (so umask, your uid, etc all come into play)17:03
*** pcaruana has joined #openstack-kolla17:04
*** mnasiadka has quit IRC17:06
*** ntpttr_laptop has joined #openstack-kolla17:07
*** mdnadeem has quit IRC17:10
*** mnasiadka has joined #openstack-kolla17:11
*** dciabrin_ has joined #openstack-kolla17:15
*** dciabrin has quit IRC17:17
*** mnasiadka has quit IRC17:18
*** mnasiadka has joined #openstack-kolla17:21
*** shardy has quit IRC17:23
*** mnasiadka has quit IRC17:26
*** mnasiadka has joined #openstack-kolla17:28
inc0good morning17:30
*** jascott1 has joined #openstack-kolla17:34
*** Pavo has joined #openstack-kolla17:35
*** mnasiadka has quit IRC17:36
*** caowei has quit IRC17:42
*** mgoddard has quit IRC17:48
*** robbbe has quit IRC17:53
*** robbbe has joined #openstack-kolla17:53
*** daidv has quit IRC17:56
*** hieulq has quit IRC17:56
*** daidv has joined #openstack-kolla17:56
*** hieulq has joined #openstack-kolla17:56
*** robbbe has quit IRC17:59
*** unicell has quit IRC18:03
*** gfidente is now known as gfidente|afk18:05
spsuryamorning inc018:10
jascott1hey inc0 what should storage_provider be when I dont have anything18:10
jascott1trying to run cinder-less as we discussed18:11
jascott1maria, rabbit and glance are freaking out without that setting18:11
*** zshi has quit IRC18:12
inc0hmm...how can we specify it to use local volume?18:14
*** harlowja has joined #openstack-kolla18:16
*** ntpttr_laptop has quit IRC18:16
jascott1ah I will look into it18:17
*** zshi has joined #openstack-kolla18:26
jascott1inc0 its supposed to be 'host' i think18:32
inc0possibly18:32
openstackgerritJustin Scott proposed openstack/kolla-kubernetes master: WIP deploy with ansible inside orchestration container  https://review.openstack.org/52084718:32
*** TxGirlGeek has joined #openstack-kolla18:33
sdakesup inc018:35
inc0good good sdake18:35
*** oanson has quit IRC18:44
spsuryasdake: hi...18:46
spsuryahope you are doing well18:47
spsuryagood night all18:48
*** unicell has joined #openstack-kolla18:50
*** TxGirlGeek has quit IRC18:51
*** sambetts is now known as sambetts|afk19:05
*** janonymous has quit IRC19:05
*** dave-mcc_ has quit IRC19:21
sdakesup spsurya19:32
sdakeya busy19:32
sdakethanks tho :)19:32
*** benjamin_diaz has joined #openstack-kolla19:44
benjamin_diazHello guys, anybody with experience with autoscaling with Gnocchi? I tried the templates that are out there, but the alarm never leaves 'insuficient_data' state.19:45
*** Pavo has quit IRC19:53
*** pcaruana has quit IRC20:12
*** amoralej is now known as amoralej|off20:14
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Quickstart rewrite  https://review.openstack.org/52620020:19
*** ntpttr_laptop has joined #openstack-kolla20:21
inc0rwellum: thanks for comments, feel free to re-review this ^ :)20:26
*** pcaruana has joined #openstack-kolla20:29
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Quickstart rewrite  https://review.openstack.org/52620020:38
*** owalsh_ has joined #openstack-kolla20:39
*** pcaruana has quit IRC20:42
*** owalsh has quit IRC20:42
*** ManoX has quit IRC20:45
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Quickstart rewrite  https://review.openstack.org/52620020:59
*** gfidente|afk has quit IRC21:11
*** benjamin_diaz has quit IRC21:21
jascott1nova-api-create-cell is logging this "Unknown Error (HTTP 504)". please let me know if anyone has seen this one21:25
jascott1ah this is hanging and returing 504: openstack hypervisor list21:28
*** threestrands_ has joined #openstack-kolla21:35
SamYaple504 as in bad gateway? are you doing a load balance jascott1?21:50
*** aagate has quit IRC21:52
jascott1SamYaple I dont *think* so22:04
jascott1yeah seems all the openstack commands get bad gateway22:14
jascott1hmm well not all22:15
*** Pavo has joined #openstack-kolla22:17
jascott1openstack endpoint list works22:22
jascott1I can curl all the endpoints22:22
jascott1imean they all giving 401 but at least I can connect to them22:22
SamYaplejascott1: is the endpoint in endpoint list the service directly, or haproxy?22:30
jascott1i dont see any haproxy anything22:31
SamYaplecan you paste your endpoint list?22:31
jascott1sure thanks http://paste.openstack.org/show/629607/22:32
SamYapleoh. hmm22:32
SamYapleso do you have dns name resolution for 'nova-api'?22:32
SamYaple`dig nova-api`22:32
jascott1i should (did before) but let me double check22:33
*** mchlumsky has quit IRC22:33
SamYaplewhen it does resolve, does it resolve to the ip that your nova-api service is bound too?22:34
*** andrewwebber_ has joined #openstack-kolla22:37
andrewwebber_When deploying the mariadb job for kolla-kubernetes I get the following error: MountVolume.SetUp failed for volume "pvc-8d896fb2-e69e-11e7-8bf8-000c2951e108" : rbd: image kubernetes-dynamic-pvc-8d8dd0cf-e69e-11e7-afbf-0a580b020004 is locked by other nodes22:38
jascott1SamYaple can curl by name for nova-api22:39
jascott1get 40122:39
jascott1checking IP22:39
jascott1svc/nova-api 's clusterIP matches that IP22:40
SamYapleis this kolla-k8a?22:40
jascott1yeah22:41
SamYapleshould have guess :/ then its got that iptables load balancer and i dont know how to help :(22:41
jascott1oh ok thanks for trying mang22:41
SamYaplegateway is timeing out, so i would assume its being blocked somewhere22:42
jascott1prob haproxy container like I said22:42
jascott1sorry, like you said :)22:42
SamYaplei dont think it uses haproxy, which makes it a bit more... different to debug22:43
jascott1there is one inside the nova-api pod but it says its ready22:44
jascott1surely its related if its a gateway timeout22:45
jascott1SamYaple if all the endpoints are curl'able and `openstack endpoint list` works but `openstack network list` and  `openstack hypervisor list` dont...22:48
*** salv-orlando has quit IRC22:49
*** salv-orlando has joined #openstack-kolla22:51
*** salv-orlando has quit IRC22:54
andrewwebber_is there a recommended commit/tag to test kolla-kubernetes22:54
*** rwsu has quit IRC22:55
*** zshi has quit IRC22:57
*** andrewwebber_ has quit IRC23:03
*** goldyfruit has quit IRC23:05
inc0jascott1: endpoint list is keystone and network list is neutron23:05
inc0seems like just nova misbehaving23:05
jascott1nova list also failing23:05
jascott1er rather hypervisor list23:06
jascott1inc0 any idea why id get a 504?23:07
inc0check haproxy container23:07
inc0or pod23:07
jascott1so I execd into that and curl'd the local port from the config and it at least gave me 40123:08
jascott1seems like its wired up23:08
*** zshi has joined #openstack-kolla23:10
inc0check /var/log/haproxy23:10
SamYapleinc0: does kolla-k8s use a haproxy pod? or the k8s service loadbalancing via iptables?23:16
inc0haproxy pod23:16
jascott1in nova-api's haproxy container but nova-api log, found this23:17
jascott1keystonemiddleware.auth_token [-] Unable to validate token: Unable to establish connection to http://10.10.10.254:35357/v3/auth/tokens23:18
inc0that sounds like 40123:18
jascott1theres a stack trace and [Errno 104] Connection reset by peer23:19
inc0oh, how helpful23:19
jascott1there is no haproxy log tho23:19
jascott1meaning nothing named haproxy in /var/log23:19
inc0kolla/haproxy?23:19
inc0well it should be there somewhere23:20
jascott1in haproxy container there is only /var/log/kolla/nova/nova-api.log23:20
jascott1hmm23:20
jascott1ps just show a bunch of haproxy cmds, dumb-init and bash -c kolla_start23:21
inc0check /etc/haproxy/haproxy.cfg to look for logs:)23:24
jascott1inc0 says `log /var/lib/kolla/heka/log local0`23:25
jascott1but /var/lib/kolla is empty :(23:26
inc0maybe we need to tweak it...23:26
inc0since heka is no more23:26
jascott1oops i meant /var/lib/kolla/haproxy is empty23:26
jascott1so maybe configured for dir that doesnt exist?23:26
inc0how about syslog23:27
inc0?23:27
jascott1in haproxy.cfg it has 0.0.0.0:877423:28
jascott1if I curl 127.0.0.1:8774 i get response23:28
inc0 /var/log/syslog23:28
inc0(for buntu)23:28
jascott1but if I do `ip a` get the IP and use that with that port i get connection refused23:28
inc0that's....interesting.23:29
jascott1i *think* 127 goes to pod?23:29
jascott1so every conainer in a pod can hit the others at whatever port23:30
inc0well ip a inside pdo should also go to pod23:30
jascott1should I change log path to something that exists?23:31
jascott1would assume woulld have to have new container23:31
jascott1on cenots23:31
jascott1centos23:31
jascott1and there is no /var/log/messages23:31
inc0:(23:32
jascott1oh journalctl23:32
jascott1duh23:32
jascott1nope23:32
inc0yeah systemd doesn't run in container23:32
jascott1oh i thought ubuntu pod had syslog.. cant remember23:33
jascott1anyway23:33
*** kbyrne has quit IRC23:33
jascott1inc0 can I run without haproxy if just to elimate it as the problem?23:39
*** kbyrne has joined #openstack-kolla23:39
inc0well if you can access pod itself23:40
inc0all you need to do is to edit endpoint to point to pod23:40
inc0or service23:40
jascott1hmm23:41
jascott1i can curl and get 401 to nova-api from problem container23:41
jascott1imean in problem container I can curl nova-api by name23:42
jascott1inc0 anything jump out about endpoint list? http://paste.openstack.org/show/629607/23:48
inc0looks good23:49
*** salv-orlando has joined #openstack-kolla23:55
jascott1yeah definitely neutron related23:56
jascott1using my handy dandy log viewer, ovs, nuetron, placement api all have errors23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!