Tuesday, 2017-12-05

*** signed8bit is now known as signed8bit_Zzz00:04
*** salv-orlando has quit IRC00:09
*** salv-orlando has joined #openstack-kolla00:09
*** keper7 has joined #openstack-kolla00:11
*** salv-orlando has quit IRC00:14
*** yangyapeng has quit IRC00:21
*** vhosakot_ has joined #openstack-kolla00:23
*** signed8bit_Zzz has quit IRC00:23
*** vhosakot has quit IRC00:26
*** athomas has quit IRC00:27
*** jtriley has quit IRC00:29
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode scenarios in zuulv3  https://review.openstack.org/51277900:32
*** jbrooks has joined #openstack-kolla00:33
*** dardelean has quit IRC00:33
*** masber has quit IRC00:33
Jeffrey4lhrw, hi, no we do not have. but i'd like to add it.00:36
SamYapleJeffrey4l: i thought kolla wasnt going to add lumionous support?00:36
Jeffrey4lSamYaple, yes. i am trying to add this.00:36
SamYapleah ok00:36
*** Pavo has quit IRC00:36
*** vhosakot has joined #openstack-kolla00:37
*** vhosakot_ has quit IRC00:38
openstackgerritJeffrey Zhang proposed openstack/kolla master: Bump to ceph luminous  https://review.openstack.org/52521600:38
*** masber has joined #openstack-kolla00:41
*** tovin07__ has joined #openstack-kolla00:43
*** kiennt26 has joined #openstack-kolla00:44
*** ntpttr_laptop has quit IRC00:44
*** cloud has joined #openstack-kolla00:44
*** cloud is now known as Guest156300:45
*** vhosakot has quit IRC00:46
Guest1563I use k8s-kolla. "sudo kolla-ansible genconfig" will fail00:46
Guest1563I use the workaroud "sudo sed -i "s/.*ansible_play_batch.*length.*//" /usr/local/share/kolla-ansible/ansible/site.yml"00:47
Guest1563it can success but I don't know it hurt or not00:47
*** masber has quit IRC00:47
Guest1563I follow the installation guide step by step.00:47
*** duonghq has joined #openstack-kolla00:48
Guest1563and I hit second issue: "kollakube res create configmap" fails00:48
*** dardelean has joined #openstack-kolla00:49
Guest1563but I don't know how to debug it00:49
*** affie has quit IRC00:49
*** klindgren__ has quit IRC00:50
*** jtriley has joined #openstack-kolla00:50
*** dardelean has quit IRC00:53
rwellumHi Guest1563 - the dev guide is a little out of date, and theres a patch set to fix it in review. The genconfig command should be:00:55
rwellumhttps://www.irccloud.com/pastebin/tYY0EUOP/00:55
*** tvignaud has quit IRC00:55
rwellumIf gen config fails, then the second step will not succeed.00:56
Guest1563thanks. rwellum. current my workaoud can pass but "kollakube res create configmap" fails. I will use your way00:57
Guest1563so I need to use your way before "sudo kolla-ansible genconfig"? or just replace it00:58
rwellumReplace it.00:58
rwellumGuest1563: see: https://review.openstack.org/#/c/524125/2/doc/source/deployment-guide.rst00:59
rwellum(In review)00:59
*** sai_ has quit IRC00:59
*** klindgren has joined #openstack-kolla01:01
*** zhurong has joined #openstack-kolla01:03
*** keper7 has quit IRC01:04
Guest1563great. thanks, rwellum:)01:05
rwellumSure np01:06
rwellumif you have further issues, it's kind of late for me here - but if you come on IRC a bit earlier there are people to help.01:06
rwellumWe also have playbooks and a script to get you going to if needed.01:07
*** Pavo has joined #openstack-kolla01:07
Jeffrey4lduonghq, hi ,your idea works. https://review.openstack.org/52231801:09
Guest1563Ubuntu does not currently work. Use centos01:09
Guest1563current, I use ubuntu.01:09
*** salv-orlando has joined #openstack-kolla01:10
Guest1563thanks, rwellum01:12
*** tvignaud has joined #openstack-kolla01:13
*** salv-orlando has quit IRC01:15
*** yangyapeng has joined #openstack-kolla01:16
*** caowei has joined #openstack-kolla01:18
lxdongwhat's this issue about http://paste.openstack.org/show/628127/?01:21
*** signed8bit has joined #openstack-kolla01:22
duonghqJeffrey4l, nice01:23
duonghqthank you01:23
*** keper7 has joined #openstack-kolla01:29
*** Pavo has quit IRC01:35
*** keper7 has quit IRC01:45
*** jbrooks has quit IRC01:46
*** mrunge has quit IRC01:56
*** hawi has joined #openstack-kolla01:56
*** mrunge has joined #openstack-kolla01:59
*** kevzha01 has joined #openstack-kolla02:00
*** kevzha01 is now known as kevinz02:00
*** zhurong has quit IRC02:00
*** salv-orlando has joined #openstack-kolla02:11
*** salv-orlando has quit IRC02:16
openstackgerritJeffrey Zhang proposed openstack/kolla master: Bump to ceph luminous  https://review.openstack.org/52521602:18
*** signed8b_ has joined #openstack-kolla02:20
*** pbourke_ has quit IRC02:23
*** signed8bit has quit IRC02:24
rwellumGuest1563: I don't agree with that statement about ubuntu which is why I commented on it in the review.02:25
*** pbourke_ has joined #openstack-kolla02:25
*** zhurong has joined #openstack-kolla02:30
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Add note about overwriting policy.json file  https://review.openstack.org/51118602:30
*** zhangfei has joined #openstack-kolla02:35
*** signed8b_ is now known as signed8bit_Zzz02:38
*** masber has joined #openstack-kolla02:39
*** signed8bit_Zzz has quit IRC02:43
*** jtriley has quit IRC02:48
*** jtriley has joined #openstack-kolla02:59
*** dmsimard has joined #openstack-kolla03:04
spsuryamorning guys03:07
spsuryasup duonghq03:07
*** klindgren has quit IRC03:09
*** salv-orlando has joined #openstack-kolla03:12
duonghqhi spsurya03:13
*** lukepatrick has quit IRC03:14
spsuryaduonghq: hope you doing well these days03:14
duonghqspsurya,  you too, do you have plan joining next PTG?03:16
*** salv-orlando has quit IRC03:17
spsuryasame ques i was also about to ask :)03:17
spsuryaHope so03:17
*** unicell has quit IRC03:18
duonghqnice, currently, I have not had any plan yet03:18
*** lukepatrick has joined #openstack-kolla03:18
inc0ah damn03:25
inc0https://review.openstack.org/#/c/512779/03:25
inc0I've seen this issue in my local env - ubuntu seems to dislike ceph - is that because it's already L in cloud archive?03:26
inc0hello everyone btw:)03:26
*** threestrands has joined #openstack-kolla03:27
spsuryahi....inc003:32
lxdongIs it due to osd_crush_chooseleaf_type = 0 for osds down?03:38
*** jbrooks has joined #openstack-kolla03:39
lxdongsingle node ceph situation03:51
*** dave-mccowan has quit IRC04:00
*** klindgren has joined #openstack-kolla04:03
lxdong all osds are down after creating in single node situation with luminous 12.2.204:04
*** klindgren_ has joined #openstack-kolla04:06
*** lrensing has joined #openstack-kolla04:06
*** coolsvap has joined #openstack-kolla04:09
*** klindgren has quit IRC04:10
*** janonymous has joined #openstack-kolla04:13
*** salv-orlando has joined #openstack-kolla04:13
*** salv-orlando has quit IRC04:18
*** skramaja has joined #openstack-kolla04:18
*** lukepatrick has quit IRC04:27
*** zhurong has quit IRC04:27
*** lukepatrick has joined #openstack-kolla04:35
*** jtriley has quit IRC04:36
*** zhurong has joined #openstack-kolla04:54
*** jaosorior has joined #openstack-kolla04:55
*** lrensing has quit IRC04:59
*** gfidente has quit IRC05:07
*** heigao_1212 has joined #openstack-kolla05:11
*** salv-orlando has joined #openstack-kolla05:14
*** heigao_1982 has quit IRC05:14
*** heigao_1982 has joined #openstack-kolla05:15
*** heigao_1212 has quit IRC05:15
*** salv-orlando has quit IRC05:18
*** jbrooks has quit IRC05:18
*** jtriley has joined #openstack-kolla05:20
*** k_mouza has joined #openstack-kolla05:22
*** threestrands has quit IRC05:25
*** dklyle has joined #openstack-kolla05:25
*** k_mouza has quit IRC05:27
*** david-lyle has quit IRC05:29
*** dklyle has quit IRC05:30
*** gfidente has joined #openstack-kolla05:34
*** lukepatrick has quit IRC05:35
SamYapleinc0: yea its a known issue. kolla-pike and above ceph is borked05:52
inc0SamYaple: cuz of Luminous?05:52
SamYapleinc0: you can potentially do external lumionous to makeit work05:52
SamYapleyea05:52
*** unicell has joined #openstack-kolla05:54
*** janki has joined #openstack-kolla05:58
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: Fix deployment of ceph  https://review.openstack.org/52547005:59
*** unicell1 has joined #openstack-kolla05:59
*** unicell has quit IRC06:01
*** gfidente has quit IRC06:03
*** jtriley has quit IRC06:14
*** salv-orlando has joined #openstack-kolla06:14
*** salv-orlando has quit IRC06:19
*** oanson has quit IRC06:20
*** oanson has joined #openstack-kolla06:20
openstackgerritKevin Zhao proposed openstack/kolla-kubernetes master: Bump haproxy_exporter version from v0.7.1 to v0.8.0  https://review.openstack.org/52547306:29
*** zhangfei has quit IRC06:38
*** skramaja is now known as skramaja|afk06:38
*** jtriley has joined #openstack-kolla06:39
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: ansiblize the init-runonce script  https://review.openstack.org/52307206:48
*** kiennt26 has quit IRC06:48
*** dardelean has joined #openstack-kolla06:49
*** zhangfei has joined #openstack-kolla06:50
*** dardelean has quit IRC06:54
*** egonzalez has joined #openstack-kolla07:00
*** jtriley has quit IRC07:00
*** cah_link has joined #openstack-kolla07:05
*** kiennt26 has joined #openstack-kolla07:07
*** salv-orlando has joined #openstack-kolla07:15
*** zhurong has quit IRC07:16
*** salv-orlando has quit IRC07:20
*** salv-orlando has joined #openstack-kolla07:26
*** jtriley has joined #openstack-kolla07:27
*** jtriley has quit IRC07:32
*** pcaruana has joined #openstack-kolla07:32
*** jtriley has joined #openstack-kolla07:33
openstackgerritKevin Zhao proposed openstack/kolla master: Creates directory /var/run/mysqld on aarch64 when building MariaDB docker image  https://review.openstack.org/52147907:33
*** zhubingbing has joined #openstack-kolla07:34
*** zhubingbing_ has quit IRC07:34
*** genek has quit IRC07:35
*** genek has joined #openstack-kolla07:36
openstackgerritJianghua Wang proposed openstack/kolla-ansible master: XenAPI: support XenAPI in nova configuration  https://review.openstack.org/50819607:36
openstackgerritMerged openstack/kolla-ansible master: fix shell error about tools/init-vpn  https://review.openstack.org/51755307:40
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Implement blazar ansible role  https://review.openstack.org/45475207:46
*** zhubingbing_ has joined #openstack-kolla07:46
*** zhubingbing has quit IRC07:46
openstackgerritMerged openstack/kolla-ansible stable/ocata: Use openstack_service_workers variable for gnocchi metricd workers  https://review.openstack.org/52515307:48
openstackgerritMerged openstack/kolla-ansible master: Fix deploy ceph failed when contoller hava no cluster_interface  https://review.openstack.org/50752007:48
*** jtriley has quit IRC07:49
*** zhurong has joined #openstack-kolla07:49
*** jtriley has joined #openstack-kolla07:53
*** athomas has joined #openstack-kolla07:54
*** codenamelxl has joined #openstack-kolla07:55
*** salv-orlando has quit IRC07:57
*** salv-orlando has joined #openstack-kolla07:57
codenamelxlhi everyone, i just set up a new openstack on Pike using kolla-ansible. "enable_neutron_dvr" set to yes. My problem is any floating IPs that stay on the network node is unreachable.07:58
codenamelxlFloating IPS located on compute node can be reach just fine07:58
*** tvignaud has quit IRC07:58
*** tvignaud has joined #openstack-kolla07:59
*** skramaja|afk is now known as skramaja08:00
*** jtriley has quit IRC08:00
*** egonzalez has quit IRC08:01
*** salv-orlando has quit IRC08:02
*** jtriley has joined #openstack-kolla08:03
*** heigao_1212 has joined #openstack-kolla08:04
*** heigao_1982 has quit IRC08:04
*** jascott1 has quit IRC08:07
*** jascott1 has joined #openstack-kolla08:07
*** egonzalez has joined #openstack-kolla08:09
*** jtriley has quit IRC08:10
*** jaosorior_ has joined #openstack-kolla08:10
*** b_bezak has joined #openstack-kolla08:11
*** jascott1 has quit IRC08:12
openstackgerritJeffrey Zhang proposed openstack/kolla master: Bump to ceph luminous  https://review.openstack.org/52521608:12
*** jaosorior has quit IRC08:13
openstackgerritChason Chan proposed openstack/kolla master: Add the missing murano package files                                                                                                                                                                                                            https://review.openstack.org/52251608:16
*** jtriley has joined #openstack-kolla08:16
openstackgerritMerged openstack/kolla master: Change more repo url to infra mirror  https://review.openstack.org/52231808:17
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Implement ceph-mgr service  https://review.openstack.org/52549508:17
openstackgerritJeffrey Zhang proposed openstack/kolla stable/pike: Change more repo url to infra mirror  https://review.openstack.org/52549608:19
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: Change more repo url to infra mirror  https://review.openstack.org/52549708:19
*** jtriley has quit IRC08:23
*** shardy has joined #openstack-kolla08:24
*** genek has quit IRC08:26
*** unicell1 has quit IRC08:26
*** mdnadeem has joined #openstack-kolla08:27
*** jtriley has joined #openstack-kolla08:27
codenamelxlhi everyone, i just set up a new openstack on Pike using kolla-ansible. "enable_neutron_dvr" set to yes. My problem is any floating IPs that stay on the network node is unreachable. Floating IP located on compute node can be reach just fine08:29
lxdongI ran into this issue:http://paste.openstack.org/show/628135/  can not create instance in single external ceph node cluster.08:30
*** heigao_1212 has quit IRC08:31
*** jtriley has quit IRC08:33
openstackgerritJeffrey Zhang proposed openstack/kolla master: Bump to ceph luminous  https://review.openstack.org/52521608:33
*** gkadam has joined #openstack-kolla08:38
*** jtriley has joined #openstack-kolla08:38
chasonegonzalez Hi, any work need us to do here? https://bugs.launchpad.net/kolla/+bug/173620608:39
openstackLaunchpad bug 1736206 in kolla "Deployer is unable to set MariaDB memory usage" [Wishlist,Triaged]08:40
*** pcaruana has quit IRC08:40
egonzalezchason, you can add a var into mariadb defaults08:41
chasonegonzalez OK, I will do it.08:42
chasonegonzalez Thanks! :D08:42
*** jtriley has quit IRC08:45
*** jaosorior_ is now known as jaosorior08:47
*** salv-orlando has joined #openstack-kolla08:47
hrwmorning08:48
hrwhttps://review.openstack.org/#/c/521479 - can someone look?08:48
*** jtriley has joined #openstack-kolla08:49
*** k_mouza has joined #openstack-kolla08:53
*** jtriley has quit IRC08:55
*** skramaja is now known as skramaja|lunch08:55
*** salv-orlando has quit IRC08:57
*** salv-orlando has joined #openstack-kolla08:57
*** magicboiz has quit IRC09:01
*** salv-orlando has quit IRC09:02
*** jtriley has joined #openstack-kolla09:03
*** jtriley has quit IRC09:10
*** zhangfei has quit IRC09:12
*** magicboiz has joined #openstack-kolla09:12
*** dardelean has joined #openstack-kolla09:15
*** jtriley has joined #openstack-kolla09:16
*** magicboiz has quit IRC09:17
*** magicboiz has joined #openstack-kolla09:17
*** ktibi has joined #openstack-kolla09:20
*** jtriley has quit IRC09:22
*** Zophar has joined #openstack-kolla09:28
*** jtriley has joined #openstack-kolla09:29
*** k_mouza has quit IRC09:29
*** k_mouza_ has joined #openstack-kolla09:29
*** heigao has joined #openstack-kolla09:29
*** affie has joined #openstack-kolla09:30
*** zhangfei has joined #openstack-kolla09:30
*** k_mouza_ has quit IRC09:30
*** k_mouza has joined #openstack-kolla09:31
*** k_mouza has quit IRC09:32
*** affie_ has joined #openstack-kolla09:32
*** k_mouza has joined #openstack-kolla09:32
ktibimorning all ;)09:33
dardeleanhi09:33
*** skramaja|lunch is now known as skramaja09:34
*** jtriley has quit IRC09:35
spsuryahrw: done09:35
hrwthanks09:35
*** affie has quit IRC09:35
*** gfidente has joined #openstack-kolla09:37
*** gfidente has quit IRC09:37
*** gfidente has joined #openstack-kolla09:37
*** jtriley has joined #openstack-kolla09:37
chasono/09:39
chasonhttps://review.openstack.org/#/c/512576/09:39
chasoncan someone review this patch? :)09:39
ktibiI have issue with telegraf and kolla. Who use telegraf here ?09:43
ktibithe bug : https://github.com/influxdata/telegraf/issues/135209:43
*** jtriley has quit IRC09:44
ktibibug can fix on centos with service proc-sys-fs-binfmt_misc.automount restart09:46
openstackgerritMerged openstack/kolla master: Creates directory /var/run/mysqld on aarch64 when building MariaDB docker image  https://review.openstack.org/52147909:51
*** owalsh_ is now known as owalsh09:52
*** heigao has quit IRC09:52
*** jtriley has joined #openstack-kolla09:52
*** bjolo has quit IRC09:54
openstackgerritMartin André proposed openstack/kolla master: Validate Dockerfile indentation in pep8  https://review.openstack.org/52551609:55
mandreegonzalez: ^^ you'll love it I'm sure09:59
*** gfidente has quit IRC10:00
*** jtriley has quit IRC10:07
*** gfidente has joined #openstack-kolla10:09
*** gfidente has quit IRC10:09
*** gfidente has joined #openstack-kolla10:09
codenamelxlI feel like centralized floating IP on DVR mode  deployed with Kolla is broken.10:10
codenamelxlAny floating IPs that stay on the network node is unreachable. Floating IP located on compute node can be reach just fine.10:11
codenamelxlOr am i doing something wrong?10:11
*** sambetts|afk is now known as sambetts10:14
*** jtriley has joined #openstack-kolla10:17
*** kevinz has quit IRC10:17
egonzalezcodenamelxl, in dvr, floating IPs should not be at network hosts, only in compute10:20
*** kbaegis has joined #openstack-kolla10:20
*** genek has joined #openstack-kolla10:21
kolla-slack<aslancimbom100> Morning/evening10:21
codenamelxlIt will be on network node if i create a centralized router or distributed router with this use case https://bugs.launchpad.net/neutron/+bug/158369410:21
openstackLaunchpad bug 1583694 in neutron "[RFE] DVR support for Allowed_address_pair port that are bound to multiple ACTIVE VM ports" [Wishlist,Fix released] - Assigned to Swaminathan Vasudevan (swaminathan-vasudevan)10:21
*** kiennt26 has quit IRC10:22
kolla-slack<aslancimbom100> Anyone know if I have to disable local chronyd to use enable_chrony?10:22
*** jtriley has quit IRC10:23
codenamelxl@egonzalez So basically, if i use DVR, Octavia won't work since the floating IP of Octavia  will be on the network node. But it's not only Octavia, any floating IPs located on network node is unreachable10:26
*** genek has quit IRC10:26
*** k_mouza_ has joined #openstack-kolla10:29
*** jtriley has joined #openstack-kolla10:30
*** tovin07__ has quit IRC10:30
zhangfeiHi, need help review https://review.openstack.org/#/c/513353/, thanks10:30
*** k_mouza has quit IRC10:33
*** k_mouza_ has quit IRC10:34
*** k_mouza has joined #openstack-kolla10:35
*** jtriley has quit IRC10:37
*** jtriley has joined #openstack-kolla10:38
*** jtriley has quit IRC10:43
*** daidv has quit IRC10:50
*** jtriley has joined #openstack-kolla10:51
*** codenamelxl has quit IRC10:53
*** jtriley has quit IRC10:58
*** kbaegis has quit IRC10:58
*** serlex has joined #openstack-kolla11:00
*** kbaegis has joined #openstack-kolla11:03
*** daidv has joined #openstack-kolla11:05
*** jtriley has joined #openstack-kolla11:05
*** zhangfei has quit IRC11:06
*** jascott1 has joined #openstack-kolla11:10
*** jtriley has quit IRC11:11
*** jascott1 has quit IRC11:14
*** jtriley has joined #openstack-kolla11:15
*** mgoddard has joined #openstack-kolla11:18
*** caowei has quit IRC11:21
*** jtriley has quit IRC11:22
*** jtriley has joined #openstack-kolla11:29
*** k_mouza_ has joined #openstack-kolla11:31
*** mwynne has quit IRC11:33
*** k_mouza has quit IRC11:34
*** jtriley has quit IRC11:36
*** janki has quit IRC11:39
*** mwynne has joined #openstack-kolla11:41
*** jtriley has joined #openstack-kolla11:43
*** jtriley has quit IRC11:50
*** mwynne has quit IRC11:54
*** mwynne has joined #openstack-kolla11:55
*** jtriley has joined #openstack-kolla11:59
*** kevinz has joined #openstack-kolla12:02
*** kbaegis has quit IRC12:03
*** kbaegis has joined #openstack-kolla12:06
*** jtriley has quit IRC12:06
*** kbaegis has quit IRC12:08
*** serlex has quit IRC12:14
*** jtriley has joined #openstack-kolla12:15
*** k_mouza_ has quit IRC12:17
*** k_mouza has joined #openstack-kolla12:21
*** jtriley has quit IRC12:36
*** skramaja has quit IRC12:36
*** zhurong has quit IRC12:36
*** jtriley has joined #openstack-kolla12:41
*** zhurong has joined #openstack-kolla12:45
*** jtriley has quit IRC12:47
*** david-lyle has joined #openstack-kolla12:48
*** jtriley has joined #openstack-kolla12:51
*** gfidente has quit IRC12:54
*** gfidente has joined #openstack-kolla12:56
*** gfidente has quit IRC12:56
*** gfidente has joined #openstack-kolla12:56
openstackgerritMerged openstack/kolla-ansible master: Removed -e Docker flag in ovsdpdk task  https://review.openstack.org/52349812:57
*** jtriley has quit IRC12:57
*** david-lyle has quit IRC13:01
*** zhurong has quit IRC13:02
*** keper7 has joined #openstack-kolla13:02
*** zhurong has joined #openstack-kolla13:02
openstackgerritMerged openstack/kolla-ansible master: Add content about cluster_network interface  https://review.openstack.org/51454413:05
*** jtriley has joined #openstack-kolla13:06
*** rhallisey has joined #openstack-kolla13:10
*** jtriley has quit IRC13:12
Jeffrey4linc0, egonzalez could you review this https://review.openstack.org/52549713:17
Jeffrey4lspsurya, ^^13:17
*** jtriley has joined #openstack-kolla13:22
*** signed8bit has joined #openstack-kolla13:24
*** genek has joined #openstack-kolla13:25
*** zhurong has quit IRC13:27
*** jtriley has quit IRC13:28
*** jtriley has joined #openstack-kolla13:30
*** yangyapeng has quit IRC13:33
*** yangyapeng has joined #openstack-kolla13:34
*** kevzha01_ has joined #openstack-kolla13:34
*** jtriley has quit IRC13:36
*** kevinz has quit IRC13:37
*** yangyapeng has quit IRC13:38
openstackgerritPierre Hanselmann proposed openstack/kolla-ansible master: Fix dhcp_agent configuration for vmware DVS  https://review.openstack.org/52560613:40
*** janki has joined #openstack-kolla13:40
openstackgerritChason Chan proposed openstack/kolla-ansible master: Add vitrage ansible role  https://review.openstack.org/43260813:42
*** jtriley has joined #openstack-kolla13:44
ktibiJeffrey4l Do you think we can expose ceph osd socket for monitoring the cluster with by example calamari ??13:48
Jeffrey4lktibi, hrm, how? and how about use ceph-mgr, which i am trying to implement.13:49
*** jtriley has quit IRC13:51
dmsimardJeffrey4l: I haven't kept up with https://review.openstack.org/#/c/521450/ very much13:53
dmsimardJeffrey4l: does it yield any improvements in performance after all ?13:53
Jeffrey4ldmsimard, yes. at least some package is cached by squid13:54
*** signed8bit has quit IRC13:54
Jeffrey4lhttp://logs.openstack.org/50/521450/40/check/kolla-build-centos-source/7ea6506/logs/system_log/squid/access.txt.gz13:55
Jeffrey4ldmsimard, here is the squid access logs.13:55
*** signed8bit has joined #openstack-kolla13:55
*** jtriley has joined #openstack-kolla13:55
dmsimardJeffrey4l: I was hoping for something with the squid implementation, maybe you can tell me if it'd work or not -- it's what I had discussed with inc013:56
Jeffrey4lthere are lots of package download multi time in kolla. with squid, the downloaded package is get very fast.13:56
Jeffrey4lit works13:56
dmsimardJeffrey4l: some of the kolla images have 30+ layers13:56
Jeffrey4lyes. this is another story ;(13:57
Jeffrey4lor another issue.13:57
dmsimardWhen doing a "docker push", the docker client does a "GET" for every layer of an image and the registry returns a 200 OK if found (client will not push) or 404 Not found (client will push)13:57
dmsimardThose lookups are very expensive when doing a docker push of 125+ images13:57
Jeffrey4lis this will be a bottle neck?13:57
dmsimardYeah, RDO builds those 125+ images and pushes them to a registry sitting in the same network and it's definitely a large part of the push bottleneck13:58
dmsimardTailing the logs, you can just see the Docker client hammering the registry with image layer lookups13:58
*** dave-mcc_ has joined #openstack-kolla13:58
Jeffrey4lwe can implement --squash in "docker build"13:58
dmsimardNot a good option, at least for us, it would heavily increase the storage usage13:58
dmsimardbecause it kills the savings we get from sharing layers13:58
Jeffrey4ldmsimard, no. just use one layer for single image.13:58
dmsimardso I was hoping we'd be able to cache the 200 OK's from the registry13:58
Jeffrey4lit won't increase the size.13:59
dmsimardIt would increase the size *on disk* for the registry13:59
dmsimardbecause you have 125 large images with one layer each, instead of 125 images which share a bunch of layers13:59
Jeffrey4lhrm. i am not sure whether it will squash parent image. if so, this won't be a good idea.14:00
Jeffrey4lso do you have any idea for this? dmsimard14:00
* Jeffrey4l is checking the squash parameter.14:01
*** jtriley has quit IRC14:01
ktibiJeffrey4l We just need to add a volume on /var/run/ceph/14:02
*** mchlumsky has joined #openstack-kolla14:02
ktibilike haproxy : https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/haproxy/defaults/main.yml#L1414:02
*** mmethot|away is now known as mmethot14:02
Jeffrey4lktibi, i think this is OK.14:02
*** jtriley has joined #openstack-kolla14:03
*** mchlumsky has quit IRC14:04
*** mchlumsky has joined #openstack-kolla14:06
openstackgerritMerged openstack/kolla master: Remove duplicated statement of parameter choices of config options  https://review.openstack.org/51460814:06
*** jtriley has quit IRC14:09
openstackgerritMerged openstack/kolla master: Update swift for replication issues  https://review.openstack.org/52283814:09
*** jascott1 has joined #openstack-kolla14:12
DosBarbos********************14:13
DosBarbosHello all,14:14
DosBarboshow can I fix this problem? anyone knows? https://bugs.launchpad.net/kolla-kubernetes/+bug/166602214:14
openstackLaunchpad bug 1666022 in kolla-kubernetes "kolla-kubernetes run command abnormal" [Critical,Invalid]14:14
DosBarbosoutput of command kolla-kubernetes and kollakube is the same :(14:16
*** jascott1 has quit IRC14:16
DosBarbosi try to use the bugfix from this link https://review.openstack.org/#/c/447356/8/doc/source/deployment-guide.rst@41 ( git fetch git://git.openstack.org/openstack/kolla-kubernetes refs/changes/40/439740/17 && git cherry-pick FETCH_HEAD ) but it didnt help14:17
*** dardelean has quit IRC14:18
DosBarbosanyone? where are kolla-kube masters? :)14:19
DosBarbosplease ...14:19
dmsimardJeffrey4l: have you tested the squid implementation when doing pushes ?14:21
dmsimardlike for example we shouldn't cache 404's but we should cache 200's14:22
Jeffrey4ldmsimard, push image won't use squid.14:22
dmsimardJeffrey4l: why not ?14:22
Jeffrey4lbut i can add this to check what will happen14:22
Jeffrey4lsquid is transport proxy. and should export http_proxy explicity.14:22
dmsimardJeffrey4l: ah so it's set a transparent right now, not http ?14:22
Jeffrey4lit is a http proxy. and a transparent http proxy.14:23
Jeffrey4lit is a http proxy. not a transparent http proxy.14:24
*** signed8bit is now known as signed8bit_Zzz14:30
ktibiJeffrey4l on fresh install, I have cinder-volume error with ceph ==>  Error connecting to ceph cluster.: TimedOut: error connecting to the cluster14:31
DosBarbosвщ нщг рфму фтн швуф фищге ьш вуздщньуте зкщидуь,14:33
DosBarbossorry14:33
DosBarbosdo you have any idea about my problem14:33
DosBarbos?14:33
*** signed8bit_Zzz is now known as signed8bit14:36
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Add config necessary to use Barbican with Murano  https://review.openstack.org/52563114:36
DosBarbosjeffrey41 dmsimard ?14:36
*** kevzha01_ has quit IRC14:40
*** magicboiz has quit IRC14:40
*** jtriley has joined #openstack-kolla14:41
rwellumDosBarbos: hi14:42
rwellumI have DosBarbos - I've never seen this before - not sure what it is, where it came from?14:43
*** lrensing has joined #openstack-kolla14:44
*** jaosorior has quit IRC14:46
rwellumDosBarbos: if you're having issues, please try: https://github.com/RichWellum/k8s - the tool ko.py will launch an AIO kolla-kubernetes cluster. You can then see if your environment is good or not.14:46
DosBarbosHello rwellum, it is after installations stap (sudo pip install -U kolla-ansible/ kolla-kubernetes/) regarding the official documentations https://docs.openstack.org/kolla-kubernetes/latest/deployment-guide.html14:46
DosBarbosthx a lot - will chk14:47
DosBarbosseems it is your own deployment way14:48
rwellumDosBarbos: yes you don't run kolla-kubernetes directly as per your bug written. Not that I'm aware of anyway.14:48
keper7hey I'm trying to get both ceph and kolla-k8s on the same hardware. The docs says the kolla-k8s ceph code isn't ready. Is the current best practice to make a kolla-ansible inventory with just the storage nodes populated and install ceph that way?14:48
rwellumDosBarbos: it's basically the deployment guide written in python14:49
rwellumDosBarbos: if you run in --demo mode it will explain each step. The deployment guide is a little out of date and theres a PS currently in review.14:50
DosBarbosas i understand I can deploy it on openstack VM.14:50
rwellumkeper7: some users have deployed ceph in an AIO environment with kolla-k8s. I haven't yet.14:50
rwellumDosBarbos: yes I have done that many times.14:50
rwellumDosBarbos: networking can be a little interesting, but you can certainly get a viable kolla-k8s cluster up and running.14:51
DosBarbosthx a lot again! i will try to use your easiest way to deploy14:51
*** goldyfruit has joined #openstack-kolla14:52
DosBarbosok-i need POC to check "what is that". After that i will try to expand cluster14:53
keper7rwellum: I'm trying to do a 3-node deployment currently, for HA. How do you currently deploy ceph?14:53
rwellumDosBarbos: if you want pike instead of ocata then you need to do: 'ko.py eth1 eth2 -iv pike -dr rwellum' - as the pike images are not pushed officially to dockerhub.14:53
DosBarboswhat is your opinion: This solutions is ready to prod? I mean kolla-kubernates14:53
rwellumkeper7: I haven't used ceph yet (just use a basic LVM backend). But there is a playbook in review (https://review.openstack.org/#/c/520847/) which has ceph to multiple hosts you can look at.14:55
*** lpetrut has joined #openstack-kolla14:55
rwellumkeper7: also the author can help when he's online (jascott1)14:55
rwellumDosBarbos: no it is not production ready; release one is planned for Rocky.14:55
*** manheim has joined #openstack-kolla14:57
*** manheim is now known as dardelean14:57
dardeleanis it mandatory to use fqdn when using TLS?14:58
keper7rwellum: Regarding production ready-ness, is kolla-ansible production-ready?14:58
rwellumkeper7: yes production ready, very stable, at least 9% of deployments in the World use it AFAIK.15:00
*** lrensing has quit IRC15:00
*** lrensing has joined #openstack-kolla15:03
*** zhubingbing has joined #openstack-kolla15:05
zhubingbinghi guys15:06
*** shardy is now known as shardy_afk15:08
DosBarboshi15:09
*** lrensing_ has joined #openstack-kolla15:11
*** janki has quit IRC15:13
*** david-lyle has joined #openstack-kolla15:13
*** janki has joined #openstack-kolla15:14
*** lrensing has quit IRC15:14
*** lrensing_ is now known as lrensing15:14
*** janki has quit IRC15:14
*** janki has joined #openstack-kolla15:15
*** cah_link1 has joined #openstack-kolla15:22
*** cah_link has quit IRC15:24
*** cah_link1 is now known as cah_link15:24
*** vhosakot has joined #openstack-kolla15:28
*** cduchesne has quit IRC15:31
*** cah_link has quit IRC15:31
*** cah_link has joined #openstack-kolla15:32
*** DosBarbos has quit IRC15:32
*** hongbin has joined #openstack-kolla15:33
openstackgerritMarcin Juszkiewicz proposed openstack/kolla stable/pike: Creates directory /var/run/mysqld on aarch64 when building MariaDB docker image  https://review.openstack.org/52566315:36
*** cduchesne has joined #openstack-kolla15:36
*** david-lyle has quit IRC15:39
hrwthis may be one of last stable/pike backports15:39
*** shardy_afk is now known as shardy15:42
*** cah_link1 has joined #openstack-kolla15:44
*** cah_link has quit IRC15:45
*** cah_link1 is now known as cah_link15:45
*** goldyfruit has quit IRC15:47
*** cah_link has quit IRC15:48
*** openstackgerrit has quit IRC15:48
*** cah_link has joined #openstack-kolla15:48
*** goldyfruit has joined #openstack-kolla15:49
*** dardelean has quit IRC15:50
*** dardelean has joined #openstack-kolla15:51
*** dardelean has quit IRC15:56
*** cah_link has quit IRC15:57
*** lrensing_ has joined #openstack-kolla15:58
*** lrensing_ has quit IRC15:59
*** lrensing has quit IRC16:00
*** egonzalez has quit IRC16:00
*** codenamelxl has joined #openstack-kolla16:02
*** zhubingbing has quit IRC16:05
*** coolsvap has quit IRC16:08
*** diablo_rojo has joined #openstack-kolla16:11
*** athomas has quit IRC16:12
*** k_mouza_ has joined #openstack-kolla16:14
*** jascott1 has joined #openstack-kolla16:14
*** k_mouza_ has quit IRC16:15
*** k_mouza has quit IRC16:16
*** k_mouza has joined #openstack-kolla16:16
*** jascott1 has quit IRC16:18
diablo_rojoinc0, is it cool to go ahead with the kolla-k8 like today? Any certain time you would prefer?16:20
kolla-slack<inc0> diablo_rojo I'm at opnfv plugfest today, tomorrow tho..16:21
rwellumWhat's the ask diablo_rojo ? Regarding storybook or something else?16:22
Zarahi! I promised folk some kolla questions a while ago! Here's a question!16:29
Zaraso aiui kolla needs pretty specific config in dockerfiles and ansible roles to deploy things.16:29
ZaraAnd also aiui it's not really possible to use kolla to build and deploy just one container in isolation16:29
Zarafor development purposes, various openstack services have to come along for the ride.16:29
ZaraI'm wondering if anyone knows of any tools to reuse dockerfiles used with kolla for local development.16:29
ZaraSince I think it's possible to use the same ansible roles (just have the variables expand to different16:29
Zaravalues) but I'm not sure what's out there for parsing the jinja-templated dockerfiles that kolla16:29
Zaraexpects.16:29
ZaraI'm not sure if this makes sense, I'm still working out the problem atm and I'm pretty new to16:29
Zaraeverything. :)16:29
*** dardelean has joined #openstack-kolla16:30
*** dardelean has quit IRC16:34
*** lukepatrick has joined #openstack-kolla16:39
diablo_rojorwellum, yeah just asking if its cool to do the migration today, but sounds like we are waiting till tomorrow.16:41
kfox1111morning.16:47
*** jascott1 has joined #openstack-kolla16:51
*** jbrooks has joined #openstack-kolla16:57
rwellumdiablo_rojo: maybe I mis-estimated the impact, I saw it as more as an enablement so I can create a storyboard - than a full migration.16:58
*** rmart04 has joined #openstack-kolla16:59
*** rmart04 has quit IRC17:00
rwellumZara: You can build a specific image in kolla np. Deploying it to an existing cluster is harder. But there's a new dev mode in kolla-ansible that might work for you.17:00
*** janki has quit IRC17:02
Zarahm, okay. if it helps, the workflow that I want is 'local dev has some dockerfiles and an ansible role to build an image and deploy a container based on it. those can then be slotted into the rest of the infra for building and deploying all desired services with kolla.' atm I'm using docker-compose locally to build and deploy things and then pretty much copying the instructions with modifications for17:06
Zara kolla (config etc) so that kolla can use them.17:06
Zara(btw, it might help users in the future if the history of the kolla project were noted somewhere; it took me a while to realise that 'kolla-ansible' used to be within 'kolla' and is now a separate project, and 'kolla' just refers to the docker part of kolla (except for things like 'openstack-kolla' which refers to both kolla and kolla-ansible).)17:13
Zara(orrr if I've gotten that wrong then maybe that proves it! :D)17:15
*** mdnadeem has quit IRC17:20
*** pcaruana has joined #openstack-kolla17:22
*** lpetrut has quit IRC17:23
inc0whats aiui?:)17:26
inc0Zara:17:26
*** pcaruana has quit IRC17:26
Zaraoh sorry, 'as I understand it'17:27
inc0ah17:27
ZaraI'm pretty new to everything so it's very possible I've missed things17:27
*** pcaruana has joined #openstack-kolla17:27
Zaraso I'm qualifying every single statement with that, haha17:27
*** ntpttr_laptop has joined #openstack-kolla17:28
inc0I'm not sure if I understand your question, do you want to build single container? and then deploy single container?17:28
Zarayeah, with the idea that that we could then reuse the build and deploy instructions in a setup where we had lots of containers, including those for openstack services17:29
inc0well just building is fine17:30
*** b_bezak has quit IRC17:30
inc0you can really build however and whatever you want, even single img17:30
*** gkadam has quit IRC17:30
codenamelxlHi everyone, it looks like that Kolla with DVR on fails on Floating IP located on network node17:30
codenamelxlFIP on compute node is OK17:30
inc0that's how DVR work tho17:31
inc0fip lands on cpu17:31
inc0(afair, not expert on dvr)17:31
codenamelxlnot with centralized router and VIP like Octavia. For those use case, the FIP will be located on network node17:31
codenamelxlFIP on network node can't be reach17:32
inc0hmm...interesting, can you confirm that you have running l3 agent on network node?17:32
codenamelxlYes, it's running17:33
codenamelxli can ping the FIP from SNAT/centralized router namespace17:34
inc0hmm...I think our inventory is tailored for dvr on cpu17:34
inc0so where is it terminated?17:34
codenamelxlsorry, can you be clearer?17:35
inc0well if you can ping fip from router, it has to be somewhere right?17:35
codenamelxlit reach the VM/VIP correctly17:35
inc0ahh but not octavia?17:36
codenamelxlNo, it's floating IP on network node in general17:36
inc0correct me if I'm wrong, but octavia lb lives inside vm?17:36
codenamelxlyes17:37
*** affie_ has quit IRC17:37
kfox1111octavia is weird though.17:37
kfox1111amphora live as vm's in the cloud,17:37
codenamelxlbut the VIP FIP is on network node17:37
kfox1111but use keepalived for service if  Iremember correctly.17:37
kfox1111but they have fip that binds to the vip.17:37
kfox1111so the fip lives on the network node, so it is fixed, while the vip moves around17:37
*** openstackgerrit has joined #openstack-kolla17:38
openstackgerritMerged openstack/kolla stable/pike: Move a release note generated in wrong place  https://review.openstack.org/50910317:38
kfox1111(not how I would have chosen to implement it. :/ )17:38
inc0right, that seems to be conflicting with what DVRs are supposed to be right?17:38
kfox1111kind of, yeah.17:38
kfox1111it use to be flat out broken when dvr was enabled,17:39
kfox1111 so a step up at least. :/17:39
Zarainc0: thanks, I need to go back over the problem because I think I'm not expressing it clearly (or possibly just confused haha) :)17:39
codenamelxlhttps://bugs.launchpad.net/neutron/+bug/158369417:39
openstackLaunchpad bug 1583694 in neutron "[RFE] DVR support for Allowed_address_pair port that are bound to multiple ACTIVE VM ports" [Wishlist,Fix released] - Assigned to Swaminathan Vasudevan (swaminathan-vasudevan)17:39
kfox1111I kind of gave up on octavia. :(17:39
*** signed8bit is now known as signed8bit_Zzz17:39
kfox1111yeah. that issue.17:39
codenamelxlAnd this fixed in neutron make the FIP binded to VIP stay on the network node17:39
kfox1111I waited a long time for them to fix it.17:39
kfox1111yeah. that should fix it. just I dont know of too many people who have tested that.17:40
kfox1111does regular network node traffic work?17:41
kfox1111ie, vm without a fip accessing data from outside the cloud?17:41
codenamelxlyes, inside to outside is OK, the other way is not17:43
codenamelxloh wait, let me check that again17:45
*** sambetts is now known as sambetts|afk17:45
rwelluminc0: I think Zara is talking more about a dev environment?17:46
inc0well, we do it differently tho17:46
inc0also it's still wip17:46
*** k_mouza_ has joined #openstack-kolla17:48
*** jmccarthy has joined #openstack-kolla17:51
codenamelxlkfox1111 let me check that in 15mins17:51
*** k_mouza has quit IRC17:52
*** k_mouza_ has quit IRC17:53
Zaraokay I think my issue was that anything built with kolla had to be deployed with kolla, but there was no way to deploy something in isolation via kolla without other openstack services being configured. so we can't use the results of kolla build to develop locally.17:57
keper7hey all, so i've given up on kolla-k8s for now since it's not prod ready and gone with kolla-ansible. I'm seeing the cinder API container repeatedly restarting with error "Permission denied: AH00058: Error retrieving pid file /run/httpd/httpd.pid" though. Looks like its this problem https://bugs.launchpad.net/kolla-ansible/+bug/1728916. Does anyone know a workaround?17:57
openstackLaunchpad bug 1728916 in kolla-ansible "centos-source-fluentd:4.0.0 and cinder api keep restarting" [Undecided,Confirmed]17:57
*** pcaruana has quit IRC17:58
*** itlinux has joined #openstack-kolla17:58
keper7I see the comment saying "please use the same version of kolla and kolla-ansible", but I just used the quickstart instructions for those so I'm not sure what to do differently18:01
*** diablo_rojo has quit IRC18:02
inc0keper7: that's strange, never seen this issue before (httpd.pid perms)18:03
inc0did you build your containers?18:03
*** codenamelxl has quit IRC18:03
*** signed8bit_Zzz is now known as signed8bit18:03
*** mgoddard has quit IRC18:04
*** gfidente is now known as gfidente|afk18:06
*** ntpttr_laptop has quit IRC18:07
jmccarthyIf a neutron plugin has no limits applied on an item in it's requirements.txt (i.e. https://github.com/openstack/networking-vpp/blob/master/requirements.txt) and it's installed with: pip --no-cache-dir install --upgrade -c requirements/upper-constraints.txt /plugins/*;  it can override the upper-constraint ? (oslo.serialization)18:07
keper7no I pulled them. I do have some that I built but I didn't use them this time18:07
*** codenamelxl has joined #openstack-kolla18:07
*** jbrooks has quit IRC18:07
*** dardelean has joined #openstack-kolla18:08
inc0ok, if you pulled 4.0.0 they're *very* old18:08
dardeleanhi18:08
keper7I have `openstack_release: "4.0.0"`, and the rest of the container/build settings left as default in my globals18:08
keper7oh18:08
inc0fortunately we have fresh and more supported ones18:08
keper7where? the newest i see is on docker hub. I can use the ones i just put up on jfrog but it was my first time trying that so i dont know if they're good or not18:08
inc0if you want pike containers just type "pike" in openstack_releease18:08
keper7oh neat. Thanks, I'll wipe the nodes and try again with that18:09
inc0caveat, only source builds are there18:09
codenamelxlI just checked, VM can reach other VM through their IP, can't reach public IP18:09
inc0so centos-source or ubuntu-source18:09
keper7openstack_release: "pike", kolla_install_type: "source", kolla_base_distro: "ubuntu"    should work?18:09
inc0https://hub.docker.com/r/kolla/ubuntu-source-glance-api/tags/18:10
inc0yes18:10
inc0do kolla-ansible destroy --include-images before18:10
inc0just to be sure18:10
dardeleanif I want to use TLS, is it mandatory to provide fqdn?18:10
inc0yes dardelean18:11
keper7oh neat i can do that instead of re-deploying my base-os? (ubuntu)18:11
dardeleanI see18:11
inc0keper7: yes18:11
keper7great!18:11
dardeleanplanning to write a blog post on deploying kolla in HA, I know I had some issues with that18:12
inc0well HA is default18:12
inc0let me know what issues you have18:13
keper7dardelean: what didnt work as HA?18:13
inc0codenamelxl: seems like router in general is fleaky, check neutron logs (/var/lib/docker/volumes/kolla_logs/_data/neutron)18:14
dardeleanit's not that it didn't work, just some stuff that we're not documente, e.g. was expecting glance to be present on al the control nodes, turned out that if I use file backend it does not shace18:14
dardeleanwhich is normal, it just wasn't obvious for me at the time18:14
inc0right, if you use file backend to glance you'll have issues18:14
dardeleanalso I had some issues with TLS at that time on stable/pike, did not manage to solve them at the time18:15
codenamelxlinc0: OK, i'm tailing log on l3 agent of network node18:15
dardeleananyways, just want to create some traction for kolla with an HA article, ppl tent to like that18:17
inc0dardelean: yup, I love that too:)18:17
Zara(so if kolla-build isn't an option to make an image that can be deployed locally without configuring other openstack services, and we wanted to reuse the dockerfiles we were using with kolla-build, we'd need another tool that could parse jinja-templated dockerfiles and build them.)18:18
inc0kolla18:18
inc0kolla-ansible doesn't make images18:18
inc0kolla itself make images and they totally can and are deployed outside kolla-ansible18:18
*** jmccarthy has left #openstack-kolla18:19
inc0kolla-build is tool that does that:)18:19
Zarahm, perhaps that's the step I was missing, as far as I was aware things built by kolla-build could only be deployed with kolla-ansible18:19
keper7inc0: `kolla-ansible destroy --include-images -i multinode --yes-i-really-really-mean-it` failed. Several pages of red...  Lots of "reference does not exist".    It did remove all images and containers from my nodes though, so I think thats ok right?18:19
inc0should be fine18:20
keper7thanks18:20
inc0Zara: right, they're just images, but default CMD is kinda specific with kolla18:20
inc0you can either override default CMD or make use of it (it is useful;))18:21
codenamelxlinc0: OK so what do i expect from the log :) I'm kinda new on this18:21
inc0not sure...we need to debug what's wrong:)18:21
inc0Zara: this file is a key https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/cinder/templates/cinder-api.json.j218:22
codenamelxlhold on, let's me switch my IRC client18:22
inc0I don't have much time now, but I'll try to write some docs to use it18:22
Zaranp, my memory's hazy and I'm ill anyway so planning to vanish shortly :) I just remember a while back I tried `kolla-build foo`, then effectively `docker run foo` and it said the container didn't have some config and it failed to start.18:23
inc0right, you'll need few ENVs18:24
inc0but you can totally do it assuming you provide them18:24
Zaraokay, hm, is there a list anywhere?18:24
*** lpetrut has joined #openstack-kolla18:25
inc0so if you override CMD, you just do what you specify18:25
inc0so if you run docker run -v my-nova-conf:/etc/nova nova-api nova-api18:25
inc0that should be fine18:25
*** codenamelxl_ has joined #openstack-kolla18:26
codenamelxl_INFO neutron.agent.common.ovs_lib [req-3ddb49e3-38da-423f-8cb7-f7e726ae1941 - - - - -] Port 372ac4ed-6fcc-4224-bbe8-be30768dbe38 not present in bridge br-int18:27
Zarathis is already very useful, thanks :) are there any examples out there already anywhere?18:27
codenamelxl_ok, this is the most suspicous line18:27
codenamelxl_that port does not show up when i do a port list18:27
*** codenamelxl has quit IRC18:27
inc0Zara: well you can definetly reverse engineer tripleo (they're using just containers) but well18:28
inc0reverse engineering tripleo is not easierst thing18:28
Zarahaha18:28
ZaraI'll probably try to set something up tomorrow since it'll be easier to show where I'm getting stuck with the env vars when I can point to actual error messages.18:32
Zarathanks for all your help so far :)18:33
rwelluminc0: we have pike images on dockerhub now. :)18:35
rwelluminc0: Somehow missed this18:36
inc0yeah just got merged18:41
inc0Zara: tomorrow I'll also have more time (I'm on conference now) so I can walk you through it18:41
rwelluminc0: +1 :)18:41
rwellumZara: I deploy kolla images outside of kolla-ansible fwiw18:41
dardeleanrwellum where are the pike images? cannot see the tag on the images on kollas dockerhub18:42
inc0dardelean: they're ther18:42
rwellumdardelean: https://hub.docker.com/r/kolla/ubuntu-source-glance-api/tags/18:42
inc0just source images - tag is pike18:42
dardeleanhmm, interesting18:43
*** alexasap has joined #openstack-kolla18:43
dardeleanubuntu source collectd does not have pike18:43
*** mgoddard has joined #openstack-kolla18:43
*** alexasap has left #openstack-kolla18:43
dardeleanhttps://hub.docker.com/r/kolla/ubuntu-source-collectd/tags/18:43
*** alexasap has joined #openstack-kolla18:43
alexasapHello18:43
alexasapplease help asap18:43
alexasaphttps://gist.github.com/anonymous/4f42bf260e2506c9645f531b5063e0fb18:43
alexasapinstalled openstack kolla with ODL18:44
alexasapno traffic for DHCP interface and ODL carbon crashing18:44
codenamelxl_i'm confused as where to report this FIP problem to kolla or neutron  now since i can confirm that it's easy to reproduce18:45
alexasapsorry?18:47
alexasapany idea how to fix that (at least I want to just boot VMs and get DHCP address)18:48
rwellumdardelean: could be because it's not pushed yet? inc0 said it was just merged.18:49
inc0no it's there18:49
alexasapit doesnt work for the private virtual and physical networks too18:49
alexasapie. ODL in Kolla is broken18:50
inc0but I don't think collectd is in profiles we psh18:50
dardeleani see, indeed for keystone there are new images18:50
*** kbaegis has joined #openstack-kolla18:52
kbaegisHi all. troubleshooting a config issue with kolla-ansible18:52
inc0alexasap: not sure what to make out of these logs, I'm not expert at odl18:53
kbaegisspecifically /etc/kolla/config/{cinder,glance,nova} with ceph external18:54
kbaegisAre there requirements for the permissions? I need to use the deploy user or the files aren't detected/deployed?18:54
kbaegisspecifically, do I need to chown -R root:root or kolla:kolla for /etc/kolla/config/?18:55
*** codenamelxl_ has quit IRC18:55
kbaegisAnd are there any requirements for rwx?18:55
*** david-lyle has joined #openstack-kolla18:55
*** shardy has quit IRC18:58
alexasap2017-12-05 18:58:31,523 | ERROR | pool-44-thread-1 | OutboundQueueProviderImpl        | 314 - org.opendaylight.openflowplugin.impl - 0.4.1.Carbon | No queue present, failing request 2017-12-05 18:58:31,523 | WARN  | pool-44-thread-1 | RpcContextImpl                   | 314 - org.opendaylight.openflowplugin.impl - 0.4.1.Carbon | Xid cannot be reserved for new RequestContext, node:openflow:5324471993737118:58
alexasapany idea?18:58
inc0you might want to check with ODL people about that19:00
*** david-lyle has quit IRC19:01
dardeleanhmm, i can see that "tunnel_interface" can be used for guest traffic19:06
dardeleanbut what if I want it on non-tunnel network19:07
dardeleanfor example my neutron-agents on the compute nodes don't support vxlan19:07
dardeleantill now I created a br-data by hand19:07
dardeleanis there a better way in your opinion?19:08
inc0well depends on what kind of neutron net you want19:09
inc0if it's vlan, you'll have to do some fiddling19:09
inc0not much, but some19:09
kbaegisI'm seeing "VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster."19:10
*** TxGirlGeek has joined #openstack-kolla19:10
kbaegisin cinder_volume logs19:10
dardeleanvlan or flat19:10
*** janonymous has quit IRC19:10
inc0kbaegis: can you connect to ceph cluster?:)19:10
inc0dardelean: flat should work if your br-ex on cpu nodes is connected19:10
inc0and you spawn vm in it19:11
kbaegisSure. Ceph seems solid. osdmap e661: 24 osds: 24 up, 24 in19:11
dardeleanthis is what I do https://paste.ubuntu.com/26120370/19:11
inc0vlan - you'll need to do conf override for ml2_conf.ini where you'll specify physnet, vlan ranges and whatnot19:11
inc0like you would normally19:11
dardeleaneth2 being my dedicated guest network19:11
dardeleanworks19:12
inc0kbaegis: hmm...even from node that runs cinder volume?19:12
dardeleanbut I am thinking if I can do it better19:12
inc0then if you just specify neutron_external_interface: "eth2"19:12
inc0create flat network19:12
inc0and then spawn vm in it19:12
inc0that should work19:12
dardeleanI don't get this, why neutron external interface? that (br-ex) is only on the control nodes19:13
dardeleani use tenant networks with dedicated guest network19:14
*** TxGirlGeek has quit IRC19:14
kbaegisinc0: Yeah. Ceph is healthy. It's the config/notes on ceph-external that appear to have some gaps maybe?19:15
inc0well, only on network nodes technically19:15
kbaegisinc0: I'm attempting to get ceph to connect from the container directly19:15
*** TxGirlGeek has joined #openstack-kolla19:16
dardeleancontrol = network nodes in my case19:16
kbaegisI'm guessing I misconfigured something19:16
inc0should be fine, we use net=host in kolla-ansible everywhere19:16
inc0dardelean: well technically network nodes are ones to connect to external network19:16
inc0vxlans provides horizontal traffit19:17
inc0I think you should be able to spawn vm in flat network (haven't tried that myself)19:17
dardeleanyeah, east-west traffic19:18
dardeleanwhile br-ex is north-south19:18
kbaegisinc0: So /etc/kolla/config/cinder/ contains ceph.client.cinder.keyring ceph.conf cinder-volume.conf & cinder-backup.conf19:18
inc0yeah19:18
dardeleanit would be nice if I could create a east-west network without vxlan19:18
dardeleanbasically creating br-data19:18
kbaegisI see none of those in the cinder_volume container19:19
dardeleanon all nodes, cpu and controllers19:19
dardeleanbr-tun is create afaik if tunnel_traffic interfaceis used19:19
kbaegisinc0: /etc/kolla/cinder-volume/ contains only ceph cinder and config.json19:20
kbaegisno keys19:20
inc0kbaegis: interesting, it should be there in /etc/ceph19:20
inc0ahh19:20
kbaegisonly ceph.conf and rbdmap19:20
kbaegisNo keys in the container19:20
inc0well19:21
inc0so keyring hasn't been copied to /etc/ceph in container?19:21
kbaegiscorrect. Nor is it in the appropriate folder for kolla-ansible19:21
kbaegisfor the deploy node19:22
kbaegisIf I copy it manually will it work? cp /etc/kolla/config/cinder/ceph.*.keyring /etc/kolla/cinder-volume/19:22
inc0check if it's present in /var/lib/kolla/config_files inside container19:23
kbaegisonly 3 exist19:23
kbaegisno keyrings19:23
inc0try to copy it manually and restart cinder-volume container19:23
inc0yeah nothing would copy ceph confs there, seems like a bug19:25
kbaegisdocker cp /etc/kolla/config/cinder/ceph.client.cinder.keyring cinder_volume:/etc/ceph/ ->>Driver post RPC initialization completed successfully19:25
*** jbrooks has joined #openstack-kolla19:25
kbaegisWell I'm glad it wasn't just me :)19:26
*** alexasap has quit IRC19:29
kbaegisHmm19:29
kbaegisWell will inserting the file into /etc/kolla/cinder-volumes at least get it under /var/lib/kolla/config_files?19:30
kbaegisinc0?19:30
inc0yes19:31
kbaegisAwesome. Makes a workaround easier until that's fixed19:31
inc0however that has to be done on every cinder-volume ndoes19:31
kbaegisI can't just do it on the deployment node and then do a reconfigure?19:31
inc0no:(19:32
kbaegis*sigh19:32
kbaegisinc0: Thanks19:32
kbaegisinc0: Nice to have a sanity check on this stuff :)19:32
inc0but if you copy it to all nodes it should be safe to reconfigure or whatnot19:32
*** lukepatrick has quit IRC19:33
kbaegisLet's see- the other component that really needs this is nova-compute19:36
kbaegisso cinder_volume and nova_compute, or is it nova_libvirt?19:36
inc0copy to both to be safe;)19:37
inc0btw ansible -m copy -i your-inv -a "src=your-keyring-file dest=/etc/kolla/nova-compute/" storage compute19:38
inc0ansible ad-hoc commands are dope19:38
kbaegisnice, tyvm19:40
kbaegisbetter than my: for i in $(cat /etc/hosts|grep bumblebee|cut -d\  -f2);do scp /etc/kolla/config/nova/*.keyring $i:/etc/ceph/;done19:40
inc0yeah I strongly recommend getting familiar with this, I use ansible -m shell -a "whatever oneliner" all routnely19:41
inc0-m script -a "src=/path/to/some/shell.sh" is also mighty useful19:42
kbaegisERROR! Extraneous options or arguments19:44
kbaegisansible -m copy -i ~/bb -a "src=/etc/kolla/config/nova/ceph.client.cinder.keyring dest=/etc/kolla/nova-compute/" storage compute19:44
kbaegisstorage and compute need a flag presumably? Can't seem to find it19:45
kbaegisOkay, got all the files squared away19:49
kbaegisNow kolla-ansible reconfigure will work?19:49
*** lukepatrick has joined #openstack-kolla19:51
*** jbrooks has quit IRC19:57
*** jbrooks has joined #openstack-kolla19:58
*** wojdec1 has joined #openstack-kolla20:04
*** gkadam has joined #openstack-kolla20:05
*** jrist has quit IRC20:05
*** lpetrut has quit IRC20:11
*** mgoddard has quit IRC20:13
kbaegisWell you're right about ansible. I should invest20:15
kbaegisinc0: But here's my hideous yet functional one-liner- or i in $(cat tmp);do ssh $i docker exec -it -u root glance_api bash -c 'cp /var/lib/kolla/config_files/*.keyring /etc/ceph/'&& docker restart glance_api;docker exec -it -u root cinder_volume bash -c 'cp /var/lib/kolla/config_files/*.keyring /etc/ceph/'&& docker restart cinder_volume;docker exec -it -u root nova_libvirt bash -c 'cp /var/lib/kolla/config_files/*.keyring /etc/c20:15
kbaegiseph/'&& docker restart nova_libvirt;docker exec -it -u root nova_compute bash -c 'cp /var/lib/kolla/config_files/*.keyring /etc/ceph/'&& docker restart nova_compute;do20:15
kbaegis*f20:16
*** ktibi has quit IRC20:17
*** yangyapeng has joined #openstack-kolla20:20
*** yangyapeng has quit IRC20:24
*** b_bezak has joined #openstack-kolla20:31
*** wojdec has joined #openstack-kolla20:31
*** rwsu has quit IRC20:31
*** k_mouza has joined #openstack-kolla20:31
*** rwsu has joined #openstack-kolla20:31
*** wojdec1 has quit IRC20:33
*** jbrooks has quit IRC20:34
*** wojdec has quit IRC20:35
*** k_mouza has quit IRC20:35
*** b_bezak has quit IRC20:35
*** wojdec has joined #openstack-kolla20:36
*** jbrooks has joined #openstack-kolla20:36
wojdecHi Folks… I'm trying to locally build a customized container using the kolla tools/build.py with packages installed from an external repo. The repo's yum settings  are "COPY" 'ed  in the dockerfile for the container. On running the build however I get an error saying that the yum repo couldn't be resolved, but resolution works fine from the host running the build. Any suggestions on how to enable external repos, without a http20:39
wojdec accessibe repo file?20:39
*** k_mouza has joined #openstack-kolla20:43
*** unicell has joined #openstack-kolla20:48
*** k_mouza has quit IRC20:54
*** kelv_ has joined #openstack-kolla20:55
kbaegiswojdec: I'd check your docker config21:09
kbaegislikely a dns config issue (--dns=…)21:09
*** Pavo has joined #openstack-kolla21:10
inc0also try creating fake dockerfile with just COPY your repo configs and do docker run -it << my fake image>> bash21:11
inc0and see if it works21:11
*** kbaegis has quit IRC21:14
*** k_mouza has joined #openstack-kolla21:15
*** jbrooks has quit IRC21:22
wojdeckbaegis: I'm using the same pattern as the other dockerfiles.j2 files… not passing DNS anywhere, nor is it mentioned in the kolla docs as needed… Is it?21:27
*** gfidente|afk has quit IRC21:29
inc0wojdec: docker engine might mess with that, that's why I suggested just running clean centos7 and see if that works manually21:30
wojdecok. will try that next21:31
*** dardelea_ has joined #openstack-kolla21:31
inc0that will help figuring out if problem is kolla or docker itself21:32
*** dardelea_ has quit IRC21:35
*** k_mouza has quit IRC21:36
*** mchlumsky has quit IRC21:37
*** gkadam has quit IRC21:38
*** itlinux has quit IRC21:39
*** itlinux has joined #openstack-kolla21:44
*** k_mouza has joined #openstack-kolla21:45
*** jrist has joined #openstack-kolla21:46
*** lrensing has joined #openstack-kolla21:50
*** k_mouza has quit IRC21:56
*** kbaegis has joined #openstack-kolla21:58
*** rhallisey has quit IRC21:59
*** zshi has quit IRC22:03
*** threestrands has joined #openstack-kolla22:05
*** threestrands has quit IRC22:05
*** threestrands has joined #openstack-kolla22:05
wojdecseems like it's a docker issue, or more specifically how docker hooks up into my host that's connected to a VPN...22:07
wojdecexternal repos are reachable. internal ones not22:07
*** jtriley has quit IRC22:08
wojdecinc0: solved it by flipping the source of the package to be off our external repo than the intranet one22:08
*** dave-mcc_ is now known as dave-mccowan22:15
rwelluminc0: these pike images have the tag 'pike'? Not 5.0.0 etc?22:18
*** kbaegis has quit IRC22:19
*** kbaegis has joined #openstack-kolla22:19
*** wojdec has quit IRC22:31
*** jascott1 has quit IRC22:31
*** jascott1 has joined #openstack-kolla22:32
dardeleanrwellum did you do a presentation @summit?22:32
rwellumdardelean: yes - combined with vhosakot22:33
dardeleancan you provide the video? :D22:36
*** jascott1 has quit IRC22:36
dardeleansearched for it some time ago but could not find it22:36
rwellumdardelean: https://www.youtube.com/watch?v=14WjpNC7HKI22:41
dardeleangreat, thanks rwellum :D22:42
rwellumIt all went smoothly, except I was using his laptop, and my demo was pre-recorded, and it failed to run on his laptop. Had to log in, 2-factor authentication etc.22:42
dardeleanyou should do a blog post on this stuff22:42
dardeleani mean the kolla-k8s script you have22:42
dardeleanit would be easier for ppl to find and use it22:43
rwellumdardelean: I do have slides up: https://docs.google.com/presentation/d/1Qlqgqwbbjd-eDUCeOmj2Ugpbbke7W99qxc7KPp_BxnE/edit?usp=sharing22:43
rwellumThis is what I presented22:44
dardeleani see, but still22:44
dardeleanif you don't do it, I may do it :))22:44
rwellumCould do thing is it's just another deployer basically. I'd rather the focus was on Justin's ansible playbook.22:45
dardeleanwhat is that?22:45
rwellumko.py was/is great for me to play with the technology - but not really the way to go in this data-driven environment. :)22:45
dardeleanoh, ofc22:45
dardeleanbut a nice place to start22:45
rwellumYeah I hope so. It's gotten a lot of people unstuck and using it for the first time.22:46
rwellumhttps://review.openstack.org/#/c/520847/22:46
rwellumdardelean: ^^ that's his playbook22:47
dardeleanhmm, what does this do?22:47
dardeleanI mean, it's something like your script?22:47
rwellumMultinode kolla-kubernetes deployment playbook with ceph supported. AFAIK.22:47
dardeleanah22:47
rwellumYeah but done with ansible.22:47
rwellumIt's currently ocata only - my tool does pike22:48
rwellumas well as ocata22:48
*** kbaegis1 has joined #openstack-kolla22:51
*** kbaegis has quit IRC22:52
rwelluminc0: not sure if you're still around, but definitely I don't see all the images for pike. For example compare: https://hub.docker.com/r/rwellum/centos-source-kubernetes-entrypoint/tags/ with https://hub.docker.com/r/kolla/centos-source-kubernetes-entrypoint/tags/22:52
*** signed8b_ has joined #openstack-kolla22:54
*** affie_ has joined #openstack-kolla22:56
rwellumdardelean: sorry I missed your presentation btw - meant to attend and say hello.22:57
*** signed8bit has quit IRC22:58
*** kbaegis1 has quit IRC22:59
*** kbaegis has joined #openstack-kolla23:00
*** kbaegis has quit IRC23:03
dardeleanrwellum my presentation? I was not @summit23:05
*** kbaegis has joined #openstack-kolla23:05
rwellumdardelean: bad with names - sorry!23:06
dardelean:D23:06
rwellumOverall dardelean the summit was great, but it was a long way to travel! And very much driven by companies - not as useful as the PTG's imo.23:10
dardeleanrwellum, I get you23:11
*** lrensing has quit IRC23:16
kbaegisinc0: Even with the hack/workaround for ceph-external, I can't create volumes with cinder-volume23:17
kbaegisinc0: Might have been a keyring issue as I get an issue with rbd, but not with ceph —user cinder —cluster ceph lspools23:17
kbaegisrbd returns an Operation not permitted or something23:18
*** goldyfruit has quit IRC23:21
*** dave-mccowan has quit IRC23:23
*** kbaegis has quit IRC23:26
*** kbaegis has joined #openstack-kolla23:26
*** kbaegis has quit IRC23:26
*** signed8b_ is now known as signed8bit_Zzz23:27
*** genek has quit IRC23:34
*** Pavo has quit IRC23:35
*** lrensing has joined #openstack-kolla23:40
*** TxGirlGeek has quit IRC23:43
inc0rwellum: yeah we limited images we build, probably can remove this limit23:45
rwelluminc0: +1 then I can test pike, well outside my own images that is.23:47
*** lukepatrick has quit IRC23:47
*** Pavo has joined #openstack-kolla23:48
*** Pavo has quit IRC23:49
*** Pavo has joined #openstack-kolla23:50
*** yangyapeng has joined #openstack-kolla23:50
*** dardelean has quit IRC23:53
*** yangyapeng has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!