Wednesday, 2017-05-03

*** ssurana has joined #openstack-kolla00:01
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Fix Multi-regions nova support boot from volume  https://review.openstack.org/46193100:02
MarginHuhi,Jeffrey Zhang00:02
MarginHuI met an issue https://bugs.launchpad.net/kolla/+bug/168781900:03
openstackLaunchpad bug 1687819 in kolla "failed to get ip from ironic_dnsmasq container" [Undecided,New]00:03
MarginHucan you help me ?00:03
*** harlowja has quit IRC00:04
*** Pavo has joined #openstack-kolla00:06
*** dave-mccowan has quit IRC00:13
*** rwallner has joined #openstack-kolla00:18
*** sbezverk_ has joined #openstack-kolla00:19
openstackgerritjunbo proposed openstack/kolla-ansible master: Fix saraha-engine authencation issue  https://review.openstack.org/46181400:19
openstackgerritjunbo proposed openstack/kolla-ansible master: Fix saraha-engine authencation issue  https://review.openstack.org/46181400:21
*** sbezverk has quit IRC00:22
*** sbezverk has joined #openstack-kolla00:23
*** rwallner has quit IRC00:23
openstackgerritjunbo proposed openstack/kolla-ansible master: mount /run in sahara-engine container  https://review.openstack.org/46178400:25
*** sbezverk_ has quit IRC00:26
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Implement Cinder, Nova Placement and mistral logs for kibana  https://review.openstack.org/46193700:26
*** schwicht has joined #openstack-kolla00:29
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Fix logrotate issue on Octavia and Watcher service  https://review.openstack.org/44845200:32
*** Pavo has quit IRC00:32
*** duonghq has joined #openstack-kolla00:35
*** goldyfruit has joined #openstack-kolla00:36
*** sbezverk_ has joined #openstack-kolla00:36
duonghqsup sbezverk sbezverk_00:36
*** fooliouno has joined #openstack-kolla00:36
sbezverk_duonghq: hey how is it going?00:37
*** sbezverk has quit IRC00:37
duonghqsbezverk_, just back from short national holiday00:38
duonghqhow about you?00:38
openstackgerritMerged openstack/kolla-kubernetes master: Fix for new canal.yaml location  https://review.openstack.org/46152700:38
sbezverk_duonghq: busy, last preparations for the summit, fixing the gate, usual stuff :)00:38
fooliounosbezverk_: quick question. In a multi-node config, what should the tunnel_interface (in cloud.yaml) be set to? In a single node its "docker0", but should it be the management interface in a multi-node?00:39
*** tovin07_ has joined #openstack-kolla00:39
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: Do Not Merge, kolla-kubernetes gate tests  https://review.openstack.org/46106300:40
duonghqsbezverk_, look forward to meet you in-person :)00:40
sbezverk_fooliouno: yep, exactly, in multinode it must be your management interface00:40
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: Add libosinfo package for source installation  https://review.openstack.org/46194000:40
sbezverk_duonghq: likewise00:40
duonghqinc0, sbezverk_ will we schedule a dinning out?00:40
sbezverk_duonghq: no idea00:41
fooliounothanks!00:41
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add vmware DVS support to kolla-ansible  https://review.openstack.org/45927000:42
fooliounosbezverk_: on the same note, should the tunnel_interface_address in globals.yaml be changed from 0.0.0.0 to the management interface IP address for multinode?00:46
*** dave-mccowan has joined #openstack-kolla00:47
sbezverk_fooliouno: do not think so00:47
sbezverk_because you cannot use single IP value for multiple nodes00:47
sbezverk_in the code we pick up ip assigned to tunnel interface and start using it00:48
fooliounook. Has novncproxy been verified for multinode (as far as you are aware)00:48
sbezverk_fooliouno: well, for novncproxy I would suggest to use VIP (keepalived)00:49
sbezverk_in this case you can connect from outside regardless where you instance/vm is running00:50
*** cuongnv has joined #openstack-kolla00:51
sbezverk_at least that is how I use it and it works well for me00:51
fooliounothanks sbezverk_ . helpful as always. I will look into keepalived.00:51
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: Add missing cleanup step for heat images  https://review.openstack.org/46194200:52
*** ssurana has quit IRC01:03
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: Add pymongo in panko images  https://review.openstack.org/45656901:06
*** iniazi has quit IRC01:08
*** xinliang has quit IRC01:13
*** xinliang has joined #openstack-kolla01:16
*** eaguilar has joined #openstack-kolla01:22
*** MarginHu has quit IRC01:23
*** MarginHu has joined #openstack-kolla01:24
*** inesa-chao has joined #openstack-kolla01:26
*** unicell has quit IRC01:30
*** ducttape_ has joined #openstack-kolla01:31
*** hrw has quit IRC01:31
*** hrw has joined #openstack-kolla01:33
openstackgerritJeffrey Zhang proposed openstack/kolla master: [TEST][MASTER][KOLLA] test build  https://review.openstack.org/43290101:35
*** ducttape_ has quit IRC01:36
*** goldyfruit has quit IRC01:41
openstackgerritzhuangzejian proposed openstack/kolla-ansible master: Fix a typo and complement doc in ceph-guide.rst  https://review.openstack.org/46043101:43
*** eanylin_ has quit IRC01:46
*** zhurong has joined #openstack-kolla01:50
*** iniazi has joined #openstack-kolla02:02
*** MarginHu has quit IRC02:05
*** ducttape_ has joined #openstack-kolla02:07
*** zhurong has quit IRC02:07
openstackgerritJawon Choo proposed openstack/kolla master: Override image's meta info.  https://review.openstack.org/46166202:09
*** eaguilar has quit IRC02:10
*** MarginHu has joined #openstack-kolla02:19
*** zhubingbing_ has joined #openstack-kolla02:22
*** japestinho has joined #openstack-kolla02:24
*** ducttape_ has quit IRC02:31
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Remove device_driver option in lbaas_agent.ini  https://review.openstack.org/46196102:34
*** dave-mccowan has quit IRC02:36
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: [TEST][OCATA][KOLLA] Run Periodic  https://review.openstack.org/43775502:40
*** inesa-chao has quit IRC02:47
*** daidv has joined #openstack-kolla02:47
*** janki has joined #openstack-kolla02:55
*** gianpiet_ has joined #openstack-kolla03:11
*** gianpietro has quit IRC03:11
*** eanylin has joined #openstack-kolla03:12
*** gianpiet_ has quit IRC03:16
*** gianpietro has joined #openstack-kolla03:16
*** gianpietro has quit IRC03:19
*** zhurong has joined #openstack-kolla03:37
zhubingbing_around sdake、?03:37
sdakesup zhubingbing_03:37
*** zhurong has quit IRC03:40
*** MarginHu has quit IRC03:44
*** MarginHu has joined #openstack-kolla03:45
*** masber has quit IRC03:46
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: [TEST][OCATA][KOLLA] Run Periodic  https://review.openstack.org/43775503:49
*** jchhatbar has joined #openstack-kolla03:49
*** janki has quit IRC03:52
*** ssurana has joined #openstack-kolla04:05
*** salv-orlando has joined #openstack-kolla04:18
*** negronjl has quit IRC04:18
*** negronjl has joined #openstack-kolla04:22
*** salv-orlando has quit IRC04:23
japestinhoAnyone know why kolla-ansible get error while fetching ceph keyrings?04:25
japestinhofatal: [compute01]: FAILED! => {"failed": true, "msg": "'delegate_host' is undefined"}04:25
*** emccormick has quit IRC04:28
*** jchhatbar is now known as janki04:30
*** ducttape_ has joined #openstack-kolla04:32
*** krtaylor has joined #openstack-kolla04:33
*** ducttape_ has quit IRC04:36
*** zhurong has joined #openstack-kolla04:37
*** caowei has quit IRC04:47
*** jascott1 has quit IRC04:48
*** salv-orlando has joined #openstack-kolla04:49
*** MarginHu has quit IRC04:52
berendthas anybody tried running a docker daemon with seccomp security profiles and kolla?05:07
berendtinc0: thanks for cancelling the next 2 meetings, i think this makes sense05:07
*** caowei has joined #openstack-kolla05:22
*** unicell has joined #openstack-kolla05:22
*** cuongnv has quit IRC05:30
*** shasha___ has quit IRC05:32
*** shasha___ has joined #openstack-kolla05:33
*** reddydodda has quit IRC05:34
*** shasha___ has quit IRC05:37
*** cuongnv has joined #openstack-kolla05:38
*** salv-orl_ has joined #openstack-kolla05:46
*** salv-orlando has quit IRC05:49
*** tvignaud_ has quit IRC05:50
*** salv-orl_ has quit IRC05:50
*** tvignaud_ has joined #openstack-kolla05:50
*** newmember has joined #openstack-kolla05:52
*** masber has joined #openstack-kolla05:58
*** MarginHu has joined #openstack-kolla06:01
*** MarginHu has quit IRC06:03
*** MarginHu has joined #openstack-kolla06:03
openstackgerritMerged openstack/kolla stable/ocata: Add ssh-keyscan to unblock kolla gate  https://review.openstack.org/46186206:09
*** tovin07 has joined #openstack-kolla06:12
*** MarginHu has quit IRC06:16
*** MarginHu has joined #openstack-kolla06:16
*** jaosorior_away is now known as jaosorior06:19
*** rwsu has quit IRC06:23
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: [TEST][OCATA][KOLLA] Run Periodic  https://review.openstack.org/43775506:25
*** newmember has quit IRC06:26
*** pcaruana has joined #openstack-kolla06:27
*** rwsu has joined #openstack-kolla06:36
*** unicell has quit IRC06:40
*** unicell has joined #openstack-kolla06:40
*** jascott1 has joined #openstack-kolla06:50
*** manheim has joined #openstack-kolla06:54
*** jascott1 has quit IRC06:54
*** manheim has quit IRC06:55
*** manheim has joined #openstack-kolla06:56
*** daidv has quit IRC06:57
*** shardy has joined #openstack-kolla07:00
*** manheim has quit IRC07:01
*** manheim has joined #openstack-kolla07:01
*** unicell has quit IRC07:02
openstackgerritzhuangzejian proposed openstack/kolla-ansible master: Fix a typo and complement doc in ceph-guide.rst  https://review.openstack.org/46043107:15
*** MarginHu has quit IRC07:21
*** MarginHu has joined #openstack-kolla07:21
*** zhurong has quit IRC07:24
*** bjolo has joined #openstack-kolla07:27
*** daidv has joined #openstack-kolla07:27
apuimedoReepicheep: my laptop and desktop are arch07:29
apuimedothe desktop should really be EL07:29
apuimedoso this deployment issues don't happen07:30
bjolomorning07:31
duonghqmorning bjolo07:32
bjolohows it going_07:33
*** nathharp has joined #openstack-kolla07:33
*** salv-orlando has joined #openstack-kolla07:38
*** gfidente|afk is now known as gfidente|doc07:39
duonghqbjolo, preparing for the summit, how about you?07:40
*** zhurong has joined #openstack-kolla07:42
*** salv-orlando has quit IRC07:46
*** egonzalez has joined #openstack-kolla07:47
*** nea1 has quit IRC07:51
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/newton: [DNM] test newton branch  https://review.openstack.org/46202407:52
*** oanson has quit IRC07:52
*** oanson has joined #openstack-kolla07:53
bjolotrying to get our second cloud up (ocata)07:54
bjolobut runing into all sorts of BS :)07:54
bjoloso you are going to the summit .... nice07:54
*** ssurana has quit IRC07:56
*** masber has quit IRC07:56
*** masber has joined #openstack-kolla07:59
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/newton: Add ssh-keyscan to unblock kolla gate  https://review.openstack.org/46202507:59
egonzalezcan someone review (fix ansible gate) https://review.openstack.org/#/c/461368/  (fix horizon ubuntu binary deploy gate) https://review.openstack.org/#/c/460770/08:02
*** mgoddard has joined #openstack-kolla08:04
*** salv-orlando has joined #openstack-kolla08:05
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Apply neutron database migration  https://review.openstack.org/40792208:14
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Apply neutron database migration  https://review.openstack.org/40792208:18
*** pester has joined #openstack-kolla08:30
nathharpMorning all - can I confirm my understanding - with the currently deployment, there is no way to loadbalance API workloads across multiple controllers, instead it is all carried by whichever controller currently has the floating IP?08:33
egonzaleznathharp, there are some issues to not having active/active keepalived. There is not way to ensure a session is always managed by the same HAproxy and balanced to the same backend when sessions are needed (db access)08:41
nathharpegonzalez, ok, figured it was aomething like that, but wanted to make sure08:42
*** ducttape_ has joined #openstack-kolla08:43
egonzalezprobably can run into some network issues with arp, in the case of cisco apic as underlay network provider, need to send garp messages to apic to let him know the IP changed from node and refresh arp tables08:43
*** matrohon has joined #openstack-kolla08:44
egonzalezmaybe there are cases where active/active is worth, but most people uses active/passive to avoid issues08:45
*** zhurong has quit IRC08:47
*** ducttape_ has quit IRC08:47
nathharpegonzalez, I’m still running into issues when heavily loading an environment, and looking for extra resource to manage it08:48
*** jascott1 has joined #openstack-kolla08:51
egonzaleznathharp, is the haproxy in network nodes? or have dedicated nodes?08:53
nathharpegonzalez, simple environment, so 3x controllers/network/storage nodes (storage is external ceph), plus 5 hypervisors08:53
egonzalezmaybe need to move haproxy nodes out of network group if load balancing and network traffic is intense08:55
nathharpegonzalez,08:55
*** jascott1 has quit IRC08:56
nathharpegonzalez, it’s not necessarily that network traffic is intense - still having occasional issues with instance provisioning+DHCP+vif plugging08:56
*** salv-orlando has quit IRC08:56
*** zhurong has joined #openstack-kolla09:01
*** hieulq_ has joined #openstack-kolla09:08
openstackgerritChen proposed openstack/kolla master: Minor fix on syntax and grammar  https://review.openstack.org/46160209:10
*** pbourke has quit IRC09:11
*** hieulq_ has quit IRC09:12
*** hieulq_ has joined #openstack-kolla09:12
*** pbourke has joined #openstack-kolla09:13
*** hieulq_ has quit IRC09:20
openstackgerritMerged openstack/kolla-ansible master: Change ssh/config permissions to 600  https://review.openstack.org/46136809:21
*** britthouser has quit IRC09:23
*** yee379 has joined #openstack-kolla09:23
*** britthouser has joined #openstack-kolla09:23
*** yee37918 has quit IRC09:23
*** cuongnv_ has joined #openstack-kolla09:23
*** daidv_ has joined #openstack-kolla09:23
*** daidv has quit IRC09:23
*** cuongnv has quit IRC09:24
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Apply neutron database migration  https://review.openstack.org/40792209:24
*** Jeffrey4l has quit IRC09:25
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible stable/ocata: Change ssh/config permissions to 600  https://review.openstack.org/46205209:26
*** Jeffrey4l has joined #openstack-kolla09:27
*** Serlex has joined #openstack-kolla09:30
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible stable/ocata: Add ssh-keyscan to unblock kolla-ansible gate  https://review.openstack.org/46205209:30
*** athomas has joined #openstack-kolla09:32
*** jmccarthy has joined #openstack-kolla09:33
*** cuongnv_ is now known as cuongnv09:34
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Apply neutron database migration  https://review.openstack.org/40792209:39
*** hieulq_ has joined #openstack-kolla09:40
*** zhurong has quit IRC09:41
openstackgerritMerged openstack/kolla-ansible master: Update Keystone haproxy config to balance based on source ip  https://review.openstack.org/45135809:42
*** rmart04 has joined #openstack-kolla09:44
*** athomas has quit IRC09:44
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Apply neutron database migration  https://review.openstack.org/40792210:02
*** hieulq_ has quit IRC10:03
*** tovin07_ has quit IRC10:04
*** salv-orlando has joined #openstack-kolla10:06
*** athomas has joined #openstack-kolla10:07
*** zhurong has joined #openstack-kolla10:19
*** oanson has quit IRC10:19
*** oanson has joined #openstack-kolla10:21
*** cuongnv has quit IRC10:23
*** tvignaud_ has quit IRC10:24
openstackgerritMerged openstack/kolla stable/ocata: Fix tempest dockerfile  https://review.openstack.org/46008310:24
*** caowei has quit IRC10:25
*** salv-orlando has quit IRC10:30
*** zhurong has quit IRC10:30
*** shasha___ has joined #openstack-kolla10:33
*** tvignaud_ has joined #openstack-kolla10:36
*** shasha___ has quit IRC10:38
*** hieulq_ has joined #openstack-kolla10:39
openstackgerritMerged openstack/kolla stable/ocata: Fixing missing genisoimage in nova-compute-ironic  https://review.openstack.org/46028710:42
sdakesup bjolo - re bs - is there some technical problem, or red tape?10:49
*** MarginHu has quit IRC10:50
*** Margin has joined #openstack-kolla10:50
*** daidv_ has quit IRC10:51
*** lpetrut has joined #openstack-kolla10:51
*** jascott1 has joined #openstack-kolla10:53
*** hieulq_ has quit IRC10:54
*** Margin has quit IRC10:55
*** Margin has joined #openstack-kolla10:55
*** ssurana has joined #openstack-kolla10:57
*** jascott1 has quit IRC10:57
*** MarginHu2017 has joined #openstack-kolla11:01
*** ssurana has quit IRC11:01
*** Margin has quit IRC11:03
*** rhallisey has joined #openstack-kolla11:14
*** gfidente|doc is now known as gfidente11:23
*** athomas has quit IRC11:30
*** rwallner has joined #openstack-kolla11:42
*** rwallner has quit IRC11:46
*** MarginHu2017 has quit IRC11:47
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Don't configure external bridge on computes if DVR is disabled  https://review.openstack.org/46064511:47
*** rwallner has joined #openstack-kolla11:48
*** rwallner_ has joined #openstack-kolla11:48
*** athomas has joined #openstack-kolla11:49
openstackgerritMartin André proposed openstack/kolla stable/ocata: Add missing cleanup step for new apache images  https://review.openstack.org/46208811:49
*** srwilkers has joined #openstack-kolla11:49
srwilkersmorning11:50
*** rwallner has quit IRC11:52
*** pcaruana has quit IRC11:53
*** yingjun has joined #openstack-kolla11:54
*** goldyfruit has joined #openstack-kolla11:56
*** pcaruana has joined #openstack-kolla11:57
*** rwellum has joined #openstack-kolla11:59
*** schwicht has quit IRC11:59
*** zhurong has joined #openstack-kolla12:01
openstackgerritMerged openstack/kolla-kubernetes master: Add local registry option in deployment-guide  https://review.openstack.org/45999612:03
bjolosdake, na outside of kolla. pfsense cluster missbehaving, docker config settings to handle proxy and local rgy, elrepo kernel-ml not booting on HP G6 machines, etc12:04
*** zhubingbing_ has quit IRC12:09
*** nathharp has quit IRC12:12
*** nathharp has joined #openstack-kolla12:12
rwellumMorning sdake12:12
rwellumMorning sbezverk_12:12
*** eaguilar has joined #openstack-kolla12:14
srwilkerssup rwellum12:16
rwellumMorning srwilkers  - not much trying to apply sbezverk_ fix to canal this morning. How about you?12:20
*** rhallisey has quit IRC12:22
*** rhallisey has joined #openstack-kolla12:22
*** iniazi has quit IRC12:22
*** schwicht has joined #openstack-kolla12:29
*** srwilkers has quit IRC12:35
*** schwicht has quit IRC12:35
*** eaguilar has quit IRC12:37
*** schwicht has joined #openstack-kolla12:39
*** eaguilar has joined #openstack-kolla12:39
*** eaguilar_ has joined #openstack-kolla12:42
*** yingjun has quit IRC12:43
*** eaguilar has quit IRC12:43
*** rwallner_ has quit IRC12:44
*** ducttape_ has joined #openstack-kolla12:44
sdakemoring rwellum12:47
*** srwilkers has joined #openstack-kolla12:49
*** ducttape_ has quit IRC12:49
sdakemorning srwilkers12:50
*** eaguilar_ has quit IRC12:51
srwilkerssdake, hey man12:51
*** eaguilar has joined #openstack-kolla12:52
*** goldyfruit has joined #openstack-kolla12:53
*** schwicht has quit IRC12:53
rwellumHey sdake12:53
*** rwallner has joined #openstack-kolla12:54
*** rwallner has quit IRC12:54
*** rwallner has joined #openstack-kolla12:54
*** skramaja has joined #openstack-kolla12:56
rwellumsdake: when you get a chance, have a qn about: https://review.openstack.org/#/c/460749/312:57
rwellumsbezverk_: when you have a chance, have a qn about: https://review.openstack.org/#/c/461527/30/tests/bin/setup_canal.sh12:58
*** Pavo has joined #openstack-kolla12:58
sdakerwellum ya I already looked at that and responded12:58
rwellumI know - can you look at my latest comment please?12:59
sdakerwellum you mean the linked commit?12:59
rwellumoops - sorry I am behind, looking at your comment now12:59
sdakerwellum cool :)13:00
*** masber has quit IRC13:01
rwellumsdake: you looked at https://review.openstack.org/#/c/461527/30 - do you know what changed in canal? I am downloading the new canal.yaml - but I don't see the issue as reported.13:03
*** iniazi has joined #openstack-kolla13:03
sdakerwellum I haven't tried the new canal nor rectified the deployment documentation as of present13:05
sdakerwellum i can tell you you will need to install both rbac and canal:13:06
sdakeurl="https://raw.githubusercontent.com/projectcalico/canal/master"13:06
sdakeurl="$url/k8s-install/1.6/canal.yaml"13:06
sdakecurl "$url" -o /tmp/canal.yaml13:06
sdakeurl="https://raw.githubusercontent.com/projectcalico/canal/master"13:06
sdakeurl="$url/k8s-install/1.6/rbac.yaml"13:06
sdakecurl "$url" -o /tmp/rbac.yaml13:06
sdakekubectl create -f /tmp/rbac.yaml13:06
*** zhubingbing_ has joined #openstack-kolla13:06
sdakeand do some other magic13:06
sdakerwellum if you figure it out, perhaps you can submit a change to the docs :)13:06
rwellumYeah looking at it now sdake - but not seeing the issue as reported in the code submitted.13:07
sdakewhich issue?13:07
rwellumsdake: But I did miss the rbac piece.13:07
rwellumsdake: "NOTE(sbezverk) Temporary workaround to canal.yaml bug. It is missing net-conf.json file. The code will add it if it does not exist"13:07
rwellumI download it and it's not missing that; and it's conditional in the code. Just not sure why it's conditional. It's either missing or now?13:08
sdakeyup i'm not sure if canal folks have fixed that - sbezverk_ spoke with them a ouple days ago I believe and asked them to fix it and they said they would do so (iirc)13:08
rwellumor not..13:08
rwellumyes ok - will work on rbac and see if that's enough13:08
rwellumNow that I think about it more, the conditional makes sense - sbezverk_ has to make sure the gate works with or without.13:09
*** eanylin has quit IRC13:11
openstackgerritMerged openstack/kolla-ansible master: Update the references link for some project  https://review.openstack.org/45055213:11
pesterhi all, I`m trying AIO install and got: "TASK [haproxy : Checking if kolla_internal_vip_address and kolla_external_vip_address are not pingable from any node] *** failed" - with where ping is succesful13:15
pesterany ideas ?13:15
pesterok, check should return false..)13:17
egonzalezpester, is IP address in kolla_internal_vip_address associated in to server?13:18
pesteryes, my bad, read it wrong and assigned this ip to interface13:19
*** rwallner has quit IRC13:21
pestergot ""msg": "Hostname has to resolve to IP address of api_interface"" with ""stdout": "127.0.1.1       STREAM ubuntu-xenial\n127.0.1.1       DGRAM  \n127.0.1.1       RAW    ""13:23
*** schwicht has joined #openstack-kolla13:23
pesteraddress in /etc/hosts already and pingable13:23
*** eanylin has joined #openstack-kolla13:23
openstackgerritMerged openstack/kolla-ansible master: Magnum: add oslo_messaging_notifications config  https://review.openstack.org/45185713:25
*** eanylin_ has joined #openstack-kolla13:25
*** jtriley has joined #openstack-kolla13:25
*** salv-orlando has joined #openstack-kolla13:25
*** zhurong has quit IRC13:28
*** eanylin has quit IRC13:29
openstackgerritMerged openstack/kolla-ansible master: Fix Heat Kolla demo in contrib  https://review.openstack.org/44235013:29
openstackgerritMerged openstack/kolla-ansible master: Remove kolla_logs_volume unused variable  https://review.openstack.org/45561613:30
openstackgerritMerged openstack/kolla-ansible master: Allow nfs, rpcbind and mountd service on server firewall  https://review.openstack.org/45121013:31
openstackgerritMerged openstack/kolla master: Fix table formatting issue in README.rst  https://review.openstack.org/45995413:32
openstackgerritMerged openstack/kolla-ansible master: Fix Grafana config file template to use variables  https://review.openstack.org/45689713:33
sbezverk_rwellum: canal people proposed a fix but it has not been merged13:33
sbezverk_the workaround I propose will survive even when their fix hit the repo..13:34
pesterok, added vip IP to hosts, looks good now13:34
*** sbezverk_ is now known as sbezverk13:35
*** rmart04 has quit IRC13:35
rwellumsbezverk_: when I download canal.yaml it's not missing that net-conf.json? I tried without that fix and it came up fine. I did have to apply the rbac.yaml change.13:36
sbezverkrwellum: ok then it got finally merged, cool13:37
rwellumok thanks.13:37
pesteris there any sane all-in-one with like 'compute, network, glance, horizon, keystone' ?13:38
sbezverkrwellum: there were some other changes/clean up in that gate PS which I think should be sync up to the deployment guide13:38
*** rmart04 has joined #openstack-kolla13:39
*** jaosorior is now known as jaosorior_away13:41
*** lucasxu has joined #openstack-kolla13:44
egonzalezpester, basic compute kit is enabled by default (keystone, neutron, nova and glance)13:45
*** ducttape_ has joined #openstack-kolla13:46
pesterdo I understand correctly that "kolla-ansible deploy" will deploy eveything from all-in-one ? (like 20 openstack components)13:46
pester?13:46
openstackgerritBertrand Lallau proposed openstack/kolla-ansible stable/ocata: Fix Heat Kolla demo in contrib  https://review.openstack.org/46215913:48
*** ducttape_ has quit IRC13:48
*** ducttape_ has joined #openstack-kolla13:48
openstackgerritBertrand Lallau proposed openstack/kolla-ansible stable/ocata: Magnum: add oslo_messaging_notifications config  https://review.openstack.org/46216013:49
openstackgerritBertrand Lallau proposed openstack/kolla-ansible stable/ocata: Fix Grafana config file template to use variables  https://review.openstack.org/46216113:49
egonzalezpester, all depends on your globals.yml configuration, there you can enable all the services you want. by default only core openstack services are deployed13:50
pesterthx!13:50
*** janki has quit IRC13:53
*** jascott1 has joined #openstack-kolla13:56
*** lpetrut has quit IRC13:57
*** ducttap__ has joined #openstack-kolla13:59
*** ksumit has joined #openstack-kolla13:59
*** jascott1 has quit IRC14:00
rwellumsbezverk: yes agreed - but it's a little hard to figure out what is needed. I'm going through a full deployment now - will feed it back in.14:01
*** lucasxu has quit IRC14:02
*** zhurong has joined #openstack-kolla14:02
*** ducttape_ has quit IRC14:02
*** lucasxu has joined #openstack-kolla14:03
*** lpetrut has joined #openstack-kolla14:04
*** shasha___ has joined #openstack-kolla14:04
pesterlooks like a mess, build.py built me 4.0.0 images, kolla-ansible pull - trying to get 4.0.1 images and fails (kolla-toolbox image),  kolla-ansible pull failing with image not found 4.0.1 (fluentd)14:05
pesterall fails with error 404 not found14:06
*** manheim_ has joined #openstack-kolla14:06
*** lpetrut has quit IRC14:07
Reepicheeppester ar you building your 4.0.1 images?  AFAIK they are not in docker hub14:08
pesterI`m trying to get working env, I`m ok with 4.0.0 - how to tell kolla to use em ? I already build them14:09
*** manheim has quit IRC14:10
Reepicheepin your globals.yml file there is a "openstack_release:"14:11
egonzalezpester, set openstack_release: "4.0.0" in globals.yml, 4.0.1 are not pushed to dockerhub yet14:11
*** schwicht has quit IRC14:11
*** rwallner has joined #openstack-kolla14:14
*** lucasxu has quit IRC14:15
*** lucasxu has joined #openstack-kolla14:17
*** rwallner has quit IRC14:18
*** openstackgerrit has quit IRC14:18
*** openstackgerrit has joined #openstack-kolla14:19
openstackgerritMerged openstack/kolla master: Add Apache packages to zaqar container  https://review.openstack.org/46177914:19
*** schwicht has joined #openstack-kolla14:22
*** eaguilar has quit IRC14:22
*** lucasxu has quit IRC14:24
*** lucasxu has joined #openstack-kolla14:24
*** dave-mccowan has joined #openstack-kolla14:26
*** lucasxu has quit IRC14:26
*** yangyape_ has joined #openstack-kolla14:27
*** yangyapepeng has quit IRC14:30
*** lpetrut has joined #openstack-kolla14:31
*** reddydodda has joined #openstack-kolla14:34
*** lpetrut has quit IRC14:37
*** lpetrut has joined #openstack-kolla14:37
*** gfidente has quit IRC14:41
*** manheim_ has quit IRC14:42
*** gfidente has joined #openstack-kolla14:42
*** manheim has joined #openstack-kolla14:42
*** rwallner has joined #openstack-kolla14:44
*** manheim has quit IRC14:46
*** manheim has joined #openstack-kolla14:46
*** zhurong has quit IRC14:52
*** salv-orl_ has joined #openstack-kolla14:53
*** salv-orlando has quit IRC14:56
*** bmace has quit IRC14:56
*** bmace has joined #openstack-kolla14:56
*** shasha___ has quit IRC14:57
*** erlon has joined #openstack-kolla14:58
sdakesbezverk do you have any link on the - in namespaces not permitted for docker?15:01
sbezverksdake: please clarify, not sure what you are refering to15:02
sdakesbezverk at one point dockerhub no longer permitted - in namespaces15:02
sdakesbezverk this was recent15:02
sdakeand it broke our gates and caused all kinds of other pain15:03
sbezverkare you talking about shared namespace issue we see in kolla-kube?15:03
sdakeno15:03
sdakee.g.15:03
sdakewe cannot have openstack-kolla/image15:03
sdakeonly kolla/image15:03
sdakeopenstack-kolla doesn't work with docker or hub or something15:03
sbezverksdake: ah metadata change I think15:03
portdirecto/15:04
sdakegot a link? - need to understand why15:04
sdakesup portdirect15:04
sbezverksdake: let me look for it15:04
portdirectjust getting prepped for summit :)15:04
sbezverkportdirect: o/15:04
portdirectsbezverk: you gonna have any time this afternoon? looked through the suff you sent and it looks awesome :)15:05
portdirectalso when are you peeps getting to boston?15:05
sbezverkportdirect: I should be ok for 5pm est today15:06
sbezverkportdirect: I have made video of demo and edited it to 5 minutes15:06
sbezverkwe can give people a choice either to watch 14 minutes live or to have more time for q&A15:07
portdirectsbezverk: sounds good -I'll make time, and try and see if i can get srwilkers along too :)15:07
sbezverkportdirect: ok will send invite15:07
*** tovin07_ has joined #openstack-kolla15:09
*** athomas has quit IRC15:09
*** bmace has quit IRC15:10
openstackgerritMerged openstack/kolla-ansible master: Remove un-used Keystone test path  https://review.openstack.org/45433315:10
*** tovin07_ has left #openstack-kolla15:11
sbezverksdake: sorry cannot find anything  :(15:11
sdakesbezverk thanks for looking :)15:12
sdakeportdirect in on saturday15:13
*** jascott1 has joined #openstack-kolla15:13
portdirectsdake: nice - I'm tweaking my flights today, so may end up there on sat as well15:14
*** tovin07_ has joined #openstack-kolla15:14
*** fooliouno has quit IRC15:14
inc0hey15:15
*** tovin07_ has quit IRC15:15
jascott1o/15:15
*** tovin07_ has joined #openstack-kolla15:15
sbezverkportdirect: I drive there, so I will be sunday afternoon15:15
pestergot error with nova compute: "["docker", "exec", "kolla_toolbox", "openstack", "--os-interface", "internal", "--os-auth-url", "http://172.1.1.10:35357/v3", "--os-identity-api-version", "3", "--os-project-domain-name", "default", "--os-tenant-name", "admin", "--os-username", "admin", "--os-password", "xfmJKpSzsHH5XDWJ4nc533M9X8W4jXiJscM86mmC", "--os-user-domain-name", "default", "compute", "service15:17
*** salv-orl_ has quit IRC15:18
*** salv-orlando has joined #openstack-kolla15:18
egonzalezpester, what error?15:19
pester[nova : Waiting for nova-compute service up] - the message above is failed command15:20
*** shasha___ has joined #openstack-kolla15:20
pesterwhen I run command manually I can see service "compute_legacy"15:21
openstackgerritMerged openstack/kolla-ansible master: Add sudo command for non-root account to mount osd  https://review.openstack.org/43929715:21
egonzalezpester, do you have any compute node in the inventory and running?15:21
pesterI have AIO - so I`m not sure, nova_compute container is running,  and in inventory I have [compute] "localhost       ansible_connection=local"15:23
egonzalezpester, can you run openstack service list and paste output to paste.openstack.org, also check if rabbitmq is running and logs from /var/lib/docker/volumes/kolla_logs/_data/nova/15:24
pesterbut looks like this compute register itself in keystone as "legacy_compute" so kolla can`t find it ? how this compute intended to work ? connect to libvirt on host machine ?15:24
egonzalez* openstack compute service list15:25
*** dciabrin has quit IRC15:26
*** dciabrin has joined #openstack-kolla15:26
pesteregonzalez: http://paste.openstack.org/show/608722/15:28
pesterso, yes, compute returns nothing15:28
egonzalezpester, check nova-(scheduler/compute).log15:29
pesterstrange errors only in nova-api about cells, but I don`t need it15:29
pesterhm, scheduler is full of exceptions  "HostMappingNotFound: Host 'ubuntu-xenial' is not mapped to any cell"15:30
pestercompute is ok, reporting "Final resource view: name=ubuntu-xenial phys_ram=7817MB used_ram=512MB phys_disk=188GB used_disk=0GB total_vcpus=1 used_vcpus=0 pci_stats=[]" every minute15:30
egonzalezpester, are you using ocata or master? appear any node at openstack compute hypervisor list?15:33
*** ducttape_ has joined #openstack-kolla15:34
pesterI`m guess using master, but don`t know how to be sure, `hypervisor list` - shows no hypervisors.15:35
*** manheim has quit IRC15:35
egonzalezif nova is connected to rabbitmq and container are running, I'd likely say was deployed with ansible ocata code with master images15:37
egonzalezcheck nova version: docker exec nova_api pip freeze | grep -i nova15:37
*** ducttap__ has quit IRC15:37
pester"nova==16.0.0.0b2.dev43" & "python-novaclient==7.1.0"15:38
*** manheim has joined #openstack-kolla15:38
egonzalezto be sure what's the issue, if you execute docker exec -ti kolla_toolbox mysql -unova -p<NOVA_PASSWD> -h <MARIADB_HOST> nova -e "SELECT * from services WHERE topic = 'compute'";15:38
pesteregonzalez: http://paste.openstack.org/show/608723/15:40
*** manheim has quit IRC15:41
pesteras of me it looks like I have enabled cells and it broke nova-api15:42
egonzalezalmost sure the issue is ansible ocata code with master images15:42
egonzalezpester, cells are not optional in ocata15:42
pesterimages are 4.0.0 build locally with build.py15:42
egonzalezkolla-ansible how was installed, via pip or from git repo?15:43
pestercause kolla pul - tries to get images 4.0.1 - and got 404 error15:43
pester`pip install kolla-ansible`15:44
egonzalezpip install ocata code, while images build by build.py created master images15:44
pesterok, how can I get ocata images ?15:44
egonzalezeasiest soluction, remove containers, set openstack_release: "4.0.0" into /etc/kolla/globals/yml, kolla-ansible pull and deploy again15:45
egonzalezensure images changes when pulling, also clean up any config to your local registry or will pull the same images again15:46
*** fooliouno has joined #openstack-kolla15:51
*** ducttape_ has quit IRC15:53
*** ducttape_ has joined #openstack-kolla15:53
*** zhubingbing_ has quit IRC15:59
*** manheim has joined #openstack-kolla16:03
*** egonzalez has quit IRC16:03
*** rwallner has quit IRC16:07
openstackgerritMerged openstack/kolla master: Make sure _CLI_OPTS def style uniform  https://review.openstack.org/46045416:08
*** manheim has quit IRC16:08
openstackgerritDavid Moreau Simard proposed openstack/kolla-ansible master: Do not merge: Test ARA 0.13.0rc1  https://review.openstack.org/43996316:11
openstackgerritDavid Moreau Simard proposed openstack/kolla-ansible master: Do not merge: Test ARA 0.13.0rc1  https://review.openstack.org/43996316:11
inc0thanks dmsimard :)16:14
dmsimardinc0: np, I try not to break people16:15
inc0I need to learn more about ARA16:16
dmsimardinc0: ansible is so 2016 anyway, it's all about k8s now, no ? :P16:17
inc0ain't that true...16:17
inc0it's serverless16:17
inc0serverless compute nodes16:17
*** eaguilar has joined #openstack-kolla16:17
inc0we should deploy cloud on cloud16:17
dmsimardI'm always surprised at some big ara deployments users are sharing with me, it's pretty amazingf16:18
* dmsimard has never created something popular before16:18
inc0yeah it solves improtant problem - ansible not being easy to debug16:18
*** matrohon has quit IRC16:20
inc0I'm always amazed how simple is idea Ansible is built on16:20
*** asettle has left #openstack-kolla16:20
inc0let's put ssh in loop and write a yaml to do it16:21
dmsimardlol16:21
dmsimardIt's predecessor is probably fabric ? I don't know the history16:22
dmsimardhttp://www.fabfile.org/16:23
dmsimardYeah, looks like fabric goes all the way back to 200816:23
inc0me neither, never got into fabric really16:23
*** pcaruana has quit IRC16:24
*** shardy has quit IRC16:28
*** vhosakot has joined #openstack-kolla16:33
pbourkehey Jeffrey4l, you tried chrony recently? cant get it to generate a keyfile16:35
vhosakotinc0: rwellum: sorry, forgot to send this info earlier.  Here is the talk about kolla-kubernetes I'm giving at the Red Hat summit today - https://twitter.com/CiscoCloud/status/859084257708396544   let me know if we can meet here at the Red Hat summit.16:36
rwellumThanks vhosakot  - I'm not at the RH Summit - but some colleagues are and I'll pass this on to them.16:38
inc0vhosakot: unfortunately not, but good luck!16:38
*** nathharp has quit IRC16:39
vhosakotcool rwellum, inc0, thanks for the info.16:39
inc0vhosakot: will it be recorded?16:39
vhosakotinc0: yes, will send the link16:39
inc0cool, thanks!16:39
vhosakotinc0: np!16:40
vhosakotinc0: rwellum here is the architecture diagram I put together for kolla-k8s, would love to hear feedback from the community.  we don't have a good arch diagram in our repo :)  -  http://imgur.com/a/fBlOw16:41
*** rmart04 has quit IRC16:43
*** athomas has joined #openstack-kolla16:44
vhosakotinc0: haha, thanks for re-tweeting... I have re-activated my twitter account and slowly starting to become a tweeters.. lol, did not use my twitter account for years until recently.. lol16:45
inc0I'm new to it too16:45
vhosakotinc0: ah cool16:45
inc0a decade too late it seems;)16:45
portdirectjust remember too many tweets makes you a...16:46
spsuryavhosakot:  nice arch of kk8s16:46
portdirect;)16:46
inc0portdirect: twat?:P16:46
*** hieulq_ has joined #openstack-kolla16:47
vhosakotportdirect: haha, I know... I dont have time for tweet and facebook and instagram and those things... lol16:47
portdirect:D16:47
vhosakotspsurya: thanks! let me know if you have any feedback :)16:47
portdirectvhosakot: on a more usefull note - that diagram looks awesome16:47
vhosakotportdirect: haha, are you serious?16:47
spsuryavhosakot: sure , i will16:48
inc0bbiaf, moving to beach:P16:48
*** harlowja has joined #openstack-kolla16:48
vhosakotportdirect: I forgot the picture of the user in that diagram... lol16:48
portdirectwould be fantastic to get in repo, provided vendor names removed for agnosticism16:48
vhosakotportdirect: wanted to draw a gender-neutral stick figure as the user :)16:48
portdirectvhosakot: gender neutral kolla no?16:48
vhosakotportdirect: true, I'll remove vendor/company names when I upstream it16:48
portdirects/kolla/koala ! :)16:49
vhosakotportdirect: well, I ususally dont draw my users as a man since women audience may be unhappy ;)16:49
vhosakotportdirect: I don;t draw a woman as well.. lol16:49
vhosakotportdirect: I draw like a stick figure16:49
*** hieulq_ has quit IRC16:49
*** unicell has joined #openstack-kolla16:52
*** tovin07_ has left #openstack-kolla16:52
*** unicell has quit IRC16:53
sbezverkvhosakot: the way it presented 1 docker container runs whole neutron, which is not the case.. I am not sure though how it can be changed easily..16:55
vhosakotsbezverk: no, there are multiple containers for neutron, nova, cinder.. right... one box is not necessarily just one container.. I'm explaining it in detail in my talk16:56
*** mgoddard has quit IRC16:57
sbezverkvhosakot: well then I am not sure why there is docker under each box..16:57
vhosakotsbezverk: it is just the idea that neutron runs on docker (in multiple containers)16:57
sbezverkvhosakot: well make sure to clarify it, because the picture does not show it..16:58
vhosakotsbezverk: yep, I explain in detail how each OpenStack service is broken into related containers with corresponding jinja templates, config files, ansible scripts and helm16:59
vhosakotsbezverk:  thanks for the great suggestion..16:59
*** hieulq_ has joined #openstack-kolla16:59
vhosakotsbezverk: if we draw each service's containers, I thought I will end up with too many boxes to show he core services, and I did not have time to write a "click and zoom" on a particular part of the diagram in the picture :)17:00
vhosakotthat arch diagram will also change due the recent moby project for docker's arch17:01
*** rwallner has joined #openstack-kolla17:02
spsuryanight all17:03
*** jmccarthy has quit IRC17:03
*** Serlex has quit IRC17:04
*** athomas has quit IRC17:05
*** eaguilar has quit IRC17:07
*** hieulq__ has joined #openstack-kolla17:11
*** lucasxu has joined #openstack-kolla17:12
*** hieulq_ has quit IRC17:12
*** ducttap__ has joined #openstack-kolla17:16
*** ducttape_ has quit IRC17:16
vhosakotsdake gave some great feedback, adding Ironic bits that do bm orchestration in kolla-k8s in my diagram17:16
*** srwilkers has quit IRC17:20
*** unicell has joined #openstack-kolla17:25
*** unicell has quit IRC17:25
*** unicell has joined #openstack-kolla17:25
*** hieulq__ has quit IRC17:26
*** vhosakot has quit IRC17:29
*** ducttap__ has quit IRC17:32
*** gfidente is now known as gfidente|afk17:45
*** ducttape_ has joined #openstack-kolla17:45
*** ducttap__ has joined #openstack-kolla17:47
*** ducttape_ has quit IRC17:50
*** eaguilar has joined #openstack-kolla17:56
*** rwallner has quit IRC17:59
*** shasha___ has quit IRC18:01
*** egonzalez has joined #openstack-kolla18:05
*** srwilkers has joined #openstack-kolla18:05
*** ducttap__ has quit IRC18:05
*** rwallner has joined #openstack-kolla18:07
*** lucasxu has quit IRC18:11
*** lucasxu has joined #openstack-kolla18:11
*** salv-orlando has quit IRC18:12
*** krtaylor has quit IRC18:12
*** shasha___ has joined #openstack-kolla18:19
*** ducttape_ has joined #openstack-kolla18:23
*** david-lyle has quit IRC18:24
*** dklyle has joined #openstack-kolla18:24
*** jascott1 has quit IRC18:25
*** jascott1 has joined #openstack-kolla18:26
*** jascott1 has quit IRC18:27
*** fooliouno has quit IRC18:28
*** jtriley has quit IRC18:30
*** iniazi has quit IRC18:36
*** jrich523 has quit IRC18:38
*** eaguilar has quit IRC18:40
*** jtriley has joined #openstack-kolla18:44
*** shasha___ has quit IRC18:45
*** shasha___ has joined #openstack-kolla18:46
*** srwilkers has quit IRC18:48
*** shasha___ has quit IRC18:50
-openstackstatus- NOTICE: Gerrit on review.openstack.org is being restarted to accomodate a memory leak in Gerrit. Service should return shortly.18:52
*** slagle has quit IRC18:53
*** krtaylor has joined #openstack-kolla18:54
*** srwilkers has joined #openstack-kolla18:55
*** manheim has joined #openstack-kolla19:02
*** eanylin_ has quit IRC19:04
*** eanylin has joined #openstack-kolla19:06
*** srwilkers_ has joined #openstack-kolla19:09
eanylinsbezverk: I went and retry again and landed up in similar situation where neutron dhcp, l3-agent and neutron metadata-agent stays in init state while everything else came up without issues... http://paste.openstack.org/show/608737/, the error seems to be related to "references non-existent config key", i.e. http://paste.openstack.org/show/608738/19:09
eanylinwasnt too sure what I am missing...19:09
sbezverkeanylin: looking19:10
eanylinsbezverk, thanks19:10
sbezverkeanylin: http://paste.openstack.org/show/608740/ compare with what you have19:11
eanylinsbezverk: Mine looks like this http://paste.openstack.org/show/608742/19:13
eanylinhmm, seems like not exact match19:13
eanylinmissing something for those 3 that had problems19:14
*** georgem1 has joined #openstack-kolla19:19
sbezverkeanylin: can you paste you cloud.yaml?19:21
eanylinok19:22
*** srwilkers has quit IRC19:22
*** srwilkers_ is now known as srwilkers19:23
eanylinsbezverk: http://paste.openstack.org/show/608744/19:23
sbezverkeanylin: kubectl get pods -n kolla -o yaml {neutron-l3-agent pod}19:26
eanylinsbezverk: http://paste.openstack.org/show/608746/19:31
*** shasha___ has joined #openstack-kolla19:32
sbezverkeanylin: :) interesting so far everything looks ok19:35
sbezverkeanylin: let's check the actual configmap kubectl get configmap -n kolla -o yaml {neutron metadata agent}19:35
sbezverkconfigmap19:35
eanylinsbezverk: I think so too. The same configurations was working for codes from mabe 2.5 weeks ago19:36
eanylinok19:36
sbezverkeanylin: I doubt it is the code as we test that at the gate many times per day19:37
*** ducttape_ has quit IRC19:37
eanylinsbezverk: I agree. Everything looks ok19:37
eanylinsbezverk: http://paste.openstack.org/show/608747/19:37
eanylinand your workaround for canal works as well. I tested that19:38
sbezverkeanylin: the most recent change I could think of was moving config generation to ansible19:38
eanylinok19:38
*** lpetrut has quit IRC19:41
*** reddydodda has quit IRC19:41
*** salv-orlando has joined #openstack-kolla19:41
*** manheim has quit IRC19:41
sbezverkeanylin: the config map look not ok, I compared with mine and it is missing some items19:42
sbezverkhow did you generate it?19:42
eanylinsbezverk: Ok. Actually I just followed the same steps in the Deployment Guide.19:43
eanylinsbezverk: This one I think? http://paste.openstack.org/show/608749/19:44
eanylinsbezverk: Didnt deviate this time round19:44
eanylinsbezverk: And I think its same steps from before, thats the usual approach I think19:45
sbezverkeanylin: I checked kube code and it does not like the key19:50
eanylinok19:50
eanylinsbezverK: Hmm19:50
sbezverkeanylin: ok I think I got it :)19:50
eanylinsbezverk: Good, tell me :)19:51
sbezverkeanylin: are you sure you are ready for this ;)??19:51
sbezverk          configMap:19:51
sbezverk            name: neutron-metadata-agent19:51
sbezverk            items:19:51
sbezverk            - key: neutron.conf19:51
sbezverk              path: neutron.conf19:51
sbezverk            - key: config.json19:51
sbezverk              path: config.json19:51
sbezverk            - key: ml2-conf.ini19:51
sbezverk              path: ml2_conf.ini19:51
sbezverk            - key: metadata-agent.ini19:51
sbezverk              path: metadata_agent.ini19:51
eanylinsbezverk: Am ready :D19:52
sbezverkyou see the configmap expects to see these keys19:52
eanylinok19:52
sbezverkbut your configmap does not have key: ml2-conf.ini19:52
eanylinoh19:52
sbezverkif you dump the content of your configmap you will see that it is missing19:52
eanylinok19:52
eanylinbut I dont recall having to do something extra for ml2-conf.ini when we started working on the deployment guide with sdake19:53
sbezverkso the real issue was generation of cofiguration files19:53
eanylinok19:53
sbezverkansible did not place ml2_conf.ini to /etc/kolla/neutron-blah19:53
eanylinoh!19:53
eanylinmakes sense19:54
sbezverkand when config maps got generated it did not get picked up and added to the config map19:54
*** rwallner has quit IRC19:54
eanylinso issue is with kolla-ansible genconfig ?19:54
sbezverkdo not ask me why ml2_conf.ini did not get copied :) you need to ping our ansible experts ;) I would start with inc019:55
sbezverkeanylin: yes19:55
eanylinhaha, ok19:55
eanylinsbezverK: Ok. Not sure if inc0 is around19:55
eanylinsbezverk: Thanks for help though :)19:56
sbezverkeanylin: to unblock you I can dump my configmaps but you will need to edit bunch of things like password ips etc19:56
sbezverknot sure if you are up for lots of vi-ing19:57
eanylinok, I can try that19:57
egonzalezml2  files  was not used by most of neutron services  and  was  removed from them  https://review.openstack.org/#/c/447992/10/ansible/roles/neutron/tasks/config.yml19:57
eanylinhaha19:57
eanylinoh, ml2 files were removed19:57
*** reddydodda has joined #openstack-kolla19:57
sbezverkwell you should stop them to use kolla-ansible19:57
sbezverkto generate configs and you new ansible playbook inc0 has created19:58
eanylinah, that explains why it stopped working this weekend since the merge happens 6 days ago19:58
eanylinsbezverk: Ok. How to use that?19:58
sbezverkin this case we are independent to what kolla-ansible dudes do19:58
eanylinthat is true19:58
sbezverkkolla-kubernetes/tests/bin/setup_config_iscsi.sh:ansible-playbook -e ansible_python_interpreter=/usr/bin/python -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla ansible/site.yml19:59
sbezverkhere is an example we use at the gate19:59
sbezverkfor iscsi workflow19:59
eanylinsbezverk: Ok. I will take a look20:00
eanylinI remember seeing it for the iscsi one when I was trying to bring it up20:00
sbezverkyou should be able to use that command without any modifications20:00
eanylinokie20:00
eanylinthanks20:00
*** jascott1 has joined #openstack-kolla20:01
*** srwilkers has quit IRC20:01
eanylinsbezverk: Should the DG be updated in this case? I think everyone will face similar problems...20:01
sbezverkeanylin: absolutely, I thought people started using inc0's playbook20:01
eanylinsbezverK: I wasnt aware of this one, maybe sdake knows about it. Will need to ask him...20:02
sbezverkeanylin: neither was I, I created my configmaps long time ago and just keep re-using them without regenerating20:03
*** rwallner_ has joined #openstack-kolla20:03
eanylinsbezverk: Yeah, I was rebuilding my environment20:03
*** srwilkers has joined #openstack-kolla20:04
eanylinsbezverk: And found that things from 2 to 3 weeks ago was working properly. This will explain why this happened.20:04
eanylinsbezverk: Is the site.yml same as cloud.yaml or?20:05
*** rwallner_ has quit IRC20:08
*** ducttape_ has joined #openstack-kolla20:10
sbezverkeanylin: nope20:10
sbezverkcloud.yaml is specific to your kubernetes cluster environment20:10
eanylinsbezverk: Ok. More like the all-in-one, multinode.yaml ?20:10
eanylinsbezverk: Ok20:11
sbezverkeanylin: hm not sure, it is more like global.yml20:11
sbezverkin kolla-ansible terms20:11
eanylinhmmm, but you have reference to globals.yml, paswords.yml20:11
eanylinI think the only one left for kolla-ansible is all-in-one.yml20:11
eanylinfor executing against the hosts20:12
*** georgem1 has quit IRC20:12
sbezverkeanylin: with all ansible related questions, you'd better check with sdake or inc020:12
*** Pavo has quit IRC20:12
eanylinsbeverk: Okie, will do20:13
eanylinsdezverk: Thanks again20:13
sdakeeanylin which ansible are you using - kolla-ansible genconfig?20:14
sdakeor the internal ansible code?20:14
eanylinegonzalez: Thanks for your information too20:14
eanylinsdake: I am following the guide, so yeah, using kolla-ansible genconfig20:15
sdakethanks - i've got to head out for a bit so unable to help atm20:15
eanylinand it will end up with the situation that I am seeing now due to missing ml2 config20:15
sdakesounds like you ahe enough to work with tho20:15
eanylinsdake: No worries. I think for now I should be able to try other stuff and see if it comes back20:16
*** skramaja has quit IRC20:21
*** ducttap__ has joined #openstack-kolla20:22
*** lazyPower has quit IRC20:24
*** ducttape_ has quit IRC20:26
*** lucasxu has quit IRC20:28
*** ducttape_ has joined #openstack-kolla20:30
*** ducttap__ has quit IRC20:33
*** p6_ is now known as p620:35
p6anyone ever seen or have any insight on kolla-toolbox getting stuck with a ton of shm mounts (in excess of 25k) on the host?20:36
*** krtaylor has quit IRC20:39
*** dave-mccowan has quit IRC20:41
*** salv-orl_ has joined #openstack-kolla20:56
*** jtriley has quit IRC20:57
srwilkerso/20:58
*** salv-orlando has quit IRC20:59
*** ksumit has quit IRC21:05
*** bmace has joined #openstack-kolla21:09
*** rwallner has joined #openstack-kolla21:18
*** krtaylor has joined #openstack-kolla21:19
*** rwallner has quit IRC21:22
*** jascott1 has quit IRC21:32
*** Pavo has joined #openstack-kolla21:35
*** schwicht has quit IRC21:37
*** shasha___ has quit IRC21:44
*** shasha___ has joined #openstack-kolla21:45
*** gfidente|afk has quit IRC21:47
*** ducttap__ has joined #openstack-kolla21:48
*** ducttape_ has quit IRC21:48
*** brad[] has quit IRC21:52
*** jascott1 has joined #openstack-kolla21:54
*** Pavo has quit IRC21:56
*** Pavo has joined #openstack-kolla22:00
*** schwicht has joined #openstack-kolla22:02
*** brad[] has joined #openstack-kolla22:05
*** dciabrin has quit IRC22:12
*** jascott1_ has joined #openstack-kolla22:23
*** haplo37 has joined #openstack-kolla22:23
*** mgkwill_ has joined #openstack-kolla22:23
*** ediardo_ has joined #openstack-kolla22:23
*** igordcar1 has joined #openstack-kolla22:26
*** harbie_ has joined #openstack-kolla22:26
*** pron_ has joined #openstack-kolla22:27
*** srwilkers has quit IRC22:27
*** slagle has joined #openstack-kolla22:27
*** erlon has quit IRC22:28
*** haplo37_ has quit IRC22:28
*** mgkwill has quit IRC22:28
*** ediardo has quit IRC22:28
*** jascott1 has quit IRC22:28
*** igordcard has quit IRC22:28
*** pron has quit IRC22:28
*** harbie has quit IRC22:28
*** shasha_t_ has quit IRC22:28
*** harlowja has quit IRC22:28
*** harbie_ is now known as harbie22:28
*** harbie has quit IRC22:28
*** harbie has joined #openstack-kolla22:28
*** mgkwill_ is now known as mgkwill22:28
*** ediardo_ is now known as ediardo22:29
*** manheim has joined #openstack-kolla22:31
*** schwicht has quit IRC22:31
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Ensure that ironic-dnsmasq can find its IP address  https://review.openstack.org/46231022:32
*** salv-orl_ has quit IRC22:32
*** egonzalez has quit IRC22:43
*** ducttap__ has quit IRC23:10
*** srwilkers has joined #openstack-kolla23:12
*** ducttape_ has joined #openstack-kolla23:13
*** rwellum has quit IRC23:18
*** srwilkers_ has joined #openstack-kolla23:20
*** dklyle has quit IRC23:23
*** manheim has quit IRC23:28
*** harlowja has joined #openstack-kolla23:31
*** ducttape_ has quit IRC23:32
*** reddydodda has quit IRC23:33
*** esmute has joined #openstack-kolla23:36
esmuteHello, im trying to deploy Openstack using kolla on kubernetes. I am following https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html23:37
esmutebut when i get to the part to deploy mariadb, the pod is crashing23:37
esmutekubectl get log doesnt show anything meaningful23:37
esmutehow do i get more information about it?23:38
portdirecthey, whats the prefered way of enabling dashboards in the horizon image?23:38
esmutehere is the log https://gist.github.com/kokhang/017b1634d05750a9187e7b4bd189b02623:38
* inc0 back from ocean beach23:45
inc0portdirect: we have most of them installed in image already23:48
inc0enable should just be conf matter23:48
inc0esmute: try kubectl get pods and examine pod with mariadb23:49
portdirectnice  -think I'm being daft but was scrating my head over this: https://github.com/openstack/kolla/blob/master/docker/horizon/extend_start.sh#L7923:49
* inc0 not k8s expert23:49
*** schwicht has joined #openstack-kolla23:50
esmute$ kubectl -n kolla get pod NAME        READY     STATUS             RESTARTS   AGE mariadb-0   0/1       CrashLoopBackOff   13         48m23:50
inc0portdirect: you're enabling magnum on top of k8s? kube all the way down;)23:50
esmuteso the hosts are using coreos.... would that matter?23:50
inc0shouldn't matter23:50
portdirectinc0: still digging through turtles here :)23:51
* inc0 is thinking how to get to /var/log/ inside container run by k8s23:51
esmuteinc0: me too.. but the pod doesnt survive long enough for me to attach to it23:52
portdirectinc0: can just exec into it if you need - otherwise srwilkers is doing some cool stuff23:52
inc0esmute: what volume mgmt are you using? ceph?23:52
inc0exec won't work with crashing containers23:52
* portdirect realising context23:52
inc0in kolla ansible I'd simply go to volume23:52
portdirectah yeah - you could add a bindmount if its not there in the manifest if its not their already?23:53
*** ducttape_ has joined #openstack-kolla23:53
portdirectbut that could be tricky otherwise23:53
inc0let me check our code for that23:54
portdirectinc0: and of course its just as simple as setting ENABLE_MAGNUM=true, doh!23:54
inc0:D23:55
inc0esmute: try to setup storageProvider host23:55
inc0and then it's bindmount to /var/lib/kolla/volumes23:55
*** goldyfruit has quit IRC23:56
*** ducttape_ has quit IRC23:57
*** masber has joined #openstack-kolla23:58
*** eaguilar has joined #openstack-kolla23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!