Monday, 2017-03-13

*** adrian_otto has quit IRC00:00
*** mnaser has joined #openstack-kolla00:04
mnaserhas anyone done work around taking over an existing cloud with kolla (openstack components seem easy, but im curious around ceph)00:04
*** kbaegis has joined #openstack-kolla00:04
SamYaplemnaser: once because someone told me i couldn't do it00:06
SamYaplemnaser: don't recommend it00:06
SamYaplein fact, i dont recommend kolla ceph in production and I never have00:06
SamYapleits stable enough, but not flexible enough00:06
mnaserSamYaple: how is it not flexible enough? it seems to do the base that's needed00:07
mnaserSamYaple: we're doing a step by step "take-over" with kolla and we could do external ceph and move ceph eventually00:07
SamYaplewould never recommend moving external ceph to a kolla-ceph deploy00:08
SamYaplei specifically said when i wrote it that it was for small labs and such00:08
mnaseri guess we could stick to our ansible stuff to maintain ceph00:09
SamYapleits not flexible in the sense that its hard to do proper maintenance on things where you might need to fix a journal since the disk died and such00:09
mnaseri guess, in a larger environment you can just scrap the osd and rebuild (at least in our environment, we're okay with this)00:09
mnaserbut i see your concerns00:09
*** adrian_otto has joined #openstack-kolla00:09
*** gfhellma has joined #openstack-kolla00:10
SamYaplei mean you *can*. you shouldn't neccesarily do that00:10
SamYaplei just last week recovered a journal in a 1.5PB environment rather than balance out the 4TB drive00:10
SamYaplebalancing out the 4TB OSD would have moved like 300TB of data00:11
mnaserjeez00:11
SamYaplegranted that cluster is on early subset of the tuneables, but still00:11
SamYaplejust because you can doesnt me you should00:11
mnaserwe'll see, right now i'm pretty confident in us getting the openstack control plane moved00:11
*** sayantan_ has joined #openstack-kolla00:12
mnaserstarting with keystone in the first few days00:12
SamYaplemoving the control plane is a great start00:13
SamYaplecompute is harder to do, but easily doable with manual intervention00:13
mnaserthe only annoyance is that our internal network is the same as the default docker bridge network00:13
mnaser(172.17/16)00:14
mnaserso that's something that might be annoying in the bootstrap process00:14
SamYaplesince kolla-ansible doesn't use docker bridge network that network should never come up00:14
SamYapleso no interference00:14
SamYaplethen you just use the systemd daemon to set a different network for safety reasons00:15
mnaserSamYaple im fairly sure the docker0 bridge goes up when teh service is started00:15
mnaserand grabs and ip too00:15
SamYaplemnaser: nope. docker0 is down unless there a a container using the docker bridge network00:16
mnaseryup, that's what we'll have to do, kolla has been spoiling us a bit00:16
mnaseroh00:16
mnaseryou are right00:16
mnaserit is listed as down00:16
mnasermy bad00:16
SamYapleit will still respond to pings on 172.17.0.1 though00:16
mnaser172.17.0.0/16 dev docker0  proto kernel  scope link  src 172.17.0.100:17
mnaserthe route does get inserted00:17
mnaserwhich will break network connectivity00:17
mnaserits just a systemd drop in but kolla spoiled us a bit in not having to do a lot :-P00:17
SamYapledepends on ordering00:17
*** manheim has quit IRC00:18
SamYaplebut anyway. back to the original point, kolla ceph stuff is meant for fewer than 100 osds (it can't actually do more than that based on naming conventions) and is good for small labs and clusters00:18
*** adrian_otto has quit IRC00:18
SamYapleits hard to call it production00:19
mnaseri guess in that case we'll continue to maintain "external ceph" and see how things go from there00:19
SamYaplei had run it longer than anyone and even i stopped using it (converted to bare-metal ceph)00:19
mnaserdocker on compute nodes is something that i am personally a bit ~bleh~ about to be honest00:19
SamYaplegotta have imaging00:20
SamYaplewithout it no rollback00:20
SamYapleluckily it is not performance impacting at all00:20
SamYapleVMs are run in the host namepsace and the same cgroups as if they were launched from the host00:20
SamYaple100% same performance00:20
mnaseroh yeah i understand that but its just00:21
mnaserits not what we're used to :)00:21
mnaserwe might work on getting monitoring based on sensu added at some point00:21
mnaseror help around an effort surrounding taht00:21
*** sbezverk has joined #openstack-kolla00:23
*** yingjun has joined #openstack-kolla00:31
*** sbezverk has quit IRC00:32
*** sbezverk has joined #openstack-kolla00:34
*** manheim has joined #openstack-kolla00:37
*** sbezverk has quit IRC00:45
*** duonghq has joined #openstack-kolla00:54
duonghqmorning guys00:54
*** dave-mccowan has joined #openstack-kolla00:59
openstackgerritJeffrey Zhang proposed openstack/kolla master: [wip] Install panko in ceilometer container  https://review.openstack.org/44468001:05
*** daidv has joined #openstack-kolla01:05
*** jtriley has joined #openstack-kolla01:06
*** cuongnv has joined #openstack-kolla01:08
duonghqhttp://logs.openstack.org/61/444561/4/check/gate-kolla-ansible-dsvm-deploy-ubuntu-source-ubuntu-xenial-nv/e2999ea/console.html#_2017-03-12_06_55_40_351982 seem that we got ara error again?01:09
*** zhurong has joined #openstack-kolla01:11
*** qiliang27 has joined #openstack-kolla01:16
*** zhubingbing has joined #openstack-kolla01:16
duonghqmorning zhubingbing01:16
zhubingbingmorning duonghq01:17
zhubingbing;)01:17
*** Margin has joined #openstack-kolla01:17
*** schwicht has quit IRC01:24
*** schwicht has joined #openstack-kolla01:25
spsuryaduonghq: morning01:32
spsuryazhubingbing:01:32
spsuryamrng01:32
duonghqsup spsurya01:33
zhubingbingmorning01:34
spsuryahw was your weekend guys ?01:38
*** manheim has quit IRC01:38
*** salv-orlando has joined #openstack-kolla01:41
*** salv-orlando has quit IRC01:45
mnaseris there an upgrade path in place that updates deployments from heka to fluentd01:48
mnaseras we go newton -> ocata01:48
*** schwicht has quit IRC01:52
SamYaplemnaser: there should always be an upgrade path for things marked stable01:55
SamYapleif its not implemented, its a bug01:55
duonghqmnaser, zhubingbing is the guys who worked on fluentd02:01
zhubingbingyes02:01
zhubingbingthere should always be an upgrade path for things marked stable02:01
zhubingbingagree sample02:02
duonghqzhubingbing, mnaser wants to know how we migrate from heka to fluentd when upgrade takes place02:02
zhubingbingto solve a lot of log collection problems02:03
zhubingbingbut the basic function is the same as heka02:04
MarginHi , I opened a bug https://bugs.launchpad.net/kolla/+bug/167224002:06
openstackLaunchpad bug 1672240 in kolla "destroy tool is incomplete " [Undecided,New]02:06
*** daidv has quit IRC02:06
*** MarginHu has joined #openstack-kolla02:07
*** Margin has quit IRC02:08
zhubingbinghi Margin02:08
zhubingbingdestroy tool is incomplete ? what's mean ?02:08
duonghqzhubingbing, imo, high and critical bug should always be targeted to some milestone02:18
openstackgerritJeffrey Zhang proposed openstack/kolla master: Install panko in ceilometer base container  https://review.openstack.org/44468002:19
zhubingbingem02:19
zhubingbingi will fix this bug02:19
zhubingbingnow02:19
duonghqzhubingbing, nice02:24
MarginHuzhubingbing , maybe I'm wrong02:27
zhubingbingdon't remove /var/lib/mariadb ?02:28
MarginHuI removed the /var/lib/docker/volumes02:31
MarginHubut also report error02:31
MarginHuI update the bug , please check it .02:31
zhubingbingOK02:33
kbaegisAnyone know where the diskimage builder output folder is?02:33
MarginHuI know02:34
MarginHuit is "pwd"02:34
*** Raj12 has joined #openstack-kolla02:34
kbaegisMarginHu: As in where I ran the command from? Doesn't appear to be true02:34
Raj12I have deployed multinode newton with kolla, now I need to do some changes in ml2.conf.ini how can I deploy  the changes02:35
Raj12I have tried adding config in /kolla/config but it is not changing the configure if I do reconfigure02:35
MarginHukbaegis, yes02:36
kbaegisRaj12: Best strategy is to set COPY ALWAYS in globals and just restart the container02:36
Raj12Not sure how to set COPY ALWAYS is there any doc?02:38
kbaegisRaj12: It's in /etc/kolla/globals02:38
kbaegis'config_strategy: "COPY_ALWAYS"'02:39
Raj12where I need to keep the changes?02:39
kbaegisand then docker restart nova_api or whatever02:39
kbaegisRaj12: /etc/kolla/globals.yml02:39
MarginHuI build ironic-agent image with disk-image-builder .02:40
kbaegisIt's not outputting into my current directory02:40
Raj12added 'config_strategy: "COPY_ALWAYS" and config in /etc/koll/connfig02:45
Raj12restarted container still no luck02:45
Raj12it has same old config nothing changed02:45
Raj12@kbaegis Thanks it worked now.02:49
*** Raj12 has quit IRC02:59
kbaegish2h03:00
openstackgerritJeffrey Zhang proposed openstack/kolla master: Load iscsi_tcp module when starting ironic-conductor  https://review.openstack.org/44469903:32
*** dave-mccowan has quit IRC03:32
*** adrian_otto has joined #openstack-kolla03:32
openstackgerrityunfeng zhou proposed openstack/kolla-ansible master: change log-* to flog-*.  https://review.openstack.org/44470003:34
openstackgerrityunfeng zhou proposed openstack/kolla-ansible master: change log-* to flog-*.  https://review.openstack.org/44470003:41
*** adrian_otto has quit IRC03:41
*** salv-orlando has joined #openstack-kolla03:42
*** salv-orlando has quit IRC03:46
*** adrian_otto has joined #openstack-kolla03:49
*** sayantan_ has quit IRC03:51
*** adrian_otto has quit IRC03:55
*** sayantan_ has joined #openstack-kolla03:59
*** sayantan_ has quit IRC04:04
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-ansible master: GerritReview check  https://review.openstack.org/44470504:14
openstackgerritJeffrey Zhang proposed openstack/kolla master: Load iscsi_tcp module when starting ironic-conductor  https://review.openstack.org/44469904:20
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Mount /lib/modules folder in ironic-conductor container  https://review.openstack.org/44470804:23
*** mdnadeem has joined #openstack-kolla04:25
*** jtriley has quit IRC04:34
*** sayantan_ has joined #openstack-kolla04:38
openstackgerritJeffrey Zhang proposed openstack/kolla master: Install panko in ceilometer base container  https://review.openstack.org/44468004:38
*** sayantan_ has quit IRC04:42
*** sudipto has joined #openstack-kolla04:45
*** sayantan_ has joined #openstack-kolla04:50
*** sayantan_ has quit IRC04:55
*** ssurana has joined #openstack-kolla05:11
*** japestinho has quit IRC05:20
*** skramaja has joined #openstack-kolla05:23
*** trinaths has joined #openstack-kolla05:23
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Add panko.conf file in ceilometer container  https://review.openstack.org/44471905:28
*** MarginHu has quit IRC05:38
*** MarginHu has joined #openstack-kolla05:39
*** MarginHu has quit IRC05:42
*** salv-orlando has joined #openstack-kolla05:43
*** salv-orlando has quit IRC05:47
*** unicell has joined #openstack-kolla05:57
*** unicell has quit IRC05:59
*** tovin07 has joined #openstack-kolla06:00
*** salv-orlando has joined #openstack-kolla06:02
*** unicell has joined #openstack-kolla06:04
*** tovin07_ has joined #openstack-kolla06:13
*** unicell has quit IRC06:20
*** Mannheim has joined #openstack-kolla06:31
*** jrobinson has quit IRC06:36
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Add panko.conf file in ceilometer container  https://review.openstack.org/44471906:40
*** jrobinson has joined #openstack-kolla06:49
*** bmace has quit IRC06:54
*** bmace has joined #openstack-kolla06:54
*** MarginHu has joined #openstack-kolla07:06
*** satyar has joined #openstack-kolla07:09
*** saneax-_-|AFK is now known as saneax07:17
spsuryaduonghq:07:20
spsuryaping...07:20
duonghq spsurya hey there07:22
*** rabel has joined #openstack-kolla07:22
spsuryawhat you think about this https://bugs.launchpad.net/kolla/+bug/157679407:23
openstackLaunchpad bug 1576794 in kolla "drop root for keystone" [Critical,In progress] - Assigned to Surya Prakash Singh (confisurya)07:23
spsuryawhat can be better way to fix07:23
spsuryathis07:23
spsurya?07:23
*** salv-orl_ has joined #openstack-kolla07:24
duonghqnot sure, but iirc, we have some issue relate to apache if we drop root for keystone07:24
duonghqdue to Keystone use WSGI for this interface07:24
duonghqnot just add USER keystone resolve issue07:25
spsuryaduonghq: any way we should not remove root previlege from keystone07:25
duonghqif somebody can prove we cannot drop root from keystone, why can mark this bug is invalid IMO,07:26
spsuryaI was thinking same.07:26
*** salv-orlando has quit IRC07:26
duonghq"should not" is not strong evidence07:26
duonghqone possible outcome is" comment in docker file why we do not drop root from keystone07:27
duonghqit'll prevent somebody raise this issue again07:27
spsuryaduonghq: suppose a root user deploy OpenStack, and he doesn't want that any other person would create any new credential i.e any normal USER07:29
duonghqspsurya, it's not a rational reason07:30
spsuryaduonghq: and if we give root previlege to any USER then he can also distribute other credential and can list all the credential, private cloud functionality would be damaged07:32
*** manheim has joined #openstack-kolla07:32
duonghqspsurya, sorry, misunderstand your last sentence,07:32
*** salv-orl_ has quit IRC07:33
*** jrobinson has quit IRC07:33
duonghqcan you explain it again?07:33
spsuryaduonghq: i mean any user can create credential as root does.07:34
spsuryaduonghq: I think inc0 has filed this bug07:34
spsuryaduonghq: may be he can add somthing into this07:34
*** manheim has quit IRC07:36
duonghqhmm,07:38
duonghqI think we should raise it again in this week meeting?07:38
*** manheim has joined #openstack-kolla07:39
spsuryaduonghq: sure07:39
spsuryaduonghq: we should07:39
duonghqspsurya, can you add this to the agenda07:40
duonghqseem that this week we will have many topic07:40
spsuryayeh i am adding07:40
spsuryaduonghq:07:40
*** MarginHu has quit IRC07:45
*** MarginHu has joined #openstack-kolla07:46
spsuryaduonghq: done07:47
*** MarginHu has quit IRC07:48
*** matrohon has joined #openstack-kolla07:50
*** japestinho has joined #openstack-kolla07:55
*** salv-orlando has joined #openstack-kolla08:04
*** tovin07 has quit IRC08:07
*** kwazar has joined #openstack-kolla08:09
*** zioproto has joined #openstack-kolla08:10
*** ssurana has quit IRC08:11
*** tovin07 has joined #openstack-kolla08:12
*** dciabrin has joined #openstack-kolla08:15
*** Serlex has joined #openstack-kolla08:16
*** openstackgerrit has quit IRC08:18
*** pcaruana has joined #openstack-kolla08:25
*** MarginHu has joined #openstack-kolla08:38
*** egonzalez has joined #openstack-kolla08:46
MarginHuhi, how to cleanup all related info about mariadb which is generated in last deployment ?08:56
*** openstackgerrit has joined #openstack-kolla08:57
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/newton: Sahara needs to run ssh-keygen  https://review.openstack.org/44476808:57
*** spsurya__ has joined #openstack-kolla08:59
egonzalezMarginHu, cleanup in which way? remove containers, config files, images, volumes?09:00
MarginHuyes , I met a mariadb galera issue when redeploy.09:01
egonzalezwhen executing cleanup-containers mariadb will remove mariadb container and volumes09:02
egonzalezconfig files aren't an issue when redeploying, since them are replaced by new files, but can be removed with cleanup-host09:03
egonzalezboth in tools/ folder09:03
*** trinaths1 has joined #openstack-kolla09:04
*** trinaths has quit IRC09:05
MarginHuyou're right, thanks09:05
egonzalezMarginHu, in the bug opened by the same issue, how you make the destroy? through kolla-ansible destroy?09:08
MarginHubut iptable rules09:08
MarginHuyes , kolla-ansible destroy09:09
egonzaleznewton, ocata or master?09:09
MarginHuocata09:09
egonzalezhrm, the mariadb issue shouldn't happen since that role removes containers and volumes09:10
MarginHuhow to troube-shoot the issue ?09:11
openstackgerritDave Walker proposed openstack/kolla-ansible master: Remove heka_socket vol and unwedge some containers  https://review.openstack.org/44477109:13
*** spsurya__ has quit IRC09:17
*** mgoddard has joined #openstack-kolla09:18
*** gfidente has joined #openstack-kolla09:24
*** athomas has joined #openstack-kolla09:24
*** zhubingbing has quit IRC09:24
*** zhubingbing has joined #openstack-kolla09:26
openstackgerritDave Walker proposed openstack/kolla-ansible master: Remove heka_socket vol and unwedge some containers  https://review.openstack.org/44477109:29
egonzalezMarginHu, i'm seeing the issue, when executing cleanup-container <container-name> instead of cleanup-container, volumes are not properly evaluated hence not removed09:30
egonzalezzhubingbing, ^^09:33
zhubingbinghi09:33
zhubingbingyes09:33
zhubingbingegonzalez09:33
egonzalezwhat i don't know yet is why destroy role does not remove those volumes, cleanup-containers without regex works fine09:34
openstackgerritThomas O'Neill proposed openstack/kolla master: Add neutron-bgp-dragent service.  https://review.openstack.org/44373209:35
*** matrohon has quit IRC09:35
egonzalezzhubingbing, got what happen with cleanup-containers https://github.com/openstack/kolla-ansible/blob/master/tools/cleanup-containers#L1509:36
*** matrohon has joined #openstack-kolla09:37
egonzalezmay notice there are ($( and the double )) at the end of the var, removing the first ( fixes the regex issue)09:37
zhubingbing7/500009:38
zhubingbingWǒ néng chóng xiàn tā mā09:38
zhubingbingcan i reproduce it ?09:38
egonzalezwhat? hehe09:39
MarginHuI think you can reproduce it .09:39
*** tovin07 has quit IRC09:39
zhubingbingegonzalez09:40
egonzalezMarginHu, btw, before using destroy role, have you removed mariadb container?09:40
MarginHuno09:41
zhubingbingegonzalez  i go to fix this bug09:41
egonzalezzhubingbing, yep09:41
zhubingbing;)09:42
zhubingbingi thinks there volume don‘t remove ;)09:43
*** hrw has quit IRC09:44
*** athomas has quit IRC09:45
*** athomas has joined #openstack-kolla09:45
*** hrw has joined #openstack-kolla09:45
*** yingjun has quit IRC09:47
egonzalezMarginHu, zhubingbing I can't reproduce the issue with mariadb volume using destroy role09:48
egonzalezall kolla volumes are removed09:48
zhubingbingme too09:48
zhubingbingi can't reproduce this bug09:48
egonzalezthe only issue i found is using regex in cleanup-containers, but destroy role does not use that09:48
zhubingbingi will talk with MarginHu09:49
kbaegisHey, anyone using barbican/senlin/magnum?09:49
egonzalezkbaegis, in prod or testing?09:49
kbaegistesting09:49
kbaegisCertificatesToClusterFailed: Failed to create certificates for Cluster09:49
egonzalezI use senlin and magnum but not barbican09:50
kbaegisbarbican logs talking about declaring loyalty to the emperor or some bs :)09:50
kbaegisgotcha09:50
zhubingbingi use magnum and barbicna09:51
kbaegiszhubingbing: Did you have to set anything up for certificates?09:51
zhubingbinghttps://review.openstack.org/#/c/444561/09:51
zhubingbingthis patch09:51
zhubingbingi fix it09:51
zhubingbingyes have some problem barbican09:51
kbaegistyvm09:52
kbaegisYou're awesome09:52
kbaegisSo after patch, I have to pip install -U ../kolla-ansible09:53
kbaegisdo I have to rebuild the barbican images?09:53
kbaegisAnd do I have to run reconfigure?09:53
zhubingbingdon;t rebuild images09:53
kbaegisokay09:53
duonghqegonzalez, can you review zhubingbing ps: https://review.openstack.org/#/c/444561/09:54
zhubingbingyour kolla version coat?09:54
*** MarginHu has quit IRC09:54
egonzalezduonghq, done09:54
kbaegisstable/ocata09:54
zhubingbingocata09:54
*** MarginHu has joined #openstack-kolla09:54
*** MarginHu has quit IRC09:55
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Fix ERROR barbican.plugin can't find /usr/lib/libCryptoki2_64.so  https://review.openstack.org/44456109:56
zhubingbingegonzalez done09:56
duonghqzhubingbing, you have not resolved 2nd commend of egonzalez09:58
duonghq*comment09:58
*** deadnull has quit IRC09:58
zhubingbingi don't know how to solve it10:00
duonghqjust move those lines down to Barbican section ;)10:00
*** cuongnv has quit IRC10:00
zhubingbingok10:01
duonghqjust above crypto_plugin line10:01
zhubingbing;)10:01
openstackgerritMerged openstack/kolla master: Sahara needs to run ssh-keygen  https://review.openstack.org/43686910:01
egonzalezbtw zhubingbing fix the regex issue in cleanup-containers if you want ;)10:02
*** caowei has joined #openstack-kolla10:02
zhubingbingi have talk with MarginHu10:03
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Fix ERROR barbican.plugin can't find /usr/lib/libCryptoki2_64.so  https://review.openstack.org/44456110:03
zhubingbingmaybe  he Is his fault10:03
*** pbourke has joined #openstack-kolla10:03
*** satyar has quit IRC10:04
duonghqzhubingbing, so if he can confirm here, we can close this bug10:06
duonghqI think regex issue is separate bug10:06
egonzalezduonghq, yep, is different issue10:07
*** mnasiadka has joined #openstack-kolla10:07
*** rmart04 has joined #openstack-kolla10:16
*** zhurong has quit IRC10:21
*** zhurong has joined #openstack-kolla10:22
*** nathharp has joined #openstack-kolla10:25
*** nathharp has quit IRC10:26
*** rmart04 has quit IRC10:28
*** duonghq has quit IRC10:29
*** tovin07_ has quit IRC10:29
zhubingbingyes <duonghq10:33
zhubingbingyes duonghq10:33
zhubingbingi have confirm  is separate bug10:34
zhubingbing;)10:34
openstackgerritNathan Harper proposed openstack/kolla-ansible master: Nova external-ceph.yml writes keyname into libvirt.base64 Closes-Bug: #1671862 Change-Id: I6b77773238cd65a3cc009d8d9a38bdbea0035212  https://review.openstack.org/44438910:34
openstackbug 1671862 in kolla "Nova external-ceph.yml writes keyname into libvirt .base64 instead of value" [High,Triaged] https://launchpad.net/bugs/1671862 - Assigned to Nathan Harper (nathharp)10:34
kbaegisWhy does neutron suck so badly at deleting ports?10:36
*** caowei has quit IRC10:36
*** caowei has joined #openstack-kolla10:37
kbaegisNeutron is talking dirty database to me, and I'm not into that: There was a conflict when trying to complete your request.10:39
zhubingbinghttps://bugs.launchpad.net/kolla-ansible/+bug/167233410:41
openstackLaunchpad bug 1672334 in kolla-ansible "Fix separate bug in cleanup-containers" [High,Confirmed] - Assigned to zhubingbing (zhubingbing)10:41
zhubingbingcan u look it ?duonghq10:42
*** zhugaoxiao has quit IRC10:43
*** zhugaoxiao has joined #openstack-kolla10:44
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Fix separate bug in cleanup-containers  https://review.openstack.org/44480710:47
openstackgerritDave Walker proposed openstack/kolla-ansible master: init-runonce - set image distro to "linux"  https://review.openstack.org/44480810:49
openstackgerritDave Walker proposed openstack/kolla-ansible master: init-runonce - set image distro to "linux"  https://review.openstack.org/44480810:50
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Fix separate bug in cleanup-containers  https://review.openstack.org/44480710:51
*** matrohon has quit IRC10:57
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Nova oslo_messaging_notifications config error  https://review.openstack.org/44450410:57
*** trinaths1 has quit IRC10:57
*** matrohon has joined #openstack-kolla10:58
*** caowei has quit IRC11:03
egonzalezanyone have ever configured two VIP in the same keepalived interface?11:12
egonzalezlets say, internal_vip and external_vip in the same interface with different VIP addresses11:12
*** matrohon has quit IRC11:21
*** ccesario__ has quit IRC11:22
*** matrohon has joined #openstack-kolla11:23
*** zhubingbing has quit IRC11:24
*** rwallner has joined #openstack-kolla11:27
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ironic-pxe: handle non-x86 architectures  https://review.openstack.org/43481711:30
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: debian: enable all images enabled for Ubuntu  https://review.openstack.org/43278711:30
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Add support for non-x86 architectures (aarch64, ppc64le)  https://review.openstack.org/43094011:30
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: nova-libvirt: handle ppc64le  https://review.openstack.org/43481011:30
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ceph: move ceph-fuse package to cephfs-fuse image  https://review.openstack.org/44063411:30
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: nova-compute: handle rtslib(-fb) for debian-binary builds too  https://review.openstack.org/43594111:30
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: debian: move to stretch  https://review.openstack.org/43445311:30
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: debian: install gnupg and dirmngr needed for apt-key  https://review.openstack.org/43443111:30
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: openstack-base/kolla-toolbox: CentOS has mariadb-devel  https://review.openstack.org/44483211:30
*** furlongm has quit IRC11:37
*** furlongm has joined #openstack-kolla11:38
*** papacz has joined #openstack-kolla11:40
*** rhallisey has joined #openstack-kolla11:41
*** trinaths has joined #openstack-kolla11:41
*** yingjun has joined #openstack-kolla11:51
*** erlon has joined #openstack-kolla11:51
*** sudipto has quit IRC11:56
*** rwellum has joined #openstack-kolla11:59
*** matrohon has quit IRC12:03
*** matrohon has joined #openstack-kolla12:05
*** trinaths has quit IRC12:06
*** yingjun has quit IRC12:11
*** athomas has quit IRC12:12
*** matrohon has quit IRC12:13
*** saneax is now known as saneax-_-|AFK12:14
*** matrohon has joined #openstack-kolla12:16
*** manheim has quit IRC12:19
*** DuncanT has joined #openstack-kolla12:26
*** athomas has joined #openstack-kolla12:26
*** lamt has quit IRC12:28
*** jheroux has joined #openstack-kolla12:35
*** zhubingbing has joined #openstack-kolla12:38
*** spsurya__ has joined #openstack-kolla12:43
*** zhubingbing__ has joined #openstack-kolla12:43
*** zhubingbing has quit IRC12:44
*** salv-orlando has quit IRC12:50
*** manheim has joined #openstack-kolla12:57
*** yingjun has joined #openstack-kolla13:01
*** zhurong has quit IRC13:01
*** matrohon has quit IRC13:04
*** matrohon has joined #openstack-kolla13:06
*** matrohon has quit IRC13:07
*** matrohon has joined #openstack-kolla13:07
*** lamt has joined #openstack-kolla13:09
*** brice has joined #openstack-kolla13:12
hrwyo13:12
hrwwho can tell me more about DELOREAN?13:12
*** sudipto has joined #openstack-kolla13:13
hrwand why it is buried in kolla/common/config.py instead of being in docker/base/Dockerfile.j2?13:13
*** jtriley has joined #openstack-kolla13:14
*** brice has quit IRC13:14
egonzalezhrw, to make it configurable at kolla-build.conf13:14
hrwegonzalez: I am trying to find a way to make it arch dependend13:15
*** salv-orlando has joined #openstack-kolla13:15
hrwnow it is forces for all rpm builds and brakes !x86-64 ones13:15
egonzalezhrw, https://github.com/openstack/kolla/blob/master/kolla/common/config.py#L228 is used as default setup_rpm_config13:18
egonzalezin stable branch that default is removed13:18
*** jtriley has quit IRC13:19
hrwegonzalez: I will test changing DELOREAN(_OPTS) from string to set and using porper set depending on arch13:19
egonzalezhrw, makes sense13:19
*** g3ek has quit IRC13:20
*** jtriley has joined #openstack-kolla13:20
*** schwicht has joined #openstack-kolla13:22
*** skramaja has quit IRC13:23
*** salv-orl_ has joined #openstack-kolla13:24
*** gfidente^2nd has joined #openstack-kolla13:25
*** salv-orlando has quit IRC13:26
*** gfidente has quit IRC13:27
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Neutron oslo_messaging_notifications config error  https://review.openstack.org/44450613:29
*** g3ek has joined #openstack-kolla13:29
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ironic-pxe: handle non-x86 architectures  https://review.openstack.org/43481713:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: debian: enable all images enabled for Ubuntu  https://review.openstack.org/43278713:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Add support for non-x86 architectures (aarch64, ppc64le)  https://review.openstack.org/43094013:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: openstack-base/kolla-toolbox: CentOS has mariadb-devel  https://review.openstack.org/44483213:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: nova-libvirt: handle ppc64le  https://review.openstack.org/43481013:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ceph: move ceph-fuse package to cephfs-fuse image  https://review.openstack.org/44063413:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: nova-compute: handle rtslib(-fb) for debian-binary builds too  https://review.openstack.org/43594113:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: debian: move to stretch  https://review.openstack.org/43445313:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: debian: install gnupg and dirmngr needed for apt-key  https://review.openstack.org/43443113:32
hrwsorry for flood but that set got quite integrated to each other13:33
hrwdocker/base/Dockerfile.j2 is a formatting mess ;(13:35
hrwbut reformatting it in my patch would add more work for reviewers. refomatting it before my patch, submitting and then rebasing my set on it would not be better13:36
*** eaguilar has joined #openstack-kolla13:40
*** gfidente^2nd is now known as gfidente13:41
*** eaguilar has quit IRC13:45
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Replace unconditional wait for compute kit  https://review.openstack.org/44461313:46
*** srwilkers has joined #openstack-kolla13:46
*** mdnadeem has quit IRC13:46
*** sudipto has quit IRC13:50
*** salv-orl_ has quit IRC13:51
srwilkersgood morning13:51
hrwuf. finally centos/aarch64 builds are going13:52
*** Jezekus has joined #openstack-kolla14:00
*** lucasxu has joined #openstack-kolla14:00
*** yingjun has quit IRC14:02
*** manheim has quit IRC14:09
*** sudipto has joined #openstack-kolla14:10
*** Jeffrey4l has joined #openstack-kolla14:12
*** sayantan_ has joined #openstack-kolla14:15
*** spsurya__ has quit IRC14:15
*** Jezekus has quit IRC14:16
*** Jeffrey4l has quit IRC14:17
*** manheim has joined #openstack-kolla14:18
*** Jeffrey4l has joined #openstack-kolla14:18
*** kbaegis has quit IRC14:26
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Designate: manage Api and Central service workers  https://review.openstack.org/44492914:28
*** eaguilar has joined #openstack-kolla14:29
*** satyar has joined #openstack-kolla14:33
*** kbaegis has joined #openstack-kolla14:35
*** dave-mccowan has joined #openstack-kolla14:35
*** kbaegis has quit IRC14:37
egonzalezis aodh required for ceilometer alarms? without aodh are not working for me14:38
Guest35019egonzalez: hey, qq do you guys see gate failure on centos?14:42
*** Guest35019 is now known as sbezverk14:42
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Designate: manage Api and Central service workers  https://review.openstack.org/44492914:43
egonzalezsbezverk, any error in special? i've seen one today with oslo log not iterable object or something like that, after recheck issue was gone14:43
sbezverkegonzalez: http://paste.openstack.org/show/602502/14:46
sbezverklooks like bindep is failing..14:46
sbezverkegonzalez: it looks to me as infra issue but wanted to confirm if you see the same before going and complaining on infra channel ;)14:48
*** sbezverk has quit IRC14:50
*** mkoderer has joined #openstack-kolla14:53
*** manheim has quit IRC15:01
*** manheim has joined #openstack-kolla15:02
*** manheim has quit IRC15:03
*** sayantan_ has quit IRC15:05
*** adrian_otto has joined #openstack-kolla15:07
*** sayantan_ has joined #openstack-kolla15:09
*** ipsecguy_ has joined #openstack-kolla15:10
*** ipsecguy has quit IRC15:11
*** eaguilar_ has joined #openstack-kolla15:12
*** eaguilar has quit IRC15:14
*** sayantan_ has quit IRC15:16
*** yingjun has joined #openstack-kolla15:17
*** gfhellma has quit IRC15:21
*** hieulq_ has joined #openstack-kolla15:26
*** zhubingbing__ has quit IRC15:27
*** satyar has quit IRC15:28
*** yingjun has quit IRC15:33
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix heat ec2 keystone auth  https://review.openstack.org/44495315:36
egonzalezk8s guys, can anyone check if this ^^ also affect you too?15:37
kfox1111morning.15:39
kfox1111egonzalez: interesting. the v3 is required for it to function properly?15:40
*** eaguilar_ has quit IRC15:40
egonzalezkfox1111, yep, at least in ansible was not working without v3, heat was not able to create a valid ec2 auth in keystone15:40
kfox1111ok. then the k8s one should have the same fix.15:41
kfox1111we haven't tested that too much yet.15:41
*** sbezverk has joined #openstack-kolla15:44
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix heat ec2 keystone auth  https://review.openstack.org/44495315:44
*** mnasiadka has quit IRC15:47
hrwwho is using kolla to build centos based images?15:49
mnaserhrw we dont build but we consume the tagged ones15:49
*** fooliouno has joined #openstack-kolla15:50
hrwok15:50
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Add cpu_mode in quickstart guide  https://review.openstack.org/44495815:51
*** crushil has joined #openstack-kolla15:59
mnaserhas anyone used kolla to scale a control plane up (more specifically my biggest concern is how scaling of mariadb is handled)16:01
kfox1111mnaser: mariadb doesn't really scale.16:02
kfox1111all you can do is manually shard.16:03
kfox1111or throw bigger iron it it.16:03
mnaserkfox1111: well maybe scale was the wrong term but adding new nodes to the galera cluster db16:03
kfox1111more then 3?16:03
mnaserso if i have 1 controller and later decide to go up to 3 for ha16:03
kfox1111ah.16:03
kfox1111haven't tried that. :/16:03
mnaserit would be nice to have small deployments start with 1 node and still be able to go u pto 316:06
*** srwilkers has left #openstack-kolla16:07
*** srwilkers has joined #openstack-kolla16:07
*** imcsk8 has quit IRC16:07
*** imcsk8 has joined #openstack-kolla16:09
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Add panko.conf file in ceilometer container  https://review.openstack.org/44471916:09
SamYaplemnaser: are you talking about kolla-ansible?16:09
mnaseryup SamYaple16:09
SamYaplemnaser: i wrote it to go from 1-n number of nodes, yes16:10
SamYapleit scales up just fine16:10
SamYaplescaling down is trickier and requires a bit of manual work16:10
mnaserSamYaple: yeah that's not something that we want to get into, but getting the ability of scaling up is perfect16:10
mnaseri assume its as straight forward as adding it to the inventory and redeploying?16:10
SamYapleyep16:10
mnaserthat's great to hear16:11
*** kbaegis has joined #openstack-kolla16:12
fooliounoHello .. I am trying to bring up mariadb in kolla-kubernetes. It works in a single node configuration, but when I try to start mariadb on a worker node, I get this error: http://pastebin.com/b9qSPjzj16:13
fooliounoAny ideas on what might be wrong16:14
*** david-lyle has joined #openstack-kolla16:15
inc0good morning16:23
hrwis there a way to inject time into each INFO: line from kolla output?16:29
*** gfhellma has joined #openstack-kolla16:32
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Designate: manage Api and Central services workers  https://review.openstack.org/44492916:33
*** matrohon has quit IRC16:33
mnaserhrw you could probably edit this - https://github.com/openstack/kolla/blob/master/docker/base/set_configs.py16:35
*** salv-orlando has joined #openstack-kolla16:35
mnaserhowever, docker can print timestamps natively16:35
*** hieulq_ has quit IRC16:35
mnaserhrw docker logs -t <container>16:35
hrwmnaser: "INFO:kolla.image.build.gnocchi-base:Cleaning up everything" is still kolla imho ;) will check set_configs16:36
mnaserooo16:36
mnaseri was thinking the docker start up things16:36
hrw;)16:36
mnaserif it uses oslo_log it could be easy16:36
hrwmnaser: so far I do not yet care of how docker runs but rather about how kolla builds16:37
mnaseryeah it looks like it uses native python logging so https://github.com/openstack/kolla/blob/master/kolla/image/build.py#L56-L7716:37
hrwyep16:38
*** pcaruana has quit IRC16:38
*** rwsu has joined #openstack-kolla16:40
*** zioproto has quit IRC16:41
rwellumping sdake16:43
*** kbaegis has quit IRC16:49
*** sayantan_ has joined #openstack-kolla16:51
*** Jezekus has joined #openstack-kolla16:54
*** kbaegis has joined #openstack-kolla17:00
*** Mannheim has quit IRC17:01
*** kbaegis has quit IRC17:03
*** lucasxu has quit IRC17:03
*** Jezekus has quit IRC17:05
*** rabel_b1 has joined #openstack-kolla17:06
*** Jezekus has joined #openstack-kolla17:06
*** egonzalez has quit IRC17:08
*** rabel has quit IRC17:09
*** kbaegis has joined #openstack-kolla17:10
*** gfidente is now known as gfidente|afk17:15
rwellumI'm trying kolla-kubernetes, and seeing this issue: https://review.openstack.org/#/c/417331/ - which was fixed a long time ago. My full output is: http://paste.openstack.org/show/602537/17:17
*** openstackgerrit has quit IRC17:18
*** sudipto has quit IRC17:19
inc0rwellum: are you using stable kolla/ansible or master?17:19
*** Jezekus has quit IRC17:21
rwellummaster I think: git clone http://github.com/openstack/kolla-kubernetes17:22
inc0how about kolla-ansible?17:22
inc0for config gen17:22
rwellumSame: I am following: https://etherpad.openstack.org/p/kolla-kubernetes-deploy-guide-BP17:23
rwellumgit clone http://github.com/openstack/kolla-ansible17:23
*** unicell has joined #openstack-kolla17:25
*** unicell has quit IRC17:25
*** unicell has joined #openstack-kolla17:25
*** Rowan_ has joined #openstack-kolla17:26
Rowan_Hi, am trying to deploy newton using Kolla and when I deploy with default config it is deploy with flat networks.17:27
inc0Rowan_: well, yeah17:28
Rowan_I want to deploy using vlans are there parameters we can set to change it to vlans and configure our own bridges?17:28
inc0that and tunnels17:28
inc0as for "configure our own bridges" depends how much you want to configure them a17:29
inc0for vlans, hold on17:29
*** lucasxu has joined #openstack-kolla17:30
inc0//docs.openstack.org/developer/kolla-ansible/advanced-configuration.html#openstack-service-configuration-in-kolla17:31
inc0https://docs.openstack.org/developer/kolla-ansible/advanced-configuration.html#openstack-service-configuration-in-kolla17:31
inc0use this mechanism17:31
inc0to override paste.ini and provide vlan ranges17:31
inc0we should probably document how to do that17:32
inc0I'll submit bp17:32
Rowan_I tried it node_custom_config: "/etc/kolla/config" but no deploying with the changes17:32
Rowan_not*17:32
Rowan_ex: am trying to change ml2_conf.ini so created /etc/kolla/config/neutron/plugins/ml2/ml2_conf.ini and reconfigured the env17:33
*** mgoddard has quit IRC17:33
Rowan_but nodes are not getting new config17:34
inc0https://docs.openstack.org/developer/kolla-ansible/advanced-configuration.html#openstack-service-configuration-in-kolla17:35
inc0sry17:35
inc0wrong link17:35
inc0https://github.com/openstack/kolla/blob/stable/newton/ansible/roles/neutron/tasks/config.yml#L99-L10117:35
inc0correct place for ml2 would be /etc/kolla/config/neutron/ml2_conf.ini17:36
inc0Rowan_: also submitted this https://blueprints.launchpad.net/kolla-ansible/+spec/docs-config-examples17:37
Rowan_Thanks inc0 really appreciate your help and timely response.17:37
inc0everyone, feel free to add comments about other use-cases ^17:37
inc0Rowan_: np:)17:37
inc0let me know if this helps you17:37
rwelluminc0, so should I not be using master?17:38
inc0reason I ask is that I don't think kolla-ansible has this change backported to stable/ocata17:38
Rowan_Will give a try and update you about vlans config17:38
inc0so if you'd use stable/ocata, it might not have code17:39
inc0but I'll confirm that17:39
rwelluminc0, I checked and I see the code.17:39
hogepodgeugh, realized in organizing this meeting that daylight saving happened over the weekend17:39
rwellumin my clone17:39
inc0ahh nvm then17:39
inc0bug is still open about that in kolla-k8s, I'll check it out when my brain starts properly17:39
inc0hogepodge: yeah.... it's always problem17:40
inc0we can place bets on how many people will miss this weeks meeting17:40
inc0because of dst17:40
hogepodgegoing to stick with 1830 UTC and we can work out scheduling17:40
hogepodgedst is the worst17:40
hogepodgeI'll probably go back and delete all my standing meetings and resubscribe to make sure my calendar didn't do the wrong thing17:41
inc0I wonder if someone actually made research to determine how much time we *lose* because of dst17:41
inc0all the meetings without quorum and such17:41
jascott1DST is about money17:42
jascott1more shopping hours17:43
*** salv-orlando has quit IRC17:43
*** salv-orlando has joined #openstack-kolla17:43
inc0how does that help? besides making everyone sleepy and not willing to leave home17:43
jascott1haha no doubt17:43
inc0(yes, I'm working from home and thinking of taking a nap)17:44
*** eaguilar has joined #openstack-kolla17:44
jascott1well in the winter you get more daylight, which more time to shop17:44
inc0I mean I understand why that was an important thing *before* we had street lights everywhere17:44
-openstackstatus- NOTICE: restarting gerrit to address performance problems17:44
inc0btw kolla was oficially released17:46
inc0all glory to Ocata17:46
jascott1inc0  "In 1984, Fortune magazine estimated that a seven-week extension of DST would yield an additional $30 million for 7-Eleven stores, and the National Golf Foundation estimated the extension would increase golf industry revenues $200 million to $300 million."17:49
jascott1https://en.wikipedia.org/wiki/Daylight_saving_time17:49
rwelluminc0, line 136 https://etherpad.openstack.org/p/kolla-kubernetes-deploy-guide-BP - seems to suggest that kolla is up and running - but it's not afaik. Does this seem wrong to you too?17:50
*** rabel_b1 has quit IRC17:51
*** manheim has joined #openstack-kolla17:54
*** adrian_otto has quit IRC17:57
*** eaguilar_ has joined #openstack-kolla17:58
*** eaguilar has quit IRC18:00
*** Jezekus has joined #openstack-kolla18:01
*** adrian_otto has joined #openstack-kolla18:02
inc0rwellum: adding labels to nodes?18:04
rwelluminc0, yeah because at this point in guide - we don't have nodes I think.18:05
inc0rwellum: so labels are meant to tell k8s where deploy what18:06
inc0some services can be just scheduled k8s style18:06
*** crushil has left #openstack-kolla18:06
inc0but some, like compute nodes should be deployed in specific nodes18:06
portdirectrwellum: I'm starting to work through the guide and will be running a deployment tonight, but that line looks like its in the right place18:06
inc0it's for k8s more than it's for kolla itself18:06
portdirectinc0's explanation is spot on - its just to make the nodes that kolla pods will run on18:07
portdirects/make/mark18:07
inc0yeah I still fight the initial k8s deploy:/18:07
inc0halcyon hates me18:07
portdirect:( thats not good inc0, if theres any way I can help just give me a holler18:08
inc0portdirect: well, issue I'm dealing with is after I deploy k8s, I run tiller and it fails liveness probe:/18:08
inc0imho it fails to connect to underlying k8s18:08
inc0(no logs available)18:09
rwellum'minime-02' is the node according to that guid portdirect, inc0 - that doesn't make too much sense to me18:09
rwellumNow: ALLINONENODE=$(hostname)18:09
rwellumkubectl label node $ALLINONENODE kolla_compute=true18:09
portdirectinc0: does the dns test work?18:09
inc0rwellum: minime-02 is sdake's node, it's just hostname18:09
rwellumthat makes more sense18:09
rwellumright...18:09
inc0portdirect: let me rebuild whole env and start fresh18:10
inc0I'm using flannel btw18:10
rwellumportdirect, great that you're looking at this now.18:10
portdirectyeah - just relocated to the States, glad to be getting ramped back up now :D18:10
inc0oh did you?18:11
inc0cool, congrats18:11
inc0now I won't have any second thoughts to ping you18:11
inc0I always felt a little guilty before because of late night thing18:11
* portdirect moves back to UK :P18:11
sbezverkinc0: has infra fixed broken centos mirror?18:11
portdirectyeah -  I'm really looking forward o working normal hours again :)18:12
inc0I had no idea about centos broken mirror in the first place;)18:12
SamYapleportdirect: define normal18:12
sbezverkinc0: yeah this morning all centos jobs were failing18:12
SamYapleinc0: sounds like dns!18:12
*** Jezekus has quit IRC18:12
portdirect6am to 5:59am SamYaple18:12
SamYapleportdirect: whats an am?18:13
inc0anno morningum?18:13
inc0(that was bad...)18:13
inc0I hate dst18:13
SamYaplejust ignore it18:15
SamYapleitll go away18:15
*** MasterOfBugs has joined #openstack-kolla18:16
*** adrian_otto has quit IRC18:17
*** adrian_otto has joined #openstack-kolla18:19
*** Mannheim has joined #openstack-kolla18:21
*** mgoddard has joined #openstack-kolla18:21
jascott1portdirect which state did you end up in?18:23
portdirectmissouri, now I can throw things at srwilkers :)18:24
jascott1excellent!18:24
srwilkersportdirect, that would require you to have sufficient aim18:24
jascott1http://i.imgur.com/eCkyv.jpg18:25
portdirectI'll just use larger objects to compensate18:25
inc0jascott1: we should get a nerf gun18:26
inc0with API ideally18:26
jascott1i think our neighbors would revolt18:26
jascott1haha18:26
jascott1the guy between our cubes would be prob be quite pissed18:27
inc0well, they'll already know rules of engagement18:27
inc0idea for project18:27
portdirectjascott1: "collateral damage" they should know the risks18:27
jascott1maybe those that are like a mortar where we can get the trajectory dialed in18:27
inc0nerfgun to automatically shoot down nerfs18:27
jascott1lol18:27
jascott1nerf drone18:27
inc0computer vision+tons and tons of algebra to figure out ballistics18:28
jascott1nerf to shoot down nerf would be very impressive18:28
inc0probably couple hundreds of hours writing code that will do something totally useless18:28
jascott1then 2 nerfs would be raining down on Brian each time18:29
inc0not to mention hardware part18:29
jascott1it would prob get confused and just shoot him everytime he stook up ;)18:29
inc0I mean you need some really quick and precise servos out there18:29
jascott1stook/stood18:29
inc0or another idea18:29
inc0drone with nerfgun (as you mentioned)18:30
inc0but with autonomus flying and infrared target painting18:30
inc0so you can point invisible laser at someone at the outher side of openspace18:30
jascott1heheh18:30
inc0then we gradually extend range and after a while have outdoor drone for random pedestrians;)18:31
*** fooliouno has quit IRC18:31
inc0nerf headshot from 300m away would be impressive;_)18:31
inc0hmm...but that's not a bad idea for things like video18:32
inc0have drone with some pre-coded flights patterns to record video of thing you point invisible laser on18:33
inc0like idk, music band on stage18:33
jascott1Intel has already demoed a drone that can follow a moving target18:33
inc0yeah, I saw few ideas of that18:33
inc0but having target painter would be good, or invisible led18:34
inc0like I could install this thing on my paraglider and have drone flying around me taping me18:34
jascott1yeah that seems like a great idea18:34
inc0pretty fun gopro appliance idea, who wants to start a startup?:P18:35
jascott1my wife says im not allowed to play with startups anymore18:35
jascott1:D18:35
inc0:(18:35
*** sayantan_ has quit IRC18:35
inc0yeah you have morgage now18:35
jascott1only 360 easy monthly payments left!18:36
SamYaplejascott1: just do ten at a time18:36
jascott1now why didnt I think of that18:37
SamYapletoo many startups18:37
inc0or even better, one startup, sell it to intel, pay off house and still have 14.993 bil $ left18:37
jascott1i have been looking into paying down extra on principal, something like paying extra $400 mo will save me about 8+ years18:38
SamYaplejascott1: doubling payments on a 30 year pays it off in 7 years18:38
portdirectinc0 you can act on intels behalf here? My patent for cloud lego just came through this morning.18:38
portdirectjascott1: you snooze, you loose.18:38
jascott1portdirect >:(18:39
inc0portdirect: cloud lego?18:39
portdirect100% my idea18:39
inc0who uses cloud any more? cloud is passe18:39
jascott1Missouri huh18:39
portdirecthybrid cloud lego?18:39
Rowan_inc0 sorry to interrupt you.18:39
jascott1(and your important conversation)18:39
inc0Rowan_: it's not like you interrupted anything constructive;)18:39
Rowan_Deployed with vlans and having issues with networking18:40
inc0aren't we all... whats happening?18:40
Rowan_while deploying instances with vlans network unable to reach metadata18:40
Rowan_Lease of 172.24.51.54 obtained, lease time 86400 route: SIOCADDRT: File exists WARN: failed: route add -net "0.0.0.0/0" gw "172.24.51.1" cirros-ds 'net' up at 0.93 checking http://169.254.169.254/2009-04-04/instance-id failed 1/20: up 0.94. request failed18:41
SamYapleRowan_: its just the vlan networks that don't have metadata?18:41
SamYapleso you can do a vxlan with no problem?18:41
Rowan_With vxlan no issues18:41
SamYapleare the vlan networks behind neutron routers?18:41
Rowan_but atatching float IP to vxlan instances not working18:41
Rowan_nope for vlans I configured default gw18:42
SamYapleso its an isolated subnet18:43
SamYapleyou need to enable metadata on isolated subnets18:43
Rowan_am able to ping the vlan instances but metadata failing18:43
SamYapleor, alternatively, setup a metadata network18:43
inc0ok portdirect sooo....let's start with k8s debugging if you have a  moment:)18:44
portdirectnp around for 10-1518:45
*** sayantan_ has joined #openstack-kolla18:45
portdirectyou run the container here? https://docs.openstack.org/developer/kolla-kubernetes/development-environment.html#testing-the-deployed-environment18:45
inc0http://paste.ubuntu.com/24172091/18:45
Rowan_Samyaple it is not isloated subnet subnet gateway_ip enabled18:48
*** manheim has quit IRC18:48
inc0portdirect: http://paste.openstack.org/show/602551/ <- dns seems fine, but 10.96.0.10 doesn;t18:49
SamYapleRowan_: if its not connected to a neutron router it counts as isolated18:49
Rowan_ok18:51
Rowan_how to enable metadata on isolated subnets?18:54
Rowan_when I check dhcp namespace it has default to metadata18:55
SamYapledhcp agent, option enable_isolated_metadata=True18:55
Rowan_default via 172.24.51.1 dev tap28fe3684-3b 169.254.0.0/16 dev tap28fe3684-3b  proto kernel  scope link  src 169.254.169.254 172.24.51.0/24 dev tap28fe3684-3b  proto kernel  scope link  src 172.24.51.5018:55
*** kbaegis has quit IRC18:55
SamYaplemetadata, by default, is handled in the neutron router namespace18:56
SamYapleyou do not have a neutron router18:56
SamYapleyou have to tell the instance how to handle metadata now18:57
SamYaplein this case, through the dhcp agent. but you need to tell the dhcp agent that18:57
Rowan_enable_isolated_metadata is set to true by default in my env18:58
SamYapleright. so there is another piece to this when you also have an external gateway19:00
SamYapleyou have to remove the gateway_ip and set it as a host route19:00
SamYapleso you need to set an additional route of 0.0.0.0/0 via 172.24.51.119:00
SamYapleand remove the gateway_ip from the subnet for the metadata to work19:00
Rowan_so I need to remove gw for the vlan?19:00
SamYapleyea19:00
inc0yeah, pretty much if your gateway is external to cloud, nothing will redirect to metadata - it's iptables rule on routers normally19:00
Rowan_Will try now19:01
SamYapleinc0: no thats not true19:01
inc0:(19:01
SamYapleinc0: it still works19:01
SamYapledhcp injects a route19:01
SamYaplebut it only does that when gateway_ip is not set19:01
SamYapleso to get internet connectivity and metadata you ahve to inject teh default route19:01
inc0ahh, right19:01
SamYapleyou end up with 169.254.169.254/32 pointed to dhcp server and 0.0.0.0/0 pointed to the gateway19:02
Rowan_no luck19:03
Rowan_disable gw added host route19:04
Rowan_WARN: failed: route add -net "0.0.0.0/0" gw "172.24.51.1" cirros-ds 'net' up at 0.89 checking http://169.254.169.254/2009-04-04/instance-id failed 1/20: up 0.89. request failed failed 2/20: up 12.93. request failed19:04
inc0Rowan_: tcpdump time?19:04
*** athomas has quit IRC19:04
*** zhugaoxiao has quit IRC19:04
SamYapleyou removed gateway_ip from neutron subnet, restarted the agents, restarted the instance and still no go?19:04
inc0check qdhcp ns if it gets traffic from cirros19:04
*** zhugaoxiao has joined #openstack-kolla19:05
*** Jeffrey4l has quit IRC19:05
*** Jeffrey4l has joined #openstack-kolla19:05
inc0Jeffrey4l: in case you're awake, we have oficially released Kolla:)19:05
Rowan_able to ping instance from qdhcp19:06
inc0check if qdhcp gets traffic if you try http://169...19:07
inc0and if it doesn, what happends next19:07
*** eaguilar_ has quit IRC19:07
inc0check also iptables -t nat (afair) for metadata rule19:07
Rowan_sorry no unable to ping instance from qhdcp19:08
*** eaguilar has joined #openstack-kolla19:17
*** Mannheim has quit IRC19:20
*** salv-orl_ has joined #openstack-kolla19:24
*** salv-orlando has quit IRC19:27
*** openstackgerrit has joined #openstack-kolla19:32
*** mgoddard has quit IRC19:41
hrwuf. looks like centos/source build of gnocchi-base will succeed19:44
*** salv-orl_ has quit IRC19:45
hrwlooks like no one built that or it worked magically19:46
inc0hrw: I am pretty sure we build it in gates;)19:47
hrwinc0: than it is magic19:48
hrws/than/then19:48
hrwinc0: without adding two devel packages it fails to build scipy from source19:49
hrwhm. same for debian/source19:50
hrwhm. on x86-64 it built for centos, debian, ubuntu.19:50
hrwsuspect some weird stuff in repos or sth19:51
hrwno. pip fetches binary version of scipy on x86-6419:52
*** sgordon` is now known as sgordon19:52
inc0there you go:)19:52
inc0I thought it's wierd because I routinelly build whole stack of ubuntu containers19:53
inc0and I know others do the same for centos19:54
inc0not sure about debian19:54
hrwno one does debian19:54
hrwotherwise it would work ;d19:54
inc0well, we need gates19:54
inc0for debian19:55
inc0otherwise we can't ensure19:55
inc0it works19:55
hrwsure19:55
hrwgnocchi container images will be a bit larger then19:55
*** jmccarthy has joined #openstack-kolla19:56
hrwand take long time...19:57
hrwINFO:kolla.image.build.gnocchi-base:    Running setup.py install for scipy: still running...19:57
inc0yeah I remember that in nova some time ago too19:57
inc0not for scipy tho19:58
hrwon x86-64 pip fetches precompiled python packages often19:58
hrwbtw... my stack of patches gets longer and longer while some of them could be merged individually. should I split them out?19:59
inc0hrw: it's always easier to break down work to multiple patches and try to limit cross-dependency between them20:00
*** Jeffrey4l has quit IRC20:00
inc0btw, you're Marcin?;)20:00
hrwyes, I am20:01
hrw /whois hrw helps20:01
inc0hej, Michal z tej strony;)20:01
*** Jeffrey4l has joined #openstack-kolla20:01
hrwinc0: ;D20:02
inc0ok, yeah, I never really used that irc wizardry20:02
hrwinc0: will you be at OpenStack Day in Warsaw on 22nd March?20:02
*** gfidente|afk has quit IRC20:02
hrwinc0: I do irc for over 20 years ;D20:02
inc0a bit long flight for me - I live in west coast now:)20:02
hrw;D20:02
hrwinc0: I also in West... but West North corner of Poland: Szczecin ;d20:03
hrwINFO:kolla.image.build.gnocchi-base:    Running setup.py install for scipy: finished with status 'done'20:03
hrwwill try to split out some patches tomorrow for easier merging20:04
inc0I'm from Wroclaw originally20:04
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: openstack-base/kolla-toolbox: CentOS has mariadb-devel  https://review.openstack.org/44483220:05
hrwinc0: lived there for ~year20:05
*** jheroux has quit IRC20:05
inc0best city in Europe;) Portland, where I live now, has pretty similar vibe20:06
inc0also lots of bridges and hipsters;)20:06
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: debian: install gnupg and dirmngr needed for apt-key  https://review.openstack.org/43443120:06
*** rhallisey has quit IRC20:08
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ironic-pxe: handle non-x86 architectures  https://review.openstack.org/43481720:09
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: nova-compute: handle rtslib(-fb) for debian-binary builds too  https://review.openstack.org/43594120:10
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: ceph: move ceph-fuse package to cephfs-fuse image  https://review.openstack.org/44063420:12
hrwnow those patches are not depending on other ones - should be easier20:12
hrwlet's see how jenkins will fail now20:13
*** rabel has joined #openstack-kolla20:20
*** rabel_b1 has joined #openstack-kolla20:21
*** 07EAAIRVF has joined #openstack-kolla20:23
*** jtriley has quit IRC20:24
*** rabel has quit IRC20:24
*** adrian_otto has quit IRC20:27
*** jmccarthy has quit IRC20:28
*** fooliouno has joined #openstack-kolla20:41
fooliounoHello .. kolla-kubernetes documentation has a step to create bootstrap for glance-endpoints, but the jinja2 file it is missing. Is this a bug20:43
*** jmccarthy has joined #openstack-kolla20:45
*** rwallner has quit IRC20:47
*** rwallner has joined #openstack-kolla20:49
*** rabel_b1 has quit IRC20:51
*** 07EAAIRVF has quit IRC20:52
*** rwallner has quit IRC20:53
*** eaguilar has quit IRC20:54
*** satyar has joined #openstack-kolla20:59
*** gfhellma has quit IRC21:00
*** jmccarthy has quit IRC21:02
Rowan_Is there any wayt to rebuild the faulty container with existing config?21:07
*** Rowan_ has quit IRC21:14
*** sayantan_ has quit IRC21:15
*** sayantan_ has joined #openstack-kolla21:20
*** srwilkers has quit IRC21:22
*** salv-orlando has joined #openstack-kolla21:30
*** Jeffrey4l has quit IRC21:35
*** satyar has quit IRC21:35
*** Jeffrey4l has joined #openstack-kolla21:35
*** fooliouno has quit IRC21:36
*** papacz has quit IRC21:46
*** Mannheim has joined #openstack-kolla21:49
*** jrobinson has joined #openstack-kolla21:55
*** schwicht has quit IRC21:56
*** rwallner has joined #openstack-kolla21:59
*** Serlex has quit IRC21:59
*** rwallner has quit IRC22:03
sbezverkfooliouno: we are not using jinj2 templates any longer, please consider using helm charts.22:11
*** manheim has joined #openstack-kolla22:30
*** Mannheim has quit IRC22:33
*** dave-mccowan has quit IRC22:36
inc0yay I thin I know whats wrong with my k8s22:48
inc0since it timeouts on 443 and not secure connection works22:49
inc0my question is22:49
inc0how exactly kube passes self signed CA certs to containers?22:49
SamYapleinc0: is that a question in general or a kolla-kubernetes question?22:59
inc0general22:59
inc0I deployed halcyon and from within container kube-api-ip:443 is no bueno22:59
*** srwilkers has joined #openstack-kolla23:00
SamYapleim not sure it does pass in self-signed certs in general....23:01
SamYapleproduction deploys are not using self-signed23:01
inc0production deploys have ca authority somewhere?23:02
SamYaplenon-production can easily just ignore and not verify23:02
SamYapleyou can get real certs for all the controllers23:02
SamYaplethats what I do23:02
inc0well wget https://10.0.2.1:443 times out23:02
SamYapletimes out is unrelated to the cert23:03
inc0right....iptables or fragmented vxlans?23:04
SamYapleare you, you know, sure you launched the api with ssl on port 443?23:04
SamYaplei think 8081 is the other "normal" https port for the api23:05
inc0if I wouldn't, that would be connection refused right?23:05
SamYapleno23:05
SamYaplebecause of the fact that ip address doesnt live anywhere23:05
SamYapleits just iptables weirdness23:05
inc0well, 8081 did refuse my connection23:05
SamYapledo random ports refuse or timeout?23:07
inc0refuse23:07
SamYapleit really sounds like you aren't listening properly23:07
SamYaplehow did you launch the api?23:07
SamYaplecli options wise23:08
inc0riiightsume23:10
inc0right23:10
inc0so, you23:10
inc0you're right, it does deploy api with secure port 644323:11
inc0question is, why halcyon doesn't do it properly23:11
SamYaplei was right? man i just guessed for all of that23:13
*** ipsecguy_ has quit IRC23:13
SamYaplemade it all up23:13
SamYapleyay me23:13
*** ipsecguy has joined #openstack-kolla23:13
inc0well23:14
inc0I23:14
inc0I'm nto sure yet, but somethigns funky out there23:14
*** lamt has quit IRC23:15
SamYaplei do know you need to launch with ssl in mind23:16
SamYapleotherwise its all going to be http from the container23:16
SamYapleor pod23:16
SamYapleWHATEVER23:16
*** jrobinson has quit IRC23:18
*** rwellum has quit IRC23:19
*** lucasxu has quit IRC23:19
*** Pavo has joined #openstack-kolla23:20
*** Pavo has quit IRC23:21
jemcevoyinc0 - I have run into another issue with ceilometer.  This time an SSL error during install.  Do you know what the fix might be? http://paste.openstack.org/show/602559/23:22
inc0ssl...23:23
*** rhallisey has joined #openstack-kolla23:23
*** srwilkers has quit IRC23:24
*** erlon has quit IRC23:25
jemcevoyI did the certificate generate thing for a self sighed cert... I plan to setup dogtag CA with FreeIPA but have not finished a real CA... I see that you and SamYaple are discussing ssl issues too23:27
*** japestinho has quit IRC23:30
*** saneax-_-|AFK is now known as saneax23:35
*** sayantan_ has quit IRC23:41
spsuryamorning all23:41
*** sayantan_ has joined #openstack-kolla23:49
sdakejascott1 grats on the house dude ;-)23:53
jascott1hey sdake whats up?23:54
jascott1thanks, yeah we are waiting on the appraisal results but its looking good so far23:54
*** sayantan_ has quit IRC23:54
*** spsurya__ has joined #openstack-kolla23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!