| *** ayoung has joined #openstack-kolla | 00:03 | |
| *** banix has joined #openstack-kolla | 00:09 | |
| *** banix has quit IRC | 00:12 | |
| *** jasonsb has joined #openstack-kolla | 00:21 | |
| *** jasonsb has quit IRC | 00:25 | |
| ccesario_ | sbezverk: ping ? | 00:34 |
|---|---|---|
| *** ntpttr_ has joined #openstack-kolla | 00:43 | |
| *** ntpttr_ has quit IRC | 00:44 | |
| *** ntpttr_ has joined #openstack-kolla | 00:50 | |
| *** banix has joined #openstack-kolla | 00:58 | |
| *** ntpttr_ has quit IRC | 01:02 | |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add etcd Docker container https://review.openstack.org/298451 | 01:10 |
| *** ravig has quit IRC | 01:17 | |
| *** unicell has quit IRC | 01:18 | |
| openstackgerrit | Martin Matyas proposed openstack/kolla: Build logging per container and other logging improvements https://review.openstack.org/306207 | 01:24 |
| *** Slower has quit IRC | 01:26 | |
| *** vhosakot has joined #openstack-kolla | 01:27 | |
| *** Slower has joined #openstack-kolla | 01:37 | |
| ccesario_ | vhosakot: ping ? | 01:38 |
| *** mwheckmann has joined #openstack-kolla | 01:38 | |
| *** ccesario_ has quit IRC | 01:39 | |
| *** vhosakot has quit IRC | 01:39 | |
| *** weiyu_ has joined #openstack-kolla | 01:41 | |
| *** vhosakot has joined #openstack-kolla | 01:43 | |
| *** Marga__ has joined #openstack-kolla | 01:48 | |
| *** ccesario___ has joined #openstack-kolla | 01:51 | |
| *** Marga_ has quit IRC | 01:51 | |
| *** Marga__ has quit IRC | 01:52 | |
| *** weiyu_ has quit IRC | 01:57 | |
| *** ntpttr_ has joined #openstack-kolla | 02:10 | |
| *** weiyu has joined #openstack-kolla | 02:17 | |
| *** rhallisey has quit IRC | 02:34 | |
| *** yuanying has quit IRC | 02:48 | |
| *** ravig has joined #openstack-kolla | 02:49 | |
| *** coolsvap has joined #openstack-kolla | 02:53 | |
| *** ravig has quit IRC | 02:55 | |
| *** weiyu has quit IRC | 02:56 | |
| *** weiyu has joined #openstack-kolla | 02:58 | |
| openstackgerrit | jackning proposed openstack/kolla: Added webroot config to dashboard https://review.openstack.org/306228 | 03:10 |
| *** mwheckmann has quit IRC | 03:14 | |
| *** sdake_ has quit IRC | 03:14 | |
| openstackgerrit | Merged openstack/kolla: cleanup-host should not remove kolla-build.conf https://review.openstack.org/305775 | 03:16 |
| *** salv-orlando has joined #openstack-kolla | 03:16 | |
| *** sdake has joined #openstack-kolla | 03:17 | |
| *** salv-orlando has quit IRC | 03:17 | |
| *** salv-orlando has joined #openstack-kolla | 03:17 | |
| *** salv-orl_ has quit IRC | 03:20 | |
| *** salv-orlando has quit IRC | 03:22 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update base dockerfile for formatting https://review.openstack.org/304783 | 03:24 |
| *** daneyon has joined #openstack-kolla | 03:24 | |
| *** vhosakot_ has joined #openstack-kolla | 03:24 | |
| *** smekel__ has joined #openstack-kolla | 03:26 | |
| *** sdake has quit IRC | 03:27 | |
| *** coolsvap has quit IRC | 03:32 | |
| *** vhosakot has quit IRC | 03:32 | |
| *** daneyon_ has quit IRC | 03:32 | |
| *** smekel_ has quit IRC | 03:32 | |
| *** smekel__ is now known as smekel_ | 03:32 | |
| *** sdake has joined #openstack-kolla | 03:35 | |
| *** coolsvap has joined #openstack-kolla | 03:38 | |
| *** sdake_ has joined #openstack-kolla | 03:39 | |
| *** sdake has quit IRC | 03:40 | |
| sdake_ | note kolla folks expecting an invitation to the core review party, the selection criteria for invitations doesn't incldue our project | 03:45 |
| *** yuanying has joined #openstack-kolla | 03:47 | |
| vhosakot_ | sdake_: ah, didn't know there was a core review party.. and why isn;t kolla included ? | 03:49 |
| sdake_ | all projects not tagged with release:managed were excluded | 03:49 |
| vhosakot_ | sdake_: ah ok | 03:50 |
| *** weiyu has quit IRC | 03:54 | |
| *** ravig has joined #openstack-kolla | 03:55 | |
| *** tfukushima has joined #openstack-kolla | 03:55 | |
| sdake_ | the amusing part is all projects will be tagged with release:managed next cycle | 04:03 |
| sdake_ | so not sure what the new selection criteria will be | 04:03 |
| *** vhosakot_ has quit IRC | 04:04 | |
| coolsvap | oh | 04:08 |
| coolsvap | so the criteria was not applicable last cycle? | 04:08 |
| *** Marga_ has joined #openstack-kolla | 04:16 | |
| *** Marga_ has quit IRC | 04:17 | |
| *** Marga_ has joined #openstack-kolla | 04:18 | |
| *** coolsvap has quit IRC | 04:18 | |
| *** salv-orlando has joined #openstack-kolla | 04:18 | |
| *** coolsvap has joined #openstack-kolla | 04:19 | |
| *** salv-orlando has quit IRC | 04:30 | |
| *** vhosakot has joined #openstack-kolla | 04:33 | |
| openstackgerrit | Merged openstack/kolla: Drop root for swift https://review.openstack.org/306030 | 04:39 |
| *** tfukushima has quit IRC | 04:44 | |
| *** salv-orlando has joined #openstack-kolla | 04:45 | |
| *** salv-orlando has quit IRC | 04:48 | |
| *** vhosakot has quit IRC | 04:55 | |
| *** reddy has joined #openstack-kolla | 05:19 | |
| sdake_ | sbezverk nicee! | 05:21 |
| *** mdnadeem has joined #openstack-kolla | 05:25 | |
| *** mkoderer__ has quit IRC | 05:26 | |
| *** larsks has quit IRC | 05:27 | |
| *** ccesario___ has quit IRC | 05:27 | |
| *** mdnadeem_ has quit IRC | 05:28 | |
| *** banix has quit IRC | 05:28 | |
| *** jmccarthy has quit IRC | 05:28 | |
| *** jtriley has joined #openstack-kolla | 05:28 | |
| *** jmccarthy has joined #openstack-kolla | 05:29 | |
| *** Marga_ has quit IRC | 05:33 | |
| *** mkoderer__ has joined #openstack-kolla | 05:35 | |
| *** larsks has joined #openstack-kolla | 05:36 | |
| *** Marga_ has joined #openstack-kolla | 05:38 | |
| *** tfukushima has joined #openstack-kolla | 05:44 | |
| *** jtriley has quit IRC | 05:48 | |
| *** tfukushima has quit IRC | 05:49 | |
| *** salv-orlando has joined #openstack-kolla | 05:51 | |
| *** coolsvap has quit IRC | 05:54 | |
| *** mikelk has joined #openstack-kolla | 06:00 | |
| *** ravig has quit IRC | 06:02 | |
| *** sdake_ has quit IRC | 06:07 | |
| *** sdake has joined #openstack-kolla | 06:07 | |
| *** tfukushima has joined #openstack-kolla | 06:09 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla: Look for private interface's MAC address in vagrant-hostmanager https://review.openstack.org/306255 | 06:28 |
| *** salv-orlando has quit IRC | 06:31 | |
| *** Marga__ has joined #openstack-kolla | 06:55 | |
| *** ccesario___ has joined #openstack-kolla | 06:57 | |
| *** daneyon has quit IRC | 07:02 | |
| *** mkoderer__ has quit IRC | 07:03 | |
| openstackgerrit | MD NADEEM proposed openstack/kolla: WIP: Added Dockerfile for congress https://review.openstack.org/305626 | 07:03 |
| *** mikelk has quit IRC | 07:04 | |
| *** Marga_ has quit IRC | 07:04 | |
| *** larsks has quit IRC | 07:08 | |
| *** mkoderer__ has joined #openstack-kolla | 07:09 | |
| *** larsks has joined #openstack-kolla | 07:14 | |
| *** mikelk has joined #openstack-kolla | 07:14 | |
| *** Serlex has joined #openstack-kolla | 07:16 | |
| *** rmart04 has joined #openstack-kolla | 07:19 | |
| *** unicell has joined #openstack-kolla | 07:19 | |
| *** achanda has joined #openstack-kolla | 07:20 | |
| *** athomas has joined #openstack-kolla | 07:22 | |
| *** athomas has quit IRC | 07:27 | |
| *** athomas has joined #openstack-kolla | 07:31 | |
| *** Serlex has quit IRC | 07:46 | |
| *** Serlex has joined #openstack-kolla | 07:54 | |
| *** shardy has joined #openstack-kolla | 07:58 | |
| *** achanda has quit IRC | 08:06 | |
| *** openstackgerrit has quit IRC | 08:08 | |
| *** mbound has joined #openstack-kolla | 08:11 | |
| *** b_bezak has joined #openstack-kolla | 08:15 | |
| *** openstackgerrit has joined #openstack-kolla | 08:18 | |
| *** openstackgerrit has quit IRC | 08:18 | |
| *** openstackgerrit has joined #openstack-kolla | 08:18 | |
| *** gfidente has joined #openstack-kolla | 08:38 | |
| *** gfidente has joined #openstack-kolla | 08:38 | |
| *** dims_ has quit IRC | 08:46 | |
| *** dims has joined #openstack-kolla | 08:49 | |
| -openstackstatus- NOTICE: Gerrit is going to be restarted because is not processing new changes | 09:05 | |
| *** salv-orlando has joined #openstack-kolla | 09:05 | |
| *** tfukushima has quit IRC | 09:06 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Enable TLS/SSL for Kibana https://review.openstack.org/306302 | 09:07 |
| *** tfukushima has joined #openstack-kolla | 09:08 | |
| sdake | patch needs more testing ^6 | 09:10 |
| sdake | night all | 09:10 |
| *** sdake has quit IRC | 09:10 | |
| *** salv-orlando has quit IRC | 09:15 | |
| -openstackstatus- NOTICE: No jobs are being processed by gerrit and zuul . We are working to solve the problem, please be aware that no changes have been sent to the queue in the last hour, so you will need to recheck jobs for that period. | 09:20 | |
| *** ChanServ changes topic to "No jobs are being processed by gerrit and zuul . We are working to solve the problem, please be aware that no changes have been sent to the queue in the last hour, so you will need to recheck jobs for that period." | 09:20 | |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Debug oraclelinux deploy gates https://review.openstack.org/300988 | 09:28 |
| Serlex | Hi, anyone seen ""url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [119/120s]: request error "" on a instance console after rebooting the nodes? I must of misconfigured something, but I'm not sure what at the moment | 09:30 |
| Serlex | actually, this isn't a kolla problem, ignore :) | 09:38 |
| *** ccesario___ has quit IRC | 09:38 | |
| *** jmccarthy has left #openstack-kolla | 09:40 | |
| *** stvnoyes has joined #openstack-kolla | 09:41 | |
| *** ccesario___ has joined #openstack-kolla | 09:47 | |
| *** ChanServ changes topic to "Kolla IRC meetngs on Wednesdays @ 16:30 UTC even weeks, 23:00 UTC odd weeks - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA)" | 09:48 | |
| -openstackstatus- NOTICE: Zuul and gerrit are working normally now. Please recheck any jobs that may have been affected by this failure. | 09:49 | |
| *** ccesario___ has quit IRC | 09:59 | |
| *** unicell has quit IRC | 10:03 | |
| *** unicell has joined #openstack-kolla | 10:04 | |
| *** ccesario_ has joined #openstack-kolla | 10:05 | |
| *** salv-orlando has joined #openstack-kolla | 10:06 | |
| ccesario_ | sbezverk: ping ? | 10:08 |
| *** unicell1 has joined #openstack-kolla | 10:13 | |
| *** unicell has quit IRC | 10:15 | |
| *** ccesario___ has joined #openstack-kolla | 10:22 | |
| *** ccesario___ has joined #openstack-kolla | 10:22 | |
| ccesario___ | Serlex, did you check mariadb container log s? | 10:25 |
| Serlex | im using mysql, I had to recreate the network/router and the instance | 10:25 |
| Serlex | something went wrong with networking on the existing instance. | 10:26 |
| ccesario___ | Hmmm... Good to know. | 10:29 |
| *** gfidente is now known as gfidente|ghost | 10:31 | |
| *** ccesario___ has quit IRC | 10:35 | |
| *** ccesario___ has joined #openstack-kolla | 10:36 | |
| *** ccesario___ has quit IRC | 10:40 | |
| *** ccesario___ has joined #openstack-kolla | 10:40 | |
| *** chopmann has joined #openstack-kolla | 10:40 | |
| *** chopmann has quit IRC | 10:41 | |
| ccesario | sbezverk, about mariadb errors, have you news ? | 11:04 |
| *** gfidente has joined #openstack-kolla | 11:05 | |
| *** ccesario___ has quit IRC | 11:07 | |
| sbezverk | morning ccesario, yes.. | 11:13 |
| ccesario | sbezverk, morning | 11:13 |
| ccesario | sbezverk, did you get my messages ? | 11:14 |
| sbezverk | ccesario when you shutdown multinode galera cluster gets partitioned and cannot come up | 11:14 |
| sbezverk | you need to run mariadb lights out recovery | 11:14 |
| ccesario | <ccesario> sbezverk, http://sprunge.us/dHXL | 11:14 |
| ccesario | <ccesario> sbezverk, http://paste.openstack.org/show/494125/ | 11:14 |
| ccesario | <ccesario> sbezverk, steps to reproduce http://paste.openstack.org/show/494127/ | 11:14 |
| sbezverk | just shutdown multinode and then bring it back up | 11:15 |
| sbezverk | you should it, in my test bed is 100% reproducible | 11:15 |
| ccesario | sbezverk, is it documented ? | 11:15 |
| ccesario | is it a normal behaivor ? | 11:15 |
| sbezverk | I do not think so, but for now you can use lights out to recover and after the summit I will work on gracefull shutdown | 11:16 |
| *** rhallisey has joined #openstack-kolla | 11:16 | |
| ccesario | sbezverk> just shutdown multinode and then bring it back up how .... why exactly this mean!? | 11:19 |
| sbezverk | you connect to each multinode server and do "powerdown" | 11:22 |
| sbezverk | here is the command for recovery: kolla-ansible mariadb_recovery | 11:22 |
| ccesario | sbezverk, let me try | 11:23 |
| *** SiRiuS has joined #openstack-kolla | 11:25 | |
| *** weiyu has joined #openstack-kolla | 11:29 | |
| *** Marga__ has quit IRC | 11:34 | |
| *** weiyu has quit IRC | 11:40 | |
| *** weiyu_ has joined #openstack-kolla | 11:42 | |
| *** unicell1 has quit IRC | 11:44 | |
| *** unicell has joined #openstack-kolla | 11:44 | |
| *** weiyu_ has quit IRC | 11:44 | |
| *** dims has quit IRC | 11:46 | |
| *** dmsimard has quit IRC | 11:46 | |
| *** rhallisey has quit IRC | 11:47 | |
| *** dmsimard has joined #openstack-kolla | 11:47 | |
| *** rhallisey has joined #openstack-kolla | 11:51 | |
| ccesario | sbezverk, finishing the deploy (to provide tests) | 11:51 |
| *** openstack has quit IRC | 11:51 | |
| *** openstack has joined #openstack-kolla | 11:52 | |
| *** weiyu has joined #openstack-kolla | 11:53 | |
| sbezverk | rhallisey ping | 11:55 |
| rhallisey | sbezverk, hey | 11:55 |
| *** dims has joined #openstack-kolla | 11:56 | |
| ccesario | sbezverk, isn't it considered a critical problem ? | 11:56 |
| sbezverk | rhallisey have you tried iscsi on your multinode? | 11:56 |
| sbezverk | ccesario no because in DC poeple usually do not do that :-) | 11:57 |
| rhallisey | sbezverk, I haven't yet. I'll give it a shot later toady | 11:57 |
| rhallisey | I have it all setup | 11:57 |
| rhallisey | I need to remember how to use cinder properly | 11:57 |
| sbezverk | rhallisey I tried and it seems working but not very stable, some intemittent errors | 11:57 |
| *** athomas has quit IRC | 11:58 | |
| rhallisey | sbezverk, did al your stuff merge? | 11:58 |
| *** athomas has joined #openstack-kolla | 11:58 | |
| sbezverk | rhallisey :-( nope, I am still missing 1 +2 | 11:58 |
| sbezverk | only one patch got merged | 11:58 |
| ccesario | sbezverk, but problems happen and if this happen we will have a lot of problems | 11:59 |
| *** weiyu has quit IRC | 11:59 | |
| mlima_ | hello guys, we need some reviews here https://review.openstack.org/#/c/306036/ https://review.openstack.org/#/c/305840/ https://review.openstack.org/#/c/305770/ | 12:00 |
| patchbot | mlima_: patch 306036 - kolla (stable/mitaka) - Set db connection retry to infinity | 12:00 |
| patchbot | mlima_: patch 305840 - kolla (stable/mitaka) - Use proper CentOS repositories for ceph and qemu | 12:00 |
| patchbot | mlima_: patch 305770 - kolla (stable/mitaka) - Enable unbuffered output for ansible | 12:00 |
| sbezverk | ccesario that is why we will look into gracefull shutdown | 12:00 |
| rhallisey | sbezverk, ok file bugs for what you're seeing | 12:00 |
| *** SiRiuS has quit IRC | 12:01 | |
| sbezverk | rhallisey I suspect it is because everything on my 5 nodes runs on the same physical server | 12:01 |
| sbezverk | rhallisey even ovs works in 75% of the time so I do not think it is iscis specific | 12:02 |
| rhallisey | sbezverk, could be. I'll try and get to this later. Hopefully I don't see it | 12:03 |
| rhallisey | mlima_, thanks for the heads up | 12:03 |
| mlima_ | rhallisey, :) | 12:04 |
| sbezverk | rhallisey ping me if you see anytime unusual | 12:04 |
| sbezverk | I meant anything | 12:04 |
| *** reddy has quit IRC | 12:04 | |
| openstackgerrit | Merged openstack/kolla: Adds ansible code for iscsid and tgtd containers https://review.openstack.org/299995 | 12:04 |
| rhallisey | sbezverk, ya will do | 12:05 |
| openstackgerrit | Merged openstack/kolla: Add iscsid/tgtd specific parameters to Kolla infrastructure https://review.openstack.org/300004 | 12:05 |
| openstackgerrit | Merged openstack/kolla: Set db connection retry to infinity https://review.openstack.org/306036 | 12:05 |
| rhallisey | sbezverk, nice job on those patches | 12:05 |
| openstackgerrit | Merged openstack/kolla: Enable unbuffered output for ansible https://review.openstack.org/305770 | 12:06 |
| *** pbourke has quit IRC | 12:06 | |
| *** pbourke has joined #openstack-kolla | 12:07 | |
| ydeshmukh | Hi All, I've successfully deployed 3 virtual node setup on single host. rabbitmq service is not running in container. It is showing an error message: 'Error in log handler' | 12:09 |
| ydeshmukh | checked by log in to rabbitmq container and 'rabbitmqctl status' it gives 'TCP connection succeeded but Erlang distribution failed' | 12:10 |
| ydeshmukh | Any idea why it is giving error? problem with hostname? | 12:11 |
| ccesario | sbezverk, did you file bug ? | 12:11 |
| sbezverk | rhallisey thank you, I was really enjoying working on them.. I hope future holds something as interesting as this one :-) | 12:14 |
| *** tfukushima has quit IRC | 12:15 | |
| sbezverk | ccesario as it was agreed with sdake it will be taken care of after the summit.. | 12:15 |
| rhallisey | sbezverk, there's always something | 12:15 |
| *** banix has joined #openstack-kolla | 12:15 | |
| ccesario | sbezverk, right | 12:15 |
| *** dwalsh has joined #openstack-kolla | 12:15 | |
| *** che-arne has joined #openstack-kolla | 12:21 | |
| *** rhallisey has quit IRC | 12:22 | |
| *** rhallisey has joined #openstack-kolla | 12:25 | |
| *** wznoinsk has joined #openstack-kolla | 12:26 | |
| *** salv-orl_ has joined #openstack-kolla | 12:27 | |
| *** wznoinsk has quit IRC | 12:28 | |
| *** rmart04_ has joined #openstack-kolla | 12:29 | |
| *** rmart04 has quit IRC | 12:29 | |
| *** rmart04_ is now known as rmart04 | 12:29 | |
| *** salv-orlando has quit IRC | 12:30 | |
| *** b_bezak has quit IRC | 12:30 | |
| *** gfidente has quit IRC | 12:31 | |
| *** gfidente|ghost has quit IRC | 12:31 | |
| *** weiyu has joined #openstack-kolla | 12:31 | |
| *** che-arne has quit IRC | 12:33 | |
| *** che-arne has joined #openstack-kolla | 12:35 | |
| *** b_bezak_ has joined #openstack-kolla | 12:35 | |
| *** koperg has joined #openstack-kolla | 12:35 | |
| koperg | Hey guys | 12:35 |
| koperg | been having some issues deploying stable/mitaka , on a particular network setup | 12:36 |
| koperg | trying multinode deployment - with compute node on a different VLAN than other nodes | 12:36 |
| koperg | but prereq script fails - because HAProxy checks if all the network interfaces on deployment nodes are in the same subnet | 12:37 |
| *** mwheckmann has joined #openstack-kolla | 12:37 | |
| koperg | seems weird, since i would imagine that on compute node there would only be nova-libvirt and maybe openvswtich/neutron agent right ? | 12:37 |
| g3ek | koperg, I'm supposed to try the same thing this WE ^^ | 12:37 |
| koperg | heh, be prepared for some issues ;> | 12:39 |
| koperg | #JSYK: trying to use ubuntu 14.04 from sources and stable/mitaka | 12:39 |
| *** rhallisey has quit IRC | 12:39 | |
| *** b_bezak_ is now known as b_bezak | 12:41 | |
| *** gfidente|ghost has joined #openstack-kolla | 12:43 | |
| *** gfidente has joined #openstack-kolla | 12:43 | |
| *** gfidente has quit IRC | 12:43 | |
| *** gfidente has joined #openstack-kolla | 12:43 | |
| *** koperg has quit IRC | 12:44 | |
| *** rhallisey has joined #openstack-kolla | 12:46 | |
| *** mgoddard__ has joined #openstack-kolla | 12:51 | |
| *** rhallisey has quit IRC | 12:53 | |
| *** rhallisey has joined #openstack-kolla | 12:53 | |
| *** mgoddard_ has quit IRC | 12:54 | |
| sbezverk | koperg haproxy and keepalived if I ma not mistaken uses multicast, so if you have nodes on different subnets and you do not have multicast routing, then they will not be able to see each other.. | 12:59 |
| *** koperg has joined #openstack-kolla | 12:59 | |
| koperg | sry was disconnected :/ anyone had some replies ? :> | 13:00 |
| *** rhallisey has quit IRC | 13:01 | |
| sbezverk | koperg haproxy and keepalived if I ma not mistaken uses multicast, so if you have nodes on different subnets and you do not have multicast routing, then they will not be able to see each other.. and most likely they use multicast link local addresses for discovery which are not routable, hence the requirement | 13:01 |
| koperg | that makes me a sad panda :/ | 13:02 |
| koperg | so basicaly in my setup - cant do ? | 13:02 |
| koperg | or any way, to have compute node, not using HAproxy and keepalived ? :> | 13:03 |
| sbezverk | koperg only controllers run haproxy and keepalived and if you have network nodes | 13:05 |
| koperg | so why prereq checks compute node ? | 13:05 |
| koperg | should just screw prereq and run deployment ? :D | 13:06 |
| sbezverk | koperg why can't you carve extra vlan and put all your controllers in there? | 13:06 |
| koperg | sry for paste : | 13:07 |
| koperg | TASK: [prechecks | Checking if kolla_internal_vip_address is in the same network as network_interface on all nodes] *** ok: [kolla-mitaka-control] ok: [kolla-mitaka-control2] fatal: [compute5] => error while evaluating conditional: '169.254.' not in kolla_internal_vip_address and 'False' in kolla_internal_vip_address | ipaddr(ip_addr_output.stdout.split()[3]) ok: [kolla-mitaka-control3] ok: [kolla-mitaka-network] | 13:07 |
| *** rhallisey has joined #openstack-kolla | 13:07 | |
| koperg | it checks all nodes | 13:07 |
| koperg | including compute | 13:07 |
| koperg | sbezverk: its an environment out of my network controll , and additional VLANs cost $ | 13:07 |
| koperg | we got all controller in 1 VLAN | 13:08 |
| koperg | and compute node is in different VLAN | 13:08 |
| koperg | and prereq fails, saying that compute needs to be in same subnet | 13:08 |
| sbezverk | koperg I see in theory I would agree with you, for as compute node as long as there is IP connectivity to internal VIP address where all services required by compute not are bound, then it should be ok | 13:09 |
| sbezverk | koperg maybe in future this limitation will be lifted, but still controllers need to stay on the same IP segment | 13:10 |
| *** allen_gao has quit IRC | 13:10 | |
| koperg | makes sense | 13:11 |
| koperg | but i still need to deploy this stable/mitaka this weekend :> | 13:12 |
| *** ydeshmukh has quit IRC | 13:13 | |
| koperg | it gets better :D | 13:15 |
| koperg | root@kolla-mitaka-deployer:/usr/local/share/kolla/ansible/inventory# kolla-ansible deploy -i /usr/local/share/kolla/ansible/inventory/multinode -p /usr/local/share/kolla/ansible/inventory/multinode Deploying Playbooks : ansible-playbook -i /usr/local/share/kolla/ansible/inventory/multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=deploy /usr/local/share/kolla/ansible/inventor | 13:15 |
| koperg | ERROR: Syntax Error while loading YAML script, /usr/local/share/kolla/ansible/inventory/multinode Note: The error may actually appear before this position: line 5, column 1 # These hostname must be resolvable from your deployment host kolla-mitaka-control ansible_connection=ssh ansible_user=root ansible_ssh_pass=**** | 13:15 |
| koperg | root@kolla-mitaka-deployer:/usr/local/share/kolla/ansible/inventory# ping kolla-mitaka-control PING kolla-mitaka-control (10.136.228.141) 56(84) bytes of data. 64 bytes from kolla-mitaka-control (10.136.228.141): icmp_seq=1 ttl=64 time=0.584 ms | 13:16 |
| koperg | any ideas ? | 13:16 |
| *** alyson_ has joined #openstack-kolla | 13:17 | |
| *** weiyu has quit IRC | 13:19 | |
| *** sdake has joined #openstack-kolla | 13:20 | |
| sdake | morning | 13:21 |
| sdake | akwasnie ping | 13:21 |
| *** weiyu has joined #openstack-kolla | 13:22 | |
| akwasnie | sdake hi | 13:22 |
| *** sdake_ has joined #openstack-kolla | 13:22 | |
| *** gfidente|ghost has quit IRC | 13:23 | |
| *** vhosakot has joined #openstack-kolla | 13:24 | |
| sdake_ | hey folks, anyone feel up to testing my tls patch for kibana | 13:24 |
| *** sdake has quit IRC | 13:26 | |
| *** allen_gao has joined #openstack-kolla | 13:27 | |
| *** inc0 has joined #openstack-kolla | 13:28 | |
| inc0 | good morning | 13:28 |
| sdake_ | hey inc0 | 13:28 |
| inc0 | sdake_, when do you tag mitaka? | 13:28 |
| sdake_ | need m0ar testing on kibana patch | 13:28 |
| sdake_ | today sometime i assume | 13:28 |
| inc0 | I need more testing on backport | 13:28 |
| inc0 | let me re subbmit it for stable/mitaka | 13:29 |
| sbezverk | koperg get the full log and paste somewhere but not here.. | 13:29 |
| *** vhosakot has quit IRC | 13:30 | |
| ccesario | hey folks.... haproxy enabled is it mandatory !? | 13:30 |
| inc0 | ccesario by all means no | 13:31 |
| sbezverk | ccesario I guess for multinode yes but not for AIO | 13:31 |
| inc0 | in fact I usually disable it | 13:31 |
| ccesario | when haproxy is disable mariadb does not goes UP .... http://paste.openstack.org/show/494220/ | 13:31 |
| inc0 | but you need haproxy when you want ha | 13:31 |
| inc0 | ccesario buy you did change external_ip right? | 13:31 |
| inc0 | to something that is not vio | 13:31 |
| ccesario | it try usage VIP address instead node adderss | 13:32 |
| ccesario | inc0, yes, I know... it only tests :) | 13:32 |
| sdake_ | sbezverk ccesario can you guys test this https://review.openstack.org/#/c/306302/1 | 13:32 |
| patchbot | sdake_: patch 306302 - kolla - Enable TLS/SSL for Kibana | 13:32 |
| sdake_ | this is probably the lsat peice of work going into stable/mitaka | 13:32 |
| koperg | sbezverk: http://paste.openstack.org/raw/29Rq4IrK7SAWmrYtm6ls/ - tried with both hostnames and IP's | 13:32 |
| sbezverk | inc0 multinode config, does it not mean always HA? I have nove seen option to say I want mariadb on one nod and that is it | 13:33 |
| koperg | did syntax change from stable/liberty in multinode.yml or what ? | 13:33 |
| *** salv-orlando has joined #openstack-kolla | 13:33 | |
| inc0 | ccesario without haproxy you need node address | 13:33 |
| inc0 | haproxy and keepalived goes together | 13:33 |
| inc0 | sbezverk, no, if you have 3 nodes of mariadb but always use just one of them | 13:34 |
| inc0 | and this one dies | 13:34 |
| ccesario | inc0, any special change into globals for this ? | 13:34 |
| inc0 | ccesario just enable_haproxy: no and change address to one of controllers | 13:34 |
| *** allen_gao has quit IRC | 13:34 | |
| sbezverk | koperg please paste your multinode inventory | 13:34 |
| ccesario | inc0, http://paste.openstack.org/show/494223/ | 13:34 |
| inc0 | koperg, full invertory please | 13:35 |
| ccesario | I m testing with this inventory | 13:35 |
| *** gfidente|ghost has joined #openstack-kolla | 13:35 | |
| *** salv-orl_ has quit IRC | 13:35 | |
| ccesario | inc0, in this case need I change kolla_internal_vip_address: pointing to 192.168.201.2 ... ? | 13:35 |
| koperg | sbezverk , inc0 : http://paste.openstack.org/raw/29Rq4IrK7SAWmrYtm6ls/ | 13:36 |
| inc0 | ccesario correct | 13:36 |
| inc0 | koperg, /usr/local/share/kolla/ansible/inventory/multinode << I need full content of this file | 13:37 |
| ccesario | inc0, thanks by info | 13:37 |
| ccesario | let me try | 13:37 |
| koperg | inc0: sorry my clipboard hates me : http://paste.openstack.org/raw/6fckpLrFs50idEIBBnN8/ | 13:38 |
| ccesario | inc0, but in this case, prechecks will fail :P | 13:38 |
| ccesario | due 192.168.201.2 it is an IP address in use | 13:39 |
| *** allen_gao has joined #openstack-kolla | 13:39 | |
| koperg | ok nvm , stupid me | 13:41 |
| koperg | didnt pointed to site.yml | 13:41 |
| koperg | lets see where will deployment fail now :D | 13:41 |
| inc0 | koperg, banish the bad emotions! you make servers anxious | 13:42 |
| *** stvnoyes has left #openstack-kolla | 13:42 | |
| *** weiyu has quit IRC | 13:43 | |
| koperg | :D | 13:43 |
| *** SiRiuS has joined #openstack-kolla | 13:44 | |
| koperg | inc0: #never_easy http://paste.openstack.org/show/494227/ | 13:46 |
| koperg | :> | 13:46 |
| sbezverk | koperg any more details ;-) it is a bit hard to guess based on just 5 lines of logs you pasted | 13:47 |
| *** tfukushima has joined #openstack-kolla | 13:47 | |
| koperg | looking through now for some details :> | 13:48 |
| koperg | want full deployment logs pasted ? | 13:48 |
| sbezverk | koperg of course | 13:48 |
| koperg | one sec | 13:48 |
| *** che-arne has quit IRC | 13:50 | |
| koperg | sbezverk: thats from 2nd try : http://paste.openstack.org/raw/op6YRryvt97DtYLbuRI0/ | 13:51 |
| koperg | sbezverk: thats from 1st: http://paste.openstack.org/raw/494229/ | 13:52 |
| inc0 | hmmm nothing failed there | 13:53 |
| inc0 | also not a full log | 13:53 |
| inc0 | btw enable_cinder: yes if you're running ceph | 13:54 |
| inc0 | we disable cinder for now because we didnt have iscsi in Mitaka | 13:54 |
| inc0 | it's perfectly working with ceph tho | 13:54 |
| sdake_ | inc0 can you review with yur eyeballs; https://review.openstack.org/#/c/306302/1 | 13:55 |
| patchbot | sdake_: patch 306302 - kolla - Enable TLS/SSL for Kibana | 13:55 |
| sdake_ | where is persistent data for kibana stored | 13:56 |
| koperg | inc0: b_bezak will paste it in a sec - need to go afk for a while | 13:56 |
| sdake_ | it needs a named volume | 13:56 |
| inc0 | sdake_, we shouldn't expose kibana to external tho | 13:56 |
| inc0 | since we don't have auth there | 13:56 |
| sdake_ | we do hae auth | 13:56 |
| sdake_ | where ya been ;) | 13:56 |
| *** rhallisey has quit IRC | 13:56 | |
| inc0 | I just deployed it yesterday and it serves me content like there is no tomorrow | 13:57 |
| inc0 | sdake_, elasticsearch is where we store data | 13:57 |
| sdake_ | it assk me for apwd try broked.selfip.net:5601 | 13:57 |
| sdake_ | does es have a named volume? | 13:58 |
| inc0 | sdake_, I've sent you a link yesterday, that's vanilla kobana | 13:58 |
| sdake_ | we must ahave config diffferences | 13:59 |
| inc0 | ahh | 13:59 |
| b_bezak | inc0, sbezverk: paste.openstack.org was trimming the log - here is the full from 1st deployment http://pastebin.com/raw/rpyiGs06 | 13:59 |
| sdake_ | are you using two vips | 13:59 |
| inc0 | but I have haproxy disabled | 13:59 |
| sdake_ | well yu need haproxy enabled to get the uath working | 13:59 |
| inc0 | b_bezak, docker logs mariadb on controllers please | 14:00 |
| inc0 | ok, that was the issue | 14:00 |
| *** alyson_ has quit IRC | 14:00 | |
| *** dims has quit IRC | 14:01 | |
| inc0 | b_bezak, sooo, something bad have happend on 10.136.228.141 | 14:01 |
| inc0 | ansible failed to get facts | 14:01 |
| inc0 | can you confirm if it's working? | 14:01 |
| openstackgerrit | Carlos Cesario proposed openstack/kolla: Add haproxy support to Kibana elasticsearch.url param https://review.openstack.org/306443 | 14:01 |
| *** alyson_ has joined #openstack-kolla | 14:02 | |
| b_bezak | inc0 - yes .141 is doing fine. 2nd deployment didn't have issues with the gathering facts phase | 14:02 |
| b_bezak | let me gather mariadb logs | 14:02 |
| sdake_ | ccesario does that patch work multinode? | 14:03 |
| sbezverk | b_bezak I saw similar issue when I did not clean up in var/lib/docker/* from previous deployments | 14:03 |
| *** dims has joined #openstack-kolla | 14:04 | |
| sbezverk | I am not syaing it is your issue, but it is better to make sure | 14:04 |
| b_bezak | sbezverk: on controllers? or everywhere? ;) | 14:04 |
| sbezverk | <b_bezak on controllers since mariadb runs only on controller nodes | 14:05 |
| *** reddy has joined #openstack-kolla | 14:06 | |
| *** mwheckmann has quit IRC | 14:07 | |
| ccesario | sdake_, in my tests, yes :) ... I shutdown one controller node and Kibana remain working .... but it would be cool if someone could test it too | 14:07 |
| b_bezak | inc0, sbezverk - mariadb docker on controllers keep restarting with this http://paste.openstack.org/raw/7qVF22lMon5ngihjvFxD/ | 14:07 |
| b_bezak | I think I just restore snapshots of controllers/network nodes and try deploy once again | 14:08 |
| inc0 | check /usr/lib/docker/volumes/kolla_logs/_data/mariadb/mariadb.log | 14:08 |
| inc0 | b_bezak, and ofc you don't have any mysql running on server by default have you? | 14:09 |
| b_bezak | nope, they are clean 14.04.4 vms | 14:09 |
| ccesario | b_bezak, did you reboot the controllers node ? | 14:10 |
| *** rhallisey has joined #openstack-kolla | 14:10 | |
| b_bezak | no I didn't | 14:10 |
| inc0 | check logs before you restore images plz | 14:10 |
| b_bezak | http://pastebin.com/raw/QJsyeRpb | 14:11 |
| inc0 | mariadb is acting up from time to time, we need to understand it | 14:11 |
| b_bezak | maria logs | 14:11 |
| inc0 | hmm, you have any ports blocked between machines? | 14:11 |
| *** salv-orlando has quit IRC | 14:12 | |
| sdake_ | need reviews on https://review.openstack.org/#/c/306302/1 | 14:12 |
| patchbot | sdake_: patch 306302 - kolla - Enable TLS/SSL for Kibana | 14:12 |
| sdake_ | critical bug - neeeds to go into stable/mitaka | 14:12 |
| b_bezak | inc0 - no all ports are opened | 14:12 |
| b_bezak | I have double checked that | 14:13 |
| inc0 | hmm | 14:14 |
| inc0 | can you check your iptables for trash | 14:15 |
| inc0 | ? | 14:15 |
| inc0 | thats a long shot | 14:15 |
| *** ravig has joined #openstack-kolla | 14:16 | |
| b_bezak | iptables looks good | 14:16 |
| b_bezak | only DOCKER chains | 14:16 |
| *** salv-orlando has joined #openstack-kolla | 14:17 | |
| inc0 | hmm, try to remove containers and rebuild | 14:18 |
| inc0 | and redeploy | 14:18 |
| b_bezak | will do | 14:18 |
| b_bezak | rebuild registry too? | 14:20 |
| *** banix has quit IRC | 14:20 | |
| inc0 | just images and push them up | 14:20 |
| b_bezak | kk | 14:20 |
| inc0 | build.py --push should be enough | 14:21 |
| *** rhallisey has quit IRC | 14:27 | |
| *** reddy has quit IRC | 14:28 | |
| *** achanda has joined #openstack-kolla | 14:30 | |
| *** rhallisey has joined #openstack-kolla | 14:30 | |
| *** achanda has quit IRC | 14:32 | |
| *** athomas has quit IRC | 14:34 | |
| *** ccesario___ has joined #openstack-kolla | 14:37 | |
| sdake_ | folks | 14:43 |
| sdake_ | the governance repository will be changed with a new releaee model | 14:43 |
| sdake_ | see: https://review.openstack.org/#/c/300752/ | 14:44 |
| patchbot | sdake_: patch 300752 - governance - Change kolla to release:cycle-trailing | 14:44 |
| inc0 | good or bad for us? | 14:44 |
| sdake_ | fantastic for us | 14:44 |
| *** SiRiuS has quit IRC | 14:44 | |
| sdake_ | pretty much my complaining and governance changes got this change triggered | 14:44 |
| sdake_ | my original change had a 1 week timer | 14:44 |
| *** haplo37 has joined #openstack-kolla | 14:44 | |
| sdake_ | this one has a two week timer | 14:44 |
| sdake_ | so i think it makes sense to give ourselves the full two weeks if we need | 14:44 |
| sdake_ | the question I have is, do we need the full two weeks | 14:44 |
| inc0 | yeah, I mean we are as stable as other services | 14:45 |
| rhallisey | nice | 14:45 |
| inc0 | so we need some time to make sure we work with other released | 14:45 |
| *** mgoddard has joined #openstack-kolla | 14:46 | |
| sdake_ | http://releases.openstack.org/mitaka/schedule.html | 14:46 |
| sdake_ | 4/7 is the mitaka schedule as is | 14:46 |
| *** mgoddard__ has quit IRC | 14:46 | |
| sdake_ | this means we could release on the 21st and be complaint with that new release model wording | 14:47 |
| inc0 | sdake_, your call but well, backport has to happen after release | 14:47 |
| inc0 | and I am not super fond of us not testing it | 14:48 |
| inc0 | nobody will deploy mitaka this release | 14:48 |
| sdake_ | i'm not super fond of us not testing stable/mitaka | 14:48 |
| inc0 | everybody will deploy liberty | 14:48 |
| sdake_ | i did heavy teting this week and all kinds of things fell over | 14:49 |
| b_bezak | inc0 - FYI - mariadb containers have been created correctly now. Thx. Other components are being deployed as we speak | 14:50 |
| inc0 | b_bezak, yeah, I think we need to refactor how we do mariadb | 14:50 |
| inc0 | lots of unneeded complexity there imho | 14:51 |
| *** pmisiak has quit IRC | 14:51 | |
| *** mwheckmann has joined #openstack-kolla | 14:53 | |
| *** mbound has quit IRC | 14:55 | |
| ccesario | sdake_, trying test your patch now | 14:55 |
| *** dave-mcc_ has joined #openstack-kolla | 14:56 | |
| koperg | back | 14:57 |
| koperg | inc0: only 1 fail left ! http://paste.openstack.org/raw/494238/ | 14:57 |
| *** mlima_ has quit IRC | 14:57 | |
| inc0 | akwasnie jsut had same error | 14:58 |
| inc0 | logs of neutron plz | 14:58 |
| inc0 | /usr/lib/docker/volumes/kolla_logs/_data/neutron | 14:58 |
| *** rhallisey has quit IRC | 14:58 | |
| sbezverk | koperg are you on ubuntu? | 14:59 |
| koperg | my deployment is on ubuntu 14.04 | 15:00 |
| koperg | yes | 15:00 |
| sbezverk | mount --make-shared /run | 15:00 |
| sbezverk | make sure you have run it | 15:01 |
| koperg | i am sure i did, but will do it again just in case | 15:01 |
| sbezverk | have you made it to run persistently on reboot? | 15:01 |
| *** rhallisey has joined #openstack-kolla | 15:02 | |
| koperg | nope | 15:02 |
| koperg | i am one of those morons, that follow instructions like a sheep ;] | 15:02 |
| sbezverk | koperg I have done the same ;-) | 15:03 |
| *** mbound has joined #openstack-kolla | 15:06 | |
| koperg | Thanks for making me feel better about myself! :> | 15:06 |
| koperg | btw. deployment full success | 15:06 |
| koperg | huge thanks for all the help guys! | 15:06 |
| koperg | our weekend is saved ! | 15:07 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: check /e/s/s/docker.service.d/kolla.conf in redhat family https://review.openstack.org/305611 | 15:07 |
| b_bezak | yeap :) | 15:07 |
| b_bezak | thx! | 15:07 |
| inc0 | glad to hear that | 15:07 |
| *** ccesario___ has quit IRC | 15:07 | |
| inc0 | still upset by mariadb | 15:07 |
| inc0 | but I guess I found myself a job for newton | 15:08 |
| koperg | ps. any plans for multiregion support soonish ? :> | 15:08 |
| *** ccesario___ has joined #openstack-kolla | 15:08 | |
| sbezverk | inc0 count me in too, I do not want to suffer galera's weirdness | 15:08 |
| inc0 | well, not really, but with a bit of magic you can do this | 15:09 |
| koperg | inc0: would greatly appreciate some tips/tricks to do so :} | 15:09 |
| inc0 | so you might need to manually create endpoints | 15:09 |
| inc0 | and override configs of services | 15:10 |
| inc0 | so you can deploy 1 region with ks in it | 15:10 |
| koperg | ok, where is the place to override the configs, so it will not be lost during docker restarts ? | 15:10 |
| inc0 | /etc/kolla/confs I believe | 15:11 |
| inc0 | you can override config options one by one | 15:11 |
| akwasnie | sbezverk: thanks!, I also forgot about 'mount --make-shared /run'....it helped:) | 15:11 |
| *** serverascode has quit IRC | 15:11 | |
| inc0 | hold on I'll find correct docs | 15:11 |
| koperg | you sure ? i rememeber making some changes there when having fun with stable/liberty - it didnt survive reboot :> | 15:11 |
| ccesario | sdake_, | 15:11 |
| ccesario | http://puu.sh/oj8LS/11713e5e9a.png | 15:11 |
| sbezverk | <koperg> /etc/kolla/config/keystone.conf | 15:11 |
| *** sdake_ has quit IRC | 15:11 | |
| sbezverk | when you have copy always | 15:12 |
| *** macsz has quit IRC | 15:12 | |
| koperg | yeah, but we used copy once in Liberty | 15:12 |
| koperg | thats why | 15:12 |
| sbezverk | it will grab new config from this file and add it to runing keystone.conf | 15:12 |
| *** vhosakot has joined #openstack-kolla | 15:12 | |
| inc0 | https://github.com/openstack/kolla/blob/master/doc/quickstart.rst ctrl+f qemu | 15:12 |
| *** harmw_ has quit IRC | 15:12 | |
| *** manjeets has quit IRC | 15:12 | |
| inc0 | you have example there | 15:12 |
| koperg | with copy once, you need to make changes and redeploy / upgrade deployment right ? | 15:12 |
| koperg | inc0: thanks | 15:12 |
| inc0 | koperg, nope | 15:13 |
| inc0 | kolla-ansible reconfigure | 15:13 |
| *** rmart04 has quit IRC | 15:13 | |
| koperg | yeah | 15:13 |
| koperg | that is missing in stable/liberty | 15:13 |
| *** mlima_ has joined #openstack-kolla | 15:13 | |
| koperg | only kolla-ansible deploy available for ya ;] | 15:13 |
| inc0 | koperg, sooo | 15:14 |
| koperg | ok, so Copy Always is default policy in stable/mitaka - right ? | 15:14 |
| inc0 | we'll do big backport | 15:14 |
| koperg | inc0: yeah, i've read about it ;] | 15:14 |
| inc0 | and liberty will essentially be deployed by mitaka code | 15:14 |
| koperg | inc0: i was hoping for april 15 :D | 15:14 |
| inc0 | actually you can help me testing | 15:15 |
| koperg | what do you need ? | 15:15 |
| inc0 | https://review.openstack.org/#/c/299563/ | 15:15 |
| patchbot | inc0: patch 299563 - kolla - Making Mitaka deploying liberty | 15:15 |
| koperg | i should have running stable/liberty somewhere | 15:15 |
| inc0 | only I'll be hones | 15:15 |
| *** timonwong has joined #openstack-kolla | 15:15 | |
| inc0 | stable/liberty is not very stable | 15:15 |
| inc0 | if you re-run deploy after rebuilding containers you may lose your data | 15:16 |
| koperg | yeah I've seen the warning :D | 15:16 |
| inc0 | ansible had a major bug which prevented us from using docker 1.9 and named volumes | 15:16 |
| inc0 | this is reason why we do this big backport thing | 15:16 |
| *** rhallisey has quit IRC | 15:16 | |
| koperg | yeah, downgrading docker-engine and docker-py helped when deploying | 15:16 |
| *** mgoddard_ has joined #openstack-kolla | 15:16 | |
| *** alyson_ has quit IRC | 15:16 | |
| *** ravig_ has joined #openstack-kolla | 15:16 | |
| inc0 | not the case with stable/mitaka any more | 15:17 |
| koperg | thats one of the reasons we switched aswell. | 15:17 |
| inc0 | so if you are thinking about deploying kolla on prod, I would strongly suggest to wait till we deal with this | 15:17 |
| *** mlima has joined #openstack-kolla | 15:17 | |
| inc0 | also you will appreciate network namespaces sharing;) | 15:18 |
| koperg | sounds good! | 15:18 |
| inc0 | debugging networking without it was a bitch | 15:18 |
| koperg | and yeah, we are thinking prod | 15:18 |
| inc0 | with docker 1.10 you can run ip netns exec on host | 15:18 |
| koperg | currently PoC phase - but running out of time soon ;] | 15:18 |
| *** manjeets has joined #openstack-kolla | 15:18 | |
| inc0 | so plan is when we tag mitaka | 15:19 |
| inc0 | I'm going to cherry pick this code to stable/mitaka | 15:19 |
| inc0 | after that observe this patchset | 15:19 |
| inc0 | it will become stable/liberty at some point | 15:19 |
| inc0 | but it should be functional even now | 15:19 |
| *** mlima__ has joined #openstack-kolla | 15:19 | |
| inc0 | with caveat that now it only works for source builds | 15:19 |
| *** ravig_ has quit IRC | 15:19 | |
| inc0 | so centos source or ubuntu source | 15:20 |
| *** sdake has joined #openstack-kolla | 15:20 | |
| inc0 | that will be fixed as well | 15:20 |
| *** mgoddard has quit IRC | 15:20 | |
| inc0 | so one thing what you could do with stable/liberty is to hack around containers | 15:20 |
| inc0 | data_containers | 15:20 |
| *** tonyb_ has joined #openstack-kolla | 15:20 | |
| inc0 | if you really need it right now | 15:20 |
| *** mlima__ has quit IRC | 15:21 | |
| inc0 | let me see | 15:21 |
| inc0 | koperg, https://github.com/openstack/kolla/blob/stable/liberty/ansible/roles/mariadb/tasks/bootstrap.yml#L53 this is perpetrator | 15:22 |
| inc0 | data container | 15:22 |
| inc0 | so all your persistend data will be in container | 15:22 |
| inc0 | you rebuild container, you lose data | 15:22 |
| *** mlima has quit IRC | 15:22 | |
| inc0 | what you can do is to change playbooks in a way that instead of binding to data container it will bind to host | 15:23 |
| inc0 | but either way, you will lose upgrades | 15:23 |
| inc0 | so I would storngly suggest to wait till backport | 15:23 |
| koperg | yeah , will patiently wait most likely | 15:23 |
| inc0 | let me know when your time runs out and I'll help you get started | 15:24 |
| koperg | only thing we need from Liberty is integration to some other tool, that stopped their development on Liberty :/ | 15:24 |
| inc0 | best thing you can do is to help me testing this patch | 15:24 |
| inc0 | :) | 15:24 |
| inc0 | more tests means more happiness | 15:24 |
| koperg | I'll try to deploy it somewhere over the weekend and let you know. | 15:25 |
| *** mlima_ has quit IRC | 15:25 | |
| *** ravig has quit IRC | 15:25 | |
| *** dims has quit IRC | 15:25 | |
| *** gfidente has quit IRC | 15:25 | |
| *** mikelk has quit IRC | 15:25 | |
| *** ntpttr_ has quit IRC | 15:25 | |
| *** kbyrne has quit IRC | 15:25 | |
| *** tonyb has quit IRC | 15:25 | |
| *** jidar has quit IRC | 15:25 | |
| *** ccesario has quit IRC | 15:25 | |
| inc0 | koperg, you're running centos or buntu? | 15:26 |
| koperg | ubuntu | 15:26 |
| inc0 | kk | 15:26 |
| koperg | from sources currently | 15:26 |
| *** smekel_ has quit IRC | 15:26 | |
| koperg | binary - still have issues | 15:26 |
| inc0 | no ubuntu binary in kolla as for now | 15:26 |
| koperg | yeah | 15:26 |
| inc0 | we simply don't have repos;) | 15:26 |
| *** mlima has joined #openstack-kolla | 15:26 | |
| *** dims has joined #openstack-kolla | 15:26 | |
| koperg | fair enough | 15:26 |
| inc0 | but ubuntu source is what I'm running and what I tested it with | 15:26 |
| inc0 | also when you're at it | 15:27 |
| inc0 | try kibana:D | 15:27 |
| inc0 | enable_central_logging: yes | 15:27 |
| b_bezak | yeap we did enable it | 15:27 |
| b_bezak | and it is cool | 15:27 |
| b_bezak | I am playing with it now :) | 15:27 |
| koperg | yeah, looks nice | 15:27 |
| *** ccesario___ has quit IRC | 15:28 | |
| koperg | some managers will love - "Status Green" :D | 15:28 |
| *** ccesario___ has joined #openstack-kolla | 15:28 | |
| inc0 | yeah, I used to run kibana on prod at fair scale | 15:28 |
| inc0 | it saved my ass more times than I can count | 15:28 |
| koperg | it soo much better , than going into dockers 1by1 and checking the problem :] | 15:29 |
| inc0 | so what you can do is with mitaka | 15:30 |
| inc0 | is to access logs from docker folumes | 15:30 |
| inc0 | volumes | 15:30 |
| *** jidar has joined #openstack-kolla | 15:30 | |
| koperg | true | 15:30 |
| koperg | thats also nice | 15:31 |
| inc0 | or with stable liberty docker exec -it log_data bash | 15:31 |
| inc0 | and you have all logs from node there | 15:31 |
| koperg | i think it was rsyslog container wasnt it ? | 15:31 |
| inc0 | however no multi node log aggregation | 15:31 |
| inc0 | rsyslog and log_data has the same stuff | 15:31 |
| inc0 | btw no more rsyslog in mitaka;) | 15:31 |
| inc0 | we do heka now;) | 15:31 |
| inc0 | it's so much better.. | 15:32 |
| *** gfidente has joined #openstack-kolla | 15:32 | |
| *** gfidente has quit IRC | 15:32 | |
| *** gfidente has joined #openstack-kolla | 15:32 | |
| openstackgerrit | Carlos Cesario proposed openstack/kolla: Make Kibana elasticsearch.url param point to VIP with haproxy enabled https://review.openstack.org/306443 | 15:32 |
| *** kbyrne has joined #openstack-kolla | 15:32 | |
| *** harmw has joined #openstack-kolla | 15:32 | |
| *** gfidente has quit IRC | 15:33 | |
| *** rhallisey has joined #openstack-kolla | 15:34 | |
| *** macsz has joined #openstack-kolla | 15:34 | |
| *** ccesario has joined #openstack-kolla | 15:34 | |
| *** smekel_ has joined #openstack-kolla | 15:39 | |
| *** mark-casey has joined #openstack-kolla | 15:45 | |
| *** timonwong has quit IRC | 15:45 | |
| d_code | inc0: btw, switching from device mapper to btrfs cleared our problems right up…after digging through journalctl for docker, it was crashing because it ran out of ability to mount and unmount loopback volumes | 15:47 |
| *** serverascode has joined #openstack-kolla | 15:47 | |
| mark-casey | d_code: where is that configured/done? I'm not having an issue I'm just wondering what I'm using | 15:47 |
| d_code | I created a new partition using btrfs and mounted it under /var/lib/docker. Docker will detect the filesystem and switch backend drivers | 15:48 |
| mark-casey | d_code: makes sense, thanks! | 15:49 |
| *** sdake has quit IRC | 15:49 | |
| *** alyson_ has joined #openstack-kolla | 15:49 | |
| d_code | mark-casey: I found this article insightful for general approach: http://www.projectatomic.io/blog/2015/06/notes-on-fedora-centos-and-docker-storage-drivers/ | 15:50 |
| d_code | and here’s the deets on docker + btrfs love: https://docs.docker.com/engine/userguide/storagedriver/btrfs-driver/ | 15:50 |
| d_code | it seemed easier to do than to setup the lvm volume backend | 15:50 |
| *** sdake has joined #openstack-kolla | 15:50 | |
| inc0 | d_code, also I heard bad things about lvm | 15:53 |
| *** gfidente|ghost is now known as gfidente | 15:53 | |
| inc0 | aufs is pretty much no-op | 15:54 |
| inc0 | all you need to do is to upgrade kernel and add option | 15:54 |
| inc0 | but btrfs is better than aufs | 15:54 |
| d_code | shrug… LVM is the recommended backend for project atomic currently…and some other kubernetes installs that I’ve seen | 15:54 |
| d_code | yeah…I’m keen on keeping selinux capability too….so aufs is a non-starter | 15:55 |
| *** ravig has joined #openstack-kolla | 15:55 | |
| inc0 | yeah, well I'm glad that btrfs worked | 15:55 |
| inc0 | I'd run my prod on btrfs | 15:55 |
| d_code | which…I know the docs suggest that you shouldn’t run selinux…but I’ve had no problems using the latest RDO packages b/c they have policies included | 15:55 |
| sdake | ok folks | 15:55 |
| sdake | here is plan | 15:55 |
| sdake | i am tagging now a new rc | 15:55 |
| sdake | we have atleast 2 more bugs which absolutely need resolutions | 15:56 |
| sdake | and i'd like people to start seriously testing stable/mitaka | 15:56 |
| inc0 | tonyb_, around? | 15:57 |
| *** MarMat has joined #openstack-kolla | 15:58 | |
| sdake | lvm backend wont work for upgrade d_code | 15:58 |
| inc0 | sdake, btw..what is exact reason | 15:58 |
| inc0 | ? | 15:58 |
| sdake | inc0 lvm locks layers | 15:58 |
| sdake | they can't be removed | 15:59 |
| sdake | ugprade returns an error 500 from docker engine | 15:59 |
| inc0 | duh | 15:59 |
| *** mark-casey1 has joined #openstack-kolla | 15:59 | |
| d_code | ah…well…. I’m using btrfs anyway | 15:59 |
| *** v1k0d3n_ has quit IRC | 15:59 | |
| *** rmart04 has joined #openstack-kolla | 16:00 | |
| *** severion has joined #openstack-kolla | 16:00 | |
| *** mgoddard__ has joined #openstack-kolla | 16:01 | |
| *** mark-casey has quit IRC | 16:01 | |
| *** mgoddard_ has quit IRC | 16:04 | |
| *** rmart04 has quit IRC | 16:05 | |
| *** rmart04 has joined #openstack-kolla | 16:05 | |
| *** mark-casey1 has quit IRC | 16:13 | |
| *** alyson_ has quit IRC | 16:13 | |
| *** serverascode has quit IRC | 16:13 | |
| *** rmart04 has quit IRC | 16:13 | |
| *** alyson_ has joined #openstack-kolla | 16:14 | |
| *** gfidente has quit IRC | 16:14 | |
| ccesario | sdake, when I try access by VIP address -> http://paste.openstack.org/show/494255/ | 16:14 |
| *** serverascode has joined #openstack-kolla | 16:14 | |
| sdake | talking with dave-mcc_ my approach may not work because of ca policie | 16:15 |
| sdake | as in they wont sign a non-routale address certificate | 16:15 |
| *** manjeets has quit IRC | 16:15 | |
| *** ntpttr has quit IRC | 16:15 | |
| sdake | this is the main thing holding up the tag of 2.0.0 | 16:16 |
| sdake | and he eis busy today | 16:16 |
| *** mbound has quit IRC | 16:17 | |
| *** SiRiuS has joined #openstack-kolla | 16:17 | |
| MarMat | Hi, I've proposed improvements regarding build logging, https://blueprints.launchpad.net/kolla/+spec/build-logging-per-container, https://review.openstack.org/#/c/306207/ | 16:17 |
| patchbot | MarMat: patch 306207 - kolla - Build logging per container and other logging impr... | 16:17 |
| *** mwheckmann has quit IRC | 16:18 | |
| openstackgerrit | Alicja Kwasniewska proposed openstack/kolla: Add Kibana quick start guide https://review.openstack.org/306019 | 16:18 |
| *** ntpttr has joined #openstack-kolla | 16:19 | |
| *** mark-casey has joined #openstack-kolla | 16:20 | |
| *** MarMat_ has joined #openstack-kolla | 16:20 | |
| MarMat_ | vhosakot I've assigned you and Sam Yaple as reviewers, it would be good if you can check and provide feedback | 16:20 |
| MarMat_ | I'll be around for discussion if needed | 16:20 |
| sdake | MarMat_ i set the blueprint to dicusion phase | 16:21 |
| *** ntpttr_ has joined #openstack-kolla | 16:21 | |
| mlima | guys, we need review here https://review.openstack.org/#/c/305840/ | 16:21 |
| patchbot | mlima: patch 305840 - kolla (stable/mitaka) - Use proper CentOS repositories for ceph and qemu | 16:21 |
| sdake | MarMat_ samyaple has departed the community so he wont be reviewing i suspect | 16:21 |
| *** MarMat has quit IRC | 16:22 | |
| MarMat_ | odake Oh I see, thanks for info... Can you set the right persons for review? | 16:22 |
| vhosakot | MarMat_: it would have been nice if you had asked me before submitting that patch set :), as the bug that patch set fixes (https://bugs.launchpad.net/kolla/+bug/1546718) was filed by me and I have already implement some fix for but not pushed my patch set for review yet :) | 16:22 |
| openstack | Launchpad bug 1546718 in kolla "Add logging for kolla-build" [Wishlist,Confirmed] - Assigned to Vikram Hosakote (vhosakot) | 16:22 |
| vhosakot | MarMat_: thanks for working on it, I will review it when I get a chance... no worries :) | 16:23 |
| vhosakot | sdake: yes, let us discuss that bp... https://blueprints.launchpad.net/kolla/+spec/build-logging-per-container I need more feedback to review/implement.. cool | 16:24 |
| MarMat_ | vhosakot I know, sorry. I have it implemented for some time already and waited for formalities to be sorted out before I could post a patchset. But I'm open to any changes to that. | 16:24 |
| *** unicell has quit IRC | 16:25 | |
| *** ccesario___ has quit IRC | 16:25 | |
| vhosakot | MarMat_: cool, no worries :) that bug was assigned to me and I have been working on it.. not a problem at all! we can implement together :) | 16:25 |
| sdake | vhosakot might as well close the bug as invalid since there is a blueprint for it now | 16:25 |
| openstackgerrit | Alicja Kwasniewska proposed openstack/kolla: Add Kibana quick start guide https://review.openstack.org/306019 | 16:26 |
| *** salv-orlando has quit IRC | 16:26 | |
| vhosakot | sdake: ok.. I will mark it invalid.. yes, we can use the bp.. cool | 16:27 |
| MarMat_ | vhosakot the bp is a bit different than your bug report, even though it's similar. | 16:27 |
| vhosakot | MarMat_: I will review it later today! | 16:27 |
| vhosakot | MarMat_: let us capture all the things in the bp | 16:27 |
| MarMat_ | vhosakot: ok! | 16:27 |
| vhosakot | MarMat_: cool | 16:27 |
| vhosakot | MarMat_: I have done it myself.. debug and fix a bug assigned to somebody else :) happens... cheers! | 16:28 |
| MarMat_ | Guys I've been following the kolla irc for come time... but it seems that from beginning of April the archive http://eavesdrop.openstack.org/irclogs/%23kolla/ is not recording... is there any alternative to that? | 16:31 |
| vhosakot | MarMat_: http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/ | 16:32 |
| vhosakot | MarMat_: it has been moved... | 16:32 |
| vhosakot | MarMat_: this message is seen as the headline banner when we join the #openstack-kolla channel here... IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA) | 16:33 |
| MarMat_ | vhosakot: great, thanks! It would be good to change link in https://wiki.openstack.org/wiki/IRC | 16:34 |
| vhosakot | MarMat_: good catch.. changing... | 16:35 |
| *** cfarquhar has quit IRC | 16:37 | |
| vhosakot | MarMat_: I see it already there... search for "#openstack-kolla" in http://eavesdrop.openstack.org/irclogs/ | 16:37 |
| *** elmiko_ is now known as elmiko | 16:38 | |
| MarMat_ | vhosakot: ok, I looked into "#kolla" only. | 16:38 |
| vhosakot | MarMat_: that is also there in case old logs need to be referred | 16:39 |
| *** smekel_ has quit IRC | 16:41 | |
| *** smekel_ has joined #openstack-kolla | 16:42 | |
| *** ntpttr_ has quit IRC | 16:42 | |
| *** cfarquhar has joined #openstack-kolla | 16:43 | |
| *** cfarquhar has joined #openstack-kolla | 16:43 | |
| *** ravig has quit IRC | 16:44 | |
| *** mark-casey1 has joined #openstack-kolla | 16:46 | |
| *** unicell has joined #openstack-kolla | 16:48 | |
| *** ntpttr_ has joined #openstack-kolla | 16:49 | |
| *** mark-casey has quit IRC | 16:50 | |
| *** banix has joined #openstack-kolla | 16:52 | |
| sdake | rhallisey can you set the topic prperly | 16:52 |
| *** unicell has quit IRC | 16:53 | |
| *** ntpttr has quit IRC | 16:55 | |
| rhallisey | sdake, what do you want it set to | 16:55 |
| *** unicell has joined #openstack-kolla | 16:56 | |
| *** allen_gao has quit IRC | 16:56 | |
| openstackgerrit | Merged openstack/kolla: Use proper CentOS repositories for ceph and qemu https://review.openstack.org/305840 | 16:56 |
| sdake | rhallisey this is the url shortened for the log link http://goo.gl/tG5gQl | 16:57 |
| *** ntpttr has joined #openstack-kolla | 16:57 | |
| rhallisey | sdake, I shorted that one posted there myself | 16:57 |
| rhallisey | it works | 16:57 |
| *** allen_gao has joined #openstack-kolla | 16:58 | |
| *** ccesario___ has joined #openstack-kolla | 16:58 | |
| sdake | rhallisey its kola, the channel logs are openstack-kolla.. | 16:58 |
| rhallisey | sdake, I have both linked up there | 16:59 |
| sdake | oh got it | 16:59 |
| sdake | ya that looks good | 16:59 |
| *** unicell has quit IRC | 17:00 | |
| vhosakot | rhallisey: sdake: the channel topic is correct | 17:00 |
| rhallisey | I have both in case you need t quick link to look back in the old days | 17:01 |
| *** achanda has joined #openstack-kolla | 17:04 | |
| *** ccesario___ has quit IRC | 17:04 | |
| *** daneyon has joined #openstack-kolla | 17:05 | |
| *** ravig has joined #openstack-kolla | 17:05 | |
| vhosakot | rhallisey: yep, that's cool.. old logs are useful | 17:05 |
| *** daneyon_ has joined #openstack-kolla | 17:06 | |
| *** ravig has quit IRC | 17:06 | |
| *** achanda has quit IRC | 17:08 | |
| *** ravig has joined #openstack-kolla | 17:08 | |
| *** ccesario___ has joined #openstack-kolla | 17:08 | |
| *** mgoddard__ has quit IRC | 17:10 | |
| *** mgoddard has joined #openstack-kolla | 17:10 | |
| *** daneyon has quit IRC | 17:11 | |
| *** ravig has quit IRC | 17:12 | |
| *** ravig has joined #openstack-kolla | 17:12 | |
| *** salv-orlando has joined #openstack-kolla | 17:14 | |
| *** ccesario___ has quit IRC | 17:16 | |
| *** SiRiuS__ has joined #openstack-kolla | 17:17 | |
| *** ccesario___ has joined #openstack-kolla | 17:17 | |
| *** salv-orl_ has joined #openstack-kolla | 17:17 | |
| *** yuanying_ has joined #openstack-kolla | 17:18 | |
| *** mark-casey has joined #openstack-kolla | 17:18 | |
| *** ravig_ has joined #openstack-kolla | 17:18 | |
| *** mlima_ has joined #openstack-kolla | 17:18 | |
| *** jidar has quit IRC | 17:20 | |
| *** mwheckmann has joined #openstack-kolla | 17:20 | |
| *** daneyon has joined #openstack-kolla | 17:21 | |
| *** che-arne has joined #openstack-kolla | 17:21 | |
| *** kbyrne has quit IRC | 17:21 | |
| *** jasonsb has joined #openstack-kolla | 17:22 | |
| *** jidar has joined #openstack-kolla | 17:23 | |
| *** mark-casey2 has joined #openstack-kolla | 17:23 | |
| *** mark-casey has quit IRC | 17:24 | |
| vhosakot | sdake: all: google added google cloud as cinder backup driver http://www.infoworld.com/article/3055293/open-source-tools/google-open-source-hybrid-cloud-gets-new-openstack-backup-driver.html | 17:24 |
| *** sdake_ has joined #openstack-kolla | 17:26 | |
| *** manjeets has joined #openstack-kolla | 17:26 | |
| *** salv-orlando has quit IRC | 17:26 | |
| *** ravig has quit IRC | 17:26 | |
| *** daneyon_ has quit IRC | 17:26 | |
| *** mark-casey1 has quit IRC | 17:26 | |
| *** cfarquhar has quit IRC | 17:26 | |
| *** smekel_ has quit IRC | 17:26 | |
| *** SiRiuS has quit IRC | 17:26 | |
| *** serverascode has quit IRC | 17:26 | |
| *** alyson_ has quit IRC | 17:26 | |
| *** rmart04 has joined #openstack-kolla | 17:27 | |
| *** ccesario has quit IRC | 17:27 | |
| *** mlima has quit IRC | 17:27 | |
| *** haplo37 has quit IRC | 17:27 | |
| *** inc0 has quit IRC | 17:27 | |
| *** dwalsh has quit IRC | 17:27 | |
| *** openstackgerrit has quit IRC | 17:27 | |
| *** yuanying has quit IRC | 17:27 | |
| *** unicell has joined #openstack-kolla | 17:28 | |
| *** che-arne has quit IRC | 17:29 | |
| *** sdake has quit IRC | 17:29 | |
| *** ccesario___ has quit IRC | 17:30 | |
| sbezverk | what is user name for kibana when connect to port 5601? | 17:30 |
| *** rmart04 has quit IRC | 17:32 | |
| *** allen_gao has quit IRC | 17:32 | |
| vhosakot | sbezverk: kibana https://github.com/openstack/kolla/blob/master/ansible/group_vars/all.yml#L227 | 17:32 |
| sbezverk | vhosakot thank you | 17:33 |
| vhosakot | sbezverk: cool | 17:33 |
| *** ccesario___ has joined #openstack-kolla | 17:33 | |
| *** b_bezak has quit IRC | 17:33 | |
| vhosakot | sbezverk: https://github.com/openstack/kolla/blame/master/doc/quickstart.rst#L647-L665 | 17:33 |
| *** mwheckmann has quit IRC | 17:35 | |
| sbezverk | vhosakot that was what I was following and there was no mentioning about user at least in first paragraph | 17:36 |
| vhosakot | ahok | 17:36 |
| *** alyson_ has joined #openstack-kolla | 17:37 | |
| *** ravig_ has quit IRC | 17:37 | |
| *** ravig has joined #openstack-kolla | 17:37 | |
| *** jasonsb has quit IRC | 17:38 | |
| *** ravig has quit IRC | 17:38 | |
| *** sdake_ has quit IRC | 17:38 | |
| *** cfarquhar has joined #openstack-kolla | 17:39 | |
| *** cfarquhar has joined #openstack-kolla | 17:39 | |
| *** kbyrne has joined #openstack-kolla | 17:40 | |
| *** tfukushima has quit IRC | 17:42 | |
| *** sdake has joined #openstack-kolla | 17:43 | |
| *** daneyon has quit IRC | 17:44 | |
| *** smekel_ has joined #openstack-kolla | 17:45 | |
| *** allen_gao has joined #openstack-kolla | 17:45 | |
| *** dwalsh has joined #openstack-kolla | 17:46 | |
| dave-mcc_ | sdake i have a good fix for kibana over tls. testing one more tweak to it now and i'll send up the patch. | 17:46 |
| sdake | nice! | 17:46 |
| *** inc0 has joined #openstack-kolla | 17:46 | |
| sdake | i still think i will slip us to thursday | 17:47 |
| sdake | to give approatriate itme for testing everything | 17:47 |
| sdake | but i'll wait for your solution to merge | 17:47 |
| *** alyson_ has quit IRC | 17:47 | |
| *** openstackgerrit has joined #openstack-kolla | 17:48 | |
| *** ccesario has joined #openstack-kolla | 17:49 | |
| *** serverascode has joined #openstack-kolla | 17:51 | |
| *** shardy has quit IRC | 17:51 | |
| *** haplo37 has joined #openstack-kolla | 17:52 | |
| *** ravig has joined #openstack-kolla | 17:56 | |
| *** mark-casey has joined #openstack-kolla | 17:56 | |
| MarMat_ | sdake can you please update me what is current plan for separating ansible/deployment part of kolla into a new repo? I noted discussion about that here but did not find any info in recent kolla meetings... So will this happen? Is there any time plan? | 17:59 |
| *** mark-casey2 has quit IRC | 17:59 | |
| sdake | here is the 2.0.0 bug list: https://launchpad.net/kolla/+milestone/2.0.0 | 18:00 |
| sdake | MarMat_ will it happen - undefined - i dont want it, I dont see value in it, but othersi nt he community do | 18:00 |
| *** ByPasS has quit IRC | 18:00 | |
| sdake | MarMat_ it is something we will discuss at summit I am certain and will require a majority vote to execute | 18:00 |
| sdake | personally I think we have bigger things to accomplish then to break apart our repo into two | 18:01 |
| sdake | fwiw I think it would take 2-3 months to do properly | 18:01 |
| sdake | and result in an inability to backport bug fixes | 18:01 |
| *** ByPasS has joined #openstack-kolla | 18:02 | |
| sdake | or atleast make backporting super painful | 18:02 |
| *** SiRiuS__ has quit IRC | 18:02 | |
| *** Marga_ has joined #openstack-kolla | 18:03 | |
| sdake | dave-mcc_ do you think your kibana tls work will land today or should we push that past rc4? | 18:04 |
| MarMat_ | sdake ok, please note also my other blueprint which is somehow related to it: https://blueprints.launchpad.net/kolla/+spec/kolla-deploy-container | 18:04 |
| MarMat_ | I have a implementation of it running in my ci environment. | 18:05 |
| dave-mcc_ | it can land today. easy/obvious patch to get tls working. i'm trying to tweak now to get http:// to automatically redirect to https://. i'll send up the patch now without the redirect. | 18:05 |
| *** mark-casey1 has joined #openstack-kolla | 18:07 | |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Add TLS to Kibana Web Interface https://review.openstack.org/306571 | 18:07 |
| dave-mcc_ | sdake ^^ | 18:08 |
| sdake | dave-mcc_ that is alot simpmle then my approach ;-) | 18:08 |
| dave-mcc_ | when i was working on this a couple nights ago, i forgot about x-forwarded-proto. that's all that was missing. | 18:08 |
| dave-mcc_ | all the blogs complaining about basePath regressions led us in the wrong direction. | 18:09 |
| mlima_ | we still have 9 bugs in mitaka :/ | 18:09 |
| mlima_ | https://launchpad.net/kolla/+milestone/2.0.0 | 18:10 |
| mlima_ | these bugs will be fixed? | 18:10 |
| *** mark-casey has quit IRC | 18:10 | |
| dave-mcc_ | sdake i'll open a new bug to add redirect from http:// to https://. that can be lower priority. it's standard for web based things, but not critical. | 18:13 |
| *** vhosakot has quit IRC | 18:13 | |
| sdake | mlima targeted | 18:13 |
| sdake | who knows what will happen inthe next 4 ddays ;) | 18:14 |
| sdake | I booted out a bunch of lower priority bugs to 2.0.1 | 18:14 |
| sdake | https://launchpad.net/kolla/+milestone/2.0.1 | 18:14 |
| *** robcresswell has quit IRC | 18:14 | |
| mlima_ | sdake, https://bugs.launchpad.net/kolla/+bug/1558832 this bug was fixed in master | 18:15 |
| openstack | Launchpad bug 1558832 in kolla mitaka "rabbitmq upgrade stops containers even if image hasnt changed" [High,Triaged] - Assigned to Serguei Bezverkhi (sbezverk) | 18:15 |
| *** robcresswell has joined #openstack-kolla | 18:15 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Rabbitmq Upgrade restarts container only when image was changed. https://review.openstack.org/306576 | 18:15 |
| sdake | need acks on this bug please ; https://review.openstack.org/#/c/306576/ | 18:16 |
| patchbot | sdake: patch 306576 - kolla (stable/mitaka) - Rabbitmq Upgrade restarts container only when imag... | 18:16 |
| sdake | rather onthis review | 18:16 |
| sdake | thanks mlima | 18:16 |
| mlima_ | I can't test now, but who knows this also solves the bug to mitaka | 18:16 |
| *** mwheckmann has joined #openstack-kolla | 18:17 | |
| sdake | inc0 can you ack that | 18:18 |
| sdake | or rhallisey | 18:18 |
| sdake | or jpeeler | 18:18 |
| sdake | anyone will do - its a backport :) | 18:18 |
| mlima_ | what ack means? sdake | 18:20 |
| sdake | lets get some reviews on this https://review.openstack.org/306571 | 18:21 |
| sdake | ack means +2 it | 18:21 |
| mlima_ | oh :) | 18:21 |
| sdake | in a rubber stamp fahsion :) | 18:21 |
| mlima_ | i looked and did'nt see any bug that i can fix and test now. :/ | 18:23 |
| mlima_ | didn't* | 18:24 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add TLS to Kibana Web Interface https://review.openstack.org/306582 | 18:24 |
| sdake | those 2.0.0 bugs in triaged need confirming too | 18:25 |
| sdake | or they will be kicked to 2.0.1 | 18:25 |
| mlima_ | deadline is today? | 18:26 |
| sdake | today is rc4 | 18:26 |
| sdake | deadline for 2.0.0 is 21st | 18:26 |
| sdake | based upon changes in how the governance repo will be working in the future | 18:26 |
| sdake | under our current release model we can release whenever we like | 18:27 |
| sdake | what we relaly want is this https://review.openstack.org/#/c/300752/ | 18:27 |
| patchbot | sdake: patch 300752 - governance - Change kolla to release:cycle-trailing | 18:27 |
| mlima_ | it will be fixed with ansible 2.0 https://bugs.launchpad.net/kolla/+bug/1520728 ? | 18:28 |
| openstack | Launchpad bug 1520728 in kolla mitaka "fatal: [openstack002] => One or more undefined variables: 'dict object' has no attribute 'stdout'" [Critical,Triaged] | 18:28 |
| sdake | i dont eeven know if that is still a bug | 18:28 |
| sdake | "fixed in ansible 2.0" is not an acceptable answer imo | 18:29 |
| sdake | it needs a workaroun | 18:29 |
| sdake | we worked daround another ansible 2.0 bug | 18:29 |
| mlima_ | hmm, but as this is ansible limitation, i think this is not a critical bug. | 18:30 |
| mlima_ | maybe medium | 18:30 |
| sdake | i dont know this problem still exists | 18:30 |
| sdake | anything that causes deployment to fail is a critical bug | 18:30 |
| mlima_ | oh | 18:30 |
| mlima_ | :) | 18:30 |
| mlima_ | ok | 18:30 |
| sdake | if its a dependency problem we need to sort out how to work around it | 18:30 |
| MarMat_ | isn't https://bugs.launchpad.net/kolla/+bug/1520728 it the same issue as https://bugs.launchpad.net/kolla/+bug/1546789 which has been workarounded in site.yml in recently? | 18:33 |
| openstack | Launchpad bug 1520728 in kolla mitaka "fatal: [openstack002] => One or more undefined variables: 'dict object' has no attribute 'stdout'" [Critical,Triaged] | 18:33 |
| openstack | Launchpad bug 1546789 in kolla mitaka "neutron failed to deploy in a multi-node deployment" [Critical,Fix committed] - Assigned to Steven Dake (sdake) | 18:33 |
| sdake | MarMat_ i think its a dupe but witing for confirmation | 18:33 |
| *** mwheckmann has quit IRC | 18:33 | |
| sdake | https://review.openstack.org/#/c/306582/1 needs an ack plz | 18:35 |
| patchbot | sdake: patch 306582 - kolla (stable/mitaka) - Add TLS to Kibana Web Interface | 18:35 |
| openstackgerrit | Merged openstack/kolla: Add TLS to Kibana Web Interface https://review.openstack.org/306571 | 18:48 |
| *** vhosakot has joined #openstack-kolla | 18:48 | |
| *** sdake_ has joined #openstack-kolla | 18:49 | |
| *** mark-casey1 has quit IRC | 18:50 | |
| *** mark-casey has joined #openstack-kolla | 18:50 | |
| mlima_ | sdake, how should be the output of this https://blueprints.launchpad.net/kolla/+spec/diag-env ? | 18:51 |
| mlima_ | all in one file? | 18:51 |
| sdake_ | yup all lin one fie | 18:51 |
| sdake_ | tools/diag | 18:52 |
| sdake_ | all the stuff we ask people to create during a ive deploy session | 18:52 |
| sdake_ | so we can get all of the information in one go | 18:52 |
| *** sdake has quit IRC | 18:52 | |
| sdake_ | without having to ask a million questions | 18:52 |
| sdake_ | over and over ;) | 18:52 |
| mlima_ | yes | 18:52 |
| mlima_ | better create it than a session of questions and answers frequently ^^ | 18:53 |
| sdake_ | agreed | 18:53 |
| sdake_ | at this point 95% of deployment failures are user errors | 18:54 |
| mlima_ | well, i'll start it | 18:54 |
| sdake_ | we cover most of those in our prechecks | 18:54 |
| vhosakot | sdake_: I like that diag bp... we should add some docker commands as well like "docker images", "docker ps -a", "docker images", "ps aux | grep docker", etc etc | 18:54 |
| sdake_ | vhosakot that is what the whiteboard is for - feel free to add em in :) | 18:54 |
| mlima_ | i'll start and you can suggest more diags | 18:55 |
| vhosakot | sdake_: yep... it should dump everything in operator's env and we can call it dump_world.sh :) | 18:55 |
| vhosakot | mlima_: cool.. sdake_: ok, will add more commands | 18:56 |
| inc0 | I think we need to ask ops what they really need;) | 18:56 |
| *** banix has quit IRC | 18:56 | |
| vhosakot | ports, processes, docker daemon status, containers, images, everything | 18:56 |
| vhosakot | inc0: ys, agreed | 18:57 |
| inc0 | I'd rather make good ops guide | 18:57 |
| inc0 | once we're running, how to keep kolla running | 18:57 |
| inc0 | how to access logs, how to check status of stuff | 18:58 |
| mimizone | should I worry about the fact that some images do get built? magnum/designate/mistral/gnocchi? | 18:58 |
| mimizone | don't get built I meant | 18:58 |
| vhosakot | mimizone: in gate ? | 18:58 |
| inc0 | mimizone, depends, do you want to deploy them? | 18:58 |
| mlima_ | in bp summary have systemctl | grep failed | 18:58 |
| mlima_ | which ubuntu command? | 18:59 |
| mimizone | inc0: this one may be a problem ERROR:kolla.cmd.build:neutron-openvswitch-agent Failed with status: error | 18:59 |
| mimizone | the other not much | 18:59 |
| inc0 | yeah, you kinda want to debug this | 18:59 |
| inc0 | failure should be in logs above | 18:59 |
| inc0 | so jsut grep logs by neutron-openvswitch-agent and see what's wrong | 18:59 |
| ccesario | mlima_, sdake_ in my tests https://bugs.launchpad.net/kolla/+bug/1520728 this is already fixed | 19:00 |
| openstack | Launchpad bug 1520728 in kolla mitaka "fatal: [openstack002] => One or more undefined variables: 'dict object' has no attribute 'stdout'" [Critical,Triaged] | 19:00 |
| mimizone | My shell didn't keep 10min of logs. are the logs saved somewhere? | 19:00 |
| inc0 | ccesario "fixed", as in doesn't appear any more, but still can | 19:01 |
| inc0 | however we will move to ansible 2.0 and it will get truly fixed then | 19:02 |
| vhosakot | mlima_: that command is for systemd based OS's like CentOS and Ubuntu (15.10), on Ubuntu 14.04, it would be "service --status-all" | 19:02 |
| mlima_ | vhosakot, thanks | 19:03 |
| vhosakot | mlima_: cool | 19:03 |
| inc0 | let's not make shortcut to shortcut plz | 19:03 |
| mimizone | how do I remove all images/clean up everything to attempt a new build? | 19:03 |
| inc0 | operator should know how to use systemd | 19:03 |
| inc0 | mimizone, tools/cleanup-containers | 19:03 |
| inc0 | and cleanup-images | 19:03 |
| inc0 | on every node | 19:03 |
| ccesario | inc0, I agree, fixing my statement "Does not happen" :) | 19:03 |
| mlima_ | well, I'll start with some commands here and we will continually improving it vhosakot | 19:04 |
| vhosakot | mlima_: yep.. cool | 19:04 |
| openstackgerrit | Merged openstack/kolla: Add TLS to Kibana Web Interface https://review.openstack.org/306582 | 19:06 |
| ccesario | hey guys ... does someone any comments about this ps https://review.openstack.org/#/c/306443/ | 19:08 |
| patchbot | ccesario: patch 306443 - kolla - Make Kibana elasticsearch.url param point to VIP w... | 19:08 |
| ccesario | in a short talk with dave-mcc_ maybe is neeeded more opinion about it .... | 19:08 |
| ccesario | rigth dave-mcc_ ?! :) | 19:08 |
| dave-mcc_ | ccesario +1 | 19:09 |
| dave-mcc_ | the config for heka and kibana made opposite design decisions. both use elasticsearch as a backend. | 19:09 |
| *** sdake_ has quit IRC | 19:10 | |
| dave-mcc_ | haproxy first load balances between kibana instances (one on each control node). then kibana chooses which backend instance of elasticsearch to use. | 19:10 |
| dave-mcc_ | should kibana always use the one that runs on the same node? or should kibana let haproxy decide which elasticsearch to use via load balancing? | 19:11 |
| vhosakot | dave-mcc_: great question | 19:11 |
| dave-mcc_ | in the code today, heka let's haproxy decide. kibana always chooses the local one. | 19:11 |
| *** sdake has joined #openstack-kolla | 19:12 | |
| *** mgoddard has quit IRC | 19:14 | |
| vhosakot | dave-mcc_: isn't it good if kibana load balances (thru heka/haproxy) incase local ES is down ? | 19:14 |
| ccesario | vhosakot, good point | 19:15 |
| dave-mcc_ | what if the remote ES is down? we don't do any app layer monitoring. | 19:15 |
| ccesario | in this case the proposed ps works to solve it | 19:15 |
| dave-mcc_ | keepalived only monitors in the network interface. if there is an app layer failure, haproxy won't know and can't help. | 19:16 |
| *** aginwala has joined #openstack-kolla | 19:17 | |
| vhosakot | dave-mcc_: yes, there is no ES monitoring... remote ES being down is a valid use case.. in the current way, if the remote ES (chosen by haproxy) is down, then kibana cannot access ES... | 19:17 |
| vhosakot | dave-mcc_: is there a way to make haproxy aware of the status if the remote ES chosen during load-balacing (app level monitoing done by haproxy) ? | 19:18 |
| *** mlima__ has joined #openstack-kolla | 19:18 | |
| *** v1k0d3n has joined #openstack-kolla | 19:19 | |
| *** Marga__ has joined #openstack-kolla | 19:19 | |
| *** aginwala_ has joined #openstack-kolla | 19:19 | |
| dave-mcc_ | vhosakot yes. but, if we're adding service level monitoring, there are higher priority services to monitor first. (like keystone) | 19:19 |
| *** ravig__ has joined #openstack-kolla | 19:19 | |
| *** salv-orlando has joined #openstack-kolla | 19:20 | |
| dave-mcc_ | we should determine if the kibana/elasticsearch connection is a long-lived one or if a new connection is made for each query. | 19:21 |
| *** unicell1 has joined #openstack-kolla | 19:21 | |
| *** serverascode_ has joined #openstack-kolla | 19:22 | |
| *** manjeets_ has joined #openstack-kolla | 19:22 | |
| *** serverascode has quit IRC | 19:22 | |
| *** manjeets has quit IRC | 19:22 | |
| *** severion has quit IRC | 19:22 | |
| *** Marga_ has quit IRC | 19:22 | |
| *** ravig has quit IRC | 19:22 | |
| *** unicell has quit IRC | 19:22 | |
| *** salv-orl_ has quit IRC | 19:22 | |
| *** ntpttr_ has quit IRC | 19:22 | |
| *** aginwala has quit IRC | 19:22 | |
| *** mlima_ has quit IRC | 19:22 | |
| *** aginwala_ has quit IRC | 19:23 | |
| *** dwalsh has quit IRC | 19:23 | |
| *** serverascode_ is now known as serverascode | 19:23 | |
| vhosakot | dave-mcc_: is app level-monitoring port-moniroting on all controllers ? | 19:25 |
| *** ravig__ has quit IRC | 19:25 | |
| sdake | not for mitaka ;) | 19:26 |
| dave-mcc_ | there is no app level or port level monitoring at all. | 19:26 |
| *** ravig has joined #openstack-kolla | 19:27 | |
| *** aginwala has joined #openstack-kolla | 19:27 | |
| vhosakot | dave-mcc_: right... what exactly is app-level monitoring ? is it the same as port-monitoring (haproxy checks if the port of the chosen service is up, if up, then use it, if down, check the port on the next contoller..something like this ?) | 19:28 |
| *** ravig has quit IRC | 19:29 | |
| vhosakot | sdake: yes, not for mitaka... did you tag yet ? just checking | 19:29 |
| dave-mcc_ | vhosakot something like that. we can design it later. | 19:30 |
| *** banix has joined #openstack-kolla | 19:30 | |
| sdake | i will be tagging rc4 shortly | 19:30 |
| sdake | 2.0.0 tag deadline is 21st | 19:30 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: [WIP] Diagnostics for Environment https://review.openstack.org/306608 | 19:30 |
| vhosakot | ah ok... | 19:30 |
| sdake | 1.1.0 tag deadline is 21st if we can get it workign in time ;) | 19:30 |
| vhosakot | dave-mcc_: cool.. I did not know every query to kibana is _double_ load balanced | 19:30 |
| vhosakot | sdake: cool | 19:31 |
| mimizone | inc0: here the specific error "The command '/bin/sh -c apt-get install -y --no-install-recommends neutron-plugin-openvswitch openvswitch-switch neutron-plugin-openvswitch-agent && apt-get clean' returned a non-zero code: 100 " | 19:31 |
| dave-mcc_ | vhosakot it's not. that is what ccesario is proposing in the patch set we're discussing. | 19:31 |
| mimizone | and the full logs for the neutron-openvswitch-agent http://paste.openstack.org/show/494276/ | 19:31 |
| mlima__ | I need more suggestions here https://review.openstack.org/#/c/306608/1 | 19:31 |
| patchbot | mlima__: patch 306608 - kolla - [WIP] Diagnostics for Environment | 19:31 |
| inc0 | look up in logs what was the error of apt-get installs | 19:31 |
| vhosakot | dave-mcc_: ah ok ok... now I understand fully.. it is ccesario's proposal.. ok.. I will check ccesario's PS then.. cool | 19:32 |
| mimizone | inc0: maybe this ? "neutron-plugin-openvswitch : Depends: neutron-common (= 1:2014.1.5-0ubuntu4) but 2:8.0.0-0ubuntu1~cloud0 is to be installed" | 19:32 |
| *** salv-orl_ has joined #openstack-kolla | 19:33 | |
| inc0 | mimizone, yeah | 19:33 |
| *** salv-orl_ has quit IRC | 19:33 | |
| sdake | ubuntu binary build? | 19:33 |
| inc0 | looks bad | 19:33 |
| mimizone | yep : binary build ubuntu | 19:33 |
| inc0 | ubuntu binary isnt working | 19:33 |
| dave-mcc_ | to answer the question (to double-lb or not to double-lb), we need to better understand the nature of the connection between kibana and elasticsearch, and the load caused by elasticsearch. | 19:33 |
| inc0 | use ubuntu-source | 19:33 |
| mimizone | inc0: arrrrr | 19:33 |
| mimizone | inc0: will do that :) | 19:33 |
| dave-mcc_ | vhosakot otherwise, we're just guessing. | 19:33 |
| *** salv-orl_ has joined #openstack-kolla | 19:34 | |
| inc0 | we don't really have good repos for ubuntu binary | 19:34 |
| sdake | 2.0.1 will address ubuntu binary more consistently | 19:34 |
| *** SiRiuS__ has joined #openstack-kolla | 19:34 | |
| sdake | its a repo problem for the most part | 19:34 |
| vhosakot | dave-mcc_: yep, I'm checking the code... | 19:34 |
| mimizone | I guess building from source is longer? | 19:34 |
| sdake | mimizone takes aboutsaem amount of time oddly | 19:35 |
| dave-mcc_ | mimizone about the same on centos | 19:35 |
| sdake | ya 14 vs 16 minutes son centos on my gear | 19:35 |
| ccesario | dave-mcc_, vhosakot great details :) | 19:36 |
| *** salv-orlando has quit IRC | 19:36 | |
| sdake | i am waiting on this patch to merge to tag rc4 https://review.openstack.org/#/c/306576/ | 19:38 |
| patchbot | sdake: patch 306576 - kolla (stable/mitaka) - Rabbitmq Upgrade restarts container only when imag... | 19:38 |
| *** salv-orl_ has quit IRC | 19:39 | |
| *** achanda has joined #openstack-kolla | 19:42 | |
| dave-mcc_ | ccesario vhosakot if we find that the kibana/es connection is longlived, we should go with the local one. if we find the connection are short lived and queries are CPU intensive, then we should load balance each query. | 19:44 |
| vhosakot | dave-mcc_: this is the point in code that connects kibana with ES... https://github.com/openstack/kolla/blob/master/ansible/roles/kibana/templates/kibana.yml.j2#L4 | 19:44 |
| sdake | say, does elastic search use a named volume | 19:45 |
| vhosakot | dave-mcc_: if each kibaan query eats the contoller's CPU, good to loadbalance ES | 19:45 |
| sdake | tonyb_ what tag did you suggest i use for the liberty 1.0.0 EOL | 19:47 |
| *** aginwala has quit IRC | 19:48 | |
| sdake | followed up by a liberty 1.1.0 tag later after we totally replace the repo contents? | 19:48 |
| openstackgerrit | Merged openstack/kolla: Rabbitmq Upgrade restarts container only when image was changed. https://review.openstack.org/306576 | 19:51 |
| *** Serlex has quit IRC | 19:51 | |
| sdake | ok rc4 tag incoming | 19:52 |
| vhosakot | I think the "longevity" of the connection between kibana and ES depends on the "size" of the log requested by the user in Kibana GUI | 19:53 |
| vhosakot | sdake: cool... | 19:53 |
| *** aginwala has joined #openstack-kolla | 19:53 | |
| *** mimizone has quit IRC | 19:53 | |
| *** manjeets_ is now known as manjeets | 19:53 | |
| *** mimizone has joined #openstack-kolla | 19:54 | |
| dave-mcc_ | vhosakot i would guess that too. but it's also possible that kibana keeps a connection open for ten queries in a row. | 20:00 |
| vhosakot | dave-mcc_: right, that is possible.. | 20:01 |
| *** aginwala has quit IRC | 20:01 | |
| *** cfarquhar has quit IRC | 20:03 | |
| *** aginwala has joined #openstack-kolla | 20:04 | |
| *** aginwala has quit IRC | 20:04 | |
| *** jidar has quit IRC | 20:04 | |
| *** aginwala has joined #openstack-kolla | 20:05 | |
| *** achanda has quit IRC | 20:05 | |
| *** cfarquhar has joined #openstack-kolla | 20:06 | |
| *** cfarquhar has joined #openstack-kolla | 20:06 | |
| ccesario | vhosakot, dave-mcc_ http://everythingshouldbevirtual.com/highly-available-elk-elasticsearch-logstash-kibana-setup there are some infos | 20:06 |
| ccesario | indeed interesting configs | 20:07 |
| *** mlima__ has quit IRC | 20:11 | |
| *** unicell1 has quit IRC | 20:11 | |
| *** unicell has joined #openstack-kolla | 20:11 | |
| openstackgerrit | Alicja Kwasniewska proposed openstack/kolla: Add Kibana quick start guide https://review.openstack.org/306019 | 20:18 |
| vhosakot | ccesario: ping | 20:19 |
| vhosakot | ccesario: is 192.168.201.100 your kolla_internal_vip_address in globals.yml ? | 20:20 |
| *** aginwala has quit IRC | 20:20 | |
| ccesario | vhosakot, pong | 20:20 |
| ccesario | vhosakot, yes | 20:20 |
| *** aginwala has joined #openstack-kolla | 20:21 | |
| *** aginwala has quit IRC | 20:22 | |
| vhosakot | ccesario: you have just one control node ? 192.168.201.2 ? | 20:22 |
| *** mimizone has quit IRC | 20:23 | |
| ccesario | vhosakot, two | 20:23 |
| ccesario | server kolla-control 192.168.201.2:9200 check inter 2000 rise 2 fall 5 | 20:23 |
| ccesario | server kolla-network 192.168.201.3:9200 check inter 2000 rise 2 fall 5 | 20:23 |
| *** aginwala has joined #openstack-kolla | 20:23 | |
| ccesario | the names are not suggestive :P | 20:23 |
| ccesario | but only to tests | 20:24 |
| vhosakot | ccesario: https://bugs.launchpad.net/kolla/+bug/1570550 and http://paste.openstack.org/show/493819/ do not have 192.168.201.3 | 20:24 |
| openstack | Launchpad bug 1570550 in kolla "elasticsearch.url param point to host address when haproxy it is enabled" [Undecided,In progress] - Assigned to Carlos Cesario (ccesario) | 20:24 |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add Kuryr ansible role https://review.openstack.org/298894 | 20:24 |
| ccesario | vhosakot, look my comment in ps | 20:24 |
| ccesario | vhosakot, I need update it | 20:25 |
| ccesario | wrong config in this paste | 20:25 |
| *** aginwala has quit IRC | 20:25 | |
| ccesario | vhosakot, my inventory file | 20:26 |
| ccesario | http://paste.openstack.org/show/494284/ | 20:26 |
| *** aginwala has joined #openstack-kolla | 20:27 | |
| vhosakot | ccesario: do you have to type 192.168.201.2:5601 in the broswer to open kibana GUI ? | 20:27 |
| ccesario | yes | 20:27 |
| ccesario | in both | 20:27 |
| ccesario | 192.168.201.2:5601 and 192.168.201.3:5601 | 20:27 |
| vhosakot | ccesario: on respective control nodes ? | 20:27 |
| *** mimizone has joined #openstack-kolla | 20:28 | |
| ccesario | yes | 20:28 |
| vhosakot | ccesario: you need to type 192.168.201.2:5601 on first control node and type 192.168.201.3:5601 on the seocnd control node ? | 20:28 |
| *** achanda has joined #openstack-kolla | 20:29 | |
| ccesario | vhosakot, I did it and kibana Gui it is accessed.... | 20:30 |
| openstackgerrit | Hui Kang proposed openstack/kolla: Implement ceilometer ansible role https://review.openstack.org/300574 | 20:31 |
| *** jidar has joined #openstack-kolla | 20:31 | |
| ccesario | vhosakot, need talk with a customer.... give some minutes ... | 20:32 |
| vhosakot | ccesario: cool! | 20:32 |
| sdake | inc0 if your to be a ptl some day you shoud live in the openstack-release channel ;) | 20:33 |
| openstackgerrit | Hui Kang proposed openstack/kolla: Implement ceilometer ansible role https://review.openstack.org/300574 | 20:33 |
| *** dwalsh has joined #openstack-kolla | 20:33 | |
| sdake | rhallisey ^^ | 20:34 |
| *** aginwala has quit IRC | 20:34 | |
| vhosakot | sdake: got a bunch of emails with subject "Fix included in openstack/kolla 2.0.0.0rc4" from dims :) | 20:34 |
| sdake | note inc0 rhallisey neither of you can tag anymore - nor can I, we have to submit via the releases repo | 20:34 |
| *** achanda has quit IRC | 20:34 | |
| vhosakot | sdake: does it mean rc4 tagging is done ? | 20:34 |
| dims | vhosakot : sdake made me do it! | 20:35 |
| sdake | yup rc4 is tagged | 20:35 |
| rhallisey | kk | 20:35 |
| sdake | plesae test | 20:35 |
| vhosakot | nice!... rc4 tagged | 20:35 |
| sdake | email inc to ml :) | 20:35 |
| openstackgerrit | Hui Kang proposed openstack/kolla: Implement ceilometer ansible role https://review.openstack.org/300574 | 20:36 |
| *** salv-orlando has joined #openstack-kolla | 20:38 | |
| *** salv-orlando has joined #openstack-kolla | 20:38 | |
| *** ccesario___ has quit IRC | 20:45 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Backport megapatch https://review.openstack.org/306625 | 20:56 |
| inc0 | here goes nothing | 20:57 |
| inc0 | yay for stackalytics bump | 20:57 |
| inc0 | sdake, ^^ | 20:57 |
| sdake | inc0 go to #kolla | 20:57 |
| sdake | rather go to #openstack-releae | 20:57 |
| sdake | inc0 rofl | 20:58 |
| sdake | inc0 reading tonyb's log kicked off a thought in my head | 20:58 |
| sdake | dangerous I knwo :) | 20:58 |
| rhallisey | 20k lines | 20:58 |
| rhallisey | o.o | 20:58 |
| sdake | inc0 if you ptu your patch on top it would be enice to see if the gates work ;) | 20:58 |
| inc0 | I might go for a record of biggest patch in history of openstack | 20:58 |
| sdake | as in a second patch on top of the first | 20:58 |
| sdake | so here was tonyb's idea I think | 20:59 |
| sdake | tag kolla with liberty-early-demise | 20:59 |
| sdake | rather tag liberty branch | 20:59 |
| sdake | at its current head state | 20:59 |
| sdake | then delete liberty branch | 20:59 |
| inc0 | if you can do this | 20:59 |
| sdake | then create liberty branch from mitaka branch | 20:59 |
| sdake | that keeps all history! | 20:59 |
| sdake | dims thoughts? | 21:00 |
| sdake | that is a perfect solution imo | 21:01 |
| sdake | inc0 join openstack-infra | 21:01 |
| sdake | lets boucne this idea off them | 21:01 |
| dims | sdake : +1 to get opinions | 21:02 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Making Mitaka deploying liberty https://review.openstack.org/299563 | 21:02 |
| *** ccesario___ has joined #openstack-kolla | 21:02 | |
| *** SiRiuS__ has quit IRC | 21:03 | |
| sdake | inc0 you also have to live on openstack-infra ;) | 21:04 |
| sdake | please join | 21:04 |
| inc0 | I'll add both to autojoin... | 21:04 |
| sdake | sounds good | 21:05 |
| sdake | rhallisey recommend the same for you | 21:05 |
| inc0 | ok ready for beating | 21:05 |
| sdake | ;-) | 21:06 |
| sdake | ill take the beating | 21:06 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Making Mitaka deploying liberty https://review.openstack.org/306630 | 21:12 |
| *** dwalsh has quit IRC | 21:12 | |
| sdake | dims would you be willing to tag as such if infra is good with this approach? | 21:12 |
| dims | sdake : could it wait till monday AM? | 21:14 |
| sdake | dims yup | 21:15 |
| sdake | just wondering if it is somethign you would consider | 21:15 |
| sdake | if not, then no point asking infra if they are good with that model | 21:15 |
| dims | sdake : asking infra would cement the plan we can present to dhellmann | 21:18 |
| sdake | sounds good | 21:18 |
| tonyb_ | sdake: Yeah the tagging preserves the history bt you don't need to deleet the branch just resetting it to the contents of mitaka is what you neeed | 21:20 |
| sdake | tonyb how do you do that? | 21:21 |
| tonyb_ | you're going to need the branch to add the "make it install liberty tarballs NOT mitaka ones | 21:21 |
| sdake | yes w ecan do that seprately | 21:21 |
| tonyb_ | sdake: let me hack something up on github | 21:22 |
| sdake | but the quetion is how to reset the repo to stable/mitaka? again, far from git expert | 21:22 |
| sdake | I teach people git rebase -i all the time and they think Im a wizard... but far f rom it ;) | 21:22 |
| tonyb_ | sdake: git reset --hard origin/stable/mitaka | 21:23 |
| sdake | while in the liberty branch? | 21:23 |
| inc0 | tonyb_, so reset will change HEAD | 21:24 |
| inc0 | so basically what will happen is we'll end up having 1000 patches | 21:24 |
| inc0 | but what I did is reset --hard on mitaka, then manually hange .git/HEAD and .gitreview | 21:25 |
| inc0 | or reather reset --soft | 21:25 |
| inc0 | I did lot of resets.. | 21:25 |
| tonyb_ | Shouldn't be needed | 21:26 |
| tonyb_ | let me hack this up | 21:26 |
| *** ccesario___ has quit IRC | 21:26 | |
| *** ccesario___ has joined #openstack-kolla | 21:26 | |
| tonyb_ | Ah you're right you *do* need to delete the branch because our setup prevents non-fast-forward pushes | 21:29 |
| tonyb_ | here's what the repo will look like : https://github.com/tbreeds/kolla/commits/stable/liberty | 21:31 |
| *** ccesario___ has quit IRC | 21:31 | |
| sdake | tonyb_ arey u on openstack-infra ml? | 21:33 |
| sdake | if not, would you like me to cc you on this email? | 21:34 |
| tonyb_ | sdake: Yeah | 21:34 |
| tonyb_ | that yeah i'm on the ML | 21:34 |
| sdake | which one :) | 21:34 |
| sdake | ok | 21:34 |
| sdake | thanks | 21:34 |
| inc0 | tonyb_, so if we can bypass gerrit to make this change, cool | 21:35 |
| tonyb_ | Here's how I did it: http://paste.openstack.org/show/494295/ | 21:35 |
| inc0 | well if we have control over github;) | 21:36 |
| tonyb_ | inc0: Yeah I think that just needs an extra step but infra will have to weigh in on that. | 21:36 |
| tonyb_ | inc0: that was justa quick way to show you what the repo would look like ;P | 21:36 |
| inc0 | sdake, gates failed because of gate setup | 21:39 |
| *** ccesario___ has joined #openstack-kolla | 21:39 | |
| sdake | email away to infra | 21:39 |
| sdake | inc0 given the workload, not sure if this will be landable by 21st as well | 21:39 |
| inc0 | duh | 21:40 |
| inc0 | we can always merge megapatch;) | 21:40 |
| sdake | rather then work towards some arbitrary deadline, lets get the right answer from infra | 21:41 |
| tonyb_ | inc0: Yes you could as a fallback but that's a really really last resort | 21:41 |
| sdake | i expect they will be happy to offer a best path foward, even if it means megapatch | 21:41 |
| sdake | these folks are git experts | 21:41 |
| tonyb_ | sdake: yeah open the dialouge I'll chime in as needed. | 21:41 |
| inc0 | sure, let's do that | 21:42 |
| *** ccesario___ has quit IRC | 21:43 | |
| *** ccesario___ has joined #openstack-kolla | 21:43 | |
| tonyb_ | It's <8am on Saturday here I need to unplug | 21:43 |
| tonyb_ | Thanks all. | 21:43 |
| sdake | thanks tobnby | 21:43 |
| sdake | appreciat theh elp | 21:43 |
| inc0 | ok, I'm going out as well, my work here is done;) | 21:47 |
| inc0 | will play around it over weekend probabl | 21:47 |
| inc0 | y | 21:47 |
| *** ccesario___ has quit IRC | 21:48 | |
| *** inc0 has quit IRC | 21:51 | |
| *** rhallisey has quit IRC | 21:52 | |
| *** haplo37 has quit IRC | 22:00 | |
| *** mwheckmann has joined #openstack-kolla | 22:00 | |
| *** ccesario___ has joined #openstack-kolla | 22:18 | |
| *** mdnadeem_ has joined #openstack-kolla | 22:21 | |
| *** vhosakot has quit IRC | 22:21 | |
| *** mdnadeem has quit IRC | 22:24 | |
| *** mwheckmann has quit IRC | 22:29 | |
| *** banix has quit IRC | 22:35 | |
| *** rhallisey has joined #openstack-kolla | 23:01 | |
| *** banix has joined #openstack-kolla | 23:05 | |
| *** rhallisey has quit IRC | 23:13 | |
| *** koperg has quit IRC | 23:13 | |
| *** ravig has joined #openstack-kolla | 23:20 | |
| *** weiyu_ has joined #openstack-kolla | 23:21 | |
| *** tonyb_ is now known as tonyb | 23:33 | |
| *** MarMat_ has quit IRC | 23:33 | |
| *** vhosakot has joined #openstack-kolla | 23:36 | |
| *** mark-casey has quit IRC | 23:37 | |
| *** weiyu_ has quit IRC | 23:38 | |
| *** ravig has quit IRC | 23:41 | |
| *** weiyu_ has joined #openstack-kolla | 23:43 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Patch for Serguei to test Cinder round robin https://review.openstack.org/306662 | 23:43 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Patch for Serguei to test Cinder round robin https://review.openstack.org/306662 | 23:45 |
| *** weiyu_ has quit IRC | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
