Wednesday, 2015-07-22

« Tuesday, 2015-07-21 Index Thursday, 2015-07-23 »
*** newbie_2 has quit IRC00:03
*** pmackinn has joined #openstack-trove00:22
*** ashleighfarnham has quit IRC00:24
*** _amrith_ is now known as amrith00:25
*** esmute_ has quit IRC00:29
*** esmute__ has joined #openstack-trove00:29
*** topshare has quit IRC00:34
*** topshare has joined #openstack-trove00:34
*** topshare has quit IRC00:36
*** saurabhs1 has joined #openstack-trove00:37
*** topshare has joined #openstack-trove00:38
*** esmute__ has quit IRC00:45
*** newbie_2 has joined #openstack-trove00:55
*** mingdang1 has joined #openstack-trove01:08
*** topshare has quit IRC01:09
*** topshare has joined #openstack-trove01:10
*** erkules_ has joined #openstack-trove01:11
*** esmute_ has joined #openstack-trove01:12
*** erkules has quit IRC01:13
*** newbie_2 has quit IRC01:20
*** pmackinn has quit IRC01:20
*** newbie_2 has joined #openstack-trove01:22
*** esmute_ has quit IRC01:24
*** newbie_2 has quit IRC01:25
*** esmute_ has joined #openstack-trove01:33
*** newbie_2 has joined #openstack-trove01:49
*** newbie_2 has quit IRC01:52
*** amrith is now known as _amrith_01:58
*** esmute_ has quit IRC02:08
*** esmute_ has joined #openstack-trove02:13
*** _amrith_ is now known as amrith02:30
*** newbie_2 has joined #openstack-trove02:30
*** saurabhs1 has quit IRC02:46
*** sabeen has joined #openstack-trove02:59
*** saurabhs1 has joined #openstack-trove03:04
*** newbie_2 has quit IRC03:21
*** exploreshaifali has joined #openstack-trove03:29
*** bhunter71 has quit IRC03:33
*** h00327910_ has quit IRC03:35
*** exploreshaifali has quit IRC03:36
*** georgelorch has quit IRC03:38
*** amrith is now known as _amrith_03:39
*** _amrith_ is now known as amrith03:40
*** exploreshaifali has joined #openstack-trove03:49
*** coolsvap|away is now known as coolsvap04:01
*** mingdang1 has quit IRC04:07
*** mingdang1 has joined #openstack-trove04:07
*** sgotliv has joined #openstack-trove04:12
openstackgerritSharika Pongubala proposed openstack/python-troveclient: Root enablement for Vertica clusters/instances  https://review.openstack.org/19736004:22
*** amrith is now known as _amrith_04:24
openstackgerritSharika Pongubala proposed openstack/trove: Modifying Vertica clusters to have a master node  https://review.openstack.org/20427104:34
openstackgerritOpenStack Proposal Bot proposed openstack/trove: Updated from global requirements  https://review.openstack.org/20315204:34
*** georgelorch has joined #openstack-trove04:56
*** exploreshaifali has quit IRC04:57
openstackgerritSergey Vilgelm proposed openstack/trove: [WIP] Switch to the oslo_log library  https://review.openstack.org/20382804:59
openstackgerritOpenStack Proposal Bot proposed openstack/python-troveclient: Updated from global requirements  https://review.openstack.org/20314504:59
openstackgerritOpenStack Proposal Bot proposed openstack/trove: Updated from global requirements  https://review.openstack.org/20315205:00
*** exploreshaifali has joined #openstack-trove05:06
*** newbie_2 has joined #openstack-trove05:11
*** esmute_ has quit IRC05:13
*** esmute_ has joined #openstack-trove05:16
*** ViswaV has joined #openstack-trove05:41
*** topshare has quit IRC05:51
*** ViswaV_ has joined #openstack-trove05:53
*** ViswaV has quit IRC05:53
*** ViswaV_ has quit IRC05:55
*** ViswaV has joined #openstack-trove05:56
*** newbie_2 has quit IRC05:57
*** topshare has joined #openstack-trove06:14
*** exploreshaifali has quit IRC06:20
*** topshare has quit IRC06:36
*** ViswaV has quit IRC06:38
*** sgotliv has quit IRC06:43
*** pmackinn has joined #openstack-trove06:44
*** ViswaV has joined #openstack-trove06:51
*** k4n0 has joined #openstack-trove06:55
*** coolsvap is now known as coolsvap|away07:01
*** esmute_ has quit IRC07:12
*** ViswaV has quit IRC07:21
-openstackstatus- NOTICE: zuul is undergoing an emergency restart. Jobs will be re-queued but some events may be lost.07:24
*** topshare has joined #openstack-trove07:47
*** topshare has quit IRC07:52
*** esmute_ has joined #openstack-trove08:13
*** exploreshaifali has joined #openstack-trove08:15
-openstackstatus- NOTICE: zuul jobs are currently stuck while problems with gearman are debugged08:16
*** ChanServ changes topic to "zuul jobs are currently stuck while problems with gearman are debugged"08:16
*** esmute_ has quit IRC08:17
*** pmackinn has quit IRC08:44
*** sgotliv has joined #openstack-trove08:48
*** ViswaV has joined #openstack-trove08:54
*** sgotliv has quit IRC09:01
*** sgotliv has joined #openstack-trove09:01
*** exploreshaifali has quit IRC09:10
*** exploreshaifali has joined #openstack-trove09:22
*** coolsvap|away is now known as coolsvap09:28
openstackgerritSushil Kumar proposed openstack/trove-specs: Datastore Registration API spec  https://review.openstack.org/18807209:32
*** ViswaV has quit IRC09:44
*** coolsvap is now known as coolsvap|away10:16
*** k4n0 has quit IRC11:31
*** exploreshaifali_ has joined #openstack-trove11:48
*** exploreshaifali has quit IRC11:48
*** david-lyle has joined #openstack-trove11:52
*** mingdang1 has quit IRC12:05
*** exploreshaifali_ has quit IRC12:24
*** tomblank has joined #openstack-trove12:37
openstackgerritMatthew Van Dijk proposed openstack/trove: MongoDB user management - access grant/revoke/show  https://review.openstack.org/19082112:51
*** lexholden has joined #openstack-trove12:58
*** radez_g0n3 is now known as radez12:59
*** _amrith_ is now known as amrith13:00
*** sgotliv_ has joined #openstack-trove13:05
*** sgotliv has quit IRC13:05
*** rnowling has joined #openstack-trove13:20
*** david-lyle has quit IRC13:27
*** mingdang_ has joined #openstack-trove13:40
*** amrith is now known as _amrith_13:46
*** _crobertsrh is now known as crobertsrh13:50
*** sabeen has quit IRC13:56
*** tomblank_ has joined #openstack-trove13:59
*** tomblank has quit IRC13:59
*** tomblank_ is now known as tomblank13:59
*** sabeen has joined #openstack-trove14:14
-openstackstatus- NOTICE: CI is currently recovering from an outage overnight. It is safe to recheck results with NOT_REGISTERED errors. It may take some time for zuul to work through the backlog.14:17
*** ChanServ changes topic to "CI is currently recovering from an outage overnight. It is safe to recheck results with NOT_REGISTERED errors. It may take some time for zuul to work through the backlog."14:17
*** georgelorch has quit IRC14:25
*** haomaiwang has joined #openstack-trove14:25
*** sabeen3 has joined #openstack-trove14:32
*** sabeen has quit IRC14:32
*** ChanServ changes topic to "Trove Openstack Database as a Service | Docs https://github.com/openstack/trove | Channel Logs http://eavesdrop.openstack.org/irclogs/ | IRC must read http://sackheads.org/~bnaylor/spew/away_msgs.html"14:35
-openstackstatus- NOTICE: CI is slowly recovering, please be patient while the backlog is worked through.14:35
*** georgelorch has joined #openstack-trove14:39
*** lexholden has quit IRC14:44
openstackgerritMerged openstack/trove: MongoDB cluster taskmanager's add_shard not called  https://review.openstack.org/19796214:55
*** mingdang_ has quit IRC14:55
*** haomaiwang has quit IRC15:00
*** marzif has quit IRC15:00
*** haomaiwang has joined #openstack-trove15:01
openstackgerritPetr Malik proposed openstack/trove: Add generic int-test classes  https://review.openstack.org/20327415:05
*** david-lyle has joined #openstack-trove15:09
*** haomaiwang has quit IRC15:18
*** haomaiwang has joined #openstack-trove15:19
*** ashleighfarnham has joined #openstack-trove15:39
*** rnowling has quit IRC15:40
*** dmitryme has quit IRC15:44
*** dmitryme has joined #openstack-trove15:44
*** tomblank has quit IRC15:47
*** tomblank has joined #openstack-trove15:51
*** pmackinn has joined #openstack-trove15:53
*** coolsvap|away is now known as coolsvap15:56
*** johnma has joined #openstack-trove15:57
*** tomblank_ has joined #openstack-trove16:14
*** pmackinn has quit IRC16:15
*** tomblank has quit IRC16:15
*** tomblank_ is now known as tomblank16:16
*** k4n0 has joined #openstack-trove16:19
*** newbie_2 has joined #openstack-trove16:22
*** esmute_ has joined #openstack-trove16:23
*** k4n0 has quit IRC16:34
*** coolsvap is now known as coolsvap|away16:38
*** saurabhs1 has joined #openstack-trove16:47
*** saurabhs1 has quit IRC16:52
*** pmackinn has joined #openstack-trove16:58
*** saurabhs1 has joined #openstack-trove17:00
*** lexholden has joined #openstack-trove17:06
*** pmackinn has quit IRC17:07
*** saurabhs1 has quit IRC17:14
*** saurabhs has joined #openstack-trove17:15
*** _amrith_ is now known as amrith17:18
cp16netyo amrith17:27
*** amrith is now known as _amrith_17:31
*** tomblank has quit IRC17:34
*** tomblank has joined #openstack-trove17:36
*** sgotliv_ has quit IRC17:41
*** esmute_ has quit IRC17:43
*** cweid has joined #openstack-trove17:46
*** shayneburgess has quit IRC17:49
*** shayneburgess has joined #openstack-trove17:49
*** esmute_ has joined #openstack-trove17:52
*** esmute_ has quit IRC17:57
*** rjaiswal has joined #openstack-trove18:00
*** bhunter71 has joined #openstack-trove18:02
*** esmute_ has joined #openstack-trove18:04
*** tomblank has quit IRC18:13
*** sriram_tesora has joined #openstack-trove18:21
*** tomblank has joined #openstack-trove18:25
*** tomblank_ has joined #openstack-trove18:28
*** tomblank has quit IRC18:29
*** tomblank_ is now known as tomblank18:29
openstackgerritPetr Malik proposed openstack/trove: Fix race conditions in config overrides tasks  https://review.openstack.org/19889118:35
openstackgerritMorgan Jones proposed openstack/trove: Redis Cluster Initial Implementation  https://review.openstack.org/20370218:37
*** tomblank has quit IRC18:40
openstackgerritPetr Malik proposed openstack/trove: Fix race conditions in config overrides tasks  https://review.openstack.org/19889118:40
*** tomblank has joined #openstack-trove18:41
openstackgerritSushil Kumar proposed openstack/trove-specs: Datastore Registration API spec  https://review.openstack.org/18807218:46
openstackgerritSharika Pongubala proposed openstack/python-troveclient: Root enablement for Vertica clusters/instances  https://review.openstack.org/19736019:00
*** exploreshaifali has joined #openstack-trove19:11
*** atomic77 has left #openstack-trove19:11
*** atomic77 has joined #openstack-trove19:11
*** esmute_ has quit IRC19:19
*** johnma has quit IRC19:27
*** esmute_ has joined #openstack-trove19:30
*** sgotliv_ has joined #openstack-trove19:31
*** esmute_ has quit IRC19:34
*** radez is now known as radez_g0n319:47
*** pmackinn has joined #openstack-trove19:50
*** esmute_ has joined #openstack-trove20:08
cp16netomg this is annoying the shit out of me...20:12
cp16netpercona wont commit the os_admin user grant privleges20:13
cp16neti added a ton of logs and see the transaction to add the user20:13
cp16neti even query and see the user inside the txn20:13
cp16netthen it commits and it disappears20:13
*** pmackinn has quit IRC20:17
dougshelley66cp16net, i've been doing a bunch of testing with percona 5.6 recently and i don't recall having that issue20:38
*** pmackinn has joined #openstack-trove20:39
cp16netreally?20:46
cp16neti'm using percona xtradb cluster server 5.5 package20:46
cp16netnot sure if there is something special with it or not yet20:47
openstackgerritSergey Vilgelm proposed openstack/trove: [WIP] Switch to the oslo_log library  https://review.openstack.org/20382821:07
*** tomblank has quit IRC21:13
*** david-lyle has quit IRC21:17
*** exploreshaifali has quit IRC21:18
*** crobertsrh is now known as _crobertsrh21:24
*** exploreshaifali has joined #openstack-trove21:44
*** sriram_tesora has quit IRC21:51
dougshelley66cp16net, i haven't used that package so don't know21:52
dougshelley66maybe georgelorch can help21:52
georgelorchsomeone incant my name?21:53
* georgelorch is reading21:54
georgelorchI'm not really a PXC guy but that certainly isn't normal, also if you are going to base anything on PXC you _really_ should be doing it on 5.621:54
cp16nethmm21:55
georgelorchif you need some help, I may be able to get one of our support guys to donate some time to help figure it out21:55
cp16netits really strange21:55
cp16neti add the grant for the os_admin user21:55
cp16netand then stop mysql and put the configs in place restart it and its gone21:56
georgelorchso I assume it is a single node, not part of a cluster yet, not really configured as a cluster member, wsrep not doing anything?21:56
cp16neti've been adding logging and this is where i am right now21:56
cp16netyea just a single node21:56
cp16neti'm starting to wonder if its because of the config changes21:57
cp16neti'm going to hack on it some more tonight but i'm about to go do some errands i need to do21:58
cp16netif someone would be around tomorrow that might be really helpful21:58
georgelorchcould be... cp16net, I'm sending out a casting call to our support folks to see if someone can help you out, either in here or exchanging emails21:59
cp16netok cool21:59
cp16netbtw i was using 5.5 because it was in the ubuntu trusty repo22:00
cp16netin using the percona-xtradb-cluster-server package22:00
georgelorchahh yeah cp16net, in vivid it is upgraded to 5.6 and 5.5 is getting dropped22:02
georgelorchcp16net, I have a couple of fish on the line that should be able to help you work through things...just a minute or two and should be here to get you introduced22:03
*** pmackinn has quit IRC22:05
cp16netok22:06
georgelorchcp16net, PM me your email and I'll send an intro to you and some help22:09
*** lefred has joined #openstack-trove22:12
lefredHi georgelorch ;-)22:12
georgelorchOhh we even have a lefred here as well!22:13
cp16nethi lefred22:13
lefredhi cp16net and all22:14
georgelorchlefred, cp16net here is having some issues working with PXC 5.5 ad he tries to implement PXC support within Trove22:14
cp16netyup22:14
georgelorchI was going to put him in touch w/ David but you beat me to it :)22:14
cp16netso i built an ubunut trusty with percona-xtradb-cluster-server package installed it has 5.522:15
lefredwhat's the problem ? (I may not have much time now, I'm in GMT+1)22:15
cp16netour process in trove runs a grant to create a user os_admin to manage the instance22:15
cp16netthen stops mysql and replaces the default config file with a new one that has our settings for each flavor22:16
cp16netand once mysql service starts up its missing the os_admin user in the mysql.user table22:16
lefredok some questions then:22:17
lefred1. how many nodes22:17
cp16net122:17
lefredat reboot too ?22:17
cp16nethavnt gotten further than that22:17
cp16netthe instance was never rebooted22:18
cp16netjust stop/start the mysql service a few times22:18
lefredsorry I meant restart of mysqld22:18
cp16netclear the ibd log files22:18
lefredwhat is the wsrep_cluster_address then in my.cnf when you start it ?22:19
cp16netprobally not set22:20
cp16netthe cnf file was the default one we used for the community mysql version22:20
lefredare you sure that your node (mysqld) is running and accepting your queries ?22:20
cp16netjust trying to figure out why it would be missing after the change22:20
cp16netyeah because i can make the calls22:20
lefredcp16net: if you installed PXC andhave the wsrep/galera libm you need to boostrap your node instead of starting it22:21
cp16netoh before anything else happens?22:21
lefredit depends what you have in the config (if you load the wsrep provider lib, then yes you need)22:22
lefredif not, then how do you create that user ?22:22
lefredGRANT...  or insert into mysql.user22:23
cp16netthe grant22:24
lefredok, are your mysql system tables myisam or innodb ?22:24
cp16neti'd stay the default22:24
cp16netwhich is probably myisam22:24
lefredok, let me verify then, could you paste me somewhere the my.cnf used to start mysql and add that user ?22:25
cp16netok give me a min22:26
cp16netforgive how dirty this is but its all the stuff i've collected22:29
cp16nethttps://gist.github.com/cp16net/b07b285493d3590eeff622:29
cp16netfirst logs part is the guest agent we have setting up the mysql service22:30
cp16netyou can see the GRANT call22:30
cp16netwith commit22:30
lefredcommit is not necessary with GRANT ;)22:31
cp16netand then a select for mysql.user and see the grant for os_admin22:31
cp16netits the way we have a localmysql client in the code that we reuse22:31
cp16netthen you see my manual select for users and its gone22:32
cp16neti make the same grant call22:32
cp16netand restart mysql and select again and its still there22:32
cp16netit just doesnt make sure why that would disappear when we are not deleting the data22:33
cp16neti dont see any errors in the transaction or anything22:33
*** newbie_2 has quit IRC22:38
lefredcp16net: I'm still checking22:38
cp16netok22:39
lefredcp16net: but the second time you run the select (line 68) I still see the user22:40
cp16netyeah after it was gone the first time22:41
*** exploreshaifali has quit IRC22:41
cp16netso the restart of the service didnt cause the problme22:42
cp16netthats what i was verifying22:42
cp16netoh right L6822:42
cp16netbut L107 you dont see it22:43
*** ashleighfarnham has quit IRC22:43
lefredcp16net: so you run the program, it creates the user, then the same program check the if the admin exists does the select sees it again (L67). then you stop mysql, you start it and the user when you check from the mysql client (command line) is not there anymore, right ?22:44
cp16netyea thats the scenario that i have22:44
cp16neti'll gist the whole guest agent file so you can see the other steps it does22:45
lefredok good22:45
cp16nethttps://gist.github.com/cp16net/5f2120eb93d7ce30ecca22:47
lefredsudo rm -f -R /var/lib/mysql22:53
lefredyou delete the full datadir22:53
lefred?22:53
cp16netwhere?22:54
cp16netsudo rm -f -R /var/lib/mysql/ib_logfile022:54
cp16netsudo rm -f -R /var/lib/mysql/ib_logfile122:54
lefredRunning cmd (subprocess): sudo rm -f -R /var/lib/mysql22:54
lefredah yes sorry it was wrapped22:55
lefred;-)22:55
cp16netok22:55
cp16net:-P22:55
cp16neti was a bit worried when you said that22:55
cp16neti know we do a few other things22:55
lefredcould you paste me /tmp/overrides.cnf22:59
cp16netits blank22:59
lefredok22:59
cp16netthere were none22:59
lefredcan you do ls -lh /var/lib/mysql/mysql/user.*23:03
cp16netyup23:03
cp16netfrm MYD MYI files23:03
lefredand the time on them ?23:04
lefredyou also do this "sudo mv /tmp/my.cnf.tmp /etc/mysql/my.cnf" will it be possible to see the content of both files ?23:06
cp16netthe instance i got this from i messed up23:07
cp16neti remove the package and tryed to reinstall23:07
cp16netto see what would happen23:07
cp16netanother instance that the same thing happened to i see the user.* files23:08
cp16netbut the times wont match up23:08
cp16netare you thinking they were never updated?23:09
lefredno no, I think they were updated too recently, or recreated or meanwhile somewhere else23:10
lefredbecause (I don't have ubuntu vm ready right now), but IIRC, we don't provide in our Percona/PXC packages a debian-sys-maint user, this comes from debian, it would be good to see when/how ect..23:11
lefredhowever, what I would like you to try (I need to go to bed, it's 1.15AM and I have to wake up early, soon), it's comment in your process the changes related my.cnf and see what it does23:12
cp16netthis is probably from the ubuntu repo package23:12
georgelorchlefred, I owe you a full case of bud-light next time we meet up :-p23:13
cp16neti dont blame ya :-P23:13
cp16netlefred: if i see you i owe you one :-D23:13
cp16neti was trying to see what the "original" my.cnf file is23:14
lefred;-) let's fix it first ;), leave your test results, gists here I will check later23:14
cp16netok i'll hack on this some more and see where i get tonight23:14
cp16netits 6:14pm here23:14
lefredcp16net: usually we didn't provide any my.cnf in our packaged, and I liked it, but it seems this has changed recently23:15
lefredcheers23:15
cp16netbecause i also want to try the ofiical percona package from their repo instead of ubuntu23:15
cp16netgot a few things to try as well as try 5.6 pxc23:15
georgelorchyeah, and the ubuntu 5.5 version is quite old23:16
cp16neti'm sure23:16
cp16net:-P23:16
cp16netlefred: cheer23:16
georgelorchwe tried updating it a little back but they wouldn't have any of that23:16
cp16netyeah its a pain to update repos like that23:16
cp16neti appreciate everyones time today!23:19
cp16neti'll update tomorrow where i am23:19
cp16neti think if i can get past this then i'l have most of clustering done for pxc in trove :)23:20
cp16neti think i got alot of new things to check out tho :)23:21
*** esmute_ has quit IRC23:25
*** esmute_ has joined #openstack-trove23:27
*** sabeen3 has quit IRC23:48
*** saurabhs has quit IRC23:50
« Tuesday, 2015-07-21 Index Thursday, 2015-07-23 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!