Thursday, 2014-05-01

*** ChanServ changes topic to "OpenStack Queuing and Notification Service || Smile :D || Meetings every Tuesday @ 15:00 UTC || Wiki: https://wiki.openstack.org/wiki/Marconi || Paste: http://paste.openstack.org/ || Send messages and make some noise :D"00:04
-openstackstatus- NOTICE: the gate is still fairly backed up, though nodepool is back on track and chipping away at remaining changes. some py3k/pypy node starvation is slowing recovery00:04
*** ayoung_DadMode is now known as ayoung00:35
*** ayoung is now known as ayoung_food00:36
*** mwagner_lap has joined #openstack-marconi01:17
*** ayoung_food is now known as ayoung01:19
*** nosnos has joined #openstack-marconi01:49
*** vkmc has quit IRC01:59
*** Gammo has quit IRC02:28
*** Gammo has joined #openstack-marconi02:29
*** haomai___ has joined #openstack-marconi02:51
*** haomaiwang has quit IRC02:54
*** nosnos has quit IRC03:46
*** ayoung has quit IRC04:02
*** prashanthr_ has quit IRC04:02
*** nosnos has joined #openstack-marconi04:36
*** Manishanker has joined #openstack-marconi05:41
*** Manishanker has quit IRC05:59
*** AAzza has joined #openstack-marconi06:15
*** chandan_kumar has joined #openstack-marconi06:44
*** AAzza has quit IRC06:51
*** chandan_kumar has quit IRC07:07
*** rossk_ has joined #openstack-marconi07:24
*** rossk has quit IRC07:25
*** rossk_ has quit IRC07:28
*** flaper87|afk is now known as flaper8708:37
*** flaper87 is now known as flaper87|afk08:57
*** Manishanker has joined #openstack-marconi09:05
*** Manishanker has quit IRC09:15
*** Manishanker has joined #openstack-marconi09:23
*** nosnos has quit IRC09:54
*** chandan_kumar has joined #openstack-marconi10:15
*** chandan_kumar has quit IRC10:52
*** malini_afk is now known as malini11:55
*** Manishanker has quit IRC12:31
*** sriram has joined #openstack-marconi12:54
*** mpanetta has joined #openstack-marconi13:16
*** amitgandhi has joined #openstack-marconi13:20
*** Obulpathi has joined #openstack-marconi13:25
*** alcabrera|afk is now known as alcabrera13:28
alcabreragood morning, all. o/13:32
malinioyeee!!!13:32
mpanettawoo!13:32
sriramaye aye13:32
maliniHere comes the first bencmark results http://166.78.249.105:8000/report.html13:36
mpanettaThat was fast13:36
mpanettaHmmm13:38
sriramget claims are super fast13:39
*** amitgandhi has quit IRC13:41
*** amitgandhi has joined #openstack-marconi13:41
* alcabrera reads the thing13:42
alcabrerahmmm13:42
alcabrerathese are looking pretty speedy13:42
alcabrera'cept maybe post message13:42
mpanettaWhich?13:42
maliniI think it gets slowed down with messages in the DB13:42
alcabreramakes sense13:43
maliniWe should try with empty DB/ vs DB with messsages in it13:43
sriramthis was on fresh DB right?13:43
alcabrerayup13:43
alcabreraerr13:43
alcabrerasriram: I have no idea. :P13:43
sriramalcabrera: thats ok :)13:43
malinihttp://166.78.249.105:8000/images/graphes-Transactions-mean.png13:44
malinipost_message takes longer towards the end of the test13:44
malinitht means , as DB starts getting populated13:44
*** ayoung has joined #openstack-marconi13:44
openstackgerritOpenStack Proposal Bot proposed a change to openstack/marconi: Updated from global requirements  https://review.openstack.org/9155913:45
mpanettaYeah, but you also have more concurrent connections at that time13:45
sriramhmm, interesting.13:45
mpanettaOuch, 5 seconds13:45
malinimpanetta: but why is it just slow for post_messages in tht case?13:45
mpanettaGood question13:45
mpanettaI am trying to find a mongodb thing to run while testing13:45
srirampost_claim should also slow down then.. if its slowing down for posts. but that isnt the case.13:46
malinimpanetta: We used to have a mongo DB monitoring thing, rt?13:46
maliniwhere we wud login to their website?13:46
*** balajiiyer has joined #openstack-marconi13:46
mpanettaOh, no we don't pay for that13:47
malinihmm..I thought it was a free one...13:47
mpanettaOh, not that I know of.  I only know of MMS13:48
maliniyeah,..tht is the one13:48
maliniis it a paid service?13:48
mpanettaI thought it was13:48
mpanettahttp://docs.mongodb.org/manual/administration/monitoring/13:49
malinilet me try to find my old login13:49
mpanettaYou may want to run mongotop on all 3 RS members when doing the test13:49
*** Obulpathi has quit IRC13:49
mpanettaAlso you could port forward the http console back to yourself13:50
*** Obulpathi has joined #openstack-marconi13:50
openstackgerritOpenStack Proposal Bot proposed a change to openstack/python-marconiclient: Updated from global requirements  https://review.openstack.org/9156513:50
maliniis mongotop similar to top?13:50
mpanettaIt displays all the reads and writes13:51
mpanettaI think it may show lock times as well13:51
mpanettaThe link above talks about it13:51
malinisriram: can you configure mongotop on our RS?13:52
sriramworking on it13:52
malinithx!!13:52
*** cpallares has joined #openstack-marconi13:53
mpanettasriram: Did you see the link I posted above with the different monitoring tools?13:54
sriramyep13:54
maliniI can still see our servers & basic profiling data in the mms site13:54
malinilooks like some of it might be free13:54
mpanettaOh?13:54
maliniI'll add mpanetta & sriram to the group we have13:54
mpanettaHow do I access it?13:55
maliniI'll add you & I think you'll get an email13:55
mpanettaOk13:55
mpanettayep got an email lol13:56
malinisriram, mpanetta: added you both to the group13:56
maliniWe will need to add our mongo servers to mms13:56
maliniinstall mms client in mongo servers?13:56
mpanettaHmmm13:58
*** shakamunyi has joined #openstack-marconi13:59
mpanettasriram: Can you do that?13:59
sriramlet me check on that, mongotop works on the individual boxes. I'm looking at connecting directly to the replica set13:59
srirammpanetta: ok13:59
*** cpallares has left #openstack-marconi13:59
*** cpallares has joined #openstack-marconi13:59
*** cpallares has quit IRC13:59
malinimpanetta: does mongotop give more info than MMS would?14:00
mpanettaDoubtful14:00
maliniwondering if it might be easier to set up MMS?14:00
mpanettaall you have to do to use mongotop is run it14:00
*** cpallares has joined #openstack-marconi14:00
*** chandan_kumar has joined #openstack-marconi14:01
sriramyeah, that works14:01
sriramneed to connect to entire replica set at once.14:01
malinisriram: "need to connect to entire replica set at once." Can you do that ?14:02
maliniI mean is it possible to do that?14:02
*** kgriffs|afk is now known as kgriffs14:02
sriramI'm looking at it now. it seems possible according to the docs14:02
malinihello kgriffs!!14:04
maliniwe have started doing the benchmarks today14:05
maliniThis is our first report http://166.78.249.105/benchmarks.html/20140501-1331/report.html14:05
srirammpanetta: replica set seed name, does it refer to the primary?14:07
mpanettaI think it may be the name of the replica set14:08
mpanettatest-rs1 I believe is the name for the shard, the catalog will be catalog-rs114:08
sriramok cool! :)14:09
flwangkgriffs: alcabrera: malini: ping14:15
alcabreraflwang: o/14:15
malinisriram: can you run the tests again (after cleaning out the dbs), once you have mongo monitoring in place?14:16
flwangmy travel is cancelled :(14:16
maliniflwang: :(14:16
alcabreranoooo. :(14:16
malinisriram: I have a meeting with my boss in 10 minutes14:16
flwangI assume you guys understand the reason14:16
sriramflwang: :(14:17
flwangmalini: sorry for the absence14:17
srirammalini: ok will do14:17
alcabreraflwang: I think so. you'll be missed!14:17
mpanettaAww man, that sucks flwang :(14:17
mpanettaI wanted to get to meet you14:17
flwangsorry guys, it's bad i know14:18
flwangSurvey: anybody is using Mac OS as development environment?14:26
alcabreraxubuntu 13.10 here.14:26
sriramI tried, but shifted to linux.14:27
flwangthanks, guys, I'm wondering if i should shift to Mac, seems the answer is no14:28
alcabreraI think kgriffs is using mac os14:28
amitgandhiflwang: some ppl here use a mac but install linux on it14:29
amitgandhii run it with osx14:29
srirammac runs a different version of python, also having multiple installations of python can be a pain on mac.. macports,pythonbrew lots of customizations..14:29
amitgandhiyeh you need to install homebrew to get the initial configuration set up14:30
amitgandhibut then once you have virtualenv it tends to be ok14:31
flwangnow I'm using Thindpad x230 and installed devstack + eclipse + pydev as my dev env, now I'm thinking if Mac is good os for openstack development14:31
amitgandhibut sometimes updating osx to the next major version messes with the python14:31
flwangamitgandhi: thanks a lot14:31
flwangand ubuntu 12.04 to get devstack worked perfect14:32
sriramamitgandhi: how do you deal with multiple python versions on a mac? 2.7 and 3.314:33
sriramI couldnt get tox to work properly on a mac. :|14:33
amitgandhisriram: havent had to deal with that yet =P14:33
alcabrerasriram, amitgandhi: I hope it's like working with py2/py3 on linux. python3.3 gets symlinked as python3.3 and python3 on the path and python2.7 gets symlinked as python, python2, and python2.714:34
flwanganother question, can Mac work with Mac OS + Ubuntu perfectly?14:34
alcabrerasimilarly, I also have symlinks for python2.6 and python3.4, as well as pypy and pypy314:34
sriramflwang: yeah you can dual boot using rEFIt14:36
flwangsriram: got it, thx14:36
sriramI had some problems with wifi in my earlier macbook, had to modify broadcom driver and build it from source :|14:37
sriramso wifi issues might be there, maybe its fixed with recent macbooks :)14:37
flwangthanks for the tips14:40
sriramhttp://166.78.249.105/benchmarks.html/mongo_log/mongo.log , the results of mongotop on the replica set while running a tsung test14:41
kgriffssriram: do you use homebrew on your mac?14:47
mpanettaOh man, those results suck :(14:49
sriramkgriffs: yep14:49
*** flwang has left #openstack-marconi14:49
kgriffshmm, so do I. After brew install python and python3, tox should work14:51
sriramyeah, I must revisit it. This was some time ago14:51
sriramI ran into some weird errors and just switched to using linux.14:52
kgriffsoic. If you can share a gist with the errors or something I can help14:52
sriramsure, I'll try it out soon :)14:53
mpanettasriram: You know what is weird about that log, even the reads are taking an insane amount of time.14:53
mpanettaTo local oplog anyway14:54
sriramyeah.. something is going wrong somewhere14:54
mpanettaThis is insanity14:54
*** flwang has joined #openstack-marconi14:56
alcabreramalini, kgriffs, flwang, flaper87|afk: https://wiki.openstack.org/wiki/Marconi#How_does_Marconi_compare_to_oslo.messaging.3F  -- I added a section detailing differences between Marconi and oslo.messaging15:07
alcabrerareview requested. :)15:07
alcabrerathe formatting is a little wonky15:07
flwangalcabrera: cool, thanks for working on this15:07
alcabreraI really want nested bullets in the wiki. :/15:07
kgriffsalcabrera: cool, I'll check it out15:08
alcabreranp15:08
kgriffsalcabrera: **15:08
alcabrerapfft15:08
alcabrerathanks, kgriffs!15:08
* kgriffs hates mediawiki syntax15:08
alcabreramuch better15:08
alcabreraE_TOO_MUCH_MARKUP15:09
kgriffsE_EYE_BLEED15:09
flwangalcabrera: should we add a simple statement like: oslo.messaging is a python library, Marconi is a messaging service(via REST API)?15:10
alcabreraflwang: I may have expressed that as "oslo.messsaging is a library -- Marconi is an HTTP API". Do you feel this is clear enough?15:11
flwangalcabrera: yep, i missed that line :)15:12
kgriffs<random>Etcd looks like a pretty sweet replacement for much of what you might use zookeeper for.</random>15:12
flwangalcabrera: but TBH, I would like to highlight we're service15:12
kgriffsalcabrera, balajiiyer, mpanetta: https://github.com/mailgun/vulcan15:14
kgriffs"connection and rate limiting"15:14
alcabreraflwang: "Marconi is a service over HTTP API" -- done. :)15:14
flwangalcabrera: awesome15:14
alcabrerakgriffs: more leaky buckets. :)15:16
*** chandan_kumar has quit IRC15:39
*** jchai has joined #openstack-marconi15:40
cpallareso/15:41
cpallaresHappy may day everyone!15:41
alcabreracpallares: heya!15:41
alcabreramay today be may day15:41
sriramlol15:41
cpallareslol15:41
* cpallares can't wait for may 4th and say may the 4th be with you15:42
* cpallares remembers alcabrera hasn't watched starwars 15:42
alcabreralol15:43
alcabreraI got that, though, cpallares! :P15:44
alcabreraat 12:01am EDT, a Tweet will arrive in cpallares stream15:44
cpallaresalcabrera: :D15:44
alcabrera"@cpallares May the 4th be with you."15:44
cpallareshaha15:44
cpallaresalcabrera: May's the best. There's also Geek Pride Day/Towel day/Discworld Day.15:45
alcabreraTowel Day?15:46
cpallaresTowel day = Hitchhiker's guide to the galaxy15:46
alcabreraoh!15:46
cpallaresalcabrera: They take towels15:46
cpallaresfor nutrition and comfort15:46
mpanettaDiscworld Day?15:46
alcabreraDiscworld day makes me want to Postal. :)15:46
alcabrera*go Postal15:46
mpanettaI'm not familiar with that one15:47
alcabreraDiscworld -- I believe that's a universe crafted by Terry Pratchet, mpanetta15:47
cpallaresalcabrera: I looove the discworld series :D15:47
alcabreraGoing Postal was one of the books in that universe15:47
alcabreraI've read just the one book, cpallares, but there's sooooo much to read. :D15:48
mpanettaI'm familiar with the books, just not what the day is for.15:48
alcabrerawhich is great15:48
mpanettaOh!15:48
mpanettaI only read one... I think it was discworld, so long ago...15:48
cpallaresalcabrera: they're so funny! I really like them15:48
cpallaresalcabrera: you should read more of them :P15:48
mpanettaI know it was pratchett...15:48
mpanettaHrm15:49
* alcabrera imagines a whale being born in space, alongside a flowerpot, and both plumeting seconds after15:49
mpanettahah15:49
alcabrerathat was the most memorable part of Hitchhiker's GttG for me. :)15:49
mpanettaInfinitely improbable heh15:50
alcabrerawhen is Dune Day?15:51
alcabrerawhen will the spices flow?15:51
cpallaresalcabrera: aw that part is sad15:52
cpallaresalcabrera: I like the babelfish part :P15:53
cpallaresalcabrera: and also the part where he's in some weird jungle where space is distorted, I can't remember which book that one is in.15:54
alcabrerahahaha, the babelfish15:54
alcabrera"http://en.wikipedia.org/wiki/Babel_fish_%28The_Hitchhiker%27s_Guide_to_the_Galaxy%29#Babel_fish"15:54
alcabreralovely quotes there15:54
alcabreracpallares: I don't remember that part. I do remember vaguely a planet of beds. :P15:55
*** balajiiyer1 has joined #openstack-marconi15:55
*** balajiiyer has quit IRC15:56
cpallaresalcabrera: I googled it, it's the old man on the pole in Mostly Harmless.15:57
alcabrerabeware of typos: Mostly Haremless -> Mostly Harmless -- a mistake I just made. :P15:58
cpallaresalcabrera: hahaha15:58
mpanettaThat is a fun one alcabrera15:59
cpallaresmpanetta: which discworld novel did you read?16:01
mpanettaThe first I think.  I can't remember if it was discworld or ringworld though.16:01
mpanettaOr I am horribly confused16:01
cpallaresmpanetta: The color of magic?16:02
* mpanetta is running on like 3 hrs sleep16:02
mpanettaYes!16:02
mpanettaOk, someone has apparently written a compatible replacement for MongoDB called TokuMX...16:03
mpanettaHmm, or maybe it *IS* mongodb... but highly optimized..16:04
mpanettahttp://www.tokutek.com/products/tokumx-for-mongodb/16:05
alcabrerait's some serious marketing. @_@16:05
alcabreraI'm so dubious16:05
mpanettaBleh *Patented*16:05
alcabreralike all my red flags have been set off16:05
mpanettaYeah16:05
mpanettaAny time someone says *patented* in a flyer...16:05
mpanettaI go...  Ok NEXT16:05
alcabrerayup16:06
mpanettaSo what if it is patented...16:06
mpanettaPatented means jack all... heh16:06
alcabreraif anything, patented tells me that I have every reason to mistrust them, because they stand in the way of progress16:06
mpanettaPretty much16:06
alcabrerashow me a research paper, tell me how the development of your product contributed to Science, but don't tell me you used Secret Sauce16:07
mpanettaExactly16:07
*** Obulpathi has quit IRC16:51
*** rossk has joined #openstack-marconi16:54
*** shakamunyi has quit IRC17:09
*** Obulpathi has joined #openstack-marconi17:11
*** balajiiyer1 has quit IRC17:12
*** reed has joined #openstack-marconi17:30
*** reed has quit IRC17:32
*** reed has joined #openstack-marconi17:32
*** jchai is now known as jchai_afk17:44
*** balajiiyer has joined #openstack-marconi17:53
*** ykaplan has joined #openstack-marconi17:54
*** alcabrera is now known as alcabrera|afk19:07
*** kgriffs is now known as kgriffs|afk19:44
*** schofield has joined #openstack-marconi19:56
*** schofield has quit IRC19:56
*** schofield has joined #openstack-marconi19:57
*** ykaplan has quit IRC20:03
*** anamalagon has joined #openstack-marconi20:03
*** schofield has left #openstack-marconi20:04
*** kgriffs|afk is now known as kgriffs20:05
*** flaper87|afk is now known as flaper8720:24
*** malini is now known as malini_afk20:24
*** malini_afk is now known as malini20:25
*** kgriffs is now known as kgriffs|afk20:27
*** malini is now known as malini_afk20:27
*** malini_afk is now known as malini20:29
*** sriram has quit IRC20:33
*** kgriffs|afk is now known as kgriffs20:38
*** jchai_afk is now known as jchai20:38
*** balajiiyer1 has joined #openstack-marconi20:46
*** balajiiyer has quit IRC20:48
*** balajiiyer has joined #openstack-marconi20:48
*** cpallares has quit IRC20:50
*** balajiiyer1 has quit IRC20:51
*** jchai is now known as jchai_afk20:55
maliniheyy flaper87!!21:00
flaper87hey :D21:00
flaper87isn't it holiday in the US too?21:01
maliniI havent seen you all day! But am not sure if it was bcoz of me not being here or you :D21:01
flaper87I guess not21:01
* flaper87 thought Labors day was a worldwide holiday21:01
maliniflaper87: you have a holiday today?21:01
maliniUS has a different labor day :D21:01
flaper87oh, there you go21:01
maliniits September 121:01
maliniI dont know why21:01
flaper87I was forced to leave home and do something away from my laptop21:02
flaper87but I'm back now21:02
flaper87:P21:02
maliniwe have started running some benchmarks today21:02
flaper87malini: AWESOME!21:06
flaper87malini: using tsung?21:06
maliniyes.21:06
malinihttp://166.78.249.105/benchmarks.html/20140501-1522/report.html21:06
flaper87what criteria are we using ?21:06
flaper87cool, thanks21:06
flaper87malini: this is a single Marconi node, single mongodb node21:07
flaper87right ?21:07
maliniwe need to do some tuning & hopefully can start sharing the numbers in ML next week21:07
maliniflaper87: no :(21:07
*** cpallares has joined #openstack-marconi21:07
malini4 webheads & a 3 node replica set21:07
flaper87malini: dang21:07
flaper87I... don't like these numbers21:08
flaper87Lets organize a performance hackathon at the summit21:08
malinitht is a good idea!21:08
flaper87we can use the developer's lounge and profile marconi21:09
flaper87fix whatever can be fixed21:09
flaper87and make it fast21:09
malinibut it'll be good if we can do some tuning before that & start sharing the numbers21:09
flaper87malini: TBH, I don't think anyone will pay attention to those numbers at this point21:10
flaper87we're basically 1 week away from the summit21:10
flaper87and folks are heads down in design sessions21:11
malinigood point..everybody are busy polishing up their stuff21:11
flaper87talks21:11
flaper87yeah21:11
*** jchai_afk is now known as jchai21:13
*** flaper87 is now known as flaper87|afk21:23
*** cpallares has quit IRC21:24
mpanettaIs it nap time yet?21:31
*** Obulpathi has quit IRC21:35
*** balajiiyer has quit IRC21:36
*** Obulpathi has joined #openstack-marconi21:37
*** Obulpathi has quit IRC21:39
kgriffsmalini, flaper87|afk: those numbers look sort of wonky to me. Why is there such a huge difference between "highest 10sec mean" and "lowest 10sec mean" ?21:42
kgriffsmalini: can we corroborate these numbers with a different tool like New Relic?21:44
*** jchai has quit IRC21:48
*** amitgandhi has quit IRC21:57
*** mpanetta has quit IRC22:03
*** ayoung is now known as ayoung_exercise22:26
*** rossk_ has joined #openstack-marconi23:01
*** rossk has quit IRC23:02
*** amitgandhi has joined #openstack-marconi23:07
*** amitgandhi has quit IRC23:12
*** shakamunyi has joined #openstack-marconi23:42
openstackgerritChad Lung proposed a change to openstack/marconi: Fixing some spelling, grammar, etc in docs  https://review.openstack.org/9170523:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!