Wednesday, 2013-09-18

*** kgriffs_afk is now known as kgriffs00:09
*** jergerber has quit IRC00:09
*** kgriffs is now known as kgriffs_afk00:09
*** amitgandhi has joined #openstack-marconi00:16
*** nosnos has joined #openstack-marconi00:18
*** malini_afk is now known as malini00:22
*** kgriffs_afk is now known as kgriffs00:31
*** amitgandhi has quit IRC00:37
*** zzs has joined #openstack-marconi01:00
*** zzs has left #openstack-marconi01:04
*** kgriffs is now known as kgriffs_afk01:15
*** kgriffs_afk is now known as kgriffs01:20
*** tedross has joined #openstack-marconi01:37
*** kgriffs is now known as kgriffs_afk01:58
*** oz_akan_ has joined #openstack-marconi02:05
*** tedross has quit IRC02:19
*** whenry has joined #openstack-marconi02:43
*** oz_akan_ has quit IRC02:56
*** oz_akan_ has joined #openstack-marconi02:56
*** oz_akan_ has quit IRC03:01
*** ayoung is now known as ayoung-ZzZzZzZ03:07
*** malini is now known as malini_afk03:38
*** oz_akan_ has joined #openstack-marconi04:07
*** oz_akan_ has quit IRC04:12
*** openstack has joined #openstack-marconi09:17
*** nosnos has quit IRC10:18
*** cthulhup_ has joined #openstack-marconi11:33
*** cthulhup_ has quit IRC11:33
*** cthulhup has joined #openstack-marconi11:34
*** flaper87|afk is now known as flaper8711:44
openstackgerritFlavio Percoco proposed a change to openstack/marconi: Update HACKING file with the new structure  https://review.openstack.org/4713311:54
*** key3 has quit IRC12:01
*** cppcabrera has joined #openstack-marconi12:05
cppcabreraGood morning! :)12:06
cppcabreraflaper87: o/12:06
flaper87cppcabrera: yo yo!12:07
flaper87how are you doing?12:07
cppcabreraAwesomely. It's a good day! How'd your talk go yesterday at PythonMilan? :)12:09
flaper87It went pretty well, folks were curious and we also endded up discussiong oslo-incubator :D12:10
flaper87They agreed with me that this is one of those cases where the worst idea is the best one12:10
flaper87:D12:10
cppcabreraas in, "some idea" is better than "no idea"? :P12:11
flaper87cppcabrera: yeaaaahhhh!!! It was nice to have more folks brainstorming and discussing strategies about what the best thing to do would be12:11
cppcabreraI think oslo-incubator is a great start. It wouldn't hurt to have an Oslo committee (is there such a thing?) to keep contributors and oslo projects on track.12:12
cppcabreraThat committee idea came to mind because I see that haskell is now putting together a "Core Libraries Commiitee" (http://www.haskell.org/haskellwiki/HaskellImplementorsWorkshop/2013/Kmett)12:13
flaper87cppcabrera: there is. Oslo itself is a program which covers oslo-incubator and oslo.* modules12:13
cppcabreraawesome!12:13
flaper87there are core contributors in oslo, per-module (as in .py files) maintainers and meetings on demand12:13
flaper87as in any other projects12:14
flaper87plus, design sessions at the summit12:14
cppcabrerasounds like the state of things is better than I thought! What ideas were proposed at PyMilan for making it even better? :)12:14
flaper87cppcabrera: the first one was git submodules but that doesn't work. The difficult part is explaining why release + pinning doesn't work for modules in incubation12:15
flaper87I mean, since OS is not opinionated plus has to have support for all kind of environments, we force distros / users to have each module installed in separted venvs12:16
*** tedross has joined #openstack-marconi12:16
cppcabreraahh, yeah. I could see git-submodules not working. :x12:16
flaper87hence we need to make sure oslo's dependencies throughout openstack don't clash12:16
flaper87etc12:16
flaper87yeah12:16
flaper87but, we got to the point were they also said: "You know what? Just copy that #$@#$!#$ file around"12:17
flaper87^^12:17
cppcabreralol12:18
*** key3 has joined #openstack-marconi12:20
flaper87cppcabrera: saw that most of the proxy patches got merged12:25
flaper87awesome!12:25
flaper87sorry if I wasn't that helpful with the review process there12:25
flaper87btw, could you take a look here? https://review.openstack.org/#/q/status:open+project:openstack/marconi+branch:master+topic:change-marconi-structure,n,z12:25
cppcabreraflaper87: you were helpful with the reviews, for sure!12:34
cppcabreraYou caught several issues that I didn't notice. :D12:35
cppcabreraThe proxy is getting much nicer with a concrete solid driver in place.12:35
cppcabreraStill has rough edges, but once oslo.config and oslo.cache get involved, it should be very close to beta quality.12:35
* cppcabrera checks out the link12:36
cppcabrera+2'd on both, flaper87.12:37
*** flaper87 is now known as flaper87|afk12:37
*** oz_akan_ has joined #openstack-marconi12:46
*** oz_akan_ has quit IRC12:50
*** oz_akan_ has joined #openstack-marconi12:51
*** amitgandhi has joined #openstack-marconi13:18
*** malini_afk is now known as malini13:38
*** cthulhup has quit IRC13:49
*** cthulhup has joined #openstack-marconi13:51
*** EmilienM has quit IRC14:05
*** EmilienM has joined #openstack-marconi14:07
openstackgerritA change was merged to openstack/marconi: Fix Functional Tests  https://review.openstack.org/4685114:09
cppcabreramalini: Thanks for the functional tests fix!14:10
malinicppcabrera: my pleasure ;)14:12
maliniI am super excited all our tests are now toxic :)14:12
cppcabrerahaha, me too!14:20
cppcabreraIt's making the CI side of things super easy.14:21
openstackgerritFlavio Percoco proposed a change to openstack/python-marconiclient: Refactor client's structure to reflect Marconi's  https://review.openstack.org/4716914:21
openstackgerritFlavio Percoco proposed a change to openstack/python-marconiclient: Update requirements and pbr version  https://review.openstack.org/4625914:21
openstackgerritFlavio Percoco proposed a change to openstack/python-marconiclient: Adding first oslo module and config  https://review.openstack.org/4627514:21
openstackgerritFlavio Percoco proposed a change to openstack/python-marconiclient: Added a base class for tests  https://review.openstack.org/4628814:21
cppcabreramy my, look at all those... rebasings?14:21
*** ayoung-ZzZzZzZ is now known as ayoung14:22
*** flaper87|afk is now known as flaper8714:23
flaper87cppcabrera: :D14:24
openstackgerritFlavio Percoco proposed a change to openstack/python-marconiclient: Refactor client's structure to reflect Marconi's  https://review.openstack.org/4716914:25
cppcabreraooohh, a restructuring patch.14:26
cppcabreraflaper87 has been busy restructuring everything lately. :)14:26
flaper87:D14:27
* flaper87 has been using hammer, nails, drills and what not!14:27
flaper87making Marconi pieces look nice14:27
cppcabreraThe addition of queues is making happier every day. The proxy implementation pretty much mirrors the queues implementation wrt to storage.14:28
cppcabreraSo I can things like: 'open queues/storage/base.py; open proxy/storage/base.py' to easily compare and learn.14:29
cppcabrera**can do things like...14:29
amettsflaper87:  how are things looking on python-marconiclient?  I didn't realize you were working on it (I've been a bit disconnected from marconi lately).14:29
flaper87ametts: so, lot of discussions lately, I think we kind of got to an agreement on how the API should look like14:30
flaper87ametts: that means I already started coding it14:31
flaper87and will have something ready soon, hopefully14:31
amettsThe client has an API?  I'm so confused...14:31
cppcabreralol14:31
flaper87ametts: programmatic API14:31
flaper87:D14:31
cppcabrera+2'd on all marconiclient patches, flaper87.14:32
cppcabreraimportutils is cooler than I thought, now that I took a closer look.14:32
amettsflaper87: Sounds good -- looking forward to trying it out!14:32
flaper87ametts: stay tunned!14:32
flaper87btw, al-maisan said he'll help out with it as well14:33
flaper87I'm sitting right next to him right now14:33
flaper87he's fighting with wifi14:33
flaper87:D14:33
amettsTell him I said "Hi".  Too bad he couldn't make it to Atlanta.14:33
zyuancppcabrera: review14:38
cppcabrerazyuan: which patchset? link?14:39
cppcabrerafalcon?14:39
zyuan?? my msg posted on falconframework partially appears here?!14:39
zyuancppcabrera: review: https://github.com/racker/falcon/pull/18414:39
cppcabreraunit tests find all the marconi-proxy errors. :P14:54
* cppcabrera is writing proxy mongo storage unit tests14:54
zyuanwhich?14:54
openstackgerritMonty Taylor proposed a change to openstack/marconi: Update pbr and requirements to be current  https://review.openstack.org/4717914:56
openstackgerritMonty Taylor proposed a change to openstack/marconi: Migrate to testr  https://review.openstack.org/4718014:56
cppcabrerazyuan: Those that cover the interface exposed in proxy/storage/base.py14:57
maliniflaper87: I have been looking at testscenarios.My biggest gripe is you have to define the 'scenarios' (a.k.a. the different data) at a class level & not per test case :(15:09
maliniI think it makes better sense, when you are trying to test scenarios - a combination of dependent tests  in a class15:09
*** flaper87 is now known as flaper87|afk15:12
*** kgriffs_afk is now known as kgriffs15:21
cppcabreraHaha, I'm finding the limits of mongodb... :P15:30
cppcabrera"database error: You cannot currently mix including and excluding fields. Contact us if this is an issue."15:30
openstackgerritMonty Taylor proposed a change to openstack/marconi: Migrate to testr  https://review.openstack.org/4718015:31
cppcabrerawoot - mongo tests are now passing on the catalog. Updating the patchset in a moment.15:54
openstackgerritAlejandro Cabrera proposed a change to openstack/marconi: Add mongo driver for marconi proxy storage  https://review.openstack.org/4637516:07
cppcabreramalini: I've added some test code to the patch proposed above (mongo driver). It follows the conventions used by tests/unit/storage, and adds a new configuration file. Could you look over the test code to see if there's anything that stands out as clearly wrong? :D16:18
maliniSure cppcabrera16:18
cppcabrerayay, thanks!16:18
al-maisanametts: I haven't given up on Atlanta :-) Plan to visit during my next travel to the US16:29
cppcabreraal-maisan: looking forward to seeing you! :)16:29
* cppcabrera is eating lunch from ATL at this very moment16:29
amettsal-maisan: Good to hear!16:30
al-maisancppcabrera: likewise :-)16:30
* al-maisan looks froward to flaper87|afk's mongodb talk :)16:32
cppcabrerawhen is he giving that talk, al-maisan? :D16:33
cppcabreraI'm sure flaper87|afk has mentioned it, but I've forgotten at this point between all the exciting talks going on. :P16:33
al-maisanin 5 minutes or so16:33
cppcabreraoohhh16:33
cppcabrerasweet16:33
al-maisanwe are at the university of Geneva here -- he's 1.5 metres away from me16:34
* cppcabrera hopes slides and/or video are available afterwards16:34
al-maisanslides probably -- don't see video kit here, so no video :(16:34
cppcabreraah well, I'm happy with access to slides, at least. :)16:35
*** cppcabrera is now known as cppcabrera_afk17:13
*** whenry has quit IRC17:28
*** kgriffs is now known as kgriffs_afk17:40
*** EmilienM has quit IRC17:46
*** EmilienM has joined #openstack-marconi17:49
*** kgriffs_afk is now known as kgriffs17:52
*** cppcabrera_afk is now known as cppcabrera18:00
*** cppcabrera has quit IRC18:19
*** cthulhup has quit IRC18:21
*** EmilienM has quit IRC18:29
*** EmilienM has joined #openstack-marconi18:32
*** cppcabrera has joined #openstack-marconi18:32
*** malini is now known as malini_afk19:03
*** amitgandhi has quit IRC19:04
*** amitgandhi has joined #openstack-marconi19:04
*** amitgandhi has quit IRC19:08
*** oz_akan_ has quit IRC19:09
*** amitgandhi has joined #openstack-marconi19:09
*** zyuan has quit IRC19:09
*** ametts has quit IRC19:09
*** cppcabrera has quit IRC19:09
*** oz_akan_ has joined #openstack-marconi19:10
*** ametts has joined #openstack-marconi19:10
*** oz_akan__ has joined #openstack-marconi19:11
*** zyuan has joined #openstack-marconi19:11
*** oz_akan__ has quit IRC19:11
*** oz_akan__ has joined #openstack-marconi19:12
*** oz_akan_ has quit IRC19:14
*** cppcabrera has joined #openstack-marconi19:16
openstackgerritA change was merged to openstack/marconi: Update pbr and requirements to be current  https://review.openstack.org/4717919:39
openstackgerritA change was merged to openstack/marconi: Fix old import paths  https://review.openstack.org/4697319:41
openstackgerritA change was merged to openstack/marconi: Update HACKING file with the new structure  https://review.openstack.org/4713319:41
openstackgerritAlejandro Cabrera proposed a change to openstack/marconi: Add mongo driver for marconi proxy storage  https://review.openstack.org/4637519:43
*** zzs has joined #openstack-marconi19:45
kgriffsguys, we NEED to fix the ddt output with that monster string19:47
zyuanAHHHHHAHAHAHAHA19:47
kgriffsanyone already got a patch in the works for it?19:47
cppcabreralol19:47
*** malini_afk is now known as malini19:47
zyuanno. it only happens on functional tests19:47
kgriffsmy eyes bleed everytime I see it scroll by in my console19:47
cppcabreraagreed, kgriffs19:47
zyuanhahahahahahhaa19:47
maliniwe will be moving away from ddt soon19:47
malinitht's the reason I didnt fix it yet19:48
kgriffsoh19:48
kgriffsok19:48
cppcabrerato testscenarios?19:48
zyuanfound something better?19:48
cppcabrera+ testresources?19:48
kgriffsi'll get a blood transfusion and try to hold out until that happens19:48
malininot sure abt the better part19:48
kgriffsheh19:48
kgriffsmalini: +119:48
malinibut openstack uses testscenarios. so we'll ove to it19:49
cppcabrerathat's not a good enough reason, IMO, to move to a different test tool.19:49
kgriffsimproving testscenarious would be good hackday project19:49
kgriffshmm, guess we need to be more careful about making snap decisions like this in the future...19:50
maliniAs is, testscenarios allows data params only at class level19:50
* kgriffs wasn't included in the discussion19:50
kgriffsmalini: that is a pretty strong case to make against migrating19:50
malinikgriffs: you still have time to chip in19:50
kgriffshave you discussed with monty/doug?19:50
maliniI talked to monty & they are going to add function level support to it19:51
kgriffslike, "hey i am trying to migrate but running into this problem that can't specify data at the test level"19:51
kgriffsoic19:51
kgriffsETA?19:51
kgriffsI mean, i think we should hold off migrating until that lands19:51
cppcabrera+1 kgriffs19:52
maliniyes..We cannot migrate till tht lands, without HUGE refactoring in marconi tests19:52
kgriffsok, that seals the deal. it's a blocker for us19:52
malinimonty has started work on it though19:52
kgriffshmmm19:52
cppcabreraI looked at testresources just now, and I like the way they're approaching class level data management. However, w/o function-level data management, it seems like testresources and ddt are orthogonal. Both can be used.19:52
malinishud be soon19:52
kgriffsyou mean, he is working on method-level data?19:53
malinikgriffs: yes19:53
kgriffssweeeeet19:53
cppcabrerathat'd be awesome. Hopefully, they can package that using decorator syntax. :)19:53
maliniIts not going to be decorators :(19:53
kgriffssrlsy?!19:54
maliniddt was sweet (even with a terrible name)19:54
kgriffs#fail19:54
cppcabrera:/19:54
kgriffsis testscenario an oslo thing?19:55
malinioslo thing as in ?19:55
kgriffshttps://github.com/openstack/oslo-incubator19:55
maliniit was written by Robert Collins, & that name pops up in all OS test tools19:55
zyuani just don't use ddt to build spam test19:55
zyuanso i'm fine with that19:55
kgriffshttps://pypi.python.org/pypi/testscenarios/0.419:56
maliniwith method support in testscenarios, we can get the same functionality as ddt19:57
cppcabreraI'm inclined to think that ddt + testscenarios actually work together.19:58
cppcabreraSo migrating to testscenarios doesn't preclude using ddt.19:58
kgriffsyuck. it's hosted in Bazaar on launchpad19:58
cppcabreraAFAICT19:58
malinicppcabrera: method level support in testscenarios , will help us acheieve data driven tests. why do u think we need both?19:58
cppcabrerait all depends on the syntax, malini. If testscenarios is designed in such a way that the way to specify method level tests is not friendly, it may be in our best interests to take the best from both worlds.20:00
maliniThe feel I get is, OS always ends up writing its own libraries :(20:00
zyuanmalini: you said the thing we don't say...20:00
maliniI love the ddt syntax!20:01
openstackgerritKurt Griffiths proposed a change to openstack/marconi: fix(mongodb): Marker generation creates a bottleneck under heavy load  https://review.openstack.org/4724220:01
cppcabrerakgriffs: Is that your giga patch?20:01
cppcabrera;)20:01
kgriffsmalini: good point. Next time you notice something like this, let me know and i can push back20:01
* cppcabrera notices that its fairly sizable.20:02
cppcabrera*it's20:02
malinikgriffs: can I send you a list? :d20:02
kgriffs(recommend NOT rebuilding things that doesn20:02
kgriffsalready exist)20:02
kgriffsmtg time20:02
cppcabreraThanks for looking into this, malini, re: test tooling. :D20:02
kgriffscppcabrera: yes20:02
cppcabrerabrb meeting20:02
*** cppcabrera is now known as cppcabrera_afk20:03
*** amitgandhi has quit IRC20:03
*** amitgandhi1 has joined #openstack-marconi20:03
*** cppcabrera_afk is now known as cppcabrera20:51
cppcabrerakgriffs: did a preliminary review on your patch.21:04
cppcabrera+1 - tenatively, 'til I can dedicate more time to look at it.21:04
cppcabreraIt looks good so far.21:04
cppcabreraI just need to dig into the logic a little more.21:04
kgriffsyes, please do21:04
kgriffsit it is very possible i missed a race condition or something21:05
cppcabreraI'll think on it tomorrow morning, just before diving into implementing the oslo.cache components.21:05
kgriffsi just started the last schema-changing patch21:05
kgriffswill get that up for review21:05
cppcabreracool21:05
cppcabreraI introduced a really silly issue with the marconi-proxy mongo storage patch. I hardcoded the driver details in app.py, and py26 gate does not like that.21:06
kgriffsonce we have multi-db and this patch i am working on now (combine p and q into a single field) we can close this bp. https://blueprints.launchpad.net/marconi/+spec/api-perf-tuning21:06
cppcabreraMy guess is that there's a mongodb installed on gate-py27.21:06
kgriffscppcabrera: oic21:06
cppcabrera+1 kgriffs21:06
cppcabreraWith the py26 issue in mind, could you review the proxy storage patch at your earliest convenience?21:07
kgriffscppcabrera: could be. How about creating a dummy/in-memory reference driver for testing?21:07
kgriffss/dummy/mock21:07
kgriffsyes, I've got it on my shortlist of TODOs21:07
cppcabrera+1 kgriffs, to the memory driver.21:08
cppcabreraI've considered it.21:08
cppcabreraI just need to figure out how long it'd take to write it. :P21:09
cppcabreraalso, lol @ short list of TODOs. :P21:09
cppcabreraour TODOs are lengthy indeed for awhile longer.21:10
kgriffsmalini: this will need to be benchmarked to see if it helps anything - https://review.openstack.org/#/c/47242/21:11
kgriffspls. +1 or −1 based on the results21:11
kgriffsoz_akan: ^^^21:11
zyuankgriffs: i hope the your side counter patch can pause reviewing for a while21:19
zyuanit heavily conflicts with multidb patch21:20
zyuani'm rebasing multidb patch; i don't want to do it again after i finished cleaning up....21:20
kgriffsi don't think there will be any major changes to it, maybe a few tweaks, before merging21:21
kgriffsso future rebasing should be a lot less painful21:21
cppcabreraI'm out for the night. oslo.cache, memory drivers, and more await me tomorrow.21:22
cppcabreraHave a good night, guys!21:22
kgriffsciao!21:22
zyuansee u21:22
cppcabreraget some sleep. ;)21:22
*** cppcabrera has quit IRC21:22
openstackgerritKurt Griffiths proposed a change to openstack/marconi: perf(mongodb): Combine project and queue message fields  https://review.openstack.org/4725921:33
*** oz_akan__ has quit IRC21:38
*** oz_akan_ has joined #openstack-marconi21:38
*** oz_akan_ has quit IRC21:43
*** tedross has quit IRC22:10
*** zzs has left #openstack-marconi22:14
openstackgerritAlejandro Cabrera proposed a change to openstack/marconi: Adds drivers for marconi proxy storage.  https://review.openstack.org/4637522:15
*** amitgandhi1 has quit IRC22:26
openstackgerritKurt Griffiths proposed a change to openstack/marconi: fix(mongodb): Marker generation creates a bottleneck under heavy load  https://review.openstack.org/4727222:44
openstackgerritKurt Griffiths proposed a change to openstack/marconi: perf(mongodb): Combine project and queue message fields  https://review.openstack.org/4725922:44
*** oz_akan_ has joined #openstack-marconi22:49
*** cthulhup has joined #openstack-marconi22:52
*** oz_akan_ has quit IRC22:54
*** cthulhup has quit IRC22:55
*** kgriffs is now known as kgriffs_afk22:56
*** megan_w has quit IRC22:58
*** megan_w has joined #openstack-marconi22:59
*** malini is now known as malini_afk23:34
*** amitgandhi has joined #openstack-marconi23:58
*** whenry has joined #openstack-marconi23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!