| *** crxz0193 has quit IRC | 04:36 | |
| *** crxz0193 has joined #magnetodb | 05:25 | |
| *** crxz0193_ has joined #magnetodb | 05:25 | |
| *** crxz0193_ has quit IRC | 05:25 | |
| *** crxz0193 has quit IRC | 05:25 | |
| *** crxz0193 has joined #magnetodb | 05:25 | |
| *** jeromatron has joined #magnetodb | 07:20 | |
| *** idegtiarov has joined #magnetodb | 07:54 | |
| miarmak | good morning all! | 08:21 |
|---|---|---|
| *** crxz0193 has quit IRC | 08:27 | |
| *** crxz0193 has joined #magnetodb | 08:41 | |
| isviridov | Hello guys | 09:17 |
| setho | Hello! | 09:19 |
| *** dukhlov__ has quit IRC | 09:22 | |
| *** dukhlov has quit IRC | 09:24 | |
| *** dukhlov_ has quit IRC | 09:24 | |
| *** dukhlov has joined #magnetodb | 09:24 | |
| *** crxz0193 has quit IRC | 09:32 | |
| *** dmakogon_ is now known as denis_makogon | 09:44 | |
| *** ominakov has joined #magnetodb | 09:44 | |
| *** crxz0193 has joined #magnetodb | 09:45 | |
| *** achudnovets has joined #magnetodb | 09:51 | |
| achudnovets | hi guys! | 09:52 |
| miarmak | achudnovets: hello) | 09:54 |
| isviridov | miarmak, here is BP https://blueprints.launchpad.net/magnetodb/+spec/lightweight-put-item | 09:58 |
| isviridov | what are your plans about it? | 09:58 |
| *** jeromatron has quit IRC | 09:58 | |
| miarmak | isviridov: we talked with ikhudoshyn yesterday about it, we will postpone this task. it is not clear now what we'll do in this direction | 10:00 |
| *** ominakov has quit IRC | 10:00 | |
| isviridov | miarmak, fair enough. But we have dependencies from other BPs. | 10:06 |
| isviridov | miarmak, what about https://blueprints.launchpad.net/magnetodb/+spec/bulk-data-load ? As for me it is done. | 10:06 |
| isviridov | miarmak, any other work you are going to do in that direction? | 10:06 |
| *** jeromatron has joined #magnetodb | 10:14 | |
| *** jeromatron has joined #magnetodb | 10:15 | |
| *** jeromatron has quit IRC | 10:18 | |
| *** idegtiarov has quit IRC | 11:02 | |
| *** ominakov has joined #magnetodb | 11:10 | |
| miarmak | isviridov: I would like to look on it after dukhlov's fixes, cause maybe it is needs some improvements | 11:11 |
| *** jeromatron has joined #magnetodb | 11:13 | |
| *** idegtiarov has joined #magnetodb | 11:13 | |
| isviridov | miarmak, ok. Moved to next milestone | 11:16 |
| ominakov | isviridov, i registered bp for moving "OK" tests from in_progress to stable after bug fixing (it's long term task), please verify it | 11:17 |
| ominakov | https://blueprints.launchpad.net/magnetodb/+spec/move-passed-tempest-tests-to-stable | 11:17 |
| miarmak | miarmak: it can be smth like either Logging and progress visualization or add possibility to specify cassandra nodes ip's | 11:17 |
| miarmak | isviridov: ^ | 11:17 |
| miarmak | isviridov: so, I would postpone closing it | 11:18 |
| miarmak | isviridov: maybe we should set priority for this | 11:18 |
| isviridov | ominakov, if I got you right it is continuous activity. And we have to do it every new functionality is implemented and covered by tests? | 11:25 |
| isviridov | ominakov, or it is lire revision of current test state and updating it? | 11:25 |
| isviridov | * like | 11:26 |
| ominakov | isviridov, yes it's continuous activity | 11:26 |
| isviridov | ominakov, what will trigger that action? | 11:26 |
| ominakov | isviridov, updating the tests state to actual | 11:28 |
| isviridov | ominakov, how often are you going to do it? | 11:30 |
| *** crxz0193 has quit IRC | 11:43 | |
| isviridov | ominakov, it is interesting point how to organize such activities. | 11:46 |
| isviridov | ominakov, I would like to do it per milestone release. And ready to approve it, but with a bit other description. Like 'actualize functional tests' | 11:47 |
| isviridov | ominakov, what do you think? | 11:47 |
| *** crxz0193 has joined #magnetodb | 12:04 | |
| *** idegtiarov has quit IRC | 12:08 | |
| ominakov | isviridov, yes i think this a good point, i'll rename bp | 12:08 |
| *** idegtiarov has joined #magnetodb | 12:08 | |
| openstackgerrit | Maksym Iarmak proposed a change to stackforge/magnetodb: Move test_describe_table to stable https://review.openstack.org/86301 | 12:09 |
| miarmak | guys, welcome to review ^^^^^^^^^ | 12:10 |
| *** aostapenko has joined #magnetodb | 12:26 | |
| aostapenko | Hello | 12:27 |
| ominakov | aostapenko, hello | 12:36 |
| isviridov | aostapenko, hello | 12:39 |
| isviridov | aostapenko, how are you today? | 12:39 |
| ominakov | isviridov, i fixed description for this bug https://bugs.launchpad.net/magnetodb/+bug/1304384 | 13:04 |
| *** [o__o] has quit IRC | 13:08 | |
| *** [o__o] has joined #magnetodb | 13:20 | |
| achudnovets | bye guys! | 13:28 |
| *** achudnovets has left #magnetodb | 13:28 | |
| miarmak | bye | 13:38 |
| miarmak | isviridov: around? | 13:38 |
| *** denis_makogon has quit IRC | 13:44 | |
| isviridov | miarmak, here | 13:47 |
| miarmak | isviridov: could you review this bp please? https://blueprints.launchpad.net/magnetodb/+spec/bulk-data-load-rest | 13:47 |
| aostapenko | isviridov: hello, better, thank you | 13:48 |
| isviridov | miarmak, I have a lot of Qs about it | 13:49 |
| isviridov | Why do we really need one more component? | 13:49 |
| miarmak | isviridov: it is a reult of our yesterday's discussion with ikhudoshyn | 13:49 |
| miarmak | result* | 13:49 |
| miarmak | in his POC pywsgi is used | 13:50 |
| miarmak | instead of our wsgi | 13:50 |
| isviridov | miarmak, it doesn't look like common approach | 13:52 |
| isviridov | miarmak, I believe it is need feature, but we have to discuss it in ML or IRC to find out the best way how to do it. | 13:53 |
| miarmak | isviridov: our wsgi does not support streaming | 13:54 |
| isviridov | miarmak, what about pecan& | 13:54 |
| isviridov | ? | 13:54 |
| isviridov | achuprin, here? | 13:56 |
| isviridov | ominakov, bug has been scheduled | 13:57 |
| isviridov | ominakov, would you like to fix it? | 13:57 |
| miarmak | isviridov: hm, we didn't discuss about it in this cas | 13:57 |
| miarmak | e | 13:58 |
| achuprin | isviridov: yes | 13:58 |
| isviridov | achuprin, anything left here from your point of view https://blueprints.launchpad.net/magnetodb/+spec/third-party-ci ? | 13:59 |
| isviridov | achuprin, can be marked as implemented? | 13:59 |
| isviridov | miarmak, I'm approving it for discussion | 14:00 |
| achuprin | isviridov: I now look at it | 14:00 |
| isviridov | miarmak and ikhudoshyn look as drivers of this feature | 14:00 |
| isviridov | achuprin, one more think can we enable voting for https://blueprints.launchpad.net/magnetodb/+spec/external-testing-infra? | 14:01 |
| achuprin | isviridov: about this bp - https://blueprints.launchpad.net/magnetodb/+spec/third-party-ci | 14:03 |
| achuprin | isviridov: our devstack job now in non-voting mode | 14:04 |
| achuprin | isviridov: so, status beta implementation | 14:04 |
| achuprin | isviridov: so, after moved in viting mode, we can be change status to implemented | 14:05 |
| isviridov | achuprin, any reason to not to move on? | 14:05 |
| achuprin | isviridov: main reason is stability | 14:07 |
| miarmak | isviridov: so, what about implementation? Should we start it now? | 14:08 |
| isviridov | achuprin, what do you mean? | 14:08 |
| isviridov | miarmak, we have start with concept first and least discuss it here | 14:09 |
| achuprin | isviridov: if all core developers and contributers think that our job stable and always return true result, we can moved our devstack job to gate pipe line on voting mode right now | 14:10 |
| isviridov | miarmak, I would like to see it as part of our main functionality. Even if it is separate component, we have to think about keystone authorization there. | 14:10 |
| miarmak | isviridov: for now, it is continuing of Illi's POC. It is based on it. | 14:11 |
| isviridov | dukhlov, any objections to make our gating jobs voting? | 14:12 |
| achuprin | isviridov: I think will be better if we do that after adding keystone integration on magnetodb devstack lib | 14:13 |
| ominakov | isviridov, this functionality implemented by achudnovets and he more competent in error handling | 14:13 |
| isviridov | miarmak, I know, but we have to do next step. If we need ikhudoshyn, np. | 14:13 |
| isviridov | achuprin, not sure. What is ETA to make it voting? | 14:14 |
| isviridov | ominakov, you can become competent as well, specially your are reporter, so know how to test. But it is up to you, sure | 14:16 |
| *** SpyRay has joined #magnetodb | 14:16 | |
| isviridov | SpyRay, hey | 14:17 |
| isviridov | SpyRay, you are our night shift. Welcome | 14:17 |
| SpyRay | hi all | 14:17 |
| miarmak | setho: | 14:17 |
| miarmak | SpyRay: hi there | 14:17 |
| achuprin | isviridov: creating a patch not take much time, but we must get approval from Infra team. I do not know how long it takes. | 14:20 |
| dukhlov | about gating jobs voting | 14:21 |
| dukhlov | I have no objections | 14:21 |
| isviridov | achuprin, please go ahead with it and add me as reviewer if you don't mind. | 14:21 |
| miarmak | isviridov: okey, let's discuss it, when ikhudoshyn joins us | 14:22 |
| isviridov | achuprin, and I moving it to next milestone. | 14:22 |
| achuprin | isviridov: ок | 14:23 |
| miarmak | isviridov: btw, could you review it please https://review.openstack.org/#/c/86301/ | 14:23 |
| isviridov | miarmak, just moving. +2 | 14:24 |
| miarmak | isviridov: aha, thanks) | 14:24 |
| isviridov | miarmak, ikhudoshyn we have to discuss the streaming API together. | 14:24 |
| miarmak | miarmak: okey, I think we'll be able to do it in 1 hour | 14:26 |
| miarmak | isviridov: ^ | 14:26 |
| miarmak | dukhlov: around? | 14:26 |
| miarmak | isviridov: so, if it is simple moving, maybe it can be merged? | 14:28 |
| dukhlov | miarmak: hi | 14:29 |
| miarmak | dukhlov: could ypu review it please? https://review.openstack.org/#/c/86301/ | 14:29 |
| miarmak | dukhlov: hello_) | 14:30 |
| dukhlov | ok, sure | 14:30 |
| miarmak | dukhlov: thanks! | 14:30 |
| ominakov | guys, Amazon DynamoDB has limits for table name such as: 1) length grater than or equal 3 symbol; 2) length smaller than or equal 255 symbol; 3) match pattern [a-zA-Z0-9_.-]+ But in Cassandra column family names shouldn't be grater than 48 symbols and match pattern [0-9A-Za-z]+ include symbol "_". What do you think? | 14:51 |
| miarmak | ominakov: we already have it: https://bugs.launchpad.net/magnetodb/+bug/1271576 | 14:53 |
| ominakov | miarmak, thanks | 14:59 |
| miarmak | ominakov: you're welcome | 14:59 |
| setho | #startmeeting Syncup 140409 | 15:02 |
| openstack | Meeting started Wed Apr 9 15:02:40 2014 UTC and is due to finish in 60 minutes. The chair is setho. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:02 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:02 |
| openstack | The meeting name has been set to 'syncup_140409' | 15:02 |
| setho | #topic Bulk import | 15:04 |
| setho | #info ikhudoshin works on Indexed Data | 15:05 |
| setho | #info miarmak works on REST part | 15:06 |
| setho | #info vnaboychenko? works on Environement | 15:06 |
| setho | #topic Indexing | 15:07 |
| setho | #info achuprin works on Indexing tests. they are interconnected and sometimes complicated | 15:07 |
| setho | #info Please add new tasks to backlog | 15:08 |
| setho | #topic Async table creation | 15:09 |
| *** idegtiarov has quit IRC | 15:10 | |
| setho | #info dukhlov works on an important bug. It makes impossible to execute more than 1 request with condition in the same time | 15:10 |
| setho | #info dukhlov isviridov ikhudoshin discuss design approach | 15:11 |
| setho | #info charles can start implementation of the bluprint | 15:13 |
| setho | #info charles has had a baby | 15:13 |
| setho | #topic Keystone integration | 15:13 |
| setho | #topic isviridov prepares Nova research | 15:14 |
| setho | #topic achudnovets? works on storage for Domain-project | 15:15 |
| setho | #topic Multi-domain write | 15:16 |
| *** dukhlov has quit IRC | 15:16 | |
| setho | #info achudnovets starts Filter for write requests | 15:16 |
| *** dukhlov has joined #magnetodb | 15:16 | |
| setho | #info musichenko? started tests | 15:17 |
| setho | #topic QA | 15:17 |
| setho | #info aminakov? works on Negative tests for PutItem | 15:18 |
| setho | #topic No updates required for CI | 15:18 |
| setho | #topic DevStack integration | 15:19 |
| setho | #info avonogradov? works on integration approach | 15:19 |
| setho | #info achuprin started big chunk on Hbase | 15:21 |
| setho | #endmeeting | 15:21 |
| openstack | Meeting ended Wed Apr 9 15:21:14 2014 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:21 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/syncup_140409/2014/syncup_140409.2014-04-09-15.02.html | 15:21 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/syncup_140409/2014/syncup_140409.2014-04-09-15.02.txt | 15:21 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/syncup_140409/2014/syncup_140409.2014-04-09-15.02.log.html | 15:21 |
| isviridov | setho, thank you for logging | 15:24 |
| * isviridov bowing | 15:24 | |
| isviridov | Congratulations guys we have finished with 2.0.2, I'm fighting with releasing it now. | 15:27 |
| isviridov | Current development focus is https://launchpad.net/magnetodb/+milestone/2.0.3 | 15:27 |
| miarmak | isviridov, ikhudoshyn around? | 15:29 |
| ikhudoshyn | yep | 15:32 |
| miarmak | isviridov: ? | 15:32 |
| ikhudoshyn | guys, I'd really love if u spend some time at https://review.openstack.org/#/c/85574/ | 15:32 |
| ikhudoshyn | i'd want it in, so I could continue work on it. And I think miarmak will use it as well | 15:33 |
| miarmak | ikhudoshyn: so, I've created a bp: https://blueprints.launchpad.net/magnetodb/+spec/bulk-data-load-rest Could you look on it please and tell me, if something wrong there | 15:34 |
| ikhudoshyn | miarmak: will do | 15:34 |
| openstackgerrit | A change was merged to stackforge/magnetodb: Move test_describe_table to stable https://review.openstack.org/86301 | 15:34 |
| miarmak | ikhudoshyn: thanks! and after that come back, because isviridovhas a couple of questions on it. And we'll have discussion on this topic | 15:34 |
| isviridov | ikhudoshyn, here and ready for discussion | 15:35 |
| ikhudoshyn | miarmak: when we r goin to gather? | 15:36 |
| isviridov | dukhlov, would you join us? | 15:36 |
| miarmak | ikhudoshyn: if you don't mind, now =) | 15:36 |
| ikhudoshyn | ok, but i'll prefer voice | 15:37 |
| isviridov | ikhudoshyn, what the main BP we will use for tracking this feature? There are several actually. Let us pin one | 15:39 |
| miarmak | ikhudoshyn: is it correct? (bp) | 15:39 |
| ikhudoshyn | https://blueprints.launchpad.net/magnetodb/+spec/bulk-data-load-over-network | 15:41 |
| ikhudoshyn | miarmak: looks good, but I'd also add something about results expected | 15:41 |
| ikhudoshyn | miarmak: could u please? | 15:42 |
| miarmak | ikhudoshyn: ok, thanks! I'l add it | 15:42 |
| ikhudoshyn | isviridov, miarmak do we want to discuss anything? | 15:45 |
| isviridov | ikhudoshyn, sorry interrapted | 15:45 |
| ikhudoshyn | isviridov: do you find anything wrong with current stare and work on bulk? | 15:47 |
| isviridov | ikhudoshyn, actually yes. | 15:47 |
| isviridov | What the reason to have additional component? | 15:47 |
| isviridov | ikhudoshyn, are you going to start one more service for that> | 15:48 |
| isviridov | ? | 15:48 |
| ikhudoshyn | isviridov: I wrote in launchpad, bulk load is CPU hungry | 15:48 |
| ikhudoshyn | and it uses geventlet.pywsgi INSTEAD of geventlets.wsgi | 15:49 |
| isviridov | ikhudoshyn, sorry, where i can read it? | 15:49 |
| isviridov | Could you give me a link? | 15:49 |
| ikhudoshyn | I thought we could run it as a separate process, but having a paste app will help us utilize authoring and other filters | 15:49 |
| miarmak | isviridov: it is here: https://blueprints.launchpad.net/magnetodb/+spec/bulk-data-load-rest | 15:50 |
| ikhudoshyn | ^ ^ tnx miarmak | 15:51 |
| isviridov | miarmak, thx. A lot BPs with almost the same | 15:51 |
| isviridov | ikhudoshyn, do you think we can use geventlets.wsgi for that? | 15:53 |
| ikhudoshyn | miarmak: i'd suggest renaming bulk-data-load to bulk-data-load-cli, it is confusing | 15:53 |
| ikhudoshyn | isviridov: no way. geventlets.wsgi does not support streaming | 15:53 |
| isviridov | It is third BP, I think | 15:53 |
| isviridov | What about pecan? | 15:54 |
| ikhudoshyn | isviridov: didnt look at that beast yet | 15:54 |
| ikhudoshyn | ikhudoshyn: if it does, we can have move to it all together | 15:55 |
| isviridov | I believe that we will use that beast very soon even for current API | 15:55 |
| ikhudoshyn | but once again, serving one bulk request eats above %100 CPU | 15:55 |
| isviridov | ikhudoshyn, but it is still API | 15:56 |
| ikhudoshyn | there is one more issue with that | 15:56 |
| isviridov | We can also kill api instance with a lot of requests | 15:56 |
| ikhudoshyn | it does not utilize storage API layer (PutItemRequers and stuff) | 15:57 |
| ikhudoshyn | so it is the API, but kind of another sort of it | 15:57 |
| miarmak | ikhudoshyn: done! it is renamed | 15:57 |
| ikhudoshyn | miarmak: tnx | 15:57 |
| isviridov | miarmak, please put link here. Thx | 15:57 |
| ikhudoshyn | isviridov: anyway, we can merge it into single API instance once it ready and we moved to pecan | 15:58 |
| miarmak | isviridov: https://blueprints.launchpad.net/magnetodb/+spec/bulk-data-load-cli | 15:58 |
| isviridov | ikhudoshyn, we have to keep code base solid. And if we are adding some extra things, it means we have to handle it properly. | 15:59 |
| isviridov | ikhudoshyn, are you planning to keep own storage implementation? | 15:59 |
| isviridov | What the reason not to extend current storage api with streaming featues? | 16:00 |
| ikhudoshyn | isviridov: cos its freakin slow | 16:00 |
| ikhudoshyn | 4 hours for parsing and inserting half gig data | 16:01 |
| isviridov | ikhudoshyn, sorry what is slow? | 16:01 |
| ikhudoshyn | and streaming won't help here | 16:01 |
| ikhudoshyn | 1) parsing JSON into PutRequestItem 2)CassandraStorageImpl.put_item(PutRequestItem) | 16:02 |
| isviridov | What part is slow? | 16:02 |
| ikhudoshyn | 1) and transforming PutRequestItem into CQL | 16:02 |
| ikhudoshyn | 1) is way to slow | 16:02 |
| ikhudoshyn | PutItemRequest is good for pluggability and strong consistent storage API but way to slow for big amounts of data | 16:04 |
| isviridov | But all cassandra is hidden under storage interface you can do there anything you want. | 16:04 |
| ikhudoshyn | u mean pass JSON to storage layer? | 16:08 |
| ikhudoshyn | then the question would be why in earth all that models if we could do with simple jsons | 16:08 |
| ikhudoshyn | 2 mins, gonna make some coffee | 16:11 |
| isviridov | sure | 16:11 |
| ikhudoshyn | im back | 16:14 |
| ikhudoshyn | what I thought about was, like having rest API and behalf it storage API and behalf it pluggable implementation | 16:15 |
| ikhudoshyn | but bulk load is rather implementation specific, so it stands aside | 16:16 |
| ikhudoshyn | we could put it as close to cassandra (i mean source code tree) as you wish, but it still sounds like an optional and separate stuff | 16:17 |
| ikhudoshyn | otherwise no one would like to use slow 'put_item' just because it is made 'properly' | 16:18 |
| ikhudoshyn | btw i heard Dima started doing another pluggability, he seems to love the word a lot | 16:19 |
| ikhudoshyn | isviridov: r u there? | 16:19 |
| miarmak | ikhudoshyn: do you mean parallel batch? | 16:20 |
| ikhudoshyn | miarmak: i haven't seen working asyncronous parallel batch yet. so no, i dont mean batch | 16:21 |
| isviridov | Are you going to support indexes? | 16:22 |
| miarmak | ikhudoshyn: there is no any working parallel batch yet) Dima said, that faced with some bugs there | 16:22 |
| ikhudoshyn | thats what i'm in rite now | 16:22 |
| ikhudoshyn | im adding index support to bulk | 16:23 |
| isviridov | As separate code? | 16:23 |
| ikhudoshyn | as addition to my poc code | 16:23 |
| isviridov | Are you implementing your own or using storage? | 16:24 |
| ikhudoshyn | which i'd like to be in btw | 16:24 |
| ikhudoshyn | now i take code from storage and cut edges | 16:24 |
| isviridov | You are creating a code island in project | 16:25 |
| ikhudoshyn | i can quit if u want | 16:26 |
| ikhudoshyn | look it's data transformation that is that slow | 16:26 |
| isviridov | Do you think we will merge it eventually? | 16:26 |
| ikhudoshyn | and it all done in python what makes it even slower | 16:26 |
| ikhudoshyn | we have now nice and pretty storage api but it is really slow | 16:27 |
| ikhudoshyn | I hope it get merget as soon as u look at it | 16:27 |
| ikhudoshyn | actually | 16:27 |
| isviridov | I understand your passion, but if we adding one more independent component we have to handle it correctly from deployment and testing point of view. | 16:28 |
| ikhudoshyn | agree | 16:28 |
| ikhudoshyn | i'd love to have it all-in-one | 16:28 |
| ikhudoshyn | just cant see how | 16:28 |
| isviridov | So we have 2 opinions now: | 16:28 |
| isviridov | 1. make it as all in one and follow and extend current architecture | 16:29 |
| isviridov | 2. introduce one more component for specific needs | 16:29 |
| ikhudoshyn | yep | 16:30 |
| ikhudoshyn | we might gather others and vote | 16:30 |
| ikhudoshyn | i personally really hate 2, but 1 just not an option for me | 16:30 |
| isviridov | With 1. we are still keeping DB backend pluggubility, what is defined by interface. | 16:31 |
| miarmak | maybe mailing list? | 16:31 |
| miarmak | ask guys from other projects | 16:31 |
| ikhudoshyn | miarmak: ML is good, but it would trtake another week to decide | 16:31 |
| miarmak | ikhudoshyn: yeah, you're right | 16:32 |
| isviridov | With 2 we have one more cassandra client, deployed separately and no interface, and no interface for DB as well. | 16:32 |
| isviridov | I see 2 possible only if we add additional abstraction layer for that | 16:33 |
| isviridov | Like there are two interfaces you have to implement for DB pluggability: sync and async | 16:34 |
| ikhudoshyn | isviridov: that's funny. i just told that having 2 abstraction layers make it slow and u suggest to add the third to make it faster | 16:34 |
| ikhudoshyn | i used to be java, too | 16:34 |
| ikhudoshyn | i wouldn't even bother about havinng abstract interface for bulk load cos it kinda of no use. bulk load seems to be too intimate to backend | 16:36 |
| ikhudoshyn | say if we have hbase as a backend and wanna bulk load data from another hbase.. | 16:37 |
| isviridov | And I'm sure it will cost more to implement and keep working both | 16:38 |
| ikhudoshyn | sure it will. I will cost more to us, or others just won't buy | 16:39 |
| isviridov | With two layers I mean sync and async storage api | 16:40 |
| ikhudoshyn | there is another option which I hate even more than your 2nd, at least for now | 16:40 |
| isviridov | What is this? | 16:40 |
| ikhudoshyn | take a very close look on storage API -- and make it as lightweight as possible | 16:41 |
| ikhudoshyn | what i c now, it is kinda make really abstract, but we only looked at dynamodb and cassandra when made it | 16:41 |
| ikhudoshyn | so we have very abstract iface between dynamo and cassandra, but don't even know if it is reasonable for other backends | 16:43 |
| ikhudoshyn | like YAGNI u know | 16:43 |
| ikhudoshyn | easy sample, we have models.AttributeType, that has element_type and collection_type and so complicated superclass with frosenset(set(....)) as a hash and stuff | 16:44 |
| ikhudoshyn | but for our case it is just 'S' or 'SS' or 'NS' | 16:45 |
| ikhudoshyn | and we dont know whether this AttributeType will be any good for other backends. I e.g. feel that it won't for Mongo, and not sure for hbase | 16:46 |
| isviridov | BTW could you reuse indexing logic in your current POC? | 16:46 |
| isviridov | Just asking how far it is from current implementation. | 16:47 |
| ikhudoshyn | what d'u mean by 'logic'? use existing code? i could, but i actually aldeady dont. cos that logic is bounded tight to AttributeTypes and stuff | 16:47 |
| isviridov | Sounds like a good reason to make it better. | 16:50 |
| ikhudoshyn | yup | 16:51 |
| ikhudoshyn | but i wouldn't doing that half-ass, and making it rite would be huge, i mean really huge amount of work, especially for qa guys | 16:52 |
| isviridov | Ok, let us involve dukhlov to discussion. | 16:54 |
| ikhudoshyn | we should have done it hour ago ))) | 16:54 |
| ikhudoshyn | i actually told him, he promised to take a look but now seems to be busy with other stuff | 16:55 |
| isviridov | Let is catch him, his eyes would be very helpful | 16:56 |
| isviridov | But i think only tomorrow | 16:56 |
| *** vnaboychenko has joined #magnetodb | 16:58 | |
| ikhudoshyn | isviridov: man its late for u | 16:58 |
| ikhudoshyn | lets continue tomorrow indeed | 16:59 |
| miarmak | ikhudoshyn: so, what should i do with that bp? | 16:59 |
| ikhudoshyn | miarmak: u mean *-rest bp? | 17:21 |
| ikhudoshyn | miarmak: sry, was away | 17:22 |
| *** ominakov has quit IRC | 17:23 | |
| miarmak | ikhudoshyn: yes, with it | 17:25 |
| *** jeromatron has quit IRC | 17:32 | |
| isviridov | ikhudoshyn, see you tomorrow | 17:34 |
| miarmak | bye guys | 17:50 |
| *** vnaboychenko has quit IRC | 17:52 | |
| *** vnaboychenko has joined #magnetodb | 17:53 | |
| *** ominakov has joined #magnetodb | 18:24 | |
| *** vnaboychenko has quit IRC | 18:41 | |
| *** pshchelo has joined #magnetodb | 18:51 | |
| *** dukhlov_ has joined #magnetodb | 19:19 | |
| *** openstackgerrit has quit IRC | 19:34 | |
| *** openstackgerrit has joined #magnetodb | 19:42 | |
| *** jeromatron has joined #magnetodb | 19:56 | |
| *** pshchelo has quit IRC | 19:58 | |
| *** pas-ha has quit IRC | 20:01 | |
| *** pas-ha has joined #magnetodb | 20:01 | |
| *** openstackstatus has quit IRC | 20:15 | |
| *** openstackstatus has joined #magnetodb | 20:15 | |
| *** jeromatron has quit IRC | 20:20 | |
| *** dukhlov_ has quit IRC | 20:38 | |
| *** jeromatron has joined #magnetodb | 20:59 | |
| *** pas-ha has quit IRC | 21:04 | |
| *** jeromatron has quit IRC | 21:07 | |
| *** openstackstatus has quit IRC | 21:14 | |
| *** jeromatron has joined #magnetodb | 21:18 | |
| *** openstackstatus has joined #magnetodb | 21:21 | |
| *** openstack has joined #magnetodb | 22:34 | |
| *** jeromatron has joined #magnetodb | 22:37 | |
| *** jeromatron has quit IRC | 22:41 | |
| *** ominakov has quit IRC | 23:30 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!