| *** alcabrera has joined #openstack-marconi | 00:34 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: fix(mongo): Negative age reported in stats https://review.openstack.org/50790 | 00:35 |
|---|---|---|
| alcabrera | kgriffs: o/ | 00:38 |
| alcabrera | I make it in just in time to catch the bug fix. | 00:38 |
| kgriffs | heh | 00:39 |
| kgriffs | indeed | 00:39 |
| alcabrera | kgriffs: +2'd - this mistake happened before, same fix strategy as last time. Flip the things! | 00:39 |
| kgriffs | yep. I thought we fixed all the places, but I guess one got missed | 00:39 |
| alcabrera | yup, the stats endpoint. Ah well, nice catch. Always good to see a bug getting closed. | 00:40 |
| alcabrera | kgriffs: any developments while I was away? | 00:41 |
| *** amitgandhi has joined #openstack-marconi | 00:44 | |
| *** fifieldt has quit IRC | 00:52 | |
| *** nosnos has joined #openstack-marconi | 00:59 | |
| *** amitgandhi has quit IRC | 01:26 | |
| kgriffs | alcabrera: Not much to speak of. I ended up having to do email and stuff and didn't get to updating the sharding patch. I will have to try to get that done later tonight or first thing in the morning. | 01:28 |
| alcabrera | kgriffs: ah, alright. Email does tend to pile up, even within a period of a few hours. :P | 01:43 |
| alcabrera | I'm working on the mongodb storage driver for the shards patch, so there's still time. I'm curious to see how the foundational code evolves. :) | 01:46 |
| *** kgriffs is now known as kgriffs_afk | 01:50 | |
| *** reed has quit IRC | 01:58 | |
| *** fifieldt has joined #openstack-marconi | 02:04 | |
| *** alcabrera has quit IRC | 03:24 | |
| *** kgriffs_afk is now known as kgriffs | 03:25 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards mongodb driver + tests https://review.openstack.org/50815 | 03:42 |
| *** kgriffs is now known as kgriffs_afk | 04:00 | |
| *** flaper87|afk is now known as flaper87 | 07:16 | |
| *** reed has joined #openstack-marconi | 07:26 | |
| openstackgerrit | A change was merged to openstack/marconi: fix(mongo): Negative age reported in stats https://review.openstack.org/50790 | 07:34 |
| *** dafter has joined #openstack-marconi | 07:37 | |
| flaper87 | yo yo! Sup' world? | 07:45 |
| *** ykaplan has joined #openstack-marconi | 07:53 | |
| *** yassine has joined #openstack-marconi | 07:57 | |
| *** dafter has quit IRC | 08:09 | |
| *** jarret has quit IRC | 08:26 | |
| *** nosnos has quit IRC | 09:06 | |
| *** ykaplan has quit IRC | 09:15 | |
| *** reed has quit IRC | 09:15 | |
| *** dafter has joined #openstack-marconi | 09:20 | |
| *** dafter has quit IRC | 09:25 | |
| *** dafter has joined #openstack-marconi | 10:05 | |
| *** alcabrera has joined #openstack-marconi | 10:13 | |
| alcabrera | flaper87: o/ | 10:14 |
| alcabrera | Morning. :) | 10:14 |
| flaper87 | alcabrera: HELLOOOO!!! | 10:14 |
| alcabrera | w00t | 10:14 |
| flaper87 | alcabrera: Man, you woke up early today | 10:14 |
| flaper87 | :D | 10:14 |
| alcabrera | lol, yup. Yesterday, too. In order to make this storage-0sharding thing work asap, I've been doing ~10 hours a day, and getting up this early works best for my wife. :) | 10:15 |
| alcabrera | (first one in the office, yeah!) | 10:15 |
| flaper87 | alcabrera: I call that a Success Story | 10:15 |
| flaper87 | :D | 10:15 |
| alcabrera | hahaha | 10:16 |
| alcabrera | flaper87: so we're going to continue with the openstack review process, and I agree with Kurt - we'll miss out on some really cool ideas and some subtle details if we don't all pull together. :D | 10:17 |
| alcabrera | The storage pipeline idea is one of those awesome ideas that is already changing what we might have done apart. | 10:17 |
| flaper87 | alcabrera: I agree with that as well | 10:18 |
| flaper87 | alcabrera: and guess what? | 10:18 |
| flaper87 | alcabrera: (I mean, I agree with continue with the review process) | 10:18 |
| alcabrera | flaper87: I'm guessing - the POC is done. ;o | 10:19 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Implement common pipeline class https://review.openstack.org/50889 | 10:22 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Use the pipeline for the storage layer https://review.openstack.org/50890 | 10:22 |
| flaper87 | alcabrera: not just the POC ^^^^^^ | 10:22 |
| flaper87 | alcabrera: dude, first time you get here before I've had lunch!!!! | 10:22 |
| flaper87 | :D | 10:22 |
| flaper87 | Take a look there and drop comments. There may be some docstrings missing as well | 10:23 |
| flaper87 | but that's the genera;, working, idea | 10:23 |
| flaper87 | general* | 10:23 |
| alcabrera | flaper87: oooohhh | 10:24 |
| alcabrera | Yup, I'm diving right in! | 10:24 |
| alcabrera | I addressed the last question you had on the multi-update proxy patch. Turns out I wasn't looking closely ennough. :P | 10:24 |
| alcabrera | *enough | 10:24 |
| alcabrera | Mind if I address that particular refactoring in a later patch (fields = {...})? I've got it generalized in the last patch I submitted (shards: mongo storage driver) | 10:25 |
| flaper87 | alcabrera: Sounds like a good plan! Thanks for taking care of that! | 10:26 |
| flaper87 | alcabrera: I seriously doubted of what I was seeing | 10:26 |
| alcabrera | np. Generalizing is fun. :D | 10:26 |
| flaper87 | I was, "mmh, they look very similar" | 10:26 |
| alcabrera | lol | 10:26 |
| flaper87 | :P | 10:27 |
| flaper87 | alcabrera: +2'd | 10:27 |
| alcabrera | flaper87: woot! | 10:27 |
| flaper87 | no no no, I don't like py26 functional tests not being deterministic!!! I've been in that hell before!!! | 10:28 |
| flaper87 | I hate those random failures | 10:28 |
| flaper87 | alcabrera: anyway, Jenkins is happy with the pipeline changes, let me know! | 10:29 |
| flaper87 | I'll have lunch now! | 10:29 |
| flaper87 | bbib | 10:29 |
| alcabrera | kk, enjoy lunch! | 10:29 |
| alcabrera | flaper87: James. Bond. :P | 10:34 |
| *** ykaplan has joined #openstack-marconi | 10:45 | |
| *** dafter has quit IRC | 11:25 | |
| flaper87 | alcabrera: :P | 11:47 |
| * flaper87 is back | 11:47 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: split queues API into public/admin https://review.openstack.org/50701 | 11:48 |
| flaper87 | alcabrera: not sure I understand your comment here: https://review.openstack.org/#/c/50890/1/marconi/queues/storage/pipeline.py | 11:50 |
| flaper87 | the first one | 11:50 |
| flaper87 | I mean, pipeline is already implemented in common/pipeline.py | 11:50 |
| flaper87 | ahhhhh | 11:50 |
| flaper87 | you mean that storage_pipeline can be used for the proxy as well | 11:51 |
| flaper87 | alcabrera: Did I understand correctly? | 11:51 |
| alcabrera | flaper87: yup. :D | 11:52 |
| alcabrera | The storage_pipeline idea is pretty generic, and plays nice with any storage layers we use in the future. | 11:52 |
| flaper87 | alcabrera: yeah, totally agree. I'll rename the function to get_storage_pipeline, though | 11:53 |
| flaper87 | so we can have get_transport_pipeline in case we need it | 11:53 |
| *** tedross has joined #openstack-marconi | 11:54 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: add shard management resource https://review.openstack.org/50702 | 11:54 |
| alcabrera | flaper87: good idea. | 11:54 |
| flaper87 | alcabrera: what are your impressions so far? | 11:55 |
| alcabrera | hmmm... | 11:55 |
| alcabrera | I like the idea. I'll have stronger impressions once I can see what a configuration file entry will look like. :) | 11:55 |
| alcabrera | That's the part where I'm drawing a blank. | 11:56 |
| alcabrera | Because the notion of a list of functions/objects as a pipeline makes perfect sense. | 11:56 |
| alcabrera | (I think I broke the dependency chain in the two patches above, hmmm...) | 11:56 |
| * alcabrera experiences the dangers of rebasing from time to time | 11:56 | |
| flaper87 | alcabrera: I'm writing the docstring right now | 11:57 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: storage sharding schema for marconi-queues https://review.openstack.org/50456 | 11:59 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: split queues API into public/admin https://review.openstack.org/50701 | 11:59 |
| alcabrera | flaper87: awesome@ | 11:59 |
| alcabrera | !! | 11:59 |
| openstack | alcabrera: Error: "!" is not a valid command. | 11:59 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: add shard management resource https://review.openstack.org/50702 | 12:00 |
| flaper87 | alcabrera: btw, I was also thinking! Since we now have separate files for marconi-* services, I don't think it makes sense to have the config params prefixed with *:something | 12:02 |
| flaper87 | I feel like we already discussed this, didn't we? | 12:03 |
| flaper87 | anyway, I'm thinking about removing (queues:|proxy:) from all our configu groups | 12:03 |
| flaper87 | instead of having queues:storage:blah we'd have storage:blah | 12:03 |
| flaper87 | which makes having get_storage_pipeline as a common function easier | 12:04 |
| alcabrera | flaper87: +1 | 12:04 |
| alcabrera | lemme find the bug report... :) | 12:04 |
| alcabrera | flaper87: https://bugs.launchpad.net/marconi/+bug/1231669 | 12:04 |
| alcabrera | Yeah, we discussed it before. | 12:04 |
| flaper87 | ahhh, we did discussed this then | 12:05 |
| flaper87 | :D | 12:05 |
| alcabrera | Or, at least, I remember putting together this bug report as a result. :P | 12:05 |
| flaper87 | cool beans | 12:05 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards storage controller interface https://review.openstack.org/50721 | 12:05 |
| *** dafter has joined #openstack-marconi | 12:05 | |
| *** dafter has quit IRC | 12:05 | |
| *** dafter has joined #openstack-marconi | 12:05 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards mongodb driver + tests https://review.openstack.org/50815 | 12:08 |
| alcabrera | flaper87: woot! Rebasing complete after I reviewed my own patches and found little errors. ;) | 12:09 |
| alcabrera | I wonder how helpful it is as a general practice in a review pipeline to have the author of a patch review their patches the day after they submit? At the very least, it might ease review by others a little down the line as the author creates a tour of a particular patch, but there's the possibility of introducing bias into the review if it isn't done carefully (maybe?). | 12:10 |
| flaper87 | alcabrera: AWESOME, I'll submit this with the docstring, review your stuff and then heads down for the rest of the day working on my PyconIE presentation | 12:10 |
| flaper87 | gotta finish it | 12:11 |
| alcabrera | flaper87: oh yeah! Good luck with that! It's (PyconIE) is going to be awesome, and I hope the videos of the talks are released soon after. :D | 12:11 |
| *** dafter has quit IRC | 12:29 | |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Implement common pipeline class https://review.openstack.org/50889 | 12:30 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Use the pipeline for the storage layer https://review.openstack.org/50890 | 12:30 |
| flaper87 | alcabrera: docstring ^ | 12:30 |
| alcabrera | flaper87: +2 for the first patch, and now... :) | 12:32 |
| alcabrera | flaper87: oooohh, this docstring is awesome. | 12:33 |
| * alcabrera keeps reading | 12:33 | |
| flaper87 | alcabrera: glad you like it. I'm sure it's full of grammatical errors :D | 12:34 |
| alcabrera | flaper87: there's a few. :P I'll point them out in review comments! This patch is ready for a +2 from me after the docstring is cleaned up, but yeah...! I really like this idea. | 12:35 |
| flaper87 | alcabrera: cooooooool, thanks! :) | 12:40 |
| alcabrera | flaper87: all set. The suggestions have been submitted. :D | 12:41 |
| flaper87 | alcabrera: awesome, let me address those so it's ready to be reviewed by kgriffs_afk when he's around | 12:42 |
| alcabrera | cool, cool. :) | 12:43 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Use the pipeline for the storage layer https://review.openstack.org/50890 | 12:47 |
| alcabrera | flaper87: +2'd :D | 12:52 |
| flaper87 | alcabrera: thank you, Sir! :D | 12:52 |
| alcabrera | I've cleaned up my last 5 storage-sharding patches, so time allowing, they're ready for you. :) | 12:53 |
| flaper87 | alcabrera: looking | 12:54 |
| flaper87 | alcabrera: not sure this patch should depend on the schema one: https://review.openstack.org/#/c/50701/4 | 12:56 |
| flaper87 | alcabrera: can I ask what your working copy looks like? | 12:56 |
| flaper87 | I mean, are you working on a single branch ? | 12:56 |
| alcabrera | flaper87: Hmm... | 12:58 |
| alcabrera | How to put it... | 12:58 |
| flaper87 | :P | 12:58 |
| flaper87 | try me! | 12:58 |
| alcabrera | I suppose I made them dependent because I expected to use it very soon, heh. :) | 12:58 |
| alcabrera | Which I do - in the patch right after, where I implement the shards_resource. | 12:58 |
| alcabrera | But I see your point. ;) | 12:58 |
| flaper87 | hahaha | 12:59 |
| flaper87 | I mean, I'm ok with that | 12:59 |
| flaper87 | what always worries me about these dependencies is that -1 one will block the whole queue | 12:59 |
| alcabrera | flaper87: agreed. It's happened to me before. | 12:59 |
| alcabrera | Heh. :P | 12:59 |
| flaper87 | and as I mentioned yday, I'm not sure the transport is the right place for this schema to live in | 12:59 |
| alcabrera | hmmm | 13:00 |
| flaper87 | or is it? | 13:00 |
| alcabrera | It's a good thought to explore. | 13:00 |
| alcabrera | Where else could it live? | 13:00 |
| flaper87 | Isn't this schema related to the sharding thing ? | 13:00 |
| alcabrera | Yup. I considered doing marconi/*/transport/schemas/shards.py | 13:01 |
| alcabrera | Since I'm 95% certain that the schemas will be applied at the transport layer. | 13:01 |
| flaper87 | I was thinking about marconi/queues/storage/shards/ | 13:01 |
| flaper87 | and there we could put the schemas used for sharding, the shard manager and all sharding related code | 13:02 |
| flaper87 | or marconi/queues/shard | 13:02 |
| flaper87 | so we can keep it separte from the storage | 13:02 |
| flaper87 | and still be used by it | 13:02 |
| alcabrera | hmmm | 13:02 |
| flaper87 | or actually, by both, if needed. | 13:02 |
| flaper87 | Although, I'd prefer to keep the transport completely storage / shard agnostic | 13:03 |
| alcabrera | if they might be used by both, the place to put them would be marconi/common/schemas/* | 13:03 |
| alcabrera | something like that. | 13:03 |
| flaper87 | by both I meant transport and storage | 13:03 |
| alcabrera | And +1 for keeping storage shard-agnostic. That's what the pipeline is for. :) | 13:03 |
| flaper87 | not exactly queues and proxy | 13:03 |
| *** oz_akan_ has joined #openstack-marconi | 13:03 | |
| alcabrera | err, transport. | 13:03 |
| alcabrera | flaper87: I thought that's what you meant - storage vs. transport. | 13:04 |
| flaper87 | TBH, if the pipeline idea makes sense and we all like how those 2 patches work, I'd love to see the shard code using the pipeline right away | 13:05 |
| alcabrera | I like the approach of putting them in marconi/commmon/schemas/* because then I can do something like 'marconi.common.schemas.shard.create'. :) | 13:05 |
| flaper87 | that will isolate changes in a single package and feature | 13:05 |
| alcabrera | flaper87: +1. kgriffs_afk just arrived in the office, too, so we'll be able to make progress on getting his shard storage foundation idea based off of pipelines, or something like that. | 13:06 |
| alcabrera | thoughts for moving schema.py -> marconi/common/schemas/shards.py? | 13:06 |
| flaper87 | alcabrera: I think I'm missing something about this schema but yeah, marconi/common/schemas/* makes sense to me as well | 13:06 |
| *** mpanetta has joined #openstack-marconi | 13:06 | |
| flaper87 | alcabrera: d000d, I'm slow... | 13:06 |
| flaper87 | :D | 13:06 |
| alcabrera | lol | 13:07 |
| alcabrera | sweet | 13:07 |
| alcabrera | already, I'll make that move shortly. | 13:07 |
| flaper87 | awesome, lemme know and I'll +2 that patch | 13:07 |
| *** mpanetta has quit IRC | 13:08 | |
| *** mpanetta has joined #openstack-marconi | 13:09 | |
| *** kgriffs_afk is now known as kgriffs | 13:13 | |
| *** amitgandhi has joined #openstack-marconi | 13:13 | |
| flaper87 | kgriffs: morning | 13:14 |
| kgriffs | o/ | 13:14 |
| kgriffs | I was just reading the log | 13:14 |
| *** mpanetta has quit IRC | 13:14 | |
| *** mpanetta has joined #openstack-marconi | 13:14 | |
| kgriffs | so, I think the pipeline thing makes sense, but I would like to get the shard patch in and then circle back when the pipeline is ready | 13:15 |
| kgriffs | flaper87: have you already started working on the pipeline? | 13:15 |
| flaper87 | kgriffs: https://review.openstack.org/#/q/status:open+project:openstack/marconi+branch:master+topic:storage-pipeline,n,z | 13:15 |
| flaper87 | kgriffs: yup :) | 13:15 |
| flaper87 | it already works | 13:15 |
| flaper87 | and as a POC, I migrated both storage backends to it | 13:16 |
| alcabrera | flaper87 has done an awesome job rolling this out so quickly. | 13:16 |
| alcabrera | I'm almost done updating my schema patch with the suggestions given. | 13:17 |
| kgriffs | eeeeeenteresting | 13:17 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: storage sharding schema for marconi-queues https://review.openstack.org/50456 | 13:18 |
| alcabrera | flaper87: ready. ^^ | 13:18 |
| alcabrera | I'll add the deps on in a moment. :P | 13:18 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: split queues API into public/admin https://review.openstack.org/50701 | 13:19 |
| alcabrera | flaper87: dependency on schema removed for this patch. ^^ | 13:20 |
| *** fifieldt has quit IRC | 13:20 | |
| *** ykaplan has quit IRC | 13:21 | |
| kgriffs | flaper87: so, I was just looking over those pipeline patches | 13:22 |
| flaper87 | alcabrera: COOL, I'll take a look! | 13:23 |
| flaper87 | kgriffs: shoot | 13:23 |
| kgriffs | There doesn't seem any way to influence the flow of the pipe | 13:23 |
| flaper87 | If you return something != None, the pipe will stop and return | 13:24 |
| kgriffs | the shard manager needs to be able to pick one of N storage controllers to pass control to | 13:24 |
| kgriffs | basically... | 13:25 |
| flaper87 | kgriffs: yup, the shard manager can have an instance of each controller and then pick the storage, call the final method from that storage and return | 13:25 |
| flaper87 | kgriffs: that stops the pipeline | 13:25 |
| kgriffs | i guess I was thinking it would be nice if the pipeline handled that | 13:26 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: storage sharding schema for marconi-queues https://review.openstack.org/50456 | 13:27 |
| flaper87 | Yeah, I had that in mind as well, thing is that chaining returned values into the next call requires to change the method signature | 13:27 |
| *** oz_akan_ has quit IRC | 13:27 | |
| flaper87 | the other thing is that the pipeline need to be agnostic of what the pipes return and do | 13:28 |
| *** oz_akan_ has joined #openstack-marconi | 13:28 | |
| flaper87 | unless we subclass it and make it storage specific | 13:28 |
| kgriffs | for reference: http://paste.openstack.org/show/48210/ | 13:29 |
| * kgriffs thinking | 13:30 | |
| flaper87 | Yeah, exactly | 13:30 |
| * kgriffs smells smoke | 13:30 | |
| mpanetta | kgriffs: You should be careful wit that | 13:30 |
| flaper87 | I thought about implementing that by checking if the returned value was an instance of Pipe | 13:30 |
| *** malini_afk is now known as malini | 13:30 | |
| alcabrera | malini, mpanetta: o/ | 13:31 |
| flaper87 | if ShardManager returns a Pipeline instance, then the pipeline will consume that instead of going forward with the current pipeline | 13:31 |
| flaper87 | mpanetta: LOL | 13:31 |
| flaper87 | mpanetta: morning | 13:31 |
| mpanetta | Morning people :) | 13:31 |
| flaper87 | that's the cleanest way to implement "Pipeline hijacking" | 13:32 |
| kgriffs | It would be nice if the individual controllers didn't have to know/care about the pipe | 13:32 |
| flaper87 | kgriffs: exactly | 13:32 |
| flaper87 | with the current implementation they don't | 13:32 |
| kgriffs | referring to the ascii art... | 13:32 |
| flaper87 | you just have to implement a __getattr__ on the ShardManager | 13:32 |
| flaper87 | make that method call the lookup | 13:32 |
| flaper87 | get the controller instance | 13:32 |
| flaper87 | and getattr(controller, method)(*args, **kwargs) | 13:33 |
| flaper87 | the returned value will stop the pipeline execution | 13:33 |
| *** ykaplan has joined #openstack-marconi | 13:33 | |
| flaper87 | and the controller instance (which is at the end of the pipeline) won't be called | 13:33 |
| kgriffs | yep. | 13:33 |
| alcabrera | a little bit of magic for hijacking things. | 13:34 |
| flaper87 | TBH, I'm a bit worried about depending on return values != None | 13:34 |
| flaper87 | but couldn't think of a better way to do it | 13:35 |
| alcabrera | flaper87: I feel like it should be the other way around. | 13:35 |
| alcabrera | Possibly using an enum | 13:35 |
| * flaper87 notices that alcabrera is an Enum fan | 13:35 | |
| flaper87 | :D | 13:35 |
| alcabrera | pipeline.Continue, pipeline.Exit, etc. | 13:35 |
| alcabrera | Make a decision based off of that ^ | 13:35 |
| flaper87 | alcabrera: yeah, but that requires making the pipes "pipeline-aware" | 13:35 |
| flaper87 | which they are not | 13:36 |
| alcabrera | hmmm... true. | 13:36 |
| flaper87 | or shouldn't be | 13:36 |
| alcabrera | good point | 13:36 |
| * flaper87 once implemented a workflow engine that was also BPMN20 compliant | 13:37 | |
| *** dafter has joined #openstack-marconi | 13:37 | |
| alcabrera | flaper87: I remember reading about pipelines somewhere else over the past 3 days. I realize now that this might be relevant to what we're trying to solve, though I'm not sure how: http://hackage.haskell.org/package/pipes-4.0.0/docs/Pipes-Core.html | 13:38 |
| alcabrera | * not sure how (yet) | 13:38 |
| alcabrera | ooohhh, I think I found something. | 13:40 |
| alcabrera | The idea behind this library decouples the pipe-connection mechanism from the transformer in the pipe. | 13:40 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: split queues API into public/admin https://review.openstack.org/50701 | 13:43 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: add shard management resource https://review.openstack.org/50702 | 13:43 |
| alcabrera | hmmm... trying to figure out multiple dependencies. :/ | 13:44 |
| alcabrera | Almost got it. | 13:44 |
| alcabrera | bah, it doesn't seem like gerrit supports multiple deps. :/ | 13:47 |
| alcabrera | flaper87: ^^ | 13:47 |
| *** jraim_ has joined #openstack-marconi | 13:47 | |
| flaper87 | alcabrera: mmh, it does | 13:48 |
| flaper87 | :/ | 13:48 |
| flaper87 | wait, what do you mean exactly? | 13:48 |
| flaper87 | kgriffs: so, thoughts? | 13:49 |
| kgriffs | I was just thinking, we could define the notion of a pipeline meta-stage that would return a callable to the pipeline that the pipeline would then actually call, but that seems like cleverness for the sake of being clever. Might as well just have the pipeline stage call the method itself and appear externally to behave like any other stage. | 13:49 |
| kgriffs | anyway | 13:50 |
| kgriffs | a couple things | 13:50 |
| alcabrera | I want the resource patch to depend on both the 'split API' patch and the 'schema' patch without the 'split API' patch depending on the 'schema' patch. :x | 13:50 |
| kgriffs | 1. Seems like __getattr__ is slow-ish, but I can't remember if it is enough to worry about | 13:50 |
| flaper87 | 1. Not sure how slow it is, TBH. But that's the way to put the pipeline in the middle without making the transport pipline-aware | 13:52 |
| kgriffs | 2. I was actually already planning to make a psuedo "ShardDriver" that just picks the right storage instance behind the scenes and fowards the call to it. That work seems orthogonal to the pipeline work | 13:52 |
| flaper87 | 2. Awesome!!! | 13:52 |
| kgriffs | flaper87: Well, you could make pipeline "controller" classes that implement the various controller interfaces and then just forward things on. | 13:54 |
| kgriffs | OR | 13:54 |
| kgriffs | with some python magic | 13:54 |
| kgriffs | you could... | 13:54 |
| alcabrera | kgriffs: getattr() seems to add about 80ns based on a few local experiments. (Python 2.7.5 and 3.3.2) | 13:55 |
| flaper87 | kgriffs: yeah, I meant, without changing current code | 13:55 |
| kgriffs | dynamically generate those proxy controllers | 13:55 |
| flaper87 | kgriffs: well, we could make it call getattr just once | 13:55 |
| flaper87 | if at least one of the pipes implement it, we can set it as an attribute of the current pipeline instance | 13:56 |
| flaper87 | and that will reduce the cost of __getattr__ if that method is called again | 13:56 |
| kgriffs | ah, since the next time python will find a method on Pipeline with the name, so won't have to fall back to __getattr__ | 13:57 |
| flaper87 | we could also cache the pipes that actually implement those methods and reduce the pipeline for each method | 13:57 |
| flaper87 | kgriffs: yup | 13:57 |
| flaper87 | so for method "create" just pipe 1 and 3 implement it, next time getattr won't be called on pipe 2 | 13:58 |
| kgriffs | +1 | 13:58 |
| flaper87 | I'll implement those optimizations in a separate patch! | 13:58 |
| flaper87 | if you guys like current implementation, lets get it merge | 13:58 |
| flaper87 | so we can base the shard work on that | 13:58 |
| flaper87 | (if we all agree) | 13:59 |
| alcabrera | I'm in favor. | 13:59 |
| kgriffs | FWIW, I just did a quick benchmark | 14:00 |
| * flaper87 hides | 14:00 | |
| alcabrera | woot, data! | 14:00 |
| kgriffs | let me paste that | 14:00 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: add shard management resource https://review.openstack.org/50702 | 14:00 |
| kgriffs | http://paste.openstack.org/show/48211/ | 14:01 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards storage controller interface https://review.openstack.org/50721 | 14:01 |
| kgriffs | I think that perf difference will actually be noticeable when serving lots of concurrent requests, so I'm in favor of the lazy caching idea. | 14:03 |
| alcabrera | __getattr__ gets pretty expensive. :P | 14:03 |
| alcabrera | I was comparing getattr(obj, 'name') to obj.name in my local experiments. | 14:03 |
| zyuan | might appears to be more expensive in pypy | 14:03 |
| flaper87 | kgriffs: yeah, agree, I think setting those methods as attributes of the pipeline instace might make sense | 14:03 |
| zyuan | because pypy focuses on compile time optimization | 14:04 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards storage controller interface https://review.openstack.org/50721 | 14:04 |
| flaper87 | we could actually override __getattribute__ which is *ALWAYS* called | 14:05 |
| flaper87 | and memoize that method instead | 14:05 |
| * flaper87 wonders what the other 241 lines of kgriffs's iPython history have | 14:06 | |
| alcabrera | flaper87: the secrets of the universe. :P | 14:06 |
| flaper87 | LOL | 14:06 |
| kgriffs | ok | 14:06 |
| kgriffs | so I just timed this | 14:07 |
| kgriffs | http://paste.openstack.org/show/48213/ | 14:07 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards mongodb driver + tests https://review.openstack.org/50815 | 14:07 |
| kgriffs | and it was super fast again | 14:07 |
| kgriffs | so, I think that optimization is the way to go | 14:07 |
| flaper87 | kgriffs: cool! | 14:07 |
| flaper87 | so, I'll add that in the next patch | 14:07 |
| kgriffs | kk | 14:07 |
| flaper87 | if you guys agree | 14:07 |
| flaper87 | :D | 14:07 |
| kgriffs | sounds like a plan | 14:07 |
| flaper87 | kgriffs: awesome! | 14:07 |
| alcabrera | flaper87: go for it! :D | 14:08 |
| flaper87 | kgriffs: btw, if you get a chance today, could you take a look at client's patches ? | 14:08 |
| alcabrera | I just finished my rebasing stream, woot. | 14:08 |
| flaper87 | alcabrera: ^ (I think there's still 1 waiting for you) | 14:08 |
| alcabrera | flaper87: yup, queue's API methods. :) | 14:08 |
| alcabrera | I thought I had them all, heh. | 14:08 |
| flaper87 | alcabrera: hehe, that one may need some iterations but it shows how the whole thing works, FINALLY | 14:09 |
| kgriffs | flaper87: sure, but things are pretty crazy today here, so it may not be until later today or tomorrow morning. I'll get to them ASAP. | 14:09 |
| flaper87 | kgriffs: no worries, that works as well. I don't think I'll be able to do much on that tomorrow anyway | 14:09 |
| flaper87 | but a general thought / comment about the API would be nice | 14:09 |
| flaper87 | if there's something to fix, I can tackle it down on Monday's morning | 14:10 |
| alcabrera | flaper87: I'm happy to see the system coming together! I'll give the last part review when I find some time. :) | 14:10 |
| flaper87 | anyway, way to go guys!! This whole Shard + Pipeline thing looks great | 14:10 |
| alcabrera | flaper87: could I get your +2 on schemas? | 14:10 |
| flaper87 | alcabrera: sure you can! Thanks | 14:10 |
| alcabrera | woot | 14:11 |
| alcabrera | I feel like today, I finally understood the secrets of the gerrit-rebase cycle. | 14:12 |
| flaper87 | alcabrera: it's about nutella, isn't it? | 14:12 |
| alcabrera | With long dependency chains, one just has to *really* know what they're doing between 'git review' and... and, yes, nutella. :) | 14:12 |
| * alcabrera gives flaper87 gerritella | 14:12 | |
| flaper87 | LOOL | 14:13 |
| kgriffs | flaper87: can you add a test for calling a method that has *both* args and kwargs? | 14:16 |
| kgriffs | https://review.openstack.org/#/c/50889/2/tests/unit/common/test_pipeline.py | 14:16 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/marconi: Tests for Client-ID validation https://review.openstack.org/50928 | 14:16 |
| flaper87 | kgriffs: mmh, damn, I thought I did! Yup, lemme do that | 14:17 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/marconi: Tests for Client-ID validation https://review.openstack.org/50928 | 14:18 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Implement common pipeline class https://review.openstack.org/50889 | 14:20 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Use the pipeline for the storage layer https://review.openstack.org/50890 | 14:20 |
| flaper87 | kgriffs: alcabrera ther you go ^ | 14:20 |
| alcabrera | flaper87: +2 on the first one, and... | 14:24 |
| alcabrera | I was already +2 on the second one. :P | 14:24 |
| alcabrera | brb | 14:25 |
| flaper87 | alcabrera: thanks | 14:25 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/marconi: Tests for Client-ID validation https://review.openstack.org/50928 | 14:27 |
| alcabrera | malini: fast! +2. :) | 14:28 |
| malini | thanks alcabrera!! | 14:28 |
| kgriffs | come on jenkins... | 14:29 |
| flaper87 | kgriffs: :P | 14:29 |
| flaper87 | kgriffs: both patches got +1 | 14:30 |
| flaper87 | from jenkins | 14:30 |
| kgriffs | rock on | 14:31 |
| alcabrera | woot | 14:31 |
| * alcabrera is looking forward to merges | 14:31 | |
| kgriffs | flaper87: https://review.openstack.org/#/c/50890/4/marconi/common/pipeline.py | 14:35 |
| kgriffs | would there be any benefit to using stevedore to load the pipes? | 14:35 |
| kgriffs | BTW, the words "pipe" and "pipeline" are easily confused | 14:36 |
| kgriffs | we may want to talk about "stages" or something like that to be more clear | 14:36 |
| kgriffs | "filters" is a common term as well | 14:36 |
| flaper87 | kgriffs: I'll rename those to stages in another patch, if that sounds good! | 14:37 |
| kgriffs | sure | 14:37 |
| flaper87 | kgriffs: I thought about using stevedore there | 14:37 |
| flaper87 | but I then thought it was a bit overengineered for this task | 14:37 |
| flaper87 | I may be wrong, though | 14:38 |
| kgriffs | i was just thinking that for get_storage_pipeline it would make sure things implement the storage driver interface, right? | 14:38 |
| flaper87 | actually, maybe using stevedore makes more sense too. By using entrypoints, we can let those filters live elsewhere | 14:39 |
| kgriffs | yeah, that too. | 14:39 |
| kgriffs | :D | 14:39 |
| flaper87 | kgriffs: cool, I'll do that as well. :D | 14:40 |
| *** jraim_ has quit IRC | 14:40 | |
| kgriffs | ok | 14:40 |
| flaper87 | kgriffs: can I do it in a bit? I gotta finish this slides | 14:40 |
| kgriffs | sure, no problem | 14:40 |
| flaper87 | or maybe in a separate patch as the other changes | 14:40 |
| flaper87 | so you can start integrating the shard thing with this | 14:41 |
| kgriffs | it can be a follow-up patch, sounds good | 14:41 |
| alcabrera | +1 for follow up patch. Get those PyconIE slides done. :P | 14:41 |
| flaper87 | kgriffs: alcabrera awesome, thanks guys!!! | 14:41 |
| * flaper87 really wanted to help with the sharding part somehow, this is it! :D | 14:42 | |
| alcabrera | :) | 14:42 |
| kgriffs | alcabrera: I feel like "location" isn't quite the right term, not sure why... | 14:45 |
| kgriffs | https://review.openstack.org/#/c/50456/7/marconi/common/schemas/shards.py | 14:46 |
| kgriffs | how about just calling it "uri" | 14:46 |
| kgriffs | or…something :p | 14:46 |
| alcabrera | kgriffs: 'uri' works for me. I'll take note of that and update it in the current patch. | 14:46 |
| kgriffs | ok | 14:47 |
| kgriffs | thanks! | 14:47 |
| alcabrera | kgriffs: https://etherpad.openstack.org/storage-sharding-issues | 14:47 |
| alcabrera | :D | 14:47 |
| alcabrera | np | 14:47 |
| kgriffs | ok, I'm off to do some major surgery on my sharding patch | 14:47 |
| alcabrera | kgriffs: +2 approve maybe on the schema patch? ;) | 14:48 |
| *** jraim_ has joined #openstack-marconi | 14:49 | |
| kgriffs | alcabrera: Sure. Since we're friends, I'll give you a deal. Only $25. | 14:50 |
| kgriffs | (per patch) | 14:50 |
| flaper87 | kgriffs: d00d, sooo cheap. $25 / LOC | 14:51 |
| alcabrera | Maaaan | 14:51 |
| alcabrera | Alright, here you go... | 14:52 |
| * alcabrera gives kgriffs $25e-13 / LOC | 14:52 | |
| openstackgerrit | A change was merged to openstack/marconi: Implement common pipeline class https://review.openstack.org/50889 | 14:52 |
| openstackgerrit | A change was merged to openstack/marconi: Use the pipeline for the storage layer https://review.openstack.org/50890 | 14:52 |
| openstackgerrit | A change was merged to openstack/marconi: Tests for Client-ID validation https://review.openstack.org/50928 | 14:59 |
| openstackgerrit | A change was merged to openstack/marconi: feat: storage sharding schema for marconi-queues https://review.openstack.org/50456 | 15:20 |
| *** Alex_Gaynor has quit IRC | 15:31 | |
| *** jcru has joined #openstack-marconi | 15:37 | |
| *** Alex_Gaynor has joined #openstack-marconi | 15:40 | |
| kgriffs | flaper87: I think we will need to hoist the pipe creation up a layer, out of storage/base.py | 15:48 |
| kgriffs | the problem is this: | 15:48 |
| kgriffs | the shard manager will load N drivers, one per partition, each with a unique connection string (uri) | 15:49 |
| kgriffs | but when each of those loads up | 15:49 |
| kgriffs | it will create it's own pipeline | 15:49 |
| kgriffs | which will then load in the shard manager | 15:49 |
| kgriffs | which will then load drivers | 15:49 |
| kgriffs | which will then load the pipelline | 15:49 |
| openstackgerrit | Zhihao Yuan proposed a change to openstack/marconi: feat(mongo): starts to use aggregation for stats https://review.openstack.org/50969 | 15:49 |
| kgriffs | which will then load shard manager | 15:50 |
| kgriffs | ... | 15:50 |
| kgriffs | :p | 15:50 |
| * flaper87 (plop) | 15:50 | |
| flaper87 | you completely lost me there :P | 15:50 |
| kgriffs | ok, let me try again. :D | 15:50 |
| flaper87 | the shard manager shouldn't create the pipeline, the pipeline is create by the driver | 15:50 |
| flaper87 | ohhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh | 15:51 |
| kgriffs | right | 15:51 |
| flaper87 | that means, I know what you mean | 15:51 |
| flaper87 | :D | 15:51 |
| kgriffs | heh | 15:51 |
| flaper87 | but | 15:51 |
| flaper87 | (silence in my brain) | 15:52 |
| kgriffs | how about constructing the pipeline in the bootstrap or something? | 15:52 |
| flaper87 | so, you're basically saying. Let's have a ShardManager that creates as many storage.driver instances as needed | 15:52 |
| flaper87 | and gets controllers from there | 15:52 |
| flaper87 | by calling lookup | 15:52 |
| kgriffs | yeah | 15:53 |
| flaper87 | sounds good to me | 15:53 |
| kgriffs | that way the drivers don't know they are being sharded | 15:53 |
| flaper87 | kgriffs: I thought about the bootstrap thing but I didn't do it because I didn't think about this issue. I thought that being able to controll the controllers isntanciation was enough | 15:53 |
| kgriffs | and you can mix-and-match different driver types if you want | 15:54 |
| flaper87 | but now, it sounds like a better option | 15:54 |
| kgriffs | ok | 15:54 |
| flaper87 | kgriffs: +1 sounds good! | 15:54 |
| kgriffs | did you want to make that change? | 15:54 |
| kgriffs | or I can do it, doesn't matter to me | 15:55 |
| flaper87 | kgriffs: if you've some extra time to work on that, it'd be great. Otherwise, I'll take it | 15:55 |
| * flaper87 is stuck with the slides | 15:55 | |
| flaper87 | mmh | 15:56 |
| flaper87 | 1 thing | 15:56 |
| flaper87 | how will you know what controller it is targetting ? | 15:57 |
| flaper87 | I mean, if you've just 1 pipeline | 15:57 |
| *** dafter has quit IRC | 15:57 | |
| flaper87 | ah, nevermind | 15:57 |
| flaper87 | this is what happens when you start working on presentations and stop coding! | 15:58 |
| * flaper87 STFU | 15:58 | |
| kgriffs | heh | 15:58 |
| kgriffs | i guess the pipeline will end up looking like | 15:58 |
| * alcabrera catches up, goes back to writing unit tests | 15:58 | |
| ametts | http://www.youtube.com/watch?v=bWGtjqv19ZA | 15:58 |
| kgriffs | ametts: LOL | 15:59 |
| kgriffs | ==> Stage X ==> Controller | 16:00 |
| kgriffs | but then if you insert shard manager in the pipeline config, after Stage X, | 16:00 |
| kgriffs | ==> Stage X ==> Shard Manager [==> Controller] | 16:00 |
| kgriffs | where, the last stage never gets executed, actually | 16:01 |
| flaper87 | yeah, exactly | 16:01 |
| flaper87 | but, you'll have 1 pipeline per controller anyway | 16:01 |
| *** yassine has quit IRC | 16:01 | |
| flaper87 | ametts: LOOOOOL | 16:02 |
| kgriffs | yeah, the bootstrap will create 1 pipeline per controller | 16:02 |
| alcabrera | ametts: hahaha, nice. | 16:02 |
| *** mpanetta has quit IRC | 16:02 | |
| *** dafter has joined #openstack-marconi | 16:02 | |
| *** mpanetta has joined #openstack-marconi | 16:03 | |
| alcabrera | brb | 16:04 |
| *** mpanetta_ has joined #openstack-marconi | 16:05 | |
| *** mpanetta_ has quit IRC | 16:06 | |
| *** dafter has quit IRC | 16:06 | |
| openstackgerrit | Zhihao Yuan proposed a change to openstack/marconi: feat(mongo): starts to use aggregation for stats https://review.openstack.org/50969 | 16:06 |
| *** mpanetta_ has joined #openstack-marconi | 16:06 | |
| *** dafter has joined #openstack-marconi | 16:06 | |
| *** mpanetta has quit IRC | 16:07 | |
| *** jraim_ has quit IRC | 16:08 | |
| *** alcabrera has quit IRC | 16:09 | |
| *** jraim_ has joined #openstack-marconi | 16:11 | |
| *** ykaplan has quit IRC | 16:18 | |
| openstackgerrit | Chad Lung proposed a change to openstack/marconi: Updated Marconi ReadMe file with correct git repo address https://review.openstack.org/50976 | 16:20 |
| *** flaper87 is now known as flaper87|afk | 16:20 | |
| *** ykaplan has joined #openstack-marconi | 16:25 | |
| *** alcabrera has joined #openstack-marconi | 16:25 | |
| *** dafter has quit IRC | 16:42 | |
| *** dafter has joined #openstack-marconi | 17:26 | |
| *** dafter has quit IRC | 17:42 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: split queues API into public/admin https://review.openstack.org/50701 | 17:42 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: add shard management resource https://review.openstack.org/50702 | 17:43 |
| *** ykaplan has quit IRC | 17:44 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards storage controller interface https://review.openstack.org/50721 | 17:45 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards mongodb driver + tests https://review.openstack.org/50815 | 17:47 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/marconi: Validation for age returned by queue/stats https://review.openstack.org/50995 | 17:52 |
| *** jraim_ has quit IRC | 18:06 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: integrate shard storage with transport https://review.openstack.org/50998 | 18:07 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/marconi: Validation for age returned by queue/stats https://review.openstack.org/50995 | 18:14 |
| *** mpanetta has joined #openstack-marconi | 18:14 | |
| *** mpanetta_ has quit IRC | 18:16 | |
| *** mpanetta has quit IRC | 18:16 | |
| *** mpanetta has joined #openstack-marconi | 18:16 | |
| *** jraim_ has joined #openstack-marconi | 18:21 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: WIP: AutoConnect retry logic https://review.openstack.org/51006 | 18:21 |
| *** dafter has joined #openstack-marconi | 18:22 | |
| *** dafter has quit IRC | 18:22 | |
| *** dafter has joined #openstack-marconi | 18:22 | |
| *** mpanetta has quit IRC | 18:26 | |
| *** mpanetta has joined #openstack-marconi | 18:26 | |
| *** acabrera has joined #openstack-marconi | 18:34 | |
| *** alcabrera has quit IRC | 18:38 | |
| openstackgerrit | Zhihao Yuan proposed a change to openstack/marconi: feat(mongo): starts to use aggregation for stats https://review.openstack.org/50969 | 18:49 |
| *** mpanetta has quit IRC | 18:54 | |
| *** ykaplan has joined #openstack-marconi | 18:59 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: chore: Remove GC cruft from storage driver base class https://review.openstack.org/51016 | 19:14 |
| acabrera | I'm heading home. See you guys later. :) | 19:55 |
| *** acabrera has quit IRC | 19:56 | |
| *** malini is now known as malini_afk | 19:57 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: chore: Remove GC cruft from storage driver base class https://review.openstack.org/51016 | 20:03 |
| openstackgerrit | A change was merged to openstack/marconi: Updated Marconi ReadMe file with correct git repo address https://review.openstack.org/50976 | 20:09 |
| *** jdaggett1 has joined #openstack-marconi | 20:17 | |
| *** jdaggett1 has quit IRC | 20:21 | |
| *** jdaggett1 has joined #openstack-marconi | 20:21 | |
| *** vkmc has joined #openstack-marconi | 20:29 | |
| *** vkmc has quit IRC | 20:29 | |
| *** vkmc has joined #openstack-marconi | 20:29 | |
| *** jraim_ has quit IRC | 20:36 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: Setup storage pipeline in the boostrap instead of driver base https://review.openstack.org/51049 | 20:43 |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: Setup storage pipeline in the boostrap instead of driver base https://review.openstack.org/51049 | 20:45 |
| *** oz_akan_ has quit IRC | 21:02 | |
| *** etoews1 has joined #openstack-marconi | 21:14 | |
| etoews1 | in the service catalog, what is the "type" for marconi? is it "queues"? | 21:21 |
| kgriffs | We discussed that in our last team mtg. Nothing is final, but it will be either queues or queueing | 21:25 |
| kgriffs | probably the latter | 21:25 |
| *** whenry has joined #openstack-marconi | 21:25 | |
| kgriffs | for example | 21:26 |
| kgriffs | queueing | 21:26 |
| kgriffs | https://review.openstack.org/#/c/47999/1/files/keystone_data.sh | 21:26 |
| kgriffs | etoews1: does that help? | 21:26 |
| *** jraim_ has joined #openstack-marconi | 21:28 | |
| etoews1 | kgriffs: yes it does. so the type is "queuing". | 21:30 |
| etoews1 | do you have a feeling for when the devstack impl will be done? | 21:31 |
| kgriffs | not sure. Flavio (flaper87) was working on that | 21:34 |
| kgriffs | I forgot to ask him for an update at the last team mtg | 21:34 |
| etoews1 | k. thx for the pointer. | 21:35 |
| *** jcru has quit IRC | 21:57 | |
| *** dafter has quit IRC | 22:05 | |
| *** dafter has joined #openstack-marconi | 22:05 | |
| *** dafter has quit IRC | 22:10 | |
| *** etoews1 has quit IRC | 22:16 | |
| *** tedross has quit IRC | 22:18 | |
| *** jraim_ has quit IRC | 22:19 | |
| *** jraim_ has joined #openstack-marconi | 22:27 | |
| *** ykaplan has quit IRC | 22:30 | |
| *** amitgandhi has quit IRC | 22:34 | |
| *** jdaggett1 has quit IRC | 22:45 | |
| *** jraim_ has quit IRC | 22:56 | |
| *** jraim_ has joined #openstack-marconi | 22:59 | |
| *** jraim_ has quit IRC | 22:59 | |
| *** metral has joined #openstack-marconi | 23:34 | |
| metral | trying to setup marconi locally with mongo on a clean ubuntu 12.04 vm. upon doing $ python setup.py develop, im getting the following error: TypeError: dist must be a Distribution instance. there are also warnings such as: UserWarning: Unknown distribution option: 'pbr' and UserWarning: Unknown distribution option: 'setup_requires'. anyone care to help? | 23:35 |
| vkmc | Hey metral, you should try with virtualenv | 23:39 |
| metral | hey vkmc - thx for the help & reply | 23:39 |
| metral | i am using virtualenv | 23:39 |
| vkmc | Oh, great | 23:39 |
| vkmc | Well, some time ago I had the same problem and I solved it with virtualenv... but if you are already using it, it should be for another reason I'm not aware of :/ | 23:40 |
| metral | could be user error ;), so ill try again from scratch just to make sure | 23:40 |
| metral | ;) | 23:40 |
| vkmc | Ok :) If you cannot solve it, probably you will find more people around tomorrow at 10-11am UTC | 23:41 |
| metral | sounds good! appreciate the info | 23:42 |
| vkmc | Np, let us know what happens | 23:42 |
| metral | will do | 23:44 |
| metral | tried from scratch on a clean vm with virtualenv done & source/bin/activated before i did $python setup.py develop and i still get the error: TypeError: dist must be a Distribution instance | 23:49 |
| kgriffs | hmm | 23:54 |
| kgriffs | I haven't ran into that issue myself, but seems like python-setuptools is out of date or something | 23:54 |
| kgriffs | If found this, not sure if it will help you (scroll down to the bottom): http://code.doughellmann.com/virtualenvwrapper/issue/199/packaging-problem-unable-to-use | 23:54 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!