| *** igor_ has joined #openstack-marconi | 00:00 | |
| *** igor_ has quit IRC | 00:04 | |
| *** igor_ has joined #openstack-marconi | 01:00 | |
| *** igor_ has quit IRC | 01:05 | |
| *** nosnos has joined #openstack-marconi | 01:47 | |
| *** malini is now known as malini_afk | 01:53 | |
| *** igor_ has joined #openstack-marconi | 02:01 | |
| *** igor_ has quit IRC | 02:05 | |
| *** vkmc has quit IRC | 02:37 | |
| *** whenry has quit IRC | 02:40 | |
| *** igor_ has joined #openstack-marconi | 03:02 | |
| *** nosnos has quit IRC | 03:07 | |
| *** igor_ has quit IRC | 03:07 | |
| *** mpanetta has joined #openstack-marconi | 03:34 | |
| *** mpanetta has quit IRC | 03:36 | |
| *** mpanetta has joined #openstack-marconi | 03:36 | |
| *** nosnos has joined #openstack-marconi | 03:47 | |
| *** nosnos has quit IRC | 04:01 | |
| *** prashanthr_ has joined #openstack-marconi | 04:10 | |
| *** prashanthr_ has quit IRC | 04:15 | |
| *** prashanthr_ has joined #openstack-marconi | 04:25 | |
| *** nosnos has joined #openstack-marconi | 04:30 | |
| *** mkoderer has joined #openstack-marconi | 04:34 | |
| *** haomaiwang has joined #openstack-marconi | 04:41 | |
| *** igor_ has joined #openstack-marconi | 05:04 | |
| *** prashanthr_ has quit IRC | 05:06 | |
| *** mpanetta has quit IRC | 05:08 | |
| *** igor_ has quit IRC | 05:08 | |
| *** prashanthr_ has joined #openstack-marconi | 05:10 | |
| *** igor has joined #openstack-marconi | 06:04 | |
| *** igor has quit IRC | 06:10 | |
| *** AAzza has joined #openstack-marconi | 06:10 | |
| *** igor has joined #openstack-marconi | 06:24 | |
| *** igor has quit IRC | 06:26 | |
| *** flaper87|afk is now known as flaper87 | 07:10 | |
| *** igor has joined #openstack-marconi | 07:28 | |
| *** igor has quit IRC | 07:32 | |
| *** flwang has joined #openstack-marconi | 08:06 | |
| flwang | flaper87: helllllllllllllllllllllllllllllo | 08:06 |
|---|---|---|
| flaper87 | flwang: what's up???????????????????????????????????????????? | 08:07 |
| flwang | flaper87: nooooooooooooo thing, just say hi | 08:07 |
| flaper87 | flwang: how are you doing? | 08:07 |
| flwang | flaper87: good, packing stuff for the move | 08:07 |
| flaper87 | haha, that's the not-so-fun part of moving | 08:08 |
| flaper87 | :D | 08:08 |
| flwang | flaper87: how about the summit? | 08:08 |
| flwang | what's the final name for Marconi? | 08:08 |
| flaper87 | flwang: we haven't decided yet, we need to do that ASAP. | 08:10 |
| flaper87 | flwang: any ideas? | 08:11 |
| flaper87 | the summit was great, as usual! :D | 08:11 |
| flwang | flaper87: maybe we can follow the rename of Ceilometer | 08:13 |
| *** rwsu has quit IRC | 08:26 | |
| *** jamie_h has joined #openstack-marconi | 08:27 | |
| *** ykaplan has joined #openstack-marconi | 08:27 | |
| *** igor has joined #openstack-marconi | 08:28 | |
| *** igor has quit IRC | 08:33 | |
| *** ykaplan has quit IRC | 08:59 | |
| *** ykaplan has joined #openstack-marconi | 09:04 | |
| *** igor has joined #openstack-marconi | 09:24 | |
| *** igor has quit IRC | 09:28 | |
| *** flaper87 has quit IRC | 09:41 | |
| *** ykaplan has quit IRC | 09:57 | |
| *** igor has joined #openstack-marconi | 09:59 | |
| *** flwang has quit IRC | 10:12 | |
| *** prashanthr_ has quit IRC | 10:18 | |
| *** ykaplan has joined #openstack-marconi | 10:32 | |
| *** haomaiwang has quit IRC | 10:39 | |
| *** haomaiwang has joined #openstack-marconi | 10:39 | |
| *** haomaiw__ has joined #openstack-marconi | 10:43 | |
| *** haomaiwang has quit IRC | 10:45 | |
| *** haomaiwang has joined #openstack-marconi | 10:54 | |
| *** haomaiw__ has quit IRC | 10:55 | |
| *** tedross has joined #openstack-marconi | 11:29 | |
| *** ykaplan has quit IRC | 11:33 | |
| *** prashanthr_ has joined #openstack-marconi | 11:37 | |
| *** AAzza has quit IRC | 11:59 | |
| *** AAzza has joined #openstack-marconi | 11:59 | |
| *** ykaplan has joined #openstack-marconi | 12:06 | |
| openstackgerrit | Alex Bettadapur proposed a change to openstack/marconi: V1 Tests JsonSchema https://review.openstack.org/94212 | 12:14 |
| *** vkmc has joined #openstack-marconi | 12:16 | |
| *** vkmc has quit IRC | 12:16 | |
| *** vkmc has joined #openstack-marconi | 12:16 | |
| *** abettadapur has joined #openstack-marconi | 12:17 | |
| *** flaper87|afk has joined #openstack-marconi | 12:27 | |
| *** flaper87|afk is now known as flaper87 | 12:27 | |
| *** sriram has joined #openstack-marconi | 12:36 | |
| *** sriram has quit IRC | 12:37 | |
| *** sriram has joined #openstack-marconi | 12:37 | |
| *** prashanthr_ has quit IRC | 12:45 | |
| *** flaper87 has quit IRC | 12:46 | |
| *** flaper87 has joined #openstack-marconi | 12:46 | |
| *** ChanServ sets mode: +o flaper87 | 12:46 | |
| *** prashanthr_ has joined #openstack-marconi | 12:48 | |
| *** jchai has joined #openstack-marconi | 12:57 | |
| *** nosnos has quit IRC | 13:02 | |
| *** Obulpathi has joined #openstack-marconi | 13:10 | |
| *** mwagner_lap has quit IRC | 13:11 | |
| *** Obulpathi has quit IRC | 13:13 | |
| *** Obulpathi has joined #openstack-marconi | 13:13 | |
| *** AAzza has quit IRC | 13:15 | |
| *** jay-atl has quit IRC | 13:15 | |
| *** ykaplan has quit IRC | 13:33 | |
| *** ykaplan has joined #openstack-marconi | 13:39 | |
| *** LaceyKite has joined #openstack-marconi | 13:45 | |
| *** LaceyKite has left #openstack-marconi | 13:45 | |
| *** cpallares has joined #openstack-marconi | 13:46 | |
| *** prashanthr_ has quit IRC | 13:56 | |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Revert "Disable Metadata write operations on v1.1" https://review.openstack.org/94374 | 14:15 |
| *** AAzza has joined #openstack-marconi | 14:18 | |
| *** jmckind has joined #openstack-marconi | 14:22 | |
| *** oz_akan_ has joined #openstack-marconi | 14:28 | |
| *** alcabrera|afk is now known as alcabrera | 14:31 | |
| *** jay-atl has joined #openstack-marconi | 14:31 | |
| *** alcabrera is now known as alcabrera|afk | 14:38 | |
| *** LaceyKite has joined #openstack-marconi | 14:45 | |
| *** mpanetta has joined #openstack-marconi | 14:46 | |
| *** prashanthr_ has joined #openstack-marconi | 14:47 | |
| *** mwagner_lap has joined #openstack-marconi | 14:48 | |
| *** alcabrera|afk is now known as alcabrera | 14:54 | |
| alcabrera | o/ | 14:54 |
| *** megan_w|afk is now known as megan_w | 14:54 | |
| mpanetta | Hi | 14:54 |
| alcabrera | marconi meeting in 1 minute? >.> | 14:59 |
| alcabrera | flaper87: ^ | 14:59 |
| *** kgriffs|afk is now known as kgriffs | 14:59 | |
| flaper87 | alcabrera: yup | 14:59 |
| alcabrera | ready! | 14:59 |
| *** mpanetta has quit IRC | 14:59 | |
| *** tjanczuk has joined #openstack-marconi | 15:00 | |
| *** whenry has joined #openstack-marconi | 15:00 | |
| *** mpanetta has joined #openstack-marconi | 15:00 | |
| *** balajiiyer has joined #openstack-marconi | 15:00 | |
| *** tjanczuk has left #openstack-marconi | 15:00 | |
| *** abettadapur has quit IRC | 15:02 | |
| alcabrera | vkmc, prashanthr_, cpallares: team meeting in #openstack-meeting-alt. Join us if you can. :) | 15:03 |
| prashanthr_ | alcabrera: Good morning ! :) | 15:03 |
| prashanthr_ | was just joining now | 15:03 |
| *** malini_afk is now known as malini | 15:03 | |
| alcabrera | good morning! :D | 15:03 |
| *** sriram_p has joined #openstack-marconi | 15:04 | |
| *** mpanetta has quit IRC | 15:05 | |
| *** mpanetta has joined #openstack-marconi | 15:05 | |
| mpanetta | grrr | 15:05 |
| *** sriram_p has quit IRC | 15:08 | |
| *** sriram_p has joined #openstack-marconi | 15:09 | |
| *** jchai is now known as jchai_afk | 15:11 | |
| vkmc | o/ | 15:12 |
| balajiiyer | vkmc can you join us on #openstack-meeting-alt? | 15:12 |
| *** prashanthr_ has left #openstack-marconi | 15:13 | |
| vkmc | I'm there :) didn't want to interrupt | 15:13 |
| *** prashanthr_ has joined #openstack-marconi | 15:16 | |
| *** flwang has joined #openstack-marconi | 15:21 | |
| *** ykaplan has quit IRC | 15:23 | |
| *** jchai_afk is now known as jchai | 15:27 | |
| *** ykaplan has joined #openstack-marconi | 15:42 | |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Revert "Disable Metadata write operations on v1.1" https://review.openstack.org/94374 | 15:48 |
| flaper87 | dinner, brb | 16:01 |
| *** tjanczuk has joined #openstack-marconi | 16:01 | |
| kgriffs | vkmc: prashanthr is going to do the redis driver? | 16:01 |
| *** LaceyKite has quit IRC | 16:01 | |
| prashanthr_ | kgriffs : Yes i am planning for the redis and openstack swift drivers | 16:02 |
| alcabrera | I'm taking down the meeting minutes | 16:02 |
| vkmc | kgriffs, yeah, he is already working in that direction | 16:02 |
| prashanthr_ | just started with the coding today: https://github.com/PrashanthRaghu/marconi-redis and https://github.com/PrashanthRaghu/marconi-swift | 16:03 |
| kgriffs | prashanthr_: let's focus on redis first | 16:03 |
| vkmc | kgriffs, if there is a storage backend that fits Marconi future plans better, then I'd love to work on that | 16:03 |
| balajiiyer | kgriffs: FYI, Juno release schedule is here https://wiki.openstack.org/wiki/Juno_Release_Schedule If you need to realign roadmap based on this. (and before sending an email to ML) | 16:03 |
| prashanthr_ | kgriffs: Sure will do that. | 16:03 |
| kgriffs | vkmc: let's wait for flaper87 to get back from dinner. AMQP could be a good one for you to tackle, or riak, idk - we will need to see what he thinks | 16:04 |
| kgriffs | balajiiyer: thanks, I'll take a look! | 16:04 |
| *** LaceyKite has joined #openstack-marconi | 16:04 | |
| vkmc | kgriffs, sure :) | 16:04 |
| tjanczuk | Here is where Rabbit's amqp 1.0 plug in stands: https://www.rabbitmq.com/plugins.html. It is "experimental" and not among the supported set. An euphemism for a toy. | 16:05 |
| vkmc | kgriffs, I added in that etherpad the ones that most developers were interested about when I did the starting research... I remember flaper87 talking about elasticsearch and flwang about redis | 16:06 |
| kgriffs | tjanczuk: sounds like a toy at the moment. the question is, do they plan to mature it over the next 6 months or is it not being worked on? | 16:07 |
| kgriffs | vkmc: we heard kafka a lot at the summit | 16:07 |
| kgriffs | so, I added it | 16:07 |
| kgriffs | (for sake of discussion) | 16:07 |
| vkmc | kgriffs, awesome, thanks | 16:07 |
| tjanczuk | AFAIK they are not working on it. Given the history behind 0.9-1.0 schism I would be surprised if Rabbit decided to ever productize it. | 16:08 |
| kgriffs | prashanthr_: do you have a minute to discuss the redis driver? | 16:08 |
| prashanthr_ | kgriffs: Yes sure. | 16:11 |
| kgriffs | tjanczuk: booh. that's too bad. | 16:11 |
| tjanczuk | re rabbit and amqp 1.0: this note from 2011 is not very encouraging: https://groups.google.com/forum/#!topic/rabbitmq-discuss/9Hj0FzgyLQk/discussion. It is also telling that nothing really has changed in this picture in the last 3 years. So I would not put my money on AMQP 1.0 being supported by Rabbit any time soon. | 16:12 |
| kgriffs | prashanthr_: so, I've been thinking a lot about how we can make an operator's life easier when deploying Marconi | 16:12 |
| kgriffs | a couple of concerns they have | 16:13 |
| kgriffs | first is what to do about really long queues. | 16:14 |
| malini | flaper87: ping me when you want to chat abt refactoring functional tests | 16:14 |
| *** megan_w is now known as megan_w|afk | 16:14 | |
| kgriffs | second is what to do about really hot queues, meaning how do we ensure QoS without having to use conservative rate limits | 16:15 |
| kgriffs | oz_akan_, mpanetta: BTW - I think you will find this conversation interesting | 16:15 |
| prashanthr_ | kgriffs: That's interesting. I had seen some info on these lines when i was reading the blueprints. | 16:16 |
| kgriffs | prashanthr_: Going forward, I think we need to start thinking of storage more in terms of a "pool" | 16:18 |
| mpanetta | Any issue that can fill a shard or take down queues I am interested in finding a way to fix. | 16:18 |
| kgriffs | mpanetta: I thought you might be interested. :) | 16:18 |
| kgriffs | so here is the thing | 16:18 |
| mpanetta | Yeppers :) | 16:18 |
| kgriffs | the less pool-like the storage, the less efficient and predictable | 16:19 |
| prashanthr_ | mpanetta: :) | 16:19 |
| prashanthr_ | kgriffs: Does "pool" mean an abstraction for storage ? | 16:19 |
| kgriffs | prashanthr_: think of "pool" in the sense of swift. It uses a massive pool of disks, and splits objects across those disks | 16:19 |
| *** whenry has quit IRC | 16:20 | |
| kgriffs | two things you need to create an efficient pool | 16:20 |
| kgriffs | a large number of nodes | 16:20 |
| kgriffs | and a somewhat even distribution of messages across those nodes | 16:21 |
| kgriffs | say you only have 3 servers | 16:21 |
| kgriffs | you assign a group of queues to server A, a second group to server B, etc. | 16:22 |
| *** ykaplan has quit IRC | 16:22 | |
| kgriffs | the trouble is, you can't know in advance whether a given queue will be long and/or hot | 16:22 |
| kgriffs | so, you have to be very conservative in your capacity planning | 16:22 |
| vkmc | maybe you can control the load balance of the server before asigning a queue to the server... but we need to figure out how to do it without losing performance | 16:23 |
| kgriffs | i.e., you have to leave a lot of spare capacity on each server. If you don't and one queue becomes really hot, it will starve requests going to other queues | 16:23 |
| kgriffs | Another issue: you can't break out of the single server. A queue can't "grow" out of a single server. Less of a problem for disk-based storage, but a real issue for RAM stores | 16:24 |
| *** haomaiwang has quit IRC | 16:24 | |
| *** ametts has quit IRC | 16:25 | |
| alcabrera | project-quotas are going to be critical, imo | 16:25 |
| alcabrera | e.g. | 16:25 |
| kgriffs | vkmc: you could, but it would only help a little bit. You would base it on which server is heavily loaded at the moment. But you still don't know what queues will be hot in the future | 16:25 |
| alcabrera | "you can only use X MB of memory for your PID" | 16:25 |
| alcabrera | or X storage | 16:25 |
| mpanetta | alcabrera: ++ | 16:25 |
| alcabrera | guarantees at least fair use of a public resource across all tenants | 16:26 |
| prashanthr_ | kgriffs: I get the point related to spare capacity. | 16:26 |
| prashanthr_ | vkmc: Also i guess once the queue is assigned to the redis server migration is tough. | 16:26 |
| kgriffs | I think quotas and rate limits are always going to be needed. However, we want to be as generous as we can. | 16:26 |
| kgriffs | the way you become more generous is thinking about a large pool of storage | 16:26 |
| kgriffs | because QoS and running out of space are mitigated somewhat, the large the pool you have | 16:26 |
| vkmc | kgriffs, makes sense | 16:27 |
| mpanetta | Also don't forget we need to have better security on the admin endpoint... speaking of managing queues... | 16:27 |
| kgriffs | mpanetta: noted | 16:27 |
| vkmc | prashanthr_, I wasn't thinking about migrating the queues but controlling the load of the server before assigning it... but yeah, in that case, that approach seems to expensive | 16:28 |
| prashanthr_ | vkmc: That's correct. The problem is the non-deterministic nature of the load prediction. | 16:29 |
| vkmc | prashanthr_, totally | 16:29 |
| kgriffs | prashanthr_: at the summit we discussed some ways to do migration with zero downtime. one of the people there needs to write up a migration bp based on that discussion (either myself or someone else - I've got a note to followup) | 16:29 |
| kgriffs | so, let's think about each driver in turn | 16:29 |
| kgriffs | 1. MongoDB | 16:29 |
| kgriffs | how would we make mongo act more like a pool? | 16:30 |
| *** rwsu has joined #openstack-marconi | 16:30 | |
| prashanthr_ | kgriffs: Can I say "10GB" of RAM as large ? caz currently i have just so much capacity to develop. | 16:30 |
| *** Obulpathi has quit IRC | 16:31 | |
| kgriffs | mpanetta, alcabrera: BTW, I think we still need the concept of a "cell" or whatever on the backend - it can be useful to segregate groups of queues by network switch or something, but inside that group you want a really big pool | 16:31 |
| mpanetta | Hmm | 16:31 |
| mpanetta | I wish pools could be dynamic... | 16:32 |
| mpanetta | Why can't they be? | 16:32 |
| alcabrera | kgriffs: could you elaborate on cell? | 16:32 |
| alcabrera | I'm favorable towards moving away from the notion of queue <-> shard, since that enables partitioning messages across different shards (better load balancing) | 16:33 |
| alcabrera | though it increases the lookup cost | 16:33 |
| *** abettadapur has joined #openstack-marconi | 16:33 | |
| alcabrera | it'll no longer be a single query | 16:33 |
| mpanetta | Yeah, what exactly do you mean by cell? | 16:35 |
| *** openstackgerrit has quit IRC | 16:35 | |
| *** Obulpathi has joined #openstack-marconi | 16:36 | |
| *** megan_w|afk is now known as megan_w | 16:36 | |
| *** openstackgerrit has joined #openstack-marconi | 16:36 | |
| *** tjanczuk has quit IRC | 16:36 | |
| vkmc | brb | 16:37 |
| kgriffs | re cell | 16:38 |
| kgriffs | hmm | 16:38 |
| kgriffs | I'll probably have to draw some pictures | 16:38 |
| kgriffs | for now, let's cross out cell and just say "pool" | 16:38 |
| kgriffs | in the code today we call these "shards" | 16:38 |
| kgriffs | but I want to call them pools and get us thinking more along those lines | 16:39 |
| kgriffs | within a pool you can shard a single queue across multiple nodes | 16:39 |
| kgriffs | make sense so far? | 16:39 |
| kgriffs | for example | 16:39 |
| kgriffs | with mongodb you could create a bunch of replica sets, and shard a single queue across multiple sets | 16:40 |
| alcabrera | gotcha | 16:40 |
| alcabrera | so instead of a shard being a discrete entity | 16:40 |
| alcabrera | it is a set of entities | 16:40 |
| alcabrera | so we can at least mitigate lookup cost to that set | 16:40 |
| *** rossk has joined #openstack-marconi | 16:41 | |
| alcabrera | and gain the benefit of dynamic capacity within a cell | 16:41 |
| kgriffs | with Redis we will need to do the sharding ourselves. Redis Cluster is not going to work for time-series data (which a message queue is, basically) | 16:41 |
| kgriffs | alcabrera: right | 16:41 |
| kgriffs | a pool is still a ton of servers, but is constrained by your datacenter design and gear | 16:42 |
| kgriffs | I think we still want to be able to migrate between pools | 16:42 |
| *** LaceyKite has quit IRC | 16:43 | |
| kgriffs | because someone may need to decomission a region in a DC in order to get in an do hardware upgrades | 16:43 |
| alcabrera | yup | 16:43 |
| alcabrera | good point | 16:43 |
| kgriffs | prashanthr_: see my note about redis ^^^ | 16:44 |
| kgriffs | with mongo, we may or may not be able to use it's native sharding | 16:44 |
| *** tjanczuk has joined #openstack-marconi | 16:44 | |
| kgriffs | we have to see if there is an opportunity for race conditions when creating the marker | 16:44 |
| prashanthr_ | kgriffs: Sure ! where can I find it ? | 16:44 |
| kgriffs | prashanthr_: "it"? | 16:45 |
| prashanthr_ | kgriffs: "the notes" ? | 16:45 |
| kgriffs | oh, sorry, I just met what I said here in IRC about having to do our own sharding | 16:46 |
| kgriffs | with AMQP, we can use qpid dispatch to scale out | 16:47 |
| kgriffs | with mongodb we *may* be able to use it's native sharding, but I suspect our monotonic counter is going to give us trouble. We may end up just having to shard by queue name, and employ super fast SSD and quotas | 16:48 |
| *** tjanczuk has quit IRC | 16:48 | |
| prashanthr_ | kgriffs I read it :). Was just trying to read a bit about Redis cluster from my book I have. | 16:49 |
| prashanthr_ | I learnt about Redis data structures over the last week. Just the clustering part remaining. | 16:49 |
| kgriffs | prashanthr_: with redis we will need to do it in the driver (application-level sharding). Also we will need to do replication ourselves. Let's chat about that for a bit. | 16:49 |
| kgriffs | prashanthr_: ok, as you will find, redis just scatters incoming writes based on the key | 16:50 |
| prashanthr_ | kgriffs: Sure. | 16:50 |
| kgriffs | the trouble is, that makes us lose our ordering | 16:50 |
| kgriffs | it also means we have to query several members of the cluster every time when reading the queue | 16:51 |
| kgriffs | prashanthr_: so, here is my idea | 16:51 |
| alcabrera | brb | 16:51 |
| *** whenry has joined #openstack-marconi | 16:52 | |
| kgriffs | we implement our own clustering but make use of the fact that messages are ordered | 16:52 |
| kgriffs | allow me to explain | 16:53 |
| prashanthr_ | kgriffs: Sure. | 16:53 |
| kgriffs | when a message arrives, we choose one of N servers to write the message to | 16:53 |
| kgriffs | so, there is some setting of N that is configurable by the operator - you set it at deployment time and never change it | 16:54 |
| kgriffs | the way you choose the server is critical | 16:54 |
| kgriffs | rather than just hashing with mmh3 or md5 or something, you keep a running counter of messages that have gone to that queue so far | 16:54 |
| *** LaceyKite has joined #openstack-marconi | 16:55 | |
| prashanthr_ | kgriffs: Will that counter of messages be in the memory cache / in redis ? | 16:55 |
| kgriffs | hmmm. I think you would have a HA set of Redis instances (3) for each pool that store the counters | 16:57 |
| kgriffs | the alternative is to use a timestamp | 16:57 |
| kgriffs | let's come back to this in a moment | 16:57 |
| prashanthr_ | kgriffs: Sure. I have a basic idea of it now. | 16:58 |
| kgriffs | let's call this a "clock" whether it is a shared counter or a timestamp | 16:58 |
| kgriffs | so, from the clock, you determine which server (0...N) the message should go through | 16:59 |
| kgriffs | the idea is to batch up messages based on the clock | 16:59 |
| kgriffs | so for X number of clock "ticks" messages go to server 0. For the next X ticks, they go to server 1, etc. | 16:59 |
| kgriffs | wrapping around to server 0 when you reach the end | 17:00 |
| kgriffs | now, when a request comes in to read messages | 17:00 |
| kgriffs | you simply read them off in the same order | 17:00 |
| *** abettadapur has quit IRC | 17:00 | |
| kgriffs | the way you know where the client last read was they give you a marker | 17:00 |
| prashanthr_ | kgriffs: That's really interesting.Now it's pretty clear to me. | 17:01 |
| kgriffs | the marker contains the server ID as well as the message ID last read (note the message ID must be monotonic, similar to mongodb's) | 17:01 |
| kgriffs | that way, we don't have to keep track of client state - they do it for themselves | 17:01 |
| flaper87 | kgriffs: is that what we talked during the summit? or is it based on a new idea? | 17:01 |
| kgriffs | flaper87: same thing | 17:02 |
| flaper87 | kgriffs: cool, I just wanted to avoid reading the backlog. keep going | 17:02 |
| kgriffs | prashanthr_: first step, though, is to create a driver for just a single redis instance. don't so sharding or replicas or anything for the first iteration of the driver | 17:02 |
| kgriffs | flaper87: the backlog gives my justification for wanting to do this | 17:02 |
| kgriffs | (FWIW) | 17:03 |
| kgriffs | prashanthr_: so, we have two things left to sort out for marconi's redis sharding implementation | 17:03 |
| flaper87 | kgriffs: do you think we should do this before scale-up? I'm of the idea that we should do scale-up first and then this. I think it'll give us the bases for the scale-out work | 17:03 |
| kgriffs | flaper87: I think migration of a queue needs to happen right away if possible | 17:04 |
| prashanthr_ | kgriffs: Sure. Will try to get the basic redis operations working. | 17:04 |
| kgriffs | I was thinking j-2 or j-3 to look at mongodb native sharding, and redis sharding | 17:04 |
| kgriffs | prashanthr_: so, two things for the redis sharding | 17:05 |
| flaper87 | kgriffs: agreed, I think j-2 would be better. These kind of features could use a milestone for tests | 17:05 |
| kgriffs | first, what to do when a client does not give us a marker | 17:05 |
| flaper87 | I'd be a bit afraid of releasing it in j-3 | 17:05 |
| kgriffs | second, whether to use a counter or system clock | 17:06 |
| flaper87 | also, it'd need to land before the feature freeze | 17:06 |
| kgriffs | mmm | 17:06 |
| kgriffs | good point | 17:06 |
| * flaper87 STFU and let kgriffs talk w/ prashanthr_ | 17:06 | |
| malini | did I hear milestone for tests? | 17:06 |
| kgriffs | we should just spend all of j-3 for testing and tuning | 17:07 |
| * kgriffs is trying to get on malini's good side | 17:07 | |
| malini | kgriffs: you are making great progress! | 17:07 |
| malini | keep up the good work ;) | 17:07 |
| kgriffs | prashanthr_: my hunch is that we can't simply start the client off at node "0" if they don't supply a marker | 17:07 |
| kgriffs | that may not be the tail of the queue | 17:07 |
| prashanthr_ | kgriffs: counters provide pretty good ordering I guess. What is the default behaviour currently when the marker is not provided ? | 17:08 |
| malini | flaper87: are you going to update this https://review.openstack.org/#/c/87937/ ? | 17:08 |
| malini | I can add CLI tests in Tempest once this lands | 17:08 |
| flaper87 | malini: I will, most likely tomorrow | 17:09 |
| kgriffs | well, we just give them the first n messages from the queue, according to the limit they specified | 17:09 |
| kgriffs | the trouble is that when you shard a single queue, now you have to figure out which node has those first n messages. :p | 17:09 |
| malini | flaper87: cool!! I will ask again tomorrow ;) | 17:09 |
| kgriffs | prashanthr_: let's think about it and discuss more in a few weeks once you have the basic driver done | 17:10 |
| kgriffs | prashanthr_: oh, I just realized I didn't explain reading very well | 17:10 |
| kgriffs | you would read from the last position (according to the marker supplied by the client). | 17:11 |
| kgriffs | say the client asks for 20 messages given a marker for Server 1 (s-1) | 17:11 |
| kgriffs | but after querying s-1 you only get back 5 messages | 17:12 |
| kgriffs | the simple thing to do is just return 5, since the API does not guarantee giving back the requested number anyway | 17:12 |
| kgriffs | the slightly more complex thing to do is to then query s-2 and splice those messages on to the end of the ones you got from s-1 | 17:13 |
| kgriffs | since this boundary condition should happen only rarely, there shouldn't be much extra load on the overall storage pool if you read from 2 nodes to fulfill a single request | 17:13 |
| *** abettadapur has joined #openstack-marconi | 17:14 | |
| *** alcabrera is now known as alcabrera|afk | 17:14 | |
| kgriffs | prashanthr_: do we already have a redis bp? | 17:14 |
| prashanthr_ | kgriffs: Hmm tat's true. More bookkeeping work in the application layer has to be done. Also with redis the operations will be faster. | 17:14 |
| prashanthr_ | kgriffs: What does bp mean ? | 17:15 |
| kgriffs | blueprint | 17:15 |
| kgriffs | let me check on that | 17:15 |
| vkmc | there is one yeah :) | 17:15 |
| vkmc | currently assigned to alcabrera | 17:15 |
| prashanthr_ | kgriffs yes : https://blueprints.launchpad.net/marconi/+spec/redis-storage-driver | 17:15 |
| *** tjanczuk has joined #openstack-marconi | 17:16 | |
| kgriffs | can you assign that to yourself? | 17:16 |
| prashanthr_ | kgriffs: Sure. | 17:16 |
| vkmc | he cannot, the approver has to do it | 17:16 |
| kgriffs | oic | 17:16 |
| kgriffs | I'll do it then | 17:17 |
| kgriffs | btw, here is the juno calendar: https://wiki.openstack.org/wiki/Juno_Release_Schedule | 17:17 |
| prashanthr_ | kgriffs: Yes i do not have the permissions. | 17:17 |
| kgriffs | I'd like to have the basic redis driver (no replica, no sharding) done by end of june | 17:17 |
| kgriffs | so, I will schedule redis bp for juno-1 and it may bleed over to juno-2. Then Let's create a second blueprint for redis-pool | 17:18 |
| kgriffs | prashanthr_: I forgot to mention, we also need to handle replication ourself for HA. 2 nodes should be enough, although I suppose it could be configurable | 17:18 |
| kgriffs | prashanthr_: the reason we can't use redis master-slave is because they ACK *before* waiting on the propagation to the slave | 17:19 |
| kgriffs | so, my idea was, just send two writes to two nodes in the storage pool simultaneously (using async) | 17:19 |
| kgriffs | we can talk about that some more in a few weeks when the basic driver is done | 17:20 |
| prashanthr_ | kgriffs: Sure i can make the number of nodes configurable. | 17:20 |
| prashanthr_ | I will also make a list of to-do's from the discussion today | 17:20 |
| prashanthr_ | so that we can discuss again after the basic driver is up. | 17:21 |
| kgriffs | sounds good. I'll register another blueprint and you can help me flesh out the wiki page for it | 17:21 |
| kgriffs | prashanthr_: btw, I was thinking that you could use a lua "stored procedure" for inserting messages | 17:22 |
| kgriffs | that way you can generate a monotonic message ID and insert the message in a single, atomic transaction | 17:22 |
| kgriffs | We have to do a really bad hack in the mongodb driver since you can't do transactions (or at least, before 2.6 - may be able to do it now) | 17:23 |
| prashanthr_ | kgriffs: that's correct. No need to seperate out the counting and insertion. Else it's tough to accomodate the counter in the redis transaction, | 17:24 |
| prashanthr_ | thanks for the idea :) | 17:24 |
| kgriffs | prashanthr_: the lua thing works because only one can run at a time, so the counter you get won't be jumped by a different request in parallel | 17:24 |
| kgriffs | rock on | 17:24 |
| vkmc | kgriffs, this work plan also applies to the storage I'll be working on, right? | 17:25 |
| kgriffs | prashanthr_: I'm showing two accounts in launchpad for "Prashanth Raghu" | 17:25 |
| vkmc | kgriffs, now that flaper87 is around we can make the selection | 17:25 |
| prashanthr_ | kgriffs: https://launchpad.net/~p-is-prashanth this is the account I use. | 17:26 |
| kgriffs | prashanthr_: good, I guessed right. :) | 17:27 |
| kgriffs | vkmc: yes, I think so. but, depends on whether we need to do extra work for sharding or the backend can do it natively for us | 17:27 |
| kgriffs | in any case you would want to deliver a basic driver by end of june, and then spend remainder of j-2 polishing/completing | 17:28 |
| prashanthr_ | kgriffs :) Awesome. | 17:28 |
| prashanthr_ | To just repeat our discussion: | 17:28 |
| prashanthr_ | 1. Implement a basic driver. | 17:28 |
| prashanthr_ | 2. Make a list of to-do's to accomodate Redis pools. | 17:28 |
| prashanthr_ | 3. Make the pooling work :) | 17:28 |
| prashanthr_ | Is this right ? | 17:28 |
| vkmc | kgriffs, great, I'll submit the corresponding blueprints then | 17:28 |
| kgriffs | flaper87: around? | 17:28 |
| AAzza | by the way, can someone assign this blueprint to me? https://blueprints.launchpad.net/marconi/+spec/py3k-support | 17:30 |
| kgriffs | flaper87: when you have a minute, can you help vkmc come up with a shortlist (2-3) options for her driver, then we can make the final decision after that. | 17:30 |
| prashanthr_ | kgriffs , vkmc , flaper87 , alcabrera , malini : Me retiring for the day . Have a great day ahead :) | 17:30 |
| kgriffs | AAzza: sure thing | 17:30 |
| kgriffs | prashanthr_: take care! Thanks for the chat! | 17:30 |
| malini | prashanthr_: good night! | 17:31 |
| vkmc | prashanthr_, ttyt! enjoy the rest of your evening :) | 17:31 |
| AAzza | me is https://launchpad.net/~grafinya-uvarova | 17:31 |
| prashanthr_ | thank you all ! :) | 17:31 |
| *** prashanthr_ has quit IRC | 17:31 | |
| kgriffs | AAzza: you're all set! | 17:32 |
| AAzza | kgriffs: many thanks) | 17:32 |
| kgriffs | sriram: ping | 17:33 |
| *** igor has quit IRC | 17:34 | |
| sriram | kgriffs:pong | 17:35 |
| kgriffs | hey good buddy | 17:35 |
| *** igor_ has joined #openstack-marconi | 17:35 | |
| kgriffs | we need to chart a course for marconi-bench | 17:35 |
| sriram | yep! | 17:36 |
| kgriffs | for j-1 I'd like to basically take what we've done so far, create a "conductor" that will let us start each client remotely and collect the results, then dump them out to a gnuplot file | 17:37 |
| kgriffs | or, we could do JSON and use flot | 17:37 |
| kgriffs | I think our clients are mostly there | 17:38 |
| kgriffs | mine needs to post to graphite still so we can watch for anomalizes, and we need to figure out how to measure and report "attempted req/sec" | 17:38 |
| *** balajiiyer has quit IRC | 17:38 | |
| *** balajiiyer has joined #openstack-marconi | 17:38 | |
| kgriffs | where, "attempted req/sec" roughly translates to "how many simultaneous requests are we trying here?" | 17:39 |
| kgriffs | hmm | 17:39 |
| *** jmckind has quit IRC | 17:39 | |
| kgriffs | I was going to try to add the "attempted req/sec" and graphite reporting to the producer. Then I will update the gist, and how about you pull that down and submit a patch with both to gerrit? | 17:40 |
| sriram | Sure, I can do that. | 17:40 |
| kgriffs | ok. I will make up a blueprint just for this first iteration | 17:40 |
| sriram | kgriffs: \m/ | 17:42 |
| *** reed has joined #openstack-marconi | 17:43 | |
| kgriffs | sriram: I don't think we need to enable keystone for our first test | 17:45 |
| sriram | yeah, I agree | 17:46 |
| kgriffs | are we now on HAProxy? | 17:46 |
| sriram | kgriffs: I had one setup, which didnt do that much. | 17:47 |
| kgriffs | sriram: oh, is it not configured to balance to the api servers? | 17:48 |
| sriram | kgriffs: it can be setup, just a matter of updating a few conf files. | 17:48 |
| sriram | shouldnt take too long. | 17:48 |
| sriram | it was pointing to the old environment, but the old environment needs to be fixed. | 17:49 |
| kgriffs | ok | 17:50 |
| kgriffs | how many web heads (api servers) are there? | 17:50 |
| sriram | 4 | 17:51 |
| kgriffs | 1. HAProxy load balancer | 17:52 |
| kgriffs | 2. 4x web heads running uwsgi, with Keystone auth disabled (we will bench it enabled later) | 17:52 |
| kgriffs | 3. 1x3 mongo replica set | 17:52 |
| kgriffs | 4. 1x graphite box | 17:52 |
| kgriffs | 3. Load generators (1x producer, 1x consumer, one conductor) | 17:52 |
| kgriffs | does that look correct? | 17:52 |
| kgriffs | oops. last one was a typo | 17:53 |
| kgriffs | 5. 3x Load generator boxes (1x producer, 1x consumer, one conducto | 17:53 |
| sriram | yes | 17:53 |
| sriram | That looks perfect | 17:53 |
| sriram | the graphite box is also setup. | 17:53 |
| sriram | I was able to send messages and create a graph the other day, but it was from the same machine. | 17:54 |
| sriram | havent tried it from another box. | 17:54 |
| kgriffs | sriram: what is your launchpad ID? | 17:56 |
| sriram | thesriram | 17:56 |
| sriram | I think | 17:56 |
| *** alcabrera|afk is now known as alcabrera | 17:56 | |
| sriram | yes thats correct. | 17:56 |
| kgriffs | sriram: https://blueprints.launchpad.net/marconi/+spec/basic-benchmarking | 17:57 |
| sriram | woot! | 17:58 |
| kgriffs | we can add more blueprints later... I don't want to waste time trying to predict the future *too* far in advance | 17:58 |
| kgriffs | because... | 17:58 |
| kgriffs | people may become suspicious | 17:58 |
| * kgriffs has to keep a low profile as a transdimensional being | 17:58 | |
| *** martyntaylor has joined #openstack-marconi | 18:00 | |
| *** abettadapur has quit IRC | 18:05 | |
| sriram | heh | 18:05 |
| *** jamie_h has quit IRC | 18:07 | |
| * flaper87 back | 18:20 | |
| flaper87 | vkmc: kgriffs so | 18:20 |
| flaper87 | kgriffs: re the third-party CI. Are we going to do it? | 18:20 |
| vkmc | hey :) | 18:22 |
| malini | jchai: ping | 18:22 |
| jchai | malini: hello | 18:22 |
| *** abettadapur has joined #openstack-marconi | 18:22 | |
| malini | jchai: I am trying to figure out why https://review.openstack.org/#/c/87762/ is failing jenkins | 18:23 |
| malini | But I have no clue | 18:23 |
| malini | Can you try pinging in #openstack-qa | 18:23 |
| malini | Somebody there might have an idea | 18:23 |
| flaper87 | malini: re functional tests, whenever you want | 18:24 |
| malini | flaper87: It comes with a lot of downs, which makes me wonder if its worth it | 18:24 |
| malini | But right now we are duplicating effort | 18:25 |
| flaper87 | kgriffs: malini https://review.openstack.org/#/c/94374/ | 18:25 |
| flaper87 | malini: mind sharing what the work is about? | 18:25 |
| flaper87 | What does tempest expect us to do? | 18:25 |
| malini | nobody has asked us to update our functional tests to use tempest libs | 18:26 |
| malini | We can continue as is if we choose to | 18:26 |
| malini | But we'll end up writing two sets of tests, for each version | 18:26 |
| malini | which might not be sustainable | 18:26 |
| malini | This is from neutron https://review.openstack.org/#/c/72585/ | 18:28 |
| flaper87 | malini: ok but I still don't know what the change is about :) | 18:28 |
| * flaper87 clicks | 18:28 | |
| peoplemerge | @kgriffs: vkmc: flaper87: did I hear right there's interest in a riak backend? Or did sb mistype and mean redis? | 18:28 |
| malini | flaper87: can I ping you later to discuss this more? am in a meeting now | 18:28 |
| flaper87 | peoplemerge: not exactly interest but it's still a valid option | 18:29 |
| vkmc | peoplemerge, we already have someone working on redis... and we are also considering riak | 18:29 |
| flaper87 | malini: oh sure, please, lets talk later | 18:29 |
| flaper87 | vkmc: is your intership *just* about working on a storage driver? | 18:30 |
| flaper87 | internship* | 18:30 |
| peoplemerge | vkmc: flaper87: good to know, I did a 3 month project this year using riak | 18:30 |
| *** martyntaylor has left #openstack-marconi | 18:30 | |
| vkmc | flaper87, that would be just a part of it... I'll also would like to contribute with another tasks | 18:30 |
| flaper87 | peoplemerge: you could write a riak driver as an external plugin | 18:30 |
| flaper87 | that would be cool | 18:30 |
| *** martyntaylor has joined #openstack-marconi | 18:31 | |
| vkmc | flaper87, but I guess that the central topic should be implementing the driver | 18:31 |
| peoplemerge | flaper87: will take a crack at it, let me familiarize myself with the code | 18:31 |
| flaper87 | vkmc: ok, I'm asking because we've discussed about how many drivers we want to keep in the code base and the number comes down to 2 more for now. That is, sqla+mongo+redis+amqp | 18:31 |
| flaper87 | peoplemerge: please ask. I think there's a cookiecutter template for sotrage drivers | 18:31 |
| * flaper87 tries to find it | 18:32 | |
| flaper87 | vkmc: ok | 18:32 |
| flaper87 | vkmc: I guess it has to be merged into the codebase in order to be considered valid for the internship, right? | 18:32 |
| vkmc | flaper87, I see... well, prashanthr will be working on Redis and he also mentioned Swift | 18:32 |
| peoplemerge | flaper87: that should help | 18:33 |
| vkmc | flaper87, I guess so, not so sure how that work in GSoC | 18:33 |
| vkmc | flaper87, Maybe I could take AMQP? | 18:34 |
| flaper87 | vkmc: sure, we'll need to work close on that one. | 18:36 |
| flaper87 | vkmc: 2 secs | 18:36 |
| *** AAzza has quit IRC | 18:37 | |
| vkmc | flaper87, sure | 18:37 |
| *** peoplemerge has quit IRC | 18:40 | |
| *** peoplemerge has joined #openstack-marconi | 18:40 | |
| *** oz_akan_ has quit IRC | 18:45 | |
| *** oz_akan_ has joined #openstack-marconi | 18:46 | |
| *** megan_w is now known as megan_w|afk | 18:49 | |
| *** tjanczuk has quit IRC | 18:51 | |
| *** AAzza has joined #openstack-marconi | 18:53 | |
| flaper87 | vkmc: back | 18:56 |
| flaper87 | so, yeah, that sounds good to me | 18:56 |
| vkmc | flaper87, awesome | 18:56 |
| flaper87 | we've been having lots of discussions as to whether support 0.9 or 1.0 | 18:56 |
| flaper87 | the thing is that I think 1.0 is the way to go | 18:56 |
| flaper87 | but probably having 0.9 makes sense too. So, you probably will work on one of those | 18:57 |
| vkmc | flaper87, so... we forget all about elasticsearch and rethinkdb? | 18:57 |
| flaper87 | :D | 18:57 |
| flaper87 | vkmc: thing is that, as of now, we don't want those in the codebase | 18:57 |
| flaper87 | :/ | 18:57 |
| flaper87 | Unless people think otherwise | 18:57 |
| flaper87 | By we don't want them is that, we don't believe the Marconi community, as-is, will be able to maintain them all un the long run | 18:58 |
| vkmc | flaper87, sounds reasonable | 18:58 |
| flaper87 | so, it's better to keep the codebase small and clean by letting other drivers live in external repositories | 18:58 |
| *** peoplemerge has quit IRC | 18:58 | |
| *** peoplemerge has joined #openstack-marconi | 18:59 | |
| vkmc | flaper87, well, in fact, I prefered to ask for feedback before proposing anything because I want to contribute with something that fits the project plans | 18:59 |
| vkmc | flaper87, I agree that we should better focus on less things and make them count | 19:00 |
| peoplemerge | flaper87: hope I didn't miss your msg re: cookie-cutter persistence | 19:01 |
| flaper87 | peoplemerge: you didn't, I did miss the thought in my head, though. :P | 19:02 |
| flaper87 | lemme get that for you | 19:02 |
| vkmc | flaper87, so... back to ampq, you consider that we should go for 0.9 or 1.0 | 19:03 |
| *** kgriffs is now known as kgriffs|afk | 19:04 | |
| * peoplemerge grins | 19:06 | |
| flaper87 | 1.0 all the way :D | 19:06 |
| flaper87 | peoplemerge: https://github.com/FlaPer87/cookiecutter-marconi-transport | 19:06 |
| peoplemerge | flaper87: thx | 19:06 |
| flaper87 | damn, it's for transport | 19:06 |
| flaper87 | :( | 19:06 |
| peoplemerge | ah right | 19:06 |
| flaper87 | I guess you could copy mongodb's :D | 19:06 |
| flaper87 | or write a cookiecutter template for storage drivers :D | 19:06 |
| * flaper87 is taking advantage of peoplemerge | 19:06 | |
| peoplemerge | flaper87: probably easier than eating cookie dough | 19:06 |
| *** abettadapur has quit IRC | 19:07 | |
| *** abettadapur has joined #openstack-marconi | 19:07 | |
| flaper87 | vkmc: so, re amqp1.0 | 19:09 |
| flaper87 | mmh, where should I start?... | 19:09 |
| flaper87 | mmh | 19:10 |
| flaper87 | vkmc: btw, when does your internship start? | 19:10 |
| vkmc | flaper87, yesterday | 19:10 |
| vkmc | lol | 19:10 |
| flaper87 | holy crap | 19:10 |
| vkmc | that's why I'm being such a pain in the neck :D | 19:11 |
| flaper87 | so, my suggestion for you is to start reading about AMQP 0.9 and AMQP 1.0 | 19:11 |
| flaper87 | Both protocols are completely different | 19:12 |
| flaper87 | However, AMQP 1.0 is now a standard whereas 0.9 is not. | 19:12 |
| alcabrera | I'll start reading, too, so I can actually help with this. :D | 19:12 |
| vkmc | cool, will do that | 19:12 |
| vkmc | I'm a bit lost though... because I was relating 'storage backend' with something more like mongo | 19:13 |
| vkmc | and AMPQ is a queuing protocol | 19:13 |
| *** tjanczuk has joined #openstack-marconi | 19:13 | |
| alcabrera | the idea is to use marconi as a medium to communicate messages to an amqp system | 19:14 |
| flaper87 | vkmc: it's a protocol but it'll talk to a queuing broker | 19:14 |
| mpanetta | So... we want to store our queues in a queue :P | 19:14 |
| flaper87 | mpanetta: shhhhhhhhhhhhhh | 19:14 |
| alcabrera | mpanetta: exactly -- queue-ception | 19:14 |
| vkmc | yes, that's the feeling mpanetta hahaha | 19:14 |
| mpanetta | haha | 19:14 |
| flaper87 | one of marconi's goals is to support existing technologies | 19:14 |
| mpanetta | Sounds good to me :) | 19:14 |
| flaper87 | hence the will to support existing brokers | 19:14 |
| flaper87 | mpanetta: no voodoo for ya' | 19:15 |
| alcabrera | flaper87: I've found two spec links | 19:15 |
| alcabrera | 1.0: http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-complete-v1.0.pdf | 19:15 |
| alcabrera | 0.9.1: http://www.amqp.org/specification/0-9-1/amqp-org-download | 19:15 |
| alcabrera | are these good places to get into the details? or is that diving too deep? | 19:15 |
| flaper87 | that's probably too detailed | 19:16 |
| flaper87 | 0.9: http://en.wikipedia.org/wiki/AMQP | 19:16 |
| flaper87 | erm, sorry | 19:17 |
| flaper87 | that's for both | 19:17 |
| alcabrera | ah, alright | 19:17 |
| alcabrera | thanks! | 19:17 |
| vkmc | thanks flaper87, alcabrera for the docs | 19:17 |
| alcabrera | I'll give that wiki page a read soon | 19:17 |
| vkmc | just one nit... are we getting closer to what rabbitmq does by adding support for ampq? | 19:18 |
| flaper87 | 0.9: http://www.rabbitmq.com/tutorials/amqp-concepts.html | 19:18 |
| vkmc | I thought that was something we wanted to keep away from | 19:18 |
| flaper87 | vkmc: nope, we're supporting rabbitmq | 19:18 |
| alcabrera | vkmc: no -- we'd in fact be running something like rabbit in the background | 19:18 |
| alcabrera | and marconi would delegate to it, but only in marconi's terms -- queue, claims, messages and their operations | 19:18 |
| vkmc | cool | 19:19 |
| flaper87 | that rabbitmq.com link has enough information about 0.9 | 19:19 |
| flaper87 | I'd recommend reading that one | 19:19 |
| alcabrera | the rabbitmq docs are pretty thorough, it seems. :) | 19:19 |
| flaper87 | these are the brokers supporting 1.0 http://en.wikipedia.org/wiki/AMQP#AMQP_1.0_broker_Implementations | 19:20 |
| alcabrera | it'd be fun to play with this: https://hackage.haskell.org/package/amqp | 19:20 |
| alcabrera | even though it only supports rabbit/0.9.1 | 19:20 |
| vkmc | yes, I also found a comparation between 0.9 and 1.0 here https://www.rabbitmq.com/specification.html | 19:21 |
| flaper87 | brb | 19:22 |
| vkmc | flaper87, kgriffs assign me to https://blueprints.launchpad.net/marconi/+spec/support-amq when you have a moment | 19:23 |
| vkmc | or alcabrera ^^ :) | 19:23 |
| alcabrera | vkmc: assigned | 19:23 |
| alcabrera | :) | 19:23 |
| vkmc | yay thanks | 19:23 |
| alcabrera | np~ | 19:23 |
| alcabrera | I'm going to head home for the day. | 19:24 |
| *** jdprax has joined #openstack-marconi | 19:24 | |
| vkmc | alcabrera, ttfn, enjoy the rest of your day! o/ | 19:24 |
| alcabrera | thanks, and you too, vkmc! | 19:25 |
| *** alcabrera is now known as alcabrera|afk | 19:25 | |
| flaper87 | vkmc: please, lets discuss the details of the amqp driver. it'll probably require changing some bits of the API too. | 19:25 |
| vkmc | flaper87, sure | 19:26 |
| flaper87 | I'd recommend reading those docs so we can discuss it further in the next couple of days | 19:26 |
| *** tjanczuk has left #openstack-marconi | 19:26 | |
| flaper87 | I need to write down the implementations details | 19:26 |
| *** tjanczuk_ has joined #openstack-marconi | 19:26 | |
| flaper87 | I have put thoughts on that | 19:26 |
| flaper87 | but I gtg now, for real this time :P | 19:27 |
| flaper87 | bbib | 19:27 |
| *** tjanczuk_ has left #openstack-marconi | 19:27 | |
| *** jdprax has quit IRC | 19:27 | |
| *** whenry has quit IRC | 19:28 | |
| vkmc | flaper87, yeah that seems better to me too :) | 19:29 |
| vkmc | flaper87, I'll read the docs in the meantime | 19:29 |
| vkmc | flaper87, thanks for the pointers :) ttfn! o/ | 19:30 |
| *** tjanczuk_ has joined #openstack-marconi | 19:30 | |
| *** kgriffs|afk is now known as kgriffs | 19:35 | |
| *** whenry has joined #openstack-marconi | 19:43 | |
| *** tjanczuk_ has left #openstack-marconi | 19:43 | |
| *** tjanczuk has joined #openstack-marconi | 19:43 | |
| *** tjanczuk has left #openstack-marconi | 19:44 | |
| cpallares | vkmc: would you happen to know what command to use when tox gives you gibberish? | 19:49 |
| cpallares | I should have asked flaper87 before he left | 19:49 |
| kgriffs | what do you mean by "gibberish"? | 19:50 |
| * flaper87 back | 19:54 | |
| flaper87 | cpallares: you probably want $ nosetest tests.unit.queues..... | 19:55 |
| cpallares | kgriffs: like has a bunch of nonsense text like --> \xde\xcd\xe1\xa9\xb3)\x01@ | 19:56 |
| vkmc | cpallares, can you paste the output? | 19:56 |
| vkmc | oh :) | 19:56 |
| cpallares | thanks flaper87! | 19:56 |
| * cpallares writes it down | 19:56 | |
| vkmc | I'm not familiar with tests yet... this is useful https://github.com/openstack/marconi/blob/master/tests/functional/README.rst | 19:57 |
| *** oz_akan_ has quit IRC | 19:57 | |
| malini | tht readme needs some love :( | 19:58 |
| vkmc | didn't we run the tests with testr though? I added that to the wiki, hope it's not wrong | 19:58 |
| flaper87 | vkmc: we use testr | 19:58 |
| vkmc | flaper87, but it is for unit tests right? | 19:59 |
| flaper87 | vkmc: everything | 19:59 |
| flaper87 | vkmc: functional and unit tests | 19:59 |
| flaper87 | we use tsung for benchmarks, though. | 20:00 |
| *** whenry has quit IRC | 20:00 | |
| *** tjanczuk_ has joined #openstack-marconi | 20:01 | |
| *** tjanczuk_ has left #openstack-marconi | 20:01 | |
| *** tjanczuk_ has joined #openstack-marconi | 20:02 | |
| *** oz_akan_ has joined #openstack-marconi | 20:02 | |
| vkmc | flaper87, good to know :) | 20:03 |
| *** balajiiyer1 has joined #openstack-marconi | 20:06 | |
| *** kgriffs is now known as kgriffs|afk | 20:06 | |
| *** balajiiyer has quit IRC | 20:06 | |
| *** balajiiyer1 has quit IRC | 20:07 | |
| *** balajiiyer has joined #openstack-marconi | 20:07 | |
| balajiiyer | abettadapur: can you take care of this bp ? https://blueprints.launchpad.net/marconi/+spec/api-v1.1-header-changes | 20:09 |
| abettadapur | balajiiyer: looking at it now | 20:09 |
| *** whenry has joined #openstack-marconi | 20:12 | |
| *** whenry has quit IRC | 20:15 | |
| *** AAzza has left #openstack-marconi | 20:20 | |
| *** whenry has joined #openstack-marconi | 20:26 | |
| *** abettadapur has quit IRC | 20:32 | |
| *** martyntaylor has left #openstack-marconi | 20:32 | |
| vkmc | malini, I'm facing another problem with Postman, not sure what is missing this time http://paste.openstack.org/show/81019/ | 20:39 |
| vkmc | oh,.. it's not on the paste, Postman is getting a 204 | 20:40 |
| *** sriram has quit IRC | 20:42 | |
| vkmc | maybe somebody else can give me a hint on that :) I'm trying to use Postman to interact with the API | 20:43 |
| *** shakamunyi has joined #openstack-marconi | 20:46 | |
| *** sriram_p has quit IRC | 20:47 | |
| *** LaceyKite has quit IRC | 20:48 | |
| *** tedross has quit IRC | 20:54 | |
| *** jraim has quit IRC | 21:04 | |
| *** jraim has joined #openstack-marconi | 21:05 | |
| *** oz_akan_ has quit IRC | 21:07 | |
| *** whenry has quit IRC | 21:12 | |
| *** mwagner_lap has quit IRC | 21:19 | |
| *** Obulpathi has quit IRC | 21:27 | |
| balajiiyer | vkmc can you try removing additional headers in postman, perhaps? X-Project-ID | 21:40 |
| vkmc | I'll try that, thanks balajiiyer | 21:46 |
| *** balajiiyer has left #openstack-marconi | 21:48 | |
| *** balajiiyer has joined #openstack-marconi | 21:49 | |
| *** jergerber has joined #openstack-marconi | 21:49 | |
| flaper87 | vkmc: btw, if you've any questions please, feel free to ask at any time, drop emails etc. | 21:54 |
| *** balajiiyer has left #openstack-marconi | 21:55 | |
| vkmc | flaper87, will do, thanks for that! :) | 21:55 |
| *** mpanetta has quit IRC | 22:02 | |
| *** Obulpathi has joined #openstack-marconi | 22:13 | |
| *** mwagner_lap has joined #openstack-marconi | 22:26 | |
| *** Obulpathi has quit IRC | 22:30 | |
| *** jchai has quit IRC | 22:51 | |
| *** flaper87 is now known as flaper87|afk | 23:15 | |
| *** malini is now known as malini_afk | 23:43 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!