| *** reed has quit IRC | 00:11 | |
| *** whenry has joined #openstack-marconi | 00:44 | |
| *** amitgandhi has joined #openstack-marconi | 00:46 | |
| *** cpallares has quit IRC | 00:46 | |
| *** haomaiwang has joined #openstack-marconi | 01:10 | |
| *** haomaiw__ has joined #openstack-marconi | 01:12 | |
| *** flwang has quit IRC | 01:13 | |
| *** haomaiwang has quit IRC | 01:15 | |
| *** nosnos has joined #openstack-marconi | 01:33 | |
| *** amitgandhi has quit IRC | 01:51 | |
| *** amitgandhi has joined #openstack-marconi | 01:53 | |
| *** reed has joined #openstack-marconi | 02:20 | |
| *** whenry has quit IRC | 02:22 | |
| *** amitgandhi has quit IRC | 02:23 | |
| *** reed has quit IRC | 02:25 | |
| *** rwsu has quit IRC | 02:52 | |
| openstackgerrit | Victoria MartÃnez de la Cruz proposed a change to openstack/marconi: Refactor auxiliary methods for MongoDB unit tests https://review.openstack.org/98916 | 03:09 |
|---|---|---|
| *** vkmc has quit IRC | 03:15 | |
| *** haomaiw__ has quit IRC | 03:27 | |
| *** haomaiwang has joined #openstack-marconi | 03:27 | |
| *** nosnos has quit IRC | 03:43 | |
| *** whenry has joined #openstack-marconi | 04:06 | |
| *** nosnos has joined #openstack-marconi | 04:24 | |
| *** whenry has quit IRC | 04:41 | |
| *** whenry has joined #openstack-marconi | 04:57 | |
| *** whenry has quit IRC | 06:13 | |
| *** mkoderer has joined #openstack-marconi | 06:21 | |
| *** jamie_h has joined #openstack-marconi | 06:26 | |
| *** whenry has joined #openstack-marconi | 06:39 | |
| *** nosnos has quit IRC | 06:59 | |
| *** whenry has quit IRC | 07:04 | |
| *** nosnos has joined #openstack-marconi | 07:05 | |
| *** flaper87|afk is now known as flaper87 | 07:06 | |
| *** AAzza is now known as AAzza_afk | 07:27 | |
| *** shivamshukla has quit IRC | 07:31 | |
| *** mwagner_lap has quit IRC | 07:34 | |
| *** mwagner_lap has joined #openstack-marconi | 07:36 | |
| *** ykaplan has joined #openstack-marconi | 08:04 | |
| *** nosnos has quit IRC | 08:40 | |
| *** nosnos has joined #openstack-marconi | 08:43 | |
| *** nosnos has quit IRC | 08:44 | |
| *** nosnos has joined #openstack-marconi | 09:03 | |
| *** ykaplan has quit IRC | 09:39 | |
| *** AAzza_afk has quit IRC | 09:48 | |
| *** AAzza_afk has joined #openstack-marconi | 09:48 | |
| *** AAzza_afk is now known as AAzza | 09:48 | |
| *** jamie_h has quit IRC | 10:17 | |
| *** jamie_h has joined #openstack-marconi | 10:18 | |
| *** jamie_h_ has joined #openstack-marconi | 10:29 | |
| *** jamie_h has quit IRC | 10:32 | |
| *** ykaplan has joined #openstack-marconi | 10:34 | |
| *** ekarlso has quit IRC | 10:57 | |
| *** ekarlso has joined #openstack-marconi | 10:58 | |
| *** mwagner_lap has quit IRC | 12:26 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/marconi: Updated from global requirements https://review.openstack.org/99033 | 12:28 |
| *** sriram has joined #openstack-marconi | 12:37 | |
| *** sriram has quit IRC | 12:39 | |
| *** sriram has joined #openstack-marconi | 12:39 | |
| *** ykaplan has quit IRC | 12:56 | |
| sriram | Good Morning! :) | 13:05 |
| flaper87 | sriram: yo | 13:06 |
| sriram | \m/ | 13:06 |
| *** amitgandhi has joined #openstack-marconi | 13:24 | |
| *** Obulpathi has joined #openstack-marconi | 13:32 | |
| *** Obulpathi has quit IRC | 13:34 | |
| *** Obulpathi has joined #openstack-marconi | 13:35 | |
| *** malini1 has joined #openstack-marconi | 13:41 | |
| *** amitgandhi has quit IRC | 13:47 | |
| *** nosnos has quit IRC | 13:50 | |
| *** balajiiyer has joined #openstack-marconi | 14:08 | |
| *** prashanthr_ has joined #openstack-marconi | 14:15 | |
| *** amitgandhi has joined #openstack-marconi | 14:19 | |
| *** amitgandhi has quit IRC | 14:19 | |
| *** ykaplan has joined #openstack-marconi | 14:20 | |
| *** amitgandhi has joined #openstack-marconi | 14:20 | |
| sriram | hey all, we need some eyes on https://review.openstack.org/#/c/97534/ :) | 14:23 |
| sriram | flaper87, malini1 : ^ | 14:24 |
| *** mwagner_lap has joined #openstack-marconi | 14:25 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/marconi: Updated from global requirements https://review.openstack.org/99033 | 14:35 |
| *** kgriffs|afk is now known as kgriffs | 14:38 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/python-marconiclient: Updated from global requirements https://review.openstack.org/99093 | 14:39 |
| *** reed has joined #openstack-marconi | 14:54 | |
| *** alcabrera|afk is now known as alcabrera | 14:56 | |
| flaper87 | meeeeeeeeeeeeeeeeeeeeeeeeeeeeeting | 14:57 |
| malini1 | #openstack-meeting-alt | 14:58 |
| malini1 | tht is a single click meeting | 14:58 |
| kgriffs | be there or be square | 14:59 |
| openstackgerrit | Alex Bettadapur proposed a change to openstack/marconi: Decoupled Unit Tests https://review.openstack.org/97534 | 14:59 |
| *** abettadapur has joined #openstack-marconi | 15:00 | |
| openstackgerrit | Alex Bettadapur proposed a change to openstack/marconi: Re-enable tests affected by #1321840 https://review.openstack.org/99117 | 15:03 |
| abettadapur | malini: ^ | 15:04 |
| malini1 | I'll +2 to it as soon as it gets the jenkins blessing | 15:05 |
| malini1 | thanks abettadapur | 15:05 |
| *** vkmc has joined #openstack-marconi | 15:06 | |
| *** vkmc has quit IRC | 15:06 | |
| *** vkmc has joined #openstack-marconi | 15:06 | |
| *** abettadapur has quit IRC | 15:08 | |
| *** peoplemerge1 has joined #openstack-marconi | 15:13 | |
| *** peoplemerge1 has quit IRC | 15:14 | |
| *** AAzza is now known as AAzza_afk | 15:18 | |
| *** vkmc has quit IRC | 15:21 | |
| *** ykaplan has quit IRC | 15:22 | |
| *** vkmc has joined #openstack-marconi | 15:29 | |
| openstackgerrit | Alex Bettadapur proposed a change to openstack/marconi: V1.1 Functional Tests https://review.openstack.org/93295 | 15:33 |
| *** cpallares has joined #openstack-marconi | 15:36 | |
| openstackgerrit | Alex Bettadapur proposed a change to openstack/marconi: V1.1 Functional Tests https://review.openstack.org/93295 | 15:48 |
| flaper87 | I'll be back in a bit | 16:01 |
| *** AAzza_afk is now known as AAzza | 16:02 | |
| tjanczuk_ | I could not agree more with kgriff's last comment about API being the product. That is exactly why the shape of the API requires most discussion. | 16:03 |
| vkmc | flaper87, sorry for the delayed response, I'm having problems with my Internet connection | 16:03 |
| vkmc | too much lag -.- | 16:03 |
| vkmc | kgriffs, as you mentioned during the meeting, we have to discuss which are the benefits of supporting AMQP since it's semantics is quite difficult to map to Marconi | 16:08 |
| vkmc | s/is quite/are quite | 16:09 |
| tjanczuk_ | vkmc: so what problems did you run into with AMQP 1.0? Let's compare notes with 0.9. | 16:12 |
| vkmc | tjanczuk_, first of all, the queues management are not possible to map with current API | 16:14 |
| *** cath6184 has joined #openstack-marconi | 16:14 | |
| vkmc | I think it's the same for AMQP 0.9, right? | 16:14 |
| tjanczuk_ | vkmc: on 0.9 you can create, check for existence, and delete a queue. But listing queues is not possible. However, with Rabbit you can call into HTTP based management endpoint to list queues. | 16:15 |
| vkmc | tjanczuk_, well there is more control in AMQP 0.9 over queues, in AMQP 1.0 only creation is available | 16:16 |
| vkmc | I should check out if there is a way to delete (at least) without depending on the Proton library | 16:17 |
| kgriffs | vkmc, tjanczuk_: I am assuming it is not possible to page through messages given a marker? | 16:18 |
| tjanczuk_ | No, paging is not possible. Strict queue semantics applies. | 16:19 |
| vkmc | kgriffs, same goes for AMQP 1.0 ^^ | 16:19 |
| vkmc | also, the notion of Id present in AMQP 1.0 is not the same as the one in Marconi | 16:20 |
| tjanczuk_ | One other key problem with AMQP 0.9 is that messages can only be acknowledged through the same AMQP channel they were originally received on. That means the "claim creation" and "message deletion" requests must have server affinity. Is that a problem with 1.0 as well? | 16:20 |
| vkmc | Ids in AMQP are intended to guarantee uniqueness, so we cannot get messages by id or delete messages by id as we do in Marconi | 16:21 |
| kgriffs | vkmc: oh, so an ID is not used for requests, it is just for the clients to distinguish copies of the same message? | 16:22 |
| vkmc | kgriffs, exactly | 16:22 |
| tjanczuk_ | Message IDs are also a non-concept in AMQP 0.9. The only meaningul IDs are the delivery tags roughly corresponding to claim IDs in Marconi. These are transient IDs issued at the time message is consumed that can be used to permanently delete it. Similar to SQS, and Azure. | 16:22 |
| vkmc | no tjanczuk_, in AMQP 1.0 acks can be sent over a new connection | 16:22 |
| tjanczuk_ | vkmc, great, so AMQP 1.0 fixed that problem. | 16:23 |
| vkmc | yeah :) | 16:24 |
| tjanczuk_ | vkmc: how do you identify the message you want to ack in AMQP 1.0? | 16:24 |
| vkmc | tjanczuk_, messages have an additional property, reply-to, which is later used by the consumer to send the ack | 16:26 |
| tjanczuk_ | Is there a concept of "ack all messages I received until a particular reply-to"? | 16:27 |
| tjanczuk_ | sort of batch ack? | 16:27 |
| vkmc | in conjunction with the correlation-id, another field in AMQP 1.0 message | 16:27 |
| tjanczuk_ | So how does that model map onto the Marconi's claim model which maps 1 claim ID to multiple message IDs? | 16:28 |
| *** abettadapur has joined #openstack-marconi | 16:29 | |
| vkmc | well that's something up to the application using Proton... as far as I know there is no such concept in the lib | 16:30 |
| vkmc | tjanczuk_, the correlation-id is a client-specific id that can be used for claims operations | 16:31 |
| tjanczuk_ | well, *you* are writing that application. How would you map the 1-N Marconi model onto AMQP 1.0? I am asking because it does not map to AMQP 0.9 too well. Basically the "create claim" implementation with 0.9 would have to always return a single message to normalize the claim-message relationship to 1-1. | 16:32 |
| tjanczuk_ | (Which, needless to say, would diminish performance) | 16:32 |
| vkmc | the main problem with claims is that we require two ids, the message-id and the claim-id | 16:34 |
| vkmc | in AMQP 1.0 we don't have a direct way to get messages by id | 16:34 |
| tjanczuk_ | Yes, this is also my observation. Would it help AMQP 1.0 if claimIDs were associated with individual messages and created at the time they are consumed? This is the SQS and Azure model, and maps very well to AMQP 0.9. | 16:36 |
| vkmc | currently I cannot come up with an algorithm to tackle this because AMQP follows, as you mentioned, an strict queue sematic | 16:36 |
| vkmc | it happens the same as with AMQP 0.9, but I'm looking into new attributes in AMQP messages to maybe find a decent solution | 16:37 |
| vkmc | that would help yeah | 16:37 |
| tjanczuk_ | Any other take-aways from AMQP 1.0? | 16:39 |
| vkmc | I'll get in touch with Proton devs before the end of this week, maybe they will be able to clear some of this doubts | 16:40 |
| vkmc | tjanczuk_, no, just what we discussed so far... unfortunately these conform the core functionality of Marconi | 16:41 |
| kgriffs | I just want to thank you both for your work on this so far. It is yielding a ton of good info! | 16:42 |
| tjanczuk_ | After I got blocked implementing Rabbit driver under current APIs, I took the liberty to experiment with what a "core" set of HTTP APIs that reflect messaging semantics could look like. | 16:43 |
| * flaper87 back | 16:43 | |
| vkmc | thanks kgriffs | 16:44 |
| flaper87 | re kafka http://www.infoq.com/articles/apache-kafka | 16:44 |
| vkmc | lunch brb | 16:45 |
| tjanczuk_ | I was trying to take a miminalist approach that allows a performant implementation with NoAQL as well as AMQP. It also adds WebSockets. A prototype is at https://github.com/tjanczuk/narconi. The protocol has only 6 endpoints. I am curious about your thoughts? | 16:45 |
| tjanczuk_ | vkmc: in particular, would this ^^^ HTTP API facilitate AMQP 1.0 implementation? I know it does AMQP 0.9 because I have it working. | 16:46 |
| *** mwagner_lap has quit IRC | 16:48 | |
| vkmc | thanks tjanczuk_, I'll take a look :) | 16:51 |
| *** cath6184 has quit IRC | 17:03 | |
| *** balajiiyer has quit IRC | 17:03 | |
| kgriffs | btw, everyone, please please please spend some time today reviewing our patch backlog. We need to cut the j-2 milestone tomorrow | 17:13 |
| kgriffs | see also: | 17:13 |
| kgriffs | https://launchpad.net/marconi/+milestone/juno-1 | 17:13 |
| kgriffs | and: | 17:13 |
| kgriffs | https://review.openstack.org/#/q/status:open+project:openstack/marconi,n,z | 17:14 |
| kgriffs | thanks in advance! | 17:14 |
| kgriffs | malini1: ping | 17:16 |
| *** balajiiyer has joined #openstack-marconi | 17:19 | |
| sriram | kgriffs: lazy queue create requires refactored unit tests. -> https://review.openstack.org/#/c/97534/ | 17:22 |
| sriram | do we want to put that as well in j-1? | 17:22 |
| kgriffs | yes, I am reviewing it now | 17:22 |
| sriram | thanks! I'll review as well. | 17:23 |
| openstackgerrit | Alex Bettadapur proposed a change to openstack/marconi: V1.1 Functional Tests https://review.openstack.org/93295 | 17:27 |
| *** prashanthr_ has quit IRC | 17:28 | |
| *** tonytan4ever has joined #openstack-marconi | 17:31 | |
| *** tonytan4ever has quit IRC | 17:33 | |
| *** tonytan4ever has joined #openstack-marconi | 17:36 | |
| kgriffs | abettadapur: I added some comments to https://review.openstack.org/#/c/97534/ | 17:38 |
| malini1 | kgriffs: pong | 17:41 |
| *** haomaiwang has quit IRC | 17:52 | |
| *** rwsu has joined #openstack-marconi | 18:02 | |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/marconi: Implement POP in v1.1 API https://review.openstack.org/90202 | 18:04 |
| *** rossk has joined #openstack-marconi | 18:04 | |
| kgriffs | malini1: re "check queue exists" | 18:04 |
| kgriffs | in v1.1 | 18:04 |
| kgriffs | i think we agreed to remove it | 18:05 |
| kgriffs | would you mind taking care of the paper work? | 18:05 |
| kgriffs | meaning, update the v1.1 spec (remove that section, also remove from the summary list of endpoints, and add a note to the breaking changes list) | 18:05 |
| malini1 | kgriffs: sure | 18:06 |
| kgriffs | finally, register a blueprint and make the umbrella v1.1 spec depend on it | 18:06 |
| malini1 | on it now | 18:06 |
| kgriffs | the bp should be scheduled for j-2 with high priority | 18:06 |
| malini1 | ok | 18:06 |
| kgriffs | thanks! | 18:06 |
| malini1 | yw | 18:06 |
| abettadapur | @kgriffs: thanks :) | 18:11 |
| abettadapur | kgriffs: with regards to v1_1/test_claims.py:test_lifecyle, is there a good reason for this to be such a large test? | 18:12 |
| abettadapur | https://review.openstack.org/#/c/97534/5/marconi/tests/queues/transport/wsgi/v1_1/test_claims.py,unified | 18:13 |
| abettadapur | kgriffs: ^ | 18:13 |
| kgriffs | abettadapur: i suppose we wanted to test an end-to-end scenario from start to finish - post a message, read it, claim it, delete it | 18:13 |
| kgriffs | FWIW everyone, I just added some more content to our FAQ: https://wiki.openstack.org/wiki/Frequently_Asked_Questions_(Marconi) | 18:14 |
| kgriffs | you can point people there when they have questions on the ML or IRC if you get tired of repeating yourself | 18:14 |
| kgriffs | :) | 18:14 |
| kgriffs | malini1, flaper87: ^^^ | 18:14 |
| flaper87 | kgriffs: awesome, thanks | 18:15 |
| flaper87 | kgriffs: btw, can we get rid of the (Marconi) ? | 18:15 |
| flaper87 | Why isn't that page under wiki/Marconi/ ? | 18:15 |
| kgriffs | short answer: mediawiki sucks | 18:15 |
| flaper87 | T_T | 18:16 |
| flaper87 | link parsers don't like () :P | 18:16 |
| kgriffs | mediawiki was designed with the tenant in mind that the world is flat | 18:16 |
| flaper87 | kgriffs: waaaaaaaaaaaaait a minute, isn't it? | 18:16 |
| kgriffs | well, you know | 18:16 |
| kgriffs | kindasorta | 18:16 |
| kgriffs | :p | 18:16 |
| flaper87 | LOL | 18:16 |
| kgriffs | anyway, i spent a lot of time fighting MediaWiki to get it to display nice page titles when you use a hierarchical page scheme | 18:17 |
| kgriffs | also looked at some other project's pages | 18:17 |
| kgriffs | for the most part, other projects seem to keep most of their stuff on a single page | 18:17 |
| kgriffs | so that didn't give me any clues | 18:17 |
| kgriffs | then I dug into the bowels of the MediWiki docs | 18:18 |
| *** AAzza is now known as AAzza_afk | 18:18 | |
| kgriffs | I found some special markup that might allow you to customize page titles | 18:18 |
| kgriffs | but it didn't work in OpenStack's deployment | 18:18 |
| kgriffs | so... | 18:19 |
| kgriffs | the only way to have nice page titles is to have flat titles | 18:19 |
| kgriffs | sorry, I mean flat URL page names | 18:19 |
| flaper87 | erm, not worth wasting our time there | 18:19 |
| kgriffs | otherwise your titles end up like "Marconi/Foo/Bar Bang thing" | 18:19 |
| kgriffs | anyway, I decided to make peace with MediaWiki and do it the "WikiPedia" way | 18:20 |
| kgriffs | personally, I still think migrating to MediaWiki was a bad decision, but I rest my case. | 18:20 |
| flaper87 | I don't even remember what were we using before media wiki | 18:21 |
| flaper87 | was it just LP ? | 18:21 |
| kgriffs | no, it was some python-based thing. | 18:22 |
| * kgriffs tries to remember | 18:22 | |
| openstackgerrit | Sriram Madapusi Vasudevan proposed a change to openstack/marconi: Implement Lazy Create Queue in v1.1 API https://review.openstack.org/91804 | 18:22 |
| flaper87 | moin moin ? | 18:22 |
| flaper87 | I don't remember | 18:22 |
| kgriffs | that sounds right | 18:22 |
| malini1 | kgriffs: I found this one from the sahara team https://blueprints.launchpad.net/sahara/+spec/savanna-renaming | 18:26 |
| malini1 | hope they have done all the homework for us :) | 18:27 |
| flaper87 | malini1: savanna is now sahara | 18:27 |
| malini1 | flaper87: yeah..this one is the work they did for renaming | 18:27 |
| flaper87 | oh cool | 18:27 |
| * flaper87 steals ideas | 18:27 | |
| malini1 | we might have to just follow the same steps | 18:27 |
| malini1 | flaper87: wait..tht is my expertise ;) | 18:28 |
| * flaper87 bows and steps away | 18:28 | |
| malini1 | :D | 18:28 |
| *** tonytan4ever has quit IRC | 18:29 | |
| kgriffs | good find | 18:32 |
| kgriffs | i gotta run for a bit | 18:32 |
| kgriffs | malini1, flaper87: please please help review our patch backlog. I will be doing that myself when I get back. I'd like to get as much as possible in j-1 | 18:32 |
| kgriffs | (otherwise, things have a tendency for going on forever - student syndrome :p) | 18:33 |
| kgriffs | flaper87, malini1, alcabrera, sriram: also this is blocking our release of the client: https://review.openstack.org/#/c/98901/ | 18:33 |
| kgriffs | ttfn | 18:34 |
| *** kgriffs is now known as kgriffs|afk | 18:35 | |
| vkmc | tum tum tum | 18:36 |
| vkmc | tjanczuk_, your implementation looks great | 18:40 |
| flaper87 | kgriffs|afk: +@ | 18:42 |
| flaper87 | kgriffs|afk: +2 | 18:42 |
| *** balajiiyer has quit IRC | 18:48 | |
| *** tonytan4ever has joined #openstack-marconi | 18:48 | |
| *** balajiiyer has joined #openstack-marconi | 18:49 | |
| openstackgerrit | A change was merged to openstack/python-marconiclient: chore: Bump version to 0.0.2 in preparation for bugfix release https://review.openstack.org/98901 | 18:51 |
| vkmc | kgriffs|afk, flaper87 when the poc for AMQP should be ready? | 18:52 |
| malini1 | flaper87: how do I handle config options tht have been renamed in v1.1 . for the pop stuff I renamed max_messages_per_claim, & the v1 functional tests are failing because of tht | 19:01 |
| malini1 | I tried adding the deprecated_name option, tht doesnt help :( | 19:02 |
| malini1 | See https://review.openstack.org/#/c/90202/5/marconi/queues/transport/validation.py | 19:03 |
| sriram | it should be deprecated_opts i think | 19:03 |
| malini1 | thanks sriram! I'll give tht a try | 19:07 |
| *** jay-atl has quit IRC | 19:14 | |
| *** mpanetta has joined #openstack-marconi | 19:26 | |
| *** malini2 has joined #openstack-marconi | 19:34 | |
| *** malini2 has quit IRC | 19:34 | |
| *** mpanetta has quit IRC | 19:35 | |
| *** malini1 has quit IRC | 19:37 | |
| tjanczuk_ | vkmc: thanks! What issues would you see in supporting AMQP 1.0 behind these HTTP APIs? | 19:50 |
| *** vkmc has quit IRC | 20:04 | |
| *** amitgandhi has quit IRC | 20:05 | |
| *** vkmc has joined #openstack-marconi | 20:08 | |
| *** amitgandhi has joined #openstack-marconi | 20:10 | |
| *** vkmc has quit IRC | 20:18 | |
| *** jamie_h has joined #openstack-marconi | 20:23 | |
| *** jamie_h_ has quit IRC | 20:25 | |
| *** mpanetta has joined #openstack-marconi | 20:31 | |
| *** sriram has quit IRC | 20:32 | |
| *** tmu has quit IRC | 20:33 | |
| *** tmu has joined #openstack-marconi | 20:33 | |
| *** abettadapur has quit IRC | 20:33 | |
| *** tmu_ has joined #openstack-marconi | 20:39 | |
| *** mpanetta has quit IRC | 20:39 | |
| *** balajiiyer has left #openstack-marconi | 20:44 | |
| *** kgriffs|afk is now known as kgriffs | 20:46 | |
| *** vkmc has joined #openstack-marconi | 20:51 | |
| kgriffs | malini: that's odd about deprecated_name | 20:53 |
| kgriffs | deprecated_name was used in marconi/marconi/queues/transport/validation.py | 20:54 |
| *** tmu has quit IRC | 20:54 | |
| *** jamie_h has quit IRC | 20:54 | |
| *** Obulpathi has quit IRC | 20:54 | |
| *** seiflotfy has quit IRC | 20:54 | |
| *** openstackgerrit has quit IRC | 20:54 | |
| *** alcabrera is now known as alcabrera|afk | 20:54 | |
| *** seiflotfy has joined #openstack-marconi | 20:58 | |
| flaper87 | malini: deprecated_name and deprecated_opts should work | 21:06 |
| *** mwagner_lap has joined #openstack-marconi | 21:06 | |
| flaper87 | vkmc: mmh, before the feature freeze for sure, it'd be nice to have it by the end of J-2 | 21:07 |
| vkmc | flaper87, ok, I'm going all in :) | 21:08 |
| *** Obulpathi has joined #openstack-marconi | 21:08 | |
| *** openstackgerrit has joined #openstack-marconi | 21:08 | |
| kgriffs | my question is how much do we need to implement before we can decide what to do with our API - if/whether it makes sense to do backends that don't support message feed part of the API? | 21:09 |
| kgriffs | I guess to settle the performance question we need redis driver with pooling as well as a broker like qpid or kafka, right? | 21:10 |
| kgriffs | unfortunately, that will require implementing most of the drivers - but perhaps cheating on stuff like dealing with adding/removing nodes (in the case of redis) - we may be able to make some simplifying assumptions for AMQP as well | 21:11 |
| vkmc | kgriffs, IMHO it's important for Naav to *try* adding support for widely used technologies as AMQP | 21:14 |
| vkmc | kgriffs, but it doesnt' make much sense to change every detail in our API just to add support for some backend | 21:14 |
| vkmc | kgriffs, I'm trying to simplify as much as possible the implementation by learning what are the core functionalities expected from AMQP users and mapping it with Naav | 21:16 |
| kgriffs | vkmc: I just want to make sure we understand what we are getting ourselves into! We are doing some rather involved science projects right now, but we aren't sure how they will turn out | 21:16 |
| flaper87 | kgriffs: I think we'll know when it's enough. The fact that we have some doubts already means that it won't be long before we make our final decision | 21:16 |
| vkmc | yeah of course :/ | 21:16 |
| vkmc | +1 flaper87 | 21:17 |
| flaper87 | another question to ask is: If we find out AMQP is not the way to go, would we still have time to support another backend? | 21:17 |
| kgriffs | also, I just want to try and get these drivers to the point that we can make reasonable guesses as to their long-term viability without having to go all the way to a production-ready implementation | 21:17 |
| flaper87 | That question is raised by the fact that redis is not a drop-in replacement for mongodb | 21:17 |
| flaper87 | which means we would get to the end of J with still just mongodb | 21:17 |
| kgriffs | hmm | 21:17 |
| *** cpallares has quit IRC | 21:18 | |
| flaper87 | I don't believe that's something we should do | 21:18 |
| kgriffs | yeah, that makes me nervous. I think if this pans out, we would have something comparable to mongo: https://blueprints.launchpad.net/marconi/+spec/redis-pool | 21:18 |
| kgriffs | well, we have redis, AMQP 1.0 and Kafka as the forerunners, right? | 21:19 |
| flaper87 | kgriffs: right, although I was digging more into kafka now and the driver would still require some changes in the API | 21:20 |
| kgriffs | bleh | 21:20 |
| flaper87 | but don't take my word on that | 21:20 |
| flaper87 | I need to investigate more | 21:20 |
| * flaper87 speaks as if people usually take his word | 21:20 | |
| flaper87 | also, I really don't like the fact it basically depends on zookeeper | 21:22 |
| kgriffs | flaper87: btw, we need to cut j-1 tomorrow. We got behind on our reviews and so we may have to push most of the patches into j-2 | 21:22 |
| kgriffs | but, I am trying to get things reviewed as quickly as possible. Let's keep reminding people to catch up on the backlog. I want to minimize how much of j-2 is taken up by j-1 slips | 21:23 |
| flaper87 | damn, ok! TBH, J-1 happened too soon, I didn't remember it was this week | 21:23 |
| flaper87 | kgriffs: I'll do a bunch of reviews tomorrow | 21:23 |
| flaper87 | kgriffs: ignore my /shard/pool/ patch and the flavor ones | 21:23 |
| kgriffs | flaper87: the first milestone always seems like a rush since it takes time to deal with the fallout from the summit | 21:23 |
| flaper87 | those can be reviewed for J-2 | 21:23 |
| kgriffs | kk | 21:23 |
| flaper87 | kgriffs: yeah | 21:24 |
| kgriffs | #link https://launchpad.net/marconi/+milestone/juno-1 | 21:24 |
| kgriffs | thanks! | 21:24 |
| flaper87 | lets move the health one | 21:24 |
| flaper87 | and the redis storage | 21:25 |
| kgriffs | flaper87: that bp was just to have a POC - didn't need to be merged into the main repo (indeed, I think all these new drivers should exist externally until we sort out which ones are going to be viable) | 21:26 |
| kgriffs | (re redis) | 21:26 |
| kgriffs | and health, yeah | 21:27 |
| kgriffs | definitely | 21:27 |
| flaper87 | kgriffs: so based on my readings, kafka API is based on streams | 21:27 |
| kgriffs | flwang popped in a couple days ago then disapeared again. :( | 21:27 |
| flaper87 | no way to claim, access messages etc | 21:27 |
| kgriffs | ok, I give up | 21:28 |
| kgriffs | let's just create a broker provisioning service | 21:28 |
| * kgriffs runs away | 21:28 | |
| flaper87 | HAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHAHAHAHAHAHAHA | 21:28 |
| flaper87 | (ROFL) | 21:28 |
| flaper87 | (ROFL) | 21:28 |
| flaper87 | (ROFL) | 21:28 |
| flaper87 | (ROFL) | 21:28 |
| flaper87 | (ROFL) | 21:28 |
| flaper87 | (ROFL) | 21:28 |
| flaper87 | (ROFL) | 21:28 |
| * flaper87 is still loughing | 21:29 | |
| * kgriffs had no idea a person could roll around on the floor like that. wow. | 21:29 | |
| * kgriffs starts recording animated gif | 21:29 | |
| flaper87 | n a more serious note, we probably need to have specific stream endpoints that will support such stores | 21:29 |
| flaper87 | LOL yeah, you have no idea | 21:29 |
| flaper87 | until those endpoint are defined, we should focus on stores like elasticsearch, swift etc | 21:30 |
| kgriffs | at some point I think you just have to say, that's a different project. | 21:30 |
| flaper87 | yeah | 21:30 |
| flaper87 | lets put more thoughts on this | 21:30 |
| kgriffs | hmm. | 21:30 |
| kgriffs | elasticsearch | 21:30 |
| kgriffs | eeeeenteresting | 21:30 |
| flaper87 | but I think we're getting more and more information leading us to the right path | 21:31 |
| flaper87 | too bad we still don't know what that path is | 21:31 |
| flaper87 | :P | 21:31 |
| flaper87 | kk, gtg now. I'll probably be back in a bit | 21:31 |
| tjanczuk_ | For whatever it is worth, I looked at Kafka in a different context in the past. Based on that I think implementing a Kafka driver for Naav is going to more challanging than AMQP. | 21:31 |
| kgriffs | interesting idea for big features from openstack-meeting-alt | 21:31 |
| kgriffs | "Build a chain of patches, with the first one having a -2" | 21:32 |
| kgriffs | "Once all the rest of the chain is approved, remove that first -2" | 21:32 |
| *** flaper87 is now known as flaper87|afk | 21:32 | |
| kgriffs | "They merge as a block " | 21:32 |
| kgriffs | </random> | 21:32 |
| kgriffs | tjanczuk_: gtk | 21:32 |
| *** Obulpathi has quit IRC | 21:32 | |
| tjanczuk_ | At the same time, Kafka has a number of properties that would make it a rather good fit for the "feed" part of the API. | 21:33 |
| tjanczuk_ | As it allows "replays". | 21:34 |
| *** mpanetta has joined #openstack-marconi | 21:36 | |
| *** mpanetta has quit IRC | 21:40 | |
| *** amitgandhi has quit IRC | 21:45 | |
| *** tonytan4ever has quit IRC | 21:53 | |
| *** sriram has joined #openstack-marconi | 22:01 | |
| *** rossk has quit IRC | 22:18 | |
| *** sriram has quit IRC | 22:18 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/python-marconiclient: doc: Update README.rst with correct installation and contibution info https://review.openstack.org/99224 | 22:24 |
| kgriffs | could use some reviews: ^^^ | 22:25 |
| *** vkmc_ has joined #openstack-marconi | 22:39 | |
| *** vkmc has quit IRC | 22:43 | |
| *** reed_ has joined #openstack-marconi | 23:22 | |
| *** reed has quit IRC | 23:22 | |
| *** reed_ has quit IRC | 23:27 | |
| *** kgriffs is now known as kgriffs|afk | 23:35 | |
| *** reed_ has joined #openstack-marconi | 23:38 | |
| *** vkmc_ has quit IRC | 23:43 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!