| openstackgerrit | Angus Salkeld proposed a change to openstack/python-marconiclient: Fix some links in the README https://review.openstack.org/53710 | 00:04 |
|---|---|---|
| *** amitgandhi has joined #openstack-marconi | 00:04 | |
| *** amitgandhi has quit IRC | 00:07 | |
| *** amitgandhi has joined #openstack-marconi | 00:07 | |
| *** wirehead_ has quit IRC | 00:09 | |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Fix some links in the README https://review.openstack.org/53710 | 00:10 |
| *** jamieh__ has quit IRC | 00:20 | |
| *** tedross has quit IRC | 00:20 | |
| *** wirehead_ has joined #openstack-marconi | 00:21 | |
| *** jcru has quit IRC | 00:26 | |
| *** adrian_otto has joined #openstack-marconi | 00:47 | |
| *** adrian_otto has quit IRC | 01:06 | |
| *** nosnos has joined #openstack-marconi | 01:13 | |
| *** nosnos has quit IRC | 01:13 | |
| *** nosnos has joined #openstack-marconi | 01:14 | |
| *** ayoung has joined #openstack-marconi | 01:20 | |
| *** amitgandhi has quit IRC | 01:32 | |
| *** oz_akan_ has quit IRC | 01:39 | |
| *** oz_akan_ has joined #openstack-marconi | 01:40 | |
| *** oz_akan_ has quit IRC | 01:44 | |
| *** cpallares has quit IRC | 02:21 | |
| *** oz_akan_ has joined #openstack-marconi | 02:22 | |
| *** nosnos_ has joined #openstack-marconi | 02:27 | |
| *** haomaiwang has joined #openstack-marconi | 02:30 | |
| *** nosnos has quit IRC | 02:30 | |
| *** amitgandhi has joined #openstack-marconi | 02:33 | |
| *** amitgandhi has quit IRC | 02:39 | |
| *** amitgandhi has joined #openstack-marconi | 02:43 | |
| *** amitgandhi has quit IRC | 02:47 | |
| *** fifieldt has joined #openstack-marconi | 02:59 | |
| *** amitgandhi has joined #openstack-marconi | 03:36 | |
| *** amitgandhi has quit IRC | 03:40 | |
| *** oz_akan_ has quit IRC | 03:53 | |
| *** amitgandhi has joined #openstack-marconi | 04:37 | |
| *** amitgandhi has quit IRC | 04:42 | |
| *** nosnos_ has quit IRC | 04:44 | |
| *** nosnos has joined #openstack-marconi | 04:44 | |
| *** amitgandhi has joined #openstack-marconi | 04:47 | |
| *** amitgandhi has quit IRC | 04:51 | |
| *** adrian_otto has joined #openstack-marconi | 05:17 | |
| *** adrian_otto has quit IRC | 05:37 | |
| *** amitgandhi has joined #openstack-marconi | 05:37 | |
| *** amitgandhi has quit IRC | 05:42 | |
| *** amitgandhi has joined #openstack-marconi | 06:38 | |
| *** adrian_otto has joined #openstack-marconi | 06:39 | |
| *** amitgandhi has quit IRC | 06:42 | |
| *** flaper87|afk is now known as flaper87 | 07:08 | |
| *** nosnos_ has joined #openstack-marconi | 07:11 | |
| *** nosnos has quit IRC | 07:14 | |
| *** adrian_otto has quit IRC | 07:30 | |
| *** nosnos_ has quit IRC | 07:31 | |
| *** nosnos has joined #openstack-marconi | 07:32 | |
| *** amitgandhi has joined #openstack-marconi | 07:38 | |
| *** amitgandhi has quit IRC | 07:43 | |
| *** Krast has joined #openstack-marconi | 08:32 | |
| *** amitgandhi has joined #openstack-marconi | 08:39 | |
| *** amitgandhi has quit IRC | 08:43 | |
| *** amitgandhi has joined #openstack-marconi | 08:49 | |
| *** amitgandhi has quit IRC | 08:54 | |
| *** yassine has joined #openstack-marconi | 09:14 | |
| *** yassine_ has joined #openstack-marconi | 09:18 | |
| *** jamieh__ has joined #openstack-marconi | 09:20 | |
| *** nosnos_ has joined #openstack-marconi | 09:33 | |
| *** nosnos has quit IRC | 09:37 | |
| *** amitgandhi has joined #openstack-marconi | 09:40 | |
| *** amitgandhi has quit IRC | 09:44 | |
| *** Krast has quit IRC | 10:07 | |
| *** Krast has joined #openstack-marconi | 10:07 | |
| *** Krast has quit IRC | 10:07 | |
| *** amitgandhi has joined #openstack-marconi | 10:41 | |
| *** amitgandhi has quit IRC | 10:45 | |
| *** amitgandhi has joined #openstack-marconi | 10:51 | |
| *** amitgandhi has quit IRC | 10:55 | |
| *** amitgandhi has joined #openstack-marconi | 11:41 | |
| *** amitgandhi has quit IRC | 11:45 | |
| *** nosnos_ has quit IRC | 11:52 | |
| *** nosnos has joined #openstack-marconi | 11:53 | |
| *** nosnos has quit IRC | 11:53 | |
| *** nosnos has joined #openstack-marconi | 11:54 | |
| *** nosnos has quit IRC | 11:58 | |
| *** nosnos has joined #openstack-marconi | 11:59 | |
| *** nosnos has quit IRC | 12:04 | |
| *** fifieldt has quit IRC | 12:25 | |
| *** yassine_ has quit IRC | 12:33 | |
| *** yassine has quit IRC | 12:33 | |
| *** yassine has joined #openstack-marconi | 12:33 | |
| *** amitgandhi has joined #openstack-marconi | 12:42 | |
| *** amitgandhi has quit IRC | 12:47 | |
| *** amitgandhi has joined #openstack-marconi | 12:52 | |
| *** amitgandhi has quit IRC | 12:56 | |
| *** tedross has joined #openstack-marconi | 12:57 | |
| *** mcpierce has left #openstack-marconi | 13:47 | |
| *** amitgandhi has joined #openstack-marconi | 14:06 | |
| *** jcru has joined #openstack-marconi | 14:06 | |
| *** amitgandhi has quit IRC | 14:07 | |
| *** amitgandhi has joined #openstack-marconi | 14:07 | |
| *** mpanetta has joined #openstack-marconi | 14:16 | |
| *** whenry has quit IRC | 14:18 | |
| *** alcabrera has joined #openstack-marconi | 14:19 | |
| alcabrera | Good morning! | 14:26 |
| *** whenry has joined #openstack-marconi | 14:26 | |
| mpanetta | Morning :) | 14:27 |
| *** oz_akan_ has joined #openstack-marconi | 14:27 | |
| alcabrera | :) | 14:28 |
| mpanetta | How is your cat doing alcabrera? | 14:33 |
| alcabrera | She's okay-ish. She's a little sniffly this morning, but was happy to eat. | 14:35 |
| mpanetta | That is good to hear :) | 14:35 |
| flaper87 | alcabrera: mpanetta goood morning!!!!! | 14:36 |
| *** ykaplan has joined #openstack-marconi | 14:37 | |
| alcabrera | flaper87: hey! :D | 14:37 |
| * flaper87 likes how goood with 3 Os looks | 14:37 | |
| flaper87 | :D | 14:37 |
| mpanetta | flaper87: Hello! | 14:37 |
| * flaper87 back as a full-day developer | 14:37 | |
| mpanetta | Welcome back! :) | 14:37 |
| alcabrera | w00t | 14:37 |
| *** malini_afk is now known as malini | 14:46 | |
| *** jamieh__ has quit IRC | 15:13 | |
| *** flwang has joined #openstack-marconi | 15:17 | |
| flaper87 | flwang: welcome! :) | 15:18 |
| flaper87 | alcabrera: mpanetta come on guys, lets say Hi to flwang :D | 15:18 |
| flwang | flaper87: haha, yep | 15:18 |
| alcabrera | flwang: hey! :) | 15:18 |
| flaper87 | flwang: this is a low-hanging-fruit you could start working on: https://bugs.launchpad.net/marconi/+bug/1222933 | 15:18 |
| flwang | hey guys, just joined a session in the summit, so I'm interested in it | 15:18 |
| flwang | hope I can contribute a little bit :) | 15:19 |
| flaper87 | please, feel free to yell if something doesn't make sense | 15:19 |
| * flaper87 pints out that flwang is also a Glance-core fellow | 15:19 | |
| malini | o/ | 15:19 |
| flaper87 | points* | 15:19 |
| malini | welcome flwang | 15:19 |
| alcabrera | yeah, ask away. Glad to have you joining us! | 15:20 |
| flwang | flaper87: seems Zhihao Yuan is working on that bug? | 15:20 |
| flaper87 | zyuan: are you? ^ | 15:20 |
| flaper87 | it's assinged to him but no patch has been submitted, yet! | 15:21 |
| flaper87 | :D | 15:21 |
| * flaper87 waits for a patch in 3, 2, 1.... | 15:21 | |
| flaper87 | aaaawwww :( | 15:21 |
| flaper87 | didn't work | 15:21 |
| flwang | flaper87: got it, I think I need a little bit time to dive into the code and then ask some stupid questions :) | 15:22 |
| flaper87 | flwang: there are no stupid questions! You may also want to watch the talk we gave at the summit | 15:23 |
| flaper87 | flwang: www.youtube.com/watch?v=IgE3Jskqtek | 15:23 |
| flaper87 | ah damn | 15:23 |
| flaper87 | youtube <-> china | 15:23 |
| flwang | flaper87: you got it | 15:23 |
| flaper87 | flwang: not sure how I can help there but I know you know what to do ;) | 15:23 |
| mpanetta | Hi flwang! | 15:25 |
| flwang | flaper87: you know, cross the wall is the basic skill for Chinese developer http://img1.gtimg.com/cq/pics/19946/19946210.jpg | 15:25 |
| flaper87 | LOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOL | 15:25 |
| flwang | mapanetta: hey there | 15:25 |
| *** jergerber has joined #openstack-marconi | 15:26 | |
| flwang | flaper87: let me ask the first stupid question, what's the mission of Marconi? provide a message service for cloud user, instead of Openstack itself, is it? | 15:27 |
| zyuan | flaper87: i'm not working on that, i'm not sure how much it helps | 15:27 |
| mpanetta | flwang: It is an extension to openstack, a new api. | 15:28 |
| flwang | mapanetta: got, so what's the extensions for? I mean who is the consumer? | 15:29 |
| flwang | I assume it's cloud tenant user, instead of openstack itself, right? | 15:30 |
| alcabrera | flwang: the consumer would typically be a cloud user. | 15:31 |
| flwang | alcabrera: got it, thx | 15:31 |
| alcabrera | I can see Marconi being used as a messaging bus for local use, as well, since it is pretty simple to get up and running. :) | 15:31 |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Add list of required fields to the API definition https://review.openstack.org/51850 | 15:31 |
| alcabrera | It's one of the friendliest messaging systems I've seen so far, but then again, I'm both biased and not widely versed in messaging systems. :P | 15:31 |
| flwang | alcabrera: thanks, so seems like it's an implement of SQS/SNS in OpenStack | 15:33 |
| alcabrera | exactly. :) | 15:33 |
| flwang | alcabrera: cool | 15:34 |
| flaper87 | wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwhhhhhhhhhhhaaaaaaaaaaaaattttttttttttt??? a patch merged in the client? | 15:34 |
| alcabrera | The SQS part is mostly done (for a v1.0 release). The SNS portion is still in the design phases. | 15:34 |
| * flaper87 cries | 15:34 | |
| flaper87 | kgriffs_afk: THANK YOUUU!! | 15:34 |
| flaper87 | kgriffs_afk: now, next one :D | 15:34 |
| flwang | btw, I realllllllllllllly like the pronunciation of Marconi | 15:35 |
| alcabrera | flaper87: congrats! | 15:36 |
| alcabrera | flwang: it *does* roll off the tongue nicely. :) | 15:36 |
| flwang | alcabrera: yep :) | 15:37 |
| flwang | very cool | 15:37 |
| mpanetta | Of course, it is italian :P | 15:37 |
| flaper87 | mpanetta: hahahah | 15:38 |
| mpanetta | :) | 15:38 |
| flwang | flaper87: I want an Italian name :P | 15:38 |
| flaper87 | flwang: I can give you one and start calling you like that! :D | 15:40 |
| * flaper87 is trying to keep religion out of this conversation | 15:40 | |
| flaper87 | :D | 15:40 |
| flwang | flaper87: waiting and may use it as my nickname :) | 15:40 |
| mpanetta | But they all end in vowels :P hehe | 15:41 |
| * mpanetta is being silly | 15:41 | |
| flwang | don't tell me all of you guys are Italian :) | 15:41 |
| mpanetta | I'm not, only by name. | 15:42 |
| mpanetta | I've visited my relatives in Italy though, does that count? :P | 15:42 |
| flwang | hehe, hope I have chance to join the summit in Paris next year, then we can go to Italy, I'm a fans of Juventus club | 15:43 |
| malini | I am so excited abt the next summit in Atlanta ! | 15:44 |
| malini | home sweet home !! | 15:44 |
| flwang | haha | 15:44 |
| malini | not that I wud mind going to Paris ;) | 15:45 |
| mpanetta | When is that one malini? | 15:45 |
| mpanetta | I want to go to Paris! | 15:45 |
| malini | next April ? flaper87 might know better | 15:45 |
| mpanetta | malini: I mean the one here in atl | 15:45 |
| flaper87 | April -> Atlanta, November -> Paris | 15:45 |
| mpanetta | Oh 2 in one year? | 15:46 |
| mpanetta | Sweet | 15:46 |
| flaper87 | mpanetta: every 6 months | 15:46 |
| mpanetta | Good, that means I get to attend one ;) | 15:46 |
| malini | we wud be graduating in the ATL summit | 15:46 |
| malini | tht got me motivated to start tinkering with tempest | 15:46 |
| *** ykaplan has quit IRC | 15:47 | |
| *** kgriffs_afk is now known as kgriffs | 15:47 | |
| * alcabrera still needs to learn him some Heat for great good | 15:47 | |
| mpanetta | heat? tempest? | 15:47 |
| mpanetta | What are these? | 15:47 |
| malini | mpanetta: tempest is the openstack integration test suite | 15:48 |
| flaper87 | btw, did I mentioned that Heat, Horizon and Ceilometer are looking forward to use Marconi ? | 15:48 |
| flaper87 | kgriffs: 'sup buddy? How are you doing? | 15:49 |
| flaper87 | good morning | 15:49 |
| mpanetta | malini: Oh? | 15:49 |
| flaper87 | malini: how's the work on tempest integration going? | 15:49 |
| mpanetta | I will have to investigate | 15:49 |
| mpanetta | So many new things to investigate | 15:49 |
| * mpanetta is reminded of Cat on Red Dwarf and his investigations | 15:50 | |
| malini | flaper87: I have just started on it..nothing substantial so far. But I hope to have something solid by end of this month | 15:50 |
| alcabrera | ahh, the great anvestor Cat that started it all... | 15:50 |
| kgriffs | o/ | 15:50 |
| kgriffs | speaking of graduation | 15:50 |
| flaper87 | malini: AWESOME! | 15:51 |
| flaper87 | looking forward to your patches | 15:51 |
| kgriffs | Judging by the last TC list digest, I don't think they necessarily have us on the radar for Icehouse graduation | 15:51 |
| flwang | flaper87: i'm interested in that you mentioned Heat, Horizon and CM will use Marconi, any more details? | 15:52 |
| kgriffs | "OpenStack is currently 9 integrated projects in Havana, 10 once we get to Icehouse (+Trove)." | 15:52 |
| mpanetta | alcabrera: Red Dwarf fan? | 15:52 |
| alcabrera | mpanetta: only slightly. It was fun to watch, but I remember very little. I like to think of myself as "Red Dwarf aware". :) | 15:52 |
| flwang | kgriffs: will Marconi graduate in Icehouse and to be in integrated state? | 15:52 |
| mpanetta | alcabrera: Ahh! Cool :) | 15:53 |
| kgriffs | I would like it to be in Icehouse, but we need to prove to the TC that we are ready in plenty of time, like 8+ weeks before hand | 15:53 |
| flwang | kgriffs: got, thanks | 15:53 |
| kgriffs | flaper87: you were working on devstack-gate, right? | 15:54 |
| flaper87 | kgriffs: TBH, I'd like it to be in Icehouse as well but, I'm also fine with us taking our time to make things right | 15:54 |
| flaper87 | kgriffs: I am | 15:54 |
| kgriffs | kewl | 15:54 |
| flaper87 | I think we've enough time to make it | 15:54 |
| kgriffs | we also need "useful tests in the integrated gate" | 15:54 |
| malini | kgriffs: are these all the stuff we need to graduate https://wiki.openstack.org/wiki/Marconi/Incubation/Graduation ? | 15:54 |
| flaper87 | I just don't want us to 'freak out' if we don't | 15:54 |
| malini | kgriffs: "useful tests" as in tempest tests, rt? | 15:54 |
| kgriffs | flaper87: right. We will have a solid, stable release in the Icehouse timeframe whether or not we are officially integrated | 15:55 |
| kgriffs | malini: good question, idk | 15:55 |
| kgriffs | I was going to ask if anyone knew what Sean was talking about | 15:55 |
| malini | Sean Dague? | 15:55 |
| kgriffs | All integrated projects should be in the integrated gate, as this is | 15:55 |
| kgriffs | the only way we provably know that they can all work together, at the | 15:55 |
| kgriffs | same level of requirements, in a consistent way. | 15:55 |
| kgriffs | " | 15:55 |
| flaper87 | kgriffs: correct, that's how I see it | 15:55 |
| kgriffs | malini: right | 15:56 |
| kgriffs | sounds like Tempest to me | 15:56 |
| flaper87 | kgriffs: that's mostly related to having support for devstack, tempest and making marconi tests run in the same environment with all other services running | 15:56 |
| malini | I'll ping him to figure out what exactly is needed | 15:56 |
| kgriffs | malini: thanks! | 15:56 |
| flwang | flaper87: i'm interested in that you mentioned Heat, Horizon and CM will use Marconi, any more details? | 15:57 |
| kgriffs | fwiw, here is the complete email: | 15:57 |
| kgriffs | http://paste.openstack.org/show/52436/ | 15:57 |
| flaper87 | flwang: The three of them want to use the notification part of MArconi | 15:57 |
| kgriffs | malini: question re. Tsung reports | 15:58 |
| flaper87 | Heat to figure when service's bootstrap has been completed, AFAIU. Ceilometer to send alarms and notify others | 15:58 |
| flwang | so what's the relationship between marconi's notification and the oslo.messaging? | 15:58 |
| kgriffs | I was looking at this: http://198.61.239.147:8000/log/20131112-1628/graph.html | 15:58 |
| flaper87 | and same for Horizon | 15:58 |
| *** jergerber has quit IRC | 15:58 | |
| kgriffs | malini: what is a "page"? | 15:58 |
| kgriffs | Looking at transactions, for example, I see ~3000 "pages" per sec | 15:58 |
| malini | kgriffs:a page is a group of request not separated by a thinktime | 15:59 |
| kgriffs | what is a think time? | 16:00 |
| malini | it's a delay you add between requests | 16:00 |
| malini | for us the page response time does not really give any useful insight | 16:00 |
| *** jamieh__ has joined #openstack-marconi | 16:00 | |
| malini | kgriffs: http://198.61.239.147:8000/log/20131112-1628/mike-high-load-multiple-endpoints.xml | 16:00 |
| kgriffs | so, looking at this report, we can currently sustain only 1000 tps? | 16:00 |
| kgriffs | is that our max for one shard? | 16:01 |
| flaper87 | brb | 16:01 |
| malini | kgriffs: requests is what we are interested in this report | 16:01 |
| malini | we can sustain ~4000 rps with 3 shards | 16:01 |
| kgriffs | oh, so ignore that transactions graph, and look at "Requests"? | 16:02 |
| kgriffs | http://198.61.239.147:8000/log/20131112-1628/images/graphes-Perfs-rate.png | 16:02 |
| malini | kgriffs: yes | 16:02 |
| kgriffs | according to this, we sit at around 3000 | 16:02 |
| kgriffs | this graph was for a single shard iirc | 16:03 |
| alcabrera | brb | 16:03 |
| *** alcabrera is now known as alcabrera|afk | 16:03 | |
| *** jergerber has joined #openstack-marconi | 16:03 | |
| kgriffs | malini: once we have these tests running automatically (nightly or triggered for each patch) it would be cool to plot our performance over time | 16:04 |
| kgriffs | so we can see trends | 16:04 |
| kgriffs | like, "look, we are 1.5x faster than we were a month ago" | 16:04 |
| malini | kgriffs: tht wud be cool.we also need to include more API's than what we have here | 16:04 |
| kgriffs | +1 | 16:04 |
| malini | & we can also try a heavier load than the one here | 16:05 |
| malini | last week was weird..we found out that one of the reasons we hit a plateau was because we didn't have a heavy enough load :D | 16:06 |
| kgriffs | gtk | 16:07 |
| kgriffs | In other news, has anyone come across any leaked information about SQS architecture? | 16:08 |
| kgriffs | Right now it is just a black box as far as I know. | 16:09 |
| mpanetta | malini: Should I add tsung-8 to the xml files? | 16:10 |
| malini | mpanetta: sure..it'll be good to add tht | 16:13 |
| mpanetta | Ok | 16:13 |
| malini | kgriffs: is tht geek gossip ;) | 16:14 |
| kgriffs | yes. I'm looking for gossip. | 16:21 |
| kgriffs | :D | 16:21 |
| kgriffs | flaper87: I | 16:29 |
| kgriffs | am reviewing the client patches | 16:29 |
| kgriffs | btw, can you also look at the sharding patches? | 16:30 |
| *** adrian_otto has joined #openstack-marconi | 16:33 | |
| * flaper87 back | 16:34 | |
| mpanetta | wb | 16:34 |
| flaper87 | kgriffs: yeah, I started doing that today, I got distracted by some internal stuff but I'll get back to that asap | 16:35 |
| kgriffs | flaper87: RabbitMQ can do something like 40K messages/sec, right? I haven't benchmarked it myself, but that is what I am seeing reported in blogs. | 16:36 |
| kgriffs | flaper87: kewl, thanks. BTW, I am planning on taking notes from the summit and converting to blueprints and stuff as appropriate over the next couple of weeks | 16:37 |
| flaper87 | kgriffs: roughly yes! | 16:37 |
| kgriffs | also cleaning up the wiki | 16:37 |
| kgriffs | flaper87: OK, I just wanted to make sure I wasn't crazy. | 16:37 |
| *** amitgandhi has quit IRC | 16:37 | |
| flaper87 | kgriffs: +1, I can take core of the notificaion and AMQP sessions | 16:37 |
| kgriffs | flaper87: that would be great, thx | 16:38 |
| flaper87 | kgriffs: from the first session, I think we agreed one putting more thoughts on the deserialization / content-type thing, and the qack endpoint | 16:38 |
| flaper87 | The former kinda made sense but we didn't want it | 16:39 |
| flaper87 | the later, instead, is something we definitely need | 16:39 |
| flaper87 | alcabrera|afk: proposed it as a topic for the next meeting | 16:39 |
| kgriffs | by "qack" you are thinking basically "pop"? | 16:39 |
| * flaper87 had a call with amit, alcabrera and zyuan on monday about 'What happened in the summit' | 16:39 | |
| flaper87 | kgriffs: yeah | 16:40 |
| flaper87 | :D | 16:40 |
| kgriffs | kk | 16:40 |
| flaper87 | I just wanted to create some confussions | 16:40 |
| kgriffs | I was worthless monday | 16:40 |
| flaper87 | confussion | 16:40 |
| kgriffs | got home Monday morning at 2am after 32 hours in transit | 16:40 |
| flaper87 | MAN!!! | 16:40 |
| kgriffs | (due to delayed flight in HK which caused a missed connection in SFO) | 16:40 |
| flaper87 | T_T | 16:40 |
| kgriffs | so, sorry I wasn't around! | 16:41 |
| flaper87 | no worries, hopefully I didn't forget anything. I used the ehterpads so, they seem to have everything we talked about | 16:41 |
| kgriffs | cool beans | 16:42 |
| kgriffs | flaper87: I wonder how fast we can get Marconi given a redis backend and a TCP transport | 16:43 |
| kgriffs | I don't want to chase RabbitMQ necessarily | 16:44 |
| kgriffs | at some point it may make sense to start a broker provisioning service ala Trove under the Marconi program umbrella | 16:45 |
| kgriffs | like, spin me up an HA, single-tenant AMQP broker and give me nice monitoring and admin features around it | 16:46 |
| flaper87 | kgriffs: agreed | 16:46 |
| kgriffs | it would obviously cost more in terms of compute resources, but I can see some people willing to pay for it | 16:46 |
| flaper87 | so, re speed, I think we can improve it a lot by using a tcp transport and some other backend | 16:47 |
| kgriffs | tbh, I think the main value proposition for the TCP transport in queues is providing push to notification consumers | 16:47 |
| flaper87 | I still think we could improve mongodb's one, though | 16:47 |
| kgriffs | So, maybe we target something like 10-20K tps | 16:48 |
| *** alcabrera|afk is now known as alcabrera | 16:48 | |
| * alcabrera catches up | 16:48 | |
| kgriffs | and for people who want higher throuput, they would use a RabbitMQ or MQ series or whatever directly | 16:48 |
| kgriffs | flaper87: definitely | 16:48 |
| flaper87 | kgriffs: also, at some point, we could have something like 'direct-access' to the backend | 16:48 |
| *** amitgandhi has joined #openstack-marconi | 16:48 | |
| kgriffs | Mongo once can be improved | 16:48 |
| kgriffs | esp. if we make FIFO and high durability optional | 16:49 |
| flaper87 | that's something I've been thinking about but I'm still unsure as to how we could achieve that | 16:49 |
| kgriffs | if we didn't have to generate the marker we would be really fast | 16:49 |
| kgriffs | plus we have some indexing work to look at | 16:49 |
| flaper87 | kgriffs: indeed! | 16:49 |
| kgriffs | flaper87: idk about direct access to the backend. I think that would be a separate service like I said before, since it would imply single-tenancy anyway, wouldn't it? | 16:50 |
| flaper87 | kgriffs: well, you know, since we'll have a discoverable API and we'll also ship partially implemented APIs, making FIFO optional in the mongodb driver sounds reasonable | 16:50 |
| kgriffs | actually, I could see that being a queue "flavor" | 16:50 |
| kgriffs | you could say "i don't need strict ordering" | 16:50 |
| kgriffs | and then we put that on a shard where the driver is configured to just use timestamp-based markers | 16:51 |
| flaper87 | kgriffs: exactly, I think we should start writing some notes wrt queue's flavors | 16:51 |
| kgriffs | not just strict ordering, but you might also get duplicate messages - client would have to watch out for that and skip things it has seen already | 16:51 |
| kgriffs | flaper87: +1 | 16:51 |
| alcabrera | +1 for queue flavors and optional FIFO | 16:51 |
| kgriffs | i think flavors is going to turn out to be VERY useful | 16:51 |
| alcabrera | I'm a big fan of the queue flavors idea | 16:51 |
| alcabrera | Declarative queueing | 16:52 |
| kgriffs | can someone register bp and start a wiki page? | 16:52 |
| flaper87 | kgriffs: o/ | 16:52 |
| alcabrera | Goes *really* well with heterogeneous backends + sharding | 16:52 |
| kgriffs | i can't remember if we have one already | 16:52 |
| flaper87 | not for flavors | 16:52 |
| flaper87 | not that I know | 16:52 |
| kgriffs | ok | 16:52 |
| alcabrera | I don't think we have anything for queue flavors yet | 16:52 |
| *** yassine has quit IRC | 16:52 | |
| kgriffs | let's check and create one if not there already, and seed a wiki page | 16:52 |
| kgriffs | speaking of bps | 16:53 |
| kgriffs | we have a backlog to triage, also bugs | 16:53 |
| alcabrera | oh yes... | 16:53 |
| kgriffs | I was thinking we could spend 30 mins each day at 1600 UTC going over those | 16:53 |
| alcabrera | A lot of BPs, a lot of bugs. | 16:53 |
| alcabrera | +1 | 16:54 |
| alcabrera | 1600 UTC works for me. | 16:54 |
| kgriffs | flaper87: ^^^ | 16:55 |
| flaper87 | kgriffs: +1 | 16:55 |
| flaper87 | sounds like a plan | 16:55 |
| alcabrera | kgriffs: I'm giving a tech talk today we might be able to link to the falconframework page - "The Definitive Introduction to Falcon" | 16:55 |
| flaper87 | https://blueprints.launchpad.net/marconi/+spec/marconi-queue-flavors | 16:55 |
| alcabrera | flaper87: +1 | 16:55 |
| kgriffs | alcabrera: oh, nice | 16:55 |
| kgriffs | I need to do a Falcon sprint in the near future | 16:56 |
| kgriffs | alcabrera: let me send you some slides in case you want to plagerize | 16:56 |
| alcabrera | kgriffs: thanks! | 16:56 |
| alcabrera | I'm tempted to do a docstring sprint for falcon. | 16:56 |
| alcabrera | Get us on RTD and such. | 16:56 |
| alcabrera | (esp. since RTD looks *really* snazzy now) | 16:57 |
| *** jamieh__ has quit IRC | 16:57 | |
| kgriffs | http://slid.es/kurtgriffiths/introducing-falcon | 16:59 |
| kgriffs | alcabrera: yes!!!!! | 16:59 |
| kgriffs | along with that, I am tempted to create a google comment style ---> rtd/sphinx tool | 16:59 |
| kgriffs | alcabrera: Keep an eye on the nav arrows. Sometimes you have to go "down" to get to the next slide. | 17:00 |
| alcabrera | oohhh | 17:00 |
| alcabrera | I missed that. | 17:00 |
| kgriffs | flaper87: re WSME vs. JSON schema | 17:00 |
| kgriffs | personally, I think WSME's fundamental philosophy of ORM-based, static definitions if flawed | 17:01 |
| kgriffs | Given that nova and glance will be using JSON schema | 17:01 |
| kgriffs | and it is looking like we will as well | 17:01 |
| kgriffs | I think we should do a survey of Python serialization and validation frameworks | 17:02 |
| flaper87 | jsonschema all the way down! | 17:02 |
| flaper87 | :D | 17:02 |
| kgriffs | serialization is an interesting one | 17:02 |
| kgriffs | I am suspicious that an ORM can generate truly XML-y XML and JSON-y JSON | 17:02 |
| flaper87 | damn, 2 secs. brb! | 17:02 |
| flaper87 | I don't think we need WSME | 17:03 |
| flaper87 | jsonschema will work better for us and the cross-transport support | 17:03 |
| flaper87 | WSME is *too* HTTP / Rest oriented, AFAIU | 17:03 |
| kgriffs | flaper87: actually, it is ironic | 17:03 |
| kgriffs | I don't think it is actually RESTful | 17:03 |
| kgriffs | REST media types tend to be more flexible, and WSME seems to be more static | 17:04 |
| kgriffs | I suspect that is due to it's SOAP heritage | 17:04 |
| *** jergerber has quit IRC | 17:14 | |
| kgriffs | anybody play around with this? | 17:17 |
| kgriffs | https://code.google.com/p/httpplus/ | 17:17 |
| *** jamieh__ has joined #openstack-marconi | 17:18 | |
| alcabrera | I've never heard of it. :/ | 17:20 |
| kgriffs | I just saw it proposed as an alternative to httplib for swift client | 17:20 |
| alcabrera | and it doesn't have a pypi registry. | 17:20 |
| kgriffs | https://gist.github.com/anonymous/9f806d7811662275e41a | 17:21 |
| *** jamieh__ has quit IRC | 17:22 | |
| alcabrera | ah, I see. | 17:22 |
| alcabrera | I wonder if they've considered requests? | 17:22 |
| kgriffs | idk | 17:22 |
| ekarlso | what about the oslo apiclient stuff ? | 17:22 |
| kgriffs | I recall something someone said at the summit in passing about requests | 17:22 |
| kgriffs | like, it didn't work in once case - wish I could remember what that was | 17:23 |
| kgriffs | s/once/one | 17:23 |
| kgriffs | I feel like we should just contribute patches to request | 17:23 |
| kgriffs | if it isn't quite where it needs to be for something | 17:24 |
| kgriffs | (i mean) | 17:24 |
| ekarlso | I just used the apiclient stuff for python-libraclient | 17:24 |
| ekarlso | works awesomely which is based on request | 17:24 |
| *** jergerber has joined #openstack-marconi | 17:25 | |
| *** jamieh__ has joined #openstack-marconi | 17:26 | |
| *** cindy has joined #openstack-marconi | 17:32 | |
| *** reed has joined #openstack-marconi | 17:34 | |
| *** cindy is now known as cpallares | 17:37 | |
| *** adrian_otto has left #openstack-marconi | 18:09 | |
| *** gleicon___ has quit IRC | 18:40 | |
| *** alcabrera is now known as alcabrera|afk | 18:45 | |
| *** alcabrera|afk is now known as alcabrera | 19:03 | |
| *** jamieh__ has quit IRC | 19:25 | |
| *** jamieh__ has joined #openstack-marconi | 19:28 | |
| amitgandhi | kgriffs: you around? | 19:37 |
| amitgandhi | <--- meeting | 19:38 |
| kgriffs | yes, sorry, didn't realize i wasn't connected to the other IRC server | 19:38 |
| *** alcabrera is now known as alcabrera|afk | 20:02 | |
| *** mpanetta_ has joined #openstack-marconi | 20:05 | |
| *** alcabrera has joined #openstack-marconi | 20:06 | |
| *** mpanetta has quit IRC | 20:08 | |
| *** alcabrera|afk has quit IRC | 20:09 | |
| *** kgriffs is now known as kgriffs_afk | 20:12 | |
| flaper87 | kgriffs_afk: when you're back, could you https://review.openstack.org/#/c/52389/ ? :D | 20:17 |
| *** kgriffs_afk is now known as kgriffs | 20:22 | |
| *** malini is now known as malini_afk | 20:22 | |
| *** mpanetta_ has quit IRC | 20:45 | |
| *** mpanetta has joined #openstack-marconi | 20:46 | |
| *** acabrera has joined #openstack-marconi | 20:48 | |
| *** alcabrera has quit IRC | 20:49 | |
| *** acabrera is now known as alcabrera | 20:57 | |
| flaper87 | alcabrera: just wanted to give you another 'Way to go' for the work you've been doing on the partition bp | 21:05 |
| alcabrera | flaper87: thanks! :D | 21:26 |
| * flaper87 is writing a blog post after a long time of pure silence | 21:37 | |
| cpallares | hi flaper87! | 21:38 |
| flaper87 | cpallares: THERE YOU ARE!!!! | 21:38 |
| cpallares | flaper87: how was HK? :D | 21:38 |
| flaper87 | and I meant to yell | 21:39 |
| flaper87 | :D | 21:39 |
| cpallares | haha | 21:39 |
| flaper87 | cpallares: it was GREAT!!!! | 21:39 |
| flaper87 | cpallares: so, how are you doing? | 21:39 |
| cpallares | flaper87: I haven't seen you around! I'm doing good :) | 21:39 |
| flaper87 | cpallares: yeah, Last week was pretty damn busy. | 21:40 |
| flaper87 | cpallares: any new contribution to Marconi? | 21:40 |
| flaper87 | you better say yes | 21:41 |
| cpallares | flaper87: Haha I'm still waiting on my other one to get reviewed! | 21:41 |
| * alcabrera goes to check review queue - it's been awhile. | 21:42 | |
| alcabrera | :P | 21:42 |
| alcabrera | cpallares: hey! :D | 21:42 |
| cpallares | alcabrera: hey :) | 21:42 |
| flaper87 | OpenStack Queuing and Notification Service || Smile :D || Meetings every Monday 16:00 UTC || Wiki: ttps://wiki.openstack.org/wiki/Marconi || Paste: http://paste.openstack.org/ || Send messages and make some noise :D | 21:44 |
| *** flaper87 changes topic to "OpenStack Queuing and Notification Service || Smile :D || Meetings every Monday 16:00 UTC || Wiki: ttps://wiki.openstack.org/wiki/Marconi || Paste: http://paste.openstack.org/ || Send messages and make some noise :D" | 21:44 | |
| cpallares | hahaha | 21:44 |
| alcabrera | w00t | 21:44 |
| alcabrera | "Send messages and make some noise :D" | 21:44 |
| alcabrera | will do | 21:44 |
| flaper87 | YEAH!!! | 21:44 |
| * cpallares makes noise | 21:45 | |
| flaper87 | gotta keep this channel alive! There are 41 nicks in here! | 21:45 |
| flaper87 | 36 of those are bots | 21:45 |
| flaper87 | nah, kidding! | 21:45 |
| * alcabrera suspects amitgandhi is a bot | 21:45 | |
| flaper87 | all real people, silent people | 21:45 |
| flaper87 | amitgandhi: you a bot? | 21:45 |
| * flaper87 stares | 21:46 | |
| mpanetta | yes he is :P | 21:46 |
| amitgandhi | awaiting command... | 21:46 |
| kgriffs | My friends call me R. Daneel. | 21:46 |
| amitgandhi | !amitgandhi help | 21:46 |
| openstack | amitgandhi: Error: "amitgandhi" is not a valid command. | 21:47 |
| *** kgriffs is now known as rdaneel | 21:47 | |
| flaper87 | amitgandhi: a bot would've insulted me | 21:47 |
| mpanetta | LOL | 21:47 |
| cpallares | lol! | 21:47 |
| alcabrera | :P | 21:47 |
| flaper87 | :D | 21:47 |
| rdaneel | you are all part of my master plan to save humanity | 21:47 |
| mpanetta | amitgandhi: just got owned by openstack | 21:47 |
| mpanetta | I need to read that book... | 21:47 |
| amitgandhi | flaper87: bots need some gelato | 21:47 |
| amitgandhi | mpanetta: lol | 21:48 |
| mpanetta | Mmm gelato | 21:48 |
| flaper87 | I sent some. Didn't rdaneel give it to you? | 21:48 |
| mpanetta | Probably not, he eats everything | 21:50 |
| alcabrera | rdaneel, mpanetta: makes sense, given a loose interpretation of the 0th law of robotics. You've gotta prevent them from eating themselves to death. ;) | 21:50 |
| mpanetta | haha | 21:51 |
| rdaneel | anybody know if there's an executor for eventlet? | 21:52 |
| rdaneel | https://pypi.python.org/pypi/futures | 21:52 |
| *** rdaneel is now known as kgriffs | 21:53 | |
| alcabrera | kgriffs: I've seen no such compatibility layer, though it makes me think of something like this: http://eventlet.net/doc/threading.html#tpool-simple-thread-pool | 21:55 |
| flaper87 | given the fact that eventlet does not support python 3.3 and hell it won't support python 3.2, I doubt there's support of it | 21:55 |
| flaper87 | however, it may make sense for python 2.7 | 21:55 |
| flaper87 | s/may// | 21:56 |
| kgriffs | i heard rumors that eventlet work for py3 was planned | 21:56 |
| kgriffs | did I hear wrong? | 21:56 |
| flaper87 | kgriffs: owwww reallly??? | 21:56 |
| flaper87 | I've heard all kind of things but that | 21:56 |
| kgriffs | heh | 21:56 |
| flaper87 | kgriffs: maybe gevent ? | 21:56 |
| kgriffs | maybe | 21:56 |
| alcabrera | kgriffs, flaper87: https://github.com/eventlet/eventlet/issues/6#issuecomment-22100946 (eventlet on py3k) | 21:57 |
| alcabrera | that's the latest | 21:57 |
| kgriffs | hmm | 21:59 |
| kgriffs | looks like a lot of talk but no action | 21:59 |
| alcabrera | yup | 21:59 |
| kgriffs | flaper87: I was just thinking, the client lib could return a future instead of taking a callback. or something | 21:59 |
| kgriffs | we could then implement an executor for eventlet or gevent | 22:00 |
| kgriffs | and then require py 3.4 with tulip or whatever | 22:00 |
| kgriffs | for running on Py3 | 22:00 |
| kgriffs | basically, using the Executor abstraction as our portability layer | 22:00 |
| flaper87 | kgriffs: yup, that's kinda the idea, I think we'll still need the callback, though. | 22:03 |
| flaper87 | I thought about always returning a Future but I don't want people not interested in async results to get a `future` back | 22:04 |
| flaper87 | I thought about having async=True|False and return a Future there | 22:04 |
| flaper87 | but, I don't want to break the functions consistency, I'd like them to always return the result or None. So, I thought about having a callback and using an Actor like execution model | 22:06 |
| kgriffs | brb | 22:07 |
| flaper87 | sure | 22:07 |
| flaper87 | I've to admit, that future library looks sexy | 22:08 |
| flaper87 | and kinda makes sense | 22:08 |
| flaper87 | but I was looking more into Pykka | 22:10 |
| flaper87 | http://www.pykka.org/en/latest/ | 22:10 |
| flaper87 | anywho, I think we should discuss it further, maybe in one of our meetings. | 22:11 |
| alcabrera | flaper87: I remembered something about actors, and then I found this post - http://pchiusano.blogspot.com/2010/01/actors-are-not-good-concurrency-model.html | 22:27 |
| alcabrera | Just a thought, and I agree - we need more discussion on the client concurrency primitives. :) | 22:27 |
| alcabrera | Consistency + 1 | 22:27 |
| flaper87 | alcabrera: wow, thanks for sharing. I'll read it tomorrow, I don't think my brain can take that post right now | 22:28 |
| flaper87 | alcabrera: I'm not an Actors fan, TBH | 22:28 |
| flaper87 | I read the original paper - did you know the whole model cam out in 1985~ ? - and some of the papers that improved the original one | 22:29 |
| alcabrera | My brain can't take it either, atm. :D | 22:29 |
| flaper87 | and still, there are some things I don't like much | 22:29 |
| alcabrera | flaper87: it feels like all the concurrency models have been around at least 10+ years | 22:29 |
| flaper87 | however, it seems to make sense for something like marconilib that needs the executor part to be forward-backward compatible | 22:30 |
| flaper87 | and it doesn't have 'strong' concurrent requirements | 22:30 |
| alcabrera | yeah, it's pretty much just 'send' and 'retrieve result' | 22:31 |
| alcabrera | Like futures. | 22:31 |
| flaper87 | alcabrera: yeah, it can return futures | 22:32 |
| *** oz_akan_ has quit IRC | 22:33 | |
| alcabrera | More thoughts next time! :D | 22:33 |
| alcabrera | I'm out for the night. | 22:33 |
| *** kgriffs is now known as kgriffs_afk | 22:33 | |
| alcabrera | Take care. :) | 22:33 |
| *** alcabrera has quit IRC | 22:33 | |
| *** oz_akan__ has joined #openstack-marconi | 22:34 | |
| *** ryanpetrello has joined #openstack-marconi | 22:37 | |
| *** oz_akan__ has quit IRC | 22:39 | |
| *** flaper87 is now known as flaper87|afk | 22:47 | |
| *** kgriffs_afk is now known as kgriffs | 22:49 | |
| *** whenry has quit IRC | 22:50 | |
| *** ryanpetrello has left #openstack-marconi | 22:52 | |
| *** jamieh__ has quit IRC | 23:00 | |
| *** mpanetta has quit IRC | 23:08 | |
| *** tedross has quit IRC | 23:08 | |
| *** jergerber has quit IRC | 23:18 | |
| *** adrian_otto has joined #openstack-marconi | 23:29 | |
| *** adrian_otto has quit IRC | 23:33 | |
| *** amitgandhi has quit IRC | 23:34 | |
| *** amitgandhi has joined #openstack-marconi | 23:35 | |
| *** amitgandhi has quit IRC | 23:39 | |
| *** kgriffs is now known as kgriffs_afk | 23:50 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
