| *** harshs has joined #openstack-performance | 00:13 | |
| *** harlowja has joined #openstack-performance | 00:20 | |
| *** wootehfoot has joined #openstack-performance | 00:21 | |
| *** wootehfoot has quit IRC | 00:21 | |
| *** wootehfoot has joined #openstack-performance | 00:22 | |
| *** wootehfoot has quit IRC | 00:26 | |
| *** wootehfoot has joined #openstack-performance | 00:26 | |
| *** arnoldje has joined #openstack-performance | 00:44 | |
| *** wootehfoot has quit IRC | 00:50 | |
| *** badari has quit IRC | 01:32 | |
| *** badari has joined #openstack-performance | 01:45 | |
| *** arnoldje has quit IRC | 01:48 | |
| *** badari has quit IRC | 02:12 | |
| *** badari has joined #openstack-performance | 02:19 | |
| *** badari has quit IRC | 02:25 | |
| *** arnoldje has joined #openstack-performance | 02:42 | |
| *** badari has joined #openstack-performance | 02:48 | |
| *** badari_ has joined #openstack-performance | 03:01 | |
| *** badari has quit IRC | 03:04 | |
| *** badari_ has quit IRC | 03:15 | |
| *** badari has joined #openstack-performance | 03:28 | |
| *** badari has quit IRC | 03:41 | |
| *** harshs has quit IRC | 04:01 | |
| *** manand has joined #openstack-performance | 04:32 | |
| *** manand has quit IRC | 05:22 | |
| *** harshs has joined #openstack-performance | 05:27 | |
| *** badari has joined #openstack-performance | 05:29 | |
| *** badari has quit IRC | 05:33 | |
| *** harshs has quit IRC | 05:43 | |
| *** harshs has joined #openstack-performance | 05:46 | |
| *** arnoldje has quit IRC | 06:11 | |
| DinaBelova | #startmeeting Performance Team | 08:01 |
|---|---|---|
| openstack | Meeting started Tue Nov 10 08:01:05 2015 UTC and is due to finish in 60 minutes. The chair is DinaBelova. Information about MeetBot at http://wiki.debian.org/MeetBot. | 08:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 08:01 |
| openstack | The meeting name has been set to 'performance_team' | 08:01 |
| DinaBelova | #topic test | 08:01 |
| DinaBelova | #topic test2 | 08:01 |
| DinaBelova | #endmeeting | 08:01 |
| openstack | Meeting ended Tue Nov 10 08:01:41 2015 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 08:01 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/performance_team/2015/performance_team.2015-11-10-08.01.html | 08:01 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/performance_team/2015/performance_team.2015-11-10-08.01.txt | 08:01 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/performance_team/2015/performance_team.2015-11-10-08.01.log.html | 08:01 |
| DinaBelova | okб умукнерштп ыууьы ещ цщкл щл Ж) | 08:02 |
| DinaBelova | wow :) | 08:02 |
| DinaBelova | everything seems to work ok :) | 08:02 |
| kun_huang | I thought I had missed this meeting when seeing the notification "meeting end log..." in my phone | 08:17 |
| *** P has joined #openstack-performance | 08:30 | |
| *** P is now known as Guest5307 | 08:31 | |
| *** harshs has quit IRC | 09:34 | |
| DinaBelova | kun_huang - please sorry for that :) I just got all changes related to OpenStack bots merged today morning, and wanted to make sure they work ok :) | 11:55 |
| *** pwolsza has joined #openstack-performance | 12:16 | |
| *** pwolsza has left #openstack-performance | 12:20 | |
| *** patrykw has joined #openstack-performance | 12:23 | |
| *** wootehfoot has joined #openstack-performance | 13:31 | |
| *** wootehfoot has quit IRC | 13:43 | |
| *** dburnazyan has joined #openstack-performance | 13:50 | |
| *** ilyashakhat_mobi has joined #openstack-performance | 13:55 | |
| *** ilyashakhat_mobi has quit IRC | 14:06 | |
| *** gals has joined #openstack-performance | 14:11 | |
| *** arnoldje has joined #openstack-performance | 14:11 | |
| *** gals has quit IRC | 14:22 | |
| *** gals has joined #openstack-performance | 14:27 | |
| *** Guest5307 has quit IRC | 14:30 | |
| *** patrykw has quit IRC | 14:30 | |
| *** gals has quit IRC | 14:35 | |
| *** gals has joined #openstack-performance | 14:36 | |
| *** arnoldje has quit IRC | 14:45 | |
| *** regXboi has joined #openstack-performance | 14:46 | |
| *** badari has joined #openstack-performance | 14:52 | |
| *** listomin has joined #openstack-performance | 14:55 | |
| DinaBelova | heh, people are joining the channel :) | 14:57 |
| *** deepika has joined #openstack-performance | 14:57 | |
| *** arnoldje has joined #openstack-performance | 14:59 | |
| DinaBelova | #startmeeting Performance Team | 15:00 |
| openstack | Meeting started Tue Nov 10 15:00:00 2015 UTC and is due to finish in 60 minutes. The chair is DinaBelova. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| openstack | The meeting name has been set to 'performance_team' | 15:00 |
| DinaBelova | Hello everyone! o/ | 15:00 |
| DinaBelova | who's here? :) | 15:00 |
| kun_huang | +1 | 15:00 |
| DinaBelova | harlowja - were you able to wake up? :) | 15:00 |
| *** Kristian_ has joined #openstack-performance | 15:01 | |
| DinaBelova | hope some other channel members are here as well :) | 15:01 |
| badari | +1 | 15:01 |
| DinaBelova | regXboi, SpamapS - are you around as well? | 15:01 |
| *** augiemena3 has joined #openstack-performance | 15:02 | |
| DinaBelova | ok, let's wait for few more minutes | 15:02 |
| Kristian_ | so not used to this type of meeting, more of an ATT Connect user, but #it works | 15:02 |
| DinaBelova | :D | 15:02 |
| Kristian_ | #itworks | 15:02 |
| DinaBelova | while people are waking up - here is our agenda | 15:02 |
| *** UK has joined #openstack-performance | 15:02 | |
| DinaBelova | #link https://wiki.openstack.org/wiki/Meetings/Performance#Agenda_for_next_meeting | 15:02 |
| *** UK is now known as Guest11617 | 15:03 | |
| DinaBelova | I decided to start with something related to the passed summit | 15:03 |
| DinaBelova | #topic Summit follow-up | 15:03 |
| DinaBelova | as you remember we had kick-off session in Tokyo | 15:03 |
| DinaBelova | #link https://etherpad.openstack.org/p/mitaka-cross-project-performance-team-kick-off | 15:03 |
| regXboi | regXboi is here | 15:03 |
| DinaBelova | yay! | 15:03 |
| DinaBelova | I see one moment in the etherpad that seems interesting to me and not covered during the summit session. | 15:04 |
| DinaBelova | #info What's good for performance test (time to do $thing) vs scale test (behavior of doing $thing in parallel) | 15:04 |
| * regXboi is also over in the neutron drivers meeting so running between two rooms | 15:04 | |
| DinaBelova | at Mirantis we use Rally for this purpose with different coefficients. | 15:04 |
| DinaBelova | yeah, that's sad... | 15:04 |
| DinaBelova | regXboi - hope you were able to go through the etherpad? | 15:04 |
| regXboi | unfortunately ... no | 15:04 |
| regXboi | :( | 15:04 |
| DinaBelova | heh :) | 15:05 |
| regXboi | or at least - not yet | 15:05 |
| DinaBelova | ok, i'll kindly ask you to do that in some nearest future :) | 15:05 |
| *** patrykw has joined #openstack-performance | 15:05 | |
| regXboi | there is an RFE over in the neutron drivers meeting that I want to reflect up to oslo because I think it has impact to this effort | 15:05 |
| patrykw | hi all patrick@intel | 15:05 |
| DinaBelova | cool, thank you for visiting it | 15:05 |
| *** jeffcb has joined #openstack-performance | 15:05 | |
| DinaBelova | patrykw o/ | 15:05 |
| regXboi | ok, did a quick scan :) | 15:06 |
| DinaBelova | so quick description of what happened on summit and what feedback was collected | 15:06 |
| DinaBelova | the main input from different community people on the summit for this team was to start with common methodologies, tool sets, ways to compare different deployments, etc. and eventually work on recommendations for projects, writing up testing methodologies and best deployment practices, and share results of the testing. | 15:06 |
| DinaBelova | so we had many people, many tools, lots scenarios, etc. | 15:06 |
| DinaBelova | and I decided to start with basic standardisation | 15:07 |
| DinaBelova | like "what tools to name as the canonical ones?" | 15:07 |
| DinaBelova | etc. | 15:07 |
| regXboi | I guess I'd ask "what do people mean by performance?" as that word means different things to different people | 15:07 |
| DinaBelova | yes, that's true | 15:08 |
| kun_huang | regXboi: agreed... | 15:08 |
| regXboi | (sorry for the meta question) | 15:08 |
| DinaBelova | at mirantis we use to measure two things during the performance testing | 15:08 |
| DinaBelova | control plane performance (operations over OpenStack resources) | 15:08 |
| kun_huang | perf = benchmark, tracing, tuning and so on | 15:08 |
| kun_huang | there're a lots of tools on for different purposes | 15:09 |
| regXboi | DinaBelova: ok, so what do you mean by "control plane performance?" | 15:09 |
| badari | DinaBelova: Control plane performance without any data plane load at all? | 15:09 |
| DinaBelova | like what time it'll take to boot Vm, assign floating IP, etc. in different load | 15:09 |
| kun_huang | rally is good at benchmark of API level | 15:09 |
| DinaBelova | and data plane separately :) | 15:09 |
| Kristian_ | we use Rally now and modify some of the out of the box .json scenarios according to our architecture :) | 15:09 |
| DinaBelova | kun_huang I'll agree | 15:09 |
| DinaBelova | cool :) | 15:09 |
| Kristian_ | mainly because its Free and we are saving on test tool licensing. but it is a flexible, great too;l | 15:10 |
| regXboi | well, I'll say I take a slightly different view of control plane performance | 15:10 |
| augiemena3 | we use Rally as well and make modifications as well for our use | 15:10 |
| DinaBelova | and for data plane we use Shaker https://github.com/openstack/shaker (Functionality is being moved from Shaker to Rally) | 15:10 |
| Kristian_ | we are looking at jmeter next for load/VM testing as well | 15:10 |
| DinaBelova | regXboi - please share the idea | 15:10 |
| Kristian_ | @DinaBelova good news! | 15:10 |
| andreykurilin | hi all:) | 15:10 |
| regXboi | I tend to look *very closely* at execution time of operations in the agents/API/etc | 15:10 |
| kun_huang | Kristian_: is there some good points on jmeter other than rally? | 15:10 |
| regXboi | because at it's core - the messaging RPC is just a queuing system | 15:11 |
| regXboi | er queueing | 15:11 |
| DinaBelova | well, it looks like I need to change topic and return back to something else in open discussion | 15:11 |
| DinaBelova | #topic Rally as a control plane benchmarking tool agreement | 15:11 |
| patrykw | in our team we have different approach, we are simulating LOB applications on cloud and "simulating real" (strange :)) workload by gatling | 15:11 |
| augiemena3 | I do think it's important to categorize the performance tools: benchmarks, benchmark harnesses, workloads, profiling/tracing, monitoring, etc | 15:11 |
| Kristian_ | @regXboi we are looking at it for virtual machine testing, something Rally does not yet do | 15:12 |
| kun_huang | augiemena3: +1 | 15:12 |
| patrykw | applications do not stressing the resources on max | 15:12 |
| Kristian_ | we need something open-source based and easier to learn, community supported as well | 15:12 |
| DinaBelova | augiemena3 +1 | 15:12 |
| regXboi | ok, so Rally is very good at supplying API load, which lets you stress the API and the input to the RPC | 15:12 |
| DinaBelova | Kristian_ - it's going to support that type of testing in future | 15:12 |
| patrykw | plus during the test checking with rally how the API is working | 15:12 |
| DinaBelova | Kristian_ - workloads testing | 15:13 |
| DinaBelova | andreykurilin - am I right? | 15:13 |
| regXboi | but AFAICT, OpenStack lacks the ability to report on execution time in the agents | 15:13 |
| kun_huang | regXboi: if you just need execution time on agent and rpc level, there would be a lot of tools | 15:13 |
| regXboi | and note: I'm not just saying "profiling" | 15:13 |
| andreykurilin | DinaBelova: yes | 15:13 |
| *** rohanion has joined #openstack-performance | 15:13 | |
| rohanion | hi everyone | 15:13 |
| kun_huang | regXboi: I think boris' osprofiler is design for that | 15:13 |
| DinaBelova | ok, so we can assume Rally is good for the API testing | 15:13 |
| Kristian_ | @DinaBelova that's right, workload as well as VM characteristics measurements we are looking at. we need to develop SLAs in the integrated cloud , and that is difficult to do w/o a standard set of measurement | 15:14 |
| patrykw | +1 | 15:14 |
| andreykurilin | DinaBelova: kun_huang is also core of rally, so he can answer to all rally related question too :) | 15:14 |
| DinaBelova | for profiling and tracing we can eventually use osprofiler | 15:14 |
| kun_huang | regXboi: but I just read his writings without real test | 15:14 |
| regXboi | kun_huang: it's possible, but it has to be something built into the system and turned off and on with configuration | 15:14 |
| regXboi | DinaBelova: please note what I just said :) | 15:14 |
| DinaBelova | and eventually rally will support dataplane as well | 15:14 |
| DinaBelova | yep, sorry :) | 15:14 |
| kun_huang | regXboi: haha, have you tried systemtap/ftrace? | 15:14 |
| *** amaretskiy1 has joined #openstack-performance | 15:14 | |
| DinaBelova | regXboi - that's true, although if it will be deeply integrated with openstack projects - why not in fact? | 15:15 |
| *** oanufriev has joined #openstack-performance | 15:15 | |
| regXboi | DinaBelova: because I'm trying to get to the point where I can do continuous test for performance | 15:15 |
| regXboi | and I think that means the tools have to be there and turned on/off via a simple config switch | 15:16 |
| kun_huang | regXboi: "continuous test for performance" +1 for this good point | 15:16 |
| regXboi | er sorry ... tools was a bit of a misnomer | 15:16 |
| DinaBelova | regXboi - so you want like one place to have this swithed on/off | 15:16 |
| regXboi | DinaBelova: so let me give you an example: https://etherpad.openstack.org/p/hyper-scale | 15:17 |
| regXboi | right now to do that, I'm having to decorate files manually to get increasing levels of execution time data | 15:17 |
| DinaBelova | looking | 15:17 |
| regXboi | and anything manual doesn't work | 15:17 |
| DinaBelova | yeah, that's true.. | 15:17 |
| DinaBelova | well, it's one more moment to define | 15:18 |
| regXboi | so whatever we use for execution time data has to be built in and has to be easily turned on and off so that we don't pollute logs | 15:18 |
| regXboi | once you *have* that | 15:18 |
| regXboi | then we get into the whole statistical tool discussion | 15:18 |
| DinaBelova | although it more related to the tracing and profiling | 15:18 |
| DinaBelova | if we are talking about API benchmarking Rally is pretty good here | 15:18 |
| DinaBelova | I suppose we all can agree hee | 15:18 |
| DinaBelova | here* | 15:18 |
| regXboi | as I said, the RPC is essentially a queueing system | 15:19 |
| regXboi | Rally drives the arrival rate | 15:19 |
| DinaBelova | yes, so we need to extend it with other tools, but I suppose we still need it | 15:19 |
| regXboi | but something needs to document the processing time so that you know what's causing you problems if the RPC starts to break | 15:19 |
| DinaBelova | agree | 15:19 |
| regXboi | s/if/when/ | 15:19 |
| DinaBelova | :d | 15:19 |
| DinaBelova | :D | 15:19 |
| DinaBelova | yes, that's true | 15:19 |
| regXboi | so... I agree that Rally is great for the front end :) | 15:20 |
| patrykw | testing data plane with Rally ?? really ? | 15:20 |
| DinaBelova | and we need something else for deeper analysis | 15:20 |
| DinaBelova | patrykw - eventually :) | 15:20 |
| *** harlowja_at_home has joined #openstack-performance | 15:20 | |
| DinaBelova | we have rally contributor here :) | 15:20 |
| harlowja_at_home | DinaBelova, i made it! | 15:20 |
| harlowja_at_home | better late than never :-P | 15:20 |
| DinaBelova | harlowja_at_home o/ | 15:20 |
| harlowja_at_home | :) | 15:20 |
| harlowja_at_home | only half zombine | 15:20 |
| DinaBelova | patrykw - andreykurilin is Rally core | 15:20 |
| Kristian_ | question: does Rally log all API objects it creates/destroys (i believe it does) and does it have a separate log report, similar to the reports that it comes pre-packaged with? | 15:20 |
| kun_huang | regXboi: https://github.com/openstack/scalpels/blob/master/scripts/rbt-trace.py at least, this script could be used RPC tracing | 15:20 |
| * regXboi looks | 15:21 | |
| DinaBelova | Kristian_ - yep, it logs, yes it has separated log report for everything it does | 15:21 |
| regXboi | kun_hunag: I'm looking to go deeper | 15:21 |
| regXboi | that will tell me about the RabbitMQ, which is certainly useful | 15:21 |
| andreykurilin | DinaBelova: we have a lot of rally contributors here: kun_huang, oanufrief | 15:21 |
| DinaBelova | kun_huang - thanks for the link, I'll need to go through it further | 15:22 |
| Kristian_ | @DinaBelova thank you! looking for a runtime type of report | 15:22 |
| DinaBelova | andreykurilin - yep, sure | 15:22 |
| andreykurilin | :) | 15:22 |
| regXboi | but I'm looking to root cause the delays in processing to get them fixed in the code that serves the RPC | 15:22 |
| *** patrykw_ has joined #openstack-performance | 15:22 | |
| kun_huang | regXboi: I use dtrace to catch python function call time... | 15:22 |
| regXboi | kun_huang: automagically? | 15:23 |
| kun_huang | but that is not implemented in Ubuntu | 15:23 |
| DinaBelova | so I think we can assume the following: we have Rally now for front-end API testing, it will be able to do dataplane with its help in future, and we need something else to catch what's going on deeper while request processing | 15:23 |
| Kristian_ | @kun_huang does it provide stress against RabbitMQ? | 15:23 |
| kun_huang | regXboi: script only, I'm working on my prototype | 15:23 |
| andreykurilin | DinaBelova: +1 | 15:23 |
| DinaBelova | in fact it's not only RPC, it's various drivers work, it's DB performance, etc | 15:23 |
| kun_huang | Kristian_: that rbt-trace.py, don't; it is just rpc log | 15:23 |
| DinaBelova | so it looks like it'll be some number of additional tools | 15:24 |
| andreykurilin | osprofiler can be really useful for profiling/tracing stuff, but it need more contribution:( | 15:24 |
| Kristian_ | i see. because that is something also we are looking for, as a stressful/load component against RPC queues and RabbitMQ | 15:24 |
| kun_huang | andreykurilin: yep, it is | 15:24 |
| *** patrykw has quit IRC | 15:24 | |
| DinaBelova | kun_huang, Kristian_, regXboi - would you agree with the points I mentioned? | 15:24 |
| kun_huang | andreykurilin: about osprofiler, I think boris need bring more and good demo for users | 15:24 |
| kun_huang | DinaBelova: I agree with Rally is a good openstack benchmark framwork | 15:25 |
| * harlowja_at_home will try to work on osprofiler stuff soon, will do some tweaks to it maybe later this week... | 15:25 | |
| * DinaBelova as well | 15:25 | |
| andreykurilin | kun_huang: the main problem of osprofiler is a cores of other projects, since osprofiler needs integration with them | 15:25 |
| andreykurilin | harlowja_at_home: nice | 15:25 |
| regXboi | DinaBelova: I would summarize as a profiler that is built into all OpenStack projects (likely via oslo) | 15:25 |
| Kristian_ | yes, we need some community tools that do back-end stress to the VMs and databases w/ a front-end manager | 15:25 |
| regXboi | and if osprofiler fits the bill, sure | 15:26 |
| kun_huang | andreykurilin: I think we should educate them, there fantastic tracing tool has to built into native codes | 15:26 |
| DinaBelova | regXboi - agreed | 15:26 |
| harlowja_at_home | andreykurilin, yes that, we need to make a joint effort to fix what cores have complained about and figure out why its not integrated yet... | 15:26 |
| kun_huang | s/there/every | 15:26 |
| DinaBelova | ok, let me write something with >agreed< tag | 15:26 |
| regXboi | is osprofiler part of oslo? | 15:26 |
| harlowja_at_home | kun_huang, +1 education, and working through the issues they have with it... | 15:26 |
| regXboi | if not, it should be | 15:26 |
| kun_huang | regXboi: nope | 15:26 |
| harlowja_at_home | regXboi, not yet | 15:26 |
| regXboi | er it *must* be | 15:26 |
| regXboi | ok, that's #1 | 15:26 |
| patrykw_ | ku_huang: from the perspecive of end user, which using VM, the numbers from Rally is what he wanted to see ??, or he just want to have information how well the VM is working and plus if infrastructure is fine | 15:26 |
| andreykurilin | harlowja_at_home: as far as I remember, they were against check `if profiler == None`, since it can produce performance issues :D | 15:27 |
| harlowja_at_home | andreykurilin, yes, we have to bust that kind of dumb stuff | 15:27 |
| harlowja_at_home | but in a politically nice way, ha | 15:27 |
| kun_huang | patrykw_: I think you are looking for a "vm workload benchmark framework" | 15:27 |
| DinaBelova | #agreed Rally is good benchmarking solution that we want to use widely for OpenStack API benchmarking and dataplane testing in future. Although we need something else for profiling that is built into all OpenStack projects | 15:27 |
| kun_huang | patrykw_: I'm not sure ;( | 15:27 |
| augiemena3 | @DinaBelova +1 | 15:28 |
| patrykw_ | kun_huang: what we want to benchmark ?? only APIs or something more ?? | 15:28 |
| andreykurilin | kun_huang, patrykw_: we have serie of patches for "vm workload", but they need more reviewers | 15:28 |
| andreykurilin | *we = Rally | 15:28 |
| patrykw_ | :) | 15:28 |
| kun_huang | patrykw_: both API level benchmark and vm workload benchmark | 15:28 |
| regXboi | DinaBelova, can we go further and say the profiling has to be controlled via config as well as by decoration? | 15:28 |
| DinaBelova | regXboi - I'll agree here | 15:28 |
| regXboi | I mean, I'd like to have decoration for one offs :) | 15:29 |
| regXboi | but CT says config (unless somebody can find a better solution) | 15:29 |
| DinaBelova | we need single opportunity to switch this possibility on/off | 15:29 |
| patrykw_ | andreykurilin: it is fine to use Rally to test API, we are doing this same but vm workload is something else | 15:29 |
| DinaBelova | patrykw_ - this is just new commits and new Rally functinality | 15:29 |
| DinaBelova | that's in progress now :) | 15:30 |
| andreykurilin | patrykw_: What is the difference?) | 15:30 |
| kun_huang | patrykw_: yes it is... | 15:30 |
| harlowja_at_home | regXboi, as for oslo, let's work on that, and chat with dims and such, i'd like to capture a list of why it isn't integrated into say nova, keystone... and attempt to resolve those (either by dispelling myths or fixing code) before osprofiler ---> oslo though.... | 15:30 |
| Kristian_ | @DinaBelova - we also need a way to isolate troublesome VMs from a test, that it picks up from statistics during runtime by IP/name/node identification | 15:30 |
| regXboi | patrykw_: I suspect that we need to come up with a common vm workload to handle that stressing | 15:30 |
| DinaBelova | harlowja_at_home +1 | 15:30 |
| regXboi | harlowja_at_home: well, that's a bit of a chicken and egg problem | 15:30 |
| DinaBelova | Kristian_ - I suppose eventually it's possible, although not sure how to do that right now | 15:31 |
| patrykw_ | DinaBelova: want to see that :) currently we are using mix of PKB + gatling to achieve some level of workload | 15:31 |
| regXboi | I'd like to have that conversation begin with "this is going into oslo, and you really need to integrate to support CT, so what can we do to help with that" | 15:31 |
| DinaBelova | patrykw_ brrr... | 15:31 |
| kun_huang | I think today's meeting is one hour open discussion ;) | 15:31 |
| DinaBelova | yep, it'll be very cool to have everything in one box | 15:31 |
| andreykurilin | kun_huang: :D | 15:31 |
| Kristian_ | @DinaBelova as long as we can get requirement in for future development in Rally or a back-end version of Rally early, i'm fine w/ that answer | 15:31 |
| Kristian_ | :) | 15:31 |
| DinaBelova | kun_huang, andreykurilin - do you have links to the workloads patches? | 15:32 |
| andreykurilin | regXboi: there was a spec of moving osprofiler to oslo namespace | 15:32 |
| DinaBelova | kun_huang - well, yes | 15:32 |
| andreykurilin | DinaBelova: one moment please | 15:32 |
| DinaBelova | I was too stupid to create an agenda | 15:32 |
| harlowja_at_home | regXboi, sure, i think some of the issues that will be captured aren't 'it isn't in oslo so i won't integrate it' | 15:32 |
| regXboi | andreykurilin: is there a #link? | 15:32 |
| DinaBelova | andreykurilin sure | 15:32 |
| kun_huang | #link https://review.openstack.org/#/q/status:open+project:openstack/rally+branch:master+topic:bp/vm-workloads-framework,n,z | 15:32 |
| DinaBelova | kun_huang thank you sir | 15:32 |
| andreykurilin | :) | 15:32 |
| regXboi | harlowja_at_home: yes I don't want to be blocked by that quote :) | 15:32 |
| harlowja_at_home | regXboi, we can resolve that quote if it happens rather quickly imho... | 15:33 |
| Kristian_ | @DinaBelova you were asking about other tools, we are using jmeter and looking at traffic engineering with SmartBits iTest / Hyperscale solution for scale test | 15:33 |
| harlowja_at_home | but saying something like 'shutup u crazy person, it will go into oslo once we resolve the other issues' | 15:33 |
| harlowja_at_home | ^ not exactly like that , ha | 15:33 |
| Kristian_ | we don't have a PoC in place yet | 15:33 |
| andreykurilin | regXboi: https://review.openstack.org/#/c/103825/ | 15:33 |
| DinaBelova | Kristian_ - a-ha, thank you sir | 15:33 |
| regXboi | harlowja_at_home: yes, that isn't political (rotflmao) | 15:33 |
| harlowja_at_home | :) | 15:33 |
| andreykurilin | regXboi: as you can see there were a lot of + for osprofiler:) | 15:33 |
| regXboi | andreykurilin: on my list to read and comment on today | 15:34 |
| regXboi | and thx :) | 15:34 |
| kun_huang | andreykurilin: I'm 100% sure that osprofiler will be a good solution | 15:34 |
| patrykw_ | i was working on other cloud solutions and main think always was how much load (instances) I can put without issues, issues to the resources, and issues to the management plane | 15:34 |
| DinaBelova | so let's write something in OSprofiler topic to have beautiful notes | 15:34 |
| regXboi | DinaBelova: I assume we want to put of the "statistical" discussion for another day? | 15:34 |
| DinaBelova | #topic Taking part in the OSprofiler initiative | 15:34 |
| kun_huang | google's dapper and twitter's zikpin examples | 15:34 |
| DinaBelova | regXboi - I suppose yes | 15:34 |
| regXboi | er off | 15:34 |
| andreykurilin | btw, everyone saw the rexample of osprofiler reports? http://boris-42.github.io/ngk.html | 15:35 |
| regXboi | DinaBelova: I'm good with thta :) | 15:35 |
| * regXboi looking | 15:35 | |
| patrykw_ | will the rally tell us: hold done, you are going to far ?? :) | 15:35 |
| DinaBelova | so it looks like during Mitaka there will be some activity on making osprofiler workable enough for all projects | 15:35 |
| DinaBelova | harlowja_at_home, me and Boris at least will try to do that | 15:35 |
| andreykurilin | DinaBelova: don't forget about me:) | 15:36 |
| DinaBelova | and if this solution will be fully ok to be used for tracing, I believe it'll become an oslo standard | 15:36 |
| andreykurilin | I'll try to help too | 15:36 |
| regXboi | does osprofiler only do db and rpc or can it go to specific python functions? | 15:36 |
| DinaBelova | andreykurilin sorry sir :) | 15:36 |
| kun_huang | regXboi: yep | 15:36 |
| regXboi | kun_hunag: yep == ??? | 15:36 |
| harlowja_at_home | regXboi, == yes | 15:36 |
| andreykurilin | :) | 15:36 |
| regXboi | no, I asked an A or B - > yes doesn't answer | 15:36 |
| kun_huang | regXboi: osprofiles covers db level things | 15:36 |
| regXboi | ok, and RPC | 15:37 |
| Kristian_ | will chk out osprofiler, thx for the tool tip @andreykurilin | 15:37 |
| kun_huang | regXboi: in document, it covers too... | 15:37 |
| regXboi | but *does it cover specific functions in the agent code* | 15:37 |
| kun_huang | regXboi: no it doesn't | 15:37 |
| harlowja_at_home | if u make it do so :) | 15:37 |
| DinaBelova | harlowja_at_home ++ | 15:37 |
| regXboi | ok... *that's* a problem | 15:37 |
| regXboi | that means osprofiler doesn't in its current form go deep enough | 15:37 |
| harlowja_at_home | it doesn't integrate with pdb to automatically do all the things | 15:38 |
| DinaBelova | I thing it can be developed after making current unctioanlity workable | 15:38 |
| harlowja_at_home | its not magic ;) | 15:38 |
| kun_huang | regXboi: that's the reason I am using systemtap | 15:38 |
| DinaBelova | regXboi - right now - yes | 15:38 |
| DinaBelova | but if we want unified way of tracing, we need to work on this... | 15:38 |
| andreykurilin | regXboi: I suppose osprofiler can be a base for good profiling tool, but it need some work on it | 15:38 |
| DinaBelova | regXboi - otherwise everyone will be using something they use to have and the results won't be comparable at all | 15:39 |
| regXboi | all - I'm not saying we shouldn't improve it, I'm just working to understand what it would need for this | 15:39 |
| andreykurilin | :) | 15:39 |
| regXboi | or I should say what I think it would need for this | 15:39 |
| DinaBelova | and that's something I want to avoid | 15:39 |
| DinaBelova | sure | 15:39 |
| DinaBelova | regXboi - I'm very glad to collect your feedback, it's very useful :) | 15:39 |
| regXboi | and believe me, I'll make these comments on that commit :) | 15:40 |
| regXboi | because what I see, I *mostly* like | 15:40 |
| DinaBelova | :D | 15:40 |
| regXboi | but it needs some more pieces to make it sing | 15:40 |
| regXboi | another question - are we sold on Ceilometer only for reporting? what about simple logs? | 15:40 |
| DinaBelova | ok, so it looks like we have solid opinion here - OSprofiler may become one day the tracing / deeper analysis tool, but it requires some work to be done | 15:41 |
| andreykurilin | regXboi: it supports text files | 15:41 |
| regXboi | andreykurilin: ok, I missed that on the quick scan | 15:41 |
| andreykurilin | also, there is a spec for multi backend - https://github.com/openstack/osprofiler/blob/master/doc/specs/in-progress/multi_backend_support.rst | 15:41 |
| DinaBelova | regXboi - one of the boris-42 concerns was to make more collector backends, but simple text was available afair | 15:41 |
| andreykurilin | #link http://lists.openstack.org/pipermail/openstack-dev/2015-November/078845.html | 15:42 |
| DinaBelova | #link https://github.com/openstack/osprofiler/blob/master/doc/specs/in-progress/multi_backend_support.rst | 15:42 |
| andreykurilin | the latest thread related to osprofiler | 15:42 |
| DinaBelova | nd all specs that need to be covered at least are placed there as well | 15:42 |
| regXboi | andreykurilin: yes I saw that :) | 15:42 |
| DinaBelova | ok, so it looks like we're done with osprofiler | 15:43 |
| DinaBelova | any more notes about it? | 15:43 |
| DinaBelova | ok | 15:43 |
| DinaBelova | #topic Data plane testing tools | 15:43 |
| andreykurilin | rally! | 15:43 |
| andreykurilin | :D | 15:43 |
| DinaBelova | ok, so later rally will support it | 15:43 |
| DinaBelova | ;) | 15:43 |
| andreykurilin | ;( | 15:43 |
| DinaBelova | Kristian_ is using jmeter and looking at traffic engineering with SmartBits iTest / Hyperscale solution for scale test | 15:43 |
| DinaBelova | for some of the bits | 15:44 |
| DinaBelova | Mirantis uses shaker right now | 15:44 |
| badari | DinaBelova: we are looking at using "cbtool" currently | 15:44 |
| kun_huang | I know vmtp/kloudbuster | 15:44 |
| DinaBelova | #link https://github.com/openstack/shaker | 15:44 |
| DinaBelova | ok, so it looks like until we'll have nice Rally solution it'll be a complete mess here :) | 15:44 |
| regXboi | So, here is where I think it may make sense to come up with a standard image that can be bundled with OpenStack? | 15:44 |
| DinaBelova | regXboi - that will be very logical I believe | 15:45 |
| Kristian_ | @DinaBelova can u explain to me in a nutshell what shaker does? is it L2/L3 stress tool? | 15:45 |
| patrykw_ | regXboi :) | 15:45 |
| DinaBelova | Kristian_ - in short, yes | 15:45 |
| regXboi | DinaBelova: I thought I saw somebody suggest that earlier in the thread (or get most of the way there) | 15:45 |
| Kristian_ | just heard of it so will read the docs shortly :) | 15:45 |
| kun_huang | haha | 15:45 |
| DinaBelova | Kristian_ - currently it's only about networking | 15:46 |
| DinaBelova | and no need to make it wider due to the current Rally changes | 15:46 |
| DinaBelova | regXboi - can you find a thred please? | 15:46 |
| Kristian_ | so shaker only touches/focuses on neutron component? | 15:46 |
| regXboi | DinaBelova: it was in this meeting... let me go back and look | 15:46 |
| DinaBelova | regXboi, oh, I missedit :) | 15:46 |
| DinaBelova | Kristian_ - yes, that's true :) | 15:47 |
| regXboi | It was patrykw_ that got most of the way there: http://eavesdrop.openstack.org/irclogs/%23openstack-performance/%23openstack-performance.2015-11-10.log.html#t2015-11-10T15:29:35 | 15:48 |
| DinaBelova | a-ha, thank you sir | 15:48 |
| DinaBelova | #link http://eavesdrop.openstack.org/irclogs/%23openstack-performance/%23openstack-performance.2015-11-10.log.html#t2015-11-10T15:29:35 | 15:48 |
| patrykw_ | yep, for data plane we are using Gatling + PKB | 15:48 |
| DinaBelova | ok, so it looks like we need to enforce standard image creation/assumption | 15:49 |
| DinaBelova | and then try to understand if we can compare at least somehow currently gathered data | 15:49 |
| regXboi | if the image has what is needed for stressing of the various components, that makes sense | 15:49 |
| patrykw_ | as data source mediawiki + magentoshop | 15:49 |
| DinaBelova | patrykw_ - interesting, thank you sir | 15:50 |
| DinaBelova | patrykw_ - do you have this data somewhere published? | 15:50 |
| augiemena3 | Badari: yes, https://github.com/ibmcb/cbtool | 15:50 |
| DinaBelova | patrykw_ - or is it private info? | 15:50 |
| patrykw_ | DinaBelova: not yet, currently gathering multinoda data with top bin CPUs | 15:51 |
| patrykw_ | "multinode" | 15:51 |
| DinaBelova | patrykw_ - will you be able to share the results once they'll be collected? I mean, it's interesting experience | 15:51 |
| DinaBelova | and personally I would love to take a look on the results | 15:51 |
| kun_huang | okay guys, this meeting is close to end; I'm working on my prototype, scalpels, which is debugfs system with systemtap/ftrace support, if you're interested in this, welcome to #openstack-scalpels | 15:52 |
| DinaBelova | #topic Open Discussion | 15:52 |
| patrykw_ | DinaBelova: probably yes, and probably the automated scripts | 15:52 |
| DinaBelova | patrykw_ - thank you sir | 15:53 |
| DinaBelova | kun_huang - thanks, will join | 15:53 |
| DinaBelova | I would ask everyone here who uses rally to prepare and send to this channel scenarios you use to run | 15:53 |
| regXboi | where did the hour go :) | 15:54 |
| DinaBelova | regXboi :D | 15:54 |
| DinaBelova | #action everyone who uses Rally - please share the scenarios you're running | 15:54 |
| patrykw_ | DinaBelova: +1 good point | 15:54 |
| DinaBelova | I think we can try to unify the scenarios list | 15:54 |
| regXboi | DinaBelova: can we start an etherpad for that? | 15:55 |
| DinaBelova | I think we can use etherpads or whatever for this purpose | 15:55 |
| kun_huang | using rally to run complex load to openstack | 15:55 |
| DinaBelova | https://etherpad.openstack.org/p/rally_scenarios_list | 15:55 |
| regXboi | DinaBelova: thx | 15:55 |
| Kristian_ | @DinaBelova - if you need inputs on what can be put into "out-of-the-box" scenarios for testing i can do that collectively here @AT^T | 15:55 |
| DinaBelova | I've quickly copy-passed the scenarios names, but I'll add their configs somewhere too | 15:55 |
| DinaBelova | Kristian_ - this will be perfect!!!! | 15:56 |
| Kristian_ | for Rally testing in particular | 15:56 |
| DinaBelova | Kristian_ - thank you sir | 15:56 |
| Kristian_ | yw :) | 15:56 |
| DinaBelova | #action Kristian_ - have internal att session to collect feedback on what can be added to out-of-box- rally | 15:57 |
| DinaBelova | ok, so it looks like we're running out of time | 15:57 |
| DinaBelova | any specific points to cover? | 15:57 |
| regXboi | other than another channel or two to watch? no :) | 15:57 |
| DinaBelova | ;D | 15:57 |
| Kristian_ | @DinaBelova and group - will do | 15:57 |
| kun_huang | DinaBelova: we need a focus next time | 15:57 |
| regXboi | yep, time to post an agenda :) | 15:58 |
| DinaBelova | kun_huang - that's true | 15:58 |
| kun_huang | on benchmark topic, or tracing, or people's need | 15:58 |
| Kristian_ | agenda would be nice, agreed | 15:58 |
| kun_huang | it helps us to find more details | 15:58 |
| DinaBelova | please feel free to clear https://wiki.openstack.org/wiki/Meetings/Performance#Agenda_for_next_meeting and fill with your items | 15:58 |
| DinaBelova | :) | 15:58 |
| Kristian_ | context would be good too. performance testing means different things to different engineers/techies :P | 15:59 |
| *** rohanion has left #openstack-performance | 15:59 | |
| DinaBelova | ok, thanks everyone, and let's have a bit clearer meeting next tim :) | 15:59 |
| DinaBelova | #endmeeting | 15:59 |
| patrykw_ | http | 15:59 |
| openstack | Meeting ended Tue Nov 10 15:59:19 2015 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:59 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/performance_team/2015/performance_team.2015-11-10-15.00.html | 15:59 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/performance_team/2015/performance_team.2015-11-10-15.00.txt | 15:59 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/performance_team/2015/performance_team.2015-11-10-15.00.log.html | 15:59 |
| *** Kristian_ has quit IRC | 15:59 | |
| DinaBelova | I've cleared agenda for the next meeting | 16:00 |
| DinaBelova | please feel free to fill it | 16:00 |
| *** jeffcb has quit IRC | 16:00 | |
| *** bapalm has joined #openstack-performance | 16:02 | |
| *** augiemena3 has quit IRC | 16:04 | |
| *** listomin has quit IRC | 16:09 | |
| *** mdorman has joined #openstack-performance | 16:24 | |
| *** patrykw_ has quit IRC | 16:33 | |
| *** harlowja_at_home has quit IRC | 16:42 | |
| *** wootehfoot has joined #openstack-performance | 16:57 | |
| *** wootehfoot has quit IRC | 17:00 | |
| SpamapS | DinaBelova: sorry I could not join the meeting. 0800 Pacific is basically never going to be free for me (taking kids to school.. :-P ) | 17:03 |
| SpamapS | so I can join whe daylight savings time happens again and it is 0900 | 17:04 |
| SpamapS | Will try to find a more-eastern colleague to represent me. ;) | 17:04 |
| *** amaretskiy1 has quit IRC | 17:11 | |
| *** badari has quit IRC | 17:29 | |
| DinaBelova | SpamapS, np :) | 17:40 |
| DinaBelova | thanks for letting know :) | 17:40 |
| *** manand has joined #openstack-performance | 17:53 | |
| *** harshs has joined #openstack-performance | 18:23 | |
| *** wootehfoot has joined #openstack-performance | 18:27 | |
| *** ilyashakhat_mobi has joined #openstack-performance | 18:35 | |
| *** Guest11617 has quit IRC | 18:44 | |
| *** gals has quit IRC | 19:20 | |
| *** wootehfoot has quit IRC | 19:39 | |
| *** wootehfoot has joined #openstack-performance | 19:56 | |
| *** deepika has quit IRC | 20:04 | |
| *** badari has joined #openstack-performance | 20:26 | |
| *** ilyashakhat_mobi has quit IRC | 20:30 | |
| *** wootehfoot has left #openstack-performance | 20:34 | |
| regXboi | DinaBelova: we need to get that osprofiler oslo-spec resubmitted for Mitaka - who should take that action item? | 20:51 |
| *** rvasilets___ has joined #openstack-performance | 21:48 | |
| *** boris-42 has joined #openstack-performance | 22:30 | |
| *** regXboi has quit IRC | 22:33 | |
| *** harshs has quit IRC | 22:52 | |
| notmorgan | DinaBelova: ooh meeting is 0800 | 23:40 |
| notmorgan | ? | 23:40 |
| notmorgan | doh | 23:40 |
| *** arnoldje has quit IRC | 23:40 | |
| *** manand has quit IRC | 23:49 | |
| *** mdorman has quit IRC | 23:50 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!