| *** yujunz has joined #openstack-performance | 00:33 | |
| *** tovin07_ has joined #openstack-performance | 00:51 | |
| *** tovin07 has joined #openstack-performance | 01:42 | |
| *** zz_dimtruck is now known as dimtruck | 01:57 | |
| *** masber has quit IRC | 02:51 | |
| *** masber has joined #openstack-performance | 02:51 | |
| *** yujunz has quit IRC | 03:02 | |
| *** yujunz has joined #openstack-performance | 03:09 | |
| *** aojea has joined #openstack-performance | 03:31 | |
| *** aojea has quit IRC | 03:35 | |
| *** yujunz has quit IRC | 04:01 | |
| *** dimtruck is now known as zz_dimtruck | 05:12 | |
| *** yujunz has joined #openstack-performance | 05:31 | |
| *** yujunz has quit IRC | 05:52 | |
| *** yujunz has joined #openstack-performance | 05:58 | |
| *** faizy has joined #openstack-performance | 06:35 | |
| *** aojea has joined #openstack-performance | 06:38 | |
| *** yujunz has quit IRC | 06:54 | |
| *** yujunz has joined #openstack-performance | 07:15 | |
| *** pcaruana has joined #openstack-performance | 07:50 | |
| *** yujunz-zte has joined #openstack-performance | 07:50 | |
| *** yujunz has quit IRC | 07:52 | |
| *** yujunz-zte has quit IRC | 08:02 | |
| *** furlongm has quit IRC | 08:35 | |
| *** rmart04 has joined #openstack-performance | 09:07 | |
| *** yee379 has quit IRC | 09:20 | |
| *** yee379 has joined #openstack-performance | 09:20 | |
| *** klindgren_ has joined #openstack-performance | 09:32 | |
| *** abalutoiu_ has quit IRC | 09:34 | |
| *** klindgren has quit IRC | 09:35 | |
| *** aojea has quit IRC | 10:00 | |
| *** tovin07_ has quit IRC | 10:03 | |
| *** aojea has joined #openstack-performance | 10:21 | |
| *** aojea has quit IRC | 10:26 | |
| *** jkilpatr has quit IRC | 10:39 | |
| *** klindgren__ has joined #openstack-performance | 10:43 | |
| *** abalutoiu has joined #openstack-performance | 10:44 | |
| *** klindgren_ has quit IRC | 10:45 | |
| *** openstackgerrit has joined #openstack-performance | 11:10 | |
| openstackgerrit | Oleg Basov proposed openstack/performance-docs master: Influx HA as an external storage for Prometheus https://review.openstack.org/442020 | 11:10 |
|---|---|---|
| *** abalutoiu has quit IRC | 11:15 | |
| *** bvanhav has joined #openstack-performance | 11:18 | |
| *** jkilpatr has joined #openstack-performance | 11:19 | |
| *** jkilpatr has quit IRC | 11:27 | |
| openstackgerrit | Oleg Basov proposed openstack/performance-docs master: Influx HA as an external storage for Prometheus https://review.openstack.org/442020 | 11:34 |
| *** jkilpatr has joined #openstack-performance | 11:40 | |
| *** faizy has quit IRC | 11:40 | |
| *** faizy has joined #openstack-performance | 12:25 | |
| *** abalutoiu has joined #openstack-performance | 12:26 | |
| *** rfolco has joined #openstack-performance | 12:29 | |
| *** faizy has quit IRC | 12:34 | |
| *** aojea has joined #openstack-performance | 12:45 | |
| *** abalutoiu has quit IRC | 12:57 | |
| *** catintheroof has joined #openstack-performance | 12:58 | |
| *** catintheroof has quit IRC | 12:59 | |
| *** catintheroof has joined #openstack-performance | 12:59 | |
| *** bvanhav_ has joined #openstack-performance | 13:11 | |
| *** markvoelker has quit IRC | 13:11 | |
| *** bvanhav has quit IRC | 13:11 | |
| *** aojea has quit IRC | 13:15 | |
| *** markvoelker has joined #openstack-performance | 13:16 | |
| *** abalutoiu has joined #openstack-performance | 13:21 | |
| *** faizy has joined #openstack-performance | 13:21 | |
| *** catinthe_ has joined #openstack-performance | 13:41 | |
| *** catintheroof has quit IRC | 13:41 | |
| *** catintheroof has joined #openstack-performance | 13:45 | |
| *** catinthe_ has quit IRC | 13:45 | |
| *** zz_dimtruck is now known as dimtruck | 14:18 | |
| *** faizy_ has joined #openstack-performance | 14:58 | |
| *** faizy has quit IRC | 15:00 | |
| *** tovin07_ has joined #openstack-performance | 15:11 | |
| *** bvanhav_ has quit IRC | 15:17 | |
| *** ronana has joined #openstack-performance | 15:29 | |
| DinaBelova | #startmeeting Performance Team | 15:30 |
| openstack | Meeting started Tue Mar 14 15:30:03 2017 UTC and is due to finish in 60 minutes. The chair is DinaBelova. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:30 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:30 |
| openstack | The meeting name has been set to 'performance_team' | 15:30 |
| tovin07_ | hi o/ | 15:30 |
| DinaBelova | hello folks! | 15:30 |
| ronana | o/ | 15:30 |
| DinaBelova | let's wait for a few moments :) | 15:30 |
| DinaBelova | ronana tovin07_ o/ | 15:30 |
| DinaBelova | ronana new faces :) | 15:30 |
| *** ronana is now known as rcherrueau | 15:30 | |
| rcherrueau | :) | 15:30 |
| DinaBelova | rcherrueau heh :) | 15:30 |
| DinaBelova | non new :D | 15:30 |
| DinaBelova | akrzos o/ | 15:31 |
| DinaBelova | ok, so let's get started | 15:33 |
| DinaBelova | #topic Action Items | 15:33 |
| *** denaitre has joined #openstack-performance | 15:33 | |
| DinaBelova | the only action item we had was on myself regarding asking Vipin about his future participance in OSprofiler | 15:33 |
| DinaBelova | sadly I had really hard weeks, so I did not manage to do this | 15:33 |
| DinaBelova | #action DinaBelova ping Vipin on his participance in OSprofiler - he seemed not to be really active last few weeks, need to check if he's going to continue work | 15:34 |
| DinaBelova | I'll keep this till next week | 15:34 |
| DinaBelova | so we can jump to the current progress | 15:34 |
| DinaBelova | #topic Current progress on the planned tests | 15:34 |
| DinaBelova | rcherrueau ? | 15:34 |
| tovin07_ | DinaBelova: :D | 15:34 |
| rcherrueau | Nothing really new on our side | 15:34 |
| DinaBelova | rcherrueau ok :) just usual work and smooth progress I guess | 15:35 |
| tovin07_ | last week, I had some disscusions with rcherrueau about kolla | 15:35 |
| *** bvanhav_ has joined #openstack-performance | 15:35 | |
| rcherrueau | We started some test in a WAN environment | 15:35 |
| akrzos | DinaBelova: o/ | 15:35 |
| DinaBelova | tovin07_ indeed. rcherrueau afair tovin07_ has proposed to push BP to kolla regarding osprofiler support | 15:36 |
| rcherrueau | there is a WAN connection between keystone and an OpenStack | 15:36 |
| DinaBelova | rcherrueau ok, so we should expect some results soon :) | 15:36 |
| *** msimonin has joined #openstack-performance | 15:37 | |
| rcherrueau | yes | 15:37 |
| msimonin | all o/ | 15:37 |
| DinaBelova | rcherrueau ok, thank you. What's about pushing new Bp to kolla about osprofiler support? | 15:37 |
| DinaBelova | msimonin o/ | 15:37 |
| rcherrueau | hey msimonin, can you give some info about the WAN experiment? | 15:37 |
| rcherrueau | such as the latency | 15:38 |
| msimonin | sure | 15:38 |
| msimonin | First of all did you know that the summit talk is accepted ? | 15:38 |
| DinaBelova | rcherrueau msimonin btw congrats on your presentation choosing for the summit | 15:38 |
| DinaBelova | ahaha | 15:38 |
| msimonin | heh :) | 15:38 |
| DinaBelova | I'm just in time with my congrats | 15:38 |
| rcherrueau | DinaBelova: thx | 15:38 |
| DinaBelova | #link https://www.openstack.org/summit/boston-2017/summit-schedule/events/17952/toward-fog-edge-and-nfv-deployments-evaluating-openstack-wanwide?BackURL=https%3A%2F%2Fwww.openstack.org%2Fsummit%2Fboston-2017%2Fsummit-schedule%2Fspeakers%2F11238%23eventId%3D17952 | 15:38 |
| msimonin | I guess the talk will be on the methodology we use and some results | 15:39 |
| tovin07_ | congrats rcherrueau | 15:40 |
| DinaBelova | msimonin any details on the experiment for now? | 15:40 |
| rcherrueau | tovin07_: thx | 15:40 |
| DinaBelova | or do you think some more time is needed for the sharing? | 15:40 |
| msimonin | about the experiments we are using some of the rally scenarios with various latency/bandwidth constraints | 15:41 |
| msimonin | and actually | 15:41 |
| DinaBelova | msimonin yes sir? | 15:42 |
| msimonin | the granularity of the results (I mean the timing of the steps) is to big | 15:42 |
| msimonin | so we'll use osprofiler + rally to get more fine grained results | 15:42 |
| DinaBelova | msimonin gotcha | 15:43 |
| DinaBelova | msimonin ack, so I think we need to wait for some results :) hope to see them soon :) | 15:43 |
| msimonin | and extract what operations/functions suffer of high latency | 15:43 |
| rcherrueau | DinaBelova: on that point of osprofiler + rally, some students here change the rally client to an `--os-profile YOUR_KEY` | 15:43 |
| rcherrueau | DinaBelova: to add* | 15:44 |
| rcherrueau | so now we can trace a specific scenario :) | 15:44 |
| tovin07_ | if you use openstackclient, —os-profile is the option | 15:44 |
| tovin07_ | `—os-profile` | 15:44 |
| msimonin | I think they changed in the rally code directly | 15:44 |
| DinaBelova | msimonin I believe you're right | 15:45 |
| rcherrueau | tovin07_: this an implementation of the `--os-profile` into the rally client | 15:45 |
| tovin07_ | oh | 15:45 |
| tovin07_ | got it | 15:45 |
| DinaBelova | rcherrueau I believe not only to the client, but to the rally itself as well :) | 15:45 |
| tovin07_ | it’s strange with other projects | 15:45 |
| DinaBelova | to pass the header further | 15:45 |
| DinaBelova | rcherrueau msimonin anyway, it's a good activity | 15:46 |
| DinaBelova | thanks for the news | 15:46 |
| rcherrueau | DinaBelova: in fact, the change is really small and located into the cli part | 15:46 |
| DinaBelova | #info work on adding osprofiler support to rally has been started by students in Inria to trace specific scenario | 15:46 |
| msimonin | Could it be interesting to push this change to rally ? | 15:46 |
| DinaBelova | msimonin I believe yes, but we need to ping Andrey Kurilin (Rally PTL) | 15:47 |
| DinaBelova | msimonin can you please do this? | 15:47 |
| rcherrueau | tovin07_: yes you are right, we did it like that because it suited our needs, but I'm not sure this makes sens for a PR for instance | 15:47 |
| msimonin | sure | 15:47 |
| msimonin | I can take the action | 15:47 |
| tovin07_ | yup | 15:47 |
| DinaBelova | #action msimonin ping Rally PTL regarding adding osprofiler support to rally code base | 15:47 |
| DinaBelova | msimonin thank you sir | 15:48 |
| msimonin | DinaBelova: ack :) | 15:48 |
| DinaBelova | msimonin rcherrueau anything else to share today? | 15:48 |
| msimonin | maybe one | 15:48 |
| rcherrueau | DinaBelova: just to be clear, the idea is to trace OS based on a scenario executed by, not rally itself | 15:48 |
| DinaBelova | rcherrueau sure, I meant this :) | 15:49 |
| msimonin | we'll add the support for a new provider to Enos : chameleon cloud | 15:49 |
| DinaBelova | rcherrueau "osprofiler support" sounds confusing, I agree | 15:49 |
| DinaBelova | :D | 15:49 |
| msimonin | https://www.chameleoncloud.org/ | 15:49 |
| tovin07_ | yes, too confusing :v | 15:49 |
| DinaBelova | msimonin ack, so your tool is covering more and more :) | 15:50 |
| msimonin | :) | 15:50 |
| msimonin | I guess that's all | 15:51 |
| DinaBelova | ok, akrzos any news? | 15:51 |
| DinaBelova | btw several talks by RedHat is accepted as well if i'm not mistaken | 15:51 |
| akrzos | So Ceilometer Collector appears to be my biggest bottleneck | 15:51 |
| akrzos | Yes | 15:51 |
| DinaBelova | #link https://www.openstack.org/summit/boston-2017/summit-schedule/events/17530/openstack-telemetry-and-the-10000-instances | 15:51 |
| akrzos | Also one lightening talk for browbeat | 15:52 |
| DinaBelova | #link https://www.openstack.org/summit/boston-2017/summit-schedule/events/18245/performance-and-scale-analysis-of-openstack-using-browbeat?BackURL=https%3A%2F%2Fwww.openstack.org%2Fsummit%2Fboston-2017%2Fsummit-schedule%2Fspeakers%2F9457%23eventId%3D18245 | 15:52 |
| DinaBelova | yeah, posted it as well | 15:52 |
| DinaBelova | akrzos I did not go through most recent Ceilo architecture, but that has been an issue since Kilo at least | 15:53 |
| akrzos | I added a collectd plugin for rabbitmq monitoring since ceilometer collector appears to be choking on how many messages it is trying to consume and push into Gnocchi backlog | 15:53 |
| akrzos | also Gnocchi is not behaving well with Ceph | 15:53 |
| akrzos | if it's not configured it will create many small objects | 15:53 |
| akrzos | (16 byte objects) | 15:53 |
| DinaBelova | akrzos so some recommendations are needed on how to tune Ceph, indeed | 15:53 |
| akrzos | and can push other more important data out of caches because it can create so many | 15:53 |
| akrzos | So still on the battle to get more instances, but we can pretty much blame the collector whcih should be gone by Pike | 15:54 |
| akrzos | Also later this week I intend on pushing my rally code into a review | 15:54 |
| akrzos | thats it for me | 15:54 |
| DinaBelova | akrzos the issue with collector / rabbit looks like the issues we've observed with classical Ceilometer some time ago... I remember issue like "collector is slowly taking messages from rabbitmq (as the speed depends on writing speed to the backend, and it may be slow), therefore queue size is constantly growing and at some point rabbitmq crashes" | 15:55 |
| *** markvoelker has quit IRC | 15:56 | |
| akrzos | so the collector will grab too much off all at once | 15:56 |
| akrzos | if the prefetch is not limited | 15:56 |
| DinaBelova | yeah | 15:56 |
| akrzos | so if you limit prefetch you can prevent that | 15:56 |
| akrzos | but that doesn't stop it from being too slow | 15:56 |
| DinaBelova | indeed | 15:56 |
| akrzos | there is some settings surrounding the thread count and worker count | 15:56 |
| akrzos | which could use a bit more playing with to see if it improves much | 15:57 |
| DinaBelova | akrzos thanks for your update | 15:57 |
| DinaBelova | from Mirantis side: we're currently stressing k8s API and ETCD with huge number of pods (on 450 nodes). At the same time we're trying to evaluate k8s benefits from LCM point of view - we've started working on upgrades testing of containerized OpenStack environment | 15:58 |
| *** markvoelker has joined #openstack-performance | 15:58 | |
| DinaBelova | sadly progress is slow due to several issues with the lab itself from hw perspective and from deployment perspective as well | 15:58 |
| DinaBelova | so nothing to share so far | 15:58 |
| DinaBelova | it looks like we're done with the progress update | 15:59 |
| DinaBelova | so I believe we may jump to the open discussion | 15:59 |
| DinaBelova | #topic Open Discussion | 15:59 |
| tovin07_ | from me | 15:59 |
| tovin07_ | continue to work on these patches | 15:59 |
| tovin07_ | #link OSprofiler in Ironic https://review.openstack.org/#/c/358142/ | 16:00 |
| tovin07_ | #link OSprofiler in Murano https://blueprints.launchpad.net/murano/+spec/osprofiler-support-in-murano | 16:00 |
| tovin07_ | #link review OSprofiler in Zun https://review.openstack.org/#/c/444776/ | 16:00 |
| DinaBelova | #info also I propose to add tovin07_ to OSprofiler core team | 16:00 |
| DinaBelova | tovin07_ ^^ | 16:00 |
| DinaBelova | I'll discuss this with Vipin, but I think he'll be really glad | 16:00 |
| tovin07_ | patches in Murano will be push this week | 16:00 |
| tovin07_ | DinaBelova: | 16:00 |
| tovin07_ | DinaBelova: thanks :D | 16:00 |
| DinaBelova | tovin07_ good to know! | 16:00 |
| tovin07_ | i think that I’m ready for that | 16:01 |
| DinaBelova | :D you should yeah :D | 16:01 |
| tovin07_ | hope to see it soon | 16:01 |
| tovin07_ | :D | 16:01 |
| DinaBelova | tovin07_ ;) | 16:01 |
| tovin07_ | besides, just realize that we bump version of OSprofiler recently (current 1.7.0) | 16:01 |
| DinaBelova | tovin07_ yeah, so it includes all nice things that were done about new drivers, etc. | 16:02 |
| DinaBelova | and some of your improvements as well | 16:02 |
| tovin07_ | yes | 16:02 |
| DinaBelova | tovin07_ anything else to share today? | 16:03 |
| tovin07_ | oh | 16:03 |
| DinaBelova | tovin07_ it might be not, do not worry :) | 16:03 |
| tovin07_ | Boris said that he came back | 16:03 |
| tovin07_ | however, I did not see him around | 16:04 |
| tovin07_ | :D | 16:04 |
| rcherrueau | tovin07_:, DinaBelova: I have a question about osprofiler, can I? | 16:04 |
| tovin07_ | yes sir | 16:05 |
| DinaBelova | tovin07_ yeah, but upcoming summit rally update will be processed by him and Andrey together | 16:05 |
| DinaBelova | #link https://www.openstack.org/summit/boston-2017/summit-schedule/events/18612/project-update-rally?BackURL=https%3A%2F%2Fwww.openstack.org%2Fsummit%2Fboston-2017%2Fsummit-schedule%2Fglobal-search%3Ft%3Dpavlovic%23eventId%3D18612 | 16:05 |
| DinaBelova | rcherrueau sure :) | 16:05 |
| rcherrueau | tovin07_: In the *-stop part of a trace, is it possible to get the result of the function or sql evaluation? | 16:05 |
| tovin07_ | DinaBelova: yes, I see :D | 16:05 |
| DinaBelova | rcherrueau hm, I believe it's possible, but it will require addtional developer work in osprofiler | 16:06 |
| tovin07_ | rcherrueau: I think yes | 16:06 |
| DinaBelova | rcherrueau right now it's not possible | 16:06 |
| DinaBelova | tovin07_ can you please evaluate this work? what needs to be done? | 16:07 |
| tovin07_ | however, current implementation mainly focus on request parameters and arguments (and kwargs) | 16:07 |
| tovin07_ | DinaBelova: sure | 16:07 |
| DinaBelova | #action tovin07_ evaluate what needs to be done to add result of the function or sql evaluation in the *-stop part of osprofiler trace | 16:07 |
| rcherrueau | tovin07_: yes you are right, but during debug it could be helpful to get the result also | 16:07 |
| DinaBelova | rcherrueau yeah, it's good use case | 16:07 |
| DinaBelova | thanks for sharing | 16:08 |
| tovin07_ | rcherrueau: yes | 16:08 |
| tovin07_ | thanks | 16:08 |
| DinaBelova | let's try to get some answer next week | 16:08 |
| tovin07_ | (y) | 16:08 |
| rcherrueau | great | 16:08 |
| DinaBelova | msimonin akrzos anything else to share? | 16:08 |
| akrzos | i'm good | 16:09 |
| DinaBelova | ok, so thanks everyone for participating and see you next week! | 16:09 |
| DinaBelova | #endmeeting | 16:09 |
| openstack | Meeting ended Tue Mar 14 16:09:37 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:09 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/performance_team/2017/performance_team.2017-03-14-15.30.html | 16:09 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/performance_team/2017/performance_team.2017-03-14-15.30.txt | 16:09 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/performance_team/2017/performance_team.2017-03-14-15.30.log.html | 16:09 |
| tovin07_ | bye | 16:09 |
| tovin07_ | have a nice week :D | 16:09 |
| rcherrueau | bye | 16:09 |
| *** jkilpatr has quit IRC | 16:10 | |
| *** jkilpatr has joined #openstack-performance | 16:23 | |
| *** tovin07_ has left #openstack-performance | 16:37 | |
| *** rcherrueau has quit IRC | 16:43 | |
| *** msimonin has quit IRC | 17:14 | |
| *** rmart04 has quit IRC | 17:17 | |
| *** boyfawkes has joined #openstack-performance | 17:26 | |
| *** boyfawkes has quit IRC | 17:31 | |
| *** faizy_ has quit IRC | 18:00 | |
| *** aojea has joined #openstack-performance | 18:02 | |
| *** pcaruana has quit IRC | 18:03 | |
| *** mgagne_ is now known as mgagne | 18:05 | |
| *** rmart04 has joined #openstack-performance | 18:10 | |
| *** jkilpatr has quit IRC | 18:12 | |
| *** jkilpatr has joined #openstack-performance | 18:13 | |
| *** bvanhav__ has joined #openstack-performance | 18:19 | |
| *** abalutoiu has quit IRC | 18:19 | |
| *** bvanhav_ has quit IRC | 18:20 | |
| *** rmart04 has quit IRC | 18:24 | |
| *** bvanhav__ has quit IRC | 19:40 | |
| *** rmart04 has joined #openstack-performance | 19:48 | |
| *** rmart04 has quit IRC | 20:25 | |
| *** jkilpatr has quit IRC | 20:37 | |
| *** dimtruck is now known as zz_dimtruck | 20:38 | |
| *** jkilpatr has joined #openstack-performance | 20:58 | |
| *** zz_dimtruck is now known as dimtruck | 21:03 | |
| *** rmart04 has joined #openstack-performance | 21:18 | |
| *** rmart04 has quit IRC | 21:20 | |
| *** abalutoiu has joined #openstack-performance | 21:20 | |
| *** dimtruck is now known as zz_dimtruck | 21:36 | |
| *** zz_dimtruck is now known as dimtruck | 21:37 | |
| *** catintheroof has quit IRC | 21:46 | |
| *** rfolco has quit IRC | 21:53 | |
| *** rmart04 has joined #openstack-performance | 22:27 | |
| *** rmart04 has quit IRC | 22:32 | |
| *** aojea has quit IRC | 22:41 | |
| *** aojea has joined #openstack-performance | 22:42 | |
| *** aojea has quit IRC | 22:46 | |
| *** rfolco has joined #openstack-performance | 22:54 | |
| *** rmart04 has joined #openstack-performance | 23:28 | |
| *** dimtruck is now known as zz_dimtruck | 23:31 | |
| *** rmart04 has quit IRC | 23:33 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!