| openstackgerrit | Bradley Klein proposed stackforge/monasca-persister: Add support for influxdb 0.9.0 https://review.openstack.org/151050 | 00:02 |
|---|---|---|
| *** dschroeder has quit IRC | 00:02 | |
| *** rbak has quit IRC | 00:04 | |
| openstackgerrit | Tim Kuhlman proposed stackforge/monasca-agent: Changed it so you have to explicitly choose the detailed process metrics. https://review.openstack.org/151053 | 00:17 |
| *** ericpeterson has quit IRC | 00:18 | |
| *** ddieterly has quit IRC | 00:20 | |
| *** openstackgerrit has quit IRC | 00:21 | |
| *** openstackgerrit has joined #openstack-monasca | 00:21 | |
| *** dlfryar has quit IRC | 00:33 | |
| *** rbak has joined #openstack-monasca | 00:34 | |
| *** dlfryar has joined #openstack-monasca | 00:40 | |
| *** rbak has quit IRC | 00:43 | |
| *** craigbr has quit IRC | 00:44 | |
| *** rbak has joined #openstack-monasca | 00:55 | |
| *** openstackgerrit has quit IRC | 01:05 | |
| *** openstackgerrit has joined #openstack-monasca | 01:05 | |
| *** craigbr has joined #openstack-monasca | 01:07 | |
| *** garyh_ has quit IRC | 01:07 | |
| *** rbak has quit IRC | 01:09 | |
| *** amalagon_ has quit IRC | 01:19 | |
| *** dlfryar has quit IRC | 01:28 | |
| *** craigbr has quit IRC | 01:52 | |
| *** craigbr has joined #openstack-monasca | 01:54 | |
| *** amalagon has joined #openstack-monasca | 02:06 | |
| *** craigbr has quit IRC | 02:16 | |
| *** theresa_ has joined #openstack-monasca | 02:20 | |
| *** openstackgerrit has quit IRC | 02:20 | |
| *** openstackgerrit has joined #openstack-monasca | 02:21 | |
| *** th_ has joined #openstack-monasca | 02:23 | |
| *** theresa_ has quit IRC | 02:27 | |
| *** rbak has joined #openstack-monasca | 02:58 | |
| *** rbak has quit IRC | 03:12 | |
| *** openstackgerrit has quit IRC | 03:20 | |
| *** openstackgerrit has joined #openstack-monasca | 03:20 | |
| *** boris-42 has quit IRC | 04:13 | |
| *** amalagon has quit IRC | 04:30 | |
| *** sanjayu has joined #openstack-monasca | 05:01 | |
| *** sanjayu_ has joined #openstack-monasca | 05:04 | |
| *** sanjayu_ has quit IRC | 05:06 | |
| *** sanjayu_ has joined #openstack-monasca | 05:06 | |
| *** th_ has quit IRC | 05:20 | |
| *** dlfryar has joined #openstack-monasca | 05:40 | |
| *** dlfryar has quit IRC | 05:46 | |
| *** sanjayu__ has joined #openstack-monasca | 07:05 | |
| *** Miouge has joined #openstack-monasca | 07:11 | |
| *** Miouge has quit IRC | 07:12 | |
| *** Miouge has joined #openstack-monasca | 07:48 | |
| *** Miouge has quit IRC | 07:54 | |
| *** Miouge has joined #openstack-monasca | 08:05 | |
| *** Miouge has quit IRC | 08:26 | |
| *** Miouge has joined #openstack-monasca | 08:38 | |
| *** Miouge has quit IRC | 09:05 | |
| *** Miouge has joined #openstack-monasca | 09:45 | |
| *** Miouge has quit IRC | 10:12 | |
| *** Miouge has joined #openstack-monasca | 11:08 | |
| *** Miouge has quit IRC | 11:43 | |
| *** eglynn-officeafk is now known as eglynn-lunch | 12:15 | |
| *** Miouge has joined #openstack-monasca | 12:16 | |
| *** Miouge has quit IRC | 13:09 | |
| *** Miouge has joined #openstack-monasca | 13:12 | |
| *** eglynn-lunch is now known as eglynn-office | 13:17 | |
| *** Miouge has quit IRC | 13:20 | |
| *** Miouge has joined #openstack-monasca | 13:23 | |
| *** boris-42 has joined #openstack-monasca | 13:31 | |
| sandywalsh_ | http://svds.com/post/avoiding-common-mistake-time-series | 13:32 |
| *** rbak has joined #openstack-monasca | 13:32 | |
| *** Miouge has quit IRC | 13:36 | |
| *** eglynn-office is now known as eglynn-officeafk | 13:40 | |
| *** rbak has quit IRC | 13:41 | |
| *** Miouge has joined #openstack-monasca | 13:55 | |
| *** rbak has joined #openstack-monasca | 14:00 | |
| *** LeanReox has joined #openstack-monasca | 14:13 | |
| *** rbak has quit IRC | 14:16 | |
| *** MVenesio has joined #openstack-monasca | 14:20 | |
| *** sanjayu__ has quit IRC | 14:37 | |
| *** sanjayu_ has quit IRC | 14:38 | |
| *** sanjayu has quit IRC | 14:42 | |
| *** rbak has joined #openstack-monasca | 14:52 | |
| *** ddieterly has joined #openstack-monasca | 14:57 | |
| *** ericpeterson has joined #openstack-monasca | 15:01 | |
| *** rbak has quit IRC | 15:02 | |
| *** craigbr has joined #openstack-monasca | 15:16 | |
| *** rbak has joined #openstack-monasca | 15:16 | |
| LeanReox | craigbr, morning, i did an awesome finding yesterday | 15:28 |
| craigbr | LeanReox: That would be great as I have no idea of what is happening | 15:28 |
| LeanReox | craigbr, i discover that only the networking metrics are getting posted ok | 15:31 |
| LeanReox | and the disk are getting 422 | 15:31 |
| craigbr | Interesting. Do you have a tcpdump or something of what the disk metrics look like? | 15:31 |
| LeanReox | ill do it right away | 15:33 |
| MVenesio | craigbr: Hello craigbr, do you know how can i add regular expressions to match specific dimensions rules ? | 15:35 |
| craigbr | Mvenesio: Where would you like to use the regular expressions? | 15:36 |
| LeanReox | craigbr, http://pastebin.com/UbU4Y6UW | 15:36 |
| MVenesio | craigbr: in the API call to monasca-api when i try to get metrics from a group of compute nodes for example | 15:37 |
| MVenesio | craigbr: for example i need to pass dimension hostname=os-nova-compute02 and hostname=os-nova-compute03 | 15:37 |
| MVenesio | craigbr: i can use hostname=os-nova-compute.* to get all hostnames but i cant select a subgroup | 15:38 |
| craigbr | Currently there is no regular expressions for the metric or measurement list. Do they share a service dimension? | 15:39 |
| MVenesio | craigbr: yes the service is monasca for all the compute nodes by default | 15:39 |
| MVenesio | craigbr: but we are coding a monitoring dashboard and we need to filter for a specific group of computes | 15:40 |
| craigbr | Today, you would have to make separate API calls. We don't implement something like hostname=os-nova-compute0[23] | 15:41 |
| MVenesio | craigbr: mmmm ok, and why this is working hostname=os-nova-compute.* | 15:42 |
| MVenesio | craigbr: with that call you get all the hosts | 15:42 |
| craigbr | I'm honestly not sure. I'm bringing up my monasca-vagrant now to check | 15:42 |
| *** dlfryar has joined #openstack-monasca | 15:43 | |
| MVenesio | craigbr: okk no problem, if you have any news please tell me | 15:43 |
| craigbr | MVenesio: Will do. Will just take some time for investigation | 15:44 |
| *** garyh has joined #openstack-monasca | 15:48 | |
| craigbr | MVenesio: hostname=os-nova-compute.* works for you because you are probably using a slightly older version of the API. There was a bug where the dimension names and values were matching as "startsWith" not "equals" | 15:53 |
| craigbr | I bet that hostname=os-nova-compute works just as well. Can you try that? | 15:54 |
| *** craigbr has quit IRC | 15:57 | |
| *** krtaylor has quit IRC | 16:08 | |
| *** craigbr has joined #openstack-monasca | 16:09 | |
| LeanReox | craigbr, did you get a chance to see the post ? | 16:13 |
| craigbr | LeanReox: Sorry, missed it with the other conversation. Looking at it now | 16:13 |
| LeanReox | craigbr, great | 16:13 |
| LeanReox | craigbr, ive tried the call that MVenesio said, and hostname=os-nova-compute doestn bring anything, but hostname=os-nova-compute.* does | 16:15 |
| LeanReox | craigbr, just to clarify that one | 16:15 |
| craigbr | Interesting. I'll have to go back to the old code and check | 16:15 |
| *** amalagon has joined #openstack-monasca | 16:16 | |
| *** krtaylor has joined #openstack-monasca | 16:22 | |
| craigbr | LeanReox: I can successfully create that metric on my system. What version of the API are you using? | 16:22 |
| craigbr | dpkg -i | grep monasca-api | 16:22 |
| *** dschroeder has joined #openstack-monasca | 16:25 | |
| *** subscope has quit IRC | 16:26 | |
| *** MVenesio has quit IRC | 16:31 | |
| *** MVenesio has joined #openstack-monasca | 16:31 | |
| *** LeanReox has quit IRC | 16:32 | |
| *** LeanReox has joined #openstack-monasca | 16:33 | |
| LeanReox | craigbr, ii monasca-api 0.1.0-1421267729582-476e21 all Monasca-API | 16:35 |
| *** craigbr has quit IRC | 16:35 | |
| *** LeanReox has quit IRC | 16:35 | |
| *** LeanReox has joined #openstack-monasca | 16:36 | |
| *** craigbr has joined #openstack-monasca | 16:50 | |
| LeanReox | craigbr, | 16:51 |
| LeanReox | finally find it !!! | 16:51 |
| craigbr | LeanReox: Cool. What was it? | 16:51 |
| LeanReox | take a look at this one: http://pastebin.com/40CTaU2v | 16:51 |
| LeanReox | im running monasca-api 0.1.0-1421267729582-476e21 all Monasca-API | 16:51 |
| craigbr | LeanReox: Yes, that colon will cause the 422. It needs to be changed into something else or removed. We probably need to add a clean method to monclient.py | 16:54 |
| LeanReox | if i change: meta.OS-EXT-AZ:availability_zone to meta.OS-EXT-AZ.availability_zone | 16:55 |
| LeanReox | a cleanup method that replaces ":" in the value, for a "." should work | 16:55 |
| craigbr | Yep. Although it would be good if it handled more than : | 16:56 |
| LeanReox | ill work on that and let you know | 16:57 |
| craigbr | The only thing that confuses me is why this wasn't being logged in your /var/log/monasca/monasca-api.log. I just tried that dimension and I see this error message: | 16:58 |
| craigbr | ERROR [2015-01-29 16:57:36,829] monasca.api.resource.exception.Exceptions: UNPROCESSABLE_ENTITY 181fa42d8a989867 - Dimension name meta.OS-EXT-AZ:availability_zone may only contain: a-z A-Z 0-9 _ - . | 16:58 |
| LeanReox | yes thats weird, i have the monasca log on debug | 16:59 |
| *** krtaylor has quit IRC | 17:03 | |
| *** craigbr has quit IRC | 17:03 | |
| *** krtaylor has joined #openstack-monasca | 17:05 | |
| *** ddieterly has quit IRC | 17:08 | |
| *** ddieterly has joined #openstack-monasca | 17:09 | |
| *** theresa_ has joined #openstack-monasca | 17:24 | |
| *** craigbr has joined #openstack-monasca | 17:26 | |
| *** MVenesio has quit IRC | 17:29 | |
| *** MVenesio has joined #openstack-monasca | 17:29 | |
| openstackgerrit | Henrique Truta proposed stackforge/monasca-api: Integrating monasca-api with keystonemiddleware https://review.openstack.org/151325 | 17:29 |
| craigbr | MVenesio: Figured it out. The reason .* works is because of how the metrics are stored and looked up in influxdb. A regular expression is used to find the metrics. | 17:30 |
| craigbr | However the metric name and dimensions are URL encoded so not all regular expressions pass through | 17:31 |
| MVenesio | craigbr: ohhh well | 17:31 |
| MVenesio | craigbr:so maybe there are some influxdb expression to use and get a group of servers | 17:33 |
| craigbr | influxdb supports standard regular expressions | 17:34 |
| LeanReox | craigbr, var = "meta.OS-EXT-AZ:availability_zone" => re.sub('[^A-Za-z0-9-.]+', '.', var) => 'meta.OS-EXT-AZ.availability.zone' | 17:34 |
| LeanReox | craigbr, how about that ? getting replaced by "." | 17:34 |
| craigbr | LeanReox: Looks good to me | 17:34 |
| htruta | ddieterly: am I the only one who's having these pep8 issues ? http://logs.openstack.org/25/151325/1/check/gate-monasca-api-pep8/8014200/console.html | 17:37 |
| *** MVenesio has quit IRC | 17:41 | |
| *** MVenesio has joined #openstack-monasca | 17:42 | |
| *** krtaylor has quit IRC | 17:51 | |
| *** MVenesio has quit IRC | 17:55 | |
| *** MVenesio_ has joined #openstack-monasca | 17:55 | |
| *** krtaylor has joined #openstack-monasca | 18:04 | |
| *** craigbr has quit IRC | 18:30 | |
| *** craigbr has joined #openstack-monasca | 18:35 | |
| ddieterly | htruta: have you tried a 'git pull' recently? | 18:37 |
| ddieterly | htruta: sometimes the errors are dormant until someone refactors the code. then they get detected. you need to make the necessary changes to make it pass the pep8 checks. | 18:38 |
| htruta | yeah... I clonned the fresh monasca-api and applied my changes in this patch | 18:38 |
| *** krtaylor has quit IRC | 18:43 | |
| htruta | ddieterly: actually, I already did in this patch: https://review.openstack.org/#/c/148676/ | 18:47 |
| *** openstackgerrit has quit IRC | 18:50 | |
| *** openstackgerrit has joined #openstack-monasca | 18:51 | |
| ddieterly | htruta: did you get all the problems fixed? | 18:51 |
| ddieterly | htruta: if the pep8 checker is complaining about imports, it should be easy to fix the problems | 18:51 |
| htruta | ddieterly: I think I did... Jenkins is not complaining in this patch I sent | 18:52 |
| htruta | btw you +1 it | 18:52 |
| ddieterly | oh, your patch did not get merged | 18:52 |
| ddieterly | hang on | 18:52 |
| ddieterly | +2 | 18:53 |
| ddieterly | should get merged shortly | 18:53 |
| htruta | cool. | 18:54 |
| *** krtaylor has joined #openstack-monasca | 18:54 | |
| craigbr | LeanReox: Take a look at https://review.openstack.org/151345 | 18:59 |
| craigbr | I used the regular expression from the actual check in the API | 18:59 |
| openstackgerrit | Merged stackforge/monasca-api: Removing imports of classes instead of modules https://review.openstack.org/148676 | 18:59 |
| LeanReox | craigbr, lets see | 18:59 |
| LeanReox | craigbr, awesome looks really good to me | 19:01 |
| craigbr | Hope so. Its almost all your code :) | 19:01 |
| LeanReox | craigbr, :) , ill register on the gerrit stuff to not bother you anymore with this changes :) | 19:02 |
| htruta | ddieterly: looks like 'recheck' is not working. should I send another patch to make jenkins run again? | 19:03 |
| craigbr | LeanReox: I get notified about all changes so I'll see it but we'll be happy to have your help | 19:03 |
| LeanReox | in the meantime, its a normal behavior that monasca-api returns a error 500 if you ask for a time period that has data in the middle but not in the very beginning ? | 19:03 |
| LeanReox | craigbr, great ill help as much as i can | 19:04 |
| *** dlfryar has quit IRC | 19:04 | |
| craigbr | Monasca-api should never return a 500. What is the stack trace from monasca-api.log? | 19:05 |
| htruta | ddieterly: nevermind. it worked. Jenkins +1'd. | 19:07 |
| LeanReox | craigbr, let me check | 19:11 |
| LeanReox | hmmm seems that im hitting the influxdb master on the cluster (i have an ifluxdb cluster with 3 members) | 19:16 |
| LeanReox | craigbr, and it seems that hitting the master makes imposible to list series from a shard that lives on other node | 19:16 |
| LeanReox | the stacktrace is like this ! java.lang.RuntimeException: No servers up to query shard 2 | 19:16 |
| LeanReox | craigbr, full stacktrace -> http://pastebin.com/dtfn1dcH | 19:18 |
| craigbr | LeanReox: I'll need to get somebody who knows influxdb on | 19:18 |
| LeanReox | that will be great | 19:18 |
| LeanReox | it seems that til 0.9.0 ... influxdb cluster is pretty inmature | 19:19 |
| LeanReox | but could be a bad configuration cfrom my side | 19:19 |
| craigbr | I think that influxdb realizes that. We're going to support 0.9.0 as soon as we can | 19:19 |
| LeanReox | craigbr, implementing elasticsearch as the backend still WIP isn it ? | 19:19 |
| craigbr | That isn't really in plan | 19:20 |
| *** dlfryar has joined #openstack-monasca | 19:21 | |
| craigbr | Influxdb is a better solution | 19:22 |
| LeanReox | craigbr, i misread something then | 19:22 |
| LeanReox | it seems to take a looot of troughput, but sharding is a must with high volume | 19:22 |
| LeanReox | and availability of course | 19:24 |
| LeanReox | craigbr, let me know when the influxdb yoda jumps in | 19:25 |
| craigbr | You probably missed: ElasticSearch isn't really in plan. Influxdb is a better solution | 19:26 |
| craigbr | lunch time here, so people are out | 19:26 |
| LeanReox | craigbr, yes, but if clustering isnt working like suposed to, a single instance is kinda no-production ready | 19:27 |
| LeanReox | craigbr, roger | 19:27 |
| *** ddieterly has quit IRC | 19:33 | |
| *** ddieterly has joined #openstack-monasca | 19:34 | |
| *** dlfryar has quit IRC | 19:36 | |
| *** dlfryar has joined #openstack-monasca | 19:43 | |
| *** ddieterl_ has joined #openstack-monasca | 19:51 | |
| ddieterl_ | LeanReox: clustering in influxdb is flaky | 19:53 |
| ddieterl_ | we'll be moving to 0.9.0 shortly and we'll look for influxdb clustering improvements | 19:53 |
| LeanReox | ddieterl_, indeed its seems that the replicas across shardings are not working as supposed to | 19:55 |
| LeanReox | so no matter if i have 3 nodes with replica count on 3 | 19:55 |
| LeanReox | the master still fails to get a series ... | 19:55 |
| LeanReox | do you guys benchmarked one influxdb node ? ddieterl_ | 19:56 |
| LeanReox | should we wait till 0.9.0 for clustering ? | 19:56 |
| *** ddieterly has quit IRC | 19:56 | |
| ddieterl_ | LeanReox: i think your questions would be better answered here https://groups.google.com/forum/#!forum/influxdb | 19:57 |
| LeanReox | roger ddieterl_ | 19:57 |
| ddieterl_ | LeanReox: we're using 0.8.8 in a cluster and it seems to work for us | 19:58 |
| LeanReox | ok, ddieterl_ how many shards and replicas | 19:58 |
| htruta | a few months ago, I went into #influxdb here on freenode and asked how I could do a cluster setup, and they answered: DON'T | 20:00 |
| *** ddieterl_ is now known as ddieterly | 20:00 | |
| htruta | they told me to wait the next release, that may be the 0.9.0 | 20:00 |
| ddieterly | 3 servers, 2 shards | 20:01 |
| LeanReox | ddieterly, ok, and replication factor ? | 20:01 |
| ddieterly | shared duration is 7 days, retention policy 365 days | 20:01 |
| ddieterly | rf=3 | 20:01 |
| LeanReox | any chance that you can share your config.toml to us ? | 20:01 |
| LeanReox | and you especify just the master on the monasca api influxdb section isnt it ? | 20:02 |
| *** openstackgerrit has quit IRC | 20:04 | |
| *** openstackgerrit has joined #openstack-monasca | 20:04 | |
| openstackgerrit | Merged stackforge/monasca-ui: Remove some old unused code https://review.openstack.org/150549 | 20:10 |
| ddieterly | LeanReox: https://gist.github.com/ddieterly/3088cbd01b96566d7499 | 20:22 |
| LeanReox | thanks ddieterly this will help a lot | 20:24 |
| LeanReox | ill test again with this setup | 20:24 |
| LeanReox | and let you know | 20:24 |
| ddieterly | you're welcome | 20:24 |
| LeanReox | cause i was getting a lof of No servers up to query shard 2 , and all the servers were up | 20:25 |
| *** Miouge has quit IRC | 20:40 | |
| openstackgerrit | Rob Raymond proposed stackforge/monasca-ui: Fix filter action logic to allow server side filtering in future https://review.openstack.org/151378 | 20:44 |
| openstackgerrit | Bradley Klein proposed stackforge/monasca-persister: Working example of influxdb 0.9.0 class https://review.openstack.org/151383 | 21:12 |
| *** ddieterly has quit IRC | 21:32 | |
| *** MVenesio_ has quit IRC | 21:45 | |
| *** LeanReox has quit IRC | 21:45 | |
| *** theresa_ has quit IRC | 21:52 | |
| *** ddieterly has joined #openstack-monasca | 22:02 | |
| *** theresa_ has joined #openstack-monasca | 22:08 | |
| *** th_ has joined #openstack-monasca | 22:10 | |
| ddieterly | htruta: did you run the code in https://review.openstack.org/#/c/151325/1 | 22:11 |
| ddieterly | { | 22:11 |
| ddieterly | "title": "Forbidden", | 22:11 |
| ddieterly | "description": "Tenant ID is missing a required role to access this service" | 22:11 |
| ddieterly | } | 22:11 |
| ddieterly | htruta: i'm getting ^^ | 22:12 |
| *** craigbr has quit IRC | 22:12 | |
| *** theresa_ has quit IRC | 22:14 | |
| *** MVenesio has joined #openstack-monasca | 22:22 | |
| *** MVenesio has quit IRC | 22:26 | |
| *** henrique_ has joined #openstack-monasca | 22:40 | |
| *** henrique_ has left #openstack-monasca | 22:49 | |
| *** openstackgerrit has quit IRC | 22:51 | |
| *** openstackgerrit has joined #openstack-monasca | 22:51 | |
| *** th_ has quit IRC | 23:27 | |
| *** dlfryar has quit IRC | 23:30 | |
| *** craigbr has joined #openstack-monasca | 23:32 | |
| *** craigbr has quit IRC | 23:46 | |
| *** rbak has quit IRC | 23:50 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!