*** promulo_ has joined #openstack-ceilometer | 00:18 | |
*** primechuck has joined #openstack-ceilometer | 00:20 | |
*** promulo__ has quit IRC | 00:22 | |
*** changbl has joined #openstack-ceilometer | 01:17 | |
*** changbl has quit IRC | 01:25 | |
*** changbl_ has joined #openstack-ceilometer | 01:36 | |
*** changbl has joined #openstack-ceilometer | 01:38 | |
*** lan has joined #openstack-ceilometer | 01:39 | |
*** changbl_ has quit IRC | 01:43 | |
*** ddieterly has quit IRC | 01:43 | |
*** ankita_wagh has quit IRC | 01:46 | |
*** deepthi has joined #openstack-ceilometer | 01:47 | |
*** changbl has quit IRC | 01:49 | |
*** lexholden has quit IRC | 01:51 | |
*** liusheng has joined #openstack-ceilometer | 02:03 | |
*** Kennan2 has joined #openstack-ceilometer | 02:05 | |
*** Kennan has quit IRC | 02:05 | |
*** Kennan2 is now known as Kennan | 02:07 | |
openstackgerrit | Merged openstack/ceilometer: Add necessary executable permission https://review.openstack.org/202353 | 02:11 |
---|---|---|
openstackgerrit | Merged openstack/ceilometer: Updated from global requirements https://review.openstack.org/200659 | 02:12 |
*** changbl has joined #openstack-ceilometer | 02:17 | |
*** ankita_wagh has joined #openstack-ceilometer | 02:39 | |
*** ddieterly has joined #openstack-ceilometer | 02:44 | |
*** primechuck has quit IRC | 02:46 | |
*** ddieterly has quit IRC | 02:48 | |
*** ddieterly has joined #openstack-ceilometer | 03:11 | |
*** ankita_wagh has quit IRC | 03:25 | |
*** ankita_wagh has joined #openstack-ceilometer | 03:41 | |
*** ankita_wagh has quit IRC | 03:41 | |
*** ankita_wagh has joined #openstack-ceilometer | 03:42 | |
*** swann has quit IRC | 03:48 | |
*** liusheng has quit IRC | 04:06 | |
*** liusheng has joined #openstack-ceilometer | 04:06 | |
*** htruta has quit IRC | 04:15 | |
*** ddieterly has quit IRC | 04:17 | |
*** changbl has quit IRC | 04:19 | |
*** changbl has joined #openstack-ceilometer | 04:33 | |
*** swann has joined #openstack-ceilometer | 04:36 | |
*** ankita_w_ has joined #openstack-ceilometer | 04:42 | |
*** ankita_wagh has quit IRC | 04:46 | |
*** yprokule has joined #openstack-ceilometer | 04:58 | |
*** ddieterly has joined #openstack-ceilometer | 05:18 | |
*** ddieterly has quit IRC | 05:22 | |
*** rex_lee has quit IRC | 05:35 | |
*** serverascode has quit IRC | 05:35 | |
*** h00327910_ has quit IRC | 05:36 | |
*** chinmay_g has quit IRC | 05:36 | |
*** serverascode has joined #openstack-ceilometer | 05:39 | |
*** cmyster has quit IRC | 05:39 | |
*** cmyster has joined #openstack-ceilometer | 05:40 | |
*** h00327910_ has joined #openstack-ceilometer | 05:48 | |
*** rex_lee has joined #openstack-ceilometer | 05:49 | |
*** chinmay_g has joined #openstack-ceilometer | 05:51 | |
*** ankita_w_ has quit IRC | 05:55 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/ceilometer: Imported Translations from Transifex https://review.openstack.org/202891 | 06:10 |
*** lsmola has joined #openstack-ceilometer | 06:12 | |
*** ddieterly has joined #openstack-ceilometer | 06:19 | |
*** ddieterly has quit IRC | 06:23 | |
*** eglynn__ has joined #openstack-ceilometer | 06:27 | |
*** boris-42 has joined #openstack-ceilometer | 06:38 | |
*** cmyster has left #openstack-ceilometer | 06:39 | |
*** _nadya_ has joined #openstack-ceilometer | 06:43 | |
*** ankita_wagh has joined #openstack-ceilometer | 06:44 | |
*** eglynn__ has quit IRC | 06:47 | |
*** afazekas_ has joined #openstack-ceilometer | 07:09 | |
*** ildikov has quit IRC | 07:11 | |
*** shardy has joined #openstack-ceilometer | 07:15 | |
*** ddieterly has joined #openstack-ceilometer | 07:20 | |
*** ddieterly has quit IRC | 07:24 | |
_nadya_ | prad: hi Pradeep! I have one question about declarative notifications. What's the plan? Will we migrate all plugins into declarative stuff and remove code from plugins? | 07:28 |
*** tshtilma has joined #openstack-ceilometer | 07:28 | |
*** jasonamyers has quit IRC | 07:31 | |
*** rex_lee has quit IRC | 07:31 | |
*** zhiyan has quit IRC | 07:31 | |
*** serverascode has quit IRC | 07:31 | |
*** Ala has joined #openstack-ceilometer | 07:34 | |
*** belmoreira has joined #openstack-ceilometer | 07:36 | |
*** jasonamyers has joined #openstack-ceilometer | 07:38 | |
*** ildikov has joined #openstack-ceilometer | 07:38 | |
*** ankita_wagh has quit IRC | 07:39 | |
*** pcaruana has quit IRC | 07:42 | |
*** zhiyan has joined #openstack-ceilometer | 07:48 | |
*** safchain has joined #openstack-ceilometer | 07:55 | |
*** eglynn__ has joined #openstack-ceilometer | 07:55 | |
*** serverascode has joined #openstack-ceilometer | 07:57 | |
*** rex_lee has joined #openstack-ceilometer | 08:01 | |
*** eglynn__ has quit IRC | 08:02 | |
*** eglynn__ has joined #openstack-ceilometer | 08:03 | |
*** ddieterly has joined #openstack-ceilometer | 08:21 | |
*** ddieterly has quit IRC | 08:25 | |
*** belmoreira has quit IRC | 08:30 | |
*** Ala has quit IRC | 08:36 | |
*** pcaruana has joined #openstack-ceilometer | 08:43 | |
*** shardy_ has joined #openstack-ceilometer | 08:44 | |
*** shardy has quit IRC | 08:45 | |
*** Ala has joined #openstack-ceilometer | 08:47 | |
*** eglynn__ has quit IRC | 08:47 | |
*** shardy_ has quit IRC | 08:49 | |
*** shardy has joined #openstack-ceilometer | 08:50 | |
*** eglynn__ has joined #openstack-ceilometer | 09:04 | |
*** eglynn__ has quit IRC | 09:13 | |
*** ddieterly has joined #openstack-ceilometer | 09:21 | |
openstackgerrit | Merged openstack/ceilometer: add Trove(DBaaS) events https://review.openstack.org/200746 | 09:24 |
*** ddieterly has quit IRC | 09:26 | |
*** belmoreira has joined #openstack-ceilometer | 09:48 | |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: api: return 410 if only Gnocchi is enabled https://review.openstack.org/202925 | 09:56 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: api: Redirect request to aodh if available https://review.openstack.org/202926 | 09:56 |
*** lexholden has joined #openstack-ceilometer | 09:57 | |
*** shardy_ has joined #openstack-ceilometer | 10:03 | |
*** shardy has quit IRC | 10:04 | |
*** shardy_ has quit IRC | 10:08 | |
*** shardy has joined #openstack-ceilometer | 10:09 | |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: gnocchi: reduce the number of patch to gnocchi API https://review.openstack.org/202589 | 10:10 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Improves send_test_data tools https://review.openstack.org/203366 | 10:10 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Adds batch dispatcher https://review.openstack.org/203349 | 10:10 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: gnocchi: concurrency post resource/measures https://review.openstack.org/203553 | 10:10 |
*** Ala has quit IRC | 10:14 | |
*** tshtilma has quit IRC | 10:16 | |
*** belmoreira has quit IRC | 10:16 | |
*** ddieterly has joined #openstack-ceilometer | 10:22 | |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: gnocchi: reduce the number of patch to gnocchi API https://review.openstack.org/202589 | 10:26 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Improves send_test_data tools https://review.openstack.org/203366 | 10:26 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Adds batch dispatcher https://review.openstack.org/203349 | 10:26 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: gnocchi: concurrency post resource/measures https://review.openstack.org/203553 | 10:26 |
*** ddieterly has quit IRC | 10:27 | |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: gnocchi: reduce the number of patch to gnocchi API https://review.openstack.org/202589 | 10:28 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Improves send_test_data tools https://review.openstack.org/203366 | 10:28 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Adds batch dispatcher https://review.openstack.org/203349 | 10:28 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: gnocchi: concurrency post resource/measures https://review.openstack.org/203553 | 10:28 |
*** tshtilma has joined #openstack-ceilometer | 10:35 | |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: api: return 410 if only Gnocchi is enabled https://review.openstack.org/202925 | 10:39 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: api: Redirect request to aodh if available https://review.openstack.org/202926 | 10:39 |
*** yprokule has quit IRC | 10:52 | |
*** yprokule has joined #openstack-ceilometer | 10:52 | |
openstackgerrit | liusheng proposed openstack/python-ceilometerclient: Add support for client redirecting to aodh endpoint if available https://review.openstack.org/202938 | 10:52 |
openstackgerrit | Merged openstack/ceilometer: Switch to the oslo_utils.fileutils https://review.openstack.org/199515 | 10:55 |
*** eglynn-officeafk has joined #openstack-ceilometer | 10:56 | |
*** cdent has joined #openstack-ceilometer | 11:02 | |
*** tshtilma has quit IRC | 11:05 | |
*** ig0r_ has joined #openstack-ceilometer | 11:08 | |
*** ViswaV has joined #openstack-ceilometer | 11:11 | |
*** tshtilma has joined #openstack-ceilometer | 11:15 | |
*** boris-42 has quit IRC | 11:22 | |
*** ddieterly has joined #openstack-ceilometer | 11:23 | |
*** ddieterly has quit IRC | 11:27 | |
*** ViswaV has quit IRC | 11:34 | |
*** ViswaV has joined #openstack-ceilometer | 11:37 | |
*** thorst has joined #openstack-ceilometer | 11:40 | |
*** liusheng has quit IRC | 11:47 | |
*** liusheng has joined #openstack-ceilometer | 11:48 | |
*** julim has joined #openstack-ceilometer | 11:56 | |
*** gordc has joined #openstack-ceilometer | 11:56 | |
jasonamyers | good morning | 11:57 |
*** eglynn__ has joined #openstack-ceilometer | 12:00 | |
*** ig0r_ has quit IRC | 12:05 | |
*** changbl has quit IRC | 12:09 | |
*** ViswaV has quit IRC | 12:12 | |
*** eglynn__ has quit IRC | 12:12 | |
*** gordc has quit IRC | 12:17 | |
*** gordc has joined #openstack-ceilometer | 12:17 | |
*** htruta has joined #openstack-ceilometer | 12:18 | |
*** ddieterly has joined #openstack-ceilometer | 12:24 | |
*** ddieterly has quit IRC | 12:28 | |
*** edmondsw has joined #openstack-ceilometer | 12:29 | |
openstackgerrit | Chris Dent proposed openstack/ceilometer: Pollsters now send notifications without doing transforms https://review.openstack.org/198865 | 12:33 |
*** Ala has joined #openstack-ceilometer | 12:37 | |
*** deepthi has quit IRC | 12:37 | |
*** edmondsw has quit IRC | 12:40 | |
*** cmyster has joined #openstack-ceilometer | 12:49 | |
*** cmyster has joined #openstack-ceilometer | 12:49 | |
*** yprokule has quit IRC | 12:53 | |
*** deepthi has joined #openstack-ceilometer | 12:54 | |
*** cmyster has quit IRC | 12:58 | |
*** yprokule has joined #openstack-ceilometer | 12:59 | |
*** lexholden has quit IRC | 13:06 | |
*** changbl has joined #openstack-ceilometer | 13:09 | |
*** pradk has joined #openstack-ceilometer | 13:17 | |
*** edmondsw has joined #openstack-ceilometer | 13:18 | |
*** primechuck has joined #openstack-ceilometer | 13:22 | |
cdent | gordc: gave you another response on that caching stuff. you didn't freak out about gnocchi not having metadata so I wonder if you noticed that part of my comments, so I made it again more visibly... | 13:22 |
openstackgerrit | Kamil Rykowski proposed openstack/ceilometer: Versioned object initial patch https://review.openstack.org/178675 | 13:26 |
openstackgerrit | Kamil Rykowski proposed openstack/ceilometer: Versioned object for Alarm and AlarmChange model https://review.openstack.org/180908 | 13:26 |
gordc | cdent: yeah, i got the metadata part. | 13:27 |
cdent | aight den | 13:27 |
gordc | tbh, i'm thinking of it's just best to use generic resource for gnocchi (no metadata) | 13:28 |
*** boris-42 has joined #openstack-ceilometer | 13:28 | |
gordc | cdent: have you seen the sql backend how we hash resource? | 13:30 |
cdent | I've not looked at closely | 13:30 |
cdent | why? | 13:30 |
gordc | i think your plan is what we tried in cache but had to push to db. (sql db) | 13:31 |
cdent | because of the change in metadata handling in gnocchi, the resource attributes are far easier to hash | 13:32 |
gordc | based on last statement, it seems your solution is gnocchi specific, so the caching bit becomes a bit easier. | 13:32 |
cdent | no nested dicts or other non-atomic stuff | 13:32 |
gordc | it wasn't that difficult to hash for sql solution, only drawback was we might get some false collisions due to size of what we're hashing. | 13:33 |
gordc | or theorectically we could. i don't know how that translate to real life. | 13:33 |
gordc | but have we settled that the patching is the part that is actually slow in the gnocchi dispatcher? | 13:34 |
cdent | I have this vague sense that we're still sort of not really talking about all the same stuff here. | 13:34 |
gordc | or is it the updating storage part. | 13:34 |
openstackgerrit | Pradeep Kilambi proposed openstack/ceilometer: Convert identity, sahara and image to meters yaml https://review.openstack.org/203031 | 13:35 |
cdent | I don't know what we've settled on but I do know what we've discovered: | 13:35 |
cdent | both _create_resource and _update_resource are called redundantly, several times, per-resource, per-polling/notification cycle | 13:35 |
cdent | and we know that http overhead is significant | 13:35 |
gordc | cdent: right. so another reason why i brought up sql was we had similar write issues. | 13:36 |
cdent | that POC that I've posted is simply to solve that one problem, not come up with a one size fits all solution to everything | 13:37 |
cdent | because those kinds of solutions are what leads to stuff either never working or never being created | 13:37 |
gordc | one thing we did was we store hash (in db) so we do basically your poc proposal but not cache (because of support across all db issue i mention) | 13:37 |
gordc | i brought up sql backen because the design and notes there probably might help | 13:38 |
cdent | in a sense an etag is a bit like in-db-storage of hash if you think of the gnocchi-api as the db | 13:39 |
cdent | "probably might help" what? | 13:39 |
gordc | cdent: yeah, that's what i figured when i read doc | 13:39 |
cdent | what is it that needs help? | 13:39 |
gordc | give some ideas of stuff write issues we ran into originally. | 13:40 |
cdent | ENOPARSE | 13:40 |
cdent | (I'm not being intentionally obtuse or anything here, legitimately confused on what you're trying to convey) | 13:41 |
gordc | i don't know what enoparse means. | 13:41 |
cdent | Error No Parse | 13:41 |
cdent | I'm unable to parse what you are saying | 13:41 |
*** jdandrea has joined #openstack-ceilometer | 13:42 | |
gordc | i was just saying we write issues in sql and we did certain things to fix them in that driver. maybe it will help with gnocchi | 13:42 |
gordc | or not. just a suggestion. | 13:42 |
cdent | Is that a suggestion specific to the use/implementation of caching in that patchset or just a general comment on making dispatch-to-gnocchi better? | 13:43 |
gordc | general comment. | 13:43 |
cdent | ah, okay, that makes more sense. I thought you were drawing a more direct connection | 13:43 |
cdent | and couldn't quite figure out what that connection was | 13:44 |
*** yassine__ has joined #openstack-ceilometer | 13:44 | |
*** yassine__ has quit IRC | 13:45 | |
cdent | sileht: you still have several dispatch-to-gnocchi patches pending, yeah? If that's the case there's no point trying to make the caching stuff "correct" yet. Once that stabilizes, we can look again perhaps | 13:45 |
*** yassine__ has joined #openstack-ceilometer | 13:45 | |
*** rbak has joined #openstack-ceilometer | 13:50 | |
*** exploreshaifali has joined #openstack-ceilometer | 13:50 | |
sileht | cdent, especially this one: https://review.openstack.org/#/c/202589/ | 13:54 |
* cdent nods | 13:55 | |
* cdent stills thinks we should cache all the things all the time everywhere ;) | 13:55 | |
cdent | otherwise we can't really claim to be cool | 13:56 |
*** shardy_ has joined #openstack-ceilometer | 13:56 | |
*** shardy has quit IRC | 13:57 | |
gordc | sileht: is there going to be an oslo.messaging release soon? | 13:57 |
sileht | I done some testing with my pile of patches, remove the PATCH when is not usefull would really incrase performance | 13:57 |
gordc | sileht: that's great to hear. | 13:57 |
gordc | cdent: are you planning on doing that next cycle? | 13:58 |
sileht | gordc, I'm waiting for dims_ for a last patch and then we can really a new oslo.msg | 13:58 |
gordc | sileht: awesome! | 13:58 |
cdent | gordc: next cycle I'm going to remove all the stevedore extensions | 13:58 |
*** yprokule has quit IRC | 13:58 | |
gordc | so caching this cycle? | 13:59 |
cdent | no way man, I'm just going to have ideas and let other people do the work | 13:59 |
* gordc walks away | 13:59 | |
sileht | cdent, gordc my quick test was to inject 2000 random samples from 10 resources | 13:59 |
sileht | with PATCH : 2m15, without PATH: 1m20 | 14:00 |
sileht | the metadata of the resource was always the same, so it wasn't usefull to send them to gnocchi | 14:00 |
cdent | gordc: I'm happy to do caching this time around if it makes sense, but I dont want to do it just cuz, I'd like us to chat about, see if it makes sense, etc | 14:00 |
gordc | sileht: 2000 different metric types? | 14:00 |
*** yprokule has joined #openstack-ceilometer | 14:00 | |
sileht | gordc, just 'instance' from the tool 'send_test_data.py' | 14:01 |
*** shardy_ has quit IRC | 14:01 | |
cdent | gordc: so since _not_ having caching in there yet is causing sileht to do cool and interesting things, we best wait | 14:01 |
gordc | sileht: kk. i'll give your patch a try locally. my devstack is a lot slower. :( | 14:01 |
*** shardy has joined #openstack-ceilometer | 14:02 | |
sileht | gordc, for my tests I have used the 'batch dispatcher' with size=128 (yes I have finally write it :p) | 14:02 |
gordc | cdent: makes sense. i just asked because last time i tried to do caching was in August, i ran out of time. so if someone were to start, they need to do it now. | 14:03 |
gordc | sileht: ah cool. | 14:03 |
openstackgerrit | Chris Dent proposed openstack/ceilometer: Pollsters now send notifications without doing transforms https://review.openstack.org/198865 | 14:09 |
*** cmyster has joined #openstack-ceilometer | 14:12 | |
*** pradk has quit IRC | 14:13 | |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: gnocchi: reduce the number of patch to gnocchi API https://review.openstack.org/202589 | 14:15 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Improves send_test_data tools https://review.openstack.org/203366 | 14:15 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Adds batch dispatcher https://review.openstack.org/203349 | 14:15 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: gnocchi: concurrency post resource/measures https://review.openstack.org/203553 | 14:15 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Move gnocchi resources definition in yaml file https://review.openstack.org/202500 | 14:15 |
*** rex_lee_ has joined #openstack-ceilometer | 14:38 | |
*** Taytay has joined #openstack-ceilometer | 14:38 | |
*** _nadya_ has quit IRC | 14:40 | |
gordc | sileht: if i create a new pool. and the consumer of the pool dies/leaves, that queue will continue to grow right? there's no way to kill a queue from pool. | 14:40 |
sileht | gordc, right, you have to use rabbitmq admin tools | 14:41 |
*** krotscheck_ has joined #openstack-ceilometer | 14:41 | |
*** krotscheck has quit IRC | 14:42 | |
*** swamireddy_ has quit IRC | 14:42 | |
*** rex_lee has quit IRC | 14:42 | |
*** pasquier-s has quit IRC | 14:42 | |
*** sudorandom has quit IRC | 14:42 | |
*** mstyne has quit IRC | 14:42 | |
*** briancline has quit IRC | 14:42 | |
*** eglynn-officeafk has quit IRC | 14:42 | |
*** titilambert has quit IRC | 14:42 | |
*** briancline has joined #openstack-ceilometer | 14:42 | |
*** sudorandom has joined #openstack-ceilometer | 14:42 | |
*** mstyne has joined #openstack-ceilometer | 14:42 | |
*** krotscheck_ is now known as krotscheck | 14:42 | |
*** rex_lee_ is now known as rex_lee | 14:42 | |
gordc | sileht: kk. i will continue brainstorming. | 14:43 |
*** swamireddy has joined #openstack-ceilometer | 14:43 | |
*** pasquier-s has joined #openstack-ceilometer | 14:43 | |
*** ddaskal has joined #openstack-ceilometer | 14:55 | |
*** lexholden has joined #openstack-ceilometer | 14:58 | |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: api: return 410 if only Gnocchi is enabled https://review.openstack.org/202925 | 14:59 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: api: Redirect request to aodh if available https://review.openstack.org/202926 | 14:59 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Allow to enforce API behavior without keystone https://review.openstack.org/203689 | 15:00 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Allow to enforce API behavior without keystone https://review.openstack.org/203689 | 15:01 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: api: return 410 if only Gnocchi is enabled https://review.openstack.org/202925 | 15:03 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: Allow to enforce API behavior without keystone https://review.openstack.org/203689 | 15:03 |
openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/ceilometer: api: Redirect request to aodh if available https://review.openstack.org/202926 | 15:03 |
*** exploreshaifali has quit IRC | 15:16 | |
*** jwcroppe_ is now known as jwcroppe | 15:19 | |
*** deepthi has quit IRC | 15:24 | |
*** rjaiswal has joined #openstack-ceilometer | 15:34 | |
*** tongli has joined #openstack-ceilometer | 15:35 | |
*** lexholden has quit IRC | 15:37 | |
jasonamyers | why is it every time I touch devstack it feels like starting over... | 15:38 |
jasonamyers | guess I've gotten to used to kolla containers | 15:38 |
gordc | never update | 15:38 |
jasonamyers | haha | 15:39 |
jasonamyers | I can never remember how to get ceilo setup in it | 15:39 |
jasonamyers | I have a vagrant config for the other parts | 15:39 |
gordc | http://paste.openstack.org/show/391948/ | 15:40 |
gordc | that's what i have in local.conf | 15:40 |
cdent | jasonamyers: a) never get rid of your local.conf, just always add more and more comments to it and save it away, b) add containers to devstack? | 15:40 |
*** tshtilma has quit IRC | 15:49 | |
*** mgarza_ has joined #openstack-ceilometer | 15:51 | |
openstackgerrit | Merged openstack/aodh: trust: remove useless conf imports https://review.openstack.org/203117 | 15:54 |
*** afazekas_ has quit IRC | 15:57 | |
openstackgerrit | Julien Danjou proposed openstack/gnocchi: metricd: allow to be killed by SIGTERM https://review.openstack.org/203709 | 16:00 |
openstackgerrit | Pradeep Kilambi proposed openstack/ceilometer: Enable entry points for new declarative meters https://review.openstack.org/203032 | 16:08 |
*** vishwanathj has joined #openstack-ceilometer | 16:12 | |
*** ildikov has quit IRC | 16:19 | |
*** _cjones_ has joined #openstack-ceilometer | 16:22 | |
*** julim has quit IRC | 16:22 | |
*** ankita_wagh has joined #openstack-ceilometer | 16:23 | |
*** jwcroppe has quit IRC | 16:23 | |
*** jwcroppe has joined #openstack-ceilometer | 16:23 | |
*** jwcroppe has quit IRC | 16:28 | |
*** jwcroppe has joined #openstack-ceilometer | 16:43 | |
*** Ala has quit IRC | 16:44 | |
*** ankita_wagh has quit IRC | 16:50 | |
*** shardy_ has joined #openstack-ceilometer | 17:01 | |
*** ildikov has joined #openstack-ceilometer | 17:01 | |
*** shardy has quit IRC | 17:02 | |
*** shardy_ has quit IRC | 17:06 | |
*** shardy has joined #openstack-ceilometer | 17:07 | |
*** yprokule has quit IRC | 17:10 | |
*** ankita_wagh has joined #openstack-ceilometer | 17:12 | |
*** openstackgerrit has quit IRC | 17:16 | |
*** openstackgerrit has joined #openstack-ceilometer | 17:17 | |
*** safchain has quit IRC | 17:17 | |
*** tshtilma has joined #openstack-ceilometer | 17:20 | |
*** jwcroppe has quit IRC | 17:21 | |
*** jwcroppe has joined #openstack-ceilometer | 17:21 | |
*** lexholden has joined #openstack-ceilometer | 17:22 | |
openstackgerrit | Rohit Jaiswal proposed openstack/ceilometer: Handles dns.domain.exists event in Ceilometer https://review.openstack.org/202772 | 17:23 |
*** jwcroppe has quit IRC | 17:26 | |
*** lexholden has quit IRC | 17:27 | |
*** lexholden has joined #openstack-ceilometer | 17:27 | |
*** Ephur has joined #openstack-ceilometer | 17:29 | |
*** tshtilma has quit IRC | 17:29 | |
openstackgerrit | Pradeep Kilambi proposed openstack/ceilometer: Delete old notification handlers https://review.openstack.org/203740 | 17:29 |
*** tshtilma has joined #openstack-ceilometer | 17:30 | |
*** lexholden has quit IRC | 17:30 | |
*** lexholden has joined #openstack-ceilometer | 17:32 | |
*** changbl has quit IRC | 17:33 | |
*** Ephur has quit IRC | 17:33 | |
*** lexholden has quit IRC | 17:37 | |
*** lexholden has joined #openstack-ceilometer | 17:39 | |
*** tshtilma has quit IRC | 17:42 | |
*** rjaiswal has quit IRC | 17:44 | |
*** lexholden has quit IRC | 17:44 | |
*** lexholden has joined #openstack-ceilometer | 17:45 | |
*** shardy_ has joined #openstack-ceilometer | 17:59 | |
*** pcaruana has quit IRC | 18:00 | |
*** shardy has quit IRC | 18:01 | |
*** shardy_ has quit IRC | 18:05 | |
*** shardy has joined #openstack-ceilometer | 18:05 | |
*** tshtilma has joined #openstack-ceilometer | 18:06 | |
gordc | cdent: darn. i think i need your pollster-no-transform patch to send single samples... you win. | 18:07 |
cdent | oh yeah? how did you figure that out? | 18:07 |
gordc | cdent: i'm trying to figure out a way to scale out notification agent. | 18:08 |
gordc | i think i can bucketise them using resource_id (or some configurable value later) | 18:08 |
*** primechu_ has joined #openstack-ceilometer | 18:09 | |
gordc | only problem is if it comes in a batch, i will always need to break it down into single datapoints or else the bucketise logic don't work | 18:09 |
*** primechuck has quit IRC | 18:10 | |
* cdent neds | 18:10 | |
cdent | nods | 18:10 |
gordc | i can break it down in notification agent... but maybe it's cleaner to just have it single always so we need to have a check all the time to see if it's single or list... | 18:11 |
cdent | gordc: so I can see two ways to go: a) single samples b) pre-process into lumped by resource on the sending end | 18:11 |
cdent | I think b is a horrible idea because it makes assumptions about the consumer that aren't always going to be true | 18:12 |
gordc | cdent: yeah. b won't be true always. | 18:12 |
gordc | cdent: i'll try breaking it down in notification agent for now... but maybe we can look at single samples in payload across everything later... as a clean up | 18:13 |
cdent | I can change it now if that's useful (especially since it is and always has been the right way to do it ;) ) | 18:13 |
gordc | ... i take it back. i don't want you to be right | 18:13 |
cdent | hah! | 18:14 |
*** tshtilma has quit IRC | 18:14 | |
gordc | maybe make it a second patch... i don't know if anyone will get around to cleaning up payload==single datapoint | 18:14 |
*** lexholden has quit IRC | 18:14 | |
openstackgerrit | Chris Dent proposed openstack/gnocchi: Remove special configuration of heat plugin https://review.openstack.org/203755 | 18:18 |
gordc | cdent: i guess we can batch some places and unbatch at notification agent... i've no idea. i just know it has to happen at notification agent at the latest. will dig more later. | 18:18 |
openstackgerrit | Chris Dent proposed openstack/ceilometer: Send a notification per sample, do not batch https://review.openstack.org/203757 | 18:27 |
cdent | there go gordc ^ | 18:27 |
ekarlso | gordc: how did the discussion ofthe notification stuff go ? | 18:28 |
gordc | cdent: cool cool | 18:29 |
gordc | ekarlso: we didn't come up with formal schema, we will track schema work done elsewhere ie. nova. | 18:29 |
ekarlso | gordc: so stuff like Avro isnt in the concideration space ? | 18:30 |
gordc | ekarlso: we talked about moving our declarative meters to the respective projects so the projects could see the 'schema' ceilometer expects and so projects could control their own metrics | 18:30 |
gordc | ekarlso: no. tbh, we started talking about it and we realised if we scope something large, it won't happen in OpenStack. | 18:31 |
ekarlso | gordc: what's so large with avro ? | 18:31 |
gordc | not avro itself. but to plan something beyond our own project (ceilometer) | 18:32 |
ekarlso | gordc: yeah, sucks that operators dont hammer down on stuff like this :p | 18:32 |
ekarlso | and the big guys' customers | 18:33 |
ekarlso | the confluent schema registry stuff, kafka and avro would be a coolie for notifications / events | 18:33 |
gordc | ekarlso: yeah, i'm not sure exactly sure who needs to initialise it, but it's definitely not a common developer. | 18:33 |
cdent | you're pretty uncommon gordc | 18:38 |
gordc | naw, there's over 1.5 billion of me. i'm very common. :) | 18:40 |
*** lexholden has joined #openstack-ceilometer | 18:40 | |
* gordc wonders if it's cool to talk about race... meh, too late. | 18:40 | |
* cdent sends gordc for reconditioning | 18:44 | |
gordc | lol i still didn't do new hire orientation yet. i blame that. | 18:45 |
jasonamyers | haha | 18:45 |
gordc | jasonamyers: i had a question, do you guys watch pan am games in the states? | 18:48 |
jasonamyers | no | 18:48 |
jasonamyers | at least not in the southern states :P | 18:49 |
gordc | i see.. yeah canada beat usa at baseball (and softball)... i take it we're just facing the B-team... or the F-team. | 18:49 |
jasonamyers | honestly until you just asked me about it, I forgot those even existed | 18:50 |
gordc | lol. yeah, i don't think i watch it normally. it just happens to be in my hometown currently. | 18:51 |
* gordc appreciates the nightly fireworks. | 18:51 | |
*** jwcroppe has joined #openstack-ceilometer | 18:53 | |
*** nijaba has quit IRC | 18:54 | |
jasonamyers | ./stack.sh .... error.... ./unstack.sh | 18:54 |
jasonamyers | repeat forevers | 18:55 |
cdent | jasonamyers: what errors? | 18:55 |
* cdent apparently have devstack kung fu | 18:55 | |
jasonamyers | blew up during migrations this last time | 18:55 |
jasonamyers | waiting for this current stack to do a thing | 18:55 |
*** lexholden has quit IRC | 18:56 | |
*** adam_g` is now known as adam_g | 19:01 | |
*** thorst has quit IRC | 19:01 | |
*** adam_g has quit IRC | 19:01 | |
*** adam_g has joined #openstack-ceilometer | 19:01 | |
jasonamyers | https://www.irccloud.com/pastebin/cnEvtzXF/ | 19:07 |
jasonamyers | cdent that's the current blow up ^ | 19:07 |
jasonamyers | gonna run a clean and retry | 19:07 |
* cdent reads | 19:08 | |
cdent | that's from cinder I believe? I have cinder turned off these days | 19:08 |
cdent | I tend turn off anything not relevant to the current batch of stuff | 19:09 |
cdent | which is good&bad | 19:09 |
cdent | jasonamyers: usually I try to fix stuff as I find it, but as I've never had cause to learn _anything_ about cinder that's one of the ones I skip | 19:14 |
cdent | gordc: do we have any numbers on the kind read concurrency we'd like gnocchi to be able to support, or the read concurrency that existing ceiometer installations do support? | 19:20 |
cdent | jd__: you too on ^ | 19:20 |
jd__ | cdent: BILLIONS | 19:21 |
cdent | maybe nine billion? | 19:21 |
jd__ | cdent: I think currently the concurrency is pretty bad due to mongo etc, but with Gnocchi it should be pretty high and easily scalable for metrics | 19:21 |
jd__ | is there a more deep sense in your question I'm missing? | 19:21 |
cdent | I'm doing some simple apachebench tests with GETs on aggregation | 19:22 |
cdent | trying to see where it goes bad | 19:22 |
cdent | and thus far it is pretty good, but I have a very small amount of data | 19:22 |
gordc | cdent: don't think we have target for reads. not from concurrency pov | 19:23 |
cdent | no deep sense in the question, jd__, just wondering: it is hard to know where to focus energy and analysis when we don't have metrics | 19:23 |
cdent | IRONY! | 19:23 |
gordc | cdent: "not worse than existing dbs" | 19:24 |
jd__ | cdent: yeah I think we're already pretty good on that front so far :) | 19:25 |
cdent | gordc: are you from management? | 19:25 |
gordc | cdent: you see my suit! | 19:26 |
gordc | i expect some "not worse than existing db" charts | 19:26 |
*** _nadya_ has joined #openstack-ceilometer | 19:27 | |
*** _nadya__ has joined #openstack-ceilometer | 19:28 | |
*** _nadya_ has quit IRC | 19:28 | |
*** _nadya__ has quit IRC | 19:29 | |
*** _nadya_ has joined #openstack-ceilometer | 19:30 | |
gordc | prad: is it easier to just enable meter.yaml when it is empty? | 19:31 |
*** changbl has joined #openstack-ceilometer | 19:31 | |
gordc | it might be easier for you to debug, so you can get 1:1 match, turn off one, enbale one | 19:31 |
prad | gordc, we could enable it empty first, if that makes it easier to review too | 19:32 |
prad | i already have a deprecation commit, i'll move the populated meters to that | 19:32 |
gordc | prad: yeah, it'd be easier for me. but i think that's just how my brain works. | 19:32 |
prad | gordc, though we already merged the populated meters.yaml in previous commit | 19:33 |
gordc | prad: yeah... :( | 19:34 |
gordc | it's not doing anything so it's ok to remove i think. | 19:34 |
gordc | prad: because i don't think your expect samples should increase. there's something wrong.. or being duplicated. | 19:35 |
prad | gordc, hmm that would mess up https://review.openstack.org/#/c/203031/3 | 19:35 |
*** miand has joined #openstack-ceilometer | 19:35 | |
prad | gordc, k lemme try that locally first and see | 19:35 |
gordc | prad: yeah, so suggestion is just to switch the order. | 19:35 |
*** miand has quit IRC | 19:35 | |
gordc | enable it and then start adding more definitions and deleting corresponding proces_notification code | 19:36 |
*** thorst has joined #openstack-ceilometer | 19:41 | |
openstackgerrit | Merged openstack/ceilometer: Imported Translations from Transifex https://review.openstack.org/202891 | 19:48 |
openstackgerrit | Merged openstack/ceilometer: make notifier default event publisher https://review.openstack.org/201630 | 19:53 |
prad | gordc, k so narrowed it down to instance event meters, if i remove those from the yaml i get the right samples.. looks like the event_type compute.instance.* is matching more than one meter | 20:18 |
*** _nadya_ has quit IRC | 20:19 | |
*** _nadya_ has joined #openstack-ceilometer | 20:19 | |
prad | gordc, i'll include those separately, i might just have to make the event type explicit instead of .* | 20:20 |
openstackgerrit | Pradeep Kilambi proposed openstack/ceilometer: Enable entry points for new declarative meters https://review.openstack.org/203032 | 20:21 |
*** _nadya_ has quit IRC | 20:24 | |
openstackgerrit | Pradeep Kilambi proposed openstack/ceilometer: Convert identity, sahara and image to meters yaml https://review.openstack.org/203031 | 20:28 |
openstackgerrit | Pradeep Kilambi proposed openstack/ceilometer: Enable entry points for new declarative meters https://review.openstack.org/203032 | 20:28 |
gordc | prad: should the event_type match whatever we have defined in code? | 20:28 |
prad | gordc, it does, but in this case more than one meter has the same event type compute.instance.* | 20:29 |
prad | gordc, instead we probably want the event type to be compute.instance.memory or compute.instance.disk or something like that | 20:29 |
gordc | prad: i'm thinking that's more a bug... i don't really see why it should change. | 20:29 |
cdent | All this breaking stuff has been awesome. Finding lots of assumptions. | 20:30 |
gordc | cdent: i don't like your optimism | 20:30 |
cdent | we're supposed to find bugs | 20:31 |
cdent | that's our primary job | 20:31 |
cdent | it is _not_ to add features | 20:31 |
cdent | however it seems like the only way we can actively find bugs is to create features | 20:31 |
cdent | features which are outside the norm | 20:31 |
gordc | we don't have bugs. | 20:31 |
cdent | bend the rules | 20:31 |
cdent | etc | 20:31 |
* cdent rotflcoptrs | 20:32 | |
gordc | prad: that said, i think it's a bug in your patch. not existing bug. | 20:32 |
gordc | cdent: just fyi, the new release of oslo.messaging is in gate. if we still see random failures we gots problems. | 20:33 |
* cdent lights some candles | 20:33 | |
prad | gordc, i think its just how we define it in the definition.. | 20:34 |
gordc | prad: can you put a breakpoint in your tests to see what the samplesare | 20:34 |
gordc | samples are* just so we can see what the 3 samples are when it should be 2 | 20:34 |
*** jwcroppe has quit IRC | 20:35 | |
gordc | prad: https://github.com/openstack/ceilometer/blob/master/ceilometer/tests/test_notification.py#L203 | 20:36 |
gordc | we only have 2 meters defined, i'm not sure why you'd get 3 meters from one notifications. | 20:36 |
*** jwcroppe has joined #openstack-ceilometer | 20:36 | |
*** yassine__ has quit IRC | 20:37 | |
prad | gordc, yea already did.. i see 1 memory and two instance samples | 20:37 |
gordc | prad: hmm... weird.. do you define it twice in yaml? | 20:37 |
prad | gordc, instead of 1 instance.. and i'm thinking thats because i dint deprecate the instance meters from setup.cfg yet | 20:37 |
gordc | seems like it. | 20:38 |
*** ankita_w_ has joined #openstack-ceilometer | 20:39 | |
*** jwcroppe has quit IRC | 20:40 | |
*** jwcroppe has joined #openstack-ceilometer | 20:40 | |
*** ankita_wagh has quit IRC | 20:42 | |
*** rjaiswal has joined #openstack-ceilometer | 20:45 | |
*** changbl has quit IRC | 20:46 | |
*** thorst has quit IRC | 20:50 | |
openstackgerrit | gordon chung proposed openstack/ceilometer: distributed coordinated notifications https://review.openstack.org/203818 | 20:55 |
*** changbl has joined #openstack-ceilometer | 20:58 | |
* cdent attempts to understand gordc code | 20:59 | |
gordc | cdent: https://docs.google.com/presentation/d/1QgjDOLRnKDboqP8P1LvV0kR5aQEv_VJsDtMlh6u7tIY/edit?usp=sharing | 21:02 |
gordc | i have no idea if slide 7 makes any sense, but that's what i'm trying to do. | 21:02 |
cdent | the commit message made relatively good sense (probably because I was warmed up to the idea already) | 21:03 |
gordc | cdent: coolio. i should also add, i need to test this. | 21:04 |
gordc | i'm guessin we need to enable workload partitioning by default now (especially if we merge your single datapoint patch) | 21:05 |
cdent | gordc: It may just be because I'm sleepy but I can't figure out where you're accounting for agents leaving and joining the group? | 21:13 |
cdent | or more specifically how self.transporters is being kept up to date | 21:14 |
gordc | cdent: i reuse existing logic. https://github.com/openstack/ceilometer/blob/master/ceilometer/notification.py#L164-L165 | 21:21 |
gordc | hmm | 21:22 |
gordc | although i now see i need to reload pipemanager too | 21:23 |
gordc | cdent: good question | 21:23 |
* cdent was getting cake | 21:24 | |
gordc | ankita_w_: what were the error message you weren't getting? the audit ones? | 21:24 |
ankita_w_ | Hi gordc : I am trying to catch nova 500 errors using ceilometer. I see all 200s and 400s in ceilometer but when a 500 error is generated. The outcome says "unknown" . I have enabled the audit middleware | 21:25 |
ankita_w_ | Is this a known issue? | 21:25 |
ankita_w_ | am not able to see any nova 500 errors | 21:26 |
gordc | ankita_w_: do you see unknown event or no event at all? | 21:26 |
ankita_w_ | the outcome of the event says unknown | 21:26 |
*** cdent has quit IRC | 21:26 | |
gordc | ah i see.. it might be some classification logic issue in middleware. | 21:27 |
gordc | i'll take a look | 21:27 |
gordc | ankita_w_: are you using keystonemiddleware or pycadf copy? | 21:28 |
ankita_w_ | am using pycadf copy | 21:28 |
ankita_w_ | since am generating events on nova icehouse | 21:29 |
gordc | hmm. weird. seems like it only sets outcome as unknown if you have no response. | 21:29 |
ankita_w_ | actually sorry....I dont see any events at all | 21:30 |
ankita_w_ | for the 500s | 21:30 |
gordc | oh. | 21:30 |
ankita_w_ | the unknowns are a different issue | 21:30 |
gordc | do you see any errors in nova api logs? | 21:30 |
ankita_w_ | lemme check | 21:31 |
ankita_w_ | I see the error in nova-api | 21:35 |
ankita_w_ | 2015-07-20 21:35:01.954 28966 INFO nova.api.openstack [req-74cbf773-a2b2-4c98-913f-9108d1d9aa1d nova 872bc6b2edd84bbf91c9ceb5bbefd84c] http://nova-api.dvt2.symcpe.net:8774/v2/872bc6b2edd84bbf91c9ceb5bbefd84c/flavors/LMM_ZO_KA returned with HTTP 500 | 21:35 |
gordc | ankita_w_: hmm... do you see the request event or do you not see either request or response event? | 21:37 |
ankita_w_ | no request or response | 21:40 |
*** changbl has quit IRC | 21:43 | |
gordc | ankita_w_: that's weird, if it was an issue with nova i'd think you would still get a request notification | 21:45 |
gordc | seems like it's with middleware itself... do you see any pycadf specific logs? | 21:45 |
ankita_w_ | nopes | 21:48 |
*** htruta_ has joined #openstack-ceilometer | 21:48 | |
ankita_w_ | Can anybody plz confirm, about nova 500 errors | 21:53 |
gordc | gordc: that's strange. the only other place would be notification agent is failing to build sample. i don't see why it'd work for others but fail for error | 21:57 |
gordc | ankita_w_: ^ | 21:57 |
*** ildikov has quit IRC | 21:57 | |
*** rbak_ has joined #openstack-ceilometer | 22:00 | |
*** tongli has quit IRC | 22:00 | |
*** rbak has quit IRC | 22:02 | |
*** changbl has joined #openstack-ceilometer | 22:15 | |
*** edmondsw has quit IRC | 22:19 | |
ankita_w_ | Hi gordc : I tested again...Here's the thing. Its copying entire stacktrace of nova-api error in payload | 22:25 |
*** primechu_ has quit IRC | 22:27 | |
*** changbl has quit IRC | 22:27 | |
gordc | ankita_w_: and it's too large for db? | 22:27 |
ankita_w_ | it doesnt get the reason.reasonCode | 22:28 |
ankita_w_ | I need that | 22:28 |
gordc | i don't quite understand, so you do get the notification but it's missing info or is it so large it's breaking stuff? | 22:29 |
openstackgerrit | gordon chung proposed openstack/ceilometer: distributed coordinated notifications https://review.openstack.org/203818 | 22:36 |
*** changbl has joined #openstack-ceilometer | 22:40 | |
*** ddaskal has quit IRC | 22:52 | |
*** gordc has quit IRC | 22:54 | |
*** rbak_ has quit IRC | 22:54 | |
*** jwcroppe has quit IRC | 23:07 | |
*** prad has quit IRC | 23:09 | |
*** changbl has quit IRC | 23:15 | |
*** bitblt has joined #openstack-ceilometer | 23:20 | |
*** jwcroppe has joined #openstack-ceilometer | 23:22 | |
*** henriquetruta has joined #openstack-ceilometer | 23:24 | |
*** jwcroppe has quit IRC | 23:27 | |
*** htruta_ has quit IRC | 23:28 | |
*** changbl has joined #openstack-ceilometer | 23:29 | |
*** changbl has quit IRC | 23:37 | |
*** changbl has joined #openstack-ceilometer | 23:38 | |
*** jwcroppe has joined #openstack-ceilometer | 23:39 | |
*** mgarza_ has quit IRC | 23:47 | |
*** primechuck has joined #openstack-ceilometer | 23:54 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!