| *** maksimov has joined #openstack-metering | 00:00 | |
| *** dina_belova has joined #openstack-metering | 00:33 | |
| *** dina_belova has quit IRC | 00:37 | |
| *** alexb_ has quit IRC | 00:39 | |
| *** flwang has joined #openstack-metering | 00:41 | |
| *** shaneduan is now known as shaneduan[afk] | 00:47 | |
| *** flwang has quit IRC | 01:10 | |
| *** sandywalsh has quit IRC | 01:14 | |
| *** alexb_ has joined #openstack-metering | 01:16 | |
| *** gordc has joined #openstack-metering | 01:28 | |
| *** dina_belova has joined #openstack-metering | 01:33 | |
| *** sandywalsh has joined #openstack-metering | 01:34 | |
| *** dina_belova has quit IRC | 01:38 | |
| *** fnaval_ has joined #openstack-metering | 01:50 | |
| *** fnaval_ has joined #openstack-metering | 01:51 | |
| *** d34dh0r53 has joined #openstack-metering | 01:59 | |
| *** anteaya has quit IRC | 02:00 | |
| *** d34dh0r53 has quit IRC | 02:02 | |
| *** flwang has joined #openstack-metering | 02:16 | |
| openstackgerrit | Shuangtai Tian proposed a change to openstack/ceilometer: Handle the metrics sent by nova notifier https://review.openstack.org/42838 | 02:22 |
|---|---|---|
| *** dina_belova has joined #openstack-metering | 02:34 | |
| *** dina_belova has quit IRC | 02:38 | |
| *** shaneduan[afk] is now known as shaneduan | 02:46 | |
| openstackgerrit | A change was merged to openstack/ceilometer: Support for wildcard in pipeline https://review.openstack.org/43702 | 02:48 |
| *** zul has quit IRC | 03:00 | |
| *** zul has joined #openstack-metering | 03:02 | |
| *** giroro_ has quit IRC | 03:32 | |
| *** dina_belova has joined #openstack-metering | 03:34 | |
| *** changbl has quit IRC | 03:35 | |
| *** dina_belova has quit IRC | 03:39 | |
| *** gordc has quit IRC | 04:01 | |
| *** dina_belova has joined #openstack-metering | 04:35 | |
| *** dina_belova has quit IRC | 04:39 | |
| *** boris-42 has joined #openstack-metering | 05:04 | |
| *** SergeyLukjanov has joined #openstack-metering | 05:17 | |
| *** dina_belova has joined #openstack-metering | 05:35 | |
| *** shang has joined #openstack-metering | 05:36 | |
| *** dina_belova has quit IRC | 05:40 | |
| flwang | jd__: around? | 05:44 |
| *** dina_belova has joined #openstack-metering | 06:00 | |
| openstackgerrit | Jenkins proposed a change to openstack/ceilometer: Imported Translations from Transifex https://review.openstack.org/43405 | 06:04 |
| *** shang has quit IRC | 06:12 | |
| *** tasdomas_afk is now known as tasdomas | 06:12 | |
| *** shang has joined #openstack-metering | 06:16 | |
| *** shang has quit IRC | 06:25 | |
| *** shang has joined #openstack-metering | 06:41 | |
| openstackgerrit | A change was merged to openstack/ceilometer: Use system locale when Accept-Language header is not provided https://review.openstack.org/43747 | 06:46 |
| *** dina_belova has quit IRC | 06:47 | |
| *** dina_belova has joined #openstack-metering | 06:47 | |
| *** maksimov has quit IRC | 06:50 | |
| *** dina_belova has quit IRC | 06:52 | |
| *** yolanda has joined #openstack-metering | 06:54 | |
| *** Alexei_987 has joined #openstack-metering | 07:13 | |
| openstackgerrit | Mehdi Abaakouk proposed a change to openstack/ceilometer: 'and' and 'or' operations for alarms combination https://review.openstack.org/42832 | 07:14 |
| openstackgerrit | Mehdi Abaakouk proposed a change to openstack/ceilometer: Add alarm combination API https://review.openstack.org/41971 | 07:14 |
| *** sileht has quit IRC | 07:19 | |
| *** sileht has joined #openstack-metering | 07:20 | |
| *** SergeyLukjanov has quit IRC | 07:27 | |
| *** shang has quit IRC | 07:28 | |
| *** shaneduan is now known as shaneduan[afk] | 07:45 | |
| *** shardy_afk is now known as shardy | 07:45 | |
| *** dina_belova has joined #openstack-metering | 07:58 | |
| *** dina_belova has quit IRC | 08:02 | |
| *** eglynn has joined #openstack-metering | 08:13 | |
| boris-42 | jd__ hi | 08:18 |
| jd__ | hey boris-42 | 08:18 |
| jd__ | I was thinking about you | 08:18 |
| boris-42 | jd__ I will take today finally that bug=) | 08:18 |
| boris-42 | jd__ I have some question about https://blueprints.launchpad.net/ceilometer/+spec/ceilometer-db-sync-models-with-migrations | 08:18 |
| jd__ | awesome! | 08:18 |
| boris-42 | jd__ why it is not approved?) | 08:19 |
| jd__ | because nobody asked to do | 08:19 |
| boris-42 | heh=) | 08:20 |
| boris-42 | thanks=) | 08:20 |
| jd__ | though I'm not sure it's going to make it for H3 at this point? | 08:20 |
| boris-42 | jd__ this thing is almost ready https://review.openstack.org/#/c/42307/ | 08:20 |
| jd__ | or is completed? the status is unknown | 08:20 |
| boris-42 | jd__ WIP | 08:20 |
| jd__ | ok I'll set it to Low for H3 | 08:21 |
| boris-42 | jd__ we are trying to make common tests that checks that models and shcemas are synced in different backned | 08:21 |
| boris-42 | jd__ seems reasonable | 08:21 |
| boris-42 | jd__ it is not blocking bp | 08:21 |
| jd__ | yeah that makes sense to have this into oslo.db | 08:21 |
| boris-42 | jd__ then we will be able to avoid using migrations in test | 08:21 |
| boris-42 | jd__ we will use BASE.metadata.create_all() | 08:22 |
| jd__ | yeah I saw Jay was complaining about it | 08:22 |
| boris-42 | yeah he is speaking=) and we already working about it half year=) | 08:22 |
| boris-42 | jd__ it is not so simple to transfer such simple ideas into code in OpenStack=) | 08:25 |
| openstackgerrit | Terri Yu proposed a change to openstack/ceilometer: Adds group by statistics for MongoDB driver https://review.openstack.org/43043 | 08:26 |
| jd__ | indeed | 08:26 |
| jd__ | well done terriyu | 08:26 |
| terriyu | oh | 08:26 |
| terriyu | don't congratulate me yet | 08:26 |
| terriyu | I forgot to run the tests on my rebase | 08:26 |
| boris-42 | ^_^ | 08:27 |
| boris-42 | I see u like risk terriyu | 08:27 |
| boris-42 | =) | 08:27 |
| terriyu | boris-42: haha | 08:28 |
| terriyu | well I'm running the tests now | 08:28 |
| terriyu | and Jenkins is too | 08:28 |
| boris-42 | =) | 08:33 |
| boris-42 | jd__ oh i lost what tests fails? | 08:41 |
| *** Ruetobas has joined #openstack-metering | 08:53 | |
| *** dina_belova has joined #openstack-metering | 08:59 | |
| jd__ | boris-42: tests.unit.db.sqlalchemy.test_utils.TestMigrationUtils.test_util_drop_unique_constraint_with_not_supported_sqlite_type | 08:59 |
| terriyu | jd__: my rebase for the MongoDB group by patch worked. It would be good if we could chat about the API group by tests sometime today, that is if you have time... | 09:03 |
| *** dina_belova has quit IRC | 09:03 | |
| jd__ | terriyu: yes just replied to your mail :) | 09:06 |
| terriyu | jd__: thanks, I'll try to catch you later then. Have a safe trip. | 09:08 |
| jd__ | thanks! | 09:08 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Base Alarm history peristence model https://review.openstack.org/43848 | 09:20 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Plug alarm history logic into the API https://review.openstack.org/43849 | 09:20 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Alarm history storage implementation for mongodb https://review.openstack.org/43850 | 09:20 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Base Alarm history peristence model https://review.openstack.org/43848 | 09:25 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Plug alarm history logic into the API https://review.openstack.org/43849 | 09:25 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Alarm history storage implementation for mongodb https://review.openstack.org/43850 | 09:25 |
| *** tasdomas is now known as tasdomas_afk | 09:27 | |
| *** tasdomas_afk is now known as tasdomas | 09:27 | |
| openstackgerrit | Julien Danjou proposed a change to openstack/ceilometer: api: update v1 for Flask >= 0.10 https://review.openstack.org/43478 | 09:33 |
| *** dina_belova has joined #openstack-metering | 09:59 | |
| *** dina_belova has quit IRC | 10:04 | |
| *** eglynn has quit IRC | 10:05 | |
| *** flwang has quit IRC | 10:23 | |
| *** bogdando has quit IRC | 10:27 | |
| *** eglynn has joined #openstack-metering | 10:33 | |
| *** eglynn is now known as eglynn-dentist | 10:47 | |
| dhellmann_ | jd__: ping | 10:51 |
| jd__ | dhellmann_: pong | 10:53 |
| dhellmann_ | jd__: I'm looking at that thread about a primary key for the meter query, and I'm confused. How does the id from the sample table help? | 10:53 |
| jd__ | dhellmann_: it doesn't for meter query, just for sample query | 10:54 |
| *** dhellmann_ is now known as dhellmann | 10:54 | |
| jd__ | llu corrected this point I think | 10:54 |
| dhellmann | I fail to see how the sample id is right for the pagination, though. It's not a unique key for meters. | 10:55 |
| jd__ | that's why they start talking about using a combination | 10:56 |
| * dhellmann re-reads | 10:56 | |
| jd__ | I mainly responded to the fact they wanted to add an id to the schema where there's already two id in the meter table | 10:56 |
| jd__ | meter table which is actually the sample table | 10:56 |
| dhellmann | right | 10:56 |
| dhellmann | ok, I like the idea of combining some values | 10:56 |
| jd__ | Ceilometer, the most confusing project I ever dealt with :) | 10:56 |
| dhellmann | I think we really only need the combination of resource id and meter name, right? | 10:57 |
| jd__ | and source | 10:57 |
| jd__ | and user_id | 10:57 |
| jd__ | and project_id | 10:57 |
| jd__ | and probably other fields of Samples | 10:57 |
| dhellmann | source, user, and project are implied by the resource id | 10:57 |
| dhellmann | or the meter name | 10:57 |
| jd__ | well, not really | 10:57 |
| jd__ | we never enforced such a thing | 10:57 |
| jd__ | e.g. we never enforced that a resource always belong to always the same user | 10:58 |
| jd__ | that can be true in OpenStack, but not in other metered systems | 10:58 |
| dhellmann | but we also aren't grouping by that in the query we're using to build the result set for the list of meters, right? | 10:58 |
| jd__ | hm I don't know | 10:59 |
| jd__ | we should? :) | 10:59 |
| dhellmann | no | 10:59 |
| dhellmann | that list is "what type of data has been collected?" | 10:59 |
| *** bogdando has joined #openstack-metering | 10:59 | |
| dhellmann | the grouping really only needs to be done in the statistics query | 10:59 |
| jd__ | right | 10:59 |
| jd__ | we don't return project/user etc anyway | 11:00 |
| *** dina_belova has joined #openstack-metering | 11:00 | |
| dhellmann | right | 11:00 |
| dhellmann | it's not relevant here | 11:00 |
| jd__ | so basically the marker should be all fields returned in a Meter | 11:00 |
| dhellmann | no, no | 11:00 |
| jd__ | not in a Sample | 11:00 |
| jd__ | I mean in a Meter | 11:00 |
| dhellmann | the marker only needs to uniquely identify the meter, and the fields you need for that are the meter name and the resource id | 11:00 |
| dhellmann | we have a rule that no 2 meters with the same name should have different units, so that's not needed | 11:01 |
| dhellmann | we don't care that the same meter was collected for more than one user | 11:01 |
| dhellmann | a resource can only belong to one project | 11:01 |
| dhellmann | (or tenant, I forget which word we're using this week) | 11:01 |
| jd__ | that's true for source=openstack | 11:01 |
| jd__ | not necessarily for others | 11:01 |
| jd__ | there isn't any rule for that | 11:02 |
| jd__ | we lack rules. :) | 11:02 |
| dhellmann | if I ask, "what kinds of things do you know about my instance" I don't care that you know that 2 users did things to it and you measured them separately | 11:02 |
| jd__ | agreed | 11:02 |
| dhellmann | if I *do* care, I can ask for the actual statistics separately | 11:02 |
| dhellmann | so the list of meters does not need to include the user, or tenant, or whatever | 11:02 |
| jd__ | but basically, the models.Meter is wrong in this regard | 11:03 |
| dhellmann | but I guess you're strictly right that a resource can move between tenants | 11:03 |
| jd__ | it returns one project_id where they can be several | 11:03 |
| dhellmann | well, let's address that when an actual case comes up :-) | 11:03 |
| jd__ | yeah | 11:03 |
| jd__ | but my point is, it's better to build on rules than on assumptions | 11:04 |
| dhellmann | fair enough | 11:04 |
| *** dina_belova has quit IRC | 11:04 | |
| dhellmann | does the actual marker need to be "transparent" so that API users know what it is? | 11:04 |
| jd__ | I don't know but I'm worried about it being storage driver agnostic | 11:05 |
| dhellmann | because we need to be able to do some sort of comparison for the sort? | 11:05 |
| jd__ | though that may rely on transparent | 11:05 |
| jd__ | well sort is on fields not markers so I'm not sure it's connected? | 11:05 |
| dhellmann | what I was thinking was if we combine some values and base64 encode them, we can keep the caller from knowing what's in the value but still split it apart ourselves to use in different columns in a query | 11:06 |
| dhellmann | I mean, it's not *really* secret, but it's at least demonstrating that they shouldn't pay attention to the value per se | 11:06 |
| dhellmann | we'll have to sort on the marker for it to make any sense | 11:06 |
| jd__ | I see what you mean | 11:06 |
| dhellmann | otherwise there's no way in hell of returning the right results | 11:07 |
| dhellmann | ok, I'll propose that, too | 11:07 |
| jd__ | ah you mean sorting on the fields transported by the marker? | 11:07 |
| dhellmann | yeah | 11:07 |
| jd__ | got it | 11:07 |
| jd__ | sure | 11:07 |
| dhellmann | so if I send resource_id:meter_name base64 encoded, it gets split and then we sort on those 2 columns | 11:07 |
| jd__ | yes | 11:09 |
| jd__ | you need to keep sort= always the same anyway | 11:09 |
| dhellmann | right | 11:09 |
| jd__ | I got to go, lunch's waiting for me | 11:09 |
| dhellmann | and sort based on the marker before any of the user's sort requirements | 11:09 |
| dhellmann | bon appetite | 11:09 |
| jd__ | thanks! | 11:10 |
| *** flwang has joined #openstack-metering | 11:16 | |
| *** dina_belova has joined #openstack-metering | 12:00 | |
| openstackgerrit | litong01 proposed a change to openstack/ceilometer: install manual last few sections format needs to be fixed https://review.openstack.org/43499 | 12:01 |
| *** litong has joined #openstack-metering | 12:04 | |
| *** dina_belova has quit IRC | 12:04 | |
| *** dina_belova has joined #openstack-metering | 12:14 | |
| *** dina_belova has quit IRC | 12:18 | |
| *** eglynn-dentist is now known as eglynn | 12:37 | |
| *** gordc has joined #openstack-metering | 12:46 | |
| openstackgerrit | Svetlana Shturm proposed a change to openstack/ceilometer: Add test for checking models with migrations https://review.openstack.org/43872 | 12:48 |
| *** bpokorny has joined #openstack-metering | 12:50 | |
| *** fnaval_ has quit IRC | 13:02 | |
| openstackgerrit | Svetlana Shturm proposed a change to openstack/ceilometer: Add test for checking models with migrations https://review.openstack.org/43872 | 13:03 |
| *** changbl has joined #openstack-metering | 13:07 | |
| boris-42 | jd__ ping | 13:10 |
| boris-42 | jd__ there was a bug in tests | 13:11 |
| boris-42 | jd__ https://review.openstack.org/#/c/43864/ | 13:11 |
| boris-42 | jd__ here is the fix | 13:11 |
| *** anteaya has joined #openstack-metering | 13:12 | |
| boris-42 | jd__ we don't patch sqla.migrate if we run only this test -> it tries to run Alter in Sqlite -> instead of our approach with create new table copy and drop old table -> produce that bug | 13:12 |
| boris-42 | jd__ so there is no bug in oslo db code hooo | 13:12 |
| openstackgerrit | Svetlana Shturm proposed a change to openstack/ceilometer: Add test for checking models with migrations https://review.openstack.org/43872 | 13:13 |
| gordc | sileht: if you have time and want to look at oslo patches -- can't seem to find any +2, so hoping for some +1s :) : https://review.openstack.org/#/c/41423/17 https://review.openstack.org/#/c/43864/ | 13:14 |
| *** dina_belova has joined #openstack-metering | 13:14 | |
| *** dina_belova has quit IRC | 13:19 | |
| *** changbl has quit IRC | 13:23 | |
| *** fnaval_ has joined #openstack-metering | 13:23 | |
| *** sandywalsh has quit IRC | 13:44 | |
| *** sandywalsh has joined #openstack-metering | 13:57 | |
| openstackgerrit | litong01 proposed a change to openstack/ceilometer: install manual last few sections format needs to be fixed https://review.openstack.org/43499 | 14:02 |
| *** thomasm has joined #openstack-metering | 14:04 | |
| thomasm | Hey all! | 14:05 |
| sandywalsh | yo | 14:05 |
| *** shaneduan[afk] is now known as shaneduan | 14:07 | |
| *** shaneduan is now known as shaneduan[afk] | 14:09 | |
| openstackgerrit | Svetlana Shturm proposed a change to openstack/ceilometer: Add test for checking models with migrations https://review.openstack.org/43872 | 14:09 |
| *** dina_belova has joined #openstack-metering | 14:15 | |
| sandywalsh | jd__, any thoughts of jumping in on this mid-icehouse meetup movement going on the ML? | 14:15 |
| sandywalsh | would be great for CM I think ... we're getting large enough that it would be really useful. | 14:16 |
| *** dina_belova has quit IRC | 14:19 | |
| openstackgerrit | Svetlana Shturm proposed a change to openstack/ceilometer: Add test for checking models with migrations https://review.openstack.org/43872 | 14:20 |
| openstackgerrit | Cyril Roelandt proposed a change to openstack/ceilometer: Network: process metering reports from Neutron https://review.openstack.org/43892 | 14:23 |
| sandywalsh | Logstash joins Elastic Search http://www.elasticsearch.com/blog/welcome-jordan-logstash/ | 14:26 |
| *** changbl has joined #openstack-metering | 14:30 | |
| thomasm | Ooooo | 14:31 |
| *** dina_belova has joined #openstack-metering | 14:32 | |
| *** alexb_ has quit IRC | 14:37 | |
| openstackgerrit | gordon chung proposed a change to openstack/ceilometer: API FunctionalTest class lacks doc strings https://review.openstack.org/43897 | 14:50 |
| *** alexb_ has joined #openstack-metering | 14:57 | |
| openstackgerrit | gordon chung proposed a change to openstack/ceilometer: API FunctionalTest class lacks doc strings https://review.openstack.org/43897 | 15:04 |
| *** shaneduan[afk] is now known as shaneduan | 15:15 | |
| *** shang has joined #openstack-metering | 15:31 | |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Plug alarm history logic into the API https://review.openstack.org/43849 | 15:39 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Alarm history storage implementation for mongodb https://review.openstack.org/43850 | 15:39 |
| openstackgerrit | gordon chung proposed a change to openstack/ceilometer: API FunctionalTest class lacks doc strings https://review.openstack.org/43897 | 15:39 |
| *** shadower is now known as shadower|ragequi | 15:40 | |
| *** shadower|ragequi is now known as shadower | 15:40 | |
| *** tasdomas is now known as tasdomas_afk | 15:53 | |
| *** shang has quit IRC | 15:58 | |
| *** yjiang5_away is now known as yjiang5 | 16:03 | |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Base Alarm history peristence model https://review.openstack.org/43848 | 16:18 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Plug alarm history logic into the API https://review.openstack.org/43849 | 16:18 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Alarm history storage implementation for mongodb https://review.openstack.org/43850 | 16:18 |
| *** shang has joined #openstack-metering | 16:32 | |
| *** herndon_ has joined #openstack-metering | 16:36 | |
| *** dina_belova has quit IRC | 16:39 | |
| *** dina_belova has joined #openstack-metering | 16:40 | |
| *** dina_belova has quit IRC | 16:44 | |
| openstackgerrit | Svetlana Shturm proposed a change to openstack/ceilometer: Add test for checking models with migrations https://review.openstack.org/43872 | 16:45 |
| openstackgerrit | A change was merged to openstack/ceilometer: missing resource in middleware notification https://review.openstack.org/43585 | 16:56 |
| openstackgerrit | A change was merged to openstack/ceilometer: api: update v1 for Flask >= 0.10 https://review.openstack.org/43478 | 16:57 |
| *** shaneduan is now known as shaneduan[afk] | 16:59 | |
| *** dina_belova has joined #openstack-metering | 17:00 | |
| openstackgerrit | A change was merged to openstack/ceilometer: API FunctionalTest class lacks doc strings https://review.openstack.org/43897 | 17:02 |
| *** Alexei_987 has quit IRC | 17:04 | |
| openstackgerrit | Svetlana Shturm proposed a change to openstack/ceilometer: Add test for checking models with migrations https://review.openstack.org/43872 | 17:08 |
| dhellmann | eglynn: ping? | 17:19 |
| eglynn | dhellmann: hey | 17:19 |
| dhellmann | hey, I'm looking at the alarm history stuff | 17:19 |
| eglynn | dhellmann: thanks | 17:19 |
| dhellmann | do we need a new table for that? or could we just use a meter? | 17:19 |
| dhellmann | I thought we talked about using a meter at the summit, but maybe I misremembered | 17:20 |
| eglynn | well I wanted to have the alarm history retention period independent of the TTL for meters in general | 17:20 |
| eglynn | seemed the best way of achieving this was a via a seperate table | 17:21 |
| dhellmann | that's a fair point | 17:21 |
| dhellmann | I've been out of the loop a good bit, do we have TTL handling now? | 17:21 |
| eglynn | (admittedly I'm still implementing the retention logic, so not in the current patch series) | 17:21 |
| eglynn | we do have TTL for meters | 17:21 |
| eglynn | controlled via config, off my default | 17:22 |
| eglynn | s/my/by/ | 17:22 |
| dhellmann | and I guess that doesn't allow for different TTL by meter name | 17:22 |
| eglynn | dhellmann: yep exactly, it's accross the board for everything | 17:23 |
| dhellmann | are we at least emitting meters when alarms change state? | 17:23 |
| eglynn | not currently | 17:23 |
| dhellmann | ok | 17:23 |
| eglynn | it would be straight-forward to add I think, but time is getting very tight at this stage for h3 | 17:24 |
| dhellmann | yeah | 17:24 |
| dhellmann | ok, I guess we can go ahead with a separate table, then | 17:24 |
| eglynn | dhellmann: cool :) | 17:25 |
| dhellmann | for get_alarm_changes() the first argument is an alarm_id -- is that a uuid? | 17:25 |
| dhellmann | because the second argument seems to be filtering on tenant, which I wouldn't expect for a specific alarm | 17:25 |
| *** shang has quit IRC | 17:26 | |
| eglynn | dhellmann: yep, first arg is a UUID | 17:26 |
| dhellmann | isn't an alarm always owned by a tenant? I don't understand the 2nd arg. | 17:27 |
| eglynn | dhellmann: the idea of the on_behalf_of is to allow the threshold evaluator make changes visible to the owner of the alarm | 17:27 |
| eglynn | dhellmann: so we record the project_id of the initiator of the change | 17:28 |
| eglynn | dhellmann: and the tenant ID that the change is made on behalf of | 17:28 |
| eglynn | dhellmann: then when history is accessed, it's scoped by the X-Project-Id matching the on_behalf_of fields | 17:29 |
| dhellmann | is a single alarm ever going to have changes because of multiple tenants? | 17:29 |
| dhellmann | maybe you're counting changes to the definition of the alarm as a change? | 17:29 |
| dhellmann | so the owner of the alarm and the owner of the resource being watched might be different? | 17:29 |
| eglynn | yes, rule changes and state transition all count | 17:29 |
| dhellmann | got it, ok | 17:30 |
| *** dina_belova has quit IRC | 17:30 | |
| dhellmann | so if I own an alarm I can see the changes I make and if I own a monitored resource I can see the changes related to the resource | 17:30 |
| *** dina_belova has joined #openstack-metering | 17:31 | |
| eglynn | dhellmann: that's a shortcoming currently that I have on my list | 17:31 |
| eglynn | dhellmann: i.e. to restrict the visibility of the resources to scope by the tenant owning the alarm | 17:32 |
| dhellmann | I thought that the way it works now, where a tenant can only see a subset of the changes, was the way you wanted it to work | 17:33 |
| eglynn | dhellmann: yes that is the way I want it to work for alarm history | 17:34 |
| eglynn | (I was talking about a similar change I also intend to make to the threshold evaluation logic) | 17:35 |
| dhellmann | ah | 17:35 |
| *** dina_belova has quit IRC | 17:35 | |
| dhellmann | eglynn: is this written up somewhere? I'm looking through the blueprints but the first couple I hit have no external link | 17:35 |
| sandywalsh | eglynn, dhellmann wouldn't an alarm be an event? | 17:36 |
| eglynn | dhellmann: no I didn't write up the on_behalf_of idea on the wiki or anywhere similar | 17:36 |
| eglynn | dhellmann: but I could do so if that would be considered useful? | 17:36 |
| dhellmann | sandywalsh: yes, it will need to be eventually | 17:36 |
| dhellmann | sandywalsh: are you suggesting using the event table instead of a separate alarm table? will that work with the TTL requirements? | 17:38 |
| eglynn | sandywalsh: I hadn't considered that | 17:38 |
| eglynn | sandywalsh: is the event logic implemented for all the storage drivers? | 17:38 |
| sandywalsh | dhellmann, yes, exactly. An alarm event would be no different than a "create.instance" event | 17:38 |
| sandywalsh | eglynn, no, it's only sqlalchemy currently, but we may be switching to mongo anyway, so it'll need to go there. | 17:39 |
| sandywalsh | (it's pretty simple) | 17:39 |
| eglynn | sandywalsh: I took the opposite approach and implemented for mongo first | 17:39 |
| eglynn | (i.e. for the proposed alarm history) | 17:40 |
| sandywalsh | eglynn, right, our storage requirements were looking like mongo would be a bad choice initially. We've re-calculated and think we can do away with needing shards | 17:40 |
| eglynn | sandywalsh: interesting, so the focus shifting to mongo? | 17:41 |
| sandywalsh | eglynn, looking that way ... still running scaling tests | 17:41 |
| sandywalsh | eglynn, if you use the event table, you'll get the trigger pipeline and all kinds of other goodies (vs a new table) | 17:41 |
| sandywalsh | eglynn, and alarming will *definitely* want the trigger pipeline | 17:42 |
| eglynn | sandywalsh: yep, for icehouse, I'm feeling some reconciliation of the alarms and events logic would be useful | 17:43 |
| eglynn | sandywalsh: (but probably premature for H, or?) | 17:43 |
| dhellmann | eglynn: I'd rather focus effort on making alarm history stuff use events than build something completely separate that we're going to throw away | 17:44 |
| sandywalsh | eglynn, icehouse certainly | 17:44 |
| dhellmann | because if we use events now, we don't need an api that we'll end up deprecating, right? | 17:45 |
| *** dina_belova has joined #openstack-metering | 17:45 | |
| eglynn | dhellmann: well except that the events stuff is still a moving target by the sounds of it, and I need to get something simple in place for H | 17:46 |
| eglynn | dhellmann: the alarm history API could facade over the event stuff later on down the road, or? | 17:47 |
| dhellmann | eglynn: yeah, it could | 17:47 |
| sandywalsh | eglynn, is anyone likely to deploy alarms from H? | 17:47 |
| eglynn | sandywalsh: well, for heat autoscaling at least, I think so | 17:48 |
| dhellmann | I thought I saw a bunch of work on the event stuff, was that just on the storage side? | 17:48 |
| * dhellmann is too far out of the loop | 17:48 | |
| sandywalsh | most of the event stuff has been in oslo actually (retries, etc) ... but the api stuff is there too now. | 17:49 |
| sandywalsh | and dragondm has the event->meter schema branch up for review as well (very very cool) | 17:49 |
| sandywalsh | data driven event->meter mapping :) | 17:49 |
| sandywalsh | I'm wrestling with alembic :( | 17:49 |
| *** shaneduan[afk] is now known as shaneduan | 17:50 | |
| sandywalsh | storage has been a big part of it though, yes. jaypipes and herndon_ have been tweaking it a lot | 17:50 |
| * dhellmann is torn | 17:51 | |
| eglynn | I feel it would be difficult to rebase alarm history on events at this late stage, but it's useful enough to get it for H | 17:52 |
| eglynn | (by "it's useful enough", I mean a standalone simple alarm history API) | 17:53 |
| sandywalsh | I'd probably try to cram it into Meter for now if that's the case. You're correct that api shouldn't be affected and that's the biggest touch point. | 17:55 |
| sandywalsh | (and it's less code to write) | 17:55 |
| sandywalsh | hmm | 17:56 |
| sandywalsh | actually | 17:56 |
| sandywalsh | would there need to be an alarm history API if it's using events? | 17:57 |
| sandywalsh | wouldn't you just look for event-type = 'XXX-alarm' ? | 17:57 |
| dragondm | sandywalsh: actually it's notification->event mapping up atm. event-> meter (sample) is next :-> | 17:57 |
| eglynn | as a convenience facade | 17:57 |
| herndon_ | dhellmann: event api is hung up on sqlalchemy migration, probably not going to make H release at this point | 17:57 |
| sandywalsh | dragondm, sorry, yes | 17:57 |
| sandywalsh | eglynn, hmm, how big a convenience is it? :) | 17:58 |
| gordc | dhellmann: when you have time, do you think this is a valid test case for the process_request failure in middleware.notifier (i've included error decorator in middleware.notifier code: http://paste.openstack.org/show/45223/ | 17:58 |
| sandywalsh | herndon_, I'm hung on it too | 17:58 |
| sandywalsh | jd__, ping? | 17:58 |
| herndon_ | apparently mirantis are working on it, hoping this will solve all of our problems: https://review.openstack.org/#/c/43872 | 17:58 |
| sandywalsh | I'm wondering if I should switch my alembic migrations back to sqlalchemy-migration until this alembic thing is fixed | 17:59 |
| herndon_ | given that there are already alembic migrations in, I'm not sure that will work. | 17:59 |
| dhellmann | gordc: that looks like a good start for testing process_request() | 18:00 |
| *** SergeyLukjanov has joined #openstack-metering | 18:00 | |
| herndon_ | (and I got -1'd for using sqlachemy-migration) | 18:00 |
| eglynn | yeah, I thought we were pretty much stuck with alembic now, or? | 18:00 |
| dhellmann | gordc: it may be tricky to mock notify() so it works in process_request but fails in process_response | 18:00 |
| sandywalsh | herndon_, you can reorder alembic migrations pretty easily ... I'm just concerned about 1000+ mysql errors :( | 18:00 |
| sandywalsh | eglynn, I went down the alembic route (being a good boy), but it's a bag of holes currently. | 18:01 |
| dhellmann | herndon_, sandywalsh : let's talk about migrations at the meeting this week -- I agree we should drop alembic if it's going to hold up the release, I thought it was already working. | 18:01 |
| sandywalsh | dhellmann, +1 | 18:01 |
| herndon_ | sqlite + alembic is a problem. I agree | 18:01 |
| thomasm | What was the motivation for Alembic? | 18:02 |
| eglynn | sandywalsh: is there even an option to be a bad boy and revert to sqlalchemy-migrate? | 18:02 |
| dhellmann | eglynn: sqlalchemy-migrate is still there, so yes | 18:03 |
| eglynn | k | 18:03 |
| sandywalsh | https://wiki.openstack.org/wiki/Meetings/MeteringAgenda#Agenda | 18:03 |
| dhellmann | the plan was to make alembic work and then stop adding new migrations with s-m | 18:03 |
| dhellmann | thomasm: alembic has better handling for merge conflicts, and sqlalchemy-migrate is no longer maintained by the author | 18:03 |
| thomasm | Ohhhh okay | 18:03 |
| sandywalsh | it's much better, but just not fully baked in CM yet | 18:03 |
| dhellmann | yeah | 18:04 |
| thomasm | Gotcha | 18:04 |
| gordc | dhellmann: you're right, not sure what i could mock that could pass one and fail the other. did the decorator look fine to you? | 18:04 |
| thomasm | :) | 18:04 |
| sandywalsh | eglynn, is there an api proposal for alarm history? | 18:04 |
| sandywalsh | (looking to read up) | 18:04 |
| dhellmann | sandywalsh: are there changes under review to fix the problems you and jay found? | 18:04 |
| dhellmann | gordc: mock.patch lets you provide multiple return values, I think | 18:05 |
| sandywalsh | dhellmann, jay's is up, I've been fixing mine locally, but there's a paste in the ML discussion on the topic. | 18:05 |
| dhellmann | gordc: but you could just write tests that call the methods directly | 18:05 |
| eglynn | sandywalsh: the skeletal history API is already landed | 18:05 |
| sandywalsh | eglynn, thanks, I'll give it a read | 18:05 |
| dhellmann | gordc: yes, I think so. maybe log_and_ignore_error()? | 18:05 |
| dhellmann | gordc: we want to make it clear that the error doesn't go any further | 18:06 |
| eglynn | sandywalsh: the current set of patches fill out the storage model and plug that into the (currently inoperative) API | 18:06 |
| gordc | dhellmann: ok i'll make that adjustment. i'll try calling methods directly to see if the decorator still gets triggered. | 18:07 |
| sandywalsh | eglynn, I think the only sticking point against Events (currently) would be the json payload. apmelton is working on the Event Body branch (fast access json for sqlalchemy) and you'd need that. | 18:08 |
| sandywalsh | eglynn, or you could stuff the json into Traits for the time being. | 18:09 |
| sandywalsh | (a little icky) | 18:09 |
| eglynn | sandywalsh: I'll take a look at the events stuff in more detail tmrw (gotta hit the road from the office fairly soon) | 18:10 |
| eglynn | (but I'd be nervous of changing course this late for H) | 18:10 |
| sandywalsh | yep, just something to consider | 18:11 |
| eglynn | sandywalsh: yep, absolutely | 18:11 |
| * eglynn is just back from two weeks vacation, so still a bit off the pace in terms of recent developments | 18:11 | |
| litong | hi folks, can you look at this patch, it is to fix the format problems in the manual. https://review.openstack.org/#/c/43499/ | 18:18 |
| litong | here is the results from the build. http://docs-draft.openstack.org/99/43499/6/check/gate-ceilometer-docs/fa503d2/doc/build/html/install/manual.html | 18:18 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Base Alarm history peristence model https://review.openstack.org/43848 | 18:25 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Plug alarm history logic into the API https://review.openstack.org/43849 | 18:25 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/ceilometer: Alarm history storage implementation for mongodb https://review.openstack.org/43850 | 18:25 |
| *** dina_belova has quit IRC | 18:27 | |
| *** dina_belova has joined #openstack-metering | 18:28 | |
| *** eglynn has quit IRC | 18:30 | |
| *** dina_belova has quit IRC | 18:32 | |
| *** anteaya has quit IRC | 18:33 | |
| *** shaneduan is now known as shaneduan[afk] | 18:39 | |
| gordc | dhellmann: could you take a look at this patch as well: https://review.openstack.org/#/c/42179/ and if you have any issues with it aside from log/ignore exceptions in process_request/response. | 18:42 |
| *** shaneduan[afk] is now known as shaneduan | 18:43 | |
| openstackgerrit | Svetlana Shturm proposed a change to openstack/ceilometer: Extra indexes cleanup https://review.openstack.org/42716 | 18:48 |
| *** maksimov has joined #openstack-metering | 18:52 | |
| *** anteaya has joined #openstack-metering | 18:54 | |
| *** maksimov has quit IRC | 18:58 | |
| *** maksimov has joined #openstack-metering | 18:58 | |
| terriyu | asalkeld: did you write some of the API Statistics tests? | 19:03 |
| terriyu | oops, I think it's 5 am in Australia | 19:17 |
| thomasm | lol | 19:17 |
| terriyu | I'm still learning what time zone everyone is in :) | 19:18 |
| thomasm | You and me both. =P | 19:19 |
| dragondm | heh ya. | 19:20 |
| dragondm | well, my body is in cdt, my brain thinks it's pdt. :> | 19:21 |
| *** mberwanger has joined #openstack-metering | 19:21 | |
| dragondm | Anyone know what breed of chicken I need to sacrifice to get someone to review a patch in openstack/requirements :> ? | 19:23 |
| thomasm | A silkie | 19:23 |
| thomasm | Pretty sure that's it, yeah. | 19:23 |
| dragondm | *floof* | 19:24 |
| thomasm | They're so fluffy! | 19:24 |
| *** dina_belova has joined #openstack-metering | 19:28 | |
| *** dina_belova has quit IRC | 19:33 | |
| *** dina_belova has joined #openstack-metering | 19:39 | |
| *** mberwanger has quit IRC | 19:41 | |
| *** dina_belova has quit IRC | 19:43 | |
| terriyu | dhellmann: can I ask you some quick questions about API testing? | 19:49 |
| *** eglynn has joined #openstack-metering | 19:56 | |
| *** eglynn has quit IRC | 20:02 | |
| *** eglynn has joined #openstack-metering | 20:10 | |
| openstackgerrit | gordon chung proposed a change to openstack/ceilometer: add tests for _query_to_kwargs func https://review.openstack.org/43796 | 20:10 |
| dhellmann | terriyu: sure, what's up? | 20:10 |
| terriyu | dhellmann: thanks for taking time out of your busy schedule to help me out | 20:10 |
| dhellmann | dragondm: add me as a reviewer for your change | 20:11 |
| terriyu | I just wanted to ask you some general questions about API testing | 20:11 |
| dhellmann | terriyu: sure | 20:11 |
| terriyu | jd__ and I wrote the group by implementation in the storage drivers and I also wrote the matching storage driver tests | 20:11 |
| terriyu | I'm working on the API tests for the group by implementation and I'm wondering if it's supposed to be very similar to the storage tests, or something different? | 20:12 |
| dhellmann | unit tests should focus on each layer separately | 20:12 |
| dhellmann | so for the storage tests I would expect to see tests that verify that the driver stores and retrieves data properly | 20:12 |
| dhellmann | for the api I would expect to see that the storage driver is called with the right arguments and that errors are handled correctly | 20:12 |
| dragondm | dhellmann: done! | 20:13 |
| dhellmann | that way you don't duplicate test setup for the 2 layers, and if there is a problem you can isolate it with a test | 20:13 |
| dhellmann | dragondm: why do we need a new json parser? | 20:14 |
| dragondm | it's not a json parser. it's basically a json version of xpath | 20:14 |
| dragondm | it provides a quick syntax for fishing an element out of a deserialized json object. | 20:15 |
| dhellmann | ah | 20:15 |
| dhellmann | I misread it | 20:15 |
| dhellmann | is that for event queries or something? | 20:15 |
| terriyu | dhellmann: so for the API tests, I would pass some arguments to get_json() and then check the results? And not worry about what the storage drivers are doing? | 20:15 |
| dragondm | for the configurable-events branch. For writing a config to transform notifications -> event traits. | 20:16 |
| dhellmann | terriyu: you will probably want to replace the storage driver with a mock so you can control what it returns | 20:16 |
| dhellmann | use mock.patch to do that, and then call get_json() to make the REST call | 20:16 |
| jd__ | dhellmann: actually we test the API on real storage drivers to be sure they all returns the same data | 20:16 |
| dragondm | (and ultimately for the event-pipelines) | 20:16 |
| dhellmann | dragondm: why do we want that to be configurable? to cut down on the amount of data stored in the traits table? | 20:17 |
| jd__ | dhellmann: terriyu: it's damn more reasonable in the end, since we had terrrrrrrrrrrrrrible differences between MongoDB and SQLAlchemy back in the early daus | 20:17 |
| dhellmann | jd__: I thought we were handling that in the db tests now? | 20:17 |
| dragondm | Yah, and to configure what we need. | 20:18 |
| dragondm | So we don't need to edit code every time we want to add a trait to an event. | 20:18 |
| jd__ | dhellmann: we do both now, it's safer -- I fixed a lot of bugs with that before we have the models | 20:18 |
| dhellmann | ok | 20:19 |
| dragondm | (and, for example RAX will have custom traits that will be used by event-pipelines ) | 20:19 |
| dhellmann | terriyu: what jd__ said :-) | 20:19 |
| jd__ | :-) | 20:19 |
| dhellmann | dragondm: well, I would have thought we'd just flatten the entire payload of the event into traits | 20:19 |
| terriyu | dhellmann: thanks for your help :) | 20:19 |
| dragondm | yah, mostly. But there are some complexities in that. Notifications have nested structure, event traits are flat. | 20:21 |
| dragondm | And while alot of the notification fields would become traits, we really only want the ones we will be actively using. (for db size reasons) | 20:23 |
| dragondm | (We've calculated we will be storing 10's of millions of events. It adds up) | 20:24 |
| dhellmann | dragondm: the size thing is the more compelling reason :-) | 20:25 |
| dragondm | Yah. Plus we can add custom traits for our installation w/o code patches. (we have internal values that are screwball bitfields derived from other fields in the notification) | 20:27 |
| *** yolanda has quit IRC | 20:27 | |
| dragondm | Or do minor data massaging. (nova has an annoying habit of using "" for null dates for instance. ) | 20:27 |
| *** dina_belova has joined #openstack-metering | 20:28 | |
| dragondm | Anyway, the configurable-events code is up for review at: https://review.openstack.org/#/c/42713/ I've tried to make the document in the linked bp fairly informative. | 20:30 |
| dragondm | (and thanks for taking care of the requirements thing. :> It got derailed in the testr/subunit bug fiasco on friday) | 20:31 |
| dhellmann | dragondm: no problem :-) | 20:32 |
| openstackgerrit | Monsyne Dragon proposed a change to openstack/ceilometer: Add configuration-driven conversion to Events https://review.openstack.org/42713 | 20:34 |
| *** mberwanger has joined #openstack-metering | 20:38 | |
| openstackgerrit | Cyril Roelandt proposed a change to openstack/ceilometer: Implement 'reset_on' for mongodb https://review.openstack.org/42355 | 20:48 |
| openstackgerrit | Cyril Roelandt proposed a change to openstack/ceilometer: Statistics: Add a "reset_on" query parameter https://review.openstack.org/41899 | 20:48 |
| *** mberwanger has quit IRC | 20:51 | |
| *** mberwanger has joined #openstack-metering | 20:53 | |
| *** thomasm has quit IRC | 20:53 | |
| *** openstackgerrit has quit IRC | 20:55 | |
| *** openstackgerrit has joined #openstack-metering | 20:56 | |
| *** eglynn has quit IRC | 21:00 | |
| *** Ruetobas has quit IRC | 21:01 | |
| herndon_ | dragondm: worth looking at your branch yet, or should I hold off for a while longer? | 21:07 |
| dragondm | Yah, I have to get jenkins to recheck when the change to openstack/requirements lands, but that's been approved, so the events code shouldn't change. Have at it. | 21:08 |
| *** litong has quit IRC | 21:13 | |
| *** mberwanger has quit IRC | 21:13 | |
| *** yolanda has joined #openstack-metering | 21:17 | |
| *** sandywalsh has quit IRC | 21:20 | |
| *** maksimov has quit IRC | 21:20 | |
| *** dina_belova has quit IRC | 21:29 | |
| *** dina_belova has joined #openstack-metering | 21:29 | |
| *** sandywalsh has joined #openstack-metering | 21:33 | |
| *** dina_belova has quit IRC | 21:34 | |
| *** yolanda has quit IRC | 21:34 | |
| *** changbl has quit IRC | 21:41 | |
| *** evanjfraser has quit IRC | 21:42 | |
| *** boris-42 has quit IRC | 22:02 | |
| *** SergeyLukjanov has quit IRC | 22:05 | |
| *** bpokorny has quit IRC | 22:10 | |
| *** shardy is now known as shardy_afk | 22:13 | |
| *** evanjfraser has joined #openstack-metering | 22:18 | |
| *** dina_belova has joined #openstack-metering | 22:40 | |
| *** dina_belova has quit IRC | 22:45 | |
| *** thomasm has joined #openstack-metering | 22:53 | |
| *** herndon_ has quit IRC | 23:16 | |
| *** anteaya has quit IRC | 23:21 | |
| *** alexb_ has quit IRC | 23:22 | |
| *** herndon_ has joined #openstack-metering | 23:29 | |
| *** dina_belova has joined #openstack-metering | 23:40 | |
| *** herndon_ has quit IRC | 23:41 | |
| *** dina_belova has quit IRC | 23:45 | |
| *** fnaval_ has quit IRC | 23:48 | |
| *** maksimov has joined #openstack-metering | 23:52 | |
| *** jergerber has quit IRC | 23:56 | |
| *** gordc has quit IRC | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!