Thursday, 2013-09-05

*** shaneduan is now known as shaneduan[afk]00:13
*** dina_belova has joined #openstack-metering00:14
*** shaneduan[afk] is now known as shaneduan00:15
*** eglynn-is-tired has quit IRC00:16
*** dina_belova has quit IRC00:19
*** nosnos has joined #openstack-metering00:36
*** osphy has quit IRC00:55
*** dina_belova has joined #openstack-metering01:15
*** changbl has joined #openstack-metering01:17
*** dina_belova has quit IRC01:19
*** shaneduan is now known as shaneduan[afk]01:22
*** shakayumi has quit IRC01:22
*** d34dh0r53 has joined #openstack-metering01:59
*** d34dh0r53 has quit IRC02:01
*** shaneduan[afk] is now known as shaneduan02:02
*** dina_belova has joined #openstack-metering02:15
*** dina_belova has quit IRC02:20
*** shaneduan is now known as shaneduan[afk]02:31
*** evanjfraser_ has joined #openstack-metering02:33
*** evanjfraser has quit IRC02:34
*** evanjfraser_ has quit IRC02:38
openstackgerritHaomeng,Wang proposed a change to openstack/ceilometer: Change resource.resource_metadata column type to Text to store entire JSON string without truncating.  https://review.openstack.org/4506402:55
openstackgerritLianhao Lu proposed a change to openstack/ceilometer: Adding hardware-agent  https://review.openstack.org/4307203:06
openstackgerritHaomeng,Wang proposed a change to openstack/ceilometer: Change resource.resource_metadata column type to Text  https://review.openstack.org/4506403:06
*** dina_belova has joined #openstack-metering03:16
*** dina_belova has quit IRC03:20
*** sandywalsh has joined #openstack-metering03:26
*** sandywalsh_ has quit IRC03:27
*** terriyu has quit IRC03:46
*** evanjfraser has joined #openstack-metering03:51
*** sdake_ has joined #openstack-metering04:00
*** sdake_ has quit IRC04:00
*** sdake_ has joined #openstack-metering04:00
*** dina_belova has joined #openstack-metering04:16
*** gordc has joined #openstack-metering04:20
*** dina_belova has quit IRC04:21
*** dina_belova has joined #openstack-metering05:17
*** evanjfraser has quit IRC05:17
openstackgerritHaomeng,Wang proposed a change to openstack/ceilometer: Change resource.resource_metadata column type to Text  https://review.openstack.org/4506405:20
*** dina_belova has quit IRC05:21
*** tasdomas_afk is now known as tasdomas05:21
*** shang has quit IRC05:22
*** gordc has quit IRC05:39
*** SergeyLukjanov has joined #openstack-metering05:40
*** dina_belova has joined #openstack-metering05:57
openstackgerritJenkins proposed a change to openstack/ceilometer: Imported Translations from Transifex  https://review.openstack.org/4517506:07
openstackgerritLianhao Lu proposed a change to openstack/ceilometer: Fixed nova nofitier unit test  https://review.openstack.org/4517606:09
llu06:10
lluif anyone wakes up, please look at the patch https://review.openstack.org/45176, our Jenkin's got stuck by the latest nova changes.06:11
*** eglynn-is-tired has joined #openstack-metering06:16
*** dina_belova has quit IRC06:36
*** evanjfraser has joined #openstack-metering06:42
*** SergeyLukjanov has quit IRC06:45
*** dina_belova has joined #openstack-metering06:49
openstackgerritLianhao Lu proposed a change to openstack/ceilometer: Fixed nova nofitier unit test  https://review.openstack.org/4517606:49
*** evanjfraser has quit IRC06:50
*** dina_belova has quit IRC06:53
openstackgerritFengqian.gao proposed a change to openstack/ceilometer: Add pagination support for sqlalchemy database  https://review.openstack.org/3545406:55
openstackgerritFengqian.gao proposed a change to openstack/ceilometer: Change pagination query method in mongodb  https://review.openstack.org/4186906:55
*** anteaya has quit IRC07:04
*** shardy_afk is now known as shardy07:05
*** eglynn-is-tired has quit IRC07:07
*** shang has joined #openstack-metering07:37
*** eglynn-is-tired has joined #openstack-metering07:47
*** boris-42 has quit IRC07:51
*** graflu0 has joined #openstack-metering07:56
*** lexx has joined #openstack-metering07:57
*** eglynn-is-tired is now known as eglynn07:58
*** graflu0 has quit IRC07:58
silehteglynn, hi08:01
eglynnsileht: hey08:01
silehteglynn, I work a bit on meta-alarm, dhellmann_ have a interresting remarks about this08:02
silehteglynn, if you have some times to take a look: https://review.openstack.org/#/c/41971/13/ceilometer/api/controllers/v2.py08:02
eglynnsileht: looking now ...08:02
lluhi guys, please look at the patch https://review.openstack.org/45176, our Jenkin's got stuck by the latest nova changes.08:03
jd__llu: can you fix the comments we made08:06
llujd__: just saw it. doing it right now.08:07
*** Alexei_987 has joined #openstack-metering08:07
jd__I'll approve https://review.openstack.org/#/c/44465/ then08:07
silehtjd__, I have updated this for the mongod issue to use the workaround only for test https://review.openstack.org/#/c/44465/08:12
jd__I saw08:12
jd__I'm waiting for llu's patch to go in first as I don't think yours will work on its own wrt our nova notifier failure08:13
silehtjd__, sure08:13
openstackgerritLianhao Lu proposed a change to openstack/ceilometer: Fixed nova notifier unit test  https://review.openstack.org/4517608:18
llui've updated the nova notifier one08:19
Alexei_987sileht: ping08:21
silehtAlexei_987, o/08:23
Alexei_987sileht: regarding this pooling feature - do you plan to keep this workaround? or it's just a temporary solution to get tests working properly?08:23
silehtAlexei_987, I'm looking for more details about how we can fix our tests to really resolve the mongod maxconn issue08:23
Alexei_987sileht: ah ok I'll try to prepare patch for it today08:24
Alexei_987__jd sileht and I also have a stupid question: what is db2 backend and why it uses mongo client?08:27
silehtAlexei_987, db2 is a mongodb like from ibm, but we some differences08:28
silehtAlexei_987, I just don't understand why when in test the teardown is called and then the conn.close() of pymongo, the socket is not closed.08:29
silehtAlexei_987, perhaps I need to wait your patch to understand :p I'm just curious08:30
Alexei_987sileht: the problem is that there are several references on MongoClient object held somewhere08:33
Alexei_987and object is not destroyed08:33
Alexei_987and the client object keeps sockets open for later use08:33
Alexei_987so we need either ensure that MongoClient object is destroyed08:34
Alexei_987or share this object between tests08:34
Alexei_987sileht: I plan to go with option 2 since 1 requires huge amount of debuging08:34
Alexei_987sileht: right now we have our own pooling of mongoclient08:35
Alexei_987sileht: but it's based on full connection URL08:35
Alexei_987sileht: which is ok for production deployments08:35
Alexei_987sileht: but for tests we generate a random database for each test - e.g. URL is different08:35
Alexei_987sileht: so we need to improve our pooling to use host:port pair without the database part08:36
Alexei_987sileht: in such case it will work correctly for tests08:36
openstackgerritA change was merged to openstack/ceilometer: Fixed nova notifier unit test  https://review.openstack.org/4517608:50
openstackgerritSvetlana Shturm proposed a change to openstack/ceilometer: Fix wrong migrations  https://review.openstack.org/4519608:53
jd__sileht: https://review.openstack.org/#/c/44465/ needs rebasing08:53
openstackgerritMehdi Abaakouk proposed a change to openstack/ceilometer: Disable the pymongo pooling feature for tests  https://review.openstack.org/4446508:54
openstackgerritSvetlana Shturm proposed a change to openstack/ceilometer: Fix wrong migrations  https://review.openstack.org/4453908:55
openstackgerritSvetlana Shturm proposed a change to openstack/ceilometer: Fix wrong index in 10 migration for mysql  https://review.openstack.org/4453908:57
silehtAlexei_987, I have already write some code to list the test that seems keeping ref to the object08:57
openstackgerritA change was merged to openstack/ceilometer: Disable the pymongo pooling feature for tests  https://review.openstack.org/4446509:04
*** sandywalsh has quit IRC09:10
openstackgerritLianhao Lu proposed a change to openstack/ceilometer: Added hardware agent's inspector and snmp implementation  https://review.openstack.org/4307309:16
openstackgerritLianhao Lu proposed a change to openstack/ceilometer: Added pollsters for the hardware agent  https://review.openstack.org/4307409:16
openstackgerritSvetlana Shturm proposed a change to openstack/ceilometer: Add testing of migrations from oslo  https://review.openstack.org/4387209:19
openstackgerritSvetlana Shturm proposed a change to openstack/ceilometer: Implement testing of migrations  https://review.openstack.org/4480909:20
openstackgerritA change was merged to openstack/ceilometer: Update to tox 1.6 and setup.py develop  https://review.openstack.org/4512409:21
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Improved MongoClient pooling to avoid out of connections error  https://review.openstack.org/4520209:22
openstackgerritYuuichi Fujioka proposed a change to openstack/ceilometer: doc: Fix service name for devstack  https://review.openstack.org/4520409:33
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Improved MongoClient pooling to avoid out of connections error  https://review.openstack.org/4520209:37
openstackgerritShuangtai Tian proposed a change to openstack/ceilometer: Handle the metrics sent by nova notifier  https://review.openstack.org/4283809:40
jd__we are feature frozen09:42
*** asalkeld has quit IRC10:00
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: Clean-ups related to alarm history patches  https://review.openstack.org/4513510:02
*** asalkeld has joined #openstack-metering10:02
*** mmcardle has joined #openstack-metering10:18
openstackgerritSwann Croiset proposed a change to openstack/ceilometer: Update openstack.common.policy from oslo-incubator  https://review.openstack.org/4521010:26
*** Ruetobas has quit IRC10:44
*** Ruetobas has joined #openstack-metering10:44
openstackgerritSwann Croiset proposed a change to openstack/ceilometer: Update openstack.common.policy from oslo-incubator  https://review.openstack.org/4521010:45
openstackgerritJulien Danjou proposed a change to openstack/python-ceilometerclient: alarm: rename counter_name to meter_name  https://review.openstack.org/4507611:00
jd__wtf11:01
jd__https://review.openstack.org/45210 failure scares me11:02
lsmolajd__, hello, can I have two quick questions?11:06
*** nosnos has quit IRC11:24
openstackgerritFengqian.gao proposed a change to openstack/ceilometer: Change test case get_alarm_history_on_create  https://review.openstack.org/4521911:25
openstackgerritFengqian.gao proposed a change to openstack/ceilometer: Add pagination support for sqlalchemy database  https://review.openstack.org/3545411:25
openstackgerritFengqian.gao proposed a change to openstack/ceilometer: Change pagination query method in mongodb  https://review.openstack.org/4186911:25
jd__lsmola: sure11:25
eglynndumb question ...11:29
eglynnso if I've got a scenario test failing against sqlalchemy, how can I access the sqlite log to debug the issue?11:29
eglynn(or even suppress deletion of the temporary sqlite DB, so that I can look at it after the fact ...)11:30
eglynnbbiab11:31
*** eglynn is now known as hungry-eglynn11:31
swannjd__: I didn't understand errors, you're ok that's an other reason than the change introduce in policy.py?11:33
jd__hungry-eglynn: no se11:33
*** dhellmann_ is now known as dhellmann11:33
lsmolajd__, first one about BP workflow in client, when I find something missing in client, can i create BP for this?11:33
jd__swann: I'm afraid so11:33
dhellmanngood morning, team11:33
jd__hi dhellmann11:33
lsmolajd__, like this https://blueprints.launchpad.net/python-ceilometerclient/+spec/alarm-audit-client-api11:34
jd__lsmola: sure11:34
lsmolajd__, ok cool, I need to track dependencies, so bp is the best11:34
lsmolajd__, second one about Alerting11:34
dhellmannjd__: Where do we stand? I'm catching up on the backlog, looks like there were some nova-related issues?11:35
lsmolajd__, I walked through BP's and seems there is only Alarms for alerting, meaning when I want to track soemthing, I need to set an Alarm11:35
jd__dhellmann: yes, this has been fixed, the mongodb connection pooling too, and milestone-proposed as been cut so havana's done11:35
jd__holy crap11:36
dhellmannwell, then11:36
jd__I can reproduce the failure from https://review.openstack.org/4521011:36
jd__so Ceilometer is broken *again*11:36
jd__good news, removing the nova notifier is now on top of my TODO list11:36
dhellmannyeah11:37
* jd__ frowns11:37
dhellmannshall we turn off the tests?11:37
lsmolajd__, what about notifications like disk failures, network failures, failures in general... Is there some API for getting alerts like this? Or it should all be done by Alarms?11:37
dhellmanndelete the whole thing?11:37
jd__lsmola: if failure can have a threshold, probably11:38
jd__lsmola: if you consider failure to be 0 when the normal value is 1, that can be done11:38
*** dina_belova has joined #openstack-metering11:38
lsmolajd__, yeah some of them could probably be simulated through alarms11:38
jd__lsmola: we just don't have code metering this meters11:38
jd__s/this/these/11:38
jd__dhellmann: I think we should fix the test first :)11:39
dhellmannfix it?11:39
dhellmannif we're going to remove it?11:39
dhellmannthe notifier, I mean11:39
lsmolajd__, so there are notifications about the failures, you just don't save them for now, right?11:39
dhellmannlsmola: there are no specific notifications for disk or network failures11:40
dhellmannlsmola: however, if you can derive a failure state from the data that is being collected, you could specify an alarm based on those rules11:41
jd__dhellmann: err I don't think we can remove for Havana and H3 is cut now, so…11:41
dhellmannif h3 is *broken* then it's a bug, and we can fix it11:42
jd__yes, I just warned ttx about it, I opened a bug https://bugs.launchpad.net/ceilometer/+bug/122117311:42
dhellmannftr, this is why I wanted that notifier to talk to our agent via rpc11:42
jd__dhellmann: but removing the nova notifier means replacing it actually :)11:42
jd__and I don't see that happens in a snap11:43
dhellmannno11:43
jd__no as I'm right or I'm wrong?11:43
dhellmannyou're right, it won't be replaced quickly11:43
jd__ok :)11:44
dhellmannwe could build a new notifier that uses rpc to talk to us without depending on nova internals, or we can make nova collect the stats we want11:44
jd__(that makes me half happy 'cause I'm right and half sad because I wish I was wrong)11:44
dhellmannyeah :-/11:44
lsmoladhellmann, ok, cool, what about statistics like disk health? I am just askin because we want to show something like this in tuskar-ui11:45
lsmoladhellmann, though is will use hardware agent11:45
dhellmannlsmola: we have a full list of the data we collect at http://docs.openstack.org/developer/ceilometer/measurements.html11:46
dhellmannlsmola: could we do this via email? we're trying to clean up a release candidate right this second11:46
lsmoladhellmann, https://blueprints.launchpad.net/ceilometer/+spec/monitoring-physical-devices11:46
lsmoladhellmann, sorry :-)11:46
dhellmannlsmola: that's for bare metal monitorin?11:47
dhellmannmonitoring11:47
lsmoladhellmann, yes11:47
lsmoladhellmann, I suppose it doesn't make sense to check hardware,network... failures for VM's11:47
dhellmannlsmola: someone is going to have to write new data collection plugins for ceilometer, or something else, to send data to us -- we don't have most of what you'll need about the hypervisors, we just collect data about the vms for now11:48
dhellmannIOW, if it's not already on the list of measurements we've published, we don't have it yet11:48
*** shang has quit IRC11:48
lsmoladhellmann, ok, thank you very much11:51
*** shang has joined #openstack-metering12:00
*** boris-42 has joined #openstack-metering12:04
*** mmcardle has quit IRC12:09
*** hungry-eglynn is now known as eglynn12:11
eglynnjd__: "no se" == "not possible"?12:11
jd__eglynn: "i don't know" in spanish12:15
eglynnjd__: a-ha, cool12:15
*** dina_belova has quit IRC12:16
silehteglynn, thx for you review12:17
eglynnnp, did it make sense?12:17
silehteglynn, yes I think, I will just take a look into wsme to ensure this is possible without hackism things (I want ensure type check/validation are done correctly)12:20
eglynnsileht: cool!12:20
silehteglynn, sandy have pointed to me a insteresting code into nova scheduler: https://github.com/openstack/nova/blob/master/nova/scheduler/filters/json_filter.py12:22
silehteglynn, I think about using a json string into a rules attibutes instead of having combination_alarms and combination_operator12:22
eglynnhmmm, that is an interesting idea12:23
eglynncertainly more flexible12:23
*** shang has quit IRC12:25
silehtI will finish my patch about correct default for normal alarm first, and (If dhellmann is ok) I will create the the new attribute to identify a alarm.12:26
dhellmannsileht: you don't mean posting json encoded in a string in a json payload, do you?12:26
silehtdhellmann, from the API point of view this just a flexible json structure: ie: alarm have a attributes 'rules' with a real json not a string :)12:28
dhellmannsileht: I guess I need to see an example to understand what you mean12:29
silehtdhellmann, sure, example is comming12:31
*** zul has quit IRC12:32
*** mmcardle has joined #openstack-metering12:37
*** zul has joined #openstack-metering12:37
silehtdhellmann, http://paste.openstack.org/show/45801/ I have take the syntax from the nova json_filter12:39
silehteglynn, ^12:39
dhellmannsileht: what would the WSME structure for that look like?12:40
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: nova_notifier: fix tests wrt instance_update_and_get_original proto change  https://review.openstack.org/4522212:40
silehtdhellmann, good question, If I correctly remember wsme needs to have a static schema of the json ?12:41
dhellmannsileht: yes12:41
*** dina_belova has joined #openstack-metering12:41
dhellmannthe idea is to describe the API in a way that is easy to document12:41
dhellmannsileht: I was hoping the "rules" would also apply to the threshold stuff, so we were consistent12:42
dhellmannit seems like there's a rule type, an operator, and then some arguments12:42
silehtdhellmann, in my example, I have missed to removed statistic and threshold12:43
dhellmannthreshold, =, [200]12:43
dhellmannalarm_id, in, [(uuid)]12:43
dhellmannalarm_id, in, [(uuid), (uuid)]12:43
dhellmannso I think you need an enum for the rule type, an enum for the operator, and a list of string arguments that are given to the rule12:44
dhellmannyour flexibility comes in the arguments12:44
dhellmannthe rules can't really be nested, that way, though12:44
silehtthreshold is particular because it needs a meter_name, a period and a evaluation_period to be computed12:44
dhellmannyeah12:44
dhellmannwell, does evaluation_period not apply in the other cases?12:44
silehtno,12:45
dhellmannI wonder if we should just create a little DSL and not use json12:45
dhellmannthat doesn't really solve the problem so much as work around WSME, though12:45
silehta other problem is the complexity of the evaluator12:47
dhellmannthey rules wouldn't be any more complicated12:48
dhellmann*the12:48
*** sandywalsh has joined #openstack-metering12:51
silehtdhellmann, we have two use case: do something if the statistic of a meter reach a threshold, and do something if state of many alarms change12:53
silehtdhellmann, separating both allow to compute it easly without make a complex evaluator12:54
openstackgerritJulien Danjou proposed a change to openstack/python-ceilometerclient: alarm: rename counter_name to meter_name  https://review.openstack.org/4507612:55
dhellmannif you push all of the settings down into rules, then each rule type can have its own evaluation class12:55
dhellmannjd__: I think I have a fix for the nova notifier issue12:55
jd__dhellmann: ^^^12:55
silehtdhellmann, yes but you need to reevalute every thing each time12:56
jd__dhellmann: https://review.openstack.org/4522212:56
dhellmannah12:56
dhellmannjd__: fails pep812:56
jd__pfff12:56
jd__damn me12:56
dhellmannsileht: you're going to have to do that anyway, right?12:56
eglynnyep the evaluator has a separate strategy per rule type12:57
eglynnas suggested in comment on https://review.openstack.org/#/c/41971/13/ceilometer/api/controllers/v2.py12:57
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: nova_notifier: fix tests  https://review.openstack.org/4522212:57
silehtdhellmann, actually the time to get the next state of a alarm is constant12:57
eglynnthat should simplify the evaluation logic12:57
eglynntho' maybe we should re-name threshold_eval to alarm_eval12:58
dhellmannsileht: what is being "reevaluated" every time? I don't understand the objection12:58
eglynn(as combination alarms do not have a threshold associated)12:58
dhellmanneglynn: an alarm type of "alarm_eval"? threshold seems more descriptive12:58
eglynndhellmann: no I meant the code that implements the evaluation cycle is currently in a module called threshold_evaluation12:59
dhellmanneglynn: got it, yeah, renaming that might make sense12:59
eglynndhellmann: (whereas for the combination rule type, there is no threshold to evaluate against)12:59
dhellmannright12:59
eglynnwhat is being re-evaluated each time is the combination of current alarms states, IIUC13:00
dhellmannthat's necessary, right?13:01
eglynnabsolutely13:01
dhellmannok, that's what I expected13:01
eglynnnot sure if sileht was objecting to it, more just pointing it out?13:01
dhellmannwhy doesn't a combination alarm have an evaluation period?13:01
dhellmannok13:01
silehtI'm ok13:01
eglynndhellmann: evaluation_periods * period = window over which statistics are compared against threshold for normal alarms13:02
dhellmannhow does the system know how often to evaluate a combination alarm?13:02
eglynndhellmann: whereas for combination alarms, stats are not queried13:02
dhellmannah, got it, that applies to the stats13:02
eglynndhellmann: it uses the configured eval interval13:02
dhellmannand "period" is how often to check the given alarm13:03
eglynnno period is the timeslot over which the stats are aggregated13:03
dhellmannok, clearly these docstrings need to be expanded :-)13:03
eglynnso say period = 300s, eval periods = 313:03
eglynnand eval interval = 60s13:03
*** gordc has joined #openstack-metering13:04
dhellmannI don't see "interval" in the data structure13:04
dhellmannis that global?13:04
eglynnthen every 60s, we look back 15mins into the past to get 3 data points each aggregated over 5 mins13:04
dhellmannacross all alarms?13:04
eglynnyep, global config13:04
dhellmannah, ok, that's what threw me13:04
eglynnwe use cloud-watch like terminology13:05
eglynnfor the evaluation_periods and period13:05
dhellmannI'm trying to sort out which fields are needed for which types of alarms so I can help with the WSME structure13:05
eglynnwhich isn't the clearest I admit13:05
dhellmannyeah, it's just some of the docstrings say "foo is the foo for the alarm"13:06
dhellmanni.e., threshold :-)13:06
eglynnI took a first stab at suggesting which fields need to go where in that gerrit comment13:06
eglynnhttps://review.openstack.org/#/c/41971/13/ceilometer/api/controllers/v2.py13:06
dhellmannthe problem is, we can't have 2 types of rule structures13:06
eglynn(may be incomplete tho' ...)13:06
silehtdhellmann, L945 they are 3 list13:06
*** litong has joined #openstack-metering13:06
dhellmannWSME doesn't do subclasses13:06
dhellmannat least not at the API layer13:07
eglynna-ha, ok13:07
dhellmannso you need an AlarmRule that is flexible enough13:07
silehtdhellmann, Does wsme ComplexType is usable for this ?13:07
eglynnI see13:07
dhellmannI want to add a union type to the complex type, but that won't happen in time for us to use it13:07
silehtdhellmann, oh ok I have understand13:07
dhellmannso if we go back to what I was saying a while ago, with rule_type, operator, and a list of args, can we express the rules that way13:09
dhellmannthreshold, =, [stats_name, value]13:09
dhellmanncombination, any, [alarm1, alarm2]13:10
dhellmanncombination, all, [alarm1, alarm2]13:10
dhellmannthe problem is expressing the search metadata for that threshold alarm, it's so much more complex than the combination alarm13:10
dhellmannbtw, the matching_metadata needs to support field types so we can do the search properly13:11
dhellmannhmm13:11
*** lexx has quit IRC13:11
dhellmannit needs to be a Query object I think13:11
eglynnthe rule needs to be Query object?13:12
eglynnis that rich enough to capture a threshold-oriented rule?13:12
dhellmannI don't mean the rule, I mean the field "matching_metadata"13:12
eglynni.e. not just a field, op, value but also period etc.?13:12
eglynndhellmann: a-ha, gotcha13:13
dhellmanna dict of {str:str} is not sufficiently rich to describe a query13:13
dhellmannin fact, you'll never match any numerical values13:13
eglynnyep, that's fair13:13
*** graflu0 has joined #openstack-metering13:14
dhellmannnoted on the review for posterity13:14
jd__https://review.openstack.org/#/c/45222/ is ready to fix the gate btw13:15
dhellmannjd__: approved to fast-track13:15
jd__thanks dhellmann13:16
*** tasdomas is now known as tasdomas_afk13:16
*** tasdomas_afk is now known as tasdomas13:17
dhellmanneglynn, sileht : changing matching_metadata to a [Query] means you can drop the meter_name field (that's consistent with the statistics API)13:20
dhellmanneglynn, sileht : you may also want to think about whether the object the user posts to the API is the same as the one they get back then they ask for details13:20
dhellmannmaybe separate Alarm and AlarmStatus structures make sense, since there are a lot of fields that the user either can't control or don't make sense to set when posting a new alarm13:21
silehtdhellmann, the user is allowed to set the alarm status for debugging13:23
openstackgerritgordon chung proposed a change to openstack/ceilometer: add tests for _query_to_kwargs func  https://review.openstack.org/4379613:23
dhellmannsileht: so they can trigger the alarm by changing its state?13:23
dhellmannmaybe that should be a separate endpoing in the API?13:24
silehtdhellmann, no trigger only set the current state13:24
dhellmannchanging the state of the alarm does not trigger it?13:24
eglynndhellmann: that's correct13:25
dhellmannok13:25
eglynndhellmann: the actions are only triggered for a true state transition13:25
dhellmannlet me spend some time today putting together a proposal13:26
*** anteaya has joined #openstack-metering13:26
dhellmannI'll write something up in the wiki and send an email13:26
dhellmannbecause I think what we have now is just exposing the schema for alarms, and is not really providing an API for working with alarms in discrete ways13:26
dhellmannand maybe I can help with the rule expression thing, too13:26
jd__anyone up for reviewing https://review.openstack.org/#/c/45196/ ?13:27
dhellmannjd__: I looked at that, but don't understand what's going on13:27
dhellmannoh, wait, that's a new one13:27
eglynngordc: I'd proposed those query tests in https://review.openstack.org/#/c/45135 (as discussed on IRC yesterday)13:27
jd__that seems like a better version of what I tried to do13:27
dhellmannI thought that was yours13:28
dhellmannyeah13:28
eglynngordc: d'oh! I hadn't realized you were on it also13:28
gordceglynn: oh whoops, i didn't see discussion on irc.13:28
gordceglynn: did you want to pick up the changes i did to the func as well? i'll just drop it then.13:29
* eglynn looking ...13:29
gordceglynn: relating to this bug https://bugs.launchpad.net/ceilometer/+bug/121712913:30
openstackgerritA change was merged to openstack/ceilometer: nova_notifier: fix tests  https://review.openstack.org/4522213:31
dhellmannjd__: crossing my fingers and giving a +213:31
eglynngordc: k, that's being a bugfix probably best to keep separate from a clean-up patch13:32
*** thomasm has joined #openstack-metering13:33
thomasmHey all13:34
gordceglynn: i'll put mine as WIP since yours is related to alarm stuff.13:35
eglynncool13:35
gordchey thomasm13:37
thomasmhow's the day treating you?13:37
gordcthomasm: just woke up.lol13:37
thomasmgordc, lol13:38
gordcthomasm: i think you can apply some of the comments i made to your patch -- just skip over the ones you weren't sure of.13:39
thomasmgordc, Sure thing. I'll do that this morning13:39
gordcthomasm: cool cool13:40
openstackgerritA change was merged to openstack/ceilometer: Fix wrong migrations  https://review.openstack.org/4519613:41
litong@thomasm, I added some comments as well, if you need or have any question, please let me know.13:42
thomasmlitong, Will do. Thank you. Just getting my code downloaded to work locally. :)13:43
*** lexx has joined #openstack-metering13:44
*** graflu0 has quit IRC13:45
thomasmgordc, regarding those comments in the impl_hbase changes, I guess the reason I went for more readability was because I prefer a more declarative approach. If I can abstract away the imperative, I can just read the code like a book.13:47
thomasmgordc, Of course, each line of code is a culprit, so I dunno - that's just something I do sometimes. :P13:47
openstackgerritJulien Danjou proposed a change to openstack/python-ceilometerclient: alarm: rename counter_name to meter_name  https://review.openstack.org/4507613:48
*** Fengqian has joined #openstack-metering13:49
gordcthomasm: i'm ok with the breaking them into functions, i'm of the opposite philosophy but its the same logic at the end of the day so no problem for me.13:49
thomasmgordc, No worries, I'm actually just curious. I like to hear the rationale from developers more experienced than I.13:50
zuljd__:  ping13:51
jd__zul: pong13:51
zuljd__:  mongodb should be the hitting the cloud archive this week13:51
jd__zul: aaaaaaaaaaaahhhhhhhhhhhhhhh13:51
silehtzul, \o/13:51
silehtzul, thanks !13:51
* jd__ runs like crazy13:51
thomasmSomebody catch 'em!13:52
gordcthomasm: you probably won't want to quote me if it's base on experience then.lol maybe more ceilometer experience?13:52
thomasmgordc, Lol, fair. Both are valuable. I've been doing this type of work for 5 months. Before that I was a front end engineer for 2 years, never touched databases or mid-tier code.13:54
thomasmSo, I learn quickly, but I always love to get a bunch of different perspectives to figure out what I really think about it.13:54
thomasmanywhooo, yeah. I guess experience has really shown that oftentimes the abstraction is where the root cause is.13:55
thomasmSo, you have to know how it's doing what it's doing.13:55
sandywalsheglynn, do we have any detailed docs on the alarm stuff (other than the blueprints)? I'm going to spend the day digging into that whole thing.14:01
eglynnsandywalsh: just the blueprints and the code so far14:02
eglynnsandywalsh: "digging in" ... in what sense?14:02
dhellmanneglynn, sileht : https://wiki.openstack.org/wiki/Ceilometer/blueprints/alarm-api14:02
dhellmannsandywalsh: ^^ that's my proposed set of changes to the Alarm structures in the API14:03
* sileht is looking14:04
*** graflu0 has joined #openstack-metering14:04
dhellmannit doesn't include a generic rule, but I think it at least clears up which settings are needed when14:04
gordcthomasm: agreed, always different valid ways to approach stuff.14:05
sandywalsheglynn, cool ... I'll give them a re-read. I want to understand the requirements better so I can understand how the code maps to it. I can't really make good suggestions for refactoring without really knowing the requirements.14:05
sandywalshdhellmann, thanks14:05
gordcthomasm: some of the comments i put are probably wrong sometimes so always good to question it if you think it makes no sense.14:05
*** krtaylor has joined #openstack-metering14:06
thomasmgordc, Sure, same here. =]14:07
silehtdhellmann, does I understand that on PUT operation, the user needs to provide the full alarm ?14:07
dhellmannsileht: WSME is going to give you an instance of the class specified as the argument type for the operation, with whatever fields the user provides14:08
dhellmannit doesn't make sense to ask them to PUT part of an Alarm14:08
eglynndhellmann: how is the mutual exclusivity between threshold_rule and combination_rule captured?14:08
dhellmanneglynn: there's a type setting on the Alarm14:09
dhellmannduring icehouse maybe we can add a union type to wsme, and then we can nest the type and rule fields in that14:09
eglynndhellmann: sure I saw that, but the representation itself could have both set (I thought the idea was to do something union-like)14:09
eglynnk14:10
dhellmannyeah, the threshold thing is just too much more complex14:10
dhellmannlet me write down what I was trying to express in irc before...14:10
dhellmanneglynn: https://wiki.openstack.org/wiki/Ceilometer/blueprints/alarm-api#Alternate_Rule_Spec14:13
eglynnhmmm the alternate more generic rule spec isn't very user friendly14:15
dhellmanneglynn: right14:15
dhellmannthe two rule types are just *so* different, there's not much in common between them14:16
dhellmannbut at least by splitting the two rule types out as in the first alternate, the user can see which fields are needed in which case14:16
eglynnyep14:16
eglynnyes I think that make the mutual exclusivity much clearer14:16
eglynn*makes14:17
*** terriyu has joined #openstack-metering14:17
silehtAnd this allow to easly extends to new type of rule without breaking API14:18
eglynntrue also14:18
eglynnI like14:18
eglynnI'm nore sure we need to change the action representation though14:18
eglynn(as part of the combination work)14:19
dhellmannsee also the new DSL section at the bottom, although that's not very well thought out yet14:19
dhellmannI really do think we should go with the classes, but wanted to capture that idea, too14:19
eglynncool, worth considering, but agreed14:19
dhellmanneglynn: are we likely to add more action types at any point? more states?14:19
silehtdhellmann, sure14:19
eglynndhellmann: I can't see a real need TBH14:20
dhellmannok, I wasn't sure if this was a MVP with plans to extend14:20
dhellmanneglynn: I can go along with separate lists of URLs, then14:22
eglynndhellmann: cool14:22
dhellmanneglynn: I'll update the page to reflect that14:23
eglynnthanks!14:23
dhellmanneglynn: done14:25
silehtdhellmann, eglynn thanks, I'm sure the code will be more clear and more concise now14:27
silehtdhellmann, by force the user to PUT the full alarm, all the default value can be in the wsme model (I have see somehing like wsproperty(xxxx, default='') in the documentation)14:28
silehtdhellmann, eglynn, so can I start to work on this reworker API ?14:30
dhellmannsileht: new alarms should be created with POST, because the user does not control the alarm id. Updates via PUT would require the entire alarm to be provided, in order to be idempotent. If you want to let them change individual pieces of the alarm, then some other mechanism is needed14:30
dhellmannsileht: that other mechanism could be PUT to just the field in question (/alarms/<uuid>/state)14:31
silehtdhellmann, I clearly prefer use wsme tools, and force it to PUT the full alarm14:31
dhellmannsure, that makes sense for updating the definition14:31
eglynnsileht: do you mean, should you start on reworking the alarm API as discussed above?14:32
silehtdhellmann, cool a new endpoint :p, this can be done later14:32
silehteglynn, yes14:32
eglynnsileht: absolutely! :)14:32
dhellmannsileht: right, if we find we need to make some operations more convenient14:32
eglynndhellmann: offtopic ... I tried to address your non-blocking suggestions for the alarm history patches here: https://review.openstack.org/4513514:32
dhellmanneglynn: looking14:34
*** shang has joined #openstack-metering14:40
openstackgerritJulien Danjou proposed a change to openstack/python-ceilometerclient: alarm: rename counter_name to meter_name  https://review.openstack.org/4507614:42
*** dina_belova has quit IRC14:43
*** dina_belova has joined #openstack-metering14:45
*** dina_bel_ has joined #openstack-metering14:47
*** dina_belova has quit IRC14:47
*** graflu0 has quit IRC14:52
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: Alarm history storage implementation for sqlalchemy  https://review.openstack.org/4524414:57
*** Fengqian has quit IRC15:06
*** lexx has quit IRC15:10
*** dina_bel_ has quit IRC15:10
*** SergeyLu_ has joined #openstack-metering15:23
*** changbl has quit IRC15:23
*** SergeyLu_ has quit IRC15:23
*** SergeyLu_ has joined #openstack-metering15:24
*** shang has quit IRC15:29
*** SergeyLu_ has quit IRC15:29
*** SergeyLukjanov has joined #openstack-metering15:30
*** tasdomas is now known as tasdomas_afk15:35
eglynnmeh, we're seeing that IndexError from wsme/type.py again15:47
eglynni.e. ... if bases[0] is not object and '__registry__' not in dct:15:47
eglynne.g. in https://jenkins02.openstack.org/job/gate-ceilometer-python26/471/consoleFull15:48
eglynnI thought that had been resolved by constraining the six dependency version not to clash with WSME?15:48
jd__O_o15:50
jd__eglynn: this patch changes six bounding15:50
eglynngrrr!15:51
*** boris-42 has quit IRC15:51
jd__be careful of devstack, it seems it changes these file magically now15:51
eglynn(yep, I took it from devstack)15:51
jd__dumb idea IMHO15:51
eglynnsorry for the noise, fixing now ...15:51
jd__:)15:51
eglynnyeah, agreed, dumb and v. annoying ...15:51
*** shaneduan[afk] is now known as shaneduan15:53
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: Alarm history storage implementation for sqlalchemy  https://review.openstack.org/4524415:55
*** Ruetobas has quit IRC16:01
*** shang has joined #openstack-metering16:01
*** Ruetobas has joined #openstack-metering16:04
*** shang has quit IRC16:07
*** shang has joined #openstack-metering16:08
*** Ruetobas has quit IRC16:08
*** SergeyLukjanov has quit IRC16:13
openstackgerritA change was merged to openstack/ceilometer: Clean-ups related to alarm history patches  https://review.openstack.org/4513516:13
openstackgerritgordon chung proposed a change to openstack/ceilometer: add MAINTAINERS file  https://review.openstack.org/4526416:13
*** Ruetobas has joined #openstack-metering16:15
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: Fix wrong migrations  https://review.openstack.org/4526516:15
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: nova_notifier: fix tests  https://review.openstack.org/4526616:15
openstackgerritgordon chung proposed a change to openstack/ceilometer: add MAINTAINERS file  https://review.openstack.org/4526416:19
openstackgerritThomas Maddox proposed a change to openstack/ceilometer: Fix to return latest resource metadata  https://review.openstack.org/4427716:22
*** shaneduan is now known as shaneduan[afk]16:30
*** shang has quit IRC16:34
*** shaneduan[afk] is now known as shaneduan16:40
*** mmcardle has quit IRC16:42
openstackgerritAlex Gaynor proposed a change to openstack/python-ceilometerclient: Added support for running the tests under PyPy with tox  https://review.openstack.org/4527616:57
*** boris-42 has joined #openstack-metering16:59
*** Alexei_987 has quit IRC17:04
*** SergeyLukjanov has joined #openstack-metering17:10
*** yjiang5_away is now known as yjiang517:11
*** dhellmann is now known as dhellmann_17:11
*** nealph has joined #openstack-metering17:15
*** changbl has joined #openstack-metering17:16
*** shang has joined #openstack-metering17:20
*** dina_belova has joined #openstack-metering17:21
*** dina_belova has quit IRC17:25
*** dina_belova has joined #openstack-metering17:26
*** mmcardle has joined #openstack-metering17:42
openstackgerritThomas Maddox proposed a change to openstack/ceilometer: Fix to return latest resource metadata  https://review.openstack.org/4427717:48
*** mmcardle has quit IRC17:50
thomasmHmmm? I didn't see any tests actually run in the two gate-tempest tests that failed on my patch? are there known issues with these tests?17:52
litong@thomasm, I was surprised as well seeing that results. the last two tests are new to me.18:02
thomasmlitong, It looks like keystone didn't start for devstack and failed the build. :\18:03
thomasmFor both of them18:04
litongyeah, that was from the log file, but I do not remember seeing the last two tests for other patchsets though.18:04
thomasmYeah, I think they were just added today18:04
litonghope that the tests will run successfully this time.18:05
thomasmMe too! *crosses fingers*18:05
litongI will be back online in about one hour.18:06
thomasmOkay18:06
*** litong has quit IRC18:06
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: Alarm history storage implementation for sqlalchemy  https://review.openstack.org/4524418:11
thomasmPassed this time. Yay!18:16
*** mmcardle has joined #openstack-metering18:17
*** mmcardle has quit IRC18:21
*** sdake has quit IRC18:22
*** eglynn has quit IRC18:22
*** sdake has joined #openstack-metering18:22
*** sdake has joined #openstack-metering18:22
openstackgerritgordon chung proposed a change to stackforge/pycadf: ensure unique id and timestamp defaults  https://review.openstack.org/4530118:40
*** dina_belova has quit IRC18:40
*** litong has joined #openstack-metering18:40
thomasmlitong, They passed this time18:43
litong@thomasm, good job.18:43
litongglad it did.18:43
thomasmlikewise18:44
openstackgerritgordon chung proposed a change to stackforge/pycadf: ensure unique id and timestamp defaults  https://review.openstack.org/4530118:44
openstackgerritA change was merged to stackforge/pycadf: ensure unique id and timestamp defaults  https://review.openstack.org/4530118:53
openstackgerritA change was merged to openstack/python-ceilometerclient: Added support for running the tests under PyPy with tox  https://review.openstack.org/4527618:53
*** SergeyLukjanov has quit IRC18:54
*** eglynn has joined #openstack-metering18:58
*** sdake_ has quit IRC18:59
*** sdake_ has joined #openstack-metering19:01
*** sdake_ has quit IRC19:01
*** sdake_ has joined #openstack-metering19:01
*** shang has quit IRC19:04
openstackgerritDan Prince proposed a change to openstack/ceilometer: Force MySQL to use InnoDB.  https://review.openstack.org/4530619:04
*** sdake_ has quit IRC19:10
*** krtaylor has quit IRC19:12
*** sdake_ has joined #openstack-metering19:13
*** sdake_ has quit IRC19:13
*** sdake_ has joined #openstack-metering19:13
*** mmcardle has joined #openstack-metering19:17
*** krtaylor has joined #openstack-metering19:17
*** mmcardle has quit IRC19:22
*** eglynn has quit IRC19:23
*** sdake_ has quit IRC19:33
*** dina_belova has joined #openstack-metering19:41
*** dina_belova has quit IRC19:45
*** dina_belova has joined #openstack-metering19:51
*** dina_belova has quit IRC19:55
thomasmgordc: I was thinking we could start applying a builder pattern to that whole driver, since there's so much boilerplate query filtering and what-not with variations that could be built off of the basic filtered object?19:57
thomasm(mongodb)19:57
openstackgerritThomas Maddox proposed a change to openstack/ceilometer: Fix to return latest resource metadata  https://review.openstack.org/4427719:59
*** sandywalsh has quit IRC20:00
openstackgerritThomas Maddox proposed a change to openstack/ceilometer: Fix to return latest resource metadata  https://review.openstack.org/4427720:01
thomasmMan? I'm just being a baddie today. Forgot the commit message change too.20:03
thomasmgordc, Updated that patch. Sorry for the missed changes!20:09
*** anteaya has quit IRC20:11
*** sandywalsh has joined #openstack-metering20:14
*** mmcardle has joined #openstack-metering20:19
*** sandywalsh has quit IRC20:20
openstackgerritA change was merged to openstack/ceilometer: add MAINTAINERS file  https://review.openstack.org/4526420:22
*** dhellmann_ is now known as dhellmann20:23
*** mmcardle has quit IRC20:23
*** sdake_ has joined #openstack-metering20:25
*** sdake_ has quit IRC20:25
*** sdake_ has joined #openstack-metering20:25
*** mmcardle has joined #openstack-metering20:28
*** sandywalsh has joined #openstack-metering20:33
*** evanjfraser has joined #openstack-metering20:45
*** mmcardle has quit IRC20:45
*** dina_belova has joined #openstack-metering20:51
*** dina_belova has quit IRC20:56
*** shaneduan is now known as shaneduan[afk]21:04
*** shaneduan[afk] is now known as shaneduan21:05
*** mmcardle has joined #openstack-metering21:15
gordcthomasm: whoops, was on another machine.21:15
thomasmgordc, No worries21:16
gordcthomasm: builder pattern sounds interesting... something we can explore in Icehouse.21:16
gordcthanks for applying fixes btw21:16
thomasmgordc, Sure thing. Yeah, the way we apply sorting/pagination is a little mixed, it feels like21:17
thomasmgordc, I'd like to be able to do query.sort(sort_keys).paginate(limit=100)21:18
thomasmor something like that21:18
gordcthomasm: :) yeah there's a lot of discussion around pagination if you look at the current patchsets21:18
thomasmI noticed that. I also saw one discussion around removing it completely.21:18
*** mmcardle has quit IRC21:20
openstackgerritgordon chung proposed a change to openstack/ceilometer: add tests for _query_to_kwargs func  https://review.openstack.org/4379621:20
thomasmOf course, as soon as I mention that, I can't find the discussion I'm referring to.21:20
gordcthomasm: lol, yeah looks like we'll have time to talk about pagination since it's blocked by feature freeze21:22
thomasmgordc, gotcha21:23
openstackgerritDan Prince proposed a change to openstack/ceilometer: Force MySQL to use InnoDB!  https://review.openstack.org/4530621:33
*** shaneduan is now known as shaneduan[afk]21:46
*** shaneduan[afk] is now known as shaneduan21:47
*** thomasm has quit IRC21:47
*** changbl has quit IRC21:51
*** dina_belova has joined #openstack-metering21:52
*** dina_belova has quit IRC21:55
*** shaneduan is now known as shaneduan[afk]22:04
*** shaneduan[afk] is now known as shaneduan22:05
*** mmcardle has joined #openstack-metering22:16
*** mmcardle has quit IRC22:21
*** shaneduan is now known as shaneduan[afk]22:26
*** shaneduan[afk] is now known as shaneduan22:30
*** shaneduan is now known as shaneduan[afk]22:31
*** shaneduan[afk] is now known as shaneduan22:31
*** evanjfraser_ has joined #openstack-metering22:45
*** evanjfraser has quit IRC22:49
*** thomasm has joined #openstack-metering22:51
*** dina_belova has joined #openstack-metering22:52
*** dina_belova has quit IRC22:57
*** evanjfraser_ has quit IRC23:03
*** shaneduan is now known as shaneduan[afk]23:03
openstackgerritgordon chung proposed a change to openstack/ceilometer: validate counter_type when posting samples  https://review.openstack.org/4534423:09
*** nijaba has quit IRC23:12
*** nijaba has joined #openstack-metering23:13
*** evanjfraser has joined #openstack-metering23:15
*** boris-42 has quit IRC23:16
*** mmcardle has joined #openstack-metering23:17
*** mmcardle has quit IRC23:22
*** shaneduan[afk] is now known as shaneduan23:26
openstackgerritgordon chung proposed a change to openstack/ceilometer: validate counter_type when posting samples  https://review.openstack.org/4534423:31
*** evanjfraser has quit IRC23:32
*** gordc has quit IRC23:32
*** evanjfraser has joined #openstack-metering23:42
*** dina_belova has joined #openstack-metering23:53
*** dina_belova has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!