Tuesday, 2016-02-23

openstackgerritClaudiu Belu proposed openstack/ceilometer: Hyper-V: replaces in-tree hyper-v utils usage with os_win  https://review.openstack.org/28045400:04
*** thumpba has quit IRC00:08
*** jwcroppe has quit IRC00:09
*** cheneydc has joined #openstack-telemetry00:59
*** thumpba has joined #openstack-telemetry00:59
*** jwcroppe has joined #openstack-telemetry01:06
*** _nadya_ has joined #openstack-telemetry01:17
*** jwcroppe has quit IRC01:20
*** jwcroppe has joined #openstack-telemetry01:20
*** _nadya_ has quit IRC01:22
openstackgerritliusheng proposed openstack/aodh: Add composite rule alarm API support  https://review.openstack.org/25772201:50
openstackgerritliusheng proposed openstack/aodh: Add composite alarm usage description  https://review.openstack.org/28012101:50
*** hparekh has quit IRC01:53
*** hparekh has joined #openstack-telemetry01:56
openstackgerritliusheng proposed openstack/aodh: Add composite rule alarm API support  https://review.openstack.org/25772201:57
*** claudiub|3 has quit IRC01:58
openstackgerritliusheng proposed openstack/aodh: Add composite alarm usage description  https://review.openstack.org/28012102:05
*** ljxiash has joined #openstack-telemetry02:06
openstackgerritliusheng proposed openstack/aodh: Add composite rule alarm API support  https://review.openstack.org/25772202:11
*** mdih has joined #openstack-telemetry02:20
*** mdih has quit IRC02:21
*** mdih has joined #openstack-telemetry02:23
*** mdih has quit IRC02:25
*** mdih has joined #openstack-telemetry02:30
*** mdih has quit IRC02:30
*** mdih has joined #openstack-telemetry02:31
*** mdih has quit IRC02:33
*** mdih has joined #openstack-telemetry02:34
*** mdih has quit IRC02:34
*** mdih has joined #openstack-telemetry02:35
*** mdih has quit IRC02:35
*** mdih has joined #openstack-telemetry02:36
*** thorst has quit IRC02:38
*** thorst has joined #openstack-telemetry02:39
openstackgerritliusheng proposed openstack/aodh: Add composite alarm usage description  https://review.openstack.org/28012102:40
*** thorst has quit IRC02:47
*** _nadya_ has joined #openstack-telemetry03:18
*** _nadya_ has quit IRC03:22
*** achatterjee has joined #openstack-telemetry03:32
*** Ich has joined #openstack-telemetry03:36
*** ccz has joined #openstack-telemetry03:36
*** Ich has quit IRC03:36
*** ccz has quit IRC03:36
*** Ich has joined #openstack-telemetry03:36
*** Ich is now known as Guest8080503:37
*** ccz has joined #openstack-telemetry03:37
openstackgerritEdwin Zhai proposed openstack/aodh: Fix tempest test path  https://review.openstack.org/28337203:38
*** Guest80805 has quit IRC03:38
*** ccz has quit IRC03:38
*** ccz has joined #openstack-telemetry03:40
*** thorst has joined #openstack-telemetry03:45
*** ccz has quit IRC03:46
*** 7GHAAB00A has joined #openstack-telemetry03:50
*** 64MAADWC4 has joined #openstack-telemetry03:50
*** thorst has quit IRC03:52
*** links has joined #openstack-telemetry04:03
*** wolsen has quit IRC04:10
openstackgerritliusheng proposed openstack/ceilometer: TEST: Fix gate  https://review.openstack.org/28338204:27
openstackgerritliusheng proposed openstack/aodh: Add composite alarm usage description  https://review.openstack.org/28012104:34
*** ljxiash has quit IRC04:48
*** ljxiash has joined #openstack-telemetry04:48
*** _nadya_ has joined #openstack-telemetry04:48
*** thorst has joined #openstack-telemetry04:53
*** _nadya_ has quit IRC04:53
*** ljxiash has quit IRC04:53
*** thorst has quit IRC04:58
*** yprokule has joined #openstack-telemetry05:01
*** thumpba has quit IRC05:31
*** ljxiash has joined #openstack-telemetry05:31
*** ljxiash has quit IRC05:45
*** ljxiash has joined #openstack-telemetry05:45
*** thorst has joined #openstack-telemetry05:56
*** thorst has quit IRC06:02
*** _nadya_ has joined #openstack-telemetry06:34
openstackgerritliusheng proposed openstack/ceilometer: Change the SERVICE_TENANT_NAME to SERVICE_PROJECT_NAME  https://review.openstack.org/28338206:37
*** _nadya_ has quit IRC06:38
*** thorst has joined #openstack-telemetry07:00
*** lsmola has joined #openstack-telemetry07:04
*** thorst has quit IRC07:07
*** liamji has joined #openstack-telemetry07:08
*** mdih has quit IRC07:43
*** mdih has joined #openstack-telemetry07:43
*** mdih has quit IRC07:43
*** boris-42 has quit IRC07:54
*** pcaruana has joined #openstack-telemetry07:58
*** _nadya_ has joined #openstack-telemetry08:02
*** thorst has joined #openstack-telemetry08:06
*** pcaruana is now known as pcaruana|afk|08:08
*** pcaruana|afk| is now known as pcaruana08:08
*** thorst has quit IRC08:13
openstackgerritliusheng proposed openstack/aodh: TEST  https://review.openstack.org/28343508:15
*** fwdit has joined #openstack-telemetry08:29
liushengthe fix of jenkins job failure should be fixed in both aodh and ceilometer devstack plugin, but they need to depends on each other :(08:48
liushenghttps://launchpad.net/bugs/154863408:48
openstackLaunchpad bug 1548634 in Ceilometer ""openstack role add" command failure in devstack installation" [Undecided,In progress] - Assigned to Liusheng (liusheng)08:48
*** safchain has joined #openstack-telemetry08:54
*** ccz has joined #openstack-telemetry09:02
*** 64MAADWC4 has quit IRC09:02
*** 7GHAAB00A has quit IRC09:04
liamjiliusheng: you are in a dead lock :)09:05
liamjimaybe need to merge the code forcely in one side09:06
*** therve has joined #openstack-telemetry09:08
*** gmoro has quit IRC09:09
therveliusheng, So what do we do to unblock the gate?09:10
therveShould we revert the devstack patch?09:10
*** thorst has joined #openstack-telemetry09:11
liushengliamji, therve I am thinking if we can skip the integration test job temporarily to merged the aodh and ceilometer patches.09:13
therveliusheng, Okay. That sounds like a plan, can you push that?09:14
therveIt block heat gate09:15
liamjiliusheng: good idea, we need to break the lock09:15
liushengtherve: yes, but I didn't find a way to skip the integration test in ceilometer yet :(09:18
*** thorst has quit IRC09:18
therveYeah I hope it doesn't require a project-config change09:18
therveOtherwise we'd better ask for the devstack revert09:18
*** yassine__ has joined #openstack-telemetry09:19
liushengtherve: yeah, I am trying :)09:21
*** ljxiash has quit IRC09:23
*** ljxiash has joined #openstack-telemetry09:24
*** efoley has joined #openstack-telemetry09:24
*** ljxiash has quit IRC09:28
*** cheneydc has quit IRC10:01
openstackgerritliusheng proposed openstack/ceilometer: Change the SERVICE_TENANT_NAME to SERVICE_PROJECT_NAME  https://review.openstack.org/28338210:07
*** cdent has joined #openstack-telemetry10:14
*** thorst has joined #openstack-telemetry10:15
*** bdossant has joined #openstack-telemetry10:16
*** belmoreira has joined #openstack-telemetry10:17
*** thorst has quit IRC10:23
*** ccz has quit IRC10:36
*** cdent has quit IRC10:38
*** _nadya_ has quit IRC10:39
*** ccz has joined #openstack-telemetry10:39
openstackgerritliusheng proposed openstack/ceilometer: Change the SERVICE_TENANT_NAME to SERVICE_PROJECT_NAME  https://review.openstack.org/28338210:55
*** yassine__ has quit IRC11:02
*** thorst has joined #openstack-telemetry11:21
*** cdent has joined #openstack-telemetry11:23
*** ljxiash has joined #openstack-telemetry11:24
*** thorst has quit IRC11:28
*** ljxiash has quit IRC11:29
openstackgerritliusheng proposed openstack/ceilometer: Change the SERVICE_TENANT_NAME to SERVICE_PROJECT_NAME  https://review.openstack.org/28338211:39
*** ildikov_ has quit IRC11:40
openstackgerritliusheng proposed openstack/ceilometer: Change the SERVICE_TENANT_NAME to SERVICE_PROJECT_NAME  https://review.openstack.org/28338211:41
swamireddyHi11:44
swamireddyI getting the ceilometer-api  <defunct> very frequently, after restarting the celimeter-api, defunct goes way...but come again in next 2-3 min...11:45
swamireddyany hint to solve this issue?11:45
openstackgerritliusheng proposed openstack/aodh: Change the SERVICE_TENANT_NAME to SERVICE_PROJECT_NAME  https://review.openstack.org/28343511:46
*** _nadya_ has joined #openstack-telemetry11:49
silehtliusheng, we can't fix your gate, disabling tests is not an acceptable solution, devstack have to be fixed to allow to migrate to the new thing12:03
silehtliusheng, I have proposed https://review.openstack.org/28353112:03
liushengsileht: thanks, sounds good :) I have update the ceilometer fix, it is like change you made in devstack12:05
silehtliusheng, Cool, I have ping qa when they wake up to get the devstack patch landed quickly12:05
liushengsileht: https://review.openstack.org/#/c/283382/6/devstack/plugin.sh12:05
silehtliusheng, is that works ?12:06
*** kairo has joined #openstack-telemetry12:06
*** ildikov has joined #openstack-telemetry12:06
liushengsileht: I am watching the zuul page :)12:06
silehtliusheng, oki, that mainly depends of the order devstack plugin setup other12:07
*** efoley has quit IRC12:07
silehtrephrased, that mainly depends on the plugin loading order done by devstack12:08
liushengsileht: but I agree your patch, I heard that the magnum also met this issue, and the ceilometer and aodh changes also needed12:08
liushengsileht: yeah, I assumed it according to the order in localrc of devstack12:10
liushengsileht: the zuul page shows that the ceilometer's fix also works :)12:13
*** rvasilets has joined #openstack-telemetry12:22
rvasiletsHi to all, our Rally gates looks like broken now you could see this here https://review.openstack.org/#/c/265857/4 In the logs we fond that http://logs.openstack.org/57/265857/4/check/gate-rally-dsvm-rally/c981960/logs/devstacklog.txt.gz#_2016-02-23_10_36_26_577 https://github.com/openstack/ceilometer/blob/master/devstack/plugin.sh#L321 $SERVICE_TENANT_NAME variable is empty but I can't fount where we need to define it to fix our gates.12:22
rvasiletsCould you you help me please12:22
cdentrvasilets: sileht proposed this https://review.openstack.org/283531 which seems related12:23
*** ljxiash has joined #openstack-telemetry12:24
rvasiletscdent, yes looks like this is it12:24
rvasiletsthx12:24
*** thorst has joined #openstack-telemetry12:25
rvasiletsMerged=)12:29
*** thorst has quit IRC12:30
*** yassine__ has joined #openstack-telemetry12:31
*** yassine__ has quit IRC12:31
*** yassine__ has joined #openstack-telemetry12:31
*** gordc has joined #openstack-telemetry12:34
*** liamji has quit IRC12:36
*** efoley has joined #openstack-telemetry12:37
*** links has quit IRC12:38
*** ljxiash has quit IRC12:40
*** ljxiash has joined #openstack-telemetry12:41
*** r-mibu has quit IRC12:42
*** r-mibu has joined #openstack-telemetry12:43
*** ljxiash has quit IRC12:45
*** thorst has joined #openstack-telemetry12:46
*** kairo has quit IRC12:59
*** ccz has quit IRC13:01
*** ljxiash has joined #openstack-telemetry13:06
*** ljxiash has quit IRC13:10
*** links has joined #openstack-telemetry13:23
*** ccz has joined #openstack-telemetry13:28
*** pradk has joined #openstack-telemetry13:38
*** pradk has quit IRC13:38
*** _nadya_ has quit IRC13:39
*** wolsen has joined #openstack-telemetry13:47
*** wolsen has quit IRC13:47
*** claudiub has joined #openstack-telemetry13:49
*** pradk has joined #openstack-telemetry13:50
claudiubliusheng: hello. If you have some time, can you take a look at a patch? It had a +2, It is pretty straightforward and simple: https://review.openstack.org/#/c/280454/ Thanks!13:56
*** julim has joined #openstack-telemetry13:57
gordcclaudiub: thans for updating.... liusheng's based in China so good chance he isn't online.14:01
gordcthanks*14:01
claudiuboh, I see. thanks for the info. :)14:01
*** pcaruana has quit IRC14:02
gordcsileht: i'm going to fast approve this unless you have concerns https://review.openstack.org/#/c/283382/14:05
silehtgordc, go ahead14:05
gordcsileht: cool cool. done.14:06
gordcsileht: you need me to look at anything?14:06
silehtgordc, qa asked me to test that before approving my change14:07
silehtgordc, so I will abandon my devstack patch :)14:07
gordcsileht: lol14:07
gordcsileht: that seems easier.14:07
*** ljxiash has joined #openstack-telemetry14:10
*** _nadya_ has joined #openstack-telemetry14:12
gordcildikov: you have any information/pretty diagrams re:Gluon?14:12
*** _nadya_ has quit IRC14:18
ildikovgordc: hi14:19
ildikovgordc: https://www.youtube.com/watch?v=mbnLWB5cUeM14:20
gordcildikov: cool thanks14:25
ildikovgordc: is this what you were looking for?14:25
gordcildikov: it's a start. i just heard about it and was curious.14:25
ildikovgordc: ok, cool :)14:26
g3ekildikov : seems interresting, will follow the project.14:27
ildikovg3ek: cool, hopefully we will have more activities in OpenStack soon-ish14:28
g3ekcool, will help a lot with security14:28
*** links has quit IRC14:30
*** links has joined #openstack-telemetry14:31
*** julim_ has joined #openstack-telemetry14:34
*** julim has quit IRC14:35
*** ddieterly has joined #openstack-telemetry14:36
silehtgordc, lol the devstack patch have been merged too :p14:36
*** ddieterl_ has joined #openstack-telemetry14:36
gordcsileht: lol, we can test whether your devstack patch works now i guess.14:37
*** ddieterl_ has quit IRC14:38
*** ddieterly has quit IRC14:38
*** liamji has joined #openstack-telemetry14:40
*** KrishR has joined #openstack-telemetry14:41
*** ddieterly has joined #openstack-telemetry14:41
*** ddieterl_ has joined #openstack-telemetry14:41
*** _nadya_ has joined #openstack-telemetry14:44
*** nicodemus_ has joined #openstack-telemetry14:48
*** yprokule has quit IRC14:53
*** rbak has joined #openstack-telemetry14:56
*** yprokule has joined #openstack-telemetry14:58
*** claudiub|2 has joined #openstack-telemetry15:04
*** _nadya__ has joined #openstack-telemetry15:05
*** dave-mcc_ has joined #openstack-telemetry15:05
*** vishwana_ has joined #openstack-telemetry15:05
*** ekarlso- has joined #openstack-telemetry15:07
*** links has quit IRC15:07
*** vishwanathj has quit IRC15:07
*** ekarlso has quit IRC15:07
*** _nadya_ has quit IRC15:07
*** claudiub has quit IRC15:08
*** dave-mccowan has quit IRC15:08
*** rickyrem has joined #openstack-telemetry15:12
*** rbak_ has joined #openstack-telemetry15:12
*** rbak has quit IRC15:13
openstackgerritMerged openstack/ceilometer: Change the SERVICE_TENANT_NAME to SERVICE_PROJECT_NAME  https://review.openstack.org/28338215:13
*** pcaruana has joined #openstack-telemetry15:18
*** pcaruana has quit IRC15:20
*** pcaruana has joined #openstack-telemetry15:21
*** pcaruana|afk| has joined #openstack-telemetry15:21
*** fwdit has quit IRC15:26
*** pcaruana|afk| has quit IRC15:29
*** ildikov has quit IRC15:29
nicodemus_hello jd__, do you have a moment?15:30
*** ildikov has joined #openstack-telemetry15:30
*** _nadya__ has quit IRC15:31
*** pcaruana has quit IRC15:37
*** yprokule has quit IRC15:44
EmilienMsileht: trying to gate puppet openstack CI with full rbd scenario.  glance/nova/cinder are already working but gnocchi is failing: http://logs.openstack.org/62/283362/4/check/gate-puppet-openstack-integration-scenario001-tempest-dsvm-centos7/e26788c/console.html#_2016-02-23_14_32_45_37015:45
EmilienMthe ceph cluster is up and running I think15:46
EmilienMit's probably a configuration issue?15:46
silehtI think !!!15:46
silehtEmilienM, can you paste the gnocchi.conf used ?15:46
EmilienMconfig is here: http://logs.openstack.org/62/283362/4/check/gate-puppet-openstack-integration-scenario001-tempest-dsvm-centos7/e26788c/logs/etc/gnocchi/gnocchi.conf.txt.gz15:46
*** claudiub|2 has quit IRC15:47
EmilienMceph conf is here: http://logs.openstack.org/62/283362/4/check/gate-puppet-openstack-integration-scenario001-tempest-dsvm-centos7/e26788c/logs/etc/ceph/15:48
silehtEmilienM, the configuration looks correct, just ensure that gnocci user can read all of this files15:48
EmilienMsileht: nova/cinder/glance can, so...15:48
EmilienMoh but maybe they don't need to read the keyring file15:48
*** pcaruana has joined #openstack-telemetry15:49
EmilienMsileht: if it would be permissions issue, we would see an error when trying to access to the file, isn't?15:50
silehtEmilienM, ceph errors are not so shiny15:50
EmilienMsileht: why did you require this file?15:51
EmilienMother components don't15:51
EmilienMthey require rbd_secret_uuid, pool and user15:51
EmilienM(cinder for example)15:51
silehtEmilienM, you can fill a bug, if you want I add somthing similar to rbd_secret_uuid instead of the keyring file15:52
EmilienMok15:52
EmilienMthanks15:52
EmilienMI'll also investigate permissions15:52
silehtEmilienM, all your conf looks correct, pool is create, the ceph creads are created and allow to connect to the pool15:53
EmilienMright15:53
silehtEmilienM, where are the source the puppet Ceph::Keys ?15:55
EmilienMsileht: https://github.com/openstack/puppet-ceph/blob/master/manifests/key.pp15:56
silehtEmilienM, I don't see any chmod, and ceph-authtool creates files with limited permissions per default, so if File[/etc/ceph/ceph.client.openstack.keyring] doesn't change the permission, this is your issue15:57
EmilienMit's root:root15:57
EmilienMbut we can change the mod15:57
EmilienMceph::key::mode is a param15:57
EmilienMso I can put 0644 maybe15:57
silehtEmilienM, I understand why the security default is as-is15:57
EmilienMyeah15:58
silehtEmilienM, 0644 is not a good idea,15:58
EmilienMbut it's in our CI15:58
EmilienMit's for gating15:58
silehtbut people that will deploy will got the same kind of issues no ?15:58
silehtEmilienM, I think in a correct deployement gnocchi should have it's own creds, nobody else should have access to its pool15:59
EmilienMsileht: people who deploy it can manage the permissions as they want with puppet16:00
EmilienMright, gnocchi should have it's own keys16:00
EmilienMI can fix it in a further iteration16:00
EmilienMfor now, we have an "openstack" key, shared.16:00
silehtEmilienM, also the rbd_secret_uuid is definitively a good enhancement for Gnocchi too16:00
EmilienMok I'll file a bug16:01
*** julim_ has quit IRC16:01
silehtEmilienM, cool thx!16:01
silehtEmilienM, put me as assignee16:01
*** julim has joined #openstack-telemetry16:02
EmilienMsileht: I can't assign it to you, but here is: https://bugs.launchpad.net/gnocchi/+bug/154889416:04
openstackLaunchpad bug 1548894 in Gnocchi "ceph storage backend: add rbd_secret_uuid support" [Undecided,New]16:04
EmilienMthanks a ton16:04
silehtEmilienM, your welcome16:05
EmilienMsileht: wait, I already set 064416:06
EmilienMhttps://review.openstack.org/#/c/283362/4/manifests/ceph.pp16:06
silehtEmilienM, perhaps username should be client.openstack16:06
EmilienMtrying again16:07
silehtEmilienM, forget we don't put client. into the username in devstack16:08
EmilienMsileht: so I don't need it?16:08
silehtEmilienM, the file path is wrong !16:08
sileht-> /etc/ceph/ceph.client.openstack.keyring instead of /etc/ceph/client.openstack.keyring16:09
EmilienMsileht: nice catch. I swear I was not drunk16:09
EmilienMit's all gordc's fault anyway.16:09
silehtagreed16:10
*** cdent has quit IRC16:13
*** ljxiash has quit IRC16:17
*** belmoreira has quit IRC16:27
*** liamji has quit IRC16:44
*** bdossant has quit IRC16:49
*** thorst is now known as thorst_afk16:51
*** rvasilets has left #openstack-telemetry16:58
nicodemus_has anyone else come across an overflow error with gnocchi-metricd on version stable/2.0?17:02
*** _nadya_ has joined #openstack-telemetry17:13
*** _nadya_ has quit IRC17:15
*** cdent has joined #openstack-telemetry17:15
*** yassine__ has quit IRC17:15
*** _nadya_ has joined #openstack-telemetry17:18
*** _nadya_ has quit IRC17:22
gordcEmilienM: *yawn*17:23
gordcnicodemus_: he's on holiday right now.17:23
gordcwhat's your error?17:24
nicodemus_gordc: oh, didn't know that. Here's the error I see: http://paste.openstack.org/show/487919/17:24
nicodemus_same host, same config but with stable/1.3 it works like a charm17:25
*** vishwana_ is now known as vishwanathj17:33
*** thorst_afk is now known as thorst_17:38
*** _nadya_ has joined #openstack-telemetry17:39
*** AndreiaKumpera has joined #openstack-telemetry17:46
*** safchain has quit IRC17:49
*** claudiub has joined #openstack-telemetry17:59
claudiubcdent: hello. If you have a bit of time, can you take a look at a commit? It already has a +2 and it's fairly straightforward and simple: https://review.openstack.org/#/c/280454/ Thanks!18:00
cdentclaudiub: sure, but I can't promise to pass it if I'm not sure about: I'm a bit out of touch with ceilometer these days18:00
claudiubi'm sure it will be fine. :)18:01
*** _nadya_ has quit IRC18:02
cdentclaudiub: +A18:03
cdentI liked it because it had a lot of red18:03
claudiubcdent: Awesome, thanks! Yees, red is also my favorite color. Unrelated to my transylvanian origins though. :D18:04
* cdent has growing suspicions18:04
gordcnicodemus_: i'll take a look after lunch. it's related to this https://review.openstack.org/#/c/252525/ but i need to figure out what correct fix is18:06
gordcsileht: ^ if you have any ideas.18:06
*** _nadya_ has joined #openstack-telemetry18:08
*** datravis has quit IRC18:09
*** _nadya_ has quit IRC18:16
*** _nadya_ has joined #openstack-telemetry18:21
EmilienMsileht: it works now BTW thanks again18:22
*** lsmola has quit IRC18:30
*** efoley has quit IRC18:34
*** julim has quit IRC18:42
*** julim has joined #openstack-telemetry18:44
*** datravis has joined #openstack-telemetry18:45
*** jwcroppe has quit IRC18:46
*** _nadya_ has quit IRC18:48
*** dave-mcc_ is now known as dave-mccowan18:52
*** _nadya_ has joined #openstack-telemetry18:52
*** _nadya_ has quit IRC19:03
*** _nadya_ has joined #openstack-telemetry19:03
*** _nadya_ has quit IRC19:04
smurkeI was trying to use gnocchi benchmark metric show api it throws out error http://paste.openstack.org/show/487932/19:06
nicodemus_gordc: great! thanks :)19:07
gordcsmurke: i think i've seen that in client. can you open a bug against https://bugs.launchpad.net/python-gnocchiclient/+bugs?orderby=-id&start=019:12
smurkegordc: sure19:12
gordcnicodemus_: what version of pandas do you have?19:14
gordcsmurke: thanks19:15
nicodemus_gordc: 0.17.119:15
gordcsmurke: i take it you only see the error on benchmark commands?19:15
*** ccz has quit IRC19:16
smurkegordc: I think its just the benchmark metric show api19:16
nicodemus_gordc: also I can reproduce the same error smurke sees19:17
gordcsmurke: kk. same.19:18
*** KrishR has quit IRC19:26
*** diogogmt has joined #openstack-telemetry19:30
openstackgerritPradeep Kilambi proposed openstack/ceilometer: Add gnocchi dispatcher opts to config  https://review.openstack.org/28376419:32
gordcnicodemus_: yeah, it's very weird that overflowerror is getting thrown19:33
gordcnicodemus_: does it happen a lot?19:33
nicodemus_gordc: quite often, let me paste a log extract19:35
nicodemus_gordc: here's the log https://gist.github.com/nvlan/53f62e30f7dee84f336a19:38
openstackgerritMerged openstack/ceilometer: Hyper-V: replaces in-tree hyper-v utils usage with os_win  https://review.openstack.org/28045419:38
*** datravis has quit IRC19:43
gordcnicodemus_: did you have pandas 0.17.0 always?19:45
gordcerr 0.17.119:45
gordci wonder if there was a regression19:45
nicodemus_gordc: yes, this log is from an instance that I created yesterday19:46
nicodemus_gordc: however, this instance is pointing to a ceph pool that already contains data19:46
*** KrishR has joined #openstack-telemetry19:47
gordcif you can, see if you can replace the code. http://paste.openstack.org/show/487939/19:49
gordcnicodemus_: i'm curious to see what is causing overflow19:49
*** datravis has joined #openstack-telemetry19:50
gordcnicodemus_: this is the same pool that you hacked to pass upgrade?19:50
nicodemus_gordc: correct19:51
*** nicodemus_ has quit IRC19:55
*** nicodemus_ has joined #openstack-telemetry20:08
nicodemus_gordc: did you receive my last message with the log extract?20:08
gordcnicodemus_: i got "gordc: correct"20:16
gordcnicodemus_: i was going to say your hack looks wrong. do you have original pre-hack data?20:17
nicodemus_gordc: I imagined so, got disconnected. The first 3000 lines of log are all "good timestamp" lines20:19
nicodemus_gordc: but there are still those OverflowError lines among them20:21
gordcwhat's the line above the OverflowError. it should print out 'timestamp' sans good20:22
nicodemus_gordc: I wouldn't be surprised, that hack was only for gnocchi-upgrade to finish20:22
nicodemus_gordc: here are the lines before and after the overflow: http://paste.openstack.org/show/487941/20:23
nicodemus_gordc: and yes, that hack was done on our local lab. These logs are from a different deployment, and the data was not yet updated20:25
nicodemus_gordc: I misspoke when said it was the same pool, my bad20:26
gordcnicodemus_: so this is from something that hasn't been upgraded?20:27
gordchmm.. i wonder why it didn't bring the timestamp when it fails.20:27
nicodemus_gordc: yes, not upgraded20:27
*** jwcroppe has joined #openstack-telemetry20:28
gordcnicodemus_: is there anywhere in logs with "gnocchi.carbonara [-] timestamp"20:29
gordc"ERROR gnocchi.carbonara [-] timestamp"20:29
nicodemus_gordc: let me check20:29
nicodemus_gordc: no, there isn't a single line containing that string20:30
*** flwang has quit IRC20:31
gordcnicodemus_: hmm. i guess overflowerror doesn't inherit Exception somehow... maybe just change the code snippet i gave you and move the LOG.debug line above the calculation.20:32
nicodemus_gordc: ok, let me try that20:33
nicodemus_gordc: here's the log snippet: http://paste.openstack.org/show/487947/20:38
nicodemus_gordc: there's also a TypeError as well, apparently less frequent... line reads: "TypeError: unsupported operand type(s) for *: 'NoneType' and 'float'"20:39
gordcyeah, this looks like because you never upgraded.20:40
nicodemus_gordc: ok, so... just in case I should wait for bug #1548367 to be fixed before attempting to upgrade20:42
openstackbug 1548367 in Gnocchi "Gnocchi-upgrade fails when moving to latest master" [High,Triaged] https://launchpad.net/bugs/154836720:42
*** jwcroppe has quit IRC20:43
gordcnicodemus_: yeah. definitely. i need to explore that a bit more.20:43
nicodemus_gordc: great. Thanks!!20:44
*** rickyrem has quit IRC21:09
*** boris-42 has joined #openstack-telemetry21:18
*** marcin12345 has joined #openstack-telemetry21:21
marcin12345hey guys, do you have any documentation about how to create some universal Dasboard for Grafana that reads from Gnocchi?21:22
gordcmarcin12345: https://julien.danjou.info/blog/2015/openstack-gnocchi-grafana21:24
gordchave you read that?21:24
*** nicodemus_ has quit IRC21:33
marcin12345gordc: yes21:44
marcin12345I can only connect to Gnocchi using older version of Grafana 2.121:45
*** datravis has quit IRC21:45
marcin12345even if I downgrade, grafana never grapsh anything, even though I can retireve data using commands like: measures show  --resource-id <myid> --aggregation mean  cpu_uti --aggregation mean  cpu_uti21:47
marcin12345my point is it would be greate to have some default json file example to import as a dashboard21:47
gordcyeah, we haven't really worked on graphana part recently. been focused on backend perf stuff.21:49
gordci'm nto sure we've tried it on anything beyond 2.121:49
gordcmarcin12345: we're open to contributions/help... really only so much a handful of devs can do.21:50
gordcopen source isn't magic :)21:50
*** cdent has quit IRC21:50
*** cdent has joined #openstack-telemetry21:50
*** john5223 has joined #openstack-telemetry21:52
john5223Hello all. Having issues with ceilometer. My agent is sending the items to the queue but ceilometer-collector isn't picking up the items in "notifications.sample" . Would anyone have any more ideas on how to debug or why?21:53
marcin12345gordc: I know21:53
marcin12345it might be just me being new to grafana21:53
gordcjohn5223: are you using liberty? you need notification agent enabled (that's waht consumes notification.sample queue)21:54
*** julim has quit IRC21:58
john5223gordc, ceilometer-agent-notification service?21:59
gordcjohn5223: right22:01
john5223b/c its started up and running but looks like its no consuming anything22:02
john5223just sitting there when there is over 500 messages in queue for notifications.samples22:02
gordcdo you see any errors in notification agent logs?22:02
gordcif you run rabbitmqctl list_consumers, do you see anything attached to that queue?22:03
gordcmaybe ensure the agent is actually able to listen to correct rabbit host and exchange22:03
john5223no errors in the notification agent log but also no consumers in list_consumers22:04
john5223ah ok. found an error after all. wasn't written to logs. if i ran process directly i can see it22:05
john5223 no such option in group DEFAULT: notification_topics22:07
john5223thats....weird22:07
*** thorst_ has quit IRC22:10
gordcjohn5223: you should have [DEFAULT] notification_topics = notifications in your ceilometer.conf file22:13
john5223yup. i do.22:13
john5223 File "/usr/lib/python2.7/dist-packages/ceilometer/database/notifications.py", line 40, in get_targets22:13
john5223    for topic in conf.notification_topics]22:13
john5223for some reason conf cant access it though. dropping into pdb to maybe see if i can tell why22:14
gordchmm... we should probably remove listening to database notifications since magnetodb is dead22:14
john5223so that might be bug then ?22:16
gordcoh wait. i think i know what you are running into now.22:16
gordcwhich version of liberty do you have?22:17
*** thorst has joined #openstack-telemetry22:17
gordcthere was an oslo.messaging release that broke liberty at some point22:17
gordcfrom the looks of it you, you don't have the fix22:18
*** claudiub has quit IRC22:18
john5223nova version? 12.0.222:20
gordcceilometer22:20
gordcyou may just want to downgrade your oslo.messaging22:20
john5223ceilometer is 5.0.122:20
gordchttps://github.com/openstack/ceilometer/commit/cd2065df67889354703f2a5d38af669630d6776022:21
*** thorst has quit IRC22:21
john5223i did see another issue with ceilometer that is also installed an old pyinotify. had to upgrade to >=0.9.6 for keystone to work in liberty22:22
gordcjohn5223: you'll need 5.0.222:23
john5223gotcha. so looks like ubuntu repo is just a little behind22:24
john5223thanks gordc . appreciate it22:25
gordcyou can also downgrade your oslo.messaging to i believe oslo.messaging<3.0.022:26
gordcerr... oslo.messaging<3.1.022:26
gordchttps://github.com/openstack/oslo.messaging/commit/33c1010c3281804456a22b769c4bac5ac6a7cca1 you don't want this22:26
john5223ii  python-oslo.messaging                2.5.0-1ubuntu122:26
gordcubuntu packages tend to be a lot behind. i think they build of debian packages (if they remember)22:27
john5223hmm. ok i think i found it. its not ubuntu then. it should be using 2.5.0 but somehow i installed 4.1.0 via pip22:28
john5223so that'll fix it.. thanks22:28
gordchmm.. that's should work then... if the package verioning matches pypi22:28
gordcah got it22:28
gordcnp22:28
*** cdent has quit IRC22:30
*** thorst_ has joined #openstack-telemetry22:38
*** thorst_ has quit IRC22:40
openstackgerritMerged openstack/ceilometer: Enable the Load Balancer v2 for the Ceilometer(Part One)  https://review.openstack.org/27451822:48
*** KrishR has quit IRC22:58
*** diogogmt has quit IRC22:58
*** pradk has quit IRC22:58
*** sileht has quit IRC22:58
*** Ephur has quit IRC22:58
*** gtt116_ has quit IRC22:58
*** pcaruana has quit IRC22:58
*** dhellmann has quit IRC22:58
*** tasdomas has quit IRC22:58
*** marcin12345 has quit IRC22:58
*** achatterjee has quit IRC22:58
*** r-mibu has quit IRC22:58
*** mnaser has quit IRC22:58
*** albertom has quit IRC22:58
*** jd__ has quit IRC22:58
*** ryanpetrello has quit IRC22:58
*** mstyne has quit IRC22:58
*** stevelle has quit IRC22:58
*** Guest51435 has quit IRC22:58
*** elemoine_ has quit IRC22:58
*** llu has quit IRC22:58
*** davidlenwell has quit IRC22:58
*** nijaba has quit IRC22:58
*** boris-42 has quit IRC22:58
*** rbak_ has quit IRC22:58
*** gordc has quit IRC22:58
*** chlong_ has quit IRC22:58
*** patrickeast has quit IRC22:58
*** zigo has quit IRC22:58
*** larainema has quit IRC22:58
*** idegtiarov_ has quit IRC22:58
*** jroll has quit IRC22:58
*** AndreiaKumpera has quit IRC22:58
*** vishwanathj has quit IRC22:58
*** ddieterl_ has quit IRC22:58
*** ddieterly has quit IRC22:58
*** liusheng has quit IRC22:58
*** pasquier-s has quit IRC22:58
*** john5223 has quit IRC22:58
*** sheeprine has quit IRC22:58
*** tpeoples has quit IRC22:58
*** agireud has quit IRC22:58
*** EmilienM has quit IRC22:58
*** swamireddy has quit IRC22:58
*** sudorandom has quit IRC22:58
*** jdandrea has quit IRC22:58
*** kbyrne has quit IRC22:58
*** ramishra has quit IRC22:58
*** toddnni has quit IRC22:58
*** ityaptin has quit IRC22:58
*** yarkot has quit IRC22:58
*** titilambert has quit IRC22:58
*** jasonamyers has quit IRC22:58
*** ekarlso- has quit IRC22:58
*** krotscheck has quit IRC22:58
*** j_king has quit IRC22:58
*** idegtiarov has quit IRC22:58
*** tonyb has quit IRC22:58
*** gzhai has quit IRC22:58
*** DinaBelova has quit IRC22:58
*** afazekas has quit IRC22:58
*** smurke has quit IRC22:58
*** bapalm has quit IRC22:58
*** elemoine has quit IRC22:58
*** kfox1111 has quit IRC22:58
*** alextricity25_ has quit IRC22:58
*** serverascode has quit IRC22:58
*** ildikov has quit IRC22:58
*** serverascode has joined #openstack-telemetry23:01
*** alextricity25_ has joined #openstack-telemetry23:01
*** tasdomas has joined #openstack-telemetry23:01
*** dhellmann has joined #openstack-telemetry23:01
*** pcaruana has joined #openstack-telemetry23:01
*** nijaba has joined #openstack-telemetry23:01
*** davidlenwell has joined #openstack-telemetry23:01
*** llu has joined #openstack-telemetry23:01
*** elemoine_ has joined #openstack-telemetry23:01
*** stevelle has joined #openstack-telemetry23:01
*** Guest51435 has joined #openstack-telemetry23:01
*** ryanpetrello has joined #openstack-telemetry23:01
*** mstyne has joined #openstack-telemetry23:01
*** albertom has joined #openstack-telemetry23:01
*** jd__ has joined #openstack-telemetry23:01
*** mnaser has joined #openstack-telemetry23:01
*** r-mibu has joined #openstack-telemetry23:01
*** ramishra has joined #openstack-telemetry23:01
*** kbyrne has joined #openstack-telemetry23:01
*** jdandrea has joined #openstack-telemetry23:01
*** sudorandom has joined #openstack-telemetry23:01
*** swamireddy has joined #openstack-telemetry23:01
*** EmilienM has joined #openstack-telemetry23:01
*** agireud has joined #openstack-telemetry23:01
*** tpeoples has joined #openstack-telemetry23:01
*** sheeprine has joined #openstack-telemetry23:01
*** DinaBelova has joined #openstack-telemetry23:01
*** boris-42 has joined #openstack-telemetry23:01
*** rbak_ has joined #openstack-telemetry23:01
*** gordc has joined #openstack-telemetry23:01
*** chlong_ has joined #openstack-telemetry23:01
*** patrickeast has joined #openstack-telemetry23:01
*** zigo has joined #openstack-telemetry23:01
*** larainema has joined #openstack-telemetry23:01
*** idegtiarov_ has joined #openstack-telemetry23:01
*** jroll has joined #openstack-telemetry23:01
*** KrishR has joined #openstack-telemetry23:02
*** diogogmt has joined #openstack-telemetry23:02
*** pradk has joined #openstack-telemetry23:02
*** sileht has joined #openstack-telemetry23:02
*** Ephur has joined #openstack-telemetry23:02
*** gtt116_ has joined #openstack-telemetry23:02
*** ildikov has joined #openstack-telemetry23:02
*** ekarlso- has joined #openstack-telemetry23:03
*** krotscheck has joined #openstack-telemetry23:03
*** j_king has joined #openstack-telemetry23:03
*** idegtiarov has joined #openstack-telemetry23:03
*** tonyb has joined #openstack-telemetry23:03
*** gzhai has joined #openstack-telemetry23:03
*** kfox1111 has joined #openstack-telemetry23:03
*** afazekas has joined #openstack-telemetry23:03
*** smurke has joined #openstack-telemetry23:03
*** bapalm has joined #openstack-telemetry23:03
*** elemoine has joined #openstack-telemetry23:03
*** toddnni has joined #openstack-telemetry23:03
*** ityaptin has joined #openstack-telemetry23:03
*** yarkot has joined #openstack-telemetry23:03
*** titilambert has joined #openstack-telemetry23:03
*** jasonamyers has joined #openstack-telemetry23:03
*** marcin12345 has joined #openstack-telemetry23:03
*** achatterjee has joined #openstack-telemetry23:03
*** liusheng has joined #openstack-telemetry23:03
*** john5223 has joined #openstack-telemetry23:06
*** AndreiaKumpera has joined #openstack-telemetry23:06
*** vishwanathj has joined #openstack-telemetry23:06
*** ddieterl_ has joined #openstack-telemetry23:06
*** ddieterly has joined #openstack-telemetry23:06
*** 17WAACO2H has joined #openstack-telemetry23:06
*** pasquier-s has joined #openstack-telemetry23:06
*** 17WAACO2H has quit IRC23:07
*** ddieterly has quit IRC23:07
*** ddieterl_ has quit IRC23:07
*** exploreshaifali has joined #openstack-telemetry23:08
*** exploreshaifali has quit IRC23:08
*** kapil has joined #openstack-telemetry23:10
kapili have ubuntu-14.04 on my compute nodes23:10
kapiland ceilometer-agent-ipmi is running as I can see from 'pas aux | grep ceilometer-agent-ipmi'23:11
kapilhowever, ceilometer sample-list -m hardware.ipmi.node.power doesn't return any data23:12
kapilany ideas to what may be wrong ?23:12
gordcdo you have ironic?23:12
kapilumm, i am trying to collect power consumption data for the compute nodes23:13
kapili don't think that would need ironic as in the docs, right ?23:13
gordckapil: no. but you'll need to specify specific resources if you don't have resources managed by ironic.23:13
gordcone sec23:13
gordchttps://bugs.launchpad.net/ceilometer/+bug/1529539/comments/123:14
openstackLaunchpad bug 1529539 in Ceilometer "Ceilometer 5.0 can't pollster hardware snmp meter (cpu_util, memory.usage,....)" [Undecided,Invalid]23:14
kapilso i checked the logs on the compute node and i get the following-23:14
kapilPolling pollster hardware.ipmi.current in the context of meter_source23:14
kapilSkip polling pollster hardware.ipmi.current, no resources found23:14
gordckapil: that comment might help you set it up23:15
kapilcool, i'll try it23:15
openstackgerritMerged openstack/ceilometer: Add gnocchi dispatcher opts to config  https://review.openstack.org/28376423:15
*** thorst has joined #openstack-telemetry23:17
*** kapil has quit IRC23:18
*** kapil has joined #openstack-telemetry23:18
*** thorst has quit IRC23:18
*** thorst has joined #openstack-telemetry23:18
gordcdammit. i can see the gap in our migration test code now. will do this later23:20
*** jwcroppe has joined #openstack-telemetry23:21
*** thorst has quit IRC23:23
kapilcan u give me an example resource that i should specify23:29
kapilotherwise i used to just run ipmitool sdr on the compute node hypervisor to get the sensor data23:29
kapilwould it be the ipmi agent ?23:30
*** diogogmt has quit IRC23:36
gordckapil: sorry, i have to go home.23:40
gordchttps://tank.peermore.com/tanks/cdent-rhat/SnmpWithCeilometer23:41
gordcthis might help23:41
*** gordc has quit IRC23:41
*** jwcroppe has quit IRC23:54
*** rbak_ has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!