| *** Miouge has joined #openstack-monasca | 07:48 | |
| *** Miouge has quit IRC | 08:31 | |
| *** Miouge has joined #openstack-monasca | 09:17 | |
| *** Miouge has quit IRC | 11:32 | |
| -openstackstatus- NOTICE: gerrit has been restarted to restore event streaming. any change events missed by zuul (between 10:56 and 11:37 utc) will need to be rechecked or have new approval votes set | 11:39 | |
| *** htruta has joined #openstack-monasca | 12:06 | |
| *** rbak_ has quit IRC | 13:26 | |
| *** craigbr has joined #openstack-monasca | 13:31 | |
| *** craigbr has quit IRC | 13:36 | |
| *** ddieterly has joined #openstack-monasca | 13:49 | |
| *** gilliard has quit IRC | 13:53 | |
| *** dlfryar has quit IRC | 13:53 | |
| *** jklare has quit IRC | 13:53 | |
| *** stackedsax has quit IRC | 13:53 | |
| *** sc has quit IRC | 13:53 | |
| *** shaneduan has quit IRC | 13:53 | |
| *** dlfryar has joined #openstack-monasca | 13:55 | |
| *** jklare has joined #openstack-monasca | 13:55 | |
| *** stackedsax has joined #openstack-monasca | 13:56 | |
| *** gilliard has joined #openstack-monasca | 13:58 | |
| *** sc has joined #openstack-monasca | 13:58 | |
| *** shaneduan has joined #openstack-monasca | 13:58 | |
| *** sc has quit IRC | 13:59 | |
| *** craigbr has joined #openstack-monasca | 14:01 | |
| *** rbak_ has joined #openstack-monasca | 14:01 | |
| *** theresa_ has joined #openstack-monasca | 14:01 | |
| *** craigbr has quit IRC | 14:01 | |
| *** sc has joined #openstack-monasca | 14:01 | |
| *** rbak_ has quit IRC | 14:07 | |
| *** rbak has joined #openstack-monasca | 14:09 | |
| *** theresa_ has quit IRC | 14:30 | |
| openstackgerrit | Dexter Fryar proposed stackforge/monasca-agent: Exception wrapper to handle type errors https://review.openstack.org/170882 | 14:53 |
|---|---|---|
| *** theresa_ has joined #openstack-monasca | 15:10 | |
| *** dschroeder has joined #openstack-monasca | 15:10 | |
| *** ericpeterson has joined #openstack-monasca | 15:14 | |
| *** cindyo has joined #openstack-monasca | 15:36 | |
| *** theresa_ has quit IRC | 15:40 | |
| *** Miouge has joined #openstack-monasca | 15:41 | |
| *** ericpeterson has quit IRC | 15:45 | |
| *** ericpeterson has joined #openstack-monasca | 15:47 | |
| openstackgerrit | Deklan Dieterly proposed stackforge/monasca-persister: Refactor influxdb and vertica classes and code https://review.openstack.org/170632 | 15:50 |
| *** craigbr has joined #openstack-monasca | 16:05 | |
| *** theresa_ has joined #openstack-monasca | 16:10 | |
| *** ericpeterson has quit IRC | 16:12 | |
| *** craigbr has quit IRC | 16:27 | |
| *** craigbr has joined #openstack-monasca | 16:30 | |
| *** ericpeterson has joined #openstack-monasca | 16:35 | |
| *** ddieterly has quit IRC | 16:42 | |
| *** ddieterly has joined #openstack-monasca | 16:42 | |
| *** ddieterly has quit IRC | 16:42 | |
| *** ddieterly has joined #openstack-monasca | 16:43 | |
| *** Miouge has quit IRC | 17:11 | |
| openstackgerrit | Deklan Dieterly proposed stackforge/monasca-persister: Refactor influxdb and vertica classes and code https://review.openstack.org/170632 | 17:20 |
| *** craigbr has quit IRC | 17:30 | |
| *** ericpeterson has quit IRC | 17:35 | |
| *** ericpeterson has joined #openstack-monasca | 17:36 | |
| *** theresa_ has quit IRC | 17:59 | |
| *** cindyo has quit IRC | 18:08 | |
| *** ericpete_ has joined #openstack-monasca | 18:09 | |
| *** ericpeterson has quit IRC | 18:13 | |
| *** craigbr has joined #openstack-monasca | 18:13 | |
| *** ericpete_ has quit IRC | 18:14 | |
| *** ericpeterson has joined #openstack-monasca | 18:15 | |
| *** rbak_ has joined #openstack-monasca | 18:20 | |
| *** zhill has joined #openstack-monasca | 18:21 | |
| *** rbak has quit IRC | 18:23 | |
| openstackgerrit | Ryan Bak proposed stackforge/python-monascaclient: Correctly selects endpoint based on region https://review.openstack.org/170958 | 19:01 |
| *** theresa_ has joined #openstack-monasca | 19:01 | |
| *** cindyo has joined #openstack-monasca | 19:05 | |
| *** LeanReox has joined #openstack-monasca | 19:17 | |
| LeanReox | Hi guys, long time no see, can i bother you with a tresh question craigbr ddieterly ? | 19:20 |
| craigbr | LeanReox: In a meeting for 40 minutes, yet | 19:21 |
| LeanReox | no worries, i can wait craigbr , its about alarms not getting triggered | 19:22 |
| openstackgerrit | Deklan Dieterly proposed stackforge/monasca-api: Accept gzip results from influxdb https://review.openstack.org/170963 | 19:26 |
| *** zhill has quit IRC | 19:33 | |
| *** zhill has joined #openstack-monasca | 19:35 | |
| *** zhill has quit IRC | 19:40 | |
| openstackgerrit | Ryan Bak proposed stackforge/monasca-agent: Adds region option to libvirt plugin config https://review.openstack.org/170975 | 19:59 |
| *** ericpete_ has joined #openstack-monasca | 19:59 | |
| *** ericpeterson has quit IRC | 20:02 | |
| craigbr | @LeanReox: I'm back. What is your question? | 20:03 |
| LeanReox | craigbr, i have a healthy running environment, but when i set an alarm to test alarm trigger for example an easy on, cpu.idle_perc > 0 | 20:04 |
| LeanReox | i dont get any alarm | 20:04 |
| LeanReox | i see the topology on the storm cluster, metrics are persisted to influx ok | 20:04 |
| LeanReox | so i need to know what to watch | 20:05 |
| craigbr | Are you getting any messages /var/log/storm/worker_*.log? | 20:05 |
| LeanReox | craigbr, lets check | 20:05 |
| *** theresa_ has quit IRC | 20:09 | |
| *** ericpete_ has quit IRC | 20:14 | |
| LeanReox | craigbr just reconnection to the zookeepers | 20:15 |
| craigbr | How about in /var/log/storm/supervisor.log? It should be older than the worker-*.log files. If it is newer, then the workers aren't really running | 20:17 |
| LeanReox | some old communication errors with StatsD UPD client | 20:17 |
| *** ericpeterson has joined #openstack-monasca | 20:17 | |
| LeanReox | let me check craigbr | 20:17 |
| LeanReox | :q! | 20:17 |
| LeanReox | ok, here i have an iteresting error, java.lang.RuntimeException: java.io.EOFException | 20:18 |
| LeanReox | 2015-04-06 20:17:55 b.s.event [ERROR] Error when processing event | 20:19 |
| craigbr | Is that event constantly occurring? | 20:22 |
| LeanReox | let me check | 20:23 |
| LeanReox | no its from 3 days ago | 20:25 |
| craigbr | Let's see if it is consuming from kafka ok. Try: | 20:26 |
| craigbr | /opt/kafka/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --topic events -zkconnect localhost:2181 --group thresh-event | 20:26 |
| craigbr | That is assuming that zookeeper is on localhost and you haven't changed the default group in /etc/monasca/thresh-config.yml | 20:27 |
| LeanReox | sorry my bad with the last question, the error is constantly ocurring | 20:27 |
| LeanReox | im tailing the log and its constantly poping | 20:27 |
| LeanReox | on the supervisor.log | 20:28 |
| craigbr | Good. That is pretty simple. Determine your storm.local.dir by looking in your storm.yaml. Our default is in /opt/storm/current/conf/storm.yaml | 20:28 |
| craigbr | sudo service monasca-thresh stop | 20:28 |
| craigbr | sudo stop storm-supervisor | 20:29 |
| LeanReox | /opt/storm/apache-storm-0.9.2-incubating/conf/storm.yaml | 20:29 |
| LeanReox | sorry bad paste | 20:29 |
| craigbr | Stop supervisor on all your nodes running thresh | 20:29 |
| LeanReox | storm local dir is set to /var/storm | 20:30 |
| craigbr | sudo stop storm-nimbus | 20:30 |
| craigbr | Once you get all supervisors and nimbus stopped, then on all the nodes do: | 20:30 |
| craigbr | sudo rm -fr /var/storm/*/* | 20:30 |
| craigbr | There is bad data and it can't read it so it keeps crashing | 20:30 |
| LeanReox | ok ... stopping thresh , killed topology | 20:30 |
| craigbr | Then restart nimbus | 20:30 |
| craigbr | Then restart all supervisors | 20:31 |
| craigbr | Then restart monasca-thresh | 20:31 |
| craigbr | Depending on how long it has been down, it may take a while to read all the queued metrics | 20:31 |
| *** ericpeterson has quit IRC | 20:31 | |
| *** ericpeterson has joined #openstack-monasca | 20:32 | |
| *** zhill has joined #openstack-monasca | 20:35 | |
| *** theresa_ has joined #openstack-monasca | 20:37 | |
| openstackgerrit | Merged stackforge/python-monascaclient: Correctly selects endpoint based on region https://review.openstack.org/170958 | 20:40 |
| *** zhill has quit IRC | 20:40 | |
| *** theresa_ has quit IRC | 20:44 | |
| *** craigbr has quit IRC | 20:56 | |
| *** craigbr has joined #openstack-monasca | 20:57 | |
| *** craigbr has quit IRC | 20:58 | |
| *** craigbr has joined #openstack-monasca | 20:59 | |
| LeanReox | craigbr, ill do that and re-check, thanks ! | 21:13 |
| LeanReox | craigbr, ill let you know | 21:13 |
| craigbr | LeanReox: Good luck | 21:13 |
| openstackgerrit | Merged stackforge/monasca-api: Accept gzip results from influxdb https://review.openstack.org/170963 | 21:23 |
| openstackgerrit | Merged stackforge/monasca-agent: Adds region option to libvirt plugin config https://review.openstack.org/170975 | 21:32 |
| openstackgerrit | Craig Bryant proposed stackforge/monasca-api: Allow empty values for value_meta https://review.openstack.org/170985 | 21:36 |
| *** zhill has joined #openstack-monasca | 21:36 | |
| *** ericpeterson has quit IRC | 21:39 | |
| *** zhill has quit IRC | 21:41 | |
| openstackgerrit | Deklan Dieterly proposed stackforge/monasca-persister: Send request and response data to Influxdb gzip encoded https://review.openstack.org/170987 | 21:47 |
| *** ddieterly has quit IRC | 21:59 | |
| *** rbak_ has quit IRC | 22:07 | |
| *** ddieterly has joined #openstack-monasca | 22:15 | |
| *** ddieterly has quit IRC | 22:21 | |
| *** ddieterly has joined #openstack-monasca | 22:21 | |
| *** craigbr has quit IRC | 22:23 | |
| *** craigbr has joined #openstack-monasca | 22:28 | |
| *** dlfryar has quit IRC | 22:30 | |
| *** ddieterly has quit IRC | 22:30 | |
| *** zhill has joined #openstack-monasca | 22:37 | |
| *** zhill has quit IRC | 22:42 | |
| *** LeanReox has quit IRC | 22:49 | |
| *** cindyo has quit IRC | 23:03 | |
| *** dschroeder has quit IRC | 23:26 | |
| *** craigbr has quit IRC | 23:36 | |
| *** zhill has joined #openstack-monasca | 23:38 | |
| *** zhill has quit IRC | 23:42 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!