| *** ddieterly has joined #openstack-monasca | 00:15 | |
| openstackgerrit | Gary Hessler proposed a change to stackforge/monasca-agent: Updating metrics names and dimensions to be consistent https://review.openstack.org/130063 | 00:22 |
|---|---|---|
| *** rbak has joined #openstack-monasca | 01:42 | |
| *** th_ has quit IRC | 02:23 | |
| *** ericpeterson has joined #openstack-monasca | 03:16 | |
| *** ericpeterson has quit IRC | 03:17 | |
| *** ddieterly has quit IRC | 03:23 | |
| *** ddieterly has joined #openstack-monasca | 03:24 | |
| *** ddieterly has quit IRC | 03:28 | |
| *** rbak has quit IRC | 03:28 | |
| *** rbak has joined #openstack-monasca | 03:29 | |
| *** ddieterly has joined #openstack-monasca | 03:54 | |
| *** ddieterly has quit IRC | 03:58 | |
| *** garyh has quit IRC | 04:09 | |
| *** ddieterly has joined #openstack-monasca | 05:25 | |
| *** ddieterly has quit IRC | 05:30 | |
| *** garyh has joined #openstack-monasca | 05:49 | |
| *** ddieterly has joined #openstack-monasca | 05:56 | |
| *** ddieterly has quit IRC | 06:01 | |
| *** ddieterly has joined #openstack-monasca | 06:55 | |
| *** ddieterly has quit IRC | 07:00 | |
| *** ddieterly has joined #openstack-monasca | 07:24 | |
| *** ddieterly has quit IRC | 07:28 | |
| *** vmunteanu has joined #openstack-monasca | 10:03 | |
| vmunteanu | Hi there! Any monasca-api guys here? | 10:04 |
| *** eoutin has joined #openstack-monasca | 10:16 | |
| *** eglynn-ofice-afk is now known as eglynn-office | 11:27 | |
| *** ddieterly has joined #openstack-monasca | 11:55 | |
| *** ddieterly has quit IRC | 13:00 | |
| *** ddieterly has joined #openstack-monasca | 13:00 | |
| *** ddieterly has quit IRC | 13:05 | |
| *** theresa_ has joined #openstack-monasca | 13:23 | |
| *** ddieterly has joined #openstack-monasca | 13:41 | |
| *** rbak has quit IRC | 13:46 | |
| *** rbak has joined #openstack-monasca | 14:17 | |
| *** ericpeterson has joined #openstack-monasca | 14:19 | |
| openstackgerrit | Ryan Bak proposed a change to stackforge/monasca-ui: Moved files for better egg packaging https://review.openstack.org/131589 | 14:35 |
| vmunteanu | I have some issues with monasca-api connecting to kafka, can anyone lend a hand? | 14:36 |
| samuelms | ddieterly, ping | 14:42 |
| *** bklei has joined #openstack-monasca | 14:42 | |
| samuelms | vmunteanu, I'm not a monasca developer .. | 14:43 |
| samuelms | vmunteanu, I just pinged ddieterly above ^ | 14:43 |
| samuelms | vmunteanu, he participated of monasca-api development | 14:43 |
| bklei | vmunteanu is kafka running? topics created? | 14:45 |
| *** dschroeder has joined #openstack-monasca | 14:49 | |
| openstackgerrit | A change was merged to stackforge/monasca-common: JAH-775 - Monasca-api is encountering SSL problems https://review.openstack.org/131513 | 15:02 |
| vmunteanu | kafka running, topics created | 15:10 |
| vmunteanu | kafka throwing | 15:11 |
| vmunteanu | [2014-10-20 15:26:03,688] ERROR Closing socket for /127.0.0.1 because of error (kafka.network.Processor) | 15:11 |
| vmunteanu | java.io.IOException: Connection reset by peer | 15:11 |
| vmunteanu | Api saying kafka.consumer.SimpleConsumer: Reconnect due to socket error: null (not sure if this is the error) | 15:11 |
| *** th_ has joined #openstack-monasca | 15:11 | |
| *** theresa_ has quit IRC | 15:13 | |
| *** theresa_ has joined #openstack-monasca | 15:14 | |
| bklei | vmunteanu is zookeeper running locally? | 15:15 |
| vmunteanu | yes | 15:16 |
| *** th_ has quit IRC | 15:16 | |
| vmunteanu | do you maybe hint to which might be the cause | 15:18 |
| *** theresa_ has quit IRC | 15:18 | |
| vmunteanu | the monasca-api log goes like this | 15:20 |
| vmunteanu | NFO [2014-10-20 13:34:45,877] kafka.utils.VerifiableProperties: Verifying properties | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,878] kafka.utils.VerifiableProperties: Property metadata.broker.list is overridden to monasca:9092 | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,878] kafka.utils.VerifiableProperties: Property request.timeout.ms is overridden to 30000 | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,878] kafka.utils.VerifiableProperties: Property client.id is overridden to test | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,879] kafka.client.ClientUtils$: Fetching metadata from broker id:0,host:monasca,port:9092 with correlation id 0 for 1 topic(s) Set(healthcheck) | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,879] kafka.producer.SyncProducer: Connected to monasca:9092 for producing | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,881] kafka.producer.SyncProducer: Disconnecting from monasca:9092 | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,886] kafka.consumer.ConsumerFetcherManager: [ConsumerFetcherManager-1413812085814] Adding fetcher for partition [healthcheck,0], initOffset 136 to broker 0 with fetcherId 0 | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,887] kafka.consumer.ConsumerFetcherThread: [ConsumerFetcherThread-test_monasca-1413812085806-94ed5212-0-0], Starting | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,891] kafka.producer.Producer: Shutting down producer | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,892] kafka.producer.ProducerPool: Closing all sync producers | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,892] kafka.producer.SyncProducer: Disconnecting from monasca:9092 | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,894] kafka.consumer.ZookeeperConsumerConnector: [test_monasca-1413812085806-94ed5212], ZKConsumerConnector shutting down | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,894] kafka.utils.KafkaScheduler: Forcing shutdown of Kafka scheduler | 15:20 |
| vmunteanu | INFO [2014-10-20 13:34:45,894] kafka.consumer.ConsumerFetcherManager: [ConsumerFetcherManager-1413812085814] Stopping leader finder thread | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,894] kafka.consumer.ConsumerFetcherManager$LeaderFinderThread: [test_monasca-1413812085806-94ed5212-leader-finder-thread], Shutting down | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,895] kafka.consumer.ConsumerFetcherManager$LeaderFinderThread: [test_monasca-1413812085806-94ed5212-leader-finder-thread], Stopped | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,895] kafka.consumer.ConsumerFetcherManager$LeaderFinderThread: [test_monasca-1413812085806-94ed5212-leader-finder-thread], Shutdown completed | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,895] kafka.consumer.ConsumerFetcherManager: [ConsumerFetcherManager-1413812085814] Stopping all fetchers | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,895] kafka.consumer.ConsumerFetcherThread: [ConsumerFetcherThread-test_monasca-1413812085806-94ed5212-0-0], Shutting down | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,992] kafka.consumer.SimpleConsumer: Reconnect due to socket error: null | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,993] kafka.consumer.ConsumerFetcherThread: [ConsumerFetcherThread-test_monasca-1413812085806-94ed5212-0-0], Shutdown completed | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,994] kafka.consumer.ConsumerFetcherManager: [ConsumerFetcherManager-1413812085814] All connections stopped | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,994] org.I0Itec.zkclient.ZkEventThread: Terminate ZkClient event thread. | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,995] kafka.consumer.ConsumerFetcherThread: [ConsumerFetcherThread-test_monasca-1413812085806-94ed5212-0-0], Stopped | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,998] org.apache.zookeeper.ZooKeeper: Session: 0x1492d79ec6900b6 closed | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,998] kafka.consumer.ZookeeperConsumerConnector: [test_monasca-1413812085806-94ed5212], ZKConsumerConnector shut down completed | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:45,998] org.apache.zookeeper.ClientCnxn: EventThread shut down | 15:21 |
| vmunteanu | INFO [2014-10-20 13:34:46,839] kafka.consumer.ZookeeperConsumerConnector: [test_monasca-1413812085806-94ed5212], stopping watcher executor thread for consumer test_monasca-1413812085806-94ed5212 | 15:21 |
| vmunteanu | that kinda repeats when it receives data from monasca-agent | 15:21 |
| vmunteanu | kafka doesn't send the data to the persister … | 15:22 |
| bklei | the monasca developers can chime in -- but can you confirm kafka is talking to zookeeper? | 15:22 |
| bklei | grep zookeeper /opt/kafka/config/server.properties | 15:22 |
| bklei | # Zookeeper connection string (see zookeeper docs for details). | 15:22 |
| bklei | zookeeper.connect=localhost:2181 | 15:22 |
| bklei | netstat -plnt | grep 2181 | 15:22 |
| bklei | tcp6 0 0 :::2181 :::* LISTEN 1306/java | 15:22 |
| *** theresa_ has joined #openstack-monasca | 15:22 | |
| vmunteanu | nemros@monasca:~$ grep zookeeper kafka/config/server.properties | 15:24 |
| vmunteanu | # Zookeeper connection string (see zookeeper docs for details). | 15:24 |
| vmunteanu | zookeeper.connect=monasca:2181 | 15:24 |
| vmunteanu | # Timeout in ms for connecting to zookeeper | 15:24 |
| vmunteanu | zookeeper.connection.timeout.ms=2000 | 15:24 |
| vmunteanu | emros@monasca:~$ netstat -plnt | grep 2181 | 15:25 |
| vmunteanu | (Not all processes could be identified, non-owned process info | 15:25 |
| vmunteanu | will not be shown, you would have to be root to see it all.) | 15:25 |
| vmunteanu | tcp6 0 0 :::2181 :::* LISTEN - | 15:25 |
| vmunteanu | if i sudo the netstat i get the java pid as well | 15:25 |
| bklei | maybe try increasing zookeeper timeout? | 15:25 |
| bklei | zookeeper.connection.timeout.ms=1000000 | 15:25 |
| bklei | just to rule that out | 15:26 |
| vmunteanu | same error | 15:28 |
| *** th_ has joined #openstack-monasca | 15:29 | |
| bklei | i'm at a loss, has it ever worked? or is this initial install sort of issue? | 15:31 |
| *** theresa_ has quit IRC | 15:32 | |
| vmunteanu | no, it's worked | 15:32 |
| vmunteanu | i have a running installation | 15:32 |
| bklei | anything interesting in /var/log/zookeeper/zookeeper.log? | 15:33 |
| vmunteanu | nothing | 15:33 |
| vmunteanu | 014-10-20 15:49:04,809 - INFO [Thread-193:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:45018 (no session established for client) | 15:33 |
| vmunteanu | 2014-10-20 15:49:04,824 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:37500 | 15:33 |
| vmunteanu | 2014-10-20 15:49:04,825 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@839] - Client attempting to establish new session at /127.0.0.1:37500 | 15:33 |
| vmunteanu | 2014-10-20 15:49:04,827 - INFO [SyncThread:0:ZooKeeperServer@595] - Established session 0x1492d79ec6900ef with negotiated timeout 6000 for client /127.0.0.1:37500 | 15:33 |
| vmunteanu | 2014-10-20 15:49:05,002 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@476] - Processed session termination for sessionid: 0x1492d79ec6900ef | 15:33 |
| vmunteanu | 2014-10-20 15:49:05,004 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1001] - Closed socket connection for client /127.0.0.1:37500 which had sessionid 0x1492d79ec6900ef | 15:33 |
| vmunteanu | can you at least tell me if it's the producer or consumer? | 15:35 |
| bklei | i'm not sure | 15:35 |
| bklei | maybe others in the room know more | 15:36 |
| vmunteanu | Guess then I did a big boo boo :) | 15:36 |
| bklei | well if you figure it out please share your learnings... | 15:36 |
| vmunteanu | I forked the common & api code and made some small changes and serialisation probably fails somewhere but I don't know where | 15:36 |
| *** th_ has quit IRC | 15:37 | |
| vmunteanu | It's quite cumbersome to set up a test environment for monasca | 15:39 |
| *** th_ has joined #openstack-monasca | 15:39 | |
| bklei | yes, lots of moving parts :) | 15:41 |
| vmunteanu | may i ask which part you are in charge of? | 15:42 |
| bklei | just a consumer | 15:42 |
| *** th_ has quit IRC | 15:44 | |
| vmunteanu | anyway, thank you for your help, maybe i'll figure it out :( | 15:46 |
| bklei | let us know | 15:47 |
| vmunteanu | btw, is there a monasca presentation @ openstack paris? | 15:48 |
| bklei | yes, hope it's recorded | 15:49 |
| bklei | https://openstacksummitnovember2014paris.sched.org/event/f377a2f8aa53344eaaed31e5fbf88b8d#.VE_qcIvF9nI | 15:49 |
| vmunteanu | good | 15:51 |
| vmunteanu | if you had a close look at the changes / test locally, could you tell me what i did wrong? | 15:52 |
| bklei | probably don't have time for that :( | 15:53 |
| vmunteanu | was worth the try :) | 15:53 |
| bklei | are you running monasca with influxdb? | 15:54 |
| vmunteanu | yes | 15:56 |
| vmunteanu | btw, remove things like System.out.println("Received " + msg); in kafkahealthcheck.java :) | 15:56 |
| bklei | just curious about influxdb performance, size of your cluster... | 15:57 |
| bklei | STDOUT == awesome | 15:57 |
| vmunteanu | but you have the log, why use stdout :P | 15:57 |
| vmunteanu | anyway, i'm doing a postdoc on cloud incident management | 15:58 |
| vmunteanu | and monasca is really great in many aspects | 15:58 |
| vmunteanu | and what i'm trying to do is get logs processed by monasca as well | 15:59 |
| bklei | cool | 15:59 |
| vmunteanu | so I've changed the subexpressions to support regex and like | 15:59 |
| vmunteanu | and everything passes your unit tests and runs but somewhere there is a hickup and i can't find it | 16:00 |
| vmunteanu | don't know if it's in the producer or consumer or somewhere else... | 16:00 |
| vmunteanu | also, we run influx on a single machine right now, we'll change it when i get things working | 16:01 |
| *** theresa_ has joined #openstack-monasca | 16:02 | |
| bklei | same here -- going to move to a 3 node cluster shortly | 16:03 |
| vmunteanu | my colleagues will be in paris, i'll ask them to look at your presentation | 16:04 |
| *** dschroeder has quit IRC | 16:09 | |
| vmunteanu | are you guys mostly from the us or eu? | 16:10 |
| *** dschroeder has joined #openstack-monasca | 16:14 | |
| *** theresa_ has quit IRC | 16:15 | |
| *** eoutin has quit IRC | 16:18 | |
| bklei | I think the bulk of the monasca dev team -- at least the ones i know are US | 16:19 |
| *** theresa_ has joined #openstack-monasca | 16:30 | |
| *** th_ has joined #openstack-monasca | 16:32 | |
| vmunteanu | do you guys have it deployed somewhere in production or just testing? | 16:32 |
| bklei | so far, just testing | 16:33 |
| bklei | you? | 16:33 |
| vmunteanu | i'm academia so just testing :) | 16:33 |
| bklei | gotcha | 16:33 |
| vmunteanu | i'm trying to make it work for the work i do | 16:35 |
| vmunteanu | but there are a lot of inconsitencies (naming etc) | 16:36 |
| *** theresa_ has quit IRC | 16:36 | |
| vmunteanu | also, i guess you're rewriting monasca-api in python to get it into openstack? | 16:36 |
| *** Michalik has quit IRC | 16:39 | |
| bklei | right, there's a big effort underway to rewrite all the java (api, thresh, persister) in python | 16:43 |
| vmunteanu | could you, maybe, help me get in touch with any of your european colleagues? | 16:43 |
| bklei | looks like sandy is UK https://openstacksummitnovember2014paris.sched.org/speaker/sandywalsh#.VFEaLfTF9nI | 16:46 |
| vmunteanu | ok, thanks | 16:47 |
| vmunteanu | i thought monasca was fully HP and he's with Rackspace… | 16:47 |
| bklei | yeah, joint effort | 16:48 |
| openstackgerrit | Gary Hessler proposed a change to stackforge/monasca-notification: Switch to using monasca-statsd https://review.openstack.org/131822 | 16:49 |
| vmunteanu | do you think you could make me an introduction with Sandy? | 16:57 |
| bklei | sure -- for 100 euros | 16:59 |
| ericpeterson | lol | 16:59 |
| vmunteanu | lol | 17:00 |
| vmunteanu | well i can pay you back in beer when i'll meet you | 17:03 |
| bklei | :) looks like sandy is in the room | 17:04 |
| openstackgerrit | Gary Hessler proposed a change to stackforge/monasca-notification: Switch to using monasca-statsd https://review.openstack.org/131822 | 17:08 |
| *** vmunteanu has quit IRC | 17:29 | |
| *** Michalik has joined #openstack-monasca | 18:00 | |
| *** theresa_ has joined #openstack-monasca | 18:02 | |
| *** th_ has quit IRC | 18:03 | |
| *** th_ has joined #openstack-monasca | 18:06 | |
| *** theresa_ has quit IRC | 18:09 | |
| *** dschroeder has quit IRC | 18:23 | |
| *** dschroeder has joined #openstack-monasca | 18:41 | |
| *** th_ has quit IRC | 18:56 | |
| *** th_ has joined #openstack-monasca | 19:00 | |
| *** vmunteanu has joined #openstack-monasca | 19:20 | |
| *** ericpeterson has quit IRC | 20:13 | |
| *** vmunteanu has quit IRC | 20:16 | |
| *** garyh_ has joined #openstack-monasca | 20:29 | |
| *** ericpeterson has joined #openstack-monasca | 20:30 | |
| *** garyh has quit IRC | 20:31 | |
| *** garyh_ is now known as garyh | 20:31 | |
| *** vmunteanu has joined #openstack-monasca | 20:32 | |
| *** openstackgerrit has quit IRC | 21:50 | |
| *** theresa_ has joined #openstack-monasca | 21:59 | |
| *** th_ has quit IRC | 22:00 | |
| *** bklei has quit IRC | 22:02 | |
| *** theresa_ has quit IRC | 22:08 | |
| *** vmunteanu has quit IRC | 22:20 | |
| *** ericpeterson has quit IRC | 22:32 | |
| *** bklei has joined #openstack-monasca | 22:39 | |
| *** bklei has quit IRC | 22:54 | |
| *** bklei has joined #openstack-monasca | 22:54 | |
| *** rbak has quit IRC | 22:55 | |
| *** bklei has quit IRC | 22:59 | |
| *** ddieterly has quit IRC | 23:02 | |
| *** dschroeder has quit IRC | 23:25 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!