| *** liyi has joined #senlin | 01:41 | |
| *** liyi has quit IRC | 03:30 | |
| *** maestropandy has joined #senlin | 04:54 | |
| maestropandy | Hi all.. after creating cluster using senlin. status is "WARNING" why it so ? | 04:59 |
|---|---|---|
| maestropandy | here the cirros file and cluster and nova output paste.openstack.org/show/621917 | 04:59 |
| maestropandy | https:// paste.openstack.org/show/621917 | 04:59 |
| maestropandy | ruijie: I checked nova logs, no isses | 05:00 |
| chenyb4 | maestropandy, you can execute "senlin cluster-show xxx" find status_reason | 05:02 |
| maestropandy | chenyb4: status is http://paste.openstack.org/show/622002/ | 05:04 |
| maestropandy | also the nova list VM state is ERROR, where if i create stack using heat, spinning normal VM no issues.. | 05:04 |
| chenyb4 | maestropandy, Hi, you must find the vm error reson, if the nova vm appear error, senlin node create don't succee | 05:06 |
| chenyb4 | maestropandy, and　cluster status maybe unable to change to active | 05:07 |
| Qiming | yes, better check the reason why the VM created is in ERROR status | 05:07 |
| Qiming | if you do senlin node-show <node> you may find some hints there | 05:08 |
| maestropandy | sure i will check and get .. but meanwhile understand me how to create heat template and do scaling using senlin.. whether aodh/ceilometer need to be installed here, because ceilometer consumes lot of memory and jammed our rabbitmq pipeline.. we feel senlin is good | 05:09 |
| Qiming | senlin supports manual scaling and auto-scaling | 05:09 |
| Qiming | by "manual scaling", you can do "cluster-scale-out", "cluster-scale-in", "cluster-resize", "cluster-nodes-add", "cluster-nodes-del" | 05:10 |
| Qiming | by "auto-scaling", you will have to provision your own monitoring software/solution that triggers a scale change | 05:11 |
| Qiming | Senlin provides you the receiving end for such events, which is called a "receiver" | 05:11 |
| maestropandy | while exapnading created cluster getting error as "transistion failure" http://paste.openstack.org/show/622003/ | 05:12 |
| Qiming | maestropandy, time for you to dig into nova side | 05:13 |
| maestropandy | Qiming: I am thinking to use collectd to trigger alert, apart from ceilomter do you suggest better way to trigger alert which should consume less memory | 05:13 |
| Qiming | you may get more info from 'nova show', to diagnose the vm failure | 05:13 |
| Qiming | as for monitoring solution, we don't have any bias/preferences | 05:14 |
| Qiming | once you create a receiver (by specifying the cluster, the action to trigger), you will get an unique webhook URL | 05:14 |
| Qiming | you, as the solution designer/implementer, decide from where you will invoke that webhook | 05:15 |
| Qiming | it can be anything that can invoke a simple HTTP call to the webhook | 05:15 |
| maestropandy | we are holding big private cloud setup. around 500+ nodes.. and faced challenges with ceilometer.. | 05:15 |
| Qiming | understood | 05:15 |
| maestropandy | in recent summit we attended your session and holded for some time.. now resumed work and am incharge for POC senlin | 05:16 |
| Qiming | no comment to ceilomter, aodh, panko, monsca, nagios, ... | 05:16 |
| Qiming | at the very beginning, we tried to model the "alarm" component into senlin, but we soon abandoned that idea | 05:17 |
| Qiming | it is very difficult to generalize | 05:17 |
| Qiming | customer use cases are so different | 05:17 |
| maestropandy | understood, so in senlin,as now PoC done with ceilometer or other plugins.. i saw senlin presentations, its not mentioned so.. if am missed please correct e | 05:19 |
| Qiming | integration with aodh is documented as an example | 05:19 |
| maestropandy | Qiming: nova has sql connectivity issue.. but wondering while creating via heat working fine,, any way i have restarted nova service will see | 05:20 |
| maestropandy | can i get some handson documents to integrate aodh with senlin, also i dont want to use aodh. can i use aodh itself | 05:20 |
| Qiming | https://docs.openstack.org/senlin/latest/scenarios/autoscaling_ceilometer.html | 05:20 |
| maestropandy | i dont want to use mongodb in case of ceilometer | 05:21 |
| maestropandy | can i use aodh to store DB | 05:22 |
| Qiming | I think ceilometer has long abandoned mongo | 05:22 |
| Qiming | they tried mysql again | 05:23 |
| Qiming | and then they decided to build their own TSDB (timeseries database), which is called Gnocchi | 05:24 |
| maestropandy | as per your ceilometer link, if i want heat.stack profile can i change "type: os.nova.server" to "type: os.heat.server" then will work ? bcz for me need heat stack.. so | 05:24 |
| Qiming | os.heat.stack | 05:25 |
| Qiming | it should work, though you have to make sure the stack (template) contains VM servers that are properly tagged | 05:25 |
| maestropandy | Qiming: yes you are right, I tried to point gnocchi as DB to point ceilomter, but didnt worked.. do you know how to point gnoccchi to use as DB for ceilometer ? also senlin scaleout default is 3600 .. may i know wher it config file sits and to change | 05:25 |
| Qiming | whatever monitoring solution you deploy, you will have to filter and group VMs from the same cluster and then decide whether/when to scale | 05:26 |
| Qiming | 3600 default is an option that can be tuned | 05:26 |
| Qiming | it used to be a global option, but recent patch has made it configurable on a per-cluster basis | 05:27 |
| Qiming | for the global option, search 'default_action_timeout' on this page: https://docs.openstack.org/senlin/pike/configuration/ | 05:29 |
| maestropandy | as per https://docs.openstack.org/senlin/latest/scenarios/autoscaling_ceilometer.html, under receiver mentioned that "Each receiver is usually created for a specific purpose, so for different purposes you may need to create more than receivers." so its like different alarms/config value as like ceilometer ? can't i can keep two "ACTIONS" at a time in receiver like SCALE_OUT && SCALE_IN | 05:31 |
| Qiming | you will need two receivers for two actions | 05:32 |
| Qiming | most likely you will need two alarms as well | 05:32 |
| maestropandy | but i can map to single cluster with two actions & receivers ? | 05:33 |
| maestropandy | Qiming: why "most likely you will need two alarms as well" I mean what are they.. ? | 05:33 |
| Qiming | there is no limitation on the number of receivers you associate with a single cluster | 05:34 |
| Qiming | for example, you will do this: | 05:34 |
| Qiming | - when average cpu utilization is above 80% for 3 minutes, scale out by 1 node | 05:34 |
| Qiming | - when average cpu utilization is below 40% for 10 minutes, scale in by 1 node | 05:35 |
| maestropandy | understand, but need seperate receiver for each actions to.. bcz i tried like this "openstack cluster receiver create --action CLUSTER_SCALE_OUT --params count=2 --cluster mycluster r_01 --action CLUSTER_SCALE_IN --params count=2 --cluster mycluster r_01 | 05:35 |
| Qiming | oh, no ... | 05:35 |
| Qiming | you cannot do that in a single command | 05:36 |
| maestropandy | tried seperately.. as well. where i can see two alarm are mapped to cluster | 05:36 |
| Qiming | not two alarms yet, you only created two receivers for your cluster | 05:37 |
| Qiming | the next step is to trigger those receivers from your favorite monitoring software | 05:37 |
| *** liyi has joined #senlin | 05:38 | |
| Qiming | if you want, you can trigger them using cURL | 05:38 |
| Qiming | you can trigger a scaling operation from crond | 05:38 |
| maestropandy | now i have created two actions with seperate receivers to single cluster, can i see what are the alarms mapped to that cluster. bcz cluster show not showing details. | 05:39 |
| Qiming | you do receiver-list, receiver-show | 05:39 |
| maestropandy | also please let me know how to integrate gnocchi with senlin,.. didn't find answer yet for it so please | 05:40 |
| Qiming | installing and configuring gnocchi is out of senlin's scope | 05:40 |
| maestropandy | u mean "openstack cluster receiver-list" ? | 05:40 |
| Qiming | yep | 05:40 |
| Qiming | gnocchi is about meter sample storage | 05:41 |
| Qiming | we don't touch meter samples directly | 05:41 |
| Qiming | we only provide the integration with aodh (the alarming service under the telemetry project) as an example | 05:41 |
| maestropandy | understand..FYI>. if install aodh and gnocchi its has dependencies with master branch of aodh.. | 05:44 |
| maestropandy | Qiming: I am looking for senlin administraton docs / CLI for senlin.. as of now familiar with most commands, but looking cheatsheet :) or senlin administration | 05:46 |
| Qiming | no cheat :) | 05:52 |
| Qiming | most docs can be found here: https://docs.openstack.org/senlin/latest/ | 05:52 |
| *** liyi has quit IRC | 05:56 | |
| maestropandy | Qiming: I think after creatinng cluster, it state_reason shows "number of active node is below desired capacity (2) " thiugh i create two nodes and add this cluster .. do you think it status will change.. | 05:57 |
| maestropandy | senlin node-list .. totally three nodes all went to error state | 05:58 |
| Qiming | as I have mentioned | 05:58 |
| Qiming | you may first want to dig into nova to find out why you cannot create a VM successfully | 05:59 |
| maestropandy | sorry, i forgot. I will check .. | 06:00 |
| *** liyi has joined #senlin | 06:07 | |
| *** maestropandy has quit IRC | 06:11 | |
| *** maestropandy has joined #senlin | 07:00 | |
| *** maestropandy1 has joined #senlin | 07:04 | |
| *** maestropandy has quit IRC | 07:07 | |
| *** ruijie has quit IRC | 07:19 | |
| *** ruijie has joined #senlin | 07:19 | |
| *** ruijie has quit IRC | 07:20 | |
| *** maestropandy has joined #senlin | 07:35 | |
| *** maestropandy1 has quit IRC | 07:35 | |
| *** ruijie has joined #senlin | 08:08 | |
| *** liyi has quit IRC | 08:18 | |
| *** XueFeng has quit IRC | 08:31 | |
| *** XueFeng has joined #senlin | 08:32 | |
| *** ruijie_ has joined #senlin | 08:56 | |
| *** ruijie has quit IRC | 08:59 | |
| *** maestropandy has quit IRC | 09:10 | |
| *** maestropandy has joined #senlin | 09:30 | |
| *** maestropandy has quit IRC | 09:51 | |
| *** maestropandy has joined #senlin | 09:54 | |
| *** maestropandy1 has joined #senlin | 09:55 | |
| *** maestropandy has quit IRC | 09:58 | |
| *** maestropandy1 has quit IRC | 10:18 | |
| *** ruijie_ has quit IRC | 11:11 | |
| *** maestropandy has joined #senlin | 12:41 | |
| *** jmlowe has quit IRC | 12:48 | |
| *** jmlowe has joined #senlin | 13:34 | |
| *** jmlowe has quit IRC | 14:31 | |
| *** maestropandy1 has joined #senlin | 14:34 | |
| *** maestropandy has quit IRC | 14:36 | |
| *** jmlowe has joined #senlin | 14:37 | |
| *** jmlowe has quit IRC | 14:48 | |
| *** jmlowe has joined #senlin | 14:48 | |
| *** jmlowe has quit IRC | 15:03 | |
| *** jmlowe has joined #senlin | 15:04 | |
| *** Tut__ has joined #senlin | 15:11 | |
| Tut__ | Hi. As soon as I enable senlin-dashboard (stable/ocata devstack on Ubuntu) my Horizon stops to work with JS error "Uncaught Error: [$injector:unpr] Unknown provider: horizon.app.core.detailRouteProvider <- horizon.app.core.detailRoute <- horizon.cluster.profiles.service". Same behavior on pike and master. Any ideas? | 15:16 |
| *** jmlowe has quit IRC | 15:32 | |
| *** Tut__ has quit IRC | 16:02 | |
| *** Tut__ has joined #senlin | 16:02 | |
| *** maestropandy1 has quit IRC | 16:18 | |
| *** jmlowe has joined #senlin | 16:35 | |
| *** eandersson has joined #senlin | 17:54 | |
| *** XueFeng has quit IRC | 18:12 | |
| *** XueFeng has joined #senlin | 18:12 | |
| *** jmlowe has quit IRC | 19:25 | |
| *** jmlowe has joined #senlin | 23:32 | |
| *** jmlowe has quit IRC | 23:52 | |
| *** jmlowe has joined #senlin | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!