| *** ruijie has joined #senlin | 00:58 | |
| *** zhurong has joined #senlin | 01:13 | |
| *** Drago1 has quit IRC | 01:13 | |
| *** Drago1 has joined #senlin | 01:14 | |
| *** guoshan has joined #senlin | 01:21 | |
| *** EldonZhao has joined #senlin | 01:28 | |
| *** yanyanhu has joined #senlin | 01:42 | |
| *** elynn has joined #senlin | 01:43 | |
| *** chohoor has joined #senlin | 01:46 | |
| *** elynn has quit IRC | 01:48 | |
| *** elynn has joined #senlin | 01:50 | |
| openstackgerrit | Merged openstack/senlin: Remove 'validate_sort_params" https://review.openstack.org/407483 | 01:55 |
|---|---|---|
| EldonZhao | Do we support creating a node using an existed nova instance? | 01:58 |
| Qiming | EldonZhao, we discussed that | 02:03 |
| Qiming | [16:00] <Qiming> a draft idea on solving this: | 02:04 |
| Qiming | [16:01] <Qiming> 1. senlin profile-spec-create -t os.nova.server -o <spec_file> <nova_instance> | 02:04 |
| Qiming | [16:01] <Qiming> 2. edit the spec_file created, fix parameters like volume and network settings | 02:04 |
| Qiming | [16:02] <Qiming> 3. senlin profile-create -s <spec_file> profile_for_existing_instance | 02:04 |
| Qiming | [16:02] <Qiming> 4. senlin node-adopt -p profile_for_existing_instance <nova_instance> | 02:04 |
| Qiming | [16:03] <Qiming> 5. senlin cluster-node-add -n <node_id_from_step_4> <cluster_identity> | 02:04 |
| Qiming | [16:06] <Qiming> it is a lengthy flow, some steps can be merged/combined I think | 02:04 |
| Qiming | the first step would be creating a spec file from an existing nova instance | 02:05 |
| Qiming | then users are expected to do some tuning/modifications to the spec file, regarding the network and storage parameters | 02:05 |
| Qiming | then he/she creates a new profile instance using this spec | 02:06 |
| Qiming | the next step (regarding senlin's support), would be adding a node-adopt API, treating the nova instance as created from senlin | 02:06 |
| *** elynn_ has joined #senlin | 02:19 | |
| *** elynn has quit IRC | 02:20 | |
| EldonZhao | Qiming, Thanks:) | 02:20 |
| openstackgerrit | Aaron Ding proposed openstack/python-senlinclient: Add destroy_after_deletion param in`cluster members del`. https://review.openstack.org/408898 | 02:20 |
| openstackgerrit | Aaron Ding proposed openstack/senlin: Add destroy_after_deletion param when cluster del nodes. https://review.openstack.org/408899 | 02:21 |
| Qiming | EldonZhao, do you have a strong requirement on this? | 02:40 |
| Qiming | we can put it to a higher priority if so | 02:40 |
| Qiming | just talk to yanyan | 02:40 |
| Qiming | I'm afraid we won't be able to land it before Ocata-2, the window is too short, Ocata-3 would be a possibility | 02:41 |
| EldonZhao | We are discussing with users now, and I think they need us support it just like Aliyun. | 02:41 |
| Qiming | okay, understood | 02:42 |
| Qiming | do you understand why senlin needs to create a spec out of a nova server and then have the user to modify it? | 02:43 |
| *** yuanying has quit IRC | 02:50 | |
| *** yuanying has joined #senlin | 03:50 | |
| *** yuanying has quit IRC | 03:50 | |
| *** yuanying has joined #senlin | 03:51 | |
| *** guoshan has quit IRC | 03:55 | |
| *** elynn_ has quit IRC | 04:23 | |
| *** yuanying has quit IRC | 04:23 | |
| *** yuanying has joined #senlin | 04:24 | |
| *** yuanying has quit IRC | 04:24 | |
| *** yuanying has joined #senlin | 04:25 | |
| *** yuanying has quit IRC | 04:25 | |
| *** guoshan has joined #senlin | 04:56 | |
| *** guoshan has quit IRC | 05:00 | |
| *** yuanying has joined #senlin | 05:26 | |
| *** yuanying has quit IRC | 05:35 | |
| *** yuanying has joined #senlin | 05:36 | |
| *** yuanying has quit IRC | 05:49 | |
| *** elynn_ has joined #senlin | 05:52 | |
| *** elynn__ has joined #senlin | 05:56 | |
| *** elynn_ has quit IRC | 05:56 | |
| openstackgerrit | Aaron Ding proposed openstack/python-senlinclient: Get project_id and user_id when show profile https://review.openstack.org/407420 | 06:07 |
| *** guoshan has joined #senlin | 06:09 | |
| *** guoshan has quit IRC | 06:13 | |
| openstackgerrit | Merged openstack/senlin-dashboard: Replace mox with mock https://review.openstack.org/408605 | 06:14 |
| *** guoshan has joined #senlin | 06:15 | |
| openstackgerrit | Qiming Teng proposed openstack/senlin: Augment request context with api microversion https://review.openstack.org/408970 | 06:19 |
| *** yuanying has joined #senlin | 06:28 | |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: engine service support cluster-policy-list2 https://review.openstack.org/408509 | 06:39 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: api layer support cluster-policy-list2 https://review.openstack.org/408976 | 06:39 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: api layer support cluster-policy-list2 https://review.openstack.org/408976 | 06:42 |
| *** Drago1 has quit IRC | 06:42 | |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: engine service support cluster-policy-list2 https://review.openstack.org/408509 | 06:53 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: api layer support cluster-policy-list2 https://review.openstack.org/408976 | 06:53 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: remove dead code of cluster-policy-list https://review.openstack.org/408979 | 06:53 |
| *** yuanying_ has joined #senlin | 06:55 | |
| openstackgerrit | Merged openstack/senlin: Updated from global requirements https://review.openstack.org/408114 | 06:56 |
| openstackgerrit | Merged openstack/senlin: Temporarily disable failed receiver integration test https://review.openstack.org/407274 | 06:56 |
| *** zhurong has quit IRC | 06:57 | |
| *** zhurong has joined #senlin | 06:58 | |
| *** yuanying has quit IRC | 06:58 | |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: remove dead code of cluster-policy-list https://review.openstack.org/408979 | 07:01 |
| openstackgerrit | Merged openstack/senlin: Remove 'ProfileTypeNotMatch' exception type https://review.openstack.org/407938 | 07:02 |
| *** yuanying_ has quit IRC | 07:14 | |
| *** guoshan has quit IRC | 07:22 | |
| ruijie | hi, Qiming | 07:22 |
| Qiming | hi | 07:27 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Versioned request object for receiver_nofity https://review.openstack.org/408990 | 07:27 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Engine support for receiver_notify2 https://review.openstack.org/408991 | 07:27 |
| ruijie | I am thinking about a scenario | 07:28 |
| ruijie | e.g. we have a cron task to do cluster_scale_out periodly | 07:28 |
| ruijie | say every 11:00 am | 07:28 |
| ruijie | if the task failed to scale out the cluster, we need to redo it. | 07:29 |
| ruijie | count=5 succeeded=4, failed=1. then the problem will be we create 9 node if the 2nd task succeeded | 07:30 |
| ruijie | do the recover action can cover this case? | 07:30 |
| ruijie | does | 07:30 |
| *** guoshan has joined #senlin | 07:30 | |
| Qiming | it should | 07:33 |
| Qiming | so, the design philosophy is like this | 07:33 |
| Qiming | we can never ensure that all cluster operations are completely successful | 07:34 |
| Qiming | especially when you do scaling | 07:34 |
| Qiming | after each cluster operation, especially the cluster scaling operations, we update the cluster's desired_capacity | 07:35 |
| Qiming | and you can count the actual capacity by counting cluster nodes | 07:35 |
| Qiming | say you have a cluster of 3 nodes, you want to add 4 more to it, by cluster-add-nodes or cluster-scale-out | 07:36 |
| Qiming | then senlin knows the 'desired_capacity' is 7 now | 07:36 |
| Qiming | it can never ensure that the cloud can eventually give you 7 nodes to use | 07:37 |
| Qiming | you can detect that from the cluster status, by checking count of cluster-node-list | 07:37 |
| ruijie | so this working flow it totally controlled by the user | 07:38 |
| ruijie | it/is | 07:39 |
| Qiming | yes, we wanted to make some decisions for users | 07:42 |
| Qiming | but it doesn't seem the right approach | 07:42 |
| Qiming | by the way, cluster-check and cluster-recover workflows need some more hammers | 07:43 |
| ruijie | em, infact we are trying to avoid repeatedly creating resources | 07:43 |
| ruijie | the recover action is the way we are finding :) | 07:44 |
| Qiming | currently, take cluster-recover as an example, we only recover the list of nodes that are not ACTIVE | 07:44 |
| Qiming | and I'm not 100% confident that workflow is free of bugs | 07:45 |
| ruijie | we need to monitor the real status of the server? | 07:45 |
| Qiming | we have cluster-check API to do that | 07:46 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: API support for receiver_notify2 https://review.openstack.org/408997 | 07:46 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Remove dead code about receiver-notify https://review.openstack.org/408998 | 07:46 |
| Qiming | we rejected the idea to to automatic periodical checking | 07:46 |
| Qiming | we only expose such APIs for users to invoke on demand | 07:46 |
| Qiming | if you want to achieve a high availability of your cluster, you may want to create and attach a health policy | 07:47 |
| ruijie | yes, that makes sense | 07:48 |
| ruijie | or that will be heavy work | 07:48 |
| Qiming | it doesn't matter if it is a heavy work or not | 07:48 |
| openstackgerrit | Qiming Teng proposed openstack/senlin: Add version map search to base object https://review.openstack.org/409001 | 07:48 |
| Qiming | the real question is whether we are doing too much to make users hate us | 07:49 |
| ruijie | I see, Qiming, let the user themselves to build their workflow | 07:50 |
| Qiming | yes | 07:50 |
| Qiming | if they trust us, we do offer some built in workflows | 07:51 |
| Qiming | for example, the ha policy is doing a lot of that | 07:51 |
| Qiming | the policy does periodical health check on your cluster nodes and attempt a recovery if node failures are found | 07:51 |
| Qiming | it can also listen to VM failure events and then react to those events (properly) | 07:52 |
| ruijie | that's cool | 07:54 |
| ruijie | that makes the cluster more reliable | 07:54 |
| ruijie | Qiming, thanks for your time, will check the HA part | 07:55 |
| Qiming | there are still a lot of things to do in that space | 07:55 |
| Qiming | the health policy is unable to treat different events in different ways | 07:55 |
| Qiming | we are also looking into extending the workflow to execute jobs defined by users | 07:56 |
| ruijie | ah, only the active and Non-active? | 07:56 |
| Qiming | in the case of a VM failure, senlin will try reboot, rebuild, recreate that VM, and after that, we want enable senlin to invoke a mistral workflow for further recovery | 07:57 |
| Qiming | look here, ruijie | 08:00 |
| Qiming | http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/health_manager.py#n57 | 08:00 |
| ruijie | since we already could recreate a server, what do we want to to with mistral? | 08:00 |
| Qiming | because creating VMs is not the goal | 08:01 |
| Qiming | your goal is always about getting some applications/services configured and started in those VMs | 08:01 |
| Qiming | those are part of the recovery operation | 08:02 |
| ruijie | right | 08:02 |
| Qiming | and we don't know what users want to do after the VM is recovered | 08:02 |
| Qiming | if they model their operations using mistral already, we can invoke those workflows | 08:02 |
| ruijie | great, then the user could use that plugin/driver .. to run init scripts or something to let their app or service up | 08:03 |
| Qiming | yes | 08:05 |
| Qiming | the whole workflow would be something like this: | 08:05 |
| Qiming | vm failed -> detected by nova -> nova sends notification -> senlin heard the notification -> senlin decides whether fencing is needed and does it if so -> senlin initiates a vm recover operation by providing the needed info --> senlin invoke external logics to get apps resurrected ... | 08:07 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: engine service support cluster-policy-list2 https://review.openstack.org/408509 | 08:08 |
| ruijie | great | 08:10 |
| ruijie | Qiming, I will check this 2 APIs and the HA part | 08:10 |
| Qiming | okay | 08:12 |
| ruijie | thanks, Qiming :) | 08:14 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: api layer support cluster-policy-list2 https://review.openstack.org/408976 | 08:15 |
| openstackgerrit | Merged openstack/senlin: Request object for action-create https://review.openstack.org/408567 | 08:17 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: remove dead code of cluster-policy-list https://review.openstack.org/408979 | 08:18 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: engine service support cluster-policy-list2 https://review.openstack.org/408509 | 08:18 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: api layer support cluster-policy-list2 https://review.openstack.org/408976 | 08:18 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Correct the output of cluster-update https://review.openstack.org/409021 | 08:36 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Correct the output of cluster-update https://review.openstack.org/409021 | 08:39 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Correct the output of cluster-update https://review.openstack.org/409021 | 08:42 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Correct the output of node-update https://review.openstack.org/409029 | 08:50 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: fields obj support cluster-policy-get2 https://review.openstack.org/409039 | 09:12 |
| yanyanhu | hi, elynn__, around? | 09:13 |
| elynn__ | Hi yanyanhu | 09:14 |
| yanyanhu | hi | 09:14 |
| yanyanhu | I noticed integration test failed for heat service was not setup correctly during devstack-gate installation | 09:14 |
| elynn__ | any link? | 09:14 |
| yanyanhu | and seems heat is not enabled as plugin in gate job template of some other services | 09:15 |
| yanyanhu | actually heat service was not installed | 09:15 |
| elynn__ | heat? | 09:15 |
| yanyanhu | yes | 09:15 |
| yanyanhu | do you know any recent change about it? | 09:15 |
| yanyanhu | before, we only need to add heat service to enabled service list | 09:16 |
| yanyanhu | then devstack-gate will set it up | 09:16 |
| yanyanhu | like this: ENABLED_SERVICES=heat,h-api,h-api-cfn,h-api-cw,h-eng, | 09:16 |
| elynn__ | https://review.openstack.org/#/c/361542/ | 09:16 |
| yanyanhu | ah | 09:16 |
| yanyanhu | got it | 09:16 |
| yanyanhu | export DEVSTACK_LOCAL_CONFIG+=$'\n'"enable_plugin heat git://git.openstack.org/openstack/heat" | 09:16 |
| yanyanhu | need this line | 09:16 |
| elynn__ | might need to use heat devstack plugin | 09:17 |
| yanyanhu | yes, seems so | 09:17 |
| yanyanhu | just a little concern once they change it back someday, we need to change our gate job again... | 09:18 |
| yanyanhu | sigh | 09:18 |
| yanyanhu | anyway, it only influences the integration test | 09:18 |
| elynn__ | devstack plugin is the future | 09:19 |
| elynn__ | I don't think they will change it back.. | 09:20 |
| yanyanhu | I see | 09:20 |
| yanyanhu | will propose change to gate | 09:20 |
| elynn__ | And I don't think you need to add "ENABLED_SERVICES=heat,h-api,h-api-cfn,h-api-cw,h-eng," | 09:20 |
| elynn__ | These service will set by https://github.com/openstack/heat/blob/master/devstack/settings | 09:21 |
| yanyanhu | ok | 09:21 |
| yanyanhu | just noticed that those services are added to list in heat's gate job | 09:22 |
| elynn__ | Is there any variables like "enable_heat yes" to enable heat on gate? | 09:23 |
| elynn__ | So that we don't need to follow the change. | 09:23 |
| yanyanhu | nope, didn't see such variables | 09:24 |
| elynn__ | export DEVSTACK_GATE_HEAT=1 ? | 09:25 |
| elynn__ | Seems most of the projects use "enable_plugin heat" | 09:25 |
| yanyanhu | you mean just adding this line? | 09:26 |
| yanyanhu | a little projects use export devstack_gate_heat=1 | 09:26 |
| elynn__ | I mean I saw this line but I don't know if it can work or not.... | 09:26 |
| elynn__ | "enable_plugin heat" is the safer way. | 09:27 |
| yanyanhu | but that was used together with "DEVSTACK_LOCAL_CONFIG+=$'\n'"enable_plugin heat git://git.openstack.org/openstack/heat"" | 09:27 |
| yanyanhu | I see | 09:27 |
| yanyanhu | may I should try enable_plugin heat first | 09:27 |
| yanyanhu | thanks, elynn__ :) | 09:28 |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: fields obj support cluster-policy-get2 https://review.openstack.org/409039 | 09:28 |
| *** elynn__ has quit IRC | 09:29 | |
| *** yanyanhu has quit IRC | 09:39 | |
| openstackgerrit | RUIJIE YUAN proposed openstack/senlin: engine service support cluster-policy-get2 https://review.openstack.org/409048 | 09:45 |
| *** zhurong has quit IRC | 09:46 | |
| *** EldonZhao has quit IRC | 10:14 | |
| *** guoshan has quit IRC | 10:24 | |
| *** chohoor has quit IRC | 10:53 | |
| fabian4 | hi, i have a Problem, when i use senlin cluster-policy-attach lb-policy appear ERROR senlin.drivers.openstack.sdk [req-cd565cb4-33f4-44ab-aa14-fe1decac8b03 8874cf3479ad4f92ade25715d2a0f591 79797a227f1445e0aa847b93026cc81e - - -] NotFoundException: Not Found, 404 Not Found | 11:37 |
| fabian4 | the other policy not appear error | 11:38 |
| *** guoshan has joined #senlin | 12:10 | |
| *** elynn__ has joined #senlin | 13:04 | |
| *** elynn__ has quit IRC | 13:40 | |
| Qiming | fabian4, have you installed lbaas service? | 13:41 |
| Qiming | senlin only support lbaasv2 api | 13:42 |
| openstackgerrit | Merged openstack/senlin: Versioned request object for receiver_nofity https://review.openstack.org/408990 | 13:59 |
| openstackgerrit | Merged openstack/senlin: Move 'parse_bool_param' to api common https://review.openstack.org/407515 | 14:18 |
| openstackgerrit | Merged openstack/senlin: Remove 'InvalidParameter' exception type https://review.openstack.org/407932 | 14:26 |
| *** guoshan has quit IRC | 15:26 | |
| *** catintheroof has joined #senlin | 15:54 | |
| *** Drago1 has joined #senlin | 16:24 | |
| *** Drago1 has quit IRC | 17:25 | |
| *** openstack has joined #senlin | 19:48 | |
| *** Drago1 has joined #senlin | 22:00 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!