Tuesday, 2015-08-18

*** lixinhui has quit IRC00:14
openstackgerritxu-haiwei proposed stackforge/senlin: Revise cluster-scale-in/out default value  https://review.openstack.org/21394400:27
openstackgerritxu-haiwei proposed stackforge/senlin: Fix ValueError when doing cluster_scale_in action  https://review.openstack.org/21394600:32
*** mathspanda_ has joined #senlin01:13
xuhaiweimorning, mathspanda_01:16
*** Yanyanhu has joined #senlin01:16
*** Yanyan has joined #senlin01:17
*** mathspanda_ has quit IRC01:20
*** Yanyanhu has quit IRC01:21
*** mathspanda has joined #senlin01:39
openstackgerritxu-haiwei proposed stackforge/python-senlinclient: Fix TypeError when doing cluster-scale-in  https://review.openstack.org/21396001:41
xuhaiweimoring, mathspanda01:42
mathspandamorning01:42
xuhaiweiJust noticed the bug you reported yesterday is not fixed yet, so I submitted a patch01:42
mathspandaok.01:43
openstackgerritYanyan Hu proposed stackforge/senlin: Check size limitation in cluster scale in/out action  https://review.openstack.org/21396402:05
*** ChrisSen has joined #senlin02:05
openstackgerritYanyan Hu proposed stackforge/senlin: Rework some interfaces in keystone_v3 driver  https://review.openstack.org/21361702:18
Yanyanhi, xuhaiwei02:22
Yanyanabout this patch https://review.openstack.org/#/c/213946/, I guess maybe the better way is check cluster size limitation before doing scale in/out action02:22
Yanyanthis kind of size check can prevent scaling in a cluster which has no nodes02:23
Yanyanjust proposed a patch for this issue: https://review.openstack.org/21396402:25
xuhaiweiYanyan, i am back02:27
xuhaiweiI was reading your patch02:28
Yanyanhi02:28
xuhaiweiI will think about it more02:28
Yanyanok, thanks02:28
Yanyanactually, this is a historical issue since the size check logic was added when we decided to redesign the cluster size related property and logic02:29
Yanyanbut before that, scale_in and scale_out action has been implemented. So I think we forgot to do the size limit check in these two actions02:30
xuhaiweiYanyan, I think we need to test this more, the source you have added (line 573) seems not meet any IF condition of the check method02:36
Yanyanok, let me check it02:36
Yanyanumm, I think it should match this line http://git.openstack.org/cgit/stackforge/senlin/tree/senlin/engine/actions/cluster_action.py#n356 if this scaling in operation will break the low limit of cluster size02:38
xuhaiweiI also thought it was bad, if the cluster contains only 2 nodes, while it is requested to scale-in 3. But after a little thinking, it seems ok to me, just scale-in all the nodes it has02:39
Yanyanno, haiwei, actually we have made a long discussion about this issue. Since we have a property named as 'best_effort'in scaling_in/out_policy, we will try to do the operation you described.02:41
xuhaiweiok, got you02:42
Yanyanbut if user define this property as False, we will prevent those scaling request which could break the size limit02:42
xuhaiweiyes, line356 meets the situation02:42
Yanyanso for 'bare' scale in/out operatoin without any policy, we just reject it :)02:42
mathspandahi, guys. please look at this paste: http://paste.openstack.org/show/419879/02:43
Yanyanok, checking it02:44
xuhaiweimathspanda, 'senlin webhook-list' ran well in my environment02:47
Yanyanumm, I saw credential is defined as 'sqlalchemy.Text' in DB model.02:48
mathspandado you create any webhooks?02:48
Yanyannot 'types.Dict'02:48
mathspandayes02:49
Yanyanlet me think which type is correct02:49
Yanyanhi, xuhaiwei, I remeber this cmnd also worked well for me when we first time implemented webhook support02:50
Yanyannot sure about reason, I guess we did some modification and forgot to test it02:50
xuhaiweiYanyan, mathspanda, I reproduced the bug02:51
Yanyannice :)02:52
xuhaiweiby creating a new webhook02:52
*** ChrisSen has quit IRC02:52
Yanyanah, I remeber it02:53
Yanyanqiming change the type from dict to text to ensure we can encrypt the entire 'credential' rather than just 'password' field of it02:54
Yanyanlet me find the patch02:54
Yanyanhttps://review.openstack.org/#/c/203391/02:55
Yanyanthis one02:55
Yanyanso, mathspanda, I think you're right02:55
*** elynn has joined #senlin02:56
mathspandaok. i will report a bug and submit a patch asap.02:58
Yanyanok, thanks a lot :)02:58
xuhaiweiYanyan, in this patch it says credential can be ommited02:59
Yanyanyes, I remeber we will use the requester's credential as the default one02:59
xuhaiweibut it is still a 'required' option now, should fix it too ?02:59
Yanyanoh?03:00
Yanyanlet me have a look03:00
Yanyanumm, let me see03:01
xuhaiweihttps://github.com/stackforge/python-senlinclient/blob/master/senlinclient/v1/shell.py#L32403:01
YanyanI think you're right03:02
Yanyansince the design about webhook changed several times, I think we may need more tests for it03:02
Yanyansince we still don't have functional test for it...03:03
xuhaiweihmm03:03
Yanyanin original design, user have to provide credential when creating a webhook03:03
Yanyanbut later we changed the design to allow using a default one when it is not provided explicitly03:04
*** mathspanda_ has joined #senlin03:04
xuhaiweiok, I will submit a patch for it03:05
Yanyanok03:06
Yanyanthanks03:06
YanyanI doubted that there could be other issues in the workflow of webhook triggering, hope can find them using practical tests if so03:07
*** mathspanda_ has quit IRC03:08
xuhaiweiYanyan, I am also testing the patch you have summitted for scale_in/out action03:09
*** ChrisSen has joined #senlin03:10
xuhaiweithe function works well03:10
Yanyanok :)03:11
xuhaiweijust one thing which I am a little worried, if the existing nodes number is smaller than 'count', nothing will be done to the cluster, in this case, user may got no information about it03:11
xuhaiweithey may think the command succeeded03:11
xuhaiweithey may think the command succeeded03:12
Yanyanthey can show the action use 'senlin action-show $ID'03:13
Yanyanand the failed reason is stored in reason field :)03:13
xuhaiweiok03:14
Yanyanactually, for those operations(e.g. cluster create, delete) which are handled using 'action', this is the standard way for user to check the status and result of their request03:14
openstackgerritxu-haiwei proposed stackforge/python-senlinclient: Credential option should not be 'required' when creating webhook  https://review.openstack.org/21398303:17
xuhaiweiyes, in senlin's design it should be the right way03:17
xuhaiweiI will go for lunch now, see you03:18
*** Zhenqi has joined #senlin03:18
Yanyansee U03:19
*** elynn has quit IRC03:20
*** elynn has joined #senlin03:23
*** Zhenqi has quit IRC03:26
openstackgerritYanyan Hu proposed stackforge/senlin: Treat return value of sdk function call as object  https://review.openstack.org/21398803:33
openstackgerritLinPeiyu proposed stackforge/python-senlinclient: Fix crendential type problem in webhook operations in client side  https://review.openstack.org/21399003:42
openstackgerritMerged stackforge/python-senlinclient: Fix crendential type problem in webhook operations in client side  https://review.openstack.org/21399004:10
mathspandaHi, guys. another paste: http://paste.openstack.org/show/419981/04:51
mathspandais my operation wrong?04:51
Yanyanhi, mathspanda, I think there is a bug here04:54
Yanyanthe authentication operation failed in sdk for some reasons04:54
Yanyanbut the keystone_v3 driver didn't handle it correctly04:54
Yanyanthe authentication failure should be caused by some bugs in webhook implementatiion04:55
Yanyanand the incorrect exception handling will be fixed in this patch https://review.openstack.org/21361704:56
mathspandaok. i try it.04:56
Yanyanlet try to figure out why the authentication failed, I guess it was caused by incorrect decryption of webhook credential04:57
*** lkarm has joined #senlin05:00
*** lkarm has quit IRC05:05
Yanyanhi, mathspanda, I think this is because the credential we fetched back from DB is text not dictionary05:06
YanyanI think we may need to do a conversion before sending it to keystone driver for authentication05:06
Yanyanlet me make a test05:08
Yanyanhi, mathspanda, I was wrong, I think the error was caused by incorrect credential format not the conversion problem05:14
mathspandayes. i think so. the user_id is out of the user in dict credential05:18
Yanyanyes, the key should be 'user'05:21
YanyanI think we need a fix in engine service05:22
Yanyanlet me have a try05:23
mathspandaok. thanks.05:24
Yanyanno problem. I also found the tenant_id was not embedded into webhook url correctly. I think there is another bug here.05:26
Yanyanah, I suddenly feel that the fix about removing 'dict' type restrict from webhook credential param might be incorrect05:34
Yanyansince we do expect an dictionary credential input in engine side05:34
Yanyanwe just dump it using json util before encrypting it05:35
mathspandai think meybe you are right.05:36
Yanyananyway, there are several bugs related in engine side05:36
Yanyanwill go through the code and find them all05:37
mathspandaadd oil05:39
Yanyan:)05:42
xuhaiweiYanyan05:47
Yanyanhi05:47
xuhaiweicurrently the first problem is token can't get from keystone?05:47
xuhaiweiit seems user id or name is required05:48
Yanyanyes, actually I think we should use 'user' rather than 'user_id' as field name05:48
Yanyanso I first try to correct the progress of webhook creation05:48
xuhaiweiyes, this maybe just a name's problem05:49
Yanyanbut I found even we fix this issue, sdk will complain the username or password is invalid and thus gave an 401 error. So I guess maybe the credential format we are using now is incorrect05:49
xuhaiweiI found the user_id is correct05:49
Yanyansince the credential needed for doing sdk authentication is different from the one using for creating sdk connection05:50
xuhaiweiok, I will make a test too05:50
Yanyanyes05:50
Yanyanhi, xuhaiwei, mathspanda, found the reason06:02
Yanyanthe credential building progress is incorrect when requester has admin role06:02
xuhaiweicant catch you06:03
Yanyanhttp://git.openstack.org/cgit/stackforge/senlin/tree/senlin/engine/service.py#n126706:03
Yanyanwe just need the trust_id here for 'trusts' field06:03
Yanyanwe shouldn't use the entire cred which stored in DB06:03
xuhaiweijust filter the trust_id from db?06:05
xuhaiweithe other creds are not needed?06:05
Yanyanyes, I guess so06:06
Yanyanwill try it06:06
Yanyanhmm, seems incorrect. I added 'trusts' and 'auth_url' field to credential which is used to talk with sdk, but it still complained invalid username or password06:19
YanyanI made a mistake, both user name and password are needed06:22
Yanyanand also the trusts06:22
Yanyanbut for the owner of target object, we actually didn't have its password, so I think user its user id may be incorrect here06:23
Yanyanmade a test and found this is the reason06:29
Yanyanwill try to figure out this issue06:29
xuhaiweiso if the password and username and user_id are all configured, it will pass?06:29
Yanyanhi, mathspanda, could you please help to file a bug report for this?06:30
Yanyanyes06:30
mathspandaok.06:30
YanyanI think some implementation might be incorrect in last change of webhook related code06:30
Yanyanwill try to correct it06:31
Yanyanhi, xuhaiwei, the error actaully happened in the section between line 1262 to 1275 http://git.openstack.org/cgit/stackforge/senlin/tree/senlin/engine/service.py#n126206:32
YanyanI think we should use senlin service user in both two cases06:33
xuhaiweithe user name should be 'service'?06:34
Yanyanshoud be 'senlin'06:34
Yanyanactually, each senlin talk with other services on behalf of requester, the following credential will be used:06:35
xuhaiweiBut after configured username, user_id, and password, I got this error: raise exceptions.EmptyCatalog('The service catalog is missing')06:35
Yanyan{'user_name': 'senlin', 'password': 'xxx', 'user_domain_name', 'xxx', 'trusts': 'xxx'}06:35
Yanyan{u'password': u'123456', u'user_domain_name': u'Default', 'trusts': u'488e8e6f331347168460375565270e8d', u'auth_url': u'http://9.186.107.212:5000/v3', 'project_id': u'1d567ed4ef51453a85545f018b68c26d', u'user_name': u'senlin'}06:36
Yanyanthis is an example from my env06:37
Yanyanhi, xuhaiwei, you can complete all those fields and try again06:38
xuhaiweiok, just do it by hand06:38
mathspandayes, i have also get the empty catalog exception when i assgin the crendential attibute in creating webhook operation06:40
Yanyanyou mean when you try to create the webhook with specified credential?06:43
mathspandayes06:43
*** openstackgerrit_ has joined #senlin06:43
Yanyancould this be caused by the removing of dict type restrict?06:43
mathspandacreation is successful, the exception happens after trigger the webhook url06:44
Yanyanhmm, this didn't happen in my env06:44
Yanyanlet me try06:44
mathspandathe same as xuhaiwei said before06:45
Yanyanok06:48
Yanyanhi, guys, can't reproduce this error in my local env... guess there might be some problems with my environment setting06:53
Yanyananyway, will try to fix the credential problem first06:53
Yanyanthen will work on this issue06:54
Yanyanmaybe I changed something manually but forgot to change it back...06:54
xuhaiweifinally worked for me06:54
Yanyanoh :)06:55
xuhaiweimathspanda, I configured "auth_url, password, user_name, user_domain_name, trusts " in the credentials06:56
xuhaiweiI think maybe you missed one or some of them06:56
mathspandaok. i miss the trusts06:56
Yanyannice, will try to propose a patch for this error :)06:57
Yanyanactually this is one of the most complicated part of senlin's authentication design...06:57
openstackgerritYanyan Hu proposed stackforge/senlin: Fix two bugs in webhook creation progress  https://review.openstack.org/21403807:03
openstackgerritYanyan Hu proposed stackforge/senlin: Fix two bugs in webhook creation progress  https://review.openstack.org/21403807:04
Yanyanhi, guys, the patch has been proposed, you can have a try again with it07:06
xuhaiweiYanyan, the patch works well07:14
Yanyannice :)07:14
xuhaiweiBut..07:14
Yanyanbut I forgot the test case part07:14
Yanyanwill add it07:14
xuhaiweinot that07:14
Yanyanok07:14
xuhaiweigot this error message in the engine log  ' Cluster cluster1 not found07:15
xuhaiwei'07:15
Yanyanhmm, I found this problem too07:15
YanyanI think some errors happened when try to find a cluster using short id07:15
mathspandayes.07:15
YanyanI found if you specify the full ID of cluster when creating webhook, this error will not happen07:16
xuhaiweithe url is still wrong?07:16
Yanyanyou mean there is still a '$(tenant_id)s' in it?07:16
xuhaiweino07:16
mathspandause short_id or name, the cluster will not found07:17
Yanyanok, mathspanda, will try to figure this out07:17
Yanyanhi, mathspanda, I think I omitted the conversion between short_id/name and full id of cluster07:22
Yanyanwill add it07:22
mathspandaok. add oil07:24
*** lkarm has joined #senlin07:27
*** lkarm has quit IRC07:32
openstackgerritYanyan Hu proposed stackforge/senlin: Fix three bugs in webhook related workflows  https://review.openstack.org/21403807:37
Yanyanhi, guys, new patch is coming, help try again :)07:39
xuhaiweiYanyan07:39
Yanyanhi07:39
xuhaiweiI found even use the full UUID of cluster, the webhook still can't be triggered07:40
Yanyanyou mean use full UUID when creating webhook?07:40
xuhaiweiyes07:40
xuhaiweithe cluster can be found, but action is not triggered07:41
xuhaiweimaybe another bug here :)07:41
Yanyanhmm, will check it07:41
mathspandait worked in my env.07:42
Yanyanlet me try it in my env07:42
xuhaiweireally mathspanda?07:43
xuhaiweiyou used which aciton?07:43
mathspandacurl the url07:43
mathspandaCLUSTER_SCALE_OUT07:44
xuhaiweiI am using the same action07:44
xuhaiweiwhy it doesn't work :(07:44
Yanyanhmm, also worked in my env...07:44
Yanyanhi, xuhaiwei, is there any error log in engine?07:44
xuhaiweino07:46
Yanyanso after send post request to senlin api, did you get an action_id as return value?07:47
xuhaiweiyes, got one07:48
xuhaiweiand the action status is also SUCCEEDED07:48
xuhaiweiok, let me jingjing07:48
Yanyan...07:49
YanyanI think you haven't proposed the patch which changed the default count from 0 to 107:49
Yanyansince we didn't provid input param for the webhook, therefore, default count will be used when action is executed :)07:50
*** jdandrea has quit IRC07:50
xuhaiweiI think you got the point07:50
Yanyanso the action is marked as succeeded, but actually no new nodes is created07:50
xuhaiweiwhile I am in another patch, count is still 007:50
Yanyan:)07:51
Yanyanyes07:51
xuhaiweithanks, worked now07:52
Yanyannice :)07:53
mathspandai think the webhooks.rst in getting_started should be revised. some parameters of examples are incorrect.07:57
xuhaiweiyes, mathspanda08:00
xuhaiweifeel free to modify it08:00
mathspandaok. :)08:01
mathspandai have another problem. when i assign a network list in spec file of a profile,08:53
mathspandathen i create a cluster in capacity 2 with this profile,08:54
mathspandaall of the fixed ips will be assigned to the first node08:54
mathspandais it right08:54
Yanyanhi, mathspanda, what you mean by 'fixed ip'?08:55
Yanyancan you paste out your spec file?08:56
mathspandaok.08:57
YanyanI guess if you specify the fixed ip in profile spec file, there could be some problems since each IP can only be assigned to one Nova instance I think08:57
Yanyanso the creation of the second VM could meet some issues08:58
mathspandayes08:59
mathspandahttp://paste.openstack.org/show/420188/08:59
mathspandaperhaps i misunderstand the usage09:01
Yanyanyes, maybe so since all nodes will use the same profile by default09:01
Yanyanunless new profile will be created and used for creation of new node09:02
Yanyanin some special cases like cross region cluster deployment09:02
mathspandaok. i understand.:)09:04
mathspandathanks.09:04
Yanyansimilar in Heat, if you define an autoscaling group, you should define a network rather than fixed IP in the child template of member09:04
Yanyan:)09:04
mathspandabut in senlin, i cannot specify a network. can i?09:05
xuhaiweiwill go home now09:05
xuhaiweisee you guys09:06
mathspandasee U:)09:06
Yanyansee U tomorrow09:06
Yanyanmathspanda, I think you can09:06
mathspandain where?09:06
xuhaiweiwe still have a meeting tonight?09:06
Yanyanjust use the same way09:06
Yanyanxuhaiwei, not sure...09:07
Yanyanbut I guess os since qiming didn't cancel it...09:07
xuhaiweiI think so, he may access to internet from someone's home maybe:)09:08
Yanyanhi, mathspanda, I haven't tested it before, but maybe just specify the 'net-id' rather than fixed-ip?09:08
Yanyanxuhaiwei, yes :)09:08
xuhaiweiso see you tonight09:08
Yanyansee U09:09
mathspandabut in schema of NETWORKS is a list of fixed-ip and port09:09
Yanyanoh, let me have a look09:10
Yanyanmathspanda, I think we need a change here09:13
Yanyanwill ask qiming about this since this profile type was implemented by him originally09:13
mathspandayes, i very need it.09:14
Yanyanoh, I remember he mentioned that he met some issues when trying to handle network option for nova instance creation09:14
Yanyanwill confirm with him09:14
Yanyanthanks :)09:14
Yanyanmaybe a bug can be file for this issue :)09:15
mathspandaok. i will file a bug soon.09:15
Yanyanthanks09:15
*** lkarm has joined #senlin09:32
mathspandago home.:)09:32
*** lkarm has quit IRC09:36
Yanyansee U :)09:38
*** mathspanda has quit IRC09:39
openstackgerritYanyan Hu proposed stackforge/senlin: [WIP]Add fake nova_v2 driver for functional test  https://review.openstack.org/21409009:39
*** ChrisSen has quit IRC09:56
*** Yanyan has quit IRC09:56
*** elynn has quit IRC10:51
*** lkarm has joined #senlin12:04
*** yanyanhu has joined #senlin12:29
yanyanhuhi, xuhaiwei, are you there?12:43
yanyanhuhi, guys, since qiming is in travel and may not be able to join the meeting tonight, me and xuhaiwei will try to chair it tonight12:53
yanyanhuhope everyone can give some update about your workitem, thanks :)12:54
yanyanhuok, let's move to meeting channel13:01
yanyanhuanyone else there?13:02
*** jroyal has joined #senlin13:08
*** yanyan has joined #senlin13:15
*** yanyanhu has quit IRC13:19
*** yanyan has left #senlin13:23
*** yanyanhu has joined #senlin13:24
*** yanyanhu has quit IRC13:27
*** jdandrea has joined #senlin14:20
*** branw has quit IRC14:59
*** branw has joined #senlin15:14
*** jroyal has quit IRC16:50
*** jroyal has joined #senlin17:10
*** jroyal has quit IRC17:15
*** jroyal has joined #senlin17:16
*** openstackgerrit_ has quit IRC18:11
*** lkarm has quit IRC19:43
*** lkarm has joined #senlin19:52
*** jroyal has quit IRC20:00
*** jroyal has joined #senlin20:01
*** jroyal has quit IRC20:27
*** jroyal has joined #senlin20:29
*** lkarm has quit IRC21:07
*** jroyal has quit IRC21:10
*** jroyal has joined #senlin21:29
*** jroyal has quit IRC21:33
openstackgerritMerged stackforge/python-senlinclient: Fix TypeError when doing cluster-scale-in  https://review.openstack.org/21396023:05
openstackgerritMerged stackforge/python-senlinclient: Credential option should not be 'required' when creating webhook  https://review.openstack.org/21398323:06

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!