Friday, 2017-04-21

*** zhurong has joined #openstack-mistral00:50
*** bobh has joined #openstack-mistral01:47
*** gongysh has joined #openstack-mistral02:22
*** mgershen has joined #openstack-mistral03:03
*** mgershen1 has quit IRC03:03
*** portdirect has quit IRC03:16
*** portdirect has joined #openstack-mistral03:23
*** zhurong has quit IRC03:43
*** bobh has quit IRC03:48
*** bobh has joined #openstack-mistral03:50
*** bobh has quit IRC03:54
*** gongysh has quit IRC04:14
*** gongysh has joined #openstack-mistral04:43
*** sharatss has joined #openstack-mistral04:44
*** gongysh has quit IRC05:09
*** apetrich has joined #openstack-mistral05:12
rakhmerovkong: ok05:16
*** apetrich has quit IRC05:30
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: Add pagination options for workflows and actions  https://review.openstack.org/45851505:32
*** apetrich has joined #openstack-mistral05:32
rakhmerovkong, ddeja, d0ugal: please https://review.openstack.org/#/c/454411/05:36
*** apetrich has quit IRC05:38
*** apetrich has joined #openstack-mistral05:50
*** gongysh has joined #openstack-mistral05:59
*** jaosorior has joined #openstack-mistral06:06
*** tung_doan has joined #openstack-mistral06:12
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: Add pagination options for workflows and actions  https://review.openstack.org/45851506:15
*** zhurong has joined #openstack-mistral06:21
*** zhurong has quit IRC06:39
*** zhurong has joined #openstack-mistral06:42
*** livelace has joined #openstack-mistral06:52
*** zhurong has quit IRC06:54
*** chlong has quit IRC06:56
jaosoriorsharatss: did you make any progress on the keystone-session bits?07:01
sharatssjaosorior, nope :(07:01
jaosoriorsharatss: ok, putting up an environment to reproduce it07:02
sharatssjaosorior, ok.. even i will be trying on that07:02
jaosoriorsharatss: I don't run into the api_key issues you were seeing07:05
jaosoriorsharatss: actually it's failing cause we're using a default domain even if v2 is used07:06
jaosoriorso, keystone v2.0 doesn't work with the current code07:06
sharatssjaosorior, does it work with v3? coz i'm using v307:08
jaosoriorsharatss: haven't tried yet.07:08
jaosoriorthe tripleo undercloud unfortunately uses v2 as of now07:08
jaosoriorsharatss: what's the error you're getting again?07:10
sharatssjaosorior, same.. the RuntimeError you have added07:12
jaosoriorsharatss: can you try printing the values available? It really sounds like you're not giving the password07:12
sharatssjaosorior, will try it07:12
jaosoriorsharatss: or verify that you have set the OS_PASSWORD environment file07:13
sharatssjaosorior, but even jenkins gives the same error07:13
openstackgerritMerged openstack/mistral master: Optimize the link address  https://review.openstack.org/45721507:15
jaosoriorsharatss: does it?07:15
jaosoriorsharatss: the error I'm seeing is the same I get locally http://logs.openstack.org/74/455174/6/check/gate-python-mistralclient-devstack-dsvm-ubuntu-xenial/29d2d4c/console.html#_2017-04-18_10_06_32_38763807:16
jaosoriorok, got it working with v207:18
*** gongysh has quit IRC07:18
sharatssjaosorior, oh great!!07:18
jaosoriorsharatss: the issue was the defaults given to the projects and domain names from the shell07:19
jaosoriorif you don't give defaults and let the user give all the values, then it works07:19
jaosoriorcause if you try using the generic plugins and use keystone v2, it was failing since it was trying to give domain names (which it shouldn't try)07:19
jaosoriornow lets try v307:19
sharatssjaosorior, is it better to use v2 and v3 separately rather than using generic for mistral?07:20
jaosoriorsharatss: so I'm using the generic one07:21
jaosoriorI just prefer adding the generic plugins, since we never know if keystone will be releasing a v4 at some point... so it gives folks an easier time if that happens07:22
sharatssjaosorior, only if they don't make it any more complex07:24
*** shardy has joined #openstack-mistral07:24
jaosoriorsharatss: I'm getting issues with v3. for some reason it's not getting the user domain name07:25
*** zhurong has joined #openstack-mistral07:31
jaosoriorlol user error07:33
jaosoriorforgot to default it07:33
jaosorior*export07:34
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517407:37
jaosoriorsharatss: so I tried this with both v3 and v2 ^^07:37
jaosoriorsharatss: now, I'm not entirely sure how to test the "target" bits07:38
sharatssjaosorior, let us see if this works07:38
sharatssthen may be we can follow the same pattern for the target and check07:39
*** jpich has joined #openstack-mistral07:43
openstackgerritluong tuan proposed openstack/mistral master: Refactor mistral context using oslo_context  https://review.openstack.org/45540707:49
*** apetrich has quit IRC08:01
*** apetrich has joined #openstack-mistral08:02
*** gongysh has joined #openstack-mistral08:09
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517408:10
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517408:15
*** zhurong has quit IRC08:15
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517408:21
*** zhurong has joined #openstack-mistral08:22
*** jpich has quit IRC08:29
*** jpich has joined #openstack-mistral08:30
*** shardy has quit IRC08:34
*** shardy has joined #openstack-mistral08:35
*** gongysh has quit IRC08:51
*** jrist has quit IRC08:57
*** tuan_ has joined #openstack-mistral09:02
tuan_Hi guys,09:05
tuan_could someone from tripleo explain to me about the job fail09:05
tuan_of tripleo gate below09:06
tuan_http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/console.html09:06
tuan_thanks in advanced09:06
*** tuan_ has quit IRC09:24
jaosoriortung_doan: might wanna ask in #tripleo09:25
*** tuan has joined #openstack-mistral09:26
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517409:26
jaosoriortuan: what patch is that?09:27
tuanjaosorior: hi, this patch: https://review.openstack.org/#/c/455407/09:27
*** zhurong has quit IRC09:33
*** zhurong has joined #openstack-mistral09:36
*** zhurong has quit IRC09:38
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: Switch from keystoneclient to keystoneauth  https://review.openstack.org/45765609:55
*** warface has joined #openstack-mistral09:58
*** tuan has quit IRC10:24
*** tuan_ has joined #openstack-mistral10:29
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517410:38
jaosoriorrakhmerov, sharatss so, I'm almost done fixing the unit tests there, but I got stuck with mocking the responses there ^^10:38
jaosoriorwait fixing pep8 issues10:41
*** tuan_ has quit IRC10:41
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517410:42
jaosoriorrakhmerov, sharatss: Alright, fixed pep8 there. Would use help fixing the unit tests. They were quite tied to using python-requests and auth-tokens directly10:42
*** jtomasek_ has joined #openstack-mistral10:43
*** tuan has joined #openstack-mistral10:44
*** sharatss has quit IRC11:01
*** sharatss has joined #openstack-mistral11:01
tuanjaosorior: Hi, do you have some ideas about the above problem11:06
jaosoriortuan: uhm... well, it seems that mistralclient is crashing but it's hiding the issue11:10
jaosoriortuan: so first one would need to fix mistralclient to show what the issue was, and then you'll know11:11
tuanjaosorior: well, it suggests to find another log file to see the root cause11:11
tuanhow can i know which log file11:11
jaosoriortuan: so for tripleo, from here http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/ tyou gotta go to the logs directory11:12
jaosoriortuan: from the ansible output you saw that it failed in the undercloud step, so then you can go to the undercloud directory11:13
jaosorioranyway, since it failed in the undercloud install, we go to the undercloud install logs, here http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/logs/undercloud/home/jenkins/undercloud_install.log.txt.gz#_2017-04-21_08_17_3511:13
jaosoriorand there the error is not visible, mistral just fails with "app"11:14
jaosoriorwhich means that the mistralclient is hiding the error11:14
jaosoriorI did search the mistral logs, and found nothing http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/logs/undercloud/var/log/mistral/11:14
jaosoriorso it might be a client issue11:14
tuanjaosorior: sorry for my bad in tripleo gate11:16
tuanbut when i look to the log11:16
tuani do nit see the error at undercloud11:16
jaosoriortuan: no biggie, it's honestly not the best log (it took me a while to get used to it)11:17
tuan:))11:17
jaosoriorok, lets see11:17
tuanokay11:17
jaosoriortuan: so this is the first log you see http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/console.html11:17
jaosorioransible shows a play recap when it's done, so look for the last in that log"PLAY RECAP"11:18
tuanjaosorior: yep, struggled with it since today morning11:18
tuan:D11:18
tuanyep11:18
jaosorior* look for the last "PLAY RECAP" in that log11:18
tuanahha11:18
tuanfailed=111:18
jaosoriorso just above it, you can see the failure11:18
jaosoriorhere http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/console.html#_2017-04-21_08_03_29_94006611:18
jaosoriorand here it says where the log is http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/console.html#_2017-04-21_08_17_39_21735511:19
jaosoriorso now we know two things: 1) the error was in the "Install the undercloud" step  2) where the logs are.11:20
tuanyep11:20
jaosoriornow, the logs for the undercloud are usually under logs/undercloud11:20
jaosoriorhere http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/logs/undercloud/11:20
jaosoriorso we follow the path that was pointed in the log home/jenkins/undercloud_install.log.txt11:20
jaosoriorwhich is here http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/logs/undercloud/home/jenkins/undercloud_install.log.txt.gz11:20
jaosoriornow, the undercloud install is not ansible driven. So those logs will look different11:21
jaosoriorhowever, the failure usually is also at the bottom of the file11:21
jaosoriorso, going to the bottom of that file, we can see this http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/logs/undercloud/home/jenkins/undercloud_install.log.txt.gz#_2017-04-21_08_17_3911:21
tuanyep11:22
tuani am there11:22
jaosoriortuan: and that's where the issue is11:23
jaosoriortuan: unfortunately, the issue is hidden by the mistralclient11:23
jaosoriorso all we see it ERROR (app)11:23
tuan"app" means mistralclient?11:24
jaosoriorso there is a wrong try: ... except there, or something is throwing an exception that's ignored by cliff11:24
jaosoriortuan: so just above that error, you can see the command that was ran11:24
tuanmistral workbook-list11:24
tuanthis run11:24
jaosoriorright11:24
tuanand then error at 'app'11:24
jaosorioroh11:24
jaosoriorawit a minute11:24
jaosoriorwhy the hell would it be failing on mistral workbook-list11:24
jaosoriorthat makes no sense11:25
tuanyep11:25
tuan(os-refresh-config) [ERROR] during post-configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/post-configure.d']' returned non-zero exit status 1]11:25
tuanand this one also11:25
flaper87kong: that information I got was using --debug, there's no actual error11:25
jaosoriorflaper87: ??11:25
tuanjaosorior: may be they are talking about another issue11:26
tuan:D11:26
tuanjaosorior: Reason: os-refresh-config failed. See log for details.11:27
flaper87jaosorior: hey, so, yday (still today) I was having problems creating a workflow in my local mistral-server process and no error was being returned by the API11:27
jaosoriorflaper87: what do you see?11:27
flaper87jaosorior: http://paste.openstack.org/show/607392/11:28
jaosoriortuan: os-refresh-config is just what's running the everything. The issue is a mistralclient failure11:28
flaper87jaosorior: that's server side11:28
flaper87I'm going to try the same on a fully deployed undercloud now11:28
jaosoriorflaper87: and the client side?11:28
flaper87jaosorior: http://paste.openstack.org/show/607457/11:29
jaosoriorflaper87: that's exactly the issue tuan is seeing11:29
tuanjaosorior: perfect11:30
tuan:D11:30
flaper87jaosorior: oh, I wasn't reading your convo w/ tuan11:30
flaper87I should have11:30
flaper87:D11:30
tuanflaper87: do you have more info about that error of client11:30
flaper87tuan: I don't :(11:31
flaper87I'm fairly new to mistral, tbh so I'll have to debug server side11:31
flaper87I'm trying this same workflow on another instance where I should be able to debug things better11:31
jaosoriorflaper87: I think you gotta debug the clientside though, to see what was the actual exception that's happening11:31
jaosoriorcause it's being hidden by the client11:31
tuanjaosorior: do we have any modifications in client recently11:31
tuan?11:32
jaosoriorso there hadn't been a release since January, and I think one happened recently11:32
flaper87jaosorior: oh, ok! I mean, the workflow doesn't exist in the server after this error is raised11:32
flaper87lemme check11:32
jaosoriorflaper87: do you have an environment where this is happening?11:32
*** shardy is now known as shardy_lunch11:32
flaper87jaosorior: it's my laptop but I11:33
jaosoriorflaper87: if you open a tmate session I can try to help11:33
flaper87but I'm trying to replicate it somewhere I can give you access11:33
flaper87jaosorior: so, the response is 500 and the content is empty11:35
jaosoriordafuq11:35
jaosoriorand mistralserver is not logging any 500?11:36
flaper87right?11:36
flaper87nope11:36
jaosoriorflaper87: is mistral running over httpd?11:36
flaper87no, it's a mistral-server --servers all11:36
jaosoriorflaper87: anything in the keystone logs?11:36
flaper87give me 2 mins to replicate this on a fully fledge undercloud11:36
flaper87if I can't, then it must be my local env11:37
jaosoriorflaper87: tuan is seeing it in the gate actually11:37
flaper87oh, fuck11:38
flaper87pardon my language11:38
tuanflaper87: no worries at all11:38
tuan:D11:38
tuanit seems that it fails when send a POST request11:39
jaosoriorflaper87: no worries, you fucking language is no biggie :P11:39
tuancan we check the mistral api for more info11:39
jaosoriortuan: nothing in the logs http://logs.openstack.org/07/455407/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/9748a37/logs/undercloud/var/log/mistral/api.log.txt.gz11:40
tuanyep11:40
tuanialso wanna say that11:40
tuani have looked at that11:40
tuanflaper87: what about your mistral-api?11:40
flaper87it worked on the undercloud11:41
flaper87in my undercloud, at least11:42
jaosoriorflaper87: how had you reproduced it initially?11:42
*** thrash|g0ne is now known as thrash11:43
jaosoriortuan: well, I rechecked your patch just to see if it was a transcient error11:44
jaosoriortuan: maybe some middleware is throwing the 500, which is why the error is not logged in the mistral logs11:45
flaper87jaosorior: I'm running mistral-server in my laptop w/o keystone11:45
flaper87jaosorior: http://paste.openstack.org/show/607459/11:46
flaper87that's my mistral.conf11:46
jaosoriorinteresting11:46
tuanflaper87: how does mistral work w/o keystone11:46
tuanmistral works with pecan11:47
tuanbut it does not mean that mistral will not need keystone11:47
tuanall the services in openstack need keystone to work11:47
flaper87tuan: so, I got this config from running the docker image that's in the repo. Then I decided to run it in my laptop because I wanted to debug some stuff11:48
flaper87tuan: not all the services, FWIW ;)11:48
tuanflaper87: yeah :D, i think you understand wht i mean11:48
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517411:49
tuanflaper87: docker or machine does not matter here11:49
flaper87tuan: I know, but it matters to understand where I got the confi file from :)11:50
tuanjaosorior: i do not think that "flaper87" he runs his env with my patch11:50
flaper87I didn't make this config file up, it was generated by the scripts used to run the docker image11:50
tuantherefore i do not think my patch has any affects here11:50
tuanflaper87: agree11:50
tuan:D11:50
jaosoriortuan: yeah, I think the issue is unrelated to your patch11:50
tuanjaosorior: can you guys reproduce it again with tripleo11:51
tuan?11:51
tuanor at least with vanilla mistral11:51
tuanif vanilla mistral works well, so i think the error belongs to tripleo11:52
tuani need to go out for lunch:)11:53
jaosoriortuan: so I did a deployment yesterday night and it's working with the latest mistralclient. So no I haven't been able to reproduce it :/11:54
jaosorioroh, I now got an update for the mistral package11:57
jaosoriorlets see11:57
jaosoriornope12:06
jaosoriorcouldn't reproduce it12:06
*** jaosorior is now known as jaosorior_away12:12
*** shardy_lunch is now known as shardy12:25
*** dmellado has quit IRC12:26
*** jtomasek_ has quit IRC12:34
*** sharatss has quit IRC12:38
*** dmellado has joined #openstack-mistral12:47
*** jrist has joined #openstack-mistral12:59
*** dprince has joined #openstack-mistral13:02
tuanjaosorior: i'm back, sorry for lunch time13:12
tuanwell. so it means that vanilla mistral does not have problem13:12
tuani suppose13:12
*** thrash is now known as thrash|biab13:14
*** rbrady has joined #openstack-mistral13:17
tuanflaper87: are you still there13:40
tuanif you do not mind, could you reproduce it by reconfigure the mistral.conf by using keystone13:40
tuani mean commenting out the auth_enable=False in [pecan]13:41
*** igormarnat has quit IRC13:42
*** igormarnat has joined #openstack-mistral13:42
tuanif you still can reproduce it so authenticating or not does not matter here13:42
tuanthen it may be the error of pecan itself13:43
*** chlong has joined #openstack-mistral13:44
*** shardy is now known as shardy_afk13:44
*** thrash|biab is now known as thrash13:49
*** catintheroof has joined #openstack-mistral13:59
*** jaosorior_away is now known as jaosorior_mtg14:05
*** catintheroof has quit IRC14:10
*** catintheroof has joined #openstack-mistral14:10
*** shardy_afk is now known as shardy14:17
*** warface has quit IRC14:29
*** warface has joined #openstack-mistral14:35
*** warface has quit IRC14:37
*** gongysh has joined #openstack-mistral14:41
*** fultonj has quit IRC14:44
*** chlong_ has joined #openstack-mistral14:54
*** chlong_ has quit IRC14:54
*** jaosorior_mtg is now known as jaosorior14:58
*** jaosorior has quit IRC14:59
*** tuan has quit IRC15:02
*** chlong has quit IRC15:13
*** livelace has quit IRC15:21
*** gongysh has quit IRC15:33
*** thrash is now known as thrash|biab15:39
*** dprince has quit IRC15:41
*** jpich has quit IRC16:10
*** bobh has joined #openstack-mistral16:25
*** jkilpatr has quit IRC16:28
*** jkilpatr_ has joined #openstack-mistral16:28
*** dprince has joined #openstack-mistral16:29
*** bobh has quit IRC16:49
*** shardy has quit IRC16:50
*** dmellado has quit IRC16:52
*** weshay has quit IRC16:54
*** weshay has joined #openstack-mistral17:01
*** dmellado has joined #openstack-mistral17:06
*** ig0r_ has quit IRC17:24
*** jtomasek has quit IRC17:26
*** thrash|biab is now known as thrash17:27
*** ig0r_ has joined #openstack-mistral17:28
*** dprince has quit IRC17:41
*** dprince has joined #openstack-mistral17:53
*** dprince has quit IRC17:53
*** dprince has joined #openstack-mistral17:53
*** livelace has joined #openstack-mistral18:29
*** bobh has joined #openstack-mistral18:34
*** bobh has quit IRC18:37
*** bobh has joined #openstack-mistral18:44
*** bobh has quit IRC19:02
*** fultonj has joined #openstack-mistral19:19
*** livelace has quit IRC19:25
*** bobh has joined #openstack-mistral19:51
*** ig0r_ has quit IRC19:55
*** bobh has quit IRC19:56
*** bobh has joined #openstack-mistral20:09
*** dprince has quit IRC20:29
*** bobh has quit IRC20:33
*** openstackgerrit has quit IRC20:48
*** bobh has joined #openstack-mistral20:51
*** bobh has quit IRC20:54
*** mgershen1 has joined #openstack-mistral21:20
*** mgershen has quit IRC21:22
*** bobh has joined #openstack-mistral22:02
*** bobh has quit IRC22:19
*** catintheroof has quit IRC22:46

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!