Friday, 2017-05-12

« Thursday, 2017-05-11 Index Saturday, 2017-05-13 »
*** jkilpatr has quit IRC00:19
*** mbrennan has joined #openstack-mistral00:26
*** mbrennan has quit IRC00:30
*** jkilpatr has joined #openstack-mistral00:57
*** jkilpatr has quit IRC01:17
*** catintheroof has quit IRC01:18
*** catintheroof has joined #openstack-mistral02:40
*** catintheroof has quit IRC03:57
openstackgerritAdriano Petrich proposed openstack/mistral master: Adding log to db_sync  https://review.openstack.org/46401004:51
*** jaosorior_away has quit IRC04:58
*** jaosorior has joined #openstack-mistral05:41
*** sharat has joined #openstack-mistral06:46
*** jamielennox is now known as jamielennox|away06:53
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517407:01
*** mbrennan has joined #openstack-mistral07:01
*** mbrennan has quit IRC07:06
*** livelace has joined #openstack-mistral07:34
*** jpich has joined #openstack-mistral07:37
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: [WIP] Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517408:20
openstackgerritRenat Akhmerov proposed openstack/mistral master: Add hide_args=True to @profiler.trace() where it may cause problems  https://review.openstack.org/46415008:26
openstackgerritRenat Akhmerov proposed openstack/mistral master: Add 'evaluate_env' workflow parameter  https://review.openstack.org/46415808:56
*** d0ugal has quit IRC09:08
openstackgerritRenat Akhmerov proposed openstack/mistral master: Release notes for "evaluate_env"  https://review.openstack.org/46416009:11
openstackgerritRenat Akhmerov proposed openstack/mistral master: Add 'evaluate_env' workflow parameter  https://review.openstack.org/46415809:13
openstackgerritRenat Akhmerov proposed openstack/mistral master: Release notes for "evaluate_env"  https://review.openstack.org/46416009:13
*** d0ugal has joined #openstack-mistral09:16
*** d0ugal has quit IRC09:25
*** mgershen has joined #openstack-mistral09:25
xavierhardyrakhmerov: we have trouble stopping workflow and action executions09:26
xavierhardyPUT /v2/action_executions HTTP/1.1 status: 50009:26
xavierhardyWe get a MessagingTimeout09:27
xavierhardyHow do you stop stale executions in the DB when the whole backend has been restarted09:27
xavierhardy?09:27
*** mgershen1 has quit IRC09:28
rakhmerovxavierhardy: hi09:28
xavierhardyhi09:28
rakhmerovwhat do you pass to /v2/action_executions ?09:29
rakhmerovdo you pass 'state'?09:29
xavierhardyyes09:29
rakhmerovok09:29
rakhmerovwell, you seem to be doing it right then09:29
rakhmerovI need to see logs09:29
xavierhardybut it times out with MessagingTimeout09:30
rakhmerovyeah, I see09:30
xavierhardyid: UUID09:30
xavierhardystate: "ERROR"09:30
rakhmerovcan you file a bug with steps to reproduce and info from logs (engine, executor, api)?09:30
xavierhardyI will, but I have got tons of executions of stop first09:31
rakhmerovare you doing it via CLI or sending HTTP reqs directly to the endpoint?09:31
xavierhardy*to stop first09:31
xavierhardyHTTP09:31
rakhmerovtry CLI, there may be some difference or missing parameters that I don't remember now09:31
xavierhardywell, it should not timeout and it should not send back 500, so there's a problem in the API in any case.09:34
rakhmerovyou can do it like: mistral execution-update -s CANCELLED <id>09:34
xavierhardyCANCELLED ?09:34
rakhmerovor ERROR09:34
rakhmerovyes09:34
rakhmerovor SUCCESS09:35
rakhmerovwhatever you prefer09:35
xavierhardythere's a CANCELLED state ?09:35
rakhmerovyes09:35
rakhmerovso, as far as timeout yes, it's not good09:35
xavierhardyI can't pause, put it in success, put it in error, nor delete it09:36
rakhmerovwe need to see logs, otherwise it's hard to say anything certain09:36
rakhmerovwhat version of Mistral are you using?09:36
xavierhardyMistralException: MessagingTimeout: Timed out waiting for a reply to message ID d63a35427a9d44c19ce1a03a68a733cb09:37
xavierhardy2017-05-12 09:23:35,612 INFO [-] xxx.xxx.xxx.xxx "PUT  /v2/action_executions HTTP/1.1" status: 500  len: 283 time: 60.047998009:37
xavierhardy017-05-12 09:23:35,611 ERROR [-] Server-side error: "MessagingTimeout: Timed out waiting for a reply to message ID 1747586e532645e0b44ffc40528ae97b". Detail:09:38
xavierhardyTraceback (most recent call last):09:38
xavierhardyFile "/usr/local/lib/python2.7/dist-packages/wsmeext/pecan.py", line 85, in callfunction09:38
xavierhardyresult = f(self, *args, **kwargs)09:38
xavierhardyFile "/usr/local/lib/python2.7/dist-packages/mistral/api/controllers/v2/action_execution.py", line 176, in put09:38
xavierhardyFile "/usr/local/lib/python2.7/dist-packages/mistral/engine/rpc_backend/rpc.py", line 319, in decorator09:38
xavierhardy_wrap_exception_and_reraise(e)09:38
xavierhardyFile "/usr/local/lib/python2.7/dist-packages/mistral/engine/rpc_backend/rpc.py", line 294, in _wrap_exception_and_reraise09:38
xavierhardyraise exc.MistralException(message)09:38
xavierhardynothing more09:38
rakhmerovhm, nothing else in engine or executor log too?09:39
rakhmerovthere should be something..09:39
rakhmerovor your RabbitMQ simply went down09:40
xavierhardyas I said, we rebooted the whole thing09:40
xavierhardyso yes, RabbitMQ went down, but I still need a way to clean the whole thing up09:41
xavierhardywe get the same error when trying to create a new execution09:42
rakhmerovok, I see09:45
rakhmerovwhat Mistral version?09:45
xavierhardy9b39d7bc62209:48
*** d0ugal has joined #openstack-mistral09:48
rakhmerovNov 14, 201609:51
rakhmerovearly Ocata09:51
rakhmerovxavierhardy: can you check RabbitMQ? does it have lots of messages in their queues?09:52
xavierhardywait, the engine does not start09:53
xavierhardythat might be part of the problem09:53
rakhmerovok09:54
rakhmerovxavierhardy: I'll be here for about an hour, will try to help you09:56
rakhmerovthen I'll be online late evening09:56
xavierhardyI'll try fixing the engine first (Babel version incompatibility, nothing fancy here, I already fixed on one of my development branches)09:57
rakhmerovok09:59
*** sharat has quit IRC10:03
*** sharat has joined #openstack-mistral10:03
xavierhardyOK, it fixed my problem10:27
xavierhardyengine => not possible to create/modify executions10:27
rakhmerovok10:29
rakhmerovd0ugal: hi10:30
rakhmerovdo you know if there's any problems with TripleO gate now?10:30
rakhmerovone of my latest patches failed on it (https://review.openstack.org/#/c/464158) but the failure doesn't seem to be related with the changes10:31
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: [WIP] Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517410:40
therverakhmerov, I think it's a intermittent mirror issue10:42
rakhmerovtherve: ok10:42
d0ugalthanks therve :)10:46
therveNo problem :)10:48
*** jkilpatr has joined #openstack-mistral11:12
*** chlong has quit IRC11:36
*** mgershen1 has joined #openstack-mistral11:53
*** mgershen has quit IRC11:55
*** jamielennox|away is now known as jamielennox12:30
openstackgerritAdriano Petrich proposed openstack/mistral master: WIP adding mistral_lib to mistral  https://review.openstack.org/46418612:34
*** sharat has quit IRC12:35
*** mbrennan has joined #openstack-mistral13:13
*** mbrennan has quit IRC13:35
*** mbrennan has joined #openstack-mistral13:38
*** mbrennan has quit IRC13:47
*** mbrennan has joined #openstack-mistral14:02
*** chlong has joined #openstack-mistral14:09
*** jistr is now known as jistr|afk14:44
openstackgerritluong tuan proposed openstack/mistral master: Add user creds trusts to db  https://review.openstack.org/46205614:47
*** livelace has quit IRC14:53
*** dprince has joined #openstack-mistral15:11
xavierhardyIt's working fine now, thanks rakhmerov15:19
xavierhardybye15:19
*** xavierhardy has left #openstack-mistral15:19
*** jpich has quit IRC15:44
openstackgerritluong tuan proposed openstack/mistral master: Add user creds trusts to db  https://review.openstack.org/46205615:49
*** jistr|afk is now known as jistr16:06
*** jtomasek has quit IRC16:16
*** jtomasek has joined #openstack-mistral16:16
*** mbrennan has quit IRC16:29
*** zhenguo has quit IRC16:46
*** jrist is now known as jrist-afk16:51
*** jaosorior has quit IRC17:09
*** jrist-afk is now known as jrist17:12
*** jkilpatr has quit IRC17:26
*** jkilpatr has joined #openstack-mistral17:28
*** mbrennan has joined #openstack-mistral17:40
*** mbrennan has quit IRC18:22
*** mbrennan has joined #openstack-mistral18:40
*** jkilpatr has quit IRC18:50
*** jkilpatr has joined #openstack-mistral18:57
*** abishop has joined #openstack-mistral18:59
*** harlowja has quit IRC19:02
*** mgershen1 has quit IRC19:26
*** dprince has quit IRC19:51
*** abishop has quit IRC20:07
*** rbrady has joined #openstack-mistral20:29
*** jkilpatr has quit IRC21:04
*** toure is now known as toure|gone22:09
*** harlowja has joined #openstack-mistral22:40
*** chlong has quit IRC23:00
*** fultonj has quit IRC23:20
*** mbrennan has quit IRC23:20
« Thursday, 2017-05-11 Index Saturday, 2017-05-13 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!