| *** Qiming has quit IRC | 00:00 | |
| *** toddjohn has joined #openstack-mistral | 00:57 | |
| *** cheneydc has joined #openstack-mistral | 00:59 | |
| *** toddjohn has quit IRC | 01:02 | |
| *** Qiming has joined #openstack-mistral | 01:12 | |
| *** vishwanathj has joined #openstack-mistral | 02:03 | |
| *** toddjohn has joined #openstack-mistral | 02:45 | |
| hparekh | kong, Yeah OK, sure | 02:46 |
|---|---|---|
| kong | hparekh :-) | 02:47 |
| *** toddjohn has quit IRC | 02:50 | |
| *** vishwanathj has quit IRC | 03:01 | |
| openstackgerrit | Merged openstack/mistral: Fail/Success/Pause transition message https://review.openstack.org/276625 | 03:27 |
| *** stevebaker has quit IRC | 03:27 | |
| *** stevebaker has joined #openstack-mistral | 03:29 | |
| rakhmerov | kong: ok | 03:55 |
| rakhmerov | kong: did you make a release? | 04:00 |
| rakhmerov | hparekh: can you please write release notes and docs for your new cool feature (command messages) ? | 04:01 |
| hparekh | rakhmerov, Yes, I am on it. :) | 04:01 |
| rakhmerov | we generally always need to keep in mind to write docs | 04:01 |
| rakhmerov | ok, great | 04:02 |
| kong | rakhmerov: not yet, are there other patches we need to merge before release? besides hparekh's note | 04:05 |
| kong | hparekh: when you have done the release not, please ping me, let's merge it asap | 04:12 |
| hparekh | kong, Ok sure | 04:12 |
| kong | I have 10 mins in office, but I will do that when I go back home. we still have time | 04:12 |
| kong | rakhmerov: are you checking? | 04:13 |
| rakhmerov | kong: nothing else from my side | 04:13 |
| kong | rakhmerov: ok | 04:13 |
| rakhmerov | let's just wait for notes | 04:13 |
| kong | yeah | 04:13 |
| rakhmerov | kong, hparekh: my suggestin is to not wait for docs though, it can be done separately after release | 04:15 |
| kong | rakhmerov: ok, sure, then i do the job now | 04:17 |
| kong | hparekh: take your time | 04:17 |
| kong | :-) | 04:17 |
| rakhmerov | kong: no-no, I mean docs only :) | 04:17 |
| rakhmerov | not notes | 04:17 |
| kong | hah | 04:17 |
| kong | sorry misunderstand you | 04:18 |
| rakhmerov | no problem | 04:18 |
| kong | hparekh: ok, go ahead, haha | 04:18 |
| openstackgerrit | hardik proposed openstack/mistral: Release notes for fail/pause/success transition message https://review.openstack.org/324199 | 04:19 |
| openstackgerrit | hardik proposed openstack/mistral: Release notes for fail/pause/success transition message https://review.openstack.org/324199 | 04:21 |
| hparekh | kong, Done | 04:23 |
| rakhmerov | hparekh: can you please fix it a little bit? | 04:23 |
| rakhmerov | I left comments | 04:23 |
| openstackgerrit | hardik proposed openstack/mistral: Release notes for fail/pause/success transition message https://review.openstack.org/324199 | 04:25 |
| rakhmerov | hparekh: Hardik, there's a gate failure in your patch for some reason | 04:37 |
| rakhmerov | http://logs.openstack.org/99/324199/3/check/gate-mistral-releasenotes/509b4aa/ | 04:37 |
| rakhmerov | hparekh: seems like it didn't like ":" in the description because it's a YAQL markup symbol | 04:38 |
| hparekh | rakhmerov, yeah | 04:39 |
| rakhmerov | I guess we just need to use quotes | 04:39 |
| hparekh | I am checking | 04:39 |
| rakhmerov | around "- fail .... %>" | 04:39 |
| hparekh | rakhmerov, Its not working | 04:54 |
| rakhmerov | quotes? | 04:54 |
| hparekh | yeah | 04:54 |
| rakhmerov | hm... | 04:55 |
| rakhmerov | weird | 04:55 |
| rakhmerov | you can also do something like: | 04:56 |
| rakhmerov | ooh, now | 04:57 |
| rakhmerov | no | 04:57 |
| rakhmerov | wait | 04:57 |
| rakhmerov | something like this should work but in case you don't have a list: | 04:58 |
| rakhmerov | text: > | 04:58 |
| rakhmerov | My text goes here | 04:58 |
| rakhmerov | I wonder if something like this will work: | 04:58 |
| rakhmerov | - > | 04:58 |
| rakhmerov | My text goes here | 04:58 |
| hparekh | let me try | 05:01 |
| rakhmerov | yeah, it works | 05:02 |
| rakhmerov | I made a simple program that does load from YAML like this and this is what I got: | 05:02 |
| rakhmerov | {'features': ['My text goes here with colons like : and dashes like -\n']} | 05:02 |
| hparekh | rakhmerov, my bab, actually tox was unable to get new changes | 05:08 |
| openstackgerrit | hardik proposed openstack/mistral: Release notes for fail/pause/success transition message https://review.openstack.org/324199 | 05:08 |
| hparekh | also I have to remove space before the "transition" | 05:12 |
| rakhmerov | hparekh: what about tox? Don't understand | 05:16 |
| *** jtomasek has joined #openstack-mistral | 05:16 | |
| hparekh | rakhmerov, If change is in my local machine (not committed) then while running tox it cannot get the change I made. I don't understand why ? | 05:20 |
| rakhmerov | hm.. dunno | 05:21 |
| openstackgerrit | hardik proposed openstack/mistral: Release notes for fail/pause/success transition message https://review.openstack.org/324199 | 05:39 |
| *** rbrady has quit IRC | 05:58 | |
| openstackgerrit | Merged openstack/mistral: Release notes for fail/pause/success transition message https://review.openstack.org/324199 | 06:17 |
| *** rbrady has joined #openstack-mistral | 06:38 | |
| openstackgerrit | Venkata Mahesh Jonnalagadda proposed openstack/mistral: mistral actions for designate v1 api's not working https://review.openstack.org/324247 | 06:55 |
| openstackgerrit | Dougal Matthews proposed openstack/mistral: Use LOG.exception when logging exceptions https://review.openstack.org/323308 | 07:10 |
| openstackgerrit | Dougal Matthews proposed openstack/mistral: Use LOG.exception when logging exceptions https://review.openstack.org/323308 | 07:13 |
| *** vishwanathj has joined #openstack-mistral | 07:15 | |
| *** vishwana_ has joined #openstack-mistral | 07:15 | |
| *** vishwanathj has quit IRC | 07:19 | |
| rakhmerov | d0ugal: hey, pep8 is failing for some reason | 07:25 |
| rakhmerov | maybe a line is longer than 80 | 07:25 |
| *** vishwana_ is now known as vishwanathj | 07:26 | |
| openstackgerrit | Dougal Matthews proposed openstack/mistral: Use LOG.exception when logging exceptions https://review.openstack.org/323308 | 07:28 |
| d0ugal | rakhmerov: Oops. Sorry about that! I don't see a pep8 error locally :/ I think that should fix it tho' ^ | 07:29 |
| rakhmerov | ok ) | 07:29 |
| *** vishwanathj has quit IRC | 07:34 | |
| *** vishwanathj has joined #openstack-mistral | 08:14 | |
| openstackgerrit | Venkata Mahesh Jonnalagadda proposed openstack/mistral: mistral actions for designate v1 api's not working https://review.openstack.org/324247 | 08:25 |
| openstackgerrit | Merged openstack/mistral: Use LOG.exception when logging exceptions https://review.openstack.org/323308 | 08:32 |
| ddeja | Hi rakhmerov. I was thinkig a lot about what we would like to introduce into oslo and came up to conclusion that having both 'at least once' and 'at most once' delivery models is not fixing all problems. | 08:37 |
| ddeja | I took a step back and rethink the original problem which is "I want to know if executor died during the action execution" | 08:38 |
| ddeja | and even if o.m. implements at least once delivery model, it doesn't solve of problems | 08:39 |
| ddeja | as an example, let's use nova_boot action | 08:40 |
| ddeja | it is obviously idempotent, so we can't use at least once delivery, couse we can end up with two VMs, instead of one | 08:41 |
| ddeja | so we have to use 'at most once' aaaaand we are in our todays situation | 08:42 |
| ddeja | I rethink how o.m. is now processing the message and from our perpective it looks like this: It's sending ACK and then it gives control to our method | 08:43 |
| ddeja | But what if it could be change to something like that: O.M is getting the message from the queue and it is passing it to our method together with ACK method pointer | 08:44 |
| ddeja | so it would be our method which would decide when it want's to send ACK | 08:45 |
| ddeja | it would give as ability to have a flow like this: | 08:45 |
| ddeja | 1. engine sends action to do onto queue | 08:45 |
| ddeja | 2. executor catches it and DO NOT send ACK at this time | 08:46 |
| ddeja | 3. executor sends message telling "This is executor ABC, I'm doing job XYZ" | 08:46 |
| ddeja | 4. Then executor sends ACK and start processing of a message; In the same time engine stores in the DB who is executinng action XYZ | 08:47 |
| ddeja | 5. After executor ends, it sends result to engine as usual | 08:47 |
| ddeja | what it would give to mistral? | 08:48 |
| ddeja | 1. If executor dies before sending ACK it means that it hasn't started doing any actual logic of a task, so it is safe to reschedule it | 08:48 |
| ddeja | 2. If executor dies after sending ACK it means that we have a knowledge what given executor was working on, so we can then decide if we want to fail workflow (task is not idempotent) or we want to reschedule it (task is idempotent) | 08:49 |
| ddeja | So, two question remains: If you think if such desing is a good idea? | 08:50 |
| ddeja | And, of course, if such change would pass be accepted by oslo people | 08:50 |
| ddeja | s/change would pass be accepted/would be accepted/ | 08:51 |
| ddeja | rakhmerov: I didn't send it to mailing list because I thought it may be good to first talk ofline with Joshua about this idea | 08:52 |
| ddeja | but of course I can send it anytime if you guys think it would be a better way | 08:53 |
| ddeja | Any comments? :) | 08:53 |
| rakhmerov | ddeja: hi | 09:13 |
| rakhmerov | ddeja: honestly, I think it's too complicated and won't solve the problem | 09:13 |
| rakhmerov | we already tried to come up with something like this but decided not to proceed with it because | 09:14 |
| rakhmerov | 1) Your step 3. Same problem here with receiving a message from executor to engine. Engine may not receive it so it won't know who is doing what | 09:15 |
| rakhmerov | 2) A number of network hops will be higher, load on a message bus too | 09:15 |
| rakhmerov | 3) Engine -> Executor is not the only problematic place because when we send a result back to engine the same may happen | 09:16 |
| rakhmerov | 4) Same with Engine -> Engine communications | 09:16 |
| rakhmerov | So essentially, by this design we are just moving this message delivery problem somewhere else | 09:17 |
| rakhmerov | but it doesn't disapper | 09:17 |
| rakhmerov | and it won't disappear for fundamental reasons, basically CAP theorem | 09:18 |
| rakhmerov | so I think we shouldn't complicate protocols between components and come up with ideas on how to recover from those failures | 09:18 |
| rakhmerov | as far as nova boot not being idempotent, it's not so huge problem. Oftentimes you can make an idempotent action out of non-idempotent by just implementing a new action which first checks if some resource, for example, exists | 09:20 |
| rakhmerov | we can make "nova.check_and_boot", for example | 09:20 |
| rakhmerov | so it will be idempotent and safe to run it multiple times | 09:20 |
| rakhmerov | now, getting to how we can account for recovery | 09:21 |
| rakhmerov | remember we discussed what I call "maintenance mode" | 09:21 |
| ddeja | yup | 09:22 |
| rakhmerov | so we could have that maintenance mode to be able to active it if we need to recover from failures, like a number of my executors died and I see that some of my workflows/tasks are stuck in the RUNNING state | 09:22 |
| rakhmerov | I should be able to repair them while being in that mode | 09:23 |
| rakhmerov | saying, for example, "OK, I know that this action actually needs to run again because I don't see a result in my environment (e.g. VM is not created)" | 09:23 |
| rakhmerov | or "OK, I know that this action has actually completed so I just need to mark it SUCCESS" and continue | 09:24 |
| rakhmerov | this is more about operational tooling | 09:24 |
| ddeja | OK | 09:26 |
| ddeja | but still I'm thining if the change to give a user ability to decide when he wants to send ACK may be more welcome in oslo... | 09:26 |
| ddeja | becouse with this we could achive 'at least once' without any problems | 09:27 |
| ddeja | but I gues I should just fire it to 'our' mistral + oslo thread on mailing list and see what happens | 09:29 |
| ddeja | rakhmerov: and btw, Executor->Engine and Engine->Engine communication can work in 'at least once' model, so we are sure that engine gets the message | 09:32 |
| ddeja | it may not solve all the problems, but it would make it less probable | 09:33 |
| ddeja | I agree with a higher load of message point :) | 09:33 |
| rakhmerov | ddeja: yes, sure | 09:47 |
| rakhmerov | ddeja: Dawid, I'm here mostly for flexibility | 09:48 |
| rakhmerov | I think we need to provide more configurability for users | 09:48 |
| rakhmerov | because all this things may heavily depend on use cases | 09:48 |
| rakhmerov | and, for example, if we could really configure delivery model per task it may be good too | 09:48 |
| *** toddjohn has joined #openstack-mistral | 09:48 | |
| rakhmerov | so my point here is: let's not make serious limitations on the design but add more flexibility to tune messaging for those who need to adapt Mistral for certain use cases | 09:50 |
| rakhmerov | and provide means for doing recovery work | 09:50 |
| d0ugal | http://docs.openstack.org/developer/pbr/#authors-and-changelog | 09:52 |
| d0ugal | Mistral maintains it's own AUTHORS file, wouldn't pbr be easier? | 09:52 |
| d0ugal | (Since pbr is used anyway) | 09:53 |
| ddeja | rakhmerov: OK, thanks. So I'm going to write an email about letting user to decide when he/she wants to send ACK and I'm getting back to work on mistral RPC layer :) | 09:54 |
| *** shadow1234 has quit IRC | 09:54 | |
| rakhmerov | d0ugal: ok, will look at it, I wasn't even aware shouldn't be doing it | 09:55 |
| rakhmerov | well, usually it's autogenerated by pbr anyway, I guess | 09:55 |
| d0ugal | ah | 09:55 |
| rakhmerov | we just maintain .mailmap | 09:55 |
| d0ugal | I wasn't sure, I just spotted AUTHORS changes in 323697 which seemed unrelated to the change | 09:55 |
| rakhmerov | yeah, it gets updated upon any usage of pbr | 09:56 |
| rakhmerov | so sometimes it seems like it's done manually | 09:56 |
| rakhmerov | but it's not | 09:56 |
| rakhmerov | ddeja: yes, thanks man | 09:56 |
| d0ugal | I see, makes sense. I think it is just normall ignored. i.e. https://github.com/openstack/nova/blob/master/.gitignore#L20 | 09:57 |
| *** vishwanathj has quit IRC | 09:57 | |
| *** Qiming has quit IRC | 09:59 | |
| *** vishwanathj has joined #openstack-mistral | 10:07 | |
| *** cheneydc has quit IRC | 10:07 | |
| *** vishwanathj has quit IRC | 10:21 | |
| *** Qiming has joined #openstack-mistral | 10:56 | |
| *** toddjohn has quit IRC | 11:00 | |
| *** toddjohn has joined #openstack-mistral | 11:20 | |
| *** toddjohn has quit IRC | 11:44 | |
| openstackgerrit | Venkata Mahesh Jonnalagadda proposed openstack/mistral: mistral actions for designate v1 api's not working https://review.openstack.org/324247 | 11:58 |
| *** rbrady has quit IRC | 11:59 | |
| *** rbrady has joined #openstack-mistral | 12:00 | |
| *** dprince has joined #openstack-mistral | 12:02 | |
| *** bobh has joined #openstack-mistral | 12:36 | |
| *** bobh has quit IRC | 12:41 | |
| openstackgerrit | Venkata Mahesh Jonnalagadda proposed openstack/mistral: mistral actions for designate v1 api's not working https://review.openstack.org/324247 | 13:07 |
| openstackgerrit | Renat Akhmerov proposed openstack/mistral: Refactoring workflow handler https://review.openstack.org/323697 | 13:24 |
| openstackgerrit | Renat Akhmerov proposed openstack/mistral: Refactoring workflow handler https://review.openstack.org/323697 | 13:32 |
| openstackgerrit | Jeff Peeler proposed openstack/mistral: Add missing argument in exception string https://review.openstack.org/324120 | 13:35 |
| *** tonytan4ever has joined #openstack-mistral | 13:41 | |
| *** venkat has joined #openstack-mistral | 13:56 | |
| venkat | Hi, I am seeing jenkins job gate-mistral-python34 failed with "mistral.exceptions.DBEntityNotFoundException: Task execution not found [id=c9f316e3-67e5-4e66-bbf5-af30a74cde00]" | 13:57 |
| venkat | any clue here.. | 13:57 |
| venkat | http://logs.openstack.org/47/324247/4/check/gate-mistral-python34/0473098/console.html | 13:57 |
| *** bobh has joined #openstack-mistral | 14:13 | |
| *** venkat has quit IRC | 14:32 | |
| mgershen | mflobo: venkat error reminds me of the one you had (this was your summary: http://paste.openstack.org/show/497848/). Did you came to a conclusion about that issue? | 14:50 |
| *** rena9067 has joined #openstack-mistral | 14:52 | |
| mflobo | mgershen, not yet. Cloning master branch it works | 15:08 |
| mflobo | I can reproduce the error using the RPM http://cbs.centos.org/koji/buildinfo?buildID=10492, which comes com tag 2.0.0 in githhub | 15:09 |
| *** toddjohn has joined #openstack-mistral | 15:16 | |
| mgershen | mflobo: I wonder if there is a connection. Anyway, good luck to you :) | 15:19 |
| *** gyee has joined #openstack-mistral | 16:08 | |
| vgnbkr | Hi. Can anyone point me to a doc describing "workflow_input" and "params" for cron-trigger-create? Otherwise, can someone tell me what the difference is? Or, point me to the code where they are used. Thx. | 16:10 |
| *** Qiming has quit IRC | 16:11 | |
| *** toddjohn has quit IRC | 16:23 | |
| mgershen | vgnbkr: are you familiar with creating executions in mistral? execution-create also has these arguments. | 16:48 |
| vgnbkr | I have created executions with execution-create, but just specified workflow_input. What is "params" for? | 16:50 |
| vgnbkr | mgershen, ^^ | 16:50 |
| vgnbkr | If you happen to have a workflow that uses them that you could share with me, that might do the trick. | 16:51 |
| mgershen | vgnbkr: It is not mandatory. I mostly use it for defining things that are environment related. You can put values in the params json file. One of them can be "env". "env" can be accessed from the workflow using a yaql expression like so: <% env() %> | 16:53 |
| vgnbkr | mgershen, hmm... if you happened to have an example you could share, I'm pretty sure that would do the trick. | 16:55 |
| mgershen | vgnbkr: I happened to open a bug with an example and it is fixed now, so you can use it: https://bugs.launchpad.net/mistral/+bug/1568909 | 16:57 |
| openstack | Launchpad bug 1568909 in Mistral "execution of ad-hoc actions based on std.mistral_http always fails" [Medium,Fix released] - Assigned to Renat Akhmerov (rakhmerov) | 16:57 |
| mgershen | vgnbkr: hope this helps, I have to go now | 17:03 |
| vgnbkr | Definitely helps, a little ways to go, though :-) Thanks. | 17:03 |
| vgnbkr | mgershen, ^^ | 17:04 |
| *** toddjohn has joined #openstack-mistral | 17:11 | |
| *** toddjohn has quit IRC | 17:16 | |
| *** tonytan4ever has quit IRC | 17:17 | |
| *** gyee has quit IRC | 17:40 | |
| *** tonytan4ever has joined #openstack-mistral | 17:42 | |
| *** toddjohn has joined #openstack-mistral | 17:42 | |
| *** toddjohn has quit IRC | 17:46 | |
| *** toddjohn has joined #openstack-mistral | 18:17 | |
| *** harlowja has quit IRC | 18:44 | |
| *** openstack has joined #openstack-mistral | 19:41 | |
| *** dprince has quit IRC | 20:08 | |
| *** harlowja has joined #openstack-mistral | 20:24 | |
| *** gyee has joined #openstack-mistral | 20:31 | |
| *** tonytan4ever has quit IRC | 20:41 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/mistral-extra: Updated from global requirements https://review.openstack.org/324835 | 21:06 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-mistralclient: Updated from global requirements https://review.openstack.org/324858 | 21:11 |
| *** toddjohn has quit IRC | 21:24 | |
| jpeeler | rbrady: i'm sure you're gone for the day, but feel free to comment on https://review.openstack.org/#/c/324894 | 21:54 |
| openstackgerrit | Merged openstack/mistral-extra: Updated from global requirements https://review.openstack.org/324835 | 21:57 |
| openstackgerrit | Jeff Peeler proposed openstack/mistral: Add missing argument in exception string https://review.openstack.org/324120 | 22:03 |
| kong | rakhmerov: how about we remove the requirement.txt and test-requirment.txt from mistral-extra repo, it is annoying when we approve the update patches every time. | 22:12 |
| *** openstackgerrit has quit IRC | 22:19 | |
| *** openstackgerrit has joined #openstack-mistral | 22:20 | |
| *** bobh has quit IRC | 22:22 | |
| openstackgerrit | Lingxian Kong proposed openstack/mistral: Remove AUTHORS file from git tracking https://review.openstack.org/324905 | 22:28 |
| openstackgerrit | Merged openstack/mistral: Updated from global requirements https://review.openstack.org/324834 | 22:30 |
| *** tonytan4ever has joined #openstack-mistral | 23:06 | |
| *** Qiming has joined #openstack-mistral | 23:09 | |
| *** Qiming has quit IRC | 23:40 | |
| *** vishwanathj has joined #openstack-mistral | 23:49 | |
| *** rena9067 has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!