Wednesday, 2019-09-04

*** altlogbot_1 has quit IRC00:23
*** irclogbot_1 has quit IRC00:23
*** threestrands has quit IRC00:32
*** irclogbot_3 has joined #openstack-mistral01:34
*** irclogbot_3 has quit IRC01:39
*** irclogbot_1 has joined #openstack-mistral01:40
*** irclogbot_1 has quit IRC01:43
*** apetrich has quit IRC02:10
*** irclogbot_0 has joined #openstack-mistral02:54
*** irclogbot_0 has quit IRC02:59
*** irclogbot_2 has joined #openstack-mistral03:34
*** irclogbot_2 has quit IRC03:39
openstackgerritRenat Akhmerov proposed openstack/mistral master: Add "published_global" field to the task execution REST resource  https://review.opendev.org/67853003:56
openstackgerritRenat Akhmerov proposed openstack/mistral master: Fix 'with-items' expression evaluation  https://review.opendev.org/67920303:57
*** gkadam has joined #openstack-mistral03:59
*** gkadam_ has joined #openstack-mistral04:01
*** gkadam_ has quit IRC04:01
*** irclogbot_1 has joined #openstack-mistral04:02
*** gkadam has quit IRC04:03
*** irclogbot_1 has quit IRC04:07
*** ricolin_ has joined #openstack-mistral04:08
*** ricolin_ has quit IRC04:08
*** ricolin has joined #openstack-mistral04:10
*** irclogbot_2 has joined #openstack-mistral04:22
*** irclogbot_2 has quit IRC04:33
*** eyalb has joined #openstack-mistral04:50
openstackgerritAndras Kovi proposed openstack/mistral master: Improve workflow notifications and webhook data  https://review.opendev.org/67979505:20
*** irclogbot_3 has joined #openstack-mistral05:20
*** irclogbot_3 has quit IRC05:25
*** jtomasek has joined #openstack-mistral06:06
*** ricolin_ has joined #openstack-mistral06:15
*** irclogbot_0 has joined #openstack-mistral06:16
*** ricolin has quit IRC06:17
*** irclogbot_0 has quit IRC06:21
*** irclogbot_0 has joined #openstack-mistral06:24
*** irclogbot_0 has quit IRC06:29
*** pgaxatte has joined #openstack-mistral06:51
*** apetrich has joined #openstack-mistral07:23
*** irclogbot_1 has joined #openstack-mistral07:30
*** irclogbot_1 has quit IRC07:35
*** cervigni has joined #openstack-mistral07:36
*** abdelal has joined #openstack-mistral07:47
*** abdelal has quit IRC07:48
*** irclogbot_1 has joined #openstack-mistral07:51
*** irclogbot_1 has quit IRC07:53
*** abdelal has joined #openstack-mistral07:58
rakhmerovvgvoleg: hi :)08:02
rakhmerovhi everyone08:02
rakhmerovmeeting?08:02
vgvolegyes08:04
vgvoleghi08:04
vgvolegI'd like to discuss 2 blueprints08:04
rakhmerov#startmeeting Mistral08:06
openstackMeeting started Wed Sep  4 08:06:19 2019 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.08:06
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:06
*** openstack changes topic to " (Meeting topic: Mistral)"08:06
rakhmerovvgvoleg: ok08:06
openstackThe meeting name has been set to 'mistral'08:06
rakhmerovgo ahead08:06
rakhmeroveyalb, abdelal, apetrich: ^08:06
vgvolegFirst of all, this one https://blueprints.launchpad.net/mistral/+spec/mistral-task-skipping-feature08:07
apetricho/08:07
eyalbo/08:07
vgvolegIt's already done in our fork, but I didn't see any reaction here :(08:07
abdelalo/08:07
rakhmerovvgvoleg: ok, let me read (again)08:07
vgvolegI've done changes in cloudflow too to support this08:08
vgvolegJust tell me that it would be useful and I'll push it :)08:09
vgvoleghttps://sun9-12.userapi.com/c854220/v854220580/df9c0/j5PU_qZKAi8.jpg08:10
vgvolegsomething like this08:10
abdelala question regarding that , if t2 published x1 , how will you pass it to t3 since t2 is skipped ?08:11
vgvolegskipped task also has publish section08:12
vgvolegit works the same way08:12
abdelalit will publish even if it was skipped ?08:12
vgvolegoh wait08:12
vgvolegMaybe I understand you wrong08:12
vgvolegthere is 'publish-on-skip' section08:13
rakhmerovno-no, I think we can't publish anything using the regular "publish" if the state is not SUCCESS08:13
vgvolegwhere you can, for example, mock data08:13
rakhmerovvgvoleg: are you aware of different syntax that you can use to publish vars?08:13
vgvolegIt is OK with publish-on-error08:14
rakhmerovyou can have publish under "on-success", "on-error", etc.08:14
vgvolegI don't see any differences with publish-on-skip08:14
rakhmerovvgvoleg: I think we shouldn't mix these things08:14
vgvolegwe didn't mix them08:15
vgvolegit is another publish section08:15
rakhmerovdifferent states => different language key words08:15
vgvolegif task is skipped, no 'publish' will be published08:15
vgvoleg:)08:15
rakhmerovvgvoleg: that's right08:15
rakhmerovvgvoleg: regular "publish" and "publish-on-error" will be deprecated I think soon08:16
rakhmerovwe'll be encouraging people to use "publish" under "on-..."08:16
rakhmerovwhere you can define scopes (currently "branch" and "global")08:16
vgvolegthe only thing I'm not sure about is what we should do if task don't have 'on-skip' branch08:16
rakhmerovI'm thinking may be we shouldn't even introduce this "publish-on-skip"08:17
rakhmerovvgvoleg: so, it's still a bit confusing to me..08:17
vgvolegIn our fork, if we skip a task with no 'on-skip' section, we use 'on-success'08:17
rakhmerovvgvoleg: no, I disagree with this approach08:18
rakhmerovit should be a totally separate things08:18
rakhmerovthing08:18
rakhmerovvgvoleg: can you express with one phrase why this functionality is needed? :)08:18
rakhmerovthis whole thing08:18
rakhmerovI'm still struggling with the idea I guess08:19
rakhmerovapetrich, eyalb, abdelal: may be you have some thoughts08:19
rakhmerovvgvoleg: so we do skipping if what?08:19
rakhmerovon an external failure?08:20
vgvolegIf the flow execution is too long, it will be great to have an opportunity to skip a failed task in the tail of the flow08:20
apetrichsorry, I'm trying to understand it too.08:20
vgvolegif there is an external failure08:20
vgvolegso retry will not help08:20
vgvolegSo it is all about a manual control of the flow08:21
vgvoleglike rerun, cancel, pause08:21
openstackgerritEyal proposed openstack/mistral master: Add a cookiecutter template to generate custom stuff  https://review.opendev.org/67978208:22
vgvolegIf we see that result of the task is not so important right now, we can skip this task and continue the execution of the flow08:22
rakhmerovok08:22
*** irclogbot_0 has joined #openstack-mistral08:23
*** akovi has joined #openstack-mistral08:23
akovihi All!08:23
rakhmerovakovi: hi Andras08:23
rakhmerovwe're discussing https://blueprints.launchpad.net/mistral/+spec/mistral-task-skipping-feature08:23
vgvolegand we should be sure that the execution will not break08:23
vgvolegso we provide an opportunity to 'mock' any data in 'publish-on-skip'08:24
rakhmerovvgvoleg: what's the problem if it breaks?08:24
rakhmerovit will just have status "ERROR" but it will do all the work08:24
vgvolegin case we need something from 'publish' in further08:25
rakhmerovmaybe you just need some mechanisms (in your system) to interpret the results of a failed workflow correctly?08:25
*** ricolin_ is now known as ricolin08:25
vgvolegWe can't expect every situation08:25
*** irclogbot_0 has quit IRC08:25
rakhmerovvgvoleg: again, as far as "publish-on-skip", I'm not sure we need this at all08:25
rakhmerovbut may be for the sake of symmetry, we need to do both options: "publish-on-skip" and the totally new clause "on-skip" that can have "publish" inside just like for other "on-xxx" things08:26
vgvolegI'm not sure if I understand you right08:27
rakhmerovok08:27
vgvolegyou just say about global problems08:27
rakhmerovno08:27
vgvolegnot about problems of this feature?08:27
rakhmerovI'm talking about the new syntax for "publish"08:27
rakhmerovno-no, it's related08:28
rakhmerovthat's why I touched it08:28
akoviso, to clarify: the basic idea is to let a task be skipped before rerunning it. Right?08:28
vgvolegOk I got it08:28
rakhmerovakovi: even after running it08:28
rakhmerovvgvoleg: is that right?08:28
vgvolegakovi: yes, we skip the task and rerun the workflow08:28
akovithe execution fails, we skip the task and rerun08:29
akoviok08:29
rakhmerovyep08:29
vgvolegConsider the situation: in the flow there is a task that requests any data from a third-party service while it is dead. Retries will not help in this situation, the task will go to the ERROR terminal state and the workflow will finish its work. Starting the workflow from the very beginning can be very expensive - it could have been several days bef08:29
vgvolegore the fall. For such cases in the mistral there is a rerun mechanism - a certain decision maker determines whether circumstances have changed (whether third-party service has come to life), and if so, the workflow will continue its work from the fallen task.08:29
rakhmerovbecause "retry" wouldn't make sense in many cases08:29
vgvolegn fact, the environment cannot always stabilize, and it can be very expensive to adapt the workflow to work with a new environment. Also it is not always possible to automatically assess the nature of the error that led to the fall. It can be something fatal, and maybe something insignificant, which in general does not affect the execution of the w08:29
vgvoleghole workflow. The decision maker can assess how important the results of the current task are and continue the execution of the workflow if not important.08:29
vgvolegyes08:29
vgvolegit's flom blueprint :D08:29
rakhmerovI figured )08:29
vgvolegall the arguments are there08:29
akovithis is probably useful when wfs are executed under human surveillance08:30
vgvolegyes08:30
rakhmerovvgvoleg: too many arguments, usually if we can't express an idea with 1 phrase then it's a bad idea )08:30
rakhmerovakovi: well, yes, it's exactly about that08:30
rakhmerovakovi: basically, that way we provide more ways for humans to influence workflow executions08:31
akoviso my general stance on wfs is that if needed, they should be implemented in an idempotent way08:32
rakhmerovso I guess, I'm not against it if 1) It's 100% backwards compatible (shouldn't be a problem here) 2) if it's a totally separate feature that doesn't interfere with existing stuff08:32
akovihowever, it's freaking hard in many cases08:32
rakhmerovby #2 I mean that it doesn't reuse "on-success" etc.08:32
rakhmerovakovi: yeah..08:32
akoviI think this feature is a useful one.08:33
vgvolegIt would be very comfortable to reuse 'on-success' if 'on-skip' is missed08:33
akoviUnfortunately it will work in many cases only if the publish-on-skip closure is defined in the wf spec08:34
rakhmerovvgvoleg: no, let's please not do this08:34
vgvolegIn many cases we want to continue 'on-success' execution08:34
rakhmerovwell, on the other hand...08:34
vgvolegSo we will have duplicates08:34
vgvolegin every task08:34
rakhmerovI know you want, but I'm not sure at all if other people will want )08:35
rakhmerovI want to make sure we have the common sense here08:35
vgvolegakovi: yes, that's the main idea: if you want to use this feature, be sure that your flow is ready for this08:35
rakhmerovvgvoleg: they you can have "on-skip" where needed08:36
akoviwhat if we omit the publish-on-skip and substitute it with a noop task that just publishes the same values?08:36
rakhmerovvgvoleg: I somewhat don't like the idea to reuse "on-success"08:36
vgvolegrakhmerov: it's not a problem if we have 'on-skip: t1, on-success: t1'08:36
rakhmerovother people can say "we want to consider such tasks failed but w/o moving them to ERROR status)08:36
vgvolegbut if we have a long array with next tasks, duplicating them would be ugly08:36
rakhmerovvgvoleg: why?08:37
rakhmerovwhy ugly?08:37
rakhmerovit's just about your case08:37
rakhmerovbut like I said, we're a considering a completely different even that may happen in a workflow08:37
rakhmerovand different people may treat it differently08:38
rakhmerovthat's why I want to make it more generic and not let it interfere with the existing mechanisms08:38
vgvolegwell, I can write some docs... :D08:38
rakhmerovdocs for what?08:38
vgvolegFor this feature08:38
akoviwf:08:38
akovi  task1:08:38
akovi    action: some_custom_action08:38
akovi      publish:08:38
akovi        var1: <% task (). result.var1%>08:38
akovi        var2: <% task (). result.var2%>08:39
akovi        var3: <% task (). result.var3%>08:39
akovi      on-success: task208:39
akovi      on-skip: task208:39
akovi  skipped-task1:08:39
akovi    action: std.noop08:39
akovi    publish:08:39
akovi       var1: "var1"08:39
akovi       var2: "var2"08:39
vgvolegIf something is described in docs, it is legal08:39
rakhmerovvgvoleg: let me put it this way: you may want to have lots of repeating tasks in "on-success" and in "on-error". But we don't say "it's ugly to repeat them"08:39
rakhmerovbecause those a totally different cases08:39
rakhmerovfor that we actually have "on-complete" where we can move repeating stuff08:40
rakhmerovvgvoleg: no-no, I can't accept that approach ("If something is described in docs, it is legal"), sorry08:40
rakhmerovdocs must not aim to explain why we came up with a bad design08:41
vgvolegakovi: how do you want to describe them?08:41
vgvolegif we run this flow the skipped task will be executed with task1 in the parallel way :D08:42
akovino08:42
akoviI messed it up08:42
akovitask1.on-skip = skipped-task108:42
vgvolegoh08:43
akovithis way there's no need for alternative publish methods08:43
vgvolegI can't undestand why 'publish-on-error' is OK and 'publish-on-skip' is not OK08:44
akoviwhere do we usually share copy-pastes? I forgot the name of the service08:44
eyalbpastebin08:44
vgvolegI think that creating redundant instances for publish is not OK08:44
rakhmerovredundant instances?08:44
rakhmerovwhat's that?08:44
vgvolegnoop task just for publish08:45
rakhmerovguys, please let's be more accurate with terms08:45
rakhmerovnoop for publish... I fail to understand this08:45
vgvoleglook at Andras' example08:46
rakhmerovok, yes...08:46
rakhmerovwell, IMO it's not a problem to make a separate "publish-on-skip" thing08:47
rakhmerovand "publish" under "on-skip"08:47
rakhmerovit's easy08:47
rakhmerovI just don't like the idea to reuse either "on-success" or something else that already exists to handle skipping08:48
rakhmerovduplicates, in my opinion, is not a problem that we need to solve now08:48
vgvolegok08:48
abdelalthe plan is to eventually remove publish and publish-on-error right?, i dont think its wise to add publish-on-skip too08:48
rakhmerovabdelal: yes!08:48
rakhmerovabdelal: yes, right08:49
rakhmerovI just thought that maybe we still need to add it, but just for the sake of symmetry with other clauses08:49
abdelalso i think we should follow the current synyax we want,just have publish under on-skip if anything08:49
abdelalsyntax*08:49
rakhmerovthat's for sure, yes08:49
vgvolegguys I'm OK with this changes but please don't mix them to the feature discussion08:50
eyalbI agree08:50
rakhmerovI'm just saying that the language should always be symmetric around similarities that we're adding08:50
rakhmerovvgvoleg: :)08:50
rakhmerovso, if there's no serious objections then I'd say "go ahead and push a patch"08:51
abdelali think this feature is useful overall , but also as renat said , it should be as generic as much as possible and be alligned with current syntax08:51
rakhmerovthese technical nuances is something that we'll polish a bit later08:51
rakhmerovnot now08:51
rakhmerovyes08:51
akoviok, great08:51
rakhmerovvgvoleg: sounds OK?08:52
vgvolegsure :)08:52
akoviso on-skip.publish? or publish.on-skip?08:52
rakhmerovonce we see your patch we may notice something else to discuss08:52
rakhmerovakovi: I'm for adding "publish-on-skip" and "on-skip" (that may contain "publish") just the same way as for other clauses08:53
rakhmerovthen it'll be 100% symmetric08:53
vgvolegguys, in the blueprint everything is symmetric08:53
vgvoleg102%08:53
akovihmm08:53
akovithe removal of publish-on* was new info for me08:54
vgvolegIt's because they mix two different topics08:54
akoviand if this is being removed then I'm not sure why we would provide this syntax for a new feature08:54
akovibut I'm ok08:54
vgvolegrelax :)08:54
rakhmerovvgvoleg: yes, sorry for that08:55
rakhmerovbut it's kind of related08:55
rakhmerovif we are to discuss particular syntax08:55
vgvolegso the second blueprint I'd like to discuss in the next meeting08:56
rakhmerovvgvoleg: yes, please08:56
vgvolegthank you!08:56
rakhmerovwe don't have enough time today08:56
rakhmerovthanks guys, I have to wrap up )08:56
rakhmerovthanks everyone for joining08:56
rakhmerovI'd encourage you to do it more often08:56
eyalbbye08:56
rakhmerovbye08:56
vgvolegbye!08:56
akovigreat, I'm looking forward to see this working08:56
rakhmerov#endmeeting08:56
*** openstack changes topic to "Mistral the Workflow Service for OpenStack. https://docs.openstack.org/mistral/latest/"08:56
openstackMeeting ended Wed Sep  4 08:56:58 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:57
openstackMinutes:        http://eavesdrop.openstack.org/meetings/mistral/2019/mistral.2019-09-04-08.06.html08:57
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/mistral/2019/mistral.2019-09-04-08.06.txt08:57
openstackLog:            http://eavesdrop.openstack.org/meetings/mistral/2019/mistral.2019-09-04-08.06.log.html08:57
*** vgvoleg has quit IRC09:00
*** irclogbot_1 has joined #openstack-mistral09:09
*** vgvoleg has joined #openstack-mistral09:09
*** irclogbot_1 has quit IRC09:11
*** ricolin has quit IRC09:34
*** ricolin has joined #openstack-mistral09:35
*** pgaxatte has quit IRC09:40
*** pgaxatte has joined #openstack-mistral09:46
*** akovi has quit IRC10:13
*** akovi has joined #openstack-mistral10:14
*** vgvoleg has quit IRC10:27
*** irclogbot_0 has joined #openstack-mistral10:29
*** irclogbot_0 has quit IRC10:33
*** irclogbot_2 has joined #openstack-mistral10:43
*** irclogbot_2 has quit IRC10:45
*** akovi has quit IRC10:57
*** irclogbot_3 has joined #openstack-mistral10:59
*** irclogbot_3 has quit IRC11:02
*** irclogbot_0 has joined #openstack-mistral11:43
*** irclogbot_0 has quit IRC11:47
*** vgvoleg has joined #openstack-mistral12:21
*** irclogbot_1 has joined #openstack-mistral13:39
*** irclogbot_1 has quit IRC13:42
*** irclogbot_0 has joined #openstack-mistral13:59
*** irclogbot_0 has quit IRC14:02
*** irclogbot_1 has joined #openstack-mistral14:05
openstackgerritMerged openstack/mistral master: Support OpenStack services dynamic versions  https://review.opendev.org/62147014:06
*** irclogbot_1 has quit IRC14:08
*** irclogbot_2 has joined #openstack-mistral14:11
*** irclogbot_2 has quit IRC14:14
*** pgaxatte has quit IRC14:28
*** irclogbot_3 has joined #openstack-mistral14:37
*** irclogbot_3 has quit IRC14:40
*** irclogbot_1 has joined #openstack-mistral14:51
*** irclogbot_1 has quit IRC14:54
openstackgerritEyal proposed openstack/mistral master: Add "published_global" field to the task execution REST resource  https://review.opendev.org/67853014:55
*** eyalb has quit IRC15:06
*** irclogbot_3 has joined #openstack-mistral15:19
*** irclogbot_3 has quit IRC15:22
*** irclogbot_2 has joined #openstack-mistral15:33
*** abdelal has quit IRC15:35
*** irclogbot_2 has quit IRC15:36
*** irclogbot_2 has joined #openstack-mistral15:37
*** vgvoleg has quit IRC15:37
*** altlogbot_1 has joined #openstack-mistral15:39
*** ricolin has quit IRC18:50
*** irclogbot_2 has quit IRC18:54
*** irclogbot_0 has joined #openstack-mistral18:55
*** irclogbot_0 has quit IRC19:02
*** irclogbot_3 has joined #openstack-mistral19:03
*** jtomasek has quit IRC20:08
openstackgerritMerged openstack/mistral master: Fix 'with-items' expression evaluation  https://review.opendev.org/67920323:19

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!