Wednesday, 2019-07-24

*** ricolin has joined #openstack-mistral00:59
openstackgerritRenat Akhmerov proposed openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error  https://review.opendev.org/67239001:02
*** gkadam has joined #openstack-mistral03:50
*** gkadam has quit IRC03:50
*** apetrich has quit IRC04:20
*** eyalb1 has joined #openstack-mistral04:45
*** pgaxatte has joined #openstack-mistral06:01
*** pgaxatte has quit IRC06:05
*** pgaxatte has joined #openstack-mistral06:05
*** pgaxatte has quit IRC06:39
*** pgaxatte has joined #openstack-mistral06:41
openstackgerritEyal proposed openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error  https://review.opendev.org/67239007:38
openstackgerritEyal proposed openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error  https://review.opendev.org/67239007:41
openstackgerritEyal proposed openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error  https://review.opendev.org/67239007:49
*** vgvoleg has joined #openstack-mistral07:56
rakhmerovhi all08:01
eyalb1\o08:01
vgvolego/08:01
rakhmerovvgvoleg, eyalb1, d0ugal: hi08:01
d0ugalHi08:01
rakhmerov#startmeeting Mistral08:01
openstackMeeting started Wed Jul 24 08:01:47 2019 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.08:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:01
*** openstack changes topic to " (Meeting topic: Mistral)"08:01
openstackThe meeting name has been set to 'mistral'08:01
vgvolegI'd like to discuss this https://review.opendev.org/#/c/670211/08:02
rakhmerovfolks, eyalb1 is my colleague from Nokia who's now the Vitrage PTL and he's also going to help with Mistral08:02
rakhmerovjust so you know08:02
rakhmerovvgvoleg: sure, go ahead08:02
vgvolegI don't understand why Andras didn't like this :D08:03
rakhmerovlet me see08:04
vgvolegThis is just an opportunity to implement actions and to write a wf which could be run as usual or in dry-run mode08:04
vgvolegThis feature could be compared with ansible's check mode08:05
*** apetrich has joined #openstack-mistral08:05
rakhmerovwell, I've not reviewed the patch itself so it's a bit difficult to discuss it08:06
rakhmerovlooking..08:06
vgvolegSo if you want to run the flow with dry-run mode, you should be sure that actions from this flow support this08:07
rakhmerovhm.. ok08:10
rakhmerovso08:10
rakhmerovI kind of understand his point08:10
rakhmerovbecause yes, in a real workflow data is critically important for all the transitions etc.08:11
rakhmerovbut at the same time I see no harm in having this feature merged08:11
vgvolegYes, I understand this08:11
rakhmerovat least we could see if it works in some very simple configuration08:12
rakhmerovits simplest paths08:12
vgvolegIt would be great to adapt std actions for this08:12
rakhmerovso what your patch does is it makes it possible to run "test" in actions instead of "run"?08:12
rakhmerovis this it?08:12
vgvolegyes08:13
vgvolegand maybe openstack actions08:14
vgvolegto return stubs in test run08:14
rakhmerovok08:14
vgvolegbut first of all, this feature is for custom actions08:15
rakhmerovyeah08:15
rakhmerovso, I'm not against it08:15
rakhmerovthe only thing I'm concerned with is maybe we need to consider a more sophisticated approach to testing workflows08:16
rakhmerovsome kind of framework for mocking workflow steps08:16
rakhmerovbut I'm not sure.. because it would involve serious time investements08:16
rakhmerovit's basically "How can we make sure that the workflow works correctly in any possible situation"08:17
rakhmerovsort of unit testing for workflows08:17
rakhmerovbut that's complex08:17
rakhmerovwe've discussed that idea in the past but realized it'd be very difficult to make it usable enough08:18
rakhmerovnot overcomplicated08:18
apetrichMorning rakhmerov08:18
rakhmerovapetrich: hey Adriano08:19
apetrichHow was the vacations?08:19
rakhmerovlong time no see08:19
rakhmerovho's it going?08:19
rakhmerovmy vacation was good ) I had good rest08:19
rakhmerovand almost forgot about the work (which is very good) :)08:19
apetrich:)08:19
rakhmerovvgvoleg: so Oleg, let us review your patch first08:20
rakhmerovI've already glanced quickly and it looks ok but I still have some questions08:20
rakhmerovvgvoleg: but generally the idea is OK to me08:21
rakhmerovapetrich: btw, I remember you helped investigating a problem with the requirements08:22
apetrichaye08:22
rakhmerovif you have a few min could you please look at http://logs.openstack.org/90/672390/4/check/requirements-check/00c6495/job-output.txt.gz ?08:22
rakhmerovwe're having hard time to see why the requirement check keeps failing08:23
rakhmerovin stable/stein branch08:23
apetrichsure can do right now.. I'm trying a deploynment08:23
rakhmerovin master this patch is OK08:23
rakhmerovapetrich: thanks a lot08:23
apetrichrakhmerov, can I have a link to the patch?08:24
rakhmerovyes, sure08:24
rakhmerovhttps://review.opendev.org/#/c/67239008:24
*** jtomasek has joined #openstack-mistral08:25
apetrichcheers08:25
rakhmerov:)08:25
vgvolegI have a question: is it possible to "bind" execution to one engine?08:26
vgvolegso if I had a small workflow and a lot of engine replicas, it would be great not to have a huge overhead to load this definition to the cache of every engine08:28
rakhmerovvgvoleg: currently no08:28
rakhmerovwell, if the workflow is small then there won't be a huge overhead )08:29
rakhmerovit'll be huge if the workflow is big08:29
rakhmerovI think we've already discussed this with you08:30
rakhmerovit'd be good to see some BP about that08:30
vgvolegI perform a small svt, results are tangible08:30
rakhmerovsvt?08:30
vgvolegstress test08:30
rakhmerovthen share this with us somehow08:31
rakhmerovcomparison, what you did etc08:31
rakhmerovI remember we talked about the opportunity to have kind of "ad-hoc" workflows08:31
vgvolegit's not done, but I can share what's done08:32
rakhmerovthat is, we create a workflow and immediately run it08:32
rakhmerovok08:32
vgvolegoh what an amazing english skills I have08:32
vgvolegsorry :D08:32
rakhmerovis that the same thing?08:32
vgvolegso08:32
rakhmerovvgvoleg: no worries, your english is fine )08:32
vgvolegno, but it's all about one case08:32
vgvolegso the case is08:32
rakhmerovand will become even better over time ;)08:32
rakhmerovok08:32
rakhmerovwhat I've just described I believe is doable08:33
vgvolegwe run in the parallel way next scenario: create workflow -> execute workflow -> wait till it's done -> repeat08:33
rakhmerovbut that's mostly about workflow definition life-time08:33
rakhmerovnot about how we distribute workflow processing08:33
rakhmerovyes, ok08:33
vgvolegthe workflow is small - 8-10 http tasks08:34
rakhmerovok08:34
vgvolegI scale users, that performs this scenario08:34
rakhmerovand you want to bind it to one engine08:34
rakhmerovI see08:34
vgvoleghttps://docs.google.com/spreadsheets/d/1S26ImGdb3V6TnbkIVyDA85OHGx0Dvbt7o0kX-rWfmDE/edit#gid=008:35
rakhmerovvgvoleg: the thing is that binding to one engine conflicts with the fundamental idea behind Mistral08:35
rakhmerovthat every workflow step must be stored in DB08:35
vgvolegand the current results (it's not done) shows that scaling engine makes performance worse08:36
rakhmerovso that any other processing unit (engine) could continue with the processing08:36
rakhmerovvgvoleg: of course, it makes it worse08:36
rakhmerovbut it makes it more reliable08:36
rakhmerovwhich is a critically important property of any workflow engine08:36
rakhmerovit's all about scaling08:36
vgvolegvalue in google doc describes how many flows could be success i one minute08:36
vgvolegin one*08:37
rakhmerovI sent a permission request08:37
rakhmerovvgvoleg: that's all understandable, of course :)08:37
vgvolegdone08:37
rakhmerovIf I just write a program in Python it'll be even faster08:38
rakhmerovand we won't need any YAML, YAQL and scale )08:38
vgvoleg:D08:38
vgvolegyes08:38
rakhmerovbut we'll lose the most important property of the system: reliability and scalability08:39
vgvolegwhat do you mean "ad-hoc workflow"?08:39
rakhmerovif we talk about just routing messages related to some workflow to one specific engine then it's doable I think, yes08:39
rakhmerovbut what if the engine crashes?08:39
vgvolegyes I understand you08:40
rakhmerovby "ad-hoc" workflow I mean: we do something like "mistral execution-create --definition my_wf.yaml" and Mistral would create an execution based on my_wf.yaml08:40
vgvolegthe main challenge is to adapt mistral to work with one-off workflows as good as with others08:40
vgvolegyes, that's it08:41
rakhmerovbut without creating an object in DB that we could access with "mistral workflow-get"08:41
rakhmerovyep08:41
rakhmerovvgvoleg: that feature is totally OK to me08:41
rakhmerovI see no issues and I think it's not so difficult to implement08:41
vgvolegok I'll try and maybe there would be something to discuss in the next meeting :)08:42
rakhmerovbut as far as binding a workflow to one engine, well, I have serious doubts. I'll look at this Google Doc though08:42
rakhmerovyes08:42
rakhmerovsure08:42
rakhmerovlet's keep discussing it08:43
vgvolegI understand that binding is a bad idea08:44
vgvolegprobably the "ad-hoc workflow feature" will solve this problem08:45
rakhmerovok08:46
rakhmerovguys, do you have anything else to discuss?08:46
vgvolegnope08:47
rakhmeroveyalb1, apetrich: btw, I didn't release T-1 and T-2 because I kind of didn't see a big point08:47
vgvolegwait08:47
vgvolegone question08:47
rakhmerovwhat do you think we need to do moving further?08:47
rakhmerovapetrich, eyalb1: I thought about just releasing T-1 instead of T-3 when the T-3 time comes and then go to RCs08:48
vgvolegwhy don't we build and deploy docker image in CI steps?08:48
rakhmerovvgvoleg: aah, Andras knows better but he's on vacation now08:48
rakhmerovI think he'll be here next week08:49
rakhmerovthere was a reason for it but I honestly don't remember08:49
vgvolegnice08:49
rakhmerovwe used to have this job but then it was removed08:49
rakhmerovlet's ask Andras when he's back08:50
vgvolegok08:51
*** altlogbot_1 has quit IRC08:51
*** irclogbot_1 has quit IRC08:51
rakhmerovok, so let's finish the meeting now08:52
rakhmerovapetrich, eyalb1: please reply to the release question when you have a chance08:52
rakhmerov#endmeeting08:52
*** openstack changes topic to "Mistral the Workflow Service for OpenStack. https://docs.openstack.org/mistral/latest/"08:52
openstackMeeting ended Wed Jul 24 08:52:32 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:52
openstackMinutes:        http://eavesdrop.openstack.org/meetings/mistral/2019/mistral.2019-07-24-08.01.html08:52
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/mistral/2019/mistral.2019-07-24-08.01.txt08:52
openstackLog:            http://eavesdrop.openstack.org/meetings/mistral/2019/mistral.2019-07-24-08.01.log.html08:52
*** altlogbot_2 has joined #openstack-mistral08:52
*** irclogbot_1 has joined #openstack-mistral08:52
*** vgvoleg has quit IRC09:05
openstackgerritEyal proposed openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error  https://review.opendev.org/67239009:38
rakhmerovapetrich: eyalb1 managed to fix the requirements problem09:55
rakhmerovapetrich: can you please review https://review.opendev.org/#/c/672245/ ?09:55
rakhmerovit's small09:55
rakhmerovwe need it desperately )09:55
rakhmerovd0ugal: ^09:55
*** apetrich has quit IRC09:56
d0ugalrakhmerov: Seems like a hack, but I guess it is fine.09:57
*** quenti[m] has quit IRC10:08
rakhmerovd0ugal: yeah, it is a hack actually )10:10
rakhmerovbut I've no idea how to do it better10:10
d0ugalI'm not sure either :)10:12
*** pgaxatte has quit IRC10:24
*** quenti[m] has joined #openstack-mistral10:47
*** irclogbot_1 has quit IRC11:33
*** irclogbot_0 has joined #openstack-mistral11:35
*** apetrich has joined #openstack-mistral11:49
*** pgaxatte has joined #openstack-mistral12:03
*** vgvoleg has joined #openstack-mistral12:04
openstackgerritMerged openstack/mistral stable/stein: Retry a DB transaction on "Too many connections" error  https://review.opendev.org/67239012:46
*** apetrich has quit IRC13:47
*** apetrich has joined #openstack-mistral13:56
*** eyalb1 has quit IRC14:21
*** ricolin_ has joined #openstack-mistral14:42
*** ricolin has quit IRC14:45
*** bobh has joined #openstack-mistral14:49
openstackgerritElod Illes proposed openstack/mistral stable/rocky: Add bindep.txt file for binary dependencies used in unit tests  https://review.opendev.org/67254314:51
*** ricolin_ is now known as ricolin15:02
*** pgaxatte has quit IRC15:07
*** ricolin has quit IRC16:23
*** mmethot_ is now known as mmethot17:02
openstackgerritMerged openstack/mistral master: Retry a DB transaction on "Too many connections" error  https://review.opendev.org/67224517:19
*** bobh has quit IRC17:42
*** vgvoleg has quit IRC19:00
*** bobh has joined #openstack-mistral19:07
*** jtomasek has quit IRC19:39
*** bobh has quit IRC20:28
*** bobh has joined #openstack-mistral20:30
*** bobh has quit IRC20:39
openstackgerritguang-yee proposed openstack/mistral stable/rocky: disable triggering version discovery in test_generator unit tests  https://review.opendev.org/67218521:03

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!