Tuesday, 2017-05-16

*** harlowja has joined #openstack-mistral00:49
*** gongysh has joined #openstack-mistral01:07
*** harlowja has quit IRC01:42
*** harlowja has joined #openstack-mistral01:56
openstackgerritLingxian Kong proposed openstack/mistral master: Add "action_region" param for OpenStack actions  https://review.openstack.org/46035102:07
*** harlowja has quit IRC02:20
*** gongysh has quit IRC02:39
rakhmerovbreton: yes, it converts context to json. It's here: https://github.com/openstack/mistral/blob/master/mistral/context.py#L24802:53
kongrakhmerov: did we have a demo in last meeting? is there any video i can see?03:01
kongit was too late for me to attend the meeting :(03:02
rakhmerovkong: yes, we did. We recorded it, Guy Shaanan should send a link to the recording soon03:02
rakhmerovyeah, I know :(03:02
kongrakhmerov: cool, looking forward to that03:02
rakhmerovyep, it should be interesting for you03:02
kongrakhmerov, d0ugal, https://review.openstack.org/#/c/460351/ is ready03:21
kongalso tested it on my multi-region environment03:22
kongusing nova.hypervisors_statistics action03:22
rakhmerovkong: done03:23
*** mbrennan has joined #openstack-mistral03:25
kongrakhmerov: ta!03:25
rakhmerov:)03:25
rakhmerovkong: sorry that it's been taking that long03:25
kongrakhmerov: that's all right. I have used that patch on my own mistral deployment for a while.03:26
rakhmerovok03:27
rakhmerov:)03:27
*** mbrennan has quit IRC03:30
*** sharat has joined #openstack-mistral04:49
*** sharatss_ has joined #openstack-mistral05:06
*** sharat has quit IRC05:06
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: [WIP] Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517405:08
*** jamielennox is now known as jamielennox|away05:10
openstackgerritAdriano Petrich proposed openstack/mistral master: Adding log to db_sync  https://review.openstack.org/46401005:17
*** jamielennox|away is now known as jamielennox05:30
*** livelace has joined #openstack-mistral05:36
*** harlowja has joined #openstack-mistral05:47
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: [WIP] Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517405:58
*** gongysh has joined #openstack-mistral06:27
*** gongysh has quit IRC06:35
*** luigiOpenstack_ has joined #openstack-mistral06:43
luigiOpenstack_Can anyone please help me to work on Openstack workflow with luigi?06:44
*** harlowja has quit IRC06:50
*** sharatss_ has quit IRC06:52
*** jaosorior has quit IRC06:53
luigiOpenstack_any help?06:58
rakhmerovluigiOpenstack_: hey, can you share more details what you want to do?07:00
luigiOpenstack_I have a very simple scenario right now to check how Luigi will work with Openstack07:01
luigiOpenstack_so I want to list images for example07:01
luigiOpenstack_using Luigi how can I get that output after successful authentication07:01
luigiOpenstack_I think task A can be successful authentication07:02
luigiOpenstack_once authenticated, task B could be to list images07:02
rakhmerovluigiOpenstack_: do you already have some workflow example that we could look at?07:03
luigiOpenstack_Not really...this is the first one so I am struggling.07:03
*** jamielennox is now known as jamielennox|away07:07
luigiOpenstack_something like this07:08
luigiOpenstack_import luigi import os_client_config    class LookupParametersTask(luigi.Task):     # pass This could look for openstack env parameters #nova = os_client_config.make_client('compute')  class CreateEnvironmentTask(luigi.Task):      def requires(self):             def run(self):              pass     def output(self): return{'images':self.glance.images.list()}  if _name_ == '__main__':     luigi.run()07:08
*** sharatss_ has joined #openstack-mistral07:11
*** brunograz has joined #openstack-mistral07:11
luigiOpenstack_@rakhmerov : anything?07:22
*** jamielennox|away is now known as jamielennox07:34
openstackgerritMerged openstack/mistral master: Add "action_region" param for OpenStack actions  https://review.openstack.org/46035107:36
*** jaosorior has joined #openstack-mistral07:43
*** jpich has joined #openstack-mistral07:53
rakhmerovluigiOpenstack_: not sure how it relates to Mistral. Does it?08:06
*** sharatss_ has quit IRC08:07
*** sharatss_ has joined #openstack-mistral08:08
luigiOpenstack_its not related to mistral08:08
luigiOpenstack_but its kind of a replacement to mistral08:08
rakhmerov:)08:09
apetrich_I had a hard time reading those messages until the last one that I understood Luigi is a software not someone that refers to himself in the third person08:12
rakhmerovluigiOpenstack_: please use http://paste.openstack.org for posting code and log snippents08:12
rakhmerovapetrich_: same to me )08:12
rakhmerovI'm not even familiar with it08:13
rakhmerovluigiOpenstack_: what is this project for?08:13
luigiOpenstack_this is just a thought that instead of being dependent on ansible for the workflows we would want to create python scripts08:13
luigiOpenstack_and use luigi for CI08:14
rakhmerovis there any web page (wiki etc.) about it?08:14
rakhmerovit's hard to help if we don't understand well what you're trying to do08:14
luigiOpenstack_yes08:14
luigiOpenstack_its limited though08:14
luigiOpenstack_https://github.com/spotify/luigi08:15
luigiOpenstack_https://luigi.readthedocs.io/en/latest/index.html08:15
rakhmerovand here we discuss Mistral topics, not sure if it's related to us honestly08:15
luigiOpenstack_I understand08:15
luigiOpenstack_any other IRC that could be helpful?08:16
rakhmerovyou can probably use the main OpenStack mailing list08:17
rakhmerovfor general OpenStack questions08:17
d0ugalI think it is only related because Luigi is a workflow management etc.08:20
d0ugalso it is similar to Mistral08:20
d0ugalit comes from Spotify08:20
*** jaosorior is now known as jaosorior_lunch08:20
luigiOpenstack_yes08:20
rakhmerovd0ugal: yeah, lots of things name themselves "workflow management", but then it turns that it's something completely different08:21
d0ugalAirflow is another, that comes from Airbnb08:21
d0ugalrakhmerov: good to know :) I have been meaning to try it but I have not yet.08:21
d0ugaloh, Airflow is now an Apache project - interesting.08:21
d0ugalAnyway, back to Mistral :)08:22
rakhmerovd0ugal: on a different topic, when Red Hat is building RPMs out of a stable branch, what commit is taken? Only tagged with a version or just latest from the branch?08:22
d0ugalrakhmerov: well, that depends :)08:23
d0ugalrakhmerov: are you thinking about backports?08:23
rakhmerovdepends on what?08:23
rakhmerovyes08:23
d0ugalRight, then I think it is based on the stable branch - but I can check if you like08:23
d0ugalrakhmerov: honestly I don't know that much about our downstream packaging - I only know a bit about RDO packages.08:24
rakhmerovyeah, I'm trying to find out that08:24
d0ugalrakhmerov: I'll ask - which version? Ocata?08:25
rakhmerovI need to know that pretty urgently because we're expecting new RPMs and I need to know what's going to be in them08:25
rakhmerovOcata, yes08:25
d0ugalrakhmerov: I've asked in our internal IRC - it might be quicker to use the official support lines if you can08:27
d0ugalbut I'll see what I can do08:27
rakhmerovd0ugal: no worries )08:28
rakhmerovwe'll ask officially08:28
d0ugalthanks08:29
d0ugalapetrich_: I checked out the devstack errors on your patch, i'm 99% sure they are unrelated.08:29
d0ugaland they changed a bit in the recheck :-D08:29
apetrich_d0ugal, thanks for checking. I have the same conclusion08:30
d0ugalrakhmerov: if you do find out about the packaging question - I'd like to know the answer. I am struggling to find out! :/09:04
rakhmerovok09:04
openstackgerritMerged openstack/mistral master: Adding log to db_sync  https://review.openstack.org/46401009:04
*** jaosorior_lunch is now known as jaosorior09:15
bretoni am trying to fix https://bugs.launchpad.net/mistral/+bug/1690787 now09:18
openstackLaunchpad bug 1690787 in Mistral "cron trigger uses trust-scoped token to create another token" [High,New]09:18
bretonand it is quite limiting that context in mistral should be serializable09:20
*** sharatss_ has quit IRC09:20
bretonbecause a session should be shared by all openstack clients in actions/openstack/actions.py09:21
bretonand from what i can see, the only way to share something among clients is via context09:21
bretonand session cannot be serialized09:21
*** sharatss_ has joined #openstack-mistral09:23
bretonon the other hand, keystoneauth cannot take for granted that a token is fine09:24
bretonit doesn't help; but if it could, it would not fix the root cause09:28
thervebreton, You don't need to serialize the session, you can recreate it with the necessary data09:30
therveHeat uses the same pattern and it works fine09:32
bretontherve: and store it again in the context?09:36
thervebreton, Sure just set it as an attribute of the context when you load it09:36
openstackgerritRenat Akhmerov proposed openstack/mistral master: WIP: refactoring methods in db models  https://review.openstack.org/46500109:39
openstackgerritRenat Akhmerov proposed openstack/mistral master: Refactor db model methods  https://review.openstack.org/46500109:51
*** luigiOpenstack_ has quit IRC09:55
*** shardy has joined #openstack-mistral09:57
openstackgerritLingxian Kong proposed openstack/mistral master: Add release note for "action_region" support  https://review.openstack.org/46501510:13
*** sharatss_ has quit IRC10:17
*** sharatss_ has joined #openstack-mistral10:17
*** thrash|g0ne is now known as thrash10:18
openstackgerritLingxian Kong proposed openstack/mistral-specs master: Move multi-region-support to implemented folder  https://review.openstack.org/46501710:22
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: [WIP] Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517410:23
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: [WIP] Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517410:30
*** jkilpatr has quit IRC10:41
*** jkilpatr has joined #openstack-mistral10:59
*** shardy is now known as shardy_lunch11:10
bretontherve: what would be the best place to put session instantiation then? ./mistral/engine/engine_server.py? Default engine?11:21
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517411:23
thervebreton, Well, most likely when the context is created?11:23
therveYou can also simply make it a property of the context, which is loaded dynamically11:24
therve(Side note: mistral.service and mistral.services are insane...)11:24
bretonoh, i see11:28
bretonso the session will not get serialized and will be accessed by just ctx.auth_session?11:28
therveYeah11:33
*** abishop has quit IRC11:35
*** shardy_lunch is now known as shardy11:50
*** abishop has joined #openstack-mistral12:06
*** abishop has quit IRC12:06
*** sharatss_ has quit IRC12:11
*** sharatss_ has joined #openstack-mistral12:29
*** dprince has joined #openstack-mistral12:38
*** mbrennan has joined #openstack-mistral12:42
openstackgerritSharat Sharma proposed openstack/python-mistralclient master: Use keystoneauth plugins and session instead of keystoneclient  https://review.openstack.org/45517413:16
*** sharatss_ has quit IRC13:20
*** mbrennan has quit IRC13:34
*** jrist has quit IRC13:43
openstackgerritluong tuan proposed openstack/mistral master: Refactor mistral context using oslo_context  https://review.openstack.org/45540713:49
*** gfidente has joined #openstack-mistral13:51
*** jaosorior is now known as jaosorior_away14:00
tourewin 1514:15
d0ugalI assume pause-before happens at the start of a task and not for each items in with-items?14:17
gfidented0ugal I am finally back at the mistral environment issue14:20
gfidentehave a few cycles to help?14:20
d0ugalgfidente: sure, what's up?14:20
gfidenteso I think I am just looking for a working example of a workflow consuming a variable from a global environment14:20
gfidenteand one of a workflow consuming a variable from an execution environment14:20
gfidentethe $.__env. syntax doesn't even work with my version of the client14:21
gfidenteERROR (app) Lexical error: illegal character '_' at position 314:21
gfidente<% $.__env.mine.foo1 %>14:22
gfidente(I had an environment named mine with a variable named foo1)14:22
gfidentewhat I am doing wrong?14:22
d0ugalgfidente: Good question. Give me a few mins and I'll try and get an example - I have not actually used this yet.14:23
gfidenteI am not sure if the scope of the environment set to private14:23
gfidenteby default14:24
gfidenteis hiding it14:24
gfidentebut given I get a syntax error I think I am not even getting to that part14:24
gfidenteit's also unclear to me the syntax to consume variables from the execution environment14:25
gfidentelooking at execution, I think to pass an execution environment I should use params: env: key:value14:29
gfidentebut then again I am not sure how to consume it from within the workflow14:29
gfidenterakhmerov actually I see you on the execution environment spec14:41
gfidenterakhmerov maybe you can help understand how to use it?14:41
d0ugalgfidente: rakhmerov probably knows :) but he isn't usually around now14:42
gfidenteI went through the submissions linked to the spec in the hope of finding an .rst somewhere14:42
d0ugalgfidente: I think this stuff is older than specs14:42
d0ugaland older than docs lol14:43
gfidenteYaqlEvaluationException: Can not evaluate YAQL expression [expression=$.env.foo1, error=u'env', data={}14:43
gfidentethis stuff is killing me14:43
d0ugalgfidente: have you tried env()14:43
gfidenteseems like data in yaql is empty14:43
d0ugal<% env().foo1 $>14:43
d0ugal<% env().foo1 %>14:43
gfidenteI can14:43
d0ugalThat is what I am testing14:44
gfidentethat worked14:44
gfidentewow14:45
gfidenteso how do I get to a global environment?14:45
d0ugalgfidente: what do you mean by global environment?14:45
gfidenteone of those created with environment-create14:45
d0ugalgfidente: ah, that I don't know14:45
d0ugalI have been meaning to find out, because I didn't understand it but I've not yet.14:46
d0ugalgfidente: maybe you can pass env a name?14:46
d0ugalbut that might be too logical :)14:46
* d0ugal looks at the code14:46
d0ugalSeems not. It is super simple.14:47
d0ugalhttps://github.com/openstack/mistral/blob/master/mistral/utils/expression_utils.py#L99-L10014:47
d0ugalgfidente: do you just need to get it? or do you need to update it too?14:47
gfidented0ugal yeah that's yet another issue14:48
thervegfidente, I think you pass a string as a env in parameters14:48
* therve finds the code14:48
gfidentetherve yes that worked now14:48
gfidentesets an execution env14:48
thervegfidente, https://github.com/openstack/mistral/blob/master/mistral/engine/workflows.py#L42814:48
therveI don't know if you can access any arbitrary env14:49
*** bobh has joined #openstack-mistral14:49
gfidentetherve but on the other hand we probably don't want to use the global ones anyway14:49
therveYeah14:50
gfidenteso I guess, going back to the resource14:50
gfidentewe probably need to save this in the heat resource itself14:50
therveOh yeah don't create a mistral environment object14:50
gfidentelet's see if i can come up with something14:51
gfidentethanks guys14:51
d0ugali would like to know what the mistral environments are for14:52
d0ugalwhy ever create them14:52
gfidenteto talk on irc14:52
d0ugallol14:52
gfidentelike weather14:52
d0ugalwe need something to talk about14:52
d0ugalHows your environment doing today?14:52
gfidenteno I think the scope property14:52
gfidenteis interesting14:52
gfidenteif the scope could match multiple workflows with a regexp for example14:53
therveThat's a solution looking for a problem14:53
therveBut it can have its uses14:53
gfidenteright14:53
d0ugal:)14:53
therveeg: you store an environment called "private-team-X" with an image name.14:53
d0ugalSo far I have only seen it abused as a key value store :(14:54
therveWhen you update that image, you can change the id in it14:54
therveAll your workflows will benefit it14:54
therveAnd you can run you workflow in different "environments" without changing it14:54
gfidenteall your base are belong to us14:54
gfidentehttps://it.wikipedia.org/wiki/All_your_base_are_belong_to_us14:54
gfidentebetter14:55
gfidentehttps://en.wikipedia.org/wiki/All_your_base_are_belong_to_us14:55
d0ugallol14:55
gfidenteall your environment are belong to us in this case14:55
gfidentebut gfidente doesn't know how to use it14:56
gfidenteso it's worthless14:56
gfidenteanyway14:56
d0ugal:(14:57
d0ugalgfidente: How did you get env() to work?14:57
d0ugalI can't even do that now.14:57
gfidentepass {"env": {"key":"value"}} to an execution14:58
gfidenteas params14:58
gfidentewhich is what therve wants the heat resource to do as well14:58
d0ugaloh, wow, you need to include env in the json!14:58
gfidenteyeah14:59
gfidentein the execution json14:59
d0ugalThat is what I was doing wrong - got that working now anyway14:59
*** tuan_ has joined #openstack-mistral15:01
openstackgerritMerged openstack/mistral master: Remove unused logging import  https://review.openstack.org/46165415:03
openstackgerritMerged openstack/mistral master: Add release note for "action_region" support  https://review.openstack.org/46501515:03
*** thrash is now known as thrash|bbl15:18
tuan_Hi guys15:26
tuan_if someone there15:26
tuan_could you please review this patch15:27
tuan_PS1915:27
tuan_https://review.openstack.org/#/c/455407/1915:27
tuan_i was quite tired about this one since it modifies the mistral context15:27
tuan_and it has alot of conflicts since someone else is modifying context too15:27
tuan_this patch is on top of master15:27
tuan_therefore i always have to fix the conflict all the time15:28
tuan_:D15:28
tuan_and now this one has conflict again with the current modification of context15:28
tuan_which is done by Kong15:28
tuan_:)15:29
* d0ugal starts writing a patch that refactors the context15:29
d0ugal;)15:30
d0ugaltuan_: I probably wont have time today, but I'll try and look tomorrow morning first thing15:30
d0ugalfeel free to remind me!15:30
tuan_okay, i wrote your name in my calendar15:33
tuan_:)15:33
tuan_thanks a lot15:33
tuan_and the PS19 was +1 before it went to conflict15:34
*** gfidente is now known as gfidente|afk15:40
bretonwhere is MistralContext constructed after being transmitted over rpc?15:52
bretonoh, i see, in _on_message15:53
*** tuan_ has quit IRC16:06
*** jpich has quit IRC16:24
*** shardy_ has quit IRC16:46
*** thrash|bbl is now known as thrash16:52
*** mbrennan has joined #openstack-mistral17:01
*** shardy has quit IRC17:04
*** mbrennan has quit IRC17:06
*** mbrennan has joined #openstack-mistral17:27
*** mbrennan has quit IRC17:27
*** mbrennan has joined #openstack-mistral17:28
*** kbaegis has joined #openstack-mistral17:30
kbaegisHi all. Can someone walk me through the mistral operations for starting a heat template?17:30
kbaegisDo I need to use heat.stacks_template or heat.stacks_create?17:30
kbaegisSeems to be challenging converting a complex heat template w/ userdata into a json object for mistral17:32
*** harlowja has joined #openstack-mistral17:50
kbaegisAnyone?17:50
*** bobh has quit IRC17:51
*** jkilpatr_ has joined #openstack-mistral18:17
*** openstackgerrit has quit IRC18:17
*** jkilpatr has quit IRC18:20
*** bobh has joined #openstack-mistral18:21
*** mbrennan has quit IRC18:25
*** bobh has quit IRC18:54
*** harlowja has quit IRC19:17
*** gfidente|afk has quit IRC19:36
*** jkilpatr_ has quit IRC19:40
*** livelace has quit IRC19:43
*** kbaegis has quit IRC20:02
*** harlowja has joined #openstack-mistral20:04
*** bobh has joined #openstack-mistral20:13
*** dprince has quit IRC20:37
*** openstackgerrit has joined #openstack-mistral20:47
openstackgerritBob Haddleton proposed openstack/mistral master: Update AdHoc Actions to support context data references in YAQL/Jinja expressions  https://review.openstack.org/46467720:47
*** jkilpatr has joined #openstack-mistral20:53
*** kbaegis has joined #openstack-mistral20:55
*** kbaegis has left #openstack-mistral21:01
*** kbaegis has joined #openstack-mistral21:02
*** bobh has quit IRC21:02
*** bobh has joined #openstack-mistral21:03
*** kbaegis has quit IRC21:06
*** bobh has quit IRC21:08
*** jkilpatr has quit IRC21:19
*** jkilpatr has joined #openstack-mistral22:03
*** thrash is now known as thrash|g0ne22:03

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!