| *** nosnos has joined #savanna | 01:33 | |
| *** xuhanp has joined #savanna | 01:37 | |
| *** IlyaE has quit IRC | 01:57 | |
| *** rnirmal has joined #savanna | 02:31 | |
| *** rnirmal has quit IRC | 04:12 | |
| *** IlyaE has joined #savanna | 04:29 | |
| *** IlyaE has quit IRC | 04:34 | |
| *** SergeyLukjanov_ is now known as SergeyLukjanov | 04:59 | |
| *** IlyaE has joined #savanna | 05:01 | |
| *** IlyaE has quit IRC | 05:08 | |
| *** IlyaE has joined #savanna | 05:10 | |
| *** nosnos_ has joined #savanna | 05:27 | |
| *** nosnos has quit IRC | 05:28 | |
| *** IlyaE has quit IRC | 06:08 | |
| *** nosnos_ has quit IRC | 06:10 | |
| *** nosnos has joined #savanna | 06:10 | |
| *** SergeyLukjanov is now known as SergeyLukjanov_ | 06:22 | |
| *** nadya_ has joined #savanna | 06:51 | |
| *** nadya_ has quit IRC | 07:30 | |
| *** qwerty_nor has joined #savanna | 08:38 | |
| *** aignatov has joined #savanna | 08:47 | |
| *** dmitryme has joined #savanna | 09:00 | |
| *** _ruhe is now known as ruhe | 09:03 | |
| *** DinaBelova has joined #savanna | 09:42 | |
| *** ityaptin has joined #savanna | 09:44 | |
| *** nosnos has quit IRC | 09:51 | |
| *** DinaBelova has quit IRC | 09:59 | |
| *** DinaBelova has joined #savanna | 09:59 | |
| *** ruhe is now known as ruhe_away | 09:59 | |
| *** ruhe_away is now known as _ruhe | 10:09 | |
| *** dmitryme has quit IRC | 10:09 | |
| *** SergeyLukjanov_ is now known as SergeyLukjanov | 10:25 | |
| *** dmitryme has joined #savanna | 10:28 | |
| *** aignatov has quit IRC | 10:28 | |
| *** SergeyLukjanov is now known as SergeyLukjanov_ | 10:35 | |
| *** SergeyLukjanov_ is now known as SergeyLukjanov | 10:51 | |
| *** _ruhe is now known as ruhe | 10:52 | |
| *** aignatov has joined #savanna | 10:52 | |
| *** xuhanp has quit IRC | 11:03 | |
| *** dmitryme has quit IRC | 11:42 | |
| *** dmitryme has joined #savanna | 12:01 | |
| *** DinaBelova has quit IRC | 12:23 | |
| *** mattf is now known as _mattf | 12:24 | |
| *** DinaBelova has joined #savanna | 12:25 | |
| *** akuznetsov has quit IRC | 12:29 | |
| *** akuznetsov has joined #savanna | 12:29 | |
| *** DinaBelova has quit IRC | 12:38 | |
| *** DinaBelova has joined #savanna | 12:42 | |
| *** ruhe is now known as ruhe_away | 12:53 | |
| *** ruhe_away is now known as ruhe | 12:54 | |
| *** dmitryme has quit IRC | 12:59 | |
| *** dmitryme has joined #savanna | 13:14 | |
| *** _crobertsrh is now known as crobertsrh | 13:22 | |
| *** tmckay has joined #savanna | 13:23 | |
| *** dmitryme has quit IRC | 13:24 | |
| tmckay | crobertsrh, _mattf, limited to freenode for a while. | 13:25 |
|---|---|---|
| crobertsrh | ack | 13:25 |
| *** dmitryme has joined #savanna | 13:28 | |
| openstackgerrit | Alexander Ignatov proposed a change to openstack/savanna: Enable heat engine to launch cluster without keypair https://review.openstack.org/66636 | 13:33 |
| *** ruhe is now known as ruhe_away | 13:41 | |
| *** ruhe_away is now known as ruhe | 13:44 | |
| *** _mattf is now known as mattf | 13:46 | |
| *** NikitaKonovalov has joined #savanna | 13:50 | |
| *** DinaBelova has quit IRC | 13:55 | |
| *** ruhe is now known as ruhe_away | 13:59 | |
| *** DinaBelova has joined #savanna | 14:00 | |
| *** akuznetsov has quit IRC | 14:01 | |
| mattf | aignatov, thanks for the feedback | 14:02 |
| aignatov | mattf, welcome! :) | 14:07 |
| aignatov | Actually I'm waiting for new thoughts from our contributors | 14:08 |
| *** tmckay has quit IRC | 14:08 | |
| *** tmckay has joined #savanna | 14:09 | |
| *** akuznetsov has joined #savanna | 14:12 | |
| *** NikitaKonovalov has quit IRC | 14:12 | |
| *** tmckay has quit IRC | 14:16 | |
| *** DinaBelova has quit IRC | 14:23 | |
| *** mattf is now known as _mattf | 14:33 | |
| *** tmckay has joined #savanna | 14:33 | |
| *** SergeyLukjanov is now known as SergeyLukjanov_a | 14:35 | |
| *** tmckay has quit IRC | 14:39 | |
| *** _mattf is now known as mattf | 14:50 | |
| *** tmckay has joined #savanna | 14:52 | |
| *** SergeyLukjanov_a is now known as SergeyLukjanov | 15:02 | |
| *** NikitaKonovalov has joined #savanna | 15:15 | |
| *** aignatov has quit IRC | 15:23 | |
| *** IlyaE has joined #savanna | 15:30 | |
| *** aignatov has joined #savanna | 15:31 | |
| openstackgerrit | Yaroslav Lobankov proposed a change to openstack/savanna: Enable check of Heat engine for Vanilla and HDP https://review.openstack.org/65911 | 15:50 |
| openstackgerrit | A change was merged to openstack/savanna-extra: Add a swift-enabled version of WordCount to edp-examples https://review.openstack.org/66675 | 16:02 |
| *** ruhe_away is now known as ruhe | 16:04 | |
| *** dmitryme has quit IRC | 16:13 | |
| *** SergeyLukjanov is now known as SergeyLukjanov_ | 16:24 | |
| *** SergeyLukjanov_ is now known as SergeyLukjanov | 16:30 | |
| *** aignatov has quit IRC | 16:36 | |
| *** ruhe is now known as ruhe_away | 17:00 | |
| mattf | Thursday, January 16, 2013 9 a.m. Pacific/12 Noon Eastern | 17:01 |
| *** rnirmal has joined #savanna | 17:01 | |
| mattf | Project team leads: Sergey Lukjanov, Data Processing (Savanna) and | 17:01 |
| mattf | Michael Basnight, Database Service (Trove) | 17:01 |
| mattf | Register: http://openstack.enterthemeeting.com/m/Z92BJCFA | 17:01 |
| *** SergeyLukjanov is now known as SergeyLukjanov_ | 17:01 | |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna-extra: [hadoop-openstack] Added debug log on getObject method https://review.openstack.org/67231 | 17:08 |
| *** NikitaKonovalov has quit IRC | 17:08 | |
| *** ruhe_away is now known as _ruhe | 17:10 | |
| *** SergeyLukjanov_ is now known as SergeyLukjanov | 17:19 | |
| *** SergeyLukjanov is now known as SergeyLukjanov_ | 17:44 | |
| *** ErikB has joined #savanna | 17:51 | |
| *** aignatov has joined #savanna | 17:54 | |
| aignatov | savanna team meeting will be in 5-7 minutes in #openstack-meeting-alt channel | 17:55 |
| *** alazarev has joined #savanna | 18:05 | |
| *** jmaron has left #savanna | 18:15 | |
| *** sreshetnyak has joined #savanna | 18:23 | |
| *** _ruhe is now known as ruhe | 18:25 | |
| *** NikitaKonovalov has joined #savanna | 18:42 | |
| *** ruhe is now known as ruhe_away | 18:45 | |
| tmckay | aignatov, mattf, sorry I missed it. I was deep in thought about EDP | 18:46 |
| *** IlyaE has quit IRC | 18:46 | |
| tmckay | aignatov, I think there may be a bug with Pig workflows that use arguments. We have arguments as a dictionary (like params). I think it's supposed to be just a list of strings. | 18:46 |
| aignatov | tmckay: not a problem at all! | 18:47 |
| aignatov | tmckay: let me quick a look edp code | 18:47 |
| *** NikitaKonovalov has quit IRC | 18:47 | |
| *** dmitryme has joined #savanna | 18:47 | |
| *** ruhe_away is now known as ruhe | 18:48 | |
| tmckay | aignatov, everything I see in Pig Oozie tutorials points that way (and from what I can find in grepcode on the oozie pig launcher). "params" are turned into additional args and then passed to Pig. I think "arguments" should not be named. I'll play, just wanted to give a heads up. | 18:49 |
| *** NikitaKonovalov has joined #savanna | 18:50 | |
| tmckay | our edp-pig example does not use arguments :) only "params" generated from data sources. | 18:50 |
| *** openstackgerrit has quit IRC | 18:52 | |
| *** openstackgerrit has joined #savanna | 18:53 | |
| aignatov | so we should use arguments instead of params to put input and output to script, right? | 18:54 |
| aignatov | btw, the good page about pig workflow https://cwiki.apache.org/confluence/display/OOZIE/Pig+Cookbook | 18:54 |
| aignatov | I know Nadya used it when she worked on pig actions | 18:55 |
| *** NikitaKonovalov has quit IRC | 18:56 | |
| tmckay | aignatov, yes, I've seen that. well, both are okay. "params" is a shorthand, I think. They end up being turned into "-param name = value" and the whole thing is passed as an argument | 18:56 |
| tmckay | you could do the same thing by specifying "-param name = value" yourself in Savanna as an "argument". But, we force arguments to be a dictionary. :) You see? | 18:57 |
| tmckay | I'm suggesting we keep both, but "arguments" should in job_configs for Pig jobs should just be a list of strings | 18:58 |
| tmckay | going to play with this on my cluster and make sure I'm sane... | 18:59 |
| tmckay | all this from writing documentation, hehe | 18:59 |
| *** NikitaKonovalov has joined #savanna | 19:00 | |
| aignatov | tmckay: I see :) , it would be a good catch, if that is confirmed in your experiments | 19:01 |
| *** dmitryme has quit IRC | 19:02 | |
| *** SergeyLukjanov_ is now known as SergeyLukjanov | 19:03 | |
| aignatov | ruhe is now known as ruhe_away, ruhe_away is now known as ruhe, looks like Split personality | 19:04 |
| tmckay | aignatov, anyway, my missing status from the meeting -- finishing up edp.rst changes for java action and hdfs data source, then I'm going to work on the REST doc | 19:04 |
| *** NikitaKonovalov has quit IRC | 19:05 | |
| tmckay | aignatov, also, I will take my integration test out of draft when new savanna python client is available on savanna-ci | 19:05 |
| *** ErikB has quit IRC | 19:05 | |
| aignatov | tmckay: nice update, really! | 19:05 |
| tmckay | I mean the REST doc in reference to java actions and hdfs, not the changes mattf has suggested :) | 19:06 |
| tmckay | not yet anyway | 19:06 |
| *** NikitaKonovalov has joined #savanna | 19:07 | |
| SergeyLukjanov | tmckay, you can already use 0.4.1 | 19:08 |
| SergeyLukjanov | just by setting it in the requirements file | 19:08 |
| SergeyLukjanov | test-requirements.txt | 19:09 |
| SergeyLukjanov | we're installing savanna from scratch each it on savanna-ci | 19:09 |
| tmckay | SergeyLukjanov, ah, okay, thanks! I'll try that later. | 19:09 |
| SergeyLukjanov | so, it'll be used for your patch | 19:09 |
| *** NikitaKonovalov has quit IRC | 19:11 | |
| *** alazarev has quit IRC | 19:12 | |
| *** NikitaKonovalov has joined #savanna | 19:12 | |
| *** NikitaKonovalov has quit IRC | 19:16 | |
| *** IlyaE has joined #savanna | 19:17 | |
| aignatov | SergeyLukjanov: forget this news about savannaclient on the meeting :( | 19:17 |
| *** NikitaKonovalov has joined #savanna | 19:18 | |
| SergeyLukjanov | aignatov, np, I've already announced it in the ML | 19:18 |
| *** aignatov has left #savanna | 19:18 | |
| *** aignatov has joined #savanna | 19:18 | |
| SergeyLukjanov | and all guys how blocked by it are know that it's ready now | 19:18 |
| *** NikitaKonovalov has quit IRC | 19:22 | |
| *** NikitaKonovalov has joined #savanna | 19:23 | |
| *** NikitaKonovalov has quit IRC | 19:30 | |
| *** ErikB has joined #savanna | 19:32 | |
| *** NikitaKonovalov has joined #savanna | 19:35 | |
| *** NikitaKonovalov has quit IRC | 19:39 | |
| *** NikitaKonovalov has joined #savanna | 19:41 | |
| *** ErikB has quit IRC | 19:45 | |
| *** NikitaKonovalov has quit IRC | 19:52 | |
| *** qwerty_nor has quit IRC | 19:57 | |
| *** NikitaKonovalov has joined #savanna | 19:57 | |
| *** dmitryme has joined #savanna | 19:58 | |
| *** alazarev has joined #savanna | 19:58 | |
| *** NikitaKonovalov has quit IRC | 20:02 | |
| *** NikitaKonovalov has joined #savanna | 20:02 | |
| *** NikitaKonovalov has quit IRC | 20:19 | |
| *** NikitaKonovalov has joined #savanna | 20:24 | |
| *** alazarev has quit IRC | 20:31 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack/savanna: Add missed i18n configs to setup.cfg https://review.openstack.org/67290 | 20:41 |
| SergeyLukjanov | aignatov, mattf, please take a look on it asap | 20:42 |
| *** dmitryme has quit IRC | 20:53 | |
| aignatov | SergeyLukjanov, done, why is so asap? | 20:58 |
| SergeyLukjanov | aignatov, our i18n jobs are failing | 20:58 |
| SergeyLukjanov | for several month :( | 20:59 |
| tmckay | aignatov, crobertsrh, so I was right about arguments for Pig. It is a list of strings. <param>n=v</param> is a shorthand for <argument>-param</argument> <argument>n=v</argument> | 20:59 |
| tmckay | http://fpaste.org/69099/13899058/ | 20:59 |
| tmckay | from the oozie logs ^^ | 20:59 |
| tmckay | The "arguments" are essentially for (expert) users to modify Pig execution with Pig options | 21:00 |
| tmckay | I'll make a bug. | 21:01 |
| aignatov | tmckay: good investigation! :) | 21:04 |
| aignatov | btw, mattf, I'm waiting your review on https://review.openstack.org/#/c/64210 and all dependent patches :) not asap but still | 21:09 |
| tmckay | aignatov, thanks. so it's wrong, but apparently it doesn't break anything. http://fpaste.org/69108/99066921/. Looks like Pig ignores the arguments (we put all our args last in Savanna). I ran this one manually but I'll verify from Savanna. | 21:14 |
| tmckay | so the question is, whether to fix it now. The only thing lost with it broken is the ability to set specific Pig -options | 21:14 |
| tmckay | In the fpaste above, I added "Bob=Bill" | 21:15 |
| *** NikitaKonovalov has quit IRC | 21:16 | |
| aignatov | tmckay, I think we should fix it if it breaks something or wrong... not sure if this issue has high priority but should be fixed, imho | 21:20 |
| tmckay | verified, setting bogus arguments from Savanna does not break anything -- the job will still run. | 21:21 |
| tmckay | aignatov, agreed, it should be fixed. Not sure how hard it is for crobertsrh to change from dict to [], and savanna-api needs changes.... | 21:21 |
| *** qwerty_nor has joined #savanna | 21:22 | |
| tmckay | aignatov, 3 work days before the freeze :) If the freeze is end of the day on the 22nd, then it's 4. | 21:22 |
| tmckay | if it doesn't get fixed, it's not a disaster | 21:23 |
| aignatov | tmckay: no way :), not in this milestone | 21:23 |
| tmckay | lol, that's what I was wondering. Okay, no pressure then. That's why I checked whether or not it broke the job. | 21:24 |
| * tmckay makes coffee | 21:24 | |
| aignatov | btw, were you able to run integration tests successfully? | 21:25 |
| * aignatov yawns | 21:26 | |
| *** mattf is now known as _mattf | 21:29 | |
| *** jdev789 has joined #savanna | 21:31 | |
| tmckay | aignatov, didn't try yet. It took a while to run all these Oozie jobs :) | 21:36 |
| tmckay | crobertsrh, found another related issue. If you hit the "add" button for arguments on a Pig job but leave name/value blank, you get <argument>=</argument> in the xml. | 21:43 |
| tmckay | Not sure if that's the UIs fault for passing empty name/value pairs, or the api's fault for not ignoring them | 21:44 |
| tmckay | {u'output_id': u'e90b7980-b7cf-4876-9dc2-4c169ff2d319', u'input_id': u'ca9f8220-4e65-43e3-94ee-cbb77625dc30', u'job_configs': {u'configs': {}, u'args': {u'': u''}, u'params': {}}, u'cluster_id': u'302ba506-5fc7-4728-b7b1-1abf27d8175f'} | 21:46 |
| *** alazarev has joined #savanna | 21:58 | |
| aignatov | tmckay: ouch, one more EDP bug... | 21:59 |
| *** aignatov has quit IRC | 21:59 | |
| tmckay | yep :) | 22:00 |
| *** aignatov has joined #savanna | 22:00 | |
| *** aignatov has quit IRC | 22:01 | |
| *** aignatov has joined #savanna | 22:01 | |
| crobertsrh | tmckay: ack. Should be easy to fix. | 22:04 |
| *** aignatov_ has quit IRC | 22:04 | |
| *** ruhe is now known as _ruhe | 22:06 | |
| *** SergeyLukjanov is now known as SergeyLukjanov_a | 22:09 | |
| *** SergeyLukjanov_a is now known as SergeyLukjanov_ | 22:10 | |
| *** aignatov has quit IRC | 22:10 | |
| openstackgerrit | A change was merged to openstack/savanna-extra: [hadoop-openstack] Added debug log on getObject method https://review.openstack.org/67231 | 22:17 |
| *** crobertsrh is now known as _crobertsrh | 22:30 | |
| *** sreshetnyak has quit IRC | 22:31 | |
| *** tmckay has left #savanna | 22:31 | |
| *** _mattf is now known as mattf | 22:33 | |
| *** qwerty_nor has quit IRC | 22:37 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!