| *** stanchan has quit IRC | 01:39 | |
| *** chlong has quit IRC | 01:40 | |
| *** chlong has joined #openstack-sahara | 01:53 | |
| *** sgotliv__ has joined #openstack-sahara | 03:12 | |
| *** sgotliv__ has quit IRC | 03:18 | |
| *** itisha has joined #openstack-sahara | 03:52 | |
| *** sgotliv__ has joined #openstack-sahara | 04:21 | |
| *** _mattf` has quit IRC | 04:25 | |
| *** links has joined #openstack-sahara | 04:48 | |
| *** stanchan has joined #openstack-sahara | 05:01 | |
| *** nkrinner_afk is now known as nkrinner | 05:34 | |
| *** itisha has quit IRC | 06:00 | |
| *** pcaruana has joined #openstack-sahara | 06:05 | |
| *** rcernin has joined #openstack-sahara | 06:09 | |
| -openstackstatus- NOTICE: All python 3.5 jobs are failing today, we need to build new xenial images first. | 06:28 | |
| *** abalutoiu has joined #openstack-sahara | 07:17 | |
| *** tesseract- has joined #openstack-sahara | 07:21 | |
| *** akuznetsov has joined #openstack-sahara | 07:31 | |
| *** houming has joined #openstack-sahara | 07:33 | |
| *** houming has quit IRC | 07:38 | |
| *** akuznetsov has quit IRC | 08:07 | |
| *** mionkin has joined #openstack-sahara | 09:00 | |
| *** esikachev has joined #openstack-sahara | 09:24 | |
| *** links has quit IRC | 10:00 | |
| openstackgerrit | Merged openstack/sahara-image-elements: [HDP2.4] Add ambari 2.2.1.0 element https://review.openstack.org/312905 | 10:06 |
|---|---|---|
| openstackgerrit | Merged openstack/sahara-image-elements: Fix Mapr security repo https://review.openstack.org/336908 | 10:11 |
| openstackgerrit | Michael Ionkin proposed openstack/sahara-image-elements: [CDH] Support CDH 5.7 https://review.openstack.org/326035 | 10:13 |
| openstackgerrit | Merged openstack/sahara-image-elements: Allow creation of Storm 1.0.1 images https://review.openstack.org/335045 | 10:29 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-tests: Fix for uploading image in sahara_cli tests https://review.openstack.org/337101 | 10:31 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-tests: Fix for uploading image in sahara_cli tests https://review.openstack.org/337101 | 10:41 |
| openstackgerrit | Merged openstack/sahara: Fix provision events for installing services https://review.openstack.org/311110 | 10:56 |
| openstackgerrit | Merged openstack/sahara: [DOC] improve docs https://review.openstack.org/335382 | 10:56 |
| openstackgerrit | Merged openstack/sahara: Ignore Nova config drive in devices list https://review.openstack.org/325220 | 10:56 |
| openstackgerrit | Merged openstack/sahara: Fix glanceclient.v2.images https://review.openstack.org/334569 | 10:57 |
| openstackgerrit | Merged openstack/sahara: Add Impala 2.2 to MapR plugin https://review.openstack.org/315502 | 11:00 |
| openstackgerrit | Merged openstack/sahara: plugins api impl https://review.openstack.org/333374 | 11:08 |
| *** links has joined #openstack-sahara | 11:24 | |
| *** esikachev has quit IRC | 11:36 | |
| *** rodrigods has quit IRC | 11:41 | |
| *** rodrigods has joined #openstack-sahara | 11:42 | |
| *** _mattf has joined #openstack-sahara | 11:44 | |
| openstackgerrit | Alina Nesterova proposed openstack/sahara-tests: Added tests for "cluster template" and "cluster" commands for Sahara CLI https://review.openstack.org/335546 | 11:45 |
| *** esikachev has joined #openstack-sahara | 11:47 | |
| *** esikachev has quit IRC | 12:00 | |
| *** permalac has joined #openstack-sahara | 12:02 | |
| *** vgridnev has quit IRC | 12:10 | |
| *** vgridnev has joined #openstack-sahara | 12:22 | |
| vgridnev | egafford, have you saw email from Nikita? | 12:24 |
| vgridnev | just to be sure : ) | 12:24 |
| egafford | vgridnev: I did, yes. Will respond today, likely. :) | 12:25 |
| vgridnev | cool | 12:25 |
| vgridnev | egafford, about recipes, I think that I can help with setting up recipes, and with CI tests also, if needed | 12:26 |
| egafford | That'd be excellent! I'm writing unit tests, and then the basic mechanism will be complete for the CLI. It's possible tellesnobrega_af may have some cycles for recipes too after his Storm work. | 12:28 |
| openstackgerrit | Merged openstack/sahara: ability to configure endpoint type for services https://review.openstack.org/316173 | 12:30 |
| *** _crobertsrh is now known as crobertsrh | 12:38 | |
| *** esikachev has joined #openstack-sahara | 12:39 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara-image-elements: drop vanilla 2.6.0 support from elements https://review.openstack.org/338235 | 13:00 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-tests: Fix for uploading image in sahara_cli tests https://review.openstack.org/337101 | 13:01 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara-image-elements: drop vanilla 2.6.0 support from elements https://review.openstack.org/338235 | 13:02 |
| *** nkrinner is now known as nkrinner_afk | 13:03 | |
| openstackgerrit | Mikhail Lelyakin proposed openstack/sahara-specs: Implement pagination abilities https://review.openstack.org/336931 | 13:25 |
| *** tmckay has joined #openstack-sahara | 13:28 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-tests: Fix for uploading image in sahara_cli tests https://review.openstack.org/337101 | 13:28 |
| *** esikachev has quit IRC | 13:36 | |
| *** tellesnobrega_af is now known as tellesnobrega | 13:36 | |
| *** tenobreg__ has joined #openstack-sahara | 13:36 | |
| *** tenobreg__ has quit IRC | 13:36 | |
| vgridnev | tmckay, are you going to join the next meeting? | 13:37 |
| openstackgerrit | Mikhail Lelyakin proposed openstack/sahara-specs: Implement pagination abilities https://review.openstack.org/336931 | 13:39 |
| *** itisha has joined #openstack-sahara | 13:39 | |
| tmckay | vgridnev, tomorrow, correct? let me check | 13:53 |
| tmckay | vgridnev, looks like I'm free | 13:54 |
| vgridnev | tmckay, so we can just put your blueprint to discuss | 13:56 |
| vgridnev | now we are discussing specs on meetings | 13:56 |
| *** esikachev has joined #openstack-sahara | 13:57 | |
| *** links has quit IRC | 13:57 | |
| tmckay | vgridnev, absolutely, great idea | 13:57 |
| tmckay | I'll try to respond to comments today | 13:57 |
| openstackgerrit | Alina Nesterova proposed openstack/sahara-tests: Added tests for "cluster template" and "cluster" commands for Sahara CLI https://review.openstack.org/335546 | 14:04 |
| openstackgerrit | Alina Nesterova proposed openstack/sahara-tests: Added tests for "cluster template" and "cluster" commands for Sahara CLI https://review.openstack.org/335546 | 14:10 |
| *** kur1j has joined #openstack-sahara | 14:26 | |
| kur1j | I'm getting py4j.protocol.Py4JJavaError: An error occurred while calling o23.partitions. : org.apache.hadoop.fs.swift.exceptions.SwiftConfigurationException: Missing mandatory configuration option: fs.swift.service.sahara.username even though the documentation says it should be passing that information automatically | 14:33 |
| kur1j | http://docs.openstack.org/developer/sahara/kilo/userdoc/edp.html#using-data-source-references-as-arguments | 14:34 |
| openstackgerrit | Maria Malyarova proposed openstack/sahara: The addition of the parentheses for py3 https://review.openstack.org/338296 | 14:44 |
| vgridnev | kur1j, you are running spark job? | 14:45 |
| kur1j | yup | 14:45 |
| vgridnev | could you please check that spark.xml contains these values | 14:46 |
| vgridnev | ? | 14:46 |
| kur1j | vgridnev: it doesn't contain anything | 14:46 |
| kur1j | https://bugs.launchpad.net/sahara/+bug/1486544 I see that bug but it says its fixed | 14:46 |
| openstack | Launchpad bug 1486544 in Sahara "[Spark1.3] Spark jobs doesn't work with Swift" [Critical,Fix released] - Assigned to Trevor McKay (tmckay) | 14:46 |
| kur1j | but same thing | 14:46 |
| kur1j | I just tried it with HDFS and it works | 14:47 |
| vgridnev | could you please provide details about your job execution, I just want to be sure everything is up | 14:48 |
| kur1j | what do you need to know? | 14:48 |
| kur1j | all the settings I have? | 14:48 |
| kur1j | for the job? | 14:48 |
| vgridnev | yup | 14:49 |
| vgridnev | as I remember, you should have 3 parameters at least | 14:49 |
| vgridnev | 2 fs.swift | 14:50 |
| vgridnev | and edp.spark.adapt_for_swift True | 14:50 |
| kur1j | Job: configure tab : main class = main, enable swift paths (checked), Use Data source Substitution for Names and UUIDs (checked), instance arguments tab: input_file, swift://WordCountData/4300.txt, output_path swift://WordcountData/output | 14:57 |
| vgridnev | uh | 14:58 |
| vgridnev | ok | 14:58 |
| kur1j | is that not what you wanted? | 14:58 |
| vgridnev | you need to put fs.swift.service.sahara.username and fs.swift.service.sahara.password to job configs | 14:59 |
| vgridnev | that is why them was not pushed to job execution | 14:59 |
| kur1j | do I have to read them in my program? | 15:00 |
| *** thumpba has joined #openstack-sahara | 15:00 | |
| vgridnev | this values will be used to access the input path | 15:02 |
| *** abalutoiu has quit IRC | 15:03 | |
| kur1j | vgridnev: so i added those two propertis to my config fs.swift.service.sahara.password and fs.swift.service.sahara.username they are there in the spark.xml but get same error | 15:03 |
| vgridnev | you putted valid username and password? | 15:03 |
| vgridnev | you can check that in spark.xml | 15:03 |
| kur1j | https://gist.github.com/vaskokj/ddc4aa780b682a1a4602cfae7a3d5222 | 15:04 |
| kur1j | thats it | 15:04 |
| vgridnev | hm, let me check | 15:05 |
| kur1j | This is command, /opt/spark/bin/spark-submit --driver-class-path /usr/lib/hadoop/hadoop-swift.jar:/tmp/spark-edp/Spark-WordCount/c1a92522-a410-4fd0-b3ed-2d4bb112b1d8 --files spark.xml --class org.openstack.sahara.edp.SparkWrapper --jars builtin-d1e4f0d7-6fbc-45f6-88f7-85f48ae2d8fd.jar --master spark://sparkclustertest-spark-master-0:7077 --deploy-mode client PySparkWordCount.py spark.xml main swift://WordCountData.sahara/430 | 15:05 |
| *** pcaruana has quit IRC | 15:06 | |
| vgridnev | is it ambari right? | 15:07 |
| kur1j | this is just spark | 15:07 |
| kur1j | default image | 15:07 |
| vgridnev | uh, strange, that should be working for sure | 15:07 |
| openstackgerrit | Maria Malyarova proposed openstack/sahara: The addition of the parentheses for py3 https://review.openstack.org/338296 | 15:08 |
| *** rcernin has quit IRC | 15:10 | |
| kur1j | 1.6.0 | 15:18 |
| vgridnev | could you please try to do the same job but with java based job-binary? | 15:20 |
| kur1j | sure, let me find oen | 15:20 |
| kur1j | one* | 15:21 |
| vgridnev | https://github.com/openstack/sahara-tests/tree/master/sahara_tests/scenario/defaults/edp-examples/edp-spark | 15:21 |
| vgridnev | we have your spark wordcount | 15:21 |
| vgridnev | at this place | 15:21 |
| vgridnev | sahara.edp.spark.SparkWordCount is the main class of job | 15:22 |
| *** esikachev has quit IRC | 15:22 | |
| kur1j | vgridnev: I got this error. https://gist.github.com/vaskokj/3e78c4fdcd6e742336cdbe5378aee3b8 | 15:33 |
| kur1j | the machine can ping and access public.fuel.local from the spark node | 15:34 |
| *** rcernin has joined #openstack-sahara | 15:49 | |
| *** abalutoiu has joined #openstack-sahara | 15:51 | |
| *** esikachev has joined #openstack-sahara | 16:05 | |
| kur1j | .. | 16:13 |
| *** ukaynar has joined #openstack-sahara | 16:27 | |
| *** amoralej has joined #openstack-sahara | 16:35 | |
| *** amoralej is now known as amoralej|off | 16:49 | |
| *** tesseract- has quit IRC | 16:49 | |
| openstackgerrit | Merged openstack/sahara: [DOC] Update installation guide https://review.openstack.org/336447 | 17:04 |
| *** ukaynar has quit IRC | 17:30 | |
| *** permalac has quit IRC | 18:09 | |
| *** pgadiya has joined #openstack-sahara | 18:19 | |
| *** pgadiya has quit IRC | 18:19 | |
| *** anshul has joined #openstack-sahara | 18:44 | |
| *** sgotliv__ has quit IRC | 18:45 | |
| openstackgerrit | Ethan Gafford proposed openstack/sahara: CLI for Plugin-Declared Image Declaration https://review.openstack.org/335669 | 18:55 |
| *** esumerfd has joined #openstack-sahara | 18:56 | |
| *** flaviodsr has quit IRC | 19:21 | |
| openstackgerrit | Merged openstack/sahara: The addition of the parentheses for py3 https://review.openstack.org/338296 | 19:50 |
| *** esikachev has quit IRC | 19:53 | |
| *** kur1j has quit IRC | 19:54 | |
| *** rcernin_ has joined #openstack-sahara | 20:06 | |
| *** rcernin_ has quit IRC | 20:10 | |
| openstackgerrit | Merged openstack/python-saharaclient: Remove unused functional tests https://review.openstack.org/333935 | 20:14 |
| *** rcernin_ has joined #openstack-sahara | 20:15 | |
| *** rcernin_ has quit IRC | 20:19 | |
| *** rcernin_ has joined #openstack-sahara | 20:19 | |
| *** rcernin_ has quit IRC | 20:20 | |
| *** rcernin has quit IRC | 20:24 | |
| *** rcernin has joined #openstack-sahara | 20:25 | |
| *** esikachev has joined #openstack-sahara | 20:35 | |
| *** rcernin has quit IRC | 20:38 | |
| *** rcernin has joined #openstack-sahara | 20:39 | |
| *** thumpba has quit IRC | 20:46 | |
| *** esikachev has quit IRC | 20:54 | |
| *** anshul has quit IRC | 20:58 | |
| *** rcernin has quit IRC | 21:03 | |
| *** rcernin has joined #openstack-sahara | 21:04 | |
| *** rcernin has quit IRC | 21:07 | |
| *** rcernin has joined #openstack-sahara | 21:07 | |
| *** tmckay has quit IRC | 21:21 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: fix building api ref docs https://review.openstack.org/338557 | 21:24 |
| *** esumerfd has quit IRC | 21:29 | |
| *** rcernin has quit IRC | 21:37 | |
| *** david-lyle_ has joined #openstack-sahara | 22:21 | |
| *** david-lyle_ is now known as david-lyle | 22:35 | |
| openstackgerrit | Merged openstack/sahara: fix building api ref docs https://review.openstack.org/338557 | 23:13 |
| *** itisha has quit IRC | 23:40 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!