| tmckay | looking | 00:15 |
|---|---|---|
| *** melaks has quit IRC | 00:18 | |
| *** japplewhite has joined #openstack-sahara | 00:26 | |
| *** DWfuturetec has quit IRC | 00:47 | |
| *** hdd has joined #openstack-sahara | 01:16 | |
| *** melaks has joined #openstack-sahara | 01:21 | |
| openstackgerrit | Merged openstack/sahara: Add script to report uncovered new lines https://review.openstack.org/211238 | 01:27 |
| openstackgerrit | Merged openstack/sahara: Make infra engine configurable in devstack plugin https://review.openstack.org/213545 | 01:27 |
| openstackgerrit | Merged openstack/sahara: Make tools/pretty_tox.sh more informative and reliable https://review.openstack.org/214080 | 01:27 |
| openstackgerrit | Merged openstack/sahara: Set missed admin user parameters used for trusts creation in devstack https://review.openstack.org/214094 | 01:27 |
| *** melaks has quit IRC | 01:35 | |
| *** melaks has joined #openstack-sahara | 01:36 | |
| *** melaks has left #openstack-sahara | 01:38 | |
| *** hdd has quit IRC | 01:44 | |
| *** hdd has joined #openstack-sahara | 01:45 | |
| *** melaks has joined #openstack-sahara | 01:55 | |
| *** witlessb has joined #openstack-sahara | 01:56 | |
| *** witlessb has quit IRC | 02:01 | |
| *** japplewhite has quit IRC | 02:04 | |
| *** hdd has quit IRC | 02:19 | |
| *** japplewhite has joined #openstack-sahara | 02:27 | |
| *** melaks has quit IRC | 02:29 | |
| *** melaks has joined #openstack-sahara | 02:29 | |
| *** hdd has joined #openstack-sahara | 02:35 | |
| *** melaks has quit IRC | 02:38 | |
| *** Poornima has joined #openstack-sahara | 02:40 | |
| *** melaks has joined #openstack-sahara | 02:50 | |
| *** egafford has joined #openstack-sahara | 02:54 | |
| *** egafford has quit IRC | 02:58 | |
| *** tiny-hands has quit IRC | 02:59 | |
| *** tiny-hands has joined #openstack-sahara | 03:16 | |
| *** rodrigods has quit IRC | 03:16 | |
| *** tellesnobrega has quit IRC | 03:17 | |
| *** melaks has quit IRC | 03:18 | |
| *** rodrigods has joined #openstack-sahara | 03:19 | |
| *** tellesnobrega has joined #openstack-sahara | 03:20 | |
| *** melaks has joined #openstack-sahara | 03:22 | |
| *** tellesnobrega has quit IRC | 03:28 | |
| *** tellesnobrega has joined #openstack-sahara | 03:30 | |
| *** logan2 has quit IRC | 03:31 | |
| *** japplewhite has quit IRC | 03:55 | |
| *** melaks has quit IRC | 03:57 | |
| *** tellesnobrega has quit IRC | 04:19 | |
| *** melaks has joined #openstack-sahara | 04:21 | |
| *** melaks has quit IRC | 04:27 | |
| *** tellesnobrega has joined #openstack-sahara | 04:28 | |
| *** coolsvap|away is now known as coolsvap | 04:46 | |
| *** coolsvap is now known as coolsvap|away | 05:08 | |
| *** ashishb has joined #openstack-sahara | 05:20 | |
| openstackgerrit | lu huichun proposed openstack/python-saharaclient: Add job execution update api https://review.openstack.org/208001 | 05:20 |
| *** coolsvap|away is now known as coolsvap | 05:23 | |
| *** vgridnev has joined #openstack-sahara | 05:24 | |
| *** sgotliv_ has joined #openstack-sahara | 05:29 | |
| *** vgridnev has quit IRC | 05:38 | |
| *** hdd has quit IRC | 05:39 | |
| *** macjack has joined #openstack-sahara | 05:47 | |
| *** sgotliv_ has quit IRC | 05:52 | |
| *** hdd has joined #openstack-sahara | 06:06 | |
| *** coolsvap is now known as coolsvap|away | 06:10 | |
| *** vgridnev has joined #openstack-sahara | 06:22 | |
| *** esikachev has joined #openstack-sahara | 06:31 | |
| *** hdd has quit IRC | 06:35 | |
| *** nkrinner has joined #openstack-sahara | 06:39 | |
| *** stanchan has joined #openstack-sahara | 06:45 | |
| *** esikachev has quit IRC | 06:50 | |
| *** esikachev has joined #openstack-sahara | 07:03 | |
| *** vgridnev has quit IRC | 07:07 | |
| *** Nikolay_St has joined #openstack-sahara | 07:11 | |
| *** esikachev has quit IRC | 07:18 | |
| *** Poornima has quit IRC | 07:24 | |
| *** Poornima has joined #openstack-sahara | 07:42 | |
| *** witlessb has joined #openstack-sahara | 07:47 | |
| *** esikachev has joined #openstack-sahara | 07:47 | |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding Grenade support for Sahara https://review.openstack.org/193101 | 07:52 |
| openstackgerrit | Yanis Guenane proposed openstack/puppet-sahara: Reflect provider change in puppet-openstacklib https://review.openstack.org/209877 | 08:05 |
| *** macjack has quit IRC | 08:13 | |
| *** macjack has joined #openstack-sahara | 08:25 | |
| *** vgridnev has joined #openstack-sahara | 08:29 | |
| openstackgerrit | Yanis Guenane proposed openstack/puppet-sahara: Reflect provider change in puppet-openstacklib https://review.openstack.org/209877 | 08:38 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements https://review.openstack.org/212281 | 08:39 |
| openstackgerrit | lu huichun proposed openstack/sahara: [EDP] Add scheduling EDP jobs in sahara(oozie engine implementation) https://review.openstack.org/182310 | 08:45 |
| *** sgotliv has joined #openstack-sahara | 08:57 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara-image-elements: {WIP} Try fix centos7 building image https://review.openstack.org/209951 | 08:59 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara-image-elements: {WIP} Try fix centos7 building image https://review.openstack.org/209951 | 09:01 |
| *** stanchan has quit IRC | 09:02 | |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara: Remove libevent installation from MapR plugin https://review.openstack.org/214520 | 09:08 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add support of SSL in scenario tests https://review.openstack.org/212476 | 09:11 |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara-image-elements: Remove libevent installation from MapR plugin https://review.openstack.org/214526 | 09:17 |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding Grenade support for Sahara https://review.openstack.org/193101 | 09:17 |
| *** tosky has joined #openstack-sahara | 09:30 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Implement ability of creating flavor for scenario tests https://review.openstack.org/203567 | 09:33 |
| *** DWfuturetec has joined #openstack-sahara | 09:35 | |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding is_public and is_protected fields support https://review.openstack.org/195065 | 09:36 |
| *** macjack has quit IRC | 09:46 | |
| *** macjack has joined #openstack-sahara | 09:50 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/python-saharaclient: Fix backward compat issue for job interfaces https://review.openstack.org/214388 | 09:57 |
| *** DWfuturetec has quit IRC | 10:00 | |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara-image-elements: Added support of MapR 5.0.0 to sahara-image-elements https://review.openstack.org/207865 | 10:46 |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding Grenade support for Sahara https://review.openstack.org/193101 | 10:59 |
| openstackgerrit | Merged openstack/sahara: Add separated dir with fake plugin scenario for gate testing https://review.openstack.org/214216 | 11:00 |
| *** macjack has quit IRC | 11:11 | |
| *** vgridnev has quit IRC | 11:19 | |
| *** DWfuturetec has joined #openstack-sahara | 11:26 | |
| *** vgridnev has joined #openstack-sahara | 11:27 | |
| *** esikachev has quit IRC | 11:28 | |
| *** macjack has joined #openstack-sahara | 11:28 | |
| *** tiny-hands has quit IRC | 11:29 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara-specs: Add Wait Condition support to Heat Engine https://review.openstack.org/214568 | 11:37 |
| *** esikachev has joined #openstack-sahara | 11:41 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Add sample spark wordcount job https://review.openstack.org/207039 | 11:49 |
| openstackgerrit | Merged openstack/sahara: Run scenario tests for the fake plugin in gate https://review.openstack.org/213546 | 11:51 |
| openstackgerrit | lu huichun proposed openstack/sahara-specs: [EDP]enable recurrence edp job in sahara https://review.openstack.org/192097 | 11:59 |
| *** macjack has quit IRC | 12:13 | |
| *** tmckay has quit IRC | 12:14 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara-specs: Add Wait Condition support to Heat Engine https://review.openstack.org/214568 | 12:15 |
| *** ashishb has quit IRC | 12:24 | |
| *** chlong has joined #openstack-sahara | 12:26 | |
| *** egafford has joined #openstack-sahara | 12:29 | |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding clusters_update api call https://review.openstack.org/195024 | 12:38 |
| *** logan2 has joined #openstack-sahara | 12:39 | |
| *** ashishb has joined #openstack-sahara | 12:40 | |
| openstackgerrit | Merged openstack/sahara: Updated from global requirements https://review.openstack.org/212281 | 12:48 |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara-specs: Add Wait Condition support to Heat Engine https://review.openstack.org/214568 | 12:48 |
| *** _crobertsrh is now known as crobertsrh | 12:48 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara: Add wait condition for heat templates https://review.openstack.org/169338 | 12:51 |
| *** tosky has quit IRC | 12:57 | |
| *** tiny-hands has joined #openstack-sahara | 13:01 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Add sample spark wordcount job https://review.openstack.org/207039 | 13:02 |
| *** flaviosr has joined #openstack-sahara | 13:04 | |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding Grenade support for Sahara https://review.openstack.org/193101 | 13:06 |
| *** DWfuturetec has quit IRC | 13:10 | |
| *** tosky has joined #openstack-sahara | 13:16 | |
| openstackgerrit | lu huichun proposed openstack/sahara: [EDP] Add suspend_job() for sahara edp engine(oozie implementation) https://review.openstack.org/201448 | 13:19 |
| *** tmckay has joined #openstack-sahara | 13:23 | |
| *** flaviosr has quit IRC | 13:34 | |
| *** esikachev has quit IRC | 13:38 | |
| openstackgerrit | Merged openstack/sahara: add unit test cover oozie upload workflow file function https://review.openstack.org/207923 | 13:41 |
| *** sgotliv has quit IRC | 13:44 | |
| *** sgotliv has joined #openstack-sahara | 13:45 | |
| *** japplewhite has joined #openstack-sahara | 13:49 | |
| *** vgridnev has quit IRC | 14:00 | |
| openstackgerrit | Michael McCune proposed openstack/sahara: adding nova to session cache https://review.openstack.org/214205 | 14:09 |
| *** vgridnev has joined #openstack-sahara | 14:18 | |
| *** vgridnev has quit IRC | 14:19 | |
| *** hdd has joined #openstack-sahara | 14:27 | |
| *** vgridnev has joined #openstack-sahara | 14:32 | |
| elmiko | vgridnev: hey, i'm looking at https://bugs.launchpad.net/sahara/+bug/1472536 | 14:38 |
| openstack | Launchpad bug 1472536 in Sahara "Operating with outputs of heat stack can cause AttributeError" [Undecided,New] | 14:38 |
| elmiko | is it the case that a heatclient stack object can be created with an outputs? | 14:39 |
| elmiko | er, *without any outputs | 14:39 |
| vgridnev | looks like | 14:39 |
| vgridnev | not sure why | 14:39 |
| elmiko | yea, very weird. i'm looking at the heatclient code | 14:40 |
| vgridnev | Seems that cluster created, scaling started, heat stack goes to the state CREATE_COMPLETE, and after that we are trying to access to the outputs | 14:42 |
| elmiko | should be an easy fix for us though, just return early if outputs isn't there | 14:43 |
| elmiko | i'm making some notes on the bug | 14:43 |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding Grenade support for Sahara https://review.openstack.org/193101 | 14:44 |
| *** Poornima has quit IRC | 14:49 | |
| openstackgerrit | lu huichun proposed openstack/sahara: [EDP] Add suspend_job() for sahara edp engine(oozie implementation) https://review.openstack.org/201448 | 14:55 |
| vgridnev | elmiko, thanks | 14:58 |
| *** vgridnev has quit IRC | 14:58 | |
| tmckay | vgridnev, any idea why spark wordcount is failing sometimes in the gate? looks somewhat consistent | 14:59 |
| tmckay | I have a suggestion to help debug .... | 14:59 |
| *** vgridnev has joined #openstack-sahara | 14:59 | |
| tmckay | with spark, it is super easy to check on the job execution directory on the master node. I did it that way on purpose :) | 15:00 |
| tmckay | so, we can easily retrieve the launch_command.log file, which will have the whole spark trace | 15:00 |
| tmckay | vgridneve, oh, you left at just the right time to miss my comments! ^^ | 15:01 |
| *** logan2 has quit IRC | 15:01 | |
| tmckay | (10:59:25 AM) tmckay: vgridnev, any idea why spark wordcount is failing sometimes in the gate? looks somewhat consistent | 15:01 |
| tmckay | (10:59:34 AM) tmckay: I have a suggestion to help debug .... | 15:01 |
| tmckay | (10:59:46 AM) vgridnev [~vgridnev@91.207.132.76] entered the room. | 15:01 |
| *** vgridnev has quit IRC | 15:01 | |
| tmckay | vgridnev, so I'm wondering if it would help to add a debug level log entry that records the spark stack trace. It would be long, but maybe very helpful for gate failures | 15:02 |
| tmckay | vgridnev, this would be helpful for all job types, but for spark it's particularly easy | 15:02 |
| tmckay | oh, he left again! | 15:03 |
| *** logan2 has joined #openstack-sahara | 15:05 | |
| *** coolsvap|away is now known as coolsvap | 15:18 | |
| elmiko | tosky: do you have any further thoughts about https://bugs.launchpad.net/sahara/+bug/1484536 ? | 15:18 |
| openstack | Launchpad bug 1484536 in Sahara "Scenario tests fail with trace on creating datasources" [Undecided,New] - Assigned to Evgeny Sikachev (esikachev) | 15:18 |
| elmiko | can we confirm this as a bug | 15:18 |
| tosky | elmiko: I didn't see it; I have to say I didn't try Spark, but it was marked as buggy with all plugins, and I ran the tests on HDP 2.0.6 and CDH 5.3. on RHEL | 15:20 |
| elmiko | hmm | 15:21 |
| elmiko | thanks for checking, gonna have to figure out how to mark this one | 15:21 |
| *** vgridnev has joined #openstack-sahara | 15:21 | |
| *** ashishb has quit IRC | 15:22 | |
| *** coolsvap is now known as coolsvap|away | 15:22 | |
| *** melaks has joined #openstack-sahara | 15:22 | |
| vgridnev | tmckay, hey | 15:22 |
| tmckay | vgridnev, hey! maybe you can read back on eavesdrop | 15:23 |
| vgridnev | I am not sure why it's broken, really | 15:23 |
| tmckay | vgridnev, well, donewitherror means there is probably something in the trace on the master node, a spark error or exception | 15:23 |
| vgridnev | yep | 15:23 |
| tmckay | vgridnev, anyway to get to that master node after a test run? | 15:24 |
| tmckay | I think pulling the launch_command.log in debug mode on failure would be a good, easy thing to do. Lot of info, but hey | 15:24 |
| elmiko | tosky: is there some documentation about setting up the scenario tests? (like what i need for credentials.yaml, edp,yaml, ...) | 15:25 |
| tosky | elmiko: sahara/tests/scenario/README.rst | 15:26 |
| elmiko | tosky: ahh, thank you sir | 15:26 |
| elmiko | tosky: can i bug you more about this for a few minutes? | 15:30 |
| tosky | elmiko: sure! | 15:30 |
| elmiko | tosky: ok, so i setup a stack with sahara running, then create some credentials as per the README, and finally i can either create a custom yaml for the test or use the mako template with an ini file. then just let her rip? | 15:31 |
| tosky | elmiko: yes, and given that the bug says that it can be reproduced with the existing templates, go for them | 15:31 |
| elmiko | tosky: yea, was gonna try that. although my testing setup here is pathetically underpowered =( | 15:32 |
| vgridnev | tmckay, I will try to investigate what happened and will ping you later, is that possible? It's seems that job failed on CDH 540 also, and it's very strange, because it's was tested on it | 15:32 |
| tosky | elmiko: you need edp.yaml, credentials.yaml.mako, and one of the plugin-specific (hdpblabla.yaml.mako, cdhetc) | 15:32 |
| elmiko | tosky: ok, and i just setup an ini file with the values i want to fill in and provide it on the cli with my tox command then? | 15:33 |
| tosky | elmiko: yes | 15:33 |
| elmiko | tosky: wish me luck ;) | 15:33 |
| tmckay | vgridnev, sure. I may put up a patch for spark log retrieval anyway. Longterm I really want to add an error retrieval feature for all job types -- hunting around for logs in hadoop is not as easy, though | 15:34 |
| tmckay | but, it certainly would help a lot | 15:34 |
| tosky | elmiko: crossing fingers :) | 15:36 |
| elmiko | tosky: is `tox -e ~/etc/credentials.yaml ~/etc/edp.yaml ~/etc/spark-1.3.1.yaml` correct form? or should it have a `tox -e scenario ...`? | 15:36 |
| *** melaks has quit IRC | 15:37 | |
| *** vgridnev has quit IRC | 15:37 | |
| tosky | elmiko: for that case, from the sahara directory, I would use `tox -e scenario -- -V variables.ini etc/sahara-ci/scenario/credentials.yaml.mako etc/sahara-ci/scenario/edp.yaml etc/sahara-ci/scenario/spark-1.3.1.yaml.mako` | 15:39 |
| tosky | ups | 15:39 |
| tosky | elmiko: better `tox -e scenario -- -V variables.ini etc/scenario/sahara-ci/credentials.yaml.mako etc/scenario/sahara-ci/edp.yaml etc/scenario/sahara-ci/spark-1.3.1.yaml.mako` | 15:39 |
| tosky | but you get it | 15:39 |
| elmiko | tosky: yea, that makes sense | 15:40 |
| elmiko | thanks | 15:40 |
| *** nkrinner has quit IRC | 15:41 | |
| *** sanjayu_ has joined #openstack-sahara | 15:43 | |
| *** sanjayu has quit IRC | 15:48 | |
| elmiko | tosky: i take it for the image, i need to have registered something with sahara before i start the tests? | 15:52 |
| tosky | elmiko: yes, also with proper tags | 15:53 |
| elmiko | tosky: ack | 15:53 |
| tosky | this is something that could be optionally automated in fact | 15:53 |
| *** DWfuturetec has joined #openstack-sahara | 15:53 | |
| elmiko | gonna try this spark 1.3.1 first | 15:53 |
| *** Nikolay_St has quit IRC | 15:55 | |
| openstackgerrit | Merged openstack/sahara-image-elements: Updated from global requirements https://review.openstack.org/206725 | 15:58 |
| tmckay | elmiko, let me know how that goes. I think I'd like to be able to run the scenario tests too | 15:59 |
| elmiko | tmckay: ack, i'm still setting up the stack | 16:00 |
| elmiko | tmckay: also, i think you'll like something in here. you can specify networks by id *or* name ;) | 16:00 |
| tmckay | nicr | 16:00 |
| tmckay | nice even | 16:00 |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara: Add wait condition for heat templates https://review.openstack.org/169338 | 16:01 |
| *** flaviosr has joined #openstack-sahara | 16:07 | |
| *** melaks has joined #openstack-sahara | 16:20 | |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding Grenade support for Sahara https://review.openstack.org/193101 | 16:24 |
| openstackgerrit | lu huichun proposed openstack/python-saharaclient: Add job execution update api https://review.openstack.org/208001 | 16:27 |
| *** macjack has joined #openstack-sahara | 16:31 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara: Add wait condition for heat templates https://review.openstack.org/169338 | 16:41 |
| *** vgridnev has joined #openstack-sahara | 16:54 | |
| *** egafford has quit IRC | 16:59 | |
| *** Nikolay_St has joined #openstack-sahara | 16:59 | |
| openstackgerrit | Merged openstack/sahara: Adding clusters_update api call https://review.openstack.org/195024 | 17:01 |
| *** sanjayu has joined #openstack-sahara | 17:05 | |
| *** sanjayu_ has quit IRC | 17:08 | |
| *** sgotliv has quit IRC | 17:09 | |
| *** melaks has quit IRC | 17:15 | |
| *** melaks has joined #openstack-sahara | 17:16 | |
| *** saneax has joined #openstack-sahara | 17:19 | |
| *** tosky has quit IRC | 17:24 | |
| *** DWfuturetec has quit IRC | 17:28 | |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding is_public and is_protected fields support https://review.openstack.org/195065 | 17:33 |
| *** witlessb has quit IRC | 17:44 | |
| *** witlessb has joined #openstack-sahara | 17:45 | |
| tmckay | vgridnev, my working theory is that the job launch wrapper for spark in 1.3.1 is not able to set the config | 17:51 |
| tmckay | I ran the wordcount job, and I cannot get it to take the config for the swift username/password | 17:51 |
| tmckay | this is what I'm seeing. trying to debug the launcher, might try against spark 1.0.0 to verify | 17:52 |
| *** tosky has joined #openstack-sahara | 17:52 | |
| tmckay | error I get on the node is that fs.service.swift.sahara.username is missing | 17:52 |
| *** egafford has joined #openstack-sahara | 17:56 | |
| *** flaviosr has quit IRC | 17:56 | |
| vgridnev | tmckay, yep, I got the same | 17:57 |
| vgridnev | but | 17:57 |
| *** DWfuturetec has joined #openstack-sahara | 17:58 | |
| vgridnev | It was working at CDH5.4.0 a while ago | 17:58 |
| vgridnev | but know it's seems to be failed on CI | 17:58 |
| tmckay | vgridnev, is that spark 1.3.1 though? | 17:59 |
| tmckay | maybe some slightly different jar in the image? | 17:59 |
| vgridnev | maybe, I did not check that | 18:00 |
| tmckay | vgridnev, well, good validation that we are on the same path | 18:00 |
| elmiko | tosky: question about my ini file, do all the values belong in any specific section? and do i just use the form "OS_USERNAME=admin"? | 18:00 |
| tosky | elmiko: [DEFAULTS] section, then key: value (or key=value) | 18:01 |
| elmiko | tosky: awesome, thanks | 18:01 |
| vgridnev | tmckay, have you checked job on spark 1.0.0 ? | 18:02 |
| vgridnev | oh, I see above | 18:02 |
| tosky | elmiko: but good point, it should be added to the README | 18:02 |
| elmiko | tosky: +1 | 18:03 |
| elmiko | a little sample would be great | 18:03 |
| tosky | elmiko: empf, DEFAULT | 18:04 |
| tosky | it's the standard INI format from python libraries | 18:04 |
| elmiko | ack | 18:06 |
| tmckay | vgridnev, closer to verification. I just set user/pass explicitly in a scala app (sc.hadoopConfiguration.set()) and it worked | 18:10 |
| tmckay | so, spark 1.3.1 can indeed reference swift | 18:10 |
| tmckay | I'll add more print lines :) | 18:10 |
| tmckay | vgridnev, could be that interface to addResource changed -- it actually was marked deprecated if I remember correctly, at some point ... | 18:11 |
| tmckay | but was still included in hadoop, so ... | 18:11 |
| tmckay | vgridneve, spark wrapper was adapted from the oozie wrapper which handles exit conditions, etc | 18:12 |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding Grenade support for Sahara https://review.openstack.org/193101 | 18:14 |
| elmiko | tosky: is _ not allowed in string names? | 18:23 |
| tosky | elmiko: key or value? It should work | 18:23 |
| tosky | elmiko: I'm testing a doc patch, you can see the preview here: http://paste.openstack.org/show/422088/ | 18:25 |
| elmiko | tosky: very nice | 18:26 |
| elmiko | you've got my +1 ;) | 18:27 |
| elmiko | tosky: here's what i'm seeing http://paste.openstack.org/show/422103/ | 18:30 |
| *** bapalm has quit IRC | 18:31 | |
| *** bapalm has joined #openstack-sahara | 18:32 | |
| elmiko | changed name to spark1 and it's working | 18:34 |
| elmiko | well, it passed validation... | 18:35 |
| elmiko | tosky: this is pretty cool =) | 18:43 |
| tosky | elmiko: oh, because it's should be a hostname | 18:43 |
| elmiko | ah, ok | 18:43 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara: Doc, scenario tests: variables config file https://review.openstack.org/214750 | 18:48 |
| tosky | elmiko: ^ | 18:48 |
| elmiko | ack, thanks tosky ! | 18:49 |
| elmiko | tosky: one minor error | 18:50 |
| tosky | elmiko: sure | 18:51 |
| elmiko | tmckay: scenario stuff is dead easy, mostly described in the README | 18:52 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara: Doc, scenario tests: variables config file https://review.openstack.org/214750 | 18:53 |
| elmiko | thanks tosky | 18:55 |
| *** melaks has quit IRC | 19:12 | |
| elmiko | tosky: do you know if i can send arbitrary configs from these scenario templates? (i'd like to turn off ntp) | 19:28 |
| tosky | elmiko: uhm, not at that level | 19:31 |
| elmiko | k | 19:31 |
| elmiko | also, feel free to tell me to bugger off. i know it's late over there ;) | 19:31 |
| tmckay | elmiko, good to know. Still poking at this spark thing. Going to try to prove that using the exact same launch files from spark 1.3.1 where it fails, it passes on spark 1.0.0 | 19:57 |
| *** saneax has quit IRC | 19:57 | |
| tmckay | seems to be the case that the stuff in the wrapper to add default resources to the hadoop config path fails .... | 19:57 |
| tmckay | but I haven't proved it 100% yet | 19:57 |
| elmiko | ooph | 20:03 |
| elmiko | good luck tmckay | 20:03 |
| elmiko | i'm being distracted by networking issues coming out of my devstack. instances can't reach ntp =( | 20:03 |
| tmckay | yeah, I feel somewhat responsible because I worked on the spark/swift stuff | 20:03 |
| elmiko | nothing like responsibility to motivate ;) | 20:04 |
| vgridnev | elmiko, can you disable ntp on your clusters? | 20:06 |
| elmiko | vgridnev: i'm not sure how to do that through the scenario tests | 20:13 |
| vgridnev | elmiko, I thinks it's can be done like there: https://github.com/openstack/sahara/blob/master/etc/scenario/sahara-ci/spark-1.3.1.yaml.mako#L23 | 20:19 |
| elmiko | vgridnev: interesting... i'll give that a try | 20:20 |
| *** melaks has joined #openstack-sahara | 20:20 | |
| elmiko | vgridnev: so i would add "Enable NTP Service: False" ? | 20:21 |
| vgridnev | instead of HDFS should general, instead of replication should be Enable NTP Service: False | 20:21 |
| elmiko | ok | 20:21 |
| vgridnev | yep | 20:21 |
| elmiko | but in a "general:" section | 20:21 |
| elmiko | right? | 20:21 |
| vgridnev | maybe instead of False should be false, not sure | 20:21 |
| vgridnev | yep | 20:21 |
| elmiko | cool, thank you =) | 20:21 |
| vgridnev | Or just change temporary Enable NTP Service to False in code | 20:22 |
| elmiko | haha, yea could do that ;) | 20:23 |
| crobertsrh | UI patch for Manila data sources https://review.openstack.org/#/c/214754/ +1 early and often (or -1 if you wanna be that way) | 20:24 |
| elmiko | i could use a break from network madness | 20:25 |
| tosky | elmiko: internal instances? | 20:26 |
| crobertsrh | Still hosing you? | 20:26 |
| crobertsrh | Was this a fresh stack, or a stack that started misbehaving? | 20:26 |
| elmiko | tosky: yea, i have a vm running devstack. and the bridge it's creating won't allow traffic out | 20:26 |
| tosky | elmiko: any traffic or just ntp? | 20:26 |
| elmiko | any traffic | 20:26 |
| tosky | ah | 20:27 |
| elmiko | and i don't see an iptables rule that is stopping it | 20:27 |
| tosky | is it all traffic, or just a DNS issue? | 20:27 |
| elmiko | i can't even ping | 20:27 |
| elmiko | i'm getting admin prohibited trying to hit other machines on the network of the host | 20:27 |
| elmiko | so, like the host is on 192.168.122.0, and the internal vms are on 10.1.0.0 | 20:28 |
| elmiko | and i can't ping out to 192.168.122.3 (for example) from 10.1.0.2 | 20:28 |
| tosky | uhm, that network is suspicious | 20:28 |
| elmiko | i get a report back from 10.1.0.1 (vrouter) that it is admin prohibited | 20:28 |
| elmiko | which one? | 20:29 |
| *** melaks has quit IRC | 20:30 | |
| vgridnev | crobertsrh, have you participated in horizon-drivers meeting? | 20:36 |
| crobertsrh | No, I said hi, but I got distracted | 20:37 |
| vgridnev | they are approving bp-s on that; so that can you useful for you | 20:38 |
| *** melaks has joined #openstack-sahara | 20:38 | |
| *** melaks has quit IRC | 20:39 | |
| vgridnev | btw, it's safer to get several +1's on changes, and after that ask david lyle to approve and merge | 20:41 |
| *** melaks has joined #openstack-sahara | 20:41 | |
| tosky | crobertsrh, vgridnev: partially related: they are working on fixing the integration tests, a possibility would be moving horizon/sahara tests in a separate job (just for Sahara), but anyway they should receive some love, so please take a look at them too | 20:43 |
| tosky | and if you want to contribute some test, even better | 20:44 |
| tosky | :) | 20:44 |
| crobertsrh | Yes, integration tests have been on my "todo" list for awhile | 20:44 |
| crobertsrh | vgridnev: yes, I try to stay on top of our horizon reviews. It looks like you do as well. Is nkonovalov still doing horizon stuff? If so, I'll add him to my recent reviews. | 20:49 |
| vgridnev | crobertsrh, I think he'll continue doing several reviews in horizon. | 20:54 |
| crobertsrh | Great, I added him. | 20:54 |
| vgridnev | not sure about features, bugfixes. Maybe I will do that from Mirantis side instead of nkonovalov | 20:55 |
| vgridnev | crobertsrh, why we put auth token to debug logs? https://review.openstack.org/#/c/207086/7/openstack_dashboard/contrib/sahara/api/manila.py,cm , line 55 | 21:03 |
| crobertsrh | vgridnev: I just grabbed that client code from another place. I should clean it up a bit. | 21:04 |
| elmiko | good catch, vgridnev | 21:04 |
| crobertsrh | We are currently only using share_list. I can see us using a few other operations at some point though. | 21:05 |
| crobertsrh | certainly not share_delete though. | 21:05 |
| crobertsrh | Ok, I updated that patch. Thanks guys. | 21:19 |
| *** crobertsrh is now known as _crobertsrh | 21:19 | |
| *** tmckay has quit IRC | 21:34 | |
| *** vgridnev has quit IRC | 21:35 | |
| *** japplewhite has quit IRC | 21:46 | |
| *** sgotliv has joined #openstack-sahara | 21:51 | |
| *** tiny-hands has quit IRC | 22:07 | |
| *** sgotliv has quit IRC | 22:13 | |
| *** melaks has quit IRC | 22:37 | |
| *** melaks has joined #openstack-sahara | 22:38 | |
| *** saneax has joined #openstack-sahara | 22:45 | |
| *** saneax has quit IRC | 22:52 | |
| *** melaks has quit IRC | 22:56 | |
| *** melaks has joined #openstack-sahara | 22:56 | |
| *** saneax has joined #openstack-sahara | 23:00 | |
| *** DWfuturetec has quit IRC | 23:09 | |
| *** tosky has quit IRC | 23:11 | |
| *** chlong has quit IRC | 23:28 | |
| *** egafford has quit IRC | 23:29 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!