| *** Longgeek has joined #openstack-sahara | 00:01 | |
| *** IlyaE has quit IRC | 00:01 | |
| *** mahito has joined #openstack-sahara | 00:02 | |
| *** Nikolay_1t has quit IRC | 00:05 | |
| *** Nikolay_St has joined #openstack-sahara | 00:06 | |
| *** Longgeek has quit IRC | 00:18 | |
| *** hdd has quit IRC | 00:36 | |
| *** Nikolay_St has quit IRC | 00:57 | |
| *** Nikolay_St has joined #openstack-sahara | 00:59 | |
| *** Longgeek has joined #openstack-sahara | 00:59 | |
| *** bandarji has joined #openstack-sahara | 01:12 | |
| *** hdd has joined #openstack-sahara | 01:45 | |
| *** bandarji has quit IRC | 01:49 | |
| *** macjack has joined #openstack-sahara | 02:02 | |
| openstackgerrit | Jacob Bin Wang proposed openstack/sahara: Fix order of arguments in assertEqual - Part1 https://review.openstack.org/165271 | 02:07 |
|---|---|---|
| *** hdd has quit IRC | 02:12 | |
| *** hdd has joined #openstack-sahara | 02:12 | |
| *** macjack has quit IRC | 02:17 | |
| *** coolsvap|afk is now known as coolsvap | 02:19 | |
| *** IlyaE has joined #openstack-sahara | 02:22 | |
| *** Poornima has joined #openstack-sahara | 02:23 | |
| *** IlyaE has quit IRC | 02:30 | |
| *** IlyaE has joined #openstack-sahara | 02:31 | |
| *** hdd has quit IRC | 02:32 | |
| *** coolsvap is now known as coolsvap|afk | 02:33 | |
| *** chandankumar has joined #openstack-sahara | 02:44 | |
| *** coolsvap|afk is now known as coolsvap | 02:45 | |
| *** IlyaE has quit IRC | 02:57 | |
| *** chandankumar has quit IRC | 03:05 | |
| *** coolsvap is now known as coolsvap|afk | 03:09 | |
| *** coolsvap|afk is now known as coolsvap | 03:16 | |
| *** Longgeek has quit IRC | 03:29 | |
| *** IlyaE has joined #openstack-sahara | 03:31 | |
| *** hdd has joined #openstack-sahara | 03:33 | |
| *** chandankumar has joined #openstack-sahara | 03:39 | |
| *** IlyaE has quit IRC | 03:53 | |
| *** IlyaE has joined #openstack-sahara | 04:16 | |
| *** Longgeek has joined #openstack-sahara | 04:30 | |
| *** IlyaE has quit IRC | 04:35 | |
| *** Longgeek has quit IRC | 04:37 | |
| *** Longgeek has joined #openstack-sahara | 04:38 | |
| *** hdd has quit IRC | 04:38 | |
| openstackgerrit | Ken Chen proposed openstack/sahara: Notify Kerberos and Sentry do not take effect https://review.openstack.org/165289 | 04:41 |
| *** Poornima has quit IRC | 04:51 | |
| *** IlyaE has joined #openstack-sahara | 05:02 | |
| *** Poornima has joined #openstack-sahara | 05:08 | |
| *** Networkn3rd has quit IRC | 05:09 | |
| *** Networkn3rd has joined #openstack-sahara | 05:26 | |
| *** sreshetn1 has joined #openstack-sahara | 06:06 | |
| *** sreshetn1 has quit IRC | 06:10 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Documentation for scenario tests https://review.openstack.org/164424 | 06:24 |
| *** chen12 has joined #openstack-sahara | 06:26 | |
| *** Nikolay_St has quit IRC | 06:27 | |
| *** IlyaE has quit IRC | 06:36 | |
| *** mahito_ has joined #openstack-sahara | 06:46 | |
| *** mahito has quit IRC | 06:49 | |
| *** IlyaE has joined #openstack-sahara | 07:05 | |
| *** IlyaE has quit IRC | 07:13 | |
| *** hogepodge has quit IRC | 07:16 | |
| *** tosky has joined #openstack-sahara | 07:18 | |
| *** Networkn_ has joined #openstack-sahara | 07:27 | |
| *** Networkn3rd has quit IRC | 07:30 | |
| *** tnovacik has joined #openstack-sahara | 07:34 | |
| tosky | uhm, interesting question on openstack-dev, but I wouldn't trust the stability on devstack environments | 07:45 |
| *** sgotliv has quit IRC | 07:49 | |
| chen12 | tosky: hi, are you talking about my question ? [Sahara][Horizon] Can't open "Data Processing" panel after update sahara & horizon | 08:04 |
| tosky | chen12: oh, hi, yes, that one | 08:04 |
| chen12 | tosky: any idea how to fix that ？ | 08:04 |
| chen12 | tosky: I don't want to re-install my devstack because of the dashboard | 08:05 |
| chen12 | tosky: but everything I did only make things worse.... | 08:05 |
| tosky | chen12: there are some dashboard fixes still not merged, especially one which is important when you ran jobs on a cluster which is then removed | 08:08 |
| tosky | let me find it | 08:08 |
| tosky | chen12: https://review.openstack.org/#/c/125927/ | 08:09 |
| tosky | try to apply it, not sure it will fix all the problems, but it's needed at least for the scenario I described | 08:09 |
| chen12 | tosky: thanks. let me try it first | 08:10 |
| chen12 | tosky: not work..... :( Ｔhis patch do looks like corressponding my issue before I update horizon. These error in bugs looks familiar to me. | 08:18 |
| chen12 | tosky: Am I doing all things correct for horizon ??? 1.update horizon code. ２.sudo python setup.py install ３.sudo service apache2 restart | 08:18 |
| tosky | chen12: I never tried that way, to be honest; did you try using devstack itself to update? | 08:19 |
| chen12 | tosky: devstack can do update ?? | 08:19 |
| chen12 | tosky: how ? | 08:19 |
| chen12 | tosky: I don't know that. | 08:19 |
| tosky | I'm rechecking the documentation, because I'm not sure anymore if the solution I had in mind (restack.sh) really works or if it cleanup and reinstall all | 08:22 |
| chen12 | tosky: I can try anyway. The worst case is a full re-install, the way I have to do if I can't solve the issue. | 08:24 |
| tosky | chen12: so, you could try unstack.sh and stack.sh again | 08:26 |
| chen12 | tosky: yep, that's my last plan.... :) | 08:27 |
| tosky | chen12: rereading the issue: did you try to delete the job execution before deleting the job? | 08:29 |
| tosky | I think the error could be that one | 08:29 |
| tosky | it's like when you try to remove cluster templates which are in use by some running cluster (you can't), or remove node group templates used by some cluster template (you can't) | 08:30 |
| chen12 | tosky: I do deleted cluster & cluster templates. because in horizon, they're on top of others. | 08:31 |
| tosky | chen12: with jobs is quite similar: you need to delete in the right order | 08:32 |
| *** mahito_ has quit IRC | 08:32 | |
| *** uschreiber_ has joined #openstack-sahara | 08:38 | |
| chen12 | tosky: o, I think I get your point. and my issue looks much like bug https://bugs.launchpad.net/horizon/+bug/1376738 .... | 08:39 |
| openstack | Launchpad bug 1376738 in OpenStack Dashboard (Horizon) "[sahara] Job execution page not working" [High,In progress] - Assigned to Nikita Konovalov (nkonovalov) | 08:39 |
| *** uschreiber_ has quit IRC | 08:39 | |
| chen12 | #/openstack-horizon | 08:40 |
| tosky | uhm, I think it could be a matter of two different issues, but let's see | 08:41 |
| *** uschreiber_ has joined #openstack-sahara | 08:41 | |
| *** uschreiber_ has quit IRC | 08:42 | |
| *** uschreiber_ has joined #openstack-sahara | 08:42 | |
| *** uschreiber_ has quit IRC | 08:42 | |
| chen12 | tosky: I must did something wrong for horizon update... at least the panel should response to my click.. | 08:42 |
| chen12 | tosky: I get help from horizon, missed "python manage.py collectstatic" before restart apache2. | 08:58 |
| tosky | I would never ever guessed it O.o | 09:00 |
| tosky | chen12: would it please send an email to openstack-dev summarizing the results of your investigation? So that other people do not scream when they hit the same issue but without answers? | 09:01 |
| tosky | :) | 09:01 |
| *** witlessb has joined #openstack-sahara | 09:01 | |
| tosky | I mean, like this: http://xkcd.com/979/ | 09:01 |
| *** sreshetn1 has joined #openstack-sahara | 09:07 | |
| *** Networkn_ has quit IRC | 09:10 | |
| *** Networkn3rd has joined #openstack-sahara | 09:11 | |
| *** sreshetn1 has quit IRC | 09:11 | |
| *** hogepodge has joined #openstack-sahara | 09:22 | |
| *** sgotliv has joined #openstack-sahara | 09:25 | |
| *** Networkn3rd has quit IRC | 09:26 | |
| *** Networkn3rd has joined #openstack-sahara | 09:27 | |
| Poornima | SergeyLukjanov, ping | 09:34 |
| *** _crobertsrh has left #openstack-sahara | 09:59 | |
| *** sgotliv has quit IRC | 10:05 | |
| *** sgotliv has joined #openstack-sahara | 10:06 | |
| openstackgerrit | Jacob Bin Wang proposed openstack/sahara: Fix order of arguments in assertEqual - Part2 https://review.openstack.org/165347 | 10:27 |
| *** tnovacik has quit IRC | 10:30 | |
| openstackgerrit | Jacob Bin Wang proposed openstack/sahara: Fix order of arguments in assertEqual - Part3 https://review.openstack.org/165350 | 10:32 |
| *** sgotliv has quit IRC | 10:35 | |
| *** sgotliv has joined #openstack-sahara | 10:41 | |
| *** tnovacik has joined #openstack-sahara | 11:19 | |
| Poornima | Hello SergeyLukjanov, I have few queries on tempest api test coverage for sahara. | 11:38 |
| Poornima | is it a good time to have a word with you? | 11:39 |
| *** Nikolay_St has joined #openstack-sahara | 11:46 | |
| *** Nikolay_St has quit IRC | 11:57 | |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara: MapR validation rules fixed https://review.openstack.org/165386 | 12:03 |
| *** Poornima has quit IRC | 12:30 | |
| *** tmckay has joined #openstack-sahara | 12:54 | |
| *** tnovacik has quit IRC | 12:54 | |
| *** hdd has joined #openstack-sahara | 13:00 | |
| *** hdd has quit IRC | 13:04 | |
| *** tnovacik has joined #openstack-sahara | 13:07 | |
| elmiko | any of the intel folks around? | 13:13 |
| *** pnayak has joined #openstack-sahara | 13:25 | |
| *** hdd has joined #openstack-sahara | 13:32 | |
| *** sgotliv has quit IRC | 13:32 | |
| tnovacik | hi, does anyone know where is specified that sahara is being installed by default on the horizon gate? | 13:42 |
| elmiko | good question, i'm not sure about that | 13:43 |
| tosky | maybe it's more a question for the horizon channel | 13:43 |
| elmiko | yea | 13:43 |
| *** sgotliv has joined #openstack-sahara | 13:46 | |
| *** egafford has joined #openstack-sahara | 13:54 | |
| openstackgerrit | lu huichun proposed openstack/sahara: Default templates for CDH https://review.openstack.org/164991 | 14:06 |
| openstackgerrit | lu huichun proposed openstack/sahara: Default templates for CDH https://review.openstack.org/164991 | 14:06 |
| *** sgotliv has quit IRC | 14:36 | |
| openstackgerrit | Michael McCune proposed openstack/sahara: Moving cdh cloudera manager password to config file https://review.openstack.org/165077 | 14:39 |
| *** sgotliv has joined #openstack-sahara | 14:39 | |
| openstackgerrit | Jacob Liberman proposed openstack/sahara: Replace direct http requests by sahara client in Quick start guide https://review.openstack.org/165192 | 14:40 |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara: Update MapR plugin docs https://review.openstack.org/165444 | 14:55 |
| *** Longgeek has quit IRC | 14:56 | |
| openstackgerrit | Jacob Liberman proposed openstack/sahara: Replace direct http requests by sahara client in Quick start guide https://review.openstack.org/165192 | 14:56 |
| *** chandankumar has quit IRC | 14:59 | |
| *** chandankumar has joined #openstack-sahara | 14:59 | |
| *** Poornima has joined #openstack-sahara | 15:02 | |
| *** hdd has quit IRC | 15:14 | |
| *** pcaruana has joined #openstack-sahara | 15:19 | |
| *** sgotliv has quit IRC | 15:36 | |
| *** tnovacik has quit IRC | 15:37 | |
| *** sgotliv has joined #openstack-sahara | 15:37 | |
| *** hdd has joined #openstack-sahara | 15:55 | |
| *** tosky has quit IRC | 15:58 | |
| *** tosky has joined #openstack-sahara | 16:00 | |
| *** chandankumar has quit IRC | 16:03 | |
| *** sreshetn1 has joined #openstack-sahara | 16:13 | |
| openstackgerrit | Trevor McKay proposed openstack/sahara: Implement job-types endpoint support methods for MapR plugin https://review.openstack.org/165494 | 16:25 |
| elmiko | what's the command to generate the conf file? | 16:27 |
| tmckay | the sahara.conf? | 16:27 |
| elmiko | yea | 16:27 |
| elmiko | does the tools/config stuff still work? i thought we had a new way | 16:28 |
| tmckay | I forget. | 16:29 |
| elmiko | hehe, me too! | 16:29 |
| tmckay | tox -e genconfig | 16:30 |
| elmiko | ah, cool. i thought it was through tox but didn't see the option in -l | 16:30 |
| elmiko | thanks! | 16:30 |
| tmckay | it's doing something :) Not sure what. | 16:30 |
| elmiko | haha | 16:30 |
| tmckay | I looked in the tox.ini | 16:30 |
| elmiko | nice, there you go using your brain again... sheesh ;) | 16:31 |
| tmckay | heh | 16:31 |
| *** tnovacik has joined #openstack-sahara | 16:36 | |
| openstackgerrit | Denis Egorenko proposed stackforge/sahara-ci-config: Global refactoring of CI scripts https://review.openstack.org/162102 | 16:52 |
| *** openstackgerrit has quit IRC | 16:59 | |
| *** openstackgerrit has joined #openstack-sahara | 16:59 | |
| openstackgerrit | Jacob Liberman proposed openstack/sahara: Replace direct http requests by sahara client in Quick start guide https://review.openstack.org/165192 | 16:59 |
| *** Nikolay_St has joined #openstack-sahara | 17:02 | |
| *** pnayak has quit IRC | 17:22 | |
| *** Nikolay_St has quit IRC | 17:22 | |
| openstackgerrit | Trevor McKay proposed openstack/sahara: Implement job-types endpoint support methods for Fake plugin https://review.openstack.org/165524 | 17:24 |
| *** sreshetn1 has quit IRC | 17:28 | |
| tellesnobrega | tmckay, hey, does edp necessarily uses Oozie? | 17:29 |
| tmckay | tellesnobrega, nope, you can write an EDP engine to execute jobs any way you like | 17:30 |
| tmckay | spark EDP uses ssh and shell commands around spark-submit | 17:30 |
| tellesnobrega | tmckay, great. really starting to work on storm edp | 17:30 |
| tmckay | excellent, looking forward to it | 17:30 |
| *** Nikolay_St has joined #openstack-sahara | 17:30 | |
| tellesnobrega | i will ping if i run into some problems | 17:31 |
| tmckay | tellesnobrega, by the way, we recently added a Shell job type that can be implemented for any EDP engine, most likely. | 17:31 |
| *** vgridnev_ has joined #openstack-sahara | 17:34 | |
| *** sreshetn1 has joined #openstack-sahara | 17:38 | |
| *** Nikolay_St has quit IRC | 17:38 | |
| *** Nikolay_St has joined #openstack-sahara | 17:39 | |
| *** crobertsrh has joined #openstack-sahara | 17:40 | |
| vgridnev_ | hi folks, could you please help with reviewing this patches: https://review.openstack.org/#/q/status:open+project:openstack/sahara+branch:master+topic:bp/event-log,n,z | 17:41 |
| *** coolsvap is now known as coolsvap|afk | 17:41 | |
| elmiko | vgridnev_: ack, i'll take a look this afternoon | 17:42 |
| tellesnobrega | tmckay, i will take a look, see how it helps | 17:42 |
| tmckay | tellesnobrega, it gives you a way to run a general command on a cluster. You can't control which node it runs on, but you can use it to manage hdfs, for instance, or launch a java app with custom classpath, that kind of thing. | 17:44 |
| tmckay | kind of a catchall if Sahara doesn't support something a user wants to do through EDP | 17:44 |
| tmckay | you may not need it for Storm, purely optional | 17:45 |
| *** Nikolay_St has quit IRC | 17:45 | |
| tellesnobrega | tmckay, got it | 17:46 |
| *** tnovacik has quit IRC | 17:46 | |
| *** crobertsrh has quit IRC | 17:56 | |
| openstackgerrit | Michael McCune proposed openstack/sahara: Moving cdh cloudera manager password to config file https://review.openstack.org/165077 | 17:56 |
| vgridnev_ | elmiko, is it ok have plugin related option in sahara conf sample? | 18:02 |
| elmiko | vgridnev_: in this case i think so, because it's needed by the plugin | 18:03 |
| elmiko | but i could be wrong about that | 18:03 |
| *** Nikolay_St has joined #openstack-sahara | 18:04 | |
| elmiko | vgridnev_: i'm not sure how else we would inject that configuration option into sahara | 18:05 |
| vgridnev_ | maybe it's better to have plugin-related option, as we have some patches in blueprint for polling? | 18:05 |
| *** Poornima has quit IRC | 18:06 | |
| vgridnev_ | like here, i think | 18:06 |
| vgridnev_ | https://review.openstack.org/#/c/160818/10/sahara/plugins/vanilla/hadoop2/config_helper.py,cm | 18:06 |
| elmiko | vgridnev_: i can try that, i was having issue because one of the tests is trying to bring in a class that needs it. | 18:09 |
| *** sreshetn1 has quit IRC | 18:12 | |
| *** hdd has quit IRC | 18:16 | |
| Nikolay_St | hi all | 18:21 |
| Nikolay_St | I send an email about the results of discovery about our client gate job | 18:22 |
| vgridnev_ | nice to hear that | 18:24 |
| tmckay | Nikolay_St, I don't see it. Did you send it yet? Subject line? | 18:26 |
| vgridnev_ | http://lists.openstack.org/pipermail/openstack-dev/2015-March/059349.html | 18:26 |
| *** tosky has quit IRC | 18:26 | |
| *** hdd has joined #openstack-sahara | 18:28 | |
| tmckay | Nikolay_St, wow. | 18:28 |
| tmckay | nice investigation | 18:28 |
| *** hdd has quit IRC | 18:33 | |
| elmiko | Nikolay_St++ nice find! | 18:46 |
| *** sgotliv has quit IRC | 18:52 | |
| elmiko | vgridnev_: re: config_helper.py, in those cases they are importing an option that is defined elsewhere. i need to inject a new config option, i think i may need to create a new module in the cdh plugin | 18:56 |
| openstackgerrit | Merged stackforge/puppet-sahara: Rewrite puppet-sahara from scratch https://review.openstack.org/145509 | 19:39 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Apply event-log feature for HDP plugin https://review.openstack.org/152472 | 19:45 |
| *** tnovacik has joined #openstack-sahara | 19:45 | |
| Nikolay_St | tmckay: elmiko: thx, guys :) | 19:50 |
| elmiko | nice, guided mode stuff for horizon got workflow+ | 19:50 |
| Nikolay_St | it was hard to find. it wasn't something that I have expected | 19:50 |
| Nikolay_St | elmiko: good news | 19:51 |
| elmiko | Nikolay_St: yea, it's a really good effort. we've had those problems for awhile | 19:51 |
| *** devlaps has joined #openstack-sahara | 19:55 | |
| *** zigo__ has joined #openstack-sahara | 19:56 | |
| *** zigo has quit IRC | 19:56 | |
| openstackgerrit | Merged openstack/sahara: Add Spark support for MapR plugin https://review.openstack.org/163007 | 20:12 |
| *** openstackgerrit has quit IRC | 20:14 | |
| *** openstackgerrit has joined #openstack-sahara | 20:15 | |
| openstackgerrit | Jacob Liberman proposed openstack/sahara: Replace direct http requests by sahara client in Quick start guide https://review.openstack.org/165192 | 20:18 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Add usages of plugin poll - part 1 https://review.openstack.org/160769 | 20:18 |
| tmckay | elmiko, ping | 20:25 |
| elmiko | tmckay: pong | 20:25 |
| tmckay | was just looking at some sample config stuff for the templates. I think "worker" nodes can be 2, and masters 3, except for cdh where the master needs to be 4 (I need to try this). | 20:26 |
| tmckay | looking at that gave me an idea | 20:26 |
| tmckay | elmiko, the way the def template sets have been written by ethan and lu, we're using a naming convention where plugin_version are already in the name | 20:27 |
| elmiko | i know that for the vanilla plugin, 2 usually works for me across the board | 20:27 |
| tmckay | ack. So, what if the CLI will recognize just [templatename] as a config section? | 20:28 |
| tmckay | I was thinking we | 20:28 |
| tmckay | we | 20:28 |
| tmckay | aahhdh. we'd have stuff named "master" and "worker" | 20:28 |
| elmiko | are you concerned that could be confusing? | 20:29 |
| tmckay | but if we're using the convention, then the simple thing can make sense. If you *do* end up with a name collision, you can use the other form | 20:29 |
| tmckay | well, for instance [cdh_5_cdh-5-default-namenode] | 20:29 |
| tmckay | not confusing, but kind of wordy | 20:29 |
| elmiko | yea, i can see that | 20:30 |
| tmckay | not a big deal, but looks kind of weird | 20:30 |
| elmiko | i don't mind wordy so much, if it makes sense | 20:30 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Add usages of plugin poll - part 2 https://review.openstack.org/160818 | 20:30 |
| elmiko | but that seems like the cdh5 is repeated and i'm not sure why | 20:30 |
| elmiko | or at least, i might be confused as a user | 20:31 |
| tmckay | yeah. The way they are named (by convention), it's immediately apparent what the thing is when you're looking at Horizon | 20:32 |
| elmiko | yea | 20:32 |
| tmckay | that's the upside | 20:33 |
| openstackgerrit | Sebastien Badia proposed stackforge/puppet-sahara: fixtures: Unpin puppetlabs-mysql module https://review.openstack.org/145510 | 20:33 |
| elmiko | well, i think if the names are clear on the user side then it's probably ok. a small confusion for the admin running the tool could be acceptable. | 20:34 |
| tmckay | it would be pretty easy to add a [templatename] section as an option | 20:35 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Remove the sahara.conf.sample file https://review.openstack.org/163488 | 20:35 |
| elmiko | tmckay: i think given the scoping for the substitution, it's not necessary for the template name variables to be cdh-5-.... (for example) | 20:35 |
| elmiko | unless it's needed in the cli tool itself | 20:35 |
| elmiko | i dunno. i think it's fine as it is, if we decide later that it's confusing we can refactor | 20:36 |
| tmckay | elmiko, I was originally thinking that naming stuff by specific template might be less common. But, now I'm thinking it's more likely that we might mix flavors. I think my problem with cdh was on the master, not the workers | 20:37 |
| tmckay | so, we need to call out the master template and bump the flavor up, hypothetically | 20:37 |
| elmiko | tmckay: yea... tough issue when we start talking about mixing flavors and everything. i would almost just vote for making it simple, 1 flavor per config section. | 20:39 |
| elmiko | don't get into the mess of separating all these variables out. | 20:39 |
| elmiko | i think once you get to that level of customization the user should be aware of building their own temaplte | 20:39 |
| tmckay | well, it would still be just "flavor_id", I'm talking about multiple sections. You can have a section per template at the finest granularity | 20:40 |
| tmckay | So, [cdh_5_master] and [cdh_5_worker] is what I was originally thinking of | 20:40 |
| tmckay | for the section names | 20:40 |
| elmiko | yea, but that's not in there now? | 20:41 |
| *** elmiko_ has joined #openstack-sahara | 20:42 | |
| tmckay | it is. but the template name is after the last "_". So we've currently got effectively [cdh_5_cdh-5-master] because the "name" field in the template is "cdh-5-master" | 20:42 |
| elmiko_ | hmm | 20:42 |
| tmckay | so if we're going to use template names that are already unique, should we allow just [cdh-5-master] (the template name) as a section? | 20:42 |
| tmckay | wouldn't work if you like to make things like "master" and "slave" for every plugin :) | 20:43 |
| tmckay | but if you have a convention that yield unique names ... | 20:43 |
| tmckay | I think it makes sense, I'd just have to call it out clearly in the README | 20:43 |
| tmckay | if I was an admin, I'd be annoyed if I had to append stuff on my name when it was already unambiguous. But that's me | 20:44 |
| elmiko_ | that makes sense | 20:44 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Migrate to oslo.policy lib instead of copy-pasted oslo-incubator https://review.openstack.org/163386 | 20:45 |
| *** elmiko has quit IRC | 20:45 | |
| elmiko_ | i could see allowing the template name as a config section | 20:45 |
| tmckay | alright, I'll try it and see what shakes out. I'll see if I can get a cdh with "large" master and smaller datanodes, cause otherwise that thing is huge | 20:45 |
| *** elmiko_ is now known as elmiko | 20:46 | |
| elmiko | i think it's nice having all the options, my only concern is that it could become too many options (if that makes sense) | 20:47 |
| tmckay | yeah, completely. That's why I'm hesitating. But I think this is clear enough. | 20:48 |
| *** vgridnev_ has quit IRC | 20:48 | |
| elmiko | i guess if you need to look inside the template files anyways to see the plugin names and version, then it's fine | 20:48 |
| tmckay | elmiko, hmm, hold on, this is what cdh 53 name node section looks like [cdh_5.3.0_cdh-530-default-namenode] | 20:51 |
| tmckay | maybe that's not so bad. The cdh5 stuff is weird because the plugin version is short | 20:51 |
| elmiko | still slightly verbose | 20:51 |
| tmckay | lol, yeah, it is | 20:51 |
| tmckay | alright, thanks. I'll give it a try | 20:52 |
| elmiko | yea i dunno. like i said before, maybe just go with the way it is | 20:52 |
| elmiko | if we don't like it we can refactor later | 20:52 |
| elmiko | i mean how often is a user going to be messing with the default template files and configs anyways? | 20:52 |
| tmckay | maybe a lot if we don't get it right :) | 20:53 |
| elmiko | ha! | 20:53 |
| tmckay | I'm a little worried about the flavors. most of our stuff is for testing, not production workloads | 20:53 |
| tmckay | so if they do need to tweak it, I want it to be a nice experience | 20:54 |
| elmiko | that's fair | 20:54 |
| elmiko | ideally our default templates will be useful enough to get someone started using sahara | 20:54 |
| tmckay | ack | 20:54 |
| elmiko | once you understand what you need, you might not need them anymore | 20:54 |
| tmckay | good point | 20:55 |
| elmiko | otoh, once you understand how to use the default templates you might just create a bunch of them because it's easier than creating them with horizon or the sahara client cli | 20:55 |
| tmckay | agreed. I will be using this thing constantly | 20:56 |
| elmiko | i dunno, a user is comfortable writing json for their templates i'm not sure how they'll be using sahara | 20:56 |
| elmiko | well yea, but we're constantly blowing away the database ;) | 20:56 |
| elmiko | similiarly i have a python script that does everything i want, proxy domain, templates, etc... | 20:56 |
| tmckay | elmiko, yeah, I suppose you're right, the def templates out of the box are to play with Sahara and hadoop, spark, storm, etc. and get experience | 21:02 |
| tmckay | if you're going to run truly big data, you're going to have people launching massive clusters with specialized flavors | 21:03 |
| elmiko | i would imagine so | 21:04 |
| tmckay | so, I'm leaning toward the smallest flavors that make the thing work. That's "2" for just about everything, except cdh which fails to launch that way. | 21:04 |
| *** Nikolay_St has quit IRC | 21:04 | |
| elmiko | fwiw, i think you're doing the right thing by running our example jobs on the clusters created with the templates | 21:04 |
| *** Nikolay_St has joined #openstack-sahara | 21:04 | |
| tmckay | and if I'm right only the masters will need 4. So our example config will have a [DEFAULT] flavor_id =2 and then a couple overrides for cdh | 21:04 |
| elmiko | ti demonstrates clearly how someone who has never used this stuff could walk through our examples and, hopefully, gain an understanding. | 21:04 |
| *** hdd has joined #openstack-sahara | 21:05 | |
| tmckay | yeah, it's pretty simple to fire off that pig job | 21:05 |
| elmiko | i keep losing connection | 21:06 |
| tmckay | funny that we thing 2GB RAM and 20GB hard drive is "small" :) | 21:06 |
| tmckay | the 80s would be open-mouthed | 21:07 |
| elmiko | LOL | 21:08 |
| elmiko | so true | 21:08 |
| *** Nikolay_St has quit IRC | 21:10 | |
| *** Nikolay_St has joined #openstack-sahara | 21:10 | |
| tmckay | elmiko, nice, cdh5 with large master and small datanode, secondary master launched | 21:13 |
| tmckay | that is much easier to take | 21:13 |
| elmiko | nice! | 21:13 |
| elmiko | yea | 21:13 |
| elmiko | i'll have to try that if i ever get an ubuntu image to boot | 21:13 |
| tmckay | heh. I don't know what the trouble is with that, I've been using them all day | 21:13 |
| elmiko | yea, i think maybe my custom network commands might be screwing things up | 21:14 |
| elmiko | i'm trying a restack now | 21:14 |
| elmiko | it's like the images can't talk to the cloud-init server | 21:14 |
| tmckay | hmm, that job-executions from delete cluster bug is still in Horizon (at least what I'm running). I should check on that. | 21:16 |
| *** hdd has quit IRC | 21:19 | |
| *** Nikolay_St has quit IRC | 21:24 | |
| *** Nikolay_St has joined #openstack-sahara | 21:25 | |
| *** jamielennox is now known as jamielennox|away | 21:31 | |
| openstackgerrit | lu huichun proposed openstack/sahara: Minor refactor of the integration service test https://review.openstack.org/165618 | 21:40 |
| *** tmckay is now known as tmckay_bbl | 21:40 | |
| *** devlaps has quit IRC | 21:43 | |
| *** hdd has joined #openstack-sahara | 21:49 | |
| *** sgotliv has joined #openstack-sahara | 21:52 | |
| *** egafford has quit IRC | 22:02 | |
| *** jamielennox|away is now known as jamielennox | 22:12 | |
| openstackgerrit | Merged stackforge/puppet-sahara: fixtures: Unpin puppetlabs-mysql module https://review.openstack.org/145510 | 22:15 |
| openstackgerrit | Sebastien Badia proposed stackforge/puppet-sahara: Migrate postgresql backend to use openstacklib::db::postgresql https://review.openstack.org/145511 | 22:26 |
| *** sgotliv has quit IRC | 22:27 | |
| *** witlessb has quit IRC | 22:49 | |
| openstackgerrit | lu huichun proposed openstack/sahara: Add integration tests for scaling in Spark https://review.openstack.org/165640 | 22:51 |
| *** tnovacik has quit IRC | 23:14 | |
| *** hdd has quit IRC | 23:37 | |
| *** crobertsrh has joined #openstack-sahara | 23:58 | |
| *** crobertsrh has left #openstack-sahara | 23:58 | |
| openstackgerrit | Trevor McKay proposed openstack/sahara: Add a CLI tool for managing default templates https://review.openstack.org/163649 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!