Friday, 2015-10-16

*** _crobertsrh has joined #openstack-sahara00:05
*** coolsvap has joined #openstack-sahara00:05
*** hogepodge has joined #openstack-sahara00:07
*** dave-mccowan has joined #openstack-sahara00:07
*** elmiko has joined #openstack-sahara00:07
*** ekarlso has joined #openstack-sahara00:07
*** tellesnobrega has joined #openstack-sahara00:07
*** goodygum has joined #openstack-sahara00:07
*** sreshetnyak has joined #openstack-sahara00:07
*** pino|work has joined #openstack-sahara00:08
*** SergeyLukjanov has joined #openstack-sahara00:08
*** dmitryme has joined #openstack-sahara00:08
*** aignatov has joined #openstack-sahara00:08
*** Erming_ has joined #openstack-sahara00:09
*** larainema_ has joined #openstack-sahara00:09
*** raildo-afk has joined #openstack-sahara00:09
*** htruta has joined #openstack-sahara00:09
*** david-lyle has joined #openstack-sahara00:11
*** kgalanov has joined #openstack-sahara00:11
*** zhiyan has joined #openstack-sahara00:11
*** logan2 has joined #openstack-sahara00:11
*** vkmc has joined #openstack-sahara00:11
*** psusi has joined #openstack-sahara00:12
*** openstackgerrit has joined #openstack-sahara00:12
*** rodrigods has joined #openstack-sahara00:12
*** rickflare has joined #openstack-sahara00:12
*** dhellmann has joined #openstack-sahara00:12
*** alazarev has joined #openstack-sahara00:12
*** bapalm has joined #openstack-sahara00:13
*** sgotliv_ has joined #openstack-sahara00:13
*** _degorenko has joined #openstack-sahara00:13
*** _mattf has joined #openstack-sahara00:13
*** ruhe has joined #openstack-sahara00:13
*** vgridnev has joined #openstack-sahara00:13
*** shakamunyi has joined #openstack-sahara00:13
*** david-lyle has quit IRC00:14
*** DuncanT has joined #openstack-sahara00:17
*** krotscheck has joined #openstack-sahara00:17
*** jamielennox has joined #openstack-sahara00:17
*** NikitaKonovalov has joined #openstack-sahara00:17
*** zigo has joined #openstack-sahara00:17
*** stevemar_ has joined #openstack-sahara00:31
*** chlong has joined #openstack-sahara00:43
*** mathspanda has joined #openstack-sahara01:15
*** david-lyle has joined #openstack-sahara01:21
openstackgerritKen Chen proposed openstack/sahara: Modify service-role view in creating node group template  https://review.openstack.org/23514101:53
*** witlessb has joined #openstack-sahara01:56
*** witlessb has quit IRC02:00
*** psusi has quit IRC02:24
openstackgerritOpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements  https://review.openstack.org/23544703:07
*** alesilva241 has joined #openstack-sahara03:14
*** alesilva241 has quit IRC03:27
*** macjack has joined #openstack-sahara03:43
*** mathspanda has quit IRC03:49
*** jamielennox has quit IRC03:51
*** jamielennox has joined #openstack-sahara03:52
*** hdd has quit IRC04:06
*** hdd has joined #openstack-sahara04:58
*** Poornima has joined #openstack-sahara05:02
*** vsham20 has joined #openstack-sahara05:24
*** jamielennox is now known as jamielennox|away05:30
*** hdd has quit IRC05:39
*** witlessb has joined #openstack-sahara05:42
*** witlessb has quit IRC05:47
*** nkrinner has joined #openstack-sahara05:51
*** Poornima has quit IRC06:02
openstackgerritKen Chen proposed openstack/sahara-specs: Resource Check before Cluster Creation  https://review.openstack.org/23511006:11
openstackgerritKen Chen proposed openstack/sahara: Add Resource Check step and the involved base methods  https://review.openstack.org/23574506:14
*** chlong has quit IRC06:15
openstackgerritKen Chen proposed openstack/sahara-specs: Resource Check before Cluster Creation  https://review.openstack.org/23511006:17
*** vsham201 has joined #openstack-sahara06:19
*** vsham20 has quit IRC06:21
*** sgotliv_ has quit IRC06:24
*** sgotliv_ has joined #openstack-sahara06:24
*** nkrinner has quit IRC06:34
*** stevemar_ has quit IRC06:35
*** stevemar_ has joined #openstack-sahara06:36
*** nkrinner has joined #openstack-sahara06:38
*** stevemar_ has quit IRC06:38
*** vsham201 has quit IRC06:47
*** macjack has quit IRC07:12
*** vsham20 has joined #openstack-sahara07:17
*** vgridnev has quit IRC07:34
*** vgridnev has joined #openstack-sahara07:41
*** vgridnev has quit IRC07:41
*** Poornima has joined #openstack-sahara07:48
*** vsham20 has quit IRC07:52
*** vsham20 has joined #openstack-sahara07:53
*** vsham201 has joined #openstack-sahara07:57
*** vsham20 has quit IRC07:57
openstackgerritKen Chen proposed openstack/sahara: Add Resource Check step and the involved base methods  https://review.openstack.org/23574507:57
openstackgerritKen Chen proposed openstack/sahara-specs: Resource Check before Cluster Creation  https://review.openstack.org/23511007:58
*** _degorenko is now known as degorenko08:34
openstackgerritAndrey Pavlov proposed openstack/sahara: [TEST] Fix tempest  https://review.openstack.org/23578508:35
*** AndreyPavlov has joined #openstack-sahara08:35
*** vgridnev has joined #openstack-sahara08:38
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Adding Job Templates support to CLI  https://review.openstack.org/23540008:48
openstackgerritAndrey Pavlov proposed openstack/sahara: Fix tempest tests  https://review.openstack.org/23578508:57
*** Poornima has quit IRC08:59
*** vsham201 has quit IRC09:11
*** vsham20 has joined #openstack-sahara09:12
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Adding Job Templates support to CLI  https://review.openstack.org/23540009:12
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Adding Job Types support to CLI  https://review.openstack.org/23546209:19
*** AndreyPavlov has quit IRC09:27
*** sgotliv_ has quit IRC09:33
*** AndreyPavlov has joined #openstack-sahara09:34
*** tosky has joined #openstack-sahara09:55
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Always use latest glanceclient  https://review.openstack.org/23583610:29
vgridnevtellesnobrega, are you around?10:46
openstackgerritVitaly Gridnev proposed openstack/sahara: [test commit]  https://review.openstack.org/23585811:13
openstackgerritMerged stackforge/sahara-ci-config: Always use latest glanceclient  https://review.openstack.org/23583611:23
openstackgerritEvgeny Sikachev proposed openstack/sahara: [Test commit]  https://review.openstack.org/22890611:24
*** thehoffau has joined #openstack-sahara11:38
*** hdd has joined #openstack-sahara11:42
*** thehoffau has quit IRC11:47
*** sgotliv_ has joined #openstack-sahara11:48
*** stevemar_ has joined #openstack-sahara11:53
*** stevemar_ has quit IRC11:55
openstackgerritVitaly Gridnev proposed openstack/sahara: Support of Spark EDP in Ambari plugin  https://review.openstack.org/23480112:00
openstackgerritVitaly Gridnev proposed openstack/sahara: Fix issue with job types in Ambari plugin  https://review.openstack.org/23247112:00
*** AndreyPavlov has quit IRC12:01
openstackgerritVitaly Gridnev proposed openstack/sahara: Support of Spark EDP in Ambari plugin  https://review.openstack.org/23480112:05
*** vsham20 has quit IRC12:06
*** vsham20 has joined #openstack-sahara12:06
*** _crobertsrh is now known as crobertsrh12:09
tellesnobregavgridnev, just got in12:12
tellesnobregai have a set up here and it is working12:13
vgridnevtellesnobrega, are you using auto-security groups?12:14
tellesnobregalet me check12:14
tellesnobregavgridnev, no, i'm using default12:15
vgridnevok, it looks like you need to implement get_open_ports for storm plugin12:15
vgridnevtellesnobrega, ^^12:17
tellesnobregahuum, ok12:18
tellesnobregai will take a look at it now12:18
vgridnevOk, could please assign bug on yourself?12:19
rickflaremorning12:20
tellesnobregavgridnev, sure, thanks and thanks to esikachev12:21
vgridnevDo you sample storm job with detailed description?12:22
vgridnevAt least how to run it12:22
*** raildo-afk is now known as raildo12:25
*** hdd has quit IRC12:32
*** degorenko is now known as _degorenko12:41
openstackgerritVitaly Gridnev proposed openstack/sahara: Make ssh timeout configurable  https://review.openstack.org/23589412:48
*** chlong has joined #openstack-sahara12:48
openstackgerritVitaly Gridnev proposed openstack/sahara: Make ssh timeout configurable  https://review.openstack.org/23589412:49
*** AndreyPavlov has joined #openstack-sahara12:56
toskytellesnobrega: I don't think it's there; if you can provide a sample Storm scenario test configuration for the CI, it would be nice13:03
tellesnobregatosky, sure, i don't really know how the CI tests work, but i will take a look and see what I can come up with13:04
toskytellesnobrega: it's two steps: a scenario (which you can run yourself) and then the CI13:05
toskytellesnobrega: take a look at sahara/tests/scenario/README.rst and then etc/scenario/sahara-ci/ for some examples (both into sahara repository013:05
*** _degorenko is now known as degorenko13:11
openstackgerritVitaly Gridnev proposed openstack/sahara: Make ssh timeout configurable  https://review.openstack.org/23589413:13
vgridnevtellesnobrega, I can setup jobs if you will upload scenario template13:14
vgridnevon CI13:14
tellesnobregatosky, thanks, i'm working on the storm bug right now, once i'm done i will take a look and come up with the scenario13:14
tellesnobregavgridnev, sounds good, the job is very simple, just like spark13:15
vgridnevOk13:15
tellesnobregavgridnev, i'm testing the storm ui problem now13:17
vgridnevOk, great13:17
toskyvgridnev: I guess it's too late to backport the patch which makes ssh timeout configurable to Liberty (new feature)13:21
vgridnevprobably not13:22
toskyeven if it would be nice, as the issue is not so difficult to hit when you start a big cluster13:22
vgridnevI think that addition of new option is not so weird13:23
*** AndreyPavlov has quit IRC13:25
*** hdd has joined #openstack-sahara13:27
toskyvgridnev: oh, then I would vote for adding it :)13:29
*** vsham20 has quit IRC13:29
elmiko+1, if we can backport it, it is a useful fix13:29
vgridnevOk, anyway we need to wait when stable/liberty will be opened13:29
*** vsham20 has joined #openstack-sahara13:29
elmikoyea13:29
*** AndreyPavlov has joined #openstack-sahara13:33
openstackgerritEvgeny Sikachev proposed openstack/sahara: Add -f to format volumes  https://review.openstack.org/23592113:44
*** tmckay has joined #openstack-sahara13:44
*** AndreyPavlov has quit IRC13:44
openstackgerritEvgeny Sikachev proposed openstack/sahara: Add -f option to formatting volumes  https://review.openstack.org/23592113:47
*** AndreyPavlov has joined #openstack-sahara13:53
*** AndreyPavlov has quit IRC13:59
openstackgerritMerged openstack/sahara: Fix tempest tests  https://review.openstack.org/23578514:16
openstackgerritOpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements  https://review.openstack.org/23544714:23
openstackgerritSergey Lukjanov proposed openstack/sahara: Remove old sahara endpoint  https://review.openstack.org/22779114:25
openstackgerritSergey Lukjanov proposed openstack/sahara: Remove old sahara endpoint  https://review.openstack.org/22779114:25
*** abalutoiu has joined #openstack-sahara14:27
openstackgerritMerged openstack/sahara-dashboard: Updated from global requirements  https://review.openstack.org/23566514:28
*** sgotliv_ has quit IRC14:34
openstackgerritVitaly Gridnev proposed openstack/sahara: Hive job type support on CI  https://review.openstack.org/23196314:56
*** AndreyPavlov has joined #openstack-sahara14:57
*** nkrinner has quit IRC14:59
SergeyLukjanovtmckay, crobertsrh, elmiko, egafford, please, take a look on a new CLI changes - https://review.openstack.org/#/q/topic:bp/cli-as-openstackclient-plugin,n,z15:01
tmckaywill do15:01
elmikoSergeyLukjanov: ack15:01
tmckayrickflare, will you be in Tokyo?15:04
tmckayrickflare, oh, found the answer in meeting notes :)  See you in Austin, if not before15:06
*** chlong has quit IRC15:09
*** weihan has joined #openstack-sahara15:09
*** david-ly_ has joined #openstack-sahara15:11
*** david-lyle has quit IRC15:11
*** tmckay has quit IRC15:14
*** stevemar_ has joined #openstack-sahara15:18
*** stevemar_ has quit IRC15:19
*** stevemar_ has joined #openstack-sahara15:19
*** tmckay has joined #openstack-sahara15:19
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Adding Clusters support to CLI  https://review.openstack.org/23417715:28
*** adelia has joined #openstack-sahara15:29
elmikovgridnev, SergeyLukjanov, what is up with the tempest-saharaclient-tests on those cli reviews. looks like it is not able to find the cluster_timeout config option or something?15:33
*** AndreyPavlov has quit IRC15:36
SergeyLukjanovelmiko, this CLI is not related anyhow to the tempest tests15:37
*** vsham20 has quit IRC15:38
elmikoSergeyLukjanov: ok, thanks15:38
*** vsham20 has joined #openstack-sahara15:38
*** vsham20 has quit IRC15:47
*** vsham20 has joined #openstack-sahara15:48
*** AndreyPavlov has joined #openstack-sahara16:00
*** vsham20 has quit IRC16:01
*** vsham20 has joined #openstack-sahara16:04
*** vsham20 has quit IRC16:04
*** vsham20 has joined #openstack-sahara16:05
*** vgridnev has quit IRC16:09
*** stevemar_ has quit IRC16:10
*** stevemar_ has joined #openstack-sahara16:11
AndreyPavlovelmiko, there is no problem with cli patches. tempest tests didn't work until https://review.openstack.org/#/c/235785/ was merged16:13
elmikoAndreyPavlov: thanks for the clarification. i thought that was the case, just was curious.16:14
elmikoAndreyPavlov: nice work on these patches too =)16:15
AndreyPavlovelmiko, thanks)16:15
*** stevemar_ has quit IRC16:16
*** AndreyPavlov has quit IRC16:16
openstackgerritMerged openstack/python-saharaclient: Updated from global requirements  https://review.openstack.org/23566216:29
openstackgerritMerged openstack/python-saharaclient: Improve readme contents  https://review.openstack.org/23521616:32
*** vgridnev has joined #openstack-sahara16:39
*** weihan has quit IRC16:39
*** weihan has joined #openstack-sahara16:43
*** weihan has quit IRC16:48
*** weihan has joined #openstack-sahara16:48
*** weihan has quit IRC16:56
*** stevemar_ has joined #openstack-sahara17:10
*** tosky has quit IRC17:14
*** degorenko is now known as _degorenko17:17
*** agireud has joined #openstack-sahara17:18
*** vsham20 has quit IRC17:18
*** vsham20 has joined #openstack-sahara17:18
openstackgerritMerged openstack/sahara: Open Kilo 2015.1.3  https://review.openstack.org/23441017:20
openstackgerritMerged openstack/sahara: Use api-paste.ini for loading middleware  https://review.openstack.org/23198917:20
*** vsham20 has quit IRC17:28
*** vsham20 has joined #openstack-sahara17:29
openstackgerritMerged openstack/sahara: Added support for Spark 1.3.1 in MapR plugin  https://review.openstack.org/21893417:30
openstackgerritMerged openstack/sahara: Use distributed mode by default in devstack  https://review.openstack.org/23240617:30
openstackgerritMerged openstack/sahara: Modify service-role view in creating node group template  https://review.openstack.org/23514117:30
openstackgerritMerged openstack/sahara: Add ability running tests on existing cluster  https://review.openstack.org/22585417:30
openstackgerritMerged openstack/sahara: Add -f option to formatting volumes  https://review.openstack.org/23592117:30
*** tmckay has quit IRC17:35
*** tmckay has joined #openstack-sahara17:36
*** vsham20 has quit IRC17:57
*** vsham20 has joined #openstack-sahara17:57
openstackgerritVitaly Gridnev proposed openstack/sahara: Add unit tests for AuthValidator middleware  https://review.openstack.org/23338617:58
*** hdd has quit IRC18:00
openstackgerritKen Chen proposed openstack/sahara: Add CDH 5.4.0 contents in doc  https://review.openstack.org/21256218:08
*** stevemar_ has quit IRC18:17
*** stevemar_ has joined #openstack-sahara18:18
*** vsham201 has joined #openstack-sahara18:19
*** vsham20 has quit IRC18:19
*** stevemar_ has quit IRC18:22
tellesnobregavgridnev, are you around?18:23
vgridnevyep18:24
tellesnobregaI'm having some problems with the storm ui bug18:25
*** stevemar_ has joined #openstack-sahara18:25
tellesnobregai implemented the get_open_ports18:25
tellesnobregaand i can see that the config is working18:25
tellesnobregabut in utils.get_confg_value_or_default, there is a call ng.configuration()18:26
tellesnobregathis returns a dict with configs, but storm configs are empty, do you know where this dict is changed?18:26
tellesnobregai'm not sure about this, but it looks like i have to pass the configuration to the nodegroup it self18:27
vgridnevhttps://github.com/openstack/sahara/blob/master/sahara/conductor/objects.py#L13618:28
*** tmckay has quit IRC18:33
*** psusi has joined #openstack-sahara18:33
tellesnobregathanks, there is problably somewhere in the plugin that i have to add something18:33
*** abalutoiu has quit IRC18:40
*** hdd has joined #openstack-sahara18:41
*** tmckay has joined #openstack-sahara18:48
tellesnobregaelmiko, tmckay do you guys know can I make storm configs (open ports) available at ng.configuration() ?18:59
elmikotellesnobrega: not of the top of my head, sorry19:00
tellesnobregai'm working on a bug where storm ui is not available when using auto sec groups,19:00
elmikoi thought there was a function to return the ports for a plugin though19:01
elmikooh, weird19:01
elmikobecause the port isn't opened?19:01
tellesnobregai didn't have get_open_ports implemented yet19:01
tellesnobregayes19:01
elmikoi haven't looked too deeply into that code, i just remember that function19:02
tmckayhmm, unfamiliar with the port stuff except for the get_open_ports() as mentioned above19:02
tellesnobregai'm coding this get now, but at some point it gets the config from ng.configuration(), and I can't find where I can put the config in ng19:02
tellesnobregalooking at the spark code here but i can't find it19:02
*** hdd has quit IRC19:04
*** david-ly_ has quit IRC19:04
*** david-lyle has joined #openstack-sahara19:05
tellesnobregatmckay, there is an extra dict in spark (as well as storm) is that only used to store configs that are going to the cluster vms?19:05
tmckaytellesnobrega, I would have to look19:06
openstackgerritVitaly Gridnev proposed openstack/sahara: Revert "Add -f option to formatting volumes"  https://review.openstack.org/23605819:07
openstackgerritVitaly Gridnev proposed openstack/sahara: Revert "Add -f option to formatting volumes"  https://review.openstack.org/23605819:07
vgridnevtellesnobrega, did you mean this https://github.com/openstack/sahara/blob/master/sahara/conductor/objects.py#L136 ?19:08
*** hdd has joined #openstack-sahara19:10
tellesnobregavgridnev, i don't think that is the place, it clearly merges configs there, but from what i can see, i have to set this some place else19:10
elmikotellesnobrega: looking at the vanilla plugin, i don't see the ports getting set per node group, i just see them getting set for the cluster19:12
tellesnobregaelmiko, and it makes sense to me to be that way19:13
elmikoyea19:13
tellesnobregamaybe i'm looking at the wrong place here19:13
elmikolooking at the vanilla impl, it just uses the process names from the node group to map the ports19:16
elmikosee, https://github.com/openstack/sahara/blob/master/sahara/plugins/vanilla/hadoop2/config.py#L36419:17
tellesnobregahum, looking19:24
tellesnobregaelmiko, got it19:28
tellesnobregait was really, really simple and dumb mistake19:28
tellesnobregai wrote master instead of nimbus (storm master process name) on the get_open_port19:28
tellesnobregaso it was not matching19:28
tellesnobregaonce you said name I went to check that out, it looks like it worked19:29
*** weihan has joined #openstack-sahara19:30
elmikonice, i love the easy fixes ;)19:30
tellesnobregai love the easy fix, but hate the easy problem that you cant find19:31
tellesnobregaand you fill stupid when you do19:31
elmikoi think we've all been there19:31
tellesnobregafor sure19:33
tellesnobregaworked19:33
tellesnobrega:D19:33
tellesnobregathank you all tmckay, elmiko and vgridnev19:33
tmckayheh19:34
openstackgerritMerged openstack/sahara: Updated from global requirements  https://review.openstack.org/23544719:35
elmikotellesnobrega: nice19:40
*** psusi has quit IRC19:41
*** vsham201 has quit IRC19:45
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/sahara: Get Open Ports for Storm  https://review.openstack.org/23607520:04
tellesnobregavgridnev, i wrote the yaml files for storm scenario and the job as well, i'm not sure if its correct but i'll commit so you can take a look20:06
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/sahara: Adding Storm scenario and CI job  https://review.openstack.org/23607720:11
tellesnobregalook like i would have to add an storm image to CI20:11
tellesnobregaif you want i can send you one20:11
*** raildo is now known as raildo-afk20:59
*** dave-mccowan has quit IRC21:11
*** crobertsrh is now known as _crobertsrh21:18
*** tmckay has left #openstack-sahara21:18
*** hdd has quit IRC21:41
*** hdd has joined #openstack-sahara21:44
*** psusi has joined #openstack-sahara21:48
*** david-ly_ has joined #openstack-sahara21:50
*** david-lyle has quit IRC21:50
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/sahara: Get Open Ports for Storm  https://review.openstack.org/23607521:59
*** stevemar_ has quit IRC22:01
*** stevemar_ has joined #openstack-sahara22:02
*** stevemar_ has quit IRC22:06
*** adelia_ has joined #openstack-sahara22:07
*** stevemar_ has joined #openstack-sahara22:07
*** adelia has quit IRC22:10
*** adelia_ has quit IRC22:12
*** stevemar_ has quit IRC22:12
*** vgridnev has quit IRC22:14
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/sahara: Get Open Ports for Storm  https://review.openstack.org/23607522:14
openstackgerritMerged openstack/sahara: Add unit tests for AuthValidator middleware  https://review.openstack.org/23338622:27
*** hdd has quit IRC22:42

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!