Tuesday, 2015-02-24

*** mahito has joined #openstack-sahara00:06
*** hdd has joined #openstack-sahara00:06
*** tnovacik has quit IRC00:09
*** devlaps has quit IRC01:11
*** hdd has quit IRC01:24
*** mattf is now known as _mattf01:30
*** hdd has joined #openstack-sahara01:35
*** jamielennox is now known as jamielennox|away01:46
*** jamielennox|away is now known as jamielennox02:12
*** mahito_ has joined #openstack-sahara02:30
*** mahito has quit IRC02:31
*** macjack_afk has quit IRC03:01
*** macjack has joined #openstack-sahara03:12
*** hdd has quit IRC04:02
*** Poornima has joined #openstack-sahara04:04
*** devlaps has joined #openstack-sahara04:17
*** akuznetsov has joined #openstack-sahara05:06
*** devlaps has quit IRC05:17
*** hdd has joined #openstack-sahara05:23
*** jamielennox is now known as jamielennox|away05:33
*** jamielennox|away is now known as jamielennox05:34
*** sgotliv has quit IRC05:34
*** coolsvap_ is now known as coolsvap05:57
*** atmark has joined #openstack-sahara06:04
atmarkHi guys, I'm getting this error when I do 'shara cluster-list'06:06
atmarkERROR: Failed to parse response from Sahara. Check if service catalog configured properly.06:06
atmarksahara*06:06
*** hdd has quit IRC06:08
*** hdd has joined #openstack-sahara06:13
*** tnovacik has joined #openstack-sahara06:15
*** atmark has left #openstack-sahara06:35
*** akuznetsov has quit IRC06:52
*** sgotliv has joined #openstack-sahara07:20
*** sgotliv has quit IRC07:26
*** akuznetsov has joined #openstack-sahara07:33
*** chlong has quit IRC07:34
*** coolsvap is now known as coolsvap_07:36
*** coolsvap_ is now known as coolsvap07:39
*** hdd has quit IRC07:40
*** sgotliv has joined #openstack-sahara07:42
*** akuznetsov has quit IRC07:44
*** tnovacik_ has joined #openstack-sahara07:59
*** akuznetsov has joined #openstack-sahara08:01
*** akuznetsov has quit IRC08:17
openstackgerritVitaly Gridnev proposed openstack/sahara: Event log supported in new integration tests  https://review.openstack.org/15695608:18
openstackgerritVitaly Gridnev proposed openstack/sahara: Applying event log feature for CDH - part 3  https://review.openstack.org/15860408:21
openstackgerritVitaly Gridnev proposed openstack/sahara: Applying event log feature for CDH - part 3  https://review.openstack.org/15860408:22
*** witlessb has joined #openstack-sahara08:35
*** witlessb has quit IRC08:39
*** witlessb has joined #openstack-sahara08:40
*** akuznetsov has joined #openstack-sahara08:53
*** IvanBerezovskiy_ has joined #openstack-sahara08:57
*** tnovacik has quit IRC09:24
*** openstackgerrit has quit IRC09:36
*** openstackgerrit has joined #openstack-sahara09:36
*** mahito_ has quit IRC09:42
*** tnovacik has joined #openstack-sahara09:45
*** tnovacik has quit IRC09:45
openstackgerritVitaly Gridnev proposed openstack/sahara: Applying event log feature for CDH - part 3  https://review.openstack.org/15860410:29
*** tnovacik has joined #openstack-sahara10:37
*** jamielennox is now known as jamielennox|away10:42
openstackgerritSergey Reshetnyak proposed openstack/sahara: Add validation in new integration tests  https://review.openstack.org/15168010:47
*** tosky has joined #openstack-sahara10:50
SergeyLukjanovtellesnobrega, hey, do you have any updates on how storm edp is going? (I'm just checking blueprints status)10:53
openstackgerritSergey Reshetnyak proposed openstack/sahara: Add concurrency support in new integration tests  https://review.openstack.org/15128810:54
*** jamielennox|away is now known as jamielennox10:58
*** jamielennox is now known as jamielennox|away11:01
*** jamielennox|away is now known as jamielennox11:02
openstackgerritVitaly Gridnev proposed openstack/sahara: Add ability to get step id from context  https://review.openstack.org/15863411:06
*** jamielennox is now known as jamielennox|away11:08
*** sgotliv has quit IRC11:08
*** jamielennox|away is now known as jamielennox11:10
openstackgerritEvgeny Sikachev proposed openstack/sahara: Add unit-tests for new integration tests  https://review.openstack.org/15529811:17
*** sgotliv has joined #openstack-sahara11:20
openstackgerritSergey Lukjanov proposed openstack/sahara: Remove never unused code (timed decorator)  https://review.openstack.org/15864911:34
*** Poornima has quit IRC11:37
*** Poornima has joined #openstack-sahara11:38
*** jamielennox is now known as jamielennox|away11:38
tellesnobregaSergeyLukjanov, i started implementing it, but i had some priorities changed (against my will) but i will get right back at it, this week or the next11:40
openstackgerritVitaly Gridnev proposed openstack/sahara: Event log supported in new integration tests  https://review.openstack.org/15695611:40
*** jamielennox|away is now known as jamielennox11:41
openstackgerritEvgeny Sikachev proposed openstack/sahara: Add unit-tests for new integration tests  https://review.openstack.org/15529811:46
*** chandankumar has joined #openstack-sahara11:46
*** sgotliv has quit IRC11:51
*** sgotliv has joined #openstack-sahara11:51
*** chlong has joined #openstack-sahara11:56
SergeyLukjanovtellesnobrega, okay, that's awesome, thx for the update11:56
tellesnobregaSergeyLukjanov, no problem11:56
tellesnobregaSergeyLukjanov, I'm proposing a google summer of code project this year to make storm fine and completely usable11:57
tellesnobregaSergeyLukjanov, if i get accepted elmiko and eggaford have agreed to mentor me in the process. So hopefully i will have some dedicated time to work just on storm implementation, make it an awesome addition to sahara11:59
SergeyLukjanovtellesnobrega, cool!12:00
openstackgerritVitaly Gridnev proposed openstack/sahara: Add ability to get step id from context  https://review.openstack.org/15863412:17
openstackgerritVitaly Gridnev proposed openstack/sahara: Add ability to get step id from context  https://review.openstack.org/15863412:19
*** tnovacik has quit IRC12:29
*** sgotliv has quit IRC12:37
*** chlong has quit IRC12:44
*** akuznetsov has quit IRC12:44
*** Longgeek has joined #openstack-sahara13:12
*** sreshetnyak has joined #openstack-sahara13:18
*** jamielennox is now known as jamielennox|away13:18
*** chandankumar has quit IRC13:20
*** jamielennox|away is now known as jamielennox13:20
*** sgotliv has joined #openstack-sahara13:27
tellesnobregahey, what is the minimum flavor in sahara for spawning a hadoop 2.6.0 cluster?13:30
*** sgotliv has quit IRC13:31
*** sgotliv has joined #openstack-sahara13:34
*** sgotliv has quit IRC13:43
*** sgotliv has joined #openstack-sahara13:43
*** tnovacik_ has quit IRC13:46
*** _crobertsrh is now known as crobertsrh13:55
*** chandankumar has joined #openstack-sahara14:17
*** tosky has quit IRC14:24
*** chandankumar_ has joined #openstack-sahara14:36
*** chandankumar has quit IRC14:36
openstackgerritEvgeny Sikachev proposed openstack/sahara: Add support of several scenario files in integration tests  https://review.openstack.org/15871014:44
*** tmckay has joined #openstack-sahara14:54
crobertsrhI can't seem to get my ci-recheck to fire for https://review.openstack.org/#/c/157460/14:55
*** egafford has joined #openstack-sahara14:56
elmikolooks like the recheck from yesterday ran14:57
elmikohmm, not the ci though. that's odd14:57
*** tnovacik_ has joined #openstack-sahara14:59
crobertsrhYeah, I noticed that15:06
*** tosky has joined #openstack-sahara15:08
*** devlaps has joined #openstack-sahara15:24
*** devlaps has quit IRC15:24
*** chandankumar_ has quit IRC15:45
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Divide jobs log to different files  https://review.openstack.org/15873715:48
*** chandankumar has joined #openstack-sahara15:48
openstackgerritArtem Osadchiy proposed openstack/sahara: Add bare images support for MapR plugin  https://review.openstack.org/15813915:53
*** chandankumar has quit IRC15:54
*** coolsvap is now known as coolsvap_16:02
*** jamielennox is now known as jamielennox|away16:15
*** jamielennox|away is now known as jamielennox16:16
openstackgerritMerged openstack/sahara-specs: Allow placeholders in datasource URLs  https://review.openstack.org/15064016:19
openstackgerritOpenStack Proposal Bot proposed openstack/python-saharaclient: Updated from global requirements  https://review.openstack.org/15558816:22
openstackgerritOpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements  https://review.openstack.org/15875916:22
openstackgerritOpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements  https://review.openstack.org/15877516:31
*** Poornima has quit IRC16:33
*** tnovacik_ has quit IRC16:49
*** egafford has quit IRC16:52
*** egafford has joined #openstack-sahara16:53
openstackgerritSergey Lukjanov proposed openstack/sahara: Remove unused code (timed decorator)  https://review.openstack.org/15864916:55
*** tnovacik_ has joined #openstack-sahara16:56
*** IvanBerezovskiy_ has quit IRC17:15
openstackgerritMerged openstack/sahara: Add Flume support for MapR plugin  https://review.openstack.org/14810217:20
*** coolsvap_ is now known as coolsvap17:20
*** hdd has joined #openstack-sahara17:22
*** chandankumar has joined #openstack-sahara17:52
*** tnovacik_ has quit IRC17:59
*** coolsvap is now known as coolsvap_18:00
*** devlaps has joined #openstack-sahara18:11
*** Longgeek has quit IRC18:15
openstackgerritSergey Lukjanov proposed openstack/sahara: Add support for oslo_debug_helper to tox.ini  https://review.openstack.org/15881218:23
*** chandankumar has quit IRC18:44
*** amcrn has joined #openstack-sahara18:45
*** tnovacik_ has joined #openstack-sahara18:45
openstackgerritAndrew Lazarev proposed openstack/sahara: Minor - Added missing check for 'Deleting' state  https://review.openstack.org/15586118:49
openstackgerritChad Roberts proposed openstack/sahara: Adding ability to edit cluster templates  https://review.openstack.org/15746019:05
*** Longgeek has joined #openstack-sahara19:15
*** hdd has quit IRC19:24
openstackgerritChad Roberts proposed openstack/sahara: Adding ability to edit cluster templates  https://review.openstack.org/15746019:26
crobertsrhtmckay:  Not sure which mechanism for exception trapping in test cases is better.19:27
elmikoi don't think there is a "better", they both do the same thing. but provide some nice side benefits depending on what type of behavior you need.19:28
crobertsrhI kind of like the testtools version, but I'd live if we went the other way.19:28
elmikoassertRaises can be used the same way though19:29
elmikohttps://docs.python.org/3/library/unittest.html19:29
crobertsrhI'm guessing that different places use the different approaches based mostly on who wrote the original test in each class.19:32
crobertsrhHmm, maybe not...I'm seeing at least one case of the same person using different ways.19:33
*** devlaps has quit IRC19:33
*** amcrn has quit IRC19:34
elmikoyea, i think people just use whatever they think of at the moment =)19:34
crobertsrhWell, there ya have it....Openstack hacking says "thou shalt not use assertRaises"19:34
elmikoi don't think that's what they are saying though19:35
tmckayjust sent an email about stable branches and requirements changes from proposal bot19:35
tmckaynot one of the covered cases on the stable wiki19:35
crobertsrhoh, that's just for broad "Exception19:35
elmikocrobertsrh: yea... that's the real point19:35
tmckaycan't find paper on requirements changes to stable branches19:35
elmikoyou can do the same thing with assertRaises that we are doing with testtools19:35
crobertsrhYeah, as long as we don't do "Exception"....we are probably fine19:35
elmikoright19:36
elmikoi don't think it really matters, people should just use what they are comfortable with19:37
elmikoi do see how the "with self.assertRaises..." can be really useful if you need to do more inside the context or need to examine the exception19:37
elmikobut we don't really use it like that, so, no harm no foul imo19:38
crobertsrhmove along, nothing to see here19:38
elmikolol19:40
elmikoit's a good question, but it seems the only answer is that we become more educated about the different usages19:41
*** hdd has joined #openstack-sahara19:43
elmikotmckay: good question, re: stable and proposal bot19:43
tmckayelmiko, yeah, I mean, who wrote that thing?  What rules does it follow?  Why should I believe it? :)19:47
tmckaybumping requirement for stable branches seems weird to me19:47
tmckayhow does that apply?  New installs only?  What about existing installs? They'll be running with the old stuff19:47
tmckayor is it about the gate?19:48
elmikodid you see you already got a response to that email?19:49
tmckayno :)19:50
* tmckay lloks19:50
tmckayawesome19:50
tmckayI didn't know there was a requirements stable branch19:52
tmckayI should take a closer look19:52
elmikoa whole new world =)19:52
tmckayI assumed the answer would be something like that, but there weren't a lot of words about it19:53
tmckayI forget that requirements is it's own project19:54
tmckaycrobertsrh, on default templates, the ACL support for templates mentioned in the spec -- do you know the status of that? I don't see a spec for it (reviews or merged specs)20:14
openstackgerritVitaly Gridnev proposed openstack/sahara: Apply event-log feature for Vanilla plugins  https://review.openstack.org/14997220:15
crobertsrhLast I heard was the Sergey was going to be writing it up....I thought about it yesterday and meant to ping him.20:16
crobertsrhWithout ACL stuff, we may need to get a bit creative20:16
*** hdd has quit IRC20:16
tmckaycrobertsrh, hmm.  I wonder if we can patch over it, for dev at least.  The sahara admin user should be able to read anything from the db, it could ignore tenant20:19
tmckayjust return everything for everyone assuming the default marker is on20:19
crobertsrhYeah, that part is ok, but we need to cover edit/delete as well20:19
*** tosky has quit IRC20:19
crobertsrhneed to figure out a way to not allow edit/delete of those default templates20:20
tmckaythat's easy, if we have a marker field in the db20:20
crobertsrhYeah, easy if we have it...harder if we dont20:20
tmckaywrite once, read many. You can't modify an old one, only upload a newer version20:21
tmckayand the old versions I suppose can be purged as soon as they are not referenced20:21
tmckay(that might be a little harder)20:21
crobertsrhYeah, that may be a bit trickier20:21
*** egafford has quit IRC20:22
tmckaycould be a special op on upload of a new default20:22
tmckayhmm, well, I guess you'd need a periodic task too20:22
crobertsrhYeah, it would have to be part of the "updater"20:22
*** egafford has joined #openstack-sahara20:22
crobertsrhMaybe we're making this too hard...let me peek at the spec again.20:23
tmckayso in summary, we've got a CLI that talks directly to the database20:23
tmckayhmm, seems like we probably want to extend at least the client API for this (but not the REST api)20:24
crobertsrhYeah, the REST api might not be desirable20:25
tmckayI'll have to look at that more closely20:25
tmckayfingers in the db seems like a bad plan, likewise20:25
tmckayhmm, I don't think we have one, per se.  all the client stuff (including that used by the integration tests) is ultimately built on REST20:29
tmckayI wonder what sahara-db-manage does ...20:29
tmckayit's got its own cli under sahara.db.migration.  Okey doke.20:31
* tmckay is thinking out loud for the ages20:31
crobertsrhheh20:32
crobertsrhmaybe SergeyLukjanov can update us on the status of the "ACL Spec"/Impl?20:33
tmckayack.  he may have something hidden away20:33
crobertsrhIf we have that, things get slightly more sane.20:34
tmckayI think I need a CLI that ties into the conductor.  I'll have to go poke around and find the right level.20:34
*** vgridnev_ has joined #openstack-sahara20:35
*** hdd has joined #openstack-sahara20:39
tmckaycrobertsrh, hmm, we have this possibility for a remote database but it's never been implemetned20:40
crobertsrhwhaa??20:40
tmckaythat's what the conductor is all about20:40
tmckaybut it's de facto all local20:41
crobertsrhYou're thinking that this alternate db would be used for the default templates?20:41
elmikois that different than the db specified with the connection string?20:41
tmckaywell, theoretically *any* db op should be supported in a remote location.  So all db stuff should go through the conductor20:41
tmckayelmiko, should be the same.  But it does bring up the question, what happens if I point my Sahara at a database on another node.  Does it work anyway?20:42
tmckaybecause sqlalchemy is smart20:42
elmikoyes it does, i've done it20:42
elmikojust change the connection string to a different location20:43
tmckayhmmm20:43
crobertsrhYeah...I've done something similar before20:43
tmckayso I wonder what this conductor is really about20:43
elmikoi thought the conductor was to add a layer of abstraction between the api and the impl?20:43
elmikowhat's that pattern called, mediator or some such20:44
tmckaybut it has a local/remote possibility.  so you can potentially have another process somewhere else handling db ops for you20:45
elmikocool20:45
tmckaybut, if sqlalchemy can already do that ... not sure I'm understanding the motivation20:45
tmckaymaybe open discussion fodder for Thursday20:45
elmikoassuming it's an abstraction layer, i would think the point is to allow the api and impl to change freely without affecting each other. but maybe there is a history there that we are unaware of.20:46
tmckayright, well, you still have the abstraction.  Because conductor has 2 classes, local api and remote api.  Remote api is to date unimplemented20:47
tmckayjust inherits from local and doesn't add anything20:47
elmikointeresting, maybe just leaving the door open for future improvement?20:47
tmckaythe idea may have evolved away20:47
elmikothat too20:47
tmckaywe've already got a distributed mode, and sqlalchemy handling remote dbs, the conductor adds ... ?  It may be cool, I'm just not seeing it at the moment.20:48
elmikoi always assumed it was abstraction to reduce the entanglement of api and impl layers20:50
crobertsrhelmiko:  That was what I figured was going on.20:50
tmckayconductor -> manager -> database20:51
tmckayyou've got that either way.  Here it is saying that manager can be local, or a separate process20:51
tmckayit's the separate process part that I'm wondering about20:51
elmikoah20:52
elmikosounds like a SergeyLukjanov question ;)20:52
elmikoor maybe alazarev knows too20:52
tmckayelmiko, I was just lamenting the fact that when this was added I don't think we were using the spec model20:52
elmikoyea, too bad20:52
tmckayso harder to find paper20:52
tmckaybp was ng-arch-conductor20:52
alazarevI don't know details20:53
tmckayelmiko, croberstrh, do you know how to find bps that are not open any longer?20:53
elmikoi like the result of git log on the conductor package, tmckay is the earliest commit in the current repo lol20:53
alazarevit is as now for a long time20:54
tmckayheh20:54
elmikotmckay: maybe search on implemented bps?20:54
tmckayalazarev, ack, I wonder if the idea was superseded by distributed mode20:54
elmikotmckay: https://blueprints.launchpad.net/sahara/+specs?show=all shows all of them20:55
alazarevtmckay, yeap, to have ability to have single point of DB access20:55
elmikohttps://blueprints.launchpad.net/sahara/+spec/ng-arch-conductor20:55
tmckayelmiko, thanks20:55
elmikotmckay: looks like need to ping dmitryme20:55
tmckayooo, there is a full specification20:56
tmckayI thought I had seen that.  I'll read20:56
tmckayelmiko, this came up because if I'm adding a CLI for default templates, what layer in Sahara should I tap into>20:56
tmckayI think maybe conductor20:56
elmikotmckay: cli as in the "sahara" command?20:57
tmckaynew tool20:57
elmikoahh, k20:57
tmckayno REST api, just semi-direct db manipulation20:57
tmckayimportant to get it at the right level20:57
elmikoi would think conductor would make sense, take it at the same level that rest api calls use20:57
tmckaythanks for the help finding this, I read this once upon a time20:58
alazarevtmckay, will it be official tool?20:58
tmckayalazarev, yes, I think so.  It's part of the default templates specification.  Either run it as an admin, or hook it up to a cron job20:58
* tmckay looks for spec 20:58
tmckayhttps://review.openstack.org/#/c/141178/3/specs/kilo/default-templates.rst20:59
tmckayalazarev, so, similar to sahara-db-manage20:59
elmikoalazarev: while you are here, do you i think i should rewrite the improved-secret-storage spec to remove the swift replacement and just focus on creating the secrets module?20:59
tmckaythe question is how to best connect to Sahara20:59
tmckaytime to go, before the weather worsens again.  bye21:00
alazarevtmckay, ack, I read spec, looks great21:00
tmckayalazarev, if you have ideas about how to tie cli into Sahara, please comment on the spec (or email or bp)21:00
tmckayjust starting, I haven't done anything yet :)21:02
*** hdd has quit IRC21:02
*** tmckay has left #openstack-sahara21:02
alazarevelmiko, I think so, the first step could be identifying list of secrets, this is not only swift21:03
elmikoalazarev: agreed, it could take a long time to identify all the areas21:03
alazarevelmiko, but swift is a good point to start21:04
elmikoalazarev: i figure, once the module is implemented we could make bugs to address individual areas that need to use the secret store21:04
*** hdd has joined #openstack-sahara21:05
elmikoalazarev: thanks for the opinion, i think i'll rewrite it21:05
*** Longgeek has quit IRC21:24
openstackgerritAndrew Lazarev proposed openstack/sahara: Minor - Added missing check for 'Deleting' state  https://review.openstack.org/15586121:35
*** tnovacik_ has quit IRC21:40
openstackgerritMerged openstack/sahara: Updated from global requirements  https://review.openstack.org/15875921:44
*** vgridnev_ has quit IRC21:56
openstackgerritMichael McCune proposed openstack/sahara-specs: Adding improved secret storage spec  https://review.openstack.org/15743222:02
*** crobertsrh is now known as _crobertsrh22:05
*** amcrn has joined #openstack-sahara22:09
* SergeyLukjanov lurking22:17
SergeyLukjanovhey folks22:17
SergeyLukjanovelmiko, is your question answered?22:17
elmikoSergeyLukjanov: i think so, i rewrote the secret store spec to remove the swift stuff22:18
SergeyLukjanovcool!22:18
elmikoSergeyLukjanov: tmckay had some questions about the sahara.conductor package, but i think he may have found some answers22:19
elmikoonce the secret store is in place, i'll open a bug to fix the swift stuff22:19
elmikoand then hopefully we can open more bugs as we find room for improvement22:19
*** ViswaV has joined #openstack-sahara22:28
*** openstackgerrit has quit IRC22:37
*** openstackgerrit has joined #openstack-sahara22:37
*** tmckay has joined #openstack-sahara22:45
openstackgerritAndrew Lazarev proposed openstack/sahara: Implemented support of placeholders in datasource URLs  https://review.openstack.org/15890923:00
openstackgerritAndrew Lazarev proposed openstack/sahara: Minor - changed name of argument in mirgation tests  https://review.openstack.org/15891123:03
*** Longgeek has joined #openstack-sahara23:09
*** chlong has joined #openstack-sahara23:19
*** oikawa has joined #openstack-sahara23:30
*** witlessb has quit IRC23:31
*** Longgeek has quit IRC23:42
*** chlong_ has joined #openstack-sahara23:50

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!