| *** mahito has joined #openstack-sahara | 00:05 | |
| *** macjack has joined #openstack-sahara | 00:26 | |
| openstackgerrit | Merged openstack/sahara: Add Key Value Store service test in cdh plugin integration test https://review.openstack.org/158940 | 00:45 |
|---|---|---|
| openstackgerrit | Ken Chen proposed openstack/sahara: Add CM API lib into CDH plugin codes https://review.openstack.org/153806 | 01:13 |
| *** himangi has joined #openstack-sahara | 01:55 | |
| *** hdd has joined #openstack-sahara | 04:02 | |
| *** hdd has quit IRC | 04:15 | |
| *** himangi has quit IRC | 04:18 | |
| *** himangi has joined #openstack-sahara | 04:30 | |
| *** chandankumar has joined #openstack-sahara | 04:40 | |
| *** coolsvap_ is now known as coolsvap | 04:43 | |
| *** hdd has joined #openstack-sahara | 04:45 | |
| *** hdd has quit IRC | 04:52 | |
| *** chandankumar has quit IRC | 05:06 | |
| *** chandankumar has joined #openstack-sahara | 05:48 | |
| *** hdd has joined #openstack-sahara | 05:59 | |
| *** hdd has quit IRC | 06:07 | |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara: Add Sqoop support for MapR plugin https://review.openstack.org/148100 | 06:13 |
| *** tnovacik has joined #openstack-sahara | 06:36 | |
| *** himangi has quit IRC | 06:44 | |
| *** tnovacik has quit IRC | 06:53 | |
| *** hogepodge has quit IRC | 06:55 | |
| *** himangi has joined #openstack-sahara | 06:58 | |
| *** sgotliv has quit IRC | 07:11 | |
| *** mahito has quit IRC | 07:29 | |
| *** hogepodge has joined #openstack-sahara | 07:49 | |
| *** chlong has quit IRC | 08:02 | |
| openstackgerrit | Denis Egorenko proposed stackforge/sahara-ci-config: Migrate jobs to new integrations tests config https://review.openstack.org/155301 | 08:03 |
| *** witlessb has joined #openstack-sahara | 08:15 | |
| *** sgotliv has joined #openstack-sahara | 08:24 | |
| *** himangi has quit IRC | 08:27 | |
| *** sgotliv has quit IRC | 08:38 | |
| *** sgotliv has joined #openstack-sahara | 08:38 | |
| *** Poornima has joined #openstack-sahara | 08:49 | |
| *** tnovacik has joined #openstack-sahara | 09:00 | |
| *** skolekonov has joined #openstack-sahara | 09:29 | |
| *** akuznetsov has joined #openstack-sahara | 09:49 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Applying event log feature for CDH - part 3 https://review.openstack.org/158604 | 09:54 |
| *** chandankumar_ has joined #openstack-sahara | 10:06 | |
| *** chandankumar has quit IRC | 10:08 | |
| *** tosky has joined #openstack-sahara | 10:22 | |
| openstackgerrit | Nikita Konovalov proposed openstack/python-saharaclient: Add support for show_events parameter https://review.openstack.org/159451 | 10:39 |
| *** akuznetsov has quit IRC | 10:45 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Apply event-log feature for Vanilla plugins https://review.openstack.org/149972 | 10:46 |
| *** Poornima has quit IRC | 10:54 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Implement poll util and plugin poll util https://review.openstack.org/157392 | 10:58 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Implement poll util and plugin poll util https://review.openstack.org/157392 | 11:02 |
| *** sgotliv has quit IRC | 11:14 | |
| *** sgotliv has joined #openstack-sahara | 11:27 | |
| *** tnovacik has quit IRC | 11:32 | |
| openstackgerrit | lu huichun proposed openstack/sahara: Add Solr service test in cdh plugin integration test https://review.openstack.org/155857 | 11:37 |
| *** sgotliv_ has joined #openstack-sahara | 11:45 | |
| *** sgotliv has quit IRC | 11:48 | |
| *** tnovacik has joined #openstack-sahara | 11:57 | |
| openstackgerrit | lu huichun proposed openstack/sahara: Add Sentry service test in cdh plugin integration test https://review.openstack.org/157915 | 11:58 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Implement poll util and plugin poll util https://review.openstack.org/157392 | 12:03 |
| openstackgerrit | lu huichun proposed openstack/sahara: Add Sentry service test in cdh plugin integration test https://review.openstack.org/157915 | 12:03 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Implement poll util and plugin poll util https://review.openstack.org/157392 | 12:06 |
| *** coolsvap is now known as coolsvap_ | 12:06 | |
| openstackgerrit | Merged openstack/sahara-specs: Add a common HBase lib in hdfs on cluster start https://review.openstack.org/155899 | 12:10 |
| openstackgerrit | Merged openstack/sahara-specs: [EDP] Add Oozie Shell Action job type https://review.openstack.org/156795 | 12:14 |
| *** coolsvap_ is now known as coolsvap | 12:15 | |
| openstackgerrit | Merged openstack/sahara: Add bare images support for MapR plugin https://review.openstack.org/158139 | 12:21 |
| openstackgerrit | Merged openstack/sahara: Add Sqoop support for MapR plugin https://review.openstack.org/148100 | 12:22 |
| *** tmckay has quit IRC | 12:31 | |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara: Add Impala support for MapR plugin https://review.openstack.org/148114 | 12:48 |
| *** sgotliv_ has quit IRC | 12:48 | |
| *** sgotliv has joined #openstack-sahara | 12:51 | |
| *** tnovacik has quit IRC | 12:53 | |
| *** hdd has joined #openstack-sahara | 12:59 | |
| openstackgerrit | Pierre Padrixe (stannie) proposed openstack/python-saharaclient: Add cluster-delete by multiple names or ids https://review.openstack.org/149821 | 13:01 |
| *** himangi has joined #openstack-sahara | 13:01 | |
| *** hdd has quit IRC | 13:03 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Implement poll util and plugin poll util https://review.openstack.org/157392 | 13:10 |
| *** hdd has joined #openstack-sahara | 13:16 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara-image-elements: Install openjdk-jdk instead openjdk-jre for Ubuntu https://review.openstack.org/160337 | 13:18 |
| *** hdd has quit IRC | 13:31 | |
| *** hdd has joined #openstack-sahara | 13:31 | |
| *** akuznetsov has joined #openstack-sahara | 13:35 | |
| *** hdd has quit IRC | 13:36 | |
| *** _crobertsrh is now known as crobertsrh | 13:41 | |
| *** chandankumar_ has quit IRC | 13:46 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara-image-elements: Migrate to openjdk - part 2 https://review.openstack.org/146434 | 13:48 |
| *** tmckay has joined #openstack-sahara | 13:51 | |
| *** IBerezovskiy has joined #openstack-sahara | 13:51 | |
| openstackgerrit | Merged openstack/sahara-specs: Adding improved secret storage spec https://review.openstack.org/157432 | 13:55 |
| openstackgerrit | Nikolay Starodubtsev proposed openstack/sahara: Rewrite log levels and messages https://review.openstack.org/154037 | 13:55 |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara-image-elements: Use openjdk by default for java element https://review.openstack.org/159473 | 13:57 |
| openstackgerrit | Sergey Reshetnyak proposed stackforge/sahara-ci-config: Revert "Make tempest-sahara-tests jobs non-voting" https://review.openstack.org/160347 | 14:01 |
| openstackgerrit | Denis Egorenko proposed stackforge/sahara-ci-config: Migrate jobs to new integrations tests config https://review.openstack.org/155301 | 14:05 |
| *** dmitryme has joined #openstack-sahara | 14:36 | |
| *** egafford has joined #openstack-sahara | 14:40 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Changed wrong value for total during step creation https://review.openstack.org/159758 | 14:53 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Changed wrong value for total during step creation https://review.openstack.org/159758 | 14:54 |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara: Add Hue support for MapR plugin https://review.openstack.org/148117 | 15:02 |
| openstackgerrit | Denis Egorenko proposed stackforge/sahara-ci-config: Migrate jobs to new integrations tests config https://review.openstack.org/155301 | 15:10 |
| tmckay | SergeyLukjanov, ping, are you around? | 15:13 |
| SergeyLukjanov | tmckay, yeah, good morning | 15:13 |
| tmckay | crobertsrh, listen to this too :) | 15:13 |
| crobertsrh | listen to what? | 15:13 |
| elmiko | <sounds of tmckay typing> | 15:14 |
| tmckay | SergeyLukjanov, crobertsrh, I am thinking about template ACLs for default templates. I have a (rough) idea | 15:14 |
| crobertsrh | ah | 15:14 |
| tmckay | What if we create a Sahara "phantom" tenant_id (could be anything, it's a string) | 15:14 |
| tmckay | Default templates are added with the phantom id | 15:14 |
| tmckay | We have a separate table to track default templates, and the ACL tenant list is in that table | 15:14 |
| tmckay | When we retrieve templates, there is an extra step to look up the defaults and map the tenant_id of the current user into the template when it is retrieved (if they are on the ACL list) | 15:15 |
| tmckay | this way, we can easily support tenant ACLs without really adding anything to the template records | 15:16 |
| elmiko | is this ACL in addition to the policy.json? | 15:17 |
| tmckay | so, templates would be in the db with "sahara_private_tenant" or "sahara_pseduo_tenant" or something like that for tenant id | 15:17 |
| tmckay | elmiko, yes. This is a way to have objects visible to more than one tenant | 15:17 |
| tmckay | which we currently don't have | 15:18 |
| tmckay | SergeyLukjanov, do you have an idea for ACL support for tenants? Default template spec says "Sergey will write" :) | 15:18 |
| crobertsrh | At one point, SergeyLukjanov mentioned setting up some sort of ACL that [at the time] we thought would be useful for this. | 15:18 |
| tmckay | I mean, for templates | 15:18 |
| crobertsrh | :) | 15:19 |
| openstackgerrit | Ethan Gafford proposed openstack/sahara: [EDP] Add Oozie Shell Job Type https://review.openstack.org/159920 | 15:19 |
| SergeyLukjanov | tmckay, I thought about it and unfortunately was unable to find enough time to work on it | 15:19 |
| tmckay | I can add to the default template spec for this if we think it's a good idea, or make it a separate spec | 15:19 |
| tmckay | SergeyLukjanov, do you think this idea is good enough for a first pass? | 15:19 |
| SergeyLukjanov | tmckay, my idea was to have configurable through the policy.json ACL for all sahara objects | 15:19 |
| tmckay | ah, I see | 15:19 |
| elmiko | +1 for separate spec, i think this could get complicated | 15:20 |
| tmckay | I am a little worried that we are running out of time for complex features | 15:20 |
| SergeyLukjanov | tmckay, I'm afraid that with such pass will be unable to migrate it to the more complex solution like ACLs | 15:20 |
| SergeyLukjanov | tmckay, could we just create templates and move ACLs to next release? | 15:20 |
| elmiko | SergeyLukjanov: +1 | 15:21 |
| SergeyLukjanov | IMO it'll be ok to have default templates that are working like all other tmeplates | 15:21 |
| SergeyLukjanov | as a first step | 15:21 |
| tmckay | SergeyLukjanov, yes. I suppose until then we can just create the defaults under each tenant | 15:21 |
| SergeyLukjanov | tmckay, exactly | 15:21 |
| tmckay | just run the script multiple times | 15:21 |
| tmckay | Pull all the current tenants from keystone and loop :) If you want. | 15:22 |
| crobertsrh | that sounds kinda ugly | 15:22 |
| SergeyLukjanov | tmckay, yeah, I saw your CR and it means that if folks need to setup templates for the list of tenants they could make loop in a worst case | 15:22 |
| tmckay | crobertsrh, admin could do it. I'm not suggesting we make that an option. | 15:22 |
| SergeyLukjanov | crobertsrh, yeah, but it sounds better then having nothing about default templates creation or making a temp solution | 15:23 |
| crobertsrh | Ideally, whatever we come up with (if it's not optimal) can be migrated to something better at some point. | 15:23 |
| elmiko | crobertsrh: +1 | 15:23 |
| crobertsrh | I'm concerned that something ugly might wind-up sticking around | 15:24 |
| tmckay | crobertsrh, I don't think so in this case | 15:24 |
| elmiko | crobertsrh: agreed, design something simple and straight-forward for now | 15:24 |
| tmckay | If we follow on with a general feature for object ACLs across tenants, it should just apply | 15:24 |
| openstackgerrit | Nikita Konovalov proposed openstack/python-saharaclient: Add support for show_events parameter https://review.openstack.org/159451 | 15:25 |
| crobertsrh | Ok, if you're thinking it falls in-line, I'm ok with that. | 15:25 |
| *** chandankumar_ has joined #openstack-sahara | 15:25 | |
| tmckay | SergeyLukjanov, okay, thanks | 15:25 |
| SergeyLukjanov | I think general ACLs could be just applied after that (after spec approved and stuf implemented :) ) | 15:26 |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara: Add support for MapR v4.0.2 https://review.openstack.org/160375 | 15:26 |
| SergeyLukjanov | because in my vision it'll be the following types of visibility - tenant (default, current impl), public, protected, default | 15:27 |
| SergeyLukjanov | defaults = read-only public | 15:27 |
| crobertsrh | +1 | 15:28 |
| elmiko | makes sense | 15:28 |
| *** chandankumar_ has quit IRC | 15:31 | |
| tmckay | SergeyLukjanov, I like it, first feature for Liberty :) (Or is it London, I forget) | 15:31 |
| tosky | Liberty, which won over Lizard | 15:33 |
| * tosky voted Lizard | 15:33 | |
| *** skolekonov has quit IRC | 15:43 | |
| tmckay | tosky, I forget if I mentioned, there is a town near here named "Lizard Lick" | 15:45 |
| *** chandankumar_ has joined #openstack-sahara | 15:48 | |
| tosky | tmckay: is it expected that you can remove an internal job binary (job-binary-data-delete) when it's used by a job binary? | 15:51 |
| tmckay | tosky, hmmm. The reference to actual binaries is via URL, and we can't protect the ones stored in Swift. So I think we didn't try to constrain deletion of an internal binary | 15:52 |
| tosky | tmckay: ack | 15:53 |
| tmckay | Hopefully Sahara is handling that case in a nice way, and not crashing | 15:53 |
| tosky | I didn't try to submit a job with this status | 15:54 |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara: Add Hue support for MapR plugin https://review.openstack.org/148117 | 16:02 |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara: Add Hue support for MapR plugin https://review.openstack.org/148117 | 16:04 |
| openstackgerrit | Artem Osadchiy proposed openstack/sahara: Add support for MapR v4.0.2 https://review.openstack.org/160375 | 16:04 |
| *** coolsvap is now known as coolsvap_ | 16:05 | |
| tmckay | anybody know why we have an image_id field in a node_group_template object? Iimage should be selected at cluster launch, not in a node group template, right? | 16:10 |
| tosky | crobertsrh: horizon/sahara interaction, when you create a job binary with a name too long, sahara logs says ' error_name=VALIDATION_ERROR' and the error messages | 16:14 |
| tosky | crobertsrh: but horizon (git master; sahara is juno) shows just "there was an error kthxbye" (ok, not exactly like that, but...) | 16:14 |
| tosky | can horizon retrieve those details? | 16:15 |
| * tosky ready to file a bug | 16:15 | |
| *** chandankumar_ has quit IRC | 16:15 | |
| crobertsrh | Sometimes it can get those details. | 16:15 |
| crobertsrh | Let me check this case really quick. | 16:15 |
| tosky | thanks | 16:16 |
| crobertsrh | I suspect that horizon could show more info, but just isn't at the moment | 16:18 |
| tosky | uhm, worth to file a bug? | 16:18 |
| tosky | is there any special tag to mark sahara-related bugs for horizon in launchpad? | 16:19 |
| tosky | something like [sahara] in the title and that's it? | 16:19 |
| crobertsrh | I think an optional tag of "sahara" works | 16:19 |
| elmiko | you could add sahara to tags, but that might not be proper | 16:19 |
| tosky | ack, thanks | 16:19 |
| crobertsrh | I also use [data processing] in the title | 16:19 |
| tosky | ehm | 16:19 |
| crobertsrh | some use [sahara] | 16:19 |
| tosky | tag or no tag? | 16:19 |
| tosky | :D | 16:19 |
| elmiko | yea [data processing] in title seems appropriate | 16:19 |
| elmiko | imo | 16:19 |
| crobertsrh | I use [data processing] + tag of sahara | 16:19 |
| elmiko | nice, cover all bases | 16:20 |
| tosky | in the meantime, I have another isse: when I try to create another job using an existing binary, it fails with no errors even in sahara | 16:20 |
| elmiko | wierd | 16:20 |
| crobertsrh | Looks like horizon gets back an API Exception with a message of "....too long". It just says "unable to create job binary" | 16:20 |
| crobertsrh | worth a bug | 16:20 |
| tosky | also, when the job binary creating fail, the job binary data is created in the internal db | 16:20 |
| crobertsrh | Validation exception might be as specific as it can reliably be though. | 16:21 |
| tosky | well, even "validation error" is better than "there was an error" I would say :D | 16:21 |
| tosky | aaand one | 16:24 |
| tosky | so, for the other issue (you can't assign an existing internal binary to a new binary) I see an horizon exception, do you know if it's a known issue? (in a minute) | 16:26 |
| tosky | http://paste.openstack.org/show/185206/ | 16:27 |
| tosky | crobertsrh: ^ | 16:27 |
| crobertsrh | Hmm, looking now. Doesn't sound familiar. | 16:28 |
| tosky | funny error message on Horizon side: "Danger: There was an error submitting the form. Please try again. " | 16:28 |
| tosky | DANGER | 16:28 |
| tosky | usually it's Alert, Error, I didn't see Danger too much around :) | 16:29 |
| elmiko | lol | 16:29 |
| elmiko | DANGER! | 16:29 |
| crobertsrh | That's an awesome message | 16:37 |
| crobertsrh | not sure why anyone would complain about that :) | 16:38 |
| crobertsrh | tosky: I'm a bit unclear on what steps you're taking to create that problem. | 16:38 |
| tosky | Danger is more "an asteroid is going to hit this spaceship. Please remain seated." | 16:38 |
| crobertsrh | You should always be seated while operating Sahara | 16:39 |
| tosky | crobertsrh: oh, steps: I create a new job binary, "storage type" internal, "internal binary" an existing one | 16:39 |
| elmiko | crobertsrh: +1 | 16:40 |
| tosky | no upload, no create | 16:40 |
| tosky | push create, see the backtrace | 16:40 |
| crobertsrh | hmm, that is bad. | 16:40 |
| crobertsrh | Leave it to QE to break stuff. | 16:40 |
| tosky | :P | 16:40 |
| elmiko | lol | 16:40 |
| tosky | do you see it as well now? | 16:40 |
| crobertsrh | yep | 16:40 |
| tosky | ok, another bug coming | 16:41 |
| tosky | and this is all trying to *write* a selenium test case | 16:41 |
| crobertsrh | My guess is that it hasn't been tried in awhile and something has broken it along the way + no test of any sort for that :) | 16:41 |
| tosky | sure, sure, the conversion/integration process of the dashboard was painful | 16:41 |
| tosky | unfortunately the old set of tests can't be easily recycled in the new framework, as they tightly mix css selection code and testing code | 16:42 |
| crobertsrh | Heh, I'm pretty sure I see the problem. | 16:42 |
| tosky | while the new integration testing framework tries to abstract from the page | 16:42 |
| crobertsrh | I'm guessing that nobody has ever tried such an operation | 16:42 |
| crobertsrh | There is zero code in there to support it. | 16:44 |
| tosky | oh | 16:45 |
| tosky | bug filed | 16:45 |
| crobertsrh | thanks | 16:45 |
| tosky | do you need the bug number or are you subscribed to them? | 16:46 |
| *** sgotliv has quit IRC | 16:46 | |
| tosky | oh, well: generic error on job creation: https://bugs.launchpad.net/horizon/+bug/1427277 | 16:46 |
| openstack | Launchpad bug 1427277 in OpenStack Dashboard (Horizon) "[sahara] Detailed error on job binary creation is not shown" [Undecided,New] | 16:46 |
| crobertsrh | I get them...thanks though | 16:46 |
| tosky | oki | 16:46 |
| tosky | :) | 16:46 |
| tmckay | crobertsrh, (different topic), hmm, as we were noting the version structure for the plugins is not uniform. What if we just add a "default_tamplates" dir under the root of each plugin? If plugin name and (hadoop) version are required to be in each JSON file (as they would be by validation) we can just read it from the JSON if we want to filter what we apply. | 16:47 |
| tosky | backtrace when an existing job binary is used: https://bugs.launchpad.net/horizon/+bug/1427289 | 16:47 |
| openstack | Launchpad bug 1427289 in OpenStack Dashboard (Horizon) "[sahara] Back trace when a job binary is created using an existing script" [Undecided,Confirmed] | 16:47 |
| crobertsrh | tmckay: I'm ok with that. Simple and uniform. | 16:47 |
| tmckay | crobertsrh, in this case "hadoop_version" usually means plugin version. Unfortunate, we should rename that at some point. Although, it doesn't have to be. We could have multiple plugin versions touching the same hadoop version ... although I don't think anyone has done that yet. | 16:48 |
| crobertsrh | Yeah, the "hadoop_version" migration just keeps getting bigger and bigger :) | 16:48 |
| tosky | tmckay: isn't hadoop_version more the hadoop version for that plugin? | 16:48 |
| tmckay | hmm, actually, maybe we should just change hadoop_version to plugin_version everywhere | 16:48 |
| tmckay | tosky, it is, but we only have one field | 16:48 |
| tmckay | we could have plugin_version and hadoop_version, but we don't | 16:49 |
| tosky | you know that changing everything would break the existing templates? | 16:49 |
| egafford | crobertsrh: Question on Horizon: adding the Shell job type to Sahara itself will not be a breaking change to Horizon's current functionality, true? | 16:49 |
| tmckay | the numbers you see on the UI forms really map to the hadoop_version field | 16:49 |
| crobertsrh | egafford: True | 16:50 |
| tmckay | tosky, ack, we need some compatible way to do this | 16:50 |
| egafford | crobertsrh: Sweet. | 16:50 |
| tmckay | tosky, for instance, the CDH stuff uses 5 and 5.3.0. That's not hadoop version, it's the CDH package version. And there is one plugin version that maps directly to each package version. | 16:50 |
| tmckay | as long as we keep it one to one, it's fine | 16:51 |
| crobertsrh | egafford: interesting point though...actually, tmckay, when we automatically fetch the job type stuff, I think that horizon will still need to specify which job types it supports, otherwise, adding a new job type to sahara could result in unsupported options showing-up in the UI dropdown. | 16:51 |
| tmckay | crobertsrh, using the new j ob-types endpoint stuff with a filter, you can do that | 16:52 |
| tmckay | type=blah&type=blah&type=blah | 16:52 |
| tmckay | That would let you constrain | 16:52 |
| crobertsrh | ah, good :) | 16:52 |
| tmckay | I didn't think the UI would actually do it, though :) | 16:53 |
| crobertsrh | Yeah, me neither | 16:53 |
| tmckay | crobertsrh, I tried loading the plugins from the little CLI wrapper, it didn't go well. | 16:53 |
| tmckay | Some anonymous failure, no doubt we need to pull in more config | 16:54 |
| tmckay | not work it imho | 16:54 |
| crobertsrh | lots of other stuff needed, eh? | 16:54 |
| tmckay | worth it | 16:54 |
| crobertsrh | Probably not then. | 16:54 |
| tmckay | seems error prone we we can make a sane default location | 16:54 |
| elmiko | maybe something is importing a module that uses config? | 16:54 |
| tosky | tmckay: not sure I get the entire picture: if you use both plugin_name and hadoop_version together, doesn't it give kind of namespace to avoid collisions? | 16:55 |
| tmckay | crobertsrh, actually .... if plugin name and version are in the json files, with template name, we don't even need multiple dirs. Just one big default template dir. | 16:55 |
| tmckay | not sure if that is nice from a hunan standpoint though | 16:55 |
| tmckay | tosky, yes, but I'm saying that "hadoop_version" is a lie. that string is anything you want it to be, with no rhyme or reason. | 16:56 |
| tmckay | there is no necessary mapping to anything in hadoop at all | 16:56 |
| crobertsrh | tmckay: Yeah, I think it may still be handy to keep them separated by plugin | 16:56 |
| tmckay | it is really a name for the particular plugin subdir | 16:56 |
| tosky | tmckay: oh, ok, naming is bad, but renaming would be costly; maybe just add an alias, disallow the direct usage of the old name for new resources, and in 3/4 releases, when all the plugins currently supported have another version (so you don't need the old templates), kill the initial name | 16:57 |
| tosky | I'm for slow backward compatible transitions :) | 16:58 |
| tmckay | tosky, yes, something like that. | 16:58 |
| tmckay | tosky, this is why crobertsrh on the UI just puts up "Version" on the forms :) | 16:58 |
| crobertsrh | c'mon tosky, I was just about to do the rename with a quick find and replace | 16:59 |
| tmckay | everyone thinks of it as the plugin version | 16:59 |
| tosky | crobertsrh: eheh :) | 16:59 |
| tmckay | and they roughly map to the versions specified in DIB | 16:59 |
| *** akuznetsov has quit IRC | 17:02 | |
| *** hdd has joined #openstack-sahara | 17:06 | |
| tosky | crobertsrh: given the current status, do you consider an error this scenario: try to create a job binary which fails because of validation; the internal job data is created anyway (you can't reuse it now from horizon, you can from python client with --url) | 17:16 |
| elmiko | tosky: i'd say so, if it fails i don't think we should be storing it in the internal db | 17:18 |
| tosky | elmiko: oki, I will file another bug | 17:18 |
| tosky | sorry :) | 17:18 |
| elmiko | no worries | 17:18 |
| tosky | on the other side, is the internal db for storing job binaries going to stay for now? No news on glance usage for Kilo, and for Liberty? | 17:19 |
| elmiko | not that i've heard of. i think the internal db will be in usage for Kilo | 17:19 |
| tosky | here you are: https://bugs.launchpad.net/horizon/+bug/1427304 | 17:24 |
| openstack | Launchpad bug 1427304 in OpenStack Dashboard (Horizon) "[sahara] When the job binary creating fails, the job binary data is created anyway" [Undecided,New] | 17:24 |
| elmiko | cool, thanks tosky | 17:24 |
| tosky | again about job binaries: where is the "Script name" field used? | 17:47 |
| tosky | most of the time the resource is address through the "Name", also when you download it | 17:48 |
| tosky | so I wonder if it's really used | 17:48 |
| tosky | in fact you don't define it for swift resources, so... | 17:48 |
| egafford | tmckay, crobertsrh: Any thoughts on whether "Parameters" in the Shell action case should or shouldn't be relabeled in the UI as "Environment Variables" for clarity? | 17:50 |
| tmckay | egafford, yes, I think they should be relabeled | 17:50 |
| tosky | and the url field of the job-binary points to the id | 17:50 |
| egafford | tmckay: Agreed. | 17:51 |
| tosky | the only place I see it is in the output of sahara job-binary-data-list | 17:51 |
| elmiko | tosky: hmm, i'd know off the top of my head. i'll have to dig in a little more to understand the "Script name" field | 17:56 |
| tosky | elmiko: in fact it does not need to be unique; sahara adds a unique ID at the end | 17:58 |
| tosky | but it fails if the name is too long, as it can't append the ID, as it would be >50 chars | 17:59 |
| tosky | oook, another one | 17:59 |
| elmiko | tosky: at a guess the "Script name" just turns into the name field for the binaries, which is limited to 80 char in the db | 18:00 |
| tosky | yes, so there is a mechanism to allow duplicated names, but it can fail | 18:01 |
| tosky | I would just block the creation of duplicated names instead of autoguessing | 18:01 |
| elmiko | right, the name isn't unique in the db | 18:01 |
| elmiko | i think it's more for convenience | 18:01 |
| tosky | but why not in the job binary name then? If the - now broken - feature of selecting an existing script was working, I would have to select between | 18:02 |
| tosky | myscript_<uuid>, myscript_<uuid2>, ... , myscript_<uuidn> | 18:02 |
| tosky | (if I used always myscript) | 18:03 |
| tosky | but then... | 18:03 |
| tosky | ok, I will summarize the findings | 18:03 |
| elmiko | i'm not following, what do you mean by "why not in the job binary name then?" ? | 18:05 |
| openstackgerrit | Vitaly Gridnev proposed openstack/python-saharaclient: Add support for show_progress parameter https://review.openstack.org/159451 | 18:07 |
| tosky | elmiko: try to create a job binary with the name of an existing one; this safe-guard mechanism is not available, you will get an error | 18:09 |
| tosky | as expected, I would say | 18:09 |
| elmiko | ah, ok | 18:10 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Apply event-log feature for Vanilla plugins https://review.openstack.org/149972 | 18:10 |
| elmiko | can you make a duplicate named binary using the cli? (i wonder) | 18:10 |
| tosky | https://bugs.launchpad.net/horizon/+bug/1427328 | 18:11 |
| openstack | Launchpad bug 1427328 in OpenStack Dashboard (Horizon) "[sahara] The mechanism used to avoid duplicate script names in jobs binaries is fragile" [Undecided,New] | 18:11 |
| tosky | elmiko: if I try a duplicated "name" field I see an error in sahara logs | 18:11 |
| elmiko | interesting | 18:12 |
| *** Networkn3rd has joined #openstack-sahara | 18:19 | |
| crobertsrh | egafford: sorry, was eating lunch...just got back | 18:22 |
| crobertsrh | I think they probably should get their own label. It's a different enough purpose. | 18:23 |
| *** hdd has quit IRC | 18:24 | |
| *** devlaps has joined #openstack-sahara | 19:01 | |
| *** hdd has joined #openstack-sahara | 19:13 | |
| *** tosky has quit IRC | 19:20 | |
| *** shakamunyi has joined #openstack-sahara | 19:20 | |
| *** devlaps has quit IRC | 19:29 | |
| *** sgotliv has joined #openstack-sahara | 19:34 | |
| *** himangi has left #openstack-sahara | 19:35 | |
| tmckay | elmiko, SergeyLukjanov, care to give a +2 to https://review.openstack.org/#/c/157460/ ? I'll touch it with a is_default field for default_templates (no update allowed), it would be helpful if it merged. | 19:49 |
| tmckay | crobertsrh, I assume the "cant update" provision also applies to "cant destroy?" | 19:50 |
| crobertsrh | Yeah, that was the spirit :) | 19:51 |
| tmckay | I plan on adding a way to remove the default flag with the CLI, of course. So if you really want to you can get rid of them. | 19:51 |
| crobertsrh | that would be quite an update | 19:51 |
| tmckay | okay | 19:51 |
| tmckay | :) | 19:51 |
| elmiko | tmckay: ack, i'll take a look | 19:54 |
| elmiko | was mainly waiting for a good test to put +2 | 19:55 |
| tmckay | ack, me too | 19:55 |
| *** himangi has joined #openstack-sahara | 20:03 | |
| *** himangi has quit IRC | 20:04 | |
| crobertsrh | Hey guys...this bug came in as a Horizon bug....I think it's likely to be a service sort of thing though. https://bugs.launchpad.net/horizon/+bug/1427304 I will take a closer look unless anyone wants to say "no, definitely service bug". | 20:09 |
| openstack | Launchpad bug 1427304 in OpenStack Dashboard (Horizon) "[sahara] When the job binary creating fails, the job binary data is created anyway" [Undecided,New] | 20:09 |
| elmiko | crobertsrh: that was one tosky submitted, i think you might be right though about it being a service bug. kinda depends where the breakdown is. | 20:10 |
| crobertsrh | Yeah, my gut is saying Service | 20:10 |
| crobertsrh | since horizon doesn't have the power to raise a validation error and still create the entry in the database | 20:11 |
| *** juice has quit IRC | 20:34 | |
| *** tmckay has quit IRC | 21:01 | |
| *** shakamunyi has quit IRC | 21:07 | |
| *** shakamunyi has joined #openstack-sahara | 21:08 | |
| openstackgerrit | Ethan Gafford proposed openstack/sahara: [EDP] Add Oozie Shell Job Type https://review.openstack.org/159920 | 21:13 |
| *** dmitryme has quit IRC | 21:23 | |
| *** alazarev has quit IRC | 21:23 | |
| *** aignatov has quit IRC | 21:23 | |
| *** chlong has joined #openstack-sahara | 21:23 | |
| *** ruhe has quit IRC | 21:23 | |
| *** NikitaKonovalov has quit IRC | 21:24 | |
| *** SergeyLukjanov has quit IRC | 21:24 | |
| *** malini1 has joined #openstack-sahara | 21:41 | |
| malini1 | Hello Sahara Team. The OpenStack Product work group is driving an effort to collect and share OpenStack sub-project roadmaps. | 21:44 |
| malini1 | https://etherpad.openstack.org/p/kilo-product-management-socialization | 21:44 |
| malini1 | Might the PTL or cores help answer the following: | 21:44 |
| malini1 | 1. What are you delivering for Kilo? 2. What do you plan on delivering for L cycle? (we understand confidence is lower with time). 3. What do you plan on delivering for M cycle? (we understand confidence is lower with time). 4. How can the product WG help? | 21:45 |
| elmiko | malini1: you should really get in touch with SergeyLukjanov | 21:45 |
| malini1 | elmiko -- :-) | 21:46 |
| malini1 | if you see him in IRC would you please ask him to reply my email | 21:46 |
| malini1 | I have sent a message | 21:46 |
| malini1 | thank you very much | 21:46 |
| elmiko | malini1: will do, i'll mention it at our meeting too | 21:46 |
| malini1 | :) | 21:47 |
| elmiko | malini1: np, wish i could help answer your questions, but i'm afraid i won't have all thei nfo | 21:47 |
| elmiko | malini1: in terms of what we are delivering, are you looking for a list of our features and fixes? | 21:47 |
| *** chlong has quit IRC | 21:51 | |
| *** shakamunyi_ has joined #openstack-sahara | 21:53 | |
| *** shakamunyi has quit IRC | 21:54 | |
| malini1 | elmiko: features enough, high level, not fixes | 22:01 |
| elmiko | malini1: ack | 22:02 |
| *** crobertsrh is now known as _crobertsrh | 22:08 | |
| *** egafford has quit IRC | 22:21 | |
| *** aignatov has joined #openstack-sahara | 22:36 | |
| *** alazarev has joined #openstack-sahara | 22:36 | |
| *** dmitryme has joined #openstack-sahara | 22:36 | |
| *** NikitaKonovalov has joined #openstack-sahara | 22:37 | |
| *** ruhe has joined #openstack-sahara | 22:37 | |
| *** SergeyLukjanov has joined #openstack-sahara | 22:37 | |
| *** shakamunyi_ has quit IRC | 23:06 | |
| *** chlong has joined #openstack-sahara | 23:33 | |
| *** openstackgerrit has quit IRC | 23:38 | |
| *** openstackgerrit has joined #openstack-sahara | 23:38 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!