| *** tellesnobrega has joined #openstack-sahara | 00:32 | |
| *** stannie1 has quit IRC | 00:35 | |
| *** tellesnobrega has quit IRC | 00:39 | |
| *** witlessb has quit IRC | 00:49 | |
| *** tnovacik has quit IRC | 01:35 | |
| *** hdd has quit IRC | 02:15 | |
| *** hdd has joined #openstack-sahara | 02:17 | |
| *** Poornima has joined #openstack-sahara | 03:17 | |
| *** hdd has quit IRC | 03:30 | |
| *** ViswaV has quit IRC | 03:34 | |
| *** ViswaV has joined #openstack-sahara | 03:40 | |
| *** chandankumar has joined #openstack-sahara | 05:03 | |
| *** ViswaV_ has joined #openstack-sahara | 05:25 | |
| *** ViswaV has quit IRC | 05:26 | |
| *** hdd has joined #openstack-sahara | 05:34 | |
| *** Poornima has quit IRC | 05:59 | |
| *** hdd has quit IRC | 06:41 | |
| *** russellb has quit IRC | 06:43 | |
| *** russellb has joined #openstack-sahara | 06:44 | |
| *** ViswaV_ has quit IRC | 06:48 | |
| *** ViswaV has joined #openstack-sahara | 06:48 | |
| *** Longgeek has joined #openstack-sahara | 06:57 | |
| *** k4n0 has joined #openstack-sahara | 07:15 | |
| *** tnovacik has joined #openstack-sahara | 07:58 | |
| *** xianghui has joined #openstack-sahara | 08:02 | |
| *** witlessb has joined #openstack-sahara | 08:15 | |
| openstackgerrit | Kazuki OIKAWA proposed openstack/sahara-extra: Add main function wrapper for edp.java.adapt_for_oozie config https://review.openstack.org/139522 | 08:18 |
|---|---|---|
| openstackgerrit | Kazuki OIKAWA proposed openstack/sahara: Add edp.java.adapt_for_oozie config for Java Action https://review.openstack.org/115884 | 08:20 |
| *** IvanBerezovskiy1 has joined #openstack-sahara | 09:12 | |
| *** skolekonov has joined #openstack-sahara | 09:20 | |
| *** Longgeek has quit IRC | 09:32 | |
| *** Longgeek has joined #openstack-sahara | 09:35 | |
| *** llirkaz has joined #openstack-sahara | 09:56 | |
| llirkaz | hi | 09:56 |
| openstackgerrit | Denis Egorenko proposed stackforge/sahara-ci-config: Temporary disable heat for hdp2 https://review.openstack.org/140285 | 10:06 |
| *** ruhe has quit IRC | 10:48 | |
| *** SergeyLukjanov has quit IRC | 10:48 | |
| *** dmitryme has quit IRC | 10:48 | |
| *** alazarev has quit IRC | 10:48 | |
| *** NikitaKonovalov has quit IRC | 10:48 | |
| *** aignatov has quit IRC | 10:48 | |
| *** aignatov has joined #openstack-sahara | 10:49 | |
| *** alazarev has joined #openstack-sahara | 10:49 | |
| *** dmitryme has joined #openstack-sahara | 10:49 | |
| *** NikitaKonovalov has joined #openstack-sahara | 10:49 | |
| *** ruhe has joined #openstack-sahara | 10:50 | |
| *** SergeyLukjanov has joined #openstack-sahara | 10:50 | |
| *** Longgeek has quit IRC | 10:54 | |
| *** Longgeek has joined #openstack-sahara | 10:56 | |
| *** llirkaz has quit IRC | 11:05 | |
| *** tosky has joined #openstack-sahara | 11:27 | |
| *** tellesnobrega has joined #openstack-sahara | 11:42 | |
| *** tellesnobrega has quit IRC | 11:54 | |
| *** tnovacik has quit IRC | 11:58 | |
| openstackgerrit | Kazuki OIKAWA proposed openstack/sahara: Add edp.java.adapt_for_oozie config for Java Action https://review.openstack.org/115884 | 12:04 |
| *** chandankumar has quit IRC | 12:25 | |
| *** _mattf is now known as mattf | 12:31 | |
| mattf | SergeyLukjanov, sreshetnyak, ping re centos images | 12:31 |
| *** xianghui has quit IRC | 12:41 | |
| SergeyLukjanov | mattf, hey, what's up? | 12:56 |
| *** Longgeek has quit IRC | 12:58 | |
| openstackgerrit | Denis Egorenko proposed stackforge/sahara-ci-config: Rename all jobs https://review.openstack.org/139956 | 12:59 |
| *** Longgeek has joined #openstack-sahara | 13:04 | |
| *** _crobertsrh is now known as crobertsrh | 13:07 | |
| *** xianghui has joined #openstack-sahara | 13:08 | |
| *** xianghui has quit IRC | 13:14 | |
| *** skolekonov has quit IRC | 13:17 | |
| *** ylobankov has quit IRC | 13:18 | |
| *** skolekonov has joined #openstack-sahara | 13:18 | |
| *** ylobankov has joined #openstack-sahara | 13:18 | |
| mattf | SergeyLukjanov, the centos folks created http://cloud.centos.org/centos/6/images/ for us | 13:22 |
| SergeyLukjanov | mattf, oh, that's great | 13:23 |
| mattf | it has centos6 cloud-init images | 13:23 |
| SergeyLukjanov | mattf, sounds like we should move to using them | 13:23 |
| SergeyLukjanov | yay! | 13:23 |
| mattf | kb is keen to make sure they work for us w/ sahara | 13:23 |
| mattf | so they'll rev them quickly w/ fixes we need | 13:23 |
| SergeyLukjanov | mattf, ack | 13:23 |
| SergeyLukjanov | mattf, sreshetnyak is on vacation this and next week | 13:24 |
| mattf | is there anyone who typically runs through https://github.com/openstack/sahara-image-elements/blob/master/diskimage-create/Create_CentOS_cloud_image.rst available to test this week? | 13:24 |
| SergeyLukjanov | mattf, I'll try to find someone to check | 13:24 |
| mattf | gr8 | 13:24 |
| mattf | if there are any issues at all ping me and i'll connect folks w/ kb | 13:24 |
| mattf | SergeyLukjanov, also, kb mentioned we should try and use the .xz compressed versions | 13:25 |
| mattf | gerrit needs a recheck button that shows up as an action on the search/dashboard pages | 13:27 |
| mattf | maybe even a recheck-all | 13:27 |
| * mattf grins | 13:27 | |
| *** miqui has joined #openstack-sahara | 13:42 | |
| *** egafford has joined #openstack-sahara | 13:42 | |
| *** tnovacik has joined #openstack-sahara | 13:48 | |
| *** tnovacik has quit IRC | 13:48 | |
| *** tnovacik has joined #openstack-sahara | 13:48 | |
| *** Longgeek_ has joined #openstack-sahara | 13:52 | |
| *** Longgeek has quit IRC | 13:52 | |
| *** egafford has quit IRC | 13:58 | |
| *** chandankumar has joined #openstack-sahara | 14:01 | |
| *** tellesnobrega has joined #openstack-sahara | 14:09 | |
| *** tnovacik has quit IRC | 14:19 | |
| *** k4n0 has quit IRC | 14:21 | |
| *** Longgeek_ has quit IRC | 14:22 | |
| tellesnobrega | SergeyLukjanov, yesterday was a holiday here in brazil, so i only got to see that storm is part of sahara today. thanks for approving it | 14:23 |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara-image-elements: Adding ability to build images for Hadoop 2.6.0 https://review.openstack.org/138391 | 14:23 |
| *** tmckay has joined #openstack-sahara | 14:27 | |
| SergeyLukjanov | tellesnobrega, thx for implementing it :) | 14:28 |
| SergeyLukjanov | tellesnobrega, so, there were a few comments, considering them as recomendations for further improvements | 14:28 |
| *** Longgeek has joined #openstack-sahara | 14:29 | |
| tellesnobrega | SergeyLukjanov, i saw them. I will work on them asap | 14:32 |
| *** tnovacik has joined #openstack-sahara | 14:37 | |
| *** Longgeek has quit IRC | 14:39 | |
| *** Longgeek has joined #openstack-sahara | 14:44 | |
| *** hdd has joined #openstack-sahara | 14:45 | |
| *** ghenriks has quit IRC | 14:52 | |
| *** tnovacik has quit IRC | 14:56 | |
| openstackgerrit | Denis Egorenko proposed stackforge/sahara-ci-config: Rename all jobs https://review.openstack.org/139956 | 15:00 |
| *** k4n0 has joined #openstack-sahara | 15:01 | |
| openstackgerrit | Ken Chen proposed openstack/sahara-specs: Add more services into CDH https://review.openstack.org/139574 | 15:02 |
| *** ghenriks has joined #openstack-sahara | 15:08 | |
| *** tnovacik has joined #openstack-sahara | 15:09 | |
| *** egafford has joined #openstack-sahara | 15:15 | |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara-image-elements: Adding ability to build images for Hadoop 2.6.0 https://review.openstack.org/138391 | 15:17 |
| *** ylobankov has quit IRC | 15:25 | |
| *** ylobankov has joined #openstack-sahara | 15:25 | |
| *** ylobankov has quit IRC | 15:26 | |
| *** ylobankov has joined #openstack-sahara | 15:26 | |
| *** ViswaV has quit IRC | 15:28 | |
| *** ViswaV has joined #openstack-sahara | 15:29 | |
| *** tnovacik has quit IRC | 15:37 | |
| *** tnovacik has joined #openstack-sahara | 15:38 | |
| *** tnovacik has quit IRC | 15:38 | |
| openstackgerrit | Denis Egorenko proposed stackforge/sahara-ci-config: Rename all jobs https://review.openstack.org/139956 | 15:48 |
| *** hdd has quit IRC | 15:53 | |
| *** Networkn3rd has joined #openstack-sahara | 15:56 | |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Adding Hadoop 2.6.0 support to Vanilla plugin https://review.openstack.org/140390 | 16:04 |
| openstackgerrit | Denis Egorenko proposed stackforge/sahara-ci-config: Rename all jobs https://review.openstack.org/139956 | 16:17 |
| *** chandankumar has quit IRC | 16:18 | |
| *** tnovacik has joined #openstack-sahara | 16:25 | |
| *** mattf is now known as _mattf | 16:25 | |
| tmckay | elmiko, question for you on the substituion semantics | 16:27 |
| tmckay | my original thought was to keep the job exec unaltered, so that when it is retrieved from the db the fields are unaltered | 16:28 |
| tmckay | but the proxy user stuff is done up front, based on the presence of paths in the job exec | 16:28 |
| tmckay | so if the substitution happens early, it's cleaner | 16:29 |
| elmiko | ok | 16:29 |
| elmiko | i think the proxy stuff just wants the job_exec id | 16:30 |
| tmckay | in order to preserve the record unaltered, we have to do the resolution twice, or do it once and stash the resolution in job_configs in additional dict entries | 16:30 |
| tmckay | elmiko, opinion? If you submitted a job and used the subst feature, what would you expect to see when you read it back? | 16:30 |
| tmckay | resolved paths, or the original args that reference data sources by name or id? | 16:31 |
| elmiko | tmckay: as a user, i would expect to see whatever i entered originally | 16:31 |
| elmiko | now, i could see that changing if i went to some page that showed the job execution as it was sent to the cluster | 16:32 |
| elmiko | but, if i went to the job page, where i entered the configs, i would expect the un-integrated form | 16:33 |
| tmckay | okay. I'll do the resolution and stash it in an extra field, and filter it in the resource return (like proxy config) | 16:33 |
| elmiko | cool | 16:33 |
| *** k4n0 has quit IRC | 16:34 | |
| tmckay | I don't think we have an "as submitted" view -- that's more like job logs, or the oozie console | 16:34 |
| tmckay | hmm, well, it does auto fill some stuff when using the standard input/output sources | 16:34 |
| elmiko | it might make sense to show the substituted values in a job execution, but show the original form on the job page. does that make sense? | 16:38 |
| tmckay | these values will only be in the job execution, though. They don't exist in the job. They're specified at submission time. | 16:39 |
| tmckay | there's really no reason the subst values couldn't be passed back, too. It will just show paths instead of names | 16:40 |
| tmckay | I don't believe it would break anything | 16:41 |
| elmiko | got it | 16:41 |
| tmckay | then you can have your cake and eat it too | 16:41 |
| elmiko | i think for debug purposes there needs to be someway to see both forms, pre-sub and post | 16:41 |
| elmiko | not sure how to display it though | 16:41 |
| tmckay | I'm thinking of job_configs fields like this: substituted_args (copy of full arg list with changes), substituted_configs (dict, just those that changed), substituted_params(dict, just those that changed) | 16:43 |
| tmckay | if there are no substitutions they won't be there | 16:43 |
| tmckay | UI could display them or not | 16:43 |
| tmckay | I wonder what the CLI does with job_configs ... | 16:44 |
| elmiko | not sure | 16:44 |
| tmckay | well, whether to report or not is a secondary issue. It can still work. | 16:44 |
| tmckay | didn't realize this would get this thorny :) | 16:45 |
| tmckay | trying very hard to keep the semantics unchanged re precedence | 16:47 |
| *** tosky has quit IRC | 16:47 | |
| elmiko | yea, it seemed like this would be kinda squirrely | 16:55 |
| *** egafford has quit IRC | 17:01 | |
| *** crobertsrh has quit IRC | 17:06 | |
| tmckay | elmiko, one more question on this, since I'm looking at it. The way the semantics are now, if a user explicitly specifies swift_username, swift_password, swift_trustid, or swift_domain_name in their job execution that field (or fields) will not be replaced by Sahara. But you really need all 4 as set by Sahara for it to work. | 17:07 |
| tmckay | Should we try to prevent that, or at least log a warning? | 17:08 |
| elmiko | i don't think you need swift_trustid and swift_domain_name, if you are using a username/password that exist in the current project it will work | 17:09 |
| elmiko | so, if the user exists in the current context project then the swift-hadoop bridge will be able to produce a token | 17:10 |
| tmckay | I mean in the case where you've got proxy users set up. If I submitted a job exec say with just swift_user specified, I would break the execution. | 17:10 |
| elmiko | yes | 17:11 |
| elmiko | in that case, it probably would be nice to error early instead of having to read the logs to figure out that auth failed in the cluster | 17:11 |
| elmiko | but, i would think if you are specifying those values yourself, that you have created the proxy user which means you are waaaay into power user territory | 17:12 |
| tmckay | right. I'm wondering if we can define an inconsistent subset, and warn if the user submits that | 17:13 |
| tmckay | I would think that messing up any one of the 4 could break it | 17:14 |
| tmckay | maybe a subject for another CR | 17:14 |
| elmiko | +1 to adding another CR | 17:14 |
| tmckay | elmiko, also in this I am trying to preserve the creation of the proxy info early, in service/api. Would there be a problem with creating it in the job_manager? | 17:16 |
| tmckay | so, in run_job instead of up front | 17:16 |
| elmiko | hmm | 17:17 |
| tmckay | just considering options. It might save the extra fields | 17:17 |
| tmckay | but I don't want to be too disruptive | 17:17 |
| elmiko | i'd need to think about it and go through the proceedure again. | 17:17 |
| tmckay | k, I'll keep it as is. Once I have something working maybe I can try moving it | 17:18 |
| tmckay | I think that's all on this topic :) | 17:18 |
| elmiko | cool | 17:18 |
| *** skolekonov has quit IRC | 17:38 | |
| *** hdd has joined #openstack-sahara | 17:48 | |
| *** ViswaV has quit IRC | 17:54 | |
| *** egafford has joined #openstack-sahara | 17:55 | |
| *** crobertsrh has joined #openstack-sahara | 18:16 | |
| crobertsrh | tmckay: working on the template editing spec right now. It may need some more detail input for the service portion. I hope to send it up soon. | 18:18 |
| crobertsrh | Also. If anyone cares to do horizon reviews, the filtering change is up (and has been for awhile). https://review.openstack.org/#/c/136459/ | 18:19 |
| tmckay | crobertsrh, thanks | 18:20 |
| crobertsrh | For adding detail, feel free to just grab it and edit it rather than large comments that I'd just copy/paste. | 18:21 |
| crobertsrh | won't hurt my feelings | 18:21 |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara: Removing unecessary check https://review.openstack.org/140424 | 18:23 |
| *** hdd has quit IRC | 18:24 | |
| tmckay | crobertsrh, "For adding detail, feel free to just grab it and edit it rather than large comments", what is "it"? The template editing spec | 18:42 |
| tmckay | ? | 18:42 |
| crobertsrh | yes | 18:42 |
| tmckay | crobertsrh, okay, I'll wait until it doesn't say WIP :) | 18:44 |
| crobertsrh | Fair enough | 18:44 |
| tmckay | actually, the spec isn't there yet, is it? | 18:44 |
| crobertsrh | Nope | 18:44 |
| crobertsrh | it's trul a WIP | 18:44 |
| crobertsrh | truly | 18:44 |
| elmiko | truly? | 18:45 |
| crobertsrh | doh | 18:45 |
| crobertsrh | really | 18:45 |
| tmckay | he's coming to vancouver too | 18:45 |
| tmckay | pmackinn is still giving me grief for that | 18:45 |
| tmckay | Van Couver that is | 18:46 |
| crobertsrh | Related to Van Halen, no doubt | 18:46 |
| tmckay | yes, that's where they were originally from | 18:46 |
| crobertsrh | He can help us translate all that Canadian jargon too | 18:46 |
| elmiko | lol | 18:46 |
| elmiko | and help us build our foursome for the spiel ;) | 18:47 |
| tmckay | which side of the road do they drive on? | 18:47 |
| crobertsrh | Exactly | 18:47 |
| crobertsrh | The top side, of course | 18:47 |
| crobertsrh | I think it's only the southern hemisphere where they drive on the bottom | 18:47 |
| tmckay | heh | 18:47 |
| elmiko | lol | 18:47 |
| crobertsrh | As long as the real Canadians don't show up, we should win the Openstack bonspiel | 18:48 |
| elmiko | oooh, that could be a thing | 18:49 |
| crobertsrh | Obviously, drinks would be included | 18:50 |
| elmiko | just need to get brooke to make a contrib for kilo | 18:50 |
| crobertsrh | No dog pictures, please | 18:50 |
| elmiko | lol | 18:50 |
| crobertsrh | Although, when you +2 +merge it with great stealth, I'll know what's up | 18:50 |
| elmiko | haha | 18:51 |
| elmiko | you should really get worried if i try to start a new project named "kennel" or something ;0 | 18:51 |
| crobertsrh | Heh. A new project that exists solely for you to get your friends to gather at each summit. | 18:52 |
| elmiko | lol | 18:52 |
| elmiko | very popular | 18:52 |
| *** ViswaV has joined #openstack-sahara | 18:53 | |
| crobertsrh | Although, Sahara barely made it into official incubation. "Kennel" had better be very convincing. | 18:53 |
| elmiko | i'm gonna wait till the "big tent" arrives, much easier to get accepted | 18:53 |
| tmckay | kennel, it's where the BigData Dogs hang out | 19:12 |
| openstackgerrit | Chad Roberts proposed openstack/sahara-specs: Spec for template editing https://review.openstack.org/140448 | 19:19 |
| *** Longgeek has quit IRC | 19:20 | |
| *** hdd has joined #openstack-sahara | 19:21 | |
| *** Longgeek has joined #openstack-sahara | 20:20 | |
| *** Longgeek has quit IRC | 20:26 | |
| openstackgerrit | Merged openstack/sahara: Removing unecessary check https://review.openstack.org/140424 | 20:43 |
| *** ViswaV has quit IRC | 21:02 | |
| *** ViswaV has joined #openstack-sahara | 21:03 | |
| *** ViswaV has quit IRC | 21:04 | |
| *** ViswaV has joined #openstack-sahara | 21:04 | |
| *** Longgeek has joined #openstack-sahara | 21:22 | |
| *** Longgeek has quit IRC | 21:26 | |
| *** ViswaV has quit IRC | 21:30 | |
| *** ViswaV has joined #openstack-sahara | 21:35 | |
| *** crobertsrh has quit IRC | 21:58 | |
| *** tmckay has quit IRC | 22:04 | |
| *** ViswaV has quit IRC | 22:24 | |
| *** ViswaV has joined #openstack-sahara | 22:30 | |
| *** stannie1 has joined #openstack-sahara | 22:36 | |
| *** ViswaV has quit IRC | 22:37 | |
| *** IlyaE has joined #openstack-sahara | 22:38 | |
| *** ViswaV has joined #openstack-sahara | 22:46 | |
| *** ViswaV has quit IRC | 22:54 | |
| *** ViswaV has joined #openstack-sahara | 22:55 | |
| *** ViswaV has quit IRC | 22:55 | |
| *** ViswaV has joined #openstack-sahara | 22:56 | |
| *** IlyaE has quit IRC | 22:56 | |
| *** ViswaV has quit IRC | 22:58 | |
| *** Longgeek has joined #openstack-sahara | 23:10 | |
| *** Networkn3rd has quit IRC | 23:12 | |
| *** Longgeek has quit IRC | 23:15 | |
| *** egafford has quit IRC | 23:27 | |
| *** stannie1 has quit IRC | 23:40 | |
| *** ViswaV has joined #openstack-sahara | 23:50 | |
| *** _mattf is now known as mattf | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!