| *** alazarev has quit IRC | 00:08 | |
| *** IlyaE has joined #openstack-sahara | 00:19 | |
| *** alazarev has joined #openstack-sahara | 00:45 | |
| *** _elmiko is now known as elmiko | 00:55 | |
| *** liusheng has quit IRC | 00:56 | |
| *** IlyaE has quit IRC | 01:10 | |
| openstackgerrit | Michael McCune proposed a change to openstack/sahara: Adding disconnected mode fixes to hdp plugin https://review.openstack.org/94460 | 01:15 |
|---|---|---|
| openstackgerrit | Andrew Lazarev proposed a change to openstack/sahara: Added validation check that network provided for neutron https://review.openstack.org/95047 | 01:23 |
| *** alazarev has quit IRC | 01:29 | |
| openstackgerrit | Michael McCune proposed a change to openstack/sahara: Adding disconnected mode fixes to hdp plugin https://review.openstack.org/94460 | 01:34 |
| *** elmiko is now known as _elmiko | 01:52 | |
| *** skolekonov has quit IRC | 01:58 | |
| *** skolekonov has joined #openstack-sahara | 02:01 | |
| *** IlyaE has joined #openstack-sahara | 02:06 | |
| *** tellesnobrega_ has joined #openstack-sahara | 02:10 | |
| *** tellesnobrega_ has quit IRC | 02:13 | |
| *** zhiyan_ is now known as zhiyan | 02:18 | |
| *** IlyaE has quit IRC | 02:18 | |
| *** Ch00k has joined #openstack-sahara | 02:42 | |
| *** Ch00k has quit IRC | 03:32 | |
| *** akuznetsov has joined #openstack-sahara | 03:59 | |
| *** IlyaE has joined #openstack-sahara | 03:59 | |
| *** ghenriks_ has quit IRC | 04:22 | |
| *** ViswaV has joined #openstack-sahara | 04:25 | |
| *** ViswaV_ has joined #openstack-sahara | 04:26 | |
| *** ViswaV has quit IRC | 04:29 | |
| *** akuznetsov has quit IRC | 05:01 | |
| *** shakayumi has joined #openstack-sahara | 05:30 | |
| *** IlyaE has quit IRC | 05:41 | |
| *** akuznetsov has joined #openstack-sahara | 05:56 | |
| *** akuznetsov has quit IRC | 06:07 | |
| *** ViswaV_ has quit IRC | 06:11 | |
| *** akuznetsov has joined #openstack-sahara | 06:13 | |
| *** IlyaE has joined #openstack-sahara | 06:14 | |
| *** alazarev has joined #openstack-sahara | 06:16 | |
| *** tnovacik has joined #openstack-sahara | 06:50 | |
| *** zhiyan is now known as zhiyan_ | 06:52 | |
| *** Ch00k has joined #openstack-sahara | 06:58 | |
| *** witlessb has joined #openstack-sahara | 06:59 | |
| *** Ch00k has quit IRC | 07:02 | |
| *** zhiyan_ is now known as zhiyan | 07:10 | |
| *** IlyaE has quit IRC | 07:32 | |
| *** alazarev has quit IRC | 07:37 | |
| *** IlyaE has joined #openstack-sahara | 07:49 | |
| *** IlyaE has quit IRC | 07:59 | |
| *** akuznetsov has quit IRC | 08:09 | |
| *** akuznetsov has joined #openstack-sahara | 08:27 | |
| *** ylobankov has quit IRC | 08:44 | |
| *** ylobankov has joined #openstack-sahara | 08:44 | |
| *** shakayumi has quit IRC | 09:01 | |
| openstackgerrit | Ivan Berezovskiy proposed a change to stackforge/sahara-ci-config: Make diskimage-integration-centos job voting https://review.openstack.org/95114 | 09:22 |
| *** Timotey has joined #openstack-sahara | 09:40 | |
| *** skostiuchenko has left #openstack-sahara | 09:47 | |
| openstackgerrit | A change was merged to stackforge/sahara-ci-config: Make diskimage-integration-centos job voting https://review.openstack.org/95114 | 10:02 |
| *** akuznetsov has quit IRC | 10:05 | |
| *** openstack has joined #openstack-sahara | 10:21 | |
| *** akuznetsov has joined #openstack-sahara | 10:36 | |
| *** zhiyan is now known as zhiyan_ | 10:58 | |
| *** themistymay has joined #openstack-sahara | 11:01 | |
| *** tnovacik has quit IRC | 11:06 | |
| *** _crobertsrh is now known as crobertsrh | 11:18 | |
| *** witlessb has quit IRC | 11:27 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to openstack/sahara: Remove unused function from xmlutils https://review.openstack.org/94845 | 11:40 |
| *** themistymay has quit IRC | 11:52 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to openstack/sahara: Fix detaching cinder volumes https://review.openstack.org/93272 | 11:57 |
| *** shakayumi has joined #openstack-sahara | 12:23 | |
| *** ViswaV has joined #openstack-sahara | 12:26 | |
| *** ViswaV has quit IRC | 12:26 | |
| *** themistymay has joined #openstack-sahara | 12:36 | |
| *** miqui has joined #openstack-sahara | 12:54 | |
| *** _elmiko is now known as elmiko | 12:59 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to openstack/sahara: Remove unused parameters in integration tests https://review.openstack.org/95156 | 13:01 |
| *** themistymay has quit IRC | 13:06 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to openstack/sahara: Remove unused parameters in integration tests https://review.openstack.org/95156 | 13:14 |
| *** themistymay has joined #openstack-sahara | 13:19 | |
| *** skolekonov has quit IRC | 13:24 | |
| *** ghenriks has joined #openstack-sahara | 13:55 | |
| openstackgerrit | Chad Roberts proposed a change to stackforge/puppet-sahara: Allowing for RPM-based installation https://review.openstack.org/95166 | 13:57 |
| openstackgerrit | Chad Roberts proposed a change to stackforge/puppet-sahara: Allowing for RPM-based installation https://review.openstack.org/95166 | 14:00 |
| *** tmckay has joined #openstack-sahara | 14:03 | |
| *** shakayumi has quit IRC | 14:07 | |
| *** bradd1 has joined #openstack-sahara | 14:13 | |
| tmckay | venza_, ping | 14:14 |
| *** alazarev has joined #openstack-sahara | 14:27 | |
| *** IlyaE has joined #openstack-sahara | 14:41 | |
| *** alazarev has quit IRC | 15:20 | |
| elmiko | SergeyLukjanov: i'm working on the disconnected hdp plugin patch and i need to adjust how i'm doing things based on the feedback i'm getting. i have a question about using HadoopProvisionError. | 15:34 |
| elmiko | SergeyLukjanov: would it be safe for me to raise that exception in plugins.hdp.hadoopserver if we have no connection to the internet(for downloading hdx rpms) and they are not installed on the instance? | 15:35 |
| elmiko | "they" being the hdx specific rpms | 15:35 |
| *** bradd1 has quit IRC | 15:37 | |
| *** Timotey has quit IRC | 15:49 | |
| elmiko | anyone else know about HadoopProvisionError exception? | 15:51 |
| *** ghenriks has quit IRC | 15:55 | |
| *** elmiko is now known as _elmiko | 16:00 | |
| *** ghenriks has joined #openstack-sahara | 16:00 | |
| *** alazarev has joined #openstack-sahara | 16:02 | |
| *** bradd1 has joined #openstack-sahara | 16:04 | |
| *** ghenriks has quit IRC | 16:06 | |
| *** ghenriks has joined #openstack-sahara | 16:13 | |
| *** akuznetsov has quit IRC | 16:31 | |
| *** ViswaV has joined #openstack-sahara | 16:39 | |
| *** ViswaV_ has joined #openstack-sahara | 16:40 | |
| *** ViswaV has quit IRC | 16:41 | |
| *** IlyaE has quit IRC | 16:43 | |
| *** bradd1 has quit IRC | 16:45 | |
| *** akuznetsov has joined #openstack-sahara | 16:57 | |
| *** IlyaE has joined #openstack-sahara | 17:02 | |
| *** _elmiko is now known as elmiko | 17:03 | |
| *** akuznetsov has quit IRC | 17:10 | |
| *** bradd1 has joined #openstack-sahara | 17:17 | |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/sahara: Added validation check that network provided for neutron https://review.openstack.org/95047 | 17:58 |
| *** themistymay has quit IRC | 18:20 | |
| notmyname | tmckay: any followup needed for the swift tempurl stuff? | 18:23 |
| tmckay | notmyname, not yet. We've got some members on PTO currently, we'll have to talk about it. I'm wondering how secure we want/need to be | 18:24 |
| tmckay | Part of the trouble is that even with a tempurl passed to Hadoop, that url might still be exposed in logs, etc. I wonder how much of an issue that is | 18:25 |
| tmckay | and predicting the expiration might be tough -- how long will a Hadoop job run? | 18:26 |
| *** bradd1 has quit IRC | 18:35 | |
| *** themistymay has joined #openstack-sahara | 18:41 | |
| *** akuznetsov has joined #openstack-sahara | 18:43 | |
| *** IlyaE has quit IRC | 18:51 | |
| *** themistymay has quit IRC | 18:51 | |
| *** alazarev has quit IRC | 18:52 | |
| *** IlyaE has joined #openstack-sahara | 18:53 | |
| tmckay | hmm, getting a key error on SPARK in "master_port = cluster.cluster_configs.SPARK.SPARK_MASTER_PORT" | 18:55 |
| tmckay | does anyone know in the plugin stuff how cluster_configs is supposed to be extended? | 18:55 |
| *** IlyaE has quit IRC | 18:55 | |
| *** IlyaE has joined #openstack-sahara | 18:56 | |
| elmiko | tmckay: sorry, haven't touched that stuff yet | 18:59 |
| tmckay | yeah, I see a dictionary labeled "SPARK" with SPARK_MASTER_PORT in it, but I'm not sure how it's supposed to get to cluster_configs. The CR says the code is running somewhere.... but I wonder if the CR is broken. | 19:01 |
| tmckay | some places I can put breakpoints | 19:01 |
| elmiko | yea, next step for me would be print statements or a pdb.set_trace | 19:02 |
| tmckay | I'm wondering if the stuff is in there but the naming is wrong | 19:03 |
| elmiko | not sure how that all plumbs together | 19:05 |
| *** themistymay has joined #openstack-sahara | 19:08 | |
| *** IlyaE has quit IRC | 19:12 | |
| *** alazarev has joined #openstack-sahara | 19:12 | |
| *** akuznetsov has quit IRC | 19:14 | |
| *** themistymay has quit IRC | 19:25 | |
| *** IlyaE has joined #openstack-sahara | 19:31 | |
| tmckay | elmiko, looks like maybe an oversight. cluster_configs are passed as part of the cluster template, apparently. These values in question should all have sane defaults in Spark. So, the fact that cluster_configs = {} should be fine. Missing a conditional, imho. | 19:40 |
| elmiko | ok, cool. the more you know :) | 19:40 |
| tmckay | I added such conditional and it got past. But, I got a "permission denied" error during node configuration =-O | 19:41 |
| elmiko | huh | 19:41 |
| tmckay | baby steps :) | 19:59 |
| *** ViswaV_ has quit IRC | 20:16 | |
| *** ViswaV has joined #openstack-sahara | 20:18 | |
| tmckay | elmiko, looks like it was trying to create a file as the ubuntu user in a dir owned by hadoop, without world write access | 20:20 |
| tmckay | elmiko, retrying with "run_as_root=True" on the file write wrapper, fingers crossed | 20:21 |
| tmckay | I always wanted to be a sports commentator, this is a close second | 20:21 |
| tmckay | woohoo! Spark cluster is active | 20:23 |
| tmckay | what, nobody here to celebrate? Friday before a long weekend or something? Party in Vegas? Beverly tasting? | 20:24 |
| crobertsrh | +1 for beverly tasting event. | 20:24 |
| tmckay | where's themistymay when you need him | 20:25 |
| crobertsrh | I read that they have it at Epcot... | 20:25 |
| crobertsrh | maybe that will keep the kids in line when we go there | 20:25 |
| tmckay | Yeah, and that's only 12 hours away. | 20:25 |
| tmckay | Or a quick cruise to Atlanta | 20:25 |
| crobertsrh | 12? Maybe for you. | 20:25 |
| tmckay | heh | 20:25 |
| tmckay | Gotta love Dixie. | 20:25 |
| crobertsrh | Something around 20 I think the last time I did it. | 20:25 |
| tmckay | I made that trip a few times from MA | 20:26 |
| crobertsrh | I kind of like the views in KY and TN, but OH and GA are a real bore-fest | 20:27 |
| tmckay | You sound like a mountain boy | 20:27 |
| crobertsrh | Speaking of long drives, I wonder how the Mirantis crew is doing :) | 20:27 |
| tmckay | hoping they're in LA by now. Maybe got held up in Las Vegas :) | 20:27 |
| crobertsrh | somehow ran out of money in Las Vegas | 20:28 |
| *** themistymay has joined #openstack-sahara | 20:28 | |
| crobertsrh | Ok, my reviews are baking in zuul. Sounds like a fine time to call it a day. Have a good weekend. | 20:30 |
| *** crobertsrh is now known as _crobertsrh | 20:30 | |
| elmiko | tmckay: glad it worked :) | 20:31 |
| tmckay | elmiko, not sure it's the correct solution, but seems to be a workaround | 20:32 |
| tmckay | commenting on the CR now ... | 20:32 |
| elmiko | from your description it sounds like there is plenty of debugging to do | 20:32 |
| elmiko | definitely cool that it works though | 20:36 |
| tmckay | yeah. whacked that it ends up with 2 files owned by 'ubuntu' in a dir owned by 'hadoop' | 20:37 |
| elmiko | agreed | 20:37 |
| elmiko | but obviously they assumed that it would only be running on ubuntu images or something | 20:37 |
| tmckay | elmiko, yeah, some stuff is being set up as the "hadoop" user though -- either at image build time, or cluster creation. Not sure. I guess I can launch just a plain image and find out. | 20:38 |
| tmckay | could probably just fix this up with a "chown" | 20:39 |
| elmiko | then the ubuntu user is stuff is just an artifact of using ssh? | 20:39 |
| elmiko | yea | 20:39 |
| tmckay | elmiko, yeah, ssh baggage | 20:39 |
| tmckay | elmiko, /opt/spark/conf is in the image, owned by hadoop | 20:44 |
| elmiko | is that just a conf file? | 20:44 |
| tmckay | it's a dir | 20:47 |
| elmiko | presumably created by diskimage-create? | 20:47 |
| tmckay | have to look more, but it looks like what is intended is to modify conf/slaves, and copy and modify conf/spark-env.sh.template to produce conf/spark-env.sh | 20:48 |
| tmckay | elmiko, yeah, rpm install I'm guessing, or something in the spark element | 20:48 |
| tmckay | well, apt-get :) | 20:48 |
| tmckay | I'm digging holes in someone's backyard | 20:49 |
| elmiko | lol totally | 20:49 |
| _crobertsrh | I should have kept working....wife was very happy to see I was done..."Oh good, now you can mow the lawn". | 20:52 |
| tmckay | probably run as root followed by a chown or some such magic | 20:52 |
| tmckay | _crobertsrh, heh. I mow tomorrow. Wife is at a conference. | 20:52 |
| elmiko | wait, tmckay aren't some of your kids old enough to mow the lawn? | 20:59 |
| elmiko | i thought that was a main reason to have kids! | 20:59 |
| *** themistymay has quit IRC | 21:14 | |
| elmiko | is there a parameter i need to set for sahara to use qpid instead of rabbit? | 21:14 |
| *** bogdando has quit IRC | 21:15 | |
| *** themistymay has joined #openstack-sahara | 21:15 | |
| *** bogdando has joined #openstack-sahara | 21:16 | |
| tmckay | elmiko, yes, technically, some of the kids are old enough | 21:20 |
| tmckay | elmiko, not sure on the qpid/rabbit thing | 21:20 |
| tmckay | I have to teach the kids to drive the riding mower first, though | 21:21 |
| elmiko | lol, my dad kicked me out there on the old john deere when i was 13 | 21:24 |
| elmiko | man, wrestling with devstack gets old sometimes | 21:33 |
| *** elmiko has quit IRC | 21:42 | |
| *** elmiko has joined #openstack-sahara | 21:45 | |
| themistymay | sometimes -> *all the time | 21:45 |
| *** IlyaE has quit IRC | 22:09 | |
| *** openstackgerrit has quit IRC | 22:14 | |
| *** openstackgerrit has joined #openstack-sahara | 22:14 | |
| *** openstack has joined #openstack-sahara | 22:18 | |
| *** themistymay has quit IRC | 22:25 | |
| *** akuznetsov has joined #openstack-sahara | 22:37 | |
| *** IlyaE has joined #openstack-sahara | 22:47 | |
| *** akuznetsov has quit IRC | 22:47 | |
| *** IlyaE has quit IRC | 22:51 | |
| *** themistymay has joined #openstack-sahara | 23:02 | |
| *** themistymay has quit IRC | 23:16 | |
| *** themistymay has joined #openstack-sahara | 23:30 | |
| *** themistymay has quit IRC | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!