| *** tosky has quit IRC | 01:17 | |
| *** caowei has joined #openstack-sahara | 01:30 | |
| openstackgerrit | pangliye proposed openstack/sahara master: add bugs link in README.rst https://review.openstack.org/533443 | 02:12 |
|---|---|---|
| *** caowei has quit IRC | 04:30 | |
| -openstackstatus- NOTICE: The logs.openstack.org filesystem has been restored to full health. We are attempting to keep logs uploaded between the prior alert and this one, however if your job logs are missing please issue a recheck. | 04:49 | |
| *** ChanServ changes topic to "The logs.openstack.org filesystem has been restored to full health. We are attempting to keep logs uploaded between the prior alert and this one, however if your job logs are missing please issue a recheck." | 04:49 | |
| *** ChanServ changes topic to "OpenStack Sahara // IRC Meetings - http://eavesdrop.openstack.org/#OpenStack_Data_Processing_(Sahara)_Team_Meeting" | 04:53 | |
| *** links has joined #openstack-sahara | 05:11 | |
| *** pcaruana has joined #openstack-sahara | 05:23 | |
| *** caowei has joined #openstack-sahara | 05:27 | |
| *** pcaruana has quit IRC | 05:32 | |
| *** hoonetorg has quit IRC | 07:58 | |
| *** rcernin has quit IRC | 07:59 | |
| *** hoonetorg has joined #openstack-sahara | 08:11 | |
| openstackgerrit | zhongshengping proposed openstack/puppet-sahara master: Add use_journal option for logging configuration https://review.openstack.org/533558 | 08:22 |
| *** caowei has quit IRC | 10:21 | |
| *** tosky has joined #openstack-sahara | 10:30 | |
| *** pgadiya has joined #openstack-sahara | 10:40 | |
| *** pcaruana has joined #openstack-sahara | 11:44 | |
| *** tesseract has joined #openstack-sahara | 12:15 | |
| *** tellesnobrega has quit IRC | 12:22 | |
| *** tellesnobrega has joined #openstack-sahara | 12:23 | |
| tellesnobrega | tosky, about .swp I don't like having "IDE" specific stuff on .gitignore, specially with temporary files like jeremy said | 12:33 |
| tellesnobrega | it is easier to delete knowing they are there | 12:33 |
| *** tellesnobrega has quit IRC | 12:34 | |
| *** tellesnobrega has joined #openstack-sahara | 12:35 | |
| tosky | ack | 12:43 |
| tosky | I may have a patch for the dashboard issue | 12:43 |
| tellesnobrega | awesome, what was the issue? | 12:45 |
| tosky | it's after lunch :) | 12:46 |
| tellesnobrega | sounds good | 12:53 |
| *** tellesnobrega has quit IRC | 13:16 | |
| *** tellesnobrega has joined #openstack-sahara | 13:16 | |
| *** openstackgerrit has quit IRC | 13:18 | |
| *** caowei has joined #openstack-sahara | 13:53 | |
| *** dgonzalez has left #openstack-sahara | 13:56 | |
| *** openstackgerrit has joined #openstack-sahara | 14:11 | |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-dashboard master: Fix attrs list of LabeledInput.render() method https://review.openstack.org/533654 | 14:11 |
| tosky | tellesnobrega: ^^ please test and let me know :) | 14:11 |
| tellesnobrega | tosky, doing it right now | 14:11 |
| *** pgadiya has quit IRC | 14:12 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_sahara master: Replace virtualenv-tools by a script https://review.openstack.org/533663 | 14:16 |
| tellesnobrega | tosky, works :) thanks | 14:17 |
| tosky | yay | 14:18 |
| tosky | it took more to understand *why* the fix was working, and luckily I found a commit message $somewhere with the explanation | 14:18 |
| tellesnobrega | that happens sometimes | 14:19 |
| tellesnobrega | positional args vs named args, right? | 14:19 |
| tosky | yes, because now the arguments are called with their names in $someplace | 14:21 |
| tellesnobrega | I see | 14:31 |
| tellesnobrega | tosky, I'm still hitting the launch job issue that jeremy didn't see | 14:41 |
| tosky | tellesnobrega: oh, so different from the create job binary issue? | 14:42 |
| tellesnobrega | not sure | 14:42 |
| tellesnobrega | looking to see if I find something in the logs | 14:43 |
| tosky | can you please repeat the steps, so that I can try to reproduce? | 14:43 |
| tellesnobrega | but the behavior is the same | 14:43 |
| tellesnobrega | 1. Create job template | 14:43 |
| tellesnobrega | 2. Launch job | 14:43 |
| tosky | do you have a cluster already running? | 14:45 |
| *** caowei has quit IRC | 14:45 | |
| tosky | and where is the issue exactly, that launch job does not open? | 14:45 |
| *** jkunigk has joined #openstack-sahara | 14:48 | |
| jkunigk | Hi, I have a question about storage in Sahara | 14:49 |
| jkunigk | I wonder how Sahara-provisioned Hadoop clusters normally implement HDFS. That part is not definitively covered in the docs, even though I suppose it is implied: | 14:50 |
| jkunigk | https://docs.openstack.org/sahara/latest/intro/overview.html#details | 14:50 |
| jkunigk | says that Cinder _can_ be used for VM block storage | 14:51 |
| tosky | but that's unrelated with HDFS | 14:51 |
| tosky | HDFS services can be defined as any other services in the cluster template | 14:51 |
| jkunigk | ok, but most modern Hadoop distros require an HDFS service and let's assume we want to run a distro of Hadoop on top =) | 14:52 |
| tosky | an existing HDFS service, do you mean? | 14:52 |
| jkunigk | no, the distros build one (with local storage) when they install "the cluster" | 14:53 |
| jkunigk | and that's why I am hung up a bit on Cinder | 14:53 |
| tosky | uhm, I'm not sure about the question then | 14:54 |
| jkunigk | Right, didn't really get there yet. | 14:54 |
| tosky | oh | 14:54 |
| jkunigk | Is it typical to feed Cinder volumes as the underlying block devices to the HDFS service in Sahara? | 14:55 |
| tosky | it's up to you - it was discussed few times and as Sahara we just provide you the possibility | 14:55 |
| jkunigk | what other options would you say are typical for implementing HDFS then? | 14:56 |
| jkunigk | (on Sahara clusters) | 14:56 |
| jkunigk | s/typical/available | 14:56 |
| tosky | for the internal usage, ephemeral storage for HDFS; for data sources, you can point to an external HDFS | 14:58 |
| tosky | tellesnobrega: when you say "Launch job", do you mean "Launch on existing cluster"/"Launch on new cluster"? | 14:58 |
| * tosky disappears for a while | 14:58 | |
| jkunigk | Ah, ok. So the clusters would be more transient. | 14:59 |
| jkunigk | Thanks for that clarification | 15:01 |
| jkunigk | Regarding Cinder: Since HDFS canonically prefers local storage, I wonder if I can rely on any form of data locality when using Cinder and local disks in commodity server (i.e. model the classic way of running HDFS). | 15:01 |
| jkunigk | I have read that Cinder manages a set of Volumes created by Linux' LVM and a corresponding iSCSI and fibre channel target for each volume on all participating hosts which provide storage. | 15:04 |
| jkunigk | The hypervisor on a host (such as KVM) then is assigned such volumes as requested, which may reside on its local host, but also on distant hosts and communicates with the volumes as an iSCSI initiator. That would basically defeat the efforts HDFS makes w/r to data durability (replication), meaning that another layer should take care of that. | 15:04 |
| jkunigk | Is that about accurate? | 15:04 |
| tosky | I heard different opinions about this, and it's not my area of expertise, so I will refrain from commenting | 15:07 |
| tosky | I mean, the data locality of HDFS | 15:07 |
| jkunigk | yes, it's my area of expertise =) | 15:09 |
| jkunigk | If you want really good durability with HDFS at scale you need each replica on its own physical box | 15:10 |
| jkunigk | and it seems to me that Cinder would make it impossible for HDFS to know this for sure. | 15:10 |
| tosky | jkunigk: https://specs.openstack.org/openstack/sahara-specs/specs/kilo/volume-instance-locality.html | 15:11 |
| tosky | disclaimer: I don't think it has been retested recently | 15:11 |
| jkunigk | Ah great! sorry, I am still very bad with the OpenStack docs... Specs... gotta start looking at those then. Yes, btw. if another layer (Enterprise storage system, Cinder itself) takes care about durability, then it's up to the team maintaining that layer to sign off on data durability. | 15:13 |
| jkunigk | But as the docs under the link above imply performance and crazy strain on network bandwidth would be the next issue | 15:13 |
| jkunigk | but alas, many public cloud implementations just use SAN for HDFS block devices... | 15:14 |
| jkunigk | thanks for the information | 15:14 |
| *** links has quit IRC | 15:20 | |
| *** hoonetorg has quit IRC | 16:09 | |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Beginnings of APIv2 support in client https://review.openstack.org/533066 | 16:27 |
| *** dave-mccowan has joined #openstack-sahara | 16:27 | |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Beginnings of APIv2 support in client https://review.openstack.org/533066 | 16:28 |
| *** dave-mcc_ has joined #openstack-sahara | 16:29 | |
| *** dave-mccowan has quit IRC | 16:32 | |
| *** jeremyfreudberg has joined #openstack-sahara | 16:33 | |
| jeremyfreudberg | tellesnobrega, i just put out a new patchset for v2 client. the comments on the latest patchset are some TODOs, can you work on those? | 16:34 |
| tellesnobrega | jeremyfreudberg, sure | 16:39 |
| tellesnobrega | tosky, I tried Launch on existing cluster, the window opens, but the launch button doesn't | 16:39 |
| tellesnobrega | jeremyfreudberg, thanks for getting that started | 16:40 |
| tosky | jeremyfreudberg: did you see the proposed patch for horizon? I don't remember if you were affected | 16:40 |
| *** tesseract has quit IRC | 16:40 | |
| jeremyfreudberg | tellesnobrega, no problem. and thanks for helping too. if you have questions just email | 16:40 |
| tellesnobrega | will do | 16:40 |
| jeremyfreudberg | tosky: yes, i was affected, so I'll try it now | 16:40 |
| jeremyfreudberg | tellesnobrega: i am pretty sure my job launch button had no problems but i'll check again | 16:41 |
| tellesnobrega | jeremyfreudberg, thanks. I'm not sure why it is not working here, will looking into it | 16:42 |
| jeremyfreudberg | tellesnobrega, just so i have it exactly the same: what attributes does your job template have (what type of job, extra configs, etc) | 16:42 |
| tellesnobrega | I tried a Spark job | 16:42 |
| tosky | apart from jeremyfreudberg 's v2 patch, I think I commented all the recent open patches | 16:43 |
| tellesnobrega | tosky, I will follow up on all of them too, hopefully today | 16:43 |
| tellesnobrega | jeremyfreudberg, nothing specific, I was hoping it would break before it got to that (testing the binary file bug) | 16:44 |
| jeremyfreudberg | tosky: your fix works | 16:44 |
| tellesnobrega | jeremyfreudberg, go ahead and merge it | 16:45 |
| tosky | \o/ | 16:45 |
| tosky | I don't even want to think about Django >=2 | 16:46 |
| jeremyfreudberg | django 2 is a bit scary | 16:46 |
| jeremyfreudberg | we should try to antcipate those breakages | 16:46 |
| jeremyfreudberg | tellesnobrega, i still don't have any problem on launch job | 16:46 |
| tellesnobrega | jeremyfreudberg, what job did you run? | 16:47 |
| tellesnobrega | spark-pi? | 16:47 |
| jeremyfreudberg | tellesnobrega, oh, do i actually have to run it (or just test that the modal appears) | 16:47 |
| tellesnobrega | the problem is clicking the launch job button | 16:48 |
| jeremyfreudberg | oh, inside the modal | 16:48 |
| jeremyfreudberg | ok, i will try it | 16:48 |
| tellesnobrega | the modal appears no problem as well | 16:48 |
| jeremyfreudberg | cd /opt/stack/horizon | 16:50 |
| jeremyfreudberg | python manage.py runserver 0.0.0.0:PORT | 16:50 |
| jeremyfreudberg | do that, navigate to the instance of horizon running on PORT, and see if any error comes out | 16:50 |
| jeremyfreudberg | tellesnobrega, never mind, client side error | 16:53 |
| jeremyfreudberg | where client=browser js | 16:53 |
| tellesnobrega | jeremyfreudberg, where is that? | 16:54 |
| jeremyfreudberg | uh, in chrome i just hit Ctrl-Shift-I and click console | 16:54 |
| jeremyfreudberg | An invalid form control with name='streaming_reducer' is not focusable. | 16:54 |
| jeremyfreudberg | An invalid form control with name='streaming_mapper' is not focusable. | 16:54 |
| tosky | partially OT, but I can't resist: http://www.commitstrip.com/en/2018/01/08/new-year-new-frameworks/ | 16:56 |
| openstackgerrit | Merged openstack/sahara-dashboard master: Fix attrs list of LabeledInput.render() method https://review.openstack.org/533654 | 16:57 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-dashboard stable/pike: Fix attrs list of LabeledInput.render() method https://review.openstack.org/533735 | 16:58 |
| tellesnobrega | tosky, makes sense | 16:58 |
| tosky | sadly | 17:00 |
| jeremyfreudberg | so, uh, anybody know how to fix the js problem? | 17:00 |
| * tosky tries to identify the nearest exit | 17:00 | |
| tosky | joking aside, I don't have an environment where I can start a real cluster | 17:01 |
| tosky | oh, but maybe I don't need that, I just need to define it and see if it starts | 17:01 |
| jeremyfreudberg | tosky, you don't need a real cluster | 17:01 |
| jeremyfreudberg | we aren't even getting that far | 17:01 |
| tosky | I probably need at least the cluster template, I guess | 17:01 |
| tellesnobrega | you do | 17:02 |
| tosky | so any cluster template, any job template valid for that cluster? | 17:02 |
| jeremyfreudberg | well, a job template (but it can reference something fake, for the purposes of debugging this) | 17:02 |
| tellesnobrega | tosky, you can easily import that, if you want I have master and slave templates for spark | 17:02 |
| tosky | let's see if default templates still work | 17:02 |
| tellesnobrega | and job template should be easy enough | 17:02 |
| jeremyfreudberg | https://github.com/openstack/sahara-dashboard/blob/b05932f4fbb15a4be0569c4cd741111b18986a10/sahara_dashboard/content/data_processing/jobs/templates/job_templates/config_template.html#L157 | 17:03 |
| jeremyfreudberg | ^ the problem is how we are hiding some fields (probably django changed how they are generated so js cannot find them anymore) | 17:04 |
| jeremyfreudberg | whoever has to fix this can also try hiding them using django stuff instead (might not be possible) | 17:04 |
| tellesnobrega | jeremyfreudberg, that makes sense | 17:05 |
| jeremyfreudberg | we can put this off till after feature freeze (we have a month for bugs only), but yeah, my guess sounds pretty good | 17:08 |
| *** dave-mcc_ has quit IRC | 17:08 | |
| tellesnobrega | jeremyfreudberg, weird enough, the problem only happens for the streaming_mapper and streaming_reducer part | 17:12 |
| tellesnobrega | if you try to launch a streamingMapReduce job it works | 17:12 |
| jeremyfreudberg | tellesnobrega, hmm | 17:13 |
| *** hoonetorg has joined #openstack-sahara | 17:14 | |
| jeremyfreudberg | it's really weird | 17:15 |
| tosky | jeremyfreudberg: unrelated (before I go afk for a while): did you see my (small) question here? https://review.openstack.org/#/c/530659/ | 17:15 |
| jeremyfreudberg | tosky, yes, i saw that, I will respond on the patch now | 17:16 |
| jeremyfreudberg | tellesnobrega, regarding your observation it might have something do with how the fields are creating on django side (different types, settings, make a subtle difference) https://github.com/openstack/sahara-dashboard/blob/20ae6d4418185111592094c134e55276f376bc49/sahara_dashboard/content/data_processing/jobs/job_templates/workflows/launch.py#L161 | 17:20 |
| jeremyfreudberg | but i don't notice anything overtly strange | 17:20 |
| * tosky bbl | 17:21 | |
| *** links has joined #openstack-sahara | 17:21 | |
| tellesnobrega | jeremyfreudberg, I do see something interesting | 17:21 |
| tellesnobrega | required=False | 17:21 |
| tellesnobrega | its not set on streaming stuff | 17:21 |
| tellesnobrega | trying now | 17:22 |
| jeremyfreudberg | tellesnobrega, clever | 17:22 |
| tellesnobrega | jeremyfreudberg, that did it :) | 17:24 |
| jeremyfreudberg | tellesnobrega, awesome! | 17:24 |
| jeremyfreudberg | i think that's a sufficient fix, most stuff in sahara-dashboard is marked required=False even if it is logically required | 17:25 |
| jeremyfreudberg | propose a patch when you can | 17:25 |
| tellesnobrega | will do | 17:26 |
| *** jeremyfreudberg has quit IRC | 17:28 | |
| *** remixtj has left #openstack-sahara | 17:34 | |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara-dashboard master: Setting streaming options as not required https://review.openstack.org/533746 | 17:35 |
| tellesnobrega | tosky, https://review.openstack.org/#/c/533746/1 launch job fix | 17:41 |
| *** links has quit IRC | 17:50 | |
| -openstackstatus- NOTICE: Zuul has been restarted and has lost queue contents; changes in progress will need to be rechecked. | 18:23 | |
| *** jkunigk has quit IRC | 18:49 | |
| tellesnobrega | tosky, I'm facing an issue to run the job because its failing with keystone auth_url for swift (using keystone v3) | 18:58 |
| tellesnobrega | do you know if we have any special settings to make it work? | 18:58 |
| tellesnobrega | I'll afk for a while now | 18:58 |
| *** jkunigk has joined #openstack-sahara | 19:17 | |
| *** jkunigk has quit IRC | 19:17 | |
| *** jkunigk has joined #openstack-sahara | 19:41 | |
| *** jkunigk has quit IRC | 20:03 | |
| tosky | uhm, swift | 21:28 |
| openstackgerrit | Merged openstack/puppet-sahara master: Add use_journal option for logging configuration https://review.openstack.org/533558 | 21:29 |
| *** pcaruana has quit IRC | 21:30 | |
| *** dave-mccowan has joined #openstack-sahara | 21:32 | |
| *** rcernin has joined #openstack-sahara | 22:21 | |
| openstackgerrit | melissaml proposed openstack/puppet-sahara master: Update openstackdocstheme>=1.18.1 https://review.openstack.org/532893 | 22:51 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!