| openstackgerrit | Zhongyue Luo proposed a change to openstack/sahara: Fix dict iteration usage https://review.openstack.org/129824 | 00:31 |
|---|---|---|
| *** crobertsrh is now known as _crobertsrh | 01:11 | |
| *** Longgeek has joined #openstack-sahara | 01:28 | |
| *** Longgeek has quit IRC | 01:33 | |
| *** witlessb has joined #openstack-sahara | 01:56 | |
| *** tellesnobrega has quit IRC | 02:00 | |
| *** witlessb has quit IRC | 02:01 | |
| *** Longgeek has joined #openstack-sahara | 03:00 | |
| *** tellesnobrega has joined #openstack-sahara | 03:02 | |
| *** Longgeek has quit IRC | 03:04 | |
| *** Networkn3rd has joined #openstack-sahara | 03:41 | |
| *** Longgeek has joined #openstack-sahara | 04:30 | |
| *** Longgeek has quit IRC | 04:35 | |
| *** Longgeek has joined #openstack-sahara | 05:46 | |
| *** Networkn3rd has quit IRC | 05:49 | |
| *** Longgeek has quit IRC | 05:51 | |
| openstackgerrit | Zhongyue Luo proposed a change to openstack/sahara: Fix dict iteration usage https://review.openstack.org/129824 | 05:53 |
| *** openstack has joined #openstack-sahara | 06:12 | |
| *** openstackstatus has quit IRC | 06:12 | |
| *** clds_ has joined #openstack-sahara | 06:15 | |
| *** clds has quit IRC | 06:15 | |
| *** Longgeek has joined #openstack-sahara | 06:28 | |
| *** tellesnobrega has quit IRC | 06:46 | |
| *** tellesnobrega has joined #openstack-sahara | 06:47 | |
| *** coolsvap has quit IRC | 06:53 | |
| *** openstack has quit IRC | 06:59 | |
| *** openstack has joined #openstack-sahara | 14:17 | |
| *** openstackstatus has joined #openstack-sahara | 14:18 | |
| *** ChanServ sets mode: +v openstackstatus | 14:18 | |
| openstackgerrit | Denis Egorenko proposed a change to stackforge/sahara-ci-config: Add distributed Sahara to CI https://review.openstack.org/130556 | 14:32 |
| *** mattf is now known as _mattf | 14:42 | |
| *** openstackgerrit has quit IRC | 15:07 | |
| *** tmckay has quit IRC | 15:10 | |
| elmiko | crobertsrh, did you ever get to the bottom of the errors that were coming up from the swift_configs 8!=7 stuff? | 15:13 |
| crobertsrh | hmm...I dont think so | 15:14 |
| *** _mattf is now known as mattf | 15:30 | |
| crobertsrh | elmiko: refresh my memory on that. I'm not sure I recall a swift_configs problem. I thought I did, but I don't. | 15:41 |
| crobertsrh | 8 != 7 is very true except for the very small values of 8 | 15:41 |
| elmiko | lol | 15:42 |
| elmiko | sec | 15:42 |
| elmiko | http://paste.fedoraproject.org/144628/40790401/ | 15:44 |
| crobertsrh | I don't recall ever seeing that one. Are you sure it was me that was looking at it? | 15:45 |
| elmiko | maybe it was tmckay | 15:45 |
| crobertsrh | Must be | 15:46 |
| *** openstack has joined #openstack-sahara | 16:01 | |
| *** ghenriks has quit IRC | 16:01 | |
| *** tmckay has joined #openstack-sahara | 16:02 | |
| *** ghenriks has joined #openstack-sahara | 16:02 | |
| *** DenisEgorenko has left #openstack-sahara | 16:08 | |
| tmckay | anybody know how log calls work from unit tests when running tox -e py26, for example? I want to add output to tests, but unless I run them outside of tox I can't seem to get the output | 16:15 |
| *** IvanBerezovskiy has left #openstack-sahara | 16:16 | |
| tmckay | I probably can just open a file for append and do it that way | 16:16 |
| *** abonilla has joined #openstack-sahara | 16:20 | |
| abonilla | Hi - anyone related on how the Sahara images were created? | 16:21 |
| abonilla | http://lists.openstack.org/pipermail/openstack-dev/2014-September/046893.html | 16:21 |
| crobertsrh | abonilla: I believe that sreshetnyak took care of that. | 16:21 |
| crobertsrh | He may be out for the day though. Do you have a specific question about them? | 16:22 |
| abonilla | Yes, it looks like. | 16:22 |
| crobertsrh | They were generated using the sahara-image-elements repo | 16:22 |
| abonilla | crobertsrh: not really, I want to create my own but need to know how he did them | 16:22 |
| *** Longgeek has quit IRC | 16:23 | |
| crobertsrh | abonilla: https://github.com/openstack/sahara-image-elements | 16:23 |
| abonilla | crobertsrh: is that documented? I see them from CentOS and Ubuntu but I use a different distro. | 16:23 |
| crobertsrh | That repo has the scripts and such that are used to generate the images. | 16:23 |
| crobertsrh | Which distro are you looking to use? | 16:23 |
| abonilla | openSUSE/SUSE | 16:24 |
| crobertsrh | Ah, ok. I don't think that distro is currently supported in there, but you can probably make some changes to make it a supported distro. | 16:25 |
| crobertsrh | Right now, it's ubuntu/CentOS/fedora | 16:25 |
| crobertsrh | The project that sahara-image-elements uses is called diskimage-builder. People in the #tripleo channel may know more about that. | 16:26 |
| abonilla | OK | 16:27 |
| abonilla | Thanks. | 16:28 |
| crobertsrh | You don't necessarily have to use the diskimage builder though. I suppose you could use anything to generate an image that has your desired packages on it and then test to see that it works with our plugins. | 16:28 |
| crobertsrh | Good luck! | 16:28 |
| abonilla | crobertsrh: we have other methods, but I need to know what is the goal od sahara-image-elements | 16:30 |
| crobertsrh | Ah | 16:30 |
| abonilla | I guess I just need to look into it. | 16:30 |
| crobertsrh | Essentially, it installs the required packages (hadoop, java, etc) | 16:30 |
| abonilla | Cloudera and HW have repos for SUSE and putting that there would be easy, I wonder if there is any "metadata" handler or any ala cloud-init tool that needs to be put in there for Sahara to inject its magic into the image. | 16:31 |
| crobertsrh | Our images do have cloud-init "magic" so they will work with openstack | 16:31 |
| crobertsrh | The diskimage-create.sh script is fairly straightforward. Hopefully, you can see all of what is done by just looking at it. | 16:32 |
| crobertsrh | If not, just ping someone around here. | 16:32 |
| tmckay | crobertsrh, slow going, but I have reproduced the intermittent test error touching os_region_name, and determined that it happens (at least but not limited to) when test_get_swift_configs and test_url_for_regions run in the same worker process | 16:35 |
| tmckay | I bet it can also happen with some cinder tests, which also touch os_region_name | 16:35 |
| tmckay | something about the successive config override and cleanup is not right, and it shows up with multiple tests in the same worker process | 16:35 |
| tmckay | If I figure this out I want a gold star ;-) | 16:36 |
| tmckay | this kind of thing might be at the root of some of our intermittent CI erorrs. dueling config overrides | 16:36 |
| * tmckay gets a snack | 16:37 | |
| crobertsrh | wow, strange stuff | 16:39 |
| crobertsrh | You deserve a snack :) | 16:39 |
| *** mattf is now known as _mattf | 16:39 | |
| tmckay | only if I figure it out. I'm on snack credit. | 16:44 |
| tmckay | actually, read that wrong. They were in separate processes. More looking | 16:48 |
| *** openstackgerrit has joined #openstack-sahara | 16:49 | |
| *** _mattf is now known as mattf | 17:00 | |
| *** tosky has quit IRC | 17:08 | |
| tmckay | MismatchError: None != 'RegionOne' | 17:11 |
| tmckay | smoking gone | 17:11 |
| tmckay | right after a call to set_override | 17:11 |
| tmckay | very intermittent | 17:11 |
| elmiko | do we have any examples of adding a swift data source to a java job as an argument? | 17:14 |
| elmiko | or do i just set the arg as "swift://container/object" ? | 17:14 |
| tmckay | I think the java wordcount example does that | 17:14 |
| tmckay | yes | 17:14 |
| tmckay | java jobs don't take data sources | 17:14 |
| elmiko | ok, i'm finally getting to dig in on that swift job binaries bug | 17:14 |
| elmiko | how do i add an input then? | 17:15 |
| tmckay | It would be nice if you could specify a data source as an arg, with the understanding that the path would be extracted. We've talked about it, maybe kilo | 17:15 |
| elmiko | that's odd, the readme totally references adding an argument. but it is quite nebulous | 17:15 |
| tmckay | so for wordcount, it takes 2 args, input and output, which are just paths | 17:15 |
| elmiko | what are the paths relative to though? | 17:16 |
| tmckay | swift://tmckay/input, swift://tmckay/output, or you can use hdfs:// on a long running cluster | 17:16 |
| *** tellesnobrega has joined #openstack-sahara | 17:17 | |
| elmiko | ok, cool. | 17:17 |
| elmiko | and for wordcount, first arg is input, second is output? | 17:17 |
| tmckay | yeah | 17:17 |
| elmiko | hopefully this works =) | 17:17 |
| tmckay | the example dir under sahara/etc should have a whole readme file that shows how to do it | 17:18 |
| elmiko | it does, but it's thin on some details | 17:18 |
| tmckay | :) | 17:18 |
| elmiko | lol | 17:18 |
| tmckay | sorry | 17:18 |
| tellesnobrega | hi, i was trying to create a node group and cluster template with a name like: cluster-0.9.2 it showed an error saying that the name is invalid. My code is kinda old, is this suppose to be like this? is it a bug? | 17:18 |
| *** mattf is now known as _mattf | 17:18 | |
| elmiko | tellesnobrega: good question, i'm not sure about the validation on cluster names | 17:19 |
| tmckay | hmm, must be the dash or the ".", if anything | 17:19 |
| tmckay | you could try "_" | 17:19 |
| tmckay | I'm not sure what rules it's applying | 17:20 |
| tellesnobrega | i think its "." | 17:20 |
| tellesnobrega | i used with dash before | 17:20 |
| tellesnobrega | just wanted to make sure that it is a bug | 17:21 |
| tellesnobrega | or it was by design | 17:21 |
| tmckay | tellesnobrega, not sure. take a look at the validation routine | 17:29 |
| crobertsrh | I wonder if it's specifically excluding names with dots | 17:39 |
| crobertsrh | doh...I needed to scroll down a bit :) | 17:39 |
| crobertsrh | I know that we had some rather heavy-handed validations at one point, like for job binary names | 17:40 |
| tellesnobrega | ok, i will take a look | 17:41 |
| *** openstackgerrit has quit IRC | 17:48 | |
| *** openstackgerrit has joined #openstack-sahara | 17:49 | |
| *** tosky has joined #openstack-sahara | 17:53 | |
| SergeyLukjanov | meeting in 5 mins | 17:55 |
| SergeyLukjanov | elmiko, thank you for woring with ossg team! | 17:57 |
| elmiko | SergeyLukjanov: glad to help =) | 17:57 |
| SergeyLukjanov | elmiko, I've been planning to ask you to join the secgroup for sahara | 17:58 |
| elmiko | SergeyLukjanov: good timing | 17:58 |
| SergeyLukjanov | :) | 17:59 |
| *** sreshetnyak has quit IRC | 17:59 | |
| SergeyLukjanov | jinx, buy me a sode :) lol | 17:59 |
| SergeyLukjanov | tmckay, ^^ | 17:59 |
| *** sreshetnyak has joined #openstack-sahara | 17:59 | |
| *** sreshetnyak has quit IRC | 17:59 | |
| *** sreshetnyak has joined #openstack-sahara | 17:59 | |
| tmckay | jo | 18:00 |
| elmiko | LOL | 18:00 |
| elmiko | SergeyLukjanov: well played, first soda on me ;) | 18:00 |
| SergeyLukjanov | :) | 18:00 |
| SergeyLukjanov | tmckay, time to join meeting :) | 18:01 |
| tmckay | yep, typing wrong | 18:01 |
| *** _mattf is now known as mattf | 18:13 | |
| *** tnovacik has quit IRC | 18:15 | |
| *** witlessb has quit IRC | 18:27 | |
| *** stannie has quit IRC | 18:29 | |
| * tmckay gets that coffee | 18:58 | |
| *** mattf is now known as _mattf | 19:02 | |
| tmckay | hey folks, looking for an opinion -- should we use [UT] as a label for unit test bugs, or put them under [IT] with the integration tests? | 19:08 |
| * tmckay uses [IT] for now | 19:09 | |
| elmiko | tmckay: i take it the integration tests use [IT] currently? | 19:10 |
| tmckay | yeah | 19:10 |
| elmiko | +1 to [UT] then | 19:10 |
| elmiko | tmckay: does a java job get a workflow created for oozie? | 19:16 |
| tmckay | yes | 19:16 |
| elmiko | i'm hoping this sub-folder issue doesn't create a huge problem | 19:17 |
| openstackgerrit | Chad Roberts proposed a change to openstack/sahara: Fixing validation exception for valid security group https://review.openstack.org/130576 | 19:29 |
| *** sreshetnyak has quit IRC | 19:30 | |
| openstackgerrit | Trevor McKay proposed a change to openstack/sahara: Remove explicit set of CONF.os_region_name in mapr plugin tests https://review.openstack.org/130628 | 19:39 |
| tmckay | aignatov, alazarev, SergeyLukjanov, please review/+2/approve https://review.openstack.org/#/c/130628/ quickly, it fixes some intermittent unit test errors in the gate | 19:44 |
| alazarev | tmckay, done | 19:45 |
| tmckay | thank you | 19:45 |
| *** witlessb has joined #openstack-sahara | 20:21 | |
| openstackgerrit | Chad Roberts proposed a change to openstack/sahara: Fixing validation exception for valid security group https://review.openstack.org/130576 | 20:29 |
| tmckay | crobertsrh, heh, your test got hit by the os_region_name issue on patch set 2 | 20:32 |
| crobertsrh | I'll call it the tmckay bug from henceforth | 20:32 |
| tmckay | ooo, mine passed. SergeyLukjanov, aignatov, care to +2 and approve? https://review.openstack.org/#/c/130628/1 | 20:33 |
| tmckay | I'll push it through myself if it's not done by 5, it's messing stuff up | 20:33 |
| tmckay | brb | 20:36 |
| *** Networkn3rd has quit IRC | 20:46 | |
| *** crobertsrh is now known as _crobertsrh | 21:03 | |
| tellesnobrega | elmiko, i took a look into the regex that matches valid_name_hostname, from what i saw, dash is allowes | 21:04 |
| tellesnobrega | allowed | 21:04 |
| tellesnobrega | . should be if im not mistaken, but i used the regex in some online regex tester and they failed for . and in another if failed for dashes and points and number which is very weird | 21:05 |
| elmiko | tellesnobrega: sounds like a bug maybe | 21:06 |
| elmiko | i wonder if we can improve the unit tests surrounding that stuff? | 21:06 |
| tellesnobrega | i think so. I need to check properly the regex | 21:06 |
| tellesnobrega | maybe so | 21:06 |
| elmiko | that would be my first approach, see if you can mock up a test that fails with the current code. then we'll know for sure | 21:06 |
| tellesnobrega | i will try to take a look into this later today. If i can confirm the bug i will report it and work on it, and improve tests as well | 21:07 |
| tellesnobrega | sure | 21:07 |
| elmiko | awesome! | 21:07 |
| tellesnobrega | how are the preparations for the summit? | 21:07 |
| elmiko | hectic =) | 21:07 |
| elmiko | especially now that i've been nominated to chair one of our sessions | 21:07 |
| tellesnobrega | cool | 21:08 |
| elmiko | yea, i'm excited | 21:08 |
| tellesnobrega | i wish i was going | 21:08 |
| elmiko | =( | 21:08 |
| tellesnobrega | i'm back working with storm | 21:08 |
| tellesnobrega | hopefully i will have it ready soon | 21:09 |
| elmiko | that's cool | 21:09 |
| tellesnobrega | i had a little change of context, so i stopped for a while | 21:09 |
| *** _mattf is now known as mattf | 21:09 | |
| elmiko | i noticed you were quiet for a bit, glad you can dig back in =) | 21:09 |
| tellesnobrega | i hope to have something to show for next week | 21:09 |
| tellesnobrega | tanks | 21:09 |
| tellesnobrega | thanks | 21:09 |
| elmiko | nice | 21:09 |
| tellesnobrega | i would like to have it ready for sure before summit | 21:10 |
| elmiko | cool | 21:10 |
| tellesnobrega | i think stream processing is a great contribution for sahara, is a new area and can be used by lots of people. we actually have some people interested already here at the university. | 21:10 |
| elmiko | oh yea, using it with sahara? | 21:11 |
| *** witlessb has quit IRC | 21:11 | |
| tellesnobrega | once i can get it work | 21:11 |
| tellesnobrega | lol | 21:11 |
| elmiko | lol, right | 21:11 |
| *** witlessb has joined #openstack-sahara | 21:11 | |
| tellesnobrega | but its looking good. storm improved a bit, so maybe i will work with no major problems. | 21:12 |
| * elmiko fingers crossed | 21:12 | |
| tellesnobrega | :) | 21:13 |
| elmiko | then you can start working on all the fun bits like storm/swift integration! | 21:13 |
| tellesnobrega | sounds like a very fun bit | 21:14 |
| tellesnobrega | but sure, gotta make it happen | 21:14 |
| elmiko | yea, i don't know much about storm's storage options, so i'm just throwing it out there | 21:14 |
| * tmckay _tmckay | 21:15 | |
| tellesnobrega | it can be used to storage outputs if there is some. The main idea of storm is to process many and keep few | 21:16 |
| tellesnobrega | i have to go now, there is star wars movie session now | 21:18 |
| tellesnobrega | i will take a look at the name problem today and get back to you with a result | 21:18 |
| elmiko | tellesnobrega: take care, and may the force be with you ;) | 21:20 |
| openstackgerrit | A change was merged to openstack/sahara: Remove explicit set of CONF.os_region_name in mapr plugin tests https://review.openstack.org/130628 | 22:07 |
| tmckay | elmiko, heh, your neutron CR got hit by the os_region_name test bug too | 22:26 |
| elmiko | tmckay: yea, i didn't recheck because i thought you had mentioned the bug before | 22:39 |
| tmckay | yeah, I pushed the fix through and just re-launched a half dozen or so | 22:39 |
| tmckay | I think that's all of them | 22:39 |
| elmiko | tmckay: i had actually talked with _crobertsrh about it this morning, because i thought he had mentioned it | 22:39 |
| elmiko | tmckay: awesome, thanks | 22:39 |
| tmckay | np. I would love a really, really, stable CI. I wonder if there are things we can make better. | 22:40 |
| elmiko | i noticed a few design sessions about CI, might be worth investigating? | 22:40 |
| *** witlessb has quit IRC | 22:40 | |
| *** tosky has quit IRC | 23:21 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!