| *** dina_belova has quit IRC | 00:00 | |
| *** qwerty_nor has joined #savanna | 00:04 | |
| *** IlyaE has quit IRC | 00:08 | |
| *** qwerty_nor has quit IRC | 00:18 | |
| *** IlyaE has joined #savanna | 00:36 | |
| *** NikitaKonovalov has joined #savanna | 00:36 | |
| *** NikitaKonovalov has quit IRC | 00:40 | |
| *** dina_belova has joined #savanna | 00:56 | |
| *** dina_belova has quit IRC | 01:00 | |
| *** _mattf is now known as mattf | 01:26 | |
| *** NikitaKonovalov has joined #savanna | 01:37 | |
| *** ben_duyujie has joined #savanna | 01:38 | |
| *** NikitaKonovalov has quit IRC | 01:43 | |
| *** dina_belova has joined #savanna | 01:56 | |
| *** dina_belova has quit IRC | 02:01 | |
| *** NikitaKonovalov has joined #savanna | 02:37 | |
| *** NikitaKonovalov has quit IRC | 02:42 | |
| *** esmute_ has joined #savanna | 02:47 | |
| *** esmute has quit IRC | 02:47 | |
| *** esmute_ is now known as esmute | 02:47 | |
| *** dina_belova has joined #savanna | 02:57 | |
| *** dina_belova has quit IRC | 03:01 | |
| *** NikitaKonovalov has joined #savanna | 03:38 | |
| *** NikitaKonovalov has quit IRC | 03:42 | |
| *** dina_belova has joined #savanna | 03:57 | |
| *** dina_belova has quit IRC | 04:00 | |
| *** SergeyLukjanov has joined #savanna | 04:03 | |
| *** SergeyLukjanov has quit IRC | 04:30 | |
| *** SergeyLukjanov has joined #savanna | 04:31 | |
| *** NikitaKonovalov has joined #savanna | 04:38 | |
| *** NikitaKonovalov has quit IRC | 04:43 | |
| *** dina_belova has joined #savanna | 04:58 | |
| *** dina_belova has quit IRC | 05:02 | |
| *** lastidiot has quit IRC | 05:10 | |
| *** SergeyLukjanov has quit IRC | 05:24 | |
| *** akuznetsov has joined #savanna | 05:36 | |
| *** NikitaKonovalov has joined #savanna | 05:39 | |
| *** NikitaKonovalov has quit IRC | 05:45 | |
| *** NikitaKonovalov has joined #savanna | 06:40 | |
| *** nprivalova has joined #savanna | 06:42 | |
| *** NikitaKonovalov has quit IRC | 06:46 | |
| *** dina_belova has joined #savanna | 06:59 | |
| *** akuznetsov has quit IRC | 07:03 | |
| *** dina_belova has quit IRC | 07:03 | |
| *** IlyaE has quit IRC | 07:04 | |
| *** IlyaE has joined #savanna | 07:06 | |
| *** akuznetsov has joined #savanna | 07:11 | |
| *** dina_belova has joined #savanna | 07:11 | |
| *** dina_belova has quit IRC | 07:25 | |
| *** NikitaKonovalov has joined #savanna | 07:39 | |
| *** dina_belova has joined #savanna | 07:46 | |
| *** IlyaE has quit IRC | 07:58 | |
| *** SergeyLukjanov has joined #savanna | 08:08 | |
| openstackgerrit | Yaroslav Lobankov proposed a change to stackforge/savanna: IT updating for "HDP" plugin https://review.openstack.org/40369 | 08:36 |
|---|---|---|
| *** akuznetsov has quit IRC | 09:04 | |
| *** dmitryme has joined #savanna | 09:06 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to stackforge/savanna: Refactoring remote utils https://review.openstack.org/39043 | 09:09 |
| *** dina_belova has quit IRC | 09:13 | |
| *** dina_belova has joined #savanna | 09:37 | |
| openstackgerrit | Dina Belova proposed a change to stackforge/savanna: Migrate to pbr https://review.openstack.org/37480 | 09:47 |
| *** dina_belova has quit IRC | 09:59 | |
| *** NikitaKonovalov has quit IRC | 10:03 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to stackforge/savanna: Refactoring remote utils https://review.openstack.org/39043 | 10:17 |
| *** dina_belova has joined #savanna | 10:20 | |
| *** nprivalova has quit IRC | 10:21 | |
| openstackgerrit | Sergey Lukjanov proposed a change to stackforge/savanna: Update requirements to the latest versions https://review.openstack.org/40385 | 10:23 |
| *** qwerty_nor has joined #savanna | 10:31 | |
| openstackgerrit | Dina Belova proposed a change to stackforge/savanna: Migrate to pbr https://review.openstack.org/37480 | 10:33 |
| *** NikitaKonovalov has joined #savanna | 10:34 | |
| *** NikitaKonovalov has quit IRC | 10:40 | |
| *** NikitaKonovalov has joined #savanna | 10:40 | |
| *** dina_belova has quit IRC | 10:46 | |
| *** qwerty_nor has quit IRC | 10:48 | |
| *** dina_belova has joined #savanna | 10:50 | |
| *** qwerty_nor has joined #savanna | 10:51 | |
| *** nprivalova has joined #savanna | 10:56 | |
| openstackgerrit | A change was merged to stackforge/savanna: Created savanna-db-manage script for new DB https://review.openstack.org/40220 | 11:05 |
| openstackgerrit | A change was merged to stackforge/savanna: Update requirements to the latest versions https://review.openstack.org/40385 | 11:05 |
| *** ruhe has joined #savanna | 11:08 | |
| *** key has quit IRC | 11:14 | |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests for Conductor Manager API https://review.openstack.org/40391 | 11:35 |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests for Conductor Manager API https://review.openstack.org/40391 | 11:45 |
| *** dina_belova has quit IRC | 11:53 | |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests for Conductor Manager API https://review.openstack.org/40391 | 11:56 |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests for Conductor Manager API https://review.openstack.org/40391 | 11:58 |
| openstackgerrit | Dmitry Mescheryakov proposed a change to stackforge/savanna: A Resource implementation for Conductor https://review.openstack.org/40400 | 12:18 |
| *** dmitryme has quit IRC | 12:23 | |
| *** dina_belova has joined #savanna | 12:24 | |
| *** dina_belova has quit IRC | 12:32 | |
| *** akuznetsov has joined #savanna | 12:41 | |
| *** qwerty_nor has quit IRC | 12:44 | |
| *** nprivalova has quit IRC | 12:47 | |
| *** _crobertsrh is now known as crobertsrh | 12:48 | |
| *** ruhe has quit IRC | 12:50 | |
| *** dmitryme has joined #savanna | 12:53 | |
| *** nprivalova has joined #savanna | 12:53 | |
| *** tmckayrh has joined #savanna | 13:00 | |
| *** dina_belova has joined #savanna | 13:00 | |
| tmckayrh | hello all. I just replaced my wife's laptop screen. The question is, will I be a hero, or will I be like the guy who washes Batman's car? | 13:00 |
| crobertsrh | tmckayrh: forgotten by tomorrow....especially if you broke the screen to start. | 13:01 |
| tmckayrh | crobertsrh, that is the mysterious part. Nobody knows how it was broken.... | 13:01 |
| crobertsrh | Oh, it was the kids then. | 13:02 |
| tmckayrh | akuznetsov, ping, I have a question or two for you. | 13:02 |
| nprivalova | hi tmckayrh! | 13:02 |
| tmckayrh | nprivalova, hi | 13:02 |
| *** akuznetsov has quit IRC | 13:03 | |
| tmckayrh | why does that always happen when I'm looking for somebody? | 13:04 |
| nprivalova | :) | 13:04 |
| nprivalova | let's discuss your questions anyway :) | 13:04 |
| tmckayrh | I tried to get here earlier, but I had to fix the above mentioned screen :) | 13:04 |
| tmckayrh | nprivalova, it was a question about the blueprint originally titled "Job Source Component". I talked to ruhe about this yesterday. I think, historically, "Job Source Component" was really intended to store job source code (text). Then we started talking about binaries eventually, and renamed the blueprint, and then we said "Hey, we really need something to manage source code, too! And a build component" | 13:06 |
| tmckayrh | nprivalova, the current text on the Job Origin Component talks about git, mercurial, etc. I think the text is appropriate for a source code component. We suggested yesterday adding a "Job Source Code Component" blueprint. | 13:07 |
| *** ben_duyujie has quit IRC | 13:07 | |
| *** akuznetsov has joined #savanna | 13:08 | |
| tmckayrh | nprivalova, so ultimately, we have Job Origin (binary) and Job Source Code (text). the question is how to rename/make the blueprints so everything is right. | 13:08 |
| nprivalova | tmckayrh, I think all the team is confused about it. As I understand, the final decision is "JobOrigin works with binaries", "UnnamedComponent works with sources" | 13:08 |
| akuznetsov | tmckayrh hi | 13:08 |
| tmckayrh | I think 1) rename current Job Origin blueprint "Job Source Code Component" and 2) make a new Job Origin Component. | 13:09 |
| aignatov_ | hi, another option can be rename description of JobOrigin blueprint and create new bp for JobSource)) | 13:09 |
| tmckayrh | nprivalova, yes, akuznetsov, hi. Agreed :) Yesterday ruhe and I (and maybe Sergey) chatted about "Job Source Code Component" for unnamed. | 13:09 |
| nprivalova | I agree with " 1) rename current Job Origin blueprint "Job Source Code Component" and 2) make a new Job Origin Component." Let's wait for akuznetsov's decision | 13:10 |
| tmckayrh | aignatov_, yes. The current description on the Job Origin Component deals with version control systems appropriate for text, it seems. That should change (either by rename or rewrite the description) | 13:11 |
| akuznetsov | i saw this blue print, I think this functionality should be implement after we finished the first version of edp | 13:11 |
| tmckayrh | akuznetsov, nprivalova, oh, the other part of the discussion we had was this -- is build/source code the same component, or 2? We thought 2. So I made a build blueprint, and then edited the description. | 13:12 |
| tmckayrh | So, at this precise moment, there is no "job source code" component blueprint -- it doesn't exist. | 13:13 |
| crobertsrh | I'm going to hold of on much more UI impl until this all settles down a bit. | 13:13 |
| tmckayrh | at first, I had put "build" and "source" in the same one | 13:13 |
| nprivalova | I think there should be 2 separate components | 13:13 |
| aignatov_ | in other word the first version of edp, as akusnetsov said, will work only with JobOrigin(bnaries) | 13:13 |
| tmckayrh | crobertsrh, I think we're almost there :) | 13:14 |
| aignatov_ | and Job Source Code is for the further development | 13:14 |
| tmckayrh | aignatov_, yes, I think we agree on that | 13:14 |
| aignatov_ | and it will be a separate component | 13:14 |
| tmckayrh | aignatov_, +1 | 13:14 |
| aignatov_ | or it could be as further exstension of JobOrigin, eventually job sources will be compiled and represented as binaries, and finally it are benaries as from JobOrigin | 13:16 |
| openstackgerrit | Dina Belova proposed a change to stackforge/savanna: Migrate to pbr https://review.openstack.org/37480 | 13:17 |
| aignatov_ | so, let's do with only JobOrigin and binaries - jars, pig scripts and so on | 13:17 |
| tmckayrh | could be. Apologies for my part in any confusion, we seem to have migrated from talking about source code in higher level descriptions to handling binaries in the early phase. | 13:17 |
| *** akuznetsov has quit IRC | 13:17 | |
| tmckayrh | aignatov_, should we have a job source code component blueprint now, or create it later, and change the Job Origin blueprint to say "binary" and call out hdfs, gluster, swift, etc.? | 13:18 |
| *** dina_belova has quit IRC | 13:19 | |
| *** akuznetsov has joined #savanna | 13:20 | |
| aignatov_ | tmckayrh, right, change description of JobOrigin and create anther one bp for source code component, we can target it for future milestones | 13:20 |
| aignatov_ | Nadya, Alex, Chad, what do you think about this? | 13:21 |
| tmckayrh | aignatov_, okay. I don't seem to have privilege to change blueprints that I did not write :) | 13:21 |
| akuznetsov | aignatov_ what is the difference between Job Source and Job Build components? | 13:22 |
| *** NikitaKonovalov has quit IRC | 13:22 | |
| aignatov_ | akuznetsov, I think there is no difference, just differnt naming | 13:23 |
| tmckayrh | yes, there is a difference | 13:23 |
| akuznetsov | aignatov_ in this case we already has blue print for it | 13:23 |
| akuznetsov | #link https://blueprints.launchpad.net/savanna/+spec/edp-job-build-component | 13:24 |
| tmckayrh | We postulated yesterday that putting source code management and build semantics (integrating with different build systems through plugins) might be too much functionality for one component. | 13:24 |
| *** dina_belova has joined #savanna | 13:24 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to stackforge/savanna: Refactoring remote utils https://review.openstack.org/39043 | 13:25 |
| tmckayrh | So, Job Source Code Component manages only source code (adding, listing, deleting, getting) and the Job Build Component builds binaries (and accesses the Job Source Code Component) | 13:25 |
| akuznetsov | tmckayrh this is standard functionality for ci/build servers, possible we will use one for this component (e.g. Jenkis) | 13:25 |
| tmckayrh | akuznetsov, okay, that sounds fine. This was the idea ruhe and I had. I'm fine with a single component. | 13:26 |
| tmckayrh | maybe something we can decide at the meetup today, one component or two? | 13:27 |
| *** dmitryme_ has joined #savanna | 13:27 | |
| akuznetsov | I have an idea to use a build sever as one of the sources for job binaries | 13:27 |
| tmckayrh | ah | 13:27 |
| *** dmitryme has quit IRC | 13:28 | |
| *** dmitryme_ is now known as dmitryme | 13:28 | |
| tmckayrh | akuznetsov, here is a related question, just for clarification. On https://etherpad.openstack.org/savanna_API_draft_EDP_extensions | 13:28 |
| tmckayrh | The Job Origin Object points to a single binary for a job that will be looked up by id by the job manager component. Yes? | 13:29 |
| tmckayrh | at some point I started thinking about describing the endpoints themselves (filesystems, build servers, whatever) and I mixed the two ideas (wrongly, I think) | 13:30 |
| akuznetsov | for now yes | 13:30 |
| aignatov_ | I think, yes, endpoints for this case would be only filesystems I think | 13:31 |
| akuznetsov | in feature we should consider case when job consists of several components, for example a pig script with some udf | 13:31 |
| tmckayrh | okay. So, the next question is, do we need a facility to list storage resources? For example, "give me a list of all the swifts or hdfs filesystems that I might use to store this binary" | 13:31 |
| *** ruhe has joined #savanna | 13:31 | |
| tmckayrh | that's a little bit different than listing the full path for a single binary | 13:32 |
| akuznetsov | tmckayrh this is a good option for UI | 13:35 |
| tmckayrh | I'm not sure how to go about it. If the URI in each job object gives the full path for a single job, then I suppose globbing or some kind of filtering could be applied to the full list of URIs. But maybe there is a better way? | 13:37 |
| aignatov_ | that's a good question... As for me, for the earlier phase of implementation it's not needed to list all possible filesystems...I tnink we should "register" only one for that | 13:38 |
| nprivalova | I thought that storing binaries in swift is user's decision and we should do nothing with it. User just set "binaries path" and savanna copies it to user's specific folder to hdfs. Or you think we should provide any mechanism for storing jobs in swift? | 13:38 |
| *** NikitaKonovalov has joined #savanna | 13:42 | |
| akuznetsov | possible it should be a preconfigure before savanna installation, the list of possible storage for job binaries | 13:43 |
| ruhe | akuznetsov, +1 | 13:46 |
| nprivalova | anyway user should not determine where to store jobs. this is regarding "give me a list of all the swifts or hdfs filesystems that I might use to store this binary" | 13:46 |
| openstackgerrit | Dmitry Mescheryakov proposed a change to stackforge/savanna: A Resource implementation for Conductor https://review.openstack.org/40400 | 13:48 |
| ruhe | on the other hand. we need to find a solution for the following case: i'm a user. i have MR job composed of N (N > 1) jar files. How should I upload this files to Savanna? | 13:48 |
| ruhe | one possible solution - upload all files to Swift/existing HDFS/Gluster and pass that path to EDP. It'll be represented as JobOrigin internally | 13:49 |
| ruhe | *user would upload these files | 13:49 |
| tmckayrh | ruhe, so just to clarify, the JobOrigin object in that case would store a path to the whole collection of files? | 13:50 |
| tmckayrh | Single path to a "dir" that contains all of the files | 13:50 |
| ruhe | tmckayrh, yeah | 13:52 |
| *** dina_belova has quit IRC | 13:52 | |
| tmckayrh | ruhe, even for the simple case of N = 1 (single jar), a user might want to ask "where can I put this so savanna can see it?" That was my thought. | 13:52 |
| tmckayrh | Maybe the answer is "ask your site administrator :)" | 13:53 |
| akuznetsov | ruhe savanna should have a two option one is upload files from UI and get it from exiting location | 13:53 |
| tmckayrh | Or "read this config file" | 13:53 |
| akuznetsov | UI part it can be just a convenient method for uploading jobs binaries | 13:54 |
| ruhe | akuznetsov, tmckayrh, i agree with you both. i'm just trying to understand what is achievable in the scope of v0.3 | 13:54 |
| tmckayrh | ruhe, maybe a silly question, but in the "collection of files" case above, how does the job manager know which file contains the "main" program? | 13:55 |
| nprivalova | I think we should consider loading only from existing location | 13:55 |
| ruhe | tmckayrh, they'll all be in classpath. user should just provide full class name for Mapper and for Reducer. or for the class which contains main() method to run MR job | 13:56 |
| tmckayrh | ruhe, ah, in the execution details fed to the job manager? | 13:57 |
| ruhe | yes | 13:57 |
| tmckayrh | okay, thanks :) | 13:57 |
| tmckayrh | hmmm, so eventually anyway a JobOrigin object may point to a single file or to a collection of files. I wonder if we need an indicator of that in the JobOrigin record.... | 14:00 |
| akuznetsov | let for first time assume that JobOrigin contains only one file | 14:01 |
| tmckayrh | :) akuznetsov, my mind is wandering down paths with orange cones blocking them off | 14:01 |
| nprivalova | maybe just add 'lib' param? | 14:01 |
| ruhe | akuznetsov, what about Pig and Hive cases? Usually there're used along with jar files containing UserDefinedFunctions | 14:02 |
| tmckayrh | also, in the "multiple file" case, I wonder how to describe it with a single job_type parameter. | 14:03 |
| akuznetsov | ruhe let create a first working version when add a support of multiplay files for JobOrigin | 14:03 |
| nprivalova | lib lib lib | 14:03 |
| tmckayrh | :) nprivalova, is that a boolean flag? | 14:04 |
| nprivalova | no, just path to all additional jars | 14:04 |
| tmckayrh | ah, of course | 14:04 |
| akuznetsov | tmckayrh job type means that kind of job, for example Oozie workflow will consists of several files | 14:04 |
| akuznetsov | a xml with workflow description and some executable code | 14:05 |
| *** dina_belova has joined #savanna | 14:05 | |
| tmckayrh | So, before we forget, the Job Origin Component blueprint description needs to change to say "binary" and talk about swift/gluster/hdfs. Like ruhe mentioned yesterday, things supporting "diff" like most vcs systems probably aren't helpful with non-human-readable binaries. | 14:09 |
| aignatov_ | this link you cannot change? https://blueprints.launchpad.net/savanna/+spec/edp-job-origin-component | 14:11 |
| aignatov_ | Trevor, I aasigned it for you, please can you confirm that you is able to change this? | 14:13 |
| *** rnirmal has joined #savanna | 14:14 | |
| *** lastidiot has joined #savanna | 14:14 | |
| akuznetsov | I changed blue print not it contains following "Job binaries can be stored in Swift, Gluster, Internal Savanna Database or can be download from ci/build server" | 14:15 |
| tmckayrh | aignatov_, okay, looks like it's done. I'll check anway, I looked to see if I could edit yesterday but I couldn't | 14:15 |
| tmckayrh | can binaries by stored in sqlite? | 14:20 |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests for Conductor Manager API https://review.openstack.org/40391 | 14:24 |
| akuznetsov | tmckayrh possible yes in case if we have open stack installation with out swift or gluster | 14:24 |
| akuznetsov | and edp works with some NoSQL databases | 14:25 |
| tmckayrh | ok, thanks. I have never tried to put a binary file in sql :) | 14:25 |
| tmckayrh | Looks like sqlite3 python bindings support it, though | 14:26 |
| openstackgerrit | Sergey Lukjanov proposed a change to stackforge/savanna: Add check S361 for imports of savanna.db module https://review.openstack.org/40428 | 14:26 |
| *** lastidiot has quit IRC | 14:27 | |
| ruhe | no no no :) we need to use distributed storage. don't forget about next-gen architecture | 14:28 |
| akuznetsov | tmckayrh stored in sql images for some my early projects, it not very elegant but allow moves a production code from one server to another very fast :) | 14:28 |
| openstackgerrit | Dmitry Mescheryakov proposed a change to stackforge/savanna: A Resource implementation for Conductor https://review.openstack.org/40400 | 14:29 |
| akuznetsov | ruhe, why not, I am not see a problem to store binaries in MySQL with replication | 14:29 |
| ruhe | meh, looks ugly to me :) | 14:30 |
| tmckayrh | hehe, I'm just asking questions. I bet it would be very fast to move to another server, but I think I agree with "meh, ugly". Using a hammer where a screwdriver is needed, I think. | 14:31 |
| akuznetsov | it should be option, possible a recommend and default way to store job binaries will be swift | 14:37 |
| tmckayrh | be back soon... | 14:42 |
| *** tmckayrh is now known as _tmckayrh | 14:42 | |
| *** qwerty_nor has joined #savanna | 14:44 | |
| *** aignatov has joined #savanna | 14:48 | |
| *** ruhe has quit IRC | 14:48 | |
| *** aignatov_ has quit IRC | 14:49 | |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests for Conductor Manager API https://review.openstack.org/40391 | 14:52 |
| *** IlyaE has joined #savanna | 14:59 | |
| *** ruhe has joined #savanna | 15:05 | |
| openstackgerrit | A change was merged to stackforge/savanna: Add check S361 for imports of savanna.db module https://review.openstack.org/40428 | 15:05 |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests for Conductor Manager API https://review.openstack.org/40391 | 15:07 |
| *** lastidiot has joined #savanna | 15:08 | |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests and fixes for Conductor Manager API https://review.openstack.org/40391 | 15:16 |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests and fixes for Conductor Manager API https://review.openstack.org/40391 | 15:17 |
| *** NikitaKonovalov has quit IRC | 15:17 | |
| *** NikitaKonovalov has joined #savanna | 15:24 | |
| _tmckayrh | all, I have a question about database scripts. If I am adding tables to the savanna db to store JobOrigins, then I imagine I need to supply some database migration script? | 15:33 |
| *** _tmckayrh is now known as tmckayrh | 15:33 | |
| tmckayrh | I have seen mention of alembic. | 15:33 |
| SergeyLukjanov | tmckayrh, hi | 15:34 |
| SergeyLukjanov | yep, we are using alembic for migrations | 15:34 |
| SergeyLukjanov | currently we are working on improving db code | 15:34 |
| SergeyLukjanov | and we'll have an separated conductor module | 15:35 |
| SergeyLukjanov | it's already contains new models | 15:35 |
| tmckayrh | SergeyLukjanov, I saw some commits come through. Should I try to write agains db_new? | 15:35 |
| SergeyLukjanov | but we aren't port current migrations to it | 15:35 |
| SergeyLukjanov | yep, absolutely | 15:35 |
| SergeyLukjanov | I think that we'll generate migration scripts when work on upgrading db code will be finished | 15:36 |
| tmckayrh | okay. so let's say for example I run savanna without job origins, and then I add an origin table and rerun. Should I worry about migrating, or should I just wipe out the savanna server db and rerun? | 15:36 |
| SergeyLukjanov | just wipe the db for dev now | 15:37 |
| tmckayrh | okay, thanks | 15:41 |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests and fixes for Conductor Manager API https://review.openstack.org/40391 | 16:01 |
| openstackgerrit | Nikita Konovalov proposed a change to stackforge/savanna: Unit Tests and fixes for Conductor Manager API https://review.openstack.org/40391 | 16:34 |
| *** gkleiman has joined #savanna | 16:42 | |
| openstackgerrit | Dmitry Mescheryakov proposed a change to stackforge/savanna: A Resource implementation for Conductor https://review.openstack.org/40400 | 16:50 |
| *** NikitaKonovalov has quit IRC | 16:54 | |
| *** nprivalova has quit IRC | 16:59 | |
| openstackgerrit | A change was merged to stackforge/savanna: Unit Tests and fixes for Conductor Manager API https://review.openstack.org/40391 | 17:00 |
| *** dmitryme has quit IRC | 17:02 | |
| *** ruhe has quit IRC | 17:04 | |
| *** IlyaE has quit IRC | 17:27 | |
| *** dina_belova has quit IRC | 17:28 | |
| *** SergeyLukjanov has quit IRC | 17:32 | |
| *** SergeyLukjanov has joined #savanna | 17:33 | |
| *** ruhe has joined #savanna | 17:33 | |
| *** ruhe has quit IRC | 17:34 | |
| *** SergeyLukjanov has quit IRC | 17:34 | |
| *** dina_belova has joined #savanna | 17:38 | |
| *** dina_belova has quit IRC | 17:40 | |
| *** tmckayrh has quit IRC | 17:42 | |
| *** tmckayrh has joined #savanna | 17:43 | |
| *** ruhe has joined #savanna | 17:47 | |
| *** qwerty_nor has quit IRC | 17:49 | |
| *** Nadya has joined #savanna | 17:54 | |
| crobertsrh | Hmm, I just pulled the latest from savanna/master and I'm seeing the following: http://fpaste.org/30422/58116971/ any tips? | 17:55 |
| *** NikitaKonovalov has joined #savanna | 17:55 | |
| crobertsrh | doh....Looks like I have requests 1.2.3, which is not < 1.2.3 | 17:55 |
| ruhe | requirements.txt was updated today. maybe it introduced this issue | 17:58 |
| crobertsrh | ah, should it be strictly less than 1.2.3, or should 1.2.3 be fine? | 17:58 |
| ruhe | requirements is aligned across all OS projects (afaik) | 17:59 |
| ruhe | so, < 1.2.3 must be there on purpose | 18:00 |
| crobertsrh | Ok, this is on Fedora 17. I'll see what I can work out. | 18:00 |
| *** NikitaKonovalov has quit IRC | 18:00 | |
| ruhe | do you have requests installed in system path? | 18:00 |
| ruhe | because tools/install_venv should install proper versions of dependencies | 18:01 |
| crobertsrh | It is on system path (1.2.3) | 18:01 |
| ruhe | i suggest to use venv to avoid such problems | 18:02 |
| *** dina_belova has joined #savanna | 18:02 | |
| crobertsrh | Yeah, venv has 1.2.3 also :( | 18:02 |
| ruhe | hmm | 18:02 |
| crobertsrh | I am only running savanna with venv | 18:02 |
| ruhe | ah, so there might be a conflict between dependencies of savanna | 18:03 |
| crobertsrh | Not seeing it right off hand | 18:04 |
| ruhe | can you grep in .tox for requests>=1.1,<1.2.3 and find out which project has this requirement? | 18:04 |
| crobertsrh | python_novaclient seems to be the only place I'm seeing it | 18:05 |
| crobertsrh | .tox/venv/lib/python2.7/site-packages/python_novaclient-2.14.0-py2.7.egg-info/requires.txt:requests>=1.1,<1.2.3 | 18:05 |
| crobertsrh | .tox/venv/lib64/python2.7/site-packages/python_novaclient-2.14.0-py2.7.egg-info/requires.txt:requests>=1.1,<1.2.3 | 18:05 |
| *** akuznetsov has quit IRC | 18:06 | |
| ruhe | i'm checking this on the latest master now in fresh directory | 18:06 |
| crobertsrh | Thanks | 18:07 |
| ruhe | yes, i have the same error | 18:08 |
| ruhe | find . -name 'require*' -exec grep -H 'request' {} \; | 18:09 |
| ruhe | ./venv/lib/python2.7/site-packages/python_cinderclient-1.0.4-py2.7.egg-info/requires.txt:requests>=0.8 | 18:09 |
| ruhe | ./venv/lib/python2.7/site-packages/python_keystoneclient-0.3.1-py2.7.egg-info/requires.txt:requests>=0.8.8 | 18:09 |
| ruhe | ./venv/lib/python2.7/site-packages/python_novaclient-2.14.0-py2.7.egg-info/requires.txt:requests>=1.1,<1.2.3 | 18:09 |
| crobertsrh | Ah, somehow, I missed the other requires.txt entries :) | 18:09 |
| crobertsrh | Should I file a bug? | 18:11 |
| ruhe | yes | 18:11 |
| crobertsrh | will do. Thanks | 18:11 |
| ruhe | i hope SergeyLukjanov will be able to resolve this asap | 18:11 |
| ruhe | i wonder how UT and IT missed this | 18:13 |
| crobertsrh | Yeah, seems like it might be hard to miss. | 18:14 |
| crobertsrh | https://bugs.launchpad.net/savanna/+bug/1208951 | 18:14 |
| *** dmitryme has joined #savanna | 18:15 | |
| ruhe | we'll need to check all CI jobs carefully to see if there is a misconfiguration which allows such issues to pass | 18:15 |
| ruhe | crobertsrh, thanks for catching this | 18:16 |
| crobertsrh | Looks like I picked the wrong (or maybe right) day to update :) | 18:16 |
| *** SergeyLukjanov has joined #savanna | 18:16 | |
| crobertsrh | no prob. That's what I'm here for | 18:16 |
| crobertsrh | Oh. Ruhe. Did you ever come up with those images that include Oozie to try out? | 18:17 |
| ruhe | i haven't time yet to work with this part of Savanna | 18:18 |
| *** NikitaKonovalov has joined #savanna | 18:19 | |
| Nadya | I did | 18:19 |
| ruhe | afaik, they're not published yet. so you'll need to built them yourself. https://github.com/stackforge/savanna-extra/tree/master/elements | 18:20 |
| ruhe | element for oozie is there | 18:20 |
| crobertsrh | cool. I've never built one yet. Another chance to learn something! | 18:20 |
| crobertsrh | Thanks | 18:20 |
| ruhe | welcome :) | 18:20 |
| SergeyLukjanov | crobertsrh, I'll take a look at requests problem | 18:21 |
| crobertsrh | ack. Thanks Sergey! | 18:21 |
| *** ruhe has quit IRC | 18:22 | |
| SergeyLukjanov | it's absolutely strange because I performed several manual tests and unit/integration tests works too | 18:22 |
| SergeyLukjanov | the possible reason for it - pypi mirrors | 18:23 |
| *** dina_belova has quit IRC | 18:25 | |
| SergeyLukjanov | here is the update commit and the only keystone client version has been increased - https://github.com/stackforge/savanna/commit/367ad522b2a86c9a7b7eae4587edc7d8f5d9bf18 | 18:29 |
| *** dina_belova has joined #savanna | 18:34 | |
| *** mattf is now known as _mattf | 18:44 | |
| *** _mattf is now known as mattf | 18:45 | |
| *** NikitaKonovalov has quit IRC | 18:47 | |
| *** crobertsrh has quit IRC | 18:47 | |
| SergeyLukjanov | mm, here is a reason of this problem - https://review.openstack.org/#/c/37461/2/requirements.txt | 18:48 |
| *** dina_belova has quit IRC | 18:48 | |
| SergeyLukjanov | it's just a problem of checking dependencies | 18:48 |
| *** NikitaKonovalov has joined #savanna | 18:49 | |
| SergeyLukjanov | I'll merge fix for it asap to savanna | 18:49 |
| *** ruhe has joined #savanna | 18:49 | |
| *** dina_belova has joined #savanna | 18:51 | |
| tmckayrh | Hi folks. Question, when representing a tuple value in the orm stuff, how do I do that? Example, the credentials value for Job Origin and Data Source object which contains a user and password. I see a JonDictType used to build a column, is that the right thing? | 18:54 |
| SergeyLukjanov | tmckayrh, good question :) | 18:55 |
| SergeyLukjanov | btw we can add JsonTupleType... | 18:55 |
| SergeyLukjanov | currently only swift credentials will be stored here, isn't it? | 18:56 |
| openstackgerrit | Sergey Lukjanov proposed a change to stackforge/savanna: Fix requests version https://review.openstack.org/40475 | 18:57 |
| tmckayrh | SergeyLukjanov, I don't know :) Just reading the structure. I think Nadya was just chatting with me about that.... | 18:57 |
| tmckayrh | brb, have to move a rug.... | 18:57 |
| SergeyLukjanov | I think it could be currently considered as an credentials storage | 18:58 |
| Nadya | let it be just a tuple I think | 18:59 |
| Nadya | for now | 18:59 |
| Nadya | user-password | 18:59 |
| tmckayrh | SergeyLukjanov, Nadya, okay. It could be two separate string fields, or a dict, or an ordered list, or if you have an idea for a tuple that is somehow different I could try to implement that.... I think either JsonDict or JsonList is likely to work... | 19:02 |
| tmckayrh | dict makes sense if the fields are named | 19:03 |
| SergeyLukjanov | looks like that fix for requests version works good, but I can't merge it write now because broken docs build is now part of our gating proces... | 19:09 |
| SergeyLukjanov | I hope that I | 19:09 |
| SergeyLukjanov | will fix it today | 19:09 |
| openstackgerrit | Jonathan Maron proposed a change to stackforge/savanna: Made Ambari RPM location configurable https://review.openstack.org/40479 | 19:13 |
| *** ruhe has quit IRC | 19:15 | |
| *** IlyaE has joined #savanna | 19:21 | |
| *** NikitaKonovalov has quit IRC | 19:21 | |
| *** NikitaKonovalov has joined #savanna | 19:21 | |
| *** crobertsrh has joined #savanna | 19:25 | |
| *** dmitryme has quit IRC | 19:28 | |
| openstackgerrit | Sergey Lukjanov proposed a change to stackforge/savanna: Bump sphinx to the latest version https://review.openstack.org/40485 | 19:30 |
| *** Nadya has quit IRC | 19:56 | |
| *** dina_belova has quit IRC | 19:58 | |
| *** lastidiot has quit IRC | 20:01 | |
| *** lastidiot has joined #savanna | 20:02 | |
| *** NikitaKonovalov has quit IRC | 20:14 | |
| *** crobertsrh has quit IRC | 20:21 | |
| openstackgerrit | Sergey Lukjanov proposed a change to stackforge/savanna: Fix docs build https://review.openstack.org/40493 | 20:23 |
| openstackgerrit | A change was merged to stackforge/savanna: Fix docs build https://review.openstack.org/40493 | 20:36 |
| openstackgerrit | A change was merged to stackforge/savanna: Fix requests version https://review.openstack.org/40475 | 20:50 |
| *** dmitryme has joined #savanna | 20:50 | |
| *** gkleiman has quit IRC | 20:53 | |
| *** dina_belova has joined #savanna | 20:58 | |
| *** dina_belova has quit IRC | 21:02 | |
| *** dina_belova has joined #savanna | 21:08 | |
| *** SergeyLukjanov has quit IRC | 21:08 | |
| *** dina_belova has quit IRC | 21:13 | |
| *** mattf is now known as _mattf | 21:15 | |
| *** NikitaKonovalov has joined #savanna | 21:15 | |
| *** NikitaKonovalov has quit IRC | 21:19 | |
| *** tstclair is now known as _tstclair | 21:20 | |
| *** dmitryme has quit IRC | 21:28 | |
| *** dina_belova has joined #savanna | 22:09 | |
| *** dina_belova has quit IRC | 22:13 | |
| *** NikitaKonovalov has joined #savanna | 22:15 | |
| *** NikitaKonovalov has quit IRC | 22:20 | |
| *** lastidiot has quit IRC | 22:26 | |
| *** _mattf is now known as mattf | 22:36 | |
| *** dina_belova has joined #savanna | 23:09 | |
| *** dina_belova has quit IRC | 23:13 | |
| *** NikitaKonovalov has joined #savanna | 23:15 | |
| *** rnirmal has quit IRC | 23:19 | |
| *** NikitaKonovalov has quit IRC | 23:20 | |
| *** IlyaE has quit IRC | 23:34 | |
| *** lastidiot has joined #savanna | 23:49 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!