Tuesday, 2011-08-16

vishys1rp: ping00:00
anotherjessemtaylor: private message ping00:13
*** chipc has joined #openstack-dev00:18
*** heckj has quit IRC00:38
*** mgius has quit IRC00:44
*** mszilagyi has quit IRC00:45
*** rods has quit IRC00:49
*** alekibango has quit IRC00:53
*** soren has quit IRC00:53
*** alekibango has joined #openstack-dev00:53
*** soren has joined #openstack-dev00:53
*** ChanServ sets mode: +v soren00:53
*** mfer has joined #openstack-dev01:00
*** jakedahn has quit IRC01:13
*** anotherjesse has quit IRC01:15
*** Tushar has quit IRC01:40
*** anotherjesse has joined #openstack-dev01:44
*** anotherjesse has quit IRC01:44
*** mfer has quit IRC02:06
*** lorin1 has joined #openstack-dev02:13
*** lorin1 has left #openstack-dev02:13
openstackgerritJoe Heck proposed a change to openstack/openstack-ci-puppet: fixes lp827070  https://review.openstack.org/24302:56
*** nci has joined #openstack-dev03:05
*** zorzar has quit IRC03:30
*** jakedahn has joined #openstack-dev03:32
*** mwhooker has quit IRC03:33
*** zorzar has joined #openstack-dev03:39
*** pasik has quit IRC03:42
*** mwhooker has joined #openstack-dev03:45
*** jakedahn has quit IRC04:07
*** tsuzuki_ has joined #openstack-dev07:01
*** reidrac has joined #openstack-dev07:09
*** pasik has joined #openstack-dev07:37
*** alekibango has quit IRC08:02
*** negronjl has quit IRC08:13
*** alekibango has joined #openstack-dev08:38
*** negronjl has joined #openstack-dev08:39
openstackgerritYuriy Taraday proposed a change to openstack/keystone: Moved run_test logic into abstract class.  https://review.openstack.org/24409:00
*** negronjl has quit IRC09:07
*** darraghb has joined #openstack-dev09:07
*** rods has joined #openstack-dev09:12
*** phil121 has joined #openstack-dev09:17
*** tsuzuki_ has quit IRC09:20
*** negronjl has joined #openstack-dev09:35
*** mxant has joined #openstack-dev09:39
*** mxant has quit IRC09:53
*** lorin1 has joined #openstack-dev09:53
*** lorin1 has left #openstack-dev09:55
ttxsoren: anything to do on the nova side for bug 806911 ? If not, just mark it invalid10:22
uvirtbotLaunchpad bug 806911 in nova "Split nova-compute package into nova-compute-{kvm,xen,etc.}" [Wishlist,Confirmed] https://launchpad.net/bugs/80691110:22
zykes-why should one split it ?10:35
reidracit's convenient if you want to use one hypervisor and you don't want to install rependencies related the ones you don't want to use (?)10:42
zykes-ah10:43
*** jamespage has joined #openstack-dev11:09
*** markvoelker has joined #openstack-dev11:12
*** alekibango has quit IRC11:13
sorenttx: Nope, I think it's all done.11:13
*** lorin1 has joined #openstack-dev11:23
ttxsoren: ok -- marked the nova task invalid.11:36
*** reidrac has left #openstack-dev11:39
*** reidrac has joined #openstack-dev11:39
*** alekibango has joined #openstack-dev11:44
*** mfer has joined #openstack-dev11:52
*** Adri2000 has quit IRC12:08
*** Adri2000 has joined #openstack-dev12:08
*** chipc has quit IRC12:13
*** lts has joined #openstack-dev12:14
*** zykes- has quit IRC12:25
*** zykes- has joined #openstack-dev12:26
*** dprince has joined #openstack-dev12:32
*** sandywalsh has joined #openstack-dev12:36
dprinceJust killed the Jenkins tarmac job (it was hung for 14 hours). This should allow the approved merge props to land.12:39
*** bsza has joined #openstack-dev12:47
*** chipc has joined #openstack-dev12:55
openstackjenkinsProject nova build #1,253: SUCCESS in 3 min 25 sec: https://jenkins.openstack.org/job/nova/1253/13:06
openstackjenkins* Tarmac: Refactored the HostFilterScheduler and LeastCostScheduler classes so that they can be combined into a single class that can do both host filtering and host weighting, allowing subclasses to override those processes as needed. Also renamed the ZoneAwareScheduler to AbstractScheduler, for two reasons: one, the 'zone-aware' designation was necessary when the zone code was being developed; now that it is part of13:06
openstackjenkins* Tarmac: cosmetic change in test_extensions.13:06
openstackjenkinsAvoids constant merge conflicts between proposals with new extensions.13:06
openstackjenkins* Tarmac: Validate the size of VHD files in OVF containers.13:06
*** kbringard has joined #openstack-dev13:07
*** mancdaz has quit IRC13:18
*** nmistry has joined #openstack-dev13:27
*** amccabe has joined #openstack-dev13:31
dprincejk0: Can you have a look at: https://bugs.launchpad.net/nova/+bug/82737313:50
uvirtbotLaunchpad bug 827373 in nova "Booting m1.tiny instance causes 'ImageTooLarge' exception" [Undecided,New]13:50
jk0seems like having local_gb as 0 is the problem... math seems correct13:51
jk0anyone know why that would be 0?13:51
dprincejk0: local_gb 0 is the default setting in our database. And lots of people use m1.tiny for testing etc.13:52
jk0right but why would it be 0?13:52
jk0that doesn't make any sense13:52
jk0wonder if adding a migration and moving that to 1 would be the way to go13:53
dprincejk0: Yeah. That sounds reasonable. This is going to cause some pain otherwise.13:53
jk0cool. having it at 0 is silly IMHO13:54
dprincejk0: Perhaps adding a check to the _check_image_size so that if 'local_gb' is zero we log that as well.13:55
openstackgerritZiad Sawalha proposed a change to openstack/keystone: Added reponse handling for xsd static file rendering II  https://review.openstack.org/24513:55
jk0yeah, good call13:56
openstackgerritZiad Sawalha proposed a change to openstack/keystone: Ziad's changes from github.com/rackspace/openstack  https://review.openstack.org/23713:58
dprincejk0: you wanna take this one?13:58
jk0I'll try and work it in today unless someone else grabs it before then13:58
dprincesure. Thanks.13:58
*** nmistry has quit IRC14:19
*** zykes- has quit IRC14:21
openstackgerritDolph Mathews proposed a change to openstack/keystone: Added pylint message count as run_tests.sh -l  https://review.openstack.org/24614:29
*** jaypipes has quit IRC14:39
openstackgerritDolph Mathews proposed a change to openstack/keystone: Moved run_test logic into abstract class.  https://review.openstack.org/24414:42
openstackgerritDolph Mathews proposed a change to openstack/keystone: Moved run_test logic into abstract class.  https://review.openstack.org/24414:44
*** dolphm has joined #openstack-dev14:44
*** nmistry has joined #openstack-dev14:45
*** martine has joined #openstack-dev14:45
openstackgerritA change was merged to openstack/keystone: Moved run_test logic into abstract class.  https://review.openstack.org/24414:48
*** jaypipes has joined #openstack-dev14:53
openstackgerritZiad Sawalha proposed a change to openstack/keystone: Added reponse handling for xsd static file rendering III  https://review.openstack.org/23714:53
*** dprince has quit IRC14:57
*** rnirmal has joined #openstack-dev15:01
*** dragondm has joined #openstack-dev15:02
openstackgerritDolph Mathews proposed a change to openstack/keystone: Creating an artificial whitespace merge conflict  https://review.openstack.org/24715:04
*** zykes- has joined #openstack-dev15:05
*** reidrac has quit IRC15:06
openstackgerritA change was merged to openstack/keystone: Creating an artificial whitespace merge conflict  https://review.openstack.org/24715:07
*** mancdaz has joined #openstack-dev15:08
*** yogirackspace has joined #openstack-dev15:10
openstackjenkinsProject nova build #1,254: SUCCESS in 3 min 56 sec: https://jenkins.openstack.org/job/nova/1254/15:10
openstackjenkinsTarmac: Updated the EC2 metadata controller so that it returns the correct value for instance-type metadata.15:10
openstackgerritZiad Sawalha proposed a change to openstack/keystone: Added reponse handling for xsd static file rendering III Extra extension tests (for RS-KEY)  https://review.openstack.org/23715:12
*** zns has joined #openstack-dev15:14
*** kbringard has quit IRC15:15
*** mattray has joined #openstack-dev15:15
openstackgerritA change was merged to openstack/keystone: Added reponse handling for xsd static file rendering III Extra extension tests (for RS-KEY)  https://review.openstack.org/23715:16
*** nmistry has quit IRC15:19
*** heckj has joined #openstack-dev15:20
*** nmistry has joined #openstack-dev15:20
*** cp16net has joined #openstack-dev15:21
*** heckj has quit IRC15:23
*** heckj has joined #openstack-dev15:23
*** cp16net has quit IRC15:31
*** cp16net has joined #openstack-dev15:31
*** dprince has joined #openstack-dev15:37
ttxhmm; what's the correct way of running run_tests.sh with just one test ?15:44
*** Guest77784 has quit IRC15:45
*** Guest77784 has joined #openstack-dev15:46
comstudi end up just running (from root of a branch): nosetests -s <path/to/file>:Class.method15:46
jaypipesttx: ./run_tests.sh -V nova.tests.test_utils15:46
comstudttx: ^^15:46
ttxthx guys !15:46
jaypipescomstud: ./run_tests.sh takes all the nose args, too..15:47
comstudcools15:47
*** nmistry has quit IRC15:47
*** cp16net has quit IRC15:58
*** zns has quit IRC15:58
*** cp16net has joined #openstack-dev16:00
*** zns has joined #openstack-dev16:04
*** p1ra has joined #openstack-dev16:17
*** p1ra has left #openstack-dev16:17
openstackgerritMonty Taylor proposed a change to openstack/openstack-ci-puppet: Added python-webtest to quantum slave.  https://review.openstack.org/24816:27
openstackgerritMonty Taylor proposed a change to openstack/openstack-ci-puppet: Added memcached to the keystone slave.  https://review.openstack.org/20216:27
*** troytoman-away is now known as troytoman16:29
openstackgerritMonty Taylor proposed a change to openstack/openstack-ci-puppet: Added python-webtest to quantum slave.  https://review.openstack.org/24816:32
openstackgerritVerification of a change to openstack/keystone failed: Added pylint message count as run_tests.sh -l  https://review.openstack.org/24616:34
*** negronjl has quit IRC16:38
*** negronjl has joined #openstack-dev16:39
openstackjenkinsProject nova build #1,255: SUCCESS in 4 min 5 sec: https://jenkins.openstack.org/job/nova/1255/16:46
openstackjenkinsTarmac: This branch implements a nova api extension which allows you to manage and update tenant/project quotas.16:46
heckjmtaylor: did I do something wrote with https://review.openstack.org/243 that you created https://review.openstack.org/24816:48
heckjmtaylor: er, /wrote/wrong16:49
mtaylorheckj: nope! I just somehow missed yours. oops :)16:54
heckjmtaylor: heh, I wondered if you were just hammering out patches this morning and ran over it. Tried to save you some work.16:54
openstackgerritA change was merged to openstack/openstack-ci-puppet: fixes lp827070  https://review.openstack.org/24316:54
heckjmtaylor: is "bug/######" the preferred format to link to bugs in LP?16:55
heckjmtaylor: I kept with the lp###### style.16:55
mtaylorheckj: you can mention bugs in the commit message either way16:55
mtaylorheckj: bug#### bug:#### lp:##### lp #####, etc ...16:55
mtaylorheckj: ([Bb]ug|[Ll][Pp])\s*[#:]?\s*(\d+)16:56
heckjmtaylor: cool - nice! Don't have to worry about format so much then16:56
mtaylorheckj: actually is the regex16:56
mtaylorheckj: and then we're working on more integration stuff with that at the moment16:56
*** Tushar has joined #openstack-dev16:57
*** cp16net has quit IRC16:59
*** ohnoimdead has joined #openstack-dev17:03
*** YorikSar has joined #openstack-dev17:21
*** jpipes has joined #openstack-dev17:21
*** jpipes has quit IRC17:21
*** jaypipes has quit IRC17:21
*** jaypipes has joined #openstack-dev17:23
*** jaypipes has quit IRC17:24
*** jaypipes has joined #openstack-dev17:24
YorikSarHello. Any Keystone or testing fans here?17:24
jaypipesYorikSar: yes, fan of both :)17:25
YorikSarI've sent to review the first part of my suggestion. Teh easy one.17:25
jaypipesYorikSar: link?17:26
YorikSarBut now I'm thinking about shell scripts used for testing and fixturing. Is it really nesessary?17:26
jaypipesYorikSar: not sure what you mean... are you asking if run_tests.sh and run_tests.py are necessary?17:27
YorikSarjaypipes: Oh, as I see now, that request is already merged.17:27
jaypipesYorikSar: :)17:28
YorikSarjaypipes: I mean, why do we have both17:28
jaypipesYorikSar: the shell script is a little wrapper that handles the virtualenv stuff cleanly...17:28
jaypipesYorikSar: I suppose one could refactor it out.17:28
YorikSarjaypipes: Shouldn't it be better to write all this logic in one Python script17:28
jaypipesYorikSar: yep. It's just legacy, that's all. Feel free to consolidate them if you wish.17:29
YorikSarIs venv  being used in run_test.py?17:29
jaypipesYorikSar: venv is activated by the run_tests.sh script (it prepends tools/with_venv.sh to the call to run_tests.py17:30
YorikSarjaypipes: It can be activated by the script itself as well17:30
jaypipesYorikSar: so you would need to roll that functionality into the run_tests.py, which may be a bit difficult, since the shell script sets the venv environment then executes the tests within that venv...17:30
dolphmYorikSar: jaypipes: i added a pylint option to run_tests.sh, but it hasn'17:31
dolphm...hasn't been merged yet17:31
YorikSarjaypipes: I had a bit of experience with some complex import-reimport and sys.path hacking stuff17:31
YorikSardolphm: All that pylints and pep8 can be run from the Python script - that's my point.17:32
YorikSarSo as I understand, if I send a request that eliminates both .sh files tomorrow, it has chance to be merged?17:33
dolphmYorikSar: i find ./run_tests.sh and python run_tests.py to be way too many characters to type, so i actually have a fabric script in my repo so i can just run "fab test", "fab review" etc17:34
jaypipesdolphm: hmm ./run<TAB> is all I need ;)17:34
zykes-clea17:34
YorikSardolphm: ./r<tab><tab><tab> is way shorter17:35
dolphm./ is too awkward to constantly type17:35
jaypipesYorikSar: :)17:35
*** lorin1 has quit IRC17:35
* jaypipes buys dolphm typing lessons :P17:35
* dolphm can type faster than you17:35
YorikSardolphm: I even type "git fetch origin" all the time.17:35
*** jamespage has left #openstack-dev17:36
YorikSardolphm: It gives some time to think about the action before pressing enter17:36
* dolphm wouldn't mind taking a typing class for a dvorak layout17:36
dolphmjenkins is currently dependent on being able to use ./run_tests.sh, right?17:37
YorikSardolphm, jaypipes: So, one more question. Is bin/ dir really nessesary?17:37
*** joesavak has joined #openstack-dev17:37
dolphmYorikSar: a lot of the code in it should 110% be refactored into keystone/17:38
jaypipesYorikSar: uhm, yeah?17:38
dolphmbin/ should contain just a couple lines of actual python code17:38
jaypipesYorikSar: bin/ is for executables installed17:38
YorikSardolphm, jaypipes: I mean, there is smth like a convention never to write a scrips but to use a distutils script installers instead17:39
dprincecd ~/projects/nova/trunk/17:39
dprincels17:39
dprinceoops17:39
dolphm:)17:39
openstackjenkinsProject nova build #1,256: SUCCESS in 3 min 32 sec: https://jenkins.openstack.org/job/nova/1256/17:40
openstackjenkinsTarmac: * Added search instance by metadata.17:40
openstackjenkins* instance_get_all_by_filters should filter deleted17:40
jaypipesYorikSar: could you show an example of that for WSGI servers?17:40
YorikSarWell, the bin dir is there, but shouldn't it be removed just like the shell scripts?17:41
jaypipesYorikSar: I don't think so...17:41
*** cp16net has joined #openstack-dev17:42
jaypipesYorikSar: where would you put executables? In the keystone/ directory along with regular python modules?17:43
dolphmjaypipes: i think his point is that they're completely unnecessary17:43
YorikSarjaypipes: For example, SimpleHTTPServer in stblib :)17:43
jaypipesdolphm: how so?17:43
jaypipesYorikSar: you're not serious, are you?17:44
*** anotherjesse has joined #openstack-dev17:44
YorikSarjaypipes: nowhere. There should be functionality somewhere in keystone package that can be called from outside either by "python -m" or by a distutils-like script.17:45
dolphmjaypipes: 'python -m keystone' <-- instead of './bin/keystone'17:45
dolphm'python -m keystone --admin-only' <-- instead of './bin/keystone-admin'17:46
dolphm'python -m keystone --service-only' <-- instead of './bin/keystone-service'17:46
YorikSarjaypipes: I'm serious a bit. You can run 'pythom -m SimpleHTTPServer' and get a cute small server for static files.17:46
dolphmYorikSar: the fact that we're serving static files through keystone makes me cry17:46
jaypipesdolphm: And that's how you want people to call keystone?17:46
YorikSardolphm: may be 'python -m keystone.server.admin' and 'python -m keystone.server.service' should be better.17:47
jaypipesdolphm: well, I agree with you on the static files thing...17:47
dolphmYorikSar: that works!17:47
dolphmjaypipes: sure, why not? it's python, treat it like python and you'll have a cleaner project17:47
jaypipesYorikSar, dolphm: Would you guys think starting Apache (if Apache were a Python executable...) like "python -m httpd" would be appropriate? :(17:48
dolphmjaypipes: we're serving a billion more static files today than we were yesterday...17:48
Vekjaypipes: you have a moment?  Trying to puzzle out how best to approach the functional tests...17:48
YorikSarjaypipes: That's how they can be called by developers, but for human beings distutils will generate a script.17:48
jaypipesYorikSar: well, I'm certainly open to seeing a demo of it. :)17:48
YorikSarjaypipes: For example, try "cat `which pep8`"17:48
jaypipesYorikSar: easy_install is not how someone would install on a production system. That's not package-worthy...17:50
dolphmYorikSar: jaypipes: if we refactor the python code out of bin/, making 'python -m keystone', 'python -m keystone.server', 'python -m keystone.server.admin', 'python -m keystone.server.service' all work would be trivial17:50
YorikSarjaypipes: About httpd - certainly no. But nothing prevents you from keeping all that apache code in some apache package, and just create an entry point in setup.py.17:50
YorikSarjaypipes: Well, lots of debs actually just wrap this process...17:51
dolphmjaypipes: it also bugs me that keystone is NOT an executable, it's just a freaking web service, so why is it not running under a real web server (like apache/nginx) anyway? i don't see any advantage to reinventing the wheel here17:52
dolphm /vent17:52
*** lorin1 has joined #openstack-dev17:52
jaypipesYorikSar: like I said, I welcome the code :)17:52
jaypipesdolphm: you might want to ask creiht that question....17:53
YorikSardolphm: Good point. Currently we force users to use a Paste, but why not cute things like uwsgi or popular httpd?17:53
*** darraghb has quit IRC17:55
*** mxant has joined #openstack-dev17:56
*** mxant has quit IRC17:57
*** jsavak has joined #openstack-dev18:00
*** joesavak has quit IRC18:00
*** jsavak has quit IRC18:00
YorikSarjaypipes: Well, I've checked some projects and all of them actually have bin/ dir with scripts almost identical to the distutils ones.18:02
zykes-nginx :p18:02
znsYoriksar: many of the responses from Keystone are static (ex: WADL, version contracts, etc.). I agree we don't want it to be a static file server, and anyone can configure it to point to a different endpoint for the contracts, but at least installing it on a laptop and having access to all the contracts is OK imo.18:03
dolphmzykes-: nginx is awesome, but doesn't it have some kind of issue running python wsgi?18:03
zykes-dunno dolphm18:03
dolphmzykes-: i've only used it as a static file server & reverse proxy18:03
znsAnd anyone should fee free to cache the static files…18:03
znsfeel18:04
*** joesavak has joined #openstack-dev18:04
zykes-varnish :)18:04
YorikSardolphm, zykes-: It cannot run wsgi, but it has a cool uwsgi protocol implementation.18:04
znsSquid18:04
jaypipesYorikSar: aha :)18:04
zykes-zns: varnish is nicer i think :p18:04
YorikSarjaypipes: But I still don't get the reason.18:04
jaypipesSquarnish FTW!18:05
dolphmzns: if that's the acknowledged intention, it static files simply shouldn't be served by keystone in a production deployment ... the threat vectors on the static files controller are pretty scary18:05
YorikSarjaypipes: Why don't use that distutils to generate the same, but version-aware script18:06
jaypipesYorikSar: lack of knowledge on our part? :) Like I said, I'd love to see code demonstrating that18:06
znsdolphm: yes. Is there a better way to serve static files on a one-machine, one port deployment?18:08
dolphmzns: that's exactly what i've used nginx for18:09
zykes-what's bad about keystone serving static ?18:09
znsdolphm: but I don't have to install/config nginx to run keystone on my machine now…  we could offer an alternative config (like we support sqlite and MySQL for different scale deployments).18:11
jaypipeszykes-: just consumes resources that can be better served through a simple web server like nginx or lighttpd...18:11
dolphmzns nginx listens on your public port (say, :5000), checks for requests to a specific path... '/static/', if there's a match, serve the static file, if there's no match, internally forward (reverse proxy) the request to the underlying keystone server (on some arbitrary private port)18:11
zykes-that's easy to do18:11
dolphmzykes-: security is our responsibility, code overhead, limited end user configuration18:11
dolphmzns: for example, django serves your static files for you in a development environment (with zero overhead on your part), but refuses to do so in a production environment (citing security)18:13
dolphmand i believe rails follows the same pattern (?)18:14
YorikSarjaypipes: I can explain how to use it, but of course I cannot provide a full solution for Keystone right now.18:17
YorikSarjaypipes: Just add an entry_points parameter in which you provide names of the generated scripts and module:method pairs for them.18:18
dolphmYorikSar: .. why 'of course'? lol18:18
openstackgerritDolph Mathews proposed a change to openstack/keystone: Admin WADL updates  https://review.openstack.org/24918:19
YorikSarjaypipes: So after setup.py install or bdist the will be generated and put into apropriate dir under either build/ directory or somewhere in /usr/local/bin.18:19
openstackgerritDolph Mathews proposed a change to openstack/keystone: Admin WADL updates  https://review.openstack.org/24918:21
YorikSardolphm: Because I started clearing out the keystone-manage script and found there some bad force to fight with.18:21
dolphmYorikSar: you're refactoring it?18:22
YorikSardolphm: Yes.18:22
dolphmsweet18:22
YorikSardolphm: But don't expect anything too sweet. My goal is to separate options parsing from command execution at first. I want to make sampledata.py run thousand times quicker than sampledata.sh.18:23
openstackgerritDolph Mathews proposed a change to openstack/keystone: Admin WADL updates  https://review.openstack.org/24918:27
jk0dprince: I'll have a fix for you to test coming in a few minutes18:28
dolphmYorikSar: sounds like a MAJOR improvement :D18:28
*** lorin1 has quit IRC18:28
dolphmYorikSar: ... does that mean we could functional tests using an in-memory sqlite backend?18:28
znsYorikSar: +118:28
YorikSardolphm: If I will break down current command processing logic into something object-oriented, it will definetly be in separate request.18:29
dolphmYorikSar: can't wait!18:29
YorikSardolphm: I'm not sure about functional tests. I never tried to understand the big difference in details of their implementation.18:30
YorikSardolphm: btw, can you enlighten me, why do unit tests require running keystone server?18:31
dolphmYorikSar: one of the major issues with testing is that a most of them are dependent on sampledata.sh to populate the db.. which is A) slow, B) obviously not self-contained in the tests, C) sampledata.sh starts and stops the server on every call, which means an in-memory db configuration would be wiped away after every line in sampledata.sh18:31
dolphmYorikSar: most of the "unit" tests are actually functional/system tests... which require a server to run against18:32
YorikSardolphm: C) caused a huge rewrite of fakeldap from Nova18:32
YorikSardolphm: But why cannot we just exclude http server and client from testing?18:33
YorikSardolphm: I mean, make almost direct call to WSGI app and get a response instead of running a stand-alone http server and using httplib to access it.18:35
dolphmYorikSar: we have a few such tests, and more are certainly welcome, but i think we also need higher level tests to test the interaction between middleware & server... so starting keystone in various configs makes sense there18:37
*** lorin1 has joined #openstack-dev18:37
dolphmjust as an example18:37
jaypipesYorikSar: keystone already includes entry_points in setup()...18:38
jaypipesdolphm: Glance does functional testing just fine using an in-memory SQLite database.18:39
dolphmjaypipes: yeah, but our tests are currently handicapped by sampledata.sh18:39
jaypipesdolphm: yup.18:40
dolphmjaypipes: e.g. the tests shouldn't depend at all on sampledata.sh18:40
jaypipesdolphm: right. all that stuff should be done in FunctionalTestCase.setUp()...18:40
dolphmjaypipes: ++18:40
YorikSarjaypipes: But they include only entry points for paste, not console scripts that are installed the way I described.18:40
jaypipesYorikSar: that's what the scripts= param is for...18:41
YorikSardolphm, jaypipes: I'm almost ready to run back to workplace to finish work on sampledata :)18:41
jaypipesYorikSar: :)18:42
jaypipesYorikSar: where are you based?18:42
dolphmjaypipes: how do other projects bootstrap themselves (in tests or production)? i.e. keystone needs an Admin role, admin user, and a link between the two before the API can be used at all18:42
zykes-has anyone made the thought of making a common way for doing things ?18:42
*** alekibango has quit IRC18:42
dolphmzykes-: https://github.com/openstack/openstack-skeleton ?18:43
jaypipesdolphm: the API server should be able to be started without an admin role/user established... is that not the case?18:43
zykes-dolphm: more like common libraries18:43
zykes-etc18:43
YorikSarjaypipes: scripts param defines existing scripts that is already there and should be copied to disribution package and to the target system.18:43
jaypipeszykes-: in skeleton, there is an openstack/ directory which is being used as the place to put common lib stuff..18:43
jaypipeszykes-: still very new, though :(18:44
dolphmjaypipes: not that i'm aware of18:44
*** lorin1 has quit IRC18:44
zykes-jaypipes: i meant like a openstack-commons18:44
dolphmjaypipes: (you can start it, but it's pointless if you can't login as admin)18:44
zykes-project for doing "common" code18:44
YorikSarjaypipes: console_scripts section in entrypoints makes the system generate a scripts that just run a specific function in specific module.18:44
dolphmfrom openstack.common import wsgi18:44
zykes-like exceptions, wsgi helper module, utils.py etc18:44
jaypipeszykes-: I know. openstack-common is being "baked" in the skeleton project...18:44
YorikSarjaypipes: I'm in Russia, Saratov.18:44
zykes-ok18:45
jaypipeszykes-: see the skeleton project.. those things are in openstack/common/*18:45
jaypipesYorikSar: cool. :)18:45
zykes-YorikSar: you work for rackspace no ?18:45
YorikSarzykes-: No, Mirantis is my employer.18:46
zykes-ah18:46
zykes-anyone know if reddwarf is started on ?18:46
zykes-or atlas18:47
pvozykes- reddwarf has been started18:47
pvothough I don't know the status as of today.18:47
zykes-pvo: is it @ github?18:47
pvo zykes- : https://github.com/rackspace/reddwarf18:48
jk0dprince: https://code.launchpad.net/~jk0/nova/lp827373/+merge/7175918:48
zykes-oh18:48
zykes-my company is pretty interested in stuff like that18:48
zykes-maybe i can get some folks on it18:48
pvozykes-: rock on18:49
zykes-we've got more devs then infrastructure folks so :p18:49
*** lorin1 has joined #openstack-dev18:49
zykes-what's monkeyfarm ?18:50
dprincejk0: Thank you sir. I'm cool with what you did. Let me kick the tires on it...18:50
jk0cool18:50
dprinceQuestion about admin passwords in nova.18:51
dprince<for anyone>18:51
dprinceWe appear to save admin password in the instances table and I'd like to know why?18:51
dprinceThat seems like a very bad idea.18:51
*** alekibango has joined #openstack-dev18:52
dprinceI'm feel like a ticket storm is coming here....18:53
blamar_dprince: yeah....not so much of a good idea18:53
blamar_dprince: agreed on the bug report :)18:53
openstackjenkinsProject nova build #1,257: SUCCESS in 3 min 27 sec: https://jenkins.openstack.org/job/nova/1257/19:00
openstackjenkinsTarmac: Use netaddr's subnet features to calculate subnets.19:00
*** cp16net has quit IRC19:01
*** zns has quit IRC19:01
vishyheckj: http://wiki.openstack.org/UnderstandingFlatNetworking very nice.  I might suggest changing it to say FlatDhcpNetworking19:02
vishysince there is a flat mode too19:02
zykes-is there anything going on to use openvswitch =?19:04
*** jsavak has joined #openstack-dev19:04
heckjvishy: Thank you - will do. It needs to be updated with the --multi-host stuff though19:04
*** zns has joined #openstack-dev19:05
*** joesavak has quit IRC19:05
heckjvishy: was trying to describe both Flat and FlatDHCP in the same document (as opposed to VLAN)19:06
*** cp16net has joined #openstack-dev19:07
*** cp16net has quit IRC19:08
*** cp16net has joined #openstack-dev19:08
*** lorin1 has quit IRC19:12
*** lorin1 has joined #openstack-dev19:13
*** bengrue has joined #openstack-dev19:19
*** hisaharu has joined #openstack-dev19:29
zykes-wonder what's gonna happen to the Cortex thingie from citrix19:31
zykes-after they19:31
zykes-they'e got Olympus & cloud.com19:31
*** lorin1 has quit IRC19:34
*** hisaharu has quit IRC19:34
*** hisaharu has joined #openstack-dev19:37
*** mattray has quit IRC19:39
*** lorin1 has joined #openstack-dev19:42
openstackgerritDolph Mathews proposed a change to openstack/keystone: Admin WADL updates  https://review.openstack.org/24919:47
openstackgerritVerification of a change to openstack/keystone failed: Additional changes to fix minor service support stuff and increase test coverage. Also making validate token call available using service admin tokens  https://review.openstack.org/24219:48
openstackgerritDolph Mathews proposed a change to openstack/keystone: Added pylint message count as run_tests.sh -l  https://review.openstack.org/24619:50
openstackgerritA change was merged to openstack/keystone: Added pylint message count as run_tests.sh -l  https://review.openstack.org/24619:53
openstackgerritYogeshwar Srikrishnan proposed a change to openstack/keystone: Additional changes to fix minor service support stuff and increase test coverage. Also making validate token call available using service admin tokens  https://review.openstack.org/24219:56
ttxTeam meeting in one hour in #openstack-meeting19:58
*** bengrue has quit IRC19:58
zykes-ttx: nova ?19:59
*** lorin1 has quit IRC19:59
*** johnpur has joined #openstack-dev19:59
*** ChanServ sets mode: +v johnpur19:59
ttxzykes-: openstack.19:59
*** lorin1 has joined #openstack-dev20:01
*** jakedahn has joined #openstack-dev20:03
ttxlorin1: hey -- do you plan to submit code for GPU/BareMetal/Tilera ? Or not in time for Diablo ?20:03
openstackgerritVerification of a change to openstack/keystone failed: Additional changes to fix minor service support stuff and increase test coverage. Also making validate token call available using service admin tokens  https://review.openstack.org/24220:05
openstackgerritJoe Savak proposed a change to openstack/keystone: Updates to XML and JSON changes for validateToken  https://review.openstack.org/25020:07
openstackgerritA change was merged to openstack/keystone: Updates to XML and JSON changes for validateToken  https://review.openstack.org/25020:11
*** dprince has quit IRC20:11
*** joesavak has joined #openstack-dev20:12
*** zul has quit IRC20:14
*** johnpur has quit IRC20:14
*** jsavak has quit IRC20:15
*** joesavak has quit IRC20:16
*** cp16net has quit IRC20:21
zykes-ttx: tilera ?20:24
ttxzykes-: https://blueprints.launchpad.net/nova/+spec/heterogeneous-tilera-architecture-support20:24
*** lorin1 has quit IRC20:30
openstackjenkinsProject nova build #1,258: SUCCESS in 4 min 20 sec: https://jenkins.openstack.org/job/nova/1258/20:33
openstackjenkins* Tarmac: Fixes issue with exceptions getting eaten in image/s3.py if there is a failure during register.  The variables referenced with locals() were actually out of scope.20:33
openstackjenkins* Tarmac: Allow local_gb size to be 0. libvirt uses local_gb as a secondary drive, but XenServer uses it as the root partition's size. Now we support both.20:33
openstackjenkinsInclude some random PEP8 fixes.20:33
*** Gordonz has joined #openstack-dev20:36
openstackgerritKevin L. Mitchell proposed a change to openstack/keystone: Port of glance-control to keystone.  This will make writing certain keystone integration functional tests a little easier to do.  https://review.openstack.org/25120:37
jk0vishy: https://bugs.launchpad.net/nova/+bug/82763420:42
uvirtbotLaunchpad bug 827634 in nova "local_gb is not used consistently between hypervisors" [Undecided,New]20:42
vishyjk0: thx20:43
jk0np20:44
vishyjk0 if you are booting a whole disk image, how does xs handle it?20:45
jk0it just takes the local_gb value and converts it to gb20:46
jk0something like local_gb * 1024 * 1024 * 102420:47
vishyright but does it try to resize whole disk images?20:47
vishywhat if the filesystem is something funky like ntfs?20:47
jk0oh, I'm not sure of those details offhand20:48
vishyjk0: this seems like a big gap to me.  Ok guess I will dig into the code :)20:48
*** jroault has joined #openstack-dev20:48
*** jroault has quit IRC20:49
jk0I think it basically just creates a vdi and links it to the instance, and then the image picks it up from there20:49
jk0but yeah I'd have to look too20:49
jk0I'm curious how the other hypervisors use it20:49
*** jrouault has joined #openstack-dev20:49
*** jrouault has left #openstack-dev20:51
*** statik has quit IRC20:53
*** statik has joined #openstack-dev20:53
*** statik has joined #openstack-dev20:53
*** joesavak has joined #openstack-dev20:54
*** rnirmal has quit IRC20:54
vishyjk0: we use local_gb as a second drive so it works with any image.20:54
vishywe used to repartition the file system, but that is always dangerous20:55
ttxMeeting starts in 2 minutes in #openstack-meeting, please join20:58
Vekreally?  I thought the leafs would be more flammable </tounge-in-cheek>20:59
*** mfer has quit IRC20:59
*** Tushar85 has joined #openstack-dev21:00
*** Tushar has quit IRC21:00
*** Tushar85 has quit IRC21:00
openstackgerritVerification of a change to openstack/glance failed: Adds SSL configuration params to the client  https://review.openstack.org/19021:02
*** mattray has joined #openstack-dev21:02
*** mattray has quit IRC21:07
*** dolphm_ has joined #openstack-dev21:08
openstackjenkinsProject burrow build #31: SUCCESS in 11 sec: https://jenkins.openstack.org/job/burrow/31/21:16
openstackjenkinsTarmac: Test case cleanup, moved backend base test cases to test.backend (out of memory test).21:16
*** zul has joined #openstack-dev21:18
*** asomya has joined #openstack-dev21:18
*** lts has quit IRC21:20
*** markvoelker has quit IRC21:22
openstackgerritDolph Mathews proposed a change to openstack/keystone: Admin WADL updates  https://review.openstack.org/24921:24
*** jsavak has joined #openstack-dev21:32
*** joesavak has quit IRC21:32
openstackgerritZiad Sawalha proposed a change to openstack/keystone: More contract changes  https://review.openstack.org/25221:33
vishydprince, tr3buchet: can you recheck https://code.launchpad.net/~danwent/nova/linuxnet_vifplug/+merge/70254 to make sure concrns have been addressed?21:36
openstackgerritA change was merged to openstack/keystone: Port of glance-control to keystone.  This will make writing certain keystone integration functional tests a little easier to do.  https://review.openstack.org/25121:39
openstackgerritA change was merged to openstack/keystone: Admin WADL updates  https://review.openstack.org/24921:42
tr3buchetvishy: sure21:43
*** YorikSar has quit IRC21:46
*** zul has quit IRC21:52
*** chipc has quit IRC21:55
mwhookerugh. anyone else have problems with circular dependencies will all this come in __init__.py?21:57
mwhookers/come/code/21:57
*** vladimir3p has joined #openstack-dev22:00
openstackgerritJames E. Blair proposed a change to openstack/openstack-ci: Add blueprint update script.  https://review.openstack.org/25322:04
openstackgerritJames E. Blair proposed a change to openstack/openstack-ci-puppet: Put blueprint updates into production.  https://review.openstack.org/25422:07
vishymwhooker: yes, that is why i try to never put code in __init__22:08
vishymwhooker: where are you hitting it?22:08
mwhookernova/api/ec2/__init__.py22:09
mwhookerthinking of just breaking though out into another file22:09
mwhookerhttp://pastebin.com/WrMxTvXA22:10
vishymwhooker: yes, seems fine.  I had to put ec2utils into a separate file because of circular deps22:11
*** bsza has quit IRC22:12
openstackgerritA change was merged to openstack/openstack-ci: Add blueprint update script.  https://review.openstack.org/25322:12
mwhookeralright22:12
vishymwhooker: a lot of those are tesing fixtures and/or docstrings and licenses as opposed to actual code22:12
mwhookerwhile I've got you on the horn, can you help me figure out how "image_location", as is used in the ec2 api, should be validated?22:13
vishyhmm22:13
mwhookersure, but the couple near the bottom22:13
mwhookernot sure if it's a url path, a unix path, or any os path22:13
vishyit is supposed to be a bucket/manifest.xml22:14
vishyit is the s3 path to an xml file22:14
mwhookerok, so url22:14
mwhookerthough I notice os.path.split is used22:14
mwhookerso who knows how that will work where the os path separator isn't '/'22:14
mwhookerjoin, rather. not split22:15
openstackgerritA change was merged to openstack/openstack-ci-puppet: Put blueprint updates into production.  https://review.openstack.org/25422:15
vishyjaypipes, notmyname: any ideas on jason's question in #openstack ?22:18
vishybug 82766022:21
uvirtbotLaunchpad bug 827660 in glance "Cannot store files greater than 5GB in Swift" [Critical,Confirmed] https://launchpad.net/bugs/82766022:21
*** amccabe has quit IRC22:22
*** zul has joined #openstack-dev22:25
*** jsavak has quit IRC22:28
*** bengrue has joined #openstack-dev22:33
*** anotherjesse has quit IRC22:42
*** anotherjesse has joined #openstack-dev22:42
*** asomya has quit IRC22:51
*** yogirackspace has left #openstack-dev22:51
jaypipesvishy: yeah, working on it...22:52
vishyjaypipes: coolness.  As i mentioned on the bug, we should have a content-length header, no?22:52
*** martine has quit IRC22:55
jaypipesvishy: not necessarily. we do chunked-transfer encoding to avoid having to read the entire request into memory...22:55
jaypipesvishy: but that said, if the client does know the size of the image, we *can* set the content-length header and read off of that...22:56
vishyjaypipes: hmm.  Yeah, seems like it is reasoable to require the client to send in the right content-length22:56
jaypipesvishy: sure, as long as it's cheap and doesn't interfere with the chunked-transfer...22:57
*** zns has quit IRC22:57
*** zns has joined #openstack-dev22:57
openstackgerritMonty Taylor proposed a change to openstack/openstack-ci: Added a version of the rfc script from gluster.  https://review.openstack.org/23523:00
*** zns has quit IRC23:01
*** yogirackspace has joined #openstack-dev23:05
openstackjenkinsProject nova build #1,259: SUCCESS in 3 min 28 sec: https://jenkins.openstack.org/job/nova/1259/23:06
openstackjenkinsTarmac: I'm taking Thierry at his word that I should merge early and merge often :)23:06
openstackjenkinsIn Diablo-3 we introduced "vif-plugging" to the hypervisor "virt" layer, allowing flexibility in how vNICs are attached to the network switch. This allowed non-linux bridge switch technologies (e.g., Open vSwitch, 802.1qbh) to be used with nova.23:06
openstackjenkinsThis blueprint adds a similar capability to linux_net.py, allowing the L3/DHCP capabilities to be "plugged" into Quantum networks.  Like in the virt layer, we created a vif driver that represents the behavior of Nova prior to the change (LinuxBridgeInterfaceDriver) and make it the default.  We also add a new driver for Open Vswitch that can be enabled using a flag (LinuxOVSInterfaceDriver).  The code is designed to su23:06
openstackjenkinsMost of the interesting code is at the bottom of linux_net.py, where the drivers are defined.  I had to pull some common code related to setting IPs on devices out of ensure_bridge() so it could be used by either approach.  The driver's plug() method is invoked by the VlanManager's _setup_network() method.  Currently unplug() is unused, which seems to be inline with how the existing nova code works.23:06
openstackjenkinsIn many places in linux_net.py, I had to tweak functions to accept the name of the linux device to configure, rather than just assuming it was the 'bridge' field in the network object, since with OVS it could be any linux device.  The code I am least sure about are the changes to bin/nova-dhcpbridge.  I changed to this key off of the network ID, rather than the bridge name.23:06
openstackjenkinsI've tested this with the linux bridge and with the OVS vif-plugging driver.  I was able to confirm that L3 forwarding and DHCP were operating correctly.23:06
*** anotherjesse_ has joined #openstack-dev23:08
s1rpjaypipes: just added a note to the large-file glance tix23:08
s1rpjaypipes: i think we can do the same 5GB segmentation we do w/ the ibacks now23:09
s1rpjaypipes: the difference, is in the Ibacks we maintain the manifest ourselves; with swift's large-file support, we can use their manifest for keeping track of each chunk23:09
*** zns has joined #openstack-dev23:10
jaypipess1rp: sure. we still need to have the client calculate the filesize before transferring though. Not a huge deal, but that's what needs to happen...23:11
s1rpjaypipes: we do? even if we're using chunked-transfer-encoding?23:11
*** anotherjesse has quit IRC23:12
*** anotherjesse_ is now known as anotherjesse23:12
jaypipess1rp: if we're going to signal to Swift to do large-object transfer, we need the size up front...23:13
*** troytoman is now known as troytoman-away23:13
jaypipess1rp: at some point, you need to know the size of the image file... and there's no "add this chunk" method in Swift, so we can't calculate the number of bytes sent to Swift via some iterator.23:15
s1rpjaypipes: the add this chunk is the PUT request to the container, if i'm reading this correctly... then once you've PUT'd all the chunks, then you register the chunks by uploading the manifest23:16
jaypipess1rp: furthermore, calling body_file.seek() on the webob.Request leads webob to copy the entire request body into a temporary file on disk...23:16
s1rpjaypipes: as long as CTE is upported by the PUT, i think you can do this w/o knowing size upfront23:16
s1rpjaypipes: we shouldn't be seeking23:17
s1rpjaypipes: we have no reason to go backwards in the stream23:17
s1rpjaypipes: we 1) create container 2) cte upload each chunk 3) write manifest23:17
jaypipess1rp: so you would loop on: while chunk  = body_file.read(chunksize), swift_client.put_object(chunk)?23:19
s1rpjaypipes: yeah, basically, though you have to add extra code to know when to terminate the request to start the next chunk23:20
s1rpjaypipes: did this on the iback, don't see a reason why this wouldn't work in glance as well23:20
jaypipess1rp: ok, lemme stew on it...23:20
s1rpjaypipes: cools23:21
*** rods has quit IRC23:22
jaypipess1rp: or we could just set up a GlusterFS cluster and use the filesystem backend in Glance ;)23:23
*** yogirackspace has left #openstack-dev23:23
s1rpjaypipes: hehe :)23:24
*** zns has quit IRC23:27
*** ohnoimdead has quit IRC23:31
*** anotherjesse has quit IRC23:41
*** anotherjesse_ has joined #openstack-dev23:41
*** anotherjesse_ is now known as anotherjesse23:41
vishymtaylor, Daviey: ping23:47
mtaylorvishy: whazzyp?23:53
*** dragondm has quit IRC23:53
vishymtaylor: so lets suppose there is a really cool piece of software that has a couple of extra things in contrib that compile to useful binaries that i want to use23:53
mtaylorvishy: ok. good supposition23:53
vishymtaylor: and this piece of software doesn't include those binaries in the ubuntu packaging23:54
mtaylorvishy: fail23:54
vishywhat is my best strategy?23:54
vishyspecifically dnsmasq has a cool binary called dhcp_release23:54
mtaylorvishy: well - we could make packages of that for our repos that have those binaries included - and then send a patch upstream23:54
mtaylorvishy: lemme look real quick23:54
vishywhich would be quite useful for us23:54
vishyit is in contrib/wrt/23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!