Monday, 2011-08-08

*** stiekes_ has quit IRC00:42
*** stiekes_ has joined #openstack-dev00:42
*** medberry is now known as med_out01:02
openstackjenkinsProject nova build #1,214: SUCCESS in 3 min 14 sec: http://jenkins.openstack.org/job/nova/1214/01:59
openstackjenkinsTarmac: Adds ability to disable snapshots in the Openstack API.01:59
*** tyoc213 has joined #openstack-dev02:25
*** stiekes has joined #openstack-dev02:58
*** stiekes_ has quit IRC02:58
*** stiekes_ has joined #openstack-dev03:19
*** stiekes has quit IRC03:21
*** stiekes has joined #openstack-dev03:23
*** stiekes_ has quit IRC03:24
*** nci has quit IRC03:32
*** nci has joined #openstack-dev03:33
*** tyoc213 has quit IRC04:03
*** martine has quit IRC04:17
*** westmaas has quit IRC04:24
*** westmaas has joined #openstack-dev04:24
*** winston-d has quit IRC05:55
*** jakedahn has joined #openstack-dev05:59
*** nickon has joined #openstack-dev06:35
*** zigo has joined #openstack-dev07:28
*** zigo is now known as zigo-_-07:34
*** tudamp has joined #openstack-dev07:43
*** zul has joined #openstack-dev07:54
*** TimR-L has joined #openstack-dev08:13
*** alekibango has joined #openstack-dev08:44
*** darraghb has joined #openstack-dev09:23
openstackgerritYuriy Taraday proposed a change to openstack/keystone: Add proper simple_bind_s to fakeldap, removed all imports from ldap.  https://review.openstack.org/15909:27
openstackgerritYuriy Taraday proposed a change to openstack/keystone: Add LDAP schema.  https://review.openstack.org/16309:27
*** phil121 has joined #openstack-dev10:43
*** bourke has quit IRC11:00
*** markvoelker has joined #openstack-dev11:04
*** martine has joined #openstack-dev11:50
*** martine has quit IRC11:59
*** mfer has joined #openstack-dev12:03
*** mnour has joined #openstack-dev12:04
*** lorin1 has joined #openstack-dev12:14
*** lorin1 has quit IRC12:23
*** lorin1 has joined #openstack-dev12:29
*** dprince has joined #openstack-dev12:43
*** lts has joined #openstack-dev12:54
ltsHi, Gaitan, Ndk:  I'm back.  Bad power hits here yesterday.  I have one unrecoverable lv on my z400.  Moving mail over to my notebook so I can stay in touch while I work on the z400.12:55
ltsFortunately, the replacement drive arrived just before vacation, so I can install that and recover what I can from the drive and restore the rest from backups or recreate.  I've lost a few VM images, but nothing important.12:56
*** zigo-_- has quit IRC13:04
*** zigo has joined #openstack-dev13:04
*** bcwaldon has joined #openstack-dev13:20
*** blamar has quit IRC13:28
*** blamar_ has joined #openstack-dev13:29
*** bcwaldon has quit IRC13:30
*** bcwaldon has joined #openstack-dev13:30
*** bsza has joined #openstack-dev13:34
*** kbringard has joined #openstack-dev13:34
*** bsza has quit IRC13:37
*** bsza has joined #openstack-dev13:37
*** amccabe has joined #openstack-dev13:37
*** vladimir3p has joined #openstack-dev13:41
*** jshepher has joined #openstack-dev13:48
*** zigo is now known as zigo-_-13:50
*** rnirmal has joined #openstack-dev14:02
*** mattray has joined #openstack-dev14:14
*** dolphm has joined #openstack-dev14:23
*** zigo-_- has quit IRC14:25
*** Binbin has joined #openstack-dev14:26
*** chomping has joined #openstack-dev14:30
*** zigo has joined #openstack-dev14:30
*** jkoelker has joined #openstack-dev14:33
*** dolphm has quit IRC14:33
*** nickon has quit IRC14:35
tudampsandywalsh: hi, you have a minute?14:36
sandywalshtudamp, yup14:37
tudampok, you think this scenario: compute configured with raw images, so the instances take a few minutes to be up.14:38
tudamp1) launch an instance;14:38
tudamp2) scheduler consumes resources on host_list14:38
tudamp3) before the instance is up compute update scheduler14:38
*** dolphm has joined #openstack-dev14:38
tudamp4) resources consumed are overwritten14:38
tudampif you launch an instance now I will use the resources occupied by the instance that is going to get up14:40
*** nmistry has joined #openstack-dev14:40
sandywalshtudamp, correct, there will be a lag between reported resources and provisioning in-progress14:40
*** zigo has quit IRC14:41
tudampcould be avoided by updating the scheduler every time an instance is migrated, turned off, etc.14:41
sandywalshtudamp, it's really an issue when provisioning 1-off instances (server X, then server Y, then server Z, etc)14:41
sandywalshtudamp, perhaps less of an issue if you say "provision N servers" ... since the CostScheduler "consumes" resources when allocating hosts14:42
openstackgerritYuriy Taraday proposed a change to openstack/keystone: Add wrapper for real LDAP connection with logging and type converting.  https://review.openstack.org/16214:42
openstackgerritYuriy Taraday proposed a change to openstack/keystone: Fix console and debug logging.  https://review.openstack.org/16114:42
openstackgerritYuriy Taraday proposed a change to openstack/keystone: Moved password check logic to backend  https://review.openstack.org/16014:42
openstackgerritVerification of a change to openstack/keystone failed: Fix console and debug logging.  https://review.openstack.org/16114:42
*** cp16net has joined #openstack-dev14:42
sandywalshtudamp, yes, we could be smarter about when we send our updates14:42
sandywalshtudamp, but ... the schedulers may go up and down ... the periodic updates are to inform the new ones of the state of the world14:43
*** med_out is now known as medberry14:44
tudampsandywalsh: mm.. therefore, also in this case, the scheduler is not completely reliable14:47
openstackjenkinsProject swift build #304: SUCCESS in 34 sec: http://jenkins.openstack.org/job/swift/304/14:47
openstackjenkinsTarmac: PEP8 Fix14:47
*** nmistry has quit IRC14:59
sandywalshtudamp, well, it's as reliable as the data it has.15:00
sandywalshtudamp, we have to assume N scheduler and M compute nodes15:01
tudampsandywalsh, I have not seen LeastCostScheduler :p15:06
tudampsandywalsh, I thought that the  scheduler update was to keep synchronized nodes / zones / scheduler15:07
tudampsandywalsh, and not for inform the new ones of the state of the world15:08
sandywalshtudamp, there are two updates that occur. 1. the ZoneManager in each Scheduler pings the child zones for status updates and 2. The services within a Zone fanout update the schedulers with the nodes state15:08
sandywalshtudamp, But, I'm still not really clear what your question is?15:09
*** nickon has joined #openstack-dev15:10
tudampsandywalsh: I just want that an instance go running only on a host that can accommodate :)15:12
tudampsandywalsh: easy to say, less easy to do :)15:13
sandywalshtudamp, the LeastCostScheduler is what you want. We're cleaning it up now for our alpha release.15:15
sandywalshtudamp, there was some confusion on filter_hosts() and weigh_hosts(), which is also getting straightened up.15:15
tudampsandywalsh: ok, then i look for updates.15:18
tudampsandywalsh: do not read criticism in my statements, I'm just trying to figure out how it works and if I can be useful for something.15:18
sandywalshtudamp, absolutely ... we just have the framework in place, now we need to turn the knobs to get it working properly. Any testing/changes you care to make would be most welcome.15:19
*** martine has joined #openstack-dev15:23
*** Binbin has quit IRC15:23
tudampsandywalsh: great, I thought to use your framework15:24
tudampsandywalsh: to create a time-scheduler, with advanced resource reservations. But this will not happen before October15:25
tudampsandywalsh: Before LeastCost must work, is what we need now. We will be happy to test it.15:25
sandywalshtudamp, sounds great ... let me know how I can help. I don't think we're far off from getting some basic (real) scheduling in place.15:26
tudampsandywalsh: today I tried the zoneawarescheduler: without the bad data coming from the nodes seems to do its job ;)15:31
*** rnorwood has joined #openstack-dev15:32
sandywalshzone aware scheduler by itself won't do much ... it's a base class for real schedulers. HostFilterScheduler is perhaps the most basic scheduler to derive from it15:33
*** ameade has joined #openstack-dev15:38
*** mnour has quit IRC15:48
*** dprince has quit IRC15:52
*** mdomsch has quit IRC15:54
openstackgerritDolph Mathews proposed a change to openstack/keystone: Add proper simple_bind_s to fakeldap, removed all imports from ldap.  https://review.openstack.org/15915:54
jk0unit tests in trunk failing for anyone?15:56
jk0ah, I see why15:57
dolphmwhich project?15:59
jk0nova16:01
jk0but I figured it out16:01
*** yogirackspace has joined #openstack-dev16:03
sandywalshttx soren mtaylor ... need novaclient update so this can go through https://code.launchpad.net/~sandy-walsh/nova/v1_1_novaclient/+merge/7047416:09
*** Transformer has joined #openstack-dev16:13
*** tudamp has left #openstack-dev16:14
openstackgerritDolph Mathews proposed a change to openstack/keystone: Add proper simple_bind_s to fakeldap, removed all imports from ldap.  https://review.openstack.org/15916:16
*** dragondm has joined #openstack-dev16:16
*** Transformer has left #openstack-dev16:17
*** erik-s has quit IRC16:18
*** erik-s has joined #openstack-dev16:18
openstackgerritA change was merged to openstack/keystone: Add proper simple_bind_s to fakeldap, removed all imports from ldap.  https://review.openstack.org/15916:18
mtaylorsandywalsh: ok. just got the slave updated16:25
sandywalshthanks mtaylor, we'll reapprove16:26
*** zul has quit IRC16:32
*** jakedahn has quit IRC16:34
vladimir3pjaypipes: ping16:35
jaypipesvladimir3p: pong16:35
vladimir3pjaypipes: trying to merge latest nova code and getting a pretty strange error that seems to be glance related16:36
vladimir3pjaypipes: http://paste.openstack.org/show/2119/16:36
sandywalshdevcamcar, jk0 seems I need a reapprove on https://code.launchpad.net/~sandy-walsh/nova/v1_1_novaclient/+merge/7076416:36
jaypipesvladimir3p: looking...16:36
jaypipesvladimir3p: have you upgrade glance?16:37
jk0sandywalsh: looks like you might need to submit a new MP, it's not letting me reapprove16:37
vladimir3pjaypipes: it should be on diablo-3 release16:37
jaypipesvladimir3p: what does glance --version say?16:37
vladimir3pjaypipes: glance 2011.3-dev16:38
jaypipesvladimir3p: hmm...16:38
jaypipesVek: around?16:38
jaypipesvladimir3p: one sec...16:38
sandywalshjk0, just re-proposed16:39
jk0sandywalsh: still wont let me. looks like you need to delete this one and make a new MP16:39
sandywalshphooey16:39
*** zaitcev has joined #openstack-dev16:40
sandywalshjk0, https://code.launchpad.net/~sandy-walsh/nova/v1_1_novaclient/+merge/7076516:40
*** jdurgin has joined #openstack-dev16:40
daboStill looking for some nova core reviews for https://code.launchpad.net/~ed-leafe/nova/powerstate/+merge/7023016:41
s1rpvladimir3p: i ran into that error too, i think python setup.py install on glance fixed it for me16:41
vladimir3ps1rp: thanks will try it right now16:42
s1rpvladimir3p: and make sure you bounce nova after you do that16:42
*** medberry is now known as med_out16:46
*** dprince has joined #openstack-dev16:59
openstackgerritJay Pipes proposed a change to openstack/glance: Add @skip_if_disabled decorator to test.utils and integrate it into the base functional API test case. The S3 functional test case now uses test_api.TestApi as its base class and the setUp() method sets the disabled and disabled_message attributes that th  https://review.openstack.org/17716:59
openstackgerritVerification of a change to openstack/keystone failed: Fix console and debug logging.  https://review.openstack.org/16117:01
dprincemtaylor: you around?17:10
mtaylordprince: morning!17:10
dprincemtaylor: Hey. So is there a good way I get get a list of branches up for review from gerrit?17:11
dprincemtaylor: Preferably on the command line.17:11
mtaylordprince: perhaps https://review.openstack.org/Documentation/cmd-query.html17:12
dprincemtaylor: In a human readable branch form? I see gerrit is using Refspec's in the webUI.17:12
mtaylorhrm... jeblair ^^^ any thoughts?17:13
dprincemtaylor: Ideally I'd like to be able to fetch someones code w/ the same they used locally for the branch.17:13
mtaylordprince: I don't know off the top of my head - I'm _sure_ there is a way17:13
mtaylordprince: ah, well, now that I know can be done17:13
mtaylordprince: (there's a fetch thing in the web ui - just a matter of finding the right thing in the command line tool) ... lemme poke17:14
dprincemtaylor: Okay. Well I'm looking for that. I'd like to make it easy for humans to test branches up for review on Gerrit.17:14
dprincemtaylor: Using the refspecs will work but man oh man that is going to be a lot of detective work to keep up with that.17:15
mtaylordprince: you've seen the fetch/pull lines on the reviews themselves, yeah?17:15
dprincemtaylor: Yes. They all just use the refspecs. For example 'refs/changes/70/170/1'17:16
dprinceI'd like to pull the branch name with something a bit more humanly understandable. Like 'fix_osapi_server_create' or something like that.17:17
mtaylordprince: oh, I do not believe that exists17:17
mtaylordprince: I'd want to verify with jeblair to make sure - but gerrit does magic with refspecs in terms of internal storage17:18
jeblair ssh review gerrit query "status: open" --current-patch-set17:18
dprincejeblair: Thanks. Let me check that out.17:18
jeblairgets a list of the current open patches, including their refspcs17:19
*** bcwaldon has quit IRC17:19
jeblair ssh review gerrit query "status: open" --current-patch-set --format JSON17:19
jeblairdprince: ^ that could be the basis of a script that does what you want17:19
jeblairthat gets you patches, their descriptions, and refspecs17:19
jaypipesJordanRinke: around? want to know what your opinion is on the priority of https://bugs.launchpad.net/glance/+bug/819936?17:21
uvirtbotLaunchpad bug 819936 in glance "New image cache breaks Glance on Windows" [Low,Confirmed]17:21
jaypipesJordanRinke: seems to me that having the image cache be truly a disable-able middleware/extension will fix things for Windows, right?17:22
sandywalshany keystone people in the house?17:25
dprincejeblair: Okay. That command will give me 'subject' descriptions which I can then associate w/ the refspecs. That is better but I'd love it if we could use humanly readable branch names instead.17:25
dolphmsandywalsh: <--17:25
sandywalshdolphm, hey! trying to test novaclient integration with keystone17:25
sandywalshdolphm, getting http://paste.openstack.org/show/2120/17:26
sandywalshdolphm, the service is running17:26
*** jakedahn has joined #openstack-dev17:26
jeblairdprince: i think subject is the closest thing that gerrit provides.  it picks it up from the first line of the commit.17:26
sandywalshdolphm, btw, seems --host / --bind-host don't work17:26
sandywalshdolphm, (it's listening on 0.0.0.0 by default)17:27
dolphmhmm17:27
dolphmsandywalsh: on the --host issue, feel free to open an issue on github.com/rackspace/keystone17:28
sandywalshdolphm, yup, was just going to17:28
*** alekibango has quit IRC17:28
dprincejeblair/mtaylor: Another question. The Anonymous HTTP commands don't work for me.17:29
dolphmsandywalsh: cool, as for the x-server-management-url, i've seen that once before, but was unable to reproduce17:29
dolphmsandywalsh: it popped up in our test suite, and then disappeared17:29
*** darraghb has quit IRC17:29
sandywalshhmm17:29
dolphmsandywalsh: but it happens every time for you?17:30
dprincejeblair/mtaylor: So if I click on the 'Download' --> 'Anonymous HTTP' link and try to run that command in my cloned project it fails.17:30
sandywalshyeah, seems to17:30
sandywalshdolphm, I'll put some debugging in there to see what's happening17:30
dprincejeblair/mtaylor: The SSH commands do appear to work (although they'd require me SSH key to be everywhere I'd need to run them).17:30
jeblairdprince: what error do you get?17:31
*** bcwaldon has joined #openstack-dev17:31
dprincefatal: HTTP request failed17:31
mtaylordprince, jeblair: ssl cert17:32
mtaylor"error: server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none while accessing https://review.openstack.org/p/openstack/glance/info/refs"17:32
mtaylorjeblair: ^^17:32
jeblairmtaylor: thanks17:33
openstackgerritVish Ishaya proposed a change to openstack/keystone: Adds support for authenticating via ec2 signatures  https://review.openstack.org/12717:34
dolphmsandywalsh: i'll forward this to the yogi, who i believe wrote the related code in keystone17:34
sandywalshdolphm, cool, thanks. I'll let you know what I find.17:35
dolphmsandywalsh: or if you want to go ahead and open an issue on this too, i'll make sure it's assigned to him17:36
sandywalshdolphm, will do. (odd, it's getting a 305 so something is answering)17:36
*** bcwaldon has quit IRC17:36
dolphmi *think* the rest of the response is "correct" but it shouldn't include a stack trace - the 305 is indicating that you're trying to call nova without first authenticating via the indicated url (keystone)17:37
*** bcwaldon has joined #openstack-dev17:37
dolphmsandywalsh: the request contains an "x-auth-key" headers, which i'm not familiar with... keystone uses the "x-auth-token" header... but that may be what the specific test is testing for17:39
dolphmsandywalsh: ... or is that not a test? lol17:40
sandywalshdolphm, we have two merge-props in novaclient for keystone integration ... let me see if the other assembles the same request17:40
sandywalshboth are specifying x-auth-key ... lemme check what keystone is looking for (in my version)17:41
*** Tushar has joined #openstack-dev17:41
TusharI am trying to setup zone_aware_scheduler to test launching of server on specific host. But I didn't find the format of blob that I should be passing to the Create Server API. Does anyone knows the format of blob?17:43
sandywalshTushar, you don't need to parse the blob. It gets created by the /zones/select command17:50
*** jakedahn has quit IRC17:50
Tusharsandywalsh: so before calling /zones/boot I should make a call to /zones/select. Is it correct?17:52
sandywalshTushar, you should nova.scheduler.host_filter:HostFilterScheduler as your base zone-aware scheduler (or LeastCostScheduler)17:52
sandywalshTushar, no need ... the framework will do that for you. You can just call zone-boot17:52
sandywalshTushar, 'nova zone-boot servername --image X --flavor Y'17:53
sandywalshTushar, it'll call /zone/select and deal with all the blob stuff under the hood17:53
Tusharsandywalsh: I am specifically interested to launch the vm instance on specific host. Is that possible using zone_ware_scheduler?17:53
kbringardyou'd probably have to create a single compute node zone17:53
kbringardthen send it to that zone17:54
sandywalshTushar, not directly, the purpose of the scheduler is to pick the best host based on the request parameters. If your request was so specific that there was only one host for the job, that might do it.17:54
*** heckj has joined #openstack-dev17:54
sandywalshTushar, but right now we only pick hosts based on InstanceType (memory and disk)17:55
sandywalshTushar, I know there was work going on to allow extra parameters on instance type, but not sure the status of that17:55
Tusharsandywalsh: I am referring to latest trunk code and trying to understand method "_provision_resource_locally"17:57
jk0jenkins backed up?17:57
Tusharkbringard: Thanks. I got it.17:57
sandywalshTushar, that method is called when it has been decided to provision the instance in the current zone (locally). It creates a db entry for the instance and forwards the request to the appropriate compute node.17:58
kbringardin the future you could probably set a zone_capability attribute for each compute node17:58
sandywalshkbringard, agreed17:59
kbringardonce the scheduler can key off of any of those attributes17:59
openstackgerritBrian Waldon proposed a change to openstack/glance: breaking up MAX_ITEM_LIMIT and making the new values configurable  https://review.openstack.org/17018:00
Tusharsandywalsh: I am still not very clear about how to pass blob to the create server API. If I don't pass blob then I think it will raise InvalidBlob error.18:03
jeblairdprince: still here?18:04
jeblairdprince: try the https git fetch command again18:04
*** stiekes has quit IRC18:05
jeblairdprince,mtaylor: the ssl certs we got for jenkins.o.o and review.o.o had _two_ intermediate certs, but apache was only serving one of them.  i fixed that, and it works for me now.18:05
*** stiekes has joined #openstack-dev18:05
sandywalshTushar, you don't need to ... the scheduler will append it when it forwards a boot request to a child zone18:07
sandywalshTushar, if the child zone sees a blob it will attempt to use it, otherwise it will assume it's a new request18:07
sandywalshTushar, the distributed scheduler docs may help http://nova.openstack.org/devref/index.html18:08
Tusharsandywalsh: I will go through this doc to understand more about zone ware scheduler. Thanks.18:09
sandywalshTushar, np18:09
openstackgerritDolph Mathews proposed a change to openstack/keystone: Moved password check logic to backend  https://review.openstack.org/16018:13
openstackgerritVerification of a change to openstack/keystone failed: Fix console and debug logging.  https://review.openstack.org/16118:14
pvovishy: this is the additional state blueprint I said I would get going. Looking for your thoughts or feedback: https://blueprints.launchpad.net/nova/+spec/nova-additional-state'18:16
jeblairdolphm: ping18:16
dolphmjeblair: whats up18:16
jeblairi managed to get a little feedback from jenkins into gerrit about whether a change can be merged18:16
openstackgerritVerification of a change to openstack/keystone failed: Moved password check logic to backend  https://review.openstack.org/16018:17
jeblairhttps://review.openstack.org/#change,16118:17
dolphmjeblair: the fourth line? i just noticed that a few minutes ago :)18:17
jeblairyep18:17
dolphmjeblair: much appreciated!18:17
jeblairso that outputs some explanatory text, so if you see "This change was unable to be automatically merged with the current state of the repository. Please rebase your change and upload a new patchset." you know you can ignore the other failures18:18
jeblairthe string formatting is.. unfortunate...  but that is, strangely, not a quick fix18:18
jeblairmtaylor: ^18:18
mtaylorjeblair: wow. love how the \n isn't being processed18:19
jeblairyep. i changed it back, so it's not in there anymore18:19
dprincejeblair: Thank you sir. I added sslverify = false to my gitconfig as well and that worked as well.18:20
jeblairgerrit helpfully styles any messages that have trailing whitespace as preformatted.  and jenkins puts in training whitespace i can't get rid of18:20
jeblairdprince: you're welcome!18:21
dprincejeblair: Is there any way we could make the refspec show up instead of the 'branch' on the Gerrit 'status:open' report?18:21
dprincejeblair: The branch name currently always says 'master' which is kind of useless.18:21
*** jhtran has joined #openstack-dev18:22
vladimir3ps1rp: re previous glance issue: installed manually the latest version from git and it helped. thanks18:22
jeblairdprince: i don't think it always will be.  i expect we will have multiple branches for things like milestones, etc18:22
dolphmmtaylor: jeblair: can we get jay's updates to test execution in today? https://review.openstack.org/#change,15418:22
mtaylordolphm: I'd love to ... what do you need from me on my end?18:23
mtaylordolphm: do you need me to change the jenkins job before you +2 it?18:23
*** alekibango has joined #openstack-dev18:24
dolphmmtaylor: yeah, look at the last failure.. https://jenkins.openstack.org/job/keystone/101/console18:24
dprincejeblair: Okay. Sure. Is there any way to make the report display 'refspec' as well?18:24
dolphmmtaylor: instead of 'cd keystone/test' 'python run_tests.py', we can just do './run_tests.sh' from the root of the project18:24
mtaylordolphm: cool. I can do that right now if you like?18:25
dolphmmtaylor: and 'export PYTHONPATH=/home/jenkins/workspace/keystone' definitely isn't needed anymore either... although i'm not sure if you need to run 'python setup.py develop' instead18:25
dolphmmtaylor: yeah, that'd be awesome18:25
mtaylordolphm: ok. i've made the change - you wanna +2 that bad boy and see how it goes?18:26
dolphmmtaylor: doing it now..18:26
dolphmmtaylor: hopefully there's no new merge conflicts..18:26
mtaylor:)18:26
mtaylordolphm: that's the fun with these large-scale patches18:26
jeblairdprince: i don't think that's tunable, so it would probably be a patch18:26
mtaylorseemed to merge properly18:27
openstackgerritVerification of a change to openstack/keystone failed: Gets Keystone a bit more inline with the way that other OpenStack projects run tests. Basically, adds the standard run_tests.sh script, modifies the run_tests.py script to do the following:  https://review.openstack.org/15418:27
dolphmmtaylor: this patch also increases test coverage18:27
dolphmStart directory is not importable: 'keystone.test'18:27
dolphmtry adding 'python setup.py develop' before running tests18:27
dolphmto fake-install keystone18:27
mtaylordolphm: so - I'd like to avoid that if we can ... (we don't do that on other projects) - lemme poke for a sec18:28
*** martine_ has joined #openstack-dev18:28
*** martine has quit IRC18:30
dolphmwell, between path munging (what other projects do, including parts of keystone), manually setting PYTHONPATH, and running a standard setup.py command that's already built into the project's dependencies (setuptools), i'm vastly in favor of setup.py (either "... develop" or "... install")18:30
dprincejeblair/mtaylor: So long term question. If GitHub adds some features for better review tools will we ditch this Gerrit and just move to GitHub?18:31
dolphmdprince: is github planning code review and continuous integration features?18:33
dprincedolphm: Well they have code review right? I'm not sure what you mean about continuous integration features. Does Gerrit have those? I thought that was more of a Jenkins thing.18:35
dprincedolphm: Its really just an approval mechanism right?18:35
mtaylordprince: I can't really predict the future ... I would prefer to not do more project process changes in the future unless there is a very compelling reason18:35
openstackgerritA change was merged to openstack/glance: breaking up MAX_ITEM_LIMIT and making the new values configurable  https://review.openstack.org/17018:36
mtaylordprince: also, as we move forward, we're building more and more automation on top of this system, so there would be another re-tooling/dev cost that would have to be considered18:37
dprincemtaylor: Sure. We all would. To many people GitHub IS Git however. And I feel like we've side stepped into this Gerrit thing. For good reason (due to the lacking features). But if GitHub adds feature will we be open to using them?18:37
dprincemtaylor: Sorry. I'm just realizing how much detective work I'm going to have to do on top of Gerrit to be able to CI branches with something like Smokestack.18:38
mtaylordprince: well, you could use the event stream like the jenkins plugin uses18:38
*** jshepher has quit IRC18:38
*** jshepher has joined #openstack-dev18:39
mtaylordprince: but I'd think that we'd need a compelling set of project requirement changes before undertaking the cost of moving - there are a bunch of more automation tooling things that are in the pipeline to be built on top of this system18:39
dprincemtaylor: Which job should I look at to check that out.18:39
mtaylordprince: it's inside the gerrit plugin (there is very deep gerrit/jenkins integration - which is one of the reasons that it's a really nice choice for us)18:40
mtaylordprince: but look at https://review.openstack.org/Documentation/cmd-stream-events.html18:40
mtaylordprince: which is the basis for the jenkins plugiun18:40
mtaylordprince: of course - you could _also_ work with us to get the smokestack features you want in to jenkins ... ;)18:41
jk0mtaylor: is jenkins backed up right now?18:41
*** zykes- has quit IRC18:42
dprincemtaylor: So the gerrit event stream is going to catch things after they have already happened.18:43
*** zykes- has joined #openstack-dev18:43
dprincemtaylor: I'm trying to get a hook into something before we commit it into trunk. Currently it seems like using the 'refspecs' on Gerrit is my only option.18:44
mtaylorjk0: we're doing cloud server snapshots - but doing a real backup strategy is next on our todo list18:44
jk0mtaylor: ah, I mean the merge queue, sorry :)18:44
mtaylordprince: the gerrit event stream actually sends you events about approved merge requests18:45
jk0mtaylor: it doesn't seem to be taking any more Approved MPs18:45
mtaylordprince: so getting things before we commit them from the stream is exactly how jenkins operates18:45
mtaylordprince: you can catch an event for everything that happens in gerrit - we have jenkins triggering on review approval - but you can totally watch for new review being submitted and take actions on that18:46
mtaylordprince: (that's actually how the jenkins plugin works by default- we changed it to work post-approve to prevent against arbitrary malicious people submitting code for review that caused machines to run that code18:47
mtaylorjk0: what do you mean?18:48
dprincemtaylor: Thanks man. I'll check out the Jenkins plugin and see how far I can get.18:48
jk0mtaylor: I have a MP that's been sitting as 'Approved' for an hour, usually jenkins picks them up within a few minutes18:49
mtaylordprince: cool. also, jeblair has been hacking in it and has some deeper knowledge than I do18:49
mtaylorjk0: link?18:49
jk0https://code.launchpad.net/~sandy-walsh/nova/v1_1_novaclient/+merge/7076518:49
*** mwhooker_ has joined #openstack-dev18:51
mtaylorjaypipes: next time you feel like hacking on keystone tests - could you figure out why I'm having to add PYTHONPATH=`pwd` before running the tests?18:51
mtaylorjk0: it's running tests in that branch now - seems like it might be hung running them18:52
mtaylorjk0: https://jenkins.openstack.org/job/nova-tarmac/110881/console18:52
mtaylorjk0: oh wait! we're both just impatient :)18:52
jk0interesting18:52
*** mwhooker_ has quit IRC18:52
mtaylorjk0: ok. there's a thing... one sec18:52
mtaylorjk0: that MP is missing a commit message18:53
jk0sandywalsh: ^18:53
*** mwhooker is now known as mwhooker_18:53
mtaylorI'm just going to add one18:54
jk0doh, I should have caught that18:54
sandywalshdolphm, out of curiosity, why is the token auth v1.0 and username/password v2.0?18:54
mtaylorwell - the system should have caught that18:54
mtaylorjk0: no worries - in the new world order this problem goes away18:54
jk0nice18:54
*** mwhooker has joined #openstack-dev18:55
sandywalshjk0, ? looks like jenkins is stuck?18:55
vladimir3pfolks, auth-related question: taking into accout recent auth changes what might be the best way to get user details & access code using context? Should I just take user_id and based on that access users DB table?18:55
jk0sandywalsh: there was no commit message in the MP18:55
dolphmsandywalsh: 1.0/1.1 reflect rackspace's auth... 2.0 will be entirely configurable18:55
mtaylordolphm: export PYTHONPATH=`pwd` is still required - but seems to fix the test run ... I'm going to bug jaypipes to fix it18:55
mtaylordolphm: re-running18:55
mtaylorsandywalsh: no worries - I believe I've fixed it18:56
dolphmmtaylor: he also wrote some code to build a virtualenv that may solve the issue as well, but i didn't play with it18:56
mtaylordolphm: well, we _also_ don't run the jenkins tests in virtualenv :)18:56
jaypipesdolphm: well, we don't run run_tests.sh -V on the jenkins boxess..18:56
sandywalshjk0, ah, that causes it to stick?18:56
mtaylordolphm: I'm basically the hardass who exists to make your life difficult :)18:56
sandywalshdolphm, gotcha18:56
openstackgerritVerification of a change to openstack/keystone failed: Gets Keystone a bit more inline with the way that other OpenStack projects run tests. Basically, adds the standard run_tests.sh script, modifies the run_tests.py script to do the following:  https://review.openstack.org/15418:56
dolphmmtaylor: i wouldn't expect you to use a virtualenv18:56
mtaylorDAMMITY18:57
dolphmlol18:57
dolphmhmm - those are test failures that weren't there before18:57
mtaylorjaypipes, dolphm: these seem to be different (and real) errors: https://jenkins.openstack.org/job/keystone/107/console18:57
mtayloror, when I say real ... actually - ran those commands by hand on that box and they succeeded... hrm18:58
mtaylordolphm, jaypipes: is it possible that a keystone server from a different run is sneaking in here?18:59
dolphmmtaylor: what do you mean?18:59
jaypipesmtaylor: most definitely it is, because the tests all run the keystone server on the same port :(19:00
mtaylorjaypipes: LOVELY19:00
mtaylorjaypipes: should I limit us to one build thread on that box until we fix that?19:00
jaypipesmtaylor: yes, will have to.19:00
dolphmmtaylor: oh. lol19:00
mtaylorjaypipes: there is a jenkins feature where we can have jenkins create and pass in port numbers ... if that winds up being a helpful thing19:01
jaypipesmtaylor: end goal is to do a little magic like we have in glance's functional test suite that starts up the servers on unused ports in an isolated test dir..19:01
mtaylorjaypipes: oh - that's even better19:01
dolphmjaypipes: that should be possible soon19:01
mtaylorI limited us to one executor and re-triggered19:02
dolphmgrab an unused port from python, throw it into your conf template pattern, and off it goes19:02
openstackgerritVerification of a change to openstack/keystone failed: Gets Keystone a bit more inline with the way that other OpenStack projects run tests. Basically, adds the standard run_tests.sh script, modifies the run_tests.py script to do the following:  https://review.openstack.org/15419:03
dolphmldap tests again19:03
dolphmthe first test run succeeds - against a sql-only backend19:04
*** dprince has quit IRC19:04
*** amccabe has quit IRC19:07
*** deshantm_laptop has joined #openstack-dev19:08
openstackjenkinsProject nova build #1,215: SUCCESS in 3 min 19 sec: http://jenkins.openstack.org/job/nova/1215/19:10
openstackjenkinsTarmac: Adds OS API 1.1 support19:10
dolphmmtaylor: i blame one of the last two commits (both ldap related) + expanded test coverage in this commit for the test failures... is it possible to totally undo the last two commits in openstack/keystone and then apply this one first?19:14
dolphm(just verified this theory by running tests before/after rebasing)19:16
*** jshepher has quit IRC19:17
jeblairdolphm: maybe the right thing to do would be to git revert the previous commits? (you can revert the commit, which will make a new commit, which you can push to gerrit for review.)19:18
dolphmjeblair: can we then reopen the two closed reviews?19:18
jeblairdolphm: i think you'd have to propose new changes; you can re-open abandoned reviews, but i don't think you can do that for merged ones.19:19
jeblairdolphm: wait19:20
dolphmjeblair: ?19:20
jeblair"Add a "revert change"-button to a submitted patchset19:20
jeblairClicking "Revert Change" creates a new change with the inverse of the submitted patch set ready for review and submission. This makes it easy to undo a build-breaking change right from the web UI.19:20
jeblairdolphm: so, um, i think that's a button that does exactly what you originally wanted. :)19:21
openstackjenkinsProject nova build #1,216: SUCCESS in 3 min 19 sec: http://jenkins.openstack.org/job/nova/1216/19:21
openstackjenkinsTarmac: This branch adds additional capability to the hosts API extension. The new options allow an admin to reboot or shutdown a host. I also added code to hide this extension if the --allow-admin-api is False, as regular users should have no access to host API calls.19:21
jeblairdolphm: i hadn't noticed it before. :)19:21
dolphmah, i see that -- let me figure out specifically which revision it was19:21
dolphmoh - only one has made it in lol19:21
*** mwhooker has quit IRC19:21
*** amccabe has joined #openstack-dev19:22
mwhooker_hi all. I'm having a problem with mox -- I'm wondering if anyone's experienced the same or knows how to fix it. I have a test that's failing on mox.VerifyAll(), but the output of the failure is what seems to be a normal message output 1 character per line, so I have about 30k lines that look like "32097.  i" (each line is numbered, for some reason)19:23
mtaylorjeblair: wow. that's a kick-ass feature!19:23
*** mwhooker_ is now known as mwhooker19:23
openstackgerritDolph Mathews proposed a change to openstack/keystone: Revert "Add proper simple_bind_s to fakeldap, removed all imports from ldap."  https://review.openstack.org/17819:23
mtaylordolphm: I may need to re-change the jenkins job to get the inserve patch through19:23
mtaylorone sec19:23
mtaylordolphm: ok. done19:24
dolphmmtaylor: can i approve the revert?19:24
dolphm( https://review.openstack.org/#change,178 )19:24
mtaylordolphm: yes!19:25
*** lorin1 has left #openstack-dev19:26
dolphmit's about to run the tests that were just failing..19:26
openstackgerritA change was merged to openstack/keystone: Revert "Add proper simple_bind_s to fakeldap, removed all imports from ldap."  https://review.openstack.org/17819:27
dolphmmtaylor: now, retrigger https://review.openstack.org/#change,154 ?19:28
mtaylordolphm: lemme change the jenkins config back19:28
openstackgerritA change was merged to openstack/keystone: Gets Keystone a bit more inline with the way that other OpenStack projects run tests. Basically, adds the standard run_tests.sh script, modifies the run_tests.py script to do the following:  https://review.openstack.org/15419:31
dolphmyay19:31
mtaylordolphm: ^^^ yay!19:31
dolphmmtaylor: THANK YOU! :d19:31
dolphm:D * lol19:32
jaypipesdolphm, mtaylor: \o/19:36
openstackgerritMonty Taylor proposed a change to openstack/keystone: Moved run_tests.py to match other projects.  https://review.openstack.org/17919:37
mtaylorjaypipes, dolphm: ^^ that removes the need for the PYTHONPATH setting19:37
jaypipesmtaylor: rock.19:37
ironcamelmtaylor: i have a question about tarmac. i checked out the openstack-ci project, and i see the tarmac.conf19:37
mtaylorironcamel: yes19:38
ironcamelit calls test_nova.sh, which runs bash run_tests.sh -N && python setup.py sdist19:38
ironcamelmy question is what is run_tests.sh running against?19:38
ironcamelwhere is it getting the code from?19:38
ironcameldoes it do a bzr clone lp:blah19:38
mtaylorironcame12: it's running against trunk with the branch in question merged in19:38
mtaylorironcame12: it does bzr branch lp:project && bzr merge lp:~somebody/project/branch19:39
mtaylor(essentially - it doesn't _ACTUALLY_ run those shell commands :) )19:39
mtaylordamn.19:39
mtaylorironcamel: ^^19:39
mtaylor(stupid tab completion)19:39
mtaylordolphm: I lied. doesn't work.19:39
ironcameli am having trouble understanding why tarmac is having problems with https://code.launchpad.net/~rackspace-titan/nova/extensions-osapi1.1-serx/+merge/6974919:39
dolphmmtaylor: shit lol19:40
ironcamelall the tests pass for me locally19:40
mtaylordolphm: I'll keep poking19:40
mtaylorironcame1: looking ... could be deps installed with different versions on the box?19:40
ironcameland for others as well19:40
ironcamelmtaylor: i have an idea why19:40
dolphmmtaylor: can you cancel a build in progress? lol19:40
mtayloroh yeah?19:40
ironcameli have .rng (xml schema files) in the source tree19:40
mtaylordolphm: I can :)19:40
dolphmmtaylor: well, your change is building19:41
mtaylordolphm: it won't BREAK anything- it just doesn't actually fix it all the way19:41
jaypipesironcamel: XML? Well there's your problem! ;P19:41
ironcamelmtaylor: it seems as though tarmac somehow is not copying those files over19:41
mtaylorironcame1: hrm. oh - wait - is it possible that you didn't bzr add them?19:41
mtayloroh - no, I see them there19:42
ironcameljaypipes: hehe. i remember from the conference you were not a fan of xml. neither am i really. just trying to get this branch in.19:42
jaypipes:)19:42
jaypipesironcamel: I know, was just kiddin you19:42
mtaylorironcame1: actually ... /tmp/tmp_11D8R/nova/__init__.py/nova/api/openstack/schemas/v1.1/extension.rng19:43
* jk0 remembers that XML talk19:43
mtaylorironcame1: that seems like a very odd file path19:43
jk0that was my favorite talk of the summit19:43
*** ameade has quit IRC19:43
ironcamelyeah, why is it constructing it like that ... hmmm19:44
ironcamelhttp://paste.openstack.org/show/2123/19:45
ironcamelthat is the code that constructs the path to the xml schema file19:45
ironcamelis there a bug in that?19:45
mtaylorironcame1: so - the essential sequence is something like "TEMPDIR=mktempdir ; bzr branch lp:nova $TEMPDIR; cd $TEMPDIR ; bzr merge lp:~rackspace-titan/nova/extensions-osapi1.1-serx ; ./run_tests.sh"19:46
ironcamelutils.novadir() must be returning /tmp/tmp_11D8R/nova/__init__.py19:46
mtaylorironcame1: it's possible that utils.novadir() is doing something weird19:46
mtayloryeah19:46
mtaylorreturn os.path.abspath(nova.__file__).split('nova/__init__.pyc')[0]19:47
*** ameade has joined #openstack-dev19:47
ironcamelmaybe it should not be splitting on .pyc19:47
ironcamelshould it be os.path.abspath(nova.__file__).split('nova/__init__.py')[0] ?19:48
mtaylorthat works WAY better for me19:48
mtayloralthough I suppose sometimes it might return __init__.py19:48
ironcamelso i'll try changing that function, and pushing my branch again19:49
ironcameldoes that sound ok?19:49
mtaylorironcame1: I think you might break other things- I think the function needs to account for nova.__file__ possibly equalling __init__.py and __init__.pyc19:49
ironcamelok19:50
mtaylorironcamel: it's ugly - but os.path.split(os.path.split(os.path.abspath(nova.__file__))[0])[0] should work for both cases19:50
ironcamelso you think the first time .__file__ is the .py file, and each subsequent time it is the .pyc file?19:51
mtaylorironcamel: I suck ... os.path.abspath(nova.__file__).split(nova.__file__)[0]19:52
mtaylorironcamel: would be the, you know, consistent thing :)19:52
ironcamelmuch better :)19:54
openstackjenkinsProject nova build #1,217: SUCCESS in 3 min 23 sec: http://jenkins.openstack.org/job/nova/1217/19:55
openstackjenkinsTarmac: nova.exception.wrap_exception will re-raise some exceptions, but in the process of possibly notifying that an exception has occurred, it may clobber the current exception information. nova.utils.to_primitive in particular (used by the notifier code) will catch and handle an exception clobbering the current exception being handled in wrap_exception. Eventually when using the bare 'raise', it will attempt to rai19:55
openstackjenkinsThe patch saves the exception at the beginning of wrap_exception and then re-raises the original exception avoiding the possibility of a clobbered exception.19:55
kpepplejaypipes: time for a quick question ?20:01
jaypipeskpepple: of course20:01
kpepplein cactus version of glance, when configured to persist images to Swift, does it store them all under a single user/id ?20:02
jaypipeskpepple: yes. the swift_store_user/key config options.20:04
jaypipeskpepple: that said, you *can* however specify an alternate swift location via the X-Image-Meta-Location property, in which case Glance is instructed to fetch the image from an alternate Swift location...20:05
kpepplejaypipes: hhhmmm ... ok, thought so. second question: are nova and glance versions pegged -- can i use glance diablo with nova cactus ?20:06
jaypipeskpepple: no, you cannot. =20:07
jaypipeskpepple: you must use cactus for both, since there was a backwards incompatible client and API versioning change in Glance D120:07
jaypipeskpepple: though we are recommending running D3 and later for both Nova and Glance...20:08
kpepplejaypipes: ok, thought it was a long shot with all the changes ...20:08
kpepplejaypipes: the version horse left the barn several months ago ...20:08
jaypipeskpepple: indeed :)20:08
*** dprince has joined #openstack-dev20:11
kpepplejaypipes: thanks. trying to figure out billing on saved images ...20:13
jaypipeskpepple: billing on saved images?20:13
kpepplejaypipes: we bill on user's image storage20:14
jaypipesI see20:15
*** nickon has quit IRC20:17
dprincemtaylor: Any idea how long it takes the PPA to update novaclient?20:18
dprinceI need the latest version bzr70 and its still at bzr65.20:18
dprincehttps://jenkins.openstack.org/job/python-novaclient-ppa/26/label=nova,series=maverick/console20:23
openstackjenkinsProject nova build #1,218: SUCCESS in 3 min 14 sec: http://jenkins.openstack.org/job/nova/1218/20:25
openstackjenkinsTarmac: Set image progress to 100 if the image is active.20:25
dprinceAccording to that we had success uploading to the PPA. But when I go to install them the package isn't there. Additionally the 'Packages' list in the PPA is stale (still back at revision 65).20:25
*** dprince has quit IRC20:26
mtaylordprince: it can take a bit - they have a queue20:26
mtaylorwell poo20:26
openstackjenkinsProject burrow build #26: SUCCESS in 11 sec: http://jenkins.openstack.org/job/burrow/26/20:31
openstackjenkinsTarmac: Added more unit test coverage around update_messages.20:31
*** deshantm_laptop has quit IRC20:49
jaypipesbcwaldon, blamar_, dprince: w00t. Swift functional test case done in Glance: http://paste.openstack.org/show/2125/20:59
bcwaldonjaypipes: tested AND working?!20:59
jaypipesbcwaldon: indeed :)20:59
bcwaldonjaypipes: that's awesome, good work!20:59
jaypipesbcwaldon: thx :)21:01
openstackgerritA change was merged to openstack/glance: Add @skip_if_disabled decorator to test.utils and integrate it into the base functional API test case. The S3 functional test case now uses test_api.TestApi as its base class and the setUp() method sets the disabled and disabled_message attributes that th  https://review.openstack.org/17721:06
openstackgerritJay Pipes proposed a change to openstack/glance: Consolidates the functional API test cases into /glance/tests/functional/test_api.py, adds a new Swift functional test case, verified that it works on Cloud Files with a test account.  https://review.openstack.org/18021:06
*** bengrue has joined #openstack-dev21:06
jaypipesmtaylor: once https://review.openstack.org/#change,180 is approved, I'd like to upload to the glance builder slave some S3 and Swift config files with test account creds and ensure we are testing S3 and Swift drivers on each commit...21:09
*** martine_ has quit IRC21:16
*** rnirmal has quit IRC21:25
*** cp16net has quit IRC21:26
openstackgerritDolph Mathews proposed a change to openstack/keystone: Refactoring business logic behind GET /tenants to make it less convoluted.  https://review.openstack.org/18121:26
*** lts has quit IRC21:26
openstackgerritJames E. Blair proposed a change to openstack/openstack-ci-puppet: Add iptables module and rules to puppet.  https://review.openstack.org/18221:33
*** rnorwood has quit IRC21:48
*** bcwaldon has quit IRC22:02
*** anotherjesse has joined #openstack-dev22:18
anotherjessesandywalsh: does the changes we are sending in with pull 56, 65 make sense now?22:19
sandywalshanotherjesse, yes, thanks ... once we get the redirect stuff straightened out we'll pull those right in22:26
sandywalshanotherjesse (actually they may not conflict anyway)22:26
*** bsza has quit IRC22:26
anotherjessedoes the comment in the redirect about rs auth make sense?22:26
anotherjessehttps://github.com/rackspace/python-novaclient/pull/62#issuecomment-175822922:26
sandywalshyes, I think the service catalog is the thing I was missing. Still have to read to figure out how a single service is specified (if no service catalog)22:27
sandywalshie. we only have nova and nothing else, so always redirect to nova api after auth22:28
sandywalshwhere do I tell keystone that info22:28
sandywalshdegenerate case of service catalog I guess22:28
anotherjessecurrently you have to tell keystone a list of endpoints for each tenant (or have a default one for all tenants)22:28
anotherjessethat is an implementation detail though22:28
anotherjessethe API says auth token returns a catalog … how it is generated is not specified ...22:29
anotherjesserax idm will probably do something specific to your needs?22:29
*** Tushar has quit IRC22:29
*** markvoelker has quit IRC22:31
*** kbringard has quit IRC22:32
*** sandywalsh has quit IRC22:33
*** CatKiller has quit IRC22:36
*** zigo has joined #openstack-dev22:39
*** mattray has quit IRC22:39
*** amccabe has quit IRC22:44
*** sandywalsh has joined #openstack-dev22:46
mtaylorjaypipes: ok. let me know where the files are22:47
jaypipesmtaylor: I will, as soon as other Glance devs approve that proposed branch...22:48
openstackgerritYogeshwar Srikrishnan proposed a change to openstack/keystone: Changes to allow additional calls to support endpoint template CRUD and additional checks on existing method.  https://review.openstack.org/18322:48
*** alekibango has quit IRC22:52
*** CatKiller has joined #openstack-dev22:53
*** yogirackspace has left #openstack-dev22:59
*** jhtran has quit IRC23:13
*** sandywalsh has quit IRC23:25
*** CatKiller has quit IRC23:26
openstackjenkinsProject nova build #1,219: SUCCESS in 3 min 13 sec: http://jenkins.openstack.org/job/nova/1219/23:29
openstackjenkinsTarmac: Adds the ability to read/write to a local xenhost config. No changes to the nova codebase; this will be used only by admin tools that have yet to be created.23:29
*** sandywalsh has joined #openstack-dev23:38
*** vladimir3p has quit IRC23:43
*** jkoelker has quit IRC23:43
*** mfer has quit IRC23:47
*** mfer has joined #openstack-dev23:47
*** martine_ has joined #openstack-dev23:48

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!