Monday, 2011-08-29

« Sunday, 2011-08-28 Index Tuesday, 2011-08-30 »
*** mnour1 has joined #openstack-dev00:05
*** mnour has quit IRC00:05
*** mnour has joined #openstack-dev00:33
*** mnour1 has quit IRC00:36
*** thickskin has quit IRC00:47
*** mnour has quit IRC00:50
*** thickskin has joined #openstack-dev00:59
*** rods has quit IRC01:03
*** redbo has quit IRC01:23
*** redbo has joined #openstack-dev01:23
*** ChanServ sets mode: +v redbo01:23
*** rohitju has joined #openstack-dev02:16
*** alekibango has quit IRC02:16
*** jkoelker_ is now known as jkoelker02:17
*** tsuzuki_ has joined #openstack-dev02:17
*** lts has joined #openstack-dev02:22
*** mdomsch_ has joined #openstack-dev02:35
*** lts has quit IRC03:03
*** mattray has joined #openstack-dev03:32
*** mattray has quit IRC03:45
*** mdomsch_ has quit IRC03:46
*** martine has quit IRC04:25
*** tsuzuki_ has quit IRC05:49
*** tsuzuki_ has joined #openstack-dev05:50
*** rohitju has quit IRC08:21
*** rohitju has joined #openstack-dev08:33
*** AhmedSoliman has joined #openstack-dev08:36
*** chomping has quit IRC08:40
*** uvirtbot` is now known as uvirtbot08:44
*** mnour has joined #openstack-dev08:50
*** rohitju has quit IRC09:01
openstackjenkinsProject nova build #1,306: SUCCESS in 3 min 31 sec: https://jenkins.openstack.org/job/nova/1306/09:10
openstackjenkinsTarmac: Adds assertIn and assertNotIn support to TestCase for compatibility with python 2.609:10
openstackjenkinsThis is a very minimal addition which doesn't require unittest2.09:10
*** zigo has joined #openstack-dev09:44
*** zigo has quit IRC10:10
*** zigo has joined #openstack-dev10:10
*** rods has joined #openstack-dev10:28
*** jaypipes has quit IRC11:16
*** zigo has quit IRC11:17
*** zigo has joined #openstack-dev11:17
*** martine has joined #openstack-dev11:21
*** BK_man has joined #openstack-dev11:37
*** martine has quit IRC11:47
*** martine has joined #openstack-dev11:52
*** adiantum has joined #openstack-dev11:55
*** mfer has joined #openstack-dev12:10
*** lts has joined #openstack-dev12:26
*** bsza has joined #openstack-dev12:27
*** mdomsch has quit IRC12:32
*** markvoelker has joined #openstack-dev12:37
*** markmc has joined #openstack-dev12:41
openstackjenkinsProject nova build #1,307: SUCCESS in 3 min 45 sec: https://jenkins.openstack.org/job/nova/1307/12:50
openstackjenkins* Tarmac: Fixes lp:813864 by removing the broken assert.  The assert was a check for isinstance of 'int' that should have been 'long'.  But it doesn't appear this assert really belongs, anyway.12:50
openstackjenkins* Tarmac: Adds assertIn and assertNotIn support to TestCase for compatibility with python 2.612:50
openstackjenkinsThis is a very minimal addition which doesn't require unittest2.12:50
openstackjenkins* Tarmac: Bug #835952: pep8 failures do not cause the tests to fail12:50
uvirtbotLaunchpad bug 835952 in nova "pep8 failures do not cause the tests to fail" [Undecided,Fix committed] https://launchpad.net/bugs/83595212:50
openstackjenkinsAdd set -eu to run_tests.sh.  This will cause it to fail whenever anything12:50
openstackjenkinsgoes wrong, which is exactly what we want in a test script.12:50
openstackjenkinsTo do this, I had to remove the use of the "let" keyword, which has a bizarre12:50
openstackjenkinsexit status in bash.12:50
openstackjenkinsI also removed the "|| exit" after run_tests, which means that this script12:50
openstackjenkinswill now exit with status 1, not status 0, if run_tests fails.12:50
openstackjenkins* Tarmac: Bug #835964: pep8 violations in IPv6 code12:50
uvirtbotLaunchpad bug 835964 in nova "pep8 violations in IPv6 code" [Undecided,Fix committed] https://launchpad.net/bugs/83596412:50
openstackjenkinsFix pep8 violations.12:50
openstackjenkins* Tarmac: Virtual Storage Array (VSA) feature.12:50
openstackjenkins- new Virtual Storage Array (VSA) objects / OS API extensions / APIs / CLIs12:50
openstackjenkins- new schedulers for selecting nodes with particular volume capabilities12:50
openstackjenkins- new special volume driver12:50
openstackjenkins- report volume capabilities12:50
openstackjenkins- some fixes for volume types12:50
openstackjenkins* Tarmac: changes a few double quotes to be single, as the rest in the vicinity are12:50
openstackjenkins* Tarmac: Update compute API and manager so that the image_ref is set before spawning the rebuilt instance. Fixes issue where rebuild didn't actually change the image_id.12:50
*** joesavak has joined #openstack-dev12:52
*** jsavak has joined #openstack-dev12:56
*** joesavak has quit IRC12:59
*** doude has joined #openstack-dev13:06
*** kbringard has joined #openstack-dev13:07
*** kbringard1 has joined #openstack-dev13:09
*** kbringard has quit IRC13:09
*** zul has quit IRC13:14
*** zul has joined #openstack-dev13:15
*** mattray has joined #openstack-dev13:16
*** Guest49398 is now known as Glace13:24
*** Glace is now known as Guest1618413:25
*** Guest16184 is now known as Glacee13:26
*** bcwaldon has joined #openstack-dev13:44
*** cp16net has joined #openstack-dev14:01
*** nmistry has quit IRC14:04
*** amccabe has joined #openstack-dev14:14
*** martine has quit IRC14:23
*** code_franco has joined #openstack-dev14:23
*** martine_ has joined #openstack-dev14:23
*** zigo has quit IRC14:25
*** zigo has joined #openstack-dev14:25
*** bcwaldon_ has joined #openstack-dev14:32
*** rackerjoe has joined #openstack-dev14:35
*** bcwaldon has quit IRC14:35
*** mnour has quit IRC14:36
*** rnirmal has joined #openstack-dev14:43
*** mdomsch has joined #openstack-dev14:46
openstackgerritSandy Walsh proposed a change to openstack/keystone: starting on tests  https://review.openstack.org/35814:49
openstackgerritSandy Walsh proposed a change to openstack/keystone: Get Service Catalog from token  https://review.openstack.org/35914:50
*** rackerjoe has quit IRC14:58
*** bcwaldon_ has quit IRC15:02
*** dragondm has joined #openstack-dev15:02
*** bcwaldon has joined #openstack-dev15:02
*** bcwaldon has quit IRC15:09
*** heckj has joined #openstack-dev15:22
*** kbringard1 is now known as kbringard15:25
zykes-what's the diff in routes.Mapper between connect & resource ?15:27
*** rnorwood has joined #openstack-dev15:33
creihtzykes-: https://routes.groovie.org/restful.html15:38
*** kbringard has quit IRC15:46
*** kbringard has joined #openstack-dev15:46
*** bsza has quit IRC15:51
*** bsza has joined #openstack-dev15:52
openstackgerritKevin L. Mitchell proposed a change to openstack/glance: Add functional tests  https://review.openstack.org/33315:57
*** mtaylor has quit IRC16:02
*** mtaylor has joined #openstack-dev16:03
*** mtaylor has joined #openstack-dev16:03
*** ChanServ sets mode: +v mtaylor16:03
openstackgerritKevin L. Mitchell proposed a change to openstack/glance: Add functional tests  https://review.openstack.org/33316:09
*** chomping has joined #openstack-dev16:11
*** AhmedSoliman has quit IRC16:11
*** chomping_ has joined #openstack-dev16:13
*** chomping has quit IRC16:15
*** chomping_ is now known as chomping16:15
*** bengrue has joined #openstack-dev16:17
openstackgerritA change was merged to openstack/keystone: Adding guide for RAX-KSKEY-service extension. Adding guide for OS-KSEC2-service extension.  https://review.openstack.org/35516:36
openstackgerritA change was merged to openstack/keystone: Fix host/port split code in authenticate_ec2. Resolves an AttributeError: 'Ec2Credentials' object has no attribute 'partition' exception that can occur for EC2 auth validations.  https://review.openstack.org/35716:37
*** jdurgin has joined #openstack-dev16:39
*** martine has joined #openstack-dev16:43
openstackgerritYogeshwar Srikrishnan proposed a change to openstack/keystone: Adding guides for groups extension.  https://review.openstack.org/36016:44
openstackgerritDolph Mathews proposed a change to openstack/keystone: Get Service Catalog from token  https://review.openstack.org/35916:45
zykes-creiht: still i don't get the diff on resource & controller16:49
zykes-ehm, connect16:49
*** vladimir3p has joined #openstack-dev16:51
*** med_out is now known as medberry16:51
openstackgerritYogeshwar Srikrishnan proposed a change to openstack/keystone: Adding guides for groups extension.  https://review.openstack.org/36016:56
openstackgerritSandy Walsh proposed a change to openstack/keystone: Get Service Catalog from token  https://review.openstack.org/35916:58
openstackgerritA change was merged to openstack/keystone: Adding guides for groups extension.  https://review.openstack.org/36016:58
openstackjenkinsProject burrow build #36: SUCCESS in 13 sec: https://jenkins.openstack.org/job/burrow/36/17:02
openstackjenkinsTarmac: Added examples, bumped version.17:02
*** ohnoimdead has joined #openstack-dev17:02
sandywalshwhat state do I put a launchpad bug in to reopen it?17:11
sandywalshback to "new"?17:11
sandywalshsoren ttx mtaylor ^^17:14
*** bcwaldon has joined #openstack-dev17:15
bcwaldonvishy: can you update this MP with your thoguhts -> https://code.launchpad.net/~klmitch/nova/lp832369/+merge/7296417:16
openstackjenkinsProject nova build #1,308: SUCCESS in 3 min 38 sec: https://jenkins.openstack.org/job/nova/1308/17:20
openstackjenkins* Tarmac: In the XenAPI simulator, set VM.domid, when creating the instance initially,17:20
openstackjenkinsand when starting the VM.17:20
openstackjenkinsThis shows up in the logs for Bug #831599, but this fix doesn't actually17:20
openstackjenkinsfix the hang.17:20
uvirtbotLaunchpad bug 831599 in nova "Tests hang frequently in XenAPIVMTestCase.test_parallel_builds" [Medium,Confirmed] https://launchpad.net/bugs/83159917:20
openstackjenkins* Tarmac: Fixes lp:813864 by removing the broken assert.  The assert was a check for isinstance of 'int' that should have been 'long'.  But it doesn't appear this assert really belongs, anyway.17:20
bcwaldon_cerberus_: can you update your review of this branch -> https://code.launchpad.net/~jason-koelker/nova/lp819477/+merge/7193017:25
*** novas0x2a|laptop has joined #openstack-dev17:30
*** negronjl has quit IRC17:42
_cerberus_bcwaldon: maybe. What's it worth to you?17:47
bcwaldon_cerberus_: a hug17:47
_cerberus_Sold17:48
*** negronjl has joined #openstack-dev17:49
sandywalsh*21217:49
creihtzykes-: connect is the base unit of mapping a url path to a python callable17:51
creihtresource basically makes a lot of pre-defined connect statements17:51
sandywalshant, just put a merge prop in to give a little more logging if that notification exception crops up again.17:51
*** mszilagyi has joined #openstack-dev18:04
*** bcwaldon has quit IRC18:06
*** bcwaldon has joined #openstack-dev18:09
zykes-what's the groups stuff in keystone ?18:13
zykes-something rackspace specific ?18:13
dolphmzykes-: more or less... it's being planned as an extension only18:14
jsavakzykes - yes - it's an extension to support rackspace within keystone.18:14
jsavakzykes - included in the open because other contributors are interested in it18:14
zykes-what's it used for in diff to tenants ?18:15
vishybcwaldon: done18:17
bcwaldonvishy: thanks18:17
vishysandywalsh: you can set it back to in progress or triaged i suppose18:17
sandywalshvishy, yeah, that's what I did ... thx18:18
jsavakzykes - when a user auths, they'll get a list of groups that the user belongs to , roles that the user has, and service endpoints that the user has access to. Tenants are attributes to the endpoint showing what account/company that endpoint belongs to18:22
zykes-what's the groups used for then ?18:23
*** zigo has quit IRC18:23
*** martine has quit IRC18:26
jsavakzykes - i think it's an alternative for roles or enables a different way to manage roles. Ex: "Add Account" role may mean different things to the "Support" group than it does to the "Ops" group18:26
zykes-hmm k18:27
zykes-so basically just more auth stuff then18:27
jsavakyup18:27
vishyi find groups + roles confusing :)18:28
vishytoo many dimensions.  Keep things flat imo18:29
jsavakvishy - checking with ziad to see if i got that right. I could be, and often am, wrong.18:33
vishyjsavak: I'm sure there are lots of existing constraints inside rs.  I'm just saying in general I find it confusing to manage18:34
*** mdomsch has quit IRC18:39
openstackjenkinsProject nova build #1,309: SUCCESS in 3 min 34 sec: https://jenkins.openstack.org/job/nova/1309/18:40
openstackjenkinsTarmac: Update the EC2 ToToken middleware to use eventlet.green.httplib instead of httplib2. Fixes issues where the JSON request body wasn't getting sent to Keystone.18:40
*** zaitcev has joined #openstack-dev18:44
*** bcwaldon has quit IRC18:45
*** bcwaldon has joined #openstack-dev18:47
*** adiantum has quit IRC19:03
sorensandywalsh: Depends. Which bug?19:06
*** martine has joined #openstack-dev19:11
*** martine_ has quit IRC19:15
*** rackerjoe has joined #openstack-dev19:18
*** rackerjoe has quit IRC19:19
smoseranyone know where/why bug 836759 ?19:22
uvirtbotLaunchpad bug 836759 in nova "nova does not grow disk images as it does partition images" [Medium,Triaged] https://launchpad.net/bugs/83675919:22
smoseri cant see how 'truncate -s' is not done for a disk image.19:23
*** martine_ has joined #openstack-dev19:23
sorensmoser: I have a guess.19:24
sorensmoser: If you truncate -s a qcow image... What happens?19:24
smoserwell, not waht you want19:24
sorenRight.19:24
smoserbut i had asumed that it must be a raw image at that point19:24
smoseri guess that would explain it though.19:25
*** dolphm has quit IRC19:25
sorenIt doesn't convert your qcow to raw, if that's what you mean.19:25
smoserit would seem to me that it should.19:25
sorenIf you upload a qcow2 image, that's what kvm ends up using. At least I'm reasonably sure it is.19:25
smoseras a reasonable point of making things sane.19:25
smoseri think you're right.19:25
sorenqcow2 could be fine.19:26
sorenThe fact that it doesn't really notice if you're using Xen and qcow2+Xen won't work... That's a different story.19:26
smoserit would seem that to be sane, a conversion should happen to raw at some point.19:30
smoserso this verifies the other question i had...19:30
smoserif i upload a compressed qcow, it will just 'qemu-img create -b <COMPRESED_QCOW>'19:31
smoserwhich will mean all reads will go through uncompress19:31
smosersoren, would you consider it reasonable if i patch _fetch_image in nova/virt/libvirt/connection.py to result in a raw image ?19:38
sorenNot really.19:39
sorenWhy would you insist on a specific, hard-coded image type?19:39
*** bcwaldon has quit IRC19:40
sorensmoser: ^19:41
smoserewll, for simplicity.19:41
sorensmoser: If it's an image type the hypervisor can handle, why convert it?19:41
smoserthe other option is to make "extend" be able to extend correctly.19:42
sorensmoser: Yeah. It shouldn't use truncate to begin with.19:42
smoserwell, at some point things like 'truncate -s 1TB' are going to screw you.19:42
sorensmoser: It should use "qemu-img extend"19:43
sorenOr whatever it's called.19:43
smoserfor my case, it also needs convert19:43
sorenqemu-img resize19:43
sorenWhy?19:43
smoserbecause its compressed.19:43
smoserand otherwise 3 instances will all do reads() that will to trhough decompress19:43
smoserat some point you want to have reasonable backed images. i really think that having "raw" (possibly sparse) cached image is sane.19:44
sorenI think forcing a specific format is a poor idea. No matter which format.19:44
sorenHaving it available as an option sounds great.19:45
soren..but forcing it for everyone.. Certainly not a change I'd approve of at this point in the cycle.19:45
*** martine has quit IRC19:46
smoseri cant really think of anything other than a test case that would want qcow images backed by a compressed qcow image.19:46
smoser(personally, i think for anything other than a test, you dont want qcow backed images)19:46
*** markvoelker has quit IRC19:46
sorenAnd that's a perfectly valid use case.19:47
*** bcwaldon has joined #openstack-dev19:47
smoserbut... for you, soren, i would:19:47
sorenFWIW, Glance is supposed to grow conversion features eventually.19:48
sorenAt that point, we could just ask glance for a specific format and it'd do some magic to give us that.19:48
smoserthat just pushes the ubck.19:48
smoserbuck19:48
smoserand means you have to get a lot more data to glance to say "get me an image"19:48
*** heckj has quit IRC19:49
smosernever mind. you're right.19:49
smoserbut so:19:49
smoser * add option for use-raw-backed-qcow images [default false]19:49
smoser * add option for allow-compressed-backed-qcow-images [default false]19:49
smoserdoes that sound reasonable?19:51
pvosoren: can the libvirt network filters be used without libvirt managing instances?19:58
sorenpvo: The various spoofing protection things, no.19:59
sorenpvo: The other stuff, yes.19:59
sorensmoser: Can you elaborate on what each of those would do? Sorry, I really don't mean to be difficult, but I'm really not at my most useful right now :(20:00
pvosoren: thanks.20:00
pvosoren: would you see any reason why we couldn't use libvirt enable security groups on XenServer? In theory?20:01
pvoI haven't tried yet, just wondering out loud.20:01
*** vladimir3p has quit IRC20:04
sorenpvo: I don't know, really.20:04
smoseri guess you dont need two flags.20:04
sorenpvo: Generally, if you can affect traffic to/from guests using iptables in your dom0, it should work.20:05
pvosoren: ok, thats what I was thinking too. you've at least told me enough to start experimenting with it. : )20:05
sorenpvo: I've never really used Xen enough to know about its networking model.20:05
sorenpvo: :)20:05
smoserbut basically one would modify the current behavior to always qemu-img convert -O raw <source>.img <target>.img (after checking that it wasnt raw)20:05
pvosoren: thanks20:05
smoserthen secondly, special case the path where it is compressed qcow.20:06
*** cmagina has quit IRC20:10
*** vladimir3p has joined #openstack-dev20:15
*** markmc has quit IRC20:40
openstackjenkinsProject swift build #312: SUCCESS in 35 sec: https://jenkins.openstack.org/job/swift/312/20:47
openstackjenkinsTarmac: deleting a database (account or container) now also clears the metadata20:47
*** mdomsch has joined #openstack-dev20:55
*** bcwaldon has quit IRC21:12
*** martine_ has quit IRC21:21
*** jsavak has quit IRC21:21
*** jsavak has joined #openstack-dev21:22
*** jakedahn has joined #openstack-dev21:24
*** mdomsch has quit IRC21:27
*** Tushar has joined #openstack-dev21:29
*** Tushar has quit IRC21:31
chmouelsomeone would have an objection if I propose a patch that s/project/tenant/ in python-novaclient?21:34
vladimir3pvishy: ping21:37
*** jsavak has quit IRC21:40
*** bengrue has quit IRC21:49
*** cp16net has quit IRC21:51
*** lts has quit IRC21:53
*** jakedahn has quit IRC21:56
vishyvladimir3p: pong21:57
*** jakedahn has joined #openstack-dev21:57
vladimir3pvishy: Hi, looking at some very strange issue in our lab21:59
vladimir3pvishy: trying to understand how thread-safe might be the code of execute() in nova/utils22:00
vishyok22:00
vladimir3pvishy: in particular usage of PIPE in Popen22:00
vishythread safe? not at all22:00
vishyalthough why would you be using multiple threads22:00
vladimir3p:-)22:00
vishy?22:00
vladimir3pwhat we are seeing is report capabilities call from periodic tasks trying to get current states22:01
vladimir3pat the same time there is a call to create & export volume22:01
vladimir3pand export fails even without reaching the actuall app22:01
vladimir3pdoes periodic tasks & handling of new requests in manager executed in the same thread?22:02
vishyeverything is the same thread22:02
vishyit uses greenlets22:02
vishyso concurrency issues are generally external22:02
vishyfor example.  Initiating a second call to an lvm command22:03
vishywhen one is already in process22:03
vishyvladimir3p: if you can isolate which calls are conflicting you should be able to fix it by wrapping the offending section in a @utils.synchronized()22:04
vladimir3pvishy: one sec, brb22:05
vladimir3pvishy: I'm back. sorry.22:12
vladimir3pvishy: let me understand how this single thread thing work ...22:12
*** kbringard has quit IRC22:12
vladimir3pvishy: for example when scheduler sends 2 requests to volume mgr to create 2 volumes22:12
vishyok22:13
vladimir3pthat I see in logs is...22:13
vladimir3psome sort of intermixing of these cmds22:13
vladimir3pI see create_volume for vol122:13
vladimir3pafter that create vol for vol222:13
vladimir3pafter that create export for vol122:13
vishyright22:13
vishyso the event loop calls execute22:13
vishyand essentially that "thread" waits (it isn't a thread but just for understanding think about it that way)22:14
vladimir3pyeah, it is right, but create volume and create export calls are calls to driver and executed from the same func22:14
vishybut new requests come in and can go through the same process22:14
vishyso you don't get two different greenlets executing the same python code concurrently22:15
vladimir3pyes, this is how it supposed to be ...22:15
vishybut external commands like execute can happen at the same time22:15
vladimir3pone sec, let me 1st clarify re create volume & export22:15
vladimir3phere what I see in logs ... let me copy/paste it ...22:16
vladimir3phttp://paste.openstack.org/show/2303/22:16
vladimir3pthere are 2 volumes - volume-00000b9b & volume-00000b9e22:17
vladimir3pI expect that 1st of all volume mgr will call create_volume for volume-00000b9b and it will result in 2 driver calls: self.driver.create_volume(volume_ref) & self.driver.create_export(context, volume_ref)22:18
vladimir3pbut according to logs there was something for 2nd volume executed in the middle ... or probably logs are screwed ...22:18
vishyyes it eats the exception because of eventlet issue22:19
vishyhave to reraise the exception using the other format22:19
vishyyes that will happen22:19
vishybecause each "thread" will pause waiting for the results of execute22:19
vishyso if the executes can run concurrently22:20
vishyif there is some execute that is not thread safe22:20
vishythen you need to wrap it in a utils.synchronized so only one happens at a time22:20
vladimir3phmm... interesting . Actually, from our perspective executes are safe ... but I'm trying to understand if such multi-execs nature of things might be broken due to usage of PIPE22:21
vishyshouldn't be22:22
vladimir3pwhat we see is that one of create_exports is not even reaching our utility22:22
vladimir3p(at least there shoould be a print and there is no)22:22
vishyyou should get the traceback22:23
vishyit will probably help22:23
vladimir3pyep. I will change this portion and will raise the correct exception22:23
vishyi did it in volume cleanup22:24
vishyshould look like this:22:24
vishyhttp://paste.openstack.org/show/2304/22:25
vishyall the raises there need to be fixed22:25
vladimir3pah, ok ... we are raising exception.ProcessExecutionError in any error...22:26
vladimir3plike this:22:26
vladimir3phttp://paste.openstack.org/show/2305/22:27
vladimir3pI will change it to your style22:27
*** mfer has quit IRC22:38
*** mgius has joined #openstack-dev22:38
*** code_franco has quit IRC22:42
*** rnorwood has quit IRC22:44
*** mattray has quit IRC22:48
*** rnirmal has quit IRC22:56
*** dragondm has quit IRC23:39
*** novas0x2a|laptop has quit IRC23:41
*** medberry is now known as med_out23:44
*** novas0x2a|laptop has joined #openstack-dev23:46
*** novas0x2a|laptop has quit IRC23:53
*** amccabe has quit IRC23:54
*** martine_ has joined #openstack-dev23:55
« Sunday, 2011-08-28 Index Tuesday, 2011-08-30 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!