Friday, 2012-02-17

*** hsd has joined #openstack-dev00:02
*** hsd has quit IRC00:03
*** hsdhsd has joined #openstack-dev00:04
adam_gat what point is the instance cache info supposed to updated on association of a floating IP?00:05
zultermie: ok00:05
adam_gzul: im not sure if its correct, but this gets around that for me, http://paste.ubuntu.com/845180/00:07
termieadam_g: that looks like amore appropriate fix00:08
*** dtroyer has quit IRC00:10
*** jakedahn has quit IRC00:12
*** hsdhsd has quit IRC00:12
*** heckj has quit IRC00:14
*** AlanClark has quit IRC00:22
*** sniperd has quit IRC00:24
*** sleepsonthefloo has quit IRC00:24
*** mattray has quit IRC00:29
*** nati2 has joined #openstack-dev00:35
*** dtroyer has joined #openstack-dev00:38
bcwaldonjeblair: !00:38
termiemtaylor: !00:38
bcwaldonmtaylor: !00:38
termiejeblair: !00:38
bcwaldon!!00:38
openstackbcwaldon: Error: "!" is not a valid command.00:38
termie!@00:39
openstacktermie: Error: "@" is not a valid command.00:39
*** dolphm has quit IRC00:39
mtaylortermie: yes?00:40
termiebcwaldon: FIRST00:40
termiemtaylor: can we make keystoneclient changes gated on devstack?00:41
mtaylortermie: hrm. I imagine so00:41
termiemtaylor: it is in use by devstack and it looks like it may have just broken00:41
jeblair!00:42
termiejeblair: too slow00:43
termieYOU'LL NEVER WORK IN THIS CHANNEL AGAIN00:43
* mtaylor finally beats jeblair 00:43
mtaylortermie: we can totally do that - no problem...00:43
jeblairthis java shit is slowing me down.  :)00:43
mtaylortermie: it'll be easier to do if we fix the current devstack break first00:43
mtaylorso that we can confirm that our change to gate on keystone client doesn't itself break anything :)00:44
termieCURSES00:44
termiebcwaldon: sounds like a job for you00:44
mtaylor(I mean, in either direction, the current devstack break will have to get fixed first) ...00:44
* mtaylor pokes bcwaldon00:44
bcwaldonwatch yourself00:44
mtaylorBUT - jeblair will get the keystoneclient gating ready to go00:44
bcwaldonkk, let me figger out whats broke her00:44
mtaylorhow can I watch myself on to my own head?00:45
bcwaldonwat00:45
* termie high-fives mtaylor00:45
termieONWARD LACKEYS00:45
* termie cracks a whip00:45
bcwaldonMUTINY00:45
*** pixelbeat has joined #openstack-dev00:45
mtaylorw00h00!00:46
mtaylortermie: where is teh breaked devstack?00:54
termiemtaylor: bcwaldon had some failures when he ran it last, he's verifying00:55
*** adiantum has joined #openstack-dev00:56
termiemtaylor: maybe it isn't failing00:58
termiemtaylor: but gating is awesome still00:58
jeblairokay, i'll go ahead and turn on gating then00:58
bcwaldonshould be good, devstack isn't failing for me00:59
*** adiantum has quit IRC01:00
bcwaldonexercises pass, g2g01:01
*** danwent has quit IRC01:01
*** danwent has joined #openstack-dev01:02
*** jdg has quit IRC01:04
jeblairbcwaldon, termie: python-keystoneclient is gated by devstack now01:08
bcwaldonjeblair: thanks!01:08
*** reed has quit IRC01:13
*** gyee has quit IRC01:19
*** Gordonz has joined #openstack-dev01:21
*** kmansel_ has quit IRC01:27
*** torgomatic has quit IRC01:27
*** littleidea has quit IRC01:28
*** rsampaio has joined #openstack-dev01:28
*** rsampaio has quit IRC01:29
*** rackerjoe has joined #openstack-dev01:30
*** littleidea has joined #openstack-dev01:31
adam_ganyone feel like nudging this after a pep8 fix https://review.openstack.org/#change,414901:40
*** danwent_ has joined #openstack-dev01:41
*** danwent has quit IRC01:45
*** danwent_ is now known as danwent01:45
*** rods has quit IRC01:49
*** danwent has quit IRC01:51
*** danwent has joined #openstack-dev01:52
*** shevek_ has quit IRC01:54
*** danwent_ has joined #openstack-dev02:00
*** cp16net has joined #openstack-dev02:02
*** danwent has quit IRC02:04
*** danwent_ is now known as danwent02:04
*** kbringard has quit IRC02:10
*** spiffxp has quit IRC02:13
*** cp16net has quit IRC02:15
*** eglynn_ has joined #openstack-dev02:20
*** eglynn has quit IRC02:20
*** Gordonz has quit IRC02:21
*** LinuxJedi has quit IRC02:37
*** Ryan_Lane has quit IRC02:42
*** mnewby has quit IRC02:48
vishytr3buchet: ping02:49
vishydanwent: ping02:49
*** Gordonz has joined #openstack-dev02:50
vishytr3buchet, danwent: there are a few networking issues with live migration that our deploy team is working through.  I'd like to loop you guys in on the best way to handle the updates that need to be done.02:52
*** davlap has quit IRC02:57
*** maplebed has quit IRC03:02
*** pixelbeat has quit IRC03:02
*** adjohn has quit IRC03:03
vishytr3buchet, danwent: emailed03:23
danwentvishy:ack.  not sure why I didn't see your messages03:24
danwentreading email03:24
danwentvishy: when you say " the bridge and ip are never created", are you taking about the bridge interface on the correct VLAN and the IP address that would be used by the DHCP server?03:29
danwentlet's say VM was on VLAN 5, moves from host X to Y, on Y, there is never a DHCP instance setup on VLAN 5?03:30
vishydanwent: yes03:35
vishydanwent: if there was already a vm there that is taken care of03:35
vishybut if it is a new server _setup_network never gets called03:35
danwentyup, got it.03:36
vishyi gotta run, but respond to the email if you get any bright ideas03:37
*** Gordonz has quit IRC03:37
vishy:)03:37
danwentk, later03:40
*** cp16net has joined #openstack-dev04:06
*** cp16net has quit IRC04:08
*** danwent has quit IRC04:08
*** cp16net has joined #openstack-dev04:08
*** novas0x2a|laptop has quit IRC04:10
*** sniperd has joined #openstack-dev04:12
*** PotHix has quit IRC04:18
*** mdomsch has joined #openstack-dev04:23
*** Gordonz has joined #openstack-dev04:25
*** Ryan_Lane has joined #openstack-dev04:40
*** Gordonz has quit IRC04:44
*** sleepsonthefloo has joined #openstack-dev04:46
*** rackerjoe has quit IRC04:52
*** Mandell has quit IRC05:00
*** Mandell has joined #openstack-dev05:04
*** Mandell_ has joined #openstack-dev05:05
*** Mandell has quit IRC05:05
*** zigo has joined #openstack-dev05:26
*** littleidea has quit IRC05:27
*** littleidea has joined #openstack-dev05:30
*** danwent has joined #openstack-dev05:37
*** mdomsch has quit IRC05:38
*** sniperd has quit IRC05:50
*** sniperd has joined #openstack-dev06:03
*** sandywalsh has quit IRC06:14
*** sniperd has quit IRC06:21
*** zigo has quit IRC06:23
*** sandywalsh has joined #openstack-dev06:28
*** danwent has quit IRC06:36
*** danwent has joined #openstack-dev06:36
*** zaitcev has quit IRC06:50
*** Lumiere has quit IRC06:50
*** Lumiere has joined #openstack-dev07:01
*** dolphm has joined #openstack-dev07:02
*** mikal has quit IRC07:06
*** mikal has joined #openstack-dev07:09
*** danwent has quit IRC07:10
*** bepernoot has joined #openstack-dev07:11
*** mikal has quit IRC07:18
*** cp16net has quit IRC07:18
*** mikal has joined #openstack-dev07:21
*** bepernoot has quit IRC07:22
*** dolphm has quit IRC07:25
*** mikal has quit IRC07:26
*** mikal has joined #openstack-dev07:29
*** mikal has quit IRC07:34
*** mikal has joined #openstack-dev07:37
*** deshantm has quit IRC07:45
*** Mandell_ has quit IRC07:46
*** mikal has quit IRC07:49
*** mikal has joined #openstack-dev07:53
*** zigo has joined #openstack-dev08:00
*** zigo-_- has joined #openstack-dev08:05
*** zigo has quit IRC08:05
*** shang has quit IRC08:06
*** littleidea has quit IRC08:07
*** zigo-_- has quit IRC08:10
*** zigo has joined #openstack-dev08:11
*** shang has joined #openstack-dev08:12
*** shevek_ has joined #openstack-dev08:12
*** Ryan_Lane has quit IRC08:16
*** zigo has quit IRC08:19
*** zigo has joined #openstack-dev08:20
*** sleepsonthefloo has quit IRC08:30
*** bepernoot has joined #openstack-dev08:32
*** rkukura has quit IRC08:35
*** nati2 has quit IRC08:46
*** rkukura has joined #openstack-dev08:51
*** derekh has joined #openstack-dev09:04
*** journeeman has joined #openstack-dev09:19
*** justinsb has quit IRC09:20
*** pixelbeat has joined #openstack-dev09:31
*** maploin has joined #openstack-dev09:45
*** maploin has quit IRC09:45
*** maploin has joined #openstack-dev09:45
*** ncode has joined #openstack-dev10:11
*** ncode has joined #openstack-dev10:11
*** andrewsmedina has quit IRC10:17
*** shang has quit IRC10:21
*** yamahata has quit IRC10:26
*** darraghb has joined #openstack-dev10:32
*** adjohn has joined #openstack-dev10:49
*** govindkrajulu has joined #openstack-dev10:54
*** adjohn has quit IRC10:54
*** andrewsmedina has joined #openstack-dev10:55
*** eglynn_ has quit IRC10:59
*** govindkrajulu has quit IRC11:00
dayouI have a stupid question about jenkins and gerrit review.11:03
dayouHow are they connected with each other?11:04
*** zigo has quit IRC11:07
*** zigo has joined #openstack-dev11:07
*** markmc has joined #openstack-dev11:31
*** zigo has quit IRC11:52
*** eglynn_ has joined #openstack-dev11:59
*** journeeman has quit IRC12:05
*** paulormg has joined #openstack-dev12:17
*** markvoelker has joined #openstack-dev12:18
*** eglynn_ has quit IRC12:25
*** eglynn_ has joined #openstack-dev12:33
*** dprince has joined #openstack-dev12:34
*** adiantum has joined #openstack-dev12:50
*** PotHix has joined #openstack-dev13:05
*** bsza has joined #openstack-dev13:14
*** sandywalsh has quit IRC13:19
*** dayou has quit IRC13:20
*** hashar has joined #openstack-dev13:20
*** hashar has joined #openstack-dev13:21
*** vinilios has joined #openstack-dev13:26
*** martine has joined #openstack-dev13:29
*** markmc has quit IRC13:31
*** eglynn_ has quit IRC13:31
*** vinilios has quit IRC13:31
*** eglynn_ has joined #openstack-dev13:32
*** sandywalsh has joined #openstack-dev13:33
*** rsampaio has joined #openstack-dev13:40
*** flaviamissi has joined #openstack-dev13:41
*** markmc has joined #openstack-dev13:43
*** csanburn has quit IRC13:45
*** mjfork has quit IRC13:45
*** markmcclain has quit IRC13:47
*** sniperd has joined #openstack-dev13:48
*** markmc has quit IRC13:49
*** eglynn__ has joined #openstack-dev13:49
*** eglynn_ has quit IRC13:49
*** eglynn has joined #openstack-dev13:54
*** eglynn__ has quit IRC13:54
*** martine has quit IRC13:54
*** adiantum has quit IRC13:55
*** stuntmachine has joined #openstack-dev13:56
*** eglynn_ has joined #openstack-dev13:59
*** eglynn has quit IRC14:00
*** markmc has joined #openstack-dev14:01
*** eglynn__ has joined #openstack-dev14:10
*** eglynn_ has quit IRC14:10
*** jaypipes has joined #openstack-dev14:24
*** lts has joined #openstack-dev14:25
*** andrewbogott_afk has quit IRC14:26
*** andrewbogott_afk has joined #openstack-dev14:26
*** ncode has quit IRC14:34
jaypipeseglynn__: I have assigned you to https://bugs.launchpad.net/glance/+bug/93399314:39
uvirtbot`Launchpad bug 933993 in glance "glance add can fail for images larger than 2 GB on some Python installations (RHEL 6.2)" [Critical,Confirmed]14:39
jaypipeseglynn__: I figure a bright guy like you can whip up a quick solution :)14:39
eglynn__jaypipes: cool, I'm on it ... just finishing up the other critical bug on the E4 list14:40
*** markmcclain has joined #openstack-dev14:42
*** rackerjoe has joined #openstack-dev14:42
openstackgerritVerification of a change to openstack/glance failed: Ensure StorageFull only raised on space starvation  https://review.openstack.org/424714:43
jaypipeseglynn__: no worries mate. appreciated!14:43
jaypipeseglynn__: why on Earth 32-bit systems are still supported is beyond me anyway :)14:43
eglynn__yep, oh so retro ...14:44
jaypipeseglynn__: next we're going to be installing OpenStack on my Commodore 64...14:45
*** littleidea has joined #openstack-dev14:46
*** markmcclain has quit IRC14:53
*** markmcclain has joined #openstack-dev14:56
*** markmcclain has quit IRC15:09
*** markmc has quit IRC15:14
*** markmcclain has joined #openstack-dev15:17
*** markmcclain has left #openstack-dev15:17
*** yamahata has joined #openstack-dev15:25
*** markmc has joined #openstack-dev15:26
*** derekh has left #openstack-dev15:35
*** derekh has joined #openstack-dev15:35
*** dtroyer has quit IRC15:44
*** sandywalsh has quit IRC15:45
*** bengrue has quit IRC15:48
*** andrewbogott_afk has quit IRC15:50
*** andrewbogott_afk has joined #openstack-dev15:51
*** zzed has joined #openstack-dev15:52
mtaylorjaypipes: I've already got one of those going ...15:53
jaypipesmtaylor: lol15:54
*** andrewbogott_afk has quit IRC15:55
*** andrewbogott_afk has joined #openstack-dev15:55
ayoungtermie, I've got LDAP working 100% against the unit tests.  https://github.com/admiyo/keystone/tree/ldap315:56
ayoungIts been rebased against master, too.15:56
ayoungI'll squash the patches before submitting,  but I suspect it will still need more work.15:57
*** dtroyer has joined #openstack-dev15:58
*** spiffxp has joined #openstack-dev15:59
*** sandywalsh has joined #openstack-dev16:01
*** spiffxp has quit IRC16:03
*** mdomsch has joined #openstack-dev16:04
*** nati2 has joined #openstack-dev16:07
smoserjkoelker, around ?16:09
*** heckj has joined #openstack-dev16:10
*** cp16net has joined #openstack-dev16:11
*** kmansel has joined #openstack-dev16:13
*** bsza has quit IRC16:14
*** bsza has joined #openstack-dev16:14
jkoelkeri am16:14
jkoelkersmoser ^16:14
smoserlooking at commit 8f9453aeb8882509d825c9715fde4e6827b0bbf716:15
smoser(wihch i can't give you a link to because github is apparently down)16:15
smoserah. wait. its back.16:16
smoserhttps://github.com/openstack/nova/commit/8f9453aeb8882509d825c9715fde4e6827b0bbf716:16
smoserCanonical has an internal openstack cloud, its really nothing large, but has been running for 6 months or so and has had a lot of usage.16:17
jkoelkerah, yes regexing ips16:17
smoserhte metadata service is unusably slow after upgrade to essex16:17
comstudruh roh16:17
comstud:)16:17
smoser(every request is now 3 seconds at least)16:17
*** dolphm_ has joined #openstack-dev16:17
tr3bucheti blame comstud16:17
comstudlol16:17
smoseryour comment of "this could suck on large installs" is incorrect16:17
jkoelkernow that there is the network info cache, that can be moved back into compute/db16:18
smoser:)16:18
smoserin that it sucks on modest installs16:18
smoserjkoelker, oh?16:18
tr3buchetsucks on all installs is probably more correct16:18
jkoelkerwell first we must define "large" ;)16:18
comstudis metadata service doing instance lookups filtered by IP address?16:18
comstudi guess it makes sense that it would16:18
smoserthe metadata service is terribly inefficient in all sorts of ways. there was one patchset to improve it to be more careful on what it does, but that has fallen off and needs repair.16:19
comstudi don't think this was really intended to be hit heavily like that16:19
smoserbut this specific code, one of our IS team identified as a major issue, and has a hack work around.16:19
comstudreally this was meant for admin API... just finding an instance by IP address16:19
*** littleidea has quit IRC16:20
smoserhttp://paste.ubuntu.com/846027/ is what he came up with16:20
smoserand there are comments tere, stating we go from ~3.30 secondsn per query to .0216:20
comstudand tr3buchet and jkoelker need to figure out a way to search by IP via the info_cache table :)16:20
* comstud throws people under the bus.16:21
* tr3buchet saves vs bus16:21
tr3buchetcomstud: it's actually not hard, just inefficient16:22
*** maploin has quit IRC16:22
jkoelkerwe need to not search for ips via regex16:22
jkoelker;)16:22
* jkoelker thows the api under the bus16:22
comstudwell16:22
comstudeven without regex, you have a problem right now16:23
comstudbut sure16:23
comstudbut you could implement a non-regex and a regex version of the calls..16:23
comstudmake the regex be slow.. that's fine :)  it's admin only16:23
jkoelkersearch by cidr, not the string representation16:24
comstudsmoser: i think self.db.fixed-ip_get_all() assumes nova DB is authoritative for network info... which it is not always16:25
*** maplebed has joined #openstack-dev16:25
smosercomstud, well, yeah its a hack.16:25
*** mjfork has joined #openstack-dev16:26
smoserthey're grasping at straws, because after updating, instances don't boot.16:26
comstudthat's why it's slow now.. the untying ;)16:26
comstudyea16:26
smosercloud-init has (admittedly stupidly low) timeout on GETs to md of 2 seconds16:26
comstudah16:26
comstudwell, really we need to have IPs indexed somehow in instance_info_cache table16:27
comstudor yet another table16:27
comstud:-/16:27
jkoelkerwe need to figure out who is authoritative for ip information16:28
jkoelkerand then excersise it from everything that is not16:28
jkoelkerthat's why we're in this mess, because both nova and quantum/melange are authoratiative for different bits of information16:29
comstudi assumed it was network manager.. whether it was the DB or a remote service (quantummelange)16:29
comstudDB/nova DB16:29
jkoelkerright, but the quantum manager uses the nova db as a message passing, cache system as well16:29
comstudwell, I don't feel like the API should care who is 'auth'16:29
jkoelkerwe havn't had the time to fix that yet either16:29
comstudit should just have a copy of the shit indexed properly16:30
comstud:)16:30
comstudand trust it is updated!16:30
comstud(i group metadata service in with API here)16:30
jkoelkernova api or quantum api or melange api?16:30
jkoelkerwhich one we talking about? i assume nova16:30
comstudsorry, nova/ec2 api and metadata service16:31
jkoelkerit would be great if we had a caching solution for it16:31
comstudjkoelker: btw i threw up a branch removing the fallback to network API16:31
*** agy has joined #openstack-dev16:31
comstudjkoelker: i need to write a test for the periodic update i added16:31
jkoelkerexcellent!16:31
comstudjkoelker: if you want to go peak at it sometime today and tell me what you think16:32
comstudthe removal of the stuff in OS API and EC2 api is pretty obvious, i guess16:32
jkoelkerroger, I'll look at that after this16:32
comstudcools.. i'll get the test finished later today16:32
comstudyou'll see i created one that just has 'pass' in it16:33
comstudlol16:33
comstudwhich.... passes!16:33
jkoelkerhahah, nice, those are the integration tests i like ;)16:33
comstudi'm only going to do a unit test16:33
comstudi'm going to assume that get_instance_nw_info in network manager does its job properly16:34
jkoelkersmoser: so I don't know if we have a good answer as to what to do until the openstack project as a whole figures out the direction they are going in regarding ip/network stuff,16:34
smoserjkoelker, well, i'd say that this is pretty much critical at this point.16:34
jkoelkercomstud: i think we have test coverage for it16:34
jkoelkersmoser: i'd agree16:35
smoserwe simply can't have queries to the MD taking 3 seconds.16:35
smoserbug 85115916:35
uvirtbot`Launchpad bug 851159 in nova "ec2 metadata service is very slow" [High,Confirmed] https://launchpad.net/bugs/85115916:35
comstudi think the quickest solution is an IP -> instance mapping table16:35
smoseri'll update that bug, with this information and pointer to this discussion.16:35
comstudupdated at same spot instance_info_cache is updated16:35
comstud:-/16:36
comstudit could hopefully be short lived until something else is decided16:36
comstudthat's all MD service needs.. right?16:36
comstudneeds to know the instance from IP ?16:36
comstudi mean, not all, but that's the slow part?16:37
jkoelkeryea that wouuld be the quickest16:38
jkoelkeryea admitedly i have no idea what the metadata service is or does16:38
smoserwell, this was very slow. there are some other fixes we can do there too. i might try to revive the one patch that is there.16:38
smosermetadata service is something that lives at what the instance percieves to be http://some-address/16:39
smoserand it can get information about itself from there16:39
smoserthe most important thing is the user-data16:39
smoserwhich cloud-init (and others) use to do things on first boot.16:39
comstudit's ec2 compatable16:40
smoseror otherwise  tell the instance what to do from the user who laucnhe dit.16:40
smoserright.16:40
comstudnova/api/metadata16:40
smoserit sucks that it is slow16:40
smoserbut it sucks *more* that boto.instance.get_instance_metadata()16:40
smosercrawls it16:40
smosergenerating dozens of hits to it.16:40
smoser(and cloud-init uses that to get all the data on boot)16:40
smoserjkoelker, would you liek a separate bug for this ?16:41
smoserand i might try to revive the patch that was submitted for bug 851159 elsewhere.16:41
uvirtbot`Launchpad bug 851159 in nova "ec2 metadata service is very slow" [High,Confirmed] https://launchpad.net/bugs/85115916:41
*** Mandell has joined #openstack-dev16:44
jkoelkeri wonder if it would speed up at all if in handler.py instead of searching we just call over get_fixed_ip16:45
jkoelkersince the only thing it does the search for is to find the instance_id16:45
*** danwent has joined #openstack-dev16:47
openstackgerritVerification of a change to openstack/nova failed: Use new style instance logging in compute api.  https://review.openstack.org/411816:50
*** deshantm has joined #openstack-dev16:52
*** ncode has joined #openstack-dev16:52
jkoelkersmoser: can ya'll try http://paste.openstack.org/show/4860/ and see if it gets better performance16:56
jkoelkerit still makes the call over to the network manager, but it bypasses doing the compute filtering and the really expensive ip search16:57
*** Rivero has joined #openstack-dev16:57
*** bepernoot has quit IRC16:59
*** littleidea has joined #openstack-dev17:00
smoserjkoelker, i pinged our IS see if he can do it.17:01
smoserability to play is somewhat limited17:01
smoseras its kind of supposed to be running17:01
jkoelkeruptime is overrated ;)17:02
*** markmc has quit IRC17:03
smoserline 13, self.network_api.get_fixed_ip should be self.network_api.get_fixed_ip_by_address17:04
smoserright?17:04
*** eglynn__ has quit IRC17:04
comstudyeah17:05
*** bsza has quit IRC17:05
*** dolphm_ has quit IRC17:06
*** eglynn__ has joined #openstack-dev17:06
*** pixelbeat has quit IRC17:09
*** hashar has quit IRC17:13
*** LinuxJedi has joined #openstack-dev17:13
openstackgerritVerification of a change to openstack/glance failed: Adds README.rst to the tarball.  https://review.openstack.org/427417:14
smoserjkoelker, it alright if i open a bug for this ? i'm just wanting to have something to track the issue so we can make sure its fixed.17:16
jkoelkersure17:16
*** hub_cap has joined #openstack-dev17:17
*** tomoe_ has quit IRC17:17
yamahatadanwent, ping?17:21
danwentyamahata: hi.  in meetings for next few hours… is it quick?17:21
danwentotherwise, please send email and I will try to get back as soon as I can17:22
yamahatadanwent, hi. I addressed your review and updated the patch.17:22
yamahataPlease look at it.17:22
danwentyamahata: great, i think salvatore will be reviewing today.  I should be able to +2 today.17:22
*** eglynn has joined #openstack-dev17:22
*** eglynn__ has quit IRC17:23
yamahatathank you in advance.17:23
danwentyamahata: really sorry for the slow reviews.  we'll definitely get this in for e-417:23
agyjkoelker: i'm trying out the patch on my test machine and it seems to work with a minor change to line 1317:24
agysmoser: ^^17:24
jkoelkerexcellent17:25
*** corXi_ has quit IRC17:25
agysmoser: have you filed the bug yet?17:25
*** jdg has joined #openstack-dev17:25
smoserno.17:26
smoseri'm opening now, agy17:27
agyok17:29
*** dovetail1an has quit IRC17:31
smoserbug https://bugs.launchpad.net/nova/+bug/93435617:33
uvirtbot`Launchpad bug 934356 in nova "get_instance_uuids_by_ip_filter is slow, making ec2 metadata slow" [Undecided,New]17:33
smoseragy, ^17:33
*** jeremy has quit IRC17:34
*** bengrue has joined #openstack-dev17:34
*** eglynn has quit IRC17:37
*** sleepsonthefloo has joined #openstack-dev17:38
*** armaan has joined #openstack-dev17:39
smoserjkoelker, bug above.17:40
armaanhi , where can i find api _key for a  devstack environment17:41
*** shevek_ has quit IRC17:46
*** ayoung has quit IRC17:47
annegentlearmaan: look in localrc17:48
*** sniperd has quit IRC17:49
*** justinsb has joined #openstack-dev17:51
*** AlanClark has joined #openstack-dev17:51
*** deshantm has quit IRC17:51
armaananne: localrc  contains17:52
armaan MYSQL_PASSWORD17:52
armaanRABBIT_PASSWORD17:52
armaanSERVICE_TOKEN17:52
armaanADMIN_PASSWORD17:52
armaanit does not have api_key17:52
*** deshantm has joined #openstack-dev17:55
*** nati2 has quit IRC17:55
*** zzed has quit IRC17:58
*** jdurgin has joined #openstack-dev18:01
*** eglynn has joined #openstack-dev18:01
*** sandywalsh has quit IRC18:01
*** armaan has left #openstack-dev18:06
annegentlethough he left, he wants SERVICE_TOKEN I believe :)18:10
*** Ryan_Lane has joined #openstack-dev18:10
*** Rivero has quit IRC18:12
*** adjohn has joined #openstack-dev18:12
*** sandywalsh has joined #openstack-dev18:15
*** hub-cap has joined #openstack-dev18:16
*** danwent_ has joined #openstack-dev18:17
*** danwent has quit IRC18:17
*** danwent_ is now known as danwent18:17
ironcamel2i'm getting 500 errors from keystone with devstack. anyone else run into this?18:17
*** cp16net has quit IRC18:18
*** hub_cap has quit IRC18:19
*** hub-cap is now known as hub_cap18:19
*** cp16net has joined #openstack-dev18:20
*** camm has joined #openstack-dev18:21
*** eglynn has quit IRC18:22
*** hub_cap has quit IRC18:25
openstackgerritVerification of a change to openstack/nova failed: Add additional information to servers output.  https://review.openstack.org/421918:25
*** hub_cap has joined #openstack-dev18:25
*** vladimir3p has joined #openstack-dev18:27
vladimir3pvish: ping18:29
vladimir3pvishy: ping :)18:29
*** darraghb has quit IRC18:34
openstackgerritVerification of a change to openstack/glance failed: Return 503 if insufficient permission on filestore  https://review.openstack.org/425118:35
vladimir3pGuys, have anybody tried to use the latest imagecache manager for compute???18:36
vladimir3pit seems like this will not work when Keystone is used (or probably our Glance/Keystone is not configured properly)18:36
vladimir3panybody from Glance team is here?18:36
*** jeblair has quit IRC18:37
*** jeblair has joined #openstack-dev18:37
*** littleidea has quit IRC18:39
*** Gordonz has joined #openstack-dev18:43
vladimir3pjaypipes: ping18:45
*** cp16net has quit IRC18:45
*** cp16net has joined #openstack-dev18:45
*** mnewby has joined #openstack-dev18:45
*** mnewby has joined #openstack-dev18:46
jkoelkersmoser, agy: I uploaded that metadata lookup change to gerrit (https://review.openstack.org/4276)18:47
vladimir3pfolks, have anybody experienced authorization issues connecting to image_service from nova as admin (using keystone)?18:48
vladimir3pis there anyody from glance/keystone team today? or anybody from nova familiar with context/tokens :-) ?18:49
*** eglynn has joined #openstack-dev18:50
*** zaitcev has joined #openstack-dev18:50
openstackgerritVerification of a change to openstack/python-keystoneclient failed: Add tenant commands to cli  https://review.openstack.org/393318:52
heckjvladimir3p: what release of keystone? Are you the newly merged trunk, or something older?18:52
vladimir3pnope, it is Diablo18:53
vladimir3pstable/diablo18:53
vladimir3pthe issues we are having is related to the fact that admin context has no clue about auth token18:53
vladimir3pif we issue any cmd through APIs - everything is great. It picks up admin token from api-paste-keystone.ini and it works18:54
vladimir3p(things like access to image details)18:54
vladimir3pbut when things like periodic_tasks are running - they have no such info and as a result could not access image service18:54
*** novas0x2a|laptop has joined #openstack-dev18:55
jaypipesvladimir3p: pong18:55
vladimir3pwe cherry-picked latest libvirt image cache managemet that runs in periodic tasks and can't make it work properly18:56
vladimir3pjaypipes: hey18:56
vladimir3pa quick question - probably we incorrectly configured glance+keystone, but it might be somethig else as well18:56
jaypipesvladimir3p: how can I help?18:56
vladimir3plet me explain the issue18:56
*** ncode has quit IRC18:56
vladimir3pif we perform any access to image service with goot context (generated through APIs, etc) - everything is fine18:57
vladimir3pespecially admin access - it picks up token from api-paste-keystone and runs with it18:57
jaypipesok18:57
vladimir3phowever, if context was generated within Nova (like get_admin...) it couldn't access glance18:58
vladimir3pand returns 401 Unauthorized18:58
vladimir3pto things like image_service.detail18:58
vladimir3pit was not an issue before, because all operation with VMs related to images were performed from user context18:59
vladimir3pbut18:59
jaypipesvladimir3p: when did things change? what commit?18:59
jaypipesor release...18:59
vladimir3pwe stated to experience problems with instance cache on hosts filling up and we cherry-picked new changes related to libvirt image cache19:00
vladimir3pJay, what I'm trying to understand is - if it is normal/expected behavior or we have just something misconfigured19:00
*** omenar has joined #openstack-dev19:00
*** ayoung has joined #openstack-dev19:01
vladimir3pthings didn't change - if context created through APIs - everything is great19:01
vladimir3pbut if context is an admin context used by periodic tasks (for example), it can't access image service19:02
vladimir3pdoes it make sense?19:02
vladimir3p(this admin context has no token inside)19:02
*** dolphm has joined #openstack-dev19:02
*** paulormg has quit IRC19:03
vladimir3pjaypipes: and another question for you - both our glance-api.conf & glance-registry.conf have admin_token. Is it only for internal glance use? do you use it when Nova's admin trying to access glance?19:06
jaypipesvladimir3p: ok, understand much better now, thx.19:11
jaypipesvladimir3p: and yes, admin_token is for communication between glance-api and glance-registyr19:12
jaypipesvladimir3p: I need to check with bcwaldon about whether Nova's admin user also needs an admin_token when talking with glance...19:12
jaypipesbcwaldon: could you read back above and advise?19:12
bcwaldonjaypipes: on it19:13
vladimir3pthanks guys19:13
openstackgerritVerification of a change to openstack/glance failed: Return 503 if insufficient permission on filestore  https://review.openstack.org/425119:13
bcwaldonjaypipes, vladimir3p: I think Nova passes along to glance whatever token is provided to it.19:14
vladimir3pyes19:14
vladimir3pbut19:14
vladimir3pin case of periodic tasks it has no token19:14
bcwaldonright, I seem to remember this issue coming up recently19:15
bcwaldonwhat 'task' is failing?19:15
jaypipesbcwaldon: does the user running the periodic task (admin?) need some token set in nova.conf?19:15
vladimir3pthe periodic tasks are part of standard manager service19:15
bcwaldonI don't there have been any periodic tasks that required talking to glance until recently19:15
bcwaldonvladimir3p: is it a specific function that is failing?19:16
vladimir3pit has it now :-)19:16
vladimir3pas I wrote before, we were trying to test new instance cache19:16
vladimir3pprobably we are the 1st ones who is trying it, because there is a typo in a code that prevents it from running at all19:16
vladimir3pbut regardless of this typo - it in general has no auth token, so it should not be able even to connect to glance19:17
vladimir3p(when Keystone is used)19:17
bcwaldonvladimir3p: yes, reading through the 'verify_base_images' function now19:18
vladimir3pyes, look at image_service = image.get_default_image_service() and for img in image_service.detail(context):19:18
bcwaldonyep, you have absolutely uncovered a bug here19:18
vladimir3p:-) seems like a conceptual bug here19:19
bcwaldonthe image service is not being created with a context, which means no authz when it hits glance19:19
vladimir3pyou mean the contex hs no proper auth token19:19
*** Gordonz has quit IRC19:19
vladimir3pis there a way to initialize image_service differently?19:20
vladimir3pand avoid auth verification?19:20
bcwaldonactually, I think we just need to either accomplish this task without the image service, or pass in an admin token in the context in periodic tasks19:21
vladimir3phow compute service could get access to auth token?19:22
bcwaldonit would have to be in nova.conf19:23
*** torgomatic has joined #openstack-dev19:23
jaypipesvladimir3p: ah, you are talking about the new *nova* image cache...19:23
vladimir3pok, is there any way to access admin_token from api-paste-keystone.ini  ?19:24
jaypipesvladimir3p: this whole time I thought you were talking about the glance image cache :(19:24
vladimir3psorry :-)19:24
vladimir3pit is instance cache19:24
jaypipesno, it's my fault...19:24
bcwaldonyes, so nova.service.Service.periodic_tasks, it creates an arbitrary admin context19:24
bcwaldonthat function, nova.context.get_admin_context, needs to produce a context with a valid admin token19:24
bcwaldonvladimir3p: I don't know offhand how difficult it would be to pull the token out of api-paste-keystone19:25
bcwaldonvladimir3p: but I am willing to be it won't be easy19:25
*** sniperd has joined #openstack-dev19:25
*** Gordonz has joined #openstack-dev19:26
vladimir3pok, so if we will add admin_token flag it and modify get_admin_context() it will help.. almost19:26
*** torgomatic has quit IRC19:27
vladimir3pit will be also required to set strategy properly (='keystone' or noauth)19:27
*** torgomatic has joined #openstack-dev19:27
vladimir3pnot sure if such info is available19:27
*** torgomatic has quit IRC19:27
bcwaldonvladimir3p: please file a bug on this asap, so others don't run into the same frustration :)19:28
vladimir3pok :-)19:29
vladimir3pthanks. I was thinking that we have something wrong with Nova/Glance/Keystone config19:29
bcwaldonvladimir3p: nope, new code that apparently wasn't tested enough beforehand19:29
vladimir3pok, great. thanks for all your help19:30
jaypipesbcwaldon: hit up mikal as well, since he wrote the instance cache code.19:31
bcwaldonmikal: around?19:31
*** hub_cap has quit IRC19:34
openstackgerritVerification of a change to openstack/nova failed: Partial fix for bug 919051  https://review.openstack.org/384219:35
uvirtbot`Launchpad bug 919051 in nova "EC2 param validation should not be in middleware" [Wishlist,In progress] https://launchpad.net/bugs/91905119:35
*** flaviamissi has quit IRC19:36
*** andrewsmedina has quit IRC19:37
*** vladimir3p has quit IRC19:39
*** breu has joined #openstack-dev19:39
*** rackerjoe has quit IRC19:41
*** breu is now known as rackerjoe19:41
*** zzed has joined #openstack-dev19:45
*** derekh has quit IRC19:55
*** adjohn has quit IRC19:55
*** hub_cap has joined #openstack-dev19:57
heckjI'm collecting keystone use cases: http://wiki.openstack.org/KeystoneUseCases19:58
heckj(email sent to openstack list encouraging contribution and feedback)19:59
Ryan_Laneany grid dynamics people here?19:59
*** sniperd has quit IRC19:59
*** andrewsmedina has joined #openstack-dev20:01
openstackgerritVerification of a change to openstack/python-keystoneclient failed: Add tenant commands to cli  https://review.openstack.org/393320:02
Ryan_Lane(if so, we have competing dns implementations, and I'd like to talk about merging, rather than competing)20:02
openstackgerritVerification of a change to openstack/nova failed: Don't tell Qpid to reconnect in a busy loop.  https://review.openstack.org/397620:04
*** spiffxp has joined #openstack-dev20:10
*** andrewbogott_afk is now known as andrewbogott20:11
*** CAP5792 has joined #openstack-dev20:15
*** bengrue has quit IRC20:15
*** omenar has quit IRC20:18
openstackgerritVerification of a change to openstack/nova failed: Support non-UTC timestamps in changes-since filter  https://review.openstack.org/421120:29
openstackgerritVerification of a change to openstack/nova failed: Support tox-based unittests.  https://review.openstack.org/387120:40
openstackgerritVerification of a change to openstack/nova failed: fix pre_block_migration() interaction with libvirt cache  https://review.openstack.org/419620:40
*** CAP5792 is now known as omenar20:41
*** lts has quit IRC20:43
*** Salv-orlando has joined #openstack-dev20:50
openstackgerritVerification of a change to openstack/nova failed: Scheduler notifications added.  https://review.openstack.org/419420:50
*** adjohn has joined #openstack-dev20:54
*** adjohn has quit IRC20:58
*** bengrue has joined #openstack-dev21:01
*** hub-cap has joined #openstack-dev21:07
*** hub-cap has joined #openstack-dev21:08
*** hub_cap has quit IRC21:11
*** hub-cap is now known as hub_cap21:11
openstackgerritVerification of a change to openstack-dev/devstack failed: Fix troubles with git cloning https  https://review.openstack.org/382521:16
*** adjohn has joined #openstack-dev21:17
*** dprince has quit IRC21:18
*** andrewbogott is now known as andrewbogott_afk21:20
*** andrewbogott has joined #openstack-dev21:21
*** vincentricci has joined #openstack-dev21:23
*** vincentricci has quit IRC21:26
*** vincentricci has joined #openstack-dev21:27
*** byeager has quit IRC21:38
openstackgerritVerification of a change to openstack/python-keystoneclient failed: Add tenant commands to cli  https://review.openstack.org/393321:39
*** adjohn has quit IRC21:40
*** Salv-orlando has quit IRC21:41
*** jog0 has joined #openstack-dev21:42
*** adjohn has joined #openstack-dev21:44
heckjbcwaldon: ping21:46
heckjdtroyer: ping21:47
dtroyeryo21:47
heckjdtroyer: heya - I'm trying to track down why you code is failing in devstack21:48
dtroyerheckj: I know what's wrong with 3933, but am having a hell of a time getting devstack to complete here21:48
dtroyerheckj: it's in the ec2_create_credentials --tenant arg.21:48
heckjdtroyer - cool - enlighten me, because i'm feeling particularly blind and ignorant after staring at the damn thing21:48
dtroyerargparse, despite trying this by hand, insists on calling --tenant an ambiguous abbreviation for --tenant_id and --tenant_name21:49
dtroyerso putting the ec2 --tenant back to --tenant_id should work based on what I've tested here21:49
heckjgot it21:49
dtroyerI'm just going to push it up21:50
heckjholler, I'll approvize it21:50
dtroyerheckj: it's there21:50
*** dolphm has quit IRC21:50
heckjdtroyer: approved21:51
*** dolphm has joined #openstack-dev21:53
Davieymtaylor: you broke my life.21:53
heckjdtroyer: where did you see the error in https://jenkins.openstack.org/job/gate-integration-tests-devstack-vm/1605/console, or am I looking at the wrong thing?21:54
heckjdtroyer: nm - oic21:54
*** markvoelker has quit IRC21:55
dtroyerheckj: the real clue to the error is when keystone_data.sh runs21:55
openstackgerritVerification of a change to openstack/nova failed: Setting access ip values on server create.  https://review.openstack.org/415521:57
*** hashar has joined #openstack-dev21:58
*** stuntmachine has quit IRC22:00
*** hub_cap has quit IRC22:01
LinuxJediDaviey: that is his job22:03
LinuxJediDaviey: what is broken today?22:04
mtaylorDaviey: what did I do?22:06
mikalbcwaldon: I'm around now. What's up?22:08
*** Guest89321 has joined #openstack-dev22:12
openstackgerritVerification of a change to openstack/nova failed: Query directly for just the ip  https://review.openstack.org/427622:13
*** shevek_ has joined #openstack-dev22:15
openstackgerritVerification of a change to openstack/nova failed: bug 929462: compile_diagnostics in xenapi erronously catch XenAPI.Failure  https://review.openstack.org/396622:16
uvirtbot`Launchpad bug 929462 in nova "compile_diagnostics in xenapi erronously catch XenAPI.Failure" [Undecided,In progress] https://launchpad.net/bugs/92946222:16
*** littleidea has joined #openstack-dev22:17
rmkDoes anyone have 2011.3.1 packages for oneiric built and available somewhere?22:18
Davieymtaylor: you broke nova building on ubuntu and debian :)22:21
Davieymtaylor: not painless,  https://github.com/openstack/nova/commit/4a4c274c834728a03bce7e5384c562321821eaf822:22
mtaylorDaviey: oh, well, ditutils.extra wasn't working ... but there's one more commit that needs to come to finish off using translation stuff more effectively22:27
Davieymtaylor: oh?22:27
openstackgerritVerification of a change to openstack/nova failed: Don't allow EC2 removal of security group in use.  https://review.openstack.org/415422:28
*** PotHix has quit IRC22:28
mtaylorDaviey: can you point me to something so I can grok what I broke for you though?22:29
mtaylorDaviey: yeah - due wasn't installing the po files at all, and it was also breaking pip installing code into virtualenvs22:29
Davieymtaylor: heard of jenkins? :) https://jenkins.qa.ubuntu.com/view/Precise%20OpenStack%20Testing/job/precise-openstack-essex-nova-trunk/382/console22:30
mtaylorDaviey: your jenkins should report things back :)22:31
mtaylorDaviey: but awesome - thanks22:31
*** armaan has joined #openstack-dev22:31
*** bsza has joined #openstack-dev22:32
mikalbcwaldon / jaypipes / vladimi3p: I think we need to be able to talk to glance in that code... The next stage of the blueprint involves pre-fetching images which are popular, and that will very much require being able to talk to glance.22:32
mikalIt sound like this might also be a more general problem? I can imagine that other periodic tasks will want admin contexts sometime in the future.22:33
*** flaviamissi has joined #openstack-dev22:33
bcwaldonmikal: it's not just having an admin context, its talking to other services with an admin token22:33
bcwaldonmikal: this happens to be the first task that needs to do that22:34
mikalSure22:34
mikalBut that doesn't mean others wont want to do it in the future22:34
bcwaldonmikal: sure, I'm not saying that22:34
mikalRefactoring that away would just delay the pain.22:34
bcwaldonit just means that in order to fix the bug, we'll need to provide the admin token in the context22:34
mikalAnd like I said, the next stage in the blueprint definitely calls for being able to make calls to glance from inside that code22:35
mikalYep, sure. I think we're agreeing here.22:35
rmkSo, release packages?  No one?22:35
rmkGuess I'm building them myself. :)22:35
mtaylorDaviey: I'll see if I can cook up a patch for you22:35
*** cp16net has quit IRC22:36
*** adjohn has quit IRC22:36
Davieymtaylor: rockin' :)22:37
Davieymtaylor: it was my intention to just moan and us fix it.. but hey, always welcome patches :)22:37
mtaylorDaviey: well, I said I'd TRY :)22:38
Davieymtaylor: Oh, you are very trying. :P22:38
* mtaylor knows it22:38
mikalbcwaldon: so, are you opposed to having an admin context in the config file, or is that the right way of fixing this problem? I haven't done enough with contexts to know.22:38
*** adjohn has joined #openstack-dev22:38
*** stuntmachine has joined #openstack-dev22:39
*** heckj has quit IRC22:40
bcwaldonmikal: no, that's a simple way to fix it22:41
mikalAhhh, ok22:42
*** vincentricci has quit IRC22:43
*** vincentricci has joined #openstack-dev22:44
*** hashar has quit IRC22:44
openstackgerritVerification of a change to openstack/nova failed: Don't allow EC2 removal of security group in use.  https://review.openstack.org/415422:44
*** rkukura has quit IRC22:46
mikalbcwaldon: I'm sure this isn't news to you, but reading this code, the context comes from nova/service.py where Service.periodic_tasks() just calls context.get_admin_context().22:48
adam_gusing the new keystone, if i'm using keystone client to create initial tenants and users, is it correct that i should be able to authenticate using only the service token in keystone.conf? or are there other crendetials i need to supply when calling tenant-create, user-create, etc?22:50
*** stuntmachine has quit IRC22:50
*** flaviamissi has quit IRC22:50
*** flaviamissi has joined #openstack-dev22:51
*** misheska has joined #openstack-dev22:52
*** rkukura has joined #openstack-dev22:52
ironcamel2comstud, johannes, https://review.openstack.org/#change,415522:52
ironcamel2oopos22:52
comstudthanks22:53
ironcamel2thank you22:53
*** rackerjoe has quit IRC22:55
*** flaviamissi has quit IRC22:56
*** flaviamissi has joined #openstack-dev22:57
openstackgerritVerification of a change to openstack/nova failed: Re-adds ssl to kombu configuration and adds flags that are needed to pass through to kombu.  https://review.openstack.org/403323:03
*** zzed has quit IRC23:04
mtaylorDaviey: main thing would just be to move files from nova/locale to /usr/share/locale after python setup.py install23:06
mtaylorDaviey: I copied the babel setup from sphinx ... so I'll look at what they do in packaging23:07
*** bsza has quit IRC23:08
mtaylorDaviey: yup. that's what they're doing23:08
comstudrussellb: around?23:10
*** dolphm has quit IRC23:11
openstackgerritVerification of a change to openstack/nova failed: Re-adds ssl to kombu configuration and adds flags that are needed to pass through to kombu.  https://review.openstack.org/403323:11
mikalWhat's with the availability of code reviewers today? It seems like sometimes I can't find any, and then today people are just all over stuff without being asked...23:14
*** flaviamissi has quit IRC23:16
Davieymtaylor: good catch, have a patch? :)23:16
*** flaviamissi has joined #openstack-dev23:16
mtaylorDaviey: will do23:17
mtaylorDaviey: is lp:~ubuntu-openstack-pacakgers/nova/trunk still the right place? or should I do lp:ubuntu/nova ?23:17
*** armaan has left #openstack-dev23:19
adam_gmtaylor: id say propose to lp:~ubuntu-server-dev/nova/essex, we use those branches for every nova build we test on CI23:21
adam_gDaviey: what do you think?23:21
Davieyerm23:22
Davieymtaylor: where adam said23:22
DavieyThose branches form the base of CI and what actually gets uploaded.23:22
*** Gordonz has quit IRC23:23
adam_ghaving all these branches is still weird to me, but ~ubuntu-server-dev is the quickest path to an automated test, second to directly proposing to the ~openstack-ubuntu-testing/ branches23:23
mtayloradam_g: I will just propose to ~ubuntu-server-dev23:24
*** mdomsch_ has joined #openstack-dev23:27
mtayloradam_g: https://code.launchpad.net/~mordred/nova/fix-locale/+merge/9366523:29
mtaylorDaviey: ^^23:29
*** Guest89321 has quit IRC23:31
adam_gmtaylor: thanks23:37
openstackgerritVerification of a change to openstack/nova failed: Support non-UTC timestamps in changes-since filter  https://review.openstack.org/421123:39
*** AlanClark has quit IRC23:39
*** rsampaio has quit IRC23:40
*** mdomsch_ has quit IRC23:42
Davieymtaylor: thanks!23:43
*** hashar has joined #openstack-dev23:54
*** flaviamissi has quit IRC23:55
*** flaviamissi has joined #openstack-dev23:55
bodepddid the way that ec2 credentials work just completely change in keystone?23:55
bodepd(or maybe the way that devstack deploys keystone?23:56
adam_gbodepd: yeah23:56
bcwaldonbodepd: the project was completely replaced with a new codebase a couple of nights ago23:56
bodepdadam_g: I just spent way too long trying to figure that out :) I finally just pasted the access key and secret key from the database to my ~/.fog file.23:57
*** flaviamissi has quit IRC23:57
*** Turicas has joined #openstack-dev23:57
*** kmansel has quit IRC23:58
adam_gbodepd: when was the last time you had it working?  theres been a lot of breakage for people since this: https://github.com/openstack/keystone/commit/eef1f0d93ae19f04601b75cd7a2514e81b4005b923:58
bodepdI had it working with devstack earlier in the week. Just spent most of the day trynig to figure out why I couldnt auth.23:59
*** hashar has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!