Thursday, 2017-04-27

*** diablo_rojo has joined #openstack-meeting-cp00:04
*** gouthamr has joined #openstack-meeting-cp00:47
*** sdague has quit IRC00:58
*** diablo_rojo has quit IRC01:30
*** gouthamr has quit IRC01:42
*** gouthamr has joined #openstack-meeting-cp02:51
*** gouthamr has quit IRC04:01
*** eeiden has quit IRC04:08
*** eeiden has joined #openstack-meeting-cp04:33
*** MarkBaker has quit IRC07:03
*** brault has joined #openstack-meeting-cp07:37
*** MarkBaker has joined #openstack-meeting-cp08:14
*** sheeprine has quit IRC08:23
*** sheeprine has joined #openstack-meeting-cp08:27
*** MarkBaker has quit IRC09:16
*** eeiden has quit IRC09:18
*** eeiden has joined #openstack-meeting-cp09:35
*** MarkBaker has joined #openstack-meeting-cp10:47
*** sdague has joined #openstack-meeting-cp11:04
*** markvoelker_ has joined #openstack-meeting-cp11:11
*** markvoelker has quit IRC11:12
*** MarkBaker has quit IRC11:37
*** markvoelker_ has quit IRC11:38
*** MarkBaker has joined #openstack-meeting-cp11:39
*** r1chardj0n3s has left #openstack-meeting-cp12:01
*** MarkBaker has quit IRC12:04
*** MarkBaker has joined #openstack-meeting-cp12:26
*** markvoelker has joined #openstack-meeting-cp12:35
*** Guest94155 has quit IRC13:10
*** bswartz has joined #openstack-meeting-cp13:19
*** jaugustine has joined #openstack-meeting-cp13:36
*** diablo_rojo has joined #openstack-meeting-cp13:46
*** lamt has joined #openstack-meeting-cp13:49
*** diablo_rojo has quit IRC14:27
*** gouthamr has joined #openstack-meeting-cp14:31
*** MarkBaker has quit IRC14:54
*** MarkBaker has joined #openstack-meeting-cp15:06
*** zigo has quit IRC15:17
*** markvoelker has quit IRC15:30
*** markvoelker has joined #openstack-meeting-cp15:31
ildikov#startmeeting cinder-nova-api-changes16:00
openstackMeeting started Thu Apr 27 16:00:12 2017 UTC and is due to finish in 60 minutes.  The chair is ildikov. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: cinder-nova-api-changes)"16:00
openstackThe meeting name has been set to 'cinder_nova_api_changes'16:00
ildikovDuncanT ameade cFouts johnthetubaguy jaypipes takashin alaski e0ne jgriffith tbarron andrearosa hemna erlon mriedem gouthamr ebalduf patrickeast smcginnis diablo_rojo gsilvis  xyang1 raj_singh lyarwood breitz jungleboyj16:00
*** mriedem has joined #openstack-meeting-cp16:00
mriedemo/16:00
jungleboyjo/16:00
hemna\o16:00
johnthetubaguyo/16:01
ildikovhi :)16:01
smcginniso/16:01
ildikovok, let's jump in the middle16:02
stvnoyeso/16:02
ildikovmost of the ongoing items are here: https://review.openstack.org/#/q/topic:bp/cinder-new-attach-apis16:02
*** lamt has quit IRC16:03
ildikovstvnoyes updated shutdown_instance: https://review.openstack.org/#/c/456877/16:03
ildikovand local_cleanup: https://review.openstack.org/#/c/456851/16:03
ildikovwith tests16:03
ildikovthose two are standalone ones, plz review and let us know if there's anything else that should be addressed there16:04
johnthetubaguyso the etherpad is a touch out of date I guess16:04
mriedemwe should slow down because we have some open questions on some existing patches16:04
ildikovI added comments about attachment_update in the chain mriedem picked up starting with terminate_volume_connections: https://review.openstack.org/#/c/456896/16:04
johnthetubaguy#link https://etherpad.openstack.org/p/pike-nova-priorities-tracking16:04
mriedemi.e. johnthetubaguy is wondering in https://review.openstack.org/#/c/456896/ if we should delete the attachment, or update the attachment to set connector=None16:05
mriedemand i have those exact same questions in https://review.openstack.org/#/c/456971/ for swap volume16:05
johnthetubaguyso I fear its not that simple16:05
ildikovjohnthetubaguy: would adding the topic there for sub-team review make sense?16:05
johnthetubaguyildikov: I think it is there16:06
ildikovmriedem: so you cannot update the attachment with connector=None today16:06
ildikovas attachment_update blows up in that case16:06
johnthetubaguyso if we look at terminate connection, I think there are two things we could be doing16:07
ildikovthe idea what we also started to follow is to create a new attachment and delete the old one16:07
johnthetubaguy(1) deleting, (2) moving16:07
jgriffithildikov +116:07
jgriffithWe made the decision to stop moving16:07
ildikovjohnthetubaguy: terminate connection always terminates the connection regardless of calling detach or not16:08
jgriffithAttachment-update at one point was actually attachment-finalize but that was changed16:08
johnthetubaguyso moving I am talking about is a create and delete (in that order)16:08
johnthetubaguyildikov: the problem is we need to make sure the new attachment is created first16:08
*** pewp has joined #openstack-meeting-cp16:08
ildikovjohnthetubaguy: if we have a new attachment for the new volume/host that should cover the not calling detach case I think16:08
johnthetubaguyso delete attachment is replicating both terminate + detach16:08
johnthetubaguyits not like we can just delete attachments where we used to terminate16:08
*** mgiles has joined #openstack-meeting-cp16:09
johnthetubaguyildikov: it can, its messy, but it can16:09
ildikovjgriffith: I start to feel sorry for insisting to switch to 'update' from 'finalize'...:/16:09
jgriffithjohnthetubaguy we can certainly do the create and then delete if we need to16:09
mriedemone of the questions i had in the swap volume patch was,16:09
jgriffithildikov haha.. nah16:09
mriedemwill migrate_volume_completion handle that we deleted the old attachment?16:09
mriedemretype gets pretty damn nutty16:10
johnthetubaguywhere is that again?16:11
mriedemjohnthetubaguy: https://review.openstack.org/#/c/456971/5/nova/compute/manager.py@503016:11
mriedemi have TODOs in there asking what i should be doing wrt delete vs update16:11
mriedemsounds like i can't do update with connector=None so that's my answer16:12
mriedembtw, we really need to get these APIs documented16:12
ildikovmriedem: so where you call update, we can call delete there as you created a new attachment already16:12
mriedemjungleboyj: do you have minions that can document these APIs?16:12
mriedemfor 3.2716:12
ildikovwhich reserves the volume just like when you didn't call detach with the old flow16:12
jungleboyjmriedem: I have no minions right now.  :-(16:13
ildikovmriedem: what would be the purpose of update in the swap case there?16:13
ildikovmriedem: there's a patch up there to document the new attach/detach API16:13
jungleboyjsmcginnis: tommylikehu has been working on some of that, right?16:13
ildikovjungleboyj: yes, that's what I'm referring to, just don't have the link handy16:14
smcginnisjungleboyj: There have been various api-ref updates.16:14
johnthetubaguymriedem: ah, I see what you mean... I am confused in all that code16:14
ildikovAPI docs patch: https://review.openstack.org/#/c/451252/16:15
mriedemjohnthetubaguy: i wrote it and i'm confused16:15
jgriffithMriedem jungleboyj I’ll look at updating the docs we have here https://github.com/openstack/cinder/blob/master/doc/source/devref/attach_detach_conventions_v2.rst16:15
tommylikehu:)16:15
smcginnisildikov: Thanks, I could not find that patch.16:15
mriedemildikov: the purpose of the update_attachment call with connector=None was me trying to terminate the connection but keep the attachment around for migrate_volume_completion16:15
mriedemildikov: but that's why i had a todo, because i didn't know what i as the client should be doing16:15
johnthetubaguyso I remember tracing back migrate_volume_completion now, and its kinda "bi-modal" depending on if cinder started the migration16:16
smcginnisjgriffith: Can you look at that api-ref patch and make sure it's in sync with your view?16:16
tommylikehusmcginnis: +116:16
mriedemjgriffith: to be clear i'm looking for it here https://developer.openstack.org/api-ref/block-storage/v3/16:16
ildikovmriedem: what does migrate_volume_completion do exactly?16:16
jgriffithmriedem ildikov tommylikehu so if you have an alternate idea I’m open, but my intent here was that we don’t re-use attachments any more.  They’re disposable things16:16
hemnajgriffith, +116:16
mriedemjgriffith: i'm fine with not re-using it, i just didn't know16:16
mriedemhence the todo to ask16:16
jgriffithThat way we eliminate the need for any funny reference counting or corner cases on keeping things around and updating them16:17
johnthetubaguyildikov: its a crazy cinder API that is a Nova call back during a volume migrate16:17
jungleboyjjgriffith: Cool, thanks.16:17
jgriffithmriedem yeah, I got ya, just explaining my view and opening it for discussion if needed16:17
mriedemildikov: yeah it's nova telling cinder that we completed our side of the retype operatoin16:17
mriedemildikov: and we say if it failed on the nova side or not16:17
mriedemso that cinder can rollback16:17
mriedemthe new volume16:17
johnthetubaguyildikov: cinder API: volume migrate -> nova API: swap volume -> cinder API: migrate_volume_completion16:17
johnthetubaguyI think16:17
mriedemjohnthetubaguy: correct16:17
mriedemplus 30 sub-api calls in between :)16:18
johnthetubaguyalso... user creates volume and calls nova API: swap volume -> cinder API: migrate_volume_completion16:18
* johnthetubaguy nods16:18
ildikovjgriffith: I don't want to re-use them either, but we need to figure out how to get a new flow with some old things like this migrate_volume_completion stuff... :/16:18
ildikovmriedem: johnthetubaguy: that's a little bit nuts... :)16:18
johnthetubaguyI think in the first case, the volume_uuid is swapped, but its not swapped in the second case, or something like that16:18
* ildikov wishes we could simplify that somehow...16:19
tommylikehubtw, migrate_volume_completion is not ready for multi attach16:19
jgriffithildikov `git revert`16:20
johnthetubaguytommylikehu: +10016:20
ildikovtommylikehu: I think currently it might not even ready for single attach... :(16:20
hemnaheh16:20
johnthetubaguyyeah, I am not totally sure how that all works with the new API, now you mention it16:21
ildikovjgriffith: we keep the new flow for sure, just need to clean up some more old stuff16:21
johnthetubaguymriedem: is there an easier place to start, like migrate?16:21
ildikovjgriffith: and that's non-negotiable :)16:21
johnthetubaguyI just feel like if we did an easier one, it would be clearer whats missing/odd16:22
ildikovjohnthetubaguy: the two standalone things are ready for review I mentioned at the beginning of the meeting16:22
ildikovjohnthetubaguy: the last patch in that chain is migrate and I couldn't really get there to give much thought to it16:22
ildikovjohnthetubaguy: so we can play with that16:22
johnthetubaguyyeah, not seen those patches before now16:23
ildikovchanging the order of patches is simple too16:23
ildikovjohnthetubaguy: post_live_migrate: https://review.openstack.org/#/c/456988/16:23
mriedemtommylikehu: we aren't supporting anything with multiattach in nova yet16:24
mriedemjohnthetubaguy: i generally don't combine "easier" and "migrate"16:24
johnthetubaguyso doing post_live-migrate seems hard if you haven't done the first bit16:24
johnthetubaguymriedem: me neither, seemed strange saying that16:24
mriedemfeels dirty16:25
ildikovmriedem: lol, my thinking too :)16:25
ildikovjohnthetubaguy: so far we tried to find where we call terminate and replace that16:25
ildikovas we're starting with detach16:26
tommylikehumriedem: nova is going to support right?16:26
johnthetubaguyyeah, I guess migrate does bring attach into it16:26
mriedemtommylikehu: nova won't support multiattach until all of the operations are using the new attach flow16:26
tommylikehumriedem:  that make sense16:26
johnthetubaguywell, and thats because we need the new flow to be able to support it safely, AFAICT16:27
johnthetubaguyso this one is nice actually: https://review.openstack.org/#/c/45687716:27
ildikovjohnthetubaguy: that's correct16:27
johnthetubaguyits really clear, attachment_delete === terminate + detach16:27
ildikovyeah, these corner cases are tough like swap and migrate where we showed the volume attached even when it technically wasn't anymore...16:29
johnthetubaguyright, its the reason we are doing the change, telling cinder what is really happening16:30
johnthetubaguyso multi-attach is possible16:30
*** gouthamr has quit IRC16:31
ildikovok, so now we need to look into the migrate_volume_completion in Cinder and figure out what it does with the volume to see how it can work with the new flow or what we need to do to have the new flow support swap16:32
ildikovI would think we don't want update to have the connector=None option here as it does not seem to be useful for swap here anyhow16:32
mriedemildikov: ok so i'll review your comments and change the attachment_update connector=None stuff to delete16:33
ildikovmriedem: that should work, I'll look into the Cinder side16:33
ildikovis there anyone here from the Cinder team, who's familiar with that particular piece in question?16:33
ildikovin case I would go nuts for some reason while looking into it :)16:34
smcginnisWhich? Sorry, too many streams going right now.16:34
ildikovsmcginnis: migrate_volume_completion16:34
hemnaildikov, we should eliminate the connector=None wherever we can16:35
hemnathat simply doesn't work for most backends in cinder.16:35
ildikovhemna: the Cinder side returns an HTTPBadRequest for that right now16:35
ildikovI mean attachment_update does return that16:35
hemnaok phew16:36
ildikov:)16:36
mriedemhemna: to be clear, nova creates an attachment initially with connector=None16:37
mriedemby design16:37
mriedemto reserve that volume16:37
mriedemwe provide the connector once we get it from brick on the compute16:37
hemnaas part of the new API right?  but that attach call never makes it to the cinder driver afaik16:37
smcginnishemna: Yep, just creates in DB.16:38
hemnaok cool16:38
ildikovmriedem: that does not go down to the drivers16:38
hemnasorry I was thinking the old API16:38
ildikovmriedem: that's basically a DB record at that point16:38
hemnare: calling terminate_connection with a None connector16:38
hemnaignore me16:38
ildikovhemna: ah ok, no, we definitely didn't want to do that :)16:39
mriedemi'll adjust my patches then16:39
ildikovmriedem: I'll confirm the migrate_volume_completion part as soon as I figured it out16:39
*** gouthamr has joined #openstack-meeting-cp16:40
ildikovI also started to clean up the very old attach PoC so we have something to test with16:40
mriedemok16:40
johnthetubaguyso, I have an idea that may help us here16:41
johnthetubaguythinking about this one: https://review.openstack.org/#/c/45687716:42
johnthetubaguyit does terminate + detach16:42
johnthetubaguyor delete attachment16:42
johnthetubaguylets make that a single method16:42
johnthetubaguyits basically detach_from_host_and_volume16:42
johnthetubaguyor detach(keep_attached=False)16:43
johnthetubaguyso if you do dettach(keep_volume_attached_to_instance=False)16:43
johnthetubaguyI think thats a better name16:43
smcginnisdetach(keep_attached) seems a little funny, but yeah16:43
johnthetubaguyall be it too long16:44
johnthetubaguydetach_from_host(also_detatch_volume=True)16:44
hemnajohnthetubaguy, where the call to os-brick.disconnect_volume in that flow?16:44
ildikovwell, we use the terminate + detach like that in 1.5 places16:44
hemna(gerrit url)16:44
johnthetubaguyhemna: its before it I think16:44
* hemna is looking16:44
johnthetubaguyhttps://review.openstack.org/#/c/456877/2/nova/compute/manager.py@226116:45
ildikovand the keep_attached piece is tricky as sometimes we have a new volume, in the other case we have a new host16:45
*** knangia has joined #openstack-meeting-cp16:45
johnthetubaguyhemna: dang, I wonder if its missing?16:45
ildikovso I wouldn't go for consolidation at thi spoint16:45
hemnajohnthetubaguy, looks like it is16:45
johnthetubaguyildikov: maybe, but here is the next bit16:46
hemnaI don't mean to derail this, but we are having a discussing in the nova channel about the races between calling brick's disconnect_volume and calling terminate_connection16:46
johnthetubaguyso detach_from_host(also_detatch_volume=True) deletes the attachment16:46
hemnawhich is why I was looking for the disconnect_volume call in that patch16:46
johnthetubaguydetach_from_host(also_detatch_volume=False) creates a new attachment, then deletes the old one16:46
mriedemjohnthetubaguy: you're trying to combine what shutdown_instance and https://review.openstack.org/#/c/456896/5/nova/compute/manager.py is doing right?16:46
johnthetubaguymriedem: yeah16:47
johnthetubaguywell, I am thinking, what method could we replace terminate connection with16:47
johnthetubaguybut rather, its what do we replace terminate connection with AND terminate + detach with16:47
mriedemwe aren't creating a new attachment in shutdown_instance though16:47
ildikovhemna: we only changed terminate_connection, so wherever disconnect is I don't think we made anything worse16:47
hemnahehe16:48
*** lamt has joined #openstack-meeting-cp16:48
ildikovjohnthetubaguy: as a matter of fact I started with something like that with terminate_volume_connections having a param keep_attached16:48
johnthetubaguyhemna: sounds like those races should get shook out of all this work, if we do it correctly16:48
hemnaI'm hoping so16:49
ildikovjohnthetubaguy: but going into the details it seemed like a bad idea at this point16:49
ildikovjohnthetubaguy: while we're trying to figure out the individual flows16:49
hemnaI don't want to have to deal with a callback from os-brick to nova16:49
hemnanor calling cinder16:49
johnthetubaguyildikov: maybe, but this is much worse: https://review.openstack.org/#/c/456896/516:49
mriedemjohnthetubaguy: i don't see how that is worse16:50
mriedemi was -1 on the earlier patches in that series which were trying to do things that we didn't need to do yet16:50
mriedemplus it was all one big change which was hard to review16:51
ildikovmriedem: yeah, I was referring to the don't need to do those yet part here16:51
mriedemuntil we actually have to write something that does an attach and then detach, can we save it for then?16:51
johnthetubaguymriedem: so isn't _terminate_volume_connections used in loads of places?16:51
mriedemjohnthetubaguy: it's used in 2 places16:52
ildikovyep, that's what I remember too16:52
mriedemresize and revert_resize16:52
johnthetubaguymriedem: OK, so thats probably where I am getting confused16:52
mriedemwe call terminate_connection separately in swap volume16:52
mriedemand shutdown_instance16:52
mriedemand probably live migration16:52
mriedemand cold migration16:52
mriedemand foobarbilly16:53
johnthetubaguyyeah16:53
hemnajohnthetubaguy, https://github.com/openstack/nova/blob/master/nova/compute/api.py#L214916:53
hemnathis also looks like a place where terminate_connection is called w/o calling brick to disconnect16:54
hemna:(16:54
mriedemthat's local delete folks16:54
mriedemyou can't call brick b/c the compute is dead,16:54
johnthetubaguyhemna: thats because the host is dead16:54
mriedemor the instance is shelved offloaded16:54
hemnaok16:54
hemnaphew16:54
hemnaI'm just scouring through nova code16:54
mriedemwe also stash the connector since mitaka or something16:54
mriedemand pass that in if we have it16:54
mriedemhttps://github.com/openstack/nova/blob/master/nova/compute/api.py#L215716:54
johnthetubaguyso, we have really been around the houses today, 5 mins left, whats next?16:55
mriedemi have to drop for another call16:55
johnthetubaguysounds like I need to look at the other patches in this series, and re-read some of this code again16:56
johnthetubaguyafter that, it should be a bit clearer to me I hope16:56
johnthetubaguysounds like something fun for a friday morning I guess16:56
ildikovjohnthetubaguy: if we could get local_cleanup and shutdown_instance done quickly that would be great16:57
ildikovand then deal with the chain16:57
ildikovalso plz look into remove check_detach as it's just hanging there lonely at this point16:57
ildikovI will add attach to the etherpads once it's in working state16:57
ildikovI mean the attach PoC, not for merging but if anyone wants to pick it up for testing, etc.16:58
johnthetubaguyI see we have a spec update up for check_detach16:58
ildikovjohnthetubaguy: and begin_detaching16:59
ildikovI have a question there for the latter16:59
ildikovwe can chat about that on the review16:59
johnthetubaguyyeah, I added follow on questions16:59
ildikovok, cool, tnx16:59
johnthetubaguywe need to resolve begin_detaching before we do check_detach I think16:59
johnthetubaguyat least, I think resolving that would stop my concerns16:59
ildikovjohnthetubaguy: it does the checks regardless17:00
johnthetubaguymy problem is finding a place for begin_detaching in the new world17:00
johnthetubaguyif its still there, the checks can die17:00
mriedemjohnthetubaguy: did you see my spec update for begin_detaching?17:00
ildikovimprovements in the volume state machine is independent IMHO17:00
johnthetubaguymriedem: yeah, thats the patch we are discussing, I added some questions17:00
johnthetubaguyso if we keep using begin_detaching with the new flow (no multi-attach), I think we are all good17:01
ildikovjohnthetubaguy: as I see it comes before we get to the new world17:01
ildikovright now we still do, no one touched that call17:01
johnthetubaguyI think we probably have agreed that now, once thats agreed I am fine with the check_detach patch17:02
johnthetubaguyildikov: totally17:02
johnthetubaguyI think we are talking past each other on this one, I am 99% sure I have the same aims as you, just a few extra worries17:02
ildikovok, we can finish agreeing on the spec17:02
johnthetubaguyyeah, the problem is we loose the protection when we get multiple attachments17:03
ildikovjohnthetubaguy: sometimes you worry more than my mother and she's really good at it! :)17:03
johnthetubaguythe protection against two detach calls being made to one volume, basically17:03
johnthetubaguyildikov: I can out worry quite a few mothers, its a bit of a worry17:03
ildikovwell, in Pike we will not get there to have multiple attachments anyhow17:04
johnthetubaguyanyways, I guess we are done, timewise17:04
ildikovjohnthetubaguy: lol :)17:04
ildikovyeah, let's take this discussion to the spec review17:04
ildikovthank y'all for today!17:04
johnthetubaguyyup, yup17:04
ildikovC U on Gerrit and next week here!17:05
ildikov#endmeeting17:05
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"17:05
openstackMeeting ended Thu Apr 27 17:05:28 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:05
openstackMinutes:        http://eavesdrop.openstack.org/meetings/cinder_nova_api_changes/2017/cinder_nova_api_changes.2017-04-27-16.00.html17:05
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/cinder_nova_api_changes/2017/cinder_nova_api_changes.2017-04-27-16.00.txt17:05
openstackLog:            http://eavesdrop.openstack.org/meetings/cinder_nova_api_changes/2017/cinder_nova_api_changes.2017-04-27-16.00.log.html17:05
*** harlowja has quit IRC17:08
*** mgiles has quit IRC17:11
*** mriedem has left #openstack-meeting-cp17:13
*** MarkBaker has quit IRC17:21
*** harlowja has joined #openstack-meeting-cp17:51
*** pewp has quit IRC17:51
*** pewp has joined #openstack-meeting-cp17:52
*** MarkBaker has joined #openstack-meeting-cp18:07
*** ying_zuo has joined #openstack-meeting-cp18:24
*** Daviey_ has joined #openstack-meeting-cp18:38
*** Daviey has quit IRC18:38
*** eglute has quit IRC18:38
*** dhellmann has quit IRC18:38
*** eglute has joined #openstack-meeting-cp18:39
*** dhellmann has joined #openstack-meeting-cp18:39
*** jaugustine has quit IRC18:59
*** jaugustine has joined #openstack-meeting-cp19:00
*** rdopiera has joined #openstack-meeting-cp19:00
*** MarkBaker has quit IRC19:07
*** jaugustine has quit IRC19:11
*** rdopiera has left #openstack-meeting-cp19:35
*** knangia has quit IRC19:51
*** MarkBaker has joined #openstack-meeting-cp19:55
*** ying_zuo has left #openstack-meeting-cp20:14
*** jaugustine has joined #openstack-meeting-cp20:19
*** zigo has joined #openstack-meeting-cp21:32
*** jaugustine has quit IRC21:49
*** kbyrne has quit IRC21:50
*** kbyrne has joined #openstack-meeting-cp21:54
*** raj_sing- has joined #openstack-meeting-cp22:14
*** jungleboyj has quit IRC22:14
*** harlowja has quit IRC22:14
*** raj_singh has quit IRC22:14
*** nikhil has quit IRC22:17
*** soren has quit IRC22:18
*** nikhil has joined #openstack-meeting-cp22:26
*** soren has joined #openstack-meeting-cp22:26
*** diablo_rojo has joined #openstack-meeting-cp23:03
*** sdague has quit IRC23:08
*** lamt has quit IRC23:09
*** diablo_rojo has quit IRC23:17
*** harlowja has joined #openstack-meeting-cp23:23
*** markvoelker has quit IRC23:40
*** jungleboyj has joined #openstack-meeting-cp23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!