| *** valw has joined #craton | 03:11 | |
| *** valw has quit IRC | 03:15 | |
| *** valw has joined #craton | 03:18 | |
| *** valw has quit IRC | 04:26 | |
| *** tojuvone has joined #craton | 07:57 | |
| *** tojuvone_ has joined #craton | 07:57 | |
| *** tojuvone_ has quit IRC | 07:57 | |
| *** tojuvone has quit IRC | 11:37 | |
| sigmavirus | jimbaker: so | 13:03 |
|---|---|---|
| sigmavirus | It seems like people would be okay adding cachecontrol to g-r (or at least support it part of the way), | 13:04 |
| sigmavirus | That said, CacheControl follows the RFC rather strictly, so we'd need to start sending back the appropriate headers (which I don't think Flask does for you out of the box) | 13:04 |
| sigmavirus | Do we want to stick to dogpile.cache's naïve approach for now and then swap out later? We have an API that is dogpile agnostic | 13:05 |
| sulo | g-r ? | 13:07 |
| sulo | also how is cache invalidated right now ? | 13:08 |
| sulo | adding right headers is prolly not too bad but then we need to add proper etags for the client | 13:12 |
| sulo | or probably like last-mod header, which is probably better for our usecase | 13:15 |
| sigmavirus | sulo: g-r is shorthand (openstack) for global-requirements.txt (in openstack/requirements) | 13:21 |
| sigmavirus | sulo: cache is not invalidated right now, this is one of the downsides I mentioned on the patch | 13:21 |
| sigmavirus | (Cache can be expired though) | 13:22 |
| sulo | sigmavirus: ah gotcha | 13:38 |
| sigmavirus | So we have something of a chicken-egg problem, in that OSA wants caching today /but/ it'll take longer for that to be probable unless we go for the short-term solution of using dogpile | 13:43 |
| jimbaker | sigmavirus, sulo, from what i can tell, CacheControl is orthogonal to dogpile. we want both in our solution | 13:47 |
| jimbaker | re flask specifics - anything beyond http://werkzeug.pocoo.org/docs/0.11/datastructures/#werkzeug.datastructures.RequestCacheControl ? | 13:51 |
| sigmavirus | jimbaker: I'm honestly not sure. Haven't done the research | 14:01 |
| sigmavirus | jimbaker: why do you think we want both dogpile.cache and cachecontrol? | 14:02 |
| jimbaker | sigmavirus, this assumes that we are differentiating between objects in the ORM that are being cached; vs any returned queries | 14:03 |
| sigmavirus | jimbaker: so, I'm purely talking about the client side here | 14:03 |
| sigmavirus | I don't think we should be doing caching on the craton-api side | 14:04 |
| sigmavirus | dogpile (as best as I can tell) will need quite a bit of work for it to recognize (and then invalidate) somethign that modifies something already in the cache | 14:04 |
| sigmavirus | It works based off of function arguments and is at best a library for memoization, not really caching | 14:04 |
| sigmavirus | the name is not apt | 14:04 |
| jimbaker | sigmavirus, sure, dogpile is only relevant to the server side | 14:05 |
| jimbaker | and anything that's interesting in terms of consistency requires some means to expire cache entries | 14:06 |
| jimbaker | on the client side, if we implement cache control for the api, we can take advantage with CacheControl. which makes a lot of sense to me | 14:07 |
| sigmavirus | I can take a look at generating the cache headers then today | 14:09 |
| jimbaker | sigmavirus, +1 | 14:09 |
| jimbaker | also was reading up on recent work on providing in-memory alternatives to the usual mysql, such as NDB | 14:37 |
| *** syed__ has joined #craton | 14:42 | |
| jimbaker | sigmavirus, sulo, syed__, others here, we will be meeting in #openstack-meeting-4 in 10 minutes | 14:50 |
| *** valw has joined #craton | 15:01 | |
| *** Mudpuppy has joined #craton | 15:02 | |
| *** valw has quit IRC | 15:21 | |
| *** valw has joined #craton | 15:25 | |
| *** valw has quit IRC | 15:27 | |
| *** valw has joined #craton | 15:28 | |
| *** valw_ has joined #craton | 15:30 | |
| *** valw has quit IRC | 15:32 | |
| *** logan- has quit IRC | 15:54 | |
| *** logan- has joined #craton | 16:01 | |
| jimbaker | sulo, so potentially we can also leverage https://blueprints.launchpad.net/craton/+spec/craton-notifications for cache invalidation as well | 16:04 |
| sulo | jimbaker: serveside yes | 16:04 |
| jimbaker | yes, server side :) | 16:04 |
| jimbaker | sigmavirus will have the cache invalidation working quite nicely on client side this week - should be straightforward with a decent library | 16:05 |
| jimbaker | sulo, although i could see a dashboard benefiting from a such service | 16:06 |
| jimbaker | or rather, functionality | 16:06 |
| sulo | https://github.com/openstack/oslo.messaging | 16:06 |
| sulo | jimbaker: yeah its meant for dashboard | 16:06 |
| sulo | i should write more complete description | 16:07 |
| sulo | so its meants to sync dashboard view one(more) craton instances running | 16:07 |
| *** valw_ has quit IRC | 16:13 | |
| *** valw has joined #craton | 16:13 | |
| *** jovon has joined #craton | 16:32 | |
| *** valw has quit IRC | 18:03 | |
| *** valw has joined #craton | 18:05 | |
| *** valw has quit IRC | 18:42 | |
| *** valw has joined #craton | 19:23 | |
| *** Dusty has joined #craton | 19:45 | |
| sulo | sooo | 19:47 |
| sulo | adding docker to test increases the test time by atleast 5 mins | 19:47 |
| sulo | it takes on average about 5 mins to get the container setup | 19:47 |
| sigmavirus | sulo: yeah, I don't like the idea of using docker for this | 19:48 |
| sigmavirus | At least with RPC, I can almost guarantee we won't use docker to deploy craton | 19:49 |
| sulo | sigmavirus: yeah, maybe we need to look at something else for testing | 19:49 |
| jimbaker | sulo, so are you building the container incrementally? | 19:54 |
| *** Dusty has quit IRC | 19:54 | |
| jimbaker | i would assume that using a layer that's just the mysql data from the gen fake cloud, plus starting mysql + craton api would be much less than 5 min | 19:55 |
| *** Dusty has joined #craton | 19:55 | |
| jimbaker | still need to deal with docker's terrible tail distribution, but that still is on the order of a minute or so from what i have seen. may just be a mac thing too | 19:56 |
| sulo | build does not include data at all | 19:57 |
| sulo | this is simply to build the initial image | 19:57 |
| jimbaker | sulo, so that's fine. we should be able to just keep an initial image, pre startup of mysql/craton api | 19:57 |
| jimbaker | then use that as a starting point for subsequent containers | 19:58 |
| sulo | same thing .. its only fast by few seconds | 19:58 |
| sulo | since that image is X10 larger | 19:58 |
| *** Dusty has quit IRC | 19:59 | |
| jimbaker | sulo, ok... i guess we can just start mysql separately with fresh fixture data. another thing to do is to avoid fsync - http://www.tocker.ca/2013/11/04/reducing-mysql-durability-for-testing.html | 19:59 |
| jimbaker | sulo, also we should start mysql separately; i haven't done this, but presumably http://dev.mysql.com/doc/refman/5.7/en/environment-variables.html provides enough control to get different port assignment | 20:02 |
| jimbaker | if we cannot containerize, let's at least treat it more or less like it is :) | 20:03 |
| *** Mudpuppy has quit IRC | 20:08 | |
| *** Dusty has joined #craton | 20:08 | |
| sigmavirus | jimbaker: we're installing craton into a virtualenv on the container. Using a virtualenv elsewhere is treating it like we treat our containers, no? | 20:37 |
| sigmavirus | Also, I think I'm going to work on a blueprint for the caching things because I think there are complexities here that need careful consideration | 20:38 |
| jimbaker | sigmavirus, iirc, we are not installing craton into a virtualenv in the provided Dockerfile | 20:41 |
| jimbaker | but virtualenv is generally good, right? | 20:41 |
| jimbaker | sigmavirus, +1 re blueprint for client-side caching. can also ref with respect to what we want do on the server side | 20:42 |
| jimbaker | which are two separable concerns | 20:42 |
| sigmavirus | jimbaker: https://github.com/openstack/craton/blob/master/Dockerfile#L58..L73 | 20:43 |
| sigmavirus | sigmavirus: I mean a blue print for the server side | 20:43 |
| sigmavirus | the client side is a simple as tossing cachecontrol into the mix | 20:43 |
| jimbaker | sigmavirus, nm, obviously the Dockerfile states what it actually is ;) | 20:43 |
| jimbaker | vs poor flawed human memory | 20:44 |
| sigmavirus | no worries :) | 20:49 |
| sulo | ok i got the build time down to 3 mins | 21:09 |
| sulo | we might be able to further bring it down a bit | 21:09 |
| sulo | but i guess the question is | 21:09 |
| sulo | do we care that we need 3 mins to setup to run functional tests ? | 21:10 |
| sulo | i wonder how other projects are doing it | 21:12 |
| sulo | we can always do tempest | 21:14 |
| sulo | but i kinda liked this approach with docker | 21:14 |
| sigmavirus | tempest is orthogonal to docker | 21:15 |
| sigmavirus | i have to run though, later y'all | 21:15 |
| *** Mudpuppy has joined #craton | 21:15 | |
| sulo | sigmavirus: but doesnt tempest need eithr devstack or some sort of vm configured to run its test ? | 21:16 |
| sulo | sigmavirus: also i thought there was an effort to move away from tempest | 21:17 |
| *** Mudpuppy_ has joined #craton | 21:22 | |
| palendae | sulo: iirc tempest will simply hit addresses/ports it's pointed at, even if they're on the local machine | 21:24 |
| palendae | I don't know for certain, but I think devstack is used by most projects on their gates simply to get everything up locally quickly | 21:24 |
| palendae | However, OSA doesn't use devstack at all, but still runs tempest | 21:25 |
| *** Mudpuppy has quit IRC | 21:25 | |
| palendae | As to whether or not tempest is the future solution, I have no idea. | 21:25 |
| sulo | palendae: ok | 21:27 |
| *** valw has quit IRC | 21:32 | |
| *** ahsa518 has joined #craton | 21:35 | |
| *** valw has joined #craton | 21:36 | |
| jimbaker | sulo, 3 min is not bad | 21:47 |
| jimbaker | and if we can bring under tempest, that's cool too. we probably should look at how OSA does it - more likely going to be closer to what we need to do in terms of a custom fit | 21:47 |
| jimbaker | if we can run *it* under | 21:48 |
| jimbaker | sulo, at the very least, let's take a look at that branch! | 21:48 |
| *** rainya has joined #craton | 21:53 | |
| *** valw has quit IRC | 22:08 | |
| *** jovon has quit IRC | 22:17 | |
| *** jovon has joined #craton | 22:26 | |
| *** rainya has quit IRC | 22:52 | |
| *** Dusty has quit IRC | 23:17 | |
| *** ahsa518 has quit IRC | 23:39 | |
| *** Mudpuppy_ has quit IRC | 23:44 | |
| *** Mudpuppy has joined #craton | 23:44 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!