diff --git a/README.md b/README.md index 4cb9d88..d31cf18 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ Upgrading from an older version? Please see the [Breaking Change History](#break # Basic Usage -Require and instantiate superagent-cache as follows to get the [default configuration](#what-does-the-default-configuraiton-give-me): +Require and instantiate superagent-cache as follows to get the [default configuration](#what-does-the-default-configuration-give-me): ```javascript var superagent = require('superagent-cache')(); ``` @@ -69,7 +69,7 @@ By default, `superagent-cache` stores data in a bundled instance of [cacheModule # What Does the Default Configuration Give Me? -You get the 'default configurations' when you don't provide any params to the `require('superagent-cache')()` command. This will return a fresh instance of `superagent` and bundle an instance of [cacheModule](https://github.com/jpodwys/cache-service-cache-module) for storing data. `cacheModule` is a slim, in-memory cache. +You get the 'default configuration' when you don't provide any params to the `require('superagent-cache')()` command. This will return a fresh instance of `superagent` and bundle an instance of [cacheModule](https://github.com/jpodwys/cache-service-cache-module) for storing data. `cacheModule` is a slim, in-memory cache. # How Do I Use a Custom Configuration? @@ -230,7 +230,7 @@ superagent ## .expiration(seconds) -Use this function when you need to override all of your caches' `defaultExpiration` properties (set via cache-service) for a particular cache entry. +Use this function when you need to override your `cache`'s `defaultExpiration` property for a particular cache entry. #### Arguments @@ -252,6 +252,16 @@ Tell superagent-cache whether to perform an ajax call if the generated cache key * bool: boolean, default: true +## .backgroundRefresh(value) + +> See the [Using Background Refresh](#using-background-refresh) section for more information. + +Tell the underlying `cache` provided in the `require` command to enable background refresh for the generated key and value. If a function is provided, it will use the function, if a boolean is provided, it will use the boolean, if nothing is provided, it will default to true. + +#### Arguments + +* value: boolean || function || undefined, default: true + ## ._end(callback (err, response)) This is a convenience method that allows you to skip all caching logic and use superagent as normal. @@ -270,6 +280,77 @@ This is the second constructor param you handed in when you instantiated `supera superagent.cache... //You can call any function existing on the cache you passed in ``` +# Using Background Refresh + +With a typical cache setup, you're left to find the perfect compromise between having a long expiration so that users don't have to suffer through the worst case load time, and a short expiration so data doesn't get stale. `superagent-cache` eliminates the need to worry about users suffering through the longest wait time by automatically refreshing keys for you. + +#### How do I turn it on? + +By default, background refresh is off. It will turn itself on the first time you use the `.backgroundRefresh)` chainable. + +#### Setup + +`superagent-cache` relies on the background refresh feature of the `cache` param you pass into the `require` command. When you use the `.backgroundRefresh()` chainable, `superagent-cache` passes the provided value into `cache`. This means that if you're using `cache-service`, you almost certainly want `cache-service`'s `writeToVolatileCaches` property set to `true` (it defaults to `true`) so that the data set by background refresh will propogate forward to earlier caches (`cache-service` ONLY background refreshses to the final cache passed to it) + +#### Configure + +If desired, configure the following properties within `cache`: + +* `backgroundRefreshInterval` +* `backgroundRefreshMinTtl` +* `backgroundRefreshIntervalCheck` + +#### Use + +Background refresh is exposed via the `.backgroundRefresh()` chainable. + +When `true` or no param is passed to `.backgroundRefresh()`, it will generate a `superagent` call identical to the one that triggered it and pass that to `cache`. + +```javascript +superagent + .get(uri) + .backgroundRefresh() + .end(function (err, response){ + //Response will no be refreshed in the background + } +); +``` + +When a function is passed, it will use that function. Read on for background refresh function requirements. + +```javascript +var refresh = function(key, cb){ + var response = goGetData(); + cb(null, response); +} + +superagent + .get(uri) + .backgroundRefresh(refresh) + .end(function (err, response){ + //Response will no be refreshed in the background + } +); +``` + +When `false` is passed, it will do nothing. + +#### The Refresh Param + +###### refresh(key, cb(err, response)) + +* key: type: string: this is the key that is being refreshed +* cb: type: function: you must trigger this function to pass the data that should replace the current key's value + +The `refresh` param MUST be a function that accepts `key` and a callback function that accepts `err` and `response` as follows: + +```javascript +var refresh = function(key, cb){ + var response = goGetData(); + cb(null, response); +} +``` + # More Usage Examples ## .end() callback argument list options @@ -324,13 +405,3 @@ var superagent = require('superagent-cache')(null, redisCache); #### 0.2.0 * `superagent-cache` is now more flexible, allowing usage of any cache that matches `cache-service`'s API. To make it lighter, then, the hard dependency on `cache-service` was replaced with the much lighter `cacheModule`. As a result, `superagent-cache` can no longer construct a `cache-service` instance for you. If you wish to use `cache-service`, you must instantiate it externally and hand it in as `cache`--the second param in the `require` command. - -# Roadmap - -* ~~Make it so superagent-cache's `.end()` callback function does not require an `err` param~~ -* ~~Make sure that `resetProps()` gets called when `._end()` is called directly~~ -* ~~Add unit tests for the various ways headers can be added to calls~~ -* ~~Add the 'More Usage Examples' section~~ -* ~~Remove the hard dependency on `superagent-cache` and allow users to use any cache that matched `superagent-cache`'s API~~ -* Add unit tests for the other points above -* Add thorough comments and param descriptions to the code diff --git a/package.json b/package.json index bce3a6f..6ba7c1e 100644 --- a/package.json +++ b/package.json @@ -1,11 +1,11 @@ { "name": "superagent-cache", - "version": "0.2.1", + "version": "1.0.0", "description": "Superagent with flexible built-in caching.", "main": "superagentCache.js", "dependencies": { "superagent": "1.1.0", - "cache-service-cache-module": "1.0.0" + "cache-service-cache-module": "1.1.0" }, "devDependencies": { "mocha": "1.13.0", diff --git a/superagentCache.js b/superagentCache.js index 84b046a..7fd82e0 100644 --- a/superagentCache.js +++ b/superagentCache.js @@ -1,3 +1,9 @@ +/** + * superagentCache constructor + * @constructor + * @param {superagent instance} agent (optional) + * @param {cache module} cache (optional) + */ module.exports = function(agent, cache){ var superagent = (agent) ? agent : require('superagent'); @@ -14,48 +20,95 @@ module.exports = function(agent, cache){ superagent.cache = new cModule(); } + /** + * Whether to execute an http query if the cache does not have the generated key + * @param {boolean} doQuery + */ Request.prototype.doQuery = function(doQuery){ props.doQuery = doQuery; return this; } + /** + * Remove the given params from the query object after executing an http query and before generating a cache key + * @param {array of strings} pruneParams + */ Request.prototype.pruneParams = function(pruneParams){ props.pruneParams = pruneParams; return this; } + /** + * Remove the given options from the headers object after executing an http query and before generating a cache key + * @param {boolean} pruneOptions + */ Request.prototype.pruneOptions = function(pruneOptions){ props.pruneOptions = pruneOptions; return this; } + /** + * Execute some logic on superagent's http response object before caching and returning it + * @param {function} prune + */ Request.prototype.prune = function(prune){ props.prune = prune; return this; } + /** + * Retrieve a top-level property from superagent's http response object before to cache and return + * @param {string} responseProp + */ Request.prototype.responseProp = function(responseProp){ props.responseProp = responseProp; return this; } + /** + * Set an expiration for this key that will override the configured cache's default expiration + * @param {integer} expiration (seconds) + */ Request.prototype.expiration = function(expiration){ props.expiration = expiration; return this; } + /** + * Whether to cache superagent's http response object when it "empty"--especially useful with .prune and .pruneParams + * @param {string} responseProp + */ Request.prototype.cacheWhenEmpty = function(cacheWhenEmpty){ props.cacheWhenEmpty = cacheWhenEmpty; return this; } + /** + * Initialize a background refresh for the generated key and value + * @param {boolean | function} backgroundRefresh + */ + Request.prototype.backgroundRefresh = function(backgroundRefresh){ + props.backgroundRefresh = (typeof backgroundRefresh !== 'undefined') ? backgroundRefresh : true; + return this; + } + + /** + * An alias for the .end function because I use ._end and .end for other things + */ Request.prototype.execute = Request.prototype.end; + /** + * Wraps the .end function so that .resetProps gets called--callable so that no caching logic takes place + */ Request.prototype._end = function(cb){ resetProps(); this.execute(cb); } + /** + * Execute all caching and http logic + * @param {function} cb + */ Request.prototype.end = function(cb){ var curProps = props; resetProps(); @@ -81,7 +134,11 @@ module.exports = function(agent, cache){ response = gutResponse(response); } if(!isEmpty(response) || curProps.cacheWhenEmpty){ - superagent.cache.set(key, response, curProps.expiration, function(){ + var refresh = curProps.backgroundRefresh || null; + if(typeof refresh == 'boolean'){ + refresh = getBackgroundRefreshFunction(curProps); + } + superagent.cache.set(key, response, curProps.expiration, refresh, function (){ callbackExecutor(cb, err, response, key); }); } @@ -114,10 +171,18 @@ module.exports = function(agent, cache){ } } + /** + * Set this.req to null so that future http calls get a branc new req object + */ Request.prototype.reset = function(){ this.req = null; } + /** + * Generate a cache key unique to this query + * @param {object} reg + * @param {object} cProps + */ function keygen(req, cProps){ var cleanParams = null; var cleanOptions = null; @@ -125,21 +190,26 @@ module.exports = function(agent, cache){ if(!params && req.req){ params = stringToObj(req.req.path); } - var options = (req.req && req.req._headers) ? req.req._headers : {}; + var options = (req.req && req.req._headers) ? req.req._headers : null; if(cProps.pruneParams || cProps.pruneOptions){ cleanParams = (cProps.pruneParams) ? pruneObj(cloneObject(params), cProps.pruneParams) : params; cleanOptions = (cProps.pruneOptions) ? pruneObj(cloneObject(options), cProps.pruneOptions, true) : options; } return JSON.stringify({ nameSpace: superagent.cache.nameSpace, + method: req.method, uri: req.url, - params: cleanParams || params || {}, - options: cleanOptions || options || {} + params: cleanParams || params || null, + options: cleanOptions || options || null }); } + /** + * Convert an array to an object + * @param {array} arr + */ function arrayToObj(arr){ - if(arr){ + if(arr && arr.length){ var obj = {}; for(var i = 0; i < arr.length; i++){ var str = arr[i]; @@ -154,6 +224,10 @@ module.exports = function(agent, cache){ return null; } + /** + * Convert a string to an object + * @param {string} str + */ function stringToObj(str){ if(str){ var obj = {}; @@ -171,6 +245,12 @@ module.exports = function(agent, cache){ return null; } + /** + * Remove properties from an object + * @param {object} obj + * @param {array} props + * @param {boolean} isOptions + */ function pruneObj(obj, props, isOptions){ for(var i = 0; i < props.length; i++){ var prop = props[i]; @@ -182,6 +262,10 @@ module.exports = function(agent, cache){ return obj; } + /** + * Simplify superagent's http response object + * @param {object} r + */ function gutResponse(r){ var newResponse = {}; newResponse.body = r.body; @@ -193,10 +277,18 @@ module.exports = function(agent, cache){ return newResponse; } + /** + * Determine whether a value is considered empty + * @param {*} val + */ function isEmpty(val){ return (val === false || val === null || (typeof val == 'object' && Object.keys(val).length == 0)); } + /** + * Return a cloneof an object + * @param {object} obj + */ function cloneObject(obj){ var newObj = {}; for(var attr in obj) { @@ -207,10 +299,47 @@ module.exports = function(agent, cache){ return newObj; } + /** + * Reset superagent-cache's default query properties + */ function resetProps(){ props = {doQuery: true, cacheWhenEmpty: true}; } + /** + * Generate a background refresh query identical to the current query + * @param {object} curProps + */ + function getBackgroundRefreshFunction(curProps){ + return function(key, cb){ + key = JSON.parse(key); + var method = key.method.toLowerCase(); + var request = superagent + [method](key.uri) + .doQuery(curProps.doQuery) + .pruneParams(curProps.pruneParams) + .pruneOptions(curProps.pruneOptions) + .prune(curProps.prune) + .responseProp(curProps.responseProp) + .expiration(curProps.expiration) + .cacheWhenEmpty(curProps.cacheWhenEmpty); + if(key.params){ + request.query(key.params) + } + if(key.options){ + request.set(key.options); + } + request.end(cb); + } + } + + /** + * Handle the varying number of callback output params + * @param {function} cb + * @param {object} err + * @param {object} response + * @param {string} key + */ function callbackExecutor(cb, err, response, key){ if(cb.length === 1){ cb(response); @@ -226,13 +355,17 @@ module.exports = function(agent, cache){ } } + /** + * Instantates an exception to be thrown + * @param {string} name + * @param {string} message + * @return {exception} + */ function exception(name, message){ this.name = name; this.message = message; } - var noop = function(){} - if(!agent){ return superagent; } diff --git a/test/server/superagent-cache.js b/test/server/superagent-cache.js index 2dedf54..3c80220 100644 --- a/test/server/superagent-cache.js +++ b/test/server/superagent-cache.js @@ -1,7 +1,9 @@ var expect = require('expect'); var express = require('express'); var superagent = require('superagent'); -require('../../superagentCache')(superagent); +var cModule = require('cache-service-cache-module'); +var cacheModule = new cModule({backgroundRefreshInterval: 500}); +require('../../superagentCache')(superagent, cacheModule); var app = express(); @@ -120,7 +122,7 @@ describe('Array', function(){ expect(result).toBe(null); done(); }); - }, 10); + }, 20); } ); }); @@ -172,7 +174,7 @@ describe('Array', function(){ expect(key.indexOf('otherParams')).toBeGreaterThan(-1); done(); } - ) + ); }); it('.get() .query(string&string) .pruneParams() .end() should query with all params but create a key without the indicated params', function (done) { @@ -187,7 +189,7 @@ describe('Array', function(){ expect(key.indexOf('otherParams')).toBeGreaterThan(-1); done(); } - ) + ); }); it('.get() .query(string) .query(string) .pruneParams() .end() should query with all params but create a key without the indicated params', function (done) { @@ -321,4 +323,131 @@ describe('Array', function(){ }); + describe('superagentCache background refresh tests', function () { + + it('.get() .expiration() .end() background refresh should not work if the chainable is not used', function (done) { + superagent + .get('localhost:3000/one') + .expiration(1) + .end(function (err, response, key){ + expect(typeof key).toBe('string'); + expect(response.body.key).toBe('one'); + setTimeout(function(){ + superagent.cache.get(key, function (err, response, key){ + expect(response).toBe(null); + done(); + }); + }, 1500); + } + ); + }); + + it('.get() .expiration() .backgroundRefresh(true) .end() background refresh should refresh a key shortly before expiration', function (done) { + superagent + .get('localhost:3000/one') + .expiration(1) + .backgroundRefresh(true) + .end(function (err, response, key){ + expect(typeof key).toBe('string'); + expect(response.body.key).toBe('one'); + setTimeout(function(){ + superagent.cache.get(key, function (err, response){ + expect(response.body.key).toBe('one'); + done(); + }); + }, 1500); + } + ); + }); + + it('.get() .query(string&string) .expiration() .end() background refresh should not work if the chainable is not used', function (done) { + superagent + .get('localhost:3000/params') + .query('pruneParams=true&otherParams=false') + .pruneParams(['pruneParams']) + .end(function (err, response, key){ + expect(response.body.pruneParams).toBe('true'); + expect(response.body.otherParams).toBe('false'); + expect(key.indexOf('pruneParams')).toBe(-1); + expect(key.indexOf('otherParams')).toBeGreaterThan(-1); + setTimeout(function(){ + superagent.cache.get(key, function (err, response){ + expect(response).toBe(null); + done(); + }); + }, 1500); + } + ); + }); + + it('.get() .query(string&string) .expiration() .backgroundRefresh(true) .end() background refresh should refresh a key shortly before expiration', function (done) { + superagent + .get('localhost:3000/params') + .query('pruneParams=true&otherParams=false') + .pruneParams(['pruneParams']) + .backgroundRefresh(true) + .end(function (err, response, key){ + expect(response.body.pruneParams).toBe('true'); + expect(response.body.otherParams).toBe('false'); + expect(key.indexOf('pruneParams')).toBe(-1); + expect(key.indexOf('otherParams')).toBeGreaterThan(-1); + setTimeout(function(){ + superagent.cache.get(key, function (err, response){ + expect(response.body.pruneParams).toBe('true'); + expect(response.body.otherParams).toBe('false'); + expect(key.indexOf('pruneParams')).toBe(-1); + expect(key.indexOf('otherParams')).toBeGreaterThan(-1); + done(); + }); + }, 1500); + } + ); + }); + + it('.get() .expiration() .backgroundRefresh(function) .end() background refresh should refresh a key shortly before expiration', function (done) { + var refresh = function(key, cb){ + cb(null, {body:{key: 'one'}}); + } + + superagent + .get('localhost:3000/one') + .expiration(1) + .backgroundRefresh(refresh) + .end(function (err, response, key){ + expect(typeof key).toBe('string'); + expect(response.body.key).toBe('one'); + setTimeout(function(){ + superagent.cache.get(key, function (err, response){ + expect(response.body.key).toBe('one'); + done(); + }); + }, 1500); + } + ); + }); + + it('.get() .expiration() .backgroundRefresh(function) .end() background refresh should refresh a key shortly before expiration', function (done) { + var refresh = function(key, cb){ + cb(null, {body:{key: 'two'}}); + } + + superagent + .get('localhost:3000/one') + .expiration(1) + .backgroundRefresh(refresh) + .end(function (err, response, key){ + expect(typeof key).toBe('string'); + expect(response.body.key).toBe('one'); + setTimeout(function(){ + superagent.cache.get(key, function (err, response){ + expect(response.body.key).toBe('two'); + done(); + }); + }, 1500); + } + ); + }); + + }); + });