Skip to content

Commit

Permalink
add performance tests
Browse files Browse the repository at this point in the history
  • Loading branch information
theKashey committed Feb 20, 2018
1 parent fa1e1f7 commit ed08b1c
Show file tree
Hide file tree
Showing 6 changed files with 353 additions and 1 deletion.
96 changes: 95 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,101 @@ But memoize-state could disable another underlying memoizations libraries.

Uses `ES6 Proxy` underneath to detect used branches of a state (as `MobX`).
Removes all the magic from result value.
Should be slower than "manual" __reselect__ors, but faster than anything else.
Should be slower than "manual" __reselect__ors, but faster than anything else.

We have a performance test, according to the results -
- memoize-state __is not slower__ than major competitors, and __10-100x times faster__, for the "state" cases.
- lodash.memoize and fast-memoize could not handle __big__ states as input.
- memoize-one should be super fast, but it is not

But the major difference is
- memoize-one are having __highest hitratio__, than means - it were able to "memoize" most of the cases
```text
function of 3 arguments, all unchanged
memoize-one x 6563316 ops/sec ±1.50% (6 runs sampled) hitratio 100% 2 /2432667
lodash.memoize x 2988552 ops/sec ±3.98% (5 runs sampled) hitratio 100% 1 /3844342
fast-memoize x 1007010 ops/sec ±1.18% (6 runs sampled) hitratio 100% 1 /4443629
memoize-state x 3753869 ops/sec ±1.33% (6 runs sampled) hitratio 100% 1 /6175599
Fastest is memoize-one
function of 2 arguments, providing 3, all unchanged
memoize-one x 6077327 ops/sec ±1.53% (6 runs sampled) hitratio 100% 2 /2534824
lodash.memoize x 2780103 ops/sec ±1.88% (6 runs sampled) hitratio 100% 1 /3615601
fast-memoize x 928385 ops/sec ±4.59% (6 runs sampled) hitratio 100% 1 /3998562
memoize-state x 3389800 ops/sec ±1.51% (6 runs sampled) hitratio 100% 1 /5243823
Fastest is memoize-one
function of 3 arguments, all changed / 10
memoize-one x 19043 ops/sec ±1.15% (6 runs sampled) hitratio 50% 4083 /8163
lodash.memoize x 30242 ops/sec ±1.45% (5 runs sampled) hitratio 79% 6114 /28541
fast-memoize x 17442 ops/sec ±0.78% (6 runs sampled) hitratio 92% 2891 /34322
memoize-state x 16621 ops/sec ±2.16% (6 runs sampled) hitratio 92% 3225 /40772
Fastest is lodash.memoize
function with an object as argument, returning a part
memoize-one x 8822 ops/sec ±1.39% (5 runs sampled) hitratio 0% 4337 /4337
lodash.memoize x 1378671 ops/sec ±8.92% (6 runs sampled) hitratio 100% 1 /669631
fast-memoize x 1027750 ops/sec ±6.03% (6 runs sampled) hitratio 100% 1 /1246719
memoize-state x 1207975 ops/sec ±2.08% (6 runs sampled) hitratio 100% 1 /1805336
Fastest is lodash.memoize
function with an object as argument, changing value, returning a part
memoize-one x 8236 ops/sec ±1.54% (6 runs sampled) hitratio 0% 4112 /4112
lodash.memoize x 74548 ops/sec ±4.14% (6 runs sampled) hitratio 91% 4106 /45160
fast-memoize x 71851 ops/sec ±2.60% (6 runs sampled) hitratio 96% 3524 /80393
memoize-state x 61650 ops/sec ±1.28% (6 runs sampled) hitratio 98% 2632 /106706
Fastest is lodash.memoize,fast-memoize
function with an object as argument, changing other value, returning a part
memoize-one x 7683 ops/sec ±1.78% (6 runs sampled) hitratio 0% 3488 /3488
lodash.memoize x 69976 ops/sec ±2.08% (6 runs sampled) hitratio 91% 3086 /34339
fast-memoize x 66844 ops/sec ±2.26% (6 runs sampled) hitratio 96% 2308 /57408
memoize-state x 1085455 ops/sec ±2.39% (6 runs sampled) hitratio 100% 1 /399267
Fastest is memoize-state
function with 2 objects as argument, changing both value
memoize-one x 7251 ops/sec ±7.62% (6 runs sampled) hitratio 0% 3263 /3263
lodash.memoize x 7255 ops/sec ±3.94% (5 runs sampled) hitratio 56% 2591 /5855
fast-memoize x 7197 ops/sec ±2.57% (6 runs sampled) hitratio 64% 3341 /9197
memoize-state x 45612 ops/sec ±5.28% (6 runs sampled) hitratio 94% 1487 /24063
Fastest is memoize-state
when changes anything, except the function gonna to consume
memoize-one x 8003 ops/sec ±0.99% (5 runs sampled) hitratio 0% 3453 /3453
lodash.memoize x 7640 ops/sec ±2.47% (6 runs sampled) hitratio 50% 3490 /6944
fast-memoize x 7315 ops/sec ±1.38% (5 runs sampled) hitratio 73% 2576 /9521
memoize-state x 461062 ops/sec ±37.16% (6 runs sampled) hitratio 100% 1 /270082
Fastest is memoize-state
when state is very big, and you need a small part
memoize-one x 7713 ops/sec ±2.21% (6 runs sampled) hitratio 0% 3518 /3518
lodash.memoize x 198 ops/sec ±2.48% (6 runs sampled) hitratio 100% 10 /3613
fast-memoize x 201 ops/sec ±1.54% (6 runs sampled) hitratio 100% 9 /3688
memoize-state x 61350 ops/sec ±2.49% (6 runs sampled) hitratio 91% 3072 /34404
Fastest is memoize-state
```

### Even more speed

```js
function fn1(object) {
return object.value
}

// ^^ memoize state will react to any change of .value

function fn2(object) {
return {...object.value}
}

// ^^ memoize state will react to any change of the values inside the .value

// for example, if value contain booleans the X and they Y - they form 4 possible pairs
const superMemoize = memoize(fn2, { cacheSize: 4 });

// ^^ you just got uber function, which will return 4 exactly the same objects
```


## Compatibility

Expand Down
229 changes: 229 additions & 0 deletions _tests/perf.spec.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
import {expect} from 'chai';
import Benchmark from 'benchmark';
import memoizeOne from 'memoize-one';
import lmemoize from 'lodash.memoize';
import fmemoize from 'fast-memoize';
import memoizeState from '../lib/index';

describe('performance test', () => {

function toStringBench() {
var bench = this,
hz = bench.hz,
stats = bench.stats,
size = stats.sample.length,
pm = '\xb1',
result = bench.name + new Array(16 - bench.name.length).join(' '),
n = "" + hz.toFixed(0);

n = new Array(16 - n.length).join(' ') + n;

result += ' x ' + n + ' ops/sec ' + pm +
stats.rme.toFixed(2) + '% (' + size + ' run' + (size == 1 ? '' : 's') + ' sampled)';

return result;
}

let complexRun = 0;

function complexFunction() {
complexRun++;
for (let i = 0; i < 10000; i++) {
Math.random();
}
return Math.random();
}

function test(fun, methods, done) {
const mo = memoizeOne(fun);
const ml = lmemoize(fun, args => JSON.stringify(args));
const ms = memoizeState(fun, {strictArity: true});
const mf = fmemoize(fun);

const suite = new Benchmark.Suite();

const options = {
maxTime: 0.1
};

complexRun = 0;
let localRun = 0;
suite
.add('memoize-one', () => mo.apply(localRun++ && null, methods()), options)
.add('lodash.memoize', () => ml.apply(localRun++ && null, methods()), options)
.add('fast-memoize', () => mf.apply(localRun++ && null, methods()), options)
.add('memoize-state', () => ms.apply(localRun++ && null, methods()), options)

.on('cycle', function (event) {
console.log(toStringBench.call(event.target), ' hitratio', 100 - Math.round(100 * complexRun / localRun) + '%', complexRun, '/' + localRun);
complexRun = 0;
localRun++;
})
.on('complete', function () {
console.log('Fastest is ' + this.filter('fastest').map('name'));
done();
})
// run async
.run({
async: false
});
}

it('compare simple function', (done) => {
console.log('function of 3 arguments, all unchanged');
test(
function (a, b, c) {
complexFunction();
return a + b + c
},
() => [1, 2, 3],
done
)
}).timeout(10000);

it('compare simple function', (done) => {
console.log('function of 2 arguments, providing 3, all unchanged');
test(
function (a, b) {
complexFunction();
return a + b
},
() => [1, 2, 3],
done
)
}).timeout(10000);

it('compare unique params function', (done) => {
console.log('function of 3 arguments, all changed / 10');
let counter = 0;
test(
function (a, b, c) {
complexFunction();
return a + b + c
},
() => [Math.round(counter++ / 10), Math.round(counter++ / 10), Math.round(counter++ / 10)],
done
)
}).timeout(10000);

it('compare function with object as argument, returning value from object', (done) => {
console.log('function with an object as argument, returning a part');
test(
function (a) {
complexFunction();
return a.data
},
() => [{data: 1}],
done
)
}).timeout(10000);

it('compare function with object as argument, returning value from object, and changing value', (done) => {
console.log('function with an object as argument, changing value, returning a part');
let counter = 0;
test(
function (a) {
complexFunction();
return a.data
},
() => [{data: Math.round(counter++ / 10)}],
done
)
}).timeout(10000);

it('compare function with object as argument, returning value from object, and changing not used value', (done) => {
console.log('function with an object as argument, changing other value, returning a part');
let counter = 0;
test(
function (a) {
complexFunction();
return a.data
},
() => [{data: 1, somethingElse: Math.round(counter++ / 10)}],
done
)
}).timeout(10000);

it('compare function with 2 object as argument, returning value from object, and changing not used value', (done) => {
console.log('function with 2 objects as argument, changing both value');
let counter = 0;
test(
function (a, b) {
complexFunction();
return a.b.c + b.c.d.e
},
() => {
counter++;
return [{
b: {
x: counter,
c: Math.round(counter / 10),
}
}, {
c: {
d: {
x: counter,
e: Math.round(counter / 10),
}
}
}]
},
done
)
}).timeout(10000);

it('when changes anything, except the function gonna to consume', (done) => {
console.log('when changes anything, except the function gonna to consume');
let counter = 0;
test(
function (a, b) {
complexFunction();
return a.b.c + b.c.d.e
},
() => {
counter++;
return [{
b: {
x: counter,
c: 1,
},
c: {
d: {
x: counter,
e: 3,
}
}
}, {
c: {
d: {
x: counter,
e: 3,
}
}
}]
},
done
)
}).timeout(10000);

it('when state is very big, and you need a small part', (done) => {
console.log('when state is very big, and you need a small part');
const A = new Array(100000).fill(1);
let counter = 0;
test(
function (a) {
complexFunction();
return a.value
},
() => {
counter+=0.1;
return [{
A,
value: Math.round(counter)
}]
},
done
);
}).timeout(10000);

}).timeout(10000);
1 change: 1 addition & 0 deletions lib/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ function callIn(that, cache, args, func, memoizationDepth) {
if (!map) {
return proxyMap[index] = (0, _proxyequal.proxyState)(arg, index);
}
map.reset();
return map.replaceState(arg);
}
return undefined;
Expand Down
4 changes: 4 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,12 @@
"babel-cli": "^6.26.0",
"babel-eslint": "^8.2.1",
"babel-preset-env": "^1.6.1",
"benchmark": "^2.1.4",
"chai": "^4.1.2",
"eslint": "^4.18.0",
"fast-memoize": "^2.3.0",
"lodash.memoize": "^4.1.2",
"memoize-one": "^3.0.1",
"mocha": "^4.0.1",
"sinon": "^4.3.0"
},
Expand Down
1 change: 1 addition & 0 deletions src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ function callIn(that, cache, args, func, memoizationDepth, proxyMap = []) {
if (!map) {
return proxyMap[index] = proxyState(arg, index);
}
map.reset();
return map.replaceState(arg);
}
return undefined
Expand Down
Loading

0 comments on commit ed08b1c

Please sign in to comment.