Filter

Filter is a token item filter based on FilterBase. It selects objects from a stream ignoring the rest. Its filter is called for all value tokens. A shape of incoming stream is preserved: when selecting a subobject, all parent objects will be recreated to keep a stream valid.

In order to recreate parent objects correctly this filter requires that an upstream produced packed keys (keyValue tokens).

Introduction

const Filter = require('stream-json/filters/Filter');
const fs = require('fs');

// our data stream:
// {total: 123456789, meta: {...}, data: [...]}
// we want to remove all properties but 'data':
// {data: [...]}

const pipeline = fs.createReadStream('sample.json')
  .pipe(Filter.withParser({filter: 'data'}));

pipeline.on('data', data => console.log(data));

API

Filter has no special API. Based on FilterBase it uses the following options:

• filter
  • If it returns a truthy value, the current object is streamed out.
    • If we have any unselected parent objects, they will be recreated.
    • Any subobject will have a chance to be filtered.
  • It is called only for the following tokens: startObject, startArray, startString, startNumber, stringValue, numberValue, nullValue, trueValue, falseValue.
• pathSeparator
• once

See their definitions in FilterBase.

Static methods and properties

filter(options) and make(options)

make() and filter() are two aliases of the factory function. It takes options described above, and return a new instance of Filter. filter() helps to reduce a boilerplate when creating data processing pipelines:

const {chain}  = require('stream-chain');
const {parser} = require('stream-json/Parser');
const {filter} = require('stream-json/filters/Filter');

const fs = require('fs');

const pipeline = chain([
  fs.createReadStream('sample.json'),
  parser(),
  filter({filter: 'data'})
]);

let objectCounter = 0;
pipeline.on('data', data => data.name === 'startObject' && ++objectCounter);
pipeline.on('end', () => console.log(`Found ${objectCounter} objects.`));

make.Constructor

Constructor property of make() (and filter()) is set to Filter. It can be used for indirect creating of filters or metaprogramming if needed.

withParser()

withParser() takes one argument:

• options is an object described in Parser's options. It is used to initialize both streams (a Parser instance and a stream returned by make()).

It returns a stream produced by stream-chain, which wraps the pipeline. The most important utility of withParser() is that it correctly sets object modes of the returned stream: object mode for the Readable part and text mode for the Writable part.

This static method is created using withParser() utility. It simplifies a case when a stream should be immediately preceded by a parser.

const Filter = require('stream-json/filters/Filter');
const fs = require('fs');

const pipeline = fs.createReadStream('sample.json')
  .pipe(Filter.withParser({filter: 'data'}));

let objectCounter = 0;
pipeline.on('data', data => data.name === 'startObject' && ++objectCounter);
pipeline.on('end', () => console.log(`Found ${objectCounter} objects.`));