@coorpacademy/squirrel

Local mirror mecanism for ETCD

Stats

stars šŸŒŸissues āš ļøupdated šŸ› created šŸ£size šŸ‹ļøā€ā™€ļø
03Apr 4, 2020Nov 19, 2015Minified + gzip package size for @coorpacademy/squirrel in KB

Readme

Squirrel

Local mirror mecanism for ETCD

npm Build Status codecov

Keep a replication of ETCD folder locally for low latency querying. Provide an index system to access a file without scanning all nodes.

Summary

Install

$ npm install --save @coorpacademy/squirrel
import createSquirrel from '@coorpacademy/squirrel';

Usage

Node Interface

const squirrel = createSquirrel({
    hosts: 'http://localhost:2379',
    auth: null,
    ca: null,
    key: null,
    cert: null,

    cwd: '/',
    fallback: '/tmp/squirrel.json',
    indexes: ['foo', 'bar.baz']
});

Options:

  • hosts: ETCD hosts. more
  • auth: A hash containing {user: "username", pass: "password"} for basic auth. more
  • ca: Ca certificate. more
  • key: Client key. more
  • cert: Client certificate. more
  • cwd: ETCD current working directory.
  • fallback: Temporary file to save ETCD backup.
  • indexes: Array of key to index.

Methods

Consider the following folder:

/
ā”œā”€ā”€ bar
ā”‚Ā Ā  ā””ā”€ā”€ baz   { "bar": { "baz": "qux" } }
ā””ā”€ā”€ foo       { "foo": "bar" }

get(path)

Get file by path. Returns Promise;

  • path (String): the path of the file to get.
const foo = await squirrel.get('/foo');
console.log(foo); // { "foo": "bar" }

const barBaz = await squirrel.get('/bar/baz');
console.log(barBaz); // { "bar": { "baz": "qux" } }

getBy(index, key)

Get by index value. Returns Promise;

  • index (String): the path of the property to get. It needs to be declared in the indexes option
  • key (String): the value to match
const foo = await squirrel.getBy('foo', 'bar');
console.log(foo); // { "foo": "bar" }

const barBaz = await squirrel.getBy('bar.baz', 'qux');
console.log(barBaz); // { "bar": { "baz": "qux" } }

Fields can be nested, as described by _.get.

getAll(index)

Get index Map. Returns Promise;

  • index (String): the path of the property to get. It needs to be declared in the indexes option
const foo = await squirrel.getAll('foo');
console.log(foo); // { "bar": { "foo": "bar" } }

const barBaz = await squirrel.getAll('bar.baz');
console.log(barBaz); // { "qux": { "bar": { "baz": "qux" } } }

set(path, value)

Set file by path. Returns Promise;

  • path (String): the path of the file to get.
  • value (Object): An object to store in file. Will be serialized.
const foo = await squirrel.set('/foo',  { "foo": "bar" });
console.log(foo); // { "foo": "bar" }

Command Line Interface

squirrel-sync

Synchronize FS folder with ETCD folder.

$ squirrel-sync --hosts localhost:2379 ./fs-folder /etcd-folder

squirrel-watch

Watch ETCD folder changes.

$ squirrel-watch --hosts localhost:2379 /etcd-folder

squirrel-dump

Write ETCD folder in preloadedStore format.

$ squirrel-dump --hosts localhost:2379 /etcd-folder ./dump.json

Arguments

  • --hosts="host1,host2": ETCD hosts. more
  • --ca=/file.ca: Ca certificate. more
  • --key=/file.key: Client key. more
  • --cert=/file.cert: Client certificate. more

Index System

Squirrel allows to put JSON in file. In this case, it could be indexes to access directly. Consider the following ETCD directory.

/
ā”œā”€ā”€ file1 { "foo": "bar" }
ā”œā”€ā”€ file2 { "foo": "baz" }
ā””ā”€ā”€ file3 { "foo": "qux" }

First of all, we should indicate Squirrel which paths we want to index.

const squirrel = createSquirrel({
  indexes: ['foo']
});

Now, we can get the contents of file1 by searching for its foo value.

const file1 = await squirrel.getBy('foo', 'bar');
console.log(file1); // { "foo": "bar" }

We can also get the value of the index as an object.

const fooIndex = await squirrel.getAll('foo');
console.log(fooIndex);
/*
{
  "bar": { "foo": "bar" },
  "baz": { "foo": "baz" },
  "qux": { "foo": "qux" }
}
 */

If two files have the same index value, Squirrel keeps one of the two.

Squirrel scans all files, no matter how deep, that contain a JSON value.

Index could be a complex path, as long as it works with _.get.

Fallback System

By declaring a fallback path, Squirrel is able :

  • to save its state every time a change is made
  • to restore the state to be faster on the next restart even if ETCD isn't available.

Test

You may run tests with

$ npm test

Marble

If you find any bugs or have a feature request, please open an issue on github!

The npm package download data comes from npm's download counts api and package details come from npms.io.