@begin-functions/data

[ ![Codeship Status for smallwins/fun-data](https://app.codeship.com/projects/1f33e4b0-21ac-0136-c2d9-3a7e7af3da57/status?branch=master)](https://app.codeship.com/projects/285941)

Stats

stars 🌟issues ⚠️updated 🛠created 🐣size 🏋️‍♀️
@begin-functions/data
Minified + gzip package size for @begin-functions/data in KB

Readme

begin functions data

Codeship Status for smallwins/fun-data

BFD is an durable and fast key/value store for Begin Functions built on top of DynamoDB with storage/access patterns similar to Redis.

Considerations

  • A document is maximum 100KB
  • Paginated reads return 1MB per page
  • namespaces are unique to an app
  • keys are unique to a namespace
  • namespaces and keys must start with a letter
  • namespaces and keys can only contain lowercase characters, numbers, dashes and colons
  • namespaces and keys must be minimum 1 character and max 255 characters

Data Types

Namespaces contain documents. Documents stored in BFD always have the following keys:

  • ns which references the document namespace
  • key which is a unique identifier within a namespace

Optionally a document can also have a ttl key with a UNIX epoch value representing the expiry time for the document.

Every namespace also has a reserved key _count which is used for generating unique keys within the namespace

API

Grab a BFD client named data.

let data = require('@begin-functions/data')

Writes

Save a document in a namespace by key. Remember ns is always required.

data.set({
  ns: 'tacos', 
  key: 'al-pastor'
}, console.log)

key is optional. But all documents have a key. If no key is given set will generate a key unique to the namespace.

data.set({
  ns: 'tacos', 
}, console.log)

Batch save multiple documents at once by passing an array of objects.

data.set([
  {ns: 'ppl', name:'brian', email:'b@brian.io'},
  {ns: 'ppl', name:'sutr0', email:'sutr0@brian.io'},
  {ns: 'tacos', key:'pollo'},
  {ns: 'tacos', key:'carnitas'},
], console.log)

Reads

Read a document by key.

data.get({
  ns: 'tacos', 
  key: 'al-pastor'
}, console.log)

Or paginate an entire namespace.

// define a read function
function read(cursor) {

  var uery = {ns: 'tacos'}

  // if we have a cursor add it to the query
  if (cursor) {
    query.cursor = cursor
  }

  // read the namespace
  data.get(query, function _get(err, page) {
    
    // bubble any errors
    if (err) throw err
    
    // log the docs
    console.log(page.docs)

    // continue reading if there's another page
    if (page.cursor) {
      read(cursor)
    }
  })
}

// start reading..
read()

Batch read by passing an array of objects. With these building blocks you can construct secondary indexesand joins like one-to-many and many-to-many.

data.get([
  {ns:'tacos', key:'carnitas'},
  {ns:'tacos', key:'al-pastor'},
], console.log)

Deletes

Delete a document by key.

data.del({
  ns: 'tacos', 
  key: 'pollo'
}, console.log)

Batch delete documents by passinng an array of objects.

data.del([
  {ns:'tacos', key:'carnitas'},
  {ns:'tacos', key:'al-pastor'},
], console.log)

Additional Superpowers

  • Documents can be expired by setting ttl to an UNIX epoch in the future.
  • Atomic counters: data.incr and data.decr

Patterns

Coming soon! Detailed guides for various data persistence tasks:

  • denormalizing
  • pagination
  • counters
  • hashids
  • secondary indexes
  • one to many
  • many to many

If you find any bugs or have a feature request, please open an issue on github!

The npm package download data comes from npm's download counts api and package details come from npms.io.