mongodb-migrations
A Node.js migration framework for MongoDB with both programmatic and CLI API.
Table of Contents generated with DocToc
- Configuration - Creating Migrations
- [Migration functions](#migration-functions)
- Sample migration file
- Running migrations
- Creating Migrator
object- [Custom logging](#custom-logging)
- Adding migrations- [`migrator.add`](#migratoradd)
- [`migrator.bulkAdd`](#migratorbulkadd)
- migrator.migrate
- migrator.runFromDir
- migrator.rollback
- migrator.create
- migrator.dispose
Installation
npm install mongodb-migrations --save
Common Usage (CLI)
The package installs a single CLI executable —mm
.When installing locally to your project this executable can be found at
./node_modules/.bin/mm
.When installing globally (normally not recommended) the executable should automatically become accessible on your PATH.
Configuration
The CLI app expects a configuration file to be present in the directory where the app is being ran.By default (if the configuration file is not set through the command-line argument) the app checks
mm-config.json
, mm-config.js
,
and mm-config.coffee
files for existence.File name can be passed through the means of
--config
parameter.
The path is relative to the current directory:mm --config=configs/mm.json
In case of
json
the file should contain valid JSON representation of the
configuration object.In case of
js
or coffee
the file should be a CommonJS module
exporting the configuration object. This is useful when you already have configuration
data (potentially in a different format) and want to avoid duplication. See test/mm-config.coffee
for an example of this usage.In case of
coffee
the coffee-script >= 1.7.0
package must be importable
from the current directory (include it as your project's dependency).The configuration object can have the following keys:
url
— full MongoDB connection url (optional, when used the rest of the connection params (host
,port
,db
,user
,password
,replicaset
,authDatabase
) are ignored),host
— MongoDB host (optional when usingurl
orreplicaset
, required otherwise),port
optional — MongoDB port,db
— MongoDB database name,ssl
optional - boolean, iftrue
,'?ssl=true'
is added to the MongoDB URL,user
optional — MongoDB user name when authentication is required,password
optional — MongoDB password when authentication is required,authDatabase
optional - MongoDB database to authenticate the user against,collection
optional — The name of the MongoDB collection to track already ran migrations, defaults to_migrations
,directory
— the directory (path relative to the current folder) to store migration files in and read them from, used when running from the command-line or when usingrunFromDir
,timeout
optional — time in milliseconds after which migration should fail ifdone()
is not called (use 0 to disable timeout)poolSize
optional, deprecated, useoptions.server.poolSize
instead - the size of the mongo connection pool,options
optional - arbitrary options passed to the MongoClient (Note: if not set directly,options.server.poolSize
defaults to5
.),replicaset
optional - if using replica sets should be an object of the following structure:
name: 'rs-ds023680',
members: [
{
host: 'bee.boo.bar',
port: 23680
}
{
host: 'choo.choo',
port: 24610
}
]
Creating Migrations
The app simplifies creating migration stubs by providing a commandmm create MIGRATION-NAME [--coffee|-c]
This creates automatically numbered file
NNN-migration-name.js
(or .coffee
if -c
of --coffee
flag provided)
inside of the directory
defined in the
configuration file.The migration file must be a CommonJS module exporting the following:
id
— a string that's used to identify the migration
mm create
).up
optional — a function used for forward migration.down
optional — a function used for backward migration (rollback).
See Configuration if your config file has non-standard name.
Migration functions
Theup
and down
functions take a single parameter — a Node-style callback:module.exports.up = function (done) {
// call done() when migration is successfully finished
// call done(error) in case of error
}
The
up
and down
functions are executed with the scope
providing 2 convenient properties:this.db
is an open MongoDB
this.log
is a function allowing you to print
stdout
with proper indentation.
See Custom logging for advanced usage.Sample migration file
exports.id = 'create-toby';
exports.up = function (done) {
var coll = this.db.collection('test');
coll.insert({ name: 'tobi' }, done);
};
exports.down = function (done) {
var coll = this.db.collection('test');
coll.remove({}, done);
};
Running migrations
Run all migrations from thedirectory
(specified in
Configuration) by simply callingmm
or
mm migrate
The utility only runs migrations that:
- have
up
function defined, - were not ran before against this database.
Ran migrations are recorded in the
collection
specified in Configuration.NOTE: If there are some noop migrations (those without the
up
method)
they will be recorded in the collection
, too.
See migrator.migrate
for the explanation why.The migration process is stopped instantly if some migration fails (returns error in its callback).
See Configuration if your config file has non-standard name.
If you have
.coffee
migration files, coffee-script >= 1.7.0
package
must be importable from the current directory.Debugging migrations
DEBUG=true mm
Running with
DEBUG=true
will print out the error stack on the console.Programmatic usage
The library also supports programmatic usage.Start with
require
'ing it:var mm = require('mongodb-migrations');
Creating Migrator
object
Next, you have to create a Migrator
object. The syntax is:var migrator = new mm.Migrator(config, [customLogFn]);
Where
config
is an object with the keys defined in the
Configuration section (except of the directory
which does not make sense in this scenario).Custom logging
By default when migrations are ranmigrator
will log
it's progress to console — 1 line for each migration added,
indicating the status (skipped, succeeded or failed).
It will also print any custom messages you pass to
this.log
inside of the up
/ down
function.To suppress this logging pass
customLogFn = null
to the
Migrator
constructor (undefined
won't do the trick).If you want to handle the logging on your own (save it to file, or whatever else) you can pass your custom function having this signature:
function customLogFn(level, message),
where
level
is either "system"
(migration status message)
or "user"
(when you call this.log
inside of your migration),
and message
is the actual message string.Adding migrations
Once you have themigrator
object you can add migrations
definitions to it.migrator.add
To add a single migration, callmigrator.add(migrationDef),
where
migrationDef
is an object with id
, up
optional,
and down
optional keys, all having the same meaning
as described in Creating Migrations.migrator.bulkAdd
To add multiple migrations at once, callmigrator.bulkAdd(migrationDefsArray),
where
migrationDefsArray
is an array of objects explained in
migrator.add.migrator.migrate
Once you have one or more migrations added, run them with callingmigrator.migrate(doneFn, [progressFn]).
Migrations are ran in order they were added to the
migrator
.The
doneFn
is called once all migrations are ran or once one of them
fails. The function has the signaturefunction doneFn(error, results),
where
error
is null
if everything is OK, or is an error
returned by the failed migration (if any).The
results
object is always passed (even in case of an error).Its keys are
id
s of the added migrations
(till the one that failed, if any, or till the last one).
The values are result
objects having the following properties:status
—'ok'
,'skip'
, or'error'
,error
- theerror
object returned from the failed migration,reason
— the reason why the migration was skipped, can be
"no migration function for direction up"
,
"no migration function for direction down"
,
"migration already ran"
,
"migration wasn't in the recent migrate run"
, See Rollback for the explanation of the last case,code
— a more machine-friendly version ofreason
, can be"no_up"
,"no_down"
,"already_ran"
,"not_in_recent_migrate"
.
The optional
progressFn
function is called once per each migration
and has the signaturefunction progressFn(id, result),
where
id
is migration's ID, and result
object is explained above.Successfully ran migrations are recorded in the
collection
specified in Configuration.NOTE: for consistency and for the proper rollback operation migrations that do not define the
up
methods (and thus are skipped) are still recorded in
the DB as being ran which is formally true as they are essentially noop.This means that if one of your migrations has an issue, you roll back, then fix this issue and rerun the migrations, the entire set will be re-applied (as rollback assumes the DB is restored to the pre-
migrate
state).Obviously, this will lead to unexpected results if you don't properly define the
down
methods where they are required.migrator.runFromDir
In case your migrations are modules in specific directory
(see Running migrations) there's a convenience method
that reads them in order and then runs.The files must conform to the following rules:
- have their names starting with one or more digits
mm
tool,- be CommonJS modules and export
id
,up
optional,
down
optional — see Creating Migrations
for explanation,- have filenames ending in
.js
or.coffee
; - if the migration file has
.coffee
extension, the
coffee-script >= 1.7.0
package must be importable
from the current directory.To run the migrations from the
directory
callmigrator.runFromDir(directory, doneFn, [progressFn])
The
doneFn
and progressFn
have the same meaning as in
migrator.migrate.migrator.rollback
If you decide that current migration run was unsuccessful,
you can roll back all recently ran transactions. Currently this
operation is only supported through programatic interface.Do so by calling
migrator.rollback()
This runs all the migrations added to the
migrator
in the reverse order, and follows these rules:- migrations without the
down
method are skipped (but see the note below), - migrations not ran recently (potentially those
NOTE: Rolling back removes the migration records from the DB. It's true even for the migrations that do not have the
down
part.migrator.create
To programmatically create a migration stub file, callmigrator.create(directory, id, doneFn, coffeeScript=false),
where
directory
is the directory to save the file to,
id
is migration's ID, doneFn
is a callback that gets
passed the error object in case of error,
and optional coffeeScript
flag tells the library to create the stub
in CoffeeScript instead of plain JavaScript.The ID is lowercased and then dasherized. It's your responsibility to assure it's unique.
The method automatically handles files numbering and naming, and sets the ID inside of the generated file.
migrator.dispose
When you are done with the migrator you should callmigrator.dispose(cb)
to release the MongoDB connections pool. Once disposed the migrator cannot be used anymore.
The
cb
is a Node-style callback:function cb(error).