@mapbox/s3scan

Streaming operations on S3 objects

Stats

StarsIssuesVersionUpdatedCreatedSize
@mapbox/s3scan
771.0.04 years ago5 years agoMinified + gzip package size for @mapbox/s3scan in KB

Readme

Build Status

s3scan

Provides native Node.js streams for S3 operations such as

  • listing all the keys in a bucket/prefix
  • retrieving all the objects under a given bucket/prefix
  • deleting all the children of a given bucket/prefix

Also provides CLI tools:

s3keys: prints a line-delimited list of keys to stdout

$ s3keys s3://my-bucket/some-prefix

s3purge: deletes all objects beneath the provided prefix

$ s3purge s3://my-bucket/all-finished-with-these

s3scan: GET and print a prefix-worth of objects to stdout

$ s3scan s3://my-bucket/some-prefix
$ s3scan s3://my-bucket/some-prefix --gunzip
$ s3scan s3://my-bucket/some-prefix --gunzip --concurrency=1

Running tests

You can run tests against your own bucket/prefix by specifying them as environment variables:

TestBucket=my-bucket TestPrefix=my-prefix npm test

Otherwise these values default to buckets owned by Mapbox which require appropriate authentication.

If you find any bugs or have a feature request, please open an issue on github!

The npm package download data comes from npm's download counts api and package details come from npms.io.