@cumulus/integration-tests
This package provides a CLI and functions for testing Cumulus.⚠️ The documented API of this package will not change without a deprecation warning being provided in earlier releases. Code in this package that is not documented in this README may change without warning, and is not considered part of the package's public API.
About Cumulus
Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.Cumulus Documentation
Installation
$ npm install @cumulus/integration-tests
API
Modules
Collections
Exampleconst Collections = require('@cumulus/integration-test/Collections');
* [createCollection(prefix, [overrides])](#exp_module_Collections--createCollection) ⇒ <code>Promise.<Object></code> ⏏
* [~addCustomUrlPathToCollectionFiles(collection, customFilePath)](#module_Collections--createCollection..addCustomUrlPathToCollectionFiles) ⇒ <code>Array.<Object></code>
* [~buildCollection(params)](#module_Collections--createCollection..buildCollection) ⇒ <code>Object</code>
* [~collectionExists(stackName, collection)](#module_Collections--createCollection..collectionExists) ⇒ <code>boolean</code>
* [~addCollection(stackName, collection)](#module_Collections--createCollection..addCollection) ⇒ <code>Promise.<undefined></code>
* [~addCollections(stackName, bucketName, dataDirectory, [postfix], [customFilePath], [duplicateHandling])](#module_Collections--createCollection..addCollections) ⇒ <code>Promise.<Array.<Object>></code>
createCollection(prefix, overrides) ⇒ Promise.<Object>
⏏
Create a randomized collection using the Cumulus API.The default collection is very simple. It expects that, for any discovered file, the granule ID is everything in the filename before the extension. For example, a file named
gran-1.txt
would
have a granuleId of gran-1
. Filenames can only contain a single .
character.Collection defaults:
- name: random string starting with
collection-name-
- version: random string starting with
collection-version-
- reportToEms:
false
- granuleId:
'^[^.]+$'
- granuleIdExtraction:
'^([^.]+)\..+$'
- sampleFileName:
'asdf.jpg'
- files:
{
bucket: 'protected',
regex: '^[^.]+\..+$',
sampleFileName: 'asdf.jpg'
}
```Kind: Exported function
Returns:
Promise.<Object>
- the generated collection| Param | Type | Description | | --- | --- | --- | | prefix |
string
| the Cumulus stack name |
| overrides | Object
| properties to set on the collection, overriding the defaults |createCollection~addCustomUrlPathToCollectionFiles(collection, customFilePath) ⇒ Array.<Object>
Given a Cumulus collection configuration, return a list of the filetype
configs with their url_path
s updated.Kind: inner method of
createCollection
Returns:
Array.<Object>
- a list of collection filetype configs| Param | Type | Description | | --- | --- | --- | | collection |
Object
| a Cumulus collection |
| customFilePath | string
| path to be added to the end of the urlpath |createCollection~buildCollection(params) ⇒ Object
Update a collection with a custom file path, duplicate handling, and name
updated with the postfix.Kind: inner method of
createCollection
Returns:
Object
- an updated collection| Param | Type | Description | | --- | --- | --- | | params |
Object
| |
| params.collection | Object
| a collection configuration |
| params.customFilePath | string
| path to be added to the end of the urlpath |
| params.duplicateHandling | string
| duplicate handling setting |
| params.postfix | string
| a string to be appended to the end of the name |createCollection~collectionExists(stackName, collection) ⇒ boolean
Returns true if collection exists. False otherwise.Kind: inner method of
createCollection
| Param | Type | Description | | --- | --- | --- | | stackName |
string
| the prefix of the Cumulus stack |
| collection | Object
| a Cumulus collection |createCollection~addCollection(stackName, collection) ⇒ Promise.<undefined>
Add a new collection to CumulusKind: inner method of
createCollection
| Param | Type | Description | | --- | --- | --- | | stackName |
string
| the prefix of the Cumulus stack |
| collection | Object
| a Cumulus collection |createCollection~addCollections(stackName, bucketName, dataDirectory, postfix, customFilePath, duplicateHandling) ⇒ Promise.<Array.<Object>>
Add collections to databaseKind: inner method of
createCollection
Returns:
Promise.<Array.<Object>>
- - collections that were added| Param | Type | Description | | --- | --- | --- | | stackName |
string
| Cloud formation stack name |
| bucketName | string
| S3 internal bucket name |
| dataDirectory | string
| the directory of collection json files |
| postfix | string
| string to append to collection name |
| customFilePath | string
| |
| duplicateHandling | string
| |Executions
Exampleconst Executions = require('@cumulus/integration-test/Executions');
* [findExecutionArn(prefix, matcher, [queryParameters], [options])](#exp_module_Executions--findExecutionArn) ⇒ <code>Promise.<string></code> ⏏
* [getExecutionWithStatus(params)](#exp_module_Executions--getExecutionWithStatus) ⇒ <code>Promise.<Object></code> ⏏
findExecutionArn(prefix, matcher, queryParameters, options) ⇒ Promise.<string>
⏏
Find the execution ARN matching the matcher
functionKind: Exported function
Returns:
Promise.<string>
- the ARN of the matching execution| Param | Type | Default | Description | | --- | --- | --- | --- | | prefix |
string
| | the name of the Cumulus stack |
| matcher | function
| | a predicate function that takes an execution and determines if this is the execution that is being searched for |
| queryParameters | Object
| | Optional query parameters |
| options | Object
| | |
| options.timeout | integer
| 0
| the number of seconds to wait for a matching execution to be found |getExecutionWithStatus(params) ⇒ Promise.<Object>
⏏
Wait for an execution to have an expected status and return the executionKind: Exported function
Returns:
Promise.<Object>
- the execution as returned by the GET /executions/<execution-arn>
endpoint| Param | Type | Default | Description | | --- | --- | --- | --- | | params |
Object
| | |
| params.prefix | string
| | the name of the Cumulus stack |
| params.arn | string
| | the execution ARN to fetch |
| params.status | string
| | the status to wait for |
| params.callback | function
| cumulusApiClient.invokeApifunction
| an async function to invoke the API Lambda that takes a prefix / user payload |
| params.timeout | integer
| 30
| the number of seconds to wait for the execution to reach a terminal state |Granules
Exampleconst Granules = require('@cumulus/integration-test/Granules');
* [getGranuleWithStatus(params)](#exp_module_Granules--getGranuleWithStatus) ⇒ <code>Promise.<Object></code> ⏏
* [~waitForListGranulesResult(params)](#module_Granules--getGranuleWithStatus..waitForListGranulesResult) ⇒ <code>Promise.<Object></code>
getGranuleWithStatus(params) ⇒ Promise.<Object>
⏏
Wait for a granule to have an expected status and return the granuleKind: Exported function
Returns:
Promise.<Object>
- the granule as returned by the GET /granules/<granule-id>
endpoint| Param | Type | Default | Description | | --- | --- | --- | --- | | params |
Object
| | |
| params.prefix | string
| | the name of the Cumulus stack |
| params.granuleId | string
| | the granuleId
of the granule |
| params.collectionId | string
| | the collectionId
of the granule |
| params.status | string
| | the status to wait for |
| params.updatedAt | string
| | minimum updatedAt time the granule must have to return |
| params.callback | function
| cumulusApiClient.invokeApifunction
| an async function to invoke the API Lambda that takes a prefix / user payload |
| params.timeout | integer
| 30
| the number of seconds to wait for the execution to reach a terminal state |getGranuleWithStatus~waitForListGranulesResult(params) ⇒ Promise.<Object>
Wait for listGranules to return at least a single value before returning an
empty resultKind: inner method of
getGranuleWithStatus
Returns:
Promise.<Object>
- - results of a successful listGranules| Param | Type | Description | | --- | --- | --- | | params |
Object
| parameters to listGranules function |Providers
Exampleconst Providers = require('@cumulus/integration-test/Providers');
createProvider(prefix, overrides) ⇒ Promise.<Object>
⏏
Create a provider using the Cumulus APIProvider defaults:
- id: random string starting with
provider-
- protocol:
s3
- globalConnectionLimit:
10
Kind: Exported function
Returns:
Promise.<Object>
- the generated provider| Param | Type | Description | | --- | --- | --- | | prefix |
string
| the Cumulus stack name |
| overrides | Object
| properties to set on the provider, overriding the defaults |Rules
Exampleconst Rules = require('@cumulus/integration-test/Rules');
createOneTimeRule(prefix, overrides) ⇒ Promise.<Object>
⏏
Create a onetime
rule using the Cumulus APIRule defaults:
- name: random string starting with
rule_
- rule:
{ type: 'onetime' }
Kind: Exported function
Returns:
Promise.<Object>
- the generated rule| Param | Type | Description | | --- | --- | --- | | prefix |
string
| the name of the Cumulus stack |
| overrides | Object
| properties to set on the rule, overriding the defaults |CLI Usage
Usage: cumulus-test TYPE COMMAND [options]
Options:
-V, --version output the version number
-s, --stack-name <stackName> AWS Cloud Formation stack name (default: null)
-b, --bucket-name <bucketName> AWS S3 internal bucket name (default: null)
-w, --workflow <workflow> Workflow name (default: null)
-i, --input-file <inputFile> Workflow input JSON file (default: null)
-h, --help output usage information
Commands:
workflow Execute a workflow and determine if the workflow completes successfully
For example, to test the HelloWorld workflow:
cumulus-test workflow --stack-name helloworld-cumulus --bucket-name cumulus-bucket-internal --workflow HelloWorldWorkflow --input-file ./helloWorldInput.json
Contributing
To make a contribution, please see our contributing guidelines.Generated automatically using
npm run build-docs