Visual regression testing with WebdriverIO.
Installation
wdio-visual-regression-service uses wdio-screenshot for capturing screenhots.You can install wdio-visual-regression-service via NPM as usual:
$ npm install wdio-visual-regression-service --save-dev
Instructions on how to install
WebdriverIO
can be found here.An example repository using the wdio-visual-regression service can be found here.
Configuration
Setup wdio-visual-regression-service by addingvisual-regression
to the service section of your WebdriverIO config and define your desired comparison strategy in visualRegression
.// wdio.conf.js
var path = require('path');
var VisualRegressionCompare = require('wdio-visual-regression-service/compare');
function getScreenshotName(basePath) {
return function(context) {
var type = context.type;
var testName = context.test.title;
var browserVersion = parseInt(context.browser.version, 10);
var browserName = context.browser.name;
var browserViewport = context.meta.viewport;
var browserWidth = browserViewport.width;
var browserHeight = browserViewport.height;
return path.join(basePath, `${testName}_${type}_${browserName}_v${browserVersion}_${browserWidth}x${browserHeight}.png`);
};
}
exports.config = {
// ...
services: [
'visual-regression',
],
visualRegression: {
compare: new VisualRegressionCompare.LocalCompare({
referenceName: getScreenshotName(path.join(process.cwd(), 'screenshots/reference')),
screenshotName: getScreenshotName(path.join(process.cwd(), 'screenshots/screen')),
diffName: getScreenshotName(path.join(process.cwd(), 'screenshots/diff')),
misMatchTolerance: 0.01,
}),
viewportChangePause: 300,
viewports: [{ width: 320, height: 480 }, { width: 480, height: 320 }, { width: 1024, height: 768 }],
orientations: ['landscape', 'portrait'],
},
// ...
};
Options
Under the keyvisualRegression
in your wdio.config.js you can pass a configuration object with the following structure:- compare
Object
- viewportChangePause
Number
( default: 100 )
- viewports
Object[{ width: Number, height: Number }]
( default: current-viewport ) (desktop only)
all screenshots will be taken in different viewport dimensions (e.g. for responsive design tests)
- orientations
String[] {landscape, portrait}
( default: current-orientation ) (mobile only)
all screenshots will be taken in different screen orientations (e.g. for responsive design tests)
Compare Methods
wdio-visual-regression-service allows the usage of different screenshot comparison methods.VisualRegressionCompare.LocalCompare
As it's name suggests LocalCompare captures screenshots locally on your computer and compares them against previous runs.You can pass the following options to it's constructor as object:
- referenceName
Function
- screenshotName
Function
- diffName
Function
- misMatchTolerance
Number
( default: 0.01 )
- ignoreComparison
String
( default: nothing )
nothing
, colors
or antialiasing
to adjust the comparison method.For an example of generating screenshot filesnames dependent on the current test name, have a look at the sample code of Configuration.
VisualRegressionCompare.SaveScreenshot
This method is a stripped variant ofVisualRegressionCompare.LocalCompare
to capture only screenshots. This is quite useful when you just want to create reference screenshots and overwrite the previous one without diffing.You can pass the following options to it's constructor as object:
- screenshotName
Function
VisualRegressionCompare.Spectre
This method is used for uploading screenshots to the web application Spectre. Spectre is a UI for visual regression testing. It stores the screenshots and compares them which is quite useful for Continuous Integration.You can pass the following options to it's constructor as object:
- url
String
- project
String
- suite
String
- test
Function
- browser
Function
- size
Function
- fuzzLevel
Number
( default: 30 )
Example
// wdio.conf.js
var path = require('path');
var VisualRegressionCompare = require('wdio-visual-regression-service/compare');
exports.config = {
// ...
services: [
'visual-regression',
],
visualRegression: {
compare: new VisualRegressionCompare.Spectre({
url: 'http://localhost:3000',
project: 'my project',
suite: 'my test suite',
test: function getTest(context) {
return context.test.title;
},
browser: function getBrowser(context) {
return context.browser.name;
},
size: function getSize(context) {
return context.meta.viewport != null ? context.meta.viewport.width : context.meta.orientation;
},
fuzzLevel: 30
}),
viewportChangePause: 300,
viewports: [{ width: 320, height: 480 }, { width: 480, height: 320 }, { width: 1024, height: 768 }],
orientations: ['landscape', 'portrait'],
},
// ...
};
Usage
wdio-visual-regression-service enhances an WebdriverIO instance with the following commands:browser.checkViewport([{options}]);
browser.checkDocument([{options}]);
browser.checkElement(elementSelector, [{options}]);
All of these provide options that will help you to capture screenshots in different dimensions or to exclude unrelevant parts (e.g. content). The following options are available:
- exclude
String[]|Object[]
(not yet implemented)
- hide
String[]
visibility: hidden
)- remove
String[]
display: none
)- viewports
Object[{ width: Number, height: Number }]
(desktop only)
Overrides the global *viewports* value for this command. All screenshots will be taken in different viewport dimensions (e.g. for responsive design tests)
- orientations
String[] {landscape, portrait}
(mobile only)
Overrides the global *orientations* value for this command. All screenshots will be taken in different screen orientations (e.g. for responsive design tests)
- misMatchTolerance
Number
Overrides the global *misMatchTolerance* value for this command. Pass in a number between 0 and 100 that defines the degree of mismatch to consider two images as identical.
- fuzzLevel
Number
Overrides the global *fuzzLevel* value for this command. Pass in a number between 0 and 100 that defines the fuzz factor of Spectre's image comparison method.
- ignoreComparison
String
Overrides the global *ignoreComparison* value for this command. Pass in a string with value of `nothing` , `colors` or `antialiasing` to adjust the comparison method.
- viewportChangePause
Number
Overrides the global *viewportChangePause* value for this command. Wait x milliseconds after viewport change.