Mikhail Angelov
Mikhail holds a Master’s in Physics. He’s run the gamut with Node.js, Go, JavaScript SPAs, React.js, Flux/Redux, RIOT.js, and AngularJS.
Expertise
Experience
21 years
Dealing with a real database for integration and end-to-end testing can pose some challenges. One way of working around them is to use an in-memory database that integrates well with the testing framework and provides APIs to manipulate its state from test code.
In this article, Toptal Freelance Software Engineer Mikhail Angelov shows how you can write integration and end-to-end tests easily for your Node.js and MongoDB application without having to write complicated setup/teardown code.
Dealing with a real database for integration and end-to-end testing can pose some challenges. One way of working around them is to use an in-memory database that integrates well with the testing framework and provides APIs to manipulate its state from test code.
In this article, Toptal Freelance Software Engineer Mikhail Angelov shows how you can write integration and end-to-end tests easily for your Node.js and MongoDB application without having to write complicated setup/teardown code.
Mikhail holds a Master’s in Physics. He’s run the gamut with Node.js, Go, JavaScript SPAs, React.js, Flux/Redux, RIOT.js, and AngularJS.
21 years
Tests are an essential part of building a robust Node.js application. Proper tests can easily overcome a lot of shortcomings that developers may point out about Node.js development solutions.
While many developers focus on 100% coverage with unit tests, it is important that the code you write is not just tested in isolation. Integration and end-to-end tests give you that extra confidence by testing parts of your application together. These parts may be working just fine on their own, but in a large system, units of code rarely work separately.
Node.js and MongoDB together form one of the most popular duos of recent times. If you happen to be one of the many people using them, you are in luck.
In this article, you will learn how to write integration and end-to-end tests easily for your Node.js and MongoDB application that run on real instances of the database all without needing to set up an elaborate environment or complicated setup/teardown code.
You will see how the mongo-unit package helps with integration and end-to-end testing in Node.js. For a more comprehensive overview of Node.js integration tests, see this article.
Typically, for integration or end-to-end tests, your scripts will need to connect to a real dedicated database for testing purposes. This involves writing code that runs at the beginning and end of every test case/suite to ensure that the database is in a clean predictable state.
This may work well for some projects, but has some limitations:
On the other hand, using a real database makes the test environment as close to production as possible. This can be looked at as a particular advantage of this approach.
Using a real database for testing does seem to have some challenges. But, the advantage of using a real database is too good to pass on. How can we work around the challenges and keep the advantage?
Reusing a good solution from another platform and applying it to the Node.js world can be the way to go here.
Java projects widely use DBUnit with an in-memory database (e.g., H2) for this purpose.
DBUnit is integrated with JUnit (the Java test runner) and lets you define the database state for each test/testing suite, etc. It removes the constraints discussed above:
Taking from these concepts, I decided to make something similar for Node.js and MongoDB: Mongo-unit.
Mongo-unit is a Node.js package that can be installed using NPM or Yarn. It runs MongoDB in-memory. It makes integration tests easy by integrating well with Mocha and providing a simple API to manage the database state.
The library uses the mongodb-prebuilt NPM package, which contains prebuilt MongoDB binaries for the popular operating systems. These MongoDB instances can run in in-memory mode.
To add mongo-unit to your project, you can run:
npm install -D mongo-unit
or
yarn add mongo-unit
And, that is it. You do not even need MongoDB installed on your computer to use this package.
Let’s imagine you have a simple Node.js application to manage tasks:
// service.js
const mongoose = require('mongoose')
const mongoUrl = process.env.MONGO_URL || 'mongodb://localhost:27017/example'
mongoose.connect(mongoUrl)
const TaskSchema = new mongoose.Schema({
name: String,
started: Date,
completed: Boolean,
})
const Task = mongoose.model('tasks', TaskSchema)
module.exports = {
getTasks: () => Task.find(),
addTask: data => new Task(data).save(),
deleteTask: taskId => Task.findByIdAndRemove(taskId)
}
The MongoDB connection URL is not hard-coded here. As with most web application back-ends, we are taking it from the environment variable. This will let us substitute it for any URL during tests.
const express = require('express')
const bodyParser = require('body-parser')
const service = require('./service')
const app = express()
app.use(bodyParser.json())
app.use(express.static(`${__dirname}/static`))
app.get('/example', (req, res) => {
service.getTasks().then(tasks => res.json(tasks))
})
app.post('/example', (req, res) => {
service.addTask(req.body).then(data => res.json(data))
})
app.delete('/example/:taskId', (req, res) => {
service.deleteTask(req.params.taskId).then(data => res.json(data))
})
app.listen(3000, () => console.log('started on port 3000'))
This is a snippet of an example application that has a user interface. The code for the UI has been omitted for brevity. You can check out the complete example on GitHub.
To make Mocha run integration tests against mongo-unit, we need to run the mongo-unit database instance before the application code is loaded in the Node.js context. To do this, we can use the mocha --require
parameter and Mocha-prepare library, which allows you to perform asynchronous operations in the require scripts.
// it-helper.js
const prepare = require('mocha-prepare')
const mongoUnit = require('mongo-unit')
prepare(done => mongoUnit.start()
.then(testMongoUrl => {
process.env.MONGO_URL = testMongoUrl
done()
}))
The first step is to add a test to the test database (testData.json
):
{
"tasks": [
{
"name": "test",
"started": "2017-08-28T16:07:38.268Z",
"completed": false
}
]
}
The next step is to add the tests themselves:
const expect = require('chai').expect
const mongoose = require('mongoose')
const mongoUnit = require('../index')
const service = require('./app/service')
const testMongoUrl = process.env.MONGO_URL
describe('service', () => {
const testData = require('./fixtures/testData.json')
beforeEach(() => mongoUnit.initDb(testMongoUrl, testData))
afterEach(() => mongoUnit.drop())
it('should find all tasks', () => {
return service.getTasks()
.then(tasks => {
expect(tasks.length).to.equal(1)
expect(tasks[0].name).to.equal('test')
})
})
it('should create new task', () => {
return service.addTask({ name: 'next', completed: false })
.then(task => {
expect(task.name).to.equal('next')
expect(task.completed).to.equal(false)
})
.then(() => service.getTasks())
.then(tasks => {
expect(tasks.length).to.equal(2)
expect(tasks[1].name).to.equal('next')
})
})
it('should remove task', () => {
return service.getTasks()
.then(tasks => tasks[0]._id)
.then(taskId => service.deleteTask(taskId))
.then(() => service.getTasks())
.then(tasks => {
expect(tasks.length).to.equal(0)
})
})
})
And, voila!
Notice how there are just a couple of lines of code dealing with setup and teardown.
As you can see, it’s very easy to write integration tests using the mongo-unit library. We do not mock MongoDB itself, and we can use the same Mongoose models. We have full control of the database data and do not lose much on test performances since the fake MongoDB is running in memory.
This also allows us to apply the best unit testing practices for integration tests:
As a bonus, we can even run the application itself against mongo-unit. It allows us to make end-to-end tests for our application against a mocked database.
For end-to-end testing, we will be using Selenium WebDriver and Hermione E2E test runner.
First, we will bootstrap the driver and the test runner:
const mongoUnit = require('mongo-unit')
const selenium = require('selenium-standalone')
const Hermione = require('hermione')
const hermione = new Hermione('./e2e/hermione.conf.js') //hermione config
seleniumInstall() //make sure selenium is installed
.then(seleniumStart) //start selenium web driver
.then(mongoUnit.start) // start mongo unit
.then(testMongoUrl => {
process.env.MONGO_URL = testMongoUrl //store mongo url
})
.then(() => {
require('./index.js') //start application
})
.then(delay(1000)) // wait a second till application is started
.then(() => hermione.run('', hermioneOpts)) // run hermiona e2e tests
.then(() => process.exit(0))
.catch(() => process.exit(1))
We will also need some helper functions (error handling removed for brevity):
function seleniumInstall() {
return new Promise(resolve => selenium.install({}, resolve))
}
function seleniumStart() {
return new Promise(resolve => selenium.start(resolve))
}
function delay(timeout) {
return new Promise(resolve => setTimeout(resolve, timeout))
}
After filling the database with some data and cleaning it once the tests are done, we can run our first tests:
const expect = require('chai').expect
const co = require('co')
const mongoUnit = require('../index')
const testMongoUrl = process.env.MONGO_URL
const DATA = require('./fixtures/testData.json')
const ui = {
task: '.task',
remove: '.task .remove',
name: '#name',
date: '#date',
addTask: '#addTask'
}
describe('Tasks', () => {
beforeEach(function () {
return mongoUnit.initDb(testMongoUrl, DATA)
.then(() => this.browser.url('http://localhost:3000'))
})
afterEach(() => mongoUnit.dropDb(testMongoUrl))
it('should display list of tasks', function () {
const browser = this.browser
return co(function* () {
const tasks = yield browser.elements(ui.task)
expect(tasks.length, 1)
})
})
it('should create task', function () {
const browser = this.browser
return co(function* () {
yield browser.element(ui.name).setValue('test')
yield browser.element(ui.addTask).click()
const tasks = yield browser.elements(ui.task)
expect(tasks.length, 2)
})
})
it('should remove task', function () {
const browser = this.browser
return co(function* () {
yield browser.element(ui.remove).click()
const tasks = yield browser.elements(ui.task)
expect(tasks.length, 0)
})
})
})
As you can see, the end-to-end tests look very similar to the integration tests.
Integration and end-to-end testing are important for any large-scale application. Node.js applications, in particular, can benefit tremendously from automated testing. With mongo-unit, you can write integration and end-to-end testing without worrying about all the challenges that come with such tests.
You can find complete examples of how to use mongo-unit on GitHub.
An integration test is an automated test that is used to verify if multiple components of a system work correctly for various cases.
E2E is short for end-to-end, and is generally used in the context of end-to-end testing.
Located in Nizhny Novgorod, Nizhny Novgorod Oblast, Russia
Member since July 6, 2015
Mikhail holds a Master’s in Physics. He’s run the gamut with Node.js, Go, JavaScript SPAs, React.js, Flux/Redux, RIOT.js, and AngularJS.
21 years
World-class articles, delivered weekly.
World-class articles, delivered weekly.
Join the Toptal® community.