End-to-end web testing with TestCafe
A complete guide to getting started
What you’ll learn
Getting Started
What is TestCafe, how to install it and create your first test.
Using TestCafe
Write a real test using a sample project: this will show you how to query elements, observe page state, interact with elements, use assertions. We’ll also learn how to organize our files with the page object pattern and using hooks.
Running Tests
Browsers support, headless mode, device emulation, concurrency.
Continuous Integration
Using CircleCI to run tests and see results.
Features
Some features you should know exist: intercepting HTTP requests, screenshots and videos, reporters, TestCafe studio.
Initially this article was a presentation, you can download the slides support here. Sources are available on Github.
Getting Started
What is TestCafe
A node.js tool to automate end-to-end web testing. You can write tests in JS or TypeScript, run them and view results. The tool is free and open source. The v1.0.1 is out since February 2019.
Requirements and installation
Of course be sure Node and NPM are installed (also yarn if you prefer). No webdriver is needed, just one command:
yarn add --dev testcafeTest Code Structure
Here is a basic test file. It contains a fixture: a category of tests. A fixture is defined by a name and a page function that targets the start page of our group of tests. TestCafe uses a proxy server, it injects scripts in the page then we can inspect the elements.
Each test receives a test controller as a parameter. The test controller is an object that allows us to access the test api. We’ll use it for actions and assertions.
We run the test with the following command:
testcafe chrome test.jsThen the test is ran in a chrome browser (you can choose another browser):
Running tests in:
- Chrome 72.0.3626 / Mac OS X 10.13.6Getting Started
✓ My first test1 passed (2s)
The project we are going to test
A React app created with create-react-app and material-ui:
The site is deployed on xke-introduction-testcafe.surge.sh
The tests we want
First scenario:
As a user, when I’m on the home page, I can enter the app and links to articles. If I click on a link, I can see the article page.
Second scenario:
As a user, from the home page, I can access the add page with a form in order to post a new article. I can post my article.
We will enable the live mode to watch for changes with the L option:
testcafe chrome e2e/**/* -LLet’s write our first scenario skeleton
The result in our terminal:
Live mode is enabled.
TestCafe now watches source files and reruns
the tests once the changes are saved.
You can use the following keys in the terminal:
'Ctrl+S' - stops the test run;
'Ctrl+R' - restarts the test run;
'Ctrl+W' - enables/disables watching files;
'Ctrl+C' - quits live mode and closes the browsers.Watching the following files:
/Users/proustibat/workspace/xebia/xke-introduction-testcafe/e2e/index.js
Running tests in:
- Chrome 72.0.3626 / Mac OS X 10.13.6Navigation
✓ Access to a specific article from the home page1 passed (0s)
Querying elements with Selector
Selector is a function that identifies a webpage element in the test. The selector API provides methods and properties to select elements on the page and get their state.
TestCafe uses standard CSS selectors to locate elements. It’s like when you use document.querySelector in JS (learn more by reading the documentation).
In our test, we import Selector then we need to select the start button on our home page:
Actions
Test API provides a set of actions that enable you to interact with the webpage.
Actions are implemented as methods in the test controller object and can be chained.
Multiple actions are available: Click, Right Click, Double Click, Drag Element, Hover, Take Screenshot, Navigate, Press Key, Select Text, Type Text, Upload, Resize Window (read the documentation to learn more).
Observing page state
TestCafe allows you to observe the page state via:
- Selector used to get direct access to DOM elements
- ClientFunction used to obtain arbitrary data from the client side.
For example in our test, once we click on the button, we should be on the posts page, so we can access the title page and log its innerText as follows:
Assertions
We now need assertions to check data or behaviors. The test controller provides an expect function that should check the result of performed actions.
The following assertion methods are available: Deep Equal, Not Deep Equal, Ok, Not Ok, Contains, Not Contains, Type of, Not Type of, Greater than, Greater than or Equal to, Less than, Less than or Equal to, Within, Not Within, Match, Not Match.
In our test, we want to check that the title is “Posts Page”:
Finishing our tests
Now we learned the 4 main concepts of TestCafe: selectors, page state, actions and assertions. So we can complete our first test:
What about our second test
Remember for our second test we want to start from the home page, enter the site by clicking the start button (like the first part of our first test). Then we want to verify the users navigate to the form page when they click on the add button. Finally they should be able to fill the form and submit it.
So the beginning of our test is the same as the first, and we need to do what is written in comments:
Let’s begin with the well known part!
This is not new now, we are able to get elements, have them undergo some actions then check some other elements exist on the page:
To fill the form, we use the typeText method of the TestController. Submitting the form is not new, we just need to select the button then click it:
The page object pattern
Now we need to talk about the organisation of our code. Writing our tests is repetitive: we go on a page, we check elements are present, we perform actions on it, then we check some elements are present, and so on.
During development process, markups, ids and classes can often change so we need to change Selectors everywhere it’s needed.
Fortunately, we can use the page object pattern:
- Page Model is a test automation pattern that allows you to create an abstraction of the tested page and use it in test code to refer to page elements.
- Keep page representation in the Page Model, while tests remain focused on the behavior.
- Improve maintainability.
Read TestCafe documentation
Now we will refactor our tests by creating page models.
Creating home model
A model is a basic class. We define Selector in the constructor:
Now we can use our model in our tests by importing and instantiating it then accessing its selectors:
Creating a posts page model
Following the same principle, we create a model for the posts page that contains selectors for its title, its links and its add button:
Now we can use the model in our tests. Replace title and links selectors:
With:
Replace add button selector:
With:
Creating article model
In our tests, replace:
With:
Creating add page model
Use it in our tests by replacing:
with:
Note that we also chained actions. This is now more readable.
We now have 4 page models that represent the pages of our application. Each page contains its own Selectors.
Now we are going to do the same thing for actions.
Adding actions to the model pages
TestCafe allows to use Test Controller outside of test code by importing it:
import { Selector, t } from ‘testcafe’;Add this to our add page model:
Then use actions in our test:
Complete the add page model with:
Complete posts page model with:
Create a toast model as follows:
Our tests become even more simple
The first test was:
Then became:
Our second test was:
Then became:
Hooks
TestCafe allows you to specify functions that are executed before a fixture or test is started and after it is finished. Exactly like when you write unit tests.
- fixture.beforeEach( fn(t) )
- fixture.afterEach( fn(t) )
- test.before( fn(t) )
- test.after( fn(t) )
Note that test hooks override fixture hooks.
Remember we need to click start button in each of our tests. So we can use a fixture hook that will execute code before each test:
Then remove the corresponding lines from our tests.
Debugging
The debug method of TestController
TestCafe provides the t.debug method that pauses the test and allows you to debug using the browser’s developer tools.
Here we want to stop the scenario with a debugger after we enter title in the input and before we enter content in the textarea:
Debugging with screenshots on failure
Use the following command to get screenshots when the test failed:
testcafe chrome e2e/**/* --screenshots ./screenshots --screenshots-on-failsDebugging with videos
Enables TestCafe to record videos of test runs and specifies the base directory to save these videos.
testcafe chrome e2e/**/* --video videosSome options are available:
testcafe chrome e2e/**/* --video videos --video-options singleFile=true,failedOnly=trueSee documentation about videos
Running Tests
Browser support: installed browsers
Most of modern browsers locally installed can be detected: Google Chrome (Stable, Beta, Dev and Canary), Internet Explorer (11+), Microsoft Edge, Mozilla Firefox, Safari, Android browser, Safari mobile.
If these browsers are locally installed then TestCafe will detect it.
Run testcafe --list-browsers to list which browsers TestCafe automatically detects.
Browsers support: remote device
To run tests on a remote mobile or desktop device, this device must have network access to the TestCafe server.
testcafe remote e2e/**/*
testcafe remote e2e/**/* --qr-code
Browsers support: chrome device emulation
To do this, use the emulation browser parameter. Specify the target device with the device parameter:
testcafe "chrome:emulation:device=iphone 6/7/8" e2e/**/* --video artifacts/videosBrowsers support: cloud testing services
TestCafe allows you to use browsers from cloud testing services.
The following plugins for cloud services are currently provided by the TestCafe team.
You can also create your own plugin: see the doc.
Some other CLI options
Concurrent test execution:
testcafe -c 2 chrome e2e/**/*Speed:
testcafe chrome e2e/**/* — speed 0.5Multiple browsers with or without headless:
testcafe chrome:headless,firefox e2e/**/*See the documentation for more options
You can also use a .testcaferc.json file for the config.
Reporters
Reporters are plugins used to output test run reports in a certain format.
TestCafe ships with the following reporters: spec (used by default), list, minimal, xUnit, JSON.
Here are some custom reporters developed by the community: NUnit, Slack, TeamCity.
Read the documentation to know more
Continuous Integration
Prerequisites
Remember we should run yarn start before running our tests because we target localhost:3000 as the start webpage.
Our goal is to test exactly the same code that will be deployed or not (depending on results). So we need to serve built content after we run yarn build.
We will use the serve package, so install it with yarn add — dev serve
Add a npm script to the package.json file:
“serve:build”: “serve -s build”This will run our app on localhost:5000.
Modify our tests to target build sources
In e2e/index.js, replace localhost:3000 on our fixture page by localhost:5000. Then build the app with yarn build.
Serve the built content in a terminal with yarn serve:buildthen run testcafe in headless mode in another terminal:
testcafe chrome:headless e2e/**/*The all in one with the appCommand
For now we need 2 terminals to run our tests. This is not ideal.
The appCommand of TestCafe executes the specified shell command before running tests.
So we can use it to serve our built app instead of running the yarn serve:build manually. Add this script to the package.json file:
“e2e:ci”: “testcafe chrome:headless e2e/*.js — app ‘yarn serve:build’”Now we’re ready for CircleCI integration. If you don’t have any account, create it before going to the next step.
Create config.yml in .circleci
version: 2.0
jobs:
build:
docker:
- image: circleci/node:8-browsers
working_directory: /home/circleci/project
steps:
- checkout
- setup_remote_docker:
docker_layer_caching: true
# Download and cache dependencies
- restore_cache:
name: Restore Yarn Package Cache
keys:
- yarn-packages-{{ checksum "yarn.lock" }}
- run:
name: Install Dependencies
command: yarn install --frozen-lockfile
- save_cache:
name: Save Yarn Package Cache
key: yarn-packages-{{ checksum "yarn.lock" }}
paths:
- ~/.cache/yarn
# Build
- deploy:
name: Build
command: yarn build
# End-to-end tests
- run:
name: Run e2e tests
command: yarn e2e:ciAdd your repo to CircleCI
On your dashboard click on “add projects” in the left bar menu
Find your repo, then click on “Set Up Project”:
Start building the project on CircleCI
You should see your project running:
Detailed workflow on CircleCI
Use store_test_results
Add xunit reporter to the project:
yarn add — dev testcafe-reporter-xunitModify the CircleCI config:
# End-to-end tests
- run:
name: Run e2e tests
command: yarn e2e:ci
- store_test_results:
path: /tmp/test-resultsModify the npm script as follows:
“e2e:ci”: “testcafe chrome:headless e2e/*.js — app ‘yarn serve:build’ -r xunit:/tmp/test-results/res.xml”See the summary on your dashboard
Be proud! Add badges to your readme:
[](https://circleci.com/gh/proustibat/xke-introduction-testcafe/tree/master)[](https://github.com/DevExpress/testcafe)
Now CircleCI is running for each pull request
Some features you should be aware of
Intercepting HTTP requests
Logging HTTP Requests:
RequestLogger is a hook that can be attached to either a test or a fixture. It allows to record sent or received requests.
For example, here we record http request accessing an article with a get method. Then we can use assertions to check the status code of the response:
Mocking HTTP Responses
We can also mock requests with the RequestMock hook.
Here, I added Google Analytics to our project. We don’t want to send the page view event to our GA dashboard when the navigation comes from our tests. So we mock the request by intercepting it and sending a fake gif:
Framework-Specific Selectors
TestCafe team and community develop libraries of dedicated selectors for the most popular frameworks. So far, the following selectors are available.
- React
- Angular
- AngularJS
- Vue
- Aurelia
TestCafe Studio
A cross-platform IDE
- Create, edit and maintain end-to-end tests in a visual recorder without writing code. See Record Tests.
- Generates a report with overall results and details for each test after completion.
- Code Editor with syntax highlight, code completion and parameter hints.
- Write code from scratch, convert recorded tests to JavaScript to edit them later.
Recording our first test
Running with remote
Recording our second test
Code editor
Links:
- Tech doc: https://devexpress.github.io/testcafe/
- TestCafe Studio: https://docs.devexpress.com/TestCafeStudio
- Continue the tests by yourself: https://github.com/proustibat/xke-introduction-testcafe
- The tested web app: https://xke-introduction-testcafe.surge.sh/
Find all technical articles by Xebia on blog.xebia.fr
