Migrating Nightwatch Tests to Cypress

Cypress is an Electron-based integration testing framework for web applications. It provides a web-based interface to easily trace and navigate test runs. Originally, Cypress just seemed like yet another Chrome-only ecosystem so I wrote it off, sticking with Nightwatch's more open cross-browser support. Though, last year Cypress finally added support for Firefox and I've considered switching ever since.

Why Migrate?

There are a number of reasons I've wanted to make the switch. To keep it short: Cypress has better documentation, easier debugging and a consistent API. I even made Night Patrol, an interactive CLI for Nightwatch, to solve some of Nightwatch's shortfalls. But it's just more things to maintain and some of Night Patrol's core dependencies have been abandoned. While personally I prefer the simpler debugging experience Night Patrol provides, Cypress provides enough for me to compromise.

Cypress is very different from Nightwatch. It uses its own API running through Electron whereas Nightwatch uses the more common WebDriver API. That means Cypress is able to do things WebDriver can't, but there are some caveats. Maybe a lot.

Around last year, Playwright also entered the scene. A WebDriver-driven framework that's a bit like a continuation of Puppeteer geared for writing tests. Personally, I prefer Playwright's API compared to Cypress. But, Cypress's test runner interface wins me over for now.

Given all of this, it's worth not tying your tests too closely with Cypress. Much like I avoided with Nightwatch. Frameworks come and go, so you need to be capable of migrating hundreds of tests if the right one appears. Keep things simple.

Configuration

Unlike Nightwatch, Cypress does not immediately support dynamic configuration. You can only provide a JSON file or swap parameters using specific environment variables and flags.

Web development often involves multiple tools working together and it gets complicated. So I have a system set up for all of my tools to easily switch between different deployments. I pass in an APP_ENV to a command and a shared script takes that and grabs the appropriate shared configuration.

Luckily this problem is easily solved. Cypress has a programmatic API so you can execute a script which generates the config and launches Cypress with it. If it didn't you could just as easily write a script that places a configuration whereever Cypress wants before executing it.

const cypress = require("cypress");

const options = {
  config: {
    // ...
  },
  env: {
    // ...
  },
};

process.on("exit", () => {
  console.log("Exiting.");
  cleanup();
});

process.on("SIGINT", async () => {
  console.log("Caught interrupt.");
  await cleanup();
  process.exit(1);
});

process.on("uncaughtException", async (error) => {
  console.error("uncaughtException", error);
  await cleanup();
  process.exit(2);
});

process.on("unhandledRejection", async (reason) => {
  console.error("unhandledRejection", reason);
  await cleanup();
  process.exit(3);
});

cypress[process.argv[2]](options).then((result) => {
  if (result.failures) {
    console.error("Could not execute tests");
    console.error(result.message);
    process.exit(result.failures);
  }

  process.exit(result.totalFailed);
});

There are a few things to mention here. Cypress does not have a global after step like Nightwatch does. So the only way to guarantee cleanup (e.g. clearing a database) is to hook into every exit condition.

The other thing is, since this is the programmatic API, to correctly fail the process in an execution pipeline, you need to do it yourself by counting the failures.

This could be made cleaner by handling setup and teardown up a level, but it'll be yet another script -- unless if you avoid using the programmatic API entirely... which might not be a bad idea.

Browser Configuration

You may have also noticed that the browser can't be configured inside the configuration! That needs to go into plugins/index.js.

const { getUserToken } = require("../auth");

module.exports = (on, config) => {
  const { env } = config;

  on("before:browser:launch", (browser = {}, launchOptions) => {
    if (browser.family === "chromium" && browser.name !== "electron") {
      launchOptions.args.push("--mute-audio");
    }
    return launchOptions;
  });

  on("task", {
    getUserToken(id) {
      return getUserToken(id, env);
    },
  });
};

Odd. Not only that, when running in headless mode (with cypress run), that goes through Electron which only supports a limited set of preferences. In my case, I couldn't mute the audio and had to instead mute it on the OS level.

The example above also takes us to the next topic.

Tasks

In Cypress, the tests are running in a browser context. So you don't have access to NodeJS APIs like you had in Nightwatch. Stuff like filesystems. Cypress provides an API for reading and writing files specifically, but if you'd rather have full-access to NodeJS, you need to use Tasks. Tasks essentially lets Cypress hook back into its NodeJS side to execute... tasks.

To execute the above task, it's as simple as:

cy.task("getUserToken", id);

It's again odd to wrap a function in a function to call it as a string argument to another function, but that's the price we're paying right now for a nice test runner interface.

Sessions

Unlike Nightwatch, Cypress doesn't start with a fresh session on every run. By default it clears cookies and localStorage, I assume for convenience, but it doesn't clear everything else; like IndexedDB. So if you're wondering why you're logged in on a new test run, that's why.

To solve it, you need to add a shared before/after hook in an ambiguously named support/index.js.

before(() => {
  cy.clearAuth();
});

This is also where your commands will go. Which takes us to the next topic.

Commands

Nightwatch has commands, Cypress has commands. Shared functions that are hooked into the test flow. So if you need to do something and make the test runner wait for it, that's where it goes.

Cypress.Commands.add("clearAuth", () => {
  return new Promise((resolve) => {
    const req = indexedDB.deleteDatabase("authDb");
    req.onsuccess = function () {
      resolve();
    };
  });
});

Unlike Tasks, because Cypress is runnning inside the browser, and also shares the same domain as the page you're testing, the indexedDB call here is accessing the same data as your website.

Test Structure

While there are some minor differences in how you structure your test suite, for the most part it's the same. Cypress uses a Mocha (describe, it) structure with Chai assertions and Sinon mocks; a popular combination for unit tests.

// Nightwatch
module.exports = {
  "opens the menu"(browser) {
    // ...
  },
};

// Cypress (Mocha)
describe('Menu', () => {
  it('opens the menu', () => {
    // ...
  }
}

Assertions

For the most part, assertions are the same, they just use different syntax. Chances are, if you've kept things simple, you can convert your assertions carefully in bulk using Find and Replace or similar.

browser.expect.element(".js-Sidebar").to.not.be.present;

cy.get(".js-Sidebar").should("exist");

The assertions in Cypress being inside strings is a bit strange. But with TypeScript support, it's a lot easier to find what's actually available. In Nightwatch's more naked Chai-like API, you have to type each section before the next. Cypress just lists the entire string. It also avoids that awful Getter hack that can cause missed assertions from simple typos.

Page Objects

Page Objects provide a common shared structure for finding and interacting with components on a web page. Nightwatch can take these definitions and provide a dynamic API on top of it.

const page = browser.page.home().navigate();
page.expect.section("@Sidebar").to.be.visible;

Personally, I avoided using Page Objects after a few attempts. I found the API unnecessary and the magic syntax risks making migrations difficult. Simple constants for selectors and functions for interactions are more than enough.

Cypress also avoids Page Objects. I had a few left from the early days so it was a matter to finishing off some refactoring.

const homePage = "/";
const Sidebar = ".js-Sidebar";

// ...

cy.visit(homePage);
cy.get(Sidebar).should("be.visible");

Alerts, Confirms, Prompts

Nightwatch has a pretty normal API for handling native browser modals. Sadly, Cypress does not have much of an API for it. The best solution I found is to mock the functions to return a value immediately.

cy.window().then((win) => cy.stub(win, "prompt").returns("My reply"));

Conclusion

That's about everything. I've only just started using Cypress so it's hard to say if all of this was worth it. Everything's familiar once you know where to put it. Time will tell as I get a chance to write some new tests. I've archived Night Patrol so that's one thing off my mind at least.

Thanks for reading.