Migrating 500+ tests from Mocha to Node.js

By
Emanuele Stoppa
Bjorn Lu

Over a month ago, we discussed a possible migration to the Node.js test runner. While we were sufficiently happy with Mocha, we are always looking to make our CI jobs faster.

Relying on a test runner baked inside our runtime had some advantages for our main monorepo:

  • Two fewer dependencies to install and maintain in our monorepo: mocha and chai.
  • Maintainability, as there are way more people involved in the Node.js project (hence the Node.js test runner).
  • The test runner will improve with time - performance-wise and DX-wise, and eventually save some time in our CI workflows.

From an idea to a PoC

The Astro monorepo has more than 500 testing suites: between integration tests and unit tests, we have 664 suites, with a total of 1603 tests. The majority of these tests are integration tests.

An integration test, in our monorepo, means the creation of a tiny Astro project, building a project with a specific environment - development, static generation (SSG), or dynamic generation (SSR), and running assertions over the built pages. That’s right, each integration test requires vite to build and bundle the project.

We didn’t start the migration right away. Before making a final decision, we wanted to make sure that moving away from Mocha was a not mistake. Despite its quirks, Mocha is a fine test runner, it’s been around for a long time, and it is battle-tested. If you use Mocha, you are in good hands.

The idea of the PoC was to understand:

  • The flexibility of the Node.js CLI arguments and how customizable can be the test reporters.
  • The speed of execution of the testing suites.
  • The overall developer experience.

How we started

We started by migrating only one of our packages that didn’t use astro’s integration suite, create-astro. This was a good opportunity to play with the built-in assertion library node:assert, to learn about the options, and its performance compared to Mocha.

Since create-astro only had a handful of tests, it was relatively easy to migrate the test files to use node:test and node:assert, instead of mocha and chai. After that, the only thing left is to update the mocha command to node --test to execute the tests. However, we quickly ran into issues using the node --test command, including:

  1. It had trouble parsing the glob syntax when passing multiple arguments, e.g. node --test "test/*.test.js" --test-name-pattern="foo".
  2. It wasn’t possible to pass the --test-concurrency flag (only available in Node.js 21 and above), but can be worked around by using the programmatic API concurrency option.
  3. Nitpicking, but the argument names were verbose, e.g. --test-name-pattern instead of --match, -m arguments, --test-timeout instead of --timeout, -t arguments, etc.

Hence, to solve these issues, we created a custom script which can be called with the astro-scripts test command. This decision also proved to be useful to enable more workarounds as we’ll see later.

The Pandora’s box

In a second step, we attempted to migrate the testing suites of the @astrojs/node package. This integration is one of our most downloaded integrations, so we have plenty of tests. Plus, the tests of this package all have integration tests, so it was a good opportunity to check the performance of the test runner.

Once the PR was ready, we noticed that Node.js test runner was way slower than Mocha. We investigated, and we discovered that Node.js spawns a new process for each test file to assure that each testing suite is run in isolation. Running a testing suite in isolation is, generally, a good practice, because it assures that tests run in an unpolluted environment.

However, our testing suites are already isolated, in fact we were able to run our testing suites using the main thread with Mocha, without running into issues: side effects, polluted environments, etc. Unfortunately, Node.js didn’t provide an option to run all tests in the same thread, so we have to come up with a solution (Aren’t we engineers, after all? We solve problems!).

Using our internal astro-scripts test command, we are able to workaround this by creating a temporary file that imports all the testing suites, and we let Node.js test that single file. This way, only one process is spawned for the file, and we reach the same level of performance as if we were using the main process.

However, this came with a downside: if there’s a test failure or a timeout, we aren’t able to tell which test is the cause. This was the main quirk we found, and we accepted the trade-off. After all, we also accepted Mocha’s trade-offs!

Node.js assert and chai

During the migration, we had to remove the chai library for node:assert/strict. This task uncovered that with chai there are several different methods for the same assertion. For example, you can run an equality check at least in four different ways:

import { expect } from "chai";


expect("foo").to.eq("foo")
expect("foo").to.be.eq("foo")
expect("foo").to.equal("foo")
expect("foo").to.be.equal("foo")

From one point of view, it’s good to have this kind of flexibility, but on the other hand, the code of the tests becomes inconsistent. With the Node.js assertion module, you do this kind of check only in one way:

import { assert } from "node:assert/strict";


assert.equal("foo", "foo")

The Node.js assertion module provides almost all the functionalities we required, so the migration from chai wasn’t as painful as we thought. Our usage of chai was very minimal. However, we miss the .includes shortcut of chai:

import { expect } from "chai";


expect("It's a fine day").includes("fine")

The Node.js assertion module doesn’t provide such utility, so we ended up using the equality assertion with the String#includes function, which produces worse error messages on failure:

import assert from "node:assert/strict";


assert.equal("It's a fine day".includes("fine"), true)

Here comes the dragons

As mentioned before, we have a lot of test files, and we add new tests almost every day. Opening a one-off PR that migrates the whole monorepo is unfeasible, it would require a lot of work from one person, and keeping the branch updated can be stressful.

So we came up with a simple plan:

  1. Migrate first the small packages inside the monorepo
  2. Slowly migrate the main package - astro - by having Mocha and Node.js test runner in the same CI
  3. Remove Mocha

In order to achieve that, we asked help to our community. We thought this was the perfect opportunity to let people that aren’t familiar with Astro business logic to contribute to the project, and we could make the migration way faster.

We created and pinned an umbrella issue to coordinate the efforts. We used this issue as a coordination hub. Each contributor took ownership of the migration of each package, and they opened a PR for each package. Two new first-time contributors joined the efforts. It was a fantastic thing to see. In one week, we were able to migrate all packages!

Migrating the main package astro was a feat. It’s the package that contains the highest number of tests. In order to slowly migrate the tests, we had to come up with an out-of-the-box solution. We set up the Node.js test runner to test only the files called *.nodetest.js. Doing so, it allowed us to keep testing all files in the CI. Then, the rest was just a matter of coordinating the delivery:

  1. Use the umbrella issue to tell other contributors which files a contributor wanted to migrate.
  2. Rename the files to migrate from *.test.js to *.nodetest.js.
  3. Migrate the files.
  4. Open a PR, wait for a review and merge the PR.

With the help of @log101, @mingjunlu, @VoxelMC, @alexnguyennz, @xMohamd, @shoaibkh4n, @marwan-mohamed12, @at-the-vr and @ktym4a we migrated almost 300 test suites in one week!

The results

We are quite happy with the results. We haven’t seen any significant regression in the performance of our tests. The assertion module that Node.js provides has all the utilities we needed, and the describe/it pattern is supported, so the migration from Mocha was smooth.

There are still few hiccups regarding the developer experience, when comparing it to Mocha.

For example, to run one single test suite with Mocha you just need to add it.only to the specific test. However, in the Node.js test runner, the process involves a few more steps:

  1. Run the CLI using the --test-only argument.
  2. Append .only to the describe block that encloses the target it.only you want to run.
  3. If there are multiple describe blocks, each one must be marked with .only.

Another example is the use of the --test-name-patterns, which could be improved. This argument is used to run only the tests that match a particular name pattern. The DX isn’t great, because the CLI litters the terminal with messages about tests that don’t match (which aren’t run), which makes it difficult to understand which tests are actually run. Plus, the command is really slow, just for running tests that match some pattern.

Node.js test runner is still young, and it has all the right cards to become better.

The Node.js project is evaluating running tests using the main process, after we voiced our use case. In a sense, we can say that our efforts will improve Node.js!