Like most modern react frontends, we use React Testing Library. It lets us write a tonne of integration tests by querying and interacting with the DOM. However, late last year, we started noticing some flakiness in our CI. Tests were failing at random by exceeding the 5 second timeout imposed by Jest.
The quick fix was of course to temporarily increase this timeout to allow us to keep shipping. But, with most temporary things, we knew this would come back to bite us if we didn’t look into what was going on. So during some downtime, I took a closer look and found a compromise to significantly speed up our tests.
The Problem
As recommended by React Testing Library, we reach for byRole queries as a first priority. This ensures our app remains accessible without having to put in much more effort. If for some reason we cannot use byRole our next preference is byText and finally byTestId. After profiling our tests however, we discovered that byRole queries are by far the slowest part of the test. In fact, they can be significantly slower than other query types.
This chart shows how long it takes to select one element as the number of rendered elements grows. As we can see, the execution time for byText and byTestId remain relatively constant as the number of elements grows. The problem however is that byRole queries grow linearly.
This was an issue for us since the pages where tests were failing were large settings-type pages with a tonne of elements. Some of our tests were taking multiple seconds to select a single element which all adds up when running in-band in CI.
The Compromise
We didn’t want to stop using byRole selectors. Quicker CI time vs an accessible application wasn’t a worthwhile tradeoff for us. Instead, we experimented with limiting the search space for our byRole queries.
To do this, we first select a container where we expect the element to be using a byTestId query. Once we have this container, we can use the within utility to restrict the search space for our byRole query and still get all of its advantages.
This is how we might go about improving the speed of a query on our settings page. Here we’re trying to select a particular checkbox to later assert its state or interact with it.
import { screen, within } from "@testing-library/react"
// 🐌
const devModeCheckbox = screen.getByRole("checkbox", {
name: /developer mode/i,
})
// 🚀
const devModeCheckbox = within(
screen.getByTestId("advanced-options-section")
).getByRole("checkbox", { name: /developer mode/i })Results
Our new technique to limit the search space for byRole queries involves minimal rework and gives us a huge increase in performance.
Apart from the obvious outlier at position 1, we can see a massive difference between the original byRole query and our new technique. We’re steadily converting our tests to use this new technique and are extremely happy about our CI times decreasing with every push.