Test strategy

Each claim of the thesis has a type of test that can show it. A rule with an exact value gets a unit test with fake time. A fact about a browser gets a test in that browser. A claim of agreement with web-vitals gets a test that operates the two libraries on the same page. The results page shows the latest run of all tests.

The kinds of tests

Unit tests

Each monitor gets its browser APIs from outside, as duck-typed dependencies (refer to the architecture). Thus a unit test gives each monitor fake timers, a fake clock and fake observers. The tests are fast, and they give the same result each time.

The fakes copy the behavior of the browsers that the research found. For example, the fake PerformanceObserver keeps entries for takeRecords(), and the fake lifecycle sends the events in the sequence of Chromium. Each test examines one behavior, and its name states that behavior.

An adversarial review of the library wrote a proof test for each defect that it found. The unit tests include these tests. Each one first failed, and passes after its fix. Examples are a second report of a page view after pagehide, and a hang that two tabs reported two times.

Browser tests

The browser tests use Vitest browser mode with Playwright. Each test file operates in Chromium, Firefox, WebKit and Chrome. On macOS, the same files also operate in Safari, through safaridriver (the browser (safari) project, with LAG_SAFARI=1). They also operate in Safari in the iOS Simulator (the browser (ios) project, with LAG_IOS=1). LAG_IOS_UDID selects the booted simulator. The tests examine the facts that fake time cannot show:

• browser-facts.test.ts: the facts of the research about each engine. These are the supported entry types, and the order of the listeners at window (experiment E1). The test fails if an engine changes, and then the documents need an update.
• soft-navigation.test.ts (Chrome): a real soft navigation in a top-level page. The command clickInTopLevelPage opens the page and clicks with Playwright, because Chrome detects soft navigations only in the top-level frame.
• drift-accuracy.test.ts: the median lag of DriftLag on an idle page is less than 5 ms, or 10 ms in a window without focus. A 300 ms block gives more than 300 ms minus one step and 10 ms (20 ms without focus), and less than 340 ms.
• lifecycle.test.ts: the lifecycle machine notifies its subscribers before a listener that the page added earlier. It uses the time of the event, and it removes its listeners at the end.
• lag-monitors.test.ts: the monitors on a real page, and the INP of a real click on a button that blocks for 120 ms.
• worker.test.ts and hang-journal.test.ts: the worker heartbeat, a hang of 6.5 s, and the record of the hang in IndexedDB.
• worker-io.test.ts: the IndexedDB writes and the fetches of a worker during a block of 2 s. WebKit completes them only after the block, thus hang-journal.test.ts skips itself there after a probe (experiment E4). The test also measures OPFS, the Cache API, XMLHttpRequest and a WebSocket of a worker, with the expectations of each engine (experiment E6). A WebSocket server in Node (commands/probe-socket.ts) records when the message arrives.
• peer-tab.test.ts (experiment E7) and peer-hang-watch.test.ts: a second top-level page that blocks its main thread, and that the test closes during the block. The commands of commands/peer-page.ts open the page with Playwright, or as a new window with WebdriverIO in Safari. The test page watches the other page through BroadcastChannel and the Web Locks API, and the peer hang watch of the library reports the abandoned hang.
• stress.test.ts: the workload profiles of @lag/load (refer to stress tests).
• browser-apis.test.ts: the monitors that need one browser API, with the real API. Examples are the layout shifts, the real CPU pressure source, the visibility-state entries, the idle periods and performance.memory.

A test that needs an API of one engine skips itself in the other engines. The skip reason comes from features.ts, for example "Long Animation Frames (the long-animation-frame entry type) are Chromium-only." The results page shows each skip with its reason.

The web-vitals oracle

web-vitals-oracle.test.ts operates web-vitals 6.2.3 and the page-view vitals on the same page. The two libraries read the same entries with the same rules. Thus the values must be the same, with no tolerance:

• FCP, LCP and TTFB of the load.
• CLS in Chromium. In Firefox and WebKit, the browser has no layout-shift entries, and the two libraries report no CLS.
• INP of real clicks on buttons that block for 40, 100 and 180 ms, with the target, the type and the three phases of the attribution.

The oracle found a bug. The library reported a CLS of 0 in browsers without layout-shift entries. Those values were false "good" values in the CLS histogram. After the fix, the library reports no CLS there, as web-vitals does.

CDP tests

The CDP tests use the Chrome DevTools Protocol to make page states that a user makes:

• A hidden page: the timer monitors pause, and they continue when the page is visible.
• A frozen page: the timer monitors pause before the freeze, record no stall for it, and continue after the resume.
• CPU throttling: 4× throttling of the same work increases the drift and the worker lag.
• A virtual compute-pressure source: the monitor records each state of the source, as the ordinals 0 to 3.
• A checkpoint: the page-view vitals record one value for each vital, and the events, when the page becomes hidden.

CDP hides a page before it freezes it. Thus the monitors pause at the hidden transition, and no sample gets the discard reason frozen in these tests.

The CDP tests found a second bug. In Chromium on Windows, the 5 ms timer steps changed from 5.5 ms to 15.6 ms after a page was shown again. DriftLag then reported windows of 160 ms of lag on an idle page, and the worker saw less than 2 ms. After the fix, DriftLag follows a change of the timer granularity (refer to DriftLag).

A browser test found a third bug. DriftLag accepted a sustained load of equal tasks as a new timer granularity. Thus the lag of the load decreased to less than 15% within 4 s (experiment E5). The fix uses a probe of message tasks.

The browser test drift-load.test.ts then found two defects of the fix, with all four browsers at the same time. In Chrome, a probed step was shorter than an idle step. In Firefox, the steps were irregular after a load. The unit tests of DriftLag.load.test.ts simulate these behaviors on a simulated thread.

Overhead and soak

The overhead benchmark measures an idle page with all monitors and without them. It uses Performance.getMetrics of CDP, in 3 rounds of 10 s. The budgets come from these measurements and from the research:

The budgets depend on the machine. Thus the benchmark operates in pnpm results, but not in the CI workflow.

The soak test operates all monitors under a mixed workload for 3 minutes, or for LAG_SOAK_MS milliseconds. It measures the heap after a full garbage collection. A growth of more than 100 kB each minute after the warm-up is a leak. After stop(), no timer, animation frame, idle callback or worker heartbeat must stay.

Coverage and mutation testing

pnpm coverage measures the coverage of @mark1russell7/lag with V8. The command fails below the thresholds in packages/lag/vitest.config.ts.

pnpm mutation uses Stryker. Stryker makes many small changes to the code, for example - to +. It operates the unit tests on each change. A change that no test finds is a surviving mutant: a behavior that no test pins down. The command fails below the break threshold in packages/lag/stryker.config.json. The incremental mode tests again only the mutants whose code or tests changed.

Stress tests and workload profiles

The stress tests use the workload profiles of @lag/load to make main-thread load. A profile is a list of lag specs. Each spec has a weight and a distribution of durations.

Lag event durations of the heavy profile

Show the data as a table

Samples of each lag spec of the heavy profile (400 each, seed 42)

Lag spec	Weight	p50	p95	Maximum
heavy-cpu	5	96 ms	189 ms	244 ms
spike	1	79 ms	313 ms	1 s
gc-pressure	2	41 ms	67 ms	81 ms
layout-thrash	2	61 ms	99 ms	145 ms
loaf-long	1	120 ms	120 ms	120 ms

How the results get to this site

pnpm results collects the results:

1. It builds all packages.
2. It operates the unit tests with coverage, the browser tests, the overhead benchmark and the site tests. A reporter writes each test with its project and its engine.
3. It reads the coverage summary, if the unit tests of this run wrote it. With --skip-unit, the run has no coverage. It also reads the newest Stryker report if one exists, with the commit and the time of its Stryker run.
4. It writes one run to packages/site/public/data/results/. Git ignores this folder.

Some failures have no failed test. A beforeAll hook that fails skips the tests of its suite. An afterAll hook that fails keeps them passed, and an error at the import of a file gives no tests. Thus the reporter adds a failed test with the name Errors outside the tests for each file or suite with such errors. The command fails when a test or a budget failed, or when a step gave an exit code that is not 0.

The results page reads this folder. It shows the tests of each engine, the skips with their reasons, the coverage, the surviving mutants, the measurements and the budgets. A package with only skipped tests, or without tests, does not get the passed icon. A run with a failed budget shows as failed in the list of runs and on the home page, as in the collector.

The mutation report comes from the weekly mutation workflow, thus it is usually older than the run. The mutation view shows the commit and the date of the Stryker run. The Pages workflow also takes a run whose score is below the break threshold, thus a score that falls shows on the site.

Continuous integration

Limits

• Safari on iOS. The tests operate in Safari on iOS only in the iOS Simulator of a macOS runner. A simulator has the CPU and the power of the Mac, not of a phone. To open the first session, safaridriver needed up to 131 s and more than one try. Thus the iOS project has a connect timeout of 600 s.

  In the simulator, the click of safaridriver gives no INP, also in web-vitals. Thus the two INP tests skip themselves there.

• Low Power Mode. A macOS runner refuses Low Power Mode on AC power. Thus no browser test operates in Low Power Mode. DriftLag.power.test.ts uses the timer rule of the WebKit source instead: a grid of 30 ms for nested timers.

• Safari on a shared runner. Safari on a macOS runner of GitHub sometimes slows the timers of its window, for example by 9 s. Then a timing test fails for a reason that is not in the code. Thus the Safari job of CI reports its results, but it does not block a pull request.

  A stress profile that the browser slowed skips itself with the reason. The profile is slow if it took more than 1.5 times its time. It is also slow if a timer delay was more than 2 s while the worker saw less than 100 ms of block. A third case is a time without animation frames that is more than 2 s longer than the longest block. Safari on a runner sometimes renders no frames for its window. Then a load generator that waits for a frame stops the profile.

• Safari and the collector. Only macOS has Safari. Thus a macOS job exports its reports (--export-reports), and the Linux job of the site imports them (--import-reports). The converter removes the checkout folder of the macOS runner from the file names, thus a test file has the same name in each engine.

• The E2E tests need a free port. The Grafana stack uses port 3000 by default. Set GRAFANA_PORT when another program uses that port.

• Mimir labels. Mimir puts service.name into the job label. The E2E query accepts job or service_name, and a native or a classic histogram.

• Mimir and CORS. Mimir sends no CORS headers, thus a test page cannot read the answer of a query. A Vitest command sends the query from Node (commands/mimir.ts).

• Shared machines. The CI runners of GitHub are shared machines. In the first CI operation, on 8 October 2026, a runner stopped a page for 490 ms. Thus a test of an idle page or of a light load examines a median or a percentile, not the largest single window.