Why Browser Tests Pass Headed but Fail Headless, and How to Debug the Gap

A test that passes in a visible browser and fails in CI is usually not mysterious, it is underspecified. Headed and headless runs can differ in viewport defaults, available fonts, GPU and compositor behavior, animation timing, and how the test waits for the page to settle. If you want to debug the gap efficiently, do not start by guessing at the framework. Start by classifying the failure: rendering, timing, or environment setup.

The fastest path is not to make headless behave like headed forever. It is to find which assumption the test made about the browser, the page, or the machine.

What “headed vs headless” really changes

Headed mode means the browser draws a visible window. Headless mode means the browser runs without a visible UI, but it still executes the same page and automation APIs. The important part is that “same browser” does not mean “same environment.”

The browser process can still differ in:

• default window size and device pixel ratio,
• available system fonts,
• GPU acceleration and compositor paths,
• animation and layout timing,
• how screenshots are rasterized,
• whether the CI container has display, sandbox, or shared-memory constraints.

That is why the search phrase browser tests pass headed but fail headless usually points to a test that accidentally depends on one of those variables.

Short decision tree: rendering, timing, or environment?

Use this first, before you chase individual assertions.

1) Does the failure change if you force the same viewport?

• Yes: suspect responsive layout, hidden controls, or screenshot drift.
• No: continue.

2) Does the failure disappear if you slow the test and add explicit waits?

• Yes: suspect timing, animation, stale elements, or async app state.
• No: continue.

3) Does the failure only appear in CI or only in containers?

• Yes: suspect fonts, sandboxing, GPU/compositor, shared memory, or missing OS packages.
• No: suspect a browser engine or framework-specific behavior, then reduce to a minimal reproduction.

4) Does the failure reproduce with screenshots or DOM snapshots only?

• Screenshot mismatch only: rendering path.
• DOM assertion mismatch: timing, state, or environment.
• Both: likely a real product bug or a test that is reading the page too early.

The real causes that matter

1) Viewport defaults and responsive breakpoints

Headed runs often open with a large desktop window. Headless runs may use a smaller default viewport unless your test sets one explicitly. That can move elements across breakpoints, hide labels, collapse navigation, or change the order of rendered content.

This is the first thing to normalize because it is cheap and high leverage.

Repro check

In Playwright, set the viewport and screenshot size explicitly:

import { test, expect } from '@playwright/test';

test.use({ viewport: { width: 1440, height: 900 } });

test('checkout opens', async ({ page }) => {
  await page.goto('https://example.com');
  await expect(page.getByRole('heading', { name: 'Checkout' })).toBeVisible();
});

In Selenium, set the window size before navigation:

from selenium import webdriver

options = webdriver.ChromeOptions() options.add_argument(‘–window-size=1440,900’) driver = webdriver.Chrome(options=options) driver.set_window_size(1440, 900)

Failure mode to watch

A test that clicks a menu item by coordinates may pass headed on a wide screen and fail headless when the menu collapses into a hamburger button. Prefer semantic locators and explicit viewport control.

2) Font availability and text metrics

Fonts change layout. They change text wrapping, line height, truncation, and screenshot pixels. A CI image that lacks the fonts used on developer laptops can make a button label wrap to two lines or shift a call-to-action below the fold.

This matters especially for visual assertions and for tests that click near text rendered inside flexible containers.

Repro check

Compare the computed font family and text metrics in both modes. If the app depends on a custom font, make sure the test environment serves and loads it consistently.

Useful checks:

• inspect getComputedStyle(element).fontFamily,
• wait for document.fonts.ready before taking screenshots,
• verify container images install the font packages your app expects.

Example wait before visual capture:

await page.goto('https://example.com');
await page.evaluate(() => document.fonts.ready);
await expect(page).toHaveScreenshot('home.png');

Failure mode to watch

A screenshot diff that only changes line wrapping is often a font problem, not a layout bug.

3) GPU and browser compositor differences

Headless rendering may use a different compositor path than a visible window, and that can affect anti-aliasing, subpixel placement, transforms, and animation frames. You do not need to know every browser-internal detail to debug it, but you do need to recognize the pattern: the DOM is correct, the screenshot differs.

Repro check

If a screenshot diff appears but the DOM is stable:

• compare the image with browser zoom at 100%,
• disable animation for the test,
• check whether transforms, sticky headers, or canvas content are involved,
• test on a machine with and without GPU acceleration if your CI allows it.

If the DOM is stable but the pixels are not, stop looking for selector problems and start looking for rendering paths.

4) Timing changes, animation, and race conditions

Headless mode can be faster, slower, or simply different enough that a race condition becomes visible. The test may click before the button is enabled, assert before a network request completes, or read content while a transition is still running.

The fix is not “add sleep.” The fix is to wait for the actual condition the user cares about.

Better waits

• wait for a role, text, or visible element,
• wait for network completion only when the UI genuinely depends on it,
• wait for an explicit state flag in the app if you control the app under test,
• disable animations in test mode when they do not matter to the scenario.

Playwright example:

await page.goto('https://example.com');
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByText('Saved')).toBeVisible();

Selenium example:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 10) wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, ‘button.save’))).click() wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ‘.toast-success’)))

Failure mode to watch

A test that uses time.sleep(2) may pass headed and fail headless depending on load. That is not stability, it is luck.

5) Missing OS dependencies in CI

Headless failures sometimes have nothing to do with headless mode itself. They come from the container or VM running the browser. Common examples include missing fonts, an incomplete shared library set, restricted sandboxing, or a tiny /dev/shm allocation.

If tests fail only in CI, verify the runtime before changing the test.

Checklist:

• confirm the browser version matches what the CI image actually installs,
• log the viewport, user agent, and OS image,
• check whether the container has enough shared memory,
• verify file downloads, permissions, and writable temp directories,
• make sure the CI environment can reach the app and any mocked services.

6) Screenshot mismatches that are not product regressions

Visual diffs can be real, but they can also be caused by unstable test setup. If only the screenshot changes, ask whether the diff is due to:

• viewport size,
• font fallback,
• animation frame timing,
• browser zoom,
• subpixel rendering,
• OS theme or scrollbars,
• dynamic content such as timestamps or ads.

The practical rule is simple: stabilize the page before you trust a screenshot assertion. Mask or ignore dynamic regions when they are not the point of the test.

A reproducible debugging workflow

Use the same order every time so the team can learn from each failure.

Step 1, make headed and headless identical where possible

Set:

• the same browser channel or version,
• the same viewport,
• the same locale and timezone if relevant,
• the same device scale factor if screenshots matter.

Step 2, capture evidence from both modes

Collect:

• full-page screenshots,
• DOM snapshot or HTML around the failure,
• console logs,
• network failures,
• browser version and CI image info.

Step 3, reduce the test to the first failing assertion

Strip the test until it still fails. If you cannot reproduce in 20 lines of code, the problem is not fully understood yet.

Step 4, decide whether the fix belongs in the test, app, or CI image

• Test fix: bad wait, brittle selector, implicit viewport assumption.
• App fix: layout bug, inconsistent responsive behavior, animation bug.
• CI fix: missing font, browser package mismatch, container resource issue.

What to change in your test suite

Normalize the environment at the suite boundary

Do this once per suite, not ad hoc in each test.

• explicit viewport,
• explicit locale and timezone when relevant,
• stable download directory,
• consistent screenshot settings,
• a documented browser version range.

Prefer user-facing assertions

Assertions based on text, roles, and visible state are less sensitive to rendering noise than raw coordinates.

Keep visual checks narrowly scoped

Use screenshots for layout and rendering regressions, not for every functional branch. If the page is dynamic, isolate the stable region.

Separate app readiness from page load

load or domcontentloaded does not mean the UI is ready. If your app hydrates, fetches data, or re-renders after mount, wait for the final state that the user can actually use.

Compact symptom-to-cause table

Not the best fit if you need a single universal fix

There is no universal “headless bug” toggle that makes flaky tests reliable. If the failure is caused by three different assumptions at once, the correct fix may involve the app, the browser configuration, and the CI image.

This guide is not for teams looking for a one-line workaround. It is for teams that want to remove the dependency that made the test fragile in the first place.

A practical baseline to keep in your repo

If you only standardize four things, make them these:

1. viewport,
2. browser version,
3. font availability,
4. explicit wait strategy.

That set eliminates a surprising amount of headed-versus-headless drift without turning your suite into a maintenance project.

FAQ

Why do browser tests pass headed but fail headless?

Because headed and headless runs can differ in viewport, fonts, rendering path, and timing. The test may be relying on an unstated browser or environment assumption.

Is headless mode less reliable than headed mode?

Not inherently. Headless mode usually exposes environment and synchronization problems more clearly, which can make brittle tests fail sooner.

What is the first thing to check when screenshots differ?

Check viewport size and font availability first. Those two causes account for a large share of non-functional visual drift.

Should I add sleeps to fix headless failures?

No. Replace sleeps with explicit waits for visible, enabled, or application-specific ready states.

How do I tell if the problem is rendering or timing?

If the DOM is correct but the pixels are not, suspect rendering. If the page state is wrong or an element is not ready, suspect timing.

Do browser and CI versions need to match exactly?

They do not always need to match exactly, but they should be intentionally managed. Version drift is a valid source of headed-versus-headless differences and should be documented.