Browser tests that pass on a cold run and fail only after a few navigations are some of the most frustrating failures to diagnose. The app looks stable, the selectors are correct, and the test often passes when rerun in isolation. Then a warm cache changes the story. A script that worked moments ago now loads a different code path, a stale asset, or a stateful client-side branch that your test did not expect.

If you have ever seen browser tests fail when cache is warm, you are usually looking at one of two broad problems. Either the test is depending on an initialization path that only happens on the first load, or the application is behaving differently because cached assets, service workers, local storage, or memory state changed the runtime environment. In practice, both can be true.

This guide focuses on how to debug those failures systematically, not by blindly clearing the cache and hoping the problem disappears, but by learning exactly what changed, why it changed, and where the test or app needs to be made more deterministic.

What “warm cache” actually means in browser automation

In browser automation, “cache” is usually shorthand for several overlapping layers of state:

  • HTTP cache for scripts, stylesheets, fonts, and images
  • Service worker cache, which can serve app shell resources or API responses
  • Local storage, session storage, and IndexedDB
  • In-memory JavaScript state inside a single browser session
  • CDN or proxy cache, which can affect the response you get from the server

A warm cache can mean the browser has already visited the site in the same context, or that the test environment reused a browser profile, or simply that your app’s own client-side runtime kept some state across navigations.

That distinction matters because the failure may not come from the browser cache in the classic sense. For example, a test may fail after the second navigation because a service worker begins serving assets from its own cache, or because a feature flag was stored in local storage during the first page load.

If a failure only appears after the first pass, assume hidden state until proven otherwise. Cached assets are one common cause, but they are not the only one.

Why warm cache failures are so hard to trust

Warm cache failures are hard to debug because they violate a common assumption in test design, that every run starts from a clean, identical state. In reality, browser automation often reuses expensive resources to speed things up. A reused browser context, a persistent profile, or a long-lived CI container can introduce state that changes the execution path.

These failures also create false confidence when you use a naive fix. Clearing cache before each test may hide the bug, but it can also mask a real production issue. Users often do have warm caches, and real browsers do cache aggressively. If the app only works on a cold load, that is a product bug or a release risk, not just a test bug.

There is also a class of timing bugs that only becomes visible when cached assets load faster. A page that accidentally depends on a delayed script may pass on a cold run because the delay gives another piece of state time to settle. On a warm run, the script arrives earlier and triggers a race condition.

First, classify the failure

Before changing code, classify the failure as one of these categories:

1. Cached asset changes app behavior

The app loads a different JavaScript bundle, stylesheet, or API response from cache. The cached version may be stale, partially updated, or incompatible with the rest of the page.

Typical signs:

  • A button appears or disappears only after the second load
  • A JS error occurs after navigation, but not on the first visit
  • DOM structure differs between runs without any code changes in the test

2. The test relies on first-load behavior

The app sets up something only during the initial visit, such as onboarding, a cookie banner, or a session bootstrap. The second load skips that path, and the test does not account for the changed state.

Typical signs:

  • The selector exists only on first load
  • A modal appears only once per browser profile
  • An API call is skipped on warm runs because the client believes data is already present

3. State leaked across tests

A previous test changes storage, authentication, feature flags, or routing state. The next test sees a warm browser and behaves differently.

Typical signs:

  • Failures disappear when the test file runs alone
  • Failures depend on ordering
  • Reusing browser contexts makes the problem worse

4. Browser or environment cache differs from your local run

CI, Docker, or Selenium Grid may reuse profiles differently from your laptop. The bug may not be in the browser at all, but in infrastructure.

Typical signs:

  • Reproduces in CI, not locally
  • Reproduces only in one browser version
  • Reproduces only when tests run in parallel

Reproduce the failure on purpose

The quickest way to stop guessing is to make the browser warm cache behavior explicit. You want to reproduce the issue in a controlled way, not just by rerunning the flaky test and hoping it shows up again.

In Playwright, reuse a persistent context deliberately

A persistent context preserves storage and can make caching and storage issues easier to inspect.

import { chromium } from 'playwright';

(async () => { const context = await chromium.launchPersistentContext(‘./tmp-profile’, { headless: true, }); const page = await context.newPage();

await page.goto(‘https://example-app.test’); await page.goto(‘https://example-app.test/dashboard’);

await context.close(); })();

This is not the only way to reproduce warm cache behavior, but it is a useful starting point when you suspect local storage, cookies, IndexedDB, or service worker state.

In Selenium, reuse a profile when needed

If your test stack uses Selenium, a custom profile can help you inspect whether the issue is tied to profile-level persistence.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options() options.add_argument(r’–user-data-dir=/tmp/selenium-profile’)

driver = webdriver.Chrome(options=options) driver.get(‘https://example-app.test’) driver.get(‘https://example-app.test/dashboard’) driver.quit()

Use this carefully. Persistent profiles are useful for reproducing bugs, but they can create more noise if you leave them in routine test runs.

Force a warm-cache path in a single test

A good debugging trick is to intentionally load the same page twice in one test and compare behavior.

import { test, expect } from '@playwright/test';

test('compare first and second load', async ({ page }) => {
  await page.goto('https://example-app.test');
  const firstTitle = await page.title();

await page.goto(‘https://example-app.test’); const secondTitle = await page.title();

expect(secondTitle).toBe(firstTitle); });

If the second load diverges, you have a reproducible signal that the page is not idempotent across browser state.

Inspect what changed between the cold and warm run

Once you can reproduce the problem, compare the two runs at the network and storage levels.

Check network responses, not just DOM

Do not assume the same URL returned the same content. A server response can differ because of cache headers, ETags, cookies, or a service worker interception.

In Playwright, capture the response headers and status code for the resources you care about:

page.on('response', async (response) => {
  const url = response.url();
  if (url.includes('/app.js') || url.includes('/api/config')) {
    console.log(response.status(), url, response.headers()['cache-control']);
  }
});

If the same page renders differently, compare whether the app bundle, configuration endpoint, or JSON payload changed between requests.

Inspect storage before and after navigation

Look at local storage, session storage, and IndexedDB. A warm cache problem is often really a state persistence problem.

typescript

const storage = await page.evaluate(() => ({
  localStorage: { ...localStorage },
  sessionStorage: { ...sessionStorage },
}));
console.log(storage);

If a test passes only when these stores are empty, the bug may be that your app assumes a missing key or stale value never happens.

Watch for service workers

Service workers can make browser cache flakiness much harder to understand because they intercept requests outside the normal page lifecycle. They can serve stale assets, stale API responses, or app shell files even after the server changes.

In Chrome-based automation, you can inspect service worker registration from the page context:

typescript

const registrations = await page.evaluate(async () => {
  return await navigator.serviceWorker.getRegistrations();
});
console.log(registrations.length);

If a service worker is present, check whether your test environment expects it. A test that is stable without a service worker can become flaky once one is installed.

Determine whether the problem is the app or the test

Not every cache-related failure is a test bug. Some are legitimate application defects that your test finally exposed.

It is probably a test bug if:

  • The test assumes one-time onboarding on every run
  • The test does not wait for a cached resource to settle before asserting
  • The test reuses browser state between unrelated cases
  • The test hard-codes a selector that only appears on the first load path

It is probably an app bug if:

  • The UI breaks when a script is loaded from cache but not from network
  • The app crashes because a cached bundle does not match the current HTML
  • A configuration request is cached too aggressively and the client never refreshes it
  • A service worker serves stale assets after a deployment

The difference matters because the fix is different. A test bug requires more isolated setup. An app bug may require versioned assets, better cache invalidation, or cache-busting deployment strategies.

Common root causes of warm cache test failures

1. Asset version mismatch

This is one of the most common causes. The HTML changes, but cached JavaScript or CSS does not, or vice versa. The page loads, but the DOM logic and markup are no longer aligned.

This often happens when you deploy HTML separately from bundled assets, or when a CDN serves a stale file after the server has already changed references.

A practical fix is to use content-hashed filenames for build assets, such as app.3f2c1.js, so the browser always fetches the correct version.

2. Non-idempotent initialization

The first load initializes global state, sets flags, or fetches bootstrap data. The second load skips that code path and the app assumes something exists that does not.

For example, a test might pass on the first page load because a global variable is populated by an inline script. On a warm navigation, the inline script does not execute the same way, and the page crashes when the variable is missing.

3. Session or local storage pollution

A previous test may leave behind user preferences, onboarding completion flags, feature toggles, or auth tokens. A later test sees the cached state and behaves differently.

A classic example is a dismissible banner stored in local storage. The first test sees it, dismisses it, and writes a flag. The second test never sees the banner, so the UI layout changes and a selector becomes invalid.

4. Service worker stale logic

Service workers can cache static assets or network responses, but a bad update strategy can leave the browser running old code long after a deploy.

If your test suite clears cookies but not service workers, you may still see stale behavior even though the browser seems “clean.”

5. Timing differences created by cache hit speed

A resource that loads from cache is much faster than a network fetch. Faster is not always better for test stability. A race that was hidden by slow loading may now surface.

For example, a test waits for a page element, but a cached script fires earlier and updates the DOM before the test starts listening. Now the assertion sometimes misses the event.

Debug with explicit browser state resets

When you suspect persistence, test with a clean slate at the right level.

Clear storage between tests

In Playwright, a new browser context is often enough for isolation, because it starts with fresh storage.

import { test } from '@playwright/test';

test('fresh context per test', async ({ browser }) => {
  const context = await browser.newContext();
  const page = await context.newPage();
  await page.goto('https://example-app.test');
  await context.close();
});

If a fresh context fixes the issue, but your normal suite reuses a context, you have identified state leakage.

Remove service workers during debugging

If the problem only happens with a service worker installed, temporarily unregister it to isolate the effect.

typescript

await page.evaluate(async () => {
  const regs = await navigator.serviceWorker.getRegistrations();
  for (const reg of regs) {
    await reg.unregister();
  }
});

Do not ship this as a normal test step unless you truly want to test behavior without service workers. Use it as a diagnostic tool.

Disable cache selectively, not blindly

You can disable cache at the browser level during debugging, but be careful. If you disable cache everywhere, you may stop reproducing the bug.

In Chromium automation, the DevTools protocol can disable cache for a debugging session:

typescript

const client = await page.context().newCDPSession(page);
await client.send('Network.enable');
await client.send('Network.setCacheDisabled', { cacheDisabled: true });

This is useful to confirm whether the failure depends on cache. If disabling cache makes the test pass, you have narrowed the issue, but not solved it. The next step is to find what state the cache was influencing.

Add diagnostics that tell you what path the app took

A flaky test is much easier to understand if the app exposes evidence of the branch it executed.

Log version identifiers

If your frontend has a build number, commit hash, or asset manifest version, print it in the test output or expose it in a debug header. When a warm cache failure appears, you can verify whether the browser actually loaded the same app version.

Capture request and response metadata

Log cache-related headers for suspicious resources:

  • cache-control
  • etag
  • last-modified
  • age
  • x-cache

These are especially helpful when a CDN or reverse proxy is involved.

Record client-side feature flags

If your application uses feature flags, make sure the test knows which flags were active. A warm cache may preserve the flag state even if the server changes it.

Use tracing when available

Modern browser automation tools can capture traces, screenshots, and network logs. A trace from the cold run and a trace from the warm run often reveals the moment behavior diverges. The point is not to produce more artifacts for the sake of it, but to make the runtime path visible.

Fix the test without hiding real issues

A stable test suite should not depend on accidental cache cleanup. The right fix is usually one of these.

Make each test independent

Each test should create its own browser context, seed its own data, and clean up after itself. That does not mean every test must start from an empty hard drive, but it should start from a predictable state.

Wait for the actual condition, not just the page load

If the page’s DOM depends on cached JS initialization, waiting for load may be too early or too late. Wait for the state that matters, such as a specific API response, UI marker, or application-ready flag.

Avoid cross-test reusing of browser sessions unless required

Reusing sessions for speed can be tempting, especially in large suites, but it makes cache-sensitive failures harder to reason about. Use reuse only where state continuity is part of the behavior under test.

Make selectors resilient to state changes

If a warm cache changes layout or suppresses a banner, selectors tied to exact positions or transient text will break. Prefer stable roles, labels, or test IDs that reflect the user action rather than the current UI arrangement.

Fix the app when cache behavior is the real defect

If the app itself is the problem, the cure is usually in how assets and client state are managed.

Version static assets correctly

Use content hashes for JS and CSS bundles, and ensure HTML references match the current build. This reduces the chance of a browser loading a new HTML shell with old code.

Make bootstrap logic idempotent

Your initialization code should behave correctly whether it runs once, twice, or after a reload with cached resources. A page that can only initialize from a completely blank state is fragile.

Use cache headers intentionally

Do not rely on defaults. Define which resources can be cached, for how long, and under what invalidation strategy.

Be careful with service worker updates

If your app uses a service worker, make its update flow explicit. Stale service workers are a classic source of browser cache flakiness because they can pin old behavior long after a deploy.

A practical debugging checklist

When a browser test fails only with warm cache, work through this sequence:

  1. Reproduce it with a deliberate second navigation or persistent profile
  2. Compare cold and warm network responses
  3. Inspect local storage, session storage, and IndexedDB
  4. Check whether a service worker is installed
  5. Verify whether test isolation is leaking state across cases
  6. Disable cache only to confirm the hypothesis, not as the final fix
  7. Decide whether the failure belongs in the test, the app, or both

The goal is not to eliminate all caching. The goal is to make cache behavior explicit enough that failures become explainable.

Examples of cache-sensitive failure patterns

A test expects a banner to appear, but a dismissal flag in local storage hides it after the first load. Fix the test by clearing storage or asserting the post-dismissal path separately.

Analytics or bootstrap script loads from stale cache

The app shell changes, but a cached bundle still references old code. Fix the deployment pipeline so the browser cannot mix incompatible HTML and assets.

Feature flag state persists across tests

A test passes locally because it starts from a fresh browser. In CI, a reused profile keeps old flag values. Fix by isolating browser contexts and resetting storage.

Service worker serves old API response

A request is fulfilled from the service worker cache even though the server data changed. Fix the service worker strategy or test against a cleared worker state when the scenario is not about offline behavior.

When to keep cache enabled in tests

It is tempting to disable cache everywhere once you have seen a warm cache failure, but that can reduce coverage. Keep cache enabled when you want to test real-world browser behavior, especially if your app depends on client-side navigation, progressive web app features, or offline support.

Keep cache enabled if:

  • Your app intentionally uses service workers
  • You want to validate update and invalidation behavior
  • You need to reproduce production-only state interactions
  • The bug appears only after the browser has already visited the app

Disable cache temporarily if:

  • You are isolating a suspected cache interaction
  • You need a clean baseline for a specific debugging run
  • You are ruling out a network layer issue

Final thought: warm cache failures are state bugs in disguise

When browser tests fail when cache is warm, the browser is usually telling you that some hidden dependency matters more than you expected. The hidden dependency may be a cached asset, but it can also be storage, service workers, a reused profile, or a timing change caused by faster delivery.

That is why the best fix is rarely “clear the cache.” Instead, identify which state changed, make that state visible in your test, and decide whether the app should tolerate it. Real users do not always browse from a cold start, and a browser automation suite that only passes on empty state is not proving much about the system you actually ship.

For more background on testing terminology and automation practices, it can help to revisit the basics of software testing, test automation, and continuous integration. Those concepts become much more concrete once you start tracing how browser state leaks from one run to the next.