A browser test failure is only useful if someone else can replay the conditions that produced it. A stack trace alone rarely does that. What usually gets teams unstuck is a small, consistent failure bundle: enough browser test evidence to answer four questions quickly, what happened, where did it happen, what did the browser say, and what changed on the network.
The trap is over-collecting. Teams often save every artifact they can, then spend more time triaging noise than reproducing the bug. The better approach is to standardize a minimum reproducible evidence bundle and capture only the artifacts that explain browser behavior, timing, and environment drift.
The short answer
If your goal is to capture a browser test failure bundle that actually helps reproduction, start with these five items:
- Failure summary, test name, environment, browser, version, viewport, timestamp, run ID.
- Screenshot at failure time, or a short burst of screenshots if the failure is timing-sensitive.
- Video recording, when the bug depends on animation, loading order, or transient UI state.
- Console logs, especially uncaught errors, warnings from the app, and browser security errors.
- Network logs or HAR file, so engineers can inspect requests, responses, redirects, and resource timing.
If a bundle does not let an engineer answer “what did the browser see?” and “what did the page request?” it is usually not enough to reproduce an intermittent failure.
What counts as evidence, and what is just noise
A useful bundle is not a dump of everything your framework can export. It is a curated set of artifacts tied to a specific failure.
Keep
- The exact browser and browser version
- The OS or container image
- Viewport or device profile
- The test identifier and failing step
- Screenshots or video from the failure window
- Console output, including stack traces and browser errors
- Network activity, ideally as a HAR file or equivalent request log
- Any test retries, because flakiness often appears only on the first attempt
Usually ignore unless the case demands it
- Full page source snapshots from every step
- Repeated screenshots from stable steps
- Generic environment dumps with no browser context
- Huge app logs unrelated to the failing action
- Raw CI logs without timestamps or step markers
The point is to reduce the time between “it failed” and “I can see why.”
A practical bundle standard
A good standard is simple enough to automate and strict enough to compare across runs. I would define a failure bundle like this:
| Artifact |
Why it matters |
Keep by default? |
| Run metadata |
Reproduce browser, OS, viewport, and test context |
Yes |
| Failure screenshot |
Shows visible UI state at the failure moment |
Yes |
| Video |
Reveals timing, animation, focus, scrolling, and transient overlays |
Yes for flaky UI failures |
| Console logs |
Captures JS errors, warnings, CSP issues, and promise rejections |
Yes |
| Network log / HAR |
Shows request failure, slow response, redirect, and auth problems |
Yes |
| DOM snapshot |
Useful when screenshot is ambiguous or the app is inaccessible |
Optional |
| Full trace |
Valuable when supported, but can be heavy and harder to share |
Optional |
The bundle should be named and stored the same way every time. For example:
text
//////
That convention makes it easier to search, compare, and attach the right artifact to the right failure ticket.
## What to capture first when a flaky test fails
If your test can fail in multiple ways, the first artifact should explain the failure mode, not just preserve it.
### 1) Screenshot at the point of failure
A screenshot is the fastest way to confirm whether the problem is a locator issue, a stale element, a modal overlay, a layout shift, or a missing page state.
Capture it at the exact assertion or action that failed, not just at test teardown. If you wait until cleanup, the page may already have changed.
### 2) Console logs
Console output is often the shortest path to root cause for browser automation failures. Useful entries include:
- Uncaught JavaScript errors
- Promise rejection messages
- CSP violations
- CORS or mixed-content warnings
- Failed feature detection or third-party script errors
For Playwright, see the official guidance on [screenshots](https://playwright.dev/docs/screenshots), [video](https://playwright.dev/docs/videos), and [tracing](https://playwright.dev/docs/trace-viewer). Selenium users need to rely more on driver logging, browser logs, or protocol access, depending on the browser and grid setup.
### 3) Network evidence
A flaky UI failure is sometimes a backend or asset delivery issue wearing a browser mask. HAR files and request logs help answer:
- Did the request go out?
- Did the response return the expected status code?
- Was the response slow enough to trigger a race?
- Did a redirect or auth flow change the page state?
- Did a static asset fail to load, changing the DOM or layout?
If you capture a HAR, include the user action that triggered it and the test step that observed the failure. A HAR without context is hard to replay.
## When video is worth the storage cost
Video is not always necessary. It is expensive in storage, slower to upload, and harder to skim than logs. Still, it is the best artifact for failures caused by timing, such as:
- menu hover behavior
- modal open and close races
- scrolling into view
- animation and transition delays
- duplicate clicks
- spinners, skeletons, or hydration delays
Video is less useful for hard assertion failures where a single screenshot and the network log already explain the issue.
A practical rule is to record video for **failure-prone interaction flows**, not for every green test. If your pipeline records every run, set a retention policy so only failed or quarantined runs keep full media.
## How to standardize the bundle so it is actually reusable
Standardization matters more than artifact count. Without it, engineers waste time hunting for the right file, the right run, or the right environment.
### Use the same capture points everywhere
Capture artifacts at the same moments in every test suite:
- on test start, store run metadata
- on each failure, store screenshot and logs
- on final failure, store video and network evidence
- on retry, preserve the first failure separately
That last point matters. Retries can hide the original problem. If the second run passes, you still need the first failure bundle.
### Add the browser context to the filename
Browser version and viewport often explain reproducibility gaps. A bug that only appears in Chromium 124 at 1365x768 is not the same bug as one in Firefox at mobile width.
### Keep timestamps in UTC
When multiple CI nodes or grids are involved, local time can make a bundle harder to compare. UTC timestamps and a single run ID reduce ambiguity.
### Pair artifacts with step names
A bundle is easier to use when each artifact is tied to a human-readable step, such as `checkout-add-item` or `login-submit`. That makes it easier to correlate the evidence with the failing action.
## Example: Playwright failure capture pattern
Playwright makes this relatively straightforward because it can record screenshots, video, and traces per test or per failure. A minimal pattern looks like this:
```typescript
import { test, expect } from '@playwright/test';
```
```typescript
test('adds an item to cart', async ({ page }) => {
await page.goto('https://example.com');
await page.click('[data-testid="add-to-cart"]');
await expect(page.getByText('Added to cart')).toBeVisible();
});
```
In configuration, you can enable media capture for failures and keep it focused on debugging rather than permanent storage:
```typescript
import { defineConfig } from '@playwright/test';
```
export default defineConfig({
use: {
screenshot: 'only-on-failure',
video: 'retain-on-failure',
trace: 'retain-on-failure'
}
});
That gives you a reproducible trail without forcing every passing run to generate large artifacts.
## Example: browser log capture in Selenium-based flows
Selenium setups usually need a bit more plumbing because logging support varies by browser, driver, and grid. The key idea is the same, though, capture browser-side errors close to the failure.
```python
from selenium import webdriver
```
options = webdriver.ChromeOptions()
options.set_capability('goog:loggingPrefs', {'browser': 'ALL'})
driver = webdriver.Chrome(options=options)
try:
driver.get('https://example.com')
# test steps
finally:
logs = driver.get_log('browser')
print(logs)
driver.quit()
If your grid or browser setup does not support the log type you want, do not fake it with unrelated application logs. Use the browser or driver mechanism that is actually supported by your stack.
## What not to do
### Don't capture everything by default
A 500 MB artifact bundle is not more useful than a 20 MB one if both contain the same clue. Large bundles slow uploads, increase storage cost, and make people avoid opening them.
### Don't rely on screenshots alone
A screenshot tells you what the page looked like, but not why it looked that way. It does not show whether the app requested the wrong data, whether the API returned a 500, or whether the UI changed after the assertion.
### Don't overwrite the original failure
If a test retries, store the first failure separately. Otherwise you lose the evidence that explains the flake.
### Don't treat local repro and CI repro as equivalent
Browser version, font rendering, viewport, throttling, and auth state can differ enough to change the outcome. If the issue only appears in CI, the bundle needs CI metadata, not just local repro notes.
## A decision framework for teams
Different teams need different bundle depth. Use this rule of thumb:
- **Mostly DOM assertion failures**, capture screenshot, console logs, and run metadata.
- **Timing-sensitive UI failures**, add video and possibly trace data.
- **Network-related failures**, keep HAR or request/response logs.
- **Cross-browser regressions**, always include browser version, OS, and viewport.
- **Grid-only failures**, include node or container identifiers, because the host can matter.
If your team is small, optimize for readability and quick triage. If your team is large or distributed, optimize for standardization and searchable storage. The bundle format should reflect the bottleneck you actually have, not the one you wish you had.
## A minimal checklist you can adopt this week
Use this as a working standard for the next flaky failure:
- [ ] Test name and failing step
- [ ] Browser name and version
- [ ] OS or container image
- [ ] Viewport or device profile
- [ ] Failure screenshot
- [ ] Console logs
- [ ] Network log or HAR
- [ ] Video for timing-sensitive failures
- [ ] First failure preserved, even if a retry passes
- [ ] Artifact path or run ID attached to the bug ticket
If you can answer the reproduction question from that bundle without asking for another run, the bundle is doing its job.
## FAQ
### Is a screenshot enough to file a browser bug?
Usually not. A screenshot can show the symptom, but it rarely explains whether the failure came from app state, timing, console errors, or the network.
### Should every failed test save a video?
No. Save video for failures where timing, animation, or transient UI state could change the diagnosis. For simple assertion failures, screenshots and logs are often enough.
### What is the difference between a network log and a HAR file?
A HAR file is a structured HTTP Archive that records page requests and responses. A generic network log may be less structured and may not include the same detail.
### Why do flaky tests need the first failure if a retry passes?
Because the first failure is often the real one. A passing retry can hide the original timing, state, or network problem.
### What should be in the failure metadata at minimum?
Test name, run ID, browser version, OS, viewport, timestamp, and failing step. Without those, the artifacts are much harder to reproduce.
### Should I keep full traces for every run?
Only if the storage and processing cost is acceptable. For many teams, retaining traces only on failure gives most of the debugging value with less overhead.
