Visual Regression Testing — Automated UI Control

How to implement visual regression testing. Playwright screenshots, Percy, Chromatic, pixel comparison and strategies for design systems.

Introduction to Visual Testing¶

Visual regression testing automates the detection of unintended changes to your UI. Instead of manually checking every component and page after each change, visual tests capture screenshots and compare them against baseline images to identify visual regressions.

Key benefits include: - Automated UI verification - Catch visual bugs before production - Design system consistency - Ensure components look correct across updates
- Cross-browser testing - Verify appearance across different browsers - Regression prevention - Detect unintended changes early

Playwright Visual Testing¶

Playwright provides built-in visual testing capabilities:

import { test, expect } from '@playwright/test';

test('homepage visual test', async ({ page }) => {
  await page.goto('/');

  // Wait for content to load
  await page.waitForSelector('[data-testid="hero-section"]');

  // Take screenshot and compare
  await expect(page).toHaveScreenshot('homepage.png');
});

test('component visual test', async ({ page }) => {
  await page.goto('/components/button');

  // Screenshot specific element
  const button = page.locator('[data-testid="primary-button"]');
  await expect(button).toHaveScreenshot('primary-button.png');
});

Configure Playwright for visual testing:

// playwright.config.js
module.exports = {
  testDir: './tests',
  use: {
    // Global settings
    viewport: { width: 1280, height: 720 },
    screenshot: 'only-on-failure',
  },
  projects: [
    {
      name: 'chromium',
      use: { 
        ...devices['Desktop Chrome'],
        // Visual testing specific settings
        video: 'retain-on-failure',
      },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
  ],
  expect: {
    // Visual comparison settings
    threshold: 0.2, // 20% difference threshold
    mode: 'strict',
  },
};

Cloud Visual Testing Services¶

Percy by BrowserStack¶

import Percy from '@percy/playwright';

const percy = new Percy();

test('percy visual test', async ({ page }) => {
  await page.goto('/');

  // Capture for Percy
  await percy.screenshot(page, 'Homepage');

  // Multiple viewports
  await percy.screenshot(page, 'Homepage Mobile', {
    widths: [375, 768, 1280]
  });
});

Chromatic (Storybook)¶

// chromatic.yml (GitHub Actions)
name: Chromatic
on: push
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
      - run: npm ci
      - run: npm run build-storybook
      - uses: chromaui/action@v1
        with:
          projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}

Best Practices¶

1. Stable Test Environment¶

test.beforeEach(async ({ page }) => {
  // Disable animations for consistent screenshots
  await page.addStyleTag({
    content: `
      *, *::before, *::after {
        animation-duration: 0s !important;
        animation-delay: 0s !important;
        transition-duration: 0s !important;
        transition-delay: 0s !important;
      }
    `
  });

  // Mock dynamic content
  await page.route('/api/timestamp', route => {
    route.fulfill({
      json: { timestamp: '2023-01-01T00:00:00Z' }
    });
  });
});

2. Component-Level Testing¶

test.describe('Button Component', () => {
  test('primary button states', async ({ page }) => {
    await page.goto('/storybook/button');

    // Test different states
    const states = ['default', 'hover', 'disabled', 'loading'];

    for (const state of states) {
      await page.locator(`[data-state="${state}"]`).click();
      await expect(page.locator('.button-demo')).toHaveScreenshot(`button-${state}.png`);
    }
  });
});

3. Responsive Testing¶

const viewports = [
  { width: 375, height: 667 },   // Mobile
  { width: 768, height: 1024 },  // Tablet
  { width: 1280, height: 720 },  // Desktop
];

for (const viewport of viewports) {
  test(`responsive design ${viewport.width}x${viewport.height}`, async ({ page }) => {
    await page.setViewportSize(viewport);
    await page.goto('/');
    await expect(page).toHaveScreenshot(`homepage-${viewport.width}w.png`);
  });
}

4. Handling Dynamic Content¶

test('page with dynamic content', async ({ page }) => {
  // Mock API responses
  await page.route('/api/user', route => {
    route.fulfill({
      json: { 
        name: 'Test User',
        avatar: 'https://example.com/static-avatar.jpg'
      }
    });
  });

  // Hide or mock timestamps
  await page.locator('[data-testid="timestamp"]').evaluate(el => {
    el.textContent = '2023-01-01 12:00:00';
  });

  await expect(page).toHaveScreenshot();
});

Integration with CI/CD¶

# GitHub Actions
name: Visual Tests
on: [push, pull_request]

jobs:
  visual-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
      - run: npm ci
      - run: npm run build
      - run: npx playwright install
      - run: npx playwright test --reporter=html

      - name: Upload test results
        uses: actions/upload-artifact@v3
        if: always()
        with:
          name: playwright-report
          path: playwright-report/

Visual regression testing is essential for maintaining UI quality at scale. Start with critical user journeys, establish baseline screenshots, and gradually expand coverage to include component-level testing for your design system.