Automated Build & Test: A Merge-Gating Test Pyramid

The challenge

A complex real-time application has many moving parts — logic, integration, performance, and endurance behaviour — and each fails in a different way. A single flat test suite is either too slow to run often or too shallow to catch real regressions.

Our approach

We structure tests as a pyramid. The base is a large set of fast, isolated unit tests that run with no rendering and give near-instant feedback. The middle tier is integration and performance tests that run in a headless harness with explicit frame-rate and memory assertions. The top tier is soak and endurance runs sustained for minutes to catch slow leaks and frame-rate decay.

A lint gate runs bug-focused static checks before merge — catching undefined names, mutable default arguments, and closure bugs. The rule is simple: only an all-green run can merge, and any red gate blocks it.

Technical specifics

Thousands of fast unit tests running in roughly two and a half minutes for local feedback.
Hundreds of integration and soak tests, opt-in, run in a headless harness with FPS and resident-memory (RSS) assertions.
Five-minute endurance runs that detect memory growth and frame-rate decay over time.
A ruff lint gate with bug-focused rules as a pre-merge check.
Coverage across physics, combat, AI, pathfinding, save/restore, damage routing, and GPU microbenchmarks.

Example configuration

Markers separate the fast suite (default) from opt-in integration and soak tiers; ruff enforces bug-focused rules; performance tests assert on real metrics.

pyproject.tomltoml

[tool.pytest.ini_options]
# Default run = fast suite only; integration/soak are opt-in.
addopts = "-m 'not integration and not soak'"
markers = [
  "integration: headless, rendering-dependent",
  "soak: multi-minute endurance runs",
]

[tool.ruff.lint]
# Bug-focused: undefined names, mutable defaults, closure bugs.
select = ["F", "B006", "B023"]

test_performance.pypython

import pytest

@pytest.mark.integration
def test_frame_rate_stays_above_threshold(headless_game):
    fps = headless_game.run_seconds(10).mean_fps
    assert fps >= 60, f"frame rate decayed to {fps:.1f}"

@pytest.mark.soak
def test_no_memory_growth_over_five_minutes(headless_game):
    start = headless_game.rss_mb()
    headless_game.run_seconds(300)
    assert headless_game.rss_mb() - start < 25   # MB

Outcome

Developers get rapid feedback from the fast suite, while a comprehensive 3,000+ test run gates every merge. Nothing ships unless the entire suite is green — a track record of full-green cycles with zero failures.