Testfile, a self-documenting definition how to run your tests & a tooling to run them locally, on your CI/CD infrastructure, or in the cloud
A YAML file that describes the whole test setup, with the services your tests depend on — which makes runs reproducible, on your machine and on CI. Every run is recorded, and even CI runs can be inspected locally.
version: 0
test:
parallel:
- name: lint
command: npm run lint
- name: e2e
ports:
web: random
services:
web:
command: npm start
env:
PORT: ${{ ports.web }}
ready:
http: http://localhost:${{ ports.web }}/healthz
command: npm run test:e2etestfile start — that's the whole CI job.
Simple definitions
The setup is easy: a name and a command is a test, and
testfile init
writes the first version for you. Plain YAML, validated by a
schema, readable in every code review.
Scales with requirements
Grow from one command to sequences, parallel groups and matrix tests — with retries, conditions and result caching when the suite gets big. The same file runs locally, in watch mode and on CI.
Manages your services
A database — or any other service your tests need — is declared once and started on demand, as a container or a plain process: health-checked before the first test runs and reachable through random ports.
Unified viewer for local and CI runs
Browse recorded runs in the terminal (testfile tui)
or in the browser (testfile serve): statuses,
durations, per-test logs, service logs — and why a test ran
or came from the cache. A simple way to sync runs between machines,
including CI runs on your
computer, like GitHub Actions.
Everything in the box
The format
- Nested tests: sequences and parallel groups
- Matrix expansion across versions and variants
- One test per folder/file with
foreach - Includes for monorepos
- Conditions, tags and per-test timeouts
- Retries and tolerated failures
- Setup & teardown hooks, DAG ordering with
needs - Template expressions with defaults
- JSON schema validation & editor autocompletion
- A normative spec with a runner-independent conformance suite
Services & environment
- Processes or containers as services — the run picks the engine
- podman, docker, or pods on a kubernetes cluster
- Readiness by HTTP, TCP, log line or command
- Service dependencies, health-gated like
depends_on - Services shared across matrix instances
- Test bodies in containers, project mounted
- Random ports for parallel-safe runs
- Isolated environment with explicit
forwardEnv - Env files & secrets with masking
- Graceful teardown, also on Ctrl+C
Running & CI
- Filters by name, tag or matrix value
- Re-run what broke (
--failed) or what changed - Result caching with
inputs, sharding across machines - Watch mode and cache-aware
--dry-run - JUnit & JSON reports, collected artifacts
- GitHub Action: annotations, job summary, a status per test
- GitLab, Jenkins, Buildkite & CircleCI recipes
- TUI and web viewer with timeline, diffs & flaky detection
- Run sharing via S3, GitHub or GitLab; platform runs merged into one
- VS Code extension,
testfile initimporters,doctor, completions