# Test every pull request the way a careful user would

> Agentic end-to-end testing for every pull request: SHIP checks its acceptance criteria on a preview and attaches a browser trace and screenshots as proof.

Available now, for engineering teams.

SHIP's Tester checks each pull request on a preview against its acceptance criteria. Every verdict links to a browser trace and screenshots you can open.

- **For:** Teams shipping web apps with thin end-to-end test coverage.
- **Instead of:** Clicking through every preview yourself before you merge.

## Describe the outcome and keep the proof

1. **Write acceptance criteria in plain words.** Put them in the issue as a checklist. The Tester turns each one into steps it performs in a real browser.
2. **Run it on every preview.** Your own pipeline deploys the preview and the Tester drives a browser against it. A project with nothing to preview is built and exercised in the sandbox instead.
3. **Open the trace behind every verdict.** Each verdict links to a Playwright trace and screenshots that open in the browser, so you see exactly what was tested.

## Catch what unit tests miss

- **Turn a failure into a fix.** A criterion that fails goes back to a Builder with what the Tester expected and what it saw, and the Tester runs again on the fixed branch.
- **Sign in with steps you write in ship.yml.** Declare a sign-in that needs no secret, such as a demo login or a magic-link stub, and the Tester follows it before it checks a signed-in page. Password, SSO and MFA sign-ins are not supported yet.
- **Reject screenshots that prove nothing.** A capture that is blank or nearly uniform is refused before upload, so a pass never rests on an empty screenshot.
- **Pick the model that tests.** Set the Tester's harness and model in ship.yml, separately from the Builder whose work it checks.
- **Cover browser extensions and APIs too.** Without a preview, the Tester builds your project in its sandbox and exercises a web app, an unpacked browser extension or an HTTP API. SHIP detects which, and qaSurface in ship.yml overrides it.

## Compare SHIP with Momentic

- **SHIP:** Tests each pull request against its own acceptance criteria on a preview your pipeline deploys.
- **SHIP:** Sends a failure back to a Builder to fix before anyone reviews it.
- **SHIP:** Tests web apps in a real browser, plus browser extensions and HTTP APIs it builds in its sandbox.
- **Momentic:** Prices by usage credits with no per-seat charge; the free plan includes 2,000 credits a month. ([Pricing](https://momentic.ai/pricing))
- **Momentic:** Runs end-to-end tests for web, iOS and Android apps, written in plain English and stored as YAML files in your repository. ([llms.txt](https://momentic.ai/llms.txt))
- **Momentic:** Updates tests when the UI changes, and its GitHub App can open repair pull requests. ([GitHub integration](https://momentic.ai/docs/integrations/github))
- **Momentic:** Keeps video, step screenshots, and console and network logs for every run. ([Infrastructure](https://momentic.ai/infrastructure))

Last reviewed October 1, 2026, from each product's own pages.

## Frequently asked questions

### Can it test pull requests SHIP did not build?

Yes: run ship delegate pr --at qa on any open pull request in a connected repository, and SHIP's Tester checks it without a review first. In a mission SHIP builds, the Tester checks the pull request before it is marked ready to merge. [Hand over a branch](https://letsship.ai/docs/cli/commands#hand-over-a-branch)

### Which apps can it test?

Web apps in a real browser, unpacked browser extensions, and HTTP APIs. Mobile apps are not covered yet.

### When is Momentic the better fit?

Momentic fits a team that wants a maintained end-to-end regression suite across web, iOS and Android apps, written in plain English and stored as YAML in its repository, as Momentic's own pages describe. SHIP fits when each pull request should be tested against its own acceptance criteria inside the loop that builds and fixes it.

### Can it test pages behind a login?

Yes, when the sign-in needs no secret. Declare its path and steps under qa.signIn in ship.yml, such as a demo login or a magic-link stub, and the Tester follows them on the preview. Sign-ins that need a password, SSO or MFA are not supported yet, so criteria behind them fail with that reason. [QA sign-in](https://letsship.ai/docs/configuration/qa-sign-in)

### Do I need a preview environment?

No. Without one, the Tester builds and serves your app inside its sandbox and tests it there. SHIP detects how from your project, and qaSurface in ship.yml overrides it. [No preview environment](https://letsship.ai/docs/configuration/deployments#no-preview-environment)

### How is testing billed?

Testing runs inside a mission, and one SHIP credit covers one mission of up to 3 agent-hours, with one more credit for each further 3. Credits come back when a mission ends in failure or needs attention with no pull request. Model usage is billed by your own provider.

## Put a tester on every pull request

Start a mission and let every change come with the proof that it works.

Source: https://letsship.ai/platform/agentic-testing