# Test every pull request the way a careful user would

> Agentic end-to-end testing for every pull request: SHIP checks the acceptance criteria on a preview and attaches a browser trace and screenshots as proof.

Available now, for engineering teams.

SHIP's Tester checks each pull request on a preview against its acceptance criteria. Every result links to a browser trace and screenshots you can open.

- **For:** Teams with web apps and few end-to-end tests.
- **Instead of:** Manual checks of every preview before you merge.

## Describe what should work, then open the proof

1. **Write acceptance criteria in plain words.** Add them to the issue as a checklist. The Tester turns each one into steps it performs in a real browser.
2. **Run it on every preview.** Your own pipeline deploys the preview, and the Tester uses a real browser to check it. If the project has nothing to preview, the Tester builds and runs it in its sandbox instead.
3. **Open the trace behind each result.** Each result links to a Playwright trace and screenshots that open in your browser, so you see exactly what was tested.

## Catch what unit tests miss

- **Send failures back to the Builder.** When a criterion fails, the Builder gets what the Tester expected and what it saw. The Tester runs again once the branch is fixed.
- **Sign in with steps from ship.yml.** Add a sign-in that needs no secret, such as a demo login or a magic-link stub. The Tester follows it before it checks a signed-in page. SHIP does not support password, SSO or MFA sign-ins yet.
- **Reject blank screenshots.** A capture that is blank or almost one color is refused before upload, so a pass never depends on an empty screenshot.
- **Pick the model that tests.** Set the Tester's harness and model in ship.yml, separately from the Builder whose work it checks.
- **Cover browser extensions and APIs.** Without a preview, the Tester builds your project in its sandbox and tests a web app, an unpacked browser extension or an HTTP API. SHIP detects which one, and qaSurface in ship.yml overrides it.

## Compare SHIP with Momentic

- **SHIP:** Tests each pull request against its own acceptance criteria on a preview your pipeline deploys.
- **SHIP:** Sends a failure back to a Builder to fix before anyone reviews it.
- **SHIP:** Tests web apps in a real browser, plus browser extensions and HTTP APIs it builds in its sandbox.
- **Momentic:** Charges by usage credits, not per seat. The free plan includes 2,000 credits a month. ([Pricing](https://momentic.ai/pricing))
- **Momentic:** Runs end-to-end tests for web, iOS and Android apps, written in plain English and stored as YAML files in your repository. ([llms.txt](https://momentic.ai/llms.txt))
- **Momentic:** Updates tests when the UI changes, and its GitHub App can open pull requests that repair them. ([GitHub integration](https://momentic.ai/docs/integrations/github))
- **Momentic:** Keeps video, step screenshots, and console and network logs for every run. ([Infrastructure](https://momentic.ai/infrastructure))

Last reviewed October 1, 2026, from each product's own pages.

## Frequently asked questions

### Can it test pull requests SHIP did not build?

Yes. Run ship delegate pr --at qa on any open pull request in a connected repository, and the Tester checks it without a review first. In a mission SHIP builds itself, the Tester checks the pull request before it is marked ready to merge. [Hand over a branch](https://letsship.ai/docs/cli/commands#hand-over-a-branch)

### Which apps can it test?

It tests web apps in a real browser, unpacked browser extensions and HTTP APIs. It doesn't test mobile apps yet.

### When is Momentic the better fit?

Momentic fits a team that wants a maintained end-to-end regression suite for web, iOS and Android apps, as Momentic's own pages describe. The team writes its tests in plain English, and Momentic stores them as YAML in the team's repository. SHIP fits when the Tester should check each pull request against its own acceptance criteria, in the same loop that builds and fixes it.

### Can it test pages behind a login?

Yes, when the sign-in needs no secret. Add its path and steps under qa.signIn in ship.yml, such as a demo login or a magic-link stub, and the Tester follows them. SHIP does not support sign-ins that need a password, SSO or MFA yet, so criteria behind them fail with that reason. [QA sign-in](https://letsship.ai/docs/configuration/qa-sign-in)

### Do I need a preview environment?

No. Without one, the Tester builds and serves your app inside its sandbox and tests it there. SHIP works out how from your project, and qaSurface in ship.yml overrides it. [No preview environment](https://letsship.ai/docs/configuration/deployments#no-preview-environment)

### How does SHIP bill for tests?

A test is part of a mission. One SHIP credit covers a mission of up to 3 agent-hours, and each further 3 agent-hours, or part of them, uses one more. Credits come back when a mission fails or needs attention without a pull request, and your own provider bills model usage.

## Put a tester on every pull request

Start a mission, and each change comes back with proof that it works.

Source: https://letsship.ai/platform/agentic-testing
