# Write the alert rule and the runbook it points at

An alert with a runbook attached, back-tested against a window where it should have fired.

Category: Observability and operability
Source: https://letsship.ai/use-cases/write-the-alert-rule-and-runbook
Published: 2026-08-25

## Goal

Write an alert rule for a real failure condition, with the runbook it links to.

## Acceptance criteria

- The rule fires when replayed against a window where the condition occurred
- It stays quiet across a comparable healthy window
- The runbook exists and every step has been run
- It routes to a rotation rather than an unwatched channel

## What lands as proof

Replays against both a bad window and a good one, which together show the rule is neither blind nor noisy.

## Prompt

Copy this to an agent to open the ticket:

```text
Create a Linear issue in my team and assign it to the SHIP agent.

Title: Add the {condition} alert and its runbook

Description: Write an alert rule for a real failure condition, with the runbook it links to.

Acceptance criteria:
- The rule fires when replayed against a window where the condition occurred
- It stays quiet across a comparable healthy window
- The runbook exists and every step has been run
- It routes to a rotation rather than an unwatched channel

Before you create the issue, find a failure condition in this repository's services that currently has no alert, and replace {condition} with it.
```

## Questions

### What does the agent actually change?

The ticket is scoped to one outcome: write an alert rule for a real failure condition, with the runbook it links to. Work that serves that outcome is in scope, and anything outside it is left for a separate ticket, so the pull request stays reviewable.

### How do I know the work is done?

The pull request carries the evidence, not only the diff. Here that means rule back-tested against the window it should have fired in, so a reviewer can confirm the result without reproducing the work locally.

### How much oversight does this need?

The run stops once the plan is written. Somebody reads the approach and approves it before any code exists, which is the cheapest moment to redirect the work.