Examples
Seven end-to-end patterns, from transactional enqueue to a durable agentic flow.
Documentation explains one concept at a time; real code combines several. This page shows seven complete patterns you can adapt directly. Every snippet uses the public API as shipped — the concept guides linked from each pattern explain the behavior behind it.
Transactional enqueue in an API route
The problem: a request creates an order and must guarantee a fulfillment job — but only if the
order commits. Pass your open transaction as the last argument to enqueue, and the job and the
order share one commit or one rollback.
import { Admin, Pool, Queue } from "@stablemates/workhorse";
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
const queue = new Queue(pool);
const admin = new Admin(pool);
export async function createOrder(orderId: string, items: string[]) {
const client = await pool.connect();
try {
await client.query("BEGIN");
await client.query("INSERT INTO orders (id, items) VALUES ($1, $2)", [orderId, items]);
await queue.enqueue("order.fulfill", { orderId }, { queue: "orders" }, client);
await client.query("COMMIT");
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release();
}
}If the insert fails, no job exists. If the enqueue fails, no order exists. There is no window where
one is visible without the other. TypeScript applications using Drizzle, Prisma, TypeORM, or Kysely
get the same guarantee through the adapter package's forTransaction boundary.
Multi-stage handler with checkpoints and a durable sleep
The problem: a trial signup should send a welcome email now and a follow-up in seven days, and neither email may ever send twice. Checkpoints make each send a restart boundary; the durable sleep releases the worker slot for the whole week.
worker.handle("trial.lifecycle", async (payload: { to: string; followUpAt: string }, context) => {
await context.checkpoint("welcome", () => mailer.welcome(payload.to));
await context.sleepUntil("follow-up-window", new Date(payload.followUpAt));
await context.checkpoint("follow-up", () => mailer.followUp(payload.to));
return { deliveredTo: payload.to };
});The sleep stores a named timer, drops the lease, and parks the job in scheduled — no worker slot
is held for seven days. When the timer is due, a worker runs the handler from the top; the saved
welcome checkpoint returns its stored value instead of sending again. A crash at any point
replays only the stages that never committed.
Nightly job synced on deploy
The problem: an invoice run must happen every night at 03:00 with no separate scheduler service. Declare the schedule in code and synchronize it during deployment; any worker watching the namespace fires it.
// Deployment step: reconcile the namespace to exactly this set.
await queue.syncSchedules("billing-production", [
{
name: "nightly-invoice-run",
schedule: "0 3 * * *",
job: {
type: "invoice.generate",
queue: "billing",
payload: { scope: "due" },
},
},
]);
// Worker process: evaluate and fire schedules for that namespace.
const worker = new Worker(queue, {
scheduleNamespaces: ["billing-production"],
});
worker.handle("invoice.generate", generateInvoices);
await worker.run();Competing workers cannot double-fire: each occurrence has a durable key from its namespace, name, and planned time, so PostgreSQL converges every racing call on one job. Removing the definition from the array disables it on the next deploy — code stays the owner of intent.
Webhook idempotency
The problem: payment providers redeliver webhooks, and each delivery must produce exactly one job. Build the idempotency key from the provider's event ID; a replayed delivery returns the original job instead of creating another.
export async function handleStripeWebhook(event: { id: string; type: string; data: unknown }) {
const jobId = await queue.enqueue(
"stripe.event",
{ eventId: event.id, eventType: event.type },
{
queue: "webhooks",
idempotency: {
key: `stripe:${event.id}`,
scope: "stripe-webhooks",
},
},
);
return { accepted: jobId };
}An equivalent replay returns the same job ID with no new job, event, or notification — the route
stays safely retryable end to end. If a request reuses the key with a different payload,
TypeScript and Python throw EnqueueIdempotencyConflictError; Go returns the matching typed error.
No SDK silently drops the difference. The handler still runs at least once, so keep its external
effects idempotent too.
Incident redrive script
The problem: a provider outage dead-lettered a batch of jobs, the provider has recovered, and an operator needs to replay exactly those failures — safely, and only once. Dry-run first, then reuse one incident request ID across pages so a crashed script can rerun without duplicating work.
const filter = { queue: "billing", errorName: "ProviderTimeout" };
const request = {
actor: "[email protected]",
reason: "provider incident INC-2041 resolved",
requestId: "INC-2041",
};
// 1. Preview what would be redriven. Creates nothing.
const preview = await admin.redriveMany(filter, request, { dryRun: true, limit: 100 });
console.log(`${preview.results.length} jobs eligible`);
// 2. Redrive for real, page by page, reusing the same request.
let cursor = undefined;
do {
const page = await admin.redriveMany(filter, request, { limit: 100, cursor });
for (const result of page.results) {
console.log(result.status, result.sourceJobId, "->", result.targetJobId);
}
cursor = page.nextCursor ?? undefined;
} while (cursor);Each redrive creates a fresh job that copies the queue, type, payload, and retry policy; the failed source stays unchanged as evidence, and each result identifies both source and target. Repeating a page under the same request ID returns the existing targets, so the script is safe to rerun after a crash.
Cooperative cancellation
The problem: a customer withdraws an order while its export job is running. Handler execution cannot be preempted safely, so cancellation is a durable request each SDK delivers through its handler cancellation primitive.
// Operator side: record who asked and why.
await queue.cancel(jobId, {
requestedBy: "[email protected]",
reason: "customer withdrew the order",
});
// Handler side: check the signal between units of work.
worker.handle("export.build", async (payload: { parts: string[] }, context) => {
for (const part of payload.parts) {
if (context.signal.aborted) throw context.signal.reason;
await uploadPart(part, { signal: context.signal });
}
return { exported: true };
});If the job has not started, PostgreSQL cancels it immediately with no invented attempt history. If
it is active, the worker's heartbeat delivers the request. TypeScript aborts context.signal,
Python cancels context.cancellation, and Go cancels the handler's standard context. A handler that
ignores that request loses its lease eventually, and recovery materializes the cancellation instead
of retrying.
Durable agentic flow
The problem: a model plans work, several tools run independently, and execution must survive a deploy while waiting for approval. Checkpoint the model call, join tools as child jobs, cross a durable timer, then wait for an idempotent signal.
worker.handle("agent.run", async ({ prompt, conversationId }, context) => {
const plan = await context.checkpoint("plan", () =>
model.plan(prompt, { idempotencyKey: `plan:${context.job.id}` }),
);
const tools = await context.runChildrenAll(
plan.tools.map((tool) => ({
name: tool.id,
type: "agent.tool",
payload: tool,
options: { queue: "tools", concurrencyKey: conversationId },
})),
);
await context.sleep("model-cooldown", cooldownMs);
const approval = await context.waitForSignal<{ approved: boolean }>("approval");
return { plan, tools, approved: approval.approved };
});Every boundary releases the lease and restarts the handler later, so names and child requests must
stay stable across replay. The repository's typescript/examples/agentic-flow.mjs demonstrates the
whole lifecycle; run it with pnpm example:agentic-flow against an installed Workhorse database.
Python consumers can run python/examples/lifecycle.py for retry, checkpoint, timer, child, signal,
and human-decision boundaries. Go consumers can run go/examples/orchestration/main.go for child,
signal, and human-decision boundaries. Both release lanes verify the public package imports.
Next
- Quickstart — run the crash-survival demo these patterns build on
- Durable execution — the rules behind checkpoints and sleeps
- Agentic flow — child tools, timers, and approval signals together
- Dead letters — the full redrive model behind the incident script
Exact transactional, delivery, and operator semantics: architecture reference.