Your print vendor will be down at some point. So will the CDN in front of it and the cloud region behind it. The useful question is not whether, but what your system does during the twenty minutes when POST /v1/print-jobs returns nothing, and what you do in the hour after.
This applies to any service that turns an API call into a physical object. We use our own product as the worked example because we know exactly where it fails.
Every control plane fails, including ours
The recent record is short and specific. On November 18, 2025, a Bot Management feature file doubled in size and Cloudflare's core network stopped delivering traffic for several hours (Cloudflare's post-mortem). On February 20, 2026, a BGP route withdrawal took down customers using Bring Your Own IP (post-mortem). On October 19 and 20, 2025, a race condition in DynamoDB's DNS management produced an empty DNS record for the us-east-1 endpoint, and the failure cascaded into Network Load Balancer health checks and EC2 launches for most of a day (AWS post-event summary).
These are the best-run infrastructure teams on the internet. We will have an outage too. The honest design assumption is that the control plane between your backend and the printer is unavailable for some window every year, and the window is long enough for a shift to notice.
RocketPrint is a remote printing API and desktop Station for software platforms: your backend POSTs a print job, and a small app beside the printer prints it, with no inbound networking, no drivers for raw label formats, and a durable job record. That architecture, described in how a cloud printing API reaches a printer behind a firewall, has one consequence that matters here: the Station dials out to us. If we are unreachable, no new job reaches the printer until we are back.
Four questions to answer before the outage
Write the answers into your runbook. If any answer is "I do not know," that is the work.
| Question | What a good answer looks like |
|---|---|
| What happens to jobs already in flight? | You know which state each job was in and which ones the agent already holds |
| What happens to new jobs? | They are stored on your side with a stable identity and submitted later without duplicates |
| What does the floor do? | Operators have a fallback that does not depend on the vendor, and know when to use it |
| How do you reconcile afterward? | You can list what the vendor recorded, compare it to what you intended, and fix the difference |
In-flight jobs: what the agent already holds is what will print
Split in-flight work by where the bytes are. A job that reached the desktop agent will print or fail on its own, because the agent has the content and the printer is local. A job the API accepted but had not yet delivered will wait on the vendor's side. A job your process was still sending when the connection dropped is the dangerous one, because you do not know whether it was accepted.
In our API these map to states. sent, printing, and completed mean the Station has the bytes. pending means we saved the job and had not delivered it; dispatching means one server leased it for delivery. If our API is down but the Station is still running, the Station cannot report outcomes to us, so a job may print while our record still says sent. When the socket comes back, the Station replays saved outcomes from its own durable record, the same mechanism that keeps a redelivered job from printing twice (see how idempotency stops duplicate labels).
New jobs: queue on your side under the same idempotency key
The pattern is an outbox. When your business decides a label should exist, write that decision to your own database with a deterministic key, and let a worker submit it. If the vendor is down, the row sits in the outbox. When the vendor returns, the worker submits it with the same key, and the vendor either creates the job or returns the one it already created.
// Outbox row: { id, idempotencyKey, printerId, content, attempts, jobId, nextAttemptAt }
async function drainOutbox(db, rp) {
const rows = await db.outbox.due({ limit: 50 });
for (const row of rows) {
try {
const job = await rp.printJobs.create(
{ printerId: row.printerId, contentType: 'raw_base64', content: row.content, expiresInSeconds: 1800 },
{ idempotencyKey: row.idempotencyKey, timeoutMs: 10_000, retries: 2 },
);
await db.outbox.markSubmitted(row.id, job.id);
} catch (err) {
const delay = Math.min(60_000, 1000 * 2 ** row.attempts);
await db.outbox.reschedule(row.id, Date.now() + delay);
}
}
}
Two details carry the weight. The key is derived from the business object (order-1042-shipping-label), not generated per attempt, so a request that timed out mid-flight is safe to send again. And the expiry is short. A label submitted during an outage and delivered forty minutes later may print onto a parcel that already left. expiresInSeconds accepts 10 to 86,400 with a default of 3,600; we set 1,800 above so stale work becomes expired rather than paper. The tradeoffs are in what to do when a warehouse printer goes offline.
The floor needs a fallback that does not need the vendor
Operators cannot wait for a status page. The cheapest fallback is the one browser printing always had: the document exists on your server, so let a human open it and print it locally. For PDFs that is a link and the OS print dialog. For ZPL, keep the generated label bytes with the order so an operator can send them through a local raw queue.
The fallback must produce the same identity as the automated path, so that afterward reconciliation can see "order 1042 was printed manually at 10:42" and not resubmit it. Record the manual print against the order, with who and when.
Subscribe someone to the vendor's status feed, and the cloud provider's. You want to learn about the incident from the feed, not from a warehouse supervisor.
Bound retries and timeouts so their outage does not become yours
An unbounded retry loop against a dead endpoint exhausts your worker pool, then your database connections, then your own API. Set a request timeout, cap attempts per pass, back off exponentially with a ceiling, and respect Retry-After when the vendor returns 429. Our Node client retries POST only when an Idempotency-Key is present, and its retry count defaults to zero, which is a deliberate refusal to retry anything that cannot be retried safely.
While the outbox grows, orders should still be accepted and picking should still happen. The label is the last step and can be late by twenty minutes without changing anything upstream.
Reconcile afterward: the vendor's list is the record you compare against
After the outage, three sets exist: what you intended to print (your outbox), what the vendor recorded (its job list), and what physically happened (the operator's memory and the manual-print log). Reconciliation is comparing the first two and asking humans only about the residue.
Our list endpoint returns jobs newest first and filters by status and printerId, with a limit that caps at 200 per request. There is no time filter and no cursor, so read a status at a time and stop at the first createdAt before the incident window.
async function jobsSince(rp, since, status) {
const out = [];
// The API caps limit at 200 and has no pagination cursor. If a status comes
// back full, narrow it (by printerId, or by polling during the incident)
// rather than assuming this page reached the start of the window.
for (const job of await rp.printJobs.list({ status, limit: 200 })) {
if (new Date(job.createdAt) < since) break;
out.push(job);
}
return out;
}
const window = new Date('2026-09-11T14:00:00Z');
const stuck = await jobsSince(rp, window, 'pending');
const unknown = await jobsSince(rp, window, 'unknown');
Then act by state. pending jobs that are now stale should be canceled; DELETE /v1/print-jobs/:id works only while a job is pending, which is the boundary you want. expired jobs never printed and can be resubmitted under a new, intentional identity if the label is still needed. unknown jobs are the ones to walk to the printer for, because the Station could not confirm the spooler outcome and another job might mean a duplicate. Each job record carries its idempotencyKey, so joining the vendor's list back to your outbox is a lookup, not a guess.
Do this within seven days. Print content is purged seven days after a job reaches a terminal state; the job record with status, timestamps, and failure reason remains, but a reprint from the vendor's copy has a one-week window.
What RocketPrint does today, and what it does not
What exists: jobs are durable from the moment we return 201, and a job for an offline printer waits until its expiry. The Station keeps its own record of finished jobs across restarts, so a redelivery after a reconnect does not print twice. unknown is a real status with a stated policy (inspect the printer before creating another job). Pending jobs can be canceled. The list endpoint, statuses, and idempotency behavior are in the API reference, and the failure modes we cannot see are listed on the reliability page.
What does not exist: we do not ship webhooks, so your worker polls or uses the Node client's waitFor, and during our outage it learns nothing until we return. We also do not ship a client-side offline SDK. The outbox above is yours to build, and we think it should be, because it needs to live in the same transaction as your order. And the list endpoint is thinner than reconciliation wants: 200 jobs per request, no time filter, no cursor. For a long outage your own outbox, not our list, is the complete side of the comparison.
The design goal is not a print path that never fails. It is a print path where, at 14:20 on a bad Tuesday, the answer to all four questions is already written down.
END / design-print-path-for-vendor-outage
More field notes →