State machines for side effects you cannot fully observe

If your system causes something to happen outside your process, your state machine needs more than success and failure. It needs a state for "we asked and never heard back", and that state needs a written policy, because the default policy is to retry, and retrying a physical action is how you get two shipping labels, two charges, or two robot moves.

The worked example here is our own print job state machine. RocketPrint is a remote printing API and desktop Station for software platforms: your backend POSTs a print job, and a small app beside the printer prints it, with no inbound networking, no drivers for raw label formats, and a durable job record. The pattern applies wherever the acknowledgment can be lost after the effect has happened, so the state design is the transferable part, not the printing.

Two states are not enough when the effect is physical

A database write gives you a strong answer: it committed or it did not, and if your client crashed mid-call you can read back and find out which. The system that performs the effect also records it.

Physical effects break that. The effect happens in one place, the record travels over a network, and the network can drop the record while keeping the effect. Printing, SMS, payouts, robots, and door locks all share the property that the acknowledgment is a separate, failable thing from the action. A boolean is then a lie by omission: it forces every unresolved case into "succeeded" or "failed", and both are wrong in a way that costs paper, money, or trust.

Name each state after the observation that produced it

The rule that fixed our own design: a state records what somebody observed, not a guess about the world. pending observes your database. sent observes a socket write. printing observes that another program took the work. None of them observes that the physical thing occurred, and if you name them as though they did, every reader of your API will over-trust them.

The test for a state name: can you say which component observed what, and what that observation does not prove? If you cannot answer the second half, the name is marketing.

Here are nine states and what each one proves

State	Who observed what	What it proves	What it does not prove
pending	API saved the job	The instruction is durable	Nothing about any device
dispatching	One server took a lease on the job	Exactly one server is delivering it	That delivery will succeed
sent	The server wrote the job to a connected Station	The bytes left our side	That the Station processed it
printing	Station handed the job to the OS print system	The OS accepted the work	That a printer received it
completed	Station's print command returned success through the OS print path	The local print path succeeded	That paper emerged, or is readable
failed	Something returned an error, with a reason	This attempt did not succeed	That nothing was printed
canceled	A caller canceled while the job was still pending	No delivery was attempted	Nothing about a job already sent, which cannot be canceled
expired	The deadline passed with no dispatch	Nobody was there to print it	That the job was unwanted
unknown	Something was interrupted with an unconfirmed outcome	Only that we cannot say	Either outcome

dispatching exists because "one server is working on this" is an observable fact, and making it a state is what holds a job under a single lease so it is never sent twice.

completed is the row that costs us something to publish. It means the print command succeeded through the OS print path. It does not mean paper came out. A printer can be out of labels, jammed, or set to a darkness that produces a barcode no scanner will read. We could have called this state printed. We do not, because that would make the API sound better and our users' reconciliation worse. The longer version is in what remote print job statuses actually mean.

Unknown is a state, and retrying it is the dangerous default

Most systems have an unknown case. Few name it, so it gets handled by whatever the code does when it falls off the end of a switch, which is usually a retry. Retry is the correct default for idempotent reads and a hazardous one for physical effects: if the action already happened and only the acknowledgment was lost, the retry produces a second physical object.

Give the state a name and a policy. Ours:

• unknown means a job was interrupted and the outcome is not confirmed.
• The documented instruction is to inspect the printer before creating another job. A human or a sensor resolves it, not a timer.
• A late Station acknowledgment can settle an unknown job, because Station keeps a durable ledger of finished jobs across restarts. The state can resolve itself; nothing pretends to know before it does.
• unknown is terminal. It is not a waiting room that quietly becomes success.

An unknown that decays into completed after a timeout is optimism with extra steps. Two ledgers are needed rather than one clever protocol because no protocol removes the problem: see exactly-once is a lie and how idempotency prevents duplicate shipping labels.

Expiry is an outcome, so give it a state

A job for a printer that is switched off has to end somewhere, or a warehouse PC that comes back on Monday prints Friday's labels. So expiry is part of the contract, set by the caller: print jobs accept expiresInSeconds from 10 to 86,400, defaulting to 3,600, and a job not dispatched by its deadline becomes expired. Pick that number from the business meaning of the document, not a technical instinct.

"Terminal" is the word your retention policy keys off

Split your states into terminal and non-terminal in one place. Ours: completed, failed, canceled, expired, and unknown are terminal; everything else is in flight.

That split drives things outside the state machine. Our content lifecycle keys off it: print content is deleted seven days after a job reaches a terminal state, the job record with status, timestamps, failure reason, and usage is kept, and the API reports contentPurgedAt so you can tell "no content" from "content gone". Billing keys off it too, since only completed jobs are billed. If you cannot point at the code defining your terminal set, your retention policy is approximate.

Write the transition table down, then generate from it

Most state machine bugs are transitions nobody intended. The cheapest fix is a table that is data, not control flow.

export const JOB_STATES = [
  "pending",
  "dispatching",
  "sent",
  "printing",
  "completed",
  "failed",
  "canceled",
  "expired",
  "unknown",
] as const;

export type JobState = (typeof JOB_STATES)[number];

const TRANSITIONS: Record<JobState, readonly JobState[]> = {
  pending: ["dispatching", "canceled", "expired"],
  dispatching: ["sent", "pending", "printing", "completed", "failed", "unknown"],
  sent: ["printing", "completed", "failed", "unknown"],
  printing: ["completed", "failed", "unknown"],
  completed: [],
  failed: [],
  canceled: [],
  expired: [],
  unknown: ["completed", "failed"],
};

export const TERMINAL: readonly JobState[] = [
  "completed",
  "failed",
  "canceled",
  "expired",
  "unknown",
];

export function assertTransition(from: JobState, to: JobState): void {
  if (!TRANSITIONS[from].includes(to)) {
    throw new Error(`illegal transition ${from} -> ${to}`);
  }
}

Three things fall out of that.

The empty arrays are load-bearing. completed: [] is a claim you can test, and a test over every state pair catches the next person who adds a "re-open" path in a hurry.

unknown has outgoing edges and is still in TERMINAL. That looks contradictory and is exactly right: the job is over as far as your workflow goes, and a late acknowledgment may still settle what happened.

dispatching can go back to pending. A lease that expires because the server died must return the work, and if your table has no edge for that, your code will do it anyway through a path nobody tests.

The edges out of pending are narrow on purpose. A saved job can be leased, canceled, or expire, and nothing else. It cannot fail, because nothing has tried yet, and a state machine that lets an untried job fail is one that will eventually record a failure reason nobody can trace to an attempt. The edges out of dispatching are wide for the opposite reason: a Station can report a terminal outcome before our own write of sent lands, so the table has to allow the skip rather than reject the report.

Design the support view before the happy path

The state machine's real users are the people answering "did it print?" at 6 a.m. Build their view early, because it forces honesty into your state names. It needs the state, the time of each transition, the failure reason verbatim, the device and its connection status, and the business object the job came from. If support joins three tables by hand to answer one question, the model is missing a field. The failures we cannot see at all are on our reliability page.

One limit: we do not ship webhooks, so a caller building that view polls the REST API or uses the Node client's waitFor helper. That gap is ours, not your architecture's.

The same shape fits SMS, payouts, and a robot arm

The names change; the structure does not.

Domain	"Accepted"	"Handed off"	"Succeeded, sort of"	The unknown
Print	pending	sent to Station	Print command returned OK	Station interrupted mid-job
SMS	Queued at the provider	Submitted to the carrier	Carrier accepted	No delivery receipt ever arrives
Card payment	Authorization request saved	Sent to the network	Approved	Timeout after submit, before response
Payout	Instruction recorded	Sent in a batch file	Bank accepted the file	File sent, no confirmation
Robot move	Command queued	Sent to the controller	Controller reports done	Link lost mid-motion

In every row the third column is weaker than its name suggests, and the fourth is the one that hurts. If your design has no fourth column, you have not removed the case, you have hidden it.

Where our version falls short

There is no dry-run endpoint. You cannot ask "would this job be accepted" without creating one, which makes testing an integration noisier than it should be.

unknown requires a human. We can settle it when a Station reports back, but we cannot look at your printer.

And completed keeps disappointing anyone who skims. The honest word with a footnote beats the flattering word without one, but it is a real cost, paid in support conversations.

Start with the table. Four columns, and the discipline that every state name is followed by "which does not prove". That exercise will change your API before you write any code.