Skip to content

Machine states

A machine’s row answers three questions at once, and they move on their own schedules. Reading them apart is what stops an ordinary start from looking like a fault.

Where you read itWhose answer it isValues
Status in the machines listthe machine, by whether its agent is answeringOnline, Offline
Power on the machine pagethe cloud, about the instanceBooting, Running, Stopping, Stopped, No longer exists
Last action on the machine pageMachinery, about the last power call it madeStarting, Booting, Ready, Stopping, Stopped, and a failure with the cloud’s own sentence

They are different facts, and they move apart:

  • Online or Offline is whether the machine is reachable. Machinery reads it from the agent’s check-ins, so it answers “is anything running on the box”.
  • Running or Stopped is whether the cloud instance is powered. The provider owns that answer, so it answers “is the box switched on”.

A machine can be Offline and Running (the agent stopped answering while the instance stayed powered, and it is still billing), Offline and Stopped (the ordinary state of a stopped machine), or Online and Running (everything normal). One never implies the other: read Power to know what the instance is doing, and Status to know whether anything is answering.

The machine’s agent reports in while it runs, and the machine reads Online while those reports arrive. Two minutes without one marks it Offline.

That is the detection time for a crash or a lost network. A clean shutdown is reported as it happens and flips the machine within about a second, and a stop Machinery itself asked for does not raise a disconnect warning at all.

The cloud’s answer, normalized so the same word means the same thing on every provider:

Wire valueWhat it meansWhat the console shows
pendingthe provider is bringing the instance upBooting
runningthe provider says the instance is upRunning
stoppingthe provider is shutting it downStopping
stoppedthe instance is powered off, and its disks are intactStopped
movingthe instance is moving to another host, or into or out of a worker pool, and is stopped for part of itMoving
gonethe provider no longer returns the instanceNo longer exists
unknownnothing has been read for this machinenothing

unknown is the answer for a machine with no cloud identity: one enrolled by hand, or a machine Machinery did not create. There is no instance to ask about, so the field stays empty rather than claiming a state.

A stop settles on the cloud, not on the agent. The instance stays Stopping until the provider says it stopped, which is tens of seconds after the guest has already gone quiet.

Machinery records what it asked the instance to do, and how that turned out:

OutcomeWhat it means
pendingthe call is in flight, and the machine is on its way to the state you asked for
succeededthe machine arrived: a start is Ready once the agent connects, and a stop is Stopped once the cloud says so
failedit did not arrive, and the message beside it is the reason, written by the service

A start reads Starting, then Booting when the cloud says the instance is up, then Ready when the agent connects. A stop reads Stopping, then Stopped. There is no timer behind any of these words: each one is a fact the server already recorded.

A machine’s create is recorded first and launched after, so its row carries the step it reached:

StatusWhat it meansWhat the console shows
pending, launchingMachinery is creating the machine nowLaunching…
launchedthe launch finished, and the machine is running normallynothing
failedthe launch stopped, and the step names what was being attemptedLaunch failed
terminate-pending, terminatinga destroy was accepted and has not finishedTerminating
terminatedthe destroy finished and the record is retiredDeregistered
unknownnothing recorded a status, which is what a client older than the field readsnothing

Launch failed is worth reading closely, because a launch can fail after the cloud created an instance. The message beside it says whether an instance may exist that Machinery never recorded, and it is the reason that state is never shown as a calm Deregistered.

Independently of all of the above, a machine is either active or deregistered. A deregistered machine has been retired from the account and is removed from the list about an hour later.

One row can qualify for several of these at once, so the console reads them in a fixed order and shows the first that applies:

  1. a destroy that was asked for and could not be carried out, as Terminate failed
  2. a failed launch, as Launch failed
  3. the record, when it is no longer active
  4. a destroy in progress, as Terminating
  5. a launch in progress, as Launching…
  6. the machine’s own answer, Online or Offline

That is why the status column shows a transition in place of the presence badge: a machine being started is genuinely not answering yet, and Offline is the least useful thing to say about a box somebody is watching boot.