The memory adapter¶

gitlab.com/phpboyscout/go/dispatch/memory is an in-memory adapter that implements the whole adapter SPI, including seven optional interfaces (Holder, Requeuer, NativePolicyCompiler, Watcher, CessationSignal, IdentityWeakener and MultiFinder). It doesn't implement OutputReader, so it reads no output. It behaves like Slurm or like Kubeflow, as its configuration chooses. The conformance suite is run against it, and the simulator uses it as the scheduler. It is not for production.

Importing the package registers it under the name memory.

import _ "gitlab.com/phpboyscout/go/dispatch/memory"

Configuration¶

The registry's Factory decodes AdapterConfig.Settings strictly: an unknown field at any level, an unknown capability, action or tristate, or trailing data is refused with ErrInvalid. Durations are Go duration strings.

{
  "shape": "slurm",
  "maxJobId": 100,
  "minJobAge": "5m",
  "callBound": "25s",
  "slurmdTimeout": "40s",
  "watchBuffer": 64,
  "capabilities": {"AccountingHistory": {"value": "yes", "source": "probed"}},
  "rules": {"restart": {"value": "yes"}},
  "native": {"restart": {"value": "yes"}},
  "cessation": {"ceased": true, "detail": "node fenced"}
}

A native entry of yes whose rules entry is not yes is refused. In Go, memory.New takes the same settings as a memory.Config.

Shapes¶

Slurm:

• every object has the job's constant name dj-<ULID>, a wrapped job id, and a submit time truncated to the second;
• Start is gated;
• Open of an object that ended is ErrFinished, and an admin hold refuses Open;
• Stop acts on one id and submit time;
• a hold is a start time of SubmitTime + HoldBase + instance. It is refused on a running job, survives Open, and only a Release naming the current instance lifts it;
• Requeue revives a finished or cancelled object to Pending, ungated, with its restart count raised. A pending object refuses it, and the requeue bit is restored either way;
• a requeue bit left set, as RequeueExternally leaves it, makes a preemption requeue the job on Slurm's own authority (Pending, cause Requeued) instead of ending it Preempted. RestoreRequeueBit clears it; an ended object refuses the update.

Kubeflow:

• names are dj-<ulid>-s<N>, a duplicate is refused, and a stop deletes;
• preemption sends the object back to pending without touching its restart counter;
• a lost node wedges it Running with cause NodeLost, and it never ends on its own;
• pod-derived causes (NodeLost, OutOfMemory, NonZeroExit) read as Unknown once evidenceWindow has passed;
• a hold deactivates the Workload, ending a running execution with cause Held;
• Requeue and RestoreRequeueBit are ErrUnsupported.

Both shapes give each execution an opaque DISPATCH_ATTEMPT and a DISPATCH_EPOCH from model.Epoch. On Kubeflow the epoch's R counts executions, because the restart counter shown does not count re-admissions.

Deadlines and time¶

Every Hold, Release, Requeue and RestoreRequeueBit carries a CallMeta.Deadline. A call whose command started after its deadline does nothing and returns ErrExpired. The start is time.Now() at the call, unless the context carries one:

ctx = memory.WithCallStart(ctx, startedAt)
err := b.Hold(ctx, ref, model.HoldMeta{Instance: 2, CallMeta: model.CallMeta{Deadline: d}})

A HoldMeta also carries who placed the hold, until when and why. The Kubeflow shape writes them beside the hold as annotations, shown in the ground truth as HeldBy, HeldUntil and HoldReason until the hold is released. The Slurm shape keeps only the instance, as a StartTime hold does.

The adapter starts no goroutines and sets no timers. Purges, lost nodes ending and evidence vanishing are worked out from the clock inside each call, so a testing/synctest bubble's fake clock drives all of them. Watch sends from inside the calls that change an object. A full buffer, a done context or Close closes the stream, and the subscriber resubscribes to get every object's current observation.

Environment and ground truth¶

b.Env() drives the scheduler's side and reads the ground truth, which every view from b.View(readOnly) shares:

Conformance¶

conformance.Backend(t, harness) runs the backend conformance suite, and conformance.Core(t, harness) the core's. The memory adapter passes both in both shapes; see memory/conformance_test.go and memory/core_conformance_test.go for the harnesses, and write an adapter for writing one of your own. That harness injects LateCall with WithCallStart set past the deadline plus the bound, and CrashBetweenSteps with WithCrashBetweenSteps, under which a Requeue sets Requeue=1 and returns ErrCrashed.