Skip to content

Record stores

ls, err := file.Open(ctx, "/var/lib/dispatch/records", file.Options{})
if err != nil {
    return err
}

store := ledgerstore.New(ls, nil)

cluster, err := dispatch.Open(ctx, "slurm", dispatch.Config{Adapter: settings, Store: store})

The core keeps each job's record behind records.Records. Pass one as dispatch.Config.Store, and the same one to supervise.New; a cluster with no store has no Operator.

Store Package For records.Owner records.Health
*ledgerstore.Store gitlab.com/phpboyscout/go/dispatch/records/ledgerstore a host's durable records when the go/ledger store is a ledger.Owner when it is also a ledger.Health
*recordstest.Memory gitlab.com/phpboyscout/go/dispatch/records/recordstest tests and examples; it keeps nothing across a restart yes yes

records.Records

Method Does
Create(ctx, header, idempotencyKey) writes a new record, or returns the existing one and false for a repeated key
Read(ctx, job) reads the record at its current revision; records.ErrNotFound when there is none
Commit(ctx, job, commit) appends entries if commit.Revision is still current, else records.ErrConflict; with a lease, also requires current ownership where the store can check it, else records.ErrNotOwner. A non-zero commit.Settled sets the settled labels
List(ctx, query) the records that are unsettled, or settled (before SettledBefore, when set)
Tombstone(ctx, job, revision) removes a settled record and keeps its idempotency entry, so a late Submit with the same key can't recreate the job

records.CommitControl(ctx, store, job, control) is how the Operator commits a person's control intent: it re-reads and retries on ErrConflict until the context ends, then returns ErrUnavailable.

Optional capabilities

Found by type assertion on the store.

Interface Methods Used for
records.Owner Acquire(ctx, job, holder) returns a records.Lease, or records.ErrOwned; Holder(ctx, job) one supervisor per job, and Status.Unattended. It is for liveness only; safety doesn't rest on it
records.Health Beat(ctx, lease, at); Loop(ctx, job) the time the owner's last loop completed, for display; see supervise.Healthy in timings

A store with neither still works: two supervisors of one job then both work it, which is safe and wasteful.

ledgerstore

ledgerstore.New(s ledger.Store, r io.Reader) adapts any go/ledger store. r is the source of mutation ids, and nil means crypto/rand.

  • A commit whose reply was lost, which go/ledger reports as unavailable, is retried with the same mutation id until the context ends, so it is never applied twice.
  • A repeated idempotency key with a different spec is refused with ErrRejected.
  • go/ledger's errors stay beneath the core's: its conflict, not-owner, owned and not-found become records.ErrConflict, ErrNotOwner, ErrOwned and ErrNotFound; a corrupt record, an unsupported operation or a refused root becomes ErrInvalid; anything else, ErrUnavailable.
  • Acquire returns ErrUnsupported, and Holder reports nobody, when the go/ledger store has no ownership. Beat returns ErrUnsupported, and Loop reports nothing known, when it has no loop health.

go/ledger's file store, file.Open, implements both. It keeps one directory per record on a local disk bound to one host, and refuses a root on a network or unrecognised filesystem with ledger.ErrRefusedRoot; see go/ledger and its file package.

recordstest

recordstest.New() returns an empty *recordstest.Memory. It is safe for concurrent use and never blocks. Revoke(job) takes ownership away from a live holder, for tests of a supervisor that loses it.