Open the adapter from Go¶

Importing gitlab.com/phpboyscout/go/dispatch-kubeflow registers the adapter with dispatch under the name "kubeflow" and registers the kubeflow.RuntimeRef job extension. Importing for effect is enough if you open the adapter through dispatch:

import (
    "gitlab.com/phpboyscout/go/dispatch"
    _ "gitlab.com/phpboyscout/go/dispatch-kubeflow"
)

cluster, err := dispatch.Open(ctx, "kubeflow", dispatch.Config{
    Adapter: map[string]any{
        "namespace":    "team-a",
        "probeRuntime": map[string]any{"kind": "ClusterTrainingRuntime", "name": "torch-distributed"},
    },
    Store: store,
})

dispatch.Config.Adapter is marshalled to JSON and handed to the adapter as its settings. dispatch.NewBackend builds the Backend alone, for a host that composes its own supervisor.

The settings are decoded strictly: an unknown key at any level, a duplicate key, trailing data, or a null is refused with ErrInvalid. A null is refused because it would read as unset and take the default silently. Every key is in the configuration reference.

Validate settings without a cluster¶

kubeflow.ParseSettings decodes and validates a settings section, applies the defaults, and touches no cluster:

s, err := kubeflow.ParseSettings(raw)

kubeflow.Schema() returns the JSON Schema of the section, additionalProperties: false at every level, for a host to publish at its adapter section. The Schema accepts a few things the parser then refuses, listed in the configuration reference.

Open it directly¶

Factory opens the adapter from an AdapterConfig, and returns a dispatch.Backend whose concrete type is *kubeflow.Backend:

b, err := kubeflow.Factory(ctx, dispatch.AdapterConfig{
    Backend:  kubeflow.Name,
    Settings: raw,
    ReadOnly: false,
})
backend := b.(*kubeflow.Backend)

*kubeflow.Backend implements dispatch.Backend, Holder, Watcher, CessationSignal, OutputReader and IdentityWeakener. ReadOnly: true opens it without the probe and refuses every write. Token, if set, supplies the bearer token on every request.

Unless the adapter is ReadOnly or turns out observe-only, Factory takes at least ten seconds, because preflight watches a gated probe for that long. Do not call it per request.

Name a runtime in a job¶

A job names the Trainer runtime it runs on with the required kubeflow.RuntimeRef extension, and the image it runs with dispatch.Image:

spec := model.JobSpec{
    Command: []string{"torchrun", "train.py"},
    Nodes:   2,
    Extensions: model.Extensions{
        model.Image{Ref: "registry.example/train:1"},
        kubeflow.RuntimeRef{Kind: kubeflow.KindClusterTrainingRuntime, Runtime: "torch-distributed"},
    },
}

The Go field is Runtime, not Name, because Name is already the method that names the extension. In JSON it is {"kind": "...", "name": "..."}. Kind is kubeflow.KindClusterTrainingRuntime or kubeflow.KindTrainingRuntime.

The adapter accepts three extensions: dispatch/Image, dispatch/SharedCheckpoint and kubeflow/RuntimeRef. Any other is refused with ErrUnsupported, never ignored. A missing dispatch/Image or kubeflow/RuntimeRef is ErrInvalid.

Call Capabilities(ctx, &spec) before submitting. With a spec it reads the runtime and refuses with ErrUnsupported one that breaks the adapter's rules, naming every reason, so the job is refused before anything is created. See runtime requirements.

Capabilities does not check the rest of the spec. Start also limits a job's Env and command to what Trainer allows beside the adapter's own, and refuses ProcsPerNode on a runtime without mlPolicy.torch. How a job becomes a TrainJob lists every limit, and where each field of the spec goes.

Errors¶

Test with errors.Is against the dispatch sentinels. The adapter's own two exported errors, kubeflow.ErrStaleHold and kubeflow.ErrPodsRemain, are classified by sentinels too. The full table is in error mapping.

The full Go API is on pkg.go.dev.