The Kubeflow adapter¶
Run go/dispatch jobs on Kubeflow Trainer.
gitlab.com/phpboyscout/go/dispatch-kubeflow is the Kubeflow adapter for go/dispatch. It turns each submission into a TrainJob that cannot run until dispatch opens it, reads back what happened from the TrainJob, its JobSet, its pods and, where installed, Kueue, and reports a typed cause rather than "failed".
The module ships two things:
- the adapter, package
kubeflow, which registers itself with dispatch under the name"kubeflow"when you import it; cmd/dispatch-admission, an AdmissionCheck controller that you run in the cluster when you use Kueue.
The design is recorded in
spec 0001, the Kubeflow adapter,
which is APPROVED. Where this section and the spec differ, this section describes the code.
Start here¶
Run a job on a kind cluster stands up a local cluster with Trainer, JobSet, Kueue and the admission controller, then starts, opens, holds, releases and stops a two-node job through the adapter.
Do a task¶
- Install the admission controller
- Choose Kueue mode or plain mode
- Mount the publish image
- Read a preflight result
- Grant the adapter its RBAC
- Open the adapter from Go
- Run the conformance suite against kind
Understand it¶
- The start gate and the readmission gate
- Why the controller never sets Active to False
- Execution identity: R = A + J
- Never two executions at once
- Holds are Workload deactivation
- Background deletion
- The Lease heartbeat