Grant the adapter its RBAC¶
The adapter runs as whatever identity its kubeconfig, its in-cluster service account or its
Token supplies. At Factory it asks the apiserver, with a SelfSubjectAccessReview for each verb
below, whether that identity may do it, and sets itself up accordingly. It does not assume. Your
identity needs create on selfsubjectaccessreviews, which Kubernetes grants to all authenticated
users by default (system:basic-user).
Read a preflight result shows how the answers surface. This page is the other half: what to grant.
The adapter serves one namespace, so most rules go in a Role in that namespace. Three kinds of permission are different:
- Required to observe. Without them
Factoryfails withErrUnauthorized. - Required to write. Without them the adapter opens observe-only.
- Optional. Without them a named capability is lowered.
The namespaced Role¶
Replace team-a with the namespace in namespace. Everything here is for a Role bound to the
adapter's identity in that namespace.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata: {name: dispatch-adapter, namespace: team-a}
rules:
# Required to observe, and to write (create, update, delete).
- apiGroups: [trainer.kubeflow.org]
resources: [trainjobs]
verbs: [get, list, watch, create, update, delete]
- apiGroups: [trainer.kubeflow.org]
resources: [trainingruntimes]
verbs: [get, list, create, update, delete]
- apiGroups: [jobset.x-k8s.io]
resources: [jobsets]
verbs: [get, list, watch]
- apiGroups: [""]
resources: [pods]
verbs: [get, list, watch]
# Required to write: StopJob deletes the JobSets and pods background deletion orphans. pods/log is rank 0's log.
- apiGroups: [jobset.x-k8s.io]
resources: [jobsets]
verbs: [delete]
- apiGroups: [""]
resources: [pods]
verbs: [delete]
- apiGroups: [""]
resources: [pods/log]
verbs: [get]
# Kueue mode only.
- apiGroups: [kueue.x-k8s.io]
resources: [workloads]
verbs: [get, list, watch, update]
- apiGroups: [kueue.x-k8s.io]
resources: [localqueues]
verbs: [get]
The adapter creates, updates and deletes TrainingRuntimes because each job runs on a namespaced copy of its runtime, named for the TrainJob. It never touches a Secret. Credentials are mounted by name and the adapter does not read them.
The cluster-scoped rules¶
Runtimes, Nodes, ClusterQueues and AdmissionChecks are cluster-scoped, so these go in a ClusterRole:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata: {name: dispatch-adapter}
rules:
# Required to observe.
- apiGroups: [trainer.kubeflow.org]
resources: [clustertrainingruntimes]
verbs: [get]
# Optional: the Node reads behind node-loss classification and CessationSignal.
- apiGroups: [""]
resources: [nodes]
verbs: [get]
# Optional, Kueue mode: proving the readmission gate.
- apiGroups: [kueue.x-k8s.io]
resources: [clusterqueues, admissionchecks]
verbs: [get]
And the Lease, in the controller's namespace (dispatch-system unless you moved it):
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata: {name: dispatch-adapter-heartbeat, namespace: dispatch-system}
rules:
- apiGroups: [coordination.k8s.io]
resources: [leases]
verbs: [get]
Bind each to the adapter's identity with a RoleBinding (the namespaced Roles) and a ClusterRoleBinding (the ClusterRole).
What each permission is for¶
| Permission | Class | Without it |
|---|---|---|
get, list, watch trainjobs, jobsets, pods |
required to observe | Factory fails with ErrUnauthorized |
get clustertrainingruntimes; get, list trainingruntimes |
required to observe | Factory fails |
get, list, watch workloads; get localqueues (Kueue mode) |
required to observe | Factory fails |
create, update, delete trainjobs; create, update, delete trainingruntimes |
required to write | observe-only; Start refuses |
update workloads (Kueue mode) |
optional | Hold is no with the reason not permitted to update workloads.kueue.x-k8s.io |
get pods/log |
optional | OutputRead and OutputFollow are no |
get nodes |
optional | Ceased returns ErrUnsupported; node-loss classification cannot read taints and says so in the cause; StopJob never reports Unproven, and counts remaining pods as Alive |
get clusterqueues, admissionchecks; get leases (Kueue mode) |
optional | ReadmissionGate is unknown, so the adapter is observe-only |
delete jobsets, pods |
required to write | observe-only; Start refuses. StopJob deletes the orphaned JobSets and pods it finds, which background deletion makes routine, so a writable adapter must be able to |
Plain mode needs none of the Workload, LocalQueue, ClusterQueue, AdmissionCheck or Lease
permissions to run. Where Kueue is installed, grant it list on workloads all the same:
preflight's check that Kueue does not manage queue-less jobs lists the probe's Workloads, and
without that permission it skips the check silently. See Choose Kueue mode or plain
mode.
update on TrainJobs matters beyond Open. The first time the adapter observes a TrainJob with a
JobSet, it records the JobSet's UID in an annotation, which needs update. An observe-only adapter
does not do this.
The admission controller's RBAC¶
The controller has its own identity and its own ClusterRole. It is in Install the admission controller.
Running as a bearer token¶
If the host supplies AdapterConfig.Token, the adapter calls it for every request and uses the
result as the bearer token, replacing whatever credentials the kubeconfig carries. The permissions
above apply to the identity that token names.