Deployment

Configuration, installation, RBAC, and testing for the Propeller Kubernetes Operator.

Configuration

The operator is configured through command-line flags and one environment variable. There is no configuration file.

Environment Variables

Variable	Default	Description
`WATCH_NAMESPACE`	(all namespaces)	When set, the operator only watches resources in this namespace. Leave unset to watch all namespaces.

MQTT Flags

Flag	Default	Description
`--mqtt-address`	—	MQTT broker URL (required, e.g. `tcp://mqtt:1883`)
`--domain-id`	—	SuperMQ domain ID (required)
`--channel-id`	—	SuperMQ channel ID (required)
`--client-id`	—	Manager client ID for MQTT authentication (required)
`--client-key`	—	Manager client key for MQTT authentication (required)
`--mqtt-qos`	`0`	MQTT QoS level (0–2)
`--mqtt-timeout`	`30s`	Broker operation timeout

Proplet Monitoring Flags

Flag	Default	Description
`--liveliness-interval`	`10s`	How often to check proplet heartbeat timestamps
`--last-seen-threshold`	`30s`	How long without a heartbeat before marking a proplet Offline

Kubernetes Manager Flags

Flag	Default	Description
`--health-probe-bind-address`	`:8081`	Address for `/healthz` and `/readyz` endpoints
`--metrics-bind-address`	`0`	Address for Prometheus metrics (`0` disables)
`--metrics-secure`	`true`	Serve metrics over HTTPS
`--metrics-cert-path`	—	Directory containing the metrics TLS certificate
`--metrics-cert-name`	`tls.crt`	Metrics certificate filename
`--metrics-cert-key`	`tls.key`	Metrics key filename
`--leader-elect`	`false`	Enable leader election for high-availability deployments
`--enable-http2`	`false`	Enable HTTP/2 (disabled by default for security)
`--webhook-cert-path`	—	Directory containing the webhook TLS certificate
`--webhook-cert-name`	`tls.crt`	Webhook certificate filename
`--webhook-cert-key`	`tls.key`	Webhook key filename

Installing and Deploying

For a complete walkthrough of setting up the operator with SuperMQ and running a task, see the Kubernetes Operator Example.

Installing CRDs

cd propeller-k8s-operator
make install

This runs controller-gen to generate CRD manifests from the Go types, then applies them with kubectl via Kustomize.

Deploying the Controller

make deploy IMG=<your-registry>/propeller-k8s-operator:<tag>

This renders the full deployment manifest (controller Deployment, ClusterRole, ClusterRoleBinding, ServiceAccount, CRDs) via Kustomize and applies it. The operator namespace (propeller-k8s-operator-system) is created automatically.

For local development with k3d, build the image and import it directly into the cluster nodes before deploying (no registry required):

make docker-build IMG=propeller-k8s-operator:latest
k3d image import propeller-k8s-operator:latest -c k3s-default
make deploy IMG=propeller-k8s-operator:latest

Set imagePullPolicy: Never in config/manager/manager.yaml when using a locally-loaded image.

Running Locally

go run ./cmd/main.go \
  --mqtt-address="tcp://localhost:1883" \
  --domain-id="<domain-id>" \
  --channel-id="<channel-id>" \
  --client-id="<manager-client-id>" \
  --client-key="<manager-client-key>" \
  --metrics-secure=false

The operator uses the current kubectl context for cluster access.

RBAC

The operator requires a ClusterRole with the following permissions:

API Group	Resources	Verbs
`""` (core)	`configmaps`	create, delete, get, list, patch, update
`""` (core)	`pods`	get, list
`""` (core)	`secrets`	get
`apps`	`deployments`	create, delete, get, list, patch, update, watch
`batch`	`jobs`	create, delete, get, list, patch, update, watch
`propeller.propeller.abstractmachines.fr`	`tasks`, `proplets`, `propellerjobs`, `federatedjobs`, `trainingrounds`	create, delete, get, list, patch, update, watch
`propeller.propeller.abstractmachines.fr`	`*/status`	get, patch, update
`propeller.propeller.abstractmachines.fr`	`*/finalizers`	update

Container Security

When deployed in-cluster the manager pod runs with:

runAsNonRoot: true
seccompProfile: RuntimeDefault
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
All Linux capabilities dropped

Resource defaults: 10m CPU / 64Mi memory requests; 500m CPU / 128Mi limits.

Health Probes

Endpoint	Purpose
`GET /healthz`	Liveness probe — confirms the process is alive
`GET /readyz`	Readiness probe — confirms the controller manager is ready to serve

Both endpoints are served on --health-probe-bind-address (default :8081).

Testing

The operator includes a controller test suite that uses envtest to spin up a real Kubernetes API server and etcd in-process, with all five CRDs loaded from config/crd/bases.

make test

End-to-end tests run against a real cluster (Kind by default):

make setup-test-e2e   # creates a Kind cluster
make test-e2e         # runs Ginkgo e2e suite
make cleanup-test-e2e # tears down the Kind cluster

On this page