Architecture guide
How the installer, the App controller and the server turn a Kubernetes cluster into a PaaS.
On shpyrd cloud the platform is run for you: see Getting started. This guide is for running shpyrd yourself, or for knowing how it works. shpyrd is a few binaries and a set of well-known open source components: shpyrd, the CLI for people who deploy, which speaks the workspace API; shpyrd-ctl, the operator's CLI, which installs and operates the cluster through a kubeconfig; shpyrd-server, which runs inside the cluster as API, edge, controllers and dashboard (and, in other modes, as the S3 gateway and the volume helpers); and pg-gateway, the proxy that wakes a sleeping database on the first connection.
Components
| Component | Role in shpyrd |
|---|---|
| kind (local profile) | the Kubernetes cluster itself, driven as a Go library |
| cert-manager + trust-manager | certificates for every project and the dashboard; on the local profile from a development CA generated by the CLI, distributed to all namespaces as a trust bundle |
| ingress-nginx | HTTP entry point; publishes the per-host request and latency metrics behind the dashboard charts |
registry (registry:3) | in-cluster OCI registry for builder and application images, TLS from the platform CA, one platform credential, weekly garbage collection; on cloud profiles it can keep its images in object storage (a provider bucket, or the S3 gateway) |
Calico (policy only, oci profile) | enforces the per-project NetworkPolicy where the provider's CNI does not |
ExternalDNS + an OCI DNS-01 solver (oci profile, optional) | publishes records for every hostname and issues the platform's wildcard certificate |
| kpack + Paketo buildpacks | builds source into images inside the cluster; polls Git branches for new commits |
| kube-prometheus-stack | Prometheus (its data on a volume sized by SHPYRD_MONITORING_SIZE on cloud profiles), node-exporter, kube-state-metrics |
local-path provisioner (shpyrd-local) | project volumes and databases on the node's own disk: the local profile, and the disks made on a cloud profile before block storage became the default |
CloudNativePG + barman-cloud (extension postgres) | PostgreSQL databases, their WAL archive and base backups |
Dex (extension auth-local) | the OpenID Connect issuer for accounts and company sign-in connectors |
KEDA + its HTTP add-on (extension sleep) | web processes that scale to zero and wake on the first request |
Vector (extension logs-agent) | collects container logs and feeds log drains |
S3 gateway or Garage (extension object-storage) | object storage: a gateway in front of one provider bucket, or Garage in the cluster |
snapshot controller, EBS/EFS (aws), Block Volume/File Storage (oci) | the platform's own disks, snapshots and older project disks |
| shpyrd-server | API for the CLIs and the dashboard, the edge in front of apps that need sign-in, the controllers, the source archive store, the two embedded applications (the console for the operator, the workspace for its members), served by host |
| shpyrd and shpyrd-ctl | every project operation (shpyrd); cluster bootstrap and operation (shpyrd-ctl) |
Control-plane database. Workspaces, teams, project grants and the people seen signing in live in PostgreSQL, not in Kubernetes objects. With the postgres extension the control-plane-db component runs it as a CloudNativePG cluster, control-plane-pg: SHPYRD_CONTROL_PLANE_DB_INSTANCES instances (two on the cloud profiles, spread over the platform pool's nodes, the replica synchronous while it is there and asynchronous rather than blocking when it is not; one on the local profile), a volume of SHPYRD_CONTROL_PLANE_DB_SIZE each, failover in about half a minute, and, with the object gateway, Barman backups to the platform-control-plane bucket nightly with the WAL archived (14 days kept). Without the extension it is a single PostgreSQL instance with one volume; SHPYRD_DATABASE_URL names a managed database instead and the component is skipped. A platform that ran the single instance moves its data to the cluster once, at the server's turn of cluster init after the cluster is ready: the server stops for the seconds the copy takes, the old instance stays behind at zero replicas with its claim until removed by hand. Workloads (App, Volume, Postgres, Redis, LogDrain, ObjectBucket) stay custom resources reconciled by controllers. Every identifier is a native uuid; the schema is versioned with golang-migrate (embedded up and down files in pkg/store/migrations, applied at start under an advisory lock, a dirty flag when a migration was interrupted), and the server imports the Team/ProjectMember objects of older installs once. Take a database backup before upgrading across a schema migration; shpyrd-ctl cluster backup carries the database's content, and pg_dump of the control-plane-db pod is the plain alternative.
The installer
shpyrd-ctl cluster init applies a profile: an ordered list of components, each a Helm chart (installed with the Helm SDK, values embedded in the binary), a Kustomize tree (rendered in-process and applied with server-side apply), or both, plus readiness conditions: Deployments available, CRDs established, an admission webhook accepting a dry-run request, a status condition true for the current generation.
Components are grouped in runlevels (rc0...rc4). A level is applied in parallel and waited for before the next starts, so CRDs exist before the resources that use them and webhooks are serving before objects they validate. Variables (${SHPYRD_DOMAIN}, ${SHPYRD_REGISTRY_HOST}...) are substituted in the manifests before Kustomize parses them. The result is recorded in a ConfigMap (shpyrd-system/shpyrd-install) that cluster status and the dashboard read. Re-runs upgrade in place; cluster export renders everything to disk for GitOps tools.
Cloud profiles (oci and aws) swap the pieces that differ - a cloud load balancer on a reserved address, Let's Encrypt, the provider's storage - and run cluster init in two phases: everything that needs no certificate first, then a wait for the load balancer address and for DNS, then the rest (Oracle Cloud (OKE)). Explicitly given settings are recorded as overrides and carried over by later runs.
Three decisions shaped the profiles:
- Registry trust. The in-cluster registry sits on a fixed ClusterIP (
10.96.0.50:5000) and serves TLS only, with a certificate from the platform CA whose SAN is that address, plus one generated credential. kpack has no setting for a private CA and every step of a build talks to the registry, so the server runs an admission webhook that mounts the cluster's trust bundle into each container of a build pod and setsSSL_CERT_FILE(every step is a Go binary); the kpack controller and BuildKit get the same bundle, and a DaemonSet writes the CA where each node's runtime reads it (containerd'scerts.d, CRI-O'scerts.d). TLS-only matters: go-containerregistry treats RFC 1918 addresses as insecure-eligible and races plain HTTP against HTTPS, and only a registry without a plaintext listener makes that race always end on HTTPS. - Certificates per host. Each hostname that needs a certificate has its own
Certificateobject owned by the App - custom domains always, the project hostname only when no platform wildcard serves it - so a domain whose DNS is not ready never blocks the others. With a DNS provider, one wildcard certificate is ingress-nginx's default certificate and project Ingresses carry none. - Multi-arch builders. The prebuilt Paketo builder images are amd64-only, which breaks on Apple Silicon. The builder is assembled from the individual Paketo buildpackages (Go, Node.js, Java, Python, Ruby, .NET, web servers, Procfile), which ship amd64 and arm64, so kpack resolves the right architecture per node.
The App controller
Projects are App custom resources living in their own namespace (p-<id>, named by the project's id; older projects: app-<slug>) with everything derived from them, which gives garbage collection and per-project RBAC for free. The controller (controller-runtime, kubebuilder layout) reconciles an App into:
- a kpack
Imagefor the source (Git or uploaded archive), with a build cache volume, or, for thedockerfilestrategy, a rootless BuildKit Job per source revision (an init container fetches the archive or clones the revision, the build pushes to the registry with a layer cache; the digest travels in the container's termination message). Images go toapps/<workspace id>/<slug>in the registry, the workspace's id rendered in base36 (25 lowercase characters): ids never change and slugs are per workspace, so two workspaces with a project of the same name never share a repository, its tags, the build cache or the builder; - the claims and mounts of the project's
Volumeresources (single-instance volumes force one replica and a Recreate rollout) and the Secret<app>-bindingswith the config vars of attached resources; - one Deployment per process type, sized from the catalog (default
shared-mfor Node.js, Ruby, Python and Java,shared-sotherwise),webrunning the image entrypoint and other types/cnb/process/<type>, withPORTinjected and the config var Secret mounted as environment; - a Service per process with a port and an Ingress for
web; an app that needs sign-in gets its Ingress pointed at the server's edge, which checks the visitor before passing the request on.
Status is derived from the kpack Image or build Job (build progress and failures), the Deployments (instances updated and ready) and the pods behind them (instances that cannot start, with the container's reason). A release is recorded whenever the running image or the configuration hash changes; its config vars are snapshotted into a Secret, and a rollback request (annotation) restores that snapshot before pinning the release's build. Status updates use optimistic locking so a reconcile working from a stale cache never overwrites a newer state.
Other controllers reconcile the other resources the same way: Volume, Postgres (into a CloudNativePG cluster), Redis, LogDrain, ObjectBucket, and the workspaces' front doors.
The server
shpyrd-server runs the controller manager (with leader election) and an HTTP API in one process. The API serves both CLIs and the dashboard (projects, builds, logs streamed as NDJSON with Heroku-style instance names, Prometheus-backed metrics per process and cluster capacity, write-only config vars, scale/rollback/deploy/destroy) and receives source archives from shpyrd deploy. It stores them by SHA-256, on a persistent volume or, with the S3 gateway, in object storage, and serves them to builds at a cluster-internal URL.
With the object gateway, Prometheus runs the Thanos sidecar: every two-hour block is uploaded to the gateway bucket platform-metrics (kept 90 days), so metrics history survives a restart and a disk; Prometheus itself keeps a few days locally and stays what the dashboard, the alerts and the metering loop read.
GET /api/readyz is the platform's readiness: anyone on the API host gets the status word (200 "ok" or 503 "degraded"); the cluster admin (the admin token, or a signed-in cluster admin) and monitoring on the server's internal API inside the cluster get the checks, one line each: the sources store answers, the platform backup succeeded within a day, the profile's storage class exists, the registry, the wake gateway for databases, the sleep interceptor, the object gateway and every shared PostgreSQL server have a ready instance. It answers 200 "ok" or 503 "degraded"; a component the cluster does not run is skipped. /api/healthz stays the process's own liveness.
People sign in with accounts (the auth-local extension and its connectors): the server is an OpenID Connect relying party and keeps a session cookie. The CLI and integrations use personal API tokens or a CLI session from shpyrd login. The admin token generated at install time remains for the operator and for automation, and can be switched off; it is accepted as Authorization: Bearer or X-Shpyrd-Token. For apps that are not public, the server's edge checks the visitor's sign-in before the request reaches the app.
The CLI
shpyrd speaks the workspace API: signed in with shpyrd login, it needs no kubeconfig. shpyrd deploy archives the committed tree of the current directory (git archive HEAD), uploads it, applies shpyrd.yaml, then follows the build step by step and the rollout until the release is running. shpyrd-ctl is the operator's: it reads and writes the cluster directly through your kubeconfig, like flux or kubectl argo rollouts, and installs the platform.
Repository layout
cmd/shpyrd CLI for people who deploy (speaks the workspace API)cmd/shpyrd-ctl CLI for the operator (cluster, extensions, accounts, backups)cmd/shpyrd-server server: API, edge, controllers, the embedded applications, the S3 gatewaycmd/pg-gateway wake-on-connect proxy for sleeping databasesapi/v1alpha1 custom resource types: App, Volume, Postgres, Redis, LogDrain, ObjectBucket... (kubebuilder layout; `make generate`)internal/controller reconcilersinternal/cli CLI commandspkg/install runlevel installer: embedded Kustomize + Helm SDK + server-side applypkg/api HTTP API and source storepkg/edge the edge in front of apps that need sign-inpkg/ext extensions (auth-local, postgres, redis, object-storage, ...)pkg/objectgateway the S3 gatewaypkg/store the control-plane database and its migrationspkg/kind kind cluster provisioningpkg/localca development root CApkg/configvars config vars (names + metadata; values are write-only)ee/ enterprise features (shpyrd Enterprise License)deploy/ components and profiles embedded in the binarydesign/ui the component library and its gallery, shared by the applications and the siteapps/ the console and the workspace applications (Next, static files, embedded by pkg/ui) and the websitecontent/ these docsexamples/hello example project (Go, web + worker)