• Get started

    • Getting started
    • Concepts
    • Tour
  • Using shpyrd

    • Deploying
    • shpyrd.yaml
    • Resources
    • Databases and caches
    • Domains and exposure
    • Sign-in for your app
    • Teams, roles and security
    • AI assistants (MCP)
    • Logs
    • Dashboard
    • CLI reference
  • Running it yourself

    • Installation
    • Oracle Cloud (OKE)
    • AWS (EKS)
    • Extensions and sign-in
    • Platform backups
    • Architecture guide
  • Project

    • Design principles
    • Roadmap
    • How to contribute
    • Support the project
  1. Running it yourself
  2. Architecture guide
Sign in
Start free

Architecture guide

How the installer, the App controller and the server turn a Kubernetes cluster into a PaaS.

On shpyrd cloud the platform is run for you: see Getting started. This guide is for running shpyrd yourself, or for knowing how it works. shpyrd is a few binaries and a set of well-known open source components: shpyrd, the CLI for people who deploy, which speaks the workspace API; shpyrd-ctl, the operator's CLI, which installs and operates the cluster through a kubeconfig; shpyrd-server, which runs inside the cluster as API, edge, controllers and dashboard (and, in other modes, as the S3 gateway and the volume helpers); and pg-gateway, the proxy that wakes a sleeping database on the first connection.

Components

ComponentRole in shpyrd
kind (local profile)the Kubernetes cluster itself, driven as a Go library
cert-manager + trust-managercertificates for every project and the dashboard; on the local profile from a development CA generated by the CLI, distributed to all namespaces as a trust bundle
ingress-nginxHTTP entry point; publishes the per-host request and latency metrics behind the dashboard charts
registry (registry:3)in-cluster OCI registry for builder and application images, TLS from the platform CA, one platform credential, weekly garbage collection; on cloud profiles it can keep its images in object storage (a provider bucket, or the S3 gateway)
Calico (policy only, oci profile)enforces the per-project NetworkPolicy where the provider's CNI does not
ExternalDNS + an OCI DNS-01 solver (oci profile, optional)publishes records for every hostname and issues the platform's wildcard certificate
kpack + Paketo buildpacksbuilds source into images inside the cluster; polls Git branches for new commits
kube-prometheus-stackPrometheus (its data on a volume sized by SHPYRD_MONITORING_SIZE on cloud profiles), node-exporter, kube-state-metrics
local-path provisioner (shpyrd-local)project volumes and databases on the node's own disk: the local profile, and the disks made on a cloud profile before block storage became the default
CloudNativePG + barman-cloud (extension postgres)PostgreSQL databases, their WAL archive and base backups
Dex (extension auth-local)the OpenID Connect issuer for accounts and company sign-in connectors
KEDA + its HTTP add-on (extension sleep)web processes that scale to zero and wake on the first request
Vector (extension logs-agent)collects container logs and feeds log drains
S3 gateway or Garage (extension object-storage)object storage: a gateway in front of one provider bucket, or Garage in the cluster
snapshot controller, EBS/EFS (aws), Block Volume/File Storage (oci)the platform's own disks, snapshots and older project disks
shpyrd-serverAPI for the CLIs and the dashboard, the edge in front of apps that need sign-in, the controllers, the source archive store, the two embedded applications (the console for the operator, the workspace for its members), served by host
shpyrd and shpyrd-ctlevery project operation (shpyrd); cluster bootstrap and operation (shpyrd-ctl)

Control-plane database. Workspaces, teams, project grants and the people seen signing in live in PostgreSQL, not in Kubernetes objects. With the postgres extension the control-plane-db component runs it as a CloudNativePG cluster, control-plane-pg: SHPYRD_CONTROL_PLANE_DB_INSTANCES instances (two on the cloud profiles, spread over the platform pool's nodes, the replica synchronous while it is there and asynchronous rather than blocking when it is not; one on the local profile), a volume of SHPYRD_CONTROL_PLANE_DB_SIZE each, failover in about half a minute, and, with the object gateway, Barman backups to the platform-control-plane bucket nightly with the WAL archived (14 days kept). Without the extension it is a single PostgreSQL instance with one volume; SHPYRD_DATABASE_URL names a managed database instead and the component is skipped. A platform that ran the single instance moves its data to the cluster once, at the server's turn of cluster init after the cluster is ready: the server stops for the seconds the copy takes, the old instance stays behind at zero replicas with its claim until removed by hand. Workloads (App, Volume, Postgres, Redis, LogDrain, ObjectBucket) stay custom resources reconciled by controllers. Every identifier is a native uuid; the schema is versioned with golang-migrate (embedded up and down files in pkg/store/migrations, applied at start under an advisory lock, a dirty flag when a migration was interrupted), and the server imports the Team/ProjectMember objects of older installs once. Take a database backup before upgrading across a schema migration; shpyrd-ctl cluster backup carries the database's content, and pg_dump of the control-plane-db pod is the plain alternative.

The installer

shpyrd-ctl cluster init applies a profile: an ordered list of components, each a Helm chart (installed with the Helm SDK, values embedded in the binary), a Kustomize tree (rendered in-process and applied with server-side apply), or both, plus readiness conditions: Deployments available, CRDs established, an admission webhook accepting a dry-run request, a status condition true for the current generation.

Components are grouped in runlevels (rc0...rc4). A level is applied in parallel and waited for before the next starts, so CRDs exist before the resources that use them and webhooks are serving before objects they validate. Variables (${SHPYRD_DOMAIN}, ${SHPYRD_REGISTRY_HOST}...) are substituted in the manifests before Kustomize parses them. The result is recorded in a ConfigMap (shpyrd-system/shpyrd-install) that cluster status and the dashboard read. Re-runs upgrade in place; cluster export renders everything to disk for GitOps tools.

Cloud profiles (oci and aws) swap the pieces that differ - a cloud load balancer on a reserved address, Let's Encrypt, the provider's storage - and run cluster init in two phases: everything that needs no certificate first, then a wait for the load balancer address and for DNS, then the rest (Oracle Cloud (OKE)). Explicitly given settings are recorded as overrides and carried over by later runs.

Three decisions shaped the profiles:

  • Registry trust. The in-cluster registry sits on a fixed ClusterIP (10.96.0.50:5000) and serves TLS only, with a certificate from the platform CA whose SAN is that address, plus one generated credential. kpack has no setting for a private CA and every step of a build talks to the registry, so the server runs an admission webhook that mounts the cluster's trust bundle into each container of a build pod and sets SSL_CERT_FILE (every step is a Go binary); the kpack controller and BuildKit get the same bundle, and a DaemonSet writes the CA where each node's runtime reads it (containerd's certs.d, CRI-O's certs.d). TLS-only matters: go-containerregistry treats RFC 1918 addresses as insecure-eligible and races plain HTTP against HTTPS, and only a registry without a plaintext listener makes that race always end on HTTPS.
  • Certificates per host. Each hostname that needs a certificate has its own Certificate object owned by the App - custom domains always, the project hostname only when no platform wildcard serves it - so a domain whose DNS is not ready never blocks the others. With a DNS provider, one wildcard certificate is ingress-nginx's default certificate and project Ingresses carry none.
  • Multi-arch builders. The prebuilt Paketo builder images are amd64-only, which breaks on Apple Silicon. The builder is assembled from the individual Paketo buildpackages (Go, Node.js, Java, Python, Ruby, .NET, web servers, Procfile), which ship amd64 and arm64, so kpack resolves the right architecture per node.

The App controller

Projects are App custom resources living in their own namespace (p-<id>, named by the project's id; older projects: app-<slug>) with everything derived from them, which gives garbage collection and per-project RBAC for free. The controller (controller-runtime, kubebuilder layout) reconciles an App into:

  • a kpack Image for the source (Git or uploaded archive), with a build cache volume, or, for the dockerfile strategy, a rootless BuildKit Job per source revision (an init container fetches the archive or clones the revision, the build pushes to the registry with a layer cache; the digest travels in the container's termination message). Images go to apps/<workspace id>/<slug> in the registry, the workspace's id rendered in base36 (25 lowercase characters): ids never change and slugs are per workspace, so two workspaces with a project of the same name never share a repository, its tags, the build cache or the builder;
  • the claims and mounts of the project's Volume resources (single-instance volumes force one replica and a Recreate rollout) and the Secret <app>-bindings with the config vars of attached resources;
  • one Deployment per process type, sized from the catalog (default shared-m for Node.js, Ruby, Python and Java, shared-s otherwise), web running the image entrypoint and other types /cnb/process/<type>, with PORT injected and the config var Secret mounted as environment;
  • a Service per process with a port and an Ingress for web; an app that needs sign-in gets its Ingress pointed at the server's edge, which checks the visitor before passing the request on.

Status is derived from the kpack Image or build Job (build progress and failures), the Deployments (instances updated and ready) and the pods behind them (instances that cannot start, with the container's reason). A release is recorded whenever the running image or the configuration hash changes; its config vars are snapshotted into a Secret, and a rollback request (annotation) restores that snapshot before pinning the release's build. Status updates use optimistic locking so a reconcile working from a stale cache never overwrites a newer state.

Other controllers reconcile the other resources the same way: Volume, Postgres (into a CloudNativePG cluster), Redis, LogDrain, ObjectBucket, and the workspaces' front doors.

The server

shpyrd-server runs the controller manager (with leader election) and an HTTP API in one process. The API serves both CLIs and the dashboard (projects, builds, logs streamed as NDJSON with Heroku-style instance names, Prometheus-backed metrics per process and cluster capacity, write-only config vars, scale/rollback/deploy/destroy) and receives source archives from shpyrd deploy. It stores them by SHA-256, on a persistent volume or, with the S3 gateway, in object storage, and serves them to builds at a cluster-internal URL.

With the object gateway, Prometheus runs the Thanos sidecar: every two-hour block is uploaded to the gateway bucket platform-metrics (kept 90 days), so metrics history survives a restart and a disk; Prometheus itself keeps a few days locally and stays what the dashboard, the alerts and the metering loop read.

GET /api/readyz is the platform's readiness: anyone on the API host gets the status word (200 "ok" or 503 "degraded"); the cluster admin (the admin token, or a signed-in cluster admin) and monitoring on the server's internal API inside the cluster get the checks, one line each: the sources store answers, the platform backup succeeded within a day, the profile's storage class exists, the registry, the wake gateway for databases, the sleep interceptor, the object gateway and every shared PostgreSQL server have a ready instance. It answers 200 "ok" or 503 "degraded"; a component the cluster does not run is skipped. /api/healthz stays the process's own liveness.

People sign in with accounts (the auth-local extension and its connectors): the server is an OpenID Connect relying party and keeps a session cookie. The CLI and integrations use personal API tokens or a CLI session from shpyrd login. The admin token generated at install time remains for the operator and for automation, and can be switched off; it is accepted as Authorization: Bearer or X-Shpyrd-Token. For apps that are not public, the server's edge checks the visitor's sign-in before the request reaches the app.

The CLI

shpyrd speaks the workspace API: signed in with shpyrd login, it needs no kubeconfig. shpyrd deploy archives the committed tree of the current directory (git archive HEAD), uploads it, applies shpyrd.yaml, then follows the build step by step and the rollout until the release is running. shpyrd-ctl is the operator's: it reads and writes the cluster directly through your kubeconfig, like flux or kubectl argo rollouts, and installs the platform.

Repository layout

cmd/shpyrd            CLI for people who deploy (speaks the workspace API)cmd/shpyrd-ctl        CLI for the operator (cluster, extensions, accounts, backups)cmd/shpyrd-server     server: API, edge, controllers, the embedded applications, the S3 gatewaycmd/pg-gateway        wake-on-connect proxy for sleeping databasesapi/v1alpha1          custom resource types: App, Volume, Postgres, Redis, LogDrain, ObjectBucket... (kubebuilder layout; `make generate`)internal/controller   reconcilersinternal/cli          CLI commandspkg/install           runlevel installer: embedded Kustomize + Helm SDK + server-side applypkg/api               HTTP API and source storepkg/edge              the edge in front of apps that need sign-inpkg/ext               extensions (auth-local, postgres, redis, object-storage, ...)pkg/objectgateway     the S3 gatewaypkg/store             the control-plane database and its migrationspkg/kind              kind cluster provisioningpkg/localca           development root CApkg/configvars        config vars (names + metadata; values are write-only)ee/                   enterprise features (shpyrd Enterprise License)deploy/               components and profiles embedded in the binarydesign/ui             the component library and its gallery, shared by the applications and the siteapps/                 the console and the workspace applications (Next, static files, embedded by pkg/ui) and the websitecontent/              these docsexamples/hello        example project (Go, web + worker)

On this page

  • Components
  • The installer
  • The App controller
  • The server
  • The CLI
  • Repository layout
Back to top
  • AI Community
  • Vibe Coders
  • Developers
  • Businesses
  • FDEs & Implementation Partners
  • Getting Started
  • Pricing
  • Docs
  • Changelog

© 2026. shpyrd. All rights reserved.LegalTerms of ServicePrivacy Policy