Logs
Live logs, the log agent that labels and bounds them, and drains that forward them to your provider, per project or for the whole cluster.
Every process writes to stdout and stderr; shpyrd names the instance (web.1, worker.2), streams the lines live, keeps them bounded on the node, and forwards them wherever you keep your logs.
Live logs
shpyrd logs --project shop # last 100 lines of every instance (-n to change)shpyrd logs --project shop -f -p worker # follow one process typeshpyrd logs --project shop --build # the latest build's outputThe project's Logs page streams the same lines. Pick a process and a minimum level (Everything, Info and up, Warnings and up, Errors only), filter by text, switch Live off to pause, and Raw to see the lines as written. There is nothing to set up: on shpyrd cloud and on a cluster you run yourself alike, this path reads from the Kubernetes API with nothing enabled.
The log agent
The log agent runs Vector on every node. It reads the container logs of project instances and turns every line into a structured event:
{"time": "...", "project": "shop", "process": "web", "instance": "web.2", "stream": "stdout", "level": "info", "msg": "request", "fields": {"method": "GET", "path": "/", "status": 200}}Lines that are JSON get level and msg promoted and the rest kept in fields; plain lines keep msg as the text. Build output and the pods of attached databases and caches are not part of this stream.
Bounded on the node. The kubelet rotates every container's log files, so an application logging at full speed cannot fill a node. On a local cluster each container keeps at most 20 MiB (two files of 10 MiB); cloud profiles keep their provider's default. History beyond that lives wherever you drain it.
Fenced in. A network policy lets the agent reach the drains' receivers (on the internet or in a platform namespace), DNS and the API server, and nothing in a project; only the platform reads its counters. On a local cluster the enriched stream is also written to the agent's own pod log (kubectl logs -n logs-system daemonset/vector), to see the pipeline at work; the cloud profiles leave that out.
Log drains
A drain forwards lines as they are written. Two kinds of receiver:
| Receiver | URL | What arrives |
|---|---|---|
| HTTPS (Datadog, Better Stack, Axiom, your own collector) | https://... | JSON, one object per line, batched, with the headers you set (API keys) |
| Syslog (Papertrail, rsyslog, a SIEM) | syslog://host:port or syslog+tls://host:port | RFC 5424 over TCP: the project as APP-NAME, the instance as PROCID, the level as severity |
And three scopes:
- a project drain receives that project's lines; project admins add them on the project's Drains page or with
--project; - a workspace drain receives every project's lines of the workspace, labelled with the project; workspace owners and admins add them on Workspace › Log drains or with
--workspace; - a cluster drain (self-hosted) receives every project's lines of the platform; the operator of a cluster you run yourself adds them over a kubeconfig with
--cluster.
shpyrd drains add https://in.logs.betterstack.com/ --header "Authorization: Bearer ..." --project shopshpyrd drains add https://http-intake.logs.datadoghq.com/api/v2/logs --header "DD-API-KEY: ..." --processes web --project shopshpyrd drains add syslog+tls://logs.papertrailapp.com:6514 --workspace acmeshpyrd drains list --project shopshpyrd drains remove in-logs-betterstack-com --project shopThe name defaults to the receiver's host. --processes limits a drain to some process types. Header values are stored in the cluster and never shown again, in the CLI or the dashboard. The pages show each drain's delivery status (Pending, Active with the last delivery time and line count, Failing with the error) refreshed every 30 seconds; a receiver that keeps failing raises a DrainFailing warning event on the drain, and after five minutes of failed checks a drain.failing entry in the project's activity (the cluster's, for a workspace or cluster drain).
Providers
| Provider | Drain |
|---|---|
| Better Stack (Logtail) | https://in.logs.betterstack.com/ with Authorization: Bearer <source token> |
| Datadog | https://http-intake.logs.datadoghq.com/api/v2/logs with DD-API-KEY: <key> (use your site's intake host) |
| Axiom | https://api.axiom.co/v1/datasets/<dataset>/ingest with Authorization: Bearer <token> |
| Papertrail | syslog+tls://logsN.papertrailapp.com:<port> |
| Grafana Loki (push API) | https://loki.example.com/loki/api/v1/push accepts JSON lines only through a proxy; native Loki support is planned |
| Anything that accepts newline-delimited JSON over HTTP, or syslog | works |