Scaling

cavs-api

Stateless — scales horizontally. No in-memory session state; every request re-authenticates from its bearer token. Run N replicas behind a load balancer.

Design choices that keep it cheap under load:

• No write on the hot path. An authenticated request does a single indexed read to resolve the caller. The user row is written only on first sign-in and on session bootstrap — never per request.
• Bytes bypass the API entirely. Uploads and downloads return presigned URLs, so transfer throughput is bounded by object storage, not by API CPU. Doubling your transfer volume doesn’t require doubling API capacity.
• Per-instance connection pool — DB_MAX_POOL_SIZE / DB_MIN_POOL_SIZE.
• Per-instance JWKS cache for Firebase verification, refreshed hourly. No shared state.

Watch total MongoDB connections

Total connections ≈ replicas × DB_MAX_POOL_SIZE. With the default 100 and 10 replicas that’s 1000 — comfortably over a small Atlas tier’s limit. Size the pool and the replica count against your cluster’s connection budget, or you’ll hit connection exhaustion long before you hit CPU.

Rate limiting is per instance

RATE_LIMIT_RPS and RATE_LIMIT_BURST protect each instance. With 5 replicas a single token can effectively get 5× the nominal limit, depending on load-balancer distribution.

For a genuinely global limit, back the limiter with Redis. It’s isolated in internal/api/ratelimit.go precisely so it can be swapped without touching handlers.

Bottleneck order

object storage  →  MongoDB  →  API CPU

Scale API replicas first (cheap, stateless). Add MongoDB read replicas or shard by organization_id when reads dominate. Object storage is almost never your problem — that’s the point of presigned URLs.

cavs-worker

Competing consumers — scales horizontally. Multiple workers share one Pub/Sub subscription and the broker distributes messages. Per-repository ordering is preserved via the message ordering key.

• Every handler is idempotent (deduplicated by processed_events on the event id), so at-least-once redelivery and overlapping replicas are both safe.
• MaxOutstandingMessages bounds in-flight work per worker. Raise it, or add replicas.
• The cleanup ticker runs on every replica. Its operations are idempotent UpdateManys, so duplicate runs are harmless — but with many replicas, gate it behind a leader lock or move it to a scheduled job rather than paying for N× the work.
• GC, repack and index compaction are the CPU-heavy stages. Today they’re coordinators; when the Rust cavs-maintenance binary lands, size that pool independently from the event workers.

To split responsibilities, give each worker class its own subscription id and filter by event_type:

cavs-worker-notifications
cavs-worker-usage
cavs-worker-maintenance

Note

Start with one worker. The event volume from ordinary pushes is low, and a single always-on instance handles a lot. Scale when cavs_queue_depth or cavs_queue_oldest_job_age_seconds says to — not on principle.

cavs-web

A static Vite build. Serve it from any CDN or static host; it’s infinitely cacheable and has no scaling concern beyond CDN configuration.

The one caveat: in the reference deployment the web tier also reverse-proxies /api to the API service so the dashboard and API share an origin (Firebase persists auth per origin). If you front the SPA with a pure CDN, put that proxy at the edge instead.

MongoDB

• Indexes for every hot query are created on boot: unique keys for identity, slug and token lookups; compound indexes for membership, repository listing, usage rollups and the audit/notification feeds.
• Transactions — the control plane runs multi-document operations sequentially on standalone MongoDB (organization + member + subscription creation, invitation accept, upload finalize). On a replica set those can be wrapped in session transactions; the operations are already grouped in the store methods for exactly that.
• Shard key candidate: organization_id. Tenant isolation aligns with the natural query boundary, so it shards cleanly.

Tip

Use a replica set in production even at low scale. It buys transactions, read scaling and failover, and the code is already written for it.

Object storage

Content-addressed keys spread objects across prefixes (…/objects/aa/bb/<oid>), which is exactly what S3-style partitioning wants — no hot prefix, no manual key randomization.

Put a CDN in front of downloads with S3_PUBLIC_BASE_URL. It cuts both latency and egress cost, and it’s especially effective for public repositories and for the static chunk-tree export, where every path is immutable and therefore aggressively cacheable.

What to watch

Metric	Tells you
cavs_http_request_duration_seconds p95 by route	Which endpoint is slowing down
cavs_http_in_flight_requests	Whether replicas are saturated
cavs_http_rate_limited_total	Clients hitting the limiter — raise it or fix the client
cavs_queue_depth, cavs_queue_oldest_job_age_seconds	Whether workers keep up
cavs_job_duration_seconds by stage	Which stage is the expensive one
cavs_chunk_lookup_duration_seconds	Dedup index performance
MongoDB connections in use	How close you are to the cluster limit
cavs_storage_ledger_discrepancy_bytes	Metering diverging from reality

Capacity notes

Dimension	Practical guidance
API replicas	2 minimum for availability. Add on p95 latency or in-flight saturation.
Workers	Start at 1. Add on queue depth or oldest-job age.
MongoDB	Replica set from day one. Shard on organization_id when a single primary can’t take the writes.
Object storage	Effectively unbounded. Watch cost, not capacity.
Rate limits	Size for your largest repository’s push burst — the transfer agent HEAD-guards every tree file and does not back off on 429.

The one real robustness gap

The transfer agent doesn’t retry on 429. A repository with many packs can exceed the rate limit during a push, the mirror fails, and — because the LFS objects already reported complete — git push still succeeds. The objects simply never get registered.

Until the agent backs off, keep RATE_LIMIT_RPS generous, and verify pushes by checking the generation rather than the push exit code.

Next

• Observability — the metrics behind these decisions.
• Configuration reference — the knobs.
• Architecture — why it’s shaped this way.