Pushing & pulling large files

Through Git

Once connected, nothing about your Git workflow changes:

git add model.safetensors
git commit -m "Retrain with the augmented set"
git push

Three things happen in parallel, described in How it works: Git pushes the commit, the CAVS agent chunks and uploads the object, and the pre-push hook uploads the Git index.

Pulling:

git pull                # commits + LFS objects for the checked-out tree
git lfs pull            # just the LFS objects
git lfs fetch --all     # every object on every ref (careful — this can be large)

The CLI offers thin wrappers that make sure the CAVS wiring is in place first:

cav push          # → git push
cav pull          # → git pull
cav sync          # → git pull, then git push

They forward extra arguments straight to Git: cav push --force-with-lease origin main.

Selective checkout

For a repository with far more data than any one machine needs:

# Clone without fetching any LFS content
GIT_LFS_SKIP_SMUDGE=1 git clone <url> && cd <repo>

# Then fetch only what you need
git lfs pull --include="checkpoints/v3/**"
git lfs pull --include="*.safetensors" --exclude="archive/**"

Or make it the default for the repository:

git config lfs.fetchinclude "checkpoints/current/**"
git config lfs.fetchexclude "archive/**,experiments/**"

Without Git

With the CLI

Works in any repository connected with cav repo connect — it reads cavs.repo-id from the Git config.

cav upload ./checkpoints/epoch-42.safetensors ./metrics.json
cav download 9f2c8b0e…4f6b --output model.safetensors
cav verify ./checkpoints/epoch-42.safetensors    # is this exact content already stored?

cav upload hashes every file first, so a bad path fails before a session is opened. cav download refuses to overwrite an existing file without --force. cav verify hashes locally and asks the Hub whether those oids exist — a cheap way to check a release is complete before shipping it.

Add --json to any of them for machine-readable output:

cav upload ./model.bin --json | jq -r '.[].oid'

With an SDK

Python

from cavs import CAVS, sha256_file

client = CAVS.from_env()

result = client.artifacts.upload(
    path="./checkpoints/epoch-42.safetensors",
    project="vision-models",           # slug, name or UUID
    name="resnet50",
    kind="model",
    version="1.4.0",
    metadata={"epochs": 42, "val_acc": 0.942},
)
print(result.reference)

# Download by oid (or an immutable @sha256: reference).
oid = sha256_file("./checkpoints/epoch-42.safetensors")
client.artifacts.download(oid, dest="./restored/model.safetensors",
                          project="vision-models")

TypeScript

import { CAVS, sha256File } from '@cavsnode/cavs-sdk';

const client = CAVS.fromEnv();

const result = await client.artifacts.upload({
  path: './dist/bundle.tar',
  project: process.env.CAVS_REPO_ID!,   // the repository UUID
  name: 'client',
  kind: 'archive',
  version: process.env.GITHUB_SHA!,
});

const oid = await sha256File('./dist/bundle.tar');
console.log(result.reference, oid);

Caution

Two current SDK limitations: download by logical reference needs a server endpoint that isn’t deployed (use an oid or an immutable @sha256: reference — the TypeScript download() doesn’t work at all yet), and dedup statistics aren’t returned by finalize (read GET /repositories/{id}/usage instead). See Python SDK and TypeScript SDK.

With plain HTTP

Four calls to upload, one to download. See Plain HTTP for the full walkthrough, or the Quickstart’s curl tab.

What determines how much you actually transfer

1. Whether the CAVS agent is in the path. Standard git-lfs moves whole files. The agent moves chunks. This is the difference between 22 MiB and 3 MiB on a small edit.

2. What the local chunk cache already holds. On pull, the agent computes the set of chunks it’s missing and fetches only those, with parallel range requests. A machine that has the previous version pays almost nothing for the next one.

3. How your format behaves under editing. Formats that append or patch in place dedup beautifully. Formats that reshuffle every offset on any change (some archive and package formats) defeat chunking. See Deduplication explained.

4. Whether the bytes are already stored. An object whose oid already exists in the repository is a metadata operation — nothing travels.

Verifying a transfer

Every path verifies content by construction: the oid is the SHA-256, so a download that doesn’t hash back to its oid is rejected rather than written.

To check by hand:

shasum -a 256 model.safetensors     # sha256sum on Linux

To confirm the platform’s view:

cav verify ./model.safetensors
curl -sS "$API/repositories/$REPO/objects?sort=recent&limit=5" -H "$H" | jq

Reading the numbers after a push

cav storage                                          # organization totals
curl -sS "$API/repositories/$REPO/usage"     -H "$H"  # this repository
curl -sS "$API/repositories/$REPO/transfers?kind=push&limit=10" -H "$H"

transfers is the measured record the agent reported: duration, logical bytes, stored bytes, new vs reused chunks. The dashboard’s Benchmarks tab renders the same data. See Storage insights.

Note

Storage figures come from an asynchronous reconciliation, so they lag a push by seconds, not minutes. The object count and generation update immediately.

Concurrency and limits

Limit	Value
API rate limit	30 requests/second sustained, burst 60, keyed per token (or per user, or per IP when anonymous)
Concurrent uploads	plan-dependent: 2 (Free) → 64 (Business)
Max object size	plan-dependent: 5 GiB (Free) → 1 TiB (Business)
Presigned URL lifetime	15 minutes
Upload session lifetime	6 hours
Control-plane request body	1 MiB

Exceeding the rate limit returns 429 rate_limited; back off exponentially with jitter. Exceeding a plan limit returns 402 quota_exceeded with a details object naming which one. See Plans & quotas.

Interrupted transfers

Path	Behaviour
CAVS agent, pull	Resumes with HTTP range requests from where it stopped. Corrupt cache entries are quarantined and repaired.
CAVS agent, push	Already-stored chunks are skipped; only the remainder is re-sent.
Upload session	Re-run the authorize step: objects already stored are reported as such, so the upload resumes by skipping them. Then finalize.
Standard git-lfs	LFS retries the whole object.
Expired presigned URL	Re-request it — the authorize call is cheap and idempotent.

Tip

An upload session left unfinalized expires after 6 hours and its objects are swept. If a job dies mid-upload, either finalize the session or just start a fresh one — with the same Idempotency-Key you get the same session back.

Next

• Browsing your data — find what you pushed.
• Releases & snapshots — mark and capture states.
• Storage insights — measure the savings.