Uploads - Clatter Labs Docs

Every upload, small or large, works the same way:

Start it with the file's name and exact size: POST /v1/buckets/{bucket_id}/uploads. You get an upload id, chunk_size (16,777,216 bytes = 16 MiB) and chunk_count.
Send each chunk: PUT /v1/uploads/{upload_id}/chunks/{index} with the raw bytes. Chunk i is bytes i × chunk_size to (i + 1) × chunk_size - 1; only the last chunk is shorter.
Complete it: POST /v1/uploads/{upload_id}/complete.

A file under 16 MiB is one chunk.

Zips or assets

A .zip is stored as an archive and unpacked into the code tree. Upload it with "target_path": "packages/api" to replace only that folder.
Anything else is an asset. Set "kind": "asset" to store a zip as a plain file instead of unpacking it.

Parallel chunks

Chunks can go in any order and at the same time. Three or four at once is a good default. Chunk requests don't count toward your key's per-minute limit (they have their own generous limit), but they do count as tokens: 1 per request plus 1 per MiB.

Checksums

Send X-Content-SHA256: <hex> with each chunk and the API refuses it with 400 checksum_mismatch if the bytes changed on the way. Every response also returns the chunk's sha256.

Resuming

If the connection drops, nothing is lost. Ask which chunks arrived and send the rest:

Terminal

curl $CB/v1/uploads/UPLOAD_ID \
  -H "Authorization: Bearer $CLATTER_API_KEY" -H "X-Clatter-Account: $CLATTER_ACCOUNT_ID"
# { "object": "upload", "status": "uploading", "chunk_count": 8, "received_chunks": [0, 1, 2, 5], ... }

Sending a chunk twice is safe: the same bytes answer "duplicate": true, different bytes replace it.

A complete uploader

import concurrent.futures, hashlib, os, requests

CB = "https://api.codebuckets.clatterlabs.xyz"
H = {"Authorization": f"Bearer {os.environ['CLATTER_API_KEY']}", "X-Clatter-Account": os.environ["CLATTER_ACCOUNT_ID"]}

def upload(bucket_id, path, resume_id=None):
    size = os.path.getsize(path)
    if resume_id:
        up = requests.get(f"{CB}/v1/uploads/{resume_id}", headers=H).json()
    else:
        up = requests.post(f"{CB}/v1/buckets/{bucket_id}/uploads", headers=H, json={"name": os.path.basename(path), "size": size}).json()
    have = set(up.get("received_chunks") or [])

    def send(i):
        with open(path, "rb") as f:
            f.seek(i * up["chunk_size"])
            data = f.read(up["chunk_size"])
        for attempt in range(5):
            r = requests.put(f"{CB}/v1/uploads/{up['id']}/chunks/{i}", data=data,
                             headers={**H, "Content-Type": "application/octet-stream", "X-Content-SHA256": hashlib.sha256(data).hexdigest()})
            if r.ok:
                return
            if r.status_code < 500 and r.status_code != 429:
                raise RuntimeError(r.json()["error"]["message"])
        raise RuntimeError(f"chunk {i} kept failing; resume with upload id {up['id']}")

    with concurrent.futures.ThreadPoolExecutor(4) as pool:
        list(pool.map(send, [i for i in range(up["chunk_count"]) if i not in have]))
    return requests.post(f"{CB}/v1/uploads/{up['id']}/complete", headers=H).json()

print(upload("cbk_...", "storefront.zip"))
import { createHash } from "node:crypto";
import { open, stat } from "node:fs/promises";
import { basename } from "node:path";

const CB = "https://api.codebuckets.clatterlabs.xyz";
const H = { Authorization: `Bearer ${process.env.CLATTER_API_KEY}`, "X-Clatter-Account": process.env.CLATTER_ACCOUNT_ID };

async function upload(bucketId, path, resumeId) {
  const { size } = await stat(path);
  const up = resumeId
    ? await fetch(`${CB}/v1/uploads/${resumeId}`, { headers: H }).then((r) => r.json())
    : await fetch(`${CB}/v1/buckets/${bucketId}/uploads`, { method: "POST", headers: { ...H, "Content-Type": "application/json" }, body: JSON.stringify({ name: basename(path), size }) }).then((r) => r.json());
  const have = new Set(up.received_chunks ?? []);
  const todo = [...Array(up.chunk_count).keys()].filter((i) => !have.has(i));
  const fh = await open(path);
  const send = async () => {
    for (let i; (i = todo.shift()) !== undefined; ) {
      const buf = Buffer.alloc(Math.min(up.chunk_size, size - i * up.chunk_size));
      await fh.read(buf, 0, buf.length, i * up.chunk_size);
      const sha = createHash("sha256").update(buf).digest("hex");
      for (let attempt = 0; ; attempt++) {
        const r = await fetch(`${CB}/v1/uploads/${up.id}/chunks/${i}`, { method: "PUT", headers: { ...H, "Content-Type": "application/octet-stream", "X-Content-SHA256": sha }, body: buf });
        if (r.ok) break;
        if ((r.status < 500 && r.status !== 429) || attempt === 4) throw new Error((await r.json()).error.message);
      }
    }
  };
  await Promise.all([send(), send(), send(), send()]);
  await fh.close();
  return fetch(`${CB}/v1/uploads/${up.id}/complete`, { method: "POST", headers: H }).then((r) => r.json());
}

console.log(await upload("cbk_...", "storefront.zip"));

After completing

An asset is ready immediately.
An archive in a bucket with preview on is processing while its tree is built (usually a few seconds; longer for very large zips), then ready. Poll Get an upload or watch for the code.updated event.

Space

Starting an upload reserves its size. If the bucket's stored files plus unfinished uploads plus the new file would pass 10 GB, the start answers 413 bucket_full. Abort uploads you won't finish to free their space.