Git Repositories in an S3 Bucket: Running walgit on SeaweedFS

walgit is a git server that is one binary in front of an object store. Point it at a bucket and you get smart-HTTP fetch/push, LFS, a browsing web UI, and a JSON API — with no database, no leader, and no local state that matters. Every machine running walgit is a disposable cache; the bucket is the repository. It’s a Rust implementation of the architecture Cursor described in Git at any scale.

The repository lives in the bucket as an immutable write-ahead log: a tiny manifest.pb (the head sequence and live pack set), append-only log/<seq>.pb entries, and content-addressed wal/<sha>.pack files. A push uploads its objects, then rewrites manifest.pb with a compare-and-swap — that CAS is the consensus. No election, no quorum: two racing pushes cannot both win, and any replica that reads the manifest has the repository.

That design needs one thing from the store: an atomic compare-and-swap on an object (conditional PUT). SeaweedFS’s S3 gateway supports conditional writes — If-None-Match and If-Match preconditions — so walgit’s commit point works on it directly. This post runs the whole thing end to end on one machine; every command and output block is a real capture.

Step 1 — Start SeaweedFS with a bucket for walgit

Give walgit a bucket and a scoped S3 identity. Save this as walgit-s3.json:

{
  "identities": [
    {
      "name": "walgit",
      "credentials": [
        { "accessKey": "walgitkey", "secretKey": "walgitsecret1234567890" }
      ],
      "actions": ["Read:walgit", "Write:walgit", "List:walgit", "Tagging:walgit"]
    }
  ]
}

Start SeaweedFS with weed mini, pre-creating the walgit bucket and serving S3 on :8333:

weed mini -dir=./data -s3.config=walgit-s3.json -bucket=walgit

Step 2 — Build walgit

walgit has no prebuilt releases yet, so build the server binary from source. You need Rust (per its rust-toolchain.toml, 1.97 here), protoc, and Node + pnpm for the embedded web UI:

git clone https://github.com/tobi/walgit.git && cd walgit
corepack enable                                   # provides pnpm
(cd web && pnpm install --frozen-lockfile && pnpm run build)   # the React UI
cargo build --release --bin walgit-server

That produces ./target/release/walgit-server. (There’s also a Containerfile and a Nix flake if you’d rather not build the toolchain yourself.)

Step 3 — Point walgit at SeaweedFS

walgit’s store config is where SeaweedFS goes. Save this as walgit.toml — the two keys that matter for a self-hosted S3 are endpoint and force_path_style = true:

[server]
listen = "127.0.0.1:8080"
public_url = "http://127.0.0.1:8080"
auto_create_on_push = true          # git push to a new name creates the repo

[server.tls]
mode = "off"                        # plain HTTP for a local walkthrough

[server.auth]
mode = "none"                       # loopback: everyone is anon with write

[store]
backend = "s3"
bucket = "walgit"

[store.s3]
endpoint = "http://127.0.0.1:8333"  # the SeaweedFS S3 gateway
region = "us-east-1"
access_key_env = "AWS_ACCESS_KEY_ID"
secret_key_env = "AWS_SECRET_ACCESS_KEY"
force_path_style = true             # required for SeaweedFS

[cache]
dir = "/tmp/walgit-cache"

walgit reads the credentials from the environment. Start it:

AWS_ACCESS_KEY_ID=walgitkey AWS_SECRET_ACCESS_KEY=walgitsecret1234567890 \
  ./target/release/walgit-server --config walgit.toml

INFO walgit_cli::serve: opening store, backend: S3
INFO walgit_cli::serve: store ready, backend: "s3"
INFO walgit_server: walgit-server listening, addr: 127.0.0.1:8080, tls: false, url: http://127.0.0.1:8080

It connected to SeaweedFS and is serving. Leave it running.

Step 4 — Use it

A push to a name that doesn’t exist creates the repository (auto_create_on_push). Make a local project and push it:

mkdir app && cd app && git init -q -b main
echo "# Hello, SeaweedFS" > README.md
echo 'print("hello from walgit on seaweedfs")' > main.py
git add -A && git commit -qm "initial commit"
git remote add origin http://127.0.0.1:8080/acme/hello.git
git push -u origin main

remote: * walgit: acme/hello — push by anon
To http://127.0.0.1:8080/acme/hello.git
 * [new branch]      main -> main

Clone it back into a fresh directory — served straight from the bucket:

git clone http://127.0.0.1:8080/acme/hello.git cloned

Cloning into 'cloned'...

ls cloned         # README.md  main.py

A second commit and push exercises the part that makes walgit interesting — the manifest compare-and-swap:

cd cloned && echo "print('second commit')" >> main.py
git commit -qam "add a line" && git push origin main

remote: * walgit: acme/hello — push by anon
To http://127.0.0.1:8080/acme/hello.git
   f07b188..e800314  main -> main

That fast-forward went through by re-reading manifest.pb, validating the ref’s old value, and CAS-writing the new one — the round-trip SeaweedFS’s conditional writes make atomic. git ls-remote confirms the tip:

e80031409ce9c432589b50e121bfe2abc5d5df4e	refs/heads/main
e80031409ce9c432589b50e121bfe2abc5d5df4e	HEAD

The repository is a WAL in the bucket

Nothing about this is opaque — list the bucket and you see the write-ahead log itself. Two pushes produced two log entries, two content-addressed packs, and one manifest:

     256  repos/acme/hello/manifest.pb                 ← the CAS commit point
     290  repos/acme/hello/log/0000000000000001.pb     ← push 1 (immutable)
     332  repos/acme/hello/log/0000000000000002.pb     ← push 2 (immutable)
     321  repos/acme/hello/wal/03a032e4….pack          ← push 1's objects
    1156  repos/acme/hello/wal/03a032e4….idx
     306  repos/acme/hello/wal/f34df035….pack          ← push 2's objects
    1184  repos/acme/hello/wal/f34df035….idx

Only manifest.pb is ever overwritten; everything else is immutable and content-addressed. Run a second walgit anywhere pointed at this bucket and it serves the same repository — it reads the manifest and it’s there.

The web UI

walgit serves a browsing UI from the same binary. The repository, rendered from the objects in SeaweedFS:

walgit also exposes its own WAL: a health page that shows the manifest invariants hold and this instance’s local copy is reconciled with the log in the bucket.

Why SeaweedFS is a good home for it

• The bucket is the source of truth, so durability is SeaweedFS’s job. Every pack and manifest inherits SeaweedFS replication or erasure coding and can tier to cold cloud storage — your git history gets the same durability as the rest of your data, at object-store cost.
• The CAS commit point just works. walgit’s whole consistency model rests on an atomic conditional PUT; SeaweedFS’s conditional writes provide exactly that, so pushes are safe even with several walgit instances pointed at one bucket.
• Stateless, disposable servers. walgit hosts hold nothing that matters — scale them out or kill them, and the repositories are untouched in SeaweedFS. One S3 endpoint backs your data lake, your backups, and now your git hosting.

walgit is an independent open-source project by Tobias Lütke; this post configures it to run on SeaweedFS. Background reading: Cursor’s Git at any scale.