S3 Table Buckets

A table bucket is an ordinary S3 bucket that also behaves as a table catalog. You create one, point a query engine at its catalog endpoint, and read and write tables — with no external metastore to deploy: no Hive Metastore, no Nessie, no AWS Glue. The tables’ data and metadata are plain objects in the bucket, so they inherit everything SeaweedFS does for the rest of your storage — replication, erasure coding, and cloud tiering. It’s all part of open-source SeaweedFS.

One bucket, one format. The format decides which built-in catalog answers for it — and both run side by side.

Two formats, two catalogs

A table bucket holds exactly one table format, and its format decides which built-in catalog serves it:

Iceberg — Apache Iceberg tables through a built-in Iceberg REST catalog (default port 8181). Any Iceberg engine — Spark, Trino, ClickHouse, Dremio, DuckDB, PyIceberg — connects with the standard REST catalog protocol.
Lance — Lance datasets, the columnar/vector format behind LanceDB, through a built-in Lance Namespace (default port 9101). LanceDB connects with connect_namespace("rest", ...).

Both catalogs authenticate with your S3 credentials, and both run side by side — one cluster can serve Iceberg and Lance buckets at once.

Create one

weed mini (and full deployments) pre-create buckets with -tableBucket, where each name takes an optional :FORMAT suffix (default ICEBERG):

# an Iceberg warehouse and a Lance vector bucket, side by side
weed mini -dir=./data -s3.config=s3config.json -tableBucket=warehouse,vectors:LANCE

On an already-running cluster, create one from weed shell:

s3tables.bucket -create -name vectors -format LANCE -account 000000000000

The Admin UI’s Table Buckets page lists every bucket, the format it holds, and the catalog endpoint that serves it:

The SeaweedFS Admin UI S3 Tables Buckets page: a callout lists the Iceberg REST catalog endpoint at :8181 and the Lance Namespace endpoint at :9101, and a vectors bucket is listed with a green LANCE format badge, its ARN, and its catalog endpoint. — The Table Buckets page — each bucket is its own catalog, and its format badge tells you which endpoint answers for it.

Query it with

Step-by-step walkthroughs, each run end to end with real commands and output:

Querying with DuckDB — attach an Iceberg table bucket from DuckDB and run SQL.
Running ClickHouse — ClickHouse’s DataLakeCatalog engine reads and writes Iceberg tables through the catalog.
Vector search with LanceDB — declare a Lance table, build an IVF_PQ index, and run nearest-neighbor search.

Keeping tables fast

Streaming or frequent small writes leave tables with many tiny data files and stale snapshots that slow every scan. SeaweedFS compacts and cleans them automatically, on dedicated workers, with no separate compaction service to run — see Iceberg Table Maintenance. Any engine that reads through the catalog picks up the compacted tables with no changes on its side.