One Table Bucket, Many Engines: DuckDB and Trino via databow

A SeaweedFS table bucket is an S3 bucket that doubles as an Apache Iceberg warehouse, fronted by a built-in Iceberg REST catalog. Because both metadata and data live behind standard protocols — Iceberg REST for the catalog, S3 for the Parquet — the query engine is a client-side choice. Nothing on the SeaweedFS side changes when you swap engines.

That’s easier to see than to read, and the tool that makes it visible is databow: a small Rust CLI that sends SQL to any engine reachable over ADBC (Arrow Database Connectivity). Point it at an engine, hand it a query, get a result table. Swapping engines is a flag: --driver trino sends the SQL to a running Trino server; --driver duckdb runs the same query in-process.

This post runs the same table and the same queries through both engines — every command below was run end to end on one machine, and every output block is a real capture. Along the way we’ll hit a third engine that doesn’t work, which turns out to be the most instructive part.

What you’ll need

• The weed binary — table buckets and the built-in Iceberg REST catalog are part of open-source SeaweedFS. Grab a recent release from GitHub releases.
• databow — uv tool install databow, then fetch the engine drivers with its companion tool: dbc install trino and dbc install duckdb.
• Docker — only to run Trino. DuckDB needs nothing extra; it loads inside the databow process.

Step 1 — Start SeaweedFS with a table bucket

Define one S3 identity — the same access key authenticates the catalog (as OAuth2 client credentials) and the S3 data reads. Save as s3config.json:

{
  "identities": [
    {
      "name": "analyst",
      "credentials": [
        { "accessKey": "tutorialkey", "secretKey": "tutorialsecret" }
      ],
      "actions": ["Admin", "Read", "Write", "List", "Tagging"]
    }
  ]
}

Launch everything with weed mini — master, volume, filer, S3 gateway, and the Iceberg catalog in one process. The default ports were taken on this machine, so the run below pins explicit ones — drop the *.port flags to get the usual 8333/8181:

weed mini -dir=./data -ip=192.168.1.103 \
  -s3.config=s3config.json \
  -s3.externalUrl=http://host.docker.internal:8533 \
  -master.port=9444 -volume.port=8180 -filer.port=8988 \
  -s3.port=8533 -s3.port.iceberg=8581 -admin.port=24646

  All enabled components are running and ready to use:

    Master UI:       http://192.168.1.103:9444
    Volume Server:   http://192.168.1.103:8180
    Filer UI:        http://192.168.1.103:8988
    WebDAV:          http://192.168.1.103:7333
    S3 Endpoint:     http://192.168.1.103:8533
    Iceberg Catalog: http://192.168.1.103:8581
    Admin UI:        http://127.0.0.1:24646

Two endpoints matter below: the Iceberg catalog (:8581, where engines find tables) and the S3 endpoint (:8533, where the Parquet lives). The -s3.externalUrl flag controls what S3 address the catalog advertises to clients; host.docker.internal reaches the host from inside containers — Step 2’s Trino runs in one — while host-local clients like DuckDB use the endpoint in their own configuration anyway.

Create the table bucket itself:

aws s3tables create-table-bucket --name analytics \
  --endpoint-url http://localhost:8533

{
    "arn": "arn:aws:s3tables:us-east-1:analyst:bucket/analytics"
}

Step 2 — Create and load a table: databow + Trino

Start Trino in Docker with a catalog file pointing at SeaweedFS. Save as iceberg.properties:

connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=http://host.docker.internal:8581
iceberg.rest-catalog.warehouse=analytics
iceberg.rest-catalog.security=OAUTH2
iceberg.rest-catalog.oauth2.credential=tutorialkey:tutorialsecret
fs.s3.enabled=true
s3.endpoint=http://host.docker.internal:8533
s3.region=us-east-1
s3.aws-access-key=tutorialkey
s3.aws-secret-key=tutorialsecret
s3.path-style-access=true

docker run -d --name trino -p 8182:8080 \
  -v $PWD/iceberg.properties:/etc/trino/catalog/iceberg.properties \
  trinodb/trino:latest

One detail worth knowing: iceberg.rest-catalog.uri takes the catalog base URL — iceberg-java appends /v1 itself, so a /v1-suffixed URI produces /v1/v1/config requests.

Now create a schema and load a million rows — through databow, not Trino’s own CLI. Catalog and schema go in the connection URI; the SQL is ordinary Trino SQL:

databow --driver trino \
  --uri 'http://trino@localhost:8182?catalog=iceberg' \
  --query "CREATE SCHEMA sales"

databow --driver trino \
  --uri 'http://trino@localhost:8182?catalog=iceberg&schema=sales' \
  --query "CREATE TABLE events AS
    SELECT a.i * 100 + b.i AS id,
           'cat_' || CAST((a.i * 100 + b.i) % 20 AS varchar) AS category,
           from_unixtime(1700000000 + a.i * 100 + b.i) AS ts,
           ((a.i * 100 + b.i) * 7919) % 1000 AS value
    FROM UNNEST(sequence(0, 9999)) a(i)
    CROSS JOIN UNNEST(sequence(0, 99)) b(i)"

┌─────────┐
│  rows   │
├─────────┤
│ 1000000 │
└─────────┘

Trino planned the scan, wrote Parquet into the table bucket, and committed the Iceberg metadata through the SeaweedFS catalog — databow was just the messenger.

Step 3 — Same table, no server at all: databow + DuckDB

Now switch engines by changing one flag. The DuckDB driver loads an embedded engine inside the databow process — no daemon, nothing deployed. The only setup is per-invocation: each databow run is a fresh in-memory DuckDB, so the catalog ATTACH and the secrets go in the same query:

databow --driver duckdb --query "
CREATE SECRET catalog_auth (
    TYPE ICEBERG,
    CLIENT_ID 'tutorialkey',
    CLIENT_SECRET 'tutorialsecret',
    OAUTH2_SERVER_URI 'http://localhost:8581/v1/oauth/tokens'
);
CREATE SECRET s3_read (
    TYPE S3, KEY_ID 'tutorialkey', SECRET 'tutorialsecret',
    ENDPOINT 'localhost:8533', URL_STYLE 'path', USE_SSL false
);
INSTALL iceberg; LOAD iceberg; INSTALL httpfs; LOAD httpfs;
ATTACH 's3://analytics' AS lake
  (TYPE ICEBERG, ENDPOINT 'http://localhost:8581', SECRET catalog_auth);
SELECT category, count(*) AS events, avg(value) AS avg_value
FROM lake.sales.events
GROUP BY category
ORDER BY category
LIMIT 3;
"

┌──────────┬────────┬───────────┐
│ category │ events │ avg_value │
├──────────┼────────┼───────────┤
│ cat_0    │ 50000  │ 490.0     │
│ cat_1    │ 50000  │ 509.0     │
│ cat_10   │ 50000  │ 500.0     │
└──────────┴────────┴───────────┘

Rows Trino wrote, read back by DuckDB — a different engine, a different process, the same Parquet. One caveat: DuckDB attaches the REST catalog read-only — it’s a superb query engine but not a writer here, which is why Trino did the loading.

The same query through Trino for symmetry:

databow --driver trino \
  --uri 'http://trino@localhost:8182?catalog=iceberg&schema=sales' \
  --query "SELECT category, count(*) AS events, avg(value) AS avg_value
           FROM events GROUP BY category ORDER BY category LIMIT 3"

┌──────────┬────────┬───────────┐
│ category │ events │ avg_value │
├──────────┼────────┼───────────┤
│ cat_0    │ 50000  │ 490.0     │
│ cat_1    │ 50000  │ 509.0     │
│ cat_10   │ 50000  │ 500.0     │
└──────────┴────────┴───────────┘

Step 4 — The engine that doesn’t work: DataFusion

databow also ships a datafusion driver — an embedded Rust engine, same in-process shape as DuckDB. Try it on this table and it fails:

databow --driver datafusion --query "
CREATE EXTERNAL TABLE events STORED AS ICEBERG
LOCATION 's3://analytics/sales/events'"

Failed to execute statement: InvalidArguments: [datafusion] Given table options incompatible with the 's3' scheme (sqlstate: 00000, vendor_code: -2147483648)

That’s the instructive failure: “any engine can query a table bucket” has a floor. The engine needs both an Iceberg catalog client and an S3-capable filesystem reader. DataFusion’s embedded driver registers no s3:// object-store scheme in this setup, so it can’t express the table location — let alone plan the scan.

Trino and DuckDB both pass the two-part test — catalog protocol plus object-store access. DataFusion doesn’t. When evaluating engines for a table bucket, that’s the checklist.

How the engines actually differ

The SQL and the data are identical, so what does the engine choice change? Operational shape:

Measured on this walkthrough’s million-row table (median of 5 runs, wall clock per databow invocation — driver load and connection setup included):

Read it honestly: subtract the baseline and both engines scan a million local Parquet rows in tens of milliseconds. At this scale the engines are equivalent — the table mostly measures invocation overhead (DuckDB’s per-process secrets and ATTACH; Trino’s HTTP round-trip). The real divergence is operational:

• Interactive poking at a table bucket: DuckDB wins on simplicity — no service, nothing to deploy.
• Many users or big data: Trino’s scheduler, caching, and worker fleet are the point; a single-node library can’t offer them.
• Federation: Trino joins the Iceberg table against Postgres, Kafka, or Hive in one query. Embedded engines can’t.

How it works

• Warehouse addressing. Each table bucket is its own Iceberg warehouse, addressed as s3://<table-bucket> (DuckDB’s ATTACH argument) or iceberg.rest-catalog.warehouse (Trino). The catalog’s /v1/config maps it to a routing prefix; subsequent calls stay scoped to that bucket.
• One credential, two protocols. The catalog speaks OAuth2 client-credentials: the S3 access key is client_id, the secret key is client_secret, exchanged at /v1/oauth/tokens. The same pair signs Parquet reads and writes at the S3 endpoint.
• Two data paths. The catalog hands out table metadata — schemas, snapshots, file lists. Engines then fetch the Parquet directly from S3, so scans never bottleneck on the catalog.
• ADBC is the seam. databow contributes nothing but transport: dbc install <driver> drops an ADBC driver locally, and --driver picks which engine plans and executes the SQL.

Keeping tables fast

Streaming or frequent small writes leave Iceberg tables with many tiny Parquet files and stale snapshots that slow every scan. SeaweedFS compacts and cleans them automatically, on dedicated workers, with no separate compaction service to run — see Iceberg Table Maintenance. Every engine in this post reads the compacted tables with no changes on its side.

Part of a series on SeaweedFS table buckets: Querying with DuckDB · Running ClickHouse · Vector search with LanceDB · DuckDB and Trino via databow (this post).