Volume Tiering — Configuration Reference & Internals

Technical reference for Volume Tiering — the configuration document and Admin UI, selector and trigger semantics, how volume age is measured on each tier form, capacity pressure, and the Parquet footer cache’s exact behavior.

The configuration document

The entire policy is one protojson TieringConfig document stored in the filer at /etc/seaweedfs/tiering/tiering.conf. Writes go through a revision check, so two concurrent edits cannot silently overwrite each other — the Admin UI and the shell both bump the revision on every change.

Apply it from the shell (tiering.apply -file=…) or manage it interactively in the Admin UI under Storage → Volume Tiering — profile cards, the rung editor, and the binding order are shown on the feature page. A rung’s trigger criteria must all hold at once; leaving one empty ignores it, and the editor shows a live verdict of what the saved ladder would do to the volumes currently bound to it.

Selectors and classes

Selector rules match glob patterns against the normalized object path, in order, first match wins — a broad pattern above a specific one makes the specific one unreachable, and validation flags it. Your configured rules are consulted first; the built-in table-format rules apply beneath them, so a convention can be overridden by naming it:

Built-ins are inert until a binding governs their class — a cluster that upgrades and configures nothing places exactly as before.

Classification happens server-side on the write path: the filer resolves the class from the object path and injects the bound profile’s landing placement (disk type, replication, data center) into the volume assignment. An S3 PUT carries its object path into the same resolution, so S3 and filer writes classify identically. The class is part of a volume’s layout identity — (collection, replication, TTL, disk type, class) — so classes never share volumes.

A class is a preference, not a hard partition: a write whose class has no usable volume still succeeds on ordinary volumes. Placement degrades to slower reads, never to failures.

Ladder semantics

• Demotion-only. The landing is tier 0; volumes only ever descend the rungs below it, in order. Moving data back up is an operator action (ec.decode, volume.tier.move), not something the worker does.
• Triggers are the AND of their set criteria. quietForSeconds, fullnessRatio, and minSizeMb combine; an empty criterion is ignored. Validation rejects a rung with no criteria (it would match every volume) and any negative value (it would too).
• Age is measured against the right clock per form. A replicated volume’s quiet time counts from the newest ModifiedAtSecond across its replicas — a lagging replica cannot age a volume prematurely, and divergent replica sizes block the fullness evaluation rather than silently trusting one copy. An erasure-coded volume carries no meaningful modification time or fullness, so its age counts from its last (re)encode — and EC vacuum mints a fresh encode timestamp, so delete-heavy collections descend on quiet-since-last-vacuum. Fullness and minimum size are rejected on any rung positioned after the first EC rung, where they are meaningless. A legacy EC volume with no encode timestamp has unknown age: it classifies by placement and holds at its current tier rather than treating the missing stamp as infinite age.
• Tier forms. replicated (disk type + replication code + optional DC), ec (data/parity shard counts — both ≥ 1, sum within the cluster’s shard maximum — plus optional shard replica placement), remote (a backend id from master.toml [storage.backend], e.g. s3.archive; the index stays local so lookups never leave the cluster).
• The volume_tiering plugin worker performs the transitions — EC encode, EC decode back to replicated, shard relocation, replicated moves, remote upload — using the same primitives as the manual ec.encode / volume.tier.move / volume.tier.upload commands. A volume found holding two forms at once (an interrupted transition or a hand-run encode) is surfaced and left to the operator.

Capacity pressure

Per disk type, at most one pressure rule: a high watermark that starts relief, a low watermark it drains toward, a cap on moves per cycle, and a prefer-deepest-rung bias. Pressure demotes the coldest bound volumes off the filling medium ahead of their age triggers. It is configured per medium rather than per profile deliberately: the byte budget is computed once for the medium and spent across every profile sharing it — two profile-scoped policies could not do that without each freeing the space the other counts on.

The Parquet footer cache, precisely

Setting tailClass names the class footers are cached into; the Cache Parquet Footers (tail_cache) plugin job does the work. It is event-driven end to end:

1. Discovery follows the filer’s metadata stream under /buckets — no namespace rescans; cost tracks the write rate, not the object count. Candidates queue in memory (bounded; overflow is dropped, counted, and surfaced — a missed footer is a slow read, not a fault). Only objects written after the job is running are seen; pre-existing files are a backfill, an operator action.
2. Execution reads the 8-byte trailer, then the footer, and parses it to find the hot region — the ColumnIndex/OffsetIndex page index a selective reader fetches before any data page, plus Bloom filters when they fit the 8 MiB per-file budget. The cached extent is floored at 64 KiB of tail: engines commonly open a Parquet file with a single speculative read of its last 64 KiB (Arrow’s default), and a read is served per overlapping range, so a smaller extent would leave that opening read straddling onto the cold body. An encrypted (PARE) footer that cannot be parsed still gets its 64 KiB tail.
3. Placement: the bytes are uploaded into a volume of the tail class, carrying the class’s landing placement (disk type, replication, DC) — the footer lands on the same fast, replicated medium the class’s ordinary writes do.
4. Recording: the cached range is attached to the file entry as cacheChunks, separate from the body chunks, stamped newer so reads inside the extent resolve to it. The write-back carries the entry’s inode and full-precision mtime as preconditions, checked atomically under the filer’s path lock — an object rewritten mid-caching wins cleanly, and the superseded upload is orphaned rather than corrupting. Rewriting a file supersedes its cached footer; the new version is cached from its own write event.

Shell reference

tiering.apply  -file=tiering.json   # validate + apply (bumps revision)
tiering.get                         # print the current document
tiering.bind   -collection=<pat> -class=<class> -profile=<name>
tiering.unbind -collection=<pat> -class=<class>
tiering.status [-profile=<name>] [-v]   # classify volumes against their ladders
tiering.run                         # execute pending replicated→replicated moves now

The volume_tiering and tail_cache job types run on plugin workers (weed worker -admin=<host:port>), scheduled by the Admin’s plugin system; enable each job type from the Admin UI. Remote tiers additionally need a [storage.backend] entry in master.toml.

Operational notes

• Volume tiering, volume classes, and the footer cache are SeaweedFS Enterprise. The primitives they drive (-disk types, ec.encode, volume.tier.move, volume.tier.upload) are open source and remain usable by hand.
• Metadata guidance is built into the model: raise the metadata class’s replication; never erasure-code or remote-tier it. Data takes the full ladder.
• A hand-run ec.encode + ec.decode round trip outside the worker drops the volume’s configured replication (the worker re-applies it after its own decodes) — prefer tiering.run or let the worker converge.

See the feature overview at Volume Tiering and the verified end-to-end walkthrough in the blog post.