diff --git a/zonetree-dev/content/docs/benchmark/benchmark.mdx b/zonetree-dev/content/docs/benchmark/benchmark.mdx index 536b15e..bd429c9 100644 --- a/zonetree-dev/content/docs/benchmark/benchmark.mdx +++ b/zonetree-dev/content/docs/benchmark/benchmark.mdx @@ -23,7 +23,7 @@ layout: aria-labelledby="benchmark-methodology-title" >
-

Understand the numbers

+

Benchmark context

Methodology and reproduction

One indexed profile-store workload, measured across four database engines. diff --git a/zonetree-dev/content/docs/concepts/bloom-filters.mdx b/zonetree-dev/content/docs/concepts/bloom-filters.mdx index 35b006e..6088446 100644 --- a/zonetree-dev/content/docs/concepts/bloom-filters.mdx +++ b/zonetree-dev/content/docs/concepts/bloom-filters.mdx @@ -91,7 +91,7 @@ With the default capacity of 1,000,000 records, the bit arrays occupy: These figures exclude object and array overhead. Power-of-two rounding creates steps: a small setting change can double the allocation. At the maximum-size cap, a very large configured segment can receive fewer effective bits per item than requested. -## Understand false positives +## False positives A false positive sends a missing key into a B+Tree lookup that eventually fails. It costs work; it does not create a record or change key order. diff --git a/zonetree-dev/content/docs/concepts/index.mdx b/zonetree-dev/content/docs/concepts/index.mdx index 7ea0b58..d72c209 100644 --- a/zonetree-dev/content/docs/concepts/index.mdx +++ b/zonetree-dev/content/docs/concepts/index.mdx @@ -9,7 +9,7 @@ layout: ---

- + Storage model

Layers describe record generations; parts divide a logical disk segment into ranges.

 @@ -22,9 +22,9 @@ layout: - + - + diff --git a/zonetree-dev/content/docs/concepts/key-ordering.mdx b/zonetree-dev/content/docs/concepts/key-ordering.mdx index ec93d69..cabb48f 100644 --- a/zonetree-dev/content/docs/concepts/key-ordering.mdx +++ b/zonetree-dev/content/docs/concepts/key-ordering.mdx @@ -94,7 +94,7 @@ For example, a comparer that ignores case makes `"User:42"` and `"USER:42"` the Do not let a key's order depend on mutable fields, the current time, or configuration that can change while the tree is open. Avoid mutating key buffers after insertion. A key that changes position without an engine write breaks the assumptions used by lookup, iteration, sparse indexes, and merging. -## Understand strings and byte sequences +## Default ordering for common key types | Representation | Default order | Consequence | | --- | --- | --- | @@ -105,7 +105,7 @@ Do not let a key's order depend on mutable fields, the current time, or configur For text keys with embedded numbers, use an appropriate encoding or comparer; a delimiter alone does not create numeric sorting. For byte keys, make sure the byte encoding and the comparer jointly produce the intended order. Use `Memory` rather than unsupported `byte[]` keys, and retain ownership of the bytes. -## Choose forward or reverse access +## Forward and reverse traversal A forward iterator follows the configured comparer. A reverse iterator traverses that same order backward. Reverse traversal does not change equality or re-encode the stored data. diff --git a/zonetree-dev/content/docs/concepts/op-index.mdx b/zonetree-dev/content/docs/concepts/op-index.mdx index 5d33e4d..439565b 100644 --- a/zonetree-dev/content/docs/concepts/op-index.mdx +++ b/zonetree-dev/content/docs/concepts/op-index.mdx @@ -42,7 +42,7 @@ With concurrent writers, sequence assignment and method return order are differe Failed existence checks and cancelled atomic callbacks commonly report operation index `0`. That sentinel is not a new successful write. See [atomic operations](../usage/atomic-operations.mdx) for the specific return contracts. -## Understand the recovery boundary +## Operation indexes after recovery WAL records retain their original operation indexes. Reopen combines recovered WAL information with persisted high-water metadata so later allocations continue above the recovered sequence. diff --git a/zonetree-dev/content/docs/concepts/segments.mdx b/zonetree-dev/content/docs/concepts/segments.mdx index af0a090..7950e31 100644 --- a/zonetree-dev/content/docs/concepts/segments.mdx +++ b/zonetree-dev/content/docs/concepts/segments.mdx @@ -43,7 +43,7 @@ The disk abstraction is `IDiskSegment`. Both single-part and multi

Layers separate newer and older storage generations. Parts divide one logical disk segment into ordered ranges. A multipart segment still occupies one disk-segment slot or one position in the bottom queue.

-## Understand why part reuse matters +## Immutable parts and reuse diff --git a/zonetree-dev/content/docs/concepts/serializers-and-comparers.mdx b/zonetree-dev/content/docs/concepts/serializers-and-comparers.mdx index f567d26..2ba9aa3 100644 --- a/zonetree-dev/content/docs/concepts/serializers-and-comparers.mdx +++ b/zonetree-dev/content/docs/concepts/serializers-and-comparers.mdx @@ -98,7 +98,7 @@ This prints `Ada`, then `False`. JSON is an explicit format choice here; the eng The serializer can be called from concurrent application, recovery, or maintenance work. Keep it safe for concurrent use. Do not retain or mutate input buffers without owning a copy, and do not return a reusable scratch buffer that another operation can overwrite. -## Choose a stable byte representation +## Serializer format contracts | Choice | Contract to preserve | | --- | --- | @@ -109,7 +109,7 @@ The serializer can be called from concurrent application, recovery, or maintenan Changing a serializer's output changes what WAL recovery, disk reads, and backup records need to decode. Test a value written by the previous application version with the new reader before treating a deployment as compatible. -## Understand what metadata validates +## Metadata validation and compatibility The loader checks the persisted key type, value type, comparer type, key-serializer type, and value-serializer type. Metadata also records the configured hasher type and Bloom density, but those are not the same persisted-format checks. diff --git a/zonetree-dev/content/docs/concepts/value-mutability.mdx b/zonetree-dev/content/docs/concepts/value-mutability.mdx index 2f93c1f..90c6778 100644 --- a/zonetree-dev/content/docs/concepts/value-mutability.mdx +++ b/zonetree-dev/content/docs/concepts/value-mutability.mdx @@ -98,7 +98,7 @@ Returning `false` cancels the new engine write. It cannot undo a change already `TryAtomicAddOrUpdate` can retry its adder/updater when a segment freezes or fills. Build replacement values from the input of each invocation, and keep callbacks free of external effects that cannot safely repeat. An in-place increment of a shared object can be applied more than once before one write is accepted. -## Understand shallow immutability +## Shallow immutability Records, `readonly` fields, read-only interfaces, and readonly structs do not automatically make their contents immutable: diff --git a/zonetree-dev/content/docs/durability/backups.mdx b/zonetree-dev/content/docs/durability/backups.mdx index 015f93b..cdba440 100644 --- a/zonetree-dev/content/docs/durability/backups.mdx +++ b/zonetree-dev/content/docs/durability/backups.mdx @@ -11,7 +11,7 @@ description: Create and verify complete live-backup generations, choose memory c titleId="backups-title" subtitle="Create a complete live-backup generation while the tree remains open, verify a restore, and configure automatic generation and retention policies."/> -## Choose a backup shape +## Live backup, offline copy, and rebuild | Approach | What it captures | When it fits | | --- | --- | --- | @@ -81,7 +81,7 @@ Restore into an unused data directory and, if configured separately, an unused W Use the local provider, a WAL-enabled target, and compatible [data contracts](recovery.mdx#preserve-the-data-contracts). Restore copies disk files, rebuilds backed-up memory records into a read-only WAL, and creates fresh mutable WAL and metadata files. It does not need the original production WALs. -## Choose the in-memory view +## In-memory capture modes Memory records are included by default, including deletion markers needed to hide older records in copied disk files. Their batch is compressed independently of disk files, with Zstd level 0 and 1 MiB blocks by default. @@ -138,7 +138,7 @@ await backup.CreateGenerationAsync(); Console.WriteLine($"Final completed generation: {backup.CurrentGenerationId}"); ``` -The final-generation sequence stops automatic triggers and waits for active backup work before making the manual request. See [request and completion behavior](#understand-requests-and-completion) below for the individual methods. +The final-generation sequence stops automatic triggers and waits for active backup work before making the manual request. See [request and completion behavior](#backup-requests-and-completion) below for the individual methods. ### Schedule choices @@ -151,7 +151,7 @@ The final-generation sequence stops automatic triggers and waits for active back `BackupAfterMerge` defaults to `true`. Successful **normal** merges request a generation after `Start()`; bottom-segment merges do not. A schedule remains useful when writes stay in memory without enough activity to trigger a normal merge. -## Understand requests and completion +## Backup requests and completion Each coordinator runs at most one generation at a time. Requests do not form a backlog: diff --git a/zonetree-dev/content/docs/durability/index.mdx b/zonetree-dev/content/docs/durability/index.mdx index 82e7c8b..db767c6 100644 --- a/zonetree-dev/content/docs/durability/index.mdx +++ b/zonetree-dev/content/docs/durability/index.mdx @@ -9,10 +9,10 @@ layout: ---
- + - + diff --git a/zonetree-dev/content/docs/durability/recovery.mdx b/zonetree-dev/content/docs/durability/recovery.mdx index 9f6e172..4368163 100644 --- a/zonetree-dev/content/docs/durability/recovery.mdx +++ b/zonetree-dev/content/docs/durability/recovery.mdx @@ -5,7 +5,7 @@ description: Understand automatic incomplete-tail recovery, capture WAL corrupti -## Understand what detection provides +## WAL corruption diagnostics Current-format WAL records carry checksums. A mismatch is reported as a read failure; it does not repair the record or establish which bytes should replace it. Reconstructing records that fail checksum validation requires a specialized tool and a policy for deciding which records can be trusted. @@ -114,7 +114,7 @@ Use the original key/value types, serializers, comparer, compatible hasher, and See [serializers and comparers](../concepts/serializers-and-comparers.mdx) and [deletion markers and TTL](../concepts/deletion-markers-and-ttl.mdx) for those contracts. -## Choose a recovery procedure +## Inspection and reconstruction workflow 1. Preserve the affected database and any separately configured WAL location, along with the diagnostic logs. Use a separate working copy for inspection. 2. Identify the affected segment, record failures, and file format with the public readers and metadata APIs. diff --git a/zonetree-dev/content/docs/durability/wal-modes.mdx b/zonetree-dev/content/docs/durability/wal-modes.mdx index 1dbab88..a833a49 100644 --- a/zonetree-dev/content/docs/durability/wal-modes.mdx +++ b/zonetree-dev/content/docs/durability/wal-modes.mdx @@ -46,7 +46,7 @@ For codec defaults and block-size controls, see [compression](../storage/compres Existing WAL files keep their detected logging mode and recorded compression format. Changing the factory configures newly created WALs; reopening an existing file does not convert it to another mode. The built-in provider detects the existing mode from its filename, so preserve WAL filenames when moving a database. -## Understand the compressed tail +## Compressed blocks and the tail file Compressed WALs have a main file of committed blocks and a companion `.tail` file for the unfinished block. Keep both when making an offline copy. diff --git a/zonetree-dev/content/docs/getting-started.mdx b/zonetree-dev/content/docs/getting-started.mdx index 9c56eac..8fccf33 100644 --- a/zonetree-dev/content/docs/getting-started.mdx +++ b/zonetree-dev/content/docs/getting-started.mdx @@ -132,7 +132,7 @@ Run `dotnet run` from the same project directory. It prints `Persisted 20: Grace

@@ -143,7 +143,7 @@ Run `dotnet run` from the same project directory. It prints `Persisted 20: Grace

-## Choose what to learn next +## Further reading Continue with one of these guides, or [browse the Usage overview](usage/index.mdx) to compare reads, writes, atomic operations, transactions, scans, and maintenance. @@ -157,7 +157,7 @@ Continue with one of these guides, or [browse the Usage overview](usage/index.md label="Opening a tree"/> Understand maintenance + >Background maintenance

@@ -117,7 +117,7 @@ if (tree.TryGet(42, out var value)) summary="See how mutable memory, read-only segments, compaction, and disk parts work together. Use that model to understand writes, reads, and maintenance." href="/docs/concepts/lsm-tree/" icon="tabler:stack-2" - label="Understand the architecture"/> + label="LSM-tree architecture"/>
  • Choose key order and prefix layouts + >Key ordering and prefix layouts
  • Choose a write-ahead logging mode + >Write-ahead logging modes
  • -

    Understand storage costs

    +

    Storage costs

    Work out what drives disk reads, rewrite work, and memory use.

     diff --git a/zonetree-dev/content/docs/storage/compression.mdx b/zonetree-dev/content/docs/storage/compression.mdx index b7611d6..384f18a 100644 --- a/zonetree-dev/content/docs/storage/compression.mdx +++ b/zonetree-dev/content/docs/storage/compression.mdx @@ -25,7 +25,7 @@ Both paths expose a compression method, method-specific level, and block size, b Changing compression options does not rewrite existing files. A merge may write new files with the new profile while reusing eligible parts with the earlier profile. Multipart descriptors have their own metadata encoding; disk data-block settings do not configure that encoding. - +

    A compression method or level does not define when a write is durable. AsyncCompressed, SyncCompressed, plain Sync, and None have different logging and failure behavior. Choose the mode with the WAL modes guide, then tune compression within that choice.

     @@ -83,7 +83,7 @@ Console.WriteLine($"Read {value.Length} characters after merging."); The two methods are intentionally independent; this is an example profile, not a universal performance recommendation. `SetDiskSegmentCompressionBlockSize()` is a factory shorthand for the disk block-size option. -## Understand the unit of a disk read +## Block reads and caching Disk files contain individually addressable blocks. A cold read locates and reads the relevant compressed block, decompresses it, and serves the requested bytes. Subsequent reads can reuse the decompressed block while it remains cached. diff --git a/zonetree-dev/content/docs/storage/disk-segments.mdx b/zonetree-dev/content/docs/storage/disk-segments.mdx index d2421b3..169dec6 100644 --- a/zonetree-dev/content/docs/storage/disk-segments.mdx +++ b/zonetree-dev/content/docs/storage/disk-segments.mdx @@ -40,7 +40,7 @@ An illustrative multipart segment with two variable-value parts looks like this: These are segment files. Metadata and any record WALs belong to the wider database layout described in the [Storage overview](index.mdx). -## Understand the four record layouts +## Fixed-size and variable-size record layouts Each physical single-part segment selects a layout from the runtime key and value types. Types with no managed references use the fixed-size path; strings, arrays, and types containing references use the variable-size path. @@ -113,7 +113,7 @@ File IDs depend on the tree's history. An empty current disk slot may enumerate

    GetFiles() describes a segment; it does not freeze the tree or acquire a backup lease. During concurrent maintenance, use the live-backup API to coordinate segment lifetimes and capture in-memory records and restore metadata.

     -## Understand when files disappear +## File lifetime after a merge Merges publish replacement segments rather than rewriting the record data of an active segment. Once a segment is retired, its files can be reclaimed when no reader or backup still leases them. Sparse-index files are auxiliary persisted indexes, separate from that immutable record data. diff --git a/zonetree-dev/content/docs/storage/file-stream-providers.mdx b/zonetree-dev/content/docs/storage/file-stream-providers.mdx index c077d34..83d6450 100644 --- a/zonetree-dev/content/docs/storage/file-stream-providers.mdx +++ b/zonetree-dev/content/docs/storage/file-stream-providers.mdx @@ -6,7 +6,7 @@ description: Choose local or RAM-backed files, reopen an in-memory tree correctl diff --git a/zonetree-dev/content/docs/storage/index.mdx b/zonetree-dev/content/docs/storage/index.mdx index e8e05c3..f18ef7b 100644 --- a/zonetree-dev/content/docs/storage/index.mdx +++ b/zonetree-dev/content/docs/storage/index.mdx @@ -9,7 +9,7 @@ layout: ---
    - + Representation

    Sorted records become block-addressable files, indexes, and reusable parts.

     diff --git a/zonetree-dev/content/docs/tuning/disk-segments.mdx b/zonetree-dev/content/docs/tuning/disk-segments.mdx index 4e6b845..7b8e6de 100644 --- a/zonetree-dev/content/docs/tuning/disk-segments.mdx +++ b/zonetree-dev/content/docs/tuning/disk-segments.mdx @@ -17,7 +17,7 @@ All three are record counts, not byte budgets. The disk threshold is checked aft For terminology and file ownership, see [segments and parts](../concepts/segments.mdx) and [disk segment files](../storage/disk-segments.mdx). -## Choose a segment mode and target range +## Segment modes and part targets `MultiPartDiskSegment` is the default. It represents one logical segment with ordered physical parts and allows eligible parts to be reused. `SingleDiskSegment` uses a single-part layout; it still can have separate data, header, and sparse-index files. @@ -79,7 +79,7 @@ Console.WriteLine($"Physical files: {disk.GetFiles().Length}"); The small dataset produces a final part below the configured minimum, which is a normal outcome. With concurrent writers, coordinate the observation before comparing file sets or merge outputs. -## Choose block size and sparse density for reads +## Block size and sparse-index density Disk compression defaults to Zstd level 0 with 4 MiB blocks. A smaller block can reduce unrelated bytes decompressed for isolated reads, while a larger block can improve locality and compression. A value larger than one block still needs all of its payload bytes. Existing files retain their recorded block size and method; a configuration change alone does not rewrite them. See [compression](../storage/compression.mdx) for a controlled comparison. diff --git a/zonetree-dev/content/docs/tuning/index.mdx b/zonetree-dev/content/docs/tuning/index.mdx index 351e7e6..3c90d94 100644 --- a/zonetree-dev/content/docs/tuning/index.mdx +++ b/zonetree-dev/content/docs/tuning/index.mdx @@ -14,7 +14,7 @@ layout: - + diff --git a/zonetree-dev/content/docs/tuning/large-values.mdx b/zonetree-dev/content/docs/tuning/large-values.mdx index 8913a13..559cbe6 100644 --- a/zonetree-dev/content/docs/tuning/large-values.mdx +++ b/zonetree-dev/content/docs/tuning/large-values.mdx @@ -5,7 +5,7 @@ description: Translate record limits into payload costs, choose direct values or -## Choose what belongs in the value +## Payload storage models | Model | Useful when | Responsibility to account for | | --- | --- | --- | diff --git a/zonetree-dev/content/docs/tuning/write-amplification.mdx b/zonetree-dev/content/docs/tuning/write-amplification.mdx index a32386c..a001a8b 100644 --- a/zonetree-dev/content/docs/tuning/write-amplification.mdx +++ b/zonetree-dev/content/docs/tuning/write-amplification.mdx @@ -11,7 +11,7 @@ Write amplification compares internal bytes written with the application's logic Record overwrite distribution as well as volume. Ten thousand updates concentrated in one range can produce different merge work from the same count spread across the whole keyspace. -## Understand when a part can be reused +## Part reuse conditions @@ -65,7 +65,7 @@ Console.WriteLine($"New data files: {after.Except(before).Count()}"); Application writes are coordinated in this example, and both merges finish before inspection. The file counts vary with randomized boundaries. Shared filenames demonstrate reuse; use I/O measurements for the actual bytes written. The part targets here make the mechanism observable, rather than prescribe a production profile. -## Choose the lever that matches the pressure +## Rewrite costs and tradeoffs | Observation | Experiment | Tradeoff to watch | | --- | --- | --- | diff --git a/zonetree-dev/content/docs/usage/atomic-operations.mdx b/zonetree-dev/content/docs/usage/atomic-operations.mdx index 1cef706..3741cf6 100644 --- a/zonetree-dev/content/docs/usage/atomic-operations.mdx +++ b/zonetree-dev/content/docs/usage/atomic-operations.mdx @@ -53,7 +53,7 @@ tree.TryAtomicAddOrUpdate( Run it again against the same directory to increment the persisted counter. The callback reports `Added` on initialization and `Updated` on a later increment. -## Understand the coordination boundary +## Coordination across storage layers The newest value may be in mutable memory, a read-only segment, or a disk layer. Atomic methods synchronize this sequence across those layers: diff --git a/zonetree-dev/content/docs/usage/reads-and-writes.mdx b/zonetree-dev/content/docs/usage/reads-and-writes.mdx index 0d09c2b..ed67c5f 100644 --- a/zonetree-dev/content/docs/usage/reads-and-writes.mdx +++ b/zonetree-dev/content/docs/usage/reads-and-writes.mdx @@ -36,7 +36,7 @@ if (tree.TryDelete(7, out var deletedAt)) For setup and resource lifetime, see [opening a tree](opening-a-tree.mdx). -## Choose an operation +## Read and write APIs | Your data rule | API | Important detail | | --- | --- | --- | @@ -124,7 +124,7 @@ Use [transactions](transactions.mdx) when the invariant spans several keys in on Different key groups can use different write disciplines. For example, profile replacement can use `Upsert` while a separate counter uses atomic methods. Keep the discipline consistent for every writer that can change the same invariant. - Choose the next operation + Range scans and coordinated updates

    Read a range, update one key based on its current value, or coordinate a group of keys. diff --git a/zonetree-dev/content/docs/usage/transactions.mdx b/zonetree-dev/content/docs/usage/transactions.mdx index 1599416..d708b3f 100644 --- a/zonetree-dev/content/docs/usage/transactions.mdx +++ b/zonetree-dev/content/docs/usage/transactions.mdx @@ -58,7 +58,7 @@ catch (TransactionAbortedException) `OpenOrCreateTransactional()` returns the transactional API. It differs from the ordinary tree: reads and writes in a transaction take a transaction ID, and success depends on the commit outcome. -## Choose the smallest coordination scope +## Single-key and multi-key coordination | Data rule | API | | --- | --- | @@ -68,7 +68,7 @@ catch (TransactionAbortedException) Concurrent application threads alone do not require transactions. Use them for a multi-key invariant, and make the reads that decide the writes part of that transaction. -## Understand the transaction lifecycle +## Transaction lifecycle 1. `BeginTransaction()` allocates a transaction ID. 2. Reads and writes use that ID.