mirror of
https://github.com/ZoneTree/ZoneTree.git
synced 2026-10-28 04:27:32 +08:00
Cleanup ai slope.
This commit is contained in:
Ahmed Yasin Koculu
@@ -23,7 +23,7 @@ layout:
|
||||
aria-labelledby="benchmark-methodology-title"
|
||||
>
|
||||
<header class="zt-methodology-heading">
|
||||
<p class="zt-methodology-eyebrow">Understand the numbers</p>
|
||||
<p class="zt-methodology-eyebrow">Benchmark context</p>
|
||||
<h2 id="benchmark-methodology-title">Methodology and reproduction</h2>
|
||||
<p>
|
||||
One indexed profile-store workload, measured across four database engines.
|
||||
|
||||
@@ -91,7 +91,7 @@ With the default capacity of 1,000,000 records, the bit arrays occupy:
|
||||
|
||||
These figures exclude object and array overhead. Power-of-two rounding creates steps: a small setting change can double the allocation. At the maximum-size cap, a very large configured segment can receive fewer effective bits per item than requested.
|
||||
|
||||
## Understand false positives
|
||||
## False positives
|
||||
|
||||
A false positive sends a missing key into a B+Tree lookup that eventually fails. It costs work; it does not create a record or change key order.
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ layout:
|
||||
---
|
||||
|
||||
<div class="zt-docs-overview">
|
||||
<SectionHeader class="zt-guide-intro" eyebrow="CONCEPTS / BUILD THE MENTAL MODEL" title="Understand the contracts behind the APIs" titleTag="h1" titleId="concepts-title" subtitle="Connect the LSM storage model with key order, serialization, filtering, deletion, write sequencing, and value ownership."/>
|
||||
<SectionHeader class="zt-guide-intro" eyebrow="CONCEPTS / BUILD THE MENTAL MODEL" title="Storage model and API contracts" titleTag="h1" titleId="concepts-title" subtitle="Connect the LSM storage model with key order, serialization, filtering, deletion, write sequencing, and value ownership."/>
|
||||
|
||||
<Grid columns="1" columnsMd="3" class="gap-3 zt-guide-outcomes">
|
||||
<Panel class="p-0" bodyClass="p-4"><strong>Storage model</strong><p>Layers describe record generations; parts divide a logical disk segment into ranges.</p></Panel>
|
||||
@@ -22,9 +22,9 @@ layout:
|
||||
<DocCard number="02 / STORAGE UNITS" title="Segments and parts" summary="Distinguish memory and disk layers, logical segments, reusable parts, and the counters that describe them." href="/docs/concepts/segments/" icon="tabler:database" label="Name the storage units"/>
|
||||
<DocCard number="03 / QUERY SHAPE" title="Key ordering" summary="Put related records into contiguous comparer order and implement a complete composite-key example." href="/docs/concepts/key-ordering/" icon="tabler:arrows-sort" label="Design the keyspace"/>
|
||||
<DocCard number="04 / DATA CONTRACTS" title="Serializers and comparers" summary="Separate equality, hashing, and persisted bytes. Configure custom components and preserve compatibility." href="/docs/concepts/serializers-and-comparers/" icon="tabler:code" label="Define the contracts"/>
|
||||
<DocCard number="05 / POINT READS" title="Bloom filters" summary="Skip provably absent memory-segment lookups and understand filter allocation, hashing, and false positives." href="/docs/concepts/bloom-filters/" icon="tabler:settings" label="Understand the filter"/>
|
||||
<DocCard number="05 / POINT READS" title="Bloom filters" summary="Skip provably absent memory-segment lookups and understand filter allocation, hashing, and false positives." href="/docs/concepts/bloom-filters/" icon="tabler:settings" label="How the filter works"/>
|
||||
<DocCard number="06 / VISIBILITY" title="Deletion markers and TTL" summary="Choose tombstones, model expiration, and reclaim deleted storage without exposing older values." href="/docs/concepts/deletion-markers-and-ttl/" icon="tabler:shield-check" label="Define what is live"/>
|
||||
<DocCard number="07 / WRITE IDENTITY" title="Operation indexes" summary="Use a producer's write sequence without confusing it with durable delivery, global ordering, or transaction commits." href="/docs/concepts/op-index/" icon="tabler:git-branch" label="Understand write sequencing"/>
|
||||
<DocCard number="07 / WRITE IDENTITY" title="Operation indexes" summary="Use a producer's write sequence without confusing it with durable delivery, global ordering, or transaction commits." href="/docs/concepts/op-index/" icon="tabler:git-branch" label="Write sequencing and replay"/>
|
||||
<DocCard number="08 / OWNERSHIP" title="Value mutability" summary="Keep stored objects and buffers read-only, create replacements, and make cancellation leave no shared mutation." href="/docs/concepts/value-mutability/" icon="tabler:book-2" label="Own stored values safely"/>
|
||||
</Grid>
|
||||
|
||||
|
||||
@@ -94,7 +94,7 @@ For example, a comparer that ignores case makes `"User:42"` and `"USER:42"` the
|
||||
|
||||
Do not let a key's order depend on mutable fields, the current time, or configuration that can change while the tree is open. Avoid mutating key buffers after insertion. A key that changes position without an engine write breaks the assumptions used by lookup, iteration, sparse indexes, and merging.
|
||||
|
||||
## Understand strings and byte sequences
|
||||
## Default ordering for common key types
|
||||
|
||||
| Representation | Default order | Consequence |
|
||||
| --- | --- | --- |
|
||||
@@ -105,7 +105,7 @@ Do not let a key's order depend on mutable fields, the current time, or configur
|
||||
|
||||
For text keys with embedded numbers, use an appropriate encoding or comparer; a delimiter alone does not create numeric sorting. For byte keys, make sure the byte encoding and the comparer jointly produce the intended order. Use `Memory<byte>` rather than unsupported `byte[]` keys, and retain ownership of the bytes.
|
||||
|
||||
## Choose forward or reverse access
|
||||
## Forward and reverse traversal
|
||||
|
||||
A forward iterator follows the configured comparer. A reverse iterator traverses that same order backward. Reverse traversal does not change equality or re-encode the stored data.
|
||||
|
||||
|
||||
@@ -42,7 +42,7 @@ With concurrent writers, sequence assignment and method return order are differe
|
||||
|
||||
Failed existence checks and cancelled atomic callbacks commonly report operation index `0`. That sentinel is not a new successful write. See [atomic operations](../usage/atomic-operations.mdx) for the specific return contracts.
|
||||
|
||||
## Understand the recovery boundary
|
||||
## Operation indexes after recovery
|
||||
|
||||
WAL records retain their original operation indexes. Reopen combines recovered WAL information with persisted high-water metadata so later allocations continue above the recovered sequence.
|
||||
|
||||
|
||||
@@ -43,7 +43,7 @@ The disk abstraction is `IDiskSegment<TKey, TValue>`. Both single-part and multi
|
||||
<p>Layers separate newer and older storage generations. Parts divide one logical disk segment into ordered ranges. A multipart segment still occupies one disk-segment slot or one position in the bottom queue.</p>
|
||||
</AlertBox>
|
||||
|
||||
## Understand why part reuse matters
|
||||
## Immutable parts and reuse
|
||||
|
||||
<LsmPartReuse/>
|
||||
|
||||
|
||||
@@ -98,7 +98,7 @@ This prints `Ada`, then `False`. JSON is an explicit format choice here; the eng
|
||||
|
||||
The serializer can be called from concurrent application, recovery, or maintenance work. Keep it safe for concurrent use. Do not retain or mutate input buffers without owning a copy, and do not return a reusable scratch buffer that another operation can overwrite.
|
||||
|
||||
## Choose a stable byte representation
|
||||
## Serializer format contracts
|
||||
|
||||
| Choice | Contract to preserve |
|
||||
| --- | --- |
|
||||
@@ -109,7 +109,7 @@ The serializer can be called from concurrent application, recovery, or maintenan
|
||||
|
||||
Changing a serializer's output changes what WAL recovery, disk reads, and backup records need to decode. Test a value written by the previous application version with the new reader before treating a deployment as compatible.
|
||||
|
||||
## Understand what metadata validates
|
||||
## Metadata validation and compatibility
|
||||
|
||||
The loader checks the persisted key type, value type, comparer type, key-serializer type, and value-serializer type. Metadata also records the configured hasher type and Bloom density, but those are not the same persisted-format checks.
|
||||
|
||||
|
||||
@@ -98,7 +98,7 @@ Returning `false` cancels the new engine write. It cannot undo a change already
|
||||
|
||||
`TryAtomicAddOrUpdate` can retry its adder/updater when a segment freezes or fills. Build replacement values from the input of each invocation, and keep callbacks free of external effects that cannot safely repeat. An in-place increment of a shared object can be applied more than once before one write is accepted.
|
||||
|
||||
## Understand shallow immutability
|
||||
## Shallow immutability
|
||||
|
||||
Records, `readonly` fields, read-only interfaces, and readonly structs do not automatically make their contents immutable:
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ description: Create and verify complete live-backup generations, choose memory c
|
||||
titleId="backups-title"
|
||||
subtitle="Create a complete live-backup generation while the tree remains open, verify a restore, and configure automatic generation and retention policies."/>
|
||||
|
||||
## Choose a backup shape
|
||||
## Live backup, offline copy, and rebuild
|
||||
|
||||
| Approach | What it captures | When it fits |
|
||||
| --- | --- | --- |
|
||||
@@ -81,7 +81,7 @@ Restore into an unused data directory and, if configured separately, an unused W
|
||||
|
||||
Use the local provider, a WAL-enabled target, and compatible [data contracts](recovery.mdx#preserve-the-data-contracts). Restore copies disk files, rebuilds backed-up memory records into a read-only WAL, and creates fresh mutable WAL and metadata files. It does not need the original production WALs.
|
||||
|
||||
## Choose the in-memory view
|
||||
## In-memory capture modes
|
||||
|
||||
Memory records are included by default, including deletion markers needed to hide older records in copied disk files. Their batch is compressed independently of disk files, with Zstd level 0 and 1 MiB blocks by default.
|
||||
|
||||
@@ -138,7 +138,7 @@ await backup.CreateGenerationAsync();
|
||||
Console.WriteLine($"Final completed generation: {backup.CurrentGenerationId}");
|
||||
```
|
||||
|
||||
The final-generation sequence stops automatic triggers and waits for active backup work before making the manual request. See [request and completion behavior](#understand-requests-and-completion) below for the individual methods.
|
||||
The final-generation sequence stops automatic triggers and waits for active backup work before making the manual request. See [request and completion behavior](#backup-requests-and-completion) below for the individual methods.
|
||||
|
||||
### Schedule choices
|
||||
|
||||
@@ -151,7 +151,7 @@ The final-generation sequence stops automatic triggers and waits for active back
|
||||
|
||||
`BackupAfterMerge` defaults to `true`. Successful **normal** merges request a generation after `Start()`; bottom-segment merges do not. A schedule remains useful when writes stay in memory without enough activity to trigger a normal merge.
|
||||
|
||||
## Understand requests and completion
|
||||
## Backup requests and completion
|
||||
|
||||
Each coordinator runs at most one generation at a time. Requests do not form a backlog:
|
||||
|
||||
|
||||
@@ -9,10 +9,10 @@ layout:
|
||||
---
|
||||
|
||||
<div class="zt-docs-overview">
|
||||
<SectionHeader class="zt-guide-intro" eyebrow="DURABILITY / RECOVER AND RESTORE" title="Understand logging, diagnostics, and restore" titleTag="h1" titleId="durability-title" subtitle="Choose a WAL policy, capture storage diagnostics, or restore a backup generation. The recovery guide covers the public APIs available for specialized inspection and recovery tools."/>
|
||||
<SectionHeader class="zt-guide-intro" eyebrow="DURABILITY / RECOVER AND RESTORE" title="Write-ahead logging, recovery, and backups" titleTag="h1" titleId="durability-title" subtitle="Choose a WAL policy, capture storage diagnostics, or restore a backup generation. The recovery guide covers the public APIs available for specialized inspection and recovery tools."/>
|
||||
|
||||
<Grid columns="1" columnsMd="3" class="gap-4 zt-guide-next zt-docs-paths">
|
||||
<DocCard number="01 / LOG RECENT WRITES" title="WAL modes" summary="Choose a logging path, understand compressed tails, and close the tree cleanly. AsyncCompressed is the default." href="/docs/durability/wal-modes/" icon="tabler:shield-check" label="Understand the default WAL"/>
|
||||
<DocCard number="01 / LOG RECENT WRITES" title="WAL modes" summary="Choose a logging path, understand compressed tails, and close the tree cleanly. AsyncCompressed is the default." href="/docs/durability/wal-modes/" icon="tabler:shield-check" label="Default WAL behavior"/>
|
||||
<DocCard number="02 / INSPECT AND RECOVER" title="Recovery" summary="Capture checksum and reader errors, integrate logging, and reuse public WAL and segment APIs in specialized tools." href="/docs/durability/recovery/" icon="tabler:database" label="Inspect WAL diagnostics"/>
|
||||
<DocCard number="03 / RESTORE A GENERATION" title="Backups" summary="Run a complete backup and restore example, then add memory capture policy, scheduling, and retention." href="/docs/durability/backups/" icon="tabler:stack-2" label="Create a verified backup"/>
|
||||
</Grid>
|
||||
|
||||
@@ -5,7 +5,7 @@ description: Understand automatic incomplete-tail recovery, capture WAL corrupti
|
||||
|
||||
<SectionHeader class="zt-guide-intro" eyebrow="DURABILITY / DIAGNOSTICS AND TOOLS" title="Inspect WALs and build recovery tools" titleTag="h1" titleId="recovery-title" subtitle="ZoneTree handles incomplete WAL tails during startup. For checksum and other read errors, logging and public storage APIs support diagnosis and specialized recovery tools."/>
|
||||
|
||||
## Understand what detection provides
|
||||
## WAL corruption diagnostics
|
||||
|
||||
Current-format WAL records carry checksums. A mismatch is reported as a read failure; it does not repair the record or establish which bytes should replace it. Reconstructing records that fail checksum validation requires a specialized tool and a policy for deciding which records can be trusted.
|
||||
|
||||
@@ -114,7 +114,7 @@ Use the original key/value types, serializers, comparer, compatible hasher, and
|
||||
|
||||
See [serializers and comparers](../concepts/serializers-and-comparers.mdx) and [deletion markers and TTL](../concepts/deletion-markers-and-ttl.mdx) for those contracts.
|
||||
|
||||
## Choose a recovery procedure
|
||||
## Inspection and reconstruction workflow
|
||||
|
||||
1. Preserve the affected database and any separately configured WAL location, along with the diagnostic logs. Use a separate working copy for inspection.
|
||||
2. Identify the affected segment, record failures, and file format with the public readers and metadata APIs.
|
||||
|
||||
@@ -46,7 +46,7 @@ For codec defaults and block-size controls, see [compression](../storage/compres
|
||||
|
||||
Existing WAL files keep their detected logging mode and recorded compression format. Changing the factory configures newly created WALs; reopening an existing file does not convert it to another mode. The built-in provider detects the existing mode from its filename, so preserve WAL filenames when moving a database.
|
||||
|
||||
## Understand the compressed tail
|
||||
## Compressed blocks and the tail file
|
||||
|
||||
Compressed WALs have a main file of committed blocks and a companion `.tail` file for the unfinished block. Keep both when making an offline copy.
|
||||
|
||||
|
||||
@@ -132,7 +132,7 @@ Run `dotnet run` from the same project directory. It prints `Persisted 20: Grace
|
||||
<AlertBox
|
||||
class="zt-guide-callout"
|
||||
tone="info"
|
||||
title="Choose durability before relying on crash recovery"
|
||||
title="Persistence and crash recovery"
|
||||
icon="tabler:shield-check"
|
||||
>
|
||||
<p>
|
||||
@@ -143,7 +143,7 @@ Run `dotnet run` from the same project directory. It prints `Persisted 20: Grace
|
||||
</p>
|
||||
</AlertBox>
|
||||
|
||||
## Choose what to learn next
|
||||
## Further reading
|
||||
|
||||
Continue with one of these guides, or [browse the Usage overview](usage/index.mdx) to compare reads, writes, atomic operations, transactions, scans, and maintenance.
|
||||
|
||||
@@ -157,7 +157,7 @@ Continue with one of these guides, or [browse the Usage overview](usage/index.md
|
||||
label="Opening a tree"/>
|
||||
<DocCard
|
||||
number="NEXT / DATA OPERATIONS"
|
||||
title="Choose the right API"
|
||||
title="Read and write APIs"
|
||||
summary="Match simple writes, conditional changes, single-key updates, and transactions to your data rules."
|
||||
href="/docs/usage/reads-and-writes/"
|
||||
icon="tabler:code"
|
||||
|
||||
@@ -92,7 +92,7 @@ if (tree.TryGet(42, out var value))
|
||||
Keep the maintainer alive with the tree to schedule background merges
|
||||
and clean inactive caches.
|
||||
<a href="/docs/usage/maintenance/"
|
||||
>Understand maintenance <Icon name="tabler:arrow-right"/></a>
|
||||
>Background maintenance <Icon name="tabler:arrow-right"/></a>
|
||||
</p>
|
||||
</Panel>
|
||||
</Grid>
|
||||
@@ -117,7 +117,7 @@ if (tree.TryGet(42, out var value))
|
||||
summary="See how mutable memory, read-only segments, compaction, and disk parts work together. Use that model to understand writes, reads, and maintenance."
|
||||
href="/docs/concepts/lsm-tree/"
|
||||
icon="tabler:stack-2"
|
||||
label="Understand the architecture"/>
|
||||
label="LSM-tree architecture"/>
|
||||
<DocCard
|
||||
number="03 / PRODUCTION"
|
||||
title="Make the operational choices"
|
||||
@@ -176,7 +176,7 @@ if (tree.TryGet(42, out var value))
|
||||
<ul class="list-none p-0 m-0">
|
||||
<li>
|
||||
<a href="/docs/concepts/key-ordering/"
|
||||
>Choose key order and prefix layouts</a>
|
||||
>Key ordering and prefix layouts</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="/docs/usage/iteration-and-range-scans/"
|
||||
@@ -206,7 +206,7 @@ if (tree.TryGet(42, out var value))
|
||||
<ul class="list-none p-0 m-0">
|
||||
<li>
|
||||
<a href="/docs/durability/wal-modes/"
|
||||
>Choose a write-ahead logging mode</a>
|
||||
>Write-ahead logging modes</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="/docs/durability/recovery/"
|
||||
@@ -228,12 +228,12 @@ if (tree.TryGet(42, out var value))
|
||||
aria-labelledby="docs-performance-title"
|
||||
>
|
||||
<Icon name="tabler:database" class="zt-docs-topic-icon"/>
|
||||
<h3 id="docs-performance-title">Understand storage costs</h3>
|
||||
<h3 id="docs-performance-title">Storage costs</h3>
|
||||
<p>Work out what drives disk reads, rewrite work, and memory use.</p>
|
||||
<ul class="list-none p-0 m-0">
|
||||
<li>
|
||||
<a href="/docs/storage/disk-segments/"
|
||||
>Understand disk-segment layouts</a>
|
||||
>Disk-segment layouts</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="/docs/tuning/write-amplification/"
|
||||
@@ -247,7 +247,7 @@ if (tree.TryGet(42, out var value))
|
||||
<a href="/docs/storage/memory-usage/">Account for memory usage</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="/docs/storage/compression/">Choose compression settings</a>
|
||||
<a href="/docs/storage/compression/">Compression settings</a>
|
||||
</li>
|
||||
</ul>
|
||||
</Panel>
|
||||
|
||||
@@ -25,7 +25,7 @@ Both paths expose a compression method, method-specific level, and block size, b
|
||||
|
||||
Changing compression options does not rewrite existing files. A merge may write new files with the new profile while reusing eligible parts with the earlier profile. Multipart descriptors have their own metadata encoding; disk data-block settings do not configure that encoding.
|
||||
|
||||
<AlertBox class="zt-guide-callout" tone="info" title="Choose recovery guarantees through the WAL mode" icon="tabler:shield-check">
|
||||
<AlertBox class="zt-guide-callout" tone="info" title="WAL mode and durability" icon="tabler:shield-check">
|
||||
<p>A compression method or level does not define when a write is durable. <code>AsyncCompressed</code>, <code>SyncCompressed</code>, plain <code>Sync</code>, and <code>None</code> have different logging and failure behavior. Choose the mode with the <a href="/docs/durability/wal-modes/">WAL modes guide</a>, then tune compression within that choice.</p>
|
||||
</AlertBox>
|
||||
|
||||
@@ -83,7 +83,7 @@ Console.WriteLine($"Read {value.Length} characters after merging.");
|
||||
|
||||
The two methods are intentionally independent; this is an example profile, not a universal performance recommendation. `SetDiskSegmentCompressionBlockSize()` is a factory shorthand for the disk block-size option.
|
||||
|
||||
## Understand the unit of a disk read
|
||||
## Block reads and caching
|
||||
|
||||
Disk files contain individually addressable blocks. A cold read locates and reads the relevant compressed block, decompresses it, and serves the requested bytes. Subsequent reads can reuse the decompressed block while it remains cached.
|
||||
|
||||
|
||||
@@ -40,7 +40,7 @@ An illustrative multipart segment with two variable-value parts looks like this:
|
||||
|
||||
These are segment files. Metadata and any record WALs belong to the wider database layout described in the [Storage overview](index.mdx).
|
||||
|
||||
## Understand the four record layouts
|
||||
## Fixed-size and variable-size record layouts
|
||||
|
||||
Each physical single-part segment selects a layout from the runtime key and value types. Types with no managed references use the fixed-size path; strings, arrays, and types containing references use the variable-size path.
|
||||
|
||||
@@ -113,7 +113,7 @@ File IDs depend on the tree's history. An empty current disk slot may enumerate
|
||||
<p><code>GetFiles()</code> describes a segment; it does not freeze the tree or acquire a backup lease. During concurrent maintenance, use the live-backup API to coordinate segment lifetimes and capture in-memory records and restore metadata.</p>
|
||||
</AlertBox>
|
||||
|
||||
## Understand when files disappear
|
||||
## File lifetime after a merge
|
||||
|
||||
Merges publish replacement segments rather than rewriting the record data of an active segment. Once a segment is retired, its files can be reclaimed when no reader or backup still leases them. Sparse-index files are auxiliary persisted indexes, separate from that immutable record data.
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ description: Choose local or RAM-backed files, reopen an in-memory tree correctl
|
||||
<SectionHeader
|
||||
class="zt-guide-intro"
|
||||
eyebrow="STORAGE / BACKING STORE"
|
||||
title="Choose where ZoneTree's files live"
|
||||
title="Local, in-memory, and custom file providers"
|
||||
titleTag="h1"
|
||||
titleId="file-stream-providers-title"
|
||||
subtitle="The storage engine works through file and stream abstractions. Keep the same ordered API while choosing local persistence, RAM-backed files, or an application-specific provider."/>
|
||||
|
||||
@@ -9,7 +9,7 @@ layout:
|
||||
---
|
||||
|
||||
<div class="zt-docs-overview">
|
||||
<SectionHeader class="zt-guide-intro" eyebrow="STORAGE / FROM RECORDS TO RESOURCES" title="Understand where your data lives and what it costs" titleTag="h1" titleId="storage-title" subtitle="Connect the logical tree to its physical files, backing store, and memory footprint. Then choose compression and cache settings with a clear view of their scope."/>
|
||||
<SectionHeader class="zt-guide-intro" eyebrow="STORAGE / FROM RECORDS TO RESOURCES" title="Physical storage and resource use" titleTag="h1" titleId="storage-title" subtitle="Connect the logical tree to its physical files, backing store, and memory footprint. Then choose compression and cache settings with a clear view of their scope."/>
|
||||
|
||||
<Grid columns="1" columnsMd="3" class="gap-3 zt-guide-outcomes">
|
||||
<Panel class="p-0" bodyClass="p-4"><strong>Representation</strong><p>Sorted records become block-addressable files, indexes, and reusable parts.</p></Panel>
|
||||
|
||||
@@ -17,7 +17,7 @@ All three are record counts, not byte budgets. The disk threshold is checked aft
|
||||
|
||||
For terminology and file ownership, see [segments and parts](../concepts/segments.mdx) and [disk segment files](../storage/disk-segments.mdx).
|
||||
|
||||
## Choose a segment mode and target range
|
||||
## Segment modes and part targets
|
||||
|
||||
`MultiPartDiskSegment` is the default. It represents one logical segment with ordered physical parts and allows eligible parts to be reused. `SingleDiskSegment` uses a single-part layout; it still can have separate data, header, and sparse-index files.
|
||||
|
||||
@@ -79,7 +79,7 @@ Console.WriteLine($"Physical files: {disk.GetFiles().Length}");
|
||||
|
||||
The small dataset produces a final part below the configured minimum, which is a normal outcome. With concurrent writers, coordinate the observation before comparing file sets or merge outputs.
|
||||
|
||||
## Choose block size and sparse density for reads
|
||||
## Block size and sparse-index density
|
||||
|
||||
Disk compression defaults to Zstd level 0 with 4 MiB blocks. A smaller block can reduce unrelated bytes decompressed for isolated reads, while a larger block can improve locality and compression. A value larger than one block still needs all of its payload bytes. Existing files retain their recorded block size and method; a configuration change alone does not rewrite them. See [compression](../storage/compression.mdx) for a controlled comparison.
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ layout:
|
||||
<Grid columns="1" columnsMd="2" class="gap-4 zt-guide-next zt-docs-paths">
|
||||
<DocCard number="01 / READ WORK" title="Read-path caching" summary="Separate block retention, deserialized-record caches, sequential lookup hints, and scan prefetch." href="/docs/tuning/read-path-caching/" icon="tabler:settings" label="Match caches to reads"/>
|
||||
<DocCard number="02 / MERGE WORK" title="Write amplification" summary="Understand reuse eligibility and observe unchanged data files across a localized update." href="/docs/tuning/write-amplification/" icon="tabler:git-branch" label="Measure rewrite work"/>
|
||||
<DocCard number="03 / FILE SHAPE" title="Disk segment tuning" summary="Distinguish writable capacity, disk-layer rollover, part targets, block size, and sparse density." href="/docs/tuning/disk-segments/" icon="tabler:database" label="Choose the right boundary"/>
|
||||
<DocCard number="03 / FILE SHAPE" title="Disk segment tuning" summary="Distinguish writable capacity, disk-layer rollover, part targets, block size, and sparse density." href="/docs/tuning/disk-segments/" icon="tabler:database" label="Segment and part sizing"/>
|
||||
<DocCard number="04 / PAYLOAD COST" title="Large values" summary="Translate record counts into bytes, evaluate value ownership, and compare direct payloads with references." href="/docs/tuning/large-values/" icon="tabler:stack-2" label="Budget payload work"/>
|
||||
</Grid>
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ description: Translate record limits into payload costs, choose direct values or
|
||||
|
||||
<SectionHeader class="zt-guide-intro" eyebrow="TUNING / PAYLOAD SIZE" title="Budget large values in bytes and work" titleTag="h1" titleId="large-values-title" subtitle="Compare direct payloads, references, and split records. Translate their record counts into retained memory, serialized bytes, and merge I/O."/>
|
||||
|
||||
## Choose what belongs in the value
|
||||
## Payload storage models
|
||||
|
||||
| Model | Useful when | Responsibility to account for |
|
||||
| --- | --- | --- |
|
||||
|
||||
@@ -11,7 +11,7 @@ Write amplification compares internal bytes written with the application's logic
|
||||
|
||||
Record overwrite distribution as well as volume. Ten thousand updates concentrated in one range can produce different merge work from the same count spread across the whole keyspace.
|
||||
|
||||
## Understand when a part can be reused
|
||||
## Part reuse conditions
|
||||
|
||||
<LsmPartReuse/>
|
||||
|
||||
@@ -65,7 +65,7 @@ Console.WriteLine($"New data files: {after.Except(before).Count()}");
|
||||
|
||||
Application writes are coordinated in this example, and both merges finish before inspection. The file counts vary with randomized boundaries. Shared filenames demonstrate reuse; use I/O measurements for the actual bytes written. The part targets here make the mechanism observable, rather than prescribe a production profile.
|
||||
|
||||
## Choose the lever that matches the pressure
|
||||
## Rewrite costs and tradeoffs
|
||||
|
||||
| Observation | Experiment | Tradeoff to watch |
|
||||
| --- | --- | --- |
|
||||
|
||||
@@ -53,7 +53,7 @@ tree.TryAtomicAddOrUpdate(
|
||||
|
||||
Run it again against the same directory to increment the persisted counter. The callback reports `Added` on initialization and `Updated` on a later increment.
|
||||
|
||||
## Understand the coordination boundary
|
||||
## Coordination across storage layers
|
||||
|
||||
The newest value may be in mutable memory, a read-only segment, or a disk layer. Atomic methods synchronize this sequence across those layers:
|
||||
|
||||
|
||||
@@ -36,7 +36,7 @@ if (tree.TryDelete(7, out var deletedAt))
|
||||
|
||||
For setup and resource lifetime, see [opening a tree](opening-a-tree.mdx).
|
||||
|
||||
## Choose an operation
|
||||
## Read and write APIs
|
||||
|
||||
| Your data rule | API | Important detail |
|
||||
| --- | --- | --- |
|
||||
@@ -124,7 +124,7 @@ Use [transactions](transactions.mdx) when the invariant spans several keys in on
|
||||
Different key groups can use different write disciplines. For example, profile replacement can use `Upsert` while a separate counter uses atomic methods. Keep the discipline consistent for every writer that can change the same invariant.
|
||||
|
||||
<Panel class="p-0 zt-guide-next-panel" bodyClass="p-5">
|
||||
<strong>Choose the next operation</strong>
|
||||
<strong>Range scans and coordinated updates</strong>
|
||||
<p>
|
||||
Read a range, update one key based on its current value, or coordinate a
|
||||
group of keys.
|
||||
|
||||
@@ -58,7 +58,7 @@ catch (TransactionAbortedException)
|
||||
|
||||
`OpenOrCreateTransactional()` returns the transactional API. It differs from the ordinary tree: reads and writes in a transaction take a transaction ID, and success depends on the commit outcome.
|
||||
|
||||
## Choose the smallest coordination scope
|
||||
## Single-key and multi-key coordination
|
||||
|
||||
| Data rule | API |
|
||||
| --- | --- |
|
||||
@@ -68,7 +68,7 @@ catch (TransactionAbortedException)
|
||||
|
||||
Concurrent application threads alone do not require transactions. Use them for a multi-key invariant, and make the reads that decide the writes part of that transaction.
|
||||
|
||||
## Understand the transaction lifecycle
|
||||
## Transaction lifecycle
|
||||
|
||||
1. `BeginTransaction()` allocates a transaction ID.
|
||||
2. Reads and writes use that ID.
|
||||
|
||||
Reference in New Issue
Block a user