Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Ledvar Modeling Guide

  • Companion to: the Ledvar Protocol, MAJOR 0 (SPEC.md)
  • Status: Non-normative. Guidance, not rules.

Who this is for

If you are writing a collector — anything that observes some state and produces a Snapshot — this guide is for you. The protocol (SPEC.md) guarantees one thing: the same canonical input always produces the same hash. It does not tell you how to turn a real firewall, VM, or transaction into nodes. That modeling is yours, and a handful of choices decide whether your drift is clean and honest or noisy and misleading.

None of this is required for conformance — a collector that ignores every word here is still valid Ledvar. But two collectors that disagree on these choices will produce different hashes for what a human would call “the same state.” Following the same conventions is what makes results comparable across tools and over time.

1. Choosing the identity (path) is the most important decision

A node’s path is its identity: a comparison matches old against new by identity_key = hash(path). So the path must be stable (it does not change when irrelevant things change) and unique (two different things never share it).

A bad identity produces churn — endless false Added/Removed pairs:

  • a process keyed by PID (the PID changes on every restart);
  • a firewall rule keyed by its line number (inserting one rule renumbers all the rest).

A good identity is a stable natural key, or a fingerprint of the thing’s defining traits:

  • a user keyed by name, not by an internal row id;
  • a firewall rule keyed by a hash of its match+action, with the line number kept as a content attribute — so a re-order is a single Modified, not a churn of Removed+Added.

Rule of thumb: “if this thing is unchanged but the world around it shifts, does its path stay the same?” If not, pick a different path.

2. content is an unordered, de-duplicated set — encode order and duplicates explicitly

Each attribute maps to a set of strings: the protocol sorts it and drops duplicates before hashing. Perfect when order doesn’t matter (a user’s grants, a group’s members). Wrong when it does.

If order matters (a firewall chain evaluated top-to-bottom; Ethereum log topics, where position is the meaning), encode the position into the value:

"topics": ["0:0xddf2…", "1:0x…sender", "2:0x…recipient"]

or promote each element to its own child node whose path carries the index. Without this, ["a","b","c"] and ["c","b","a"] hash identically and a reorder becomes invisible.

If duplicates matter (a true multiset — two identical line items that really are two), the set collapses them. Tag each with an instance id: ["item#1:widget", "item#2:widget"].

3. Structured and nested data — a subtree, or an encoded string

content values are flat sets of strings, not nested objects. Real things are often nested (a security group holding a list of rule objects). Two ways to model it:

  • Promote to child nodes when you want field-level drift. Each rule becomes its own node: path = [..., "sg-123", "ingress", "tcp-443"], content = {proto, from_port, to_port, cidr}. A change to one field is a Modified on that one rule.
  • Flatten into a string when coarse drift is enough: "ingress": ["tcp:443:443:0.0.0.0/0", "tcp:22:22:10.0.0.0/8"]. Any change re-hashes the whole element; you lose which field changed, but it is compact.

Never put a blob in content. Store its digest: "sha256": ["b1946ac…"]. The protocol hashes your hash.

4. Relationships: content for drift, refs for annotation

A link between nodes can live in two places — the same labels-vs-content question:

  • If a change to the relationship should be drift (a volume detached from a VM, a user removed from a role), put the target ids in content as a set: "attached_volumes": ["vol-1","vol-2"]. Detaching vol-2 changes the content hash.
  • If the relationship is pure annotation (a “this configures that” edge drawn for a diagram), use refs. Refs are never hashed, so adding or removing one is never drift.

5. Normalize your values — the protocol compares them literally

Because everything is a string, the protocol compares values byte for byte. "0.5" and "0.50" differ. "ACCEPT" and "accept" differ. "0600" and "384" (the same mode in octal vs decimal) differ.

So value normalization is your job. Pick one canonical representation per attribute — lowercase hex, octal modes with the leading zero, RFC 3339 timestamps in UTC, a fixed decimal precision — and emit it the same way every time. If two versions of your collector format the same value differently, they report drift where nothing changed. This is the most common source of false positives, and the protocol cannot catch it for you.

Unicode normalization is part of this — and it is your job, not the protocol’s. The protocol does no normalization (SPEC.md §6): “café” written as caf+U+00E9 (NFC) and as caf+e+U+0301 (NFD) are different byte sequences and hash differently, even though they look the same. macOS hands filenames back as NFD, Linux as NFC — so a cross-platform filesystem collector that does nothing here reports phantom drift for the same file. Normalize deliberately, per source: for human-facing text values, normalize to NFC; but for a value whose whole point is to match the bytes on disk — a filename — do not normalize, or your identity stops matching what is really there. Decide once per attribute and emit it the same way every time.

Beware implicitly-typed formats. Everything is a string (SPEC.md §4.4), but some serializations don’t preserve that on their own: in YAML (and TOML, and some JSON tooling) an unquoted scalar is auto-typed — 149.90 parses back as the float 149.9, 0600 may become the integer 384, yes/no become booleans. Two consumers of the same careless document then disagree on the string, and hash differently. So when you emit or read these formats, keep every value a quoted string ("149.90", "0600"), and reduce anything typed back to its canonical string form before hashing. JSON with string values and Protocol Buffers string fields don’t have this trap; YAML and TOML do.

Range-check timestamp on the raw token in floating-point languages. In JavaScript, Lua, and other double-precision languages, the default JSON parser reads a large integer as an IEEE-754 float — so 9223372036854775807 (int64 max, legal) and 9223372036854775808 (out of range, ill-formed per SPEC.md §9) both round to the same float and are indistinguishable after parsing. A validator in such a language must range-check the raw token (or parse it as a big integer), not the parsed number. This is a timestamp-only concern — it is metadata, never hashed, and real timestamps sit ten orders of magnitude below the boundary, so the practical risk is nil; the note just spares a reimplementer the debugging.

6. Snapshot scope — stable state, and the whole of it

  • Model state that is meant to be stable: configuration, posture, structure, declared limits. Do not model fast-moving metrics (live CPU %, current connection count) — they change every snapshot and bury real drift in noise. Model the configured memory limit, not the current usage.
  • A snapshot must be complete for its declared scope. A comparison treats any node present last time and absent now as Removed. If a lens fails halfway and emits fewer nodes, every missing node looks deleted — a flood of false removals. Either emit the complete state of your scope, or narrow the scope (a subtree) so that “complete” is something you can guarantee.

7. Identity and origin are separate

identity_key is hash(path) — it does not include the snapshot’s origin_id. So the file ["etc","ssh","sshd_config"] has the same identity on every host. That is a feature: it lets you compare the same file across a fleet. But it means:

  • to diff a single host’s timeline, the store keys by (origin_id, identity_key);
  • if you want nodes from different hosts to be distinct entities, put the host in the path: ["host:web-01","etc","ssh","sshd_config"].

Decide on purpose whether a node’s identity is global or per-origin, and encode that choice in the path. origin_id is metadata; it never enters a hash.

8. Numbers and binary are strings — and that’s a strength

There is no number type, and you don’t want one. Strings give you arbitrary precision for free: a Bitcoin amount in satoshis, an Ethereum uint256 balance (far beyond a 64-bit integer), a nanosecond timestamp — all exact, no overflow, no float rounding. Binary becomes a string too: hex or base64. The collector picks the representation (see §5); the protocol just hashes the bytes.

Non-Unicode bytes must be encoded, too. The protocol requires every hashed string to be valid Unicode (SPEC.md §9), but the real world hands you bytes that are not: a Linux filename can be an arbitrary byte sequence that is not valid UTF-8; a string from a UTF-16 language can carry an unpaired surrogate. You cannot put those into content raw. Encode them — hex or base64 — the same way you would any other binary value, and (if useful) keep a best-effort human-readable form in a label. Whether to encode such a node or to skip it and log the error is your decision as the collector — the protocol only insists that whatever it hashes is valid Unicode; it does not tell you which of the two to do.

A note on domain conventions

This guide is general. The specific question — exactly how to encode an iptables rule, an EC2 instance, or an Ethereum transaction so that everyone’s collectors agree — is a per-domain convention, and those are best written as their own companion documents over time (the way DIFF.md is a companion to the core). Keeping them out of the core protocol is deliberate: the protocol stays small, and each domain settles its conventions at its own pace.