# Snapshot row commitment v1 **Identifier:** `fonteum-row-merkle-jcs-v1` **Hash:** SHA-256 **Cutover:** `2026-08-01T00:00:00Z` This document freezes the byte-level construction used for new Fonteum source snapshot attestations. It is written so an independent party can reproduce a root from either the exact upstream CSV download or a canonical NDJSON snapshot without Fonteum code or infrastructure. ## Historical boundary No historical hash is rewritten at this cutover. Attestations labelled `count-observation-v1` predate the row-commitment cutover. Their hash was computed from a source identifier, observation date, and row count. Their historical `content_size_bytes` value is also a row count, not a byte measurement. Those attestations prove only that Fonteum recorded a particular count on a particular day. They do **not** commit to any value in any source row. They remain unchanged as historical evidence of exactly what the old system produced. Attestations labelled `raw-file-sha256-v1` commit to the bytes of a retained source file. Attestations at or after the cutover may use the row commitment defined below only when the complete normalized rowset has been sealed and stored. The schema migration `20260713000000_integrity_rowsets_chain_anchor_recovery.sql` remains unapplied in this branch. Pause integrity writers, deploy the #1300-compatible reader first, confirm the withdrawn legacy fields are absent, then have the operator apply the reviewed migration. Publish and check the independent witness-role directory key with owner/operator SQL access, run the smoke/audit checks, and only then resume writers. Its `BEFORE INSERT` guard handles any in-flight legacy writer that omits `hash_version` by recomputing the exact frozen count preimage `SHA256(source_id || "||" || snapshot_date || "||" || record_count)` rather than inferring semantics from a date or mutable methodology label. The migration aborts if any `count-observation-v1` row already exists at or after the proposed boundary. Complete this app-first cutover before `2026-08-01T00:00:00Z`; never roll back to a reader older than #1300 after migration apply. If that window cannot be met, choose a later boundary and change every forward reference consistently before proceeding. Do not backdate, relabel, or rewrite historical evidence. ## Exact upstream CSV normalization For a CSV source, the downloaded file itself is accepted as input. It is decoded as UTF-8 in fatal mode: malformed byte sequences are rejected, never silently replaced with U+FFFD. A streaming RFC 4180 parser preserves quoted commas, escaped quotes, and embedded line endings. CRLF, LF, and bare CR are accepted as record separators outside quoted fields. The first record is the header. It is transformed exactly as follows before JCS serialization: 1. Remove one leading U+FEFF only from the first header field. Do not trim or otherwise alter any header. 2. Give each column a six-digit, zero-padded positional prefix and a colon. Header fields `NAME,NAME` therefore become object keys `000000:NAME` and `000001:NAME`; duplicate header text cannot discard a value. 3. Preserve every parsed field as an exact string. Do not trim, coerce a number, substitute null, or normalize Unicode. 4. If a record omits trailing fields, use the empty string for each missing value. Reject a record with more fields than the header. 5. Ignore a fully empty data record. Retain all other records, including a record containing several empty fields. Thus a header `A,B,C` and data record `x,y` produce: ```json { "000000:A": "x", "000001:B": "y", "000002:C": "" } ``` This is the same positional normalization used by the production CSV capture path. It defines the normalized rows committed by the root; the raw CSV byte length is not `content_size_bytes`. ## Accepted normalized row data After CSV normalization—or directly for a canonical NDJSON export—every row must be I-JSON-compatible: - `null`, booleans, strings, finite numbers, arrays, and objects are accepted. - Object member names must be unique. Escaped-equivalent names such as `"id"` and `"\u0069d"` are duplicates. - Integer-valued JavaScript numbers must be within the safe-integer range. Database `bigint`, `numeric`, and `decimal` values must be exported as exact decimal strings. - `undefined`, bigint values, non-finite numbers, unsafe integer numbers, functions, symbols, sparse arrays, cycles, accessor properties, and non-plain JavaScript objects are rejected. - Strings and object names must not contain lone UTF-16 surrogates. Unicode is not normalized. For example, the NFC string `"é"` and the NFD string `"e\u0301"` are different source values and produce different leaf digests. ## Canonical row bytes Each row is serialized with the JSON Canonicalization Scheme in RFC 8785: 1. Object member names are sorted recursively by their UTF-16 code units. 2. Numbers use ECMAScript JSON number serialization. Thus negative zero is `0`, `0.000001` remains fixed notation, and `1e-7` uses exponent notation. 3. Strings use ECMAScript JSON escaping. 4. The canonical text is encoded as UTF-8 without Unicode normalization. The canonical snapshot export is the concatenation of: ```text UTF8(JCS(row_1)) || 0x0A || UTF8(JCS(row_2)) || 0x0A || ... || UTF8(JCS(row_n)) || 0x0A ``` The last row always has an LF. `content_size_bytes` is the exact length of that canonical export. `row_count` is recorded separately. ## Merkle construction This construction does not use RFC 6962 leaf or parent prefixes. 1. For every row, compute `SHA256(UTF8(JCS(row)))`. The export LF is not part of the leaf preimage. 2. Sort all 32-byte leaf digests lexicographically as raw bytes. 3. Retain duplicate digests. Row multiplicity is part of the commitment. 4. Pair adjacent nodes and compute `SHA256(left_raw_32_bytes || right_raw_32_bytes)`. 5. Promote an unpaired node unchanged to the next level. Do not duplicate it. 6. Repeat steps 4–5 until one 32-byte root remains. A single leaf is its own root. An empty rowset is rejected and receives no attestation. The root is stored as lowercase hexadecimal in `snapshot_attestations.content_hash`, with `hash_algorithm = 'SHA-256'` and `hash_version = 'fonteum-row-merkle-jcs-v1'`. Equivalent pseudocode: ```text level = raw_byte_sort([SHA256(UTF8(JCS(row))) for row in rows]) require level.length > 0 while level.length > 1: next = [] for i in 0, 2, 4, ...: if i + 1 < level.length: next.push(SHA256(level[i] || level[i + 1])) else: next.push(level[i]) level = next root = level[0] ``` ## Bounded implementation The production implementation and the reference verifier do not retain all leaf digests in memory. They: 1. stream-parse CSV or NDJSON and canonicalize one normalized row at a time; 2. sort a configured number of 32-byte digests in memory; 3. write each sorted run as consecutive raw 32-byte values; 4. reduce excessive run counts with bounded k-way merge passes, deleting each consumed input run after its merged output is closed; and 5. feed the final sorted stream into an O(log n) odd-promotion accumulator. Both the number of in-memory digests and open run files are bounded. Temporary binary runs are deleted on success or failure. The canonical rows themselves are banked separately as the immutable snapshot content; the scratch digest runs are not the snapshot archive. ## Standalone reproduction The reference implementation uses only Node.js built-ins and does not import the application implementation. Download the versioned public copy directly; no repository checkout or package install is required: ```bash curl -fsSLo verify-snapshot-merkle.mjs \ https://fonteum.com/methodology/snapshot-row-merkle-v1/reference.mjs node verify-snapshot-merkle.mjs UPDATED.csv node verify-snapshot-merkle.mjs snapshot.ndjson ``` The same frozen script is tracked in the repository, so a checked-out copy may instead run: ```bash node scripts/integrity/verify-snapshot-merkle.mjs UPDATED.csv node scripts/integrity/verify-snapshot-merkle.mjs snapshot.ndjson ``` To compare a download with published metadata: ```bash node scripts/integrity/verify-snapshot-merkle.mjs UPDATED.csv \ --expected-root 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef \ --expected-row-count 68055 \ --expected-content-size-bytes 12345678 ``` The `.csv`, `.ndjson`, and `.jsonl` extensions select the parser. For a file without one of those extensions, pass `--format csv` or `--format ndjson`. The process exits nonzero on a mismatch, malformed UTF-8, malformed RFC 4180, surplus CSV field, malformed NDJSON, duplicate JSON member name, invalid I-JSON value, or empty capture. `--json` emits machine-readable output. `--max-digests-in-memory` and `--max-open-runs` can lower the scratch resource bounds without changing the root. ## Input retention and five-year reproduction boundary Algorithm stability and input availability are separate obligations. This specification freezes the byte construction so the same captured CSV or canonical NDJSON produces the same root in five years. A mutable upstream URL does not by itself preserve the historical input: an agency may replace a file at the same address. For a historical re-check, use the exact downloaded CSV or canonical NDJSON that produced the attestation. For a row-Merkle attestation, `/verify/` resolves the observation to its original producer snapshot and exposes that producer's immutable `cache_url` when retained. That captured file is the preferred public input. The immutable sealed row bank is the authoritative service-side source for re-derivation, but it is service-only and is not represented as a public download. If neither retained input is available, the verifier must already possess the original download; re-downloading a mutable `source_archive_url` later is insufficient because it may produce a newer file and a different root. This availability limitation does not change the root construction. The binary digest-run files described above are scratch space only. They are deleted after sorting/reduction and cannot reproduce a snapshot. Reproduction depends on the captured CSV or canonical NDJSON, not the scratch runs. ## Initial cutover scope The daily row-level cutover enrolls only the three source namespaces whose producers bank every source row and column before sealing: - `oig_leie` from producer `oig-leie`; - `pecos_ppef` from producer `cms-pecos`; and - `hrsa_shortage_areas` from producer `hrsa-hpsa`. There is no count-only fallback after the cutover. A missing sealed capture is an error. Other ingestion paths continue to use `raw-file-sha256-v1` when they attest retained source-file bytes; they do not become row-Merkle attestations until an exact-row producer and immutable bank are wired. ## Exact upstream CSV mutation vector The repository fixture `src/lib/integrity/__fixtures__/oig-leie-upstream.csv` contains the header and first five data records from the HHS-OIG Updated LEIE Database downloaded from `https://oig.hhs.gov/exclusions/downloadables/UPDATED.csv` on 2026-07-12. The feed was listed by HHS-OIG as updated on 2026-07-10. Positional CSV normalization produces five rows and 2,387 canonical NDJSON bytes. Original fixture root: ```text f10eb3d2cdc5094c1cf629bf7f1956b187034ff7f0cc442eb44ad5a0262fc093 ``` Changing the single ASCII byte `B` to `C` in `BROOKLYN` produces: ```text b5325fe6eb8bb9a4b5c34b4ebe72d27bf04d89380f5fb466c3f9665066492f8b ``` The raw fixture and mutated fixture differ by exactly one byte. Their roots differ even though both normalize to five rows and 2,387 canonical bytes.