Write-Once (WORM) Storage for Reconciliation Audit Events #

A hash chain proves an audit event was altered; write-once storage stops the alteration from ever landing. When an examiner asks whether your reconciliation break log could have been rewritten after an incident, "we would have detected it" is weaker than "the storage physically refused the write for six years." This page sits inside the immutable audit trails reference within the broader Reg E, NACHA and audit controls framework, and shows how to put a reconciliation audit event onto write-once, read-many (WORM) media — S3 Object Lock in compliance mode — with a retention-until date, an idempotent key, and legal-hold support, then confirm the object cannot be overwritten or deleted.

WORM is the prevention half of a two-part control; the hash-chained audit log is the detection half. Detection without prevention leaves a window in which the record can be silently replaced; prevention without detection gives you no cryptographic proof of what the record contained. Regulators expect both, and 17 CFR 240.17a-4 has required non-rewriteable, non-erasable media for covered records since long before object storage existed.

Concept Spec #

S3 Object Lock enforces a retention period per object version and offers two modes with very different threat models:

• Governance mode blocks overwrite and delete for ordinary principals but lets a holder of s3:BypassGovernanceRetention remove the lock. It protects against accident, not against a privileged insider — so it is not sufficient for a regulatory WORM control.
• Compliance mode blocks overwrite and delete for everyone, including the root account, until the retention-until timestamp passes. No principal can shorten or bypass it. This is the mode a 17a-4-grade audit trail requires.

An object under compliance-mode retention with retain-until = $t_r$ cannot be deleted or overwritten while wall-clock time $t<t_r$; the guarantee is monotonic — retention can be extended but never reduced. Legal hold is an orthogonal, open-ended flag: an object under legal hold cannot be deleted regardless of $t_r$, and the hold persists until explicitly released, which is how you freeze records beyond their normal retention when litigation or an investigation demands it. The audit-defensible state for a sensitive record is compliance-mode retention and, when required, a legal hold — the two are additive, and an object is deletable only when neither applies.

Full Annotated Python Implementation #

The function below writes one audit event to a WORM bucket and returns the version it created. The bucket must have Object Lock enabled at creation (it cannot be turned on later), and the key is derived deterministically from the event's chain position so a retried write targets the same logical slot rather than silently forking history.

from __future__ import annotations

import hashlib
import json
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from typing import Any

import boto3
from botocore.exceptions import ClientError


@dataclass(frozen=True)
class WormPutResult:
    key: str
    version_id: str
    retain_until: datetime
    object_hash: str


def canonical_bytes(event: dict[str, Any]) -> bytes:
    """Deterministic bytes for both the object body and its content hash.
    Sorted keys, compact separators, fixed UTF-8; no floats permitted."""
    return json.dumps(
        event, sort_keys=True, separators=(",", ":"), ensure_ascii=False
    ).encode("utf-8")


def put_audit_event(
    client: Any,                     # a boto3 S3 client
    bucket: str,                     # a bucket created WITH Object Lock enabled
    event: dict[str, Any],
    retention_days: int,             # a regulatory input, not an ops convenience
    legal_hold: bool = False,
) -> WormPutResult:
    """Write one reconciliation audit event to WORM storage under
    compliance-mode retention.

    The key is content-addressed by the event's sequence and hash, making the
    write idempotent: replaying the same event re-derives the same key and the
    same immutable body, so a retry never forks or overwrites history.
    """
    body = canonical_bytes(event)
    object_hash = hashlib.sha256(body).hexdigest()

    # Deterministic, idempotent key: chain position + content fingerprint.
    seq = int(event["seq"])
    key = f"audit/{seq:012d}-{object_hash[:16]}.json"

    retain_until = datetime.now(timezone.utc) + timedelta(days=retention_days)

    put_kwargs: dict[str, Any] = {
        "Bucket": bucket,
        "Key": key,
        "Body": body,
        "ContentType": "application/json",
        # Compliance mode: NO principal, including root, can shorten or remove
        # this retention before retain_until. Governance mode would be bypassable.
        "ObjectLockMode": "COMPLIANCE",
        "ObjectLockRetainUntilDate": retain_until,
        # Integrity: S3 verifies the body matches this digest on write.
        "ChecksumSHA256": _b64_sha256(body),
    }
    if legal_hold:
        put_kwargs["ObjectLockLegalHoldStatus"] = "ON"

    resp = client.put_object(**put_kwargs)
    return WormPutResult(
        key=key,
        version_id=resp["VersionId"],
        retain_until=retain_until,
        object_hash=object_hash,
    )


def _b64_sha256(body: bytes) -> str:
    import base64
    return base64.b64encode(hashlib.sha256(body).digest()).decode("ascii")


def assert_immutable(client: Any, bucket: str, key: str, version_id: str) -> None:
    """Prove the object cannot be overwritten or deleted under retention.

    A compliance-mode object should reject a versioned delete with AccessDenied.
    We treat a *successful* delete as a control failure, not a success.
    """
    try:
        client.delete_object(Bucket=bucket, Key=key, VersionId=version_id)
    except ClientError as exc:
        if exc.response["Error"]["Code"] in ("AccessDenied", "InvalidRequest"):
            return  # expected: retention refused the delete
        raise
    raise AssertionError(
        f"WORM CONTROL FAILURE: {key} v{version_id} was deletable under retention"
    )

Calibration & Configuration #

The retention period and the mode are compliance inputs, not tuning knobs — get them from the rule that governs the record, not from storage cost.

• Retention period per regulation. 17 CFR 240.17a-4 generally requires many broker-dealer records to be preserved for six years (the first two readily accessible), so retention_days for those records is on the order of 2,190-plus days. Reg E dispute documentation must outlast its own resolution window and any downstream litigation, so audit events tied to a consumer dispute frequently inherit a longer hold than a routine match event. Set retention_days from a per-record-type policy table, and when in doubt, retain longer — you can extend a compliance-mode retention but never shorten it.
• Governance vs compliance mode. Use compliance mode for anything an examiner may rely on; its whole point is that not even the root account can bypass it. Reserve governance mode for internal, non-regulatory data where an authorized administrator legitimately needs an escape hatch. Mixing them up is the most common WORM misconfiguration: a "WORM" bucket in governance mode is defeated by anyone holding the bypass permission.
• Idempotent keys and versioning. Enable bucket versioning (Object Lock requires it) and derive the object key deterministically from the event's chain position and content hash. A retried write then re-targets the exact same immutable object instead of creating a divergent version, which keeps the WORM store aligned one-to-one with the hash chain.
• Clock and retention arithmetic. Compute retain_until from a trusted, NTP-synced clock in UTC. An off-by-a-timezone or naive-datetime bug can set retention hours short of the requirement — always construct timezone-aware UTC datetimes and validate that retain_until is at least the policy minimum before the write.

Validation Example: Before and After #

Write a reconciliation state-change event under a six-year compliance-mode hold, then attempt the two mutations an attacker would try.

event = {
    "seq": 4471,
    "subject_ref": "IMAD-20260716-CHASUS33-0417",
    "decision": "STATE_TRANSITION",
    "actor": "analyst_12",
    "note": "PROVISIONAL_CREDIT -> RESOLVED",
    "prev_hash": "d15c9a...",
    "event_hash": "a88022...",
}

result = put_audit_event(client, "recon-audit-worm", event,
                         retention_days=2192, legal_hold=True)
# WormPutResult(key='audit/000000004471-a88022...json',
#               version_id='3sL9...', retain_until=2032-07-15..., object_hash='...')

Before WORM (a plain versioned bucket), each of the following would succeed and quietly rewrite or remove the record:

• put_object on the same key → a new "latest" version shadows the original.
• delete_object with the version id → the audit event is gone.

After WORM (compliance mode + legal hold), both fail:

# Overwrite attempt: a new PUT creates a new version but CANNOT replace or
# erase the locked one — the original version remains, permanently readable.
client.put_object(Bucket="recon-audit-worm",
                  Key=result.key, Body=b"{}")           # new version only; original intact

# Versioned delete of the locked version is refused outright.
assert_immutable(client, "recon-audit-worm", result.key, result.version_id)
# returns normally: the delete raised AccessDenied, as required

Even the root account cannot delete result.version_id before 2032-07-15, and while the legal hold is ON it cannot be deleted after that date either — only releasing the hold, after retention has also lapsed, makes the object eligible for lifecycle expiration. That is the physical guarantee behind "non-rewriteable, non-erasable."

Failure Modes & Guardrails #

1. Governance-mode bypass mistaken for WORM. A bucket configured in governance mode looks immutable in day-to-day use but is deletable by any principal holding s3:BypassGovernanceRetention — including an insider or a compromised admin role. For regulatory audit trails, assert ObjectLockMode == "COMPLIANCE" on write and periodically re-read the object's retention configuration to confirm it was not silently created in governance mode.
2. Retention miscalculated by clock or timezone bug. A naive datetime.now() (no tzinfo), a host with a skewed clock, or day-count arithmetic that omits leap days can set retain_until short of the regulatory minimum, and once written in compliance mode you cannot correct it downward. Always build timezone-aware UTC datetimes, source time from NTP, and validate retain_until - now >= policy_minimum before the write, failing the append if it does not hold.
3. Overwrite attempt treated as success. Under versioning, a put_object to an existing key succeeds by creating a new version — it does not error — which can fool a test into believing overwrite "worked" and the bucket is therefore not WORM. Verify immutability by attempting a versioned delete of the specific locked version and asserting it is refused, exactly as assert_immutable does; never infer WORM status from a plain PUT returning 200.

Frequently Asked Questions #

Related guides in this collection #

• Immutable Audit Trails for Reconciliation Decisions — the parent reference on append-only, hash-chained, WORM-backed trails and what an examiner needs from each event.
• Building Hash-Chained Audit Logs for Reconciliation Events — the SHA-256 tamper-detection layer that pairs with the WORM prevention layer described here.
• Regulation E Error Resolution for ACH Disputes — the disputes whose evidentiary records inherit the longest retention holds.
• Reg E, NACHA & Audit Controls — the parent reference for the regulatory envelope around the reconciliation pipeline.