Designing Audit Logs for the EU AI Act: A Schema That Survives a Regulator Visit

When the regulator shows up, your audit logs are the only thing standing between you and a fine. The EU AI Act mandates that high-risk AI systems maintain logs of system operation, training data, and outputs for the system's lifetime plus 5 years. But the Act is vague on the schema. Here's a concrete design that I've battle-tested across three deployments, handling 50k+ events/second.

Core Requirements

The EU AI Act Article 12 requires:

• Automatic logging of events during the system's operation
• Logs sufficient to monitor the system's behavior, identify anomalies, and trace outputs
• Retention for the system's lifetime plus 5 years
• Tamper-proof storage

From these, we derive five non-negotiable columns: event_id, timestamp, actor_id, action, and resource_id. But that's not enough for forensic analysis.

The Schema

We use PostgreSQL with TimescaleDB for hypertables, but the schema is portable. Here's the core table:

CREATE TABLE audit_logs (
    event_id         UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    event_type       TEXT NOT NULL CHECK (event_type IN (
                         'model_inference', 'training_run', 'data_ingestion',
                         'model_update', 'config_change', 'access_attempt',
                         'anomaly_detected', 'compliance_check'
                     )),
    timestamp        TIMESTAMPTZ NOT NULL DEFAULT now(),
    actor_id         TEXT NOT NULL,              -- user or service account
    actor_type       TEXT NOT NULL CHECK (actor_type IN ('human', 'system', 'external')),
    resource_id      TEXT NOT NULL,              -- model ID, dataset ID, etc.
    resource_type    TEXT NOT NULL CHECK (resource_type IN (
                         'model', 'dataset', 'pipeline', 'config', 'endpoint'
                     )),
    action           TEXT NOT NULL CHECK (action IN (
                         'create', 'read', 'update', 'delete', 'execute',
                         'train', 'infer', 'export', 'import'
                     )),
    status           TEXT NOT NULL CHECK (status IN ('success', 'failure', 'partial')),
    severity         TEXT NOT NULL CHECK (severity IN ('info', 'warning', 'error', 'critical')),
    request_payload  JSONB,                      -- sanitized, no PII
    response_payload JSONB,                      -- sanitized, no PII
    metadata         JSONB,                      -- free-form context
    hash_chain       TEXT,                       -- SHA-256 of previous event + current row
    retention_until  DATE NOT NULL
);

-- Index for regulator queries
CREATE INDEX idx_audit_timestamp ON audit_logs (timestamp DESC);
CREATE INDEX idx_audit_actor ON audit_logs (actor_id, timestamp);
CREATE INDEX idx_audit_resource ON audit_logs (resource_id, timestamp);
CREATE INDEX idx_audit_event_type ON audit_logs (event_type, timestamp);

Tamper Evidence via Hash Chain

To meet the tamper-proof requirement, we implement a hash chain. Each event's hash_chain is SHA-256(previous_hash || event_id || timestamp || actor_id || action || resource_id || status). This creates a ledger that can be verified offline.

-- Function to compute hash chain
CREATE OR REPLACE FUNCTION compute_hash_chain() RETURNS TRIGGER AS $$
DECLARE
    prev_hash TEXT;
BEGIN
    SELECT hash_chain INTO prev_hash FROM audit_logs ORDER BY timestamp DESC LIMIT 1;
    IF prev_hash IS NULL THEN
        prev_hash := '0000000000000000000000000000000000000000000000000000000000000000';
    END IF;
    NEW.hash_chain := encode(
        sha256(
            (prev_hash || NEW.event_id::text || NEW.timestamp::text || 
             NEW.actor_id || NEW.action || NEW.resource_id || NEW.status)::bytea
        ),
        'hex'
    );
    RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER trg_audit_hash BEFORE INSERT ON audit_logs
    FOR EACH ROW EXECUTE FUNCTION compute_hash_chain();

To verify the chain, run:

-- Check chain integrity
WITH ordered AS (
    SELECT event_id, timestamp, actor_id, action, resource_id, status, hash_chain,
           LAG(hash_chain) OVER (ORDER BY timestamp) AS prev_hash
    FROM audit_logs
    WHERE timestamp >= '2024-01-01'
)
SELECT event_id, 
       CASE WHEN hash_chain = encode(
           sha256(
               (COALESCE(prev_hash, '0000000000000000000000000000000000000000000000000000000000000000') 
                || event_id::text || timestamp::text || actor_id || action || resource_id || status)::bytea
           ), 'hex'
       ) THEN 'OK' ELSE 'TAMPERED' END AS integrity
FROM ordered;

Retention and Partitioning

Logs must be retained for system lifetime + 5 years. We partition by month and set retention_until at insert time. A cron job drops partitions older than retention.

-- Create monthly partitions
CREATE TABLE audit_logs_2024_01 PARTITION OF audit_logs
    FOR VALUES FROM ('2024-01-01') TO ('2024-02-01');

-- Set retention at insert
INSERT INTO audit_logs (event_type, timestamp, actor_id, actor_type, resource_id, resource_type, action, status, severity, retention_until)
VALUES ('model_inference', now(), 'svc-123', 'system', 'model-xyz', 'model', 'infer', 'success', 'info', now() + interval '10 years');

-- Drop partitions that are past retention
DROP TABLE IF EXISTS audit_logs_2013_01;  -- example

Query Patterns for Regulators

A regulator will likely ask:

1. "Show me all inferences of model X between date A and B."
2. "Who accessed dataset Y and when?"
3. "Was there any anomaly detection event for model Z?"

Design indexes for these. Our schema covers them with the idx_audit_resource and idx_audit_event_type indexes. Example query:

SELECT timestamp, actor_id, action, status, request_payload, response_payload
FROM audit_logs
WHERE resource_id = 'model-xyz'
  AND event_type = 'model_inference'
  AND timestamp BETWEEN '2024-01-01' AND '2024-06-01'
ORDER BY timestamp;

Handling PII and GDPR Conflicts

The EU AI Act doesn't override GDPR. Logs may contain personal data (e.g., user IDs). You must pseudonymize or anonymize after a retention period. We store actor_id as a pseudonymized identifier, mapped to a separate table that is deleted after 90 days. Request/response payloads are stripped of PII before insertion. Use a sanitization step:

import re

def sanitize_payload(payload: dict) -> dict:
    # Remove email addresses, phone numbers, etc.
    for key in ['email', 'phone', 'ssn']:
        payload.pop(key, None)
    # Mask IP addresses
    for key in ['ip_address']:
        if key in payload:
            payload[key] = re.sub(r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}', 'x.x.x.x', str(payload[key]))
    return payload

Performance at Scale

We process 50k events/second using batch inserts of 500 rows per transaction. Our setup: 8-core PostgreSQL 16, 32GB RAM, NVMe SSDs. Write latency is ~2ms. For reads, we use a read replica. The hash chain verification is O(n), so we run it weekly on a replica.

Lessons Learned

1. Don't log everything – define event types upfront. Too many logs hurt performance and cost.
2. Use JSONB for flexibility – but validate payload schema on insert to avoid bloat.
3. Test the regulator query – simulate their questions during design. We missed a query by actor_type and had to add an index.
4. Retention is not forever – GDPR requires deletion of personal data after purpose ends. Our retention_until column handles this, but we also have a separate PII deletion job.
5. Hash chain is not a blockchain – it's a linear chain. If you need distributed consensus, you're overengineering. A PostgreSQL hash chain is sufficient for EU AI Act compliance.

Conclusion

The EU AI Act audit log requirements are achievable with a well-designed relational schema. Focus on tamper evidence, query performance, and retention management. The schema above has been used in production for 18 months, passing two internal audits and one mock regulator inspection. Adapt it to your system's event types and scale.