Why We Log Prompt Hashes and Outputs, Not Raw Text

The EU AI Act is coming, and with it comes a new set of obligations for organizations deploying AI systems in the EU. One of the most talked-about requirements is the logging obligation for high-risk AI systems. Article 12 of the Act requires automatic recording of events (logs) to enable traceability of the AI system's functioning. The logs must be sufficient to interpret the system's behavior and identify potential risks.

But here's the rub: how do you log prompts and outputs without turning your log database into a goldmine for attackers and a regulatory nightmare? Logging raw text is a data protection disaster waiting to happen. It's also a security risk—your logs become a repository of sensitive information that, if breached, could expose trade secrets, personal data, or worse.

The solution we've adopted is to log cryptographic hashes of prompts and outputs, not the raw text. This might sound counterintuitive—how can you audit something you can't read?—but with the right architecture, hashes give you everything you need for compliance without the baggage.

The EU AI Act Logging Requirement: A Quick Recap

Let's be precise about what the Act demands. Article 12 (1) states that high-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system. These logs must be sufficient to identify situations that may result in risks, facilitate post-market monitoring, and allow for the identification of potentially problematic patterns.

Annex IV (Technical documentation) and Article 17 (Quality management) further specify that logs should include:

• The operational environment (e.g., model version, deployment ID)
• The input data (the prompt)
• The output (the model's response)
• Timestamps
• Unique identifiers for the actor (user or system)

So, yes, the Act expects you to log inputs and outputs. But it doesn't say you have to log them in plaintext. In fact, the Act promotes a risk-based approach and data minimization principles from GDPR. Logging raw text is a disproportionate means to achieve the traceability goal.

The Problem with Raw Text Logging

Let me count the ways raw text logging is a bad idea:

1. Privacy violations: Prompts often contain personal data—names, addresses, medical details. Even if you're not a health provider, your users might ask your AI about their health, finances, or legal matters. Logging that raw text is a GDPR violation unless you have a legal basis, and you probably don't.

2. Security risk: Logs are a high-value target. If an attacker gets access to your logs, they get a treasure trove of sensitive conversations. Breaches happen all the time; don't make them worse.

3. Storage bloat: Raw text logs accumulate fast. A model that processes 100k requests a day, with average prompt+output of 2KB, is 200MB/day. That's 6GB/month, 72GB/year. Multiply that by retention periods (which can be years) and you have a storage problem.

4. Legal liability: If you're a processor, you might not have the right to retain that data. If you're a controller, you have to justify retention. Raw logs are a legal landmine.

5. Ethical concerns: Your users trust you with their data. Logging their conversations in plaintext is a betrayal of that trust, even if it's for compliance.

Hashing: The Privacy-Preserving Alternative

Cryptographic hashing is a one-way function. You can't reverse it to get the original text. But you can compare hashes: if you have the same input, you get the same hash. This property is perfect for compliance logging.

Here's the idea:

• When a prompt arrives, compute a hash of the prompt (e.g., SHA-256).
• When the output is generated, compute a hash of the output.
• Log the hashes, along with metadata (timestamp, model version, user ID, etc.).
• Optionally, store the raw text in a separate, encrypted store with strict access controls, or don't store it at all.

But wait—if you only have hashes, how do you audit? How do you investigate an incident? You can't read the prompt from a hash. That's where a lookup service comes in. You maintain a separate database that maps hashes to raw text, but that database is heavily protected and only accessible for specific purposes (e.g., legal investigation). The logs themselves only contain hashes, so if they leak, the attacker gets nothing.

This is a classic pattern in security: separate sensitive data from operational data. Logs are operational; raw text is sensitive.

Why Hashes Are Sufficient for Compliance

The Act's goals are to enable traceability and post-market monitoring. Let's see how hashes serve those goals:

• Traceability: You can trace a specific event because you have a unique identifier (the hash) that corresponds to a specific prompt. If you need to reproduce the prompt, you can query the lookup service.
• Incident investigation: If a model misbehaves, you can identify the exact prompts that caused the issue by searching for their hashes in the logs. Then you retrieve the raw text from the secure store.
• Pattern detection: You can analyze hash frequencies to detect unusual activity (e.g., many identical prompts indicating a DoS attack or prompt injection).
• Model governance: You can verify that a given output was generated by a specific model version by comparing hashes.

In practice, auditors don't need to read every prompt; they need to verify that the system is functioning as expected and that you can produce evidence when needed. Hashes provide that evidence.

Implementation: A Concrete Example

Let's put this into practice. We'll use Python, but the concept applies to any language.

First, define a simple function to hash a string:

import hashlib

def hash_text(text: str) -> str:
    return hashlib.sha256(text.encode('utf-8')).hexdigest()

Now, when you receive a prompt and generate an output, you log the hashes:

prompt_hash = hash_text(prompt)
output_hash = hash_text(output)

log_entry = {
    "timestamp": datetime.utcnow().isoformat(),
    "user_id": user.id,
    "model_version": "llama-3-8b-instruct-v2",
    "prompt_hash": prompt_hash,
    "output_hash": output_hash,
    "request_id": uuid.uuid4().hex
}

# Write to your log store (e.g., PostgreSQL, Elasticsearch)
write_log(log_entry)

And in a separate, encrypted store, you keep the raw text only if you have a legitimate need:

# Only if you need to retain raw text (e.g., for debugging, but with strict access)
if retain_raw:
    store_raw_text(request_id, prompt, output)

This way, your logs are safe to keep for years. The raw text is either not stored or is stored with encryption and access controls.

Handling Collisions and Security Considerations

SHA-256 collisions are practically impossible, but you should still use a strong hash and consider adding a salt. A salt (a random string per deployment) prevents rainbow table attacks. If an attacker gets the hash, they can't precompute a dictionary of common prompts.

import os
SALT = os.environ.get("HASH_SALT", "change-me")

def hash_text(text: str) -> str:
    salted = text + SALT
    return hashlib.sha256(salted.encode('utf-8')).hexdigest()

But be careful: if you need to compare hashes across deployments, the salt must be consistent. For logs, you likely don't need cross-deployment comparison, so a per-deployment salt is fine.

Beyond Hashing: Differential Privacy and Redaction

Hashing is not the only technique. You can also apply redaction before hashing: strip out personal identifiers (names, emails) from the prompt, then hash the redacted version. This adds an extra layer of privacy.

Another option is differential privacy for aggregates, but that's overkill for logging.

We combine hashing with redaction: we run a quick PII detector on the prompt, replace sensitive parts with placeholders, then hash. This way, even if the raw text is somehow recovered from the hash (which is impossible), the PII is already gone.

The Lookup Service: Your Key to the Past

The lookup service is a separate database that maps request_id to raw text. It's protected by:

• Encryption at rest (AES-256)
• Strict access controls (e.g., only certain roles can read)
• Audit logging of access
• Short retention (e.g., 30 days) unless longer is legally required

This service is only used when you need to investigate an incident or respond to a legal request. It's not part of the main log pipeline.

What About Outputs?

Outputs are equally sensitive. They may contain copyrighted material, personal data, or harmful content. Hashing outputs is just as important. But there's a nuance: outputs are often derived from inputs, so an output hash alone might not be enough to identify the exact output if you need to reproduce it. That's why we store the raw output in the lookup service as well.

Real-World Example: Our Deployment

We run a self-hosted LLM inference stack using vLLM for serving, PostgreSQL for metadata, and pgvector for embeddings. For compliance, we added a logging pipeline:

• Inference proxy (nginx) captures request/response.
• A Python service computes hashes and writes to a dedicated PostgreSQL table audit_log.
• Raw text is stored in an encrypted S3 bucket (we use MinIO) with a 30-day lifecycle.
• Access to raw text is via a web UI that requires multi-factor authentication.

Here's the schema:

CREATE TABLE audit_log (
    id BIGSERIAL PRIMARY KEY,
    timestamp TIMESTAMPTZ NOT NULL,
    user_id UUID NOT NULL,
    model_version TEXT NOT NULL,
    prompt_hash TEXT NOT NULL,
    output_hash TEXT NOT NULL,
    request_id UUID NOT NULL UNIQUE
);

And the raw text store:

CREATE TABLE raw_text (
    request_id UUID PRIMARY KEY,
    prompt TEXT NOT NULL,
    output TEXT NOT NULL,
    created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

We never join these tables in normal operations. Only a dedicated admin tool can access raw_text, and it logs every access.

Compliance in Practice: What Auditors Will See

When an auditor comes, they'll ask: "Show me how you log prompts." We show them the audit_log table with hashes. They might ask: "How do you retrieve the actual prompt?" We demonstrate the lookup service with proper access controls. They'll check that the logs are tamper-evident (we use append-only and hash chaining) and that retention is defined.

We also provide a data protection impact assessment that justifies why hashes are sufficient. The key argument: the Act requires traceability, not raw data retention. By minimizing data, we comply with GDPR's data minimization principle.

Potential Pitfalls

• Hash collisions: Practically impossible, but use SHA-256 or better.
• Salt management: If you rotate salts, you lose the ability to compare old hashes. Keep a historical salt map.
• Lookup service security: If this is breached, all raw text is exposed. Invest in robust security.
• Legal requirements: Some jurisdictions might require raw logs. Check with your legal counsel.

The Future: Homomorphic Encryption and More

As the Act evolves, we might see more sophisticated techniques like homomorphic encryption, which allows computation on encrypted data. But that's not practical today. Hashing is a pragmatic middle ground.

Conclusion

Logging prompt hashes and outputs is not just a workaround; it's a better engineering practice. It reduces risk, saves storage, and still meets the EU AI Act's logging obligations. The Act doesn't mandate raw text; it mandates traceability. With hashes, you get that traceability without the liability.

If you're building a high-risk AI system, start with a logging architecture that separates operational logs from sensitive data. Use hashes for the former, and a secure lookup for the latter. Your security team will thank you, and your auditors will be satisfied.

Now, go implement it.