Audit Logging Patterns for Clinical AI Agents (Code Samples Inside)

Status: Satellite draft for Healthcare RAG hub. Body to be written by writer; outline locked. Target length: 1,400 words. Schema: Article + FAQPage. Companion to /blog/audit-ready-ai-five-questions (which covers *what to ask*) — this post covers *how to build*.

The companion to our audit-ready-ai-five-questions post — that one tells you what an auditor will ask. This one tells you how to build the system that answers them. Code samples in TypeScript, deployment patterns for AWS, and the schema we ship to clients.

What to log — the seven required fields

For every model invocation in a clinical AI system, the audit log row contains:

1. request_id (UUID, server-generated)
2. tenant_id (mapped to organization, EHR system, or matter — depending on workload)
3. user_id (the human user — never the service account)
4. session_id and conversation_id (for multi-turn flows)
5. prompt (full input as sent to the model — system prompt, conversation history, retrieved context, user message)
6. retrieved_chunks (array of {document_id, chunk_id, score} objects)
7. model_output (full output, with citation references)

Plus the operational fields: model_id, model_version, latency_ms, token_counts, timestamp, cost_estimate.

Where to log — the storage tier

• Hot tier (CloudWatch Logs): searchable by analyst within 60s, 30-day retention.
• Warm tier (S3 with Object Lock): all logs forever (or for the regulated retention period — six years HIPAA minimum). Object Lock prevents tampering even with admin credentials.
• Query tier (Athena over S3): ad-hoc analyst queries with cost controls. SQL over JSON.
• Streaming tier (Kinesis Firehose): delivers from CloudWatch → S3 with batching/compression. Enables Athena queries within minutes of write.

Diagram (writer to add SVG): the log stream from app → CloudWatch → Firehose → S3 (Object Lock + KMS) → Athena.

How to redact — the audit log is PHI

The audit log itself contains PHI. Treat it like clinical data, not like operational logs:

• KMS encryption at rest with customer-managed keys (CMK).
• Access controls scoped — analysts who can run Athena queries are not necessarily the same humans who can read full prompts.
• *Optional:* PII/PHI redaction at log-write time, with a separate "decrypt-on-demand" service for incident response. Trade-off section here on the operational cost vs. compliance posture.

How to retain — the six-year minimum

• HIPAA requires six years for documentation related to compliance with the Privacy and Security Rules.
• Some state regulations require longer (e.g., California, certain pediatric records).
• S3 lifecycle policies: Standard → Standard-IA at 90 days → Glacier at 1 year → Deep Archive at 3 years.
• Object Lock in compliance mode: even root cannot delete before retention period.

How to query — the analyst experience

Sample Athena queries. Writer to render code blocks for each:

• "Every interaction by user X between dates A and B"
• "Every interaction touching patient/matter/account ID Y"
• "Every interaction where retrieval came back empty"
• "Every interaction where model output lacked citations"
• "Every interaction by model version Z"

These are the queries that come up in real audits. If the schema can't support them, the schema is wrong.

The middleware — TypeScript code sample

Writer to render a complete Hono/Express middleware example:

// Pseudocode placeholder — writer to fill in real implementation
export function withAuditLog(handler: ChatHandler): ChatHandler {
  return async (req) => {
    const requestId = randomUUID()
    const startedAt = Date.now()
    try {
      const result = await handler(req)
      await writeAuditLog({
        requestId,
        tenantId: req.tenantId,
        userId: req.userId,
        // ... all 7 required fields
        latencyMs: Date.now() - startedAt,
      })
      return result
    } catch (err) {
      await writeAuditLog({ requestId, error: err.message, /* etc */ })
      throw err
    }
  }
}

Trade-off section: synchronous write (slower request, guaranteed durability) vs. async write to Kinesis (fast request, durability bounded by Kinesis retention). For HIPAA, sync to a durable buffer (Kinesis with PutRecords) is the right answer.

What goes wrong in production

• The user_id field is captured but is the service account, not the human user.
• Retrieved chunks are not logged; only the LLM input is logged. Audits cannot trace "what did the AI see."
• Logs are written to CloudWatch only with no S3 export. CloudWatch retention costs go vertical at scale.
• Object Lock not enabled. A compromised admin credential becomes a compliance incident.
• PII redaction is enabled, but the redaction strategy is regex-based and fails on free-text clinical notes.

FAQ

For FAQPage schema. ~75-100 words each.

1. How much does this audit log infrastructure cost at 50K interactions/month?
2. Do we need to log if the model is a local model that never leaves our VPC?
3. Can we use OpenSearch instead of Athena for query?
4. Should we redact PHI at log-write or at log-read?
5. How does this integrate with our existing SIEM?
6. What if our retention requirement is longer than six years?

Where we fit

CTA — links to architecture review (Cal.com), back to the HIPAA-aligned RAG cornerstone, and to healthcare AI consulting.