Healthcare AI has a hard constraint that most other domains don't: regulatory requirements that must be satisfied before you can ship. HIPAA for US, similar frameworks in other jurisdictions. The good news: compliance-first design is a tractable engineering problem. The bad news: teams who design around it late in the project usually rebuild significant portions. This post is how we architect healthcare AI from day one to pass compliance review.
What HIPAA requires, concretely
At the 30,000-foot summary for engineers: protected health information (PHI) can only be shared with 'business associates' who have signed Business Associate Agreements (BAA). You need administrative, physical, and technical safeguards around PHI. You need audit logs that identify who accessed what. You need breach notification procedures. HIPAA is a framework, not a checklist — specific implementations are judgment calls. Don't design alone; involve compliance and legal early.
International equivalents (GDPR special-category data, India's DPDP Act for health data, UK's Data Protection Act) differ in specifics but align on principles: minimum-necessary access, auditability, explicit consent, breach notification. Building for HIPAA gets you most of the way to the others.
The compliant stack we deploy
Layer 1: VPC and network isolation
All AI workloads run in a dedicated VPC with no public internet egress. Private endpoints to the AI provider (AWS PrivateLink to Bedrock, Azure's private endpoints to OpenAI). This is both a security and a compliance control — PHI physically cannot leave the network boundary through an unexpected route.
Layer 2: PHI gateway
A middleware layer that every AI request passes through. Responsibilities: identify PHI in request (see our PII redaction post), redact with reversible tokens, log the request with PHI-free summary, forward to the model, restore placeholders in response, log completion. This layer is where you implement your minimum-necessary and auditability requirements.
Layer 3: BAA-covered AI layer
Your model provider must have a BAA. As of 2026: Anthropic offers BAAs on enterprise plans. Azure OpenAI supports BAAs on specific regions. Google Vertex AI supports BAAs. Self-hosted open models (Llama, etc.) on your own BAA-covered infrastructure (AWS/Azure/GCP have BAAs) also works. OpenAI's standard API does not support BAAs — you must use Azure OpenAI for HIPAA workloads on the OpenAI family. Always verify current BAA status with the provider's compliance documentation.
Layer 4: Application logic with role-based access
Patient access, clinician access, and administrative access are different surfaces. Role-based access control at every API endpoint. Patient-facing features never reveal other patients' data. Clinician-facing features respect care-relationship constraints (a clinician can only access patients in their panel unless they assert break-the-glass, which is logged).
Layer 5: Audit logs with immutability
Every PHI access, AI request, and AI response logged with user, timestamp, purpose, and PHI summary. Logs retained per HIPAA requirements (6 years minimum in the US). Logs immutable — append-only storage, write-once semantics. We typically use AWS CloudTrail Lake or a dedicated SIEM with legal-hold capabilities.
Specific patterns that pass review
Clinical documentation assistants: summarize encounters, draft notes, assist with billing codes. Core pattern: clinician always signs off on output before it enters the EHR. AI is a productivity tool; clinician is the decision-maker. Audit log captures AI-suggested vs clinician-approved.
Patient-facing symptom triage: guide patients to appropriate care level. Core pattern: strong safety guardrails, clear escalation to human clinicians for anything ambiguous, explicit disclaimers about AI limitations. Many regulatory frameworks treat this as software-as-medical-device; FDA pathway may apply in the US. Not a weekend project.
Revenue cycle automation: prior authorization, claim denials, eligibility verification. Lower PHI volume than clinical use cases, often higher ROI. Standard pattern applies — compliant stack, audit trail, human review of denials.
Research and population health analytics: often involves de-identified or limited-data-set variants of PHI. Different rules apply; simpler in some ways, stricter in others (de-identification is itself a regulated process). Separate workflow from clinical and patient-facing systems.