Quick Reference: Clinical AI Checklists

This appendix collects the key checklists from throughout the book for easy reference. Print this section or keep it accessible when evaluating, deploying, or using clinical AI systems.

The Clinician’s Checklist

From Chapter 18: Interpretability & Uncertainty

Before trusting an AI tool’s output, ask yourself:

• What was the training population? Does it match my patients in demographics, disease prevalence, and imaging equipment?

• What is the failure mode? When does this tool get it wrong? Are those situations I can recognize?

• What is the fallback? When the tool fails or is unavailable, what do I do?

• Can I see WHY it made this prediction? Is there an explanation I can verify against my clinical knowledge?

• Who is liable? If I follow this recommendation and it’s wrong, what is the medicolegal situation?

• How will I know if it stops working? Is there ongoing monitoring? Who gets alerted to performance degradation?

---

Pre-Deployment Checklist

From Chapter 16: AI Ops & Deployment

Before any clinical AI system goes live, verify:

Performance Validation

• Holdout test set performance meets defined thresholds
• Subgroup analysis shows no significant disparities across patient populations
• Calibration is acceptable (predicted probabilities match observed frequencies)
• Performance on edge cases and failure modes is documented

Infrastructure

• API endpoints respond within latency requirements
• Health checks are configured and monitored
• Logging captures inputs, outputs, and errors (without PHI exposure)
• Rollback procedure is tested and documented

Monitoring

• Drift detection pipeline is active
• Alerting thresholds are configured
• Dashboard shows key metrics in real-time
• On-call rotation is established

Governance

• Model card is complete and accessible
• Escalation protocols are documented and communicated
• Clinical users are trained on appropriate use and limitations
• Feedback mechanism for reporting errors is in place

Compliance

• Legal/compliance review is complete
• IRB approval obtained (if required)
• Data use agreements are in place
• Documentation meets regulatory requirements (FDA, EU MDR if applicable)

Go/No-Go Decision

• All stakeholders have signed off
• Rollback trigger conditions are defined
• Post-launch monitoring plan is scheduled

---

Field Guide Checklist

From Chapter 22: Writing the Field Guide

Every clinical AI field guide should include:

• Tool Summary: One paragraph explaining what it does in plain language

• Intended Use and Scope: What it’s for AND what it’s not for

• How It Works: Plain language explanation (no code, no jargon)

• Performance Summary: Key metrics with confidence intervals and context

• Limitations and Failure Modes: Specific, honest list of weaknesses

• Human Oversight Rules: When and how to override

• Monitoring and Feedback: How to report problems, who monitors performance

---

Model Evaluation Checklist

From Chapters 6 and 8

When evaluating a clinical AI model:

Data Quality

• Training data source and collection process documented
• Labels verified (how were ground truth labels obtained?)
• Class imbalance addressed or documented
• Data leakage checked (no test data in training)

Performance Metrics

• Metrics appropriate to clinical use case selected
• Sensitivity and specificity reported at clinical threshold
• AUROC with confidence intervals provided
• Calibration assessed (reliability diagram)

Generalization

• Tested on held-out data (not used in training or tuning)
• Tested on external data (different institution if possible)
• Subgroup performance reported (age, sex, race, disease severity)
• Domain shift risks identified

Clinical Validity

• Comparison to current standard of care
• Clinical utility demonstrated (not just statistical significance)
• Failure modes characterized
• Intended clinical workflow defined

---

Bias Audit Checklist

From Chapter 20: Fairness, Bias & Health Equity

When auditing a clinical AI system for bias:

Data Representation

• Demographics of training data documented
• Underrepresented populations identified
• Historical biases in labels assessed
• Proxy variables checked (ZIP code, insurance status)

Performance Parity

• Metrics stratified by protected attributes
• Significant disparities identified and quantified
• Error type distribution compared (FPR, FNR across groups)
• Intersectional analysis performed (e.g., elderly Black women)

Mitigation

• Fairness constraints considered during training
• Post-processing calibration evaluated
• Tradeoffs documented (equity vs. overall accuracy)
• Monitoring plan includes subgroup performance

Deployment Safeguards

• Populations where tool should NOT be used identified
• Override rules account for known disparities
• Feedback loop established for detecting emerging bias
• Regular re-audit scheduled

---

Regulatory Pathway Checklist

From Chapter 21: Regulation & Ethics

When preparing a clinical AI system for regulatory submission:

Classification (US FDA)

• Determined if device is SaMD (software as medical device)
• Risk classification established (Class I, II, or III)
• Appropriate pathway identified (510(k), De Novo, or PMA)
• Predicate device identified (for 510(k))

Classification (EU MDR)

• MDR classification determined (Class I, IIa, IIb, or III)
• Rule 11 assessment completed for software
• Notified Body selected (for Class IIa and above)
• GDPR compliance addressed

Documentation

• Device description and intended use complete
• Training data and algorithm documented
• Validation study design and results documented
• Subgroup analyses included

Post-Market

• Post-market surveillance plan in place
• Adverse event reporting procedure established
• Predetermined Change Control Plan prepared (if applicable)
• Continuous monitoring framework defined

---

SaMD Classification Quick Reference

From Chapter 21

Is it SaMD? Ask: Does the software provide information used for clinical decisions?

System Type	SaMD?	Typical Class
Administrative scheduling	No	N/A
EHR data storage	No	N/A
Diagnosis support (human in loop)	Yes	Class II
Diagnosis autonomous	Yes	Class III
Treatment recommendation	Yes	Class II/III
Triage/prioritization	Yes	Class II
Risk prediction	Yes	Class II

Key question: What is the intended use? The same algorithm with different intended uses may have different classifications.

---

Quick Ethics Framework

From Chapter 21

When facing an ethical dilemma in clinical AI:

1. Identify stakeholders: Who is affected? (Patients, clinicians, institution, society)

2. Clarify the dilemma: What values are in tension?

3. Gather facts: What do we actually know about impacts?

4. Consider principles:

   • Beneficence (does it help?)
   • Non-maleficence (does it harm?)
   • Autonomy (are choices respected?)
   • Justice (are benefits/burdens fairly distributed?)
   • Transparency (do stakeholders understand?)
   • Accountability (who is responsible?)

5. Explore options: What alternatives exist?

6. Evaluate tradeoffs: Who bears costs and benefits?

7. Decide and document: Make a reasoned decision, record the reasoning

8. Monitor and revise: Track outcomes, adjust as needed

---

Emergency Reference: When AI Fails

If you suspect a clinical AI system is providing incorrect outputs:

Immediate Actions

1. Do not rely on the output for the current patient
2. Use clinical judgment and standard-of-care protocols
3. Document the AI output and your clinical decision

Reporting

1. File a safety report through your institution’s system
2. Notify the AI system owner (see system documentation)
3. Include: Patient context (deidentified), AI output, your assessment, why you suspect error

If Widespread Problem Suspected

1. Contact the responsible AI governance committee
2. Request temporary suspension pending investigation
3. Alert colleagues who may be affected

Remember

• AI failures may affect many patients simultaneously
• Your report may protect other patients
• Even uncertain concerns should be reported