Archive \ Volume.16 2025 Issue 2

Building an Explainability Contract between Clinical Algorithms and Practicing Pharmacists

, ,
  1. Department of Explainable AI in Pharmacy, College of Pharmacy, University of Florida, Gainesville, United States.
  2. Department of Algorithm Transparency and Pharmacy Practice, Faculty of Pharmacy, National University of Singapore, Singapore.

Abstract

Clinical algorithms increasingly support medication-risk identification, prescription prioritization, alert management, and other pharmacy activities. Yet explainability is commonly treated as a technical property of a model rather than an accountable relationship among developers, health-care organizations, practising pharmacists, and affected patients. An explanation may appear plausible while omitting uncertainty, local applicability, data limitations, workflow consequences, or mechanisms for professional challenge. This article proposes an explainability contract for pharmacist-facing clinical algorithms. The contract is a non-legal, sociotechnical governance construct that links each algorithmic role in the medication-decision lifecycle to defined parties, pharmacist rights, reciprocal duties, required disclosures, and procedures for challenge, correction, and escalation. Its principal components include intended-use boundaries, data provenance, patient-specific rationale, uncertainty and calibration, local-validity information, professional override, version traceability, and documented resolution of material concerns. The framework distinguishes model output from medication authorization, explanation plausibility from fidelity, technical performance from pharmaceutical usefulness, and implementation from demonstrated benefit. Contract performance would require staged technical, clinical, human-factors, organizational, medication-safety, and equity evaluation. Relevant outcomes include explanation fidelity, appropriate reliance, workflow accessibility, challenge responsiveness, correction completeness, subgroup performance, and recurrence of known failures. The proposed contract does not establish that explanation improves medication decisions, transfer liability to pharmacists, guarantee equitable outcomes, or constitute a validated clinical, legal, regulatory, or deployment standard. Its original contribution is to reposition explainability from optional information supplied by an algorithm into a reciprocal and contestable governance relationship requiring empirical validation.


Downloads: 26
Views: 108

How to cite:
Vancouver
Anderson M, Wong L, Lee S. Building an Explainability Contract between Clinical Algorithms and Practicing Pharmacists. Arch Pharm Pract. 2025;16(2):26-34. https://doi.org/10.51847/rgnlCe8U0W
APA
Anderson, M., Wong, L., & Lee, S. (2025). Building an Explainability Contract between Clinical Algorithms and Practicing Pharmacists. Archives of Pharmacy Practice, 16(2), 26-34. https://doi.org/10.51847/rgnlCe8U0W

Download Citation
References
  1. Ghassemi M, Oakden-Rayner L, Beam AL. The false hope of current approaches to explainable artificial intelligence in health care. Lancet Digit Health. 2021;3(11). doi:10.1016/S2589-7500(21)00208-9
  2. Amann J, Blasimme A, Vayena E, Frey D, Madai VI; Precise4Q Consortium. Explainability for artificial intelligence in healthcare: A multidisciplinary perspective. BMC Med Inform Decis Mak. 2020;20(1):310. doi:10.1186/s12911-020-01332-6
  3. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: Benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. doi:10.1038/s41746-020-0221-y
  4. Graafsma J, Murphy RM, van de Garde EMW, Karapinar-Çarkit F, Derijks HJ, Hoge RHL, et al. The use of artificial intelligence to optimize medication alerts generated by clinical decision support systems: A scoping review. J Am Med Inform Assoc. 2024;31(6):1411-22. doi:10.1093/jamia/ocae076
  5. Corny J, Rajkumar A, Martin O, Dode X, Lajonchère JP, Billuart O, et al. A machine learning-based clinical decision support system to identify prescriptions with a high risk of medication error. J Am Med Inform Assoc. 2020;27(11):1695-704. doi:10.1093/jamia/ocaa154
  6. Miller T. Explanation in artificial intelligence: Insights from the social sciences. Artif Intell. 2019;267:1-38. doi:10.1016/j.artint.2018.07.007
  7. Markus AF, Kors JA, Rijnbeek PR. The role of explainability in creating trustworthy artificial intelligence for health care: A comprehensive survey of the terminology, design choices, and evaluation strategies. J Biomed Inform. 2021;113:103655. doi:10.1016/j.jbi.2020.103655
  8. Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell. 2019;1(5):206-15. doi:10.1038/s42256-019-0048-x
  9. Babic B, Gerke S, Evgeniou T, Cohen IG. Beware explanations from AI in health care. Science. 2021;373(6552):284-6. doi:10.1126/science.abg1834
  10. Char DS, Shah NH, Magnus D. Implementing machine learning in health care—addressing ethical challenges. N Engl J Med. 2018;378(11):981-3. doi:10.1056/NEJMp1714229
  11. Reddy S, Allan S, Coghlan S, Cooper P. A governance model for the application of AI in health care. J Am Med Inform Assoc. 2020;27(3):491-7. doi:10.1093/jamia/ocz192
  12. Asan O, Bayrak AE, Choudhury A. Artificial intelligence and human trust in healthcare: Focus on clinicians. J Med Internet Res. 2020;22(6). doi:10.2196/15154
  13. Price WN 2nd, Gerke S, Cohen IG. Potential liability for physicians using artificial intelligence. JAMA. 2019;322(18):1765-6. doi:10.1001/jama.2019.15064
  14. Benda NC, Das LT, Abramson EL, Blackburn K, Thoman A, Kaushal R, et al. “How did you get to this number?” Stakeholder needs for implementing predictive analytics: A pre-implementation qualitative study. J Am Med Inform Assoc. 2020;27(5):709-16. doi:10.1093/jamia/ocaa021
  15. Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Calibration: The Achilles heel of predictive analytics. BMC Med. 2019;17(1):230. doi:10.1186/s12916-019-1466-7
  16. Subbaswamy A, Saria S. From development to deployment: Dataset shift, causality, and shift-stable models in health AI. Biostatistics. 2020;21(2):345-52. doi:10.1093/biostatistics/kxz041
  17. Cabitza F, Rasoini R, Gensini GF. Unintended consequences of machine learning in medicine. JAMA. 2017;318(6):517-18. doi:10.1001/jama.2017.7797
  18. Gaube S, Suresh H, Raue M, Merritt A, Berkowitz SJ, Lermer E, et al. Do as AI say: Susceptibility in deployment of clinical decision-aids. NPJ Digit Med. 2021;4(1):31. doi:10.1038/s41746-021-00385-9
  19. McCradden MD, Joshi S, Mazwi M, Anderson JA. Ethical limitations of algorithmic fairness solutions in health care machine learning. Lancet Digit Health. 2020;2(5). doi:10.1016/S2589-7500(20)30065-0
  20. Finlayson SG, Bowers JD, Ito J, Zittrain JL, Beam AL, Kohane IS. Adversarial attacks on medical machine learning. Science. 2019;363(6433):1287-9. doi:10.1126/science.aaw4399
  21. Wiens J, Saria S, Sendak M, Ghassemi M, Liu VX, Doshi-Velez F, et al. Do no harm: A roadmap for responsible machine learning for health care. Nat Med. 2019;25(9):1337-40. doi:10.1038/s41591-019-0548-6
  22. Vollmer S, Mateen BA, Bohner G, Király FJ, Ghani R, Jonsson P, et al. Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ. 2020;368. doi:10.1136/bmj.l6927
  23. Norgeot B, Quer G, Beaulieu-Jones BK, Torkamani A, Dias R, Gianfrancesco M, et al. Minimum information about clinical artificial intelligence modeling: The MI-CLAIM checklist. Nat Med. 2020;26(9):1320-4. doi:10.1038/s41591-020-1041-y
  24. Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med. 2022;28(5):924-33. doi:10.1038/s41591-022-01772-9
  25. Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B, et al. TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385. doi:10.1136/bmj-2023-078378
  26. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, et al. Beyond adoption: A new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19(11). doi:10.2196/jmir.8775
  27. Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17(1):75. doi:10.1186/s13012-022-01245-0
  28. Sendak MP, Ratliff W, Sarro D, Alderton E, Futoma J, Gao M, et al. Real-world integration of a sepsis deep learning technology into routine clinical care: Implementation study. JMIR Med Inform. 2020;8(7). doi:10.2196/15182
  29. Hogue SC, Chen F, Brassard G, Lebel D, Bussières JF, Durand A, et al. Pharmacists’ perceptions of a machine learning model for the identification of atypical medication orders. J Am Med Inform Assoc. 2021;28(8):1712-18. doi:10.1093/jamia/ocab071
  30. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-53. doi:10.1126/science.aax2342
  31. Seyyed-Kalantari L, Zhang H, McDermott MBA, Chen IY, Ghassemi M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat Med. 2021;27(12):2176-82. doi:10.1038/s41591-021-01595-0
  32. Omiye JA, Lester JC, Spichak S, Rotemberg V, Daneshjou R. Large language models propagate race-based medicine. NPJ Digit Med. 2023;6(1):195. doi:10.1038/s41746-023-00939-z
  33. Rajkomar A, Hardt M, Howell MD, Corrado G, Chin MH. Ensuring fairness in machine learning to advance health equity. Ann Intern Med. 2018;169(12):866-72. doi:10.7326/M18-1990

 

 

 


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.