Archive \ Volume.16 2025 Issue 2

Why Better Predictions Can Still Produce Worse Pharmacy Decisions

, ,
  1. Department of Pharmacy Decision Science and AI, Faculty of Pharmacy, University of Edinburgh, Edinburgh, United Kingdom.
  2. Department of Predictive Analytics and Decision Quality, Faculty of Pharmaceutical Sciences, Utrecht University, Utrecht, Netherlands

Abstract

Artificial intelligence in pharmacy is frequently evaluated through prediction-centred measures, including discrimination, calibration, sensitivity, specificity, and alert reduction. These measures are necessary for characterizing model behaviour but do not establish whether the model improves medication-related decisions. A prediction becomes consequential only when it is interpreted by a professional, connected to an available action, introduced at an appropriate time, and applied within a workflow in which errors have asymmetric consequences. This article critically examines the prediction-to-decision translation gap: the possibility that technically improved predictions may produce unchanged or worse pharmacy decisions. Six mechanisms are distinguished: target mismatch, automation bias, treatment leakage, poor timing, actionability failure, and unequal error consequences. Decision usefulness is proposed as a conditional property arising from alignment among the prediction target, patient and medication context, uncertainty communication, professional interpretation, action feasibility, and the consequences of acting or not acting. The article develops a proposed Prediction–Interpretation–Action Framework that separates model output from professional interpretation and medication action through three translation gates: target validity, interpretation integrity, and action suitability. Evaluation should therefore extend beyond model performance to include human–AI interaction, decision consequences, workflow burden, medication safety, subgroup effects, governance, and post-implementation change. The framework is an original conceptual synthesis rather than a validated clinical model, procurement standard, or regulatory instrument. Its propositions require prospective testing across pharmacy settings, medication tasks, professional roles, patient populations, and technology configurations before decision readiness can be inferred.


Downloads: 24
Views: 115

How to cite:
Vancouver
Wilson E, De Jong F, Peters L. Why Better Predictions Can Still Produce Worse Pharmacy Decisions. Arch Pharm Pract. 2025;16(2):44-52. https://doi.org/10.51847/699h8jFRoX
APA
Wilson, E., De Jong, F., & Peters, L. (2025). Why Better Predictions Can Still Produce Worse Pharmacy Decisions. Archives of Pharmacy Practice, 16(2), 44-52. https://doi.org/10.51847/699h8jFRoX

Download Citation
References
  1. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 2019;17(1):195. doi:10.1186/s12916-019-1426-2
  2. Wiens J, Saria S, Sendak M, Ghassemi M, Liu VX, Doshi-Velez F, et al. Do no harm: A roadmap for responsible machine learning for health care. Nat Med. 2019;25(9):1337-40. doi:10.1038/s41591-019-0548-6
  3. Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW; Topic Group “Evaluating Diagnostic Tests and Prediction Models” of the STRATOS initiative. Calibration: The Achilles heel of predictive analytics. BMC Med. 2019;17(1):230. doi:10.1186/s12916-019-1466-7
  4. Graafsma J, Murphy RM, van de Garde EMW, Karapinar-Çarkit F, Derijks HJ, Hoge RHL, et al. The use of artificial intelligence to optimize medication alerts generated by clinical decision support systems: A scoping review. J Am Med Inform Assoc. 2024;31(6):1411-22. doi:10.1093/jamia/ocae076
  5. Wong A, Otles E, Donnelly JP, Krumm A, McCullough J, DeTroyer-Cooley O, et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med. 2021;181(8):1065-70. doi:10.1001/jamainternmed.2021.2626
  6. Futoma J, Simons M, Panch T, Doshi-Velez F, Celi LA. The myth of generalisability in clinical research and machine learning in health care. Lancet Digit Health. 2020;2(9). doi:10.1016/S2589-7500(20)30186-2
  7. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med. 2018;15(11). doi:10.1371/journal.pmed.1002683
  8. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: Benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. doi:10.1038/s41746-020-0221-y
  9. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-53. doi:10.1126/science.aax2342
  10. Lyell D, Coiera E. Automation bias and verification complexity: A systematic review. J Am Med Inform Assoc. 2017;24(2):423-31. doi:10.1093/jamia/ocw105
  11. Davis SE, Matheny ME, Balu S, Sendak MP. A framework for understanding label leakage in machine learning for health care. J Am Med Inform Assoc. 2024;31(1):274-80. doi:10.1093/jamia/ocad178
  12. Jabbour S, Fouhey D, Shepard S, Valley TS, Kazerooni EA, Banovic N, et al. Measuring the impact of AI in the diagnosis of hospitalized patients: A randomized clinical vignette survey study. JAMA. 2023;330(23):2275-84. doi:10.1001/jama.2023.22295
  13. Kompa B, Snoek J, Beam AL. Second opinion needed: Communicating uncertainty in medical machine learning. NPJ Digit Med. 2021;4(1):4. doi:10.1038/s41746-020-00367-3
  14. Ghassemi M, Oakden-Rayner L, Beam AL. The false hope of current approaches to explainable artificial intelligence in health care. Lancet Digit Health. 2021;3(11). doi:10.1016/S2589-7500(21)00208-9
  15. Asan O, Bayrak AE, Choudhury A. Artificial intelligence and human trust in healthcare: Focus on clinicians. J Med Internet Res. 2020;22(6). doi:10.2196/15154
  16. Shortliffe EH, Sepulveda MJ. Clinical decision support in the era of artificial intelligence. JAMA. 2018;320(21):2199-200. doi:10.1001/jama.2018.17163
  17. Tschandl P, Rinner C, Apalla Z, Argenziano G, Codella N, Halpern A, et al. Human-computer collaboration for skin cancer recognition. Nat Med. 2020;26(8):1229-34. doi:10.1038/s41591-020-0942-0
  18. Grote T, Berens P. On the ethics of algorithmic decision-making in healthcare. J Med Ethics. 2020;46(3):205-11. doi:10.1136/medethics-2019-105586
  19. Van Calster B, Wynants L, Verbeek JFM, Verbakel JY, Christodoulou E, Vickers AJ, et al. Reporting and interpreting decision curve analysis: A guide for investigators. Eur Urol. 2018;74(6):796-804. doi:10.1016/j.eururo.2018.08.038
  20. Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med. 2022;28(5):924-33. doi:10.1038/s41591-022-01772-9
  21. Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: A tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. 2019;170(1):51-8. doi:10.7326/M18-1376
  22. Reddy S, Allan S, Coghlan S, Cooper P. A governance model for the application of AI in health care. J Am Med Inform Assoc. 2020;27(3):491-7. doi:10.1093/jamia/ocz192
  23. Labkoff S, Oladimeji B, Kannry J, Solomonides A, Leftwich R, Koski E, et al. Toward a responsible future: Recommendations for AI-enabled clinical decision support. J Am Med Inform Assoc. 2024;31(11):2730-9. doi:10.1093/jamia/ocae209
  24. Bauer J, Busse M, Koch S, Schmid M, Sommer J, Fromm MF, et al. Clinical-pharmaceutical assessment of medication CDSS alerts: Content appropriateness and patient relevance in clinical practice. Front Pharmacol. 2025;16:1510425. doi:10.3389/fphar.2025.1510425
  25. Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, et al. Artificial intelligence versus clinicians: Systematic review of design, reporting standards, and claims of deep learning studies. BMJ. 2020;368. doi:10.1136/bmj.m689
  26. Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B, et al. TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385. doi:10.1136/bmj-2023-078378
  27. Seyyed-Kalantari L, Zhang H, McDermott MBA, Chen IY, Ghassemi M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat Med. 2021;27(12):2176-82. doi:10.1038/s41591-021-01595-0

 

 


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.