Archive \ Volume.16 2025 Issue 2

Governing Artificial Intelligence after Deployment in the Medication-Use System

, ,
  1. Department of AI Governance and Medication Safety, Faculty of Pharmacy, University of Ghana, Accra, Ghana.
  2. Department of Post-Deployment AI Oversight, Faculty of Pharmacy, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana.

Abstract

Artificial intelligence is increasingly embedded within medication-related decision support, prescription review, alerting, prioritization, pharmacovigilance, and operational workflows. Deployment, however, does not stabilize the data environment, clinical task, professional response, patient population, organizational context, or relationship between model output and medication decisions. Postdeployment governance must therefore extend beyond periodic technical performance checks. This article develops a proposed postdeployment-governance architecture for artificial intelligence used within medication-use systems. The architecture defines the governed object as the complete AI-enabled arrangement, including its model, data pipelines, interfaces, users, workflow position, organizational policies, external dependencies, and intended-use boundaries. It connects versioned baseline evidence with multidomain surveillance of performance, drift, equity, human–AI interaction, workflow consequences, medication incidents, pharmacovigilance signals, and patient experience. Detected signals enter structured triage and investigation before proportionate decisions concerning continued use, intensified monitoring, workflow correction, model modification, revalidation, restriction, suspension, rollback, or withdrawal. Accountability functions assign responsibility for evidence review, decision authority, communication, alternative workflow activation, and learning closure. Validation would require longitudinal and multisite assessment of technical, clinical, sociotechnical, safety, equity, organizational, and patient-relevant outcomes. The architecture does not establish universal thresholds, causal attribution rules, regulatory acceptability, or improved medication outcomes. Its original contribution is an integrated and testable governance structure that treats deployment as the beginning of continuing institutional responsibility rather than the endpoint of model development.


Downloads: 31
Views: 140

How to cite:
Vancouver
Osei D, Afriyie A, Adu K. Governing Artificial Intelligence after Deployment in the Medication-Use System. Arch Pharm Pract. 2025;16(2):62-71. https://doi.org/10.51847/hPZnbLGKmC
APA
Osei, D., Afriyie, A., & Adu, K. (2025). Governing Artificial Intelligence after Deployment in the Medication-Use System. Archives of Pharmacy Practice, 16(2), 62-71. https://doi.org/10.51847/hPZnbLGKmC

Download Citation
References
  1. Wiens J, Saria S, Sendak M, Ghassemi M, Liu VX, Doshi-Velez F, et al. Do no harm: A roadmap for responsible machine learning for health care. Nat Med. 2019;25(9):1337-40. doi:10.1038/s41591-019-0548-6
  2. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 2019;17(1):195. doi:10.1186/s12916-019-1426-2
  3. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: Benefits, risks, and strategies for success. NPJ Digit Med. 2020;3(1):17. doi:10.1038/s41746-020-0221-y
  4. Vollmer S, Mateen BA, Bohner G, Király FJ, Ghani R, Jonsson P, et al. Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ. 2020;368. doi:10.1136/bmj.l6927
  5. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, et al. Beyond adoption: A new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19(11). doi:10.2196/jmir.8775
  6. Graafsma J, Murphy RM, van de Garde EMW, Karapinar-Çarkit F, Derijks HJ, Hoge RHL, et al. The use of artificial intelligence to optimize medication alerts generated by clinical decision support systems: A scoping review. J Am Med Inform Assoc. 2024;31(6):1411-22. doi:10.1093/jamia/ocae076
  7. Corny J, Rajkumar A, Martin O, Dode X, Lajonchère JP, Billuart O, et al. A machine learning-based clinical decision support system to identify prescriptions with a high risk of medication error. J Am Med Inform Assoc. 2020;27(11):1688-94. doi:10.1093/jamia/ocaa154
  8. Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak. 2017;17(1):36. doi:10.1186/s12911-017-0430-8
  9. Finlayson SG, Subbaswamy A, Singh K, Bowers J, Kupke A, Zittrain J, et al. The clinician and dataset shift in artificial intelligence. N Engl J Med. 2021;385(3):283-6. doi:10.1056/NEJMc2104626
  10. Gaube S, Suresh H, Raue M, Merritt A, Berkowitz SJ, Lermer E, et al. Do as AI say: Susceptibility in deployment of clinical decision-aids. NPJ Digit Med. 2021;4(1):31. doi:10.1038/s41746-021-00385-9
  11. Embi PJ. Algorithmovigilance—advancing methods to analyze and monitor artificial intelligence-driven health care for effectiveness and equity. JAMA Netw Open. 2021;4(4). doi:10.1001/jamanetworkopen.2021.4622
  12. Davis SE, Embi PJ, Matheny ME. Sustainable deployment of clinical prediction tools—a 360° approach to model maintenance. J Am Med Inform Assoc. 2024;31(5):1195-8. doi:10.1093/jamia/ocae036
  13. Labkoff S, Oladimeji B, Kannry J, Solomonides A, Leftwich R, Koski E, et al. Toward a responsible future: Recommendations for AI-enabled clinical decision support. J Am Med Inform Assoc. 2024;31(11):2730-9. doi:10.1093/jamia/ocae209
  14. Cutillo CM, Sharma KR, Foschini L, Kundu S, Mackintosh M, Mandl KD, et al. Machine intelligence in healthcare—perspectives on trustworthiness, explainability, usability, and transparency. NPJ Digit Med. 2020;3(1):47. doi:10.1038/s41746-020-0254-2
  15. Reps JM, Schuemie MJ, Suchard MA, Ryan PB, Rijnbeek PR. Design and implementation of a standardized framework to generate and evaluate patient-level prediction models using observational healthcare data. J Am Med Inform Assoc. 2018;25(8):969-75. doi:10.1093/jamia/ocy032
  16. Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Calibration: The Achilles heel of predictive analytics. BMC Med. 2019;17(1):230. doi:10.1186/s12916-019-1466-7
  17. Kore A, Abbasi Bavil E, Subasri V, Abdalla M, Fine B, Dolatabadi E, et al. Empirical data drift detection experiments on real-world medical imaging data. Nat Commun. 2024;15(1):1887. doi:10.1038/s41467-024-46142-w
  18. Davis SE, Dorn C, Park DJ, Matheny ME. Emerging algorithmic bias: Fairness drift as the next dimension of model maintenance and sustainability. J Am Med Inform Assoc. 2025;32(5):845-54. doi:10.1093/jamia/ocaf039
  19. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-53. doi:10.1126/science.aax2342
  20. Kassekert R, Grabowski N, Lorenz D, Schaffer C, Kempf D, Roy P, et al. Industry perspective on artificial intelligence/machine learning in pharmacovigilance. Drug Saf. 2022;45(5):439-48. doi:10.1007/s40264-022-01164-5
  21. Feng J, Phillips RV, Malenica I, Bishara A, Hubbard AE, Celi LA, et al. Clinical artificial intelligence quality improvement: Towards continual monitoring and updating of AI algorithms in healthcare. NPJ Digit Med. 2022;5(1):66. doi:10.1038/s41746-022-00611-y
  22. Davis SE, Greevy RA Jr, Lasko TA, Walsh CG, Matheny ME. Detection of calibration drift in clinical prediction models to inform model updating. J Biomed Inform. 2020;112:103611. doi:10.1016/j.jbi.2020.103611
  23. Liu X, Glocker B, McCradden MM, Ghassemi M, Denniston AK, Oakden-Rayner L. The medical algorithmic audit. Lancet Digit Health. 2022;4(5). doi:10.1016/S2589-7500(22)00003-6
  24. Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B, et al. TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385. doi:10.1136/bmj-2023-078378
  25. Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med. 2022;28(5):924-33. doi:10.1038/s41591-022-01772-9
  26. Reddy S, Allan S, Coghlan S, Cooper P. A governance model for the application of AI in health care. J Am Med Inform Assoc. 2020;27(3):491-7. doi:10.1093/jamia/ocz192
  27. Morley J, Machado CCV, Burr C, Cowls J, Joshi I, Taddeo M, et al. The ethics of AI in health care: A mapping review. Soc Sci Med. 2020;260:113172. doi:10.1016/j.socscimed.2020.113172
  28. Mittelstadt B. Principles alone cannot guarantee ethical AI. Nat Mach Intell. 2019;1(11):501-7. doi:10.1038/s42256-019-0114-4
  29. Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17(1):75. doi:10.1186/s13012-022-01245-0
  30. Strohm L, Hehakaya C, Ranschaert ER, Boon WPC, Moors EHM. Implementation of artificial intelligence applications in radiology: Hindering and facilitating factors. Eur Radiol. 2020;30(10):5525-32. doi:10.1007/s00330-020-06946-y
  31. van der Vegt AH, Scott IA, Dermawan K, Schnetler RJ, Kalke VR, Lane T. Implementation frameworks for end-to-end clinical AI: Derivation of the SALIENT framework. J Am Med Inform Assoc. 2023;30(9):1503-15. doi:10.1093/jamia/ocad088
  32. Kamel Rahimi A, Pienaar O, Ghadimi M, Canfell OJ, Pole JD, Shrapnel S, et al. Implementing AI in hospitals to achieve a learning health system: Systematic review of current enablers and barriers. J Med Internet Res. 2024;26. doi:10.2196/49655
  33. Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, et al. Artificial intelligence versus clinicians: Systematic review of design, reporting standards, and claims of deep learning studies. BMJ. 2020;368. doi:10.1136/bmj.m689
  34. Vasey B, Ursprung S, Beddoe B, Taylor EH, Maruthappu M, Nagendran M, et al. Association of clinician diagnostic performance with machine learning-based decision support systems: A systematic review. JAMA Netw Open. 2021;4(3). doi:10.1001/jamanetworkopen.2021.1276
  35. Futoma J, Simons M, Panch T, Doshi-Velez F, Celi LA. The myth of generalisability in clinical research and machine learning in health care. Lancet Digit Health. 2020;2(9). doi:10.1016/S2589-7500(20)30186-2
  36. Ghassemi M, Oakden-Rayner L, Beam AL. The false hope of current approaches to explainable artificial intelligence in health care. Lancet Digit Health. 2021;3(11). doi:10.1016/S2589-7500(21)00208-9

 

 


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.