Why Pharmacy AI Evaluation Must Begin with Plausible Harm Scenarios

Artificial intelligence evaluation in pharmacy commonly begins with model-centred measures such as discrimination, sensitivity, specificity, calibration, alert acceptance, or processing efficiency. These measures are necessary, but aggregate performance can obscure how errors affect particular patients, professionals, medication-use tasks, and organizational conditions. This article develops an original, non-empirical Plausible-Harm Scenario Framework for Pharmacy AI Evaluation. The proposed approach begins by specifying who may be affected, which medication-use task is involved, how an AI-related failure could propagate through data, model output, interface presentation, professional interpretation, and action or inaction, and what medication-related consequence could follow. Each scenario is then examined through five distinct dimensions: severity, exposure, detectability, reversibility, and recovery. These dimensions are not combined into a numerical score. Instead, they organize the selection of technical, clinical-task, human–AI, workflow, medication-safety, equity, implementation, and lifecycle evidence. The framework further proposes governance gates for scenario completeness, evidence adequacy, residual-risk deliberation, bounded permission, monitoring, rollback, and requalification. Its principal contribution is to reposition performance metrics as scenario-dependent evidence rather than sufficient indicators of safety or readiness. Validation would require prospective and post-deployment testing across relevant users, settings, populations, workflows, model versions, and failure conditions. The framework cannot guarantee complete hazard discovery, establish universal thresholds, replace professional judgment, or demonstrate clinical benefit. It is intended as a conceptual structure for making the consequences and evidentiary assumptions of pharmacy AI evaluation more explicit.