When a healthcare center develops its own medical AI models, it assumes long-term responsibility for ensuring that performance remains consistent over time. This session will explore best practices for pre-deployment validation, post-deployment monitoring, and strategies to bridge the gap between theoretical AI performance and real-world clinical outcomes. We will discuss how rigorous local validation can uncover potential diagnostic improvements before deployment and how ongoing performance tracking can help mitigate biases, optimize radiologist-AI collaboration, and ensure sustainable AI impact. Through case studies from a large radiology practice, we will share insights on key performance indicators (KPIs), monitoring methods, and lessons learned in scaling AI in clinical practice. 

Objectives: 

• Describe key performance metrics (KPIs) for tracking AI model performance over time.
• Compare pre-deployment Enhanced Detection Rate (EDR) predictions with real-world clinical outcomes.
• Identify strategies for addressing performance deviations in AI-assisted radiology.