Medical Device Modeling: A Case Study in Building Credibility With Limited Reproducibility
Paulina Rodriguez, George Washington University
Computational modeling and simulation (CM&S) continues to show promise for risk-informed decision-making. However, trust in the medical device sector remains limited due to the complexity and inherent device uncertainties, reliability of CM&S, and non-standardized credibility assessments. This study aims to advance the credibility of CM&S with a case-study focusing on a multi-validation metric approach and a reproducible credibility workflow. We developed a CM&S for an electronic drug delivery system where coils heat fluid passing through the device and into a user's mouth cavity. A finite volume discretization of both the Navier-Stokes and energy equations were implemented to model fluid flow and conjugate heat transfer. The verification analysis included a Grid Convergence Index calculation and observed order of accuracy, achieving errors with a magnitude of 1e-5. A multi-validation metric analysis for small sample sizes (three experiments and 10 CM&S) used probabilistic metrics, including the area metric, confidence interval, and tolerance interval, implemented with a custom Python script and continuous integration testing. This revealed that the largest credibility errors stem from validation. A GitHub repository transparently captured the verification, validation, and uncertainty quantification studies. Despite the limited model reproducibility due to the proprietary nature of the commercial solver, we ensured the reliability of the credibility study results. We plan to replicate this study with an open-source solver, to achieve complete reproducibility, while increasing the statistical rigor of the validation studies with large CM&S samples using HPC parallelization. These efforts provide an approach to building credible and reliable CM&S for medical devices, enhancing their potential to inform regulatory decision-making.