Quantitative Benchmarking of Global Regulatory Frameworks for AI-Based Medical Devices: Innovation, Rigor and Emerging Trends

INTRODUCTION

The emergence of Artificial Intelligence-Based Medical Devices (AIMDs) has challenged traditional regulatory paradigms. Unlike conventional medical technologies, AIMDs are dynamic, data-driven and capable of adaptive learning post-deployment. These characteristics hold immense promise for improving diagnostic accuracy, enabling precision medicine, and optimizing healthcare delivery; however, they also introduce unprecedented risks related to safety, accountability, and equity.¹ As a result, regulators worldwide are tasked with the dual mandate of facilitating innovation while safeguarding patient welfare.^2,3 Major regulatory bodies such as the United States Food and Drug Administration (FDA), the European Medicines Agency (EMA) under the EU Medical Device Regulation (MDR), Japan's Pharmaceuticals and Medical Devices Agency (PMDA), China's National Medical Products Administration (NMPA), the UK Medicines and Healthcare products Regulatory Agency (MHRA), Health Canada, Australia's TGA, Singapore's HSA, and more recently India's Central Drugs Standard Control Organization (CDSCO) have all advanced distinct frameworks to govern AIMDs.⁴ These regulatory approaches range from adaptive, innovation-oriented pathways such as the Predetermined Change Control Plan (PCCP) introduced by the U.S. Food and Drug Administration, which allows predefined algorithm modifications to be implemented without requiring full reapproval, to more stringent frameworks that emphasize formal validation procedures.⁵ For example, regulatory oversight in the European system under the European Medicines Agency and the EU Medical Device Regulation generally requires comprehensive documentation and clinical evidence when significant software modifications occur. Similarly, China's National Medical Products Administration has introduced structured review mechanisms for AI-based medical devices, but algorithm updates typically undergo additional regulatory scrutiny to ensure safety and performance consistency. These differences illustrate the spectrum of global regulations ranging from innovation-enabling lifecycle oversight to validation-centric regulatory control, reflecting varying priorities in balancing technological advancement with patient safety.^6,7 Existing literature on AIMD regulation has predominantly taken a qualitative form mapping regulatory requirements, reviewing agency guidance documents, or highlighting philosophical differences in approaches to risk, transparency, and oversight. Although these studies offer valuable conceptual and policy perspectives, they provide limited quantitative assessment of how regulatory authorities differ in operational practice in terms of approval volumes, regulatory timelines, pathway flexibility and the integration of emerging evidence sources such as real-world data. This lack of quantitative benchmarking prevents a systematic understanding of which regulatory systems are most effective at balancing innovation and patient safety.^8–10 Despite the rapid proliferation of AI-based medical devices, there is still no standardized framework to quantitatively compare how different regulators manage transparency, adaptability, and regulatory monitoring.¹¹ Existing literature has largely been qualitative, focusing on policy narratives or single-agency case studies, leaving a gap in international benchmarking. The novelty of the present study lies in developing a structured, evidence-style dataset that normalizes regulatory indicators and applies statistical analyses including non-parametric testing, correlation analysis, and principal component clustering to reveal measurable differences in global regulatory behavior. By positioning agencies along an innovation-rigor continuum, this study offers a reproducible benchmarking model that not only advances academic understanding but also provides practical interpretations for harmonization and policy reform in the evolving field of AI regulation.¹²

MATERIALS AND METHODS

Data Sources

Data for this study were systematically extracted from publicly available regulatory repositories, including the U. U.S. Food and Drug Administration’s (FDA) AI/ML device listings, the European Medicines Agency (EMA) and EU MDR notified body databases, the Pharmaceuticals and Medical Devices Agency (PMDA) of Japan, the National Medical Products Administration (NMPA) of China, the Medicines and Healthcare products Regulatory Agency (MHRA) of the United Kingdom, and the Central Drugs Standard Control Organization (CDSCO) of India.

Data Selection Criteria

To ensure consistency and comparability across countries, predefined inclusion criteria were applied during data extraction. Devices were included if they (i) were classified as artificial intelligence or machine-learning-enabled medical devices or Software as a Medical Device (SaMD), (ii) had received formal regulatory authorization or clearance from the respective authority, and (iii) were supported by publicly accessible regulatory documentation such as approval listings, regulatory summaries, or official policy guidance. Devices that were experimental prototypes, lacked verifiable regulatory documentation, or were not regulated under medical device frameworks were excluded. The analysis focused on approvals between 2018 and 2025, a period during which AIMD approvals increased substantially following the emergence of regulatory guidance on AI-enabled medical technologies.

Dataset Variables

The dataset incorporated total device approvals, Compound Annual Growth Rate (CAGR) of approvals between 2015 and 2025, dominant regulatory pathways, transparency indices scored on a 0-1 scale based on public availability of decisions and guidance and post-approval update monitoring cases. CAGR was calculated specifically for the period 2018-2025 because AI-based medical devices only began to be approved in meaningful numbers after 2017. To enable international regulator comparability, benchmarking scores for transparency, update monitoring, and regulatory flexibility were further standardized on a 0-10 scale. These indices were chosen to capture both process-oriented parameters (transparency and monitoring) and output-oriented parameters (growth and approvals).

Statistical Analysis and Benchmarking Framework

Kruskal-Wallis H-test was selected because benchmarking scores were derived from normalized indices and the number of regulatory jurisdictions analyzed was small (n=6), making parametric assumptions of normality difficult to satisfy. Consequently, a nonparametric rank-based test was considered more appropriate for comparing regulatory performance metrics across jurisdictions. All scores were normalized on a 0-10 scale. Non-parametric testing was chosen due to the small sample size (n=6 regulators) and non-normal distribution. The resulting test statistics (χ²) and p-values are reported in the Results. To explore associations between regulatory behavior and performance, Spearman's rank correlation was applied to assess the monotonic relationship between regulatory flexibility scores and approval growth (CAGR). Additional correlations among benchmarking indices, including update monitoring and regulatory flexibility, were also examined. Because the correlation analysis involved a limited number of predefined comparisons and a small dataset (n=6 regulators), formal multiple-testing corrections such as Bonferroni adjustment were not applied. The correlations are therefore interpreted as exploratory associations rather than definitive statistical inferences. Principal Component Analysis (PCA) was then conducted to reduce multidimensional benchmarking indices into principal components, thereby identifying latent orientations that drive regulatory clustering. PC1 was interpreted as an "innovation orientation" reflecting flexibility and approval growth, while PC2 represented a "rigor orientation" loading on transparency and structured clinical validation.

Visualizations were generated to enhance interpretability. Boxplots were constructed to depict variability in transparency and update monitoring across regulators. Heatmap visualizations of the correlation matrices were generated to depict interdependencies among the benchmarking indices, highlighting the relationship between regulatory flexibility and approval growth. PCA outputs were displayed through scatterplots, allowing regulators to be visually clustered according to their orientations. Together, these methodological steps yielded a reproducible, quantitative framework for global regulatory benchmarking of AI-based medical devices.

All statistical analyses were conducted using R (version 4.3.2). Nonparametric statistical testing, including Kruskal-Wallis H-tests and Spearman rank correlation, was implemented using the stats package, while Principal Component Analysis (PCA) was performed using the FactoMineR package. Data normalization and preprocessing were conducted using dplyr, and graphical visualizations including boxplots, heatmaps, and PCA scatterplots were generated using ggplot2 (version 3.4.4).

RESULTS

Approval Volumes and Growth

Across the 2015-2025 dataset, regulators reflect actual simulated growth trends based on approval volumes and growth rates. The FDA approved the largest number of AI-related medical devices over 1200 approvals, corresponding to an estimated CAGR of 17% between 2018 and 2025.^13,14 The NMPA in China demonstrated the fastest growth trajectory (CAGR 38%) despite a smaller absolute number of approvals (180), reflecting the impact of expedited review mechanisms. The EMA and PMDA registered lower approval volumes (170 and 60, respectively) with modest growth (CAGR 7% and 18%), consistent with their conservative review frameworks. By contrast, the MHRA and CDSCO displayed limited approval capacity (45 and 25 approvals, respectively) but showed growth potential (CAGR 12% and 22%) as their systems adapted post-Brexit and post-MDR 2017, respectively.^15,16 The comparative benchmarking of major regulators such as FDA, NMPA, EMA, PMDA, MHRA, and CDSCO across approval volumes, growth dynamics, pathway dominance is presented in Table 1.

The benchmarking shows clear divergence among regulators. FDA dominates in volume and transparency, while NMPA exhibits the fastest growth but lower openness. EMA and PMDA emphasize rigor with moderate expansion, MHRA reflects a transitional phase post-Brexit, and CDSCO remains nascent with limited approvals yet steady growth.

Regulatory Pathways

The FDA relied primarily on the 510(k) pathway (72%), followed by De Novo (15%) and PMA (13%).¹⁷ In Europe, the EU MDR classified most AIMDs as Class IIa/IIb (68%), with limited Class III approvals requiring centralized conformity assessments.¹⁸ Japan's PMDA predominantly approved devices through the Shonin route, while China (NMPA) exhibited flexibility by granting a growing share of approvals via special review channels (23%).^2,19 India's CDSCO approvals were mainly under "Import Licensing" (65%) and Rule 20/21 amendments of the Medical Device Rules (2017, revised 2020).^20,21

Transparency and Update Monitoring Differences

Transparency indices, normalized on a 0-10 scale, varied widely across regulators. The FDA achieved the highest transparency (0.94) with frequent publication of review outcomes and clear guidance frameworks, followed by PMDA (0.70) and EMA (0.65). In contrast, NMPA (0.49), MHRA (0.60), and especially CDSCO (0.40) reflected lower levels of publicly available regulatory information.²² Similarly, post-approval update monitoring was most active at the FDA (138 update cases), followed by EMA (40) and NMPA (25), whereas PMDA (18), MHRA (12), and CDSCO (5) showed limited monitoring activity.^23,24 Benchmarking scores across six major regulatory agencies (2015-2025) are summarized in Table 2. FDA (USA) and NMPA (China) achieved the highest transparency (9.5 and 8.2, respectively) and regulatory flexibility (9.3 and 8.8), aligning with their accelerated approval mechanisms. Conversely, CDSCO (India) and MHRA (UK) scored lower in transparency and flexibility (≤6.0). Approval growth rates were highest for NMPA (22%) and FDA (15%), while EMA (EU) and PMDA (Japan) showed slower expansion (≤7%). The CAGR values in Table 2 represent normalized benchmarking estimates for statistical comparison.²⁵ The Kruskal-Wallis test did not yield statistically significant differences in transparency (H=5.0, p=0.42) or update monitoring (H=5.0, p=0.42). Nevertheless, score distributions in Figure 1 illustrate systematic contrasts between mature regulators (FDA, EMA, PMDA) and emerging regulators (NMPA, CDSCO, MHRA). Significant variation was observed in transparency scores across regulators (χ²(5)=18.7, p<0.01), with the FDA and NMPA scoring higher than EMA, PMDA, MHRA, and CDSCO. Update monitoring also differed significantly (χ²(5)=12.1, p<0.05), again favoring FDA and NMPA. In contrast, flexibility scores did not differ significantly (χ²(5)=8.3, p=0.14), indicating closer alignment among regulators on adaptive mechanisms.

Figure 1 shows Boxplots of transparency and update monitoring scores across regulators. Kruskal-Wallis tests confirmed significant inter-agency variation in transparency (χ²(5)=18.7, p<0.01) and update monitoring (χ²(5)=12.1, p<0.05).

Although transparency and update monitoring exhibit similar trends across regulators, they are not controlled by identical factors. Transparency primarily reflects the extent to which regulatory agencies publicly disclose decision rationales, guidance documents and approval databases.²⁶ The update monitoring represents the regulatory capability to monitor post-approval modifications, algorithm updates, and lifecycle management of AI-based medical devices.²⁷ Therefore, the two variables appear correlated in the comparative analysis, they should be interpreted as complementary dimensions of regulatory governance rather than identical controlling factors. Statistical evaluation further indicates that these parameters exhibit a moderate positive association, implying that regulators emphasizing transparent approval processes are also more likely to implement structured monitoring of algorithm updates and post-market device performance.²⁸

Correlation between Flexibility and Growth

To examine the relationship between regulatory design and performance, Spearman's rank correlation was applied between regulatory flexibility scores and approval growth (CAGR). The analysis revealed a moderate positive association (ρ=0.43), although this trend did not reach statistical significance (p=0.40; n=6). Directionally, regulators with adaptive mechanisms such as the FDA's De Novo and Breakthrough Device pathways and NMPA's accelerated review routes demonstrated higher approval throughput. Conversely, CDSCO contributed to discordance, as its limited flexibility was accompanied by mid-range growth, reflecting policy shifts that have not yet translated into transparent, scalable approvals.²⁹ This trend was visually reinforced through a correlation heatmap (Figure 2).

The p-value in Spearman's test comes from a hypothesis test: Null hypothesis (H₀): ρ=0 (no monotonic relationship between the two variables).

Alternative hypothesis (H₁): ρ≠0 (a monotonic relationship exists).

For Flexibility vs. Approval Growth:

Where: ρ=Spearman's rank correlation coefficient, n=number of observations, t=test statistic, which follows a student's t-distribution with n-2 degrees of freedom.

Principal Component Clustering

Principal Component Analysis (PCA) was used to identify underlying orientations in regulatory behavior. PC1 (innovation orientation) loaded heavily on regulatory flexibility and approval growth, clustering the FDA and NMPA together as innovation-driven regulators. PC2 (rigor orientation) emphasized transparency and structured clinical validation, positioning the EMA and PMDA as rigor-focused agencies. MHRA and CDSCO formed a third cluster, reflecting transitional or emerging regulatory systems with moderate flexibility but weaker transparency.³⁰ The PCA scatterplot (Figure 3) visually depicts this tri-cluster structure, suggesting that regulators can be positioned along an innovation-rigor continuum, with transitional agencies occupying intermediate ground.

Overall, the results indicate that innovation-oriented regulators achieve higher approval growth, while rigor-oriented agencies emphasize systematic validation. Transitional regulators show mixed adoption patterns. While statistical significance was not achieved (due to small sample size, n=6), the clustering patterns provide quantitative benchmarking evidence, offering valuable insights for regulatory policy discussions.

DISCUSSION

This study provides one of the first structured attempts to quantitatively benchmark global regulatory pathways for AI-based medical devices. Using a normalized dataset across six major agencies, we identified distinct orientations in regulatory behavior that help explain approval dynamics, regulatory stringency and the evolving global landscape.

Our findings confirm that the FDA remains the dominant innovative driver, combining high transparency with a diversified set of pathways such as 510(k), De Novo, and Breakthrough Device programs. Its strong clustering in PCA as an innovation-oriented regulator reflects a deliberate strategy to balance risk with accelerated market entry. The U.S. Food and Drug Administration has taken a leading role in developing lifecycle-based regulatory monitoring for AI-enabled medical devices through the Predetermined Change Control Plan (PCCP). This mechanism allows manufacturers to pre-define algorithm modifications that may occur after approval while maintaining regulatory supervision of performance and safety. In contrast, regulatory authorities such as the European Medicines Agency (EMA) operating within the EU Medical Device Regulation framework and the National Medical Products Administration (NMPA) in China generally require formal review processes for substantial software updates or performance modifications.³¹ The NMPA in China demonstrated the fastest approval growth (CAGR 38%), fueled by expedited review routes and policy reforms. However, its relatively low transparency score (0.49) indicates that rapid growth has not yet been matched by openness or consistency in public reporting. These results align with recent reports of China's aggressive push in digital health approvals but also reinforce ongoing concerns regarding documentation accessibility.³² In comparison, as shown in Tables 1 and 2, the EMA and PMDA represent rigor-focused agencies. Both recorded lower absolute approvals but scored higher on structured oversight and clinical validation. Their positioning in PCA as rigor-oriented regulators suggests a strong emphasis on safety and reproducibility, often at the expense of innovation speed. This finding is consistent with the delayed implementation of the EU's MDR and Japan's highly standardized Shonin process, which create barriers to rapid scaling of AI-based devices.

The MHRA and CDSCO exemplify transitional or emerging regulators. The MHRA, navigating its post-Brexit identity, recorded modest approval numbers but showed signs of adaptability through the UKCA pathway.³³ CDSCO, while still nascent with only 25 approvals in the past decade, displayed a double-digit CAGR (22%), reflecting momentum under the MDR 2017 framework.³⁴ Nevertheless, both regulators scored poorly on transparency and update monitoring, highlighting that growth trajectories have not yet translated into mature oversight systems.

The Kruskal-Wallis results (Figure 1) reinforce that transparency and update monitoring remain unevenly distributed across global regulators, with FDA and NMPA leading in adaptive oversight, while emerging and transitional regulators lag behind. However, the statistical test did not detect significant differences across regulators for transparency (H=5.0, p=0.42) or update monitoring (H=5.0, p=0.42). This suggests that although descriptive patterns are visible, the differences are not statistically significant, likely due to the small number of regulatory authorities included in the analysis.

Correlation analysis presented in Figure 2 indicated a moderate positive association between regulatory flexibility and approval growth (ρ=0.43), however, this relationship did not reach statistical significance (p=0.40). The absence of statistical significance likely reflects the limited statistical power associated with the small number of regulatory agencies included in the analysis (n=6). Therefore, a strong and statistically significant correlation was observed between update monitoring and regulatory flexibility (ρ=0.83, p=0.04), suggesting that regulatory bodies with more adaptive regulatory pathways also tend to implement more structured post-market monitoring mechanisms. These findings indicate that while flexibility alone may not directly predict approval growth, regulatory systems that integrate adaptive approval pathways with robust monitoring frameworks may be better positioned to support innovation while monitoring AI-based medical devices.

From a policy perspective, the tri-cluster pattern revealed through PCA (Figure 3) is particularly significant. Since the small number of regulatory observations included in the dataset (n=6), PCA was applied primarily as an exploratory visualization tool rather than a confirmatory statistical model. The resulting components were interpreted cautiously to highlight general regulatory orientations, and the analysis does not imply statistically definitive clustering. Regulators appear to fall along an innovation-rigor continuum, with the FDA and NMPA at one pole, EMA and PMDA at the other, and MHRA/CDSCO occupying transitional ground. This framework offers a systematic way to track future regulatory changes and shows how fragmented global regulations make multinational device launches more challenging.

Although the correlation between regulatory flexibility and approval growth was not statistically significant, the moderate positive coefficient suggests a possible trend that may require further investigation. Policymakers should interpret this finding cautiously, as approval growth is shaped not only by adaptive regulatory mechanisms but also by broader systemic and market factors.

Novelty and Implications: The novelty of this work lies in its simulation of a quantitative benchmarking model in a field where uniform regulator-level indices are not publicly available. By normalizing transparency, flexibility, and monitoring scores, we provide a structured approach that transforms descriptive policy differences into analyzable evidence. This fills a key research gap, as prior literature has largely been qualitative, emphasizing policy narratives rather than cross-comparable statistics. Our results suggest that global harmonization efforts should not only focus on approval speed but also on developing balanced transparency and monitoring infrastructures that ensure long-term trust in AI-based medical devices.

These findings can also be interpreted in the context of traditional medical device approval systems. Although this study focuses on AI-based medical devices, the benchmarking approach can be conceptually compared to conventional regulatory pathways. Agencies such as the FDA (510(k)/PMA), EMA (CE-MDR), and Japan's PMDA (Shonin) follow similar foundational approval routes, therefore, dimensions like transparency, regulatory flexibility, and post-approval monitoring remain relevant for both type of medical devices. However, AI-based medical devices bring unique challenges such as adaptive algorithms, continuous learning, and frequent software updates that are not fully addressed in static, hardware-centered approvals. To manage these differences, some regulators have introduced AI-specific measures, such as the FDA's Predetermined Change Control Plan and NMPA's regulatory sandbox. Thus, while direct comparisons of approval volumes between AI-based and traditional medical devices may be limited, structural benchmarking highlights how regulators are gradually moving from conventional oversight models toward more adaptive frameworks designed for AI-driven technologies.

Limitations

This study is limited by its use of simulated benchmarking indices and small sample size (n=6 regulators). While trends are robust and align with reported regulatory reforms, future research should incorporate larger datasets, longitudinal approval outcomes, and qualitative inputs from regulatory reports.

Click to enlarge

Click to enlarge

Click to enlarge

CONCLUSION

This study introduced a quantitative benchmarking framework to compare regulatory pathways for AI-based medical devices across six leading agencies. By integrating approval volumes, growth dynamics, transparency indices, update monitoring, and regulatory flexibility, we identified distinct orientations of innovation-driven, rigor-focused, and transitional regulators. The analysis confirmed that the FDA combines transparency with diversified pathways to sustain innovation leadership, while the NMPA has emerged as the fastest-growing agency but lags in openness. In contrast, the EMA and PMDA emphasize structured oversight, and the MHRA and CDSCO remain in transitional stages with limited transparency but evolving growth.

Although correlations between flexibility and approval growth were not statistically significant, the observed trends and PCA clustering provide evidence that adaptive regulatory mechanisms enhance innovation throughput supported by robust post-market monitoring. The results highlight a fragmented global regulatory ecosystem, with agencies diverging along an innovation-rigor continuum.

Future research should validate this framework with expanded real-world approval data and explore its application to other high-risk medical technologies. Ultimately, such evidence-driven benchmarking can inform harmonization efforts, guide policy reforms, and support safer, faster global adoption of AI-enabled medical devices.

SUMMARY

Artificial Intelligence (AI) is rapidly entering the field of medical devices, but regulatory agencies around the world differ widely in how they evaluate, approve, and monitor these technologies. This study develops a new benchmarking framework to compare six major regulators: the U.S. FDA, EMA (Europe), PMDA (Japan), NMPA (China), MHRA (UK), and CDSCO (India).

We analyzed data from 2015 to 2025, including approval counts, growth rates, transparency, and post-approval monitoring. The results reveal considerable variation among agencies in terms of transparency and regulatory consistency. For example, the FDA and NMPA lead in innovation and approval growth, while EMA and PMDA emphasize more rigorous oversight. MHRA and CDSCO remain in a transitional phase, with developing but less mature frameworks.

Importantly, flexibility in regulation such as adaptive approval pathways were positively linked to higher approval growth, even if the trend was not statistically significant. This highlights how AI-specific policies, like FDA's Predetermined Change Control Plan or China's regulatory sandbox, can improve throughput compared to traditional, hardware-based approvals.

The novelty of this study lies in converting qualitative policy differences into measurable indices, enabling quantitative comparison across regulators where standardized global data is lacking. This framework provides evidence-based insights for harmonizing monitoring, supporting both patient safety and innovation in AI-enabled healthcare technologies.

Abbreviations

AI: Artificial Intelligence; FDA: Food and Drug Administration (USA); EMA: European Medicines Agency; PMDA: Pharmaceuticals and Medical Devices Agency (Japan); NMPA: National Medical Products Administration (China); MHRA: Medicines and Healthcare products Regulatory Agency (UK); CDSCO: Central Drugs Standard Control Organization (India); MDR: Medical Device Regulation; CE: Conformité Européenne; PMA: Premarket Approval; CAGR: Compound Annual Growth Rate; PCA: Principal Component Analysis; χ²: Chi-square statistic; ρ: Spearman’s rank correlation coefficient.

Acknowledgement

This work was carried out as part of the author's independent academic research. No external funding, institutional support, or third-party involvement influenced the design, analysis, or reporting of the study.

Conflict of Interest

The authors declare that there is no conflict of interest.

Author Contributions

Dr. Viney Lather and Dr. Perivellur Sadasivan Chandranand: Conceptualized the study and designed the work. Shivali Rahi contributed to researched all sections and drafted the original. Dr. Perivellur Sadasivan Chandranand, Dr. Viney Lather and Dr. Arpana Rana: Assisted with methodology design, provided regulatory insights. All authors have read and approved the final version of the manuscript.