External Validation of AI-Aided Weaning Software Using Multicenter Retrospective Data

Using Multicenter Retrospective Data to Validate the Performance of AI-Aided Weaning Software

This multicenter retrospective study aims to externally validate an artificial intelligence-aided weaning software developed using intensive care unit data from Taichung Veterans General Hospital between 2015 and 2019. The model predicts the optimal timing for extubation using routinely collected clinical variables including ventilator parameters, physiologic measurements, and fluid and nutrition information. De-identified data from four hospitals collected between 2020 and 2024 will be used to evaluate model performance. Performance metrics include sensitivity, specificity, accuracy, area under the receiver operating characteristic curve (AUROC), and F1 score.

Critical care generates a large amount of digitized clinical data that may benefit from artificial intelligence-assisted decision support. The AI-Aided Weaning Software was previously developed using ICU data from Taichung Veterans General Hospital collected between 2015 and 2019.

This retrospective multicenter validation study will evaluate the external performance of the established model using independent datasets from four hospitals in Taiwan, including Taichung Veterans General Hospital, Mackay Memorial Hospital, Kaohsiung Medical University Chung-Ho Memorial Hospital, and Tungs' Taichung MetroHarbor Hospital.

The study population includes adult ICU patients with respiratory failure who received mechanical ventilation for at least 72 hours between January 2020 and December 2024. De-identified routine clinical records will be collected according to a predefined case report form and analyzed centrally.

The primary objective is to assess the external validity of the AI-Aided Weaning Software across different hospitals. Model performance will be evaluated using sensitivity, specificity, accuracy, AUROC, and F1 score.

Inclusion Criteria:

• Adult patients aged 20 years or older.
• Admitted to the intensive care unit (ICU) at one of the participating hospitals between January 1, 2020 and December 31, 2024.
• Received invasive mechanical ventilation for at least 72 hours.
• Availability of de-identified clinical data required for model validation.

Exclusion Criteria:

• Patients who did not receive invasive mechanical ventilation.
• Duration of mechanical ventilation less than 72 hours.
• Missing key clinical variables required for model validation.