Comparison of a Full Automatic Software With Standard Methods of Vessels Analysis

Retrospective Study of the Use of PRAEVAorta 2 Software for Vessel Geometric Analysis Versus Standard Analysis Methods in Patients With Abdominal Aortic Aneurysm

The treatment of aortic aneurysms is today based on different indicators (diameters, lengths, angles, volumes of the arteries) measured on CT scan images. Several indicators are time consuming and complicatated to measure. They demand training and practice. Nurea is developing a software for automatic measurement of these indicators, PRAEVAorta® 2, to facilitate and assist the physician in his clinical routine.

The purpose of this study is to compare the analysis realised by the software PRAEVAorta® 2 with the analysis realised by the healthcare professional on retrospective CT scan images.

Contrasted and non-contrasted, pre-operation or post-operation CT scans from 50 patients will be analysed.

The main objectif is to validate the accuracy of the software by demonstrating its adequacy to the standard method of analysis.

The second objectives are the following:

• Evaluate the security of the software PRAEVAorta® 2
• Evaluate the unanticipated risks related to the use of the software
• Validate the accessory PRAEVAorta® Web

We make the following assumption :

90% of the patients show 90% of adequacy to the healthcare professional analysis

The study PRAEVA-2-LP is realized in the University hospital of Leipzig. The treatment of aortic aneurysms is today based on different indicators (diameters, lengths, angles, volumes of the arteries) measured on CT scan images. Several indicators are time consuming and complicatated to measure. They demand training and practice. Nurea is developing a software for automatic measurement of these indicators, PRAEVAorta® 2, to facilitate and assist the physician in his clinical routine.

The purpose of this study is to compare the analysis realised by the software PRAEVAorta® 2 with the analysis realised by the healthcare professional on retrospective CT scan images.

Contrasted and non-contrasted, pre-operation or post-operation CT scans from 50 patients will be analysed.

We make the following assumption: 90% of the patients show 90% of adequacy to the healthcare professional analysis

This study is comparative, retrospective, non-interventional and monocentric. We compare the SaMD measurements to measurements realised using the standard method by a healthcare professional. The study analysis retrospective pseudonymised CT scans and deliver aggregated data. The study will be realised on a group of 50 subjects, which is small enough to have one site of investigation. Subjects will be their own witness. This strategy allows us to minimize the cost and the risks of the study without compromising the investigation. This investigation is submitted to the local ethics committee for approval according to paragraph 15 of Professional Code of the Saxon State Medical Association (Berufsordnung - BO) of 24 June 1998.

The study realised on the first version of the medical device PRAEVAorta® has shown a proportion of 93% of patient with measures calculated by the software in adequation with the measures realised following the standard method of analysis (ratio ≥ 90%). Because the new version of the software PRAEVAorta®2 has been improved, we expect to observe the same proportion.

Patient number calculation:

Hypotheses are the following:

H0: The proportion of patients with an absolute mean discrepancy > 5mm is 90% H1: The proportion of patient with an absolute mean discrepancy ≤ 5mm is strictly over 90% For this statistical analysis, variables are quantitative: aorta measurements. We consider over here the proportion of patients per sub-group who have as less or greater than 90%.

Comparison of two proportions: a theoretical proportion with an observed proportion The theoretical proportion ϕ0 is 90% The observed proportion ϕ is 93% (the observed proportion in the study of PRAEVAorta® 1) Therefore, the numbers of patients needed, with an α risk of 5% and a β risk of 0.9% is 32 in unilateral n=([1,96(φ0(1-φ0))^0,5 +u2β(φ(1-φ))^0,5]^2)/([φ0-φ]^2)=32 For security reasons and to anticipate any problem we choose to include approximately 10% more subjects. Moreover, to be sure that the amount of data is sufficient for CE marking we add another 15%. Therefore, the total of patient is 50.

The collected data comes from a database built by the hospital especially for retrospective study. The patient consent has already been collected at the creation of the database which exempt us from collecting it again.

The collected data are the gender, the birthdate of the patient, the scan manufacturer, the contrast-enhanced CT scan, the date of realization of the CT scan, the indicators, the healthcare professional trust in the results of the analysis realized by the software.

Images' analysis Pseudonymised images in DICOM format will be analysed. Pre-operation and post-operation thin-sliced CT images will be analysed.

Procedure to follow:

1. Inclusion and exclusion criteria validation
2. Demographic data and scanner references collection
3. CT scans (in DICOM format) collection
4. First Randomisation of the subjects' order
5. Determination of the measures by the physician with the standard method
6. Second randomisation of the subjects' order
7. Software analysis of the images with PRAEVAorta® 2 and Physician trust opinion In step 7, physicians will have to indicate if, based on the report generated by PRAEVAorta® 2, they trust the software measures or they do not, in order to validate clinical safety.

The comparison of data will be done during the statistical analysis.

The data will be collected in an EDC / e-CRF. The software allow authentication, account management with several level of access to the different parts and data. Exchanges are secured and data are stored on a Health Data Host accredited server. A back-up is regularly made.

The practician fills a form / report when measuring the indicators and the software PRAEVAorta® 2 is returning a pdf report. The data collected in the EDC will be compared with the data present in those two reports and in the hospital study database.

Most monitoring will be realised in remote. A monitoring visit will be realised at mid-time. A data review and validation will be realised before database lockdown. Data will then be validated by the PI and the data manager.

Data management:

Data collected are pseudonymised to ensure confidentiality and medical secret and registered in the e-CRF.

Demographic data as well as scans are available in the hospital's database, set up specifically for retrospective data studies. Measures realised on the CT scans are registered on an electronic report, for both measurements methods. The data from those reports and from the database are reported in the e-CRF. Analysed CT scans are also uploaded in the e-CRF.

Data cannot be registered in the software before the subject's inclusion has been validated.

Data entered into the software must be double-checked to limit errors.

Data validation:

This requirement is achieved through Source Data Verification, the process by which the information reported on the e-CRF by the investigator is compared with the original medical records to ensure it is complete and accurate during monitoring visits and closure visit.

In addition, a system of dynamic consistency checks is implemented in the e-CRF for rapid data validation when applicable.

Through the Source Data Validation process, the monitor should confirm accurate transcription of data from source files to the CRF and that the CRF contains all the relevant information about the patient's participation in the clinical trial.

Monitoring visit process

Process:

The first monitoring visit will be conducted after the completion of the Step 4. of the procedure has been completed, to review first part of data. The rest will be reviewed in close-out visit.

Monitoring call and conference will be realized at the end of each step and when required The Monitoring visit will review all of the responsibilities listed above. Following each monitoring visit / call, a monitoring report will be submitted to the QI.

The following activities may take place during each monitoring visit:

The following participant data will be source data verified for the indicated percentage of participants enrolled in the trial and all incidences reported since the last monitoring visit:

100% of eligibility criteria 100% of participant CRFs

Any updates and/or revisions to the following study documents since the last monitoring visit will also be reviewed:

Training documentation/records and Task Delegation Log updates Regulatory documentation including Health Canada approval/amendments

The following activities may be conducted at each monitoring visit:

Trial Master File

• Ensure that essential document files are complete and current.

Investigator and Site Personnel Responsibilities

• Ensure that the Task Delegation Log is complete and signed.
• Verify that the QI and site personnel are adhering to the protocol and conducting the study according to regulatory requirements, Procedures and NF ISO 14155:2020.
• Verify that study activities are being performed by the QI or have been delegated to personnel qualified by appropriate education and training.
• Provide and document any necessary training for the QI and site personnel, such as training on NF ISO 14155:2020, procedures, protocol, e-CRF…

CRF Review / Source Documentation Verification. Verify the following:

• Accurate, complete, and current source documentation is maintained.
• Participants' eligibility reviewed and signed off by QI/Sub-Investigator.
• All procedures outlined in the protocol were completed.
• Protocol deviations documented and reported according to the protocol.
• All dropouts of enrolled subjects are recorded in the source documentation and on the CRF.
• The QI has reviewed, signed, and dated all required e-CRF pages in a timely manner
• Data entries in the e-CRF pages coincide with the source documentation, and note any errors, omissions, or discrepancies by issuing queries within the e-CRF system and revise other bullets.
• Provide the site staff with copies of Data correction forms.
• Verify that previously outstanding data queries have been resolved, signed, and dated by the QI or designee.

Device Deficiency (DD)

• Verify all newly reported DDs against source documentation.
• Follow up on previously reported DDs
• Confirm that all DDs have been reported to the Sponsor
• Identify any unreported DDs in source documentation.
• Review DDs reporting procedures, as necessary.

Protocol Deviations

• Verify that all protocol deviations are documented appropriately in each participant's research record and on the appropriate protocol deviation form.
• Ensure that the site has reported all protocol deviations to the Ethical Committee, as defined by EC policy and/or procedures
• Address any protocol deviations with site personnel during the monitoring visit and identify ways to prevent a recurrence of similar issues.

General

• Confirm all data is verifiable against source documentation.
• Confirm no transcription errors have been made.
• Ensure corrections are lined out, dated & initialed (no erasures or "whiteouts").

Visit Conclusion

At the conclusion of the visit, the monitor will meet with the QI and site research staff to review visit findings and answer questions. The monitor will discuss the following topics at a minimum:

• Enrolment progress.
• Study conducts and documentation of study activities.
• Deviations.
• Scheduling of the next Monitoring Visit when applicable

Statistics:

Aggregated data analysed as treated and per subgroup. Descriptive analysis

• Pearson's coefficient correlation
• Absolute mean discrepancy per measurements
• Global mean and standard deviation per measurement endpoint
• Ratio for a measure
• Maximum aortic diameter ratio
• Computational analysis time The duration of the semi-automatic segmentation manually corrected by the 2 surgeons was reported and compared to the fully automatic method.
• Detectable and non-detectable error rate for safety evaluation
• Calculation of false positive, false negative, sensibility, specificity, positive predictive values, negative predictive values and likelihood ratio (LR+ and LR-).

Inferential analysis Objective: comparing an observed proportion p1 with a theoretical proportion p0 p1: proportion of adequate data p0: theoretical proportion: 90%

A one-sided right test will be realised to evaluate the comparison

Evaluation criteria:

Principal criteria:

• 90% of the patients shaw 90% of adequacy.

Secondary criteria:

Evaluation of the measurements and the segmentations:

• The mean discrepancy shall be ≤ 5mm
• Pearson correlation shall be ≥ 90%

Evaluation of risks:

• Unanticipated risks should not be critical Evaluation of the security
• The detectable error rate shall be ≥ 95%
• The non-detectable error rate shall be ≤ 5% Evaluation of PRAEVAorta® Web
• No critical bug No critical unanticipated risks

Inclusion Criteria:

• Male or female patients diagnosed with an infrarenal AAA and treated with EVAR at the University Hospital of Leipzig between 2015-2021
• Patient equal to or over 18 years old

Exclusion Criteria:

Patients with

• para-renal or thoraco-abdominal aortic aneurysms,
• associated iliac artery aneurysms,
• ruptured aneurysms,
• aortic dissections,