Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Class |
|---|---|
| Mendel AI | INDUSTRY |
Not provided
Not provided
Not provided
Not provided
Identifying eligible patients is a key process in the clinical trial enterprise. Currently, this process relies on time-intensive manual chart review, creating a rate-limiting step for trial participation. The integration of AI technology into the trial screening process has potential to improve participation rates. This study aims to assess the performance (accuracy, efficiency) of AI-augmented patient identification and inform optimal integration into clinical research screening processes.
The objective of this study is to assess and compare the accuracy and efficiency of three different approaches to abstracting clinical data used to identify oncology patients who meet the inclusion criteria for participation in clinical trials. The three approaches under evaluation include: (1) an autonomous AI algorithm (Mendel AI; developed by artificial intelligence startup company Mendel) which analyzes patient medical records to extract relevant clinical facts ("AI-alone"); (2) a human researcher who manually reviews patient charts as per the current norm/practice ("Human-alone"); and (3) a human researcher utilizing AI augmentation ("Human+AI"), where Mendel AI serves as a supportive tool in the decision-making process by providing the researcher a list of elements abstracted by the AI algorithm and a rank-order list of patients most likely to meet inclusion criteria for a trial.
The study primarily aims to compare (1) the chart-level accuracy of the Human+AI collaboration relative to Human-alone given the relevance of this comparison for real-world clinical workflows, defined by the percentage of pre-identified chart elements classified correctly compared against a predetermined "gold standard"; and (2) the efficiency of the Human+AI vs. Human-alone arms, defined by the time per chart review in minutes, measured for each chart.
Our hypotheses are (1) the Human+AI arm will be non-inferior in accuracy when compared to the Human-alone arm, in relation to a predetermined "gold standard", and (2) that a Human+AI arm will be superior in efficiency of abstraction when compared to Human-alone screening.
The identification of eligible patients for clinical trials is a critical component of clinical research, as it directly impacts patient recruitment, study enrollment, and the generalizability of research findings. Currently, the process of identifying eligible patients often relies on manual chart review by clinical research staff, which can be time-consuming, labor-intensive, and prone to human error. Consequently, eligible patients may be overlooked, and opportunities for trial participation may be missed. The integration of AI technology into the patient identification process has the potential to enhance the accuracy and efficiency of this critical task, leading to improved clinical trial recruitment and outcomes.
This study holds important implications for the field of clinical research by evaluating the effectiveness of AI-augmented patient identification compared to traditional manual methods and autonomous AI algorithms. By examining the strengths and limitations of each approach, the study will provide valuable insights into the optimal integration of AI technology in clinical research processes. Furthermore, the results of this study have the potential to benefit patients by improving their access to clinical trials and increasing awareness of available treatment options. For clinical research institutions, enhancing the efficiency of patient identification can lead to more effective use of research resources and the potential for accelerated clinical trial timelines. Ultimately, the findings of this study may contribute to advancements in clinical research practices, promoting more equitable access to trials and facilitating the development of innovative treatments for patients with cancer.
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| AI-alone |
| ||
| Human-alone |
| ||
| Human + AI |
|
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| Chart review | Other | Chart review |
|
| Measure | Description | Time Frame |
|---|---|---|
| Abstracted Chart-level Accuracy | The primary outcome measured was mean chart-level accuracy, defined as the percentage of elements identified by clinical research coordinators among all elements in the gold-standard set, measured for each chart, and averaged across all charts. Research coordinator-abstracted responses were identified as being accurate when they exactly matched with the gold-standard set. The gold-standard set was determined by 2-3 clinicians blinded to experimental arms. | 1 year |
| Measure | Description | Time Frame |
|---|---|---|
| Efficiency of Chart-level Abstraction (in Minutes) | Efficiency was calculated as the number of minutes spent on each chart abstraction. | 1 year |
Not provided
Inclusion Criteria:
Exclusion Criteria:
Not provided
Not provided
Not provided
Not provided
De-identified patient charts from community oncology practices, with a diagnosis of non-small cell lung cancer (NSCLC) or colorectal cancer (CRC).
Not provided
Not provided
| Facility | Status | City | State | ZIP | Country | Contacts |
|---|---|---|---|---|---|---|
| Emory University | Atlanta | Georgia | 30307 | United States | ||
| University of Pennsylvania |
| PubMed Identifier | Type | Citation | Retractions |
|---|---|---|---|
| 41634037 | Derived | Parikh RB, Kolla L, Beothy EA, Ferrell WJ, Laventure B, Guido M, Girard A, Li Y, Dosoky KEM, Tarabishy K, Patel PS, Andalcio A, Maloney K, Mena JU, Salloum W, Chen J, Emanuel EJ. Human-AI teaming to improve accuracy and efficiency of eligibility criteria prescreening for oncology trials: a randomized evaluation trial using retrospective electronic health records. Nat Commun. 2026 Feb 3;17(1):2306. doi: 10.1038/s41467-026-68873-8. |
Not provided
Not provided
No individual participant data will be shared.
Not provided
Not provided
Not provided
Not provided
Each record will be reviewed via all 3 arms, so the total enrollment will be 355.
Not provided
Not provided
| ID | Title | Description |
|---|---|---|
| FG000 | All Participants | Patients that underwent all chart reviews (Human-alone, AI-alone, and Human + AI) |
| Title | Milestones | Reasons Not Completed | |||||
|---|---|---|---|---|---|---|---|
| Human-alone |
|
Not provided
Not provided
| Type | Includes Protocol | Includes SAP | Includes ICF | Document Label | Document Date | Document Uploaded Date | Document File Name |
|---|---|---|---|---|---|---|---|
| Prot_SAP | Yes | Yes | No | Study Protocol and Statistical Analysis Plan | Jan 23, 2025 |
Not provided
Not provided
Not provided
Not provided
| Philadelphia |
| Pennsylvania |
| 19104 |
| United States |
| COMPLETED |
|
| NOT COMPLETED |
|
| AI-alone |
|
| Human + AI |
|
Each participated in all 3 arms, for an overall total of 355.
Not provided
| ID | Title | Description |
|---|---|---|
| BG000 | All Participants | Patients who underwent Human-alone, AI-alone, and Human+AI chart review |
| Units | Counts |
|---|---|
| Participants |
|
| Title | Description | Population Description | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Denominator Units Selected | Denominators | Classes | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Age, Customized | Count of Participants | Participants |
| ||||||||||||||||||
| Sex/Gender, Customized | Sex/gender were not collected from any participant. | Count of Participants | Participants |
| |||||||||||||||||
| Race and Ethnicity Not Collected | Race and Ethnicity were not collected from any participant. | Count of Participants | Participants |
| |||||||||||||||||
| Region of Enrollment | Number | participants |
| ||||||||||||||||||
| Cancer Type | Count of Participants | Participants |
|
| Type | Title | Description | Population Description | Reporting Status | Anticipated Posting Date | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Time Frame | Units Analyzed | Denominator Units Selected | Arm/Group Information | Denominators | Classes | Analyses | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Primary | Abstracted Chart-level Accuracy | The primary outcome measured was mean chart-level accuracy, defined as the percentage of elements identified by clinical research coordinators among all elements in the gold-standard set, measured for each chart, and averaged across all charts. Research coordinator-abstracted responses were identified as being accurate when they exactly matched with the gold-standard set. The gold-standard set was determined by 2-3 clinicians blinded to experimental arms. | The study population was drawn from a 15-physician community oncology practice in California serving patients from urban and surrounding rural communities. This study cohort consisted of unstructured medical records from patients within the dataset with 1) a diagnosis of non-small cell lung cancer (NSCLC) or colorectal cancer (CrCa), 2) a minimum of five clinical documents available, and 3) the most recent document being within five years from the time of data extraction. | Posted | Mean | Standard Deviation | % of elements correctly abstracted | 1 year |
|
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Secondary | Efficiency of Chart-level Abstraction (in Minutes) | Efficiency was calculated as the number of minutes spent on each chart abstraction. | The study population was the same as the study population described in the primary outcome analysis population description section. However, AI-alone chart reviews were NOT analyzed for efficiency. Data for the secondary outcome of efficiency was not collected for the AI-alone arm and therefore cannot be reported in the outcome table. This was prespecified, as its fully automated nature renders direct comparison with human-involved workflows inappropriate and uninformative. | Posted | Median | Inter-Quartile Range | Number of minutes spent on abstraction | 1 year |
|
|
Not applicable. Adverse events were not collected.
Not applicable. Adverse events were not collected.
Not provided
| ID | Title | Description | Deaths (Affected) | Deaths (At Risk) | Serious Events (Affected) | Serious Events (At Risk) | Other Events (Affected) | Other Events (At Risk) |
|---|---|---|---|---|---|---|---|---|
| EG000 | AI-alone | Chart review: Chart review | 0 | 0 | 0 | 0 | 0 | 0 |
| EG001 | Human-alone | Chart review: Chart review | 0 | 0 | 0 | 0 | 0 | 0 |
| EG002 | Human + AI | Chart review: Chart review | 0 | 0 | 0 | 0 | 0 | 0 |
Not provided
Not provided
Not provided
Not provided
| Title | Organization | Phone | Extension | |
|---|---|---|---|---|
| Ravi B. Parikh, MD, MPP | Emory University School of Medicine | (352) 422-4285 | ravi.bharat.parikh@emory.edu |
| Apr 7, 2025 |
| Prot_SAP_000.pdf |
| ID | Term |
|---|---|
| D009369 | Neoplasms |
Not provided
Not provided
Not provided
|
|
| Non-Inferiority |
A predefined non-inferiority margin of 0.05 was used. The primary study was powered on a retrospective, paired, non-inferiority design to evaluate abstraction accuracy of elements commonly used to help screen patients for clinical trials. |
| A Shapiro-Wilk test was used to assess normality of paired differences in chart-level accuracy (alpha = 0.05). A one-sided, paired Wilcoxon Rank Sum test (alpha = 0.05) was employed to test the null hypothesis of whether chart-level accuracy of the Human+AI arm for EHR chart abstraction was superior to the chart-level accuracy of a Human-alone arm abstraction. | Wilcoxon (Mann-Whitney) | 0.002 | Mean Difference (Final Values) | 0.02 | 2-Sided | Superiority | The primary study was powered on a retrospective, paired, superiority design to evaluate abstraction accuracy of elements commonly used to help screen patients for clinical trials. |
|
|