Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Class |
|---|---|
| Digital Diagnostics, Inc. | INDUSTRY |
| Deep Eye Care Foundation (DECF) | OTHER |
Not provided
Not provided
Not provided
Not provided
The purpose of this study is to assess the impact of using autonomous artificial intelligence (AI) system for identification of diabetic retinopathy (DR) and diabetic macular edema on productivity of retina specialists in Bangladesh.
Globally, the number of people with diabetes mellitus is increasing. Diabetic retinopathy is a chronic, progressive complication of diabetes mellitus that affects the microvasculature of the retina, which if left untreated can potentially result in vision loss. Early detection and treatment of diabetic retinopathy can prevent potential blindness.
Study Aim: To assess the impact of using autonomous artificial intelligence (AI) system for detection of diabetic retinopathy (DR) and diabetic macular edema on physician productivity in Bangladesh.
Main study question: Will ophthalmologists with clinic days randomized to use autonomous AI DR detection for all persons with diabetes (diagnosed or un-diagnosed) visiting their clinic system have a greater number of examined patients with diabetes (by either AI or clinical exam), and a greater complexity of examined patients on a recognized grading scale, per physician working hour than those randomized not to have autonomous AI screening for their diabetes population?
The investigators anticipate that this study will demonstrate an increase in physician productivity, supporting efficiency for both physicians and patients, while also addressing increased access for DR screening; ultimately, preventing vision loss amongst diabetic patients. The study has the potential to contribute to the evidence base on the benefits of AI for physicians and patients. Additionally, the study has the potential to demonstrate the benefits (and/or challenges) of implementing AI in resource-constrained settings, such as Bangladesh.
Bangladesh PRODUCTIVity in Eyecare (B-PRODUCTIVE) Trial
Study Aim: To assess the impact of using autonomous artificial intelligence (AI) for identification of diabetic retinopathy (DR) and diabetic macular edema on productivity of retina specialists in Bangladesh.
Hypothesis: Autonomous AI increases retina specialist productivity
Main Study Question: Will retina specialists complete a greater number of diabetic eye exams per working hour (including persons reviewed by AI whom the retina specialist does not need to see personally) when they use autonomous AI in a randomized clinical trial?
Design: Cluster-randomized (by clinic day) controlled trial.
Randomization: By clinic day. Each morning the clinic manager will open an opaque envelope, which informs the manager if it is an Intervention (AI) or Control (non-AI) day.
Interventions: All patients in both groups go through the eligibility checklist. If approved, they will be evaluated by autonomous AI. This is done to decrease potential bias (neither patients nor physicians know the group assignment of participants) and concealment (so that neither patients nor doctors can arrange visits on a known "Intervention Day").
Intervention Group: On randomly selected "Intervention" clinic days, if patients screen positive or have insufficient image quality, they continue to the ophthalmologist. If not eligible for autonomous AI, they proceed straight to the ophthalmologist without autonomous AI evaluation. If patients receive a negative result, they do not see the retina specialist, and are referred for a visit at the regular eye clinic (not the retina clinic) in 3 months.
Control Group: On randomly-selected "Control Days," all patients see the ophthalmologist, irrespective of the results of autonomous AI evaluation.
Masking: The retina doctors are masked both patient group assignment (that is, whether autonomous AI was used for pre-screening or not on the particular clinic day) and also masked to the results of the AI on Intervention days. Patients are also masked to group assignment and autonomous AI results.
Not provided
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| Intervention Group | Experimental | Autonomous AI results are used to evaluate if the participant needs to see the retina specialist (positive result) or not (negative result). |
|
| Control Group | No Intervention | All participants see the retina specialist irrespective of the results of their autonomous AI evaluation. |
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| Results utilized from autonomous AI diagnostic system for diabetic retinopathy and/or diabetic macular edema | Diagnostic Test | If patients receive a negative result they do not see the retina specialist |
| Measure | Description | Time Frame |
|---|---|---|
| Number of Completed Care Encounters Among Clinic Patients With Diabetes Per Retina Specialist Clinic Hour | Number of completed care encounters among clinic patients with diabetes per retina specialist clinic hour. Numerator is the number of care encounters among patients with diabetes (including persons evaluated by autonomous AI on Intervention Days who are determined not to need to see the retina specialist). The denominator is retina specialist clinic time in hours. | 105 randomized clinic days |
| Number of Completed Care Encounters Among All Clinic Patients (With and Without Diabetes) Per Retina Specialist Clinic Hour | Number of completed care encounters among all clinic patients (with and without diabetes) per retina specialist clinic hour. Numerator is the number of completed care encounters (including persons evaluated by autonomous AI on Intervention Days who are determined not to need to see the retina specialist). The denominator is retina specialist clinic working time in hours. | 105 randomized clinic days |
| Measure | Description | Time Frame |
|---|---|---|
| Specialist Productivity Adjusted for Patient Complexity for Patients With Diabetes | Specialist productivity (care encounters / specialist clinic hour) adjusted for patient complexity for patients with diabetes. The complexity score for each patient participant was calculated by a masked United Kingdom National Health Service grader using the International Grading system, adapted from Wilkinson et al. International Clinical Diabetic Retinopathy and Diabetic Macular Edema Severity Scales (no DED = 0 points, mild non-proliferative DED = 0 points, moderate or severe non-proliferative DED = 1 point, proliferative DED = 3 points and diabetic macular edema = 2 points.) The patient participant complexity score was summed across both eyes. The average complexity score for each arm was calculated. Complexity adjusted specialist productivity was calculated for intervention and control arms by multiplying the respective overall productivity (care encounters per specialist clinic hour) calculation by the respective average complexity score. |
Not provided
Inclusion Criteria:
Retina specialists regularly seeing patients with DR
Patients
Exclusion Criteria:
Retina specialists
Patients
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Name | Affiliation | Role |
|---|---|---|
| Nathan Congdon, MD, MPH | Orbis | Study Chair |
| Facility | Status | City | State | ZIP | Country | Contacts |
|---|---|---|---|---|---|---|
| Deep Eye Care Foundation | Rangpur City | Bangladesh |
| PubMed Identifier | Type | Citation | Retractions |
|---|---|---|---|
| 37794054 | Derived | Abramoff MD, Whitestone N, Patnaik JL, Rich E, Ahmed M, Husain L, Hassan MY, Tanjil MSH, Weitzman D, Dai T, Wagner BD, Cherwek DH, Congdon N, Islam K. Autonomous artificial intelligence increases real-world specialist clinic productivity in a cluster-randomized trial. NPJ Digit Med. 2023 Oct 4;6(1):184. doi: 10.1038/s41746-023-00931-7. |
Not provided
Not provided
We do not plan to share IPD
Not provided
Not provided
Not provided
Not provided
The random allocation of each cluster (clinic day) was concealed until clinic staff received an email with this information just before the start of that day's clinic. Medical staff who determined access, specialists and patient participants remained masked to the random assignment of clinic days as control or intervention. Technicians and/or specialists are not considered enrolled. The participant flow details patient participants randomized by clinic day.
Not provided
| ID | Title | Description |
|---|---|---|
| FG000 | Intervention Group | Autonomous AI results are used to evaluate if the participant needs to see the retina specialist (positive result) or not (negative result). Results utilized from autonomous AI diagnostic system for diabetic retinopathy and/or diabetic macular edema: If patients receive a negative result they do not see the retina specialist |
| FG001 | Control Group | All participants see the retina specialist irrespective of the results of their autonomous AI evaluation. |
| Title | Milestones | Reasons Not Completed | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Overall Study |
|
Not provided
Not provided
| ID | Title | Description |
|---|---|---|
| BG000 | Intervention Group | Autonomous AI results are used to evaluate if the participant needs to see the retina specialist (positive result) or not (negative result). Results utilized from autonomous AI diagnostic system for diabetic retinopathy and/or diabetic macular edema: If patients receive a negative result they do not see the retina specialist |
| Units | Counts |
|---|---|
| Participants |
|
| Title | Description | Population Description | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Denominator Units Selected | Denominators | Classes |
|---|---|---|---|---|---|---|---|---|---|
| Age, Customized | Mean |
| Type | Title | Description | Population Description | Reporting Status | Anticipated Posting Date | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Time Frame | Units Analyzed | Denominator Units Selected | Arm/Group Information | Denominators | Classes | Analyses | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Primary | Number of Completed Care Encounters Among Clinic Patients With Diabetes Per Retina Specialist Clinic Hour | Number of completed care encounters among clinic patients with diabetes per retina specialist clinic hour. Numerator is the number of care encounters among patients with diabetes (including persons evaluated by autonomous AI on Intervention Days who are determined not to need to see the retina specialist). The denominator is retina specialist clinic time in hours. | Completed care encounters among clinic patients with diabetes | Posted | Mean | 95% Confidence Interval | care encounters/specialist clinic hour | 105 randomized clinic days | care encounters pts with diabetes | care encounters pts with diabetes |
|
Through study completion, up to 5 months
Not provided
Not provided
| ID | Title | Description | Deaths (Affected) | Deaths (At Risk) | Serious Events (Affected) | Serious Events (At Risk) | Other Events (Affected) | Other Events (At Risk) |
|---|---|---|---|---|---|---|---|---|
| EG000 | Intervention Group | Autonomous AI results are used to evaluate if the participant needs to see the retina specialist (positive result) or not (negative result). Results utilized from autonomous AI diagnostic system for diabetic retinopathy and/or diabetic macular edema: If patients receive a negative result they do not see the retina specialist |
Not provided
Not provided
Not provided
| Title | Organization | Phone | Extension | |
|---|---|---|---|---|
| Dr. Nathan Congdon, Director of Research | Orbis International | 646-674-5514 | ncongdon1@gmail.com |
Not provided
| Type | Includes Protocol | Includes SAP | Includes ICF | Document Label | Document Date | Document Uploaded Date | Document File Name |
|---|---|---|---|---|---|---|---|
| Prot_SAP | Yes | Yes | No | Study Protocol and Statistical Analysis Plan | Aug 1, 2022 | Nov 13, 2023 | Prot_SAP_000.pdf |
Not provided
Not provided
Not provided
| ID | Term |
|---|---|
| D003930 | Diabetic Retinopathy |
| ID | Term |
|---|---|
| D012164 | Retinal Diseases |
| D005128 | Eye Diseases |
| D003925 | Diabetic Angiopathies |
| D014652 | Vascular Diseases |
Not provided
Not provided
Cluster-randomized (by clinic day) controlled trial.
Not provided
Not provided
The retina specialists are masked both to patient group assignment (that is, whether autonomous AI results were used or not on the particular clinic day) and also masked to the results of the autonomous AI on Intervention days. Patients are also masked to group assignment and autonomous AI screening results.
| 105 randomized clinic days |
| Number of Participants Who Were Very Satisfied or Satisfied With Autonomous AI | After the patient participant completed the autonomous AI process, a survey with a 4-point Likert scale ("very satisfied," "satisfied," "dissatisfied," "very dissatisfied") was administered, concerning the participant's satisfaction with interactions with the healthcare team, time to receive examination results, and receiving their diagnosis from the autonomous AI system. | 105 randomized clinic days |
| BG001 |
| Control Group |
All participants see the retina specialist irrespective of the results of their autonomous AI evaluation. |
| BG002 | Total | Total of all reporting groups |
| years |
|
| Sex: Female, Male | Count of Participants | Participants |
|
| Race and Ethnicity Not Collected | Race and Ethnicity were not collected from any participant. | Count of Participants | Participants |
|
| Region of Enrollment | Number | participants |
|
| OG001 | Control Group | All participants see the retina specialist irrespective of the results of their autonomous AI evaluation. |
|
|
| Primary | Number of Completed Care Encounters Among All Clinic Patients (With and Without Diabetes) Per Retina Specialist Clinic Hour | Number of completed care encounters among all clinic patients (with and without diabetes) per retina specialist clinic hour. Numerator is the number of completed care encounters (including persons evaluated by autonomous AI on Intervention Days who are determined not to need to see the retina specialist). The denominator is retina specialist clinic working time in hours. | All retina clinic care encounters | Posted | Mean | 95% Confidence Interval | care encounters/specialist clinic hour | 105 randomized clinic days | all retina clinic care encounters | all retina clinic care encounters |
|
|
|
| Secondary | Specialist Productivity Adjusted for Patient Complexity for Patients With Diabetes | Specialist productivity (care encounters / specialist clinic hour) adjusted for patient complexity for patients with diabetes. The complexity score for each patient participant was calculated by a masked United Kingdom National Health Service grader using the International Grading system, adapted from Wilkinson et al. International Clinical Diabetic Retinopathy and Diabetic Macular Edema Severity Scales (no DED = 0 points, mild non-proliferative DED = 0 points, moderate or severe non-proliferative DED = 1 point, proliferative DED = 3 points and diabetic macular edema = 2 points.) The patient participant complexity score was summed across both eyes. The average complexity score for each arm was calculated. Complexity adjusted specialist productivity was calculated for intervention and control arms by multiplying the respective overall productivity (care encounters per specialist clinic hour) calculation by the respective average complexity score. | This is the same calculation as Outcome 1 but with the addition of adjustment for complexity. (# retina care encounters*complexity score) / specialist clinic hours | Posted | Number | (exams/clinic hour)*(score on a scale) | 105 randomized clinic days | care encounters pts with diabetes | care encounters pts with diabetes |
|
|
|
| Secondary | Number of Participants Who Were Very Satisfied or Satisfied With Autonomous AI | After the patient participant completed the autonomous AI process, a survey with a 4-point Likert scale ("very satisfied," "satisfied," "dissatisfied," "very dissatisfied") was administered, concerning the participant's satisfaction with interactions with the healthcare team, time to receive examination results, and receiving their diagnosis from the autonomous AI system. | Posted | Count of Participants | Participants | 105 randomized clinic days |
|
|
|
| 0 |
| 494 |
| 0 |
| 494 |
| 0 |
| 494 |
| EG001 | Control Group | All participants see the retina specialist irrespective of the results of their autonomous AI evaluation. | 0 | 499 | 0 | 499 | 0 | 499 |
Not provided
Not provided
| D002318 |
| Cardiovascular Diseases |
| D048909 | Diabetes Complications |
| D003920 | Diabetes Mellitus |
| D004700 | Endocrine System Diseases |