Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
This is a three-arm pragmatic RCT of 238 outpatient physicians at a large academic health system, randomized 1:1:1 to one of two AI scribe tools or a usual-care control group. The two-month study will observe and compare the effects of each tool prior to system-wide roll out of selected tool (anticipated Spring 2025). We will use covariate-constrained randomization to balance the arms in terms of physician baseline time in notes, survey-measured level of burnout, and clinic days per week.
The primary purpose of the initiative is to improve quality, efficiency, and business operations at University of California, Los Angeles (UCLA) Health, and this initiative is not being done for research purposes. The results of this operational initiative will inform the widespread roll out of AI scribe tools across all providers within the UCLA Health System. Nevertheless, the UCLA study team plans to rigorously examine and publish the impact of this intervention across the health system, which is why the study team pre-registered the initiative.
This study will assess operational-oriented outcomes across all groups. Notably, all groups will eventually receive all interventions over time in this observational study of a randomized roll out of a QI initiative. Moreover, the primary purpose of this initiative is operational. In other words, based on the results of this initiative, one of these tools will be eventually selected and operationalized widely across the health system.
Enrolled participants are randomized to one of three groups. Randomization was needed to overcome secular trends, seasonal and holiday effects in December, and other factors confounding the relationship between exposure to the AI tools and the outcomes.
The primary aim of this study is to evaluate the impact of two ambient AI scribe technologies on clinician change from baseline time spent on EHR documentation, comparing each scribe to a control group. Secondary objectives include assessing the AI scribes' impact on clinician metrics such as burnout, physician satisfaction, and productivity. Additionally, the study team intends to perform an economic evaluation analysis of the tools to guide business decision making. The study team will also analyze physician reported effects of the AI tools on patient safety, equity, and any unintended consequences of the initiative.
Not provided
Not provided
Not provided
Not provided
| Label | Type | Description | Intervention Names |
|---|---|---|---|
| Nabla, Vendor of virtual AI scribe technology | Other | Participants in this arm will utilize AI scribe tools from Nabla and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note. The Nabla AI scribe tool is transcriptional and does not provide clinical decision support. |
|
| Vendor B of virtual AI scribe technology | Other | Participants in this arm will utilize AI scribe tools from Vendor B and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note. The AI scribe tool is transcriptional and does not provide clinical decision support. |
|
| No Scribe | No Intervention | Participants in this arm will not have access to AI scribe tools and will continue their usual clinical documentation processes |
| Name | Type | Description | Arm Group Labels | Other Names |
|---|---|---|---|---|
| Use Nabla AI Scribe tool provided | Other | AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes. These tools are integrated into the EHR and automatically adds the generated text to the provider note. All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked. Nabla leverages its proprietary speech-to-text to transform the conversation into a written context, combined with HIPAA compliant Large Language Models (LLM) like Azure OpenAI's GPT-4. Nabla does not store any audio. |
| Measure | Description | Time Frame |
|---|---|---|
| Change in the Time in Notes Per Note | The primary outcome measure is the change in provider mean time in notes per note in the second month of the trial from the providers baseline mean time in notes per note for the six months prior to enrollment. This change will be computed on the natural log scale. No patient level information will be collected for this outcome measure. | Study month 2 |
| Measure | Description | Time Frame |
|---|---|---|
| Provider Burnout Score | The Mini Z 2.0 Survey is a validated 10-item instrument designed to measure key factors influencing workplace satisfaction and burnout among healthcare professionals. Each item is scored on a Likert scale (1-5), with higher scores generally indicating more positive outcomes - greater job satisfaction, sufficiency of time for electronic medical record documentation, and lower levels of stress. For negatively framed items (e.g., stress due to the job or frustration with the electronic medical record), higher scores indicate lower levels of dissatisfaction. The total score ranges from 10 to 50, with scores ≥40 representing a joyful workplace. No patient level information will be collected for this outcome measure. |
Not provided
Inclusion Criteria:
Exclusion Criteria:
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
| Facility | Status | City | State | ZIP | Country | Contacts |
|---|---|---|---|---|---|---|
| UCLA Health System | Los Angeles | California | 90024 | United States |
| PubMed Identifier | Type | Citation | Retractions |
|---|---|---|---|
| 39602298 | Background | Sittig DF, Singh H. Recommendations to Ensure Safety of AI in Real-World Clinical Care. JAMA. 2025 Feb 11;333(6):457-458. doi: 10.1001/jama.2024.24598. | |
| 30128552 | Background | Gianfrancesco MA, Tamang S, Yazdany J, Schmajuk G. Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data. JAMA Intern Med. 2018 Nov 1;178(11):1544-1547. doi: 10.1001/jamainternmed.2018.3763. |
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Not provided
Of 331 providers expressing interest, 93 were excluded due to ineligibility, declining participation, incomplete surveys, or decision-making roles. The remaining 238 were randomized to the arms.
Not provided
Not provided
| ID | Title | Description |
|---|---|---|
| FG000 | Nabla, Vendor of Virtual AI Scribe Technology | Participants in this arm will utilize AI scribe tools from Nabla and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note. The Nabla AI scribe tool is transcriptional and does not provide clinical decision support. Use Nabla AI Scribe tool provided: AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes. These tools are integrated into the EHR and automatically adds the generated text to the provider note. All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked. Nabla leverages its proprietary speech-to-text to transform the conversation into a written context, combined with HIPAA compliant Large Language Models (LLM) like Azure OpenAI's GPT-4. Nabla does not store any audio. |
| FG001 | Vendor B of Virtual AI Scribe Technology | Participants in this arm will utilize AI scribe tools from Vendor B and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note. The AI scribe tool is transcriptional and does not provide clinical decision support. Use AI Scribe tool provided by Vendor B: AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes. These tools are integrated into the EHR and automatically adds the generated text to the provider note. All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked. |
| FG002 | No Scribe | Participants in this arm will not have access to AI scribe tools and will continue their usual clinical documentation processes |
| Title | Milestones | Reasons Not Completed | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Overall Study |
|
Not provided
Not provided
| ID | Title | Description |
|---|---|---|
| BG000 | Nabla, Vendor of Virtual AI Scribe Technology | Participants in this arm will utilize AI scribe tools from Nabla and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note. The Nabla AI scribe tool is transcriptional and does not provide clinical decision support. Use Nabla AI Scribe tool provided: AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes. These tools are integrated into the EHR and automatically adds the generated text to the provider note. All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked. Nabla leverages its proprietary speech-to-text to transform the conversation into a written context, combined with HIPAA compliant Large Language Models (LLM) like Azure OpenAI's GPT-4. Nabla does not store any audio. |
| Units | Counts |
|---|---|
| Participants |
|
| Title | Description | Population Description | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Denominator Units Selected | Denominators | Classes |
|---|---|---|---|---|---|---|---|---|---|
| Age, Customized | Count of Participants |
| Type | Title | Description | Population Description | Reporting Status | Anticipated Posting Date | Parameter Type | Dispersion Type | Unit of Measure | Calculate Percentage | Time Frame | Units Analyzed | Denominator Units Selected | Arm/Group Information | Denominators | Classes | Analyses | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Primary | Change in the Time in Notes Per Note | The primary outcome measure is the change in provider mean time in notes per note in the second month of the trial from the providers baseline mean time in notes per note for the six months prior to enrollment. This change will be computed on the natural log scale. No patient level information will be collected for this outcome measure. | Providers with this metric logged by Signal in the reporting period. | Posted | Mean | Standard Deviation | ln(minutes per note) | Study month 2 |
|
From the time the scribe was enabled for providers (study start date) until completion of the post-trial survey (study completion date), up to 72 days
We considered any reported event/outcome of the AI Scribe tool's use that compromised the safety of a patient under the care of a trial participant as an adverse event. Adverse event reports were accepted either ad-hoc by trial providers or in the post-trial survey.
Not provided
| ID | Title | Description | Deaths (Affected) | Deaths (At Risk) | Serious Events (Affected) | Serious Events (At Risk) | Other Events (Affected) | Other Events (At Risk) |
|---|---|---|---|---|---|---|---|---|
| EG000 | Nabla, Vendor of Virtual AI Scribe Technology | Participants in this arm will utilize AI scribe tools from Nabla and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note. The Nabla AI scribe tool is transcriptional and does not provide clinical decision support. Use Nabla AI Scribe tool provided: AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes. These tools are integrated into the EHR and automatically adds the generated text to the provider note. All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked. Nabla leverages its proprietary speech-to-text to transform the conversation into a written context, combined with HIPAA compliant Large Language Models (LLM) like Azure OpenAI's GPT-4. Nabla does not store any audio. |
Not provided
Not provided
Not provided
| Title | Organization | Phone | Extension | |
|---|---|---|---|---|
| Dr. John N. Mafi, MD, MPH | University of California, Los Angeles | 310-794-2278 | jmafi@mednet.ucla.edu |
Not provided
| Type | Includes Protocol | Includes SAP | Includes ICF | Document Label | Document Date | Document Uploaded Date | Document File Name |
|---|---|---|---|---|---|---|---|
| Prot_SAP | Yes | Yes | No | Study Protocol and Statistical Analysis Plan | Jan 28, 2025 | Feb 9, 2026 | Prot_SAP_000.pdf |
Not provided
Not provided
Not provided
Not provided
The outpatient physicians are not told which tool that they are assigned to.
|
| Use AI Scribe tool provided by Vendor B | Other | AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes. These tools are integrated into the EHR and automatically adds the generated text to the provider note. All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked. |
|
| Study month 2 |
| Provider Task Load Score | Provider task load adapted from the NASA Task Load Index (TLX), a validated tool for assessing perceived workload across six sub-scales: mental demand, physical demand, temporal demand, performance, effort, and frustration. For this study, we adapted the TLX to focus on note-writing workload, including four sub-scales (mental demand, temporal demand, physical demand, and effort) as done previously. Each sub-scale is rated from 0 (low task load) to 100 (high task load) and summed together for a total score scale of 0 (low task load) to 400 (high task load), lower is better. No patient level information will be collected for this outcome measure. | Study month 2 |
| Provider Professional Fulfillment | The Professional Fulfillment Index (PFI) is a validated 16-item instrument that uses a 5-point Likert scale (0-4) to measure professional fulfillment, work exhaustion, and interpersonal disengagement. For this study, we utilize the 4-item work exhaustion subscale which is a mean of the 4-items within that subscale, where a low score (0) indicates a lower level of exhaustion and a high (4) score indicates greater level of exhaustion. No patient level information will be collected for this outcome measure. | Study month 2 |
| Number of Physicians Who Are Considered Detractors, Passive, or Promoters | Self-reported satisfaction survey that asks physicians to consider note accuracy, patient safety, equity, and other potential unintended consequences and rate their overall likelihood to recommend use of the tool on a 1-10 scale. Higher scores (10) indicate greater satisfaction and likelihood to recommend, whereas lower scores (1) indicate dissatisfaction and unlikelihood to recommend. Providers are grouped as "Promoters" if they respond 9-10, "Passive" if they respond 7-8, and "Detractors" if they respond with a value less than or equal to 6. This grouping matches commonly accepted "Net Promoter Score" groupings. No patient level information will be collected for this outcome measure. | Study month 2 |
| Change in Provider RVU | The study team will use physician-level billing information via RVU to determine their change in productivity from a retrospective baseline 6 months prior to enrollment. No patient level information will be collected for this outcome measure. | Study month 2 |
| Change in EHR Signal (Activity) Data - Pajama Time | We will examine change from a retrospective baseline 6 months prior to enrollment in Signal metrics including pajama time per scheduled day. Using this data will determine how a providers time is utilized in the EHR. No patient level information will be collected for this outcome measure. | Study month 2 |
| Change in EHR Signal (Activity) Data - Time Outside Scheduled Hours | We will examine change from a retrospective baseline 6 months prior to enrollment in Signal metrics including time outside scheduled hours per scheduled day. Using this data will determine how a providers time is utilized in the EHR. No patient level information will be collected for this outcome measure. | Study month 2 |
| Change in EHR Signal (Activity) Data - Time on Unscheduled Days | We will examine change from a retrospective baseline 6 months prior to enrollment in Signal metrics including time spent in the system on unscheduled days where . Using this data will determine how a providers time is utilized in the EHR. No patient level information will be collected for this outcome measure. | Study month 2 |
| 35048290 | Background | Linzer M, McLoughlin C, Poplau S, Goelz E, Brown R, Sinsky C; AMA-Hennepin Health System (HHS) burnout reduction writing team. The Mini Z Worklife and Burnout Reduction Instrument: Psychometrics and Clinical Implications. J Gen Intern Med. 2022 Aug;37(11):2876-2878. doi: 10.1007/s11606-021-07278-3. Epub 2022 Jan 19. No abstract available. |
| 29196982 | Background | Trockel M, Bohman B, Lesure E, Hamidi MS, Welle D, Roberts L, Shanafelt T. A Brief Instrument to Assess Both Burnout and Professional Fulfillment in Physicians: Reliability and Validity, Including Correlation with Self-Reported Medical Errors, in a Sample of Resident and Practicing Physicians. Acad Psychiatry. 2018 Feb;42(1):11-24. doi: 10.1007/s40596-017-0849-3. Epub 2017 Dec 1. |
| 38506805 | Background | Garcia P, Ma SP, Shah S, Smith M, Jeong Y, Devon-Sand A, Tai-Seale M, Takazawa K, Clutter D, Vogt K, Lugtu C, Rojo M, Lin S, Shanafelt T, Pfeffer MA, Sharp C. Artificial Intelligence-Generated Draft Replies to Patient Inbox Messages. JAMA Netw Open. 2024 Mar 4;7(3):e243201. doi: 10.1001/jamanetworkopen.2024.3201. |
| 32908284 | Background | Cruz Rivera S, Liu X, Chan AW, Denniston AK, Calvert MJ; SPIRIT-AI and CONSORT-AI Working Group; SPIRIT-AI and CONSORT-AI Steering Group; SPIRIT-AI and CONSORT-AI Consensus Group. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med. 2020 Sep;26(9):1351-1363. doi: 10.1038/s41591-020-1037-7. Epub 2020 Sep 9. |
| 39465898 | Background | McCoy LG, Manrai AK, Rodman A. Large Language Models and the Degradation of the Medical Record. N Engl J Med. 2024 Oct 31;391(17):1561-1564. doi: 10.1056/NEJMp2405999. Epub 2024 Oct 26. No abstract available. |
| 35227443 | Background | Hendrix N, Veenstra DL, Cheng M, Anderson NC, Verguet S. Assessing the Economic Value of Clinical Artificial Intelligence: Challenges and Opportunities. Value Health. 2022 Mar;25(3):331-339. doi: 10.1016/j.jval.2021.08.015. Epub 2021 Oct 9. |
| 38787556 | Background | Rotenstein L, Melnick ER, Iannaccone C, Zhang J, Mugal A, Lipsitz SR, Healey MJ, Holland C, Snyder R, Sinsky CA, Ting D, Bates DW. Virtual Scribes and Physician Time Spent on Electronic Health Records. JAMA Netw Open. 2024 May 1;7(5):e2413140. doi: 10.1001/jamanetworkopen.2024.13140. |
| 38571698 | Background | Cao DY, Silkey JR, Decker MC, Wanat KA. Artificial intelligence-driven digital scribes in clinical documentation: Pilot study assessing the impact on dermatologist workflow and patient encounters. JAAD Int. 2024 Feb 20;15:149-151. doi: 10.1016/j.jdin.2024.02.009. eCollection 2024 Jun. No abstract available. |
| 38834180 | Background | Owens LM, Wilda JJ, Grifka R, Westendorp J, Fletcher JJ. Effect of Ambient Voice Technology, Natural Language Processing, and Artificial Intelligence on the Patient-Physician Relationship. Appl Clin Inform. 2024 Aug;15(4):660-667. doi: 10.1055/a-2337-4739. Epub 2024 Jun 4. |
| 38345343 | Background | Haberle T, Cleveland C, Snow GL, Barber C, Stookey N, Thornock C, Younger L, Mullahkhel B, Ize-Ludlow D. The impact of nuance DAX ambient listening AI documentation: a cohort study. J Am Med Inform Assoc. 2024 Apr 3;31(4):975-979. doi: 10.1093/jamia/ocae022. |
| 39240568 | Background | Liu TL, Hetherington TC, Stephens C, McWilliams A, Dharod A, Carroll T, Cleveland JA. AI-Powered Clinical Documentation and Clinicians' Electronic Health Record Experience: A Nonrandomized Clinical Trial. JAMA Netw Open. 2024 Sep 3;7(9):e2432460. doi: 10.1001/jamanetworkopen.2024.32460. |
| 30753666 | Background | Blackley SV, Huynh J, Wang L, Korach Z, Zhou L. Speech recognition for clinical documentation from 1990 to 2018: a systematic review. J Am Med Inform Assoc. 2019 Apr 1;26(4):324-338. doi: 10.1093/jamia/ocy179. |
| 31808131 | Background | Heckman J, Mukamal KJ, Christensen A, Reynolds EE. Medical Scribes, Provider and Patient Experience, and Patient Throughput: a Trial in an Academic General Internal Medicine Practice. J Gen Intern Med. 2020 Mar;35(3):770-774. doi: 10.1007/s11606-019-05352-5. Epub 2019 Dec 5. |
| 30242315 | Background | Bates DW, Landman AB. Use of Medical Scribes to Reduce Documentation Burden: Are They Where We Need to Go With Clinical Documentation? JAMA Intern Med. 2018 Nov 1;178(11):1472-1473. doi: 10.1001/jamainternmed.2018.3945. No abstract available. |
| 30242380 | Background | Mishra P, Kiang JC, Grant RW. Association of Medical Scribes in Primary Care With Physician Workflow and Patient Experience. JAMA Intern Med. 2018 Nov 1;178(11):1467-1472. doi: 10.1001/jamainternmed.2018.3956. |
| 36156143 | Background | Steinkamp J, Kantrowitz JJ, Airan-Javia S. Prevalence and Sources of Duplicate Information in the Electronic Medical Record. JAMA Netw Open. 2022 Sep 1;5(9):e2233348. doi: 10.1001/jamanetworkopen.2022.33348. |
| 27595430 | Background | Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders L, Westbrook J, Tutty M, Blike G. Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties. Ann Intern Med. 2016 Dec 6;165(11):753-760. doi: 10.7326/M16-0961. Epub 2016 Sep 6. |
| 39442042 | Background | Guille C, Sen S. Burnout, Depression, and Diminished Well-Being among Physicians. N Engl J Med. 2024 Oct 24;391(16):1519-1527. doi: 10.1056/NEJMra2302878. No abstract available. |
| 35710654 | Background | Lou SS, Lew D, Harford DR, Lu C, Evanoff BA, Duncan JG, Kannampallil T. Temporal Associations Between EHR-Derived Workload, Burnout, and Errors: a Prospective Cohort Study. J Gen Intern Med. 2022 Jul;37(9):2165-2172. doi: 10.1007/s11606-022-07620-3. Epub 2022 Jun 16. |
| 33434273 | Background | Moy AJ, Schwartz JM, Chen R, Sadri S, Lucas E, Cato KD, Rossetti SC. Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review. J Am Med Inform Assoc. 2021 Apr 23;28(5):998-1008. doi: 10.1093/jamia/ocaa325. |
| 33550392 | Background | Peccoralo LA, Kaplan CA, Pietrzak RH, Charney DS, Ripp JA. The impact of time spent on the electronic health record after work and of clerical work on burnout among clinical faculty. J Am Med Inform Assoc. 2021 Apr 23;28(5):938-947. doi: 10.1093/jamia/ocaa349. |
| 41497288 | Derived | Lukac PJ, Turner W, Vangala S, Chin AT, Khalili J, Shih YT, Sarkisian C, Cheng EM, Mafi JN. Ambient AI Scribes in Clinical Practice: A Randomized Trial. NEJM AI. 2025 Dec;2(12):10.1056/aioa2501000. doi: 10.1056/aioa2501000. Epub 2025 Nov 26. |
| 40672471 | Derived | Lukac PJ, Turner W, Vangala S, Chin AT, Khalili J, Shih YT, Sarkisian C, Cheng EM, Mafi JN. A Randomized-Clinical Trial of Two Ambient Artificial Intelligence Scribes: Measuring Documentation Efficiency and Physician Burnout. medRxiv [Preprint]. 2025 Jul 11:2025.07.10.25331333. doi: 10.1101/2025.07.10.25331333. |
| BG001 | Vendor B of Virtual AI Scribe Technology | Participants in this arm will utilize AI scribe tools from Vendor B and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note. The AI scribe tool is transcriptional and does not provide clinical decision support. Use AI Scribe tool provided by Vendor B: AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes. These tools are integrated into the EHR and automatically adds the generated text to the provider note. All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked. |
| BG002 | No Scribe | Participants in this arm will not have access to AI scribe tools and will continue their usual clinical documentation processes |
| BG003 | Total | Total of all reporting groups |
| Participants |
|
| Sex/Gender, Customized | Count of Participants | Participants |
|
| Ethnicity (NIH/OMB) | Count of Participants | Participants |
|
| Race/Ethnicity, Customized | Count of Participants | Participants |
|
| Region of Enrollment | Count of Participants | Participants |
|
| Specialty | Count of Participants | Participants |
|
| Time-in-note | Median | Inter-Quartile Range | minutes per note |
|
| Single-item burnout | The single-item burnout question is a component of the validated Mini Z 2.0 Survey. It is one of 10-items instrument designed to measure key factors influencing workplace satisfaction and burnout among healthcare professionals. The single-item burnout is scored on a Likert scale (1-5), with higher scores generally indicating more positive outcomes. Single-item scores >3 are considered "not burnt out". No patient level information will be collected for this outcome measure. | Mean | Standard Deviation | units on a scale |
|
| Clinic days per week | Mean | Standard Deviation | days/week |
|
| OG001 | Vendor B of Virtual AI Scribe Technology | Participants in this arm will utilize AI scribe tools from Vendor B and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note. The AI scribe tool is transcriptional and does not provide clinical decision support. Use AI Scribe tool provided by Vendor B: AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes. These tools are integrated into the EHR and automatically adds the generated text to the provider note. All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked. |
| OG002 | No Scribe | Participants in this arm will not have access to AI scribe tools and will continue their usual clinical documentation processes |
|
|
| Secondary | Provider Burnout Score | The Mini Z 2.0 Survey is a validated 10-item instrument designed to measure key factors influencing workplace satisfaction and burnout among healthcare professionals. Each item is scored on a Likert scale (1-5), with higher scores generally indicating more positive outcomes - greater job satisfaction, sufficiency of time for electronic medical record documentation, and lower levels of stress. For negatively framed items (e.g., stress due to the job or frustration with the electronic medical record), higher scores indicate lower levels of dissatisfaction. The total score ranges from 10 to 50, with scores ≥40 representing a joyful workplace. No patient level information will be collected for this outcome measure. | Providers who completed the follow-up survey. | Posted | Mean | Standard Deviation | score on a scale | Study month 2 |
|
|
|
| Secondary | Provider Task Load Score | Provider task load adapted from the NASA Task Load Index (TLX), a validated tool for assessing perceived workload across six sub-scales: mental demand, physical demand, temporal demand, performance, effort, and frustration. For this study, we adapted the TLX to focus on note-writing workload, including four sub-scales (mental demand, temporal demand, physical demand, and effort) as done previously. Each sub-scale is rated from 0 (low task load) to 100 (high task load) and summed together for a total score scale of 0 (low task load) to 400 (high task load), lower is better. No patient level information will be collected for this outcome measure. | Providers who completed the follow-up survey. | Posted | Mean | Standard Deviation | score on a scale | Study month 2 |
|
|
|
| Secondary | Provider Professional Fulfillment | The Professional Fulfillment Index (PFI) is a validated 16-item instrument that uses a 5-point Likert scale (0-4) to measure professional fulfillment, work exhaustion, and interpersonal disengagement. For this study, we utilize the 4-item work exhaustion subscale which is a mean of the 4-items within that subscale, where a low score (0) indicates a lower level of exhaustion and a high (4) score indicates greater level of exhaustion. No patient level information will be collected for this outcome measure. | Providers who completed the second survey. | Posted | Mean | Standard Deviation | score on a scale | Study month 2 |
|
|
|
| Secondary | Number of Physicians Who Are Considered Detractors, Passive, or Promoters | Self-reported satisfaction survey that asks physicians to consider note accuracy, patient safety, equity, and other potential unintended consequences and rate their overall likelihood to recommend use of the tool on a 1-10 scale. Higher scores (10) indicate greater satisfaction and likelihood to recommend, whereas lower scores (1) indicate dissatisfaction and unlikelihood to recommend. Providers are grouped as "Promoters" if they respond 9-10, "Passive" if they respond 7-8, and "Detractors" if they respond with a value less than or equal to 6. This grouping matches commonly accepted "Net Promoter Score" groupings. No patient level information will be collected for this outcome measure. | Scribe-assigned providers who completed the posttrial survey. | Posted | Count of Participants | Participants | Study month 2 |
|
|
|
| Secondary | Change in Provider RVU | The study team will use physician-level billing information via RVU to determine their change in productivity from a retrospective baseline 6 months prior to enrollment. No patient level information will be collected for this outcome measure. | Posted | Mean | Standard Deviation | monthly work RVU | Study month 2 |
|
|
|
| Secondary | Change in EHR Signal (Activity) Data - Pajama Time | We will examine change from a retrospective baseline 6 months prior to enrollment in Signal metrics including pajama time per scheduled day. Using this data will determine how a providers time is utilized in the EHR. No patient level information will be collected for this outcome measure. | Providers with reported data for this metric in Signal. To be included, a provider needs at least 5 appointments scheduled per week within the reporting period with Pajama Time. | Posted | Mean | Standard Deviation | minutes/day | Study month 2 |
|
|
|
| Secondary | Change in EHR Signal (Activity) Data - Time Outside Scheduled Hours | We will examine change from a retrospective baseline 6 months prior to enrollment in Signal metrics including time outside scheduled hours per scheduled day. Using this data will determine how a providers time is utilized in the EHR. No patient level information will be collected for this outcome measure. | Providers with reported data in this Signal metric. To be included, a provider needs at least 5 appointments scheduled per week within the reporting period. | Posted | Mean | Standard Deviation | minutes/day | Study month 2 |
|
|
|
| Secondary | Change in EHR Signal (Activity) Data - Time on Unscheduled Days | We will examine change from a retrospective baseline 6 months prior to enrollment in Signal metrics including time spent in the system on unscheduled days where . Using this data will determine how a providers time is utilized in the EHR. No patient level information will be collected for this outcome measure. | Providers for whom this record was reported by Signal. To be included, a provider needs at least 5 appointments scheduled per week within the reporting period. | Posted | Mean | Standard Deviation | minutes/day | Study month 2 |
|
|
|
| 0 |
| 79 |
| 0 |
| 79 |
| 0 |
| 79 |
| EG001 | Vendor B of Virtual AI Scribe Technology | Participants in this arm will utilize AI scribe tools from Vendor B and will continue their usual clinical documentation processes, supported by the scribe software, which integrates with the EHR and automatically adds the generated text to the note. The AI scribe tool is transcriptional and does not provide clinical decision support. Use AI Scribe tool provided by Vendor B: AI Scribe technologies capture physician-patient conversations to create a transcript, then summarize the transcript in the form of a clinical notes. These tools are integrated into the EHR and automatically adds the generated text to the provider note. All physicians must inform patients about the recording and obtain their verbal consent, and instances of patients declining to consent are tracked. | 0 | 79 | 0 | 79 | 0 | 79 |
| EG002 | No Scribe | Participants in this arm will not have access to AI scribe tools and will continue their usual clinical documentation processes | 0 | 80 | 0 | 80 | 0 | 80 |
Not provided
Not provided
Not provided
| Promoter (9-10) |
|