Can Feedback From a Large Language Model Improve Health Care Quality?

A Pilot Ptudy of an LLM Tool to Support Frontline Health Workers in Low-Resource Settings

The goal of this study is to learn if computer-assisted advice can help improve patient care in Nigerian health clinics. The main question it aims to answer is: does giving healthcare workers instant computer feedback help them make better decisions about patient care?

Researchers will compare patient care notes written by healthcare workers before and after they receive computer feedback to see if the feedback improves care quality. A doctor who doesn't know if feedback was given will review these notes.

Participants will:

• Be seen by a community healthcare worker who uses the computer feedback system
• Be treated by a fully trained medical doctor
• Get tested for malaria, anemia, or urinary tract infections if they have certain symptoms

This project tests whether Large Language Models (LLMs) can improve patient care in Nigerian primary care clinics by giving customized and instant feedback to the provider in natural language. An LLM-based tool integrated into an electronic patient record management system provides "second opinions" to community health extension workers (CHEWs) at two clinics in Nigeria. These second opinions are intended to mirror what a reviewing physician might advise the CHEWs after seeing or hearing their initial report on a patient.

For the main analysis, this study employs a within-patient comparison of two patient notes created by the CHEW; one during the initial patient consultation, and one after the LLM feedback was received. The patient is also seen by a fully trained medical officer who is in charge of patient care. The MO conducts a blinded review of the CHEW's patient notes to measures changes in the CHEW's care as a result of the LLM feedback. The data comes from the information captured in the electronic medical record (EMR) of the patient and from survey data collected from CHEWs, reviewing MOs, and a panel of reviewing Medical Doctors.

Inclusion Criteria:

• Patient is at the clinic for outpatient consultation
• Parent/guardian consent is required for individuals under 18

Exclusion Criteria:

• Patient does not require emergency care
• Patient is not at the clinic for a checkup (e.g. weight, blood pressure, follow up after recovery)
• Patient is not a trauma patient (visit is not for an accident, wound or injury)
• Patient is not at the clinic for a scheduled procedure or a birth