Background:

Relisten is an artificial intelligence-based software developed by Recog Analytics that improves patient care by facilitating more natural interactions between healthcare professionals and patients. Relisten extracts relevant information from recorded conversations, structuring it in the medical record and sending it to the Health Information System after the professional's approval. This approach allows professionals to focus on the patient without the need to perform clinical documentation tasks.

Method:

This Proof of Concept (PoC) study is conducted as a multi-centre trial with the participation of several health professionals in Primary Care Centres (CAPs) of Amposta, Centelles, Artés, Sallent, Súria and the Consorci d'Atenció Primària de Salut Barcelona Esquerra (CAPSBE). During the study, Relisten will be used in consultations under informed consent, followed by patient and professional surveys. Statistical analyses will be performed for each objective, using independent sample comparison tests according to normality evaluated with the Kolmogorov-Smirnov test and Lilliefors correction. The sample has been determined a priori to optimise the achievement of satisfactory results. Stratified statistical tests will also be performed to consider the variance between professionals.

Discussion:

The investigators expect an improvement in the quality of care perceived by patients and a significant reduction in the time spent taking clinical notes, with a saving of at least 30 seconds per visit. Although a high quality of the notes generated is expected, it is uncertain whether a significant improvement over the control group, which is already expected to have high quality notes, will be demonstrated.

In the medical field, medical record writing is an essential task that requires time and accuracy on the part of healthcare professionals. The medical record, which includes the patient's medical history, any symptoms they have had, treatments performed, and other relevant details, is a critical component in making appropriate medical decisions and ongoing patient follow-up.

In the modern healthcare context, there has been a transition to the digitization of these records, giving rise to the concept of the Electronic Medical Record (EMR). An EMR is the electronic representation of a patient's medical record, created and maintained by healthcare professionals. This digital approach has not only revolutionised the way medical information is stored and accessed but has also improved the efficiency of medical care by facilitating the retrieval of relevant data at the point of care. EMRs provide a centralised platform for medical information management, allowing for more accurate tracking and more coordinated care.

Traditionally, healthcare professionals have spent a significant amount of time writing medical records, which can affect the efficiency and quality of care they provide. This manual task is not only time-consuming, but can also lead to documentation errors, omissions or inconsistencies in the information recorded.

In recent years, the field of Artificial Intelligence (AI) has experienced significant advances in natural language processing and speech recognition. These advances have enabled the creation of automated tools and systems that can accurately and efficiently transform speech into text. In the healthcare setting, this technology has the potential to streamline and improve the writing of medical records, freeing up time for professionals to focus on direct patient care. However, this technology was necessary but not sufficient, and it was not until the advent of generative AI that a key part of the process could be completed to obtain sufficient quality for practical use.

In this context, the Spanish company Recog Analytics has developed Relisten, an automated clinical note writing system that stands out for its specialisation in 1) Face-to-face consultations. 2) Non-guided consultations, in colloquial language to maintain a close relationship with the patient. 3) Multi-language queries (currently supports Castilian Spanish, Catalan, English, Portuguese and others). 4) Easy integration with the electronic medical record and simple use for healthcare professionals. 5) Maximum quality and structuring of the information extracted.

Through a natural conversation between the healthcare professional and the patient, Relisten uses recordings to extract relevant fields for the medical history in a structured way and then send them to the Health Information System (after correction/approval by the healthcare professional). By employing this approach, the healthcare professional can devote full attention to the patient without the need to perform multiple simultaneous tasks to document clinical notes.

Methods/Design

This study has the following objectives: 1) To increase the quality of care perceived by patients with the use of Relisten in the consultation, measured by means of anonymous satisfaction questionnaires at the end of each consultation (Annex I). 2) To increase the satisfaction of health care personnel with the care provided in consultations, measured by means of an anonymous satisfaction survey at the end of the study and a structured interview. 3) To reduce the time spent by healthcare personnel on entering records in the EMR, measured by statistical tests comparing consultations with and without the tool.

A proof of concept (PoC) will be carried out in the context of a multi-centre study, where several health professionals from different Primary Care Centres (CAPs) will use the Relisten tool in consultations (under informed consent of the patient) and will survey patients and the professionals themselves.

The project is structured in three main phases:

1. Preparation: This includes the demonstration and initial training of the participating professionals, the preparation of the technological infrastructure (microphones) and the coordination for carrying out the surveys, with an approximate duration of 2 weeks.
2. Development: This consists of the use of Relisten during consultations, as well as data collection, including audios, clinical notes and survey results, lasting approximately 20 weeks.
3. Data analysis and presentation of results: This phase involves the processing and analysis of the data collected, followed by the presentation of the results obtained, and lasts approximately 2 weeks.

The measurement of the objectives established above will be carried out using the following methods:

• Quality of care as perceived by patients: By means of an anonymous patient survey after each consultation, in a patient-blinded study (patients are not told beforehand whether Relisten has been used in the consultation; this fact is outlined by a graphic mark in the surveys in which Relisten has been used).

• Satisfaction of healthcare personnel: An anonymous survey and an interview will be conducted at the end of the study to understand the impact the tool has had on consultations in a qualitative way. The survey will include at least an assessment of the following aspects:

  • The general usefulness of the tool
  • Ease of use of the tool
  • Quality of the notes generated by the tool
  • The extent to which it is perceived to improve attention span
  • The extent to which it is perceived to save time

• Saving time spent entering information into the EMR To determine the magnitude of savings, two measurements taken during the proof of concept will be compared:

  • The time the healthcare professional spends typing in the consultation.
  • The time the healthcare professional spends correcting Relisten's notes. The study population will be made up of patients who are visited in person at Primary Care Centres in Catalonia, both in emergency and control consultations, whose professionals participate in the study.

Given the nature of the tool and its applicability to most of the practice setting, there will be no prior selection of patients, but any patient who comes for consultation with the professionals participating in the study will be eligible.

Inclusion criteria:

• To have signed the informed consent form.
• To carry out face-to-face medical and nursing consultations in Catalan and/or Castilian Spanish.
• To be over 18 years old.

Exclusion criteria:

• Inability to understand the nature of the study.
• Not fluent in Catalan or Castilian Spanish.
• Not giving consent to be recorded.
• Existence of any technical failure in the recording (e.g., internet service downtime).

The study included patients in unscheduled visits (emergency), first visits (with history taking) and follow-up for chronic disease.

A sample size of 400 for each arm will be needed.

The randomisation was carried out in a simple format, given the sufficient sample size and the fact that the aim is to validate the hypotheses at a general level, although for information purposes the study also shows intermediate results stratified by type of consultation (first/follow-up) or by health professional.

The data collection process involves the following:

• Satisfaction surveys completed by patients at the end of the consultation (with Relisten as a test and without Relisten as a control).
• Quality surveys at the end of each consultation by healthcare professionals, on a scale of 0 to 10 with the possibility of including a qualitative comment, will be carried out directly through the Relisten digital platform.
• Satisfaction surveys completed by healthcare professionals at the end of the study, as well as interviews to better understand the impact and limitations in a qualitative manner.
• Recordings of conversations between the patient and healthcare professional (Recog will delete the recordings at the end of the study).

Study variables

• Independent:

  • Indicator of whether Relisten has been used in the consultation (Yes/No)
  • Identifier of the healthcare professional involved

• Dependent:

  • Results for each survey question, on a scale of 1 to 5.
  • Time saved in consultation (calculated from writing time measured in non-Relisten audios, i.e., where the doctor writes during the consultation, and Relisten note correction time, timed by the doctor through the Relisten digital platform). The writing time is measured individually for each recording. Relisten investigators will listen to the control recordings and measure the time spent on note writing by the professionals.

Statistical analysis

The main hypotheses of the study focus on improving the perceived quality of care and saving time in the writing of clinical notes. To assess the improvement in perceived quality of care, the normality of the sample will be checked and Student's t-test for independent samples will be applied in the case of normality, or an equivalent non-parametric test otherwise. In terms of time savings, two aspects will be measured separately: the relative time (in percentage) that the professional spends writing notes during the consultation without using Relisten, and the relative time that the doctor spends reviewing the notes generated by Relisten. Subsequently, the normality of the sample will be checked and the Student's t-test or its non-parametric equivalent will be applied as appropriate. All statistical analyses will be performed using R Studio, considering a confidence level of 95% and a statistical power of 80%.

Confidentiality

In this study, confidentiality will be rigorously protected by several procedures. The patient surveys, which will be conducted anonymously upon exiting the consultation, will be transcribed by the Recog staff into an Excel file for subsequent analysis, without any identifying data. The audios collected during the consultations will be automatically stored in the Relisten platform, guaranteeing their security by storing them in the S3 service of AWS, with controlled access, encryption, and without identifiers that allow patients to be identified.

Discussion The implementation of artificial intelligence-based technologies in healthcare has been a topic of growing interest in the last decade. These technologies promise to improve the efficiency and quality of healthcare, but their adoption depends largely on empirical evidence to support their benefits.

This study seeks to contribute to that evidence base by evaluating a specific tool that has the potential to alleviate one of the main sources of administrative burden for healthcare professionals: the writing of clinical notes. By freeing up time that would otherwise be spent on administrative tasks, Relisten could enable healthcare professionals to focus more on direct patient care, thereby improving the quality of care.

The results of the project will serve to validate the usefulness of Relisten in daily practice, from the perspectives of improving the quality of care and saving professionals'; time, which could amount to more than an hour a day that could be invested in attending to more patients, promoting adherence with the same patients, or other value-added tasks.

One of the main limitations of the study is the variability introduced by the participation of different health professionals, who can generate a significant variance in the results obtained, both in patient satisfaction and in the time saved in the consultation. To mitigate this effect, it has been decided to involve a reasonable number of professionals, seeking a balance between reducing variance and avoiding an excessive burden on participants, although this will still be a limiting factor.

Inclusion criteria:

• To have signed the informed consent form.
• To carry out face-to-face medical and nursing consultations in Catalan and/or Castilian Spanish.
• To be over 18 years old.

Exclusion criteria:

• Inability to understand the nature of the study.
• Not fluent in Catalan or Castilian Spanish.
• Not giving consent to be recorded.
• Existence of any technical failure in the recording (e.g., internet service downtime).

The datasets used and/or analysed during the current study will be available from the corresponding author on reasonable request.