Digital health technologies (DHT) are increasingly developed to support healthcare systems around the world. However, they are frequently lacking evidence-based medicine and medical validation. There is considerable need in the western countries to allocate healthcare resources accurately and give the population detailed and reliable health information enabling to take greater responsibility for their health. Intelligent patient flow management system (IPFM, product name Klinik Frontline) is developed to meet these needs. In practice, IPFM is used for decision support in the triaging and diagnostic processes as well as automatizing the management of inflow of the patients. The core of the IPFM is a clinical artificial intelligence (AI), which utilizes a comprehensive medical database of clinical correlations generated by medical doctors.

The study population of this research consists of patients from the Emergency Department of Kuopio University Hospital (KUH). Data will be gathered during 2 weeks of piloting, after which the results will be analysed. Anticipated number of patients to the study is minimum of 246 patients, with objective to be several hundreds. When attending to the hospital, patients will report their demographics, background information and symptoms using structured IPFM online form. Patients entering the unit in an ambulance or with need of immediate care of healthcare professionals due to severe and acute conditions are referred similar to normal process to ensure the patient safety. Results obtained from IPFM are blinded from the healthcare professional and IPFM does not affect professional's clinical decision making in any way. The data obtained from IPFM online form and clinical data from the emergency department and KUH will be analysed after the data collection.

The main aim of the research is to validate the use of IPFM by evaluating the association of IPFM output with 1) urgency and severity of the conditions (using Emergency Severity Index [ESI], an international triaging protocol for emergency units, and an assessment by triage nurse); and 2) actual diagnoses diagnosed by medical doctors. The main hypotheses of the research are that 1) IPFM is safe and sensitive in evaluating the urgency of the conditions of arriving patients at the emergency department and that 2) IPFM has sufficient correlation of differential diagnosis with actual diagnosis made by medical doctor.

1. Background

   Continuous development of digital technology provides new opportunities also in healthcare area. As healthcare costs are constantly growing both worldwide and in Finland (1), innovations are needed to enhance allocation of limited healthcare resources and to reduce economic burden. Digital health technologies (DHT) have the potential to reduce these costs. They are increasingly developed to support healthcare systems around the world. DHT are, however, often lacking evidence-based medicine (2) and both the importance and the lack of their validation has been acknowledged widely (3, 4).

   Work in triage and urgent care centers is highly demanding, with considerable time pressure. When operating with high patient volumes, acute conditions and severe stress, human errors are likely to result at least at some extent. Physicians have been found to have a ~5 % diagnostic error rate (5) with half of these potentially harmful (6). DHT have been suggested to have a significant role helping both patient management and triage (7), and thus, reducing also potential human errors. For instance, crude online clinical decision support tool has been found to reliably demonstrate emergency triage severity in experimental setting (7).

   The validation of DHT and decision support tools is highly crucial. Tools with false negative rates could have serious consequences when ignoring severe conditions such as cardiac ischaemia (4). DHT should be thorough and optimal in the medical field and their internal and external validity should have been evaluated. Recently, guidelines for evaluating DHT have been published (4). According to guidelines, the first step in the assessment is to show the results in an early observational study in clinical setting. In addition to safety, also efficacy and effectiveness of DHT should be evaluated.

   To meet these needs, Intelligent Patient Flow Management system (IPFM) has been developed to help the user (i.e. patient) and the healthcare professionals to evaluate user's condition and its severity. IPFM enables semi-automated triaging of patients and enhances the management of patient inflow and demand. In addition, IPFM provides written and precise information for the healthcare professionals communicated by the patients themselves of their conditions and symptoms. The core of IPFM is a broad medical database generated by medical doctors, combined with artificial intelligence (AI) and feedback loops for accuracy improvement. In other words, IPFM is an AI-induced triage and decision support tool developed and monitored by medical doctors to help healthcare professionals to evaluate the urgency of the patients. It evaluates user's condition and infers the urgency using user demographics, background information and symptoms.

   The IPFM is developed by a healthcare technology company Klinik Healthcare Solutions Oy and it has already been implemented in over 400 primary care and dental facilities in the Nordics and was recently launched in the United Kingdom and Portugal. It has been previously shown to reduce the costs in primary care by 14 % in overall total service costs of the patients by streamlining the patient flow (8). IPFM is currently classified and registered as a CE class 1 medical device, which requires continuous scientific validation research and systematic development and quality management processes. The system is used according to its intended use in this study and therefore the study does not require permission or registration for FIMEA's clinical device study register.

   Klinik's IPFM (product names Klinik Access, Klinik Frontline) provides a full-stack solution, with interface for patient input, backend for database and algorithm calculations and interface for handling of case outputs for triaging and combined statistics for healthcare staff. IPFM uses medically golden standard Bayesian methodology for inferring the effect of clinical features on the probabilities of the conditions. The severity and the urgency of the condition is inferred using specific severity symptoms and by setting a threshold for the probabilities of relevant conditions. All inflow of data is dynamically gathered and analysed ensemble and visualised, which can be used by (management) staff to monitor and optimally meet demand ad hoc.

2. Aims and objectives

   The main aim of this study is to validate the use of IPFM in a hospital setting by evaluating the association of IPFM output with 1) clinical urgency and severity of the conditions (using Emergency Severity Index [ESI], an international triaging protocol for emergency units, and an assessment by triage nurse); and 2) actual diagnoses made by the hospital doctors. The objective is also to assess the correlation of IPFM output with redirection or referral to various specialties (e.g. internist or surgeon).

   Main hypotheses are:

   • IPFM is safe (< 1 % emergency cases missed) and accurate (> 80 % agreement with triaging nurse) in evaluating the urgency of the conditions of arriving patients at the emergency department
   • IPFM has sufficient correlation (> 60 % overlap) of differential diagnosis with actual diagnosis diagnosed by medical doctor

   Validating IPFM, an AI-induced triage support and decision support tool, in real-life environment and in hospital setting is essential for its use in hospitals and clinical work in the future to support evidence-based medicine. An accurate and efficient tool will decrease delays of treatment of urgent conditions and has the potential to decrease human errors that are inevitable in environment with high patient volumes, acute conditions and severe stress. When in full implementation, it can also help to optimize emergency department resources and to target them more accurately. In addition, this study provides important information on patient history and symptoms provided by patients themselves when entering emergency department compared to clinical data.

3. Materials and methods

   The study population consists of patients entering the emergency department of Kuopio University Hospital (KUH) during a 2-week research period. The calculated needed sample size is a minimum of 246 patients (confidence level 95%, margin of error 5%, study population proportion 80%). The data consists of registry data from the electronic health records, other quantitative data (such as extracts from KUH emergency department reporting software) and qualitative data acquired through observation. Both qualitative and statistical methods will be applied to the data to infer outcomes.

   Case-specific information in the IPFM (demographics, background information, symptoms) provided by the patient will be analyzed with clinical data (Emergency Severity Class index (ESI), triage category, diagnoses) collected from the electronic patient records of the hospital (emergency and other possible departments) and IPFM professional dashboard, and the safety, specificity and sensitivity of IPFM output are analyzed.

   The urgency of the condition is evaluated by IPFM and the analyses with clinical data and validation will be performed after the data collection. They urgency is evaluated by IPFM as: 1. suitable for self-care if no prescription medication or other treatment is suggested (i.e. doesn't require doctor's evaluation, however might benefit from nurse check-up and advice); 2. non-urgent if ailment is prolonged or chronic, but benefits from doctor's evaluation and would more efficiently be managed through non-urgent appointment (> 3 days); 3. urgent if ailment is (semi-)acute and benefits from doctor's evaluation within 1-3 days; and 4. emergency if ailment requires evaluation and treatment within the same day (or immediately) to avoid risk of serious health hazard. This output is correlated with the ESI classification used by KUH emergency and other data collected from electronic health records regarding urgency of the condition.

   When attending to the hospital, patients will report their demographics, background information, symptoms, other case specific data and satisfaction using structured IPFM online form. Patients entering the unit in an ambulance or with need of immediate care due to severe and acute conditions are managed through traditional process: they do not fill the form and, hence, are out of scope of the pre-evaluation process and this study. All information gathered is transferred to the IPFM dashboard, where the patient reported clinical information, AI-induced suggestions on the urgency of the care needed and potential diagnosis (a differential of 3-10 probable diagnosis) is demonstrated. However, in this study design, healthcare professionals cannot see any clinical information in the IPFM dashboard, i.e. the output of IPFM is blinded.

   Patient data will be saved to the registry from IPFM: 1) age; 2) gender; 3) duration of the symptoms; 4) body location of the symptoms; 5) symptoms chosen from the symptom cloud(s) by the patient (symptom cloud is dynamic, consist of 430 potential symptoms, symptom selection varies depending on the previous actions of the patient); 6) severity symptoms chosen from the severity symptom cloud (severity symptom cloud is dynamic, consist of 100 potential severity symptoms, selection varies depending on the previous actions of the patient); 7) free text fields regarding the selected symptoms; 8) answers (yes/no) to the questions: a) have you used any medication or other treatment for your ailment, b) has someone (e.g. a physician or a physiotherapist) already examined or treated your ailment; 9) identification number of the form; 10) unique case id number; 11) name; 12) social security number; and 13) telephone number.

   After the primary IPFM data collection stage at the beginning of each ER visit, a secondary stage of data retrieval is initiated after the study period. During this only critical information required to assess the validity of the IPFM system outcomes will be fetched from the electronic health care system: 1) ESI classification; 2) diagnosis/diagnoses of the patient; 3) speciality & care path chosen in the urgency care; 4) traditional non-ESI urgency assessment comparable to our urgency scale (if stored). The final data points will be equivalent to these, but might due to changes in ER practices and/or data systems vary. The data points are chosen to optimally allow comparative analysis of results. This information is only used after data collection when evaluating the validity of IPFM.

   Despite excluding patients with need of acute immediate care of healthcare professionals (arriving with an ambulance), also children/adolescents and patients with otherwise restricted capabilities or developmental disorders are excluded from the study. Adult patients with mild mental disorders can be included, excluding the patients with guardian. Additionally, pregnant or breastfeeding patients are excluded from the study.

4. Ethical aspects of the research

   Participation to the study is voluntary for the patients. After receiving oral and written information on location at KUH emergency department, patients will sign an informed consent form. As described in the chapter 5, the study protocol will not harm any study subjects and will not interfere negatively in their treatment. The patients participate in the study only once, on location when arriving to the emergency department, and thus no further activity nor financial burden is generated.

   An independent research data registry will be generated for this study. When creating the registry, social security number or telephone number is used to merge the data from electronic health records and IPFM. After the data has been merged and the combined registry is created, randomly generated case ids are used as research subject identifiers and information that allows for identification of single patients is not available. The merging of the data is done by the KYS staff prior to access into research registry being granted for the research group. The combined research registry will be stored in a digital format, on a computer owned only by the research group members. There will be a password protection for a user to enter a) the computer and b) the registry file itself. The file will be stored on a local folder not accessed online (i.e. not a cloud storage folder). The registry file will be stored for five (5) years after the research has deemed finished and will be deleted after the period according to the required handling standards. The administrator of the case ids is the person responsible of the project and Klinik Healthcare Solutions Oy is responsible for storing the registry file. The study will be registered into international "ClinicalTrials.gov" research register. The results of this project will be reported in a group level and, thus, no individual information can be recognized.

   The application for the ethical committee will be submitted in May 2020.

5. Risk assessment

   The healthcare professionals are evaluating the patients by their own clinical judgement and they do not utilize output of IPFM in their decision making. IPFM data is only assessed after the data collection.

   There is a potential risk of acquiring volunteers below the intended volume for the study. This will be addressed by extending the study period or overlap of daily working hours.

   There is a hypothetical risk of causing undesired delay in the registration process of the patients that would cause critical cases to be missed due to increased waiting times for the registration. Our previous user studies have shown that filling out the IPFM form takes circa 3-5 minutes, which would not be a life threatening delay for walk-in patients. This risk will be monitored and if noticed, managed in real time.

6. Time schedule and funding plans

   This study is initially planned to be executed between 1.7.2020 and 30.6.2023. Duration of the data collection is approximately 2 weeks and is designed to complete during fall 2020.

   The project is anticipated to have a small scale economic effect for the parties. The patients in this study will not experience any economic harm. The service provider Klinik Healthcare Solutions Oy will cover for the costs of it's researchers and assisting staff in the form of a moderate hourly pay. The service provider will also provide the digital equipment used in the research in collecting the information from the patients. In the approval of the equipment it will work closely with the ICT department of Kuopio University Hospital District. The hospital or the emergency department will not have any additional costs regarding the research project. There are no additional economical compensations to the researchers or to the study population.

7. Research group, resources and conflicts of interest

   The research group consists of MD, PhD Juhani Määttä (Medical Research Center Oulu, University of Oulu), MD, PhD candidate Petteri Hirvonen (University of Helsinki), MD Rony Lindell (University of Helsinki) and PhD Tero Martikainen, chief doctor of Kuopio University Hospital A&E department.

   Juhani Määttä, Petteri Hirvonen and Rony Lindell have financial and business interests in Klinik Healthcare Solutions Oy.

8. Expected results and scientific impact

The study is expected to validate high accuracy (i.e. safe and reliable usability) of IPFM tool in KUH hospital context. The results are generalizable to other hospital emergency departments, however with caution by doing contextual analysis and process comparisons prior to implementation considerations. Through assessing IPFM output with data from patient's electronic health records, the study is expected to show: 1) strong correlation of the urgency category; and 2) good correlation between calculated differential diagnosis and the diagnoses of ED or ward doctors. The purpose is to publish the results in internationally recognized medical journals with high quality.

According to our review of the literature on this subject, an AI-supported automation of the inbound patient traffic in an emergency department setting has not been studied previously, especially not in this scale and patient variation. An increasing public and professional interest is arising regarding AI-based solutions to support professionals in their work and decision making. To meet these needs, this study provides and scientifically validates a solution which works semi-independently, taking up initial tasks normally reserved for healthcare professionals. The research group anticipates this being of great interest to the scientific community - and will add to the discussion of the possibilities of the DHT to provide marked cost-effectiveness while not compromising on patient safety and effectiveness of the use of professional resources.

Inclusion Criteria:

All adult (>18 years of age) patients independently (walking) entering emergency care ward with written consent.

Exclusion Criteria:

• Patients arriving with ambulance
• Patients needing immediate care
• Patients under 18 years of age
• Patients with restricted capabilities or developmental disorders.

Patients living in the district of North-Savo (250 000).