Development of a Model for Digital Monitoring of the Mental State of the Hospitalized Patient

Development and Validation of an Experience Sampling Method Questionnaire (ESM) for Digital Monitoring of Mental State in Psychiatric Hospitalization: Convergence of ESM Reports, Standard Clinical Assessments, and Smartwatch Data

This study presents the development and validation of a unique Digital Experience Sampling Method (ESM) questionnaire specifically adapted for monitoring changes in the mental state of patients during psychiatric hospitalization. The questionnaire was carefully crafted through focus groups involving patients and clinical staff, ensuring its relevance and applicability to the unique characteristics of mental state changes in a hospitalization setting.

To evaluate the validity of the ESM questionnaire, symptom severity trends obtained from the questionnaire will be compared with estimates derived from the Positive and Negative Syndrome Scale (PANSS) assessment. Data will be collected from 100 subjects over a 14-day psychiatric hospitalization period. In addition to the ESM questionnaire, smartwatch sensors will monitor physiological indicators.

Feasibility and patient compliance will be assessed by examining patients' willingness to use the digital ESM questionnaires and the smartwatch sensors. The study will also cross-reference self-reported sleep quality and activity levels captured in the ESM questionnaires with objective physiological indicators and nursing staff reports, providing insights into the reliability of the patient-reported data.

Furthermore, the study will evaluate the impact of the ESM data on clinical decision-making by physicians throughout the 14-day psychiatric hospitalization period. Patient satisfaction and satisfaction among the multidisciplinary team with the monitoring model will also be assessed.

This research underscores the potential of digital technologies to enhance patient-centered care and facilitate informed treatment decisions in psychiatric hospitalization settings.

The research will be carried out in two phases:

1. Development of the monitoring questionnaire - Content validity phase - This phase has been fully completed, and its results will be presented in the preliminary results chapter. At this research stage, the core content domains of the monitoring questionnaire will be developed. The research process in this phase includes the following:

1. Examining the content of scales from the Positive and Negative Syndrome Scale (PANSS) for schizophrenia and The Symptom Checklist-90-R (SCL-90-R) questionnaire: At this stage, the psychiatric symptoms and the main criteria for assessing a mental state as emerged from those questionnaires were examined for discussion in the focus groups.

2. Focus group No. 1: The first focus group included patients hospitalized in the psychiatric ward. The purpose of the focus group is to formulate the ESM core and additional content domains and the questionnaire items and to discuss the optimal measurement frequency according to the hospitalized patients' opinions. The focus group will also examine the wording that best suits how the patients describe their mental state.

   Indices The subjects' point of view regarding the psychiatric symptoms they suffered from during their hospitalization was examined during a free discussion. Emphasis was placed on the symptoms that change daily throughout the hospitalization and how much they indicate a change in their mental state. The investigators also investigated what information they would like to report to the treatment team about their mental state throughout the hospitalization. After the free discussion, the position of the subjects was examined regarding the criteria for assessing mental state as they emerged from the monitoring scales of the PROM questionnaires.

   After a qualitative analysis of the information collected at this stage, the main themes of the ESM questionnaire were formulated. According to the themes found, suggestions were developed for the questionnaire items while noting how the patient's symptoms were worded. A professional transcriber transcribed the discussion.

3. Focus group No. 2: The second focus group includes a multi-disciplinary team working in the psychiatric inpatient wards. In a free discussion, the multi-disciplinary team members examined the core issues that should appear in the questionnaire for monitoring the mental state of hospitalized patients, then explored and expressed their opinion regarding the issues raised by the patients in the first focus group. The result of this step will be an ESM questionnaire adapted to the needs of the patients and the clinical staff in the inpatient wards.

   Indices In the discussion, the participants floated important themes for monitoring according to their points of view. After a free discussion, the participants were presented with the themes raised by the patients in the previous focus group. They were also presented with the possible questionnaire items formulated after the last stage, and their position was examined regarding the appropriateness of fitting these items to the questionnaire. In addition, the discussion dealt with the optimal sampling frequency, in their opinion.

   The initial formulation of the ESM questionnaire items. The questionnaire items were formulated considering the information collected in the focus groups. An effort was made to use the patients' words to ask about their mental state. In addition, existing and valid self-report items marked as necessary by the patients and staff in the focus groups discussion were selected. Items from these questionnaires were reformulated according to the ESM approach that examines the current state. At the end of this stage, an ESM questionnaire was developed, which consists of 15 items on a five levels Likert scale. In the questionnaire, there was another item of a Visual Analogue Scale type where the subject had to mark his mood on a scale ranging from 1-100.

4. Focus group No.3: The third focus group included hospitalized inpatients for cognitive debriefing - Cognitive debriefing is the process by which a patient questionnaire is actively tested among representatives of the target population and target language group to determine whether respondents understand the questionnaire as the authors intended it to be understood.

Indices For each item in the developed ESM questionnaire, subjects were asked to describe freely what they thought the item was testing. In addition, to each item, the investigators discussed how appropriate the question was in their opinion to describe their mental state throughout the hospitalization and whether they would change anything in the wording. At the end of this part of the discussion, the subjects were asked how many times a day they would be willing to report their mental state, how many items they would be ready to answer, and what additional questions they would like to be asked to learn about their condition. In addition, the investigators examined the patients' attitudes regarding their agreement to answer questions via mobile phone and their agreement to continue answering a similar questionnaire after their discharge from the hospital.

Research Process The focus groups were held at Sheba Medical Center on 5/18/22, 08/9/22, and 11/09/22. The groups were guided by the researcher and another facilitator, who work as senior clinical psychologists in the psychiatric inpatient wards at the Sheba Medical Center. A professional transcriber attended each group and typed the conversation throughout the focus groups. The transcripts of the discussions in the focus groups were qualitatively analyzed. After collecting and processing the data from the focus group, the ESM questionnaire was improved and refined, considering the findings.

2. Running the monitoring model - feasibility phase - this phase will begin in the middle of 2023 and is expected to continue for 18 months.

Participants

To conduct the monitoring model, the investigators will recruit 100 subjects hospitalized in the inpatient wards. To receive feedback on implementing the monitoring model from the multi-disciplinary care team working in the wards, the investigators will recruit 30 multi-disciplinary team members. The running will be carried out in two stages:

Pre-Test - 20 patients and feedback from 5 multi-disciplinary - estimated time - 3 months.

Field Test - 80 additional patients and feedback from 30 multi-disciplinary staff members - estimated 12 months.

Indices Pre-test indices -

1. Selection of the ESM questionnaire items with the best distribution indices and improving the wording of items with low variability scores within and between subjects. Feedback from staff? Field test indices -

2. The effect of exposing the psychiatrists to the patient's monitoring indicators on their clinical decisions regarding him will be examined in a short questionnaire that the physician will fill out after each medication change and mark on a Likert scale of 1 5 - how much the exposure to the monitoring indicators influenced the clinical decision.

3. Feasibility and compliance:

   1. The responsiveness to participation in the monitoring model - The number of subjects who agreed to participate in the study out of the subjects the investigators offered to participate.
   2. The extent of responding to the self-report questionnaires over 14 days multiplied by three times a day.
   3. The degree of responsiveness to wearing and charging the smartwatch for 14 days.
   4. The degree of difference between subjects with different psychiatric diagnoses (psychotic spectrum, depression, mania, and personality disorders) in their adherence and responsiveness to agree to participate in the study, answer the self-report ESM questionnaires and wear and charge the smartwatch.

4. The validity of the subject's self-reports on sleep quality and activity level:

   1. The validity of the subject's self-reports on sleep quality -The subject's self-reported data about his sleep quality and number of awakenings will be compared to physiological indicators that the smartwatch will collect (number of hours of sleep, number of awakenings) and will be equalized in addition to the medical reports of the nursing staff in the ward.

   2. The validity of the subject's self-reports on his activity level:

      The subject's self-reported data about his energy and activity level will be compared to physiological indicators that the smartwatch will collect (number of steps, accelerations) and will be equalized in addition to the reports of the nursing staff in the ward.

   3. The degree of difference between subjects with different psychiatric diagnoses (psychotic spectrum, depression, mania, and personality disorders) in correspondence between the self-report and the physiological indices and the reports of the nursing staff.

5. Changes in the hospitalized patient's condition over the timeline: Changes within 14 days of hospitalization will be presented through the patient's self-report indicators, the physiological indicators collected by the sensor, and the reports of the multi-disciplinary team in the ward.

6. Patient satisfaction: After 14 days of monitoring, the subjects will report their satisfaction with using the monitoring model throughout the hospitalization through questionnaires.

7. The satisfaction of the multi-disciplinary teams: At the end of the study, the teams will report through questionnaires on their satisfaction with using this monitoring model and its influence on the team's clinical decision-making.

Research Process After recruiting the patient for the study and signing informed consent, the monitoring application (Datos Health) and the wearable smartwatch application (Garmin) will be installed on the patient's smartphone. The patient will receive an extensive explanation from the research team's representative on the self-reporting method and will experience using the application.

From this stage, the hospitalized subject will receive ongoing clinical care, alongside participation in monitoring indicators by self-reporting in the ESM questionnaires using the Datos Health application and physiological indices monitoring by using the Garmin smartwatch. The trial will end after 14 days or until the patient's discharge (whichever comes first). The ESM self-report sampling will be conducted a frequency of 3 times a day in a semi-random manner:

• The first sample will be randomly sampled between 8-10 am.
• A second sample will be randomly sampled between 1-3 pm.
• A third sample will be randomly sampled between 7-10 pm. In each clinical discussion of the subject's mental state, the ward staff will watch the dashboard of the Datos Health application, which will display the subject's self-report measures and physiological indicators. For each change in the patient's clinical status, determined by the physician (change of medication, vacation, preparation for discharge, discharge), the physician will answer a questionnaire in which he will document how much (between 1-5) the clinical decision was influenced by the monitoring indicators he observed.

The feasibility and compliance, Validity, and changes in the patient's condition over timeline indicators will be collected for 14 days and analyzed at the study's end. The patient satisfaction indicators will be tested after 14 days of monitoring through a questionnaire sent to each subject. The satisfaction indicators of the clinical staff in the ward will be tested through questionnaires sent to the clinical staff at the end of the study.

Focus group no.1 :

Inclustion Criteria-

• Hospitalized by consent in the psychiatric wards at the Sheba Medical Center.
• Signed informed consent to participate in the study. Exclusion Criteria-
• Patients with a low level of compliance to treatment.
• Patients suffering from a significant active physical illness, intellectual disability, or complex organic conditions including dementia conditions
• Patients who cannot read and write in the Hebrew language
• Patients who are not qualified to give informed consent.

Focus group no.2:

Inclusion Criteria:

• Staff members work in the inpatient wards and have a valid Israeli license.
• Signed informed consent to participate in the study.

No Exclusion Criteria.

Focus group no.3-

Inclustion Criteria:

• Hospitalized by consent in the psychiatric wards at the Sheba Medical Center.
• Signed informed consent to participate in the study.

Exclusion Criteria:

• Patients with a low level of compliance to treatment.
• Patients suffering from a significant active physical illness, intellectual disability, or complex organic conditions including dementia conditions
• Patients who cannot read and write in the Hebrew language
• Patients who are not qualified to give informed consent.

Monitoring model- The investigators will recruit 100 subjects hospitalized in the inpatient wards.

Inclusion criteria:

• Adults are consenting hospitalized in a psychiatric hospital at the Sheba Medical Center.
• Staying in the open part of the ward.
• Owns a mobile smartphone.

Exclusion criteria:

• Patients with a low level of treatment compliance.
• Patients with a developmental or intellectual disability, complex organic conditions including dementia.
• Patients who cannot read and write in the Hebrew language
• Patients who are incapable of informed consent.

The study inclusion criteria for the multi-disciplinary team:

• Multi-disciplinary team members who work in the psychiatric inpatient wards at the Sheba Medical Center.
• The staff members will have a valid Israeli license (from the professions of medicine, nursing, psychology, social work, and occupational therapy).