Introduction: in the course of the research, the investigators will develop a decision-support system (comorbidity-DSS) consisting (1) a knowledge base (KB) consisting of (a) computer-interpretable clinical guidelines for type 2 diabetes and 2 other diseases from: obstructive pulmonary disease, osteoporosis, hypertension, and osteoarthritis; and (b) an ontology of relevant general medical knowledge that could complement (a) in order to propose non conflicting treatment options not mentioned in the clinical practice guidelines; and (2) an algorithm that matches the KB with a patient's data set to identify the guidelines-based recommendations applicable for the patient and their interactions and which proposes ways to mitigate conflicting interactions (e.g., suggesting to select intervention A.2 (instead of A.1) from guideline A and intervention B.3 (instead of B.1) from guideline B together with an action B' mentioned in the general medical knowledge, because these interventions are not conflicting yet A.3 fulfills the same goals as intervention A.1 and intervention B.3 + B' together fulfill the same goal as B.1).

Research purpose: Assessing the correctness and completeness of detection of recommendation-interaction and generation of conflict-free recommendations by a comorbidity-DSS

Research question: How will the usage of the comorbidity-DSS affect the completeness and correctness of clinicians regarding (a) detection of interactions between recommendations originating from different clinical guidelines applicable for patients with comorbidities and (b) identification of interventions that fulfill the guidelines' goals and are not conflicting.

The protocol is as follows.

1. In consultation with 3 expert clinicians, the investigators will create a database of patient scenarios. The investigators will obtain deidentified extracts from medical records of 6-12 typical patients who have type 2 diabetes and at least one of the following comorbidities: obstructive pulmonary disease, osteoporosis, hypertension, and osteoarthritis will be obtained. The data obtained will include relevant observations, medications, and procedures regarding these patients. Relevancy will be determined from the clinical practice guidelines for the above-mentioned diseases, which specify which data should be collected from such patients. The data will not include information that could identify the patient, such as date of birth, name, identification number, street address, telephone number.

   The patient cases will be assembled into a database of scenarios, decomposed into several steps, each step taking place at a different point along the clinical guideline's timeline and being composed of several decision-points regarding goals of the clinical guidelines. In total, there would be 60 decision points across all patient scenarios.

2. Validation of the KB and creation of a gold standard. The three experienced clinicians will validate the KB and will also create a gold standard set of interactions between recommendations for the decision points of the patient scenarios as well as a set of recommendations that fulfill the clinical guidelines' goals for these decision points that are conflict-free.

3. Recruiting study participants. The investigators will send invitations to 50 clinicians/medical students in order to recruit at least 30 participants for our experiment. The participants will be asked to solve (detect interactions and propose non conflicting interventions) six scenarios. Anticipated time for solving the cases: 3 hours + 1 hour of introduction and signing consent documents.

4. Crossover study design will be used to compare the effect of using the comorbidity-DSS for detection of recommendations interactions and on generation of correct non-conflicting recommendations. Each participant will be given 6 scenarios: 3 will be solved with the aid of the system and 3 without it. Each scenario will be presented to the clinicians as a series of single steps. In DSS-mode the clinicians will be presented with the output of the comorbidity-DSS for each step, including the list of interactions between recommendations originating in the different clinical guidelines and the set of non-conflicting recommendations that fulfill the goals of the relevant decision point/goal. They will need to say for each interaction and for each non-conflicting recommendation whether they accept or reject it, possibly adding some free text; in the non-DSS mode, the clinicians will need to provide their set of detected interactions and proposed non-conflicting recommendations after each step, in free text.

5. While the gold standard interactions and recommendations would have been created ahead of time by the three clinicians from RAMBAM in step 1, it is possible that the comorbidity-DSS or that the study participants will identify additional interactions and non-conflicting recommendations. The three clinicians will review these as well and could potentially revise the gold standard to include a richer set of interactions and of non-conflicting recommendations for each set. Based on the revised gold standard, completeness and correctness of the interactions detected and the non-conflicting recommendations generated by the participant will be calculated, while using the comorbidity-DSS vs. without it for the different steps (decisions/goals). The overall completeness and correctness are percentages, and thus range from 0 to 1, relative to the extended gold standard.

6. Statistical analysis will be done as follows. Assuming that total completeness and correctness are two dependent variables, as they are bounded variables between zero and 1. Thus, a beta regression model with a logit link function will be used for the mean response model; a log link function will be fitted for the precision model. This model is based on the assumption that the dependent variable is beta-distributed and that its mean is related to a set of regressors through a linear predictor with unknown coefficients and a link function. The full model shall include three factors: (1) DSS mode (DSS and Non-DSS), (2) Level of training (e.g., 1st year resident, experienced resident, specialist), and (3) Scenario (as different scenarios will be used). The pseudo R2 value (squared correlation of linear predictor and link-transformed response) will be used to measure the overall goodness of fit of the model. A backward elimination algorithm will be used to assess what are the important factors. Unlike the total completeness measure, which was computed for the overall set of decision-points per scenario per clinician, the investigators will also analyze the completeness measure for each decision-point (e.g., "order lab test"), increasing resolution and looking at the clinician's guideline-based action per decision; therefore, the measured variable is binary, and a logistic regression with a logit link function will be used. As in the previous case, the full model will include three factors; a backward elimination algorithm will be used to reach the final model.

Inclusion Criteria:

• Medical students from Technion medical school or clinical experts from RAMBAM

Exclusion Criteria:

• None