How Clinical and Personal Information Shape Physicians' Risk Judgments

How Clinical and Personal Information Shape Physicians' Risk Judgments

Physicians often form quick judgments about the risk for serious disease when interacting with patients. Underestimating risk can lead to underuse of diagnostic testing and untreated illness, which can worsen patient outcomes. On the other hand, overestimating risk can lead to overuse of diagnostic testing, which is costly for health systems.

To form judgments of risk, physicians should attend to a host of validated factors that are predictive of disease. However, research suggests that physicians may rely on demographic factors-such as race and gender. Physicians' judgments could also be influenced by non-health-related, personal information about their patients (e.g., hobbies, nicknames), which may moderate the impact of demographics on those judgments.

The investigators examine these dynamics in the context of heart disease. The History, Electrocardiogram, Age, Risk factors and Troponin (HEART) Score is a validated model that specifies a correspondence between certain risk factors and the likelihood of Major Adverse Cardiac Event (MACE). Importantly, there are substantially different diagnostic tests (e.g., noninvasive stress test versus coronary angiogram) that should be used depending on a patient's MACE likelihood.

Specifically, the investigators have three research questions:

• Research Question 1 (RQ1): How accurate are physicians relative to the benchmarks from the HEART score model?
• Research Question 2 (RQ2): How do clinically-relevant risk factors (e.g., smoking history), race, gender, and personal information disclosure influence risk judgments?
• Research Question 3 (RQ3): Does personal information disclosure moderate the effects of race and gender on risk judgments?

Note that when the investigators discuss accuracy and error, they are referring to the comparison of physician judgments to the HEART score model benchmarks.

The investigators designed a survey to assess physicians' perception of MACE likelihood. Each physician rates a panel of patient profiles. The profiles randomly vary in risk factors, race, gender, and personal information disclosure (e.g., non-health related information about their hobbies). Using the HEART Score model as a benchmark, the investigators will assess how accurately physicians perceive MACE Likelihood based on the risk factors in a given profile. The investigators will further estimate how race, gender, and personal information disclosure causally affect physician judgments.

To do this, the investigators designed a mixed-design experiment. Each participant will respond to eight patient profiles that vary along three fully-crossed within-subject factors: (i) race: black vs. white, (ii) gender: man vs. woman, and (iii) risk factors: low vs. medium risk (based on risk levels from the HEART score model). Each participant will also be randomly assigned (between-subjects) to (iv) either see non-health-related personal information (e.g., hobbies) for all eight of their patients, or not see this information for any of their patients. The investigators refer to each factor as a profile attribute.

For each profile, participants indicate the perceived risk of a major adverse cardiac event in the six weeks following the visit. Our primary outcome is a measure of absolute error in perceived risk of MACE (described under the Primary Outcome section). They also indicate the diagnostic test they believe is most appropriate (Secondary Outcome #4).

Analysis plan

For all analyses, the investigators will format the data such that there are eight observations per participant, each corresponding to a patient profile the participant responded to.

• To test RQ1, the investigators will estimate an Ordinary Least Squares (OLS) regression predicting the primary outcome. The investigators will cluster standard errors by physician. The constant reflects the mean of our primary outcome, reflecting the extent to which physicians are inaccurate.
• To test RQ2, the investigators will estimate an OLS regression predicting the primary outcome. The independent variables include four indicators to model the profile attributes: an indicator for risk factors, an indicator for gender, an indicator for race, and an indicator for personal information disclosure. The model will also include fixed effects for physician specialty (emergency medicine, cardiologists, and hospitalists). The investigators will cluster standard errors by physician.
• To test RQ3, the investigators will reestimate the primary model testing RQ2 twice: In one model, the investigators will add an interaction term between personal information disclosure and race; and in the second model, the investigators will add an interaction term between personal information disclosure and gender.

As secondary analyses,

• The investigators will estimate the models above except using Secondary Outcomes Measures #1-3
• The investigators will measure participants' choice of appropriate diagnostic test for each patient as a secondary outcome (see Secondary Outcome #4). The investigators will estimate a multinomial logistic regression predicting choice of appropriate diagnostic test as the categorical outcome (with "routine follow-up" as the reference category). The independent variables include the four profile attribute indicators and physician specialty fixed effects. The investigators will cluster standard errors by physician.
• The investigators will reestimate the primary models testing RQ2, except including pairwise interactions between the physician specialty fixed effects and each of the four attributes.
• The investigators will reestimate the primary model testing RQ2, except adding all pairwise interactions between the four attributes as independent variables.
• The investigators will measure overconfidence (see Secondary Outcome Measure #5). With data at the physician-level, the investigators will estimate an OLS regression predicting overconfidence. The investigators will reestimate the model, except adding the two indicators for physician specialty and the indicator for personal information disclosure as independent variables.

The investigators will also measure and explore (i) qualitative open-ended responses about how they made their risk estimations, (ii) whether participants use the HEART score model at their jobs (or another model), (iii) if they use a model, why they use it, (iv) if they have heard of the HEART score model, (v) if they looked up anything while taking the study, and (vi) if yes, what they looked up.

The investigators plan to recruit 300 physicians using the Medscape panel.

Inclusion Criteria:

• Emergency medicine physicians
• Cardiologists
• Hospitalists (i.e., internists whose practice is primarily in the hospital)
• If there are duplicate participant IDs in the data, only the first response will be included in the analysis

Exclusion Criteria:

• Participants who indicate they are retired
• Participants who write gibberish in the open-ended items
• Incomplete responses