Motivated Responding in Studies of Factual Learning

Kabir Khanna and Gaurav Sood

Observers of contemporary public opinion often lament the seeming inability of the political left and right to agree on even basic facts. Democrats and Republicans, for example, seem to hold different beliefs about a range of facts, from the number of Americans who are unemployed to the existence of global warming to the number of people who voted illegally in the past election.

A prominent explanation for these differences is motivated learning: even when people are given the same information supporting an unambiguous conclusion, they are more likely to learn the correct conclusion when it reflects positively on their core attachments and identities. Motivated learning fuels concerns about citizens’ ability to hold governments accountable and the prospect of democratic deliberation. For instance, how can people on different sides of the aisle engage productively with one another when they see the exact same information and yet walk away with diametrically opposed beliefs about what it says?

In our article in Political Behavior, we reexamine the evidence for motivated learning. In a series of experiments, we presented people with tabular data from a putative study on a social policy, either gun control or raising the minimum wage. Following Kahan et al., we manipulated the congeniality of the result supported by the data (e.g., whether the result supports or undermines the effectiveness of gun control). We find that in some cases respondents are indeed more likely to learn the correct result when it is congenial, or in other words, when the result is consistent with their position on the issue. On the surface, this looks like textbook motivated learning.

But here is the crucial part of our study design: independently of the congeniality manipulation, we offered a random subset of respondents a small financial incentive to accurately report what they had learned. Importantly, we only told respondents about the incentive after they had seen the data and could no longer return to it. When we did so, respondents became significantly more likely to report the correct result when it was uncongenial. The incentive treatment significantly reduces estimates of motivated learning and in some cases, eliminates it entirely. But incentives do not alter responses universally. For instance, incentives made no difference among opponents of gun control, suggesting that they really did learn in a biased manner.

Overall, however, the data suggest that without incentives, some respondents give incorrect but congenial answers even when they have learned the correct result. This sort of behavior is what we mean by motivated responding. We also find that respondents are unbiased in recalling the precise numbers they saw in the table.

Our study builds on recent scholarship by Bullock et al. (2015) and Prior et al. (2015), who each demonstrate that motivated responding occurs in surveys of stored knowledge. We find a similar phenomenon in surveys of what people learn over the course of a study. This line of research has important implications for measuring factual beliefs – incentivizing answers to factual questions is likely to reduce bias in measurement.

We’ll end by noting an important wrinkle in our findings. When we incentivized respondents to faithfully report what they had learned, they became more biased in judging the credibility of the putative study. That is, anti-gun respondents judged a study with a pro-gun result more harshly, and vice versa. So, while our findings suggest that motivated learning is less common than what the literature suggests, there is also a whack-a-mole nature to bias: reducing bias in one place is offset by an increase in bias in another place.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s