Legislators as Lab Rats?

by Karl Kurtz

Yale University political scientists Daniel Butler and David Broockman have recently published an article, "Do Politicians Racially Discriminate against Constituents? A Field Experiment on State Legislators." Their research consisted of sending a fictitious e-mail message to approximately 4,800 state legislators with a request for assistance in registering to vote. Some of the messages were sent using an apparently white name (Jake Mueller) and some under a stereotypical black name (DeShawn Jackson).

They found that 61 percent of the legislators responded to the Jake alias and 55 percent to DeShawn--"a statistically significant difference of 5.1 percentage points (p=0.04)" (p. 14). They conclude:

Overall, we find that the putatively black requests receive fewer replies. We explore two potential explanations for this discrimination: strategic partisan behavior and the legislator's own race. We find that the putatively black alias continues to be differentially treated even when the emails signal partisanship, indicating that strategic considerations cannot completely explain the observed differential treatment. Further analysis reveals that white legislators of both parties exhibit similar levels of discrimination against the black alias. Minority legislators do the opposite, responding more frequently to the black alias.

It's tempting to quibble with the methodology and findings of this research. For example, while the differences are "statistically significant," is the small difference of 5 percent meaningful? Would the differences have disappeared if the "Jake" and "DeShawn" messages had included home addresses in the legislators' districts? Many legislators receive hundreds of e-mails a day and perform triage on them by responding only to the ones identifiably from their own districts. Certainly the response rates would have increased if it were an identifiable constituent making the request.

But my bigger concerns are the ethical issues of political scientists performing this kind of research.

Writing in a Washington Post blog, Ezra Klein calls this research "simple and elegant." That's one point of view.

I think of it more as treating legislators as lab rats: Performing an experiment by telling them a lie and then measuring how they respond. Ring a bell and see if they salivate.

It's a little bit like the FBI creating a crime and then trying to ensnare public officials in it. Only the political scientists' deception doesn't have an FBI sting's justification of uncovering corruption. The only saving grace is that this study reports only group behavior, not individual responses, so there's no "gotcha." There is, however, a profound disrespect for the work that legislators do, the enormous demands many of them work under, and the idea of public service.

To their credit, the authors address the ethical issues of performing experiments on public officials (pp. 12-14). They defend their research design as meeting a standard for experiments on publlic officials that they ascribe to the eminent political scientist, Robert Putnam: If it's only "slightly deceptive, but innocuous and highly revealing," then it's OK to make work for public officials.

When I asked Alan Rosenthal, an equally noted political scientist, about this, he mused, "How is 'slightly deceptive' a defense for deception? What do you think political scientists or the media would say if a legislator defended his or her misbehavior as 'slightly corrupt, but harmless and profitable?' They would make a joke of an elected official who said this."

Now that I've biased you with my rant on this subject, you can weigh in with your own opinion in this poll:

 

8/25/11 Update: Another study of legislator responsiveness to fictitious constituent requests--this one from South Africa--has drawn further attention to the ethics of experimental academic research. See "Politicians Like Constituents Like Them" in The Monkey Cage and ensuing thoughtful comments about research ethics.

Comments

[Liz]

A more interesting question is how Daniel Butler and David Broockman can blatently manipulate their data, presenting relative values as absolutes, and use this inaccurate reflection to support a pre-determined but unfounded conclusion, and be treated seriously. The results appear in a table, showing that the group that responded most to the DeShawn e-mail was "white Republicans." They had a higher response rate with Jake, but overall DeShawn got the best response from that group. The second highest percentage of responses came from white Democrats. Minority Democrats were third, but looked good only because they totally blew off poor Jake, and minority Republicans loved Jake and kind of ignored DeShawn. While I support increased numbers of minority representatives, I could never use this foolishness to back me up.