There’s More To The Harvard Racial Bias Study Than Right-Wing Media Are Reporting

Other Media Note Error Of Extrapolating From Limited Data

After The New York Times published results from Harvard economics professor Roland Fryer’s study showing that police, after making a stop, are “less likely to shoot if the suspects were black," right-wing media hyped the report headline that there was “no racial bias” involved in police shootings. They argued that high rates of black crime could instead explain the disproportionate rate of black fatalities at the hands of police. But other media outlets noted that the study’s data is limited, that it is based on testimonies of police officers, and that it “avoided the question of whether black citizens are more likely to be stopped to begin with.”

NY Times Reports On Results Of Harvard Professor’s Study Showing “No Racial Bias” In Police Shootings

NY Times: “Surprising” Research “Contradicts The Image Of Police Shootings That Many Americans Hold.” The New York Times published results from a Harvard economics professor's study which showed that while black people are “more likely to be touched, handcuffed, pushed to the ground or pepper-sprayed by a police officer,” but when it comes to police shootings, there is “no racial bias.” The Times called the result “surprising” and wrote that it “contradicts the image of police shootings that many Americans hold” after several high-profile deaths of black people at the hands of police. The Times noted the professor's emphasis that “the work is not the definitive analysis of police shootings, and that more data would be needed to understand the country as a whole,” and that “This work focused only on what happens once the police have stopped civilians, not on the risk of being stopped at all. Other research has shown that blacks are more likely to be stopped by the police.” [The New York Times, 7/11/16]

Right-Wing Media Hype Findings On Police Shootings, Fail To Note Limited Scope Of Data 

National Review: Black Crime, Not Discrimination, Is Probably “Driving Much Of The Overall Racial Disparity In Police Shootings.” Conservative columnist Robert Verbruggen wrote in the National Review that the study supports “crime data” suggesting it is “not likely” that racial bias is a factor “when police kill civilians.” Verbruggen claimed that “discrimination probably isn’t driving much of the overall racial disparity in police shootings,” but that one explanation is that the “violent-crime rates” of black people, including the percentage of black “murderers” and “cop-killers,” is a “highly suggestive” reason for the disparity in police shootings of blacks. Verbruggen pointed to Fryer’s study to bolster his suggestion that “racial bias plays a minor role at best” in police shootings. From the July 11 National Review article:

It may make a difference sometimes. But discrimination probably isn’t driving much of the overall racial disparity in police shootings.

Let’s start with a fact everyone should be able to agree on: There is a disparity. Blacks are about 13 percent of the American population, yet according to data collected by the Washington Post in 2015 and 2016 thus far, they are about 27 percent of those killed by police. (For all the numbers in this article, I exclude any cases for which the race is “unknown.”)

But this isn’t the end of the discussion. Police are allowed, indeed often expected, to kill in certain circumstances — namely, when they reasonably think it is necessary to stop a threat to life or limb. People who pose such a threat are not necessarily representative of the entire population. So the question is, if not 13 percent, what baseline should we be comparing that 27 percent against?

One possible comparison group is murderers: According to the FBI, about half are black. Another is cop-killers, i.e., those who demonstrably presented a lethal threat to police: Again according to the FBI, about 43 percent are black. Still another is violent criminals in general: Most of these commit relatively minor offenses (such as simple assault, where there is no weapon or serious injury), but according to victimization surveys, about 24 percent are black. In other words, violent-crime rates roughly explain the gap — indeed, they over-explain it in the case of murderers and cop-killers, who are far more likely to be black than police-shooting victims are.

[...]

Roland Fryer — among the nation’s leading economists studying racial matters — dug deeply into data from the NYPD and other sources, finding no evidence of bias when it came to lethal force. In fact, blacks were slightly less likely to be killed, which Fryer called “the most surprising result I have found in my entire career.” Sendhil Mullainathan, another leading racial-bias researcher, has noted — similar to the analysis above — that the percentage of arrestees who are black, as well as the percentage of offenders reported to police who are black, roughly matches the percentage of police-shooting victims who are black. [National Review, 7/11/16]

Fox’s Brit Hume: People Must Ask Whether “The High Rate Of Crime Committed By Blacks,” Rather Than Police Bias, Could Explain The Disproportionate Use Of Force By Police. Fox senior political analyst Brit Hume said on Special Report that the study found that “when it came to police shootings, there was no racial bias. Did you hear that, Mr. President? No racial bias.” Hume acknowledged that the study found that “the general treatment of blacks by police officers is rougher and tougher, and perhaps unfairly so, than it is for whites,” but added that “of course, the other question is given the high rate of crime committed by blacks as opposed to other groups in this country, you would have to be able to assess whether that has something to do with it.” From the July 11 edition of Fox News’ Special Report with Bret Baier:

BRIT HUME: Now a study led by a black Harvard law professor has examined 15 years of crime data from five major cities and two Florida counties. The study found that, while police were more often likely to get physical with black suspects than with white ones, when it came to police shootings, there was no racial bias. Did you hear that, Mr. President? No racial bias. Bret.

BRET BAIER (HOST): Brit, last week, I had Sen. Tim Scott, African-American, Republican from South Carolina. He came on and said even though he's a lawmaker, he still feels sometimes a tension because, he believes -- he's African-American, and that feeling seems to be pervasive, at least when you talk to African-Americans in different communities around the country.

HUME: That was certainly one of the findings of that Harvard study, that had found -- while it found that there's no racial bias involved in police shootings, that the general treatment of blacks by police officers is rougher and tougher, and perhaps unfairly so, than it is for whites. To understand that data fully, I suppose you would have to know about the circumstances under which these various incidents occurred over time. I don't know how you would ever do that, but to fully understand, you might have to. And of course, the other question is given the high rate of crime committed by blacks as opposed to other groups in this country, you would have to be able to assess whether that has something to do with it. But it does seem to be a grievance widely shared and certainly one that the police departments in this country should be working on. [Fox News, Special Report with Bret Baier, 7/11/16]

Fox’s Bill O’Reilly: Study Shows Black Shooting Deaths Are “Low When Compared To White Americans.” Fox host Bill O’Reilly claimed that the study revealed that the rate at which black people are fatally shot by police is “low when compared to white Americans.” Fox contributor Charles Krauthammer said the results go “against all conventional wisdom.” From the July 11 edition of Fox News’ The O’Reilly Factor:

CHARLES KRAUTHAMMER: Look, we also are making some tremendously unwarranted assumptions. There is a study that came out of Harvard today by a Professor Roland Fryer, a professor of economics at Harvard, that had a very surprising result. He studied the use of force, deadly force, shootings of black Americans by police. And he compared it to white Americans. And he found that for black Americans, the number is lower. He called it -- it's in The New York Times of all places -- “the most surprising result of my career.” And he, incidentally, the professor, is African-American. Now, it turns out that other elements of the use of force -- handcuffing, pushing up against the wall -- is somewhat higher among African-Americans. But shooting, which everybody assumes is higher among African-Americans, it is not.

BILL O'REILLY (HOST): It is low when compared to white Americans in --

KRAUTHAMMER: Exactly.

O'REILLY: Right. Right.

KRAUTHAMMER: Compared to -- but that's against all conventional wisdom. [Fox News, The O’Reilly Factor, 7/11/16]

Fox’s Dana Perino: “The Lethality Argument Is Actually Not There. It Does Not Hold Up.” Fox host Dana Perino said that though African-Americans feel “that they are unfairly targeted by police,” Fryer’s study shows that “the lethality argument is not there. It does not hold up.” From the July 11 edition of Fox News’ The Five:

ERIC BOLLING (CO-HOST): Dana, your thoughts on that equal footing for them?

DANA PERINO (CO-HOST): I think that whether I agree with President Obama or not, I think that it’s incumbent upon all of us to recognize, there are a lot of people in America who do agree with him. So then you have to decide, OK, well then from a leadership standpoint, what do we need to do? I actually think it’s kind of an evolution or a continuation of a movement and that these things sort of ebb and flow over time and that it's built up over a feeling by African-Americans that they are unfairly targeted by police. And there's a study in The New York Times that’s reported today by somebody from Harvard, who’d gone -- he’s like, let's look at the evidence. And the lethality argument is actually not there. It does not hold up. But again, from social media and the perception, President Obama is giving voice to people that he thinks have a grievance, and he's trying to find a way, I guess, to channel it. I don't necessarily agree with it, but I think we have to recognize that there are a lot of people who do feel that way. [Fox News, The Five, 7/11/16]

TheBlaze: Study Shows “Law Enforcement Officials Do Not Have Any Racial Bias In Terms Of Using Lethal Force.” A July 11 article in The Blaze wrote that the study showed that “there was no indication of racial bias [by police officers] when it comes to firing a gun” and “that police officers were more likely to shoot a suspect in incidents without having first been attacked in cases where the suspects were white.” [TheBlaze, 7/11/16]

Wash. Times: Study “Challenges The Contention By The New Wave Of Civil-Rights Groups” That “Racist Police Are Singling Out Blacks For Shootings.” A July 11 Washington Times article said the study “found no evidence of racial bias in police shootings even though officers were more likely to interact physically with non-whites than whites” and that it “challenges the contention by the new wave of civil-rights groups such as Black Lives Matter that racist police are singling out blacks for shootings.” [The Washington Times, 7/11/16]

Media Note Study Reaches Narrow Conclusions Due To Sampling Only After Police Stops Were Made And Because Data Were Gathered From Police Reports

Vox’s Dara Lind: Study Found Limited Racial Bias In Shootings After Police Stops Were Made, But “Avoided The Question Of Whether Black Citizens Are More Likely To Be Stopped To Begin With (They Are).” Vox’s Dara Lind wrote that the Harvard study’s “narrow” focus makes its findings “an addition, not a discovery, and not the last word” on racial bias in police encounters, noting that the data set is “not nearly as broad a sample as the conclusion … would suggest.” Lind noted that the study examined “how often black and white suspects who’d already been stopped by police were killed,” but “deliberately avoided the question of whether black citizens are more likely to be stopped to begin with (they are) and whether they’re more likely to be stopped without cause (yup).” Lind concluded that a series of “red flag[s]” in the study provide a reason to be “skeptical” of the findings:

As a general rule, when somebody claims that a new academic study finally looks into the data behind a controversial news issue, you should be skeptical.

[...]

The study is definitely useful in some regards: Researchers examined the circumstances of police shootings (like whether police were attacked before firing their guns) with much more detail than previous work has, for example. But it’s narrower than a lot of existing research on racial disparities in criminal justice — so narrow, in fact, that it kind of misses the point of why police killings of young black men are so frequently outrageous.

[...]

The most revealing passage in the Times article is probably the one explaining what Fryer and his team didn’t include in their study:

It focused on what happens when police encounters occur, not how often they happen. Racial differences in how often police-civilian interactions occur reflect greater structural problems in society.

In other words, Fryer and company found that there weren’t big racial disparities in how often black and white suspects who’d already been stopped by police were killed. But they deliberately avoided the question of whether black citizens are more likely to be stopped to begin with (they are) and whether they’re more likely to be stopped without cause (yup).

Avoiding those issues makes sense for the question Fryer was trying to answer. He wanted to know what happens between the moment a police officer stops someone and the moment he pulls the trigger — and how those sequences of events vary by race.

But when people talk about racial disparities in police use of force, they’re usually not asking, Is a black American stopped by police treated the same as a white American in the same circumstances? They’re making a broader critique of the “greater structural problems” in society in general and the criminal justice system in particular. They’re saying that black Americans are more likely to get stopped by police, which makes them more likely to get killed. [Vox, 7/11/16]

The Guardian: “The Data Is Misleading” And “Created ... Largely By Coding Police Narratives,” Which “Leads To Simplistic Conclusions.” Guardian data editor Mona Chalabi wrote that the study “uses a small sample of data that leads to simplistic conclusions.” Chalabi pointed out that “To understand lethal use of force, Fryer looked at police reports from just one city,” which may not be “representative of the country.” Chalabi also noted other problems in the study, including that it “assumes police reports are unbiased sources of information about facts,” and that it “has also not yet been peer-reviewed” for its results. From the July 11 Guardian article:

A study reported by the New York Times on Monday claimed to find “surprising new evidence” that there is no racial bias in police shootings. But the study, and the New York Times’ reporting, uses a small sample of data that leads to simplistic conclusions.

The author of the study, Roland G Fryer Jr, analyzed 1,332 shootings between 2000 and 2015. However, the way he and a group of student researchers created their data was largely by coding police narratives rather than considering the testimonials of witnesses or suspects (assuming that the suspects were not killed by the police in the shooting). The study therefore assumes police reports are unbiased sources of information about facts like whether or not the officer shoots the suspect before being attacked.

There are other serious weaknesses in the research. To understand lethal use of force, Fryer looked at police reports from just one city: Houston. There, he found that blacks were either less likely to be shot by an officer or there was no difference between blacks and whites. Even if the data from Houston were accurate, it is doubtful the city is representative of the country.

The Houston police department also allowed the researchers to look at “interactions with police where lethal force may have been justified”. But that data assumes that all officers make fair and objective decisions about the lethal use of force – decisions that are not affected by the race of the suspect.

Looking at just one other city would suggest very different conclusions. In Chicago, a review of the reports of each police-involved shooting looked at fatal and non-fatal shootings. Despite the city being one third black, a disproportionate 118 black males (44 of them fatal) were involved in the 150 shootings recorded since 2010. [The Guardian, 7/11/16]

Fortune: Reports On The Study Fail To Note That “Many Who Die In Police Custody Weren’t Shot” And Downplay Conclusion Of Bias In “Lower Levels Of Use Of Force.” A Fortune article noted that Fryer’s study was “making headlines … for all the wrong reasons,” explaining that, “It’s only the shootings data that shows a lack of racial bias,” and that “many who die in police custody weren’t shot.” Fortune interviewed the author of the study, who acknowledged he was concerned that “people were going to take the shooting results and pay less attention to the lower level uses of force” because “These lower level uses of force happen thousands of times per day.” From the July 11 Fortune article:

Fryer’s work analyzed thousands of police reports from a sample of U.S. cities. The reports, which were written by officers from 2000 to 2015, showed no police bias in the shootings of black suspects compared to white ones. That’s problem No. 1 with the data, which Fryer admits is limited, but there was no easy way to get access to reports from victims, or witnesses.

Problem No. 2: It’s only the shootings data that shows a lack of racial bias. Look beyond shootings, and even the self-reported data from police departments shows that officers have a stronger propensity to use physical force with African Americans than their white counterparts.

For example, the study shows police are 24% more likely to point their guns, 25% more likely to use pepper spray or a baton, and 18% more likely to push a an individual to the ground if the suspect is a black person rather than a white person.

Nonetheless, with police involved shootings making headlines last week, early media reports on the study like the Times‘ report focused strongly on the shooting category, and the lack of bias.

The Times story emphasized its “surprising” results on lack of shooting bias. The story was quickly picked up by conservative media outlets like Drudge Report and Fox News insider, which ran the headline “Harvard Economist’s Study Finds No Racial Bias in Police Shootings” above a graphic from Fryer’s study showing racial bias against blacks in every other use of force category.

In an interview with Fortune, Fryer says these misleading reports are “unfortunate,” pointing out that many high profile fatal encounters involving blacks and police—people like Eric Garner, Sandra Bland, Barbara Dawson, and Freddie Gray, for example – did not involve guns.

“One of the things I worried about was people were going to take the shooting results and pay less attention to the lower level uses of force,” Fryer tells Fortune. “These lower level uses of force happen thousands of times per day. Who would want that to happen?” [Fortune, 7/11/16]

Slate: Study’s Limited Data And Reliance On Officers’ Testimonies Mean Results Shouldn’t “Be The Last Word.” An article by Slate’s Jordan Weissmann noted that there are several shortfalls with the study’s parameters, and that the results shouldn’t “be the last word, nor should anybody treat it that way.” The study’s limitations include the fact that “much of the data on shootings comes from major cities with large black populations, which might be more sensitive about race issues in policing” and that the “study relies on reports written by officers who may well be lying about what really transpired.” Weissmann also noted that the study “doesn't tell us anything about whether people of color are being stopped excessively in the first place.” From the July 11 Slate article:

So what do we make of the findings?

In short, they're interesting, but have some important limitations, which Fryer, a MacArthur genius grant winner who was the youngest black professor to ever earn tenure at Harvard, is very open about. As he puts it, “this paper takes first steps into the treacherous terrain of understanding the nature and extent of racial differences in police use of force.” It's not meant to be the last word, nor should anybody treat it that way.

[...]

Then, of course, there are those limitations. First, much of the data on shootings comes from major cities with large black populations, which might be more sensitive about race issues in policing. This is especially striking when you consider that the most famous shooting of recent years took place in Ferguson, a small, inner-ring suburb with its own police department. “It is possible that these departments only supplied the data because they are either enlightened or were not concerned about what the analysis would reveal,” Fryer writes. “In essence, this is equivalent to analyzing labor market discrimination on a set of firms willing to supply a researcher with their Human Resources data!” Who knows how the situation may look in other cities or towns. “Relatedly, even police departments willing to supply data may contain police officers who present contextual factors at that time of an incident in a biased manner,” Fryer adds—which is a polite researcher's way of saying that the study relies on reports written by officers who may well be lying about what really transpired.

Readers also need to be cautious about how they interpret the findings. Once again, Fryer suggests blacks and Hispanics are no more likely to be shot than whites after they've been stopped by police. But as a few sharp writers have noted today, that doesn't tell us anything about whether people of color are being stopped excessively in the first place. And in parts of the country, they almost certainly are. In one-third white Ferguson, blacks made up 93 percent of arrests and 85 percent of traffic stops. Philando Castile was stopped by cops 52 times in 14 years before he was shot and killed by an officer in Minnesota last week. Fryer's paper may simply be telling us that deadly bias in American policing has less to do with how cops decide to pull the trigger and more to do with how they interact with the community overall. Hopefully, researchers will be able to replicate his in-depth analysis of Houston in many more cities to give us a better sense of whether that is actually the case.

Even if police are only biased when it comes to the use of, say, hurling suspects to the ground—that's not exactly something to celebrate. [Slate, 7/11/16]