Popular News Coverage Needlessly Makes Simple Issues Complex
Popular news media are often accused of simplifying complex issues, and no doubt this is often true. But equally often, popular news coverage makes simple issues complex. A recent example is the accounts of the retraction of an academic paper about police shootings (David J. Johnson, Trevor Tress, Nicole Burkel, Carley Taylor, and Joseph Cesario, “Officer characteristics and racial disparities in fatal officer-involved shootings.” PNAS August 6, 2019 116 (32) 15877-15882; first published July 22, 2019 ).
The offending phrase in the paper was “white officers are not more likely to shoot minority civilians than nonwhite officers.” In the absence of context, this could have two interpretations. Police officers fatally shoot about 0.5 black civilians for every white civilian. If white officers shot 0.8 black civilians for every white civilian, and black officers shot 0.2 black civilians for every white civilian, then white officers would be more likely to shoot black civilians than black officers. The paper found this was not true.
The other interpretation starts with police officers rather than shootings. The average police officer has about 1 chance in 4,000, 0.025%, of fatally shooting a black civilian in a year. If the average white officer had a 0.04% chance of fatally shooting a black civilian in a year, and the average black officer only a 0.01% chance, then white officers would be more likely to shoot black civilians than black officers. The paper did not address this question. Other papers have, by the way, and there is no evidence for this kind of racial disparity either. So someone confused about the meaning of the phrase would be incorrect about what this paper demonstrated, but not about racial disparities in police shootings.
While the phrase is ambiguous without context, it is crystal clear if you read the entire sentence, “We find no evidence of anti-Black or anti-Hispanic disparities across shootings, and White officers are not more likely to shoot minority civilians than non-White officers.” The phrase refers to disparities across shootings (the first interpretation) not officers (the second). You could also look to the title of the paper, “Officer characteristics and racial disparities in fatal officer-involved shootings” (emphasis mine). No one who read even the non-technical “interpretation” summary section, or the abstract, could be confused about the meaning, much less anyone who read the paper.
The key point is that it was critics of the paper who misinterpreted the out-of-context phrase from the non-technical summary of the paper. The paper was interpreted correctly by people arguing that individual police officer racism is not the cause of racial disparities among civilian victims. For example, Heather Mac Donald relied on the study to tell Congress, “There was no evidence that officer race (i.e., whiteness) predicted the race of the shooting victim.” That’s precisely what the study found.
Of course, no one study can prove the absence of racism among police officers. All it can do is fail to find evidence of that racism. When a study does turn up evidence, it points to ways to reduce the problem. If white officers were shooting black civilians at a higher ratio than black officers were, hiring more black officers, identifying and firing racists among white officers or better training might help. But if white officers and black officers are shooting black civilians at the same ratio as white civilians, those solutions might not be effective, at least not for the specific problem of disparities in victim race. If individual racism cannot be identified anywhere, then we need to instead look for solutions to systemic racism—that is social structures that produce disparate outcomes without any individual or law being explicitly racist. The question is not whether racial disparities exist, only whether to look for solutions on the individual officer level or the system level.
It was people who insist on racism of individual white officers who took exception to the paper. Unable to find evidence against its conclusions, or to make technical objections to the methodology stick, they grabbed on partial sentence out of context from the non-technical summary and misunderstood it, or pretended to. These people misrepresented the paper, not to support a false narrative, but to discredit the paper.
The proper response by the authors would be to tell critics, “Read the whole sentence you’re criticizing, or look at the title of the paper.” They could issue a public statement, “Heather Mac Donald’s use of our paper is accurate, and her critics are misrepresenting the paper to discredit it.”
A neutral action would have been to rewrite the non-technical summary of the paper. There’s no way to avoid any possible ambiguity of any partial sentence taken out of context. The authors could have explicitly disavowed this particular one. But it’s legal contracts that are written to prevent any possible misunderstanding, and they are unreadable to most people. The whole point of non-technical summaries is to make the work accessible to intelligent non-specialists.
Compare the clear original, “We find no evidence of anti-Black or anti-Hispanic disparities across shootings, and White officers are not more likely to shoot minority civilians than non-White officers,” to the authors’ proposed rewrite, “We find no evidence of anti-Black or anti-Hispanic disparities across shootings, and as the proportion of White officers in a fatal officer-involved shooting increased, a person fatally shot was not more likely to be of a racial minority.” I pretty sure anyone confused by the original would be more confused by the rewrite, and many people who understood the original perfectly would misunderstand the replacement.
I think the authors violated academic integrity when they placed the clarification of the non-technical summary in a correction to the paper itself. They claimed the papers results were correct and that they had been “clear about the quantity we estimated.” To “correct” things already correct and clear is not honest.
The groveling apology one author sent to the Wall Street Journal for having “overstepped” and failed to “uphold the high standard”—without conceding that anything in the paper was false or misleading—is merely unseemly, and blaming Heather Mac Donald for his problems rather than the people who actually did misrepresent his work (not to mention his own spinelessness) is pathetic.
The major academic crime is retracting the paper. Retracting a result you know to be true and clearly stated should carry the same academic consequences as knowingly publishing a false one. When the Soviet Union outlawed Darwin, or Maoist China Einstein, or Nazi Germany imposed racial and eugenic theories—the problem wasn’t that serious researchers began publishing false results, but that they retracted true ones and switched to other fields. The Catholic Church did not demand Galileo publish Ptolemaic astronomy essays, merely that he retract his heliocentric ones. When the US government imposes gentler coercion by restricting funding and access to information in fields like gun violence, nuclear testing, recreational drugs, abortion, fetal cells, nuclear winter, etc.; the problem again is not false publications in these fields, but lack of true ones.
These researchers have less excuse. They were not threatened with gulags, farm labor, concentration camps or burning at the stake. I don’t know if they feared losses of jobs or grants, or perhaps that law enforcement agencies would no longer share the data they needed for their work, but such things do happen. Anyway, I don’t like to criticize individuals without knowing the full circumstances in which they made their decisions. I’m not especially heroic myself, so I’m not going to claim I would have stood up to whatever pressures they may have felt.
But the scientific community should reject the retraction. Like publication, retraction demands justification. The one given by the authors, that Heather Mac Donald misrepresented their work is both false and inadequate.
