Speaking as a scientist, I think the partisan claim that the Republicans are "anti-science" and the Democrats "follow the science" is inaccurate. Indeed, Republican politicians often disregard science, especially during the Trump era, but both parties tend to extoll scientific findings they find politically convenient, and ignore the others. Of course science is rarely definitive, and the large majority of its results are irrelevant to daily life.
Very helpful article that manages to cover a recent argument online while still threading the needle to discuss larger implications of the findings. I think trying to do justice to "the paper"'s many dimensions is hard and I appreciate the effort shown here. I hope people will consider, as much if not more, the challenges that criminal justice reform will face if we do not focus on the needs of fragile families who are at their wits' ends because of abusive behavior by family members. The fact is, for people in those families, the only help they have in 2021 is the police. The police are the one group of people with the legal authority to remove an abusive person from the home, but that is only for things that can justify such extreme actions taken. But there are many families with abusive family members, both siblings and parents and children, whose situation is such that they are unwilling to involve the police (for frankly incredibly smart and well thought out reasons) or simply do not have legal cause to do so (e.g., emotional, verbal and financial forms of abusive behaviors) and those can also be unbelievably ravaging on family members.
I think what is unusual about this paper, though, is its judge fixed effects design and the implications of that design. By definition, this paper estimates a LATE for the complier subpopulation . These are people for whom forcible removal through imprisonment occurred because they got a strict judge, but would not have occurred with a more lenient judge. It's this "disagreement principle" as I call it that drives the design's identification. The marginal abuser, in other words, is who this paper is about. So incredibly, it seems to say we are actually not paying attention enough to the externalities associated with criminal behavior. I think that's what it means -- the lenient judge is too lenient because he disregards or doesn't understand that there are others suffering from this person's behavior.
And yet, prison is not imo the first or even second best. The disagreement principle for me means we have got to find these families ASAP and get them help. We have got to find them because the lenient judges represent the very goal we have as society! We want more leniency even towards violence -- we just want it to fair and equitably done. But that necessarily means, as Pfaff has continually pointed out, handling violence differently. But if we reduce incarceration levels and don't simultaneously allocate major resources to fragile families and hurting communities, we do no one any good. We do harm. So the paper's conclusions for me is both/and. Both reduce incarceration levels AND start solving these problems that we are using harsh judges to solve (which are themselves unbelievably socially and personally costly).
You say that "no one is suggesting anything like [being explicit and transparent about biases and ideologies]" and maybe that's true for economists and scientists. But some philosophers of science are suggesting exactly that approach. One example is Kevin Elliott: https://theconversation.com/rather-than-being-free-of-values-good-science-is-transparent-about-them-84946 Another commenter already mentioned Liam Kofi Bright, and Heather Douglas has been leading the charge against the value free ideal, which is mostly used as an excuse for scientists to avoid responsibility for the consequences of their research choices and findings.
I proposed an inexpensive observational study that would help calibrate how much predictive power for graduate success of GREs, GPA, etc. is lost due to stratification bias in single-program samples. It would be useful and timely. My colleague agreed, but demurred: "That's getting too close to the third rail."
Critics aren't arguing that we should let the People's Soviet censor all our econ papers, they're expressing deep skepticism of a study that doesn't seem to gel well with existing research on incarceration and has a pretty questionable framing.
For example, suggested reasons given for why incarceration could have a positive effect on children were that it could rehabilitate the defendant from engaging in criminal activity and lead them to be a more committed caregiver, that it could lead the non-incarcerated parent to change their behaviour and possibly desist from crime, that seeing the family member go to jail could have a deterrent effect on the kid, and that removing the person could remove a harmful influence. Basically, all the classic arguments for punitive incarceration.
But the paper said that the highest effect was observed for black children. The race for whom we have decades of evidence that incarceration is unfairly overapplied for similar crimes. I think they even said that their offender sample was biased towards first-time offenders and drug offenders, for whom institutional racism should be most evident. And if I'm reading it correctly, they found a decrease in likelihood of future incarceration, better adult neighborhood, and a null effect for academic attainment and teen pregnancy, so they're not saying it just impacted one small aspect of life outcomes. It had neutral to positive effect on pretty much the big ones.
I'm not a sociologist or an economist, but I think that if I did a study and it suggested that the justice system actually works the best for the people for whom we know it works the worst, that aspect would be worth more discussion than the zero commentary it's given in the paper. Like, are black people reaping a windfall from institutional racism in the justice system? Would they be worse off if it was fairer? The authors don't seem to have even registered the strangeness.
It's not ideological for sociologists to be skeptical of this, any more than it was ideological for climate scientists to sneer at the Nordhaus papers you mentioned about the economic effects of climate change. And it's not out of line for these sociologists to suggest that a review board that was more diverse might have caught these questions and at least required the authors to address them, because white people, even ones who think they're being fair-minded, show substantial bias when asked to make judgments about issues in which race is salient; this fact has convincing empirical verification in many different studies, however uncomfortable it may make people.
This is (at least potentially) a great example of where insisting on understanding on something that makes us uncomfortable has the potential to improve lives. I doubt prison qua prison is responsible for the observed improvements in outcomes. Understanding what is the underlying cause can help us formulate actual policy that isn't just 'throw more people in prison'.
Or the results might fail to replicate v0v
To move away from the specific example, I think there is a (growing?) strain of thought which fully rejects objectivity in science. Scientific research is, in this mindset, as much an act of activism as knowledge seeking. The idea of pure knowledge seeking is viewed with suspicion - an exercise of privileged self-delusion at best, and often worse. Academic output is judged not on its quality but on the extent to which it supports the Cause, and all sorts of findings get denounced for being contrary to political goals (this crops up a lot in social science and particularly the intersection of psychology and biology).
I don't necessarily agree with the findings of the paper, and it may not be replicated, but the right way to disagree has to be with better theses and more accurate data. In fact, other policies probably result in even better reductions in incarceration. But we need to investigate.
Otherwise we'd be lying about the state of society and that definitely is not going to result in better outcomes for anyone.
It is unsurprising that an economic paper sparks this debate. Economists are roughly divided into two tribes. One approaches research as an attempt to understand the nature of economic activity. Assuming good practices are followed in data collection and data handling, let the results all where they may. The other enters a research program with strong priors about the expected outcomes. These priors are ideologically conditioned. Results not consistent with these priors are rejected as mistaken or, ironically, ideologically driven.
I would make a distinction between a research agenda and particular discoveries. The choice about what to study and the approach you’ll take there involves a wide range of non-science ideology - what’s important and moral and valuable. But within that agenda you are applying the scientific method and should be rigorous about truth seeking.
I don’t know about these particular researchers, but the broader research agenda of “applying science to criminal justice issues” seems to me valuable and important and moral. How can we protect citizens and not destroy lives in the process? I’d rather we be asking that question in a careful way than going by society’s moral gut (-> mass incarceration).
Of course, ever trust one piece of research. A body of work has to build up over time before we can draw policy conclusions.
This episode does reveal a lot, though. People (and media) will accept and promote a single piece of research if it fits their narrative. Those who say "apply a higher standard" in special cases, as this supposedly is, reveal that they probably have a tendency not to push back on research that supports their internal narrative. I think a lot of laypeople intuit that this happens and so they discount research findings that support the narrative of the media that are reporting them. This can be read as being anti-scientific, but there's a wisdom to it, if indeed that's the dynamic that is at work. That's a good reason for media to be very careful about what and how they report research findings, and if they do not, to not glibly accuse skeptics of not believing in science.
Opinions not anchored in reality will be scorned or, worse yet, will lead to actions and decisions that create unexpected results. At least a portion of those unexpected results will be either directly harmful (disutilities) to the actor or will impose harmful effects on others. The scorn comes if the science has been done and the opinion can be seen as ignorant. The real material or psychological harm to oneself or others is a result of either ignorance or of the science not having been done and the facts not being known.
When we do harm with knowledge aforethought, then we are in the moral dimension where the harm must be compared to some countervailing good in order to weigh the net result of our actions. By this moral calculus, bystanders can decide to cast credit or blame. Knowledge thus becomes the basis for accountability.
Accordingly, ignorance is never good, and it is always preferable to have science unfettered in the pursuit of truth.
Speaking as a scientist, I think the partisan claim that the Republicans are "anti-science" and the Democrats "follow the science" is inaccurate. Indeed, Republican politicians often disregard science, especially during the Trump era, but both parties tend to extoll scientific findings they find politically convenient, and ignore the others. Of course science is rarely definitive, and the large majority of its results are irrelevant to daily life.
In the future, when people claim they are the party of science, the retort will be... only because you control what science is published.
This finding was published, but I wonder how many other findings were hidden, or the papers tweaked because the findings were inconvenient.
But thanks for explaining the controversy. I saw it on Twitter yesterday, but was to lazy to figure out the story behind it.
Very helpful article that manages to cover a recent argument online while still threading the needle to discuss larger implications of the findings. I think trying to do justice to "the paper"'s many dimensions is hard and I appreciate the effort shown here. I hope people will consider, as much if not more, the challenges that criminal justice reform will face if we do not focus on the needs of fragile families who are at their wits' ends because of abusive behavior by family members. The fact is, for people in those families, the only help they have in 2021 is the police. The police are the one group of people with the legal authority to remove an abusive person from the home, but that is only for things that can justify such extreme actions taken. But there are many families with abusive family members, both siblings and parents and children, whose situation is such that they are unwilling to involve the police (for frankly incredibly smart and well thought out reasons) or simply do not have legal cause to do so (e.g., emotional, verbal and financial forms of abusive behaviors) and those can also be unbelievably ravaging on family members.
I think what is unusual about this paper, though, is its judge fixed effects design and the implications of that design. By definition, this paper estimates a LATE for the complier subpopulation . These are people for whom forcible removal through imprisonment occurred because they got a strict judge, but would not have occurred with a more lenient judge. It's this "disagreement principle" as I call it that drives the design's identification. The marginal abuser, in other words, is who this paper is about. So incredibly, it seems to say we are actually not paying attention enough to the externalities associated with criminal behavior. I think that's what it means -- the lenient judge is too lenient because he disregards or doesn't understand that there are others suffering from this person's behavior.
And yet, prison is not imo the first or even second best. The disagreement principle for me means we have got to find these families ASAP and get them help. We have got to find them because the lenient judges represent the very goal we have as society! We want more leniency even towards violence -- we just want it to fair and equitably done. But that necessarily means, as Pfaff has continually pointed out, handling violence differently. But if we reduce incarceration levels and don't simultaneously allocate major resources to fragile families and hurting communities, we do no one any good. We do harm. So the paper's conclusions for me is both/and. Both reduce incarceration levels AND start solving these problems that we are using harsh judges to solve (which are themselves unbelievably socially and personally costly).
You say that "no one is suggesting anything like [being explicit and transparent about biases and ideologies]" and maybe that's true for economists and scientists. But some philosophers of science are suggesting exactly that approach. One example is Kevin Elliott: https://theconversation.com/rather-than-being-free-of-values-good-science-is-transparent-about-them-84946 Another commenter already mentioned Liam Kofi Bright, and Heather Douglas has been leading the charge against the value free ideal, which is mostly used as an excuse for scientists to avoid responsibility for the consequences of their research choices and findings.
Pfaff's contrast between "it works" and "it works well/optimally/importantly" is so SO important.
Also, I like your discussion of "faith" as important to "science," given how often these two words are opposed. 👍
I proposed an inexpensive observational study that would help calibrate how much predictive power for graduate success of GREs, GPA, etc. is lost due to stratification bias in single-program samples. It would be useful and timely. My colleague agreed, but demurred: "That's getting too close to the third rail."
Previous Noahpinion interviewee Liam Kofi Bright has a paper discussing the views of W.E.B. DuBois on this issue.
http://www.liamkofibright.com/uploads/4/8/9/8/48985425/approximating_value_freedom.pdf
Critics aren't arguing that we should let the People's Soviet censor all our econ papers, they're expressing deep skepticism of a study that doesn't seem to gel well with existing research on incarceration and has a pretty questionable framing.
For example, suggested reasons given for why incarceration could have a positive effect on children were that it could rehabilitate the defendant from engaging in criminal activity and lead them to be a more committed caregiver, that it could lead the non-incarcerated parent to change their behaviour and possibly desist from crime, that seeing the family member go to jail could have a deterrent effect on the kid, and that removing the person could remove a harmful influence. Basically, all the classic arguments for punitive incarceration.
But the paper said that the highest effect was observed for black children. The race for whom we have decades of evidence that incarceration is unfairly overapplied for similar crimes. I think they even said that their offender sample was biased towards first-time offenders and drug offenders, for whom institutional racism should be most evident. And if I'm reading it correctly, they found a decrease in likelihood of future incarceration, better adult neighborhood, and a null effect for academic attainment and teen pregnancy, so they're not saying it just impacted one small aspect of life outcomes. It had neutral to positive effect on pretty much the big ones.
I'm not a sociologist or an economist, but I think that if I did a study and it suggested that the justice system actually works the best for the people for whom we know it works the worst, that aspect would be worth more discussion than the zero commentary it's given in the paper. Like, are black people reaping a windfall from institutional racism in the justice system? Would they be worse off if it was fairer? The authors don't seem to have even registered the strangeness.
It's not ideological for sociologists to be skeptical of this, any more than it was ideological for climate scientists to sneer at the Nordhaus papers you mentioned about the economic effects of climate change. And it's not out of line for these sociologists to suggest that a review board that was more diverse might have caught these questions and at least required the authors to address them, because white people, even ones who think they're being fair-minded, show substantial bias when asked to make judgments about issues in which race is salient; this fact has convincing empirical verification in many different studies, however uncomfortable it may make people.
This is (at least potentially) a great example of where insisting on understanding on something that makes us uncomfortable has the potential to improve lives. I doubt prison qua prison is responsible for the observed improvements in outcomes. Understanding what is the underlying cause can help us formulate actual policy that isn't just 'throw more people in prison'.
Or the results might fail to replicate v0v
To move away from the specific example, I think there is a (growing?) strain of thought which fully rejects objectivity in science. Scientific research is, in this mindset, as much an act of activism as knowledge seeking. The idea of pure knowledge seeking is viewed with suspicion - an exercise of privileged self-delusion at best, and often worse. Academic output is judged not on its quality but on the extent to which it supports the Cause, and all sorts of findings get denounced for being contrary to political goals (this crops up a lot in social science and particularly the intersection of psychology and biology).
New publication: The Journal of Inconvenient Research”, where every article has an abstract that ends “We have suppressed our data because reasons.”
I don't necessarily agree with the findings of the paper, and it may not be replicated, but the right way to disagree has to be with better theses and more accurate data. In fact, other policies probably result in even better reductions in incarceration. But we need to investigate.
Otherwise we'd be lying about the state of society and that definitely is not going to result in better outcomes for anyone.
It is unsurprising that an economic paper sparks this debate. Economists are roughly divided into two tribes. One approaches research as an attempt to understand the nature of economic activity. Assuming good practices are followed in data collection and data handling, let the results all where they may. The other enters a research program with strong priors about the expected outcomes. These priors are ideologically conditioned. Results not consistent with these priors are rejected as mistaken or, ironically, ideologically driven.
I would make a distinction between a research agenda and particular discoveries. The choice about what to study and the approach you’ll take there involves a wide range of non-science ideology - what’s important and moral and valuable. But within that agenda you are applying the scientific method and should be rigorous about truth seeking.
I don’t know about these particular researchers, but the broader research agenda of “applying science to criminal justice issues” seems to me valuable and important and moral. How can we protect citizens and not destroy lives in the process? I’d rather we be asking that question in a careful way than going by society’s moral gut (-> mass incarceration).
Of course, ever trust one piece of research. A body of work has to build up over time before we can draw policy conclusions.
This episode does reveal a lot, though. People (and media) will accept and promote a single piece of research if it fits their narrative. Those who say "apply a higher standard" in special cases, as this supposedly is, reveal that they probably have a tendency not to push back on research that supports their internal narrative. I think a lot of laypeople intuit that this happens and so they discount research findings that support the narrative of the media that are reporting them. This can be read as being anti-scientific, but there's a wisdom to it, if indeed that's the dynamic that is at work. That's a good reason for media to be very careful about what and how they report research findings, and if they do not, to not glibly accuse skeptics of not believing in science.
Opinions not anchored in reality will be scorned or, worse yet, will lead to actions and decisions that create unexpected results. At least a portion of those unexpected results will be either directly harmful (disutilities) to the actor or will impose harmful effects on others. The scorn comes if the science has been done and the opinion can be seen as ignorant. The real material or psychological harm to oneself or others is a result of either ignorance or of the science not having been done and the facts not being known.
When we do harm with knowledge aforethought, then we are in the moral dimension where the harm must be compared to some countervailing good in order to weigh the net result of our actions. By this moral calculus, bystanders can decide to cast credit or blame. Knowledge thus becomes the basis for accountability.
Accordingly, ignorance is never good, and it is always preferable to have science unfettered in the pursuit of truth.
Thank you for stating what should be a pretty obvious belief.
This seems like more evidence that political polarization is the biggest problem America faces. it poisons everything it touches.
Given the intensely lopsided political representation inside academia, how could we _not_ expect science to be politicized?