I believe the original studies were shown to be faulty because they didn’t account for the fact that the cases were ordered. Less severe cases were seen first, which meant the more severe cases (ie those with more severe penalties) were shown later.
“Danziger etal. rely crucially on the assumption that the order of the cases is random and, thus, exogenous to the decision-making process. This assumption has been forcefully challenged. For a short and very critical reply in PNAS, Keren Weinshall-Margel and John Shapard analyzed the data of the original study—as well as other self-collected data—and conducted additional interviews with the court personnel involved.Footnote 51 They point out that the order of the cases is not random: The panel tries to deal with all cases from one prison before a break, before then moving to the cases of the next prison after a break. Most importantly, though, requests from prisoners who are not represented by a lawyer are typically dealt with at the end of each session. So, prisoners without legal representation are less likely to receive a favorable decision compared to those with legal representation.Footnote 52 Additionally, lawyers often represent several inmates and decide on the order in which the cases are presented—it might well be possible that they start with the strongest cases”
[1] Chatziathanasiou, K., 2022. Beware the lure of narratives:“hungry judges” should not motivate the use of “artificial intelligence” in law. German Law Journal, 23(4), pp.452-464.
> the fact that the cases were ordered. Less severe cases were seen first, which meant the more severe cases (ie those with more severe penalties) were shown later.
Isn't that order creating a bias for the judge? Should the cases be randomized instead?
Yes, in the words of the linked paper it injects exogenous decision making. In other words, the decision is based on more than just the judge so we can’t conclude the discrepancy is due to the judge’s personal bias.
Like any other set of tasks, the order is a practical matter. Severe cases are more random, sometimes sucking up more time than expected. Put them early in the order and any delay will impact everyone. So judges try to get the easy/predictable stuff done first, minimizing the number of people impacted by the inevitable delays. (Doctors do the same if they can, trying tk see easy patients first.)
Also, as with any other process, you want to start with a few easy wins in order to solve inevitable problems. If there is something wrong with the court tech (recorders, security etc) you can work it out during the easy cases. Save the murders for after everything is sorted.
Thirdly, prisoners are different than normal people. They do not control thier own scheduals. Put thier case early in the docket and they might miss breakfast at the jail. Prisoners are also moved as groups. Put them randomly on the docket and they all have to wait all day. Put them as a group towards the end and the group wont wait as long. So cases involving prisoners, on average the more severe cases, are placed later on the docket.
For those unfamiliar, the original study found that judges were kinder in their decisions right after lunch, and harshest right before. (I’m dramatically oversimplifying, but that’s the bit folks usually cite.)
This study contests the strength of that finding by showing that positive rulings take longer, and that you can fit more simple negative rulings in just before a break (negative rulings are denials of parole, if you’re wondering why they are faster). Judges don’t want to start complex cases that are more likely to be favorable just before break. (Again, dramatically simplifying. The article has more.)
I have cited the original study countless times, and this injects a lot more nuance for me. I’m glad it was revisited.
Without having read in-depth either original paper, it seems like the issue here is much simpler than reproduction (though reproduction is the gold standard as is totally under-appreciated these days).
Rather, it seems the authors made a much simpler mistake: hypotheses can only be refuted by evidence, not confirmed. So, in this case, if the hypothesis is "judges act more harshly when hungry", what they should have been doing is looking for evidence disproving that statement. Instead, they seem to have presented a correlation and a suggestion, which is not the same thing as a scientific finding.
There was another article related to this study that hit the HN front page. It talked about the size of effect, and argued that the effect size was ridiculously big, and if true, we should see giant spikes in car crashes around lunch, big enough for common sense to ban driving just before noon.
I think you'll enjoy the read.
> If hunger had an effect on our mental resources of this magnitude, our society would fall into minor chaos every day at 11:45. Or at the very least, our society would have organized itself around this incredibly strong effect of mental depletion.
It's always fascinating to see the vapor of trends which seem local and harmless - work sucks, i can't wait for lunch - condensed into real-world results. In the same vein is an infamous study correlating the beginning of daylight savings time with a marked increase in heart attacks [0].
Did anybody ever ask the judges/clerks about this finding? It seems like the whole thing could have been rebutted with one phone call and the judge saying “yeah, we make the schedules and intentionally backload the negative/easy ones”
They have incentive to hide it. Ordinary people want to think the justice system actually does justice. Not group cases by time slot because they are likely to be simply denied, which is clearly biased.
Personally this is more disturbing to me than the alleged earlier finding that the judge was in a bad mood because he was hungry. At least that is correctable and avoidable; systemic discrimination by case type isn't.
Statistics for stuff that has unequal length is tricky. Reminds me of the days there I did packet sampling on the LAN and as the length distribution is bimodal statistical math using means as assumption yielded misleading results.
Interesting, didn't see this before. Now I am curious about addicted judge effect (e.g. smoking) since I witnessed weird things semi-similar to this, but for smoking.
Not to mention how different vices are going to affect a judge differently. One addicted to the lunchtime brothel might rule different than one addicted to the lunchtime smoke, and different still to one addicted to the lunchtime scotch.
How much of the current failure to repliacte could be attributed to the subjects undee observation being aware of the phenomenon and actively compensating?
Study finds that when there's a time limit, even a rational judge would try the case faster, and there would be a tendency towards unfavorable ruling.
Since lunchtime (and presumably end of work day) are such time limits, we see a drop in favorable rulings as lunchtime approaches, and a restoration in favorable rulings right after lunchtime.
And yet, the study, curiously, says, "the analyses by DLA do not provide conclusive evidence for the hypothesis that extraneous factors influence legal rulings". What is lunchtime and end of work day as not extraneous factors?
Note: the above only explains a part of the original finding. And the study admits that there are definitely more factors at play.
> Study finds that when there's a time limit, even a rational judge would try the case faster, and there would be a tendency towards unfavorable ruling.
This study does not say this.
The simulated rational judges are "ideal" and their decisions are not influenced by the ordering of the cases or how long it has been since a break.
The study is saying that despite this perfect behavior, some simulated methods for choosing when to take a break will cause favorable cases to be more likely to be scheduled at the beginning of a session (in their last simulation, this effect only appears after applying the same statistical processing as the original study).
Readit News
“Danziger etal. rely crucially on the assumption that the order of the cases is random and, thus, exogenous to the decision-making process. This assumption has been forcefully challenged. For a short and very critical reply in PNAS, Keren Weinshall-Margel and John Shapard analyzed the data of the original study—as well as other self-collected data—and conducted additional interviews with the court personnel involved.Footnote 51 They point out that the order of the cases is not random: The panel tries to deal with all cases from one prison before a break, before then moving to the cases of the next prison after a break. Most importantly, though, requests from prisoners who are not represented by a lawyer are typically dealt with at the end of each session. So, prisoners without legal representation are less likely to receive a favorable decision compared to those with legal representation.Footnote 52 Additionally, lawyers often represent several inmates and decide on the order in which the cases are presented—it might well be possible that they start with the strongest cases”
[1] Chatziathanasiou, K., 2022. Beware the lure of narratives:“hungry judges” should not motivate the use of “artificial intelligence” in law. German Law Journal, 23(4), pp.452-464.
Isn't that order creating a bias for the judge? Should the cases be randomized instead?
Like any other set of tasks, the order is a practical matter. Severe cases are more random, sometimes sucking up more time than expected. Put them early in the order and any delay will impact everyone. So judges try to get the easy/predictable stuff done first, minimizing the number of people impacted by the inevitable delays. (Doctors do the same if they can, trying tk see easy patients first.)
Also, as with any other process, you want to start with a few easy wins in order to solve inevitable problems. If there is something wrong with the court tech (recorders, security etc) you can work it out during the easy cases. Save the murders for after everything is sorted.
Thirdly, prisoners are different than normal people. They do not control thier own scheduals. Put thier case early in the docket and they might miss breakfast at the jail. Prisoners are also moved as groups. Put them randomly on the docket and they all have to wait all day. Put them as a group towards the end and the group wont wait as long. So cases involving prisoners, on average the more severe cases, are placed later on the docket.
Randomness introduces inefficiency which implies delay
For those unfamiliar, the original study found that judges were kinder in their decisions right after lunch, and harshest right before. (I’m dramatically oversimplifying, but that’s the bit folks usually cite.)
This study contests the strength of that finding by showing that positive rulings take longer, and that you can fit more simple negative rulings in just before a break (negative rulings are denials of parole, if you’re wondering why they are faster). Judges don’t want to start complex cases that are more likely to be favorable just before break. (Again, dramatically simplifying. The article has more.)
I have cited the original study countless times, and this injects a lot more nuance for me. I’m glad it was revisited.
Rather, it seems the authors made a much simpler mistake: hypotheses can only be refuted by evidence, not confirmed. So, in this case, if the hypothesis is "judges act more harshly when hungry", what they should have been doing is looking for evidence disproving that statement. Instead, they seem to have presented a correlation and a suggestion, which is not the same thing as a scientific finding.
I think you'll enjoy the read.
> If hunger had an effect on our mental resources of this magnitude, our society would fall into minor chaos every day at 11:45. Or at the very least, our society would have organized itself around this incredibly strong effect of mental depletion.
https://daniellakens.blogspot.com/2017/07/impossibly-hungry-...
[0] https://www.sciencedirect.com/science/article/pii/S254245482...
Personally this is more disturbing to me than the alleged earlier finding that the judge was in a bad mood because he was hungry. At least that is correctable and avoidable; systemic discrimination by case type isn't.
Do judges give out tougher sentences when hungry? A study too good to be true - https://news.ycombinator.com/item?id=35491060 - April 2023 (202 comments)
Impossibly Hungry Judges - https://news.ycombinator.com/item?id=22020716 - Jan 2020 (1 comment)
Rebuttal to hungry judges give harsher sentences - https://news.ycombinator.com/item?id=19958435 - May 2019 (1 comment)
Impossibly Hungry Judges (2017) - https://news.ycombinator.com/item?id=18112378 - Oct 2018 (58 comments)
Impossibly Hungry Judges - https://news.ycombinator.com/item?id=14701328 - July 2017 (70 comments)
Do hungry judges give harsher sentences? - https://news.ycombinator.com/item?id=2438189 - April 2011 (1 comment)
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
There are some tricks to solve my problem. The general term for it is "periodogram".
https://en.wikipedia.org/wiki/Least-squares_spectral_analysi...
https://www.mathworks.com/help/signal/ug/spectral-analysis-o...
https://daniellakens.blogspot.com/2017/07/impossibly-hungry-...
The original effect was known to be spurious on publication.
Since lunchtime (and presumably end of work day) are such time limits, we see a drop in favorable rulings as lunchtime approaches, and a restoration in favorable rulings right after lunchtime.
And yet, the study, curiously, says, "the analyses by DLA do not provide conclusive evidence for the hypothesis that extraneous factors influence legal rulings". What is lunchtime and end of work day as not extraneous factors?
Note: the above only explains a part of the original finding. And the study admits that there are definitely more factors at play.
This study does not say this.
The simulated rational judges are "ideal" and their decisions are not influenced by the ordering of the cases or how long it has been since a break.
The study is saying that despite this perfect behavior, some simulated methods for choosing when to take a break will cause favorable cases to be more likely to be scheduled at the beginning of a session (in their last simulation, this effect only appears after applying the same statistical processing as the original study).