By Dr. Ken Broda Bahm:
According to the weather forecast, a pretty sizeable storm is headed to Denver. When this blog post goes to press, downtown Denver could be heading toward two feet of snow…or perhaps three inches, or nothing. Weather forecasts have a range of uncertainty. They’ve gotten better, and are no longer the butt of jokes like they used to be, but there are still some unknowns causing weather forecasts to be based on probabilities, not certainties. Litigation forecasts are similar. But when assessing the chances of winning in trial and the range of likely damages, lawyers are often missing something the weather forecasters have: data and Doppler radar. It is part of the attorney’s job to understand the law and the venue, to take a clear-eyed look at the case’s strenths and weaknesses, to consider the adversary’s skill, and to offer the client a reliable case assessment on the chances for success. But what do we know about how good attorneys are in that role?
Based on the research, we know they’re not all that good at it; at least not when they rely on their own subjective feelings on the case. According to a recent essay in Georgia State University Law Review by Alexis Knutson, Natalie Gordon and Edie Greene (2014), that has an impact on the chances of a good settlement. Attorneys must be able to forecast their own case’s chances of success in order to know what a good settlement would be, but “attorneys are not particularly skilled at these tasks.” They add, “allegiance to clients, immersion in the minutiae of the law, and attorneys’ own risk behaviors can blind them to the weaknesses in their case, the strengths of their adversaries’ evidence, and the likelihood of winning, thus resulting in difficulties in settling lawsuits.” That article references a number of retrospective studies showing that errors in settlement are common and spring from a variety of psychological causes. They note that mistakes in proceeding to trial when a better result could have been obtained beforehand, are more likely to affect plaintiffs, but end up being more costly to defendants. Both sides can also make the mistake of settling when a better result is likely to have been obtained in trial. The Knutson, Gordon, and Greene article is useful in pulling together all of the current research showing why poor case forecasting ability tends to be the rule and not the exception. The article also recommends a fix for those forecasting problems in the form of small group research – focus groups and mock trials – in order to provide a better foundation for settlement. This post takes a look at both the problem and the solution.
The Problem: Attorneys Fall Victim to Forecasting Errors
The article is very thorough in identifying and defining the problems, and sharing the research supporting each one. For a good and complete explanation, refer to the original article, which is available without charge at http://readingroom.law.gsu.edu/gsulr/vol31/iss4/2. For a more top-level view, here is my handy reference list of eight of the reasons Knutson, Gordon, and Greene share in explaining why attorneys’ trial forecasts aren’t going to be as reliable as the Weather Channel’s.
1. Partisan Distortion
I’m unconsciously thinking like an advocate when I assess the case, so I overweight the strengths and underweight my weaknesses.
2. Confirmation Bias
I notice, remember, and use the facts that support my case much more than I notice, remember, and use the facts that indict my case.
3. Anchoring
I’ve fixated on a (potentially arbitrary) number to represent my chances of success and the likely damages amount.
4. Endowment Effect
I’m “counting my chickens,” acting like the most favorable result is already “mine,” such that any deviation from that result seems like a loss.
5. Overconfidence
Like most attorneys, I believe I’m better than average. And I think most of my cases are winners.
6. Illusion of Control
Because I focus on the factors I can control, I tend to see those factors as the most important ones, and I discount or ignore the factors that are beyond my control.
7. Deferred Assessment
There are still unknowns due to pending motions and discovery, so I will assess the case later… and that may not happen until we’re in trial mode.
8. Sunk Costs
We’ve already invested so much into this that we need to invest a little more on the chance of a return on that investment.
One Solution: Good Quality Small Group Research
Some of those reasons can be mitigated through awareness and conscious effort, but many of them are unconscious biases. Their presence will be to some extent inevitable. The fix the authors recommend is to conduct small group research, like mock trials and focus groups. While they’re commonly seen as a way of preparing for trial, their greater utility might be in providing a foundation for your case forecast in the more likely event that the case is headed toward settlement. “Although Small Group Research does not provide crystal-ball-like predictions,” the authors write, “these exercises can shed light on the strengths and weaknesses of a case and move the lawsuit closer to resolution.” They title the article More Than a Feelin’. It’s a nod to that old Boston song…before the band collapsed in a storm of litigation that continues to this day (probably some failed case forecasting there as well). But the idea is that trial lawyers cannot afford to just rely on a feeling about their chances at trial. Having another reference point, even an imperfect one, is going to improve forecasting abilities.
After defining the services that they’re talking about, Knutson, Gordon, and Greene also point out that not all small group research is created equal. Citing the American Society of Trial Consultants, they emphasize that the project should follow reasonable standards and practice guidelines that support the validity and reliability of social science research in a trial setting. Good research will:
- Give equal or greater weight to the arguments on the other side.
- Rely on recruits that are as similar as possible to the relevant jury-eligible population.
- Obtain feedback while participants are blind regarding the actual research sponsor.
There is, however, one factor the authors don’t address, and it is critical. If the research is being conducted for the purposes of forecasting a potential range of damages, then the research needs to be designed for that purpose. In a typical mock trial, the presentations focus generally on the broad outlines of a case. To the extent that the attorneys share information on damages, it is often just a very rough sketch of what an actual jury will hear. While it is true that even when the real jury hears days or weeks of testimony, they are still often basing their case leanings on the same first impression that the mock jurors will have. But when it comes to damages, we have learned through post-trial interviews that the real jury generally will do the work of paying close attention to the details and exhibits, particularly in the categories of economic damages. So a mock trial focused on forecasting damages will need to give more of that information — possibly a lot more, depending on the complexity of your case. Where there isn’t time to do that in a standard one- or two-day mock trial, one innovation we’ve employed is to do a “call-back” mock trial. We conduct a liability-oriented mock trial as usual, and then one or two days later, invite back only the strong pro-plaintiff jurors to test what would happen with a jury that got to the point of awarding damages. We give them another day of information that is focused in detail on both side’s damages amounts, and then we watch them deliberate just on the damages questions.
This damages-focused phase of the mock trial can be less expensive than a typical mock trial because the attorneys are already prepared, the participants are already recruited, and you’re dealing with a smaller and more select group. Still, mock trial costs are often used as a reason for avoiding or delaying the research. For all the reasons covered in the research article, that is a mistake. Poor forecasting carries costs as well — sometimes very high costs. Looking at the amount of money that is often at issue, and the broad range of possible outcomes created by speculative case assessment, the costs of a well-run mock trial amount to a tiny fraction.
______
Other Posts on Case Assessment:
- Predict With Care: Adapt to Overconfidence in Case Assessment
- Beware the Bias for the “Safe” Call in Case Assessment
- Avoid ‘Vicarious Entrapment’ When Assessing Your Client’s Case
- Beware of TMI: More Information Doesn’t Lead to Better Decisions
______
Knutson, A., Gordon, N., & Greene, E. (2014). More Than a Feelin’: Using Small Group Research to Inform Settlement Decisions in Civil Lawsuits. Ga. St. UL Rev., 31, 685.
Image Credit: Colorado’s Weather Center/CBS4