By Dr. Ken Broda Bahm:
Leave it to the engineers. While we all have our subjective methods of estimating our odds in litigation, in the field of construction litigation, the act of handicapping has apparently been raised to a level of mathematical precision. Computer modelers have found a way to input known factors and predict construction trial outcomes with a success rate of 83 percent! I’m not sure if it is possible to hire those modelers or not, but the general approach tells us something important about case assessment in all contexts: Instead of giving in to your trial team’s group-think and confidence-boosting habits, give weight to multiple sources of information and strive for the most objective indicators of your chances in trial.
This post takes a quick look at the little known (to me at least) practice of mathematical case prediction in the construction litigation field, before turning to the more general question of how litigators should strive to get an unbiased sense of their chances for success or failure in trial. Specifically, I take a look at some new research on how attorneys use, and misuse, the second opinions that are so commonly sought in litigation. Apparently, asking another lawyer what they think of your case doesn’t work as well when you give that second opinion less weight than your own.
Case-Based Reasoning in Construction Litigation
In the early nineties, academics in the engineering field began applying the principle of “Case-Based Reasoning” to litigation outcomes. Chances are, you are already familiar with case-based reasoning, in general at least, from the methods applied in law school: You learn the principle by considering a case example, then another case, then another case, until a rule emerges that transcends the individual circumstances. At that level, it is just a way of reasoning by analogy. But when computers are added to the mix, case-based reasoning becomes a tool of artificial intelligence.
The researchers apparently thought of litigation as just one of the many unknowns that can affect a construction job — like the chances that a raw materials shortage will holdup the delivery of your drywall. Applying to litigation the same processes of quantitative risk assessment that were applied in other construction contexts, they built predictive models. David Arditi and Onur Tokdemir (1999), for example, looked at 43 input features (aspects about the case) and a total pool of 114 cases in order to predict outcomes in Illinois circuit courts. At first blush, experienced litigators are apt to say, “well, that will never work” because cases are so varied. Arditi and Tokdemir, however, found that there is a certain commonality to construction cases, that meant that in 83 percent of the cases the mathematical model was able to predict the verdict at trial.
Not a bad result, and the example has been followed by some other researchers as well. The lesson that applies beyond the data-driven models and outside the field of construction litigation is this: Consider as many factors as possible and weigh them objectively as you can.
Second Opinions in All Litigation
One area where that lesson applies more commonly is in the collegial practice of asking a colleague for a second opinion about a case. One recent study (Jacobson et al., 2011) discussed in the Wall Street Journal’s Law Blog, looks at lawyers’ and law students’ ability to predict jury verdicts before and after seeking second opinions from study “partners,” in the form of other lawyers making the same prediction. It turns out that when given access to another’s predictions, our accuracy improves, even when the other person has no more information than we do. However, the rub is in the basic human tendency to trust our own predictions more than the predictions made by others. Participants, especially the experienced trial attorneys, tended to give less weight to predictions made by others, and in the process, failed to benefit from the aggregation.
There are a few other findings from this study that are useful for attorneys trying to get a handle on their chances in trial:
-
The accuracy of predictions increased as the size of the group making the predictions increased. So get lots of opinions — from the team, from colleagues, from consultants, from paralegals and secretaries, and from the fellow running the A/V at the mock trial.
-
A requirement in the study to reach agreement significantly improved the accuracy of the predictions. So, when you seek other opinions, instead of just hearing someone out and internally saying, “Okay, you have your view and I have mine,” take the time to discuss it and to see where you can agree.
-
You won’t always agree, but take care in discounting the opinions of others. In the study, fully 53 percent ignored their partner’s opinion all together. On the whole, that reduced the accuracy of their predictions. The best predictions occurred when participants treated their partner as an equal, giving their opinion 50 percent weight.
Our predictions of an individual case’s chances, if and when it makes it to trial, are obviously a large part of the assessment driving settlement decisions. Given the reality that most cases settle, anything that improves our ability to make realistic and reliable case assessments helps at all stages — not just in the walk up to trial, but in the earliest stages of deciding what cases to develop and where to spend resources.
______
Other Posts on Case Assessment:
- Predict With Care: Adapt to Overconfidence in Case Assessment
- Be More Realistic Than Your Opponent
- Diagnose Your MedMal Case
______
Arditi, D. & Tokdemir, O. (1999). Using Case-Based Reasoning to Predict the Outcome of Construction Litigation Computer-Aided Civil and Infrastructure Engineering, 14(6), 385-393 DOI: 10.1111/0885-9507.00157
Jacobson, J., Dobbs-Marsh, J., Liberman, V., & Minson, J. (2011). Predicting Civil Jury Verdicts: How Attorneys Use (and Misuse) a Second Opinion Journal of Empirical Legal Studies, 8, 99-119 DOI: 10.1111/j.1740-1461.2011.01229.x
Photo Credit: Wallyg, Flickr Creative Commons