By Dr. Ken Broda Bahm:
A good piece of advice for employers: If your habit is to sweep things under the rug, then don’t commission a study to look under the rug. Last Thursday, the ABA Journal released a fascinating report with important implications on the role of research, the risks of discrimination, and the occasional absence of a self-protective instinct exhibited even by a law firm. Its name protected by a confidentiality agreement, a Wall Street law firm with national and international offices hired a consultant, M.J. Tocci, to look into its evaluation procedures involving 268 junior attorneys. In a firm where near-perfect scores are required for promotion to partnership, Tocci found the women received significantly lower scores. If that was explained by the women simply being less effective, you would expect to see the same difference in the narrative comments that accompanied the scores, but you don’t. In some ways, in fact, the women’s narratives were better.
So what is happening? The researchers concluded that gender bias is indeed alive and well at this firm, and playing a dramatic role in limiting the partnership opportunities for female associates. But the real kicker is this: What does the firm plan to do about it? Absolutely nothing.
This post is directed, not at the trial defense, but at in-house legal and HR departments, with the aim of reducing the need for a trial defense. For the law firm at issue, the only thing standing between it and a class action discrimination suit is a confidentiality agreement. But for all companies, it is important to take a close look at this study, in order to understand and address subtle discrimination and the potential vulnerabilities of your employee evaluation process.
The Study
Published in Social Psychology and Personality Science in July, the study (Biernat, Tocci & Williams, 2011) focused on the degree of consistency between numerical evaluations (e.g., 4 for “Meets Expectations,” and 5 for “Exceeds Expectations”) and the accompanying narrative comments. Counting the number of “Positive Performance Words” (e.g., “excellent,” “stellar,” etc.), and looking at themes such as competence and interpersonal warmth, the researchers discovered a disconnect that strongly correlated with the gender of the junior lawyer being evaluated. Specifically:
-
Men received significantly higher numerical scores.
-
Males’ higher scores were not explained by better narrative comments. In fact, women received more open-ended “Positive Performance Words,” but this did not translate into higher scores.
-
Evaluation of technical competence mattered more for men: Men praised for competence received higher marks than women receiving the same competence-based praise.
-
Evaluations of interpersonal warmth mattered more for women: Women criticized for low interpersonal warmth received lower marks than men criticized for the same problem.
-
Comparing the men and women who were matched based on receiving the best narrative comments, the men still received higher numerical scores.
-
The net effect of this pattern, combined with the firm’s reliance on numerical scores, is that it is nearly four times more likely that men rather than women will be promoted to partner.
Of course, this is a detailed look at one firm, but if the results are anything close to typical, then we need not wonder too much about at least one cause of under-representation of women at the partnership level.
The Response
Difficulty in ensuring gender neutrality might be understandable, especially when the discrimination hides in nuances of the evaluation system. But in this case, the law firm’s response is not understandable. As reported in an ABA Journal note this past Thursday, the unnamed firm “doesn’t intend to change anything about its performance review process.”
In fact, the consultant, M.J. Tocci, might have seen that coming when one partner told her, “We’d like for you to help us get a better outcome, but we’d rather not change anything.” Despite the fact that the firm was, according to her, “getting dinged as one of the worst places for women to work,” Tocci said that the firm continued to attract female associates and wasn’t receiving any pressure from clients for greater diversity in the partnership ranks. “I thought the results of the study were shocking,” Tocci told The Careerist last week, “But the firm didn’t care…I think they believed my conclusions. They were just unwilling to do anything about it.”
The Recommendations
1. Be Careful of Any Cost Benefit Analysis on Eliminating Discrimination. Let’s hope that this anonymous law firm is one day proven wrong, when recruiting difficulty and/or client pressure force it to change its evaluation and partnership processes. But even if it isn’t wrong, it is best to treat the elimination of both discrimination as well as the perception of discrimination as ends in themselves: Not good because they lead to better public perception, client relations, litigation defenses, or recruiting advantages, but good simply because it has intrinsic value.
2. Ensure Consistency Between Written Comments and Narrative Scores. If your company, like most companies, has an evaluation system in which employees receive both scores and numerical rankings, then take a close look at the consistency between the two, or even use a second or third pair of eyes. No employee likes to see “Fantastic job,” and a four out of five. It is a good practice to use narrative comments to explain the rank, and not just to give open-ended feedback to the evaluated individual. In other words, if the employee has a four then the narrative should at least implicitly explain why it isn’t a three and isn’t a five.
3. Develop Clear and Gender-Neutral Expectations. For example, think about how much competency and interpersonal warmth matter to the evaluation. And be conscious of the possibly “gendered” role of these and other concepts. We know from research (e.g., Rudman & Glick, 2008), for example, that even well meaning evaluators are a little more likely to emphasize competence when evaluating men, and warmth when evaluating women.
4. Law Firms: Make Sure This Isn’t Your Pattern. If you’re reading this as a law firm partner, and you’ve never heard of M.J. Tocci, then take comfort in the fact that yours isn’t the unnamed firm. However, don’t get too comfortable in the assumption that it is just another firm’s problem. As law professor Joan Williams, another of the study’s authors, notes, “This study suggests that one reason law firms can’t keep women is that firms’ evaluation systems are not correcting for implicit biases that disadvantage women,” she says. “Law firms need to adopt best practices, including having someone trained to spot gender bias reviewing all evaluations before they become final.”
______
Related Posts:
- Count Your Plaintiffs Before Certification Hatches: Class Size Matters in Some Unexpected Ways
- In Employment Cases (and All Cases), Keeping it Simple is Smart
- Assess Your Juror’s Economic Security: A Vulnerable Juror Can Make for a Vulnerable Defense (Part One)
______
Monica Biernat, M.J. Tocci and Joan C. Williams (2011). The Language of Performance Evaluations: Gender-Based Shifts in Content and Consistency of Judgment Social Psycholoigical and Personality Science, Published Online (July 18)
Photo Credit: grahamc99, Flickr Creative Commons, Photograph of original graffiti by Banksy