EQUAL EMPLOYMENT OPPORTUNITY COMMISSION, Plаintiff-Appellant, v. FREEMAN, Defendant-Appellee, and The United States Office of Personnel Management, Intervenor.
No. 13-2365
United States Court of Appeals, Fourth Circuit
Decided: Feb. 20, 2015
778 F.3d 463
For all of these reasons, we will hold that the District Court exercised sound judgment and acted within its discretion when it denied the Union‘s motion for relief from judgment.
Pacific Legal Foundation; Equal Employment Advisory Council; National Federation of Independent Business Small Business Legal Center; Retail Litigation Center; Chamber of Commerce of The United States of America, Amici Supporting, Appellee.
Argued: Oct. 29, 2014.
causes of action was designed to foster the “full and active participation by the rank and file in the affairs of the union.” Id. at 975 (quoting Hall, 412 U.S. at 7-8, 93 S.Ct. 1943). Though the District Court did not discuss the claims in these terms, it is legitimate, under common benefit analysis, to assess whether the suit “vindicat[ed] statutory policy.” Mills, 396 U.S. at 396, 90 S.Ct. 616.
Before GREGORY, AGEE, and DIAZ, Circuit Judges.
Affirmed by published opinion. Judge GREGORY wrote the opinion, in which Judge AGEE and Judge DIAZ joined. Judge AGEE wrote a separate concurring opinion.
GREGORY, Circuit Judge:
In 2001, Freeman began conducting background checks on its job applicants,
I.
Freeman is a provider of integrated services for expositions, conventions, and corporate events, with offices in major cities throughout the United States. In 2001, the company commenced background checks of job applicants’ credit and criminal justice histories. Criminal background checks were required for all applicants, and credit history checks for “credit sensitivе” positions involving money handling or access to sensitive financial information. Freeman‘s credit and criminal background check policies excluded applicants whose histories revealed certain prohibited criteria. If an applicant‘s history included one of the listed criteria, like a conviction for a crime of violence, the applicant was not hired.1 Freeman modified these criteria on July 20, 2006, and again on August 11, 2011, after which it no longer conducted credit checks.
In 2008, after an applicant who was denied a рosition filed a charge of discrimination, the EEOC began an investigation of Freeman‘s credit check policy. On September 25, 2008, it notified Freeman it was expanding this investigation to the criminal background check policy. On March 27, 2009, the EEOC issued a letter of determination finding Freeman‘s use of credit and criminal checks violated Title VII.
After conciliation failed, the EEOC filed suit under
The case proceeded to discovery. The EEOC produced a report by Kevin Murphy, an industrial/organizational psychologist, and one by Beth Huebner, an associate professor of criminology, which
The district court denied the EEOC‘S motion for leave to file a sur-reply and granted Freeman‘s motion to exclude Murphy‘s testimony on the basis that it was “rife with analytical errors” and “completely unreliable” under
II.
We review a district court‘s decision to admit or to exclude expert evidence for an abuse of discretion. See Westberry, 178 F.3d at 261. A district court abuses its discretion if it relies on an error of law or a clearly erroneous factual finding. See id. We reverse the district cоurt only if we have “a definite and firm conviction that the court below committed a clear error of judgment in the conclusion it reached upon a weighing of the relevant factors.” Id. (quoting Wilson v. Volkswagen of Am., Inc., 561 F.2d 494, 506 (4th Cir.1977)).
A.
The district court identified an alarming number of errors and analytical fallacies in Murphy‘s reports, making it impossible to rely on any of his conclusions. Freeman provided the EEOC with complete background check logs for hundreds, if not thousands, of applicants who Murphy did not include in his database of fewer than 2,014 background checks conducted largely before October 14, 2008. J.A. 1061. Only 19 post-October 14, 2008 applicants were included in Murphy‘s data-
Most troubling, the district court found a “mind-boggling” number of errors and unexplained discrepancies in Murphy‘s database. For example, looking at a subset of 41 individuals for whom the EEOC is seeking back pay, 29 had at least one error or omission. Seven were missing from the database altogether. Seven were listed in the database without a race code, “one was incorrectly coded as passing the criminal background check, two were incоrrectly coded as failing the criminal background check, one ha[d] an incorrect race code, five ha[d] incorrect gender codes, nine [we]re listed twice and double-counted in Murphy‘s results, and three who failed the credit check [we]re not coded with a credit check result.” J.A. 1064. The EEOC claims these errors were present in the original data, a contention dispelled by comparing the information from the discovery materials to Murphy‘s database. It was in fact Murphy who introduced these errors into his own analysis.5
The EEOC also contends that Murphy fixed any errors in his analysis in subsequently-filed, supplemental reports. The district court examined a third report by Murphy6 and found that he did not make certain corrections to his database, despite claims of doing so. Contrary to his assertions, Murphy did not change “incorrect coding of race and pass/fail status for several individuals.” J.A. 1065. The district court also found that Murphy “managed to introduce fresh errors into his new analysis,” like double-counting applicants who had failed their background checks. Id. And Murphy‘s new, expanded database still omitted hundreds of applicants for whom Freeman produced complete information in discovery.
The sheer number of mistakes and omissions in Murphy‘s analysis renders it “outside the range where experts might reasonably differ.” Kumho, 526 U.S. at 153, 119 S.Ct. 1167. We therefore cannot say the district court abused its discretion in
III.
We affirm the district court‘s grant of summary judgment8 to Freeman solely on the basis that the district court did not abuse its discretion in excluding EEOC‘s expert reports as unreliable under Rule 702. We decline to consider whether the district court erred in limiting the time period in which the EEOC could seek relief, as any error in this regard was inconsequential in light of Murphy‘s pervasive errors and utterly unreliable analysis. We decline to reach any other issues in the district court‘s opinion.
AFFIRMED.
AGEE, Circuit Judge, concurring:
Although I concur in Judge Gregory‘s opinion, I write separately to address my concern with the EEOC‘s disappointing litigation conduct. The Commission‘s work of serving “the public interest” is jeopardized by the kind of missteps that occurred here. Gen. Tel. Co. of the Nw. v. EEOC, 446 U.S. 318, 326, 100 S.Ct. 1698, 64 L.Ed.2d 319 (1980). And it troubles me that the Commission continues to proffer expert testimony from a witness whose work has been roundly rejected in our sister circuits for similar deficienсies to those we observe here. It is my hope that the agency will reconsider pursuing a course that does not serve it or the public interest well.
I.
As in other cases, the EEOC proffered expert testimony to establish the alleged disparate impact of Freeman‘s background check policies. Yet the expert testimony here was fatally flawed in multiple respects.
A.
The district court used harsh words to describe the work of the EEOC‘s “expert,” Kevin R. Murphy. The court found that Murphy‘s reports contained a “plethora” of “analytical fallaсies,” reflected “cherry-picked” data, produced “a meaningless, skewed statistic,” and included a “mind-boggling number of errors.” EEOC v. Freeman, 961 F.Supp.2d 783, 793-96 (D.Md.2013). Even when Murphy submitted late-in-the-day amendments, he still relied upon “a skewed database plagued by material fallacies.” Id. at 796. The slap-dash nature of Murphy‘s work convinced the district court that the EEOC had only a “theory in search of facts to support it.” Id. at 803.
The majority opinion rightly agrees with the district court‘s view, as Murphy‘s work simply did not meet the standards for expert testimony that
First, courts often caution experts against drawing broad conclusions from incomplete data. In Lilly v. Harris-Teeter Supermarket, 720 F.2d 326 (4th Cir.1983), for instance, this Court criticized an expert for using data from only a limited set of relevant locations and years to draw conclusions about a much broader class. See id. at 337 (“The first problem with this data, however, is that its scope—covering the stores and warehouse for only 1976 and only the stores for 1975—is insufficient to prove discrimination from 1974
Yet as the majority notes, Murphy made the very mistake identified in Lilly: he omitted important information from relevant periods and locations. The EEOC challenged credit check policies beginning in late March 2007 and ending in early August 2011; its criminal-background-check claims spanned November 30, 2007 to the presеnt. For reasons unknown, Murphy‘s data included barely any information on applicants after mid-October 2008—ignoring at least two-and-a-half years of relevant and available data for each claim. By arbitrarily putting aside those years, Murphy ignored 300 credit checks and 1,500 criminal background checks. Indeed, Murphy even ignored applicant data on persons that the EEOC identified as purported victims. Worse still, Murphy ignored relevant criminal background check data from 21 of Freeman‘s 39 different locations.
Neither Murphy nor the agency explained these omissions. Although the EEOC speculates that Freeman produced incomplete data, the record says differently. Among other things, Freeman produced applicant logs, datasheets, and background check forms that Murphy could have used to compile relevant information. Thus, as the majority indicates, the district court‘s finding that Freeman presented more than sufficient data is far from clearly erroneous. For his part, Murphy insisted that there was no need to look at more of the available information regardless of relevance. Yet he never explained why his model incorporated enough observations to ensure a valid statistical result.1
Second, courts have consistently excluded expert testimony that “cherry-picks” relevant data. See, e.g., Bricklayers & Trowel Trades Int‘l Pension Fund v. Credit Suisse Secs. (USA) L.L.C, 752 F.3d 82, 92 (1st Cir.2014); Greater New Orleans Fair Hous. Action Ctr. v. U.S. Dep‘t of Hous. & Urban Dev., 639 F.3d 1078, 1086 (D.C.Cir.2011); Barber v. United Airlines, Inc., 17 Fed.Appx. 433, 437 (7th Cir.2001); Fail-Safe, LLC v. A.O. Smith Corp., 744 F.Supp.2d 870, 891 (E.D.Wis.2010); In re Bextra & Celebrex Mktg. Sales Practices & Prod. Liab. Litig., 524 F.Supp.2d 1166, 1176-77 (N.D.Cal.2007). “Cherry-picking” data is essentially the converse of omitting
Murphy undeniably “cherry-picked.” The very few pieces of post-October-2008 data that Murphy included consisted of 19 applicants. Of those 19, one was a double-counted applicant, one was a “fail” miscoded as a “pass,” and the remaining were all “fails” under one or the other (or both) checks. This 100% failure rate among the 19 post-October-2008 applicants wildly varies from the 3.5% failure rate for criminal checks and 9.9% failure rate for credit checks reflected in the rest of the data. See J.A. 326 (noting that “the likelihood of failing either [cheсk] is low“). Thus, not only was Murphy capriciously selective in his use of post-October-2008 data, but the high number of “fails” among his few selections suggests that he fully intended to skew the results. The district court certainly thought so, terming Murphy‘s work “an egregious example of scientific dishonesty.” Freeman, 961 F.Supp.2d at 792.
Finally, Murphy‘s analysis contained many obvious errors and mistakes, and these “factual deficiencies” further evidence his “faulty methods and lack of investigation.” Brown v. Burlington N. Santa Fe Ry. Co., 765 F.3d 765, 773 (7th Cir.2014); see also Dart v. Kitchens Bros. Mfg. Co., 253 Fed.Appx. 395, 399 (5th Cir.2007) (noting that “basic mathematical errors and flaws in methodology” were appropriate reasons to exclude an expert); cf. Overton v. City of Austin, 871 F.2d 529, 539 (5th Cir.1989) (per curiam) (“[A] trial court should not ignore the imperfections of the data used[.]“). For example, Murphy‘s initial statistical analysis was filled with basic arithmetic mistakes. Even once those fundamental errors were corrected, problems lingered. Murphy excluded applicants with known race and gender information, inaccurately claiming incomplete information. He miscoded criminal and credit check outcomes, as well as race and gender information. And he double-counted other applicants. As the majority recounts, within a sample of 41 known “victims” in Murphy‘s database, 29 of those 41 (or more than 70%) had errors or omissions.
In sum, Murphy‘s work was riddled with fundamental errors, mistakes, and misrepresentations. I certainly agree with the majority‘s determination that the district court appropriately excluded Murphy‘s evidence.
B.
These problems would be troubling enough standing alone, but they are even more disquieting in the context of what appears to be a pattern of suspect work from Murphy.
EEOC v. Kaplan Higher Education Corp., 748 F.3d 749 (6th Cir.2014), provides only the most recent example. There, the EEOC sought to use Murphy‘s testimony to challenge an employer‘s use of credit checks, just as it did here.2 A panel of the Sixth Circuit, however, unanimously affirmed the district court‘s decision to exclude Murphy‘s determinations. Like his work in this case, Murphy‘s analysis in Kaplan was filled with errors; among other things, he again “overrepresented ‘fails’ generally” and again drew conclusions from a skewed, unrepresenta-
Murphy‘s flawed approach is not just a recent problem. Over a decade ago, in Cooper v. Southern Co., 390 F.3d 695 (11th Cir.2004), overruled in part by Ash v. Tyson Foods, Inc., 546 U.S. 454, 456-57, 126 S.Ct. 1195, 163 L.Ed.2d 1053 (2006) (per curiam), Murphy drew different but no less severe criticism. The Eleventh Circuit concluded a report from Murphy served only to “recapitulate[] the basic allegations of the plaintiffs in the guise of an expert report.” Id. at 716 n. 10. Indeed, his report lacked any “statistical evidence to substantiate [its] broad claims.” Id. Thus, the report and the “sweeping conclusions” within it were “of extremely limited use.” Id.
Other recent cases provide additional examples of Murphy‘s lax attitude towards scientific rigor. In Boelk v. AT & T Teleholdings, Inc., No. 12-cv-40-bbc, 2013 WL 3777251 (W.D.Wis. July 19, 2013), for example, Murphy attempted to offer an expert opinion premised on “common sense,” “obvious[ness],” and “foreseeab[ility].” Id. at *8. Unsurprisingly, the district court held that such testimony was “not the appropriate subject of expert testimony” and did not create a genuine dispute of material fact at summary judgment. Id. Echoing a familiar theme, the court dubbed Murphy‘s testimony “too general and speculative to be useful.” Id. The Second Circuit too has rejected Murphy‘s conclusions, holding that Murphy had incorrectly accused another expert of making unfounded assumptions in her report. See M.O.C.H.A. Soc‘y, Inc. v. City of Buffalo, 689 F.3d 263, 278-79 (2d Cir.2012).
II.
Despite Murphy‘s record of slipshod work, faulty analysis, and statistical sleight of hand, the EEOC continues on appeal to defend his testimony. Conceding that Murphy‘s report was not an “A+ report,” the EEOC nevertheless says that it meets some indeterminate threshold of reliability. In doing so, however, the Commission advances positions that are not grounded in law. Most troubling is its view that problems in an expert‘s data are an inappropriate reason to exclude that expert.
Evidence is admissible only if “it rests on a reliable foundation.” Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 597, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993). Thus,
The EEOC, however, ignores this threshold analysis by contending that the issue of the reliability of an expert‘s data is always a question of fact for the jury, except perhaps in some theoretical, rare case. See, e.g., Reply Br. 15 (“[P]urported flaws in Murphy‘s analyses concerned data . . . and therefore concerned weight/credibility issues for trial, not admissibility.“). The agency‘s contention ignores Daubert‘s instruction that the district court must act as a gatekeeper. Moreover, no court has accepted the agency‘s argument. Rather, courts widely agree that “trial judges may evaluate the data offered to support an expert‘s bottom-line opinions to determine if that data provides adequate support to mark the expert‘s testimony as reliable.” Milward v. Acuity Specialty Prods. Grp., Inc., 639 F.3d 11, 15 (1st Cir.2011) (quotation marks and alteration omitted); accord Blunt v. Lower Merion Sch. Dist., 767 F.3d 247, 276 (3d Cir.2014); In re TMI Litig., 193 F.3d 613, 697 (3d Cir.1999); United States v. City of Miami, 115 F.3d 870, 873 (11th Cir.1997). The EEOC‘s contention was not simply meritless, but unsupported and without a legal foundation.
*
The EEOC wields significant power, some of which stems from the agency‘s broad discretion to investigate, conciliate, and enforce, and some of which derives from public actions that exert influence outside the courtroom. The Commission‘s actions can be also expected to have brоader consequences than those of an ordinary litigant given the “vast disparity of resources between the government and private litigants.” EEOC v. Great Steaks, Inc., 667 F.3d 510, 519 (4th Cir.2012).
In deciding when to act, the Commission must balance sometimes-competing responsibilities. On the one hand, the agency must serve the employee‘s interest by preventing an employer from “engaging in any unlawful employment practice” under
The EEOC must be constantly vigilant that it does not abuse the power conferred upon it by Congress, as its “significant resources, authority, and discretion” will affect all “those outside parties they investigate or sue.” EEOC v. Propak Logistics, Inc., 746 F.3d 145, 156 (4th Cir.2014) (Wilkinson, J., concurring). Government “has a more unfettered hand over those it either serves or investigates, and it is thus incumbent upon public officials, high and petty, to maintain some appreciation for the extent of the burden that their actions may impose.” Id. The Commission‘s con-