Lead Opinion
Affirmеd by published opinion. Judge GREGORY wrote the opinion, in which Judge AGEE and Judge DIAZ joined. Judge AGEE wrote a separate concurring opinion.
In 2001, Freeman began conducting background checks on its job applicants,
I.
Freeman is a provider of integrated services for expositions, conventions, and corporate events, with offices in major cities throughout the United States. In 2001, the company commenced background checks of job applicants’ credit and criminal justice histories. Criminal background checks were required for all applicants, and credit history checks for “credit sensitive” positions involving money handling or access to sensitive financial information. Freeman’s credit and criminal background check policies excluded applicants whose histories revealed certain prohibited criteria. If an applicant’s history included one of the listed criteria, like a conviction for a crime of violence, the applicant was not hired.
In 2008, after an applicant who was denied a position filed a charge of discrimination, the EEOC began an investigation of Freeman’s credit check policy. On September 25, 2008, it notified Freeman it was expanding this investigation to the criminal background check policy. On March 27, 2009, the EEOC issued a letter of determination finding Freeman’s use of credit and criminal checks violated Title VII.
After conciliation failed, the EEOC filed suit under Sections 706 and 707 of Title VII.
The case proceeded to discovery. The EEOC produced a report by Kevin Murphy, an industrial/organizational psychologist, and one by Beth Huebner, an associate professor of criminology, which
The district court denied the EEOC’s motion for leave to file a sur-reply and granted Freeman’s motion to exclude Murphy’s testimony on the basis thаt it was “rife with analytical errors” and “completely unreliable” under Federal Rule of Evidence 702. The court granted Freeman’s motion for summary judgment. The EEOC timely appealed.
II.
Federal Rule of Evidence 702 governs the admissibility of expert evidence. Expert testimony under Rule 702 is admissible if it “rests on a reliable foundation and is relevant.” Westberry v. Gis-laved Gummi AB,
We review a district court’s decision to admit or to exclude expert evidence for an abuse of discretion. See Westberry,
A.
The district court identified an alarming number of errors and analytical fallacies in Murphy’s reports, making it impossible to rely on any of his conclusions. Freeman provided the EEOC with complete background check logs for hundreds, if not thousands, of applicants who Murphy did not include in his database of fewer than 2,014 background checks conducted largely before October 14, 2008. J.A. 1061. Only 19 post-October 14, 2008 applicants were included in Murphy’s data
Most troubling, the district court found a “mind-boggling” number of errors and unexplained discrepancies in Murphy’s database. For example, looking at a subset of 41 individuals for whom the EEOC is seeking back pay, 29 had at least one error or omission. Seven were missing from the database altogether. Seven were listed in the database without a race code, “one was incorrectly coded as passing the criminal background chеck, two were incorrectly coded as failing the criminal background check, one ha[d] an incorrect race code, five ha[d] incorrect gender codes, nine [we]re listed twice and double-counted in Murphy’s results, and three who failed the credit check [we]re not coded with a credit check result.” J.A. 1064. The' EEOC claims these errors were present in the original data, a contention dispelled by comparing the information from the discovery materials to Murphy’s database. It was in fact Murphy who introduced these errors into his own analysis.
The EEOC also contends that Murphy fixed any errors in his analysis in subsequently-filed, supplemental reports. The district court examined a third report by Murphy
The sheer number of mistakes and omissions in Murphy’s analysis renders it “outside the range where experts might reasonably differ.” Kumho,
III.
We affirm the district court’s grant of summary judgment
AFFIRMED.
Notes
.Freeman required a form authorizing a background search to be completed with each job application, which, according to a company handbook, Freeman thought would “deter individuals with negative information from applying.” However, the checks were not conducted until after a conditional offer of employment had been made. It appears most criteria, as well as making false statements on the job application, led to automatic disqualification. But, Freeman usually gave applicants a reasonable amount of time to resolve outstanding arrest warrants before rescinding an offer.
. The Office of Personnel Management intervened in the case to protect the confidentiality of information related to federal government background investigations, which Freeman sought.
. The EEOC’s complaint originally alleged the checks also had a disparate impact on Hispanics. After its expert found no statistically significant effect on Hispanic applicants, the parties jointly dismissed the EEOC’s claim that the criminal checks discriminated against this class.
. These factors may include “whether the reasoning or methodology underlying the expert's opinion has been or could be tested; whether the reasoning or methodology has been subject to peer review and publication; the known or potential rate of error; and the level of acceptance of the reasoning or methodology by the relevant professional community.” Westberry,
. Although Murphy contends that any errors in the data were in the discovery materials from Freeman, we do not discern any clear 'error by the district court in making this factual finding.
. The EEOC proffered a fourth report by Murphy аt the summary judgment hearing, but did not attach it to the agency's earlier motion to file a sur-reply. The district court also found the EEOC never properly offered Huebner's supplemental report. The court declined to allow the EEOC to file a surreply, and we therefore find that neither Murphy’s fourth report nor Huebner’s supplemental report are part of the record.
. The district court also held that Murphy’s third and fourth reports were not proper supplements under Federal Rule of Civil Procedure 26(e), but werе "poorly disguised attempts to counter Defendant's arguments with new expert analyses.” We agree that EEOC cannot use Rule 26(e) as a “loophole ... [to] revise [its] disclosures in light of [Freeman’s] challenges to the analysis and conclusions therein.” Luke v. Family Care & Urgent Med. Clinics,
. We emphasize that by our disposition we express no opinion on the merits of the EEOC’s claims.
Concurrence Opinion
concurring:
Although I concur in Judge Gregory’s opinion, I write separately to address my concern with the EEOC’s disappointing litigation conduct. The Commission’s work of serving “the public interest” is jeoрardized by the kind of missteps that occurred here. Gen. Tel. Co. of the Nw. v. EEOC;
I.
As in other cases, the EEOC proffered expert testimony to establish the alleged disparate impact of Freeman’s background check policies. Yet the expert testimony here was fatally flawed in multiple respects.
A.
The district court used harsh words to describe the work of the EEOC’s “expert,” Kevin R. Murphy. The court found that Murphy’s reports contained a “plethora” of “analytical fallacies,” reflected “cherry-picked” data’ produced “a meaningless, skewed statistic,” and included a “mind-boggling number of errors.” EEOC v. Freeman,
The majority opinion rightly agrees with the district court’s view, as Murphy’s work simply did not meet the standards for expert testimony that Federal Rule of Evidence 702 provides. But this was not a close question, and three problems merit special recognition.
First, courts often caution experts against drawing broad conclusions from incomplete data. In Lilly v. Harris-Teeter Supermarket,
Yet as the majority notes, Murphy made the very mistake identified in Lilly: he omitted important information from relevant periods and locations. The EEOC challenged credit check policies beginning in late March 2007 and ending in early August 2011; its criminal-background-check claims spanned November 30, 2007 to the present. For reasons unknown, Murphy’s data included barely any information on applicants after mid-October 2008 — ignoring at least two-and-a-half years of relevant and available data for each claim. By arbitrarily putting aside those' years, Murphy ignored 300 credit checks and 1,500 criminal background checks. Indeed, Murphy even ignored applicant data on persons that the EEOC identified as purported victims. Worse still, Murphy ignored relevant criminal background check data from 21 of Freeman’s 39 different locations.
Neither Murphy nor the agency explained these 'omissions. Although the-EEOC speculates that Freeman produced incоmplete data, the record says differently. Among other things, Freeman produced applicant logs, datasheets, and background check forms that Murphy could have used to compile relevant information. Thus, as the majority indicates, the district court’s finding that Freeman presented more than sufficient data is far from clearly erroneous. For his part, Murphy insisted that there was no need to look at more of the available information regardless of relevance. Yet he never explained why his model incorporated enough observations to ensure a valid statistical result.
Second, courts have consistently excluded expert testimony that “cherry-picks” relevant data. See, e.g., Bricklayers & Trowel Trades Int’l Pension Fund v. Credit Suisse Secs. (USA) L.L.C,
Murphy undeniably “cherry-picked.” The very few pieces of post-October-2008 data that Murphy included consisted of 19 applicants. Of those 19, one was a double-counted applicant, one was a “fail” miscod-ed as a “pass,” and the remaining were all “fails” under one or the other (or both) checks. This 100% failure rate among the 19 post-October-2008 applicants wildly varies from the 3.5% failure rate for criminal checks and 9.9% failure rate for credit checks reflected in the rest of the data. See J.A. 326 (noting that' “the likelihood of failing either [check] is low”). Thus, not only was Murphy capriciously selective in his use of post-October-2008 data, but the high number of “fails” among his few selections suggests that he fully intended to skew the results. The district court certainly thought so, terming Murphy’s work “an egregious example of scientific dishonesty.” Freeman,
Finally, Murphy’s analysis contained many obvious errors and mistakes, and these “factual deficiencies” further evidence his “faulty methods and lack of investigation.” Brown v. Burlington N. Santa Fe Ry. Co.,
In sum, Murphy’s work was riddled with fundamental errors, mistakes, and misrepresentations. I certainly agree with the majority’s determination that the district court appropriately excluded Murphy’s evidence.
B.
These problems would be troubling enough standing alone, but they are even more disquieting in the context of what appears to be a pattern of suspect work from Murphy.
EEOC v. Kaplan Higher Education Corp.,
Murphy’s flawed approach is not just a recent problem. Over a decade ago, in Cooper v. Southern Co., Murphy drew different but no less severe criticism. See
Other recent cases provide additional examples of Murphy’s lax attitude towards scientific rigor. In Boelk v. AT & T Teleholdings, Inc., No. 12-cv-40-bbc,
II.
Despite Murphy’s record of slipshod work, faulty analysis, and statistical sleight of hand, the EEOC continues on appeal to defend his testimony. Conceding that Murphy’s report was not an “A+ report,” the EEOC nevertheless says that it meets some indеterminate threshold of reliability. In doing so, however, the Commission advances positions that are not grounded in law. Most troubling is its view that problems in an expert’s data are an inappropriate reason to exclude that expert.
Evidence is admissible only if “it rests on a reliable foundation.” Daubert v. Mer-
The EEOC, however, ignores this threshold analysis by contending that the issue of the reliability of an expert’s data is always a question of fact for the jury, except perhaps in some theoretical, rare case. See, e.g., Reply Br. 15 (“[PJurported flaws in Murphy’s analyses concerned data ... and therefore concerned weight/credibility issues for trial, not admissibility.”). The agency’s contention ignores Daubert’s instruction that the district court must act as a gatekeeper. Moreover, no court has accepted the agency’s argument. Rather, courts widely agree that “trial judges may evaluate the data offered to support an expert’s bottom-line opinions to determine if that data provides adequate support to mark the expert’s testimony as reliable.” Milward v. Acuity Specialty Prods. Grp., Inc.,
The EEOC wields significant power,some of which stems from the agency’s broad discretion to investigate, conciliate, and enforce, and some of which derives from public actions that exert influence outside the courtroom. The Commission’s actions can be also expected to have broader consequences thаn those of an ordinary litigant given the “vast disparity of resources between the government and private litigants.” EEOC v. Great Steaks, Inc.,
In deciding when to act, the Commission must balance sometimes-competing responsibilities. On the one hand, the agency must serve the employee’s interest by preventing an employer from “engaging in any unlawful employment practice” finder Title VII. 42 U.S.C. § 2000e-5(a). On the other hand, “the EEOC owes duties to employers as well: a duty reasonably to investigate charges, a duty to conciliate in good faith, and a duty to сease enforcement attempts after learning that an action lacks merit.” EEOC v. Agro Distrib., LLC,
The EEOC must be constantly vigilant that it does not abuse the power conferred upon it by Congress, as its “significant resources, authority, and discretion” will affect all “those outside parties they investigate or sue.” EEOC v. Propak Logistics, Inc.,
. Experts may use appropriate sampling methods to draw conclusions. But determining an appropriate sample size cаn be a "tricky” question in statistics, Am. Honda Motor Co., Inc. v. Allen,
. In Kaplan, "the EEOC sued the defendants for using the same type of background check that the EEOC itself uses.”
. In the present case, Murphy submitted additional reports right up to the day of the summary judgment hearing. As the majority notes, the district court correctly saw these last-minute changes for what they were: "poorly disguised attempts to counter [Free-manj’s arguments with new expert analyses.” Freeman,