UNITED STATES, Appellee, v. Terrance MOON, Defendant, Appellant.
No. 13-2352.
United States Court of Appeals, First Circuit.
May 16, 2016.
102 LIPEZ, Circuit Judge, joined by TORRUELLA and THOMPSON, Circuit Judges, Statement Re Denial of En Banc Review.
Derege B. Demissie, Demissie & Church, Cambridge, MA, for Appellant.
Terrance Moon, Salters, SC, pro se.
Before HOWARD, Chief Judge, SOUTER, Associate Justice,* TORRUELLA, SELYA, LYNCH, LIPEZ, THOMPSON, KAYATTA and BARRON, Circuit Judges.
ORDER OF COURT
The petition for rehearing having been denied by the panel of judges who decided the case, and the petition for rehearing en banc having been submitted to the active judges of this court and a majority of the judges not having voted that the case be heard en banc, it is ordered that the petition for rehearing and the petition for rehearing en banc be denied.
TORRUELLA and THOMPSON, Circuit Judges, dissenting from denial of en banc rehearing.
LIPEZ, Circuit Judge, joined by TORRUELLA and THOMPSON, Circuit Judges, Statement Re Denial of En Banc Review.
I am disappointed that a majority of the active judges have rejected the opportunity presented by this case to reconsider en banc our aberrant and misguided law on the admission of opinion testimony by police officers. In my concurrence four years ago in United States v. Valdivia, 680 F.3d 33, 61 (1st Cir.2012), I pointed out that our approach has “created in some of our precedents an unwarranted police exception from the requirements applicable to expert testimony.” Id. at 61. That approach not only seriously misconstrues
Pedro LOPEZ, individually and on behalf of a class of individuals similarly situated; Abel Cano, individually and on behalf of a class of individuals similarly situated; Kevin Sledge, individually and on behalf of a class of individuals similarly situated; Charles De Jesús, individually and on behalf of a class of individuals similarly situated; Richard Brooks, individually and on behalf of a class of individuals similarly situated; Massachusetts Hispanic Law Enforcement Association, individually and on behalf of a class of individuals similarly situated; Robert Alvarez, individually and on behalf of a class of individuals similarly situated; Spencer Tatum, individually and on behalf of a class of individuals similarly situated; Shumeand Benfold, individually and on behalf of a class of individuals similarly situated; Angela Williams-Mitchell, individually and on behalf of a class of individuals similarly situated; Gwendolyn Brown, individually and on behalf of a class of individuals similarly situated; Lynette Praileau, individually and on behalf of a class of individuals similarly situated; Tyrone Smith, individually and on behalf of a class of individuals similarly situated; Eddy Chrispin, individually and on behalf of a class of individuals similarly situated; David E. Melvin, individually and on behalf of a class of individuals similarly situated; Steven Morgan, individually and on behalf of a class of individuals similarly situated; William E. Iraolo, individually and on behalf of a class of individuals similarly situated; José Lozano, individually and on behalf of a class of individuals similarly situated; Courtney A. Powell, individually and on behalf of a class of individuals similarly situated; James L. Brown, individually and on behalf of a class of individuals similarly situated; George Cardoza, individually and on behalf of a class of individuals similarly situated; Larry Ellison, individually and on behalf of a class of individuals similarly situated; David Singletary, individually and on behalf of a class of individuals similarly situated; Charisse Brittle Powell, individually and on behalf of a class of individuals similarly situated; Cathenia D. Cooper-Paterson, individually and on behalf of a class of individuals similarly situated; Molwyn Shaw, individually and on behalf of a class of individuals similarly situated; Lamont Anderson, individually and on behalf of a class of individuals similarly situated; Gloria Kinkead, individually and on behalf of a class of individuals similarly situated; Kenneth Gaines, individually and on behalf of a class of individuals similarly situated; Murphy Gregory, individually and on behalf of a class of individuals similarly situated; Julian Turner, individually and on behalf of a class of individuals similarly situated; Neva Grice, individually and on behalf of a class of individuals similarly situated; Delores E. Facey, individually and on behalf of a class of individuals similarly situated; Lisa Venus, individually and on behalf of a class of individuals similarly situated; Rodney O. Best, individually and on behalf of a class of individuals similarly situated; Karen Vandyke, individually and on behalf of a class of individuals similarly situated; Robert C. Young, individually and on behalf of a class of individuals similarly situated; Royline Lamb, individually and on behalf of a class of individuals similarly situated; Lynn Davis, individually and on behalf of a class of individuals similarly situated; James A. Jackson, individually and on behalf of a class of individuals similarly situated; Louis Rosario, Jr., individually and on behalf of a class of individuals similarly situated; Obed Almeyda, individually and on behalf of a class of individuals similarly situated; Devon Williams, individually and on behalf of a class of individuals similarly situated; Julio M. Toledo, individually and on behalf of a class of individuals similarly situated, Plaintiffs, Appellants, v. Marisol Nobrega, individually and on behalf of a class of individuals similarly situated, Plaintiff, v. CITY OF LAWRENCE, MASSACHUSETTS; City of Methuen, Massachusetts; John Michael Sullivan, in his capacity as Mayor of the City of Lawrence, Massachusetts; William Manzi, III, in his capacity as Mayor of the City of Methuen, Massachusetts; City of Lowell, Massachusetts; City of Worcester, Massachusetts; Appointing Authority for the City of Lowell, Massachusetts; Michael O‘Brien, in his capacity as City Manager of the City of Worcester, Massachusetts; City of Boston, Massachusetts; City of Springfield, Massachusetts; Domenic J. Sarno, Jr., in his capacity as Mayor of the City of Springfield, Massachusetts; Massachusetts Bay Transportation Authority; Daniel Grabauskas, in his capacity as General Manager, Massachusetts Bay Transportation Authority; Board of Trustees of the Massachusetts Bay Transportation Authority; William F. Martin, in his capacity as Mayor of the City of Lowell, Massachusetts; Konstantina B. Lukes, in her capacity as Mayor of the City of Worcester, Massachusetts; Commonwealth of Massachusetts; Paul Dietl, in his capacity as Personnel Administrator for the Commonwealth of Massachusetts, Defendants, Appellees,
No. 14-1952.
United States Court of Appeals, First Circuit.
May 18, 2016.
Bonnie I. Robin-Vergeer, Attorney, Department of Justice, Civil Rights Division, Appellate Section, with whom Sharon M. McGowan, Attorney, Civil Rights Division, Vanita Gupta, Acting Assistant Attorney General, P. David López, General Counsel, and Carolyn L. Wheeler, Acting Associate General Counsel, Appellate Services, Equal Employment Opportunity Commission, were on brief for amicus the United States of America.
Kay H. Hodge, with whom John M. Simon, Geoffrey R. Bok, Stoneman, Chandler & Miller LLP, Susan M. Weise, Attorney, City of Boston Law Department, and Lisa Skehill Maki, Attorney, City of Boston Law Department, were on brief, for appellee City of Boston, Massachusetts.
James F. Kavanaugh, Jr., with whom Christopher K. Sweeney, and Conn Kavanaugh Rosenthal Peisch & Ford, LLP, were on brief, for appellees Massachusetts Bay Transportation Authority, Daniel Grabauskas, and the Board of Trustees of the Massachusetts Bay Transportation Authority.
Rachel M. Brown, Assistant City Solicitor, City of Lowell Law Department, with whom Christine Patricia O‘Connor, City Solicitor, City of Lowell Law Department, was on brief for appellees City of Lowell, Massachusetts, and Appointing Authority for the City of Lowell, Massachusetts.
Tim D. Norris, with whom Joshua R. Coleman, and Collins, Loughran & Peloquin, P.C., were on brief, for appellees City of Worcester, Massachusetts, Michael O‘Brien, City Manager of Worcester, and Konstantina B. Lukes, Mayor of the City of Worcester.
Anthony I. Wilson, Associate City Solicitor, City of Springfield Law Department, with whom Edward M. Pikula, City Solicitor, and John T. Liebel, Associate City Solicitor, were on brief, for appellees City of Springfield, Massachusetts, and Mayor Domenic J. Sarno, Jr.
Raquel D. Ruano, Attorney, Office of the City Attorney, City of Lawrence, Massachusetts, and Charles D. Boddy, Jr., Attorney, Office of the City Attorney, City of Lawrence, Massachusetts, on brief for appellees City of Lawrence, Massachusetts, and Mayor John Michael Sullivan.
Kerry Regan Jenness, Attorney, Office of the City Solicitor, City of Methuen, on brief for appellees City of Methuen, Massachusetts, and Mayor William M. Manzi, III.
Michael L. Foreman, Civil Rights Appellate Clinic, Dickinson School of Law, Pennsylvania State University, on amicus brief of National Urban League and the National Association for the Advancement of Colored People.
Gary Klein, Kevin Costello, Corinne Reed, Klein Kavanagh Costello, LLP, Mark S. Brodin, Professor, Boston College Law School, and Ray McClain, Director, Employment Discrimination Project, Lawyers’ Committee for Civil Rights Under Law, on amicus brief of Massachusetts Association of Minority Law Enforcement Officers, New England Area Conference of the National Association for the Advancement of Colored People, Urban League of Eastern Massachusetts, and Professor Mark S. Brodin.
Christopher L. Brown, Christopher J. Petrini, and Petrini & Associates, P.C., on amicus brief of International Municipal Lawyers Association, Massachusetts Municipal Lawyers Association, Massachusetts Municipal Association, National Public Labor Relations Association, Massachusetts Chiefs of Police Association, Inc., and Fire Chiefs Association of Massachusetts, Inc.
Before TORRUELLA, LYNCH, and KAYATTA, Circuit Judges.
KAYATTA, Circuit Judge.
The percentage of Black and Hispanic applicants selected for promotion using the results of this test nevertheless fell significantly below the percentage of Caucasian applicants selected. Some of those Black and Hispanic applicants who were not selected for promotion sued, claiming that the use of the test resulted in an unjustified “disparate impact” in violation of Title VII notwithstanding the absence of any intent to discriminate on the basis of race.
I. Background
The plaintiffs in this suit (the “Officers“) sought promotion in the police departments operated by the Massachusetts municipalities or state agencies sued in this case. Id. at *7-8. All parties agree that affirmance of the judgment in favor of Boston would result in affirmance of the judgment in favor of the other defendants as well, so we focus our discussion for simplicity‘s sake on the evidence concerning Boston. Because this is an appeal of fact-finding and application of law to fact following a trial on the merits, we describe
A. Development of the Exams Over Time
In 1971, Congress noted that the United States Commission on Civil Rights (“USCCR“) found racial discrimination in municipal employment “more pervasive than in the private sector.” H.R.Rep. No. 92-238, at 17 (1971). According to the USCCR, nepotism and political patronage helped perpetuate pre-existing racial hierarchies. U.S. Comm‘n on Civil Rights, For All the People, By All the People: A Report on Equal Opportunity in State and Local Government Employment, 63-65, 119 (1969), reprinted in 118 Cong. Rec. 1817 (1972). Police and fire departments served as particularly extreme examples of this practice. See, e.g., Wesley MacNeil Oliver, The Neglected History of Criminal Procedure, 1850-1940, 62 Rutgers L.Rev. 447, 473 (2010) (“Officers who delivered payments to their superiors were practically assured of retention and even promotion, regardless of their transgressions.“); Nirej S. Sekhon, Redistributive Policing, 101 J.Crim. L. & Criminology 1171, 1191 (2011) (“Police departments were prime sources of patronage jobs.“).
Boston‘s police department was no exception: As far back as the nineteenth century, a subjective hiring scheme that hinged on an applicant‘s perceived political influence and the hiring officer‘s subjective familiarity with the candidate (or the candidate‘s last name) was seen as the primary culprit behind a corrupt, inept, and racially exclusive police force. See, e.g., George H. McCaffrey, Boston Police Department, 2 J. Am. Inst.Crim. L. & Criminology 672, 672 (1912) (“This system worked very unsatisfactorily, however, because places on the police force were invariably bestowed as a reward for partisan activity.“).
At both the state and local levels, Massachusetts officials eventually gravitated toward competitive exams as a tool to accomplish an important public policy of moving away from nepotism, patronage, and racism in the hiring and promoting of police. Boston Chapter, N.A.A.C.P., Inc. v. Beecher, 504 F.2d 1017, 1022 (1st Cir.1974) (“[C]ivil service tests were instituted to replace the evils of a subjective hiring process .... “); see generally League of Women Voters of Mass., The Merit System in Massachusetts: A Study of Public Personnel Administration in the Commonwealth 3-5 (1961). At the statewide level, this movement resulted in legislation and regulations aimed at ensuring that employees in civil service positions are “recruit[ed], select[ed] and advanc[ed] ... on the basis of their relative ability, knowledge and skills” and “without regard to political affiliation, race, color, age, national origin, sex, marital status, handicap, or religion.”
B. The 2005 and 2008 Exams
Born of these purposes and shaped by decades of Title VII litigation,2 the examinations at issue in this case allowed no room for the subjective grading of applications. The total score of a test-taker who sat for the promotional examination in
The subject matter tested on the 2005 and 2008 examinations can be traced back to a 1991 “validation study” or “job analysis report” performed by the state agency responsible for compiling the exam.3 See
That 1991 report was prepared by the Massachusetts Department of Personnel Administration (“DPA“), the predecessor to HRD. In preparing the report, DPA surveyed police officers in thirty-four jurisdictions nationwide, issuing a questionnaire that sought to ascertain the kinds of “knowledge[ ], skills, abilities and personnel characteristics” that police officers across the country deemed critical to the performance of a police sergeant‘s responsibilities. The report‘s authors distilled the initial results from this survey and their own knowledge regarding professional best practices into a list of critical police supervisory traits. They then distributed this list in a second survey to high-ranking police officers in Massachusetts, who were asked to rank these traits according to how important they felt each was to a Massachusetts police sergeant‘s performance of her duties. DPA further refined the ranking of key skills and traits through focused small-group discussions with police sergeants and conducted a “testability analysis” of which skills could likely be measured through the written examination or the E & E component. In 2000, HRD engaged outside consultants to refresh the findings of the 1991 examination through a process similar to, though less thorough than, DPA‘s approach in 1991.
The written question and answer component of the examination consisted of multiple choice questions that covered many topic areas, including the rules governing custodial interrogation, juvenile issues, community policing, and firearm issues, to name a few.4 The text of individual questions was often closely drawn from the text of materials identified in a reading list provided by the Boston Police Department
For example, one question on the 2008 promotional exam asked applicants to accurately complete the following statement:
According to [a criminal investigations textbook on the reading list], a warrantless search and seizure is acceptable:
A. after stopping a vehicle for a traffic violation and writing a citation.
B. after obtaining the consent of the person, regardless of whether obtained voluntarily or nonvoluntarily.
C. when possible loss or destruction of evidence exists.
D. when a quick search of the trunk of a motor vehicle is desired.
In addition to completing the question and answer component of the examination, applicants listed on the E & E rating sheet their relevant work experience, their degrees and certifications in certain areas, their teaching experience, and any licenses they held.5 Points were assigned based on the listed education and experience. For example, applicants could receive up to fifteen points in recognition of their educational attainment, with an associate‘s degree providing up to three points and a doctorate providing up to twelve.
After collecting and scoring the exams, HRD provided the municipalities with a list of passing test-takers eligible for promotion, ranked in order of their test scores.
Because many officers achieved at least the minimum passing score of seventy and there were relatively few openings for promotion to sergeant, all of those who were promoted scored well above the minimum in both 2005 and 2008. In 2005, 9 of the 224 Black and Hispanic candidates who took the exam were promoted, whereas 57 of the 401 other candidates were promoted. In 2008, 1 of the 213 Black and Hispanic test-takers was promoted, whereas 25 of the 291 other candidates were promoted. The average scores for those who the statisticians called “minority test takers” fell below the average scores for the “non-minority test takers” by 6.4 points in 2005 and 6.6 points in 2008.
II. Analysis
We recently described in another suit against Boston the elements of a disparate impact claim. Jones v. City of Boston, 752 F.3d 38, 46, 54 (1st Cir.2014). In a nutshell, litigation of such a claim in a case challenging hiring or promotion decisions focuses on three questions: Do the plaintiffs show by competent evidence that the employer is utilizing an employment prac
In this case, all parties agree that, using competent statistical analysis, the Officers have proven that Boston‘s use of the challenged exam in 2005 and 2008 did indeed have a marked disparate impact because the selection rates of Black and Hispanic officers for promotion to sergeant were so much lower than the selection rates of the other officers that we can fairly exclude random chance as the explanation for the difference.7
A. Validity
The focus of the trial thus turned to the second question: Did Boston use a “practice [that was] ‘job related for the position in question and consistent with business necessity.‘” Ricci v. DeStefano, 557 U.S. 557, 578 (2009) (quoting
In this case, Boston sought to carry its burden of proving the validity of its exams by demonstrating what the Equal Employment Opportunity Commission (“EEOC“) refers to as “content validity” under the Uniform Guidelines on Employee Selection Procedures (“Guidelines“). See
Much of the evidence at trial and many of the arguments in the briefs on appeal focus on the Guidelines’ technical testing standards. The Officers’ briefs treat the Guidelines as if they were inflexible and binding legal standards that must be rigorously applied in ascertaining whether an employment selection device significantly advances the employer‘s business needs. For two reasons, this is not so.
First, “[b]ecause ‘Congress, in enacting Title VII, did not confer upon the EEOC authority to promulgate rules and
Second, even on their own terms, the Guidelines poorly serve the controlling role assigned to them by the Officers in challenging the district court‘s findings. The Guidelines quite understandably provide no quantitative measure for drawing the line between “representative,”
All that being said, Boston did not shy away from seeking to show that its process for selecting new police sergeants in 2005 and 2008 satisfied the technical requirements of the Guidelines. To make such a showing, the City presented the testimony of Dr. James Outtz. Outtz is an industrial organizational psychologist with twenty years of experience testing and measuring employee selection systems. He has served as a consultant to numerous American municipalities and federal agencies and has assisted in the development of employment selection devices used by many public employers. Outtz has published approximately twenty academic publications in the field of industrial organizational psychology. He has worked for both plaintiffs and defendants in challenges to the validity of exams. In Ricci, for example, Outtz co-authored an amicus brief brought on behalf of industrial psychologists arguing that the New Haven Fire Department promotional examinations for captain and lieutenant were flawed and invalid. See Br. of Industrial-Organizational Psychologists as Amici Curiae at 3, Ricci, 557 U.S. 557 (Nos.07-1428, 08-328), 2009 WL 796281, at *3.
Outtz reviewed the development, application, substance, and results of the exams at issue in this case. He opined that the exams were based on job analyses that validly identified the critical skills used by actual police sergeants and that the tests covered a “representative sample” of the content of the job.
Outtz concluded that “at the end of the day” the combined “package” of the written examination and the E & E as administered tested a “representative sample” of the key supervisory skills identified by the 1991 and 2000 reports and was “minimally valid” or “acceptable” under the Guidelines. Id. He testified that the representativeness of the skills tested by the two components and the linkage of these skills to the validation reports were in line with what was contemplated by the Guidelines’ technical standards for constructing a content-valid selection device. See
This is not to say that Outtz‘s testimony trumpeted a wholehearted endorsement of the scheme used by Boston to identify candidates for promotion. He agreed with the Officers that the validity of the Boston examination could have been improved, perhaps by incorporating a “well-developed assessment center” to evaluate an officer‘s interpersonal skills through observed social interaction, or some kind of device for measuring an applicant‘s oral communication skills. Outtz was clear that his opinion solely concerned the selection device‘s compliance with his profession‘s minimum standards as translated into the EEOC‘s Guidelines.
The Officers challenged Outtz‘s conclusions on cross-examination, arguing that his testimony fell short of the mark in several respects that we will discuss, and presented the contrary opinions of their own expert, Dr. James Wiesen. Altogether, the trial testimony of these competing experts consumed the better part of nine days of the eighteen-day trial.
The district court judge who listened to these experts testify concluded that Outtz was correct: “After consideration of the evidence as a whole, I find and conclude that Dr. Outtz‘s opinion rests on adequate
On appeal, the Officers now ask us to set aside the district court‘s finding that the 2005 and 2008 exams were valid. In considering such a request, we ask whether the district court applied the correct legal standards and whether the record contained sufficient support for its findings. See, e.g., Beecher, 504 F.2d at 1022 (affirming a finding of invalidity as “supported by the record“). Since our decision in Beecher, all circuit courts that have addressed the question have reviewed a district court‘s determination that a selection method was or was not valid for clear error. See M.O.C.H.A. Soc‘y, Inc. v. City of Buffalo, 689 F.3d 263, 275 (2d Cir.2012); Ass‘n of Mex.-Am. Educators v. California, 231 F.3d 572, 584-85 (9th Cir.2000) (en banc) (“The question whether a test has been validated properly is primarily a factual question, which depends on underlying factual determinations regarding the content and reliability of the validation studies that a defendant utilized.“); Melendez v. Ill. Bell Tel. Co., 79 F.3d 661, 669 (7th Cir.1996); Hamer v. City of Atlanta, 872 F.2d 1521, 1526 (11th Cir.1989); Bernard v. Gulf Oil Corp., 890 F.2d 735, 739 (5th Cir.1989).
With this standard in mind, we consider the Officers’ critique of the district court‘s reliance on Outtz‘s opinion in finding the examinations valid. Repeatedly, the Officers suggest that Outtz‘s own characterization of the exams as “minimally valid” should render his opinion legally insufficient to carry the City‘s burden. Implicitly, the Officers ask us to read “minimally valid” as meaning, in effect, “not valid enough.” Read in context, however, Outtz was plainly testifying that he found the exams to be valid, albeit not by much. Indeed, elsewhere in his testimony he made clear that the exams were “valid” and, in his view, complied with the technical requirements of the Guidelines.
Moving more aptly from debating adverbs to discussing the law, the Officers (with the support of the United States as amicus curiae) argue that the district court misconstrued the law in finding Outtz‘s testimony sufficient. Specifically, they say that the district court did not reach its finding of content validity in accord with the Guidelines’ statement that evidence of an exam‘s content validity should “consist of data showing that the content of the selection procedure is representative of important aspects of performance on the job for which the candidates are to be evaluated.”
It is true that the district court observed that “more than half of the KSAs identified as pertinent to the job of sergeant were tested,” and that “this was sufficient to meet the ‘representative sample’ requirement of the Uniform Guidelines.” Lopez, 2014 U.S. Dist. LEXIS 124139 at *54 (quoting
The district court‘s qualitative focus on the importance of the factors that the exam tested was further highlighted by the court‘s agreement with Outtz that the written job knowledge portion of the test was not alone valid “because it could not measure some skills and abilities (as distinguished from knowledge) essential to the position.” Id. at *60. After then agreeing with Outtz that the E & E component of the exams adequately, albeit minimally, filled in this gap, the district court expressly found that the exams “were based on job analyses that considered the important tasks necessary to the successful performance of the job.” Id. at *61. The district court‘s opinion as a whole thus makes clear that the court trained its focus on critical and important knowledge, skills, and abilities called for by the job, and it did not clearly err by finding that a test that measured a large percentage of such critical and important KSAs was a test that was sufficiently “representative of important aspects of performance on the job.”
The Officers and the United States also contend that our 1974 opinion in Beecher, 504 F.2d 1017, mandates our reversal of this conclusion. Their reliance on Beecher fits this case awkwardly because of the burdens we have already detailed. In Beecher, the central question was whether the record supported the district court‘s finding of fact that a hiring exam given to would-be firefighters was not valid. See id. at 1022-23. To affirm, we needed only to find that the record did not compel a contrary finding. Id. at 1022. Here, by contrast, the Officers ask us to find that this record compels a finding contrary to that reached by the district court.
The Officers and the United States nevertheless seem to find much significance in one analogy we drew in Beecher. In assessing an exam for the position of firefighter, we compared knowledge of firefighting terminology to knowledge of baseball vocabulary possessed by a potential recruit for the Boston Red Sox “who could not bat, pitch or catch.” Id. at 1023. Here, in reviewing an exam for the supervisory position of sergeant, the more apt baseball analogy would be the hiring of a coach, who must certainly have an extensive knowledge of the rules that must be followed by those being managed. At trial, former Boston Police Commissioner Edward Davis testified that a “sergeant really has to have a strong basis of knowledge of all the rules and regulations and constitutional protections that are afforded the citizens of the Commonwealth to do the job properly,” because when officers in the field “get confused and don‘t understand something, the first thing they do is call the sergeant.” This “fundamental understanding” of “how things work,” was a “critical component” of a sergeant‘s responsibilities, according to Commissioner Davis. And, the court supportably found, those skillsets were tested by the exam.
The Officers’ reliance on Beecher is further undermined by the different approach taken in that case towards validation of the exam. We stated that for an exam to be valid, the court must be satisfied that “it demonstrably selects people who will perform better the required on-the-job behaviors after they have been hired and trained.” Id. at 1021-22. We observed that “[t]he crucial fit is not between test and job lexicon, but between the test and job performance.” Id. at 1022. This approach resembles what the Guidelines, adopted four years after Beecher, call “criterion-related validity.”
None of the remaining arguments advanced by the Officers seriously support any claim that the exams are not materially better predictors of success than would be achieved by the random selection of
B. Rank-Order Selection
When officials at the BPD received the results of the 2005 and 2008 sergeant promotional examinations from HRD, they selected as many police officers for promotion as there were vacancies currently available, beginning with the highest-scoring name at the top of the list and moving down the list, one at a time, in order of the score each candidate received. The Officers argue that this method of selection—quite independently from the written examination itself—led to a disparate impact and the district court was obligated to conduct a separate analysis of its validity under Title VII. We review the legal sufficiency of the district court‘s holding on this point de novo and its subsidiary fact-finding for clear error. E.E.O.C. v. Steamship Clerks Union, Local 1066, 48 F.3d 594, 603 (1st Cir.1995).
The Officers first argue that the district court failed altogether to wrestle with the consequences of rank-order selection. This is clearly not the case. Citing section
This finding by the district court, to the Officers, is “not enough.” Based on their reading of the Guidelines, something more is required. The Officers argue that the use of the results of an examination that is “minimally valid” insofar as it tests job-related skills may not necessarily be valid if used to select candidates solely according to their scores on that exam.
Two provisions of the Guidelines discuss an employer‘s appropriate use of a rank-ordering selection method. In the section of the Guidelines establishing “General Principles,” the EEOC has advised the following:
The evidence of both the validity and utility of a selection procedure should support the method the user chooses for operational use of the procedure, if that method of use has a greater adverse impact than another method of use. Evidence which may be sufficient to support the use of a selection procedure on a pass/fail (screening) basis may be insufficient to support the use of the same procedure on a ranking basis under these guidelines. Thus, if a user decides to use a selection procedure on a ranking basis, and that method of use has a greater adverse impact than use on an appropriate pass/fail basis (see section 5H below), the user should have sufficient evidence of validity and utility to support the use on a ranking basis.
29 C.F.R. § 1607.5(G) (emphasis supplied).
If a user can show, by a job analysis or otherwise, that a higher score on a content valid selection procedure is likely to result in better job performance, the results may be used to rank persons who score above minimum levels. Where a selection procedure supported solely or primarily by content validity is used to rank job candidates, the selection procedure should measure those aspects of performance which differentiate among levels of job performance.
Id. § 1607.14(C)(9) .
These two statements evidence some inconsistency. Section
Several courts have seemed to approach this issue by requiring more scrutiny of the validation evidence as a whole when rank ordering is used, particularly when the exams in question have led to closely bunched scores. See Johnson v. City of Memphis, 770 F.3d 464, 479-81 (6th Cir.2014), cert. denied, — U.S. —, 136 S.Ct. 81 (2015); Police Officers for Equal Rights v. City of Columbus, 916 F.2d 1092, 1102-03 (6th Cir.1990); Guardians Ass‘n of N.Y.C. Police Dep‘t, Inc. v. Civil Serv. Comm‘n of City of N.Y., 630 F.2d 79, 100-05 (2d Cir.1980).
The district court in this case expressly adopted the approach most favorable to the Officers, citing
Even accepting the district court‘s opinion that added scrutiny was called for because rank ordering was used, whatever added scrutiny one need apply here certainly falls short of the added scrutiny one would apply if rank ordering had been a material contributor to the disparate impact. Although they belatedly offer on appeal, without citation to the record, counsel‘s own calculations that “banding” in lieu of rank order selection would have caused more Black and Hispanic applicants to be “reachable” for selection by subjective “performance” criteria, the Officers made no effort to demonstrate that an increased number of Black and Hispanic applicants likely would have been selected under such an alternative approach. Rank ordering furthers the City‘s interest in eliminating patronage and intentional racism under the guise of subjective selection criteria. Such a goal is itself a reasonable enough business need so as to provide some weight against a challenge that is unaccompanied by any showing that rank order selection itself caused any disparate impact in this case.13
None of this is to suggest that Boston could not have come up with an exam that did a better job of furthering its goal of selecting the best candidates for promotion to the position of sergeant. The Officers argue persuasively that Boston could have made the exam more valid. Indeed, Outtz agreed and so, too, it would appear, does the City, which, counsel tells us, has since 2008 developed a new exam that it now uses.
The point, instead, is that the record contains detailed, professionally buttressed and elaborately explained support for the district court‘s finding “that persons who perform better under the test method are likely to perform better on the job.” Id. at *61. Given that plainly supported finding, it makes little sense to debate in the abstract how much better the exam might have been. Instead, it makes more sense to move to the next step of the inquiry to see if there is any alternative selection test that would have had less adverse impact. If so, then the court will have a meaningful gauge of validity by comparing the two tests. And if the alternative test with less adverse impact has equal or greater validity, it makes no difference how valid the employer‘s actual test is; the employee wins. Ricci, 557 U.S. at 578 (citing
C. The Officers’ Alternatives
So, the pivotal question on appellate review is whether the evidence compelled a finding “that the employer refuse[d] to adopt an available alternative employment practice that has less disparate impact and serves the employer‘s legitimate needs.” Allen v. City of Chicago, 351 F.3d 306, 313 (7th Cir.2003). To carry this burden, plaintiffs must “demonstrate a viable alternative and give the employer an opportunity to adopt it.” Id.
Outtz explained that he thought the Officers would be unlikely to carry this burden due to the very large number of applicants for relatively few open positions in Boston. On the 2008 exam, for example, where the disparate impact was much greater than in 2005, there were only 26 openings for 504 applicants. He explained that his experience is that:
[I]n dealing with adverse impact[,] the ball game is played, for the most part, in terms of selection ratio. If I come to—if an employer comes to me and says, “Look, I‘ve got five job openings and I‘ve got 5,000 people that are applying for those five jobs and I want you to develop a system that reduces adverse impact,” I‘m just going home.
The Officers’ own expert agreed that the selection ratio heavily influenced the menu of available options, offering his opinion that the degree of adverse impact caused by a selection process “depends so much on how many people you appoint.”
The Officers baldly assert that the district court did not find “that Plaintiffs failed to meet their burden of putting forward a specific less discriminatory alternative.” In fact, the district court precisely so found—twice. Lopez, 2014 U.S. Dist. LEXIS 124139, at *78 (holding that the Officers’ showing was “not enough to carry their burden on this issue” and did not “demonstrat[e] by the evidence that there was an alternative employment practice with equal validity and less adverse impact that was available and that BPD refused to adopt“).
The Officers also contend that “[i]t is undisputed that ... adding test components such as an assessment center, structured oral interview, or performance review to an exam process increases the validity of an exam while having less adverse impact on minorities.” Yet the Officers failed to offer any evidence that would have compelled the district court to find that the deployment of any of these supposedly “undisputed” solutions would have led to “a smaller racial disparity in outcomes,” Jones, 752 F.3d at 55, given the selection ratios facing authorities in Boston.
Our own review of the record does disclose testimony convincingly establishing that, as a general matter, incorporation of selection tools such as use of “hurdles,” banding, oral interviews, so-called assessment centers, and open ended “situational judgment” questions generally tend to result in less adverse impact than does a reliance on multiple choice exams. What is missing, though, is any rebuttal to Outtz‘s opinion that the low rates of job openings in the Boston sergeant ranks relative to the number of applicants made it unlikely that any alternative selection device would have materially reduced adverse impact in 2005 and 2008.
Additionally, as the district court noted, Boston‘s prior attempt to employ assessment centers with situational exercises and oral questioning in its 2002 promotional exam resulted in a cost of $1.2 million to develop the exam and the required “transporting, housing, and training a substantial number of police officers from throughout the country who acted as the assessors,” id. at *70, without generating any convincing support that repeating such an approach in 2005 or 2008 would have reduced adverse impact, id. at *73. In concluding that the City was not required to again incur such costs without any demonstration that adverse impact would be materially reduced, the district court acted well within its discretion in making the judgments called for by the applicable law.15
See Watson, 487 U.S. at 998 (opinion of O‘Connor, J.) (“Factors such as the cost or other burdens of proposed alternative selection devices are relevant in determining whether they would be equally as effective as the challenged practice in serving the employer‘s legitimate business goals.“).
Satisfying a plaintiff‘s burden on this point at trial “demands evidence that plaintiffs’ preferred alternative would have improved upon the challenged practice,” Johnson, 770 F.3d at 477, not just that such practices exist in the abstract. Furthermore, securing the reversal of a trial court‘s factual finding that the Officers’ proof on this point was not persuasive required evidence that is so compelling as to render its rejection clear error. The Officers’ scattershot listing of alternatives without any developed rejoinder to Outtz‘s testimony concerning the challenge posed by the selection ratios in 2005 and 2008 fell short of this mark.16
III. Conclusion
Given our finding that the district court applied the correct law and committed no
TORRUELLA, Circuit Judge, concurring in part and dissenting in part.
I agree with my colleagues in the majority only to the extent that the challenged tests did have a disparate impact. There is little doubt in my mind, however, that the majority‘s question, whether “the employer[s] show[ed] that the challenged employment practice creating this disparate result is nevertheless job-related for the position in question and consistent with business necessity,” supra at 111, cannot be answered in the affirmative based on this record.17 To my view, the district court committed clear error in finding that the challenged tests were valid when placed under the legal prism of Title VII,
A review of the record shows that Boston18 did not, contrary to the district court‘s finding and the majority‘s asser-
tion, “show[] that the content of the selection procedure is representative of important aspects of performance on the job for which the candidates are to be evaluated.” Supra at 111 (quoting
The most significant flaws in Boston‘s case for validity should each independently have been fatal to it: Boston failed to demonstrate (1) that the 1991 Validation Report and 2000 job analysis were applicable and reliable19 and (2) that the exams tested “representative” and critical knowledge, skills, and abilities (“KSAs“) necessary to quality for the position of police sergeant.
This first flaw stems from “the way in which the validation study was performed” and its effect on test validity. Beecher, 504 F.2d at 1025. The Validation Report and job analysis were defective. The district court acknowledged the “rule of
The Officers bolstered the presumption that a test more than eight years old is not reliable, and the common sense conclusion that a position changes over time, by pointing to specific evidence that defendants’ police departments changed practices since the Report and analysis were performed: testimony from Commissioner Edward F. Davis that Lowell implemented a community policing model and a 2002 Boston Commissioner‘s memo referring to changes in policing policy and practice. While the district court was entitled to rely on Dr. Outtz‘s testimony as to the unchanging nature of the position of sergeant, it clearly erred in doing so for the proposition it drew from his testimony, that the position of police sergeant in the defendant departments had not changed, as Dr. Outtz based his statement on “[his] experience generally” regarding the position in other municipalities, including those in other states.
The subsequent job analysis completed in 2000, within the time range to be presumed reliable, is unreliable by virtue of the way it was performed. The 2000 job analysis suggests that the eleven subject matter experts (“SMEs“), sergeants and detective sergeants, relied upon by the testing firm to evaluate KSAs and tasks for inclusion in the exam, were to do so individually; the analysis details procedures for reconciling disparate results to determine which items should make the cut. For example, “[f]or a KSA to be included as a[sic] important component of the Police Sergeant position, the KSA had to be rated by nine ... of the eleven ... SMEs” in a certain way across all five categories. Yet the eleven SMEs evaluating 160 KSAs each rated all 160 KSAs’ five attributes—job relatedness, time for learning, length of learning, differential value to performance, and necessity20—in exactly the same way, although there were 72 possible ways to rate each KSA. The same was true of task ratings, wherein each SME was supposed to rate each of 218 tasks’ frequency, importance, necessity, relationship to performance, and dimensions,21 despite the fact that each of 218 tasks could be rated in 1,674 ways. I will not speculate as to how and why this total agreement occurred but only observe that
Beyond these threshold issues, the resultant exams did not test a representative portion of KSAs. See
The 2000 job analysis identified 163 “important tasks” and 155 “important” KSAs. The district court acknowledged that the eighty-point multiple-choice portion of the exams tested primarily the “K” of the KSAs, knowledge, and failed to measure key skills and abilities, and thus would not be independently valid. López, 2014 U.S. Dist. LEXIS 124139, at *60-61. The E & E component that purportedly compensated for the “SA” deficit, edging the exams into the realm of validity, consisted of a single sheet requiring candidates to bubble in responses as to length of work experience in departmental positions by rank, educational background, and teaching ex-
perience. As the majority concedes, this component had a minimal effect on score. Supra at 113.
The conclusion that more than half, López, 2014 U.S. Dist. LEXIS 124139, at *54, or nearly half, supra at 115 n. 11, of applicable KSAs were or could be tested by the exams overestimates the number of KSAs tested by the E & E component. But even if that estimate were correct, relying upon this quantitative measure misses that representativeness is partly qualitative.
It is quite a stretch to conclude that the E & E‘s bubbles incorporated measures of the majority of key skills and abilities. It is even more difficult to conclude from the record that the skills and abilities measured received representative weight. Supra at 113. How, exactly, could this worksheet test, as the testability analysis suggests, “[k]nowledge of the various communities within the Department‘s jurisdiction and the factors which make them unique,” “[s]kill in perceiving and reacting to the needs of others,” or “[k]nowledge of the procedures/techniques when a major disaster occurs,“? And how, if it only affected the ultimate score by five to seven percent at most, supra at 113, could it be said that the KSAs for which the E & E ostensibly tested were adequately represented relative to those KSAs tested on the multiple-choice component?
The exam‘s failure to include particularly significant KSAs also precludes representativeness. See Gillespie v. Wisconsin, 771 F.2d 1035, 1044 (7th Cir.1985) (“To be representative for Title VII purposes, an employment test must neither: (1) focus exclusively on a minor aspect of the posi-
As in Beecher, “[t]here are, in sum, too many problems with the test ... to approve it here.” 504 F.2d at 1026. It cannot be anything but clear error, supra at 114, to find valid exams based on an outdated validation report and a facially flawed job analysis, exams that are not only unrepresentative but also omit critical KSAs for the position of police sergeant. To endorse the means by which these exams were created and the exams themselves here establishes a perilous precedent that all but encourages corner-cutting when it comes to Title VII.
On these grounds, I respectfully dissent.