330 F. Supp. 203 | S.D.N.Y. | 1971
The fairness and validity of competitive examinations, once described by Gilbert and Sullivan as the means of attaining “a Duke’s exalted station,”
The two named plaintiffs, who are respectively Black and Puerto Rican, have brought this purported class action on behalf of themselves and all other persons similarly situated pursuant to federal civil rights laws, 42 U.S.C. §§ 1981
Plaintiffs seek a preliminary injunction under Rule 65, F.R.Civ.P., prohibiting the alleged violations of these laws. They also seek declaratory relief
The Board of Education has not actively opposed the motion for preliminary injunction, and it agrees that plaintiffs have presented triable issues of fact. The Board of Examiners (“Board” herein), however, has vigorously opposed the motion.
An applicant for permanent appointment to a supervisory position in the New York City School System must, in addition to meeting state requirements for the position, obtain a New York City license.
Next the candidate must pass an examination procedure prepared and administered by the Board for the particular type or classification of supervisory post desired, which may take as long as two years to complete. If the candidate successfully completes the testing procedure, he or she is granted a license and placed on a list of those eligible for assignment to the type of supervisory position involved. The appropriate school governing authority — either a central board of education or a community school board under New York City’s present decentralized system — then selects the person it wishes from the eligible list to fill an open position. Since appointments of permanent supervisory personnel in the New York City School System must be made from lists of eligibles who have passed examinations, the Board from time to time announces and conducts examinations for particular supervisory posts (of which there are more than 50 different types) following which the number of persons eligible for appointment are supplemented by promulgation of lists of those who passed the latest examination. If a successful candidate, after being listed as eligible for appointment, is not appointed within four years, he or she is dropped from the list and must again pass the qualifying examinations to be listed as eligible.
Only in the cities of Buffalo and New York does state law provide for examinations in addition to state certification, N.Y.Education Law § 2573(10-a), and only the New York City School District maintains a Board of Examiners and the specific examination and licensing procedure here under attack. The Board has described itself as “a highly select group with broad professional background in education and related fields chosen through the most objective and impartially searching examination given under civil service.” (Ex. 10, Item 10, attached to 5/28/71 aff. of Richard S. Barrett)
Were it not for New York City’s special examination and licensing procedure, plaintiffs Chance and Mercado would
Plaintiff Boston M. Chance has been employed in the New York City public school system for the last 15 years and is acting principal of P.S. 104, an elementary school in the Bronx. Chance, who is of the Black race, possesses all of the basic qualifications of education and experience established by law and by the Board of Education and the Chancellor of the New York City School District for the position of principal of an elementary school. However, he lacks a city license as elementary school principal and therefore is barred at present from securing a permanent position as principal. In September, 1968, Chance took the examination given by the Board for the position of Assistant Principal, Junior High School, but he failed it and thus was not placed on the eligibility list and was not issued a license entitling him to permanent appointment.
Plaintiff Louis Mercado, a Puerto Rican who also holds a New York State license as a principal, has been serving the New York City school system for the last 12 years. He is presently acting principal of P.S. 75, an elementary school in Manhattan, but he is barred from permanent appointment because he does not have a New York City license as an elementary school principal. Mercado is in a somewhat different position from Chance in that he does not allege that he has ever taken the relevant Board of Examiners’ Supervisory Examination. While the present motion was pending — • and while the parties were collecting statistical information pursuant to our order — the Board conducted their November, 1970 series of examinations for elementary school principal. Mercado withdrew from this examination and refused to take it on the grounds that the “Board of Examiners is not the appropriate agency for qualifying school personnel” and “the examination is not relevant *
Both Chance and Mercado were selected for their present acting principalships by their respective community school boards, in accordance with New York City’s decentralized system. See generally, Council of Supervisory Associations, etc. v. Board of Education, 23 N.Y.2d 458, 297 N.Y.S.2d 547, 245 N.E.2d 204 (1969). In some instances such local school boards found, after interviewing licensed principals listed as eligible by the Board, that persons not so licensed were more qualified to serve as principals than those interviewed and that they performed their duties in a superior manner. (See Aff. of Peter J. Straus, 9/22/70).
There are approximately 1,000 licensed Principals of New York public schools of varying levels (e. g., elementary day, junior high school, high school, etc.), of whom some act as the heads of schools and others function in administrative positions. Of the 1,000 only 11 (or approximately 1%) are Black and only 1 is Puerto Rican. Furthermore, of the 750 licensed Principals of New York elementary schools only 5 (or less than 1 %) are Black, and none is Puerto Rican. Of the 180 high school administrative assistants, none is Black or Puerto Rican.
Of the 1,610 licensed Assistant Principals of New York City junior high and elementary schools, only 7% are Black and only .2% are Puerto Rican. When the list for the position of Principal, elementary school, was originally promulgated, only 6 out of the 340 persons (or about 1.8%) were Black and none was Puerto Rican; and when the list for Principal, high school, was promulgated, none of the 22 licensed people was Black or Puerto Rican. The promulgated list of licensed Assistant Principals for jun
Plaintiffs contend that the written and oral examinations of the Board are the major factor accounting for this extremely low percentage of Black and Puerto Rican supervisors in a school system where 55% of the students are Black or Puerto Rican. Plaintiffs summarize their basic argument as follows:
“[Tjhese tests place a premium on familiarity with organizational peculiarities of the New York City school system which, while having little to do with educational needs, are largely gained through coaching and assistance from present, predominately white, supervisory personnel.
if if if if if if
“The testing procedures do not indicate a candidate’s ability to do the job being tested for. There is no evidence that they measure merit or fitness, they have never been validated, and they are unreliable psychological instruments.” Amended Complaint |fjf 22, 23.
Rather than risk the endless delay that would be encountered while the parties obtained this essential evidence through pretrial discovery procedures, we directed the parties, in view of the importance of the issue, to use their best efforts to agree on a procedure whereby the Board of Examiners and the Board of Education would compile the necessary racial statistics. All parties cooperated fully and at considerable effort in working out almost all of the details of the procedure to be followed. Such differences as existed were resolved by court order.
All parties and amici have submitted briefs as to the relevance of the statistics thus adduced and the inferences that may be drawn from them. The parties also submitted affidavits by statistical experts. A hearing was held, at which each side’s expert testified and was subject to cross-examination; and we heard more oral argument on the statistical data. After declining the opportunity to examine and cross-examine any other witnesses, including those presented by affidavit, both sides rested on the record thus adduced.
Upon the evidence thus presented we find that the examinations and testing procedures prepared and administered by the Board for the purpose of determining which candidates will be licensed as supervisory personnel have the effect of discriminating against Black and Puerto Rican candidates.
EXAMINATION CAUCASIAN BLACK PUERTO RICAN BLACK AND PROBABILITY
Total % Pass Total % Pass Total % Pass Total % Pass THAN —_
Assistant Principal,
Day Elementary Schools, 1965 Examination
(PF-03) 1171 61,3% 278 45.68% 7 28.57% 285 45.26% 1/1 million
Assistant Principal,
Junior High Schools, 1968 Examination
(PF-43) 1319 48.82% 236 26.27% 14 14.29% 250 25.60% l/l million
Thus white candidates passed the examination for Assistant Principal of Junior High School at almost double the rate of Black and Puerto Rican candidates, and passed the examination for Assistant Principal of Day Elementary School at a rate one-third greater than Black and Puerto Rican candidates. The gross disparity in passing rates on these two examinations is of particular significance not only because they were taken by far more candidates than those taking any other examinations conducted in at least the last seven years, resulting in licensing of the largest number of supervisors, but also because the assistant principalship has traditionally been the route to and prerequisite for the most important supervisory position, Principal. To the extent that Black and Puerto Ricans are screened out by the examination for Assistant Principal they are not only prevented from becoming Assistant Principals but are kept out of the pool of eligibles for future examinations for the position of Principal as well. The fact that the process involves a series of examinations and that to reach the top one must pass several examinations at different times in his or her career serves to magnify the statistical differences between the white and nonwhite pass-fail rates. For instance, if we take a group of 100 Black and Puerto Rican candidates, on the one hand, and 1,000 white candidates, on the other, and assume a passing rate of 25% for the former and of 50% for the latter on
Black & Puerto Rican White
25% X 100= 25 50% X 1000 =500
Licensed Assistant Principals
The group of 525 licensed assistant principals would then form the pool of eligibles for the related principal’s examination. Assuming the same relative pass rates, we have the following results:
Black & Puerto Rican White
25% X 25 = 6.25 50% x 500 =250
Principals
Thus the true resulting difference between the Black and Puerto Rican versus the white pass rates would be even more substantial: only 6.25% of the Blacks and Puerto Ricans would pass the two successive examinations as against 25% of the whites.
When we look at all 50 examinations which were the subject of the Survey, we find that only 34 were taken by at least one member of both the white and Black-Puerto Rican racial groups. One of these examinations (Assistant Administrative Director, given Dec. 1967, PF-17) was passed by everyone taking it. Another (Director, Bureau for Children with Retarded Mental Development, given Jan. 1968, PF-18) was not completed successfully by anyone taking it.
Plaintiffs offered the testimony of Dr. Jacob Cohen, an expert in the field of statistics, in support of the validity and significance of the Survey results. The Board, in turn, adduced the testimony of Dr. Nathan Jaspen, an expert in the same field, in opposition. Both witnesses possess outstanding qualifications. After reviewing their testimony and appraising them as witnesses, we are more persuaded by the testimony of Dr. Cohen with respect to certain crucial matters affecting the significance of the figures for present purposes.
Turning first to the gross aggregate pass-fail statistics, which reveal that the overall pass rate of white candidates (44.3%) was almost half again as high as the non-white rate (31.4%), Dr. Cohen testified that on the basis of such a large sample (5,910 out of 6,201 candidates), the test results were especially valuable and formed a sound basis for drawing valid statistical conclusions as to the difference in passing rates between
In an effort to rebut Dr. Cohen’s analysis the Board, after first dismissing the aggregate figures as insignificant on the ground that the “examinations are discrete competitions related to widely-varied, particular supervisory examinations” (Second Supp. Mem. 5/7/71, p. 7), offered Dr. Jaspen’s testimony that he did not compute probabilities on the basis of the gross aggregate figures because of the possibility of “overlap,” i. e., the taking of more than one test by the same persons. We reject this excuse for several reasons. In the first place, the Board, although it had the data as to any overlap within its control and was afforded the opportunity to adduce any relevant evidence, chose not to do so. Under such circumstances we cannot assume any significant overlap. Secondly, if a “random” overlap existed (i. e., one where the number of persons repeating an examination because of a previous failure approximates those taking a second, more advanced examination after passing the first), it would not substantially affect the probabilities. Dr. Jaspen conceded, for instance, that a random overlap of as much as 50% would not substantially affect the significance of the observed difference between the white and non-white pass rates. Lastly, Dr. Cohen (plaintiffs’ expert) testified that because of the large Chi-Square value an overlap would not significantly affect the probability data. In other words, the observed difference between the aggregate pass-fail rates of whites and non-whites was too great to be a mere matter of chance, unrelated to race (the chance of such an occurrence being less than one in a billion) so that even if the probability were reduced somewhat because of overlap, the figure would still be significant, e. g., one in one million, or one in 500,000, and if the overlap were no greater than 15%, the statistical results would not be affected at all.
Turning to the 50 examinations forming the raw material of the Survey, some were taken by very few people. For instance, 41 of the 50 examinations were taken by only 83 (or 10.1%) of the total number of Black and Puerto Rican candidates. Because of the smallness of the sample in each instance the resulting figures of each such examination, when analyzed individually, cannot be accorded much weight or signficance for our purposes. Although statisticians can analyze very small samples through use of a method called the Fisher Exact Probability Test in conjunction with the Chi-Square (Yates-corrected) Test, in our opinion such a sample is less reliable than analyses based on larger samples, even after making allowances for greater margins of error in use of the small sample. Use of the small sample involves more extrapolation and theory superimposed on less fact. We prefer the greater fact content found in the larger sample. For the reasons explained by Dr. Cohen — principally the risk of spurious differences based on insufficient evidence — we do not believe that meaningful conclusions as to differences can be drawn from the meager data derived from any one of these 41 examinations.
Turning to the 9 examinations taken by 10 or more Black and Puerto Rican candidates,
Finally we are impressed with the revealing statistics comparing the percentage of Black and Puerto Rican Principals to white Principals in the five largest school systems in the country:
Total No. of % Black and
City Principals % Black % Puerto Rican Puerto Rican
Detroit 281 16.7% 16.7%
Philadelphia Los Angeles 267 1,012 16.7% 8.0% 1.7% 16.7% 9.7%
Chicago 479 6.9% 6.9%
NEW YORK 862 1.3% 0.1% 1.4%
Thus New York City has by far the lowest percentage of minority representation. The next lowest city, Chicago, has almost 5 times the percentage of minority principals found in New York City, and as the following table shows there is a similar imbalance of minority Assistant Principals:
City Total No. of Asst. Principals % Black % Puerto Rican % Black and Puerto Rican
Detroit 360 24.7% 0.2% 24.9%
Philadelphia 225 37.0% 37.0%
Los Angeles
Chicago 714 32.5% 32.5%
NEW YORK 1,610 7.0% 0.2% 7.2%
Plaintiffs also argue that discrimination may be inferred from the fact that the percentage of Black and Puerto Rican Principals and Assistant Principals in New York City schools (1.4% and 7.2%, respectively) is far below the percentage of the total student body who are Black and Puerto Rican (55.8%) and when compared with similar figures for the five largest school systems in the country (New York, Los Angeles, Chicago, Detroit, and Philadelphia) constitutes not only the lowest minority representation in the supervisory ranks, but also the lowest ratio of such minority group supervisors to minority group students. We reject this contention. Supervisors are drawn from the pool of qualified teachers, most of whom attended elementary and high school long ago, and not from present-day students. Undoubtedly the low number of minority teachers eligible to take the supervisory examinations prescribed by the Board
For the same reasons we are unimpressed with plaintiffs’ comparisons between (1) the percentage of Black and Puerto Rican members of the general population in New York City, and (2) the percentage of Black and Puerto Rican Principals and Assistant Principals found in the City’s total school supervisory personnel. Statistical comparisons to the general racial population of the community may be relevant in determining whether there is discrimination in job opportunities that are supposed to be open to the general public, see, e. g., Arrington v. Mass. Bay Transp. Auth., 306 F.Supp. 1355 (D.Mass.1969) (bus drivers); Penn v. Stumpf, 308 F.Supp. 1238 (N.D.Cal.1970) (policemen), in the selection of teachers from a pool of those already qualified and eligible for appointment, e. g., Porcelli v. Titus, 302 F.Supp. 726 (D.N.J.1969), affd., 431 F.2d 1254 (3d Cir. 1970) (per curiam), or in qualification of voters or jurors, Smith v. Texas, 311 U.S. 128, 61 S.Ct. 164, 85 L.Ed. 84 (1940); Jones v. Georgia, 389 U.S. 24, 88 S.Ct. 4, 19 L.Ed.2d 25 (1967); Coleman v. Alabama, 389 U.S. 22, 88 S.Ct. 2, 19 L.Ed.2d 22 (1967); Swain v. Alabama, 380 U.S. 202, 208-209, 85 S.Ct. 824, 13 L.Ed.2d 759 (1967). But we are here dealing with candidates who must meet preliminary eligibility requirements as to education and experience that are not possessed by most of the general population. Where the education of our children is at stake, such insistence upon the highest possible quality in our teachers is a salutary and lawful objective, provided it does not result in racial discrimination between candidates who are otherwise eligible, which is the case here.
Notwithstanding the introduction of some evidence thus found irrelevant, the evidence establishes to our satisfaction that the examinations prepared and administered by the Board for the licensing of supervisory personnel in New York City schools do have the de facto effect of discriminating significantly and substantially against qualified Black and Puerto Rican applicants. However, the existence of such discrimination, standing alone, would not necessarily entitle plaintiffs to relief. The Constitution does not require that minority group candidates be licensed as supervisors in the same proportion as white candidates. The goal of the examination procedures should be to provide the best qualified supervisors, regardless of their race, and if the- examinations appear reasonably constructive to measure knowledge, skills and abilities essential to a particular position, they should not be nullified because of a de facto discriminatory impact. We accordingly pass on to the question of whether the examinations under attack can be validated as relevant to the requirements of the positions for which they are given, i. e., whether they are “job-related.”
The parties disagree as to which side bears the burden of proving that the examinations are job-related. Plaintiffs contend that once a discriminatory impact is shown the burden is on the Board to show a compelling necessity or justification for tests having an unintended
The Board contends that in a Constitutional framework plaintiffs must show that there is no rational relationship between the examinations and the requirements of the supervisory positions for which they are given, relying on a line of cases that include Dandridge v. Williams, 397 U.S. 471, 485, 90 S.Ct. 1153, 25 L.Ed.2d 491 (1970), and Chaney v. State Bar of California, 386 F.2d 962, 964 (9th Cir. 1967), cert. denied, 390 U.S. 1011, 88 S.Ct. 1262, 20 L.Ed.2d 162 (1968).
Although the “rational relationship” standard has been applied to practices attacked as causing commercial or economic harm, e. g., McGowan v. Maryland, 366 U.S. 420, 426, 81 S.Ct. 1101, 6 L.Ed.2d 393 (1961) (statute forbidding sale of certain items on Sunday and not others); Lindsley v. Natural Carbonic Gas Co., 220 U.S. 61, 31 S.Ct. 337, 55 L.Ed. 369 (1911) (regulating drilling for natural gas); Morey v. Doud, 354 U.S. 457, 463, 464, 77 S.Ct. 1344, 1 L.Ed.2d 1485 (1957) (regulating use of premises as “currency exchanges”); Williamson v. Lee Optical of Okl., 348 U.S. 483, 489, 75 S.Ct. 461, 99 L.Ed. 563 (1954) (regulating the sale of eye glasses); Ft. Smith Light & Traction Co. v. Paving District, 274 U.S. 387, 47 S.Ct. 595, 71 L.Ed. 1112 (1926) (imposing duty to repair on street railways), we are here dealing with racial, not economic, discrimination, where even reputed strict constructionists have joined in the view that a more stringent standard must be applied. See Harper v. Virginia State Board of Elections, 383 U.S. 663, 86 S.Ct. 1079, 16 L.Ed.2d 169 (1966), and dissenting opinion of Justice Harlan in Shapiro v. Thompson, 394 U.S. 618, 659, 89 S.Ct. 1322, 22 L.Ed.2d 600 (1969). Where official conduct discriminates as to race, it is “constitutionally suspect,” Bolling v. Sharpe, 347 U.S. 497, 499, 74 S.Ct. 693, 98 L.Ed. 884 (1954), subject to “most rigid scrutiny,” Korematsu v. United States, 323 U.S. 214, 216, 65 S.Ct. 193, 89 L.Ed. 194 (1944), and bears a “very heavy burden of justification,” Loving v. Virginia, 388 U.S. 1, 9, 87 S.Ct. 1817, 18 L.Ed.2d 1010 (1967), regardless of lack of intent to discriminate racially, see Hobson v. Hansen, 269 F.Supp. 401, 426, 497 (D.D.C.1967), affd. sub. nom. Smuck v. Hobson, 132 U.S.App.D.C. 372, 408 F.2d 175 (1969). In a closely parallel situation arising under Title VII of the 1964 Civil Rights Act, 42 U.S.C. § 2000e et seq. in Griggs v. Duke Power Co., 401 U.S. 424, 432, 91 S.Ct. 849, 854, 28 L.Ed.2d 158 (1971), the Supreme Court pointed out that “good intent or absence of discriminatory intent does not redeem employment procedures or testing mechanisms that operate as ‘built in headwinds’ for minority groups and are unrelated to measuring job capability. * * * ” See also, Baker v. Columbus Municipal Separate School District, 329 F.Supp. 327 (N.D.Miss.1971); Western Addition Community Organization v.
We are satisfied that where, as here, plaintiffs show that the examinations result in substantial discrimination against a minority racial group qualified to take them, a strong showing must be made by the Board that the examinations are required to measure abilities essential to performance of the supervisory positions for which they are given. Shapiro v. Thompson, 394 U.S. 618, 627, 89 S.Ct. 1322, 22 L.Ed.2d 600 (1969); Arrington v. Mass. Bay Transp. Auth., 306 F.Supp. 1355, 1358 (D.Mass.1969); Penn v. Stumpf, 308 F.Supp. 1238 (N.D.Cal.1970).
Before considering the evidence with respect to the validity, reliability and objectivity of the examinations conducted by the Board, a few general principles must be stated. It seems to be generally accepted that before an examination will be recognized as a reliable instrument for measuring the fitness and ability of a candidate to perform tasks demanded by a given position, the examination should be validated, i. e., shown to be reasonably capable of measuring “what it purports to measure.” O. G. Stahl, Public Personnel Administration, p. 71 (5th ed. 1962). The first step toward this basic objective is to insure that as to subject matter the examination will elicit from the candidate information that is relevant to the job for which it is given. If so, it is described as having “content validity.” Otherwise the examination could be a useless and misleading tool. For instance, an examination eliciting information required to perform a bus-driver’s job would hardly be relevant in determining a candidate’s capacity to perform the duties of a policeman, lawyer or accountant.
In constructing an examination that will have “content validity,” the preferred course is first to have an empirical analysis made of the position for which it is given, usually by experts or professionals in the field. Such an analysis requires a study to be made of the duties of the job, of the performance by those already occupying it, and of the elements, aspects and characteristics that make for successful performance. Questions are then formulated, selective procedures established, and criteria prepared for examiners that should elicit information enabling them to measure these characteristics, skills and proficiency in a candidate and determine his capacity to do the job satisfactorily.
As Professor R. L. Thorndike has observed :
“Whenever a test is being tried for selection of personnel for some job specialty, it is most desirable that it be validated empirically. Experimental evidence is called for to show that the test is in fact effective in discriminating between those who are and those who are not successful in a particular job. Though it may be necessary under the press of an emergency to rely upon the professional judgment of the psychologist to establish the value of a test for personnel selection, this must be recognized as a stopgap.” R. L. Thorndike, Personnel Selection 5-6 (1949). See also, R. L. Thorndike & E. Hagen, Measurement and Evaluation in Psychology and Education 616-41 (1969); cf. 29 C.F.R. § 1607.5 (Jan. 1, 1971).
To a lesser extent the validity of an examination as a means of selecting candidates best suited for a position may also be checked or verified empirically by comparing the relative examination scores of successful candidates with their later performance on the job. If there is a significant correlation between test scores and later performance, the examination has “predictive validity.” Predictive validity is of greater significance in evaluating aptitude tests than proficiency tests. Furthermore it often takes a long time to establish such validity and even then the evaluation depends upon the reliability and fairness of the field appraisal of performance on the job.
Lastly, an examination must, of course, be administered objectively, free from bias or prejudice in favor of or against particular candidate or group, if it is to be a useful selection tool.
Notwithstanding our doubt, both sides in this dispute agree that while examination procedures may have weaknesses in testing for higher level executive positions, such procedures (including use of written tests) are essential tools in selecting supervisory personnel. However, they differ as to the areas of weakness. Plaintiffs, for instance, state that “content validity is of limited utility in selecting persons for fairly sophisticated or complex job” because of the difficulties faced in preparing tests that fairly sample the job and accurately predict a candidate’s performance. See F. Freeman, Theory and Practice of Psychological Testing, p. 91 (3d ed. 1963); A Anastasi, Principles of Psychological Testing, pp. 100-04 (3d ed. 1968). According to plaintiffs, examinations for such positions are useful only if they have predictive validity, since content validity is primarily relevant for the purpose of determining whether a candidate has learned a defined body of knowledge rather than for the purpose of determining how he will use and apply that knowledge on the job. The Board, on the other hand, takes the view that content validity is more important in determining a candidate’s proficiency or capacity to perform the duties of a Principal, and that predictive validity should be de-emphasized in judging the utility of such tests because predictive validity is more relevant to aptitude for learning than to achievement or proficiency for satisfactory performance on the job. The Board further points out that follow-up studies used to determine predictive validity are not generally used in licensing of doctors, lawyers, or other professionals.
The typical supervisory examination consists of two parts: (1) a written test, and (2) an oral interview. In addition, a problem-based conference test has been used in some years for examinations for positions such as Assistant Director and Director. The percentage weight attributed to each part of the examination varies according to the supervisory position involved. Usually about 45 to 50% is accorded to the written examination and 25 to 30% to the oral examination and another 25 to 30% to an appraisal of the candidate’s training and experience, record, written English, and physical and medical condition. More recently in an examination given for license as Principal of a Day Elementary School the written and oral interview tests were weighted 50% each with no weight apparently being given to the other factors mentioned above.
In the past the written test has usually consisted of an essay portion and a short-answer section, the latter consist
The Board contends that its examinations are valid, reliable and objective. It points to several steps taken in an effort to achieve and maintain these goals. It asserts that for each examination it has obtained from the Board of Education a statement of the “duties of the position” for which the examination is to be given. A committee or panel of experts is then assembled to specify those responsibilities considered most significant. According to the Board, well known interested educators and lay persons are also consulted with respect to the qualities for which a candidate is to be tested. With the aid of these committees, educators and lay persons, the Board’s staff then constructs questions designed to elicit the knowledge and skills required of a person holding the supervisory position for which the examination is given.
According to the Board it achieves objectivity in the scoring of written examinations by use of code book numbers on test papers, which conceal the applicant’s identity, by submitting answers on any test to more than one scorer, by use of detailed objective rating guidance, and by having answers to essay questions rated by different committees so that no one individual is responsible for rating of an entire examination paper. Oral interviews are each administered by panels of three experts.
In support of its position the Board has submitted affidavits of several respected leaders in the field of educational testing
“In general, the Board of Examiners appears to have made a conscientious and informed attempt to develop test tasks that do correspond to selected ones of the specifications set forth by the supervisory persons who set out the requirements for the job.” (Aff. of Dr. Robert L. Thorndike, dated October 9, 1970)
“The approach used by the Board of Examiners in determining the validity or relevance of its tests consists essentially of a strategy which relies on the judgments of experts and consensus among them as to what constitutes an appropriate test item.” (Aff. of Dr. Aaron Carton, dated October 25, 1970)
As thus described, the Board’s methods and procedures seem reasonable enough to one lacking expertise in the field of
“It is my recollection of the meeting that the ensuing discussion of the qualities to be sought in a candidate was initiated by the consultants, not by Mr. Rockowitz or any other representative of the Board of Examiners. Many of us expressed our dissatisfaction with the adequacy and relevance of the qualities which the Board of Examiners was apparently intending to test for, based on the established ‘duties of the position.’ We indicated that in our view it was essential to test for qualities which traditionally have never been considered by the Board of Examiners in examining principals.
“5. The meeting concluded without any sort of consensus or agreement having been reached between the consultants present at the meeting and the members of the Board of Examiners and their assistants present. There was never any follow-up to the meeting.
“6. To my knowledge, the ‘duties of the position’ previously established were in no way varied or modified as a result of the meeting. On August 28, 1969 the Board of Examiners issued an announcement for an examination for the position of Principal, Day Elementary School. Said announcement included a listing of the ‘duties of the position.’ ” (Aff. Peter J. Strauss, dated Nov. 4, 1970).
It also appears that Harvey B. Scribner, the Chancellor of the New York City School District, a distinguished and fearless educator, does not share the Board’s confidence in the validity of its examinations. In a memorandum to the Board of Education dated October 13, 1970, Dr. Scribner noted that he was “pressed to evaluate whether the present examination and licensing system, which dictates specific limitations of employment and promotion of staff for the public schools, is a help or a hardship [in the efforts of community boards to operate] * He recommended that in lieu of current employment practices the Board adopt New York State certification, plus such prescribe for those to be selected by it, criteria as each commuity board might as the minimum requirement for employment in New York City public schools. He concluded:
“For the reasons outlined in this position paper, my position with regard to the Chance and Mercado case is that I prefer not to defend myself against*220 the action. To do so would require that I both violate my own professional beliefs and defend a system of personnel selection and promotion which I no longer believe to be workable. We are facing a future in education which leaves no alternative to the selection of the most creative teachers; the most talented supervisors; the most able principals and other administrators who possess the highest level of leadership qualities possible from wherever they may be found and as are available at any given time.”
Digging deeper, we find to our dismay that the Board’s position does not appear to be supported by most of the research reports submitted by it as demonstrating the content validity of its supervisory examinations. Many of the reports are irrelevant, since they concern teachers’, not supervisors’, examinations; or they deal with objectivity or consistency; or they are mere proposals, not studies. One report merely measures the attitudes of the assistant examiners towards the “observed teaching test.” Another attempts to correlate the similarity of a 300-word English test with a 450-word English test. One report relied on by the Board does deal tangentially with the predictive validity of the examinations. It is based on a pilot study, admittedly very meager in scope. The test results showed that there was little or no correlation between success on the tests and job success (in some cases the control group did better, in others the experimental group did better, in some cases there was no statistical difference). There was no attempt in this pilot study to determine whether those who scored lower on the examination (but passed) did less well as Principals. Neither was there an attempt to determine whether those who failed the examination but who are Acting Principals are rated as successful or unsuccessful on the job. In only two out of the six years tested (1946 and 1966) did the differences reach statistically significant levels in favor of the Board’s position, and in those two years the report candidly admitted: “ * * * the relatively small population of the samples involved should be noted.” Finally, the entire study reveals only the mean rank order of the examination scores.
Reluctant as we are to invade a profession characterized by an expertise not shared by us, we must conclude on the record before us that while the Board has adopted procedures designed for content validity, it does not appear in practice to have achieved, this goal. Our conclusion, which is based upon our appraisal of affidavits of experts furnished by the parties, is confirmed by our own study of some of the examinations themselves. Turning first to the short-answer, multiple-choice tests, many of the questions strike us as having little relevance to the qualities expected of a school supervisor. Some examples are cited in the margin.
The inadequacy of such testing procedures led the Board finally to drop the short-answer test for Principal because, as Dr. Isidore Bogen, a Board member stated, “The Board of Examiners felt it was largely a matter of memory.” Notwithstanding this frank admission, the short-answer examination has been retained in tests for Assistant Principal, apparently on the theory that an Assistant Principal should know the content of the curriculum taught by teachers under his supervision. While we might be prepared to accept this compromise if it were the sole objectionable feature, the same weakness is found in many of the essay question tests, which one would expect to be used for the purpose of eliciting some of the qualities required of a Principal (e. g., accurate grammar, clarity of expression, organization and presentation of ideas, and creative thinking) as well as the candidate’s reaction to problems and his judgment and ability in analyzing and solving them. Many essay questions, however, appear to be aimed solely at the ability to memorize “duties.” Indeed the Treatman manual, swpra, gives the following advice to candidates preparing for essay questions:
“One step in this direction is the mastery of mnemonic devices, such as those found in the previous chapter, that help organize your knowledge of curriculum. Another and related valuable approach is the mastery and use of a set of mnemonic devices which will help organize your knowledge and experience in administrative and supervisory aspects of school life. Such a set of mnemonics appears below. In answering any essay question which seems to lend itself to mne*222 monic application, the candidate will have to use his own good judgment in deciding which elements of which devices are most appropriate for use. Blind exploitation of every element in a mnemonic device might result in a fatal waste of time.
* -* * * •» -X-
“A careful study of these sample answers will reveal to the candidate the technique of writing concise, credit-laden statements that outlined model answers cannot reveal. The author is convinced that the outlined model answer teaches little or nothing to the candidate, who must still see, and learn to write in, a terse style, employing many concrete illustrations, and following, wherever possible a recognizable organization * * *.” (“Principal, Assistant-To-Principal” by Paul Treatman, supra at 212-13)
Samples of some mnemonic methods and professional jargon suggested by the author are given in a footnote.
Turning to the oral interview and conference tests, we are handicapped in our ability to appraise their validity by the absence of any tangible, objective data, since the principal qualities that should be evaluated by the examining committee in the oral interview (e. g., leadership, speech, poise, comprehension, soundness of judgment and ability to present ideas and meet challenges) are intangible, and the judgment of the interviewing committee can only be as sound as the capacity of its members to recognize the presence or absence of these qualities in each candidate. In the absence of any record of such interviews or of any proof that the examiners are incompetent or that the interview problems or questions have been irrelevant, we make no finding as to the content validity of the oral examinations, standing alone.
With respect to the question of the objectivity of the examinations, adequate precautions appear to have been taken by the Board to assure objectivity in the conduct of the written examinations. As for the oral interviews, we are not persuaded by vague and speculative hearsay statements that members of the examining committees intentionally discriminate against Black or Puerto Rican candidates because of their color, use of “southernisms,” or the like. (See Aff. of Edythe J. Gaines). Since each candi
CONCLUSION
The evidence reveals that the examinations prepared and administered by the Board of Examiners for the licensing of supervisory personnel, such as Principals and Assistant Principals, have the de facto effect of discriminating significantly and substantially against Black and Puerto Rican applicants.
Despite the fact that candidates for such positions are licensed teachers who have satisfied prerequisites as to education and experience established by the Board of Education for supervisory positions and have already been certified by the State of New York for the positions sought, a survey of the results of examinations taken by 5,910 applicants (of whom 818 were Black or Puerto Rican) reveals that white candidates have received passing grades at almost 1% times the rate of Black and Puerto Rican candidates and that on one important examination given in 1968 for the position of Assistant Principal, Junior High School, white candidates passed at almost double the rate of Black and Puerto Rican candidates. (See p. 210, supra). The discriminatory effect in the latter case is aggravated by the fact that the Assistant Prineipalship has traditionally been an essential prerequisite to the more important supervisory position of Principal. (See pp. 210, 211, supra).
The existence of such de facto racial discrimination is further confirmed by the fact that only 1.4%' of the Principals, and 7.2% of the Assistant Principals in New York City schools are Black or Puerto Rican, percentages which are far below those for the same positions in the four other largest city school systems in the United States. (See pp. 212, 213, supra). For example, the percentage of Black and Puerto Rican Principals in each of the cities of Detroit and Philadelphia is 16.7%, or 12 times as high as that in New York.
Such a discriminatory impact is constitutionally suspect and places the burden on the Board to show that the examinations can be justified as necessary to obtain Principals, Assistant Principals and supervisors possessing the skills and qualifications required for successful performance of the duties of these positions. The Board has failed to meet this burden. Although it has taken some steps towards securing content and predictive validity for the examinations and has been improving the examinations during the last two years, the Board has not in practice achieved the goals of constructing examination procedures that are truly job-related. Many objectionable features remain, with the result that some 37 minority Acting Principals and 131 minority Acting Assistant Prin
There appears to be a strong likelihood that plaintiffs will prevail on the merits at trial. It further appears that plaintiffs would suffer greater harm from denial of preliminary injunctive relief than defendants would suffer from the granting of relief. Denial of relief would perpetuate existing racial discrimination, depriving plaintiffs and others similarly situated of an equal opportunity for permanent appointment and licensing as supervisors. During the long period before the case would finally be adjudicated on the merits, permanent appointments would be made from lists promulgated by the Board, which would have the effect of threatening the continued employment of those holding acting appointments, since New York law requires vacancies to be filled from the eligibility lists if such lists exist. N.Y. Education Law § 2573(2); Board of Education ByLaws § 101(3).
Granting of preliminary relief, it is true, will temporarily prevent the appointment from eligible lists of those who have, after the arduous process of taking the existing type of examinations, successfully passed and been placed on the eligible list. However, they will not be denied an equal opportunity in the future to qualify under such examination procedures as are found to be constitutionally permissible and, pending trial of the case, they would be eligible for appointment as Acting Principals or Acting Assistant Principals. Thus the balance of hardships tips decidedly in favo.r of plaintiffs, and, pending final determination of the merits, the effect of preliminary relief would be to preserve the status quo until the issues are resolved. Checker Motors Corp. v. Chrysler Corp., 405 F.2d 319, 323 (2d Cir. 1969).
Having' in mind that the existing examination system is not believed by the Chancellor of the New York City District to be a workable one, we do not envisage any great harm to the public as a result of preliminary relief. On the contrary, such relief may possibly lead the Board of Examiners, after taking another hard look at its examination procedures, to consider an overhaul that will not only eliminate racial discrimination, but lead to procedures that will be more adaptable to the Community School Board type of administration.
Lastly we cannot overlook the fact that various persons having the duty of selecting supervisory personnel, such as members of community school boards, have stated in affidavits filed with the court that they have often found that holders of licenses from the Board of Examiners do not possess the ability to perform the duties of a supervisory position for which a candidate is sought, with the result that in order to select qualified personnel it has been necessary to appoint unlicensed candidates on an acting basis. (See Affs. of Dr. Edwin J. Haas, Edythe J. Gaines, and Peter J. Strauss).
For the foregoing reasons a preliminary injunction will issue restraining defendants from (1) conducting further examinations of the type found to be unconstitutionally discriminatory against Blacks and Puerto Ricans, and from (2) promulgating eligible lists on the basis of such examination procedures.
The foregoing shall constitute our findings of fact and conclusions of law as required by Rule 52(a), F.R.Civ.P.
We take this opportunity to express appreciation to the parties for their thorough papers, and to the amici for their briefs, which were of assistance in resolving the difficult and complex issues.
Settle order on notice.
Heard the case as a District Judge and, after appointment to the Circuit Court of Appeals, was designated to sit on the District Court for the purpose of completing this phase of the case.
. See lolanthe, Act I:
“Titles shall ennoble, then,
All the Common Couneilmen:
Peers shall teem in Christendom,
And a Duke’s exalted station Be attained by Competitive Examination.”
. 42 U.S.C. § 1981:
“§ 1981. Equal rights under the law “All i)ersons within the jurisdiction of the United States shall have the same right in every State and Territory to make and enforce contracts, to sue, be parties, give evidence, and to the full and equal benefit of all laws and proceedings for the security of persons and property as is enjoyed by white citizens, and shall be subject to like punishment, pains, penalties, taxes, licenses, and exactions of every kind, and to no other.”
. 42 U.S.C. § 1983:
Ҥ 1983. Civil action for deprivation of rights
“Every person who, under color of any statute, ordinance, regulation, custom, or usage, of any State or Territory, subjects, or causes to be subjected, any citizen of the United States or other person within the jurisdiction thereof to the deprivation of any rights, privileges, or immunities secured by the Constitution and laws, shall be liable to the party injured in an action at law, suit in equity, or other proper proceeding for redress.”
. New York Constitution Art. 5, § 6:
“§ 6. \Civil service appointments and promotions; veterans’ preference and credits]
“Appointments and promotions in the civil service of the state and all of the civil divisions thereof, including cities and villages, shall be made according to merit and fitness to be ascertained, as far as practicable, by examination which, as far as practicable, shall be competitive; * *
. New York Education Law § 2590-j(3) (a) (1):
Ҥ 2590-j. Appointment and removal of persons in the teaching and supervisory service
* * * * *
“3. (a) (1) The board of examiners shall prepare and administer objective examinations to determine the merit and fitness of all candidates for teaching and supervisory service positions, other than the positions of chancellor, executive deputy city superintendent, deputy city superintendent, assistant city superintendent and community superintendent. Examinations for teaching positions may consist in part*206 of the National Teachers Examination administered by the Educational Testing Service of Princeton, New Jersey.”
. New York Education Law § 2569(1) :
“§ 2569. Board of examiners “1. In a city having a population of one million or more there shall be a board of examiners to consist of five members, one of whom shall be the superintendent of schools of the city school district of the city of New York or a deputy superintendent thereof designated by him to act in his stead during his pleasure. No person other than the chancellor, the superintendent of schools, or a deputy superintendent so designated shall, while in the supervising or teaching service of the city, serve on such board. It shall be the duty of the board to hold examinations whenever necessary, to examine all applicants who are required to be licensed or to have their names placed upon eligible lists for appointment in the schools in such city, except examiners, and to prepare all necessary eligible lists, subject to the responsibilities among the members of the board of examiners hereinafter set forth. The board shall periodically review the var lidity and reliability of examinations as well as examination procedures." (Emphasis added)
. New York Education Law § 2573(10) :
“10. In a city having a population of one million or more, recommendations for appointment to the teaching service shall be from the first three persons on appropriate eligible lists prepared by the board of examiners. Eligible lists, including competitive and qualifying lists, shall not be merged and a list shall be exhausted or have expired before nominations are made from a list of subsequent date. No competitive or qualifying eligible list except existing principals’ eligible list shall remain in force for a longer period than four years, nor have a life of less than three years, no eligible list now in force shall terminate any sooner than four years from the date on which it was promulgated. * * ® The board of education, on the recommendation of the superintendent of schools shall designate, subject to the other provisions of this chapter, the kind and grades of licenses which shall be required for service as principal, branch principal, director, assistant examiner in the board of examiners, supervisor or teacher of a special branch, head of department, assistant, school psychiatrist, school psychologist, school medical inspector, school aurist, school psychiatric social worker, school social case worker, research assistant, teacher-clerk, school clerk, school secretary, clerical assistant, industrial or trade helper in vocational schools, school librarian, laboratory assistant, placement and investigation assistant, financial assistant, machine shop assistant, tool boy, or any other position of the teaching staff together with the academic and professional qualifications required for each kind or grade of license. No person required to have a license under the provisions of this chapter in order to be employed in a position who does not have such license shall have any claim for salary, except that a person who has been assigned to teach in a subject or field not specifically covered in his license but on the same rank or level of service shall be entitled to his salary.”
. Plaintiffs have also moved for a determination that this action is a class action maintainable under Rule 23, F.R.Civ.P. Although plaintiffs brought this motion on soon after their original motion for a preliminary injunction, in accordance with Local Civil Rule 11A, all parties have agreed that our decision on this class action motion would be reserved. Defendant Board of Examiners has not yet filed any papers opposing plaintiffs’ motion making this action a class action. Defendants have 10 days from the date of this decision to file opposing papers. Otherwise the motion will be granted.
. NYABSSA lias a membership of approximately 800 New York City public school administrators and supervisors. Its purpose is “the attainment of an effective education for black children in the New York City School System.”
. ASPIRA is a nonprofit corporation “devoted to the improvement of the Puerto Rican communities of the United States through education and leadership development.” “ASPIRA” is the Spanisli word for “aspiring.”
. The PEA is a nonprofit corporation, chartered in 1899, in order “to secure and maintain the highest possible standards of public education in the City of New York.”
. N.Y.Education Law §§ 2569, 2590-j(l), (4) (b), (4) (d).
. Telegram of Luis Mercado to Board of Examiners, Xov. 3, 1970, Exhibit B to defendant Board of Examiners’ First Supplemental Memorandum of Law, received Xov. 6, 1970.
. See Court Orders of December 29, 1970, February 16, 1971, April 13, 1971, and April 29, 1971.
. Plaintiffs contended, inter alia, that the discriminatory effects of the examinations were and are well known to the Blacks and Puerto Ricans. Hence there was a “chilling effect” on their taking the examinations. Though we ordered these statistics on “withdrawal” and “not apI>earing” to be taken for the sake of completeness, we place no reliance on this argument. Merely because someone thinks an examination is discriminatory is not proof that the examination is in fact discriminatory.
. We have based our conclusion on the statistical evidence, not on “straightforward moral and constitutional arithmetic.” Hobson v. Hansen, D.C., 327 F.Supp. 844, at 859 (1971). We are not here dealing with simple arithmetic, cf. Reynolds v. Sims, 377 U.S. 533, 84 S.Ct. 1362, 12 L.Ed.2d 506 (1964) (“one man, one vote”), but with something approaching “constitutional calculus.”
Table adapted from Affidavit of plaintiffs' expert, professor Jacob Cohen, May 6, 1971, 115. The computations of probability are his; the racial statistics come directly from the Survey.
. Of the candidates identified as Puerto Rican only, 27.59% passed; of the candidates identified as Black, 31.56% passed.
. The one Black who took it failed; no Puerto Ricans took it; and 5 whites took it. Pour failed and the last one withdrew, which in effect meant that he failed because he could not successfully complete the examination process.
. See Affidavit of Dr. Jacob Colien, dated May 13, 1971, K 3, n. 2, for a list of these nine examinations.
. These examinations are :
Assistant to Principal, Day Elementary School, 4/65, PF-03 Superintendent of Home Economics, 6/67, PF-12
Superintendent, Early Childhood, 2/68, PF-20
Supervisor of Art, 3/68, PF-23 Supervisor of Music, 5/68, PF-28
Assistant Principal, Junior High School, 9/68, PF-43.
. The sources of these statistics, which have not been disputed by defendants, are set forth in detail in plaintiffs’ Memorandum filed Sept. 24, 1970, pp. 3 — 4, n. 2, as supplemented by letter of Elizabeth B. DuBois to the Court, dated June 11, 1971.
. Professor Robert L. Thorndike, Teachers College, Columbia University; Dr. Marvin Sontag, Associate Professor of Psychology and Education, Teachers College, Columbia University; Associate Professor Aaron Suss Carton, State University of New York; Dr. Esiu Kaya Carton, Professor of Educational Psychology, Hofstra University.
. 1961 Examination for Principal, Day Elementary School
“64. Of the following characters in the nursery rhyme, THE BURIAL OF POOR COOK ROBIN, the one who killed Cock Robin is the
1. Lark
2. Thrush
3. Bull
4. Sparrow”
Í965 Examination for Assistant Principal, Junior High School
“211. I’ve Got a Little List, from the Mikado is sung by
1. Nanki-Poo 2. Pish-Tush 3. Ko-Ko 4. Pooh-Bah
“212. Arthur Sullivan of the team of Gilbert and Sullivan, wrote
1. Gypsy Love Song 2. The Lost Chord
3. Ah, Sweet Mystery 4. A Kiss in the Dark of Life
“218. AVliich one of the following violin makers is NOT of the great triumvirate of Cremona?
1. Amati 2. Stradivarius 3. Guarnerius
4. Maeeini”
1. Daisy Miller
2. Elmer Gantry
3. Senry Esmond
4. Mrs. Dalloway
“74. The author of Tender is the Sight also wrote :
1. The Great Gatshy
2. Butterfield 8
3. The Sun Also Rises
4. Sanctuary
“75. A contemporary play in which one of the characters seems to turn into a rhinoceros was written by:
1. Beckett
2. Pinter
3. Ionesco
4. Shaw”
. “OBJECTIVES OP ELEMENTARY EDUCATION
Cohetkase
Character
Our American Heritage
Health
Exploration
Thinking
Knowledges and Skills
Appreciation and Expression
Social Relationships
Economic Relationships
“DEVELOPING A LEARNING EXPERIENCE
Rice
Readiness
Instructional materials provided
Carrying out experience
Evaluating the experience
“KINDS OP LEARNING EXPERIENCES
Ü.8.O.
Unit
Short-term
Ongoing (centers of interest)
“AREAS POR EVALUATION
I Am Same
Interaction among children
Atmosphere of classroom
Materials
Status (health, social, interests)
Achievement
Methods and procedures
Experiences, worth of
“USES OP EVALUATION
Pragg
Progress and grade placement
Reports to parents
Analysis of pupil difficulties
Guidance
Grouping
* * x xx
“IMPROVING SCHOOL DISCIPLINE
Pert Gages
Planning of standards and rules
Environment, improvement of
Routines, training in
Teaching, improvement of
Conferences
Analysis of difficulties
Guidance procedures
Evaluation and follow-up
Self-control emphasis”
. The Board has not adduced statistics as to the race of the examiners or examination assistants who have conducted the oral examinations other than to observe, at a recent large-scale examination, that of approximately 200 persons serving as examination assistants over 30, or 15%, were Black or Puerto Rican. Furthermore, the racial statistical survey is not broken down according to grades received by candidates on the components of examinations (i. e., scores on written, essay, short-answer, oral, appraisal of record, etc.).
