Csicseri v. Bowsher

862 F. Supp. 547 | D.D.C. | 1994

862 F.Supp. 547 (1994)

Anthony M. CSICSERI, et al., Plaintiffs,
v.
Charles A. BOWSHER, Defendant.

Civ. A. No. 88-1089 (OG).

United States District Court, District of Columbia.

September 16, 1994.

David H. Shapiro, Washington, DC, for plaintiffs.

Eric H. Holder, Jr., U.S. Atty., John D. Bates, Mark E. Nagle, Asst. U.S. Attys., Joan Hollenbach, Associate Gen. Counsel, Paul Thompson, Edda Emmanuelli-Perez, General Accounting Office, Washington, DC, for defendant.

MEMORANDUM

GASCH, District Judge.

This Court held a trial on January 21, 24, 25, 26, 27 and 28, 1994, concerning allegations of age discrimination at the General Accounting Office ("GAO"), which defendant Charles A. Bowsher heads as Comptroller General of the United States. Plaintiffs, all GS-15 ("Band III") employees older than 50, allege that they were passed over for selection to the Executive Candidate Development Program ("ECDP") in favor of younger candidates. The ECDP is the most common route of entry into the GAO's Senior Executive Service ("SES").

Plaintiffs also allege that other routes into the SES were denied them because of their age. Finally, the GAO addresses, although plaintiffs did not raise, a disparate impact claim. Such claims arise when an employment practice is neutral on its face but discriminatory in its impact.

Plaintiffs are all excellent GAO employees with long government careers about which they can be justifiably proud. The SES, however, is not the next step on the employment ladder for all those qualified as GS-15's. Rather, the SES represents a significant departure from their previous experience and a marked elevation in responsibility. Only a fraction of ostensibly qualified GS-15 or Band III employees will ever be promoted to the SES through the ECDP or otherwise. These promotions are of necessity reserved for those truly exceptional employees with sterling records and great potential.

As discussed at length in the opinion that follows, each plaintiff's employment record and the testimony of their supervisors at the GAO indicate that there are legitimate non-discriminatory reasons why they were not selected for the ECDP and promoted to the SES. The statistical evidence further supports the GAO's contention that its ECDP and SES selection process is age neutral.

I. EVIDENTIARY RECORD

The record derives from the trial testimony and the exhibits received in evidence at trial. The trial testimony is delineated as follows: Tr.Vol. I — January 21, 1994; Tr.Vol. II — January 24, 1994; Tr.Vol. III — January 25, 1994; Tr.Vol. IV — January 26, 1994; Tr. Vol. V — January 27, 1994; and Tr.Vol. VI — January 28, 1994. The exhibits will be referred to as Pl.Ex. _ or Def. Ex. _.

Based on this evidence and in accordance with Federal Rule of Civil Procedure 52(a), the Court makes the following findings of fact and conclusions of law:

II. FINDINGS OF FACT

A. The Parties

1. Defendant Charles A. Bowsher is the Comptroller General of the United States. As such, he is head of the GAO. Tr.Vol. II 183.

2. The GAO, where all plaintiffs were employed at the time of the events giving rise to this suit, is an agency of Congress employing approximately 4,600 people. What began as a mere voucher checking organization has grown into an auditing and evaluating agency that investigates and evaluates the efficacy of a wide range of government programs. Acting on congressional requests, the GAO issues approximately 1,000 reports per year and its representatives testify before Congress in excess of 200 times per year. Tr. Vol. II 234-36.

3. The GAO is divided into four program divisions and two technical divisions. The program divisions are General Government, Human Resources, National Security and International Affairs and Resources, Community and Economic Development. The technical divisions are Program, Evaluation and Methodology and Accounting and Information Management.^[1] At the head of each of these divisions is an Assistant Comptroller General who is a member of the SES. The Assistant Comptroller Generals report directly to the Comptroller General. Def.Ex. 1.

4. The divisions are divided into approximately 36 issue areas designed to cover the entire federal government. Issue areas are headed by Directors with some areas having Associate Directors on their staffs as well. The Directors and Associate Directors are also members of the SES who report to the applicable Assistant Comptroller General. Tr.Vol. II 241, 247.

5. Directors and Associate Directors oversee 50-60 audits and evaluations at any given time. They are responsible for the presentation and preparation of congressional testimony. They also represent the GAO in meetings with members of Congress and their staffs, other federal agencies and the private sector. Tr.Vol. I 179; Tr.Vol. II 235, 238, 241-42, 245; Tr. Vol. V 897.

6. Directly below the Directors and Associate Directors are Assistant Directors who are Band III or GS-15 employees.^[2] Assistant Directors supervise the staff actually conducting the audits and investigations. Tr. Vol. VI 967, 970.

7. Plaintiff Anthony Csicseri began his government service in 1953 with the Air Force, where he became an expert in computers and electronics. He remained with the Air Force until 1964 when he transferred to the Army. Mr. Csicseri retired from the military in 1972 with the rank of Chief Warrant Officer W-2. During that period, Mr. Csicseri earned a college degree in economics. Tr.Vol. II 287-90.

8. Immediately after leaving the military, Mr. Csicseri joined the GAO in its Cincinnati regional field office as a GS-12. He began as a computer specialist but soon became an evaluator. Mr. Csicseri remained with the Cincinnati office until 1979. He then came to Washington where he was assigned to the Accounting and Financial Management Division until 1984. From 1984 until his retirement in 1993,^[3] he was with the Information Management and Technology Division. During this time, Mr. Csicseri did two stints on Capitol Hill. He was detailed to the Senate Finance Committee in 1979-1980, the House Environment, Energy, and Natural Resources Subcommittee from August 1988 to June 1990, and from June 1990 to June 1991, he was detailed to the House Subcommittee on Legislation and National Security. Mr. Csicseri was promoted to GS-15 in 1981. Tr.Vol. II 285-88, 291, 315, 336, 362; Tr.Vol. V 847. Pl.Ex. 30.

9. Plaintiff Jack D. Kearns came to the GAO in 1973 from the United States Army Computer Systems Command, where he was a computer specialist. Prior to his civilian employment with the Army, Mr. Kearns worked on Capitol Hill serving as a press corps liaison in the House periodical gallery. Mr. Kearns also earned his college degree in business administration and computer science in 1973. Tr. Vol. II 342-43.

10. During his entire 20 year tenure at the GAO, Mr. Kearns was assigned to one division, the Accounting and Financial Management Division, where he rose to the rank of GS-15 in 1979. Mr. Kearns was designated a computer specialist until 1983. From 1983 until 1987 he was an evaluator, and from 1987 to 1993, he was a supervisory computer specialist. Mr. Kearns was detailed to the Senate Select Committee on Indian Affairs for most of 1988. He took advantage of the early retirement program and left the GAO in December 1993. Tr.Vol. II 344, 350-52, 362.

11. Plaintiff Julius S. Brown entered the Army after graduating from college in 1953. After 20 years of service in the Quartermasters Corps, Mr. Brown retired from the Army a Lieutenant Colonel and entered the GAO as a GS-12 in 1973. Mr. Brown was first assigned to the Logistical Communications Command as a management analyst, a job title later changed to evaluator. Tr.Vol. II 370-72.

12. In 1978 he was reassigned to the General Service and Controller Division where he remained until 1986. Mr. Brown achieved GS-15 in 1981. During that period, Mr. Brown was Director of the Office of Publishing. In 1986, Mr. Brown was moved to the National Security and International Affairs Division ("NSIAD"). He remained with that division until 1993, when he too took advantage of the GAO early retirement option. Tr.Vol. II 376-77, 384; Tr.Vol. III 396, 398.

13. Plaintiff Manohar Singh came to the GAO in 1976, as a GS-14, Supervisory General Engineer, after a number of government posts. Dr. Singh began his career in 1957 or 1958 with the District of Columbia government. He then transferred to the Army Corps of Engineers as a civilian employee. He next worked for the Commonwealth of Pennsylvania as a Supervisory Hydraulic Engineer, returning to the Army Corps of Engineers three years later. Dr. Singh then switched services, working for the Navy Facilities Engineering Command and finally for the Postal Service as a Supervisory General Engineer. Tr.Vol. III 403-07.

14. At the GAO, Dr. Singh was assigned to the Procurement System Acquisition Division as a Supervisory General Engineer. In 1982 or 1983, he transferred to the Resources Community and Economic Division, where he remains today. Tr.Vol. III 404, 407.

15. Dr. Singh holds a bachelor's and master's degree in engineering, as well as a Doctor of Science degree. He is a Registered Professional Engineer. Dr. Singh was promoted to GS-15 in 1984. Tr.Vol. III 403, 428.

16. Plaintiff Robert A. Jaxel began his federal career with the Census Bureau in 1961, moving in 1967 to the Federal Aviation Administration. He returned to the Census Bureau in 1968, moving to the Department of Commerce, Bureau of Domestic and International Commerce, in 1969. In 1972 he transferred within the Department to the National Technical Information Service. Mr. Jaxel joined the GAO in 1974. Mr. Jaxel holds a bachelor's and master's degree in Economics, and completed additional graduate work at the University of Pennsylvania and the War College. Tr.Vol. III 454-58.

17. At the GAO, Mr. Jaxel was assigned to the Financial and General Management Studies Division as an evaluator. In this division, he made GS-15 in 1977. In 1983 he went to the Accounting and Financial Management Division. From 1984 to 1986, Mr. Jaxel worked as a report reviewer, first in the Office of Quality Assurance and then in the National Security and International Affairs Division. Mr. Jaxel then remained with NSIAD as a group director, running audits. In January 1988, Mr. Jaxel went to the Office of Affirmative Action Plans to work on the Equal Promotion Review Program. This program sought to investigate racial discrimination in promotions within Bands I, II, and III. In November 1988, he was detailed to the House Appropriations Committee. In 1990, Mr. Jaxel resigned from the GAO, with full re-employment rights, to keep his assignment with the Committee. In November 1993, Mr. Jaxel exercised his re-employment rights and returned to NSIAD at the GAO, where he was assigned to the Office of Special Investigations. Tr.Vol. III 447-48, 450, 461-65, 482-83.

18. Plaintiff Frank S. Heard came to the GAO from the Army, where he was a telecommunications and computer expert. Although he was drafted into the service as a private, he rose to the rank of major before retiring in 1974. Between the Army and arriving at the GAO in 1975, Mr. Heard worked briefly for a private corporation and the General Services Administration. He received his college degree in 1978, while working at the GAO. Mr. Heard also completed some graduate work at the University of Pennsylvania and George Washington University. Tr.Vol. III 485-86, 489.

19. Mr. Heard entered the GAO as a GS-12 Communications Specialist/Evaluator in the Logistics and Communications Division. He was promoted to a GS-15 in that division in 1982. Mr. Heard's communications group bounced from division to division although it remained intact. It landed in the Information Management and Technology Division in 1983. Mr. Heard was next assigned to the now abolished Washington Regional Office in 1985. In 1988, he was reassigned to the Resources, Community, Economic Division. From September 1988 to December 1989, Mr. Heard was detailed to the District of Columbia government. He returned to the GAO proper in 1990, where he was assigned to the National Security and International Affairs Division as a report reviewer. He currently serves on the planning staff of that division. Tr.Vol. II 499, 501, 509-12, 514.

20. Unfortunately, plaintiff Ryan Yuille died on September 30, 1993. His heirs now carry on this lawsuit in his name. Mr. Yuille graduated from the Hampton Institute in 1952. Thereafter, he had a distinguished military career culminating in the rank of Lieutenant Colonel. Along the way, Mr. Yuille picked up a master's degree in Personnel Administration from George Washington University in 1969. Pl.Ex. 64.

21. Mr. Yuille came to the GAO as a GS-15 in 1979. In 1980, he was appointed Deputy Director of the Civil Rights Office at the GAO. In February 1988, he was detailed to Howard University under the Intergovernmental Personnel Act. Mr. Yuille was still with Howard at the time of his death. Pl.Ex. 64.

B. The SES Promotion Process At The GAO

i. The SES

22. The SES was established at the GAO on October 5, 1980, under the General Accounting Office Personnel Act of 1980. It replaced the supergrades GS-16 through GS-18. Def.Ex. 3.; Tr.Vol. II 197.

23. The SES is made up of positions such as the General Counsel, Assistant Comptroller General, Director of Congressional Relations and Director of Internal Revenue. For the most part, however, SES members serve as Directors or Associate Directors for the issue areas. These executives oversee approximately 50-60 audits at a time. Def.Ex. 3; Tr.Vol. II 241-42.

23. The SES is made up of positions such as the General Counsel, Assistant Comptroller General, Director of Congressional Relations and Director of Internal Revenue. For the most part, however, SES members serve as Directors or Associate Directors for the issue areas. These executives oversee approximately 50-60 audits at a time. Def.Ex. 3; Tr.Vol. II 241-42.

24. Of the approximately 4,600 GAO employees worldwide, only 145 are in the SES. This number is set by congressional act. Tr. Vol. II 236, 244.

25. Out of necessity, mainstream members of the GAO SES must have demonstrated the ability to organize and oversee a number of employees, as well as to set up and run at the same time a number of complex evaluations. SES executives must also have superior verbal and written communication skills so that they can easily and effectively testify in congressional hearings. Tr. Vol. I 112, 144-45; Tr.Vol. II 235, 237, 244-45; Tr.Vol. VI 969.

26. There are essentially three routes into the SES. They are noncompetitive limited term appointments, competitive Comptroller General career appointments and the ECDP. Approximately 80 per cent of those eventually rising to the SES come through the ECDP. Def.Ex. 4, 6(b), 6(c), 6(d); Tr. Vol. II 205-07, 223-24, 249-51.

27. The ECDP came into existence in 1981. Its purpose was and continues to be to provide the GAO with a pool of GS-15 executive candidates capable of immediately filling positions within the SES. This program also serves a training function. It provides executive candidates with the knowledge and understanding of the organization that are required to perform successfully as an executive. Additionally, it ensures that the executive candidate continues to demonstrate excellence in managerial skills. Def.Ex. 3; Tr. Vol. I 41, 115-19, 132, 144, 151, 170; Tr.Vol. II 179, 186-87, 198-99.

28. Once selected to the ECDP, the executive candidate receives intensive training to develop his management, technical and program skills. When this training period is completed, the candidate is eligible for a noncompetitive promotion to the ECDP. Def.Ex. 3.

29. Since its inception in 1981, there have been 10 ECDP classes. On the average each class has had 10-12 members, which accounts for approximately two per cent of those eligible for promotion to the SES. Since 1981, 116 people have been taken into the ECDP. Of that number only approximately seven have failed to reach the SES. Tr.Vol. I 106; Tr.Vol. II 197-99, 240, 243-44.

ii. Selection to the ECDP

30. Ultimately, selection to the ECDP is somewhat subjective, being based on a comparison of nominees in any given year. Generally, the successful candidate has demonstrated exceptional skill in performing the GAO's main tasks, namely, managing several audits and evaluations at one time and showing outstanding conceptualization and communications skills which would enable them to testify effectively before Congress. Although the selection process remains to a certain extent a subjective and comparative assessment of a candidate's skills and accomplishments, there are, nevertheless, certain guidelines which form the basis for the decisions. These guidelines are by their nature flexible and are designed to reflect potential as much as past achievement. The guidelines are delineated in Def.Ex. 4(a) and 4(b) and were the focus of much attention at trial. Tr.Vol. I 112, 132, 145; 159-60; Tr.Vol. II 179, 211-12, 243-44; Tr.Vol. V 897; Tr.Vol. VI 969, 978-80.

31. These criteria, which appear in the ECDP Job Opportunity Announcement under the title "Qualification Requirements", are as follows:

Knowledge of evaluation or auditing principles, techniques and practices, and the potential to organize and direct a comprehensive evaluation and/or audit program are required. Applicants meeting the above criteria will be further evaluated to determine the extent to which their experience, education and accomplishments are indicative of competence to accomplish the following six activity areas.

1. Integration of Internal and External Policy Issues. Understanding roles and relationships within an agency, the federal hierarchy, and the public at large; keeping abreast of key national agency-wide issues, priorities and values; and applying these factors in carrying out the responsibilities of the immediate work unit.

2. Organizational Preparation and Liaison. Establishing and maintaining relationships with key individuals and groups outside the immediate work unit, and serving as a spokesperson for the work unit and organization. Such activities would include preparing and/or presenting effective briefings, speeches, congressional testimony, interunit staff meetings, etc.

3. Direction and Guidance of Programs, Projects, or Policy Development. Understanding broad policy guidance and priorities, designing responsible plans and programs, and establishing the necessary structure and procedures for carrying them out. Necessary capabilities include: effective long-term and short-term planning, information gathering and analysis, and scheduling and monitoring progress of the work.

4. Resource Acquisition and Administration. Principles and procedures for obtaining and allocating the resources needed to implement policies and programs, including: a) staff planning and budgeting techniques and b) equitable work force recruitment and selection procedures.

5. Development and Utilization of Human Resources. Assuring that people are appropriately employed and dealt with fairly and equitably. These include: assessing individuals' capabilities and needs and assigning appropriate work in response, providing career development opportunities, establishing clear performance standards and accurate performance appraisals, and adhering to governmentwide EEO and other personal utilization programs.

6. Review of Implementation and Results. Employing program review and evaluation techniques to assure that plans are being implemented and adjusted as necessary and that the appropriate results are being achieved.

Def.Ex. 4(a), 4(b).

32. As indicated above, the ECDP application process begins with a job announcement. Applicants are limited to GS-15, Band III or employees with like rank and responsibility. In addition to GAO employees, applications are accepted from outside candidates. However, because the issue before the Court concerns only internal selections to the ECDP, it will not delve into the selection process for non-GAO employees. Def.Ex. 4; Tr.Vol. I 46, 49, 126; Tr.Vol. II 214.

33. Interested GAO GS-15's submit a memorandum to their respective division or unit heads. The unit heads then typically consult with the applicant's supervisors in an effort to determine which applicants best meet the criteria enumerated in ¶ 31. Of those GS-15's eligible, most do not apply. And, of those who do apply, only about half or sometimes a third receive their unit head's nomination. Pl.Ex. 28; Def.Ex. 4, 4(a), 4(b), 8, 25-28, 33; Tr.Vol. I 46, 50, 120, 123-24, 128-34, 157-58, 163-64; Tr.Vol. II 243; Tr. Vol. IV 722, 733-34, 752-53; Tr.Vol. V 843-44; 893-96, 915-18; Tr.Vol. VI 969-76, 978-80, 1008.

34. Those nominated by their unit heads are then passed on to the Qualifications and Performance Review Board ("Q & PRB") along with their SF-171's, GAO Form 570's as well as the nominee's supervisor's "Appraisal of Candidate Performance and Potential." The Q & PRB has six or seven members, with each being appointed by the Comptroller General and serving two year staggered non-renewable terms. All members of the Q & PRB are SES and are chosen from throughout the GAO. Comprising the Board are at least one Division Head, one Regional Manager, an Issue Area Director, a Planning and Reporting Director and a Director of Operations. Pl.Ex. 28; Def. Ex. 3, 10-22, 25-28, 33; Tr.Vol. I 50, 164.

35. The Q & PRB reviews each applicant's paperwork and scores that person's qualifications on a scale of 1, lowest, to 5, highest, in each of the six categories set forth in ¶ 31. Each applicant is given an overall score, and then a composite score is also calculated. Pl.Ex. 28; Def.Ex. 3, 4, 34; Tr. Vol. I 50, 157-58, 160, 164-65.

36. Each Q & PRB member is then assigned one or more applicants on whose qualifications they become knowledgeable. At a meeting of the full Q & PRB, that member would lead a discussion concerning the applicant's qualifications and potential. Also, the applicant's supervisor or unit head usually appears before the Board to discuss his nominees. After this meeting, the Q & PRB makes adjustments, if necessary, to the applicant's score, and it arrives at a final overall composite score. Pl.Ex. 28, Def.Ex. 3, 4, 34; Tr.Vol. I 50-53, 157-58, 160, 164, 165.

37. The Q & PRB sends what it determines to be the top 50% of the applicants, broken down into quartiles, on to the Executive Resources Board ("ERB"). The membership of the ERB is chosen by the Comptroller General. It consists of certain ex officio members as well as those personally appointed by the Comptroller General. They are all senior SES officials, including a number of Assistant Comptroller Generals as well as the General Counsel. The Chairman of the ERB throughout the relevant time period was Milton Socolar, the Special Assistant to the Comptroller General, the number two person at the GAO. Def.Ex. 3, 24; Tr. Vol. I 58-59, 118-19, 162-65; Tr.Vol. VI 994-97.

38. The Q & PRB was eliminated in 1990, when it was determined that the ERB could perform the Q & PRB's role as well as its own. Pl.Ex. 28; Tr.Vol. I 163.

39. Each member of the ERB considers a candidate's SF-171 and other paperwork, the comments of his unit head and, until 1991, the recommendation of the Q & PRB.^[4] Like the Q & PRB, the ERB scores the candidate, on a scale of 1-5, in each of the criteria enumerated in ¶ 31. The overall rating of the candidate is then factored in to create a composite score. Pl.Ex. 28; Tr.Vol. I 105, 121, 161-62, 165-66; Tr.Vol. II 194.

40. Typically, two ERB members interview each candidate using a predetermined set of questions. They then report the results of the meeting to the entire ERB. At this stage, the applicant's unit head is given an opportunity to address the ERB. Pl.Ex. 28; Tr.Vol. I 142, 166-67; Tr.Vol. II 194.

41. Each member of the ERB arrives at a final score for each applicant and from that, a final composite score is calculated. Based on these scores, the ERB forwards a list of candidates, along with the minutes of the ERB, to the Comptroller General. The Comptroller General then discusses the nominee with the Chairman of the ERB, the Assistant Comptroller General for Operations and other individuals who can contribute to the decision, such as the candidate's unit head. Pl.Ex. 28; Tr.Vol. I 124, 143, 162; Tr.Vol. II 190-93, 203, 205, 245.

42. The final decision on who wins a spot in the ECDP is the Comptroller General's. Normally, he selects all of the nominees whom the ERB forwards. On occasion, however, he has selected slightly more or slightly less. In any event, the Comptroller General is free to place anyone in the ECDP whether or not they participated in, or were eliminated during, the nomination process. The Comptroller General, however, has exercised this power sparingly over the years. Pl.Ex. 28; Tr.Vol. I 121-24, 143; Tr.Vol. II 189-90, 203, 244-45, 247-48.

C. Plaintiffs' ECDP Candidacies

43. Plaintiff Csicseri applied for his division's nomination to the ECDP in 1982 (81-14A),^[5] 1984 (84-3), 1985 (85-4), 1986 (86-2), 1987 (87-2), 1988 (88-1), and 1990 (90-1). He received his division's nomination in 1982, 1984 and 1985. In 1984 and 1985, he was referred to the ERB. The ERB did not forward his name to the Comptroller General on either occasion. Pl.Ex. 28, 29; Def.Ex. 4-6; Tr.Vol. II 298-99; Tr.Vol. VI 843, 848, 853, 855.

44. There is no question that Mr. Csicseri was highly competitive for a position in the ECDP. In addition to his achievements enumerated in ¶¶ 7-8, Mr. Csicseri's performance ratings were for the most part excellent. He received Certificates of Merit, Certificates of Accomplishment, Outstanding Achievement Awards,^[6] and numerous letters of recommendation from congressmen, senators and Capitol Hill staff. Pl.Ex. 30-34.

45. Nevertheless, there are numerous non-discriminatory reasons why Mr. Csicseri was not taken into the ECDP and promoted to the SES. His supervisors expressed legitimate concern over the depth and complexity of his GAO experience. Although an excellent employee, they believed that Mr. Csicseri did not favorably compare to other ECDP candidates in the breadth and quality of his experience in managing complex audits or evaluations, his knowledge of the GAO's institutional goals and his ability to represent the agency before high level government officials and the Congress. Also mentioned were Mr. Csicseri's somewhat inadequate writing skills, his inability to complete a project in a timely manner and his difficulty with interpersonal relationships. Indeed, even when his nomination was forwarded to the ERB, the Q & PRB ranked him in the lower two quartiles. Pl.Ex. 28, 30-34, 75-79; Def. Ex. 4, 24, 28; Tr.Vol. V 843-84; Tr.Vol. VI 996-97.

46. Mr. Csicseri's age was not a factor in the GAO's decision not to select him to the ECDP or to promote him to the SES. Pl. Ex. 28; Tr.Vol. I 121, 124, 142-145, 157-158, 160, 161, 164-165, 166-167, 176; Tr.Vol. II 190-194, 212, 260, 277-278.

47. Jack Kearns applied for his division's nomination to the ECDP in 1985 (85-4), 1986 (86-2), 1987 (87-2), 1988 (88-1), 1990 (90-1) and 1991 (91-4). Defendant states that Mr. Kearns never received a division nomination. Apparently, however, Mr. Kearns did receive a nomination in 1986, although he was never informed of his success. In any event, he never progressed beyond the Q & PRB. Pl. Ex. 28, 34; Def. Ex. 28; Tr.Vol. II 355; Tr.Vol. IV 739-42.

48. Mr. Kearns, like his co-plaintiffs, was an excellent and valued GAO employee. His record contains accolades from within and without the GAO, including a Director's Award, a Special Commendation Award and letters of commendation from a congressman and a senator. Pl.Ex. 35, 36.

49. This praise, well deserved though it is, does not entitle him to a position in the ECDP and the SES, as there are numerous nondiscriminatory reasons why Mr. Kearns was not promoted. It was the opinion of his division directors that his performance was not consistently of sufficient quality to warrant selection, and his performance evaluations did not compare favorably to others seeking promotion. Indeed, Mr. Kearns' performance appraisals show that he was most often evaluated as having "met expectations," which would equate with an average ranking.^[7] Even Mr. Kearns himself admitted that his appraisals were not "exceptional across the board". Pl.Ex. 66-74. Tr.Vol. II 366; Tr.Vol. IV 739-42; Tr.Vol. V 779-80, 796-97.

50. Mr. Kearns' lack of success in receiving a promotion to the ECDP or to the SES is not attributable to his age. Tr.Vol. V 909-10.

51. Julius Brown, according to plaintiffs, applied for every ECDP class from 1982 until his retirement. Defendant states that Mr. Brown applied for the ECDP in 1982 (81-14A), 1984 (84-3), 1988 (88-1), 1990 (90-1) and 1991 (91-4). Nevertheless, both parties agree that Mr. Brown received his division's nomination only once, in 1984 (84-3). Def. Ex. 26; Tr.Vol. VI 990-91.

52. From 1978 to 1986, Mr. Brown ran the Office of Publishing, at times called the Office of Administration and Publishing, where he supervised upwards of 100 employees. By all accounts, Mr. Brown did an exemplary job of running this office. Pl.Ex. 37, 38.

53. In 1985, the Comptroller General decided to modernize and upgrade the publishing office and to create an SES position for the eventual chief of this office. Mr. Brown applied for this position which was a competitive career appointment, not a position filled through the ECDP. To head the revamped department, the GAO looked for someone with specialized technical experience in publication, design, graphics and electronic printing. The GAO ultimately hired Larry Rolfus, a member of the SES and the director of operations at the Bureau of Engraving and Printing. Mr. Rolfus had extensive prior experience with this type of high technology publishing operation. In short, Mr. Brown was not as qualified as Mr. Rolfus. Tr.Vol. I 97-98, 168; Tr.Vol. II 220-21.

54. When Mr. Rolfus took over the reconstituted Office of Publishing, now called the Office of Publication and Communication, Mr. Brown went back to auditing, a task he had not performed since 1978 as a GS-13. As an auditor/evaluator, Mr. Brown met with some success, as indicated by letters of commendation from within and without the GAO. However, it does not appear that Mr. Brown's achievements were commensurate with selection to the ECDP and ultimately a position in the SES. Other group directors supervised four to seven audits at a time, while Mr. Brown typically supervised only one at a time. Additionally, there was some fairly substantial criticism concerning the Bath Iron Works audit which Mr. Brown oversaw. Pl.Ex. 38; Def.Ex. 24; Tr.Vol. II 389-91; Tr.Vol. IV 756; Tr.Vol. VI 985-87.

55. Age was not a factor in Mr. Brown's not being selected to the SES, as head of the Office of Publication and Communication, nor in his not being selected to the ECDP and ultimately the SES. Tr.Vol. I 98; Tr.Vol. II 220-221; Tr.Vol. IV 759; Tr.Vol. VI 975-976, 993-998.

56. Manohar Singh applied for the ECDP classes of 1986 (86-2), 1988 (88-1), 1990 (90-1) and 1991 (91-4). Dr. Singh never received his unit's nomination. Pl.Ex. 28, 29; Tr.Vol. III 430.

57. The evidence suggests that Dr. Singh's contributions as an engineer were substantial. He received kudos from within the GAO, from Capitol Hill and from the private sector. In 1993 he received a Special Commendation Award and a pay increase for his work on a report covering "smart pig" technology to be used for the mechanized inspection of natural gas pipelines. Pl.Ex. 42-44; Tr.Vol. III 407-24.

58. Being a successful engineer, a highly specialized field, does not automatically make one a successful auditor or executive candidate. Dr. Singh's division head, after consultation with Dr. Singh's immediate supervisors and other senior managers in the division, determined that Dr. Singh was not as well qualified as others who had applied from that division. Moreover, it was felt that Dr. Singh possessed deficient writing skills, had a problem translating technical material for a lay audience, and that he had an interpersonal relationship problem with his coworkers. These are two deficiencies which, when measured against the GAO's promotion criteria as enumerated in ¶ 31, would prove fatal to any application. Furthermore, Dr. Singh was never an auditor/evaluator, which would further hamper his chances of promotion. Def.Ex. 4(a), 4(b); Tr.Vol. V 921, 923; Tr. Vol. VI 928-31.

59. Dr. Singh alleges that his supervisors attempted to force him to retire. This pressure supposedly came from Herb McClure and Ken Mead, who inquired as to when Dr. Singh would retire, and asked him why he did not seem to want to retire. Mr. Mead also allegedly told Dr. Singh that as a specialist he had very little chance for promotion to the SES. Tr.Vol. III 432-33.

60. Although the matters raised in ¶ 59 certainly deal with age, questions concerning retirement by one's supervisors, who may have personnel and staffing concerns, do not necessarily indicate age bias. Furthermore, many people look forward to retirement, and asking someone why they do not take advantage of the availability of retirement may be nothing more than friendly conversation. While the final allegation dealing with specialists not making the SES is disturbing and, if true, a bad policy, it does not indicate age discrimination. In the final analysis, lack of executive potential, and not his age, kept Dr. Singh out of the ECDP and the SES. Tr.Vol. VI 932.

61. Robert Jaxel applied for only the 1988 (88-1) and 1990 (90-1) ECDP classes. Neither time did he earn a nomination. Tr. Vol. III 481; Tr.Vol. V 771-72; Tr.Vol. VI 1024.

62. Like the other plaintiffs, Mr. Jaxel was a solid GAO Band III employee but, not executive material. Mr. Jaxel's record does contain several accolades — specifically, a letter of commendation from a congressman, a Director's Award, a Career Service Award, a Certificate of Appreciation and a Special Award From The Comptroller General. Most notably, Mr. Jaxel met with great success in his work with the Equal Promotion Review Board, which conducted an investigation into racial disparity in promotions within Bands I, II and III. Pl.Ex. 39, 40, 41; Tr. Vol. III 460-63.

63. While Mr. Jaxel's record was certainly more than satisfactory, it was not exceptional, especially when evaluated against the ECDP hiring criteria enumerated in ¶ 31. Furthermore, while other group directors typically ran four to seven audits at any given time, Mr. Jaxel usually ran only one, although he claims to have run three to five at a time. Also, he spent long periods on detail and had limited experience and accomplishments as a group director. And, as the promotion criteria indicates, the quality and quantity of such experience are important factors in the ECDP decision-making process. Moreover, Mr. Jaxel received a plethora of "met expectations" ratings from his superiors which, as discussed in footnote 6, is the middle of three possible rankings. He also received several "superiors", which is the fourth of five possible rankings under the new system. While these ratings from his superiors are solid, they are not stellar, and would not put Mr. Jaxel in the top percentage of Band III employees who are taken into the ECDP. Pl.Ex. 39; Def.Ex. 4(a), 4(b); Tr.Vol. III 460-63; Tr.Vol. V 771-72; 809-11; Tr.Vol. VI 981, 1025, 1031.

64. Age was not a factor in Mr. Jaxel's failure to be chosen for the ECDP and promoted to the SES. Tr.Vol. V 772; Tr.Vol. VI 1025.

65. Frank Heard applied for the ECDP classes of 1984 (84-3), 1985 (85-4), 1986 (86-2), 1987 (87-2), 1988 (88-1) and 1990 (90-1). He received a division nomination every time he applied except in 1990. His nomination reached the ERB at least once. Plaintiffs say he reached the ERB in 1985 and 1986, while the GAO claims he reached it only in 1986. In any event, Mr. Heard's name was never passed on to the Comptroller General for appointment to the ECDP. Pl.Ex. 28, 29; Def.Ex. 6, 24; Tr.Vol. I 110-11; Tr.Vol. III 520-33; Tr.Vol. VI 990-92, 994-95.

66. There is no question that Mr. Heard was an excellent GAO employee. He received two Certificates of Appreciation, two Certificates of Merit, an Outstanding Achievement Award as well as several letters of commendation from supervisors and congressmen alike. Pl.Ex. 47, 48.

67. Despite his exemplary performance, there is ample evidence to suggest that Mr. Heard was not part of that small group of GS-15 superstars chosen for the ECDP and ultimately the SES. His many excellent performance reviews are characterized with several ratings of "met expectations", where "exceeded expectations" was the highest grade, "superior", where "exceptional" was the highest grade, "exceeds fully successful", where "outstanding" was the top, or "average" or "high", where "outstanding" was the top grade. At times Mr. Heard received nothing but the highest rating, but often he received these lesser rankings. Pl.Ex. 48.

68. Even when nominated, his superiors felt that Mr. Heard needed to improve in the areas of conceptualization, oral communication and organization. It was also felt that he had not had enough experience as a group director and that he did not quite match up to the other candidates in executive potential and leadership. Moreover, in the latter years when Mr. Heard became a report reviewer, he typically reviewed only one to three reports in a six-month period while others would review fifteen to twenty reports in the same period. This may or may not have to do with some administrative duties Mr. Heard shouldered, and the record does not indicate whether the other report reviewers had similar administrative burdens. Pl. Ex. 51-54; Tr.Vol. I 110; Tr.Vol. III 526, 540-41; Tr.Vol. IV 760-64; Tr.Vol. VI 990-92, 1012-15.

69. Age was not a factor in the GAO's not promoting Mr. Heard to the ECDP and the SES. Tr.Vol. IV 764; Tr.Vol. VI 998.

70. Ryan Yuille applied for his division's nomination to the ECDP in 1982 (81-14A), 1984 (84-3), 1985 (85-4), 1987 (87-2) and 1990 (90-1). His nomination was forwarded to the Q & PRB only once, in 1981. His nomination did not go forward to the ERB in that or any other year. Pl.Ex. 28, 29, 64; Def.Ex. 24; Tr.Vol. V 884-86.

71. Mr. Yuille's performance was generally satisfactory, especially in the early 1980's. Thereafter, it trailed off according to his immediate supervisor, Alexander A. Silva. Still, his record contains many letters of commendation from within and without the GAO. Also, and not of small note, it is clear from the record that Mr. Yuille was an outstanding member of the community. Thoughtful and caring, he stands as a role model for community involvement. Unfortunately, these desirable attributes which make one a good citizen do not necessarily make one a qualified executive. Pl.Ex. 64.

72. Throughout his GAO career, Mr. Yuille's performance was rated as having "met expectations" on the three-tier system. In 1986, he received one ranking of "did not meet expectations". According to Mr. Silva, he failed properly to supervise the work of subordinates who were preparing employment discrimination decisions. These decisions did not present the issues clearly, concisely and logically, did not include sufficient facts and data to support the conclusions and did not include correct data and references to the file. A deficiency in supervisory skills would prove fatal to any person seeking promotion to an executive level. Moreover, Mr. Yuille, who was not an auditor in the classical GAO sense, did not demonstrate the skills or potential to organize and direct several large scale audits at one time. And, as indicated in the ¶ 31 promotion criteria, such skills are a prerequisite for the ECDP. Pl.Ex. 64; Def. Ex. 30, 32; Tr.Vol. V 885, 886-88.

73. In the final analysis, most of Mr. Yuille's professional success occurred prior to his joining the GAO, and his age played no role in his not being selected for the ECDP and the SES. Pl.Ex. 64; Tr.Vol. V 885-88.

D. The Statistical Evidence

Plaintiffs allege that, in addition to their individual cases of age discrimination, there is an invidious pattern of age discrimination at the GAO. Plaintiffs point to what they call the "inexorable zero" meaning that no one over the age of 50 has been selected for the ECDP, and very few of that age have been taken directly into the SES via other mechanisms. Both sides presented copious, contradictory and, at times, confusing expert statistical evidence hoping to establish or refute such a pattern of discrimination. The Court's findings on this evidence are as follows:

74. Dr. Charles R. Mann, plaintiffs' expert, is an expert in the field of statistics. Tr.Vol. IV 571.

75. Dr. Joan G. Haworth, defendant's expert, is an expert in the fields of labor economics, econometrics and statistics. Tr.Vol. IV 637, 638.

76. The GAO provided data to both experts for ECDP classes 1981 through 1990. No information concerning applicants who failed to win their unit's nomination was available for ECDP classes 81-09 (1981), 81-14A (1982) and 84-03 (1984). There being no 1983 ECDP class, both experts began their analyses of unit nominations with 1985. The GAO also provided numbers concerning all those entering the SES, regardless of route, for the years 1984 through 1990. Pre-1984 numbers are, apparently, not available. Pl. Ex. 12; Def.Ex. 8.

77. In performing his tests, Dr. Mann ran statistical analyses based upon two age groups. He compared the treatment of applicants under 40 to those over 40. Next, he compared the treatment of applicants under 40 to those over 50. The latter comparison leaves out all those applicants between the ages of 40 and 50. Pl.Ex. 12.

78. Dr. Mann ran analyses for unit nominations,^[8] the Q & PRB's recommendations to the ERB, the ERB's recommendations to the Comptroller General, the Comptroller General's ultimate selection to the ECDP and for all four stages of the process taken together. Pl.Ex. 12.

79. For each of these stages, Dr. Mann performed a 2 × 2 contingency table analysis using the Fisher exact test^[9] and a multiple pool extension, which is also referred to as an exact probability analysis.^[10] From examining plaintiffs' exhibit 12, it would appear that Dr. Mann also used various permutations of the Chi-square test,^[11] but rejected them because of the relatively small size of the sample pool. Pl.Ex. 12.

80. In his lengthy testimony before this Court, Dr. Mann explained the procedure he used to draw his first set of conclusions with regard to the ECDP. The hypothesis he tested was that there is `no difference between the treatment of older workers and younger ones.' This type of hypothesis is called a two-tailed hypothesis because, if the opposite were to be true, that the two groups were not treated the same, the statistician would not know whether the older or the younger workers fared better. In a one-tailed test, which Dr. Mann also used, the hypothesis would read something like, `older workers are treated at least as well as younger workers.' That way, if the opposite were to be true, the statistician would know that the younger workers fared better. Tr.Vol. IV 573, 612.

81. With either method, the statistician examines the data to evaluate the likelihood, or the probability, that the numbers he arrived at are substantially the same as he predicted if the original, or null, hypothesis were true. If the numbers do differ substantially, the statistician concludes that the opposite to the null hypothesis is true, namely, that older and younger workers are not treated equally, in the two-tail test, or that younger workers are treated better, in the one-tail test. The numerical discrepancy between the null hypothesis and the data reflects the probability of rejecting the null hypothesis when it is actually true. Tr.Vol. IV 574-75.

82. Normally this discrepancy or probability rate is deemed to be legally significant if it is calculated at .05. Other ways of expressing this .05 value are 5%, 1 in 20 or 1.96 standard deviations which is commonly rounded and referred to as two standard deviations.^[12] (The value 1.96 reflects a two-tail test. For a one-tail test, 1.645 is normally rounded to two standard deviations.) In essence, this means that there is a 1 in 20 chance that the statistical conclusions are wrong. Tr.Vol. IV 579.

83. After performing a statistical analysis of the different stages of the ECDP process for each class, as well as an overall analysis for each class, Dr. Mann found statistically significant disparities between those over 40 and those under 40 in only 2 of 37 instances. Originally, Dr. Mann concluded that such a disparity existed in 3 of 37 instances, however, at trial he concluded that he had made an error and that such a disparity only existed in two instances. Those two instances were the Q & PRB's recommendation to the ERB in 1984 (84-03) and the ERB's recommendation to the Comptroller General in 1986 (86-02). Pl.Ex. 12; Tr.Vol. IV 611-15.

84. Dr. Mann also ran these analyses for employees under the age of 40 versus those over the age of 50. He concluded that statistically significant disparities existed in the same three instances as described in ¶ 83, above. On cross-examination he once again agreed that there was an error and, in reality, significant disparities existed in only two instances. The same two as described above. It was also pointed out that further errors had been made with respect to the ERB's recommendation in 86-02. While these errors do not invalidate Dr. Mann's conclusion, they certainly raise questions. Pl.Ex. 12, Tr.Vol. IV 611-15; 629-30.

85. Also, the Court notes, and Dr. Mann concurred in his testimony, that 2 of 37 equals roughly 5%. While this 5% has a different mathematical significance than the 5% referred to in ¶ 82, supra, it certainly does not indicate large scale discrimination even if Dr. Mann's conclusion is taken at face value. Tr.Vol. IV 613.

86. Dr. Mann also used a multiple pool analysis for each phase of the ECDP selection process, as well as for the overall process, for those employees under 40 versus those over 40. Of the five analyses run, Dr. Mann concluded that significant statistical deviations occurred in two instances. These are selection of candidates by the ERB for the Comptroller General's consideration and the overall ECDP selection process. Pl.Ex. 12.

87. In running the same five-stage multiple pool analysis for those employees under 40 versus those over 50, Dr. Mann concluded that statistical discrepancies of at least two standard deviations occurred in four out of the five steps analyzed. Only the analysis of the Comptroller General's consideration of those forwarded to him by the ERB was deemed statistically insignificant. Pl.Ex. 12.

88. Defendant's expert, Dr. Haworth, also reviewed the available data for the eight ECDP classes between 1981 and 1990. She analyzed the data available for each step in the ECDP selection process except for the Comptroller General's consideration of the names sent to him from the ERB. Dr. Haworth excluded this final step because the Comptroller General, with a few statistically insignificant exceptions, appointed all those whom the ERB recommended. Dr. Haworth also performed an overall analysis of the final selection to the ECDP from all the nominated candidates, as well as all the applicants where possible.^[13] For each rung along the ECDP selection ladder, Dr. Haworth evaluated the eight classes together. Thus, in effect, she performed a technique similar to Dr. Mann's pooling process. Dr. Haworth's methodology is discussed in the following paragraphs. Def.Ex. 8.

89. As alluded to above, Dr. Haworth went beyond mere statistics in performing her analyses. She wanted to determine what factors, except for age as that was the ultimate question to be tested, were determinative in the decision-making process at each step of the ECDP selection system. She began by looking at various documents produced by the decision-making boards, such as the Q & PRB and the ERB, she examined the papers submitted by the candidates themselves, and she examined other GAO documents which she deemed relevant to her inquiry. Dr. Haworth also conducted extensive interviews of the promotion decision-makers. She asked them how these decisions were made, what factors were considered, how they went about the process, how they reviewed each individual candidate, et cetera. Tr.Vol. IV 640-41.

90. Dr. Haworth used the information she gathered during this investigation to arrive at certain characteristics which she felt were statistically intrinsic to the decision-making process. These were: (1) level and field of education; (2) competitiveness of the particular ECDP class applied for; (3) office assignment, range of projects given that office; (4) experience; (5) occupation; (6) hire grade and years of GAO experience; and (7) prior applications. Pl.Ex. 65; Tr.Vol. IV 651-52.

91. Dr. Haworth incorporated these characteristics into a multiple regression model, or more precisely, a subspecies of that model known as a logistical or logit equation which is used for yes or no type inquiries.^[14] Dr. Haworth defines this process as the relationship between a candidate's being selected or not, and isolating the particular characteristics which caused that person to be selected or not. To do this, Dr. Haworth examined each applicant's particular characteristics at every stage in the ECDP process. By examining whether that candidate was successful or not, Dr. Haworth could determine whether those particular characteristics were a help or hindrance to the candidate. Pl.Ex. 65; Tr.Vol. IV 655-56, 687-97.

92. When estimating the number of older applicants who should succeed at any given step in the process, Dr. Haworth factored in this logit regression which, in theory, should reduce the number of variables other than age which could account for older applicants' not being selected. This would give her ultimate analyses greater accuracy. Def.Ex. 8.; Tr.Vol. IV 652.

93. Dr. Haworth tested her logit model and found it increased the predictive power of her tests between 2% and 6% for each factor considered, and it increased the predictive power up to 50% when all factors are taken out of the model. Using this logit model, her successful prediction rate was between 67% or 68% to 75%. Tr.Vol. IV 657-58.

94. To arrive at her ultimate conclusions, Dr. Haworth took the number of applicants still in consideration at each stage in the process. As discussed above, she next adjusted for factors such as the applicant's education, occupation, office assignment, length of service and the number of times the applicant applied. Then, she determined the percentage of older applicants from that group who, statistically, should be successful in moving on to the next step in the process. Dr. Haworth next predicted the number of applicants who should be selected from the older group, 40 or over or 45 or over depending on the test, by multiplying the total number of applicants by the percentage of older applicants who, statistically, should be successful. She then compared the spread between the number her formula predicted and the actual number of older applicants selected. She calculated the standard deviation from that spread. Again, two standard deviations is the operative number. Def.Ex. 8; Tr.Vol. IV 659-60.

95. Dr. Haworth concluded that there were no statistically significant instances of discrimination, defined as greater than two standard deviations, in either the analysis run at 40 or 45. The ECDP promotion process, in her opinion, is at every stage and in totality age neutral. Def.Ex. 8.

96. Dr. Haworth and Dr. Mann also ran analyses for all promotions to the SES regardless of route for those under 40 versus those over 40. Dr. Haworth performed this test by taking the total number of GS-15 employees and determining what percentage of those employees are over the age of 40. Using the total number of people promoted to the SES in any given year, she next calculated the percentage of those promoted who should be 40 or older. She further distilled this into a hard number, for example, if 70% of the 10 entrants to the SES in any given year should, simply based on the population of GS-15's, be 40 or over, then 7 of the 10 promoted should be 40 or older. Dr. Haworth then compared the distilled predicted number with the actual number of those 40 or older who were promoted. As with her ECDP analysis, she compared the spread between these two numbers and calculated the number of standard deviations from those spreads. Def.Ex. 8; Tr.Vol. IV 642-44.

97. Dr. Haworth ran this analysis for every year from 1984 to 1990.^[15] Taking all of those years together, Dr. Haworth arrived at a standard deviation of - 1.72.^[16] This is less than 1.96 and, therefore, she concluded that the totality of hiring into the SES was age neutral. Def.Ex. 8.

98. Dr. Mann used the same numbers and mathematical procedures as Dr. Haworth. However, he only ran his analysis for 1984 through 1987. Dr. Mann left out the years 1988-1990 because he felt that the GAO changed its hiring practices in 1987 as a result of this lawsuit.^[17] To add those years, he believes, would taint his analysis. Pl.Ex. 14; Tr.Vol. IV 596-98, 668-69.

99. Having isolated the years 1984-1987, Dr. Mann concluded that there was a standard deviation of -2.663.^[18] This figure is statistically significant and would be evidence of age bias. Pl.Ex. 14.

100. In the final analysis, it goes without saying that both experts engaged in mathematical acrobatics and statistical juggling. Nevertheless, for the reasons set forth in the following paragraphs, the Court finds that Dr. Haworth is the more credible witness. Likewise, the Court finds that her report is more credible and helpful in determining this difficult matter. See generally, Pl.Ex. 11, 12, 14; Def.Ex. 8, 9; Tr.Vol. IV 571-730.

101. First, Dr. Mann made two errors in his non-pooled analysis of the ECDP classes. The first on class 81-14A and the second on class 86-02. While Dr. Mann admitted and corrected these errors in open court, and testified that they do not change his conclusion, they nevertheless cut into the persuasiveness of his overall analysis. Pl.Ex. 12; Tr.Vol. IV 611-15; 629-30. See also, ¶¶ 83-84, supra.

102. Instead of performing an analysis for those under 50 versus those over 50, the age bracket in which plaintiffs allege that discrimination lies, Dr. Mann ran an analysis of those under 40 versus those over 50. This cuts out all those employees ages 40 to 49, who make up a large portion of GS-15's, the majority of those applying to the ECDP and the majority of those entering the SES. Tr. Vol. IV 607-10.

103. Dr. Haworth also failed to run an analysis for those under 50 as opposed to those over 50. She instead ran an analysis for those under 45 versus those over 45. Dr. Haworth explained that there were so few candidates over the age of 50, between 5% and 8% for any given class, that it could be possible to choose none and still be within the realm of chance. Therefore, in order to see if the older members of the protected class were treated differently from the younger members, she chose a cut-off point of 45 so that she could increase her pool and come up with a more accurate result. This method seems more logical to the Court than Dr. Mann's excision of the majority of the affected employees. Tr.Vol. IV 650-51.

104. The Court also notes that Dr. Mann uses a one-tail test in drawing his conclusions, while Dr. Haworth uses a two-tail test. Generally, in the District of Columbia Circuit, a two-tail test is preferred for these types of cases. However, in this case the analysis cannot end with that statement. There is little chance, from a facial review of the evidence, that applicants over the age of 50 were treated statistically better, as opposed to statistically equal or worse, than those under the age of 50, or for that matter, 40. Therefore, a one-tail test is not without merit. Nevertheless, there is an excellent chance that applicants over the age of 40 were treated statistically better than those under the age of 40. In that situation, a two-tail test is preferred. Thus, while the Court does not discredit the one-tail test which Dr. Mann used, it does note that it would have been preferable had he used a two-tail test, as did Dr. Haworth, at least for the tests comparing those over 40 with those under 40. Tr.Vol. IV 612, 662-63; see also, ¶¶ 80-81 (defining one and two-tail testing); Palmer, 815 F.2d at 92-96 (expressing this Circuit's preference of two-tail tests).

105. Dr. Mann places great emphasis on his multiple pool ECDP analysis. The Court believes he is correct in stating that when one increases the size of the sample, there should be greater accuracy in the final statistical analysis. However, aggregation of data is not without its dangers. Tr.Vol. IV 585.

106. For example, let us assume a statistician is running an aggregate analysis to determine if there is sex bias in graduate admissions at a university. Department A has an overall acceptance rate of 50%, which is the same for men and women. Department B has an overall acceptance rate of 10%, which is the same for men and women. If 80 men and 20 women apply to Department A, while 20 men and 80 women apply to Department B, then of the 100 men who applied 42 will be accepted and of the 100 women, only 18 will be accepted. The odds ratio equals 1 if each department is taken separately. However, the odds ratio is 3.7 if they are aggregated. Thus, aggregation can create the appearance of discrimination when in reality none exists.^[19]See M. Finkelstein and B. Levin, Statistics for Lawyers 242 n. 1 (Springer-Verlag 1990).

107. The above example highlights but one of many variables which can throw off an aggregate analysis. Therefore, when formulating a test using aggregate data, such as multiple pools, certain steps can be taken to try to ensure that the pools are not statistically skewed by some unforeseen variable. It appears that Dr. Mann did try to safeguard his data to a certain extent by using some type of multivariant or regression type analysis. But, exactly what steps Dr. Mann took, what variables he considered and how far he went, cannot be gleaned from his testimony or report.^[20] We do know, however, that Dr. Mann, unlike Dr. Haworth, did not engage in a lengthy study of the promotion process at the GAO prior to embarking upon his analyses. Tr.Vol. IV 585-87, 605-06.

108. Therefore, in this Court's opinion, Dr. Mann's methodology in his ECDP analyses is not as persuasive as Dr. Haworth's. Granted, under regression models, there is a danger that the variables will be chosen in order to influence the outcome or that the inclusion of a particular variable may be inherently invalid.^[21] While it is inevitable that people will disagree over the inclusion or exclusion of certain variables, it appears that Dr. Haworth was careful and accurate in her choice of variables, or characteristics as she calls them, and none appear to be inherently invalid. Furthermore, the Court does not know what, if any, variables Dr. Mann used, and, therefore, does not have a basis to judge them for good or for ill. See ¶ 90, supra and Pl.Ex. 65 for a full list of Dr. Haworth's variables. See also generally, F. Fisher, Multiple Regression in Legal Proceedings, 80 Col.Law Rev. 702 (1980).

109. In the end, Dr. Haworth's variables, chosen after much research and consultation at the GAO, provide the Court with a broader, more in-depth and, ultimately, more accurate analysis of the ECDP selection process at the GAO. Tr.Vol. IV 640-41.

110. Nevertheless, plaintiffs complain that Dr. Haworth's variables are not the same as the oft touted six qualifications appearing on Def.Ex. 4(a) and 4(b). While the qualifications and Dr. Haworth's variables are not identical, they are, nevertheless, compatible and consistent. Furthermore, the job announcements from which those six qualifications are drawn state that those qualifications will be measured within the context of an applicant's auditing skills, experience, education and accomplishments. And, these last four enumerated factors are, in fact, almost identical with the variables used by Dr. Haworth. Pl.Ex. 65; Def.Ex. 4(a), 4(b); see also, ¶ 31, supra.

111. Turning to the SES as a whole, plaintiffs criticize Dr. Haworth for her incorporation of the years 1988-1990 into her analysis of "Entry into SES by Age Group". Plaintiffs believe that the GAO took steps to amend their discriminatory ways with the filing of this lawsuit in 1988 and thus, Dr. Haworth's incorporation of those years is biased. Tr.Vol. IV 667-70.

112. In reality, the difference between the actual and predicted promotions within the protected age group does not differ that greatly pre- and post-lawsuit. The spread is as follows:

1984          -.09
1985         -3.5
1986         -1.1
1987         -2.7
---------------------
1988          1.7
1989          -.02
1990          0.0

To credit plaintiffs' theory would be to assume that the dozens of people involved in the ECDP process, from the unit heads to the Comptroller General himself, were aware of this lawsuit, had decided to promote more, older employees because of this lawsuit and acted in a conspiracy to do so. One would also have to assume that a number of those people, including the highest ranking GAO officials, had further determined to perjure themselves. Finally, the Court notes that, had the GAO wanted to effect the outcome of this lawsuit, it would have been simpler and more effective to promote plaintiffs or to promote a larger group of older employees, and thus make a bigger statistical dent in the numbers. Compare Pl.Ex. 14 and Def.Ex. 8; Tr.Vol. IV 667-70.

113. The Court finds Dr. Haworth's analysis of "Entry into SES by Age Group" to be more comprehensive and credible. Both experts used the same numbers and the same methodology for this analysis. Because her -1.72 standard deviation reflects a larger and more complete sample, namely, 1984-1990, the Court finds it to be more persuasive than Dr. Mann's -2.663 figure which reflects only the years 1984-1987. Pl.Ex. 14; Def.Ex. 8.

114. Along the same lines, Dr. Haworth is further criticized for failing to apply a continuity correction factor^[22] to her results for her "Entry into SES Positions by Age Group" analysis. Dr. Haworth testified that she did indeed employ a continuity correction factor. However, because it did not render the test results statistically significant, she did not use it in her final analysis and conclusion. During trial a continuity correction was performed for Dr. Haworth's final result of -1.72. It yielded a still statistically insignificant result of -1.84.^[23] Pl.Ex. 14; Def. Ex. 8; Tr.Vol. IV 681-82, 712.

115. Plaintiffs had a further problem with Dr. Haworth's counting people's ages as of the beginning of the fiscal year. To use Dr. Mann's example, if an applicant were to turn 40 on the second day of the fiscal year, he would be counted as young. This, Dr. Mann believes, works to her advantage. First, few people would be shifting into the protected class in any given year. Secondly, it would appear that the more people above 40 in the sample, the more advantageous it is to the GAO. Thus, it would seem that Dr. Haworth's methodology actually works against the GAO by increasing the size of the non-protected class in the sample pool. Tr.Vol. IV 710-11.

116. Finally, plaintiffs speak of the "inexorable zero." It is indeed troubling to this Court that no candidates over the age of 50 were accepted into the ECDP during the relevant time period. The Court, however, cannot take this seemingly unusual figure in isolation but, rather, must view it in the context of the other evidence. A persuasive but by no means dispositive component of that other evidence remains the fact that there were so few candidates over the age of 50 that it is possible to have chosen none and remain within the realm of statistical chance. See ¶ 104, supra.

E. Other Evidence

117. Defendant has entered into evidence the records of certain successful ECDP candidates who, it asserts, are better qualified than plaintiffs. Similarly, plaintiffs have raised certain successful candidates who, they claim, are less qualified than they. Neither party can prove plaintiffs' relative qualifications, or lack thereof, with small samplings of candidates. The records of all successful and unsuccessful candidates are not before the Court. Also, plaintiffs' evidence in this regard is mostly testimonial. Therefore, the Court does not have a proper basis for comparison. Def.Ex. 6, 10-12, 17, 20, 21, 22; Tr.Vol. I 121, 127-29, 145-48; Tr.Vol. II 217-18, 246, 268-70; Tr.Vol. VI 937-40, 1015.

118. Plaintiffs take umbrage at the GAO's listing of the successful candidates' ages in the GAO Management News. They also point to a handwritten memorandum between two support staff personnel to the ERB in which the birthdays and ages of successful candidates for one ECDP class are listed next to their names and divisions. First, the GAO Management News did not highlight or focus on age. Rather, it did statistical profiles of the most recently chosen ECDP class. In these profiles, age was listed along with several other items such as division, years at the GAO, years at GS-15, education and experience. Age, like education and experience, is typically listed in these types of statistical or biographical breakdowns. Pl.Ex. 1-9, 16; Tr.Vol. I 70-79; Tr.Vol. III 554-57.

119. Plaintiffs further argue that the GAO stopped mentioning age in the GAO Management News with the filing of this lawsuit. In reality, the GAO stopped printing the entire statistical profile of the newly chosen class. This was done out of a concern that people might assume that age was a consideration rather than a statistical result. In any event, age was still treated no differently than the other statistical results such as division or education. As for the single handwritten memorandum, it is certainly odd that birthdays and ages would be listed in such a way. However, this single note was written by and to people not involved in the hiring process, and it reflected only the age of successful candidates, not all candidates. Pl.Ex. 1-9, 16; Tr.Vol. I 70-79; Tr.Vol. III 554-57.

120. Plaintiffs also point to a 1986 internal review of the ECDP in which the GAO itself brings forth the specter of age discrimination. It does not call for affirmative action, the GAO believing at all times that its system is age neutral, but states the need for more objective hiring criteria and feedback so as to obviate any appearance of discrimination. As for the Equal Promotion Review Board's ("EPRB") finding of racial discrimination, that only applied to promotions within the GS or Band ranks and did not deal directly with age discrimination. Plaintiffs also entered into evidence a 1989 Personnel Appeals Board investigation of alleged age discrimination in the Boston Regional Office. This report concluded that a prima facie case of discrimination could be legally established but, there was not enough evidence to substantiate the allegations. This report pertains only to a GS level promotion in a regional office. It does not concern the SES, the ECDP or the GAO as a whole. The Court would also note that as a legal standard, a prima facie case of discrimination is not difficult to establish. Pl.Ex. 18-27; Tr. Vol. III 473-79.

121. Plaintiffs also point to their Exhibit 17, in which Mr. Csicseri, in a handwritten note in a comment box on an ECDP score sheet, was referred to as "old hat". This vague statement may or may not have to do with age. The reference could be to average skills, a typical rather than exceptional GS-15, an application that resembled most of the others or to his previous unsuccessful applications. Pl.Ex. 17.

III. CONCLUSIONS OF LAW

This case comes before this Court under the Age Discrimination in Employment Act ("ADEA"), 29 U.S.C. §§ 623, 633a(c).^[24] The ADEA states, quite directly:

It shall be unlawful for an employer —

(1) to fail or refuse to hire or to discharge any individual or otherwise discriminate against any individual with respect to his compensation, terms, conditions, or privileges of employment, because of such individual's age;

(2) to limit, segregate, or classify his employees in any way which would deprive or tend to deprive any individual of employment opportunities or otherwise adversely affect his status as an employee, because of such individual's age; ....

29 U.S.C. § 623(a)(1) and (2).

Suits brought under the ADEA are typically treated under the analytical framework used in cases of race and gender discrimination arising under Title VII of the Civil Rights Act of 1964. Arnold v. United States Postal Service, 863 F.2d 994, 996 (D.C.Cir.1988), cert. denied, 493 U.S. 846, 110 S.Ct. 140, 107 L.Ed.2d 99 (1989).

A Title VII plaintiff may rely on two different theories to support his claim of age discrimination. These are disparate treatment and disparate impact. A disparate treatment claim involves an isolated instance of discrimination against an individual within the protected class, or a pattern or practice of discrimination against a group of individuals within the class. A disparate impact claim involves a facially neutral employment practice that, nevertheless, works to the disadvantage of the protected class. See International Brotherhood of Teamsters v. United States, 431 U.S. 324, 335 n. 15, 97 S.Ct. 1843, 1854 n. 15, 52 L.Ed.2d 396 (1977).

A. Disparate Treatment

The Supreme Court's decision in McDonnell Douglas Corp. v. Green, 411 U.S. 792, 93 S.Ct. 1817, 36 L.Ed.2d 668 (1973), lays out the familiar shifting burden of proof to be used in Title VII and ADEA cases. First, the plaintiff must establish a prima facie case of discrimination. The burden then shifts to the employer to articulate a legitimate nondiscriminatory reason for the employee's rejection. The burden then moves once again to the plaintiff to demonstrate by a preponderance of the evidence that the employer's stated reasons were in fact pretextual and age was a determining factor in the employment decision. McDonnell Douglas, 411 U.S. at 802-05, 93 S.Ct. at 1824-26; Hayman v. National Academy of Sciences, 23 F.3d 535, 538 (D.C.Cir.1994); Cuddy v. Carmen, 694 F.2d 853, 856-858 (D.C.Cir.1982), cert. denied, 474 U.S. 1034, 106 S.Ct. 597, 88 L.Ed.2d 576 (1985).

To establish a prima facie case of age discrimination, plaintiffs must demonstrate facts sufficient to create a reasonable inference that age was a determining factor in the GAO's decision not to promote them to the ECDP or the SES. Such an inference can be established by showing: (1) plaintiffs are members of the statutorily protected age group; (2) plaintiffs were qualified for the ECDP and SES; (3) plaintiffs were not selected to the ECDP nor promoted to the SES; and (4) people not within the protected age group were selected or promoted in their place. See Hayman, 23 F.3d at 537; Cuddy, 694 F.2d at 857. See also, McDonnell Douglas, 411 U.S. at 802, 93 S.Ct. at 1824-25 (analogous elements under Title VII).

The statutorily protected age group is individuals at least 40. 29 U.S.C. § 631(a). All plaintiffs fall within the statutorily protected class. FOF^[25] ¶¶ 7, 9, 11, 13, 16, 18, 20. By the mere fact of their being GS-15/Band III's, plaintiffs were considered qualified for appointment to the ECDP and ultimate appointment to the SES. FOF ¶ 32. Plaintiffs were neither selected for the ECDP nor the SES. FOF ¶¶ 43, 47, 51, 56, 61, 65, 70. GS-15's under the age of 40 were appointed to the ECDP, and employees under the age of 40 were promoted to the SES. FOF ¶¶ 102, 103. Thus, plaintiffs have established a prima facie case, as the GAO itself admits.

As discussed above, the burden next shifts to the GAO to raise legitimate nondiscriminatory reasons why plaintiffs were not promoted. This is merely a "burden of production, not of persuasion." Cuddy, 694 F.2d at 857, 857 n. 21. See also, McDonnell Douglas, 411 U.S. at 802, 93 S.Ct. at 1824-25; Whitacre v. Davey, 890 F.2d 1168, 1170 (D.C.Cir.1989), cert. denied, 497 U.S. 1038, 110 S.Ct. 3301, 111 L.Ed.2d 810 (1990); Krodel v. Young, 748 F.2d 701, 705 (D.C.Cir. 1984), cert. denied, 474 U.S. 817, 106 S.Ct. 62, 88 L.Ed.2d 51 (1985).

In meeting this burden the GAO produced a plethora of nondiscriminatory reasons why plaintiffs were not promoted. These reasons are laid out in great detail in the Court's Findings of Fact. See generally FOF ¶¶ 43-73. Nevertheless, because this case hinges on these articulations, it proves necessary to reiterate some of these facts.

To properly frame the evidence before the Court, it is important to understand the mission of the ECDP and the SES. The comedian and film director Woody Allen is said to have remarked, "Eighty per cent of success is showing up."^[26] Not so the SES. The SES is not the next step on the career ladder for successful GS-15's. One does not merely progress GS-13, 14, 15 and SES. When an employee is promoted into the SES, he is taken into a whole other realm of responsibility and expectation. FOF ¶¶ 23-25. A GAO executive not only oversees a unit or division, and is responsible for its work product, he or she is the GAO's main representative with Congress, other agencies and the private sector. FOF ¶¶ 3, 4, 5, 23, 25.

The function of the ECDP is to train an elite group of GS-15's for eventual and near certain appointment to the SES. Therefore, the GAO is looking for people with sterling, indeed, almost flawless records. Perhaps even more importantly, they also seek individuals with unique interpersonal and communicative skills that will allow them effectively to supervise and motivate their employees as well as to represent the GAO to the outside world. FOF ¶ 27.

Of 4,600 GAO employees worldwide only 145 are SES. Of all those GS-15's eligible, only 2% will be appointed to the ECDP. Granted, many GS-15's do not apply but, nevertheless, these numbers indicate the extreme competition in winning one of these coveted spots. FOF ¶¶ 24, 29.

All of the plaintiffs were good, solid GS-15's. But, in this instance, good was not enough. Legitimate concerns were raised over the depth and complexity of Anthony Csicseri's GAO experience. Also brought forth were Mr. Csicseri's inadequate writing ability, his inability to complete projects on time and his apparent difficulty with interpersonal relationships. All of these deficiencies are disqualifications to becoming a successful executive, especially writing and interpersonal skills. FOF ¶ 45.

Jack Kearns often received mediocre performance appraisals which he admitted were not "exceptional across the board." His record, although solid, is not enough to vault him to the SES. FOF ¶ 49.

Julius Brown was passed over for a competitive SES position to run the Office of Administration and Publishing, which he had previously headed as a GS-15, in favor of a member of the SES from the Bureau of Engraving and Printing. At that time, the nature of the Office of Administration and Publishing at the GAO was changed, and Mr. Brown did not have the necessary technical skills and experience to run the revamped and modernized office. As an auditor, Mr. Brown typically supervised only one audit at a time while four to seven were the norm. Also, substantial criticism arose concerning one of the audits under Mr. Brown's supervision. FOF ¶¶ 52-54.

Manohar Singh is an engineer, not an auditor. And, although some exceptions were made, the ECDP is primarily a program for auditors. Moreover, Dr. Singh possessed deficient writing skills, had a problem explaining complex technical material to a lay audience and had interpersonal problems with his co-workers. FOF ¶ 58.

Robert Jaxel also had a work productivity problem, supervising only one audit at a time. Mr. Jaxel also spent long periods on detail, and his experience as a group director was limited and unremarkable. Furthermore, Mr. Jaxel's performance ratings can be described as solid but by no means stellar. FOF ¶ 63.

Frank Heard's generally good performance ratings are interspersed by several falling below the top categories. Furthermore, he was said to be deficient in the areas of conceptualization and oral communication, and he did not have enough exposure as a group director. Later, when Mr. Heard became a report reviewer, he reviewed only one to three reports in a six month period while others typically reviewed fifteen to twenty. FOF ¶¶ 67-68.

Ryan Yuille also generally received solid yet unexceptional performance ratings. In 1986, he received a ranking of "did not meet expectations" for failing to supervise properly the work product of a subordinate. Also, Mr. Yuille was not an auditor in the classic GAO sense, and there was legitimate concern over his ability and potential to direct several large-scale audits at one time. FOF ¶ 72.

Thus, the GAO has more than met its burden of producing legitimate nondiscriminatory reason as to why plaintiffs were not appointed to the ECDP and promoted to the SES. The burden, now heightened to a preponderance of the evidence standard, shifts back upon plaintiffs who must demonstrate that the GAO's reasoning was merely pretextual. Whitacre, 890 F.2d at 1170; Arnold, 863 F.2d at 996; Krodel, 748 F.2d at 705; Cuddy, 694 F.2d at 853. See also, McDonnell Douglas, 411 U.S. at 804, 93 S.Ct. at 1825.

In reality, there exists a two-part burden for plaintiffs at this stage. In addition to demonstrating that the GAO's reasons for not promoting them were pretextual, they must also prove, again by a preponderance of the evidence, the ultimate issue. Namely, they must prove that intentional age discrimination was a motivating factor in their not being promoted. Texas Department of Community Affairs v. Burdine, 450 U.S. 248, 253, 101 S.Ct. 1089, 1093-94, 67 L.Ed.2d 207 (1981). As the Supreme Court recently noted, "[A] reason cannot be proved to be `a pretext for discrimination' unless it is shown both that the reason was false, and that discrimination was the real reason." St. Mary's Honor Center v. Hicks, ___ U.S. ___, ___, 113 S.Ct. 2742, 2752, 125 L.Ed.2d 407 (1993) (emphasis in original).

Plaintiffs may demonstrate this by showing that a discriminatory reason more likely than not influenced the GAO. Id., 450 U.S. at 256, 101 S.Ct. at 1095. This has also been described as a "but for" analysis, in the sense that, but for the discriminatory motives, plaintiffs would have been promoted. Hayman, 23 F.3d at 538; Krodel, 748 F.2d at 706, citing Cuddy, 694 F.2d at 857, 858 n. 23. Ultimately, this Court must "decide which party's explanation of the employer's motivation it believes", just as it decides questions of fact in any other type of civil litigation. United States Postal Service v. Aikens, 460 U.S. 711, 716, 103 S.Ct. at 1478, 1482, 75 L.Ed.2d 403 (1983). See also, Krodel, 748 F.2d at 706-707 (applying the above principles to an ADEA context).

Plaintiffs brought forth much evidence in an attempt to demonstrate that the GAO's reasons for failing to promote them were merely pretextual. Plaintiffs, mainly through testimonial evidence, argued that certain successful younger ECDP candidates and SES promotees were less qualified than they. Likewise, the GAO presented evidence of certain successful younger ECDP and SES candidates who were better qualified than plaintiffs. Neither party presented evidence of the relevant qualifications of all those who applied for the ECDP or SES during the relevant time period, nor did they provide the qualifications of all successful candidates. The Court is reluctant on this small, largely testimonial sampling to try to infer that plaintiffs were among the top candidates in any given year, or for that matter, that they were not. FOF ¶ 117. Furthermore, the Supreme Court has counseled that it is not up to the defendant to prove that those hired were more qualified than the plaintiff. It always remains the plaintiff's task to show that similarly situated employees were not treated equally. Burdine, 450 U.S. at 258-259, 101 S.Ct. at 1096-97. See also, St. Mary's, ___ U.S. at ___, 113 S.Ct. at 2747 ("`[t]he ultimate burden of persuading the trier of fact that the defendant intentionally discriminated against the plaintiff remains at all times with the plaintiff ...'" quoting Burdine, 450 U.S. at 253, 101 S.Ct. at 1093). In presenting this limited evidence, plaintiffs have in no way established that they were more qualified than those chosen. FOF ¶ 117.

Plaintiffs also introduced into evidence several issues of the GAO Management News, which they argue show bias against older workers because it profiles successful candidates' ages. Plaintiffs' allegations are only half true. The GAO Management News ran statistical profiles of each new ECDP class. In addition to an average age and span of ages, the profile included statistical breakdowns of the candidate's divisions, years at the GAO, years at GS-15, education and experience. The references to age in this context were entirely innocuous and not evidence of discriminatory intent. FOF ¶¶ 118-19. See also Wilson v. Firestone Tire & Rubber Co., 932 F.2d 510, 514 (6th Cir.1991) (the mere keeping of information such as age or birthdays, absent direct evidence that it was used to make adverse employment decisions, cannot create even a circumstantial inference of discrimination).

More troubling is a handwritten memorandum between two support personnel assigned to the ERB. It lists, for one class, next to the successful ECDP candidates' names and divisions, their birthdays and ages. In weighing this evidence, however, the Court is cognizant that this memorandum was neither written by nor to promotion decision-makers. Id.

Plaintiffs also point to a 1986 internal review of the ECDP in which the GAO itself raised the specter of age discrimination. The report concludes, not that an affirmative action program is necessary, but rather that the criteria used to judge the candidates must be standardized, and unsuccessful candidates must be given feedback so that if, as here, the GAO is accused of impropriety, it will have specific standards to point to. As for the Equal Promotion and Review Board's ("EPRB") findings of racial discrimination within the GS ranks, it is of little bearing on this case. Age is only tangentially dealt with and, moreover, the scope of this review was limited to promotion within and between the different GS or Band grades. It in no way concerns promotions to or within the SES or the ECDP. Likewise, the 1989 report concerning the Boston Regional Office is limited to a promotion to GS-15 within that particular office. FOF ¶ 120.

The Court would also note that the EPRB, the 1986 ECDP report and the 1989 Boston Regional Office investigation, while damaging to defendant, also demonstrates that the GAO affirmatively seeks out and addresses discrimination within its ranks.

Plaintiffs also point to stray age-related remarks which befell two of their rank: specifically, Mr. Csicseri's being referred to as "old hat" in a handwritten comment on an ECDP score sheet, and Dr. Singh's being questioned about his retirement plans and asked why he does not wish to retire. FOF ¶¶ 59-60, 121.

The "old hat" statement is quite vague and can refer to many things in addition to age. It may mean Mr. Csicseri was a typical rather than outstanding GS-15, that his application read like the majority and did not stand out, or that they had seen his application before and found it less than impressive. FOF ¶ 120. Likewise, the statements made to Dr. Singh could be attributed to friendly conversation as many people look forward to retiring. They could also reflect personnel or staffing concerns on the part of his supervisors. FOF ¶¶ 59-60. Cf., Carter v. City of Miami, 870 F.2d 578, 582 (11th Cir.1989) (remarks around office such as "little old Jewish lady" and that a secretary was `too old and made too much money' are not direct evidence of discrimination). Such statements, however, are not wholly without evidentiary value. See, Normand v. Research Institute of America, Inc., 927 F.2d 857, 864 (5th Cir.1991); Siegel v. Alpha Wire Corp., 894 F.2d 50, 54-55 (3rd Cir.), cert. denied, 496 U.S. 906, 110 S.Ct. 2588, 110 L.Ed.2d 269 (1990).

As detailed in the Findings of Fact, both parties introduced copious, contradictory and, at times, confusing statistical evidence. See generally, FOF ¶¶ 74-116. Plaintiffs typically introduce such evidence to demonstrate a pattern or practice of discrimination in attempting to show pretext or intent. Krodel, 748 F.2d at 709, 710. See also, McDonnell Douglas, 411 U.S. at 802-805, 93 S.Ct. at 1824-25. Defendants typically introduce statistical evidence to rebut a plaintiff's statistics and to demonstrate that their hiring practices are age neutral.

The Supreme Court has stated that statistical evidence is helpful, but ordinarily not dispositive. See Furnco v. Waters, 438 U.S. 567, 580, 98 S.Ct. 2943, 2951, 57 L.Ed.2d 957 (1978). The Court has also written that such evidence is like any other kind of evidence. It may be rebutted, and its weight depends on all the surrounding facts and circumstances. Teamsters, 431 U.S. at 340, 97 S.Ct. at 1857.

Plaintiffs' expert, Dr. Charles Mann, performed an analysis of every stage of the ECDP process, as well as an overall analysis, for every class from 1981 through 1990. After adjusting his conclusion, as a result of an error, he determined that statistically significant instances of discrimination occurred in only 2 of 37 statistical comparisons. This conclusion was the same for analyses run for those under 40 versus those over 40, as well as for those under 40 versus those over 50. FOF ¶¶ 77-84. The figure 2 of 37 amounts to roughly 5% and is hardly indicative of a pattern of discrimination. FOF ¶ 85.

Recall also that the Court questioned Dr. Mann's use of the age spread of those under 40 versus those over 50. This leaves out all those between 40 and 50 who make up a large portion, if not the majority, of those applying to the ECDP and the SES. FOF ¶ 102. Dr. Mann also used a one-tail test in drawing his conclusions, when a two-tailed test is generally preferred in this circuit. See, Palmer v. Shultz, 815 F.2d 84, 92-96 (D.C.Cir.1987); FOF ¶ 104.

Dr. Mann then performed a multiple pool analysis by taking the data from all eight classes and aggregating it for each stage of the ECDP selection process. This methodology produced statistically significant results in two of five instances, for those under 40 versus those over 40, and statistically significant instances of discrimination in four of five instances for those under 40 versus those over 50. FOF ¶¶ 86-87.

Aggregate samples of data can be very useful in a statistical analysis because, generally, the more data a statistician has to work with the more accurate his results. When dealing with aggregate or multiple pools of data, however, statisticians generally perform some type of multivariant or multiple regression analysis to ensure the accuracy of their tests. By employing such an analysis they can isolate and account for certain variables that would skew their results. FOF ¶¶ 105-09.

It appears that Dr. Mann did perform some type of multivariant or regression analysis. But, in contrast to defendant's expert, Dr. Joan Haworth, Dr. Mann did not state what, if any, variables he used. In his testimony he does make passing references to complicated mathematical tests but, goes no further. It is crucial to know what variables are included in such an analysis for the inclusion or exclusion of certain variables can have a profound affect on the final result. Id.

Dr. Mann also analyzed selection into the SES. He used only the years 1984-1987 because he felt that the GAO changed its promotion policy with the filing of this lawsuit in 1988. As discussed in FOF ¶¶ 111-12, the Court gives little credence to that theory. In any event, Dr. Mann's results indicate statistically significant discrimination. Defendant's expert ran the same test, with the same numbers for all of the applicable years possible, 1984-1990, and concluded that there was no statistically significant discrimination. If the years 1988-1990 were to be added to Dr. Mann's analysis, he would arrive at the same result as Dr. Haworth. FOF ¶¶ 111-13.

Not surprisingly, defendant's expert, Dr. Haworth, found no instances of statistical discrimination in any of her methodology. FOF ¶¶ 95, 97. Like Dr. Mann, some questions arose concerning her analyses. FOF 107-16. However, the Court was generally more impressed with Dr. Haworth. FOF ¶ 100. She laid out in great detail the variables she used in her logit regression analysis, thereby giving the Court a larger and clearer picture of how her analyses took shape. Thus, the Court could be more assured that nothing lurked beneath the numbers to significantly skew her results. FOF ¶¶ 89-93.

Also, the Court preferred her use of the age spread under 45 versus over 45. Although it too was not under 50 versus over 50, she explained that there were so few applicants over 50 that it would be possible to choose none and remain within the realm of statistical possibility. Therefore, in order to see if older members of the protected class were treated differently from younger class members, she settled on a cut off at 45. FOF ¶¶ 102-03. The Court also found her use of two-tail tests superior to Dr. Mann's use of one-tail tests. FOF ¶ 105; Palmer, 815 F.2d at 92-96.

Although the Court found both Dr. Mann and Dr. Haworth to be credible, and gave evidentiary weight to both of their conclusions, for the reasons laid out in detail in the Findings of Fact and highlighted above, the Court found Dr. Haworth's analysis to be more credible and more persuasive. FOF ¶ 100.

The Supreme Court has recently stated, "[T]here is no disparate treatment under the ADEA when the factor motivating the employer is some feature other than the employee's age." Hazen Paper Co. v. Biggins, ___ U.S. ___, ___, 113 S.Ct. 1701, 1705, 123 L.Ed.2d 338 (1993). The GAO presented significant and credible nondiscriminatory reasons why plaintiffs were not appointed to the ECDP or promoted to the SES. As the discussion above indicates, plaintiffs failed to demonstrate by a preponderance of the evidence that the GAO's reasons were pretextual and that age was a motivating factor, or any factor whatsoever, in the GAO's promotion decisions. Therefore, this Court must find that plaintiffs did not suffer disparate treatment.^[27]

B. Disparate Impact

The GAO has argued the case as both a disparate treatment and disparate impact case, devoting a great deal of their post trial Proposed Conclusions of Law to a disparate impact analysis. The Court, however, is not persuaded that such an analysis is necessary. After examining the Amended Complaint at length, as well as the Court's recollection of the tenor of plaintiffs' argument, the Court does not believe that plaintiffs intended to raise a disparate impact claim.

Plaintiffs certainly argued throughout the trial that the GAO willfully and intentionally discriminated against older GS-15's. They do allege an ongoing pattern or practice of discrimination, but that goes toward establishing their disparate treatment claim. Plaintiffs never specifically articulated the elements of a disparate impact claim. Nowhere did they argue that the ECDP or SES selection processes were neutral on their face but discriminatory as applied.

Furthermore, it has not been settled that disparate impact claims can be brought under the ADEA. Hazen, ___ U.S. at ___, 113 S.Ct. at 1706 ("[W]e have never decided whether a disparate impact theory of liability is available under the ADEA ..."); id., at ___, 113 S.Ct. at 1710 (Kennedy, J. concurring) ("[N]othing in the Court's opinion should be read as incorporating in the ADEA context the so-called `disparate impact' theory ... there are substantial arguments that it is improper to carry over disparate impact analysis from Title VII to the ADEA."). See also Markam v. Geller, 451 U.S. 945, 948, 101 S.Ct. 2028, 2030, 68 L.Ed.2d 332 (1981) (Rehnquist, J., dissenting) ("[T]he Court has never held that proof of discriminatory impact can establish a violation of the ADEA ..."). This circuit has not ruled on the issue. See Arnold, 863 F.2d at 998 (noting the Supreme Court's indecision but declining to decide if a disparate impact claim can be raised in an ADEA suit). Those circuits that have ruled, however, have decided that disparate impact claims can be brought under the ADEA. See, e.g., Lowe v. Commack Union Free School District, 886 F.2d 1364, 1369 (2nd Cir.1989), cert. denied, 494 U.S. 1026, 110 S.Ct. 1470, 108 L.Ed.2d 608 (1990).

In any event, this Court does not believe that plaintiffs established or even could have established a prima facie claim of disparate impact. The Supreme Court, in Watson v. Fort Worth Bank and Trust, 487 U.S. 977, 108 S.Ct. 2777, 101 L.Ed.2d 827 (1988), articulated the elements, as well as the proper analysis of these claims. Of special significance to the present case, the Court was specifically addressing disparate impact in the context of promotion decisions based upon the exercise of personal judgment or the application of subjective criteria. Id., 487 U.S. at 984-85, 991, 108 S.Ct. at 2783-84, 2786. C.f., Griggs v. Duke Power Co., 401 U.S. 424, 91 S.Ct. 849, 28 L.Ed.2d 158 (1971) (disparate impact of objective criteria). Although much attention was paid at trial to the objective criteria enumerated in Plaintiffs' Exhibit 4(a) and 4(b), qualification requirements from the ECDP Job Opportunity Announcement, this case, nevertheless, revolves around the subjective application of that criteria. FOF ¶¶ 30-31.

Having established the applicability of the analysis to subjective employment decisions, the Court turned to the evidentiary standards to apply. Watson, 487 U.S. at 991, 108 S.Ct. at 2786. Plaintiffs must begin by identifying the specific employment practices they wish to challenge. Moreover, they are charged with isolating the specific employment practices allegedly responsible for any statistical disparities. Id., 487 U.S. at 994, 108 S.Ct. at 2788-89.

Plaintiffs have failed to isolate and identify any specific GAO practices which they feel cause discriminatory impact. Granted, they ran statistical analyses of every stage of the ECDP selection process, but they do not identify any stage as being in and of itself discriminatory. Likewise, at no time did plaintiffs state that the criteria enumerated in their Exhibits 4(a) and 4(b) were the manifestations of discrimination.

IV. CONCLUSION

Based upon the foregoing Findings of Fact and the entire record herein, it is clear that plaintiffs have not sustained their burden of proof. Moreover, defendant has established that it had legitimate, nondiscriminatory reasons for its conduct, and plaintiffs have in no way established that defendant's conduct was pretextual. Accordingly, the Court will enter an order consistent with this opinion, entering judgment for defendant.

NOTES

[1] During periodic reorganizations, the GAO changes the names and functions of its divisions. Therefore, as plaintiffs' lengthy careers are discussed in this opinion, the names of the divisions often will not be the same as those enumerated in ¶ 3. Those divisions listed in ¶ 3 represent the divisions in existence at the time of trial and the current structure of the GAO.

[2] About 1990, the GAO adopted a banding personnel classification system consisting of three bands. All Band III employees, the highest band, were classified as GS-15's under the prior system. As at trial, the terms GS-15 and Band III are used interchangeably. Tr.Vol. I 42-44.

[3] Mr. Csicseri took advantage of an early retirement program at the GAO. Under this program, if those eligible to retire did so by the end of 1993, they would receive a $25,000 payment. Plaintiffs Kearns and Brown also retired under this program. Tr.Vol. II 236, 285, 341, 369.

[4] Occasionally, the ERB would also consider a candidate who was not forwarded by the Q & PRB.

[5] The numbers in parenthesis refer to the ECDP Job Opportunity Announcement numbers.

[6] Tellingly, a memorandum dated August 5, 1983 from W.D. Campbell to Mr. Csicseri, congratulating him on his Outstanding Achievement Award, mentions that only seven per cent of GAO employees win this coveted honor. Compare that number with the mere two per cent of those eligible who will go on to the ECDP. Pl.Ex. 33; Tr.Vol. I 106; Tr.Vol. II 194-99, 240, 243-44.

[7] In the early 1980's, GAO employees were evaluated on a five-prong scale of "outstanding", "high", "average" and "low". Later, the GAO evaluated people on a chart which allowed one to check "exceeded expectations", "met expectations", "did not meet expectations" or "no basis for evaluation". Thus, "met expectations" is the middle of three possible scores for those with anything to say about an employee. In 1989, it appears, the GAO went to an expanded system using five gradations ranging from "unacceptable" to "exceptional", or later "outstanding". Tr.Vol. V 881. Under this system, Mr. Kearns was consistently rated as "fully successful" or "superior". He rarely received the highest rating of "exceptional". Pl.Ex. 66-74.

[8] Data for unit nominations are only available after 1984 as discussed in ¶ 76.

[9] The Fisher exact test is used where the statistician wants to test a hypothesis in which the sample is relatively small, generally where the pre-selection pool is less than ten times as large as the post-selection pool, and that the hypothesis tested is that the majority and minority selection rates are equal. Baldus, D. & Cole, J., Statistical Proof of Discrimination (McGraw Hill, 1980) (§ 9A.11-.12); see also, E. Shoben, Differential Pass-Fail Rates in Employment Testing: Statistical Proof Under Title VII, 91 Harv.L.Rev. 793, 812 (1978).

[10] The multiple pool or exact probability analysis refers to the totality of data in all eight pools, which are the eight ECDP classes between 1981 and 1990. Statisticians use this method to increase the amount of data they have to work with and thereby, in theory, come up with a more accurate result. This method, however, is not without its pitfalls, as discussed in ¶¶ 105-06, infra. Tr.Vol. IV 585-86.

[11] The Chi-square test is an approximation of the Fisher exact test commonly used when there is a large sample pool. Finkelstein, M. & Levin, B., Statistics for Lawyers, 159 (Springer-Verlag 1990).

[12] Palmer v. Shultz, 815 F.2d 84, 92 (D.C.Cir. 1987).

[13] It is noted, also, that data are not available for the nomination stage of the selection process until 1985. See ¶ 76.

[14] See Segar v. Smith, 738 F.2d 1249, 1261 (D.C.Cir.1984) (discusses multiple regression analysis).

[15] Apparently, all the information needed to perform the analyses was not available from 1981 to 1983. Def.Ex. 8; Tr.Vol. IV 670-71.

[16] Neither Dr. Haworth nor Dr. Mann calculated the standard deviation for each year. Rather, they calculated the standard deviation for the entire period examined (1984-1990 for Dr. Haworth and 1984-1987 for Dr. Mann).

[17] The allegation that the GAO changed its hiring practices in 1987 will be discussed further at ¶¶ 111-12, infra.

[18] The figure -2.663 reflects no continuity correction. Dr. Mann calculated a standard deviation of -2.824 with the continuity correction factor added to his analysis. The dispute concerning the use of the continuity correction factor is discussed at ¶ 114, infra.

[19] The Court is equally aware that less obvious instances of discrimination can appear fairly insignificant when parceled into discrete units although, in fact, if taken as a whole, the discrimination is fairly profound.

[20] It is clear that Dr. Mann is no stranger to multiple regression having himself used such an analysis in, at the very least, the cases of Lewis v. Bloomsburg Mills, 773 F.2d 561 (4th Cir.1985), and Bazemore v. Friday, 751 F.2d 662 (4th Cir. 1984).

[21] For example, including level of education as a variable in a multiple regression analysis studying racial discrimination in hiring may be invalid because minorities might not have had the same level of access to higher education as non-minorities.

[22] A continuity correction factor is used with samples of less than 100. It is a measure used to take into account that one is looking at discrete things but using a continuous, bell-shaped, curve. The correction is made by reducing the absolute difference between the observed and expected values by ½ unit, so that the difference between observed and expected is moved closer to zero. This correction is not applied if the observed value already differs from the expected value by less than ½ unit. Tr.Vol. IV 593-594; Finkelstein and Levin at 121.

[23] Dr. Mann's final conclusion in this section, which again only reflects the years 1984-1987, is raised from a standard deviation of -2.663 to -2.824 with the continuity correction added.

[24] Section 633a(c) is merely a mechanism by which aggrieved federal employees may bring action.

[25] Finding of Fact.

[26] Bartlett's Familiar Quotations, 16th Edition (J. Kaplan, ed., Little Brown 1992) at 767 1.17.

[27] Because the Court finds that age played no role in the GAO's decisions, an analysis under Price Waterhouse v. Hopkins, 490 U.S. 228, 109 S.Ct. 1775, 104 L.Ed.2d 268 (1989), is not necessary.