83 Fed. Cl. 643 | Fed. Cl. | 2008
OPINION
This pre-award bid protest is before the court on cross-motions for judgment on the administrative record filed under Rule 52.1(b) of the Rules of the United States Court of Federal Claims (RCFC).
EOTech filed its complaint on July 15, 2008. An administrative record (AR) was filed on July 22, 2008, and was supplemented on July 25, 2008. The parties’ cross-motions have been fully briefed, and oral argument was held on August 11, 2008. The court and counsel observed a demonstration of the securing, utilization and removal of three optical rifle sights on two rifles on August 5, 2008. For the reasons discussed below, plaintiffs motion is granted, and the Army is enjoined from proceeding with this procurement in accordance with its competitive range determination.
BACKGROUND
I. Solicitation
A. Solicitation Overview
The optical rifle sight, or close combat optic (CCO), sought by the Army in this solicitation must be compatible with both the M4 carbine and M16 series rifles. AR at 1. Aimpoint has been the incumbent contractor supplying CCOs for this purpose. Id. One goal of this procurement is to encourage competition among potential sources for CCOs, and, if possible, to award two contracts so that the Army will no longer rely on just one company to fill its demand for CCOs. Compare id. at 7 (“The Government cannot risk having only one vendor.”) with id. at 8 (“However, if two awards are not possible based on best value and cost, the Government reserves the right to award 100 percent of the entire contract, subject to availability of funds, to only one source.”).
The Army issued the solicitation on August 2, 2007. AR at 69. Proposals were due on September 4, 2007. Id. at 335. Bidders were “responsible for including sufficient details, in a concise manner, to permit a complete and accurate evaluation of each
The statement of work specifies that the CCO must be mountable both on the M4 and the M16, “with a separate piece of mounting hardware for use on the M16A2 Rifle’s carrying handle.” AR at 80. Of particular interest is the instruction given to bidders as to the submission of bid samples:
The offeror shall provide twenty (20) identical and complete Bid Samples that meet the Essential Criteria and Rated Criteria in Section M____ Each bid sample shall include a lens/dust cover, the necessary hardware for attachment to the various weapons ... and Commercial Off-The-Shelf (COTS) manuals for the items submitted for evaluation.
Id. at 134. The solicitation also specifies that “the Government intends to evaluate proposals and award a contract without discussions (except clarifications as described in FAR 15.306(a)). However, the Government reserves the right to conduct discussions if the Contracting Officer later determines that discussions are necessary.” Id.
B. Evaluation Criteria
Section M of the solicitation gives both general and very specific information as to the evaluation of proposals. The solicitation indicates that “award will be made based on the best overall (i.e. best value) proposal that is determined to be the most beneficial to the Government.” AR at 141. Bidders are “cautioned that the award may not necessarily be made to the lowest price proposal or the most highly rated proposal.” Id. As to the details of the evaluation process, the Army describes six evaluation factors, ranked in descending order of importance:
(1) Bid Sample Testing (significantly more important than the next factor)
(2) Quality System Plan
(3) Equipment Facilities and Production Rates
(4) Evaluated Price
(5) Performance Risk (including past performance)
(6) Small Disadvantaged Business Participation.
Id. at 141-52. This bid protest largely concerns bid sample testing.
The Bid Sample Testing factor is further divided into two subfaetors, Essential Criteria, which constitutes the first phase of testing and which includes fifteen testing components, and Rated Criteria, the second testing phase, which includes seven testing components. AR at 142-46. The first phase of the testing could be described as an elimination round, because “[a] failure in any one or more of the [fifteen] essential criteria as stated shall be cause for elimination from further consideration for award and offer- or[’]s submission will not be further evaluated. No extra credit will be awarded for exceeding the pass/fail requirements, in this category.” Id. at 142. If a bid sample makes it to the second round of testing for the seven Rated Criteria, each of the seven criteria produces a rating such as Excellent, Good, Satisfactory or Unsatisfactory. Id. at 144-46. These seven individual ratings in the second phase of testing are amalgamated into an “overall color rating” of Blue, Green, Yellow or Red. Id. at 146.
The fifteen Essential Criteria in the elimination round, many of which are not especially relevant to this bid protest, are: (1) Magnification, (2) Clear Aperture, (3) Dot Angular Spot Size, (4) Parallax, (5) Azimuth and Elevation Adjustments, (6) Housing, (7) Mounting, (8) Weight, (9) Power Source, (10) Endurance-Live Fire, (11) Reattachment, (12) Illuminance, (13) Equipment Compatibility/Operability, (14) Height Above Bore, and (15) Environmental Testing (for Temperature Extremes, Sand and Dust, Thermal Shock, and Sealing). AR at 142-44. Mounting, Endurance-Live Fire and Reattaehment are the Phase I testing criteria that are the most important here. As to Mounting,
The bid sample sight shall be attachable and detachable to the [M4] rail without the use of special tools[ ]. Additional hardware*646 must be provided for mounting the optic on the M16A2 rifle carry handle without using special tools----Special tools is defined as tools not contained in the armorer tool kit____ Adapting the optic mount(s) and/or the optic to either the [M4] rail or the M16A2 carrying handle may use special tools such as a knife, hex key or similar tools; at the armorer level only. Attachment and detachment of the optic to the weapon by the operator in the field shall require no special tools.
Id. at 142. As to Endurance-Live Fire testing,
The bid sample shall be mounted on the M16A2 Rifle and shall withstand a 6,000 round endurance firing with no physical damage and maintain zero within 1 Gunner’s mil after 2,000 rounds. This test shall also be performed on the M4 Carbine.
Id. at 178. Finally, as to Reattachment,
The sight shall be mounted to a M16 rifle carry handle, and, separately, a[M4] rail. These combinations shall be separately mounted to equipment capable of measuring angular deviations as small as 10 are-seconds. A reference zero shall be recorded. The sight shall be cycled on and off the mount (M16 rifle carry handle itself, and [M4] rail itself, independently), ten (10) times. The line of sight shall be maintained within 0.33 mils.
Id. at 143.
The seven Rated Criteria of the second phase of testing are: (1) Battery Performance, (2) Battery Type, (3) Shock, (4) Drop, (5) Signature, (6) Applied Force, and (7) Human Systems Integration. AR at 144-45. Detailed discussion of these criteria is not necessary at this point, other than noting that the Rated Criteria include further endurance testing, and measuring the ease of use of the CCO by the average soldier, an area of evaluation the Army terms Human Systems Integration (HSI). Id. at 144^46. These criteria could produce an overall color rating ranging from Blue, where the sample “meets all and exceeds several requirements [and] [essentially no doubt exists that the offeror will successfully provide a product that meets the Government’s requirements,” to Red, where the “[o]fferor fails to demonstrate the ability to meet several requirements [and] [significant doubt exists that the offeror will provide a product that meets the Government’s requirements.” Id. at 146.
C. Attachment Instructions
The product information in the solicitation instructs that the CCO should be attachable to either the M4 rail or the M16A2 carry handle without using special tools. AR at 159. Specifically noted was a concern that damage from overtightening during attachment should be avoided. See id. (“Attachment of the sight to either weapon shall not damage the ... rail or mount due to over-tightening to include deforming, nicking or scratching the rail or the mount.”). Attachment also must be practical for the average soldier, in a variety of attire, “without the use of tools.” Id. According to the product specifications, the CCO, when attached to the rifle, must withstand shock, vibration, live fire and drop testing. Id. at 160,162.
II. Results of the Testing of CCO Bid Samples
A. Overall Results
Seven proposals were due and were received on September 4, 2007 from four companies: Aimpoint [ ], EOTech, [ ] and [ ]. AR at 190-91, 335. Aimpoint A passed the Essential Criteria phase of testing; all other proposals failed the Essential Criteria sub-factor, and were ehminated from the competition. Id. at 190-91. Aimpoint A’s scores on the Rated Criteria were mixed, meeting some of the seven criteria but getting [] in [ ]. Id. at 191.[ ] and therefore, [ ]. Id. at 192. “The technical team also recommend[ed][ ].” Id. The technical team concluded that “Aim-point A represents the best value to the U.S. Government as the best material solution for the CCO.” Id.
B. Certain Significant Results in Essential Criteria Testing
The Essential Criteria testing procedures, in many respects, appear to conform to the testing description provided in the solicitation. AR at 195-201. Only Aimpoint A
Four of the CCO samples passed most of the Essential Criteria, but encountered difficulties withstanding extended firing of the weapons or in reattachment tests. Two bid samples, EOTech and [], failed only one Essential Criteria, the Endurance-Live Fire test on the M16A2. [] failed two Essential Criteria each: [ ] failed Endurance-Live Fire on the M4 and Reattachment on both rifles; [ ] failed Endurance-Live Fire on both rifles and Reattachment on the M4. A cursory review of the testing data suggests that the extent to which the CCOs could remain securely attached to the weapon and correctly aligned was, in essence, the “do or die” criteria in the bid sample testing, at least for five of the seven samples.
C. Certain Significant Results in Rated Criteria Testing
The testing procedures undertaken for the Rated Criteria appear to conform to the testing description given in the solicitation.
Aimpoint A also received a[ ] rating, the [ ] rating in the scale, in Human Systems Integration, which focuses on comfort, ease of use and similar concerns of the average soldier. AR at 205-06. In addition, Aimpoint A received three [ ] ratings in [ ] and [ ]. Id. at 204, 206. The Army technical team gave Aimpoint A’s Rated Criteria an overall color rating of [ ], the [ ] rating in the scale, id. at 191, which represents that:
Offeror’s Bid Sample Test demonstrate^ all
Id. at 146. Despite this definition of [] provided by the solicitation, the technical team concluded, somewhat incongruously: [ ]. Id. at 192; compare id. at 146 ([ ] rating for Rated Criteria as defined by solicitation) ([ ]) with id. ([ ] rating for Rated Criteria as defined by solicitation) ([ ]).
D. Human Systems Integration—a Comparison of Aimpoint A, [ ] and EOTech
[ ] and EOTech failed the Essential Criteria phase of testing. The solicitation indicates that any bid samples which failed the Essential Criteria phase of testing would be eliminated from further consideration and perforce would not be included in Rated Criteria testing. AR at 339. Nonetheless, when Aimpoint A was tested for Human Systems Integration (HSI), one of the Rated Criteria, [] and EOTech were also tested.
During HSI testing, some [ ]. AR at 251. Soldiers also complained of []. Id. at 252. The primary difference between Aimpoint A
Various problems were also encountered in the use of the EOTech sample. []. AR at 285.[ ]. Id. at 311.
Although the Human Systems Integration (HSI) testing was, according to the solicitation, only relevant for Aimpoint A’s bid sample testing, apparently one analyst reviewing this data, Mr. [], of the Army’s Human Resources and Engineering Directorate (HRED), believed that the end-user testing conducted by the Army also produced ratings for [ ] and EOTech. AR at 333-34. According to this analyst, all three samples were “rated on the [HSI subfactors] and rank-ordered based on an analysis of the objective (where appropriate) and subjective data collected.” Id. at 333.[ ]. Id. Mr. [] concluded that Aimpoint A and [ ] were both [ ]. Id. at 334. Mr. [] also commented that “[o]f the two candidates rated [], candidate [] votes and would be the logical choice under the Human Systems Integration section.” Id.
E. Competitive Range Determination
“After evaluation of the seven (7) proposals, the only successful proposal for CCO solicitation W15QKN-07-R-0428 (the only proposal to Pass all essential criteria) is Aim-point ... A.” AR at 349. The Army decided on March 17, 2008 that Aimpoint A constituted a competitive range of one. Id. at 351. The Army also decided to hold discussions with Aimpoint regarding, first, Aimpoint A’s [] ratings in [], requesting []. Id. at 350. Additional discussions would be held regarding the [ ], with the Army preferring that the []. Id. In the end, however, []. Id. Finally, the Army would engage Aimpoint in discussions hoping to [ ].
The Army concluded that []. AR at 351. After the conclusion of discussions, Aimpoint would submit the “Contractor’s Final Proposal Revision” and the Army would evaluate that proposal. Id. Aimpoint’s “final proposal revision w[ould] be a paper proposal only, no new bid samples w[ould] be required.” Id.
F. Protests of the Army’s Competitive Range Determination
On March 17, 2008, EOTech was informed that it had failed Endurance-Live Fire Essential Criteria testing and had been eliminated from the competition. AR at 352. Debriefing was held on March 31, 2008. Id. at 353-72. On April 3, 2008, EOTech protested to the Government Accountability Office (GAO). Id. at 373-426. At the risk of oversimplification, EOTech questioned the validity of the testing of its bid sample, in part because the Army failed to properly “[secure the locking nut” which attached EOTeeh’s gooseneck mount to the carry handle of the M16A2. See id. at 426. GAO denied the protest, reasoning that “the agency or user could reasonably conclude that hand-tightening [EOTech’s] knurled ridge locking nut was the appropriate method to secure the sight mount.” Id. at 654.
GAO characterized EOTeeh’s instructions for securing its optic mount to the M16A2 as “inadequately written.” AR at 655 n. 7. Because of this flaw in the proposal, GAO concluded that the Army was under no obligation to conduct clarifications with EOTech concerning the failure of the Endurance-Live Fire test. Id. GAO also did not find that the Army was required to hold discussions with EOTech under these circumstances. Id. Finally, GAO held that the Army made no error in eliminating EOTech from the competitive range and in pursuing modifications, through discussions, of Aimpoint A. Id. at 655-56. On July 15, 2008, one day after GAO denied its protest, EOTech filed a bid protest complaint in this court.
EOTech relies on three principal arguments to support its request for injunctive relief. First, plaintiff alleges that the Army’s testing methodology was flawed. Pl.’s Mem. at 2. Second, EOTech asserts that the Army should have engaged in clarifications with EOTech concerning the proper attachment of its mount to the M 16A2. Id. Third, EOTech contends that the Army’s treatment of Aim-point and EOTech was “grossly disparate.” Id. This final argument includes several sub-parts, which highlight, for example, the cre
DISCUSSION
I. Jurisdiction
The Tucker Act provides the United States Court of Federal Claims with bid protest jurisdiction. 28 U.S.C.A. § 1491(b)(1)-(5) (West 2006 & Supp.2008 & Supp. July 2008); Am. Fed’n of Gov’t Employees v. United States, 258 F.3d 1294, 1300 (Fed.Cir.2001) (AFGE); Hunt Bldg. Co. v. United States, 61 Fed.Cl. 243, 268-69 (2004). The statute explicitly provides that this court “shall have jurisdiction to render judgment on an action by an interested party objecting to a solicitation by a Federal agency for bids or proposals for a proposed contract or to a proposed award or the award of a contract or any alleged violation of statute or regulation in connection with a procurement or a proposed procurement.” 28 U.S.C. § 1491(b)(1). The statute also states that the Court of Federal Claims “shall have jurisdiction to entertain such an action without regard to whether suit is instituted before or after the contract is awarded.” Id. Accordingly, this court has subject matter jurisdiction to entertain this bid protest, unless some other impediment to jurisdiction prevents consideration of plaintiffs suit.
II. Standards of Review
A. Judgment on the Administrative Record
RCFC 52.1(b) provides for judgment on the administrative record. To review a motion, or cross-motions, under RCFC 52.1(b), the court asks whether, given all the disputed and undisputed facts, a party has met its burden of proof based on the evidence in the record. Bannum, Inc. v. United States, 404 F.3d 1346, 1356 (Fed.Cir.2005). The court must make fact findings where necessary. Id. The resolution of RCFC 52.1 cross-motions is akin to an expedited trial on the paper record. Id.
B. Bid Protest Review
As a threshold jurisdictional matter, the plaintiff in a bid protest must show that it has standing to bring the suit. Info. Tech. & Applications Corp. v. United States, 316 F.3d 1312, 1319 (Fed.Cir.2003) (IT AC). This may be accomplished by demonstrating that the plaintiff was an actual bidder and that it was prejudiced by the award to the successful offeror. Id. (citing AFGE, 258 F.3d at 1302). Prejudice is proven by establishing that the plaintiff had a substantial chance of receiving the contract, but for the alleged procurement error. Id. (citing Alfa Laval Separation, Inc. v. United States, 175 F.3d 1365, 1367 (Fed.Cir.1999)). Standing is an element of this court’s jurisdiction over bid protest cases. See Rex Serv. Corp. v. United States, 448 F.3d 1305, 1307-08 (Fed.Cir.2006) (upholding a dismissal for lack of jurisdiction because the protestor was not an actual bidder on the disputed contract and could not show prejudice, i.e., that it had a substantial chance of receiving the contract but for the alleged procurement errors).
As the United States Court of Appeals for the Federal Circuit has stated, “the proper standard to be applied in bid protest cases is provided by 5 U.S.C. § 706(2)(A) [ (2006) ]: a reviewing court shall set aside the agency action if it is ‘arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law.’ ” Banknote Corp. of Am. v. United States, 365 F.3d 1345, 1350-51 (Fed.Cir.2004) (Banknote) (citing Advanced Data Concepts, Inc. v. United States, 216 F.3d 1054, 1057-58 (Fed.Cir.2000)); see also 28 U.S.C. § 1491(b)(4) (describing this court’s standard of review for bid protests). Under this standard, a procurement decision may be set aside if it lacked a rational basis or if the agency’s decision-making involved a violation of regulation or procedure. Impresa Construzioni Geom. Domenico Garufi v. United States, 238 F.3d 1324, 1332 (Fed.Cir.2001) (citations omitted). De minimis errors in the procurement process, however, do not justify relief. Grumman Data Sys. Corp. v. Dalton, 88 F.3d 990, 1000 (Fed.Cir.1996) (cit
The higher the degree of discretion allotted the contracting officer, the more difficult it is for a protestor to prove that the procurement decision was arbitrary and capricious. Burroughs Corp. v. United States, 223 Ct.Cl. 53, 617 F.2d 590, 597 (1980) (citing Keco Indus., Inc. v. United States, 203 Ct.Cl. 566, 492 F.2d 1200, 1204 (1974)). Negotiated procurements give a “breadth of discretion” to the contracting officer, and impose a heavier burden of proof on a protestor. Id. at 598 (citing Keco, 492 F.2d at 1204). Similarly, “best value” contract awards give a contracting officer more discretion than awards based on price alone. Galen Med. Assocs., Inc. v. United States, 369 F.3d 1324, 1330 (Fed.Cir.2004) (citing E.W. Bliss Co. v. United States, 77 F.3d 445, 449 (Fed.Cir.1996)). Thus, the protestor’s burden is especially heavy in negotiated, best value procurements. Banknote Corp. of Am. v. United States, 56 Fed.Cl. 377, 380 (2003) (citations omitted), aff'd, 365 F.3d 1345 (Fed.Cir.2004).
The deference afforded to an agency’s decision must be even greater when a trial court is asked to review a technical evaluation. “[TJechnical ratings ... involve discretionary determinations of procurement officials that a court will not second guess.” E.W. Bliss, 77 F.3d at 449 (citations omitted); Omega World Travel Inc. v. United States, 54 Fed.Cl. 570, 578 (2002) (“It is well settled that contracting officers are given broad discretion with respect to evaluation of technical proposals.” (citing E.W. Bliss, 77 F.3d at 449)). “[W]here an agency’s decisions are highly technical in nature, ... judicial restraint is appropriate and proper.” ElectroMethods, Inc. v. United States, 7 Cl.Ct. 755, 762 (1985) (citing Isometrics v. United States, 5 Cl.Ct. 420, 423 (1984)).
“ ‘If the court finds a reasonable basis for the agency’s action, the court should stay its hand even though it might, as an original proposition, have reached a different conclusion as to the proper administration and application of the procurement regulations.’ ” Honeywell Inc. v. United States, 870 F.2d 644, 648 (Fed.Cir.1989) (quoting M. Steinthal & Co. v. Seamans, 455 F.2d 1289, 1301 (D.C.Cir.1971)). If, on the other hand, “the trial court determines [that] the government acted without rational basis or contrary to law when evaluating the bids and awarding the contract!,] ... it proceeds to determine, as a factual matter, if the bid protester was prejudiced by that conduct.” Bannum, 404 F.3d at 1351. Plaintiff again bears the burden of proof, and must “show that there was a ‘substantial chance’ [plaintiff] would have received the contract award but for the [government’s] errors in the bid process.” Id. at 1358 (citations omitted). If a protestor can show that, but for the procurement error of the agency, there was a substantial chance that it would have won the contract award, prejudice has been established. Id. at 1353 (citations omitted). “Prejudice is a question of fact.” Id. (citing Advanced Data Concepts, 216 F.3d at 1057).
C. Review of Competitive Range Determinations
According to the Federal Acquisition Regulation (FAR), when agencies intend to hold discussions with offerors in negotiated procurements:
Agencies shall evaluate all proposals in accordance with [FAR] 15.305(a), and, if discussions are to be conducted, establish the competitive range. Based on the ratings of each proposal against all evaluation criteria, the contracting officer shall establish a competitive range comprised of all of the most highly rated proposals____
FAR 15.306(c)(1), 48 C.F.R. § 15.306(c)(1) (2007). This court has interpreted the language of FAR 15.306(c)(1) to require close scrutiny when an agency establishes a competitive range of only one offeror, thus allowing that offeror to modify its proposal through discussions, while denying this opportunity to other offerors. E.g., Int’l Outsourcing Servs., LLC v. United States, 69 Fed.Cl. 40, 51 n. 12 (2005); Bean Stuyvesant, L.L.C. v. United States, 48 Fed.Cl. 303, 339 (2000) (citation omitted). Where, as here,
III. Standing
The parties do not dispute EOTech’s standing to bring this bid protest.
IV. Whether the Army’s Competitive Range Determination and Proposed Discussions with Aimpoint are Arbitrary and Capricious or Contrary to Law
The court turns first to plaintiffs contention that Ampoint and EOTeeh received, or will receive, disparate treatment. This allegedly disparate treatment is the result of the Army’s decision to create a competitive range of one offeror, which necessarily permits Ampoint to revise Aimpoint A through discussions, and the Army’s decision to permit the reconfigured Aimpoint A to proceed to award without being subjected to Endurance-Live Fire Essential Criteria testing. As discussed below, the court views this series of decisions by the Army to constitute a relaxation of the solicitation requirement that eliminated EOTeeh from further consideration. Furthermore, there were enough departures from the evaluation plan described in the solicitation to cast a shadow upon this procurement, when these irregularities are properly placed in context. Finally, the Army’s decision to secure EOTech’s goose-neck mount to the M16A2 by finger-tightening cannot be judged to be fundamentally fair or rational, [ ]. The cumulative effect of these procurement errors invalidates the Army’s competitive range determination and its proposed plan to select the modified Aim-point A without a retest of the Endurance-Live Fire criteria. If these procurement procedures were allowed to stand, the Army’s upcoming “best value” decision would be fundamentally flawed, arbitrary and capricious, and would not reflect full and open competition.
A. Disparate Treatment
“[TJhere is a presumption ... that government [procurement] officials act in good faith.” Galen Medical, 369 F.3d at 1335 (citing Ami-Pro Protective Agency, Inc. v. United States, 281 F.3d 1234, 1239 (Fed.Cir.2002)) (emphasis removed). Even when acting in good faith, however, agency officials may fail to “evaluate ... competitive proposals and make an award based solely on the factors specified in the solicitation.” 10 U.S.C. § 2305(b)(1) (2006). Here, the court considers the proposed award to Aimpoint A
In doing so, the Army would inhibit the full and open competition required by 10 U.S.C. § 2305(a)(l)(A)(i), and is acting in an arbitrary and capricious manner. As this court and GAO have repeatedly held, an “agency’s failure to follow the terms of its own Solicitation and selection of an offeror based upon different requirements than those imposed upon the only other offeror[s] are quintessential examples of conduct which lacks a rational basis.” Hunt Building, 61 Fed.Cl. at 273 (citation omitted); see also PGBA LLC v. United States, 60 Fed.Cl. 196, 207 (noting that a certain level of “uneven treatment goes against the standard of equality and fair-play that is a necessary underpinning of the federal government’s procurement process and amounts to an abuse of the agency’s discretion”) (citations omitted), aff'd, 389 F.3d 1219 (Fed.Cir.2004); Sperry Marine, Inc., B-227106, B-227106.2, 87-2 CPD ¶ 241, 1987 WL 102805 (Comp. Gen. Sept. 14, 1987) (upholding protest because the Navy relaxed two mandatory performance requirements for one offeror only). Although the government asserts that the Army is merely following solicitation procedures and is allowed to treat Aimpoint A differently than EOTech, this approach ignores the practical consequences of relaxing testing requirements for the sole offeror remaining in the competitive range.
Here, the most highly rated proposals, Aimpoint A, [ ] and EOTech, were poised to pass Essential Criteria testing until [] and EOTech both failed Endurance-Live Fire testing on the M16A2 rifle, although [] and EOTech [] on the M4. AR at 201. It is undisputed that attaching these CCOs to the M16A2 carry handle is more difficult than securing these sights to the M4 rail, because the carry handle was not designed as an “attachment point.” See id. at 192. Aim-point A received [ ] results in [ ] Rated Criteria because of [ ]. Id. at 206. Aimpoint A will be allowed to [ ]. Id. at 192, 350. The Army thus contemplates award to a modified Aim-point A, where the proposed modification [ ] and EOTech from the competitive range. There is no assurance that Aimpoint A, as modified, would pass the Endurance-Live Fire testing on the M16A2, and the Army inexplicably will not require this test.
In accordance with the record before this court, the foregoing reflects an accurate characterization of the Army’s decision-making process, a process which evinces more than a relaxation of the Endurance-Live Fire requirement: the Army contemplated waiving a solicitation requirement for what it then believed to be the equivalent of [].
B. Failure to Abide by Solicitation Evaluation Procedures
Technical evaluations should be consistent with the factors, subfactors and procedures outlined in the solicitation. See FAR 15.305(a), 48 C.F.R. § 15.305(a) (2007) (“An agency shall evaluate competitive proposals and then assess their relative qualities solely on the factors and subfactors specified in the solicitation.”); see also Dubinsky v. United States, 43 Fed.Cl. 243, 267 n. 56 (1999) (noting that FAR 15.305(a) “does not grant contracting officers carte blanche to notify offerors of one rating system in the RFP and to then apply a different system during the evaluation of proposals”) (citations omitted); Kilgore Corp., B-253672, B-253685, B-253686, 93-2 CPD ¶220 (Comp.Gen. Oct. 13, 1993) (“While procuring agencies have broad discretion in determining the evaluation plan they will use, they do not have the discretion to announce in the solicitation that one plan will be used and then follow another in the actual evaluation.”) (citation omitted); Arltec Hotel Group, B-213788, 84-1 CPD ¶381, 1984 WL 44060 (Comp.Gen. Apr. 4, 1984) (“Consequently, it is improper for an agency to depart in any material way from the evaluation plan described in the solicitation without informing the offerors and giving them an opportunity to structure their proposals with the new evaluation scheme in mind.”)
1. Testing Irregularities
Several statements in Section M of the solicitation indicate that Essential Criteria testing, and passing grades in all fifteen of these tests, were required before a sample could proceed to the Rated Criteria round of testing. See AR at 142 (“Essential Criteria Subfactor (PASS/FAIL) will be evaluated first.”); id. (“If the offeror’s bid samples fail to comply with any of the essential government requirements listed below [as Essential Criteria], the offeror will be eliminated from any further consideration for award.”); id. (“All Bid Samples that pass essential criteria evaluation will be evaluated in accordance with the rated criteria as stated.”). Although the solicitation established an elimination round in Phase I testing with fifteen Essential Criteria, the Army did not wait for the final results of the elimination round before beginning Phase II testing of the Rated Criteria. See supra note 4. When the Army tested three samples for Rated Criteria, it might have been proceeding on the assumption that at least one of those candidates would pass the Endurance-Live Fire testing that was still ongoing, or perhaps the Army then knew that Aimpoint A had passed Endurance-Live Fire testing but did not yet have the Endurance-Live Fire scores for [ ] and EOTech. In any event, the Army deviated from its proposed evaluation scheme and tested two bid samples, both of which failed to pass the Essential Criteria elimination round, for the Human Systems Integration (HSI) element of the Rated Criteria.
2. Evaluation Irregularities
Two evaluation irregularities, in particular, deserve mention here. The court previously noted that the [ ] rating given to Aimpoint A for its overall Rated Criteria test results contradicts the more glowing narrative from the technical testing team. See AR at 146, 192, 211-12. Although a[] rating signifies a[] according to the solicitation, the narrative describes Aimpoint A as having a[], language that more properly belongs to a[ ] rating. See id. at 146, 211. Similarly, although a[] rating signifies that [] that the product will successfully meet expectations, the narrative suggests that [ ] that Aimpoint A will successfully meet expectations, again language borrowed from the definition of a[ ] rating. See id. at 146, 192, 212. The [] rating was similarly upgraded in the narrative adopted by the Source Selection Authority in the competitive range determination. Id. at 344. Because the Army did not apply the color ratings in the manner it prescribed in the solicitation, the court has little confidence in the accuracy of these ratings. See Dubinsky, 43 Fed.Cl. at 267 n. 56 (describing deviations from a color rating scheme as “unorthodox” and “arbitrary and capricious”).
Another troubling evaluation irregularity occurred in the HSI component of Rated
The underlying data, however, from the ninety-page report prepared by the Army’s Soldier Battle Lab (SBL), do not match or even resemble Mr. [ ] brief summary.
In light of the SBL report’s executive summary, and especially its conclusions regarding the areas of concern mentioned in the solicitation, Mr. [ ] ratings are revealed to be inaccurate and misleading. Under adjustment of the optic, an ease of use criteria, the SBL notes [ ]. AR at 252. The only optics mentioned as []. Id. Aimpoint A and [], while EOTech [ ]. Id. For Effectiveness, the components of which reflect evaluation criteria listed in the solicitation, [ ]. See id. at 333 (Mr. [ ] chart showing that [ ]).
As to various measures of Suitability, EOTeeh appears to [ ] Aimpoint A and [ ]. Portability was [] and maintainability []. AR at 251. All optics had [ ]. Id. Aimpoint A and [ ], but the SBL report does not attach any significance to EOTech’s []. The SBL team also reviewed results achieved in using the optics while firing the rifles, and concluded that [ ]. Id. at 253. Questionnaire data was mixed on the [ ], but [ ]. Id. at 285-86. The “preferred design” question reaped 9 votes from survey participants favoring [ ].
Mr. [] two-page analysis of the SBL’s Human Systems Integration testing falls far short of accurately summarizing what the staff conducting those tests characterized as [ ] results for [ ]. In particular, his “finding”
3. Amendment 2—Changes to the Evaluation Criteria During Ongoing Bid Sample Testing
Amendment 2 of the solicitation was issued and effective on October 19, 2007, over one month after bid samples had been received and after testing of the samples had begun. See AR at 193 (report describing technical bid sample testing as occurring “5 Sept 2007—17 Jan 2008”). Although solicitations may be amended after receipt of offers, there are some regulatory restrictions on the scope of such amendments:
If, in the judgment of the contracting officer, based on market research or otherwise, an amendment proposed for issuance after offers have been received is so substantial as to exceed what prospective offerors reasonably could have anticipated, so that additional sources likely would have submitted offers had the substance of the amendment been known to them, the contracting officer shall cancel the original solicitation and issue a new one, regardless of the stage of the acquisition.
FAR 15.206(e), 48 C.F.R. § 15.206(e) (2007). The Army characterized Amendment 2 as providing merely a “clarification” of the Endurance-Live Fire Essential Criteria benchmark, and a correction of a “typographical error” in the Height Above Bore Essential Criteria benchmark. AR at 175. The court has no reason to presume that new companies would have sought to enter this eompetition had these changes been known, and thus no violation of FAR 15.206(e) occurred when the Army issued Amendment 2. Nor has there been any allegation of favoritism or bias here, and the record shows none. Nonetheless, the court notes that both of these amendments had material scoring consequences for the bid samples that were in the process of being tested and evaluated at the time Amendment 2 was issued.
First, the Height Above Bore requirement was changed in Amendment 2 from 8.2 centimeters, +/-. 5 centimeters, to 7.7 centimeters, +/-.5 centimeters, on the M16A2. AR at 175, 178. None of the bid samples tested met the original Height Above Bore requirement. See id. at 201. All of Aimpoint’s [ ], and EOTech, met the amended Height Above Bore Essential Criteria benchmark. After Amendment 2, five bid samples, not six bid samples, as reported by the Army, met the revised Height Above Bore requirement.
More importantly, Amendment 2 changed the Endurance-Live Fire requirement. AR at 175, 178. Before this change, the requirement in relevant part stated that the sample “shall withstand a 6,000 round endurance firing with no physical damage and maintain zero within 1 Gunner’s mil upon completion of endurance test.” Id. at 143, 175. The court understands this phrasing to comprise two measurable benchmarks: (1) the CCO must not sustain physical damage after 6,000 rounds, and (2) the CCO must maintain zero within 1 Gunner’s mil after 6,000 rounds. Id. The amended Endurance-Live Fire test states that the sample “shall withstand a 6,000 round endurance firing with no physical damage and maintain zero within 1 Gunner’s mil after 2,000 rounds.” Id. at 175, 178. The court reads this revised requirement to again comprise two measurable benchmarks: (1) the CCO must not sustain physical damage after 6,000 rounds, and (2) the CCO must
The Army applied what appears to be a different requirement, with four benchmarks. According to the testing team report, each sample, if stable enough to safely permit 6.000 rounds of firing, “shall withstand a 6.000 round endurance firing with no physical damage and maintain zero within 1 Gunner’s mil after each of 3 sets of 2,000 rounds.” AR at 198 (emphasis added to show variation from solicitation). Thus, looseness, or some other impairment to the CCO’s ability to maintain zero, was measured three times, at 2,000, 4,000 and 6,000 rounds, and physical damage was recorded at 6,000 rounds. Id.
To distinguish the three variants of the requirement for the Endurance-Live Fire Essential Criteria testing, the court refers to the three versions as the Original Live Fire, the Amended Live Fire, and the As Tested Live Fire requirements. As to the Original Live Fire requirement, it appears that three samples would have passed this test on both rifles, because they maintained zero well enough as measured at 6,000 rounds and sustained no damage at 6,000 rounds: Aim-point A, [ ], and [ ].
If the Endurance-Live Fire Essential Criteria testing had not acquired such importance in this procurement, eliminating six of the seven submitted bid samples, the fact that the Army modified this requirement twice, once properly by amendment, and again informally and in error, the court could regard the Army’s misinterpretation of its own requirement as a de minimis error which did not significantly affect competition. However, because the Army’s Endurance-Live Fire testing produced a competitive range of one, it is subject to close scrutiny. Under close scrutiny, the court cannot conclude that the evaluation of this Essential Criteria component adequately conformed to the procedures outlined in the solicitation, or, given the somewhat perplexing variations in scores achieved by the optics on both rifles, that the benchmarks set in this testing necessarily reflect that only Aimpoint A could perform well when extensively fired in combat conditions. Cf. OAO Corp., B-232216, B-232216.2, 88-2 CPD ¶ 546, 1988 WL 228249 (Comp.Gen. Dec. 1, 1988) (“The real question here, in our view, is whether the [technical] evaluation ... was adequate to ensure that the Army will obtain a [product] meeting the RFP requirements.”).
Aimpoint A received only a marginally passing score on Endurance-Live Fire testing according to the Original Live Fire Requirement, and clearly passed that test only under the Amended Live Fire Requirement. The Army changed the benchmark after testing had begun. It is somewhat troubling that the benchmarks for both Endurance Live-Fire and Height Above Bore were changed while testing was underway, altering the pass/fail scores of each optic then under consideration. The court’s confidence in the validity and integrity of the testing process is undermined by this series of events.
C. Failure to Seek Clarifications or to Otherwise Properly Secure EO-Tech’s CCO in Order to Produce Significant Endurance-Live Fire Testing Results for More Than One Proposal
The Army could have sought information from EOTech regarding the instructions for securing EOTech’s gooseneck mount to the M16A2. Such communication is permitted under FAR 15.306, in some circumstances, either through clarifications, or exchanges to facilitate the development of a competitive range. See 48 C.F.R. § 15.306(a)-(b). “[T]he decision to engage in clarifications [i]s discretionary.” DynCorp Int’l LLC v. United States, 76 Fed.Cl. 528, 540 (2007). Nonetheless, the decision to not seek clarifications from an offeror must be reasonable under the circumstances of the procurement. See Gen. Dynamics-Ordinance & Tactical Sys., B-295987, B-295987.2, 2005 CPD ¶ 114, 2005 WL 1468418 (Comp.Gen. May 20, 2005) (“[GAO] will review the exercise of such discretion [to seek or not seek clarifications] to ensure that it was reasonably based on the particular circumstances of the procurement.”) (citations omitted). Thus, if EO-Tech’s instructions for securing the goose-neck mount to the M16A2 carry handle were clearly deficient, the court must decide whether it was reasonable for the Army not to seek more information from EOTech before conducting the Endurance-Live Fire testing on EOTech’s optic.
Even if EOTeeh’s attachment instructions were not clearly deficient, the court must still decide whether the Army’s decision to secure EOTech’s locking nut on the gooseneck mount by finger-tightening was rational.
EOTech’s live fire test results on the M16A2 were used to justify a competitive range of one. AR at 201. The court must decide whether the Army’s finger-tightening of EOTech’s gooseneck mount to the M16A2 carry handle, in the context of the test results the Army obtained from the various optics on the Endurance-Live Fire Essential Criteria, was reasonable under close scrutiny. See Chapman Law Firm, 71 Fed.Cl. at 132 (“Case law and common sense counsel that close scrutiny is appropriate where an agency has included only one firm in the competitive range____”). The court relies less on conflicting data regarding industry standards, than on a demonstration of the hardware in question.
As a threshold matter, the record does not unequivocally establish the cause of EO-Tech’s failure to pass Endurance-Live Fire testing on the M16A2. However, the most likely cause was looseness of the gooseneck mount. Comments from soldiers testing EO-Tech noted []. AR at 311.[ ]. Id. at 657-61; see also 774-79 (noting [ ]). Although the court cannot rule out other causes for EO-Tech’s failure to adequately maintain zero in live firing on the M16A2, the evidence of record points to a loose gooseneck mount, □-
EOTech’s gooseneck mount “installation” instructions include one pertinent sentence: “Secure the locking nut.” AR at 686. The accompanying diagram [ ]. Id. Apparently the testers developed a protocol where hand-tightening was the default when no instructions to the contrary were given in a bid sample’s manual. Id. at 741. The technical testing team relied on past practices, measurement of the average hand-tightening torque of the testing team members, and study data showing the consistency of hand-tightening. Id. at 504-05.
EOTech’s locking nut has [ ]. Athough the [] of the nut would normally permit hand-tightening, the court observed that when the gooseneck mount is installed on the M16A2 carry handle, access to the nut is restricted by the carry handle and a full twist of the hand is not possible. Instead, installation without tools only permits the locking nut to be finger-tightened. Because hand-tightening of the locking nut is physically impossible, the court must reject much of the Amy’s supporting data regarding the rationality of its hand-tightening protocol. The Amy testing staff admitted that the consistency checks they conducted with a torque wrench on other locking nuts were not conducted on EOTech’s gooseneck mount locking nut, because access to the nut was restricted by the narrow space created by the carry handle.
A much more “secure” tightening of EO-Tech’s locking nut is possible by using a tool that can act as a lever, after the tool is inserted [ ]. Whether a need for tool-tightening was apparent from the design of the nut itself, or from the design of the nut and the close quarters of the location of the nut, is a close question. What is clear, however, is that the engineers had a stated objective of achieving significant test results by using
In light of [ ] on EOTech’s locking nut, the restricted access afforded the nut by the carry handle, and the testing team’s goal of “hand-tightening” consistency, the court finds that the Army’s decision to not inquire as to how this nut should be secured, and simply finger-tightening the nut before subjecting the sight to 6,000 rounds of live firing, was not reasonable. Whether this irrationality is attributed more to a failure to request a needed clarification, or to a failure to choose a tightening technique that was adequately rehable under the circumstances, the result is the same. Based on the record before the court, EOTech’s gooseneck mount was not properly secured to the M16A2 carry handle. For this reason, the Endurance-Live Fire Essential Criteria test was irrational and fundamentally flawed as it was conducted for EOTeeh’s bid sample.
V. Success on the Merits
For the foregoing reasons, the court finds that the Army’s competitive range determination of March 17, 2008 and proposed discussions with Aimpoint are arbitrary and capricious and lack a rational basis. As previously stated, EOTeeh had a substantial chance of placement in the competitive range for this procurement but for the disparate treatment of its bid sample, testing and evaluation irregularities, and the Army’s failure to properly secure its gooseneck mount on the M16A2. As a consequence thereof, EO-Tech was prejudiced by the Army’s competitive range determination and its decision to enter into discussions only with Aimpoint. EOTeeh has thus succeeded on the merits of this bid protest.
VI. Whether EOTeeh Should be Awarded Injunctive Relief
As the Federal Circuit has stated,
In deciding whether a permanent injunction should issue, a court considers: (1) whether, as it must, the plaintiff has succeeded on the merits of the case; (2) whether the plaintiff will suffer irreparable harm if the court withholds injunctive relief; (3) whether the balance of hardships to the respective parties favors the grant of injunctive relief; and (4) whether it is in the public interest to grant injunctive relief.
PGBA, LLC v. United States, 389 F.3d 1219, 1228-29 (Fed.Cir.2004) (citation omitted). Here, plaintiff has succeeded on the merits. Elimination from this competition would cause EOTeeh irreparable harm, because loss of the opportunity to compete for this sizable contract cannot be remedied other than by means of injunctive relief. See, e.g., Hosp. Klean of Tex., Inc. v. United States, 65 Fed.Cl. 618, 624 (2005) (“Here, absent injunctive relief, [the protestor] will lose the opportunity to earn the profit it would have made under this contract. Such loss of profit, stemming from a lost opportunity to compete for a contract on a level playing field has been found sufficient to constitute irreparable harm.”) (citations omitted); Overstreet Elec. Co. v. United States, 47 Fed.Cl. 728, 744 (2000) (holding that the “potential loss of valuable business” may constitute irreparable harm). Thus, plaintiff has met the first two criteria for receiving a permanent injunction.
As to the balance of hardships, the court was assured by the government that adequate supplies of CCOs are available through other contracting vehicles pending the outcome of this protest. The government has not argued, or shown, that any additional delays the Army would experience, if its competitive range determination is enjoined, would compromise the military’s weapons readiness. Thus, the balance of hardships tips in favor of EOTeeh, if it is not permitted to fairly compete with Aimpoint, when compared to any hardships the military would face. Aimpoint, for its part, fails to identify any hardships it would endure if forced to fairly compete for this procurement. See Aimpoint’s Mot. at 23. Indeed, as the incumbent provider of CCOs, Aim-point may not suffer any economic hardship if award is further delayed.
As to the public interest, the court views this factor as unquestionably favoring injunctive relief in this case. The Army began this procurement with the goal of developing more than one source for CCOs, if possible,
CONCLUSION
Plaintiff has proved that the competitive range determination for Solicitation W15QKN-07-R-0428, including only one proposal for further consideration and discussions, was arbitrary and capricious and without a rational basis. Plaintiff has also shown that it was prejudiced by the Army’s procurement errors. For these reasons, plaintiff’s bid protest is sustained. Plaintiff, in addition, is entitled to injunctive relief.
Accordingly, it is hereby ORDERED that
(1) Plaintiffs Motion for Judgment on the Administrative Record, deemed to have been filed on July 15, 2008, is GRANTED;
(2) Intervenor-Defendant’s and Defendant’s Cross-Motions for Judgment on the Administrative Record, filed July 25 and 28, 2008, respectively, are DENIED;
(3) Plaintiffs Motion for Permanent Injunctive and Declaratory Relief, filed July 15, 2008, is GRANTED as follows:
(a) the United States, by and through the Department of the Army, its officers, agents, and employees, is hereby PERMANENTLY RESTRAINED AND ENJOINED from proceeding with discussions and award of a contract based on the competitive range determination of March 17, 2008 for Solicitation W15QKN-07-R-0428;
(b) If the United States Department of the Army elects to proceed with this procurement, the Army must RETEST EOTech’s bid sample for Endurance-Live Fire Essential Criteria on the M16A2 rifle pursuant to the requirements of the amended solicitation, after CLARIFYING with EOTech how its gooseneck mount should properly be secured to the M16A2 rifle, in order to determine whether EOTech’s proposal should be placed in the competitive range;
(4) The Clerk’s Office is directed to ENTER final judgment in favor of plaintiff;
(5) On or before September 22, 2008, counsel for the parties shall CONFER and FILE with the Clerk’s Office a redacted copy of this opinion, with any material deemed proprietary marked out and enclosed in brackets, so that a copy of the opinion can then be prepared and made available in the public record of this matter; and
(6) Each party shall bear its own costs.
. Plaintiff's Motion for Permanent Injunctive and Declaratory Relief (PI.'s Mot.), filed July 15, 2008, and plaintiff’s [Amended] Memorandum of Points and Authorities in Support of Plaintiff’s Motions for a Temporary Restraining Order and Injunctive and Declaratory Relief, (Pl.’s Mem.), filed August 4, 2008, together are deemed to constitute plaintiff’s motion for judgment on the administrative record. See Order of July 17, 2008.
. One departure from the solicitation testing instructions is discussed in the next section of this opinion, as to testing for the Human Systems Integration Rated Criteria.
. The Army explained this anomaly to the Government Accountability Office (GAO), stating that due to scheduling concerns, Human Systems Integration (HSI) Rated Criteria testing was begun before Endurance-Live Fire Essential Criteria testing was complete. AR at 740 n. 4. Once [] and EOTech had failed Endurance-Live Fire testing, these samples were, according to the Army, "eliminated from the competitive range.” Id. According to the Army, the document reporting the performance of Aimpoint A, [] and EOTech on HSI testing was simply a “back up document used by the Source Selection Board to generate the Human Systems Integration subfactor rating” for Aimpoint A. Id.
. Apparently, the Army also intended to discuss []. ARat651.
. Aimpoint raises what might be called a standing argument in its reply brief, suggesting that EOTeeh does not have "standing to challenge what future steps may or may not be taken in” this procurement. Aimpoint’s Reply at 4 n. 1. Arguments raised for the first time in a reply brief need not be considered by the court. See Arakaki v. United States, 62 Fed.Cl. 244, 246 n. 9 (2004) ("The court will not consider arguments that were presented for the first time in a reply brief or after briefing was complete.”) (citations omitted). Even if Aimpoint’s argument had been timely raised, Aimpoint fails to cite to any authority, and the court has found none, for the proposition that a protestor must show that it has standing for each and every separate argumeat it has raised in requesting relief from this court. EOTeeh has standing for this protest because it is an offeror with a substantial chance of winning the contract but for the errors it alleges in this procurement. The errors alleged include testing errors and errors which would occur as a result of the Army’s proposed discussions with Aimpoint. All of these allegations question the propriety of the Army’s Essential Criteria testing and the establishment of the competitive range. Standing to bring this suit is not undermined by plaintiff’s arguments that focus on contemplated acts of the Army, because these arguments go to the totality of the circumstances of the Army’s competitive range determination.
. It was not, as the government appears to contend, obvious from the language of the solicitation that discussions would be held, see AR at 134, or that modifications of hardware critical to Essential Criteria test results would not trigger Essential Criteria retesting of the modified sample. See Def.’s Mot. at 14. Thus, EOTech's protest of disparate treatment is timely. Cf. Blue & Gold Fleet, L.P. v. United States, 492 F.3d 1308, 1313 (Fed.Cir.2007) (holding that “a parly who has the opportunity to object to the terms of a government solicitation containing a patent error and fails to do so prior to the close of the bidding process waives its ability to raise the same objection subsequently in a bid protest action in the Court of Federal Claims”) (emphasis added). And EOTech's disparate treatment challenge is of a different nature than that discussed in Aero Corp., S.A. v. United States, 38 Fed.Cl. 739, 768 (1997), cited by defendant and intervenor-defendant, a case which stands for the unremarkable proposition that harsh ratings for one proposal may be rationally explained by the severity of deficiencies noted in that proposal, compared to less severe deficiencies noted in others. Because Aero Corporation does not consider relaxed or waived solicitation testing requirements, it is not useful authority for this case.
. The Army’s decision to procure a modified CCO that will not have passed the Endurance-Live Fire requirement is not reasonable. See McRae Indus., Inc., B-287609.2, 2001 CPD ¶ 127, 2001 WL 826734 (Comp.Gen. July 20, 2001) (examining a waiver of bid sample tests to see if “the record reasonably supports the [contracting officer's] decision that the tests were not necessary to ensure that the agency will obtain items meeting its needs”).
. The Army contends that there are internal differences between Aimpoint A and []. See AR at 712. The date these differences were discovered is not provided in the record, although defendant conceded at oral argument that knowledge of the differences between Aimpoint A and [ ] post-dates the Army’s competitive range determination. Because the record shows no contemporaneous evidence of differences between Aimpoint A and [], other than [], the court cannot use such evidence to support the Army’s decision. See OMV Med., Inc. v. United States, 219 F.3d 1337, 1344 (Fed.Cir.2000) (holding that a procurement decision may not be upheld as rational for reasons not used by the agency at the time of the decision) (citations omitted). Even if this discovery were relevant to the court’s review, the differences noted between Aimpoint A and [] have not been shown to be directly related to the differences in scores achieved by these CCOs on the M16A2.
. The summary report of Rated Criteria test results describes [] and EOTech as "NOT TESTED” for Human Systems Integration (HSI). AR at 206; see also id. at 191 (technical team report summary stating that "[n]o further evaluations on [six samples, including [] and EOTech] were performed” after these samples failed Essential Criteria testing). As previously noted, HSI testing collected data from exercises involving soldier participants who used the CCOs on the M4 and M16A2. AR at 251-59.
. The Army describes the document reporting on the testing of HSI as a "back up document.” AR at 740 n. 4. The technical testing team reported that [] and EOTech were not tested for HSI. Id. at 206. One of the members of the Army’s technical team did, however, [] to Aimpoint A, [] and EOTech, respectively, for the HSI component of Rated Criteria. See id. at 333-34 (describing his ratings of the three bid samples as being "in accordance with the criteria in the Source Selection Plan").
. The Army’s Soldier Battle Lab (SBL) at Fort Benning, Georgia provided support for the technical team rating the CCO bid samples by conducting the HSI testing, but did not itself produce "recommendations or overall ratings” of the optics. AR at 245. Mr. [] is not listed among personnel conducting the HSI testing. Id. at 255. The Human Resources and Engineering Directorate (HRED) member on the SBL team was Mr. []. Id. The SBL report was based on data derived, in part, from "interviews with evaluation personnel, and through direct observation by experienced Human Research and Engineering practitioners.” Id. at 259. The record does not show that Mr. [ ] was present for any of the HSI testing, and his analysis appears to be the result of a review, conducted some months after testing was completed, of the written SBL report and underlying data. Compare id. at 244 (showing that SBL testing occurred from October 22-26, 2007 and that the SBL report was issued in December 2007) with id. at 333 (first page of Mr. [ ] two-page analysis bearing the date February 4, 2008). Mr. [] is, however, the HRED member of the technical testing team that prepared the recommendations relied upon by the Source Selection Authority for the Army's competitive range determination. Id. at 190.
. Mr. [] statement that [] is not substantiated by the record. See AR at 268-69 (survey results showing only []”).
. The Army incorrectly reported that [] passed the Height Above Bore test on the M16A2 with a measurement of 9 centimeters, which is outside of the acceptable range for this requirement, as modified by Amendment 2. AR at 201.
. Aimpoint A may not have passed the Original Live Fire requirement, depending on underlying data not in the record. The court cannot ascertain whether the [] measurement in the summary chart on page 201 of the administrative record might have been rounded down for formatting purposes.
. Aimpoint A is also on the margins of passing the As Tested Live Fire Requirement. See supra note 15.
. Amendments to evaluation criteria after evaluation procedures have begun, are not, by themselves, indicative of reversible error by the government. See Pacer Sys., Inc., B-215999, 84-2 CPD ¶ 645, 1984 WL 47057 (Comp.Gen. Dec. 10, 1994) ("[GAO] has held that the contracting officer can amend the RFP after the proposal closing date. For example, reopening competition following the closing date and receipt of ‘best and final offers’ is appropriate when an ambiguity in the solicitation is apparent.”) (citation omitted). However, many of the decisions approving
. According to the Army, the Endurance-Live Fire was the last Essential Criteria test, conducted after the other fourteen tests were complete. See AR at 740 n. 4
. The court agrees with defendant that some protests will fail when an offeror has not supplied the government with an adequately detailed proposal. See Def.’s Mot. at 10 (citing cases). But a distinction must be drawn between a proposal containing “material omissions and other serious flaws requiring an extensive revision,” see United Enter. & Assocs. v. United States, 70 Fed.Cl. 1, 27 (2006), and a proposal providing a technically viable bid sample but lacking detailed installation instructions.
. The court makes a distinction between hand-tightening and finger-tightening, based on its observations of various tightening techniques demonstrated on the CCOs and their mounts.
. Causation is not the dispositive inquiry here. Whether EOTech’s testing difficulty was caused by a loose locking nut, as plaintiff alleges, or some other attachment hardware issue, it is the narrowing of the competition to one offeror, directly related to mounting hardware used on the M16A2 carry handle, which must be reviewed here. Whether EOTech’s poor performance was solely related to the tightening of its locking nut, as the evidence tends to show, or to other factors, the Army's testing protocol must, in any case, withstand close scrutiny.