JOHN R. BROWN, Circuit Judge:

This is an appeal from a take-nothing judgment of the District Court which included a review of a determination by the United States Army Corps of Engineers denying a request for a permit to fill and construct on a lot owned by Harry Bowles. Bowles also filed a Bivens 1 action, seeking a determination that the Corps’ actions constituted a taking of his property in violation of his First and Fifth Amendment rights and compensatory damages for the unlawful taking. As the parties concede, Bowles’ claim for compensatory damages in excess of $10,000 against the United States, through the Corps of Engineers, for wrongful taking of his property, was within the sole jurisdiction of the Court of *114 Claims. Consequently, we remand to the District Court for transfer to the Court of Claims. 2 We affirm the judgment in all other respects.

Bowles is the owner of a 50 by 100 foot lot (Lot 29) which is the subject of this litigation. Lot 29 is located in a platted subdivision on Follett’s Island in Brazoria County, Texas. Bowles also owns an adjacent tract of land outside of the platted subdivision. The property has a shore line on Cold Pass, an area of salt water adjacent to the Gulf of Mexico.

Bowles initially corresponded with the Corps in March 1980, regarding possible commercial development of the larger tract located directly behind Lot 29. An on-site inspection was conducted by two Corps representatives. During the course of the inspection, the Corps representative indicated that the presence of certain grasses on the tract indicated it was a wetland subject to Corps’ jurisdiction. Consequently, a permit would be required for any development. Bowles challenged this ad hoc determination and, during an apparently heated discussion with the Corps representative, Bowles announced his intention to construct a home on Lot 29. The representative responded that Lot 29 was also wetr lands 3 and that a permit was required for construction on that lot as well.

Bowles submitted a permit application under § 404 of the Clean Water Act 4 to the Corps on May 2, 1980. At some point during the summer, Bowles withdrew his application and in September, 1980, commenced depositing clean sand to Lot 29. In December, 1980, the Corps ordered Bowles to cease his fill activity and all fill activity stopped, pursuant to the cease and desist order.

The complaint in this action was filed in April, 1981, naming as defendants the Galveston District Office of the Corps of Engineers, Colonel Sigler (the District Engineer) and his two subordinates, Marcos De La Rosa and Fred Miller, in both their official and individual capacities. Bowles raised a Bivens claim, alleging that the denial of his permit was arbitrary, capricious, retaliatory and in violation of his rights under the First and Fifth Amendments because numerous other wetlands areas in this subdivision had been covered with fill material and developed, including the adjacent lots and lots across the street.

Discovery took place throughout 1981. In March, 1982, the Corps moved to dismiss and, in the alternative, for summary judgment based on Bowles’ failure to exhaust administrative remedies within the Corps. At this time, Bowles had yet to file an application for a permit. 5 On November 23,1982, after a hearing on this motion, the District Court abated the action for 30 days to enable Bowles to apply for a Corps permit. Once the permit application was submitted, the abatement continued until the administrative proceedings were completed.

Bowles did not submit a new application, but instead asked the Corps to reconsider his prior application which had been withdrawn during the summer of 1980. Because this was an unusual request, coupled with the usual bureaucratic red tape, there was an additional delay in processing the application.

*115 While the application was still pending before the Corps, the case was set for jury-trial. 6 Bowles had repeatedly requested a jury trial on his Bivens claims, and, at one point, the district judge granted that request. Nonetheless, when trial actually commenced, the judge refused to empanel a jury. 7 Bowles testified as his own witness and called four others.

Bowles presented evidence at trial that he was the President of the Gulf Coast Wildlife Preservation Society (GCWPS), a nonprofit organization concerned with protecting Texas coastal wildlife and habitat in the Galveston and Brazoria counties area. Bowles got the Constitution in the case by contending that the permit application was denied 8 in retaliation for constitutionally protected speech made in his role as President of GCWPS. During the presentation of Bowles’ case in chief, the District Court recessed the trial to enable the Corps to complete its processing of Bowles’ application. The Corps was ordered to submit status reports every 15 days until a decision on the application was reached.

The Corps conducted a public interest review of Lot 29. The Environmental Protection Agency, National Marine Fisheries Service, United States Fish and Wildlife Service, and the Sierra Club all opposed granting the permit. Their recommendations were based on their conclusions that Lot 29 is a valuable wetlands area that provides nutrients and detritus which are necessary for the support of fish and crustaceans.

Eight months after the trial was recessed, on October 26, 1984, the Corps issued an environmental assessment and statement of findings, denying Bowles a § 404 permit. The permit was denied because Lot 29 met the definition of wetlands and it was in the public interest to deny the permit. Bowles was notified of this decision on October 30, 1984. 9

In mid-July, 1985, he requested a jury trial on issues arising from the Corps’ October 1984 denial of his permit application for Lot 29. After the case was set for trial, Bowles filed an original petition for mandamus in this Court, 10 seeking the disqualification of the District Judge. 11

The abated trial was resumed two and a half years after it was recessed. 12 Bowles did not present any new evidence at this time, but the Corps presented testimony regarding the application process and the rationale for the wetlands determination and permit denial.

The trial court affirmed the Corps’ jurisdiction and found that the Corps’ denial of the permit was neither arbitrary nor capricious. Other developments in the same subdivision were permitted because they were built pursuant to a mitigation plan filed with the Corps, pursuant to a nationwide permit, or under “grandfather” regulations.

*116 I.

Bowles' initial challenge is to the denial of his § 404 permit. A District Court’s review of the determination of a federal agency is limited by the Administrative Procedure Act (APA) 13 to whether the agency acted in an arbitrary or capricious fashion, abused its discretion or otherwise acted not in accordance with law. 14 This includes actions taken contrary to a constitutional right, 15 in excess of statutory jurisdiction 16 or without observance of procedure required by law. 17 When an agency's determination is based on an administrative record, the decision should be reviewed in light of that record. 18 If the agency decision cannot be sustained on the administrative record, then the case should be remanded to the agency for further consideration. 19

Contrary to the accepted rule of a District Court in reviewing an administrative order, the District Court permitted the taking of evidence at a trial. 20 After erroneously receiving this evidence, the District Court concluded that the lot in question was in fact wetlands, subject to the Corps' jurisdiction. The Court further determined that the § 404 permit was denied on the basis of recommendations from various federal agencies and public interest groups. There was no evidence presented to indicate that the decision of the Corps was arbitrary, capricious, an abuse of discretion or otherwise not in accordance with law. There was likewise no evidence that the denial was in any way related to, or constituted a denial of Bowles’ constitutional rights under the First Amendment. Bowles did not prove that he was treated differently from other similarly situated land owners, as the other land owners were not similarly situated. 21

This Court reviews the District Court’s decision under the traditional standard requiring us to uphold factual findings unless they are clearly erroneous. 22 As always, we review the District Court’s conclusions of law de novo. 23

The District Court, as did the Corps, found that Lot 29 was, in fact, a wetlands area commonly known as a coastal or intertidal salt marsh. The Court further found that Bowles failed to show that (i) he engaged in any constitutionally protected speech, (ii) the Corps denied the permit application in retaliation for this unidentified speech, or (iii) the unidentified speech was a motivating factor in the Corps’ decision. The Court’s finding that there was no retaliatory motive in the Corps’ decision is a finding of fact which we must uphold unless elearly erroneous. We cannot say that this finding is erroneous.

II.

Bowles’ next argument is that the denial of a fill permit by the Corps renders Lot 29 *117 devoid of economic value and constitutes a taking of his private property in violation of his constitutional rights guaranteed under the Fifth Amendment. 24 The parties concede this inverse condemnation claim in excess of $10,000 should properly be heard by the Court of Claims. 25

III.

Bowles is correct in his assertion that a Bivens plaintiff is entitled to a jury trial on those constitutional issues. 26 Although technically the refusal to accede to Bowles’ jury demand was erroneous, it was not reversible since there was no evidentiary basis for the constitutional claims.

In a case in which the plaintiffs case “would not have survived a motion for a directed verdict,” the denial of a jury trial is harmless error. Cox v. C.H. Masland & Sons, Inc., 607 F.2d 138, 145 (5th Cir.1979). 27 Although Bowles claimed that the Corps denied his permit application in retaliation for unspecified speech activities, the evidence Bowles presented on impermissible motive was insufficient to survive a directed verdict motion. The district court found that Bowles presented “no evidence showing the content of [his] speech or that it was a motivating factor in the Corps’ decision.” 28

Although it is uncontested that Bowles and one of the Corps’ representatives exchanged angry words during an inspection of the land in question, this altercation occurred after the Corps had preliminarily asserted jurisdiction. The jurisdiction was not in retaliation for the argument. Additionally, the initial assertion of jurisdiction was sustained during the later, more comprehensive, application proceeding. The proceeding itself had no retaliatory cast to it, and any assertion that Corps’ actions during the proceeding were retaliatory can only be speculation. In fact, one Corps executive testified that he knew nothing of Bowles’ environmentalist tendencies or activities prior to the filing of this lawsuit.

On the facts presented, Bowles’ case would not have survived a motion for a directed verdict. Under these circumstances, a jury would never have had the opportunity to take the case under consideration. The absence of a jury physically present in the courtroom, if error at all, was harmless error.

Conclusion

Except for the remand for transfer to the Court of Claims, the points raised by Bowles on appeal are without merit. The District Court held as a factual matter that the Corps’ decision to deny a § 404 permit was neither arbitrary nor capricious. We *118 cannot say that finding was clearly erroneous. The lack of a jury was harmless.

AFFIRMED and remanded with instructions.

Notes

. Bivens v. Six Unknown Agents, 403 U.S. 388, 91 S.Ct. 1999, 29 L.Ed.2d 619 (1971).

. While 28 U.S.C. § 1631 permits this Court to transfer the case directly to the Court of Claims, we remand here to permit a determination, in light of this opinion, of what portion of the case remains for transfer to the Claims Court.

. The parties do not contest the determination that Lot 29 is a wetlands area, only whether that fact alone is sufficient to deny a fill and construction permit.

. Federal Water Pollution Control Act Amendments of 1972, Pub.L. No. 92-500, 86 Stat. 816 (as amended).

.As Bowles had yet to give the Corps an opportunity to act formally on his proposal, there was no Corps' action for the District Court to review. As the Corps had not yet taken any action, what, pray tell, was Bowles complaining about? We do not wish to encourage such anticipatory lawsuits against the Corps. Administrative proceedings exist for a reason — litigants are expected to exhaust those remedies prior to filing suit.

. See n. 5, supra.

. The reasons for this refusal are not reflected in the record.

. We emphasize: at this time, the permit had not yet been denied.

. In mid-June, 1985, Bowles, without a Corps permit, began filling wetlands on Lot 33, located within another part of the same subdivision. In June, 1985, the Corps filed a cross-complaint seeking to enjoin his conduct. A temporary restraining order was immediately granted. At a subsequent hearing Bowles agreed to submit a new § 404 application covering the entire tract. Bowles further agreed that the temporary restraining order would continue in force until the Corps acted on his new application. Bowles never submitted any new permit application to the Corps.

. This Court ultimately denied the petition on April 24, 1986.

. While the mandamus petition was pending, Bowles, in February 1986, again placed fill on his property. The Corps sought enforcement of the previous TRO, still in force by agreement of the parties, and applied for remedial relief and sanctions. After a hearing in which Bowles was cited for contempt, the parties stipulated to an order which specified when and how Bowles would remove the fill material and restore the land. In June, 1986, the government commenced a separate action against Bowles for filling wetlands. This action is still pending in the Southern District of Texas.

. The lengthy delay was the result of scheduling conflicts in the District Court, exacerbated by the pending petition for mandamus.

. 5 U.S.C. §§ 706 et seq. See also Avoyelles Sportsmen’s League, Inc. v. Marsh, 715 F.2d 897, 905 (5th Cir.1983).

. 5 U.S.C. § 706(2)(A).

. 5 U.S.C. § 706(2)(B).

. 5 U.S.C. § 706(2)(C).

. 5 U.S.C. § 706(2)(D).

. See Avoyelles, 715 F.2d at 905; Buttrey v. United States, 690 F.2d 1170, 1183 (5th Cir.1982), ce rt. denied, 461 U.S. 927, 103 S.Ct. 2087, 77 L.Ed.2d 298 (1983). This just reemphasizes the difficulty with attempting to try this case prior to the Corps making a determination.

. Camp v. Pitts, 411 U.S. 138, 142-3, 93 S.Ct. 1241, 1244, 36 L.Ed.2d 106, 111 (1973).

. This court has previously held it to be error for the District Court to engage in this de novo review of a Corps’ wetlands determination. Avoyelles, 715 F.2d at 907. Under appropriate circumstances, the Avoyelles court indicated the proper course would be to remand to the District Court for review of the agency determination under the appropriate standard. As the District Court here upheld the Corps’ determination, we treat the decision as a finding that the Corps did not act in an arbitrary and capricious manner in its wetlands determination or in denying the permit.

. Other development was permitted either (i) because it was a water dependent project, such as a marina; (ii) pursuant to a mitigation plan filed with the Corps; or (iii) pursuant to a nationwide permit or under grandfather regulations.

. F.R.Civ.P. 52(a).

. Avoyelles, 715 F.2d at 920; Sierra Club v. Sigler, 695 F.2d 957, 967-68 (5th Cir.1983).

. The Fifth Amendment provides: "[N]or shall private property be taken for public use,” without compensation.

. Bowles’ claim for compensatory damages against the Corps of Engineers for unlawfully "taking” his property without due process constitutes a claim against the United States, founded upon the Corps’ regulatory jurisdiction, in excess of $10,000. Under Amoco Production Co. v. Hodel, 815 F.2d 352, 359 (5th Cir.1987), this claim is within the exclusive jurisdiction of the Court of Claims. See also 28 U.S.C. § 1491.

The District Court held that there had been no inverse condemnation because Bowles was not in fact denied the economically viable use of his property. Bowles consistently refused to cooperate with the Corps, steadfastly maintaining his right to build a slab house. He refused to consider the option of building a house on pillars or the possibility of negotiating a mitigation plan with the Corps. As Bowles failed to investigate these environmentally acceptable, economically feasible options, the District Court was unable to find an unlawful taking. Of course, we make no attempt to influence the Court of Claims in their determination of the merits of the taking claim.

. Carlson v. Green, 446 U.S. 14, 100 S.Ct. 1468, 64 L.Ed.2d 15 (1980). We assume, but do not decide that a Bivens claim in this context exists in this Circuit. The defendants have not challenged its existence so we do not reach the question. See Burks v. Lasker, 441 U.S. 471, 99 S.Ct. 1831, 1835-36, 60 L.Ed.2d 404 (1979); Chipser v. Kohlmeyer and Co., 600 F.2d 1061, 1067 (5th Cir.1979).

. In deciding whether to grant a motion for a directed verdict, the party against whom the motion is made (Bowles) must be given the benefit of every legitimate inference that can be drawn from the evidence. F.R.Civ.P. 50; Boeing Co. v. Shipman, 411 F.2d 365, 374 (5th Cir.1969).

. Record Excerpts at p. 18.