OPINION

JOHN S. ANDERSON, Justice.

This is an appeal from a summary judgment. involving deed restrictions. Appellants, Rutherford Investments Inc. and Jim Rutherford, individually (Rutherford), appeal the trial court’s grant of appellee’s, Terramar Beach Community (Terramar), motion for summary judgment. The trial court granted Terramar’s summary judgment, permanently enjoining Rutherford from violating deed restrictions in the future in Terramar Beach subdivision. Ter-ramar also cross-appeals the trial court’s denial of reasonable attorney’s fees. We affirm in part and reverse and remand in part.

Factual Background

Rutherford is a real estate developer who purchased twenty-seven of the fifty-four lots in Section 6 in the Terramar Beach subdivision on Galveston Island for $1.8 million. The lots Rutherford purchased in Section 6 were subject to deed restrictions requiring, among other things, that all buildings be set back ten feet from the property side lines and that no building shall be erected until the Terramar Beach Association Architectural Control Committee had approved the construction plans. It is uncontroverted that Rutherford knew of these restrictions and failed to follow them.

On January 10, 1998, two days prior to commencing construction, Rutherford attended a general Terramar Association meeting and introduced himself and his plans for development to the association. *848 Although he did not have the required specifications, he told the group about his plans, and also told them he was not required to follow the deed restrictions in his section because “it states in the deed restriction that 100% of the property owners on that section right there can really do whatever they want to do. We can voluntarily withdraw from the ceremony.” Based on these representations, the committee approved Rutherford’s plans for two homes (Lots 8 and 9) that violated the setback restriction; however, the committee did not approve the construction plans for a third home (Lot 7) that violated the restriction. Rutherford’s construction of this third home and his proposed construction of other homes in the subdivision form the basis of this suit.

II.

Procedural History

On January 20, 1998, ten days after the association meeting, the Architectural Control Committee chairman, Gordon Hopkins, told Sherman Eagleton, Rutherford’s foreman, that the plans for Lot 7 would have to comply with the setback restriction. On February 5, 1998, the association’s legal counsel wrote to Rutherford, reminding him that the lots were still subject to the setback requirements and that he needed approval by the Architectural Control Committee before beginning construction. Following these encounters, Rutherford contacted his attorney and refused to halt construction. Therefore, on February 18, 1998, Terramar filed suit in district court seeking temporary and permanent injunctions to enforce the deed restrictions. Rutherford answered, counterclaiming based on wrongful injunction and asserting the affirmative defenses of waiver, laches, estoppel, and fraud. Rutherford also sought his own injunction against Terramar to “keep it from interfering in the construction of the remaining residential homes in the development.”

The district court granted Terramar’s motion for temporary injunction, finding it likely that Terramar would prevail in its suit for a permanent injunction. Terramar then moved for summary judgment as to all of its claims against Rutherford, Rutherford’s affirmative defenses, Rutherford’s counterclaim, and its own reasonable attorney’s fees. The trial court partly granted Terramar’s motion, permanently enjoining Rutherford from further violations of the subdivision’s deed restrictions, denying Rutherford’s counterclaim, and rejecting Rutherford’s affirmative defenses. The trial court, however, denied Terramar’s summary judgment motion for recovery of attorney’s fees. On appeal, Rutherford challenges the trial court’s grant of Terra-mar’s summary judgment, and Terramar contests the trial court’s denial of its attorney’s fees.

III.

Permanent Injunction Summary Judgment

A. Standards of Review

The grant or refusal of a permanent or temporary injunction is ordinarily within the trial court’s sound discretion, and on appeal, review of the trial court’s action is limited to the question of whether the action constituted a clear abuse of discretion. See Priest v. Texas Animal Health Com’n, 780 S.W.2d 874, 875 (Tex.App.—Dallas 1989, no writ). However, where the facts conclusively show that a party is violating the substantive law, the trial court should enjoin the violation, and in such case, there is no discretion to be exercised. See id. at 876.

The standard of review becomes more complex when the permanent injunction is issued by the trial court’s grant of a motion for summary judgment. A trial court may render summary judgment only if the pleadings, depositions, admissions, and affidavits show that there is no genuine issue as to any material fact and that the moving party is entitled to judgment as a matter of law. See Tex.R. Civ. P. 166a(c); *849 see also Nixon v. Mr. Property Management Co., 690 S.W.2d 546, 548-49 (Tex.1985). In deciding whether there is a disputed material fact issue precluding summary judgment, we indulge every reasonable inference in favor of the non-mov-ant and resolve any doubts in his favor. See O’Bryant v. Century 21 South Central States, Inc., 899 S.W.2d 270, 271 (Tex.App.—Houston [14th Dist.] 1995, no writ). If the movant’s motion and summary judgment proof facially establish his right to judgment as a matter of law, then the burden shifts to the non-movant to raise fact issues precluding summary judgment. See City of Houston v. Clear Creek Basin Auth, 589 S.W.2d 671, 678 (Tex.1979). Moreover, if the non-movant relies on an affirmative defense to defeat summary judgment, he must present summary judgment proof sufficient to raise a fact issue as to each element of that defense. See Brownlee v. Brownlee, 665 S.W.2d 111, 112 (Tex.1984). With both standards of review in mind, we turn to the propriety of the trial court’s issuance of the permanent injunction.

B. Permanent Injunction Elements

On appeal, Rutherford complains the trial court erred in permanently enjoining his construction because Terramar has not demonstrated his violations of the deed restrictions will cause irreparable injury. Ordinarily, injunctive relief may only be granted upon a showing of (1) the existence of a wrongful act; (2) the existence of imminent harm; (3) the existence of irreparable injury; and (4) the absence of an adequate remedy at law. See Priest, 780 S.W.2d at 875. However, when the basis of the suit is enforcement of deed restrictions, the elements differ slightly. In Munson v. Milton, the court held “a movant seeking a temporary injunction to enforce a restrictive covenant is not required to show proof of irreparable injury.” 948 S.W.2d 813, 815 (Tex.App.—San Antonio 1997, pet. denied). Instead, the movant is only required to prove that the defendant intends to do an act that would breach the restrictive covenant. See id. Although Munson involved a temporary rather than a permanent injunction, we believe the rule applied in Munson also applies here. See Tien Too Ass’n v. Kingsbridge Park Community Ass’n, Inc., 953 S.W.2d 525, 529-530 (Tex.App.—Houston [1st Dist] 1997, no pet.) (holding a party must substantially violate a deed restriction before the trial court may issue a permanent injunction); see also Gigow-ski v. Russell, 718 S.W.2d 16, 21 (Tex. App. — Tyler 1986, writ ref d n.r.e.) (citing Protestant Episcopal Church Council v. McKinney, 339 S.W.2d 400, 403-04 (Tex. CivApp. — Eastland 1960, writ refd) for proposition that a well-settled exception in restrictive covenant cases allows issuance of a permanent injunction where a distinct or substantial breach is shown without regard to damages or irreparable injury).

The deed restrictions for Section 6, at issue here, were duly recorded with the County Clerk of Galveston County in 1969. It is undisputed that Rutherford purchased the lots with full knowledge of the setback and prior approval of building specifications deed restrictions. Because he purchased the property with knowledge of the recorded deed restrictions and then promptly violated the restrictions, the additional burden should not be upon Terra-mar to demonstrate irreparable injury. Thus, Rutherford’s contention that Terra-mar has not demonstrated irreparable injury has no effect on our analysis. See Tien Tao, 953 S.W.2d at 529-30; see also Munson, 948 S.W.2d at 816.

C. Application of Law to Facts

The trial court’s issuance of a permanent injunction was proper. First, Ter-ramar’s summary judgment proof demonstrates Rutherford substantially breached the setback and prior approval of building specifications deed restrictions. Rutherford admitted he never received approval for the setback violations for Lot 7. Further, the record clearly demonstrates the home Rutherford built on Lot 7 has side *850 setback lines seven and one-half feet from the edges of the lot, thereby violating the ten foot setback restriction by two and one-half feet. The record also demonstrates that when asked to stop construction on Lot 7 because of deed restriction violations, Rutherford did not stop. Instead, he contacted his attorney and sought his own injunction to enjoin Terra-mar from impeding his construction in the future. Moreover, Terramar submitted evidence that demonstrates Rutherford’s activities were considered substantial breaches of the deed restrictions by homeowners and board members. See Tien Tao, 953 S.W.2d at 531 (noting association’s perception of the violation as “substantial” constituted evidence of substantial breach).

Second, a balance of the equities demonstrates that those favoring Terramar significantly outweigh any equities favoring Rutherford. See Beere v. Duren, 985 S.W.2d 243, 247 (Tex.App.—Beaumont 1999, pet. denied) (citing rule from Cowling v. Colligan, 158 Tex. 458, 312 S.W.2d 943, 945-46 (1958) regarding the necessity of balancing the equities when enforcing restrictive covenants). As discussed, Rutherford purchased the property with knowledge of the restrictions. At the meeting, he represented he did not have to follow the deed restrictions. Finally, when informed his property was subject to the deed restrictions, he refused to halt construction and sought an injunction to keep Terramar at bay. By contrast, Terramar consulted its legal counsel following the meeting and repeatedly informed Rutherford he needed to comply with the existing deed restrictions. Further, when it became apparent Rutherford would not willingly comply, Terramar instituted legal action to force him into compliance with the deed restrictions. Thus, the equities of enforcing the deed restrictions favor Ter-ramar.

Based on this evidence, Terramar conclusively proved its entitlement to summary judgment as a matter of law. Therefore, the trial court did not abuse its discretion by permanently enjoining Rutherford from violating the Terramar Beach deed restrictions.

IV.

Affirmative Defenses

On appeal, Rutherford also challenges the trial court’s grant of Terramar’s motion for summary judgment based on his affirmative defenses. Rutherford insists there are fact issues to be resolved concerning three of his asserted defenses: waiver, estoppel, and laches. 1 Because Terramar moved for a Rule 166a(i) no-evidence summary judgment as to Rutherford’s affirmative defenses, we will review the trial court’s grant as we would a directed verdict. See Moore v. K Mart Corp., 981 S.W.2d 266, 269 (Tex.App.—San Antonio 1998, pet. denied); see also Judge David Hittner and Lynne Liberato, No-Evidence Summary Judgments Under the New Rule,, in State BaR of Texas PROF. Dev. Program, 20 ADVANCED Civil Trial D, D-5 (1997). The trial court may not grant a no-evidence summary judgment if the respondent brings forth more than a scintilla of probative evidence to raise a genuine issue of material fact. See Tex.R. Civ. P. 166a(i); see also Merrell Dow Pharm., Inc. v. Havner, 953 S.W.2d 706, 711 (Tex.1997). Less than a scintilla of evidence exists when the evidence is so weak as to do no more than create a mere surmise or suspicion of a fact. See Kindred v. Con/Chem, Inc., 650 S.W.2d 61, 63 (Tex.1983). The relevant inquiry is whether the non-movants asserting an affirmative defense have raised a fact issue upon each element *851 of that defense by evidence which would be admissible upon the trial of the case. See Finkelstein v. Southampton Civic Club, 675 S.W.2d 271, 278 (Tex.App.—Houston [1st Dist.] 1984, writ refd n.r.e.). We will begin our analysis with the affirmative defense of waiver.

A. Waiver

Waiver is the voluntary relinquishment of a known right. See Finkelstein, 675 S.W.2d at 278. To establish waiver in a deed restrictions case, the nonconforming user must prove that violations then existing are so great as to lead the mind of the “average man” to reasonably conclude that the restriction in question has been abandoned and its enforcement waived. See New Jerusalem Baptist Church, Inc. v. City of Houston, 598 S.W.2d 666, 669 (Tex.Civ.App.—Houston [14th Dist.] 1980). Among the factors to be considered by the “average man” are the number, nature, and severity of the then existing violations, any prior acts of enforcement of the restriction, and whether it is still possible to realize to a substantial degree the benefits intended through the covenant. See id.

First, Rutherford argues that Ter-ramar waived its setback deed restriction as to Lot 7 when it approved his five foot setbacks on Lots 8 and 9. Based on his assertions, our inquiry is whether an average person could reasonably conclude that based on the prior approvals, Terramar had abandoned its setback and approval restrictions. However, Rutherford admitted that although Terramar approved Lots 8 and 9, he knew Terramar did not approve the Lot 7 setback violation. In fact, Rutherford admitted he did not submit the required specifications in order to obtain approval for Lot 7. Further, within ten days of the association meeting, Rutherford was on notice that Terramar would not allow his violation of either the approval or setback restrictions as to Lot 7. 2

Second, Rutherford claims Terra-mar waived its setback restriction because four other homes in Section 6 of the subdivision violate the deed restrictions. 3 However, the deed restrictions in question contain a severability clause, stating “[t]he invalidity, violation, abandonment or waiver of any one or more of or any part of the Restrictions shall in no wise affect or impair the remaining Restrictions or parts therof which shall remain in full force and effect.” Therefore, we will not consider the alleged waiver of deed restrictions other than side setback violations in our analysis of the setback restriction. See Tanglewood Homes Ass’n v. Henke, 728 S.W.2d 39, 41-42 (Tex.App.—Houston [1st Dist.] 1987, writ refd n.r.e.). Thus, the only support for Rutherford’s waiver argument is the evidence of one other setback violation in Section 6.

In Tanglewood, the court was faced with five violations of a certain deed restriction which the appellees, in violation of that restriction, argued was abandoned. The court adverted to this court’s analysis of the issue of waiver of deed restrictions in New Jerusalem Baptist Church, 598 *852 S.W.2d at 669. This court in New Jerusalem stated that to establish the affirmative defense of waiver in a deed restriction case, the non-conforming user must prove that the violations then existing are so great as to lead the average man to conclude the restriction in question had been abandoned and its enforcement waived. See id. Among the factors to be considered by the average man are the number, nature, and severity of the then existing violations, any prior acts of enforcement, and whether it is still possible to realize the benefits intended through the covenant. See id.

In New Jerusalem this court held that four non-conforming uses within the subdivision did not constitute abandonment of the restrictions. See id. Similarly, in Tanglewood, the court held that five violations of the main residence restriction were insufficient in number, nature and severity to constitute a waiver of the restrictive covenant’s benefits. See Tanglewood, 728 S.W.2d at 44. Here, because Rutherford points to only one violation of the side setback restriction, we hold that one violation is not so great as to lead the mind of the average man to reasonably conclude that the restriction in question had been abandoned and its enforcement waived. See New Jerusalem, 598 S.W.2d at 669.

B. Estoppel

Estoppel has been defined as follows:

the effect of the voluntary conduct of a party whereby he is absolutely precluded, both at law and in equity, from asserting rights which might perhaps have otherwise existed, either of property, of contract, or of remedy, as against another person, who has in good faith relied upon such conduct, and has been led thereby to change his position for the worse, and who on his part acquires some corresponding right, either of property, of contract, or of remedy.

Finkelstein, 675 S.W.2d at 278. Estoppel does not apply in this case primarily because Rutherford presented no evidence that he changed his position regarding the setback violations on Lot 7 based on Terramar’s conduct.

At the meeting, Rutherford announced he intended to begin construction on Lot 7 in two days. His plans demonstrated a violation of the setback restriction at the time of the meeting and at all times thereafter. Thus, Rutherford cannot demonstrate he changed position in reliance on Terramar’s actions when his position, the violation of the setback restrictions, never changed. Having failed to present more than a scintilla of probative evidence as to each element of estoppel, Rutherford may not rely on this affirmative defense. We now address Rutherford’s final defense, laches.

C. Laches

Laches, like waiver and estop-pel, is an affirmative defense, 4 which Rutherford had the burden to prove. See Caldwell v. Barnes, 975 S.W.2d 535, 538 (Tex.1998); see also City of Houston v. Muse, 788 S.W.2d 419, 422 (Tex.App.—Houston [1st Dist.] 1990, no writ). The two elements of laches are: (1) an unreasonable delay in asserting a legal or equitable right; and (2) a good faith change of position by Rutherford to his detriment in reliance upon the delay. See Muse, 788 S.W.2d at 422. When a party takes no steps to enforce its known rights until the other party has, in good faith, so changed its position that it cannot be restored to its former state, the delay becomes inequitable and may estop the assertion of the right. See Id., see also Culver v. Pickens, 142 Tex. 87, 91, 176 S.W.2d 167, 170-71 (1943). As a general rule, laches does not bar a plaintiffs suit before the statute of limitations has run unless estoppel or “extraordinary circumstances” are present. See Green v. Parrack, 974 S.W.2d 200, 204 (Tex.App.—San Antonio 1998, no pet.). *853 Moreover, laches does not apply where the defendant has acted in open and known hostility to a plaintiffs rights and has not been misled by the plaintiffs apparent acquiescence. See Muse, 788 S.W.2d at 423.

Here, there are no extraordinary circumstances, and as previously discussed, no estoppel is present. Rutherford began construction on Lot 7, without the requisite approval, on January 12, 1998. Terra-mar’s legal counsel wrote him a letter discussing his compliance with the setback restriction on February 5, 1998. When Rutherford refused to halt construction or comply with the restriction, Terramar filed suit on February 18, 1998. Therefore, Terramar did not unreasonably delay in asserting its legal right to enforce the restrictions. Thus, Rutherford has not met his burden of bringing forth more than a scintilla of probative evidence as to both elements of laches. Because Rutherford failed to adequately support each element of his affirmative defenses, the trial court correctly granted Terramar’s no-evidence summary judgment. Accordingly, we affirm that portion of the trial court’s judgment.

Attorney’s Fees

Finally, Terramar cross-appeals the trial court’s denial of attorney’s fees. Terramar appeals this portion of the trial court’s order asserting that reasonable attorney’s fees are mandated by the property code for the prevailing party in an action to enforce deed restrictions. See Tex. PROp.Cobe Ann. § 5.006(a) (Vernon 1984); see also Beere, 985 S.W.2d at 249. We agree with Terramar, inasmuch as the issue of whether attorney’s fees are authorized in a particular case is a question of law to be determined by the court. See id. However, the determination of the amount to be awarded as reasonable attorney’s fees is a question of fact to be determined by the trier of fact. See id. Here, the trial court should have awarded reasonable attorney’s fees. Therefore, the trial court erred by refusing to award attorney’s fees to Terramar, the prevailing party. However, the award, if any, must be supported by competent evidence. See id. Although Terramar claims attorney’s fees in the amount of $25,000 at the trial level, $10,000 at the intermediate appellate level, and $7,500 if the case is appealed to the Texas Supreme Court, section 5.006 only authorizes the award of reasonable attorney’s fees. That statute also provides the trial court with four factors to consider in its determination of reasonable attorney’s fees. 5 Terramar submitted evidence of all factors for the court to consider except evidence relating to the time and labor factors. Because the amount of fees is a fact question, we remand this case to the trial court to award Terramar reasonable attorney’s fees, based on the factors outlined in section 5.006(b) of the Property Code. Accordingly, we sustain Terramar’s cross-point on appeal.

VI.

Conclusion

We affirm the trial court’s judgment permanently enjoining Rutherford from violating Terramar Beach’s deed restrictions in the future. We likewise affirm the trial court’s grant of Terramar’s motion for a no-evidence summary judgment as to Rutherford’s affirmative defenses of waiver, estoppel, and laches. However, we reverse the trial court’s denial of attorney fees for Terramar, the prevailing party in this suit to enforce deed restrictions, and remand this case to the trial court for *854 further proceedings consistent with this opinion.

. Rutherford also raised the affirmative defense of fraud at trial. However because he has not challenged the summary judgment's denial of that claim on appeal, we do not reach the issue of whether he brought forth more than a scintilla of evidence on each element of fraud so as to defeat Terramar’s no evidence summary judgment on Rutherford's affirmative defenses.

. Arguably, Rutherford was on notice seven days after the meeting that the approval restriction was still in force because his foreman, Sherman Eagleton, signed and returned the "Important Notice to All Terramar Property Owners and Builders Requirements to Build.” This document was submitted with the application fee to the Association. The Notice states: (1) that two complete plans with specifications "shall” be furnished to the Architectural Control Committee; (2) that the plans are subject to approval by the Committee; (3) that violation of these requirements "shall be sufficient cause for legal process against owner, builder, and contractor;” (4) that plan approval is required fifteen days prior to start of construction, and (5) that the Architectural Chairman "shall have the owner to halt construction if requirements are not met.” Rutherford admitted Eagleton, as his agent, had authority to sign this agreement and bind him.

. In support of his waiver argument, Rutherford submitted the affidavit of David Watson. Watson noted numerous deed restriction violations in Sections 3, 4, 6, and the Marina, but only one setback violation in Section 6.

. See Tex.R. Civ. P. 94.

. "To determine reasonable attorney’s fees, the court shall consider:

(1) the time and labor required;

(2) the novelty and difficulty of the questions;

(3) the expertise, reputation, and ability of the attorney; and

(4) any other factor.”

Tex Prop Code Ann. § 5.006(b) (Vernon 1984).