The Comptroller General shall immediately commence and pursue against all county treasurers reported by the State Treasurer as being in default in making their returns such legal measures as will be best calculated to compel an immediate compliance with the duty imposed upon them in regard to such returns and shall enforce the performance of their duty generally by all legal means. The Attorney General and the solicitor of the circuit in which such default may occur shall conduct such legal proceedings when called upon to do so by the Comptroller General.