The Governor may remove any board member for incompetence, neglect of duty or corruption upon first giving him a copy of the charges against him and an opportunity to be heard.