The governor may remove any member of the board for unprofessional conduct, incompetency, or neglect of duty after giving such member a written statement of the reasons for removal and after such member has had an opportunity to be heard thereon.