The Governor may remove any member of the board for misconduct, incompetency or neglect of duty.