Nets: Specter 'Driven Out' of GOP by 'Right Wing' and 'Fringe of Party'