That's what worries me about some of the anti-vaccine folks.
The worst case scenario is that we always have enough people without immunity that not only is there a reservoir of small-scale transmission that is going on for years, stopping the chance to eradicate it, but that within that reservoir of transmission it mutates into something that the vaccine no longer offers protection for, or something with much higher lethality, or both(!).