Also, I think people forget the political expediency of the times when the issue of who was "for" and "against" slavery comes about.
Lincoln was from Illinois, after all, a state that hasn't exactly had the cleanest running political machines.