It affects the integrity of elections, and in close elections with razor thin margins legitimacy matters most because of the large effect an election can have on public policy. So yes, depending on who you ask in which election, society can definitely be "hurt" by election fraud, or fraud of any kind. When Kennedy beat Nixon there were widespread reports of precincts reporting over 100% turnout. That election determined the opening of China to US trade, the end of Vietnam, and the formation of the EPA. And it's true, not all election fraud is caused by illegitimate voters. Some of it is due to ballot stuffing, invalid registrations, and electronic manipulation. I also disagree with any kind of centralized federal ID. I prefer to keep it at the state/local level. Indeed. It depends on how retailers react to losses from fraud.