Let’s not waste time: The neo-nazi and white supremacist degenerates who gathered in Charlottesville, Virginia over the weekend are despicable examples of human excrement. See, that’s not so hard to say.

This is not just a question of violence. Observers on the ground in Charlottesville point out that both the nazis and the anti-fa counter protestors contributed to the fighting (although notably none of the nazis ended up dead). Denouncing the violence is easy and lends itself to the type of “plague on both your houses” type of rhetoric that lets us off the hook. The problem isn’t just the violence that accompanied the nazis and racists, it’s their ideas.

Now, I’m pretty damn close to an absolutist when it comes to the first amendment. I think that if a bunch of escapees from their mothers basement want to parade around with tiki torches and preach hate, they have a legal right to do so. I’d be opposed to any law that tried to deny them that right.

But the fact that they have a right to be scum, doesn’t bind me to silence. A right to say something doesn’t mean a right to say something without consequences. Too many – including far too many libertarians – stop at the first part of the equation, the right to speak. But if we truly believe in a philosophy that is premised on the equal worth of every human being, then we have an obligation to speak out against racism (and sexism, homophobia, xenophobia, Islamophobia, and every other form of bigotry).

And please, no “whatsboutism.” Yes, some of the rhetoric issuing from the left is beyond the pale. It is wrong to use force and violence to prevent speech you disagree with. And, the idiots traipsing around in Che Guevara tshirts might as well be wearing swastikas. But A) that’s irrelevant, and B) doesn’t take into account how bigoted speech impacts people who have suffered from bigotry since this country was founded. We owe a special type of debt to the people that this country has oppressed. The least we can do is say that speech that continues to denigrate them is wrong.

“All that is necessary for the triumph of evil is for good men to do nothing,” goes the quote attributed, perhaps incorrectly, to Edmund Burke. Libertarianism cannot be so thin that it doesn’t recognize racism (and other forms of bigotry) as evil. And recognizing them as evil, we should take action to denounce them, to ostracize them, and to drive them out of civilized society.

We should demand this from our political leaders, including our president (who has so far failed the test), but most of all from ourselves. Indeed, because we believe in liberty for all, that obligation lies even heavier on us. And if we can’t see that, then how different are we really from those losers in Charlotte?