I'm not only talking about defamation, I'm talking about insults as well.
If I call you a [racist epithet], that does not damage your reputation, and would not normally be considered defamation, I believe.
Exactly, but it's still an insult, and clearly intended as such (what else?), and as such a punishable offense in many jurisdictions, including mine. I don't have a problem with that.
That's what the defamation laws are for.
Does that mean that you'd have no redress in law against simple, non-defamatory insults (somebody calling you an idiot, say, something along these lines)?
Mit "Freedom of Speech" hat das nichts zutun ...
Natürlich hat das mit Meinungsäußerungsfreiheit zu tun, was sonst? Ob eine Einschränkung (dauerhaft) gerechtfertigt ist, ist ein anderes, aber man soll nicht so tun als ob es keine wäre.