I really like to have a glass of tap water available at meals along with whatever I actually order, whether it's just a diet Coke or beer or wine. Just because I have water doesn't mean I'm not going to order a drink, or at least, usually not.
I don't really care what the tap water tastes like. I don't think I could identify chlorine or any other particular taste, but I'm used to the idea that tap water from different places just tastes different. (You all should have seen the well water at my grandparents' house, which came out of the tap yellowish-brown and smelled like sulphur.) I just assume that as long as it's in a developed country, it's safe to drink.
I've always been sort of scared to ask for tap water in Europe (well, I'm always scared to ask waiters anything if I have to do it in German), but really, those times when I have been brave and done it, I don't recall that a waiter ever said, No, you can't do that, you have to buy a bottle.
That did happen to me once (which is partly why I've been scared to ask ever since), but it was in Belgium.
So maybe a lot more places would be willing to bring tap water if people would only ask. I would think it would even be a green thing to do, especially in environmentally conscious Germany. Why waste all that plastic on bottles? (-:
One more thing: At American restaurants, the first cup of coffee or tea isn't free, only the refills.