Well, to start the ball rolling:
I don't think these are really interchangeable at all, and I'm perplexed by your perception of the difference, too.
To me, "When did I..." is the normal question in standard English. The answer could be "on several occasions" (fifty times, every time we meet, don't you ever listen to me?), not just "on one occasion"..
"When have I .." sounds more like a protest, in a context such as this:
-"You're always telling me about his cat. I get fed up of it."
"When have I [ever] told you about his cat?"
I don't think that "when have I"... would be normal outside this kind of context, and I'm not sure that it would be used in all varieties/dialects of English.
As for "when has he..." etc., it's about the same. I can imagine a conversation like this:
"Sometimes he even left the cat indoors for two days at a time"
"When has he done that, then?"
Here, too, I think this is regional rather than standard. Someone speaking an entirely standard form of British English question would ask:
"When did he do that, then?"