I think this started out from horror movies where bad things happen once someone looks into the mirror... it got planted in people's minds, they told more and more people and it became a general concept that mirrors = scary stuff.

Personally mirrors freak me out, even in real life if i stare at my reflection for a while... Try it for yourself. Just keep staring at your reflection and after a while it might feel like it's another person or that your reflection is alive or something O.o