If the voice ever does say it's your time to die, I'd imagine that would be one of the better ways to die - no disease, no pain, no suffering. I'd much prefer a voice telling me it's my time to die in the middle of the night to (skip these if you want to, I'm just listing ways to die) say, a car (or any vehicle) crash, getting shot, dying of a disease, drowning, being tortured to death, a random violent crime against me, a terrorist attack, a gas explosion, an earthquake, a flood, starvation/thirst, etc...
If that is the way you die, you have it much easier than most people.
If not, the worst that can happen is the fear you're currently experiencing.
Either way, if you worry about it all you're going to do is ruin your life with fear. I know that it's possible that New York City could be nuked by a terrorist or someone, but it isn't something I worry about. If I did, I would never get anywhere.
Someone else on the forum may give you advice on how to make it stop, or that it isn't possible (to die that way), but I'm telling you now that it isn't something to worry about.