I can think of a few more situations where voicemail is useful. If a person tries to call you and is going to be out of contact shortly thereafter, they can leave a message with what they want/need to say without you having to call them back. Also, I kind of like filtering the calls I return. If, for example, my friend calls, I'll almost always call them back. However, if my boss calls, I like to listen to the voicemail message before calling him back. One last reason I can think of where it's handy is when someone you know is trying to tell you something bad has happened (for example, a friend or relative was in a car accident). Sure, they can keep trying, but chances are you'll get the voicemail before you get another call from them. There are probably a few more obvious reasons that voicemail is useful, but I can't think of them now.
That being said, I totally understand why you don't want it. It nearly forces you to receive a message sent to you, regardless of if you're trying to avoid it or not. It's rare that it bugs, so its hard to blame it on "the system," I guess.