People that listen to music as they walk around annoy me. There's like maybe one or two guys that do it here and I just think it takes away from any possibility of social interaction. What's even worse is people that walk around the mall with their headphones on.
I think it depends on the situation. If I either don't want social interaction or am doing a long walk, I'll wear headphones (although I pull them out as soon as interaction seems possible).
I frequently walk from my parent's place to my place in the late evening (10pm or later), which is an hour-long walk down a deserted street, then down a long creek path. There is basically no chance of human interaction, so wearing headphones is perfectly justifiable to me.
Also, on the bus, I usually wear headphones and read. Also at work I wear headphones when I'm reading -- those are times when I'm actively removing the possibility of distractions (ie, social interaction). When I'm programming or studying at home, and I don't want to hear the phone ring or anything like that (I live alone), I'll also wear headphones. Again, it's to get rid of distractions.
So yeah, precluding social interaction is often the point.