Yes, you can limit bandwidth based on anything: ip, port, protocol (layer 2, 3, 4, and 7), and any other check that iptables can do.
Incidentally, that's called "packet shaping"
I have a script that sets it up, but I suddenly can't access the computer that it's on. I'll post it here later if I remember.