But I'm not all about burning them so what's it matter?
Well, for one, "fag" is a shortening of "fagot", which is "a bundle of sticks, twigs, or branches bound together and used as fuel, a fascine, a torch, etc."
1, so, by calling a homosexual a "fag", you're insinuating that they are useful for nothing more than being burned (which is what they used to do to homosexuals: burn them at the stake).