Every chess professional agrees that chess is a sport. They would know better than you.
The definition of a sport is a little ambiguous; it can either be (1) a competitive activity, or (2) a competitive activity that involves physical exertion. In the case of (1) there is no dispute. In the case of (2), I will let you know that chess professionals consider it an important part of their training to keep physically fit. The two contenders for the upcoming World Championship have both mentioned in interviews that they are doing so. Kramnik goes swimming or running every day, and Anand goes to the gym everyday. When you have to retain concentration on a board for 6 or so hours, fitness becomes a factor.