They're the Linux equivilant of "explorer.exe". They look after all the front-end graphical stuff.
X-Windows Server looks after programs connecting to the interface, where their windows are, the resolution, and where the mouse is clicked, not much more.
KDE and Gnome take care of what the Windows look like, how they are moved, and how they are resized. They also provide graphical tools for configuring your computer, icons on your desktop, a "start"-style menu, wallpaper, etc. Anything that you see, basically, is provided by your window manager (KDE and Gnome)