Hm, now that you mention it, I think X11 would take 20MB alone. 32MB of RAM might be a bottleneck, but the 400MHz wont be.
Its perfectly acceptable as a server. Unless the client computer is connected over 100Mbps ethernet, connection speed will be too bottleneckish for the processor to matter, and apache can just use the pagefile (so can X11, I guess) if needed (it really shouldn't be, but eh?).