The controller is so important because, if they are able to efficiently allocate and remap clusters during writes, performance improves.
It's kind of like accessing an array of pointers. Everything has to get remapped via the SSD controller. The controller is what is responsible for dereferencing the pointers.