Commit ffbbdd21
"spi: create a message queueing infrastructure"
Accidentally deleted the logic to disable the port
when unused leading to higher power consumption.
Fix this up.

Cc: stable@kernel.org
Cc: Vinit Shenoy <vinit.shenoy@st.com>
Signed-off-by: Virupax Sadashivpetimath <virupax.sadashivpetimath@stericsson.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>