Packed mode expects minimum transfer length of 4 bytes.

This patch fixes this by using unpacked mode for transfers less
than 4 bytes.
Signed-off-by: Sowjanya Komatineni <skomatineni@nvidia.com>
Signed-off-by: Mark Brown <broonie@kernel.org>