This reverts commit d18fddff.

This patch came on top of another patch that introduced a regression.
Revert it before addressing the culprit.
Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>