This driver was originally written to use the regmap abstraction with no
clear benefit. As the registers are always mmio and there is no sharing
of the region with other devices, we can safely read and write without
the locking that regmap provides.

This reduces the code size of the driver by about 25%.
Signed-off-by: Joel Stanley <joel@jms.id.au>
Message-Id: <20210903051039.307991-1-joel@jms.id.au>
Signed-off-by: Corey Minyard <cminyard@mvista.com>