This is only used on arm and arm64 platforms. Add COMPILE_TEST option.

Tested with 5.1-rc3+ on Fedora/RISCV. CONFIG_ARM_TIMER_SP804 no more shows
up in riscv config.
Signed-off-by: David Abdurachmanov <david.abdurachmanov@gmail.com>
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>