The commit "gpio: pxa: change the interrupt management" should have
taken care of moving an ifdef to not englobe irqdomain related
structures anymore, as they are used now for all builds.

This repairs the broken builds where CONFIG_OF=n.
Signed-off-by: Robert Jarzmik <robert.jarzmik@free.fr>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>