If power domain information are missing in the device tree, no
power domains get initialized. However, imx_gpc_remove tries to
remove power domains always in the old DT binding case. Only
remove power domains when imx_gpc_probe initialized them in
first place.

Fixes: 721cabf6 ("soc: imx: move PGC handling to a new GPC driver")
Signed-off-by: Stefan Agner <stefan@agner.ch>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Reviewed-by: Fabio Estevam <fabio.estevam@nxp.com>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>