Avoid iterating the child devices.

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/841c226efa424701161dd9f1793e0cf96b45a07c.1559308269.git.jani.nikula@intel.com