Wait with the destruction of cpufreq objects until all references
are gone (code partly based on rmk's comparable patch for the PCMCIA
subsystem.

From Dominik.