ADAPT now is added not only for HS Gear4 mode but also higher gears.  Fix
the logic for higher gears.

Link: https://lore.kernel.org/r/20220616053725.5681-3-stanley.chu@mediatek.comReviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>