We can't use "adap->dev" after it has been freed.

Fixes: 5bf4fa7d ("i2c: break out OF support into separate file")
Signed-off-by: Wen Yang <wenyang@linux.alibaba.com>
Signed-off-by: Wolfram Sang <wsa@the-dreams.de>