Casting a pointer to unsigned long only to immediately cast it back
to a pointer makes no sense.  Fix this.
Tested-by: Christian Lamparter <chunkeey@googlemail.com>
Signed-off-by: Mans Rullgard <mans@mansr.com>
Signed-off-by: Tejun Heo <tj@kernel.org>