Commit 9d84fb27 authored by Mark Rutland's avatar Mark Rutland Committed by Catalin Marinas

arm64: restore get_current() optimisation

Commit c02433dd ("arm64: split thread_info from task stack")
inverted the relationship between get_current() and
current_thread_info(), with sp_el0 now holding the current task_struct
rather than the current thead_info. The new implementation of
get_current() prevents the compiler from being able to optimize repeated
calls to either, resulting in a noticeable penalty in some
microbenchmarks.

This patch restores the previous optimisation by implementing
get_current() in the same way as our old current_thread_info(), using a
non-volatile asm statement.
Acked-by: default avatarWill Deacon <will.deacon@arm.com>
Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
Reported-by: default avatarDavidlohr Bueso <dbueso@suse.de>
Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
parent 6ef4fb38
...@@ -9,9 +9,17 @@ ...@@ -9,9 +9,17 @@
struct task_struct; struct task_struct;
/*
* We don't use read_sysreg() as we want the compiler to cache the value where
* possible.
*/
static __always_inline struct task_struct *get_current(void) static __always_inline struct task_struct *get_current(void)
{ {
return (struct task_struct *)read_sysreg(sp_el0); unsigned long sp_el0;
asm ("mrs %0, sp_el0" : "=r" (sp_el0));
return (struct task_struct *)sp_el0;
} }
#define current get_current() #define current get_current()
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment