Commit 61e98ebf authored by Anton Blanchard's avatar Anton Blanchard Committed by Michael Ellerman

powerpc: Remove static branch prediction in atomic{, 64}_add_unless

I see quite a lot of static branch mispredictions on a simple
web serving workload. The issue is in __atomic_add_unless(), called
from _atomic_dec_and_lock(). There is no obvious common case, so it
is better to let the hardware predict the branch.
Signed-off-by: default avatarAnton Blanchard <anton@samba.org>
Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
parent bb85fb58
...@@ -233,7 +233,7 @@ static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u) ...@@ -233,7 +233,7 @@ static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
PPC_ATOMIC_ENTRY_BARRIER PPC_ATOMIC_ENTRY_BARRIER
"1: lwarx %0,0,%1 # __atomic_add_unless\n\ "1: lwarx %0,0,%1 # __atomic_add_unless\n\
cmpw 0,%0,%3 \n\ cmpw 0,%0,%3 \n\
beq- 2f \n\ beq 2f \n\
add %0,%2,%0 \n" add %0,%2,%0 \n"
PPC405_ERR77(0,%2) PPC405_ERR77(0,%2)
" stwcx. %0,0,%1 \n\ " stwcx. %0,0,%1 \n\
...@@ -539,7 +539,7 @@ static __inline__ int atomic64_add_unless(atomic64_t *v, long a, long u) ...@@ -539,7 +539,7 @@ static __inline__ int atomic64_add_unless(atomic64_t *v, long a, long u)
PPC_ATOMIC_ENTRY_BARRIER PPC_ATOMIC_ENTRY_BARRIER
"1: ldarx %0,0,%1 # __atomic_add_unless\n\ "1: ldarx %0,0,%1 # __atomic_add_unless\n\
cmpd 0,%0,%3 \n\ cmpd 0,%0,%3 \n\
beq- 2f \n\ beq 2f \n\
add %0,%2,%0 \n" add %0,%2,%0 \n"
" stdcx. %0,0,%1 \n\ " stdcx. %0,0,%1 \n\
bne- 1b \n" bne- 1b \n"
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment