Commit b32aadc1 authored by Shreyas B. Prabhu's avatar Shreyas B. Prabhu Committed by Michael Ellerman

powerpc/powernv: Fix race in updating core_idle_state

core_idle_state is maintained for each core. It uses 0-7 bits to track
whether a thread in the core has entered fastsleep or winkle. 8th bit is
used as a lock bit.
The lock bit is set in these 2 scenarios-
 - The thread is first in subcore to wakeup from sleep/winkle.
 - If its the last thread in the core about to enter sleep/winkle

While the lock bit is set, if any other thread in the core wakes up, it
loops until the lock bit is cleared before proceeding in the wakeup
path. This helps prevent race conditions w.r.t fastsleep workaround and
prevents threads from switching to process context before core/subcore
resources are restored.

But, in the path to sleep/winkle entry, we currently don't check for
lock-bit. This exposes us to following race when running with subcore
on-

First thread in the subcorea		Another thread in the same
waking up		   		core entering sleep/winkle

lwarx   r15,0,r14
ori     r15,r15,PNV_CORE_IDLE_LOCK_BIT
stwcx.  r15,0,r14
[Code to restore subcore state]

						lwarx   r15,0,r14
						[clear thread bit]
						stwcx.  r15,0,r14

andi.   r15,r15,PNV_CORE_IDLE_THREAD_BITS
stw     r15,0(r14)

Here, after the thread entering sleep clears its thread bit in
core_idle_state, the value is overwritten by the thread waking up.
In such cases when the core enters fastsleep, code mistakes an idle
thread as running. Because of this, the first thread waking up from
fastsleep which is supposed to resync timebase skips it. So we can
end up having a core with stale timebase value.

This patch fixes the above race by looping on the lock bit even while
entering the idle states.
Signed-off-by: default avatarShreyas B. Prabhu <shreyas@linux.vnet.ibm.com>
Fixes: 7b54e9f213f76 'powernv/powerpc: Add winkle support for offline cpus'
Cc: stable@vger.kernel.org # 3.19+
Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
parent a8956a7b
...@@ -51,6 +51,22 @@ ...@@ -51,6 +51,22 @@
.text .text
/*
* Used by threads when the lock bit of core_idle_state is set.
* Threads will spin in HMT_LOW until the lock bit is cleared.
* r14 - pointer to core_idle_state
* r15 - used to load contents of core_idle_state
*/
core_idle_lock_held:
HMT_LOW
3: lwz r15,0(r14)
andi. r15,r15,PNV_CORE_IDLE_LOCK_BIT
bne 3b
HMT_MEDIUM
lwarx r15,0,r14
blr
/* /*
* Pass requested state in r3: * Pass requested state in r3:
* r3 - PNV_THREAD_NAP/SLEEP/WINKLE * r3 - PNV_THREAD_NAP/SLEEP/WINKLE
...@@ -150,6 +166,10 @@ power7_enter_nap_mode: ...@@ -150,6 +166,10 @@ power7_enter_nap_mode:
ld r14,PACA_CORE_IDLE_STATE_PTR(r13) ld r14,PACA_CORE_IDLE_STATE_PTR(r13)
lwarx_loop1: lwarx_loop1:
lwarx r15,0,r14 lwarx r15,0,r14
andi. r9,r15,PNV_CORE_IDLE_LOCK_BIT
bnel core_idle_lock_held
andc r15,r15,r7 /* Clear thread bit */ andc r15,r15,r7 /* Clear thread bit */
andi. r15,r15,PNV_CORE_IDLE_THREAD_BITS andi. r15,r15,PNV_CORE_IDLE_THREAD_BITS
...@@ -294,7 +314,7 @@ lwarx_loop2: ...@@ -294,7 +314,7 @@ lwarx_loop2:
* workaround undo code or resyncing timebase or restoring context * workaround undo code or resyncing timebase or restoring context
* In either case loop until the lock bit is cleared. * In either case loop until the lock bit is cleared.
*/ */
bne core_idle_lock_held bnel core_idle_lock_held
cmpwi cr2,r15,0 cmpwi cr2,r15,0
lbz r4,PACA_SUBCORE_SIBLING_MASK(r13) lbz r4,PACA_SUBCORE_SIBLING_MASK(r13)
...@@ -319,15 +339,6 @@ lwarx_loop2: ...@@ -319,15 +339,6 @@ lwarx_loop2:
isync isync
b common_exit b common_exit
core_idle_lock_held:
HMT_LOW
core_idle_lock_loop:
lwz r15,0(14)
andi. r9,r15,PNV_CORE_IDLE_LOCK_BIT
bne core_idle_lock_loop
HMT_MEDIUM
b lwarx_loop2
first_thread_in_subcore: first_thread_in_subcore:
/* First thread in subcore to wakeup */ /* First thread in subcore to wakeup */
ori r15,r15,PNV_CORE_IDLE_LOCK_BIT ori r15,r15,PNV_CORE_IDLE_LOCK_BIT
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment