- 14 Dec, 2015 32 commits
-
-
Boqun Feng authored
According to memory-barriers.txt, xchg*, cmpxchg* and their atomic_ versions all need to be fully ordered, however they are now just RELEASE+ACQUIRE, which are not fully ordered. So also replace PPC_RELEASE_BARRIER and PPC_ACQUIRE_BARRIER with PPC_ATOMIC_ENTRY_BARRIER and PPC_ATOMIC_EXIT_BARRIER in __{cmp,}xchg_{u32,u64} respectively to guarantee fully ordered semantics of atomic{,64}_{cmp,}xchg() and {cmp,}xchg(), as a complement of commit b97021f8 ("powerpc: Fix atomic_xxx_return barrier semantics") This patch depends on patch "powerpc: Make value-returning atomics fully ordered" for PPC_ATOMIC_ENTRY_BARRIER definition. Cc: stable@vger.kernel.org # 3.2+ Signed-off-by:Boqun Feng <boqun.feng@gmail.com> Reviewed-by:
Paul E. McKenney <paulmck@linux.vnet.ibm.com> Acked-by:
Peter Zijlstra (Intel) <peterz@infradead.org> Signed-off-by:
Michael Ellerman <mpe@ellerman.id.au>
-
Boqun Feng authored
According to memory-barriers.txt: > Any atomic operation that modifies some state in memory and returns > information about the state (old or new) implies an SMP-conditional > general memory barrier (smp_mb()) on each side of the actual > operation ... Which mean these operations should be fully ordered. However on PPC, PPC_ATOMIC_ENTRY_BARRIER is the barrier before the actual operation, which is currently "lwsync" if SMP=y. The leading "lwsync" can not guarantee fully ordered atomics, according to Paul Mckenney: https://lkml.org/lkml/2015/10/14/970 To fix this, we define PPC_ATOMIC_ENTRY_BARRIER as "sync" to guarantee the fully-ordered semantics. This also makes futex atomics fully ordered, which can avoid possible memory ordering problems if userspace code relies on futex system call for fully ordered semantics. Fixes: b97021f8 ("powerpc: Fix atomic_xxx_return barrier semantics") Cc: stable@vger.kernel.org # 3.2+ Signed-off-by:
-
Aneesh Kumar K.V authored
The slot information of base page size hash pte is stored in the pgtable_t w.r.t transparent hugepage. We need to make sure we don't index beyond pgtable_t size. Signed-off-by:
Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> Signed-off-by:
-
Aneesh Kumar K.V authored
This will bulk read 4 hash pte slot entries and should reduce the loop Signed-off-by:
-
Aneesh Kumar K.V authored
Remove the related functions and #defines Signed-off-by:
-
Aneesh Kumar K.V authored
They don't need to track 4k subpage slot details and hence don't need second half of pgtable_t. Signed-off-by:
-
Aneesh Kumar K.V authored
Use the #define instead of open-coding the same Signed-off-by:
-
Aneesh Kumar K.V authored
pte and pmd table size are dependent on config items. Don't hard code the same. This make sure we use the right value when masking pmd entries and also while checking pmd_bad Signed-off-by:
-
Aneesh Kumar K.V authored
For a pte entry we will have _PAGE_PTE set. Our pte page address have a minimum alignment requirement of HUGEPD_SHIFT_MASK + 1. We use the lower 7 bits to indicate hugepd. ie. For pmd and pgd we can find: 1) _PAGE_PTE set pte -> indicate PTE 2) bits [2..6] non zero -> indicate hugepd. They also encode the size. We skip bit 1 (_PAGE_PRESENT). 3) othewise pointer to next table. Acked-by:
Scott Wood <scottwood@freescale.com> Signed-off-by:
-
Aneesh Kumar K.V authored
We support THP only with book3s_64 and 64K page size. Move THP details to hash64-64k.h to clarify the same. Acked-by:
-
Aneesh Kumar K.V authored
W.r.t hugetlb, we support two format for pmd. With book3s_64 and 64K linux page size, we can have pte at the pmd level. Hence we don't need to support hugepd there. For everything else hugepd is supported and pmd_huge is (0). Acked-by:
-
Aneesh Kumar K.V authored
Only difference here is, we apply the WIMG mapping early, so rflags passed to updatepp will also be changed. Acked-by:
-
Aneesh Kumar K.V authored
Instead of open coding it in multiple code paths, export the helper and add more documentation. Also make sure we don't make assumption regarding pte bit position Acked-by:
-
Aneesh Kumar K.V authored
This is similar to 64K insert. May be we want to consolidate Acked-by:
-
Aneesh Kumar K.V authored
Convert from asm to C Acked-by:
-
Aneesh Kumar K.V authored
No real change, only style changes Acked-by:
-
Aneesh Kumar K.V authored
Signed-off-by:
-
Aneesh Kumar K.V authored
This free up 11 bits in pte_t. In the later patch we also change the pte_t format so that we can start supporting migration pte at pmd level. We now track 4k subpage valid bit as below If we have _PAGE_COMBO set, we override the _PAGE_F_GIX_SHIFT and _PAGE_F_SECOND. Together we have 4 bits, each of them used to indicate whether any of the 4 4k subpage in that group is valid. ie, [ group 1 bit ] [ group 2 bit ] ..... [ group 4 ] [ subpage 1 - 4] [ subpage 5- 8] ..... [ subpage 13 - 16] We still track each 4k subpage slot number and secondary hash information in the second half of pgtable_t. Removing the subpage tracking have some significant overhead on aim9 and ebizzy benchmark and to support THP with 4K subpage, we do need a pgtable_t of 4096 bytes. Signed-off-by:
-
Aneesh Kumar K.V authored
We should not expect pte bit position in asm code. Simply by moving part of that to C Acked-by:
-
Aneesh Kumar K.V authored
Acked-by:
-
Aneesh Kumar K.V authored
Move the booke related headers below booke/32 or booke/64 Acked-by:
-
Aneesh Kumar K.V authored
functions which operate on pte bits are moved to hash*.h and other generic functions are moved to pgtable.h Acked-by:
-
Aneesh Kumar K.V authored
This enables us to keep hash64 related bits together, and makes it easy to follow. Acked-by:
-
Aneesh Kumar K.V authored
We convert them static inline function here as we did with pte_val in the previous patch Acked-by:
-
Aneesh Kumar K.V authored
We also convert few #define to static inline in this patch for better type checking Acked-by:
-
Aneesh Kumar K.V authored
We copy only needed PTE bits define from pte-common.h to respective hash related header. This should greatly simply later patches in which we are going to change the pte format for hash config Acked-by:
-
Aneesh Kumar K.V authored
We are going to drop pte_common.h in the later patch. The idea is to enable hash code not require to define all PTE bits. Having PTE bits defined in pte_common.h made the code unnecessarily complex. Acked-by:
-
Aneesh Kumar K.V authored
We also move __ASSEMBLY__ towards the end of header. This avoid having #ifndef __ASSEMBLY___ all over the header Acked-by:
-
Aneesh Kumar K.V authored
This further make a copy of pte defines to book3s/64/hash*.h. This remove the dependency on pgtable-ppc64-4k.h and pgtable-ppc64-64k.h Acked-by:
-
Aneesh Kumar K.V authored
In this patch we do: cp pgtable-ppc32.h book3s/32/pgtable.h cp pgtable-ppc64.h book3s/64/pgtable.h This enable us to do further changes to hash specific config. We will change the page table format for 64bit hash in later patches. Acked-by:
-
Aneesh Kumar K.V authored
Acked-by:
-
Aneesh Kumar K.V authored
This is the same bug we fixed as part of 09567e7f ("powerpc/mm: Check paca psize is up to date for huge mappings"). Please check that for details. The difference here is that faults were happening on a 4K page at an address previously mapped by hugetlb. Signed-off-by:
Anshuman Khandual <khandual@linux.vnet.ibm.com> Signed-off-by:
-
- 10 Dec, 2015 3 commits
-
-
Anton Blanchard authored
Two DSCR tests have a hack in them: /* * XXX: Force a context switch out so that DSCR * current value is copied into the thread struct * which is required for the child to inherit the * changed value. */ sleep(1); We should not be working around this in the testcase, it is a kernel bug. Fix it by copying the current DSCR to the child, instead of what we had in the thread struct at last context switch. Signed-off-by:
Anton Blanchard <anton@samba.org> Signed-off-by:
-
Anton Blanchard authored
commit 152d523e ("powerpc: Create context switch helpers save_sprs() and restore_sprs()") moved the restore of SPRs after the call to _switch(). There is an issue with this approach - new tasks do not return through _switch(), they are set up by copy_thread() to directly return through ret_from_fork() or ret_from_kernel_thread(). This means restore_sprs() is not getting called for new tasks. Fix this by moving restore_sprs() before _switch(). Fixes: 152d523e ("powerpc: Create context switch helpers save_sprs() and restore_sprs()") Signed-off-by:
-
Anton Blanchard authored
Commit a0e72cf1 ("powerpc: Create msr_check_and_{set,clear}()") removed a call to check_if_tm_restore_required() in the enable_kernel_*() functions. Add them back in. Fixes: a0e72cf1 ("powerpc: Create msr_check_and_{set,clear}()") Reported-by:
Rashmica Gupta <rashmicy@gmail.com> Signed-off-by:
-
- 02 Dec, 2015 4 commits
-
-
Anton Blanchard authored
Remove a bunch of unnecessary fallback functions and group things in a more logical way. Signed-off-by:
-
Anton Blanchard authored
Most of __switch_to() is housekeeping, TLB batching, timekeeping etc. Move these away from the more complex and critical context switching code. Signed-off-by:
-
Anton Blanchard authored
Create a single function that flushes everything (FP, VMX, VSX, SPE). Doing this all at once means we only do one MSR write. Signed-off-by:
-
Anton Blanchard authored
Create a single function that gives everything up (FP, VMX, VSX, SPE). Doing this all at once means we only do one MSR write. A context switch microbenchmark using yield(): http://ozlabs.org/~anton/junkcode/context_switch2.c ./context_switch2 --test=yield --fp --altivec --vector 0 0 shows an improvement of 3% on POWER8. Signed-off-by:
-
- 01 Dec, 2015 1 commit
-
-
Anton Blanchard authored
More consolidation of our MSR available bit handling. Signed-off-by:
-
