Commit eee4818b authored by Naoya Horiguchi's avatar Naoya Horiguchi Committed by Linus Torvalds

mm: x86: move _PAGE_SWP_SOFT_DIRTY from bit 7 to bit 1

_PAGE_PSE is used to distinguish between a truly non-present
(_PAGE_PRESENT=0) PMD, and a PMD which is undergoing a THP split and
should be treated as present.

But _PAGE_SWP_SOFT_DIRTY currently uses the _PAGE_PSE bit, which would
cause confusion between one of those PMDs undergoing a THP split, and a
soft-dirty PMD.  Dropping _PAGE_PSE check in pmd_present() does not work
well, because it can hurt optimization of tlb handling in thp split.

Thus, we need to move the bit.

In the current kernel, bits 1-4 are not used in non-present format since
commit 00839ee3 ("x86/mm: Move swap offset/type up in PTE to work
around erratum").  So let's move _PAGE_SWP_SOFT_DIRTY to bit 1.  Bit 7
is used as reserved (always clear), so please don't use it for other
purpose.

Link: http://lkml.kernel.org/r/20170717193955.20207-3-zi.yan@sent.comSigned-off-by: default avatarNaoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Signed-off-by: default avatarZi Yan <zi.yan@cs.rutgers.edu>
Acked-by: default avatarDave Hansen <dave.hansen@intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com>
Cc: David Nellans <dnellans@nvidia.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Michal Hocko <mhocko@kernel.org>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent 88aaa2a1
...@@ -180,15 +180,21 @@ static inline int pgd_large(pgd_t pgd) { return 0; } ...@@ -180,15 +180,21 @@ static inline int pgd_large(pgd_t pgd) { return 0; }
/* /*
* Encode and de-code a swap entry * Encode and de-code a swap entry
* *
* | ... | 11| 10| 9|8|7|6|5| 4| 3|2|1|0| <- bit number * | ... | 11| 10| 9|8|7|6|5| 4| 3|2| 1|0| <- bit number
* | ... |SW3|SW2|SW1|G|L|D|A|CD|WT|U|W|P| <- bit names * | ... |SW3|SW2|SW1|G|L|D|A|CD|WT|U| W|P| <- bit names
* | OFFSET (14->63) | TYPE (9-13) |0|X|X|X| X| X|X|X|0| <- swp entry * | OFFSET (14->63) | TYPE (9-13) |0|0|X|X| X| X|X|SD|0| <- swp entry
* *
* G (8) is aliased and used as a PROT_NONE indicator for * G (8) is aliased and used as a PROT_NONE indicator for
* !present ptes. We need to start storing swap entries above * !present ptes. We need to start storing swap entries above
* there. We also need to avoid using A and D because of an * there. We also need to avoid using A and D because of an
* erratum where they can be incorrectly set by hardware on * erratum where they can be incorrectly set by hardware on
* non-present PTEs. * non-present PTEs.
*
* SD (1) in swp entry is used to store soft dirty bit, which helps us
* remember soft dirty over page migration
*
* Bit 7 in swp entry should be 0 because pmd_present checks not only P,
* but also L and G.
*/ */
#define SWP_TYPE_FIRST_BIT (_PAGE_BIT_PROTNONE + 1) #define SWP_TYPE_FIRST_BIT (_PAGE_BIT_PROTNONE + 1)
#define SWP_TYPE_BITS 5 #define SWP_TYPE_BITS 5
......
...@@ -99,15 +99,15 @@ ...@@ -99,15 +99,15 @@
/* /*
* Tracking soft dirty bit when a page goes to a swap is tricky. * Tracking soft dirty bit when a page goes to a swap is tricky.
* We need a bit which can be stored in pte _and_ not conflict * We need a bit which can be stored in pte _and_ not conflict
* with swap entry format. On x86 bits 6 and 7 are *not* involved * with swap entry format. On x86 bits 1-4 are *not* involved
* into swap entry computation, but bit 6 is used for nonlinear * into swap entry computation, but bit 7 is used for thp migration,
* file mapping, so we borrow bit 7 for soft dirty tracking. * so we borrow bit 1 for soft dirty tracking.
* *
* Please note that this bit must be treated as swap dirty page * Please note that this bit must be treated as swap dirty page
* mark if and only if the PTE has present bit clear! * mark if and only if the PTE/PMD has present bit clear!
*/ */
#ifdef CONFIG_MEM_SOFT_DIRTY #ifdef CONFIG_MEM_SOFT_DIRTY
#define _PAGE_SWP_SOFT_DIRTY _PAGE_PSE #define _PAGE_SWP_SOFT_DIRTY _PAGE_RW
#else #else
#define _PAGE_SWP_SOFT_DIRTY (_AT(pteval_t, 0)) #define _PAGE_SWP_SOFT_DIRTY (_AT(pteval_t, 0))
#endif #endif
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment