Commit 0ba1d0f7 authored by Andrea Arcangeli's avatar Andrea Arcangeli Committed by Linus Torvalds

ksm: cleanup stable_node chain collapse case

Patch series "KSMscale cleanup/optimizations".

There are no fixes here it's just minor cleanups and optimizations.

1/3 removes makes the "fix" for the stale stable_node fall in the
    standard case without introducing new cases.  Setting stable_node to
    NULL was marginally safer, but stale pointer is still wiped from the
    caller, this looks cleaner.

2/3 should fix the false positive from Dan's static checker.

3/3 is a microoptimization to apply the the refile of future merge
    candidate dups at the head of the chain in all cases and to skip it in
    one case where we did it and but it was a noop (to avoid checking if
    it was already at the head but now we've to check it anyway so it got
    optimized away).

This patch (of 3):

When the stable_node chain is collapsed we can as well set the caller
stable_node to match the returned stable_node_dup in chain_prune().

This way the collapse case becomes indistinguishable from the regular
stable_node case and we can remove two branches from the KSM page
migration handling slow paths.

While it was all correct this looks cleaner (and faster) as the caller has
to deal with fewer special cases.

Link: http://lkml.kernel.org/r/20170518173721.22316-2-aarcange@redhat.comSigned-off-by: default avatarAndrea Arcangeli <aarcange@redhat.com>
Cc: Evgheni Dereveanchin <ederevea@redhat.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Petr Holasek <pholasek@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Gavin Guo <gavin.guo@canonical.com>
Cc: Jay Vosburgh <jay.vosburgh@canonical.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent b4fecc67
...@@ -1392,14 +1392,18 @@ static struct stable_node *stable_node_dup(struct stable_node **_stable_node, ...@@ -1392,14 +1392,18 @@ static struct stable_node *stable_node_dup(struct stable_node **_stable_node,
ksm_stable_node_chains--; ksm_stable_node_chains--;
ksm_stable_node_dups--; ksm_stable_node_dups--;
/* /*
* NOTE: the caller depends on the * NOTE: the caller depends on the stable_node
* *_stable_node to become NULL if the chain * to be equal to stable_node_dup if the chain
* was collapsed. Enforce that if anything * was collapsed.
* uses a stale (freed) stable_node chain a */
* visible crash will materialize (instead of *_stable_node = found;
* an use after free). /*
*/ * Just for robustneess as stable_node is
*_stable_node = stable_node = NULL; * otherwise left as a stable pointer, the
* compiler shall optimize it away at build
* time.
*/
stable_node = NULL;
} else if (__is_page_sharing_candidate(found, 1)) { } else if (__is_page_sharing_candidate(found, 1)) {
/* /*
* Refile our candidate at the head * Refile our candidate at the head
...@@ -1505,7 +1509,11 @@ static struct page *stable_tree_search(struct page *page) ...@@ -1505,7 +1509,11 @@ static struct page *stable_tree_search(struct page *page)
* not NULL. stable_node_dup may have been inserted in * not NULL. stable_node_dup may have been inserted in
* the rbtree instead as a regular stable_node (in * the rbtree instead as a regular stable_node (in
* order to collapse the stable_node chain if a single * order to collapse the stable_node chain if a single
* stable_node dup was found in it). * stable_node dup was found in it). In such case the
* stable_node is overwritten by the calleee to point
* to the stable_node_dup that was collapsed in the
* stable rbtree and stable_node will be equal to
* stable_node_dup like if the chain never existed.
*/ */
if (!stable_node_dup) { if (!stable_node_dup) {
/* /*
...@@ -1623,15 +1631,13 @@ static struct page *stable_tree_search(struct page *page) ...@@ -1623,15 +1631,13 @@ static struct page *stable_tree_search(struct page *page)
replace: replace:
/* /*
* If stable_node was a chain and chain_prune collapsed it, * If stable_node was a chain and chain_prune collapsed it,
* stable_node will be NULL here. In that case the * stable_node has been updated to be the new regular
* stable_node_dup is the regular stable_node that has * stable_node. A collapse of the chain is indistinguishable
* replaced the chain. If stable_node is not NULL and equal to * from the case there was no chain in the stable
* stable_node_dup there was no chain and stable_node_dup is * rbtree. Otherwise stable_node is the chain and
* the regular stable_node in the stable rbtree. Otherwise * stable_node_dup is the dup to replace.
* stable_node is the chain and stable_node_dup is the dup to */
* replace. if (stable_node_dup == stable_node) {
*/
if (!stable_node || stable_node_dup == stable_node) {
VM_BUG_ON(is_stable_node_chain(stable_node_dup)); VM_BUG_ON(is_stable_node_chain(stable_node_dup));
VM_BUG_ON(is_stable_node_dup(stable_node_dup)); VM_BUG_ON(is_stable_node_dup(stable_node_dup));
/* there is no chain */ /* there is no chain */
...@@ -1676,13 +1682,13 @@ static struct page *stable_tree_search(struct page *page) ...@@ -1676,13 +1682,13 @@ static struct page *stable_tree_search(struct page *page)
stable_node_dup = stable_node_any; stable_node_dup = stable_node_any;
/* /*
* If stable_node was a chain and chain_prune collapsed it, * If stable_node was a chain and chain_prune collapsed it,
* stable_node will be NULL here. In that case the * stable_node has been updated to be the new regular
* stable_node_dup is the regular stable_node that has * stable_node. A collapse of the chain is indistinguishable
* replaced the chain. If stable_node is not NULL and equal to * from the case there was no chain in the stable
* stable_node_dup there was no chain and stable_node_dup is * rbtree. Otherwise stable_node is the chain and
* the regular stable_node in the stable rbtree. * stable_node_dup is the dup to replace.
*/ */
if (!stable_node || stable_node_dup == stable_node) { if (stable_node_dup == stable_node) {
VM_BUG_ON(is_stable_node_chain(stable_node_dup)); VM_BUG_ON(is_stable_node_chain(stable_node_dup));
VM_BUG_ON(is_stable_node_dup(stable_node_dup)); VM_BUG_ON(is_stable_node_dup(stable_node_dup));
/* chain is missing so create it */ /* chain is missing so create it */
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment