Commit f2e5ff85 authored by Andrea Arcangeli's avatar Andrea Arcangeli Committed by Linus Torvalds

ksm: don't fail stable tree lookups if walking over stale stable_nodes

The stable_nodes can become stale at any time if the underlying pages gets
freed.  The stable_node gets collected and removed from the stable rbtree
if that is detected during the rbtree lookups.

Don't fail the lookup if running into stale stable_nodes, just restart the
lookup after collecting the stale stable_nodes.  Otherwise the CPU spent
in the preparation stage is wasted and the lookup must be repeated at the
next loop potentially failing a second time in a second stale stable_node.

If we don't prune aggressively we delay the merging of the unstable node
candidates and at the same time we delay the freeing of the stale
stable_nodes.  Keeping stale stable_nodes around wastes memory and it
can't provide any benefit.
Signed-off-by: default avatarAndrea Arcangeli <aarcange@redhat.com>
Acked-by: default avatarHugh Dickins <hughd@google.com>
Cc: Petr Holasek <pholasek@redhat.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent ad12695f
...@@ -1177,8 +1177,18 @@ static struct page *stable_tree_search(struct page *page) ...@@ -1177,8 +1177,18 @@ static struct page *stable_tree_search(struct page *page)
cond_resched(); cond_resched();
stable_node = rb_entry(*new, struct stable_node, node); stable_node = rb_entry(*new, struct stable_node, node);
tree_page = get_ksm_page(stable_node, false); tree_page = get_ksm_page(stable_node, false);
if (!tree_page) if (!tree_page) {
return NULL; /*
* If we walked over a stale stable_node,
* get_ksm_page() will call rb_erase() and it
* may rebalance the tree from under us. So
* restart the search from scratch. Returning
* NULL would be safe too, but we'd generate
* false negative insertions just because some
* stable_node was stale.
*/
goto again;
}
ret = memcmp_pages(page, tree_page); ret = memcmp_pages(page, tree_page);
put_page(tree_page); put_page(tree_page);
...@@ -1254,12 +1264,14 @@ static struct stable_node *stable_tree_insert(struct page *kpage) ...@@ -1254,12 +1264,14 @@ static struct stable_node *stable_tree_insert(struct page *kpage)
unsigned long kpfn; unsigned long kpfn;
struct rb_root *root; struct rb_root *root;
struct rb_node **new; struct rb_node **new;
struct rb_node *parent = NULL; struct rb_node *parent;
struct stable_node *stable_node; struct stable_node *stable_node;
kpfn = page_to_pfn(kpage); kpfn = page_to_pfn(kpage);
nid = get_kpfn_nid(kpfn); nid = get_kpfn_nid(kpfn);
root = root_stable_tree + nid; root = root_stable_tree + nid;
again:
parent = NULL;
new = &root->rb_node; new = &root->rb_node;
while (*new) { while (*new) {
...@@ -1269,8 +1281,18 @@ static struct stable_node *stable_tree_insert(struct page *kpage) ...@@ -1269,8 +1281,18 @@ static struct stable_node *stable_tree_insert(struct page *kpage)
cond_resched(); cond_resched();
stable_node = rb_entry(*new, struct stable_node, node); stable_node = rb_entry(*new, struct stable_node, node);
tree_page = get_ksm_page(stable_node, false); tree_page = get_ksm_page(stable_node, false);
if (!tree_page) if (!tree_page) {
return NULL; /*
* If we walked over a stale stable_node,
* get_ksm_page() will call rb_erase() and it
* may rebalance the tree from under us. So
* restart the search from scratch. Returning
* NULL would be safe too, but we'd generate
* false negative insertions just because some
* stable_node was stale.
*/
goto again;
}
ret = memcmp_pages(kpage, tree_page); ret = memcmp_pages(kpage, tree_page);
put_page(tree_page); put_page(tree_page);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment