[PATCH] mm: flush TLB when clearing young

Traditionally we've not flushed TLB after clearing the young/referenced bit,
it has seemed just a waste of time.  Russell King points out that on some
architectures, with the move from 2.4 mm sweeping to 2.6 rmap, this may be a
serious omission: very frequently referenced pages never re-marked young, and
the worst choices made for unmapping.

So, replace ptep_test_and_clear_young by ptep_clear_flush_young throughout
rmap.c.  Originally I'd imagined making some kind of TLB gather optimization,
but don't see what now: whether worth it rather depends on how common
cross-cpu flushes are, and whether global or not.

ppc and ppc64 have already found this issue, and worked around it by arranging
TLB flush from their ptep_test_and_clear_young: with the aid of pgtable rmap
pointers.  I'm hoping ptep_clear_flush_young will allow ppc and ppc64 to
remove that special code, but won't change them myself.

It's worth noting that it is Andrea's anon_vma rmap which makes the vma
available for ptep_clear_flush_young in page_referenced_one: anonmm and
pte_chains would both need an additional find_vma for that.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

mm/rmap.c

@@ -226,7 +226,7 @@ static int page_referenced_one(struct page *page,
 	if (page_to_pfn(page) != pte_pfn(*pte))
 		goto out_unmap;
 
-	if (ptep_test_and_clear_young(pte))
+	if (ptep_clear_flush_young(vma, address, pte))
 		referenced++;
 
 	(*mapcount)--;
@@ -465,7 +465,7 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma)
 	 * skipped over this mm) then we should reactivate it.
 	 */
 	if ((vma->vm_flags & (VM_LOCKED|VM_RESERVED)) ||
-			ptep_test_and_clear_young(pte)) {
+			ptep_clear_flush_young(vma, address, pte)) {
 		ret = SWAP_FAIL;
 		goto out_unmap;
 	}
@@ -592,7 +592,7 @@ static int try_to_unmap_cluster(unsigned long cursor,
 		if (PageReserved(page))
 			continue;
 
-		if (ptep_test_and_clear_young(pte))
+		if (ptep_clear_flush_young(vma, address, pte))
 			continue;
 
 		/* Nuke the page table entry. */