Commit 3ad339fb authored by Johannes Weiner's avatar Johannes Weiner Committed by Luis Henriques

mm: page_alloc: fix zone allocation fairness on UP

commit abe5f972 upstream.

The zone allocation batches can easily underflow due to higher-order
allocations or spills to remote nodes.  On SMP that's fine, because
underflows are expected from concurrency and dealt with by returning 0.
But on UP, zone_page_state will just return a wrapped unsigned long,
which will get past the <= 0 check and then consider the zone eligible
until its watermarks are hit.

Commit 3a025760 ("mm: page_alloc: spill to remote nodes before
waking kswapd") already made the counter-resetting use
atomic_long_read() to accomodate underflows from remote spills, but it
didn't go all the way with it.

Make it clear that these batches are expected to go negative regardless
of concurrency, and use atomic_long_read() everywhere.

Fixes: 81c0a2bb ("mm: page_alloc: fair zone allocator policy")
Reported-by: default avatarVlastimil Babka <vbabka@suse.cz>
Reported-by: default avatarLeon Romanovsky <leon@leon.nu>
Signed-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
Acked-by: default avatarMel Gorman <mgorman@suse.de>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
[ luis: backported to 3.16 by Johannes Weiner ]
Signed-off-by: default avatarLuis Henriques <luis.henriques@canonical.com>
parent ffb84baa
......@@ -1965,7 +1965,7 @@ get_page_from_freelist(gfp_t gfp_mask, nodemask_t *nodemask, unsigned int order,
if (alloc_flags & ALLOC_FAIR) {
if (!zone_local(preferred_zone, zone))
continue;
if (zone_page_state(zone, NR_ALLOC_BATCH) <= 0)
if (atomic_long_read(&zone->vm_stat[NR_ALLOC_BATCH]) <= 0)
continue;
}
/*
......@@ -5694,9 +5694,8 @@ static void __setup_per_zone_wmarks(void)
zone->watermark[WMARK_HIGH] = min_wmark_pages(zone) + (tmp >> 1);
__mod_zone_page_state(zone, NR_ALLOC_BATCH,
high_wmark_pages(zone) -
low_wmark_pages(zone) -
zone_page_state(zone, NR_ALLOC_BATCH));
high_wmark_pages(zone) - low_wmark_pages(zone) -
atomic_long_read(&zone->vm_stat[NR_ALLOC_BATCH]));
setup_zone_migrate_reserve(zone);
spin_unlock_irqrestore(&zone->lock, flags);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment