Commit 5bbe3547 authored by Eric B Munson's avatar Eric B Munson Committed by Linus Torvalds

mm: allow compaction of unevictable pages

Currently, pages which are marked as unevictable are protected from
compaction, but not from other types of migration.  The POSIX real time
extension explicitly states that mlock() will prevent a major page
fault, but the spirit of this is that mlock() should give a process the
ability to control sources of latency, including minor page faults.
However, the mlock manpage only explicitly says that a locked page will
not be written to swap and this can cause some confusion.  The
compaction code today does not give a developer who wants to avoid swap
but wants to have large contiguous areas available any method to achieve
this state.  This patch introduces a sysctl for controlling compaction
behavior with respect to the unevictable lru.  Users who demand no page
faults after a page is present can set compact_unevictable_allowed to 0
and users who need the large contiguous areas can enable compaction on
locked memory by leaving the default value of 1.

To illustrate this problem I wrote a quick test program that mmaps a
large number of 1MB files filled with random data.  These maps are
created locked and read only.  Then every other mmap is unmapped and I
attempt to allocate huge pages to the static huge page pool.  When the
compact_unevictable_allowed sysctl is 0, I cannot allocate hugepages
after fragmenting memory.  When the value is set to 1, allocations
succeed.
Signed-off-by: default avatarEric B Munson <emunson@akamai.com>
Acked-by: default avatarMichal Hocko <mhocko@suse.cz>
Acked-by: default avatarVlastimil Babka <vbabka@suse.cz>
Acked-by: default avatarChristoph Lameter <cl@linux.com>
Acked-by: default avatarDavid Rientjes <rientjes@google.com>
Acked-by: default avatarRik van Riel <riel@redhat.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Christoph Lameter <cl@linux.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Michal Hocko <mhocko@suse.cz>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent a4bb3ecd
...@@ -21,6 +21,7 @@ Currently, these files are in /proc/sys/vm: ...@@ -21,6 +21,7 @@ Currently, these files are in /proc/sys/vm:
- admin_reserve_kbytes - admin_reserve_kbytes
- block_dump - block_dump
- compact_memory - compact_memory
- compact_unevictable_allowed
- dirty_background_bytes - dirty_background_bytes
- dirty_background_ratio - dirty_background_ratio
- dirty_bytes - dirty_bytes
...@@ -106,6 +107,16 @@ huge pages although processes will also directly compact memory as required. ...@@ -106,6 +107,16 @@ huge pages although processes will also directly compact memory as required.
============================================================== ==============================================================
compact_unevictable_allowed
Available only when CONFIG_COMPACTION is set. When set to 1, compaction is
allowed to examine the unevictable lru (mlocked pages) for pages to compact.
This should be used on systems where stalls for minor page faults are an
acceptable trade for large contiguous free memory. Set to 0 to prevent
compaction from moving pages that are unevictable. Default value is 1.
==============================================================
dirty_background_bytes dirty_background_bytes
Contains the amount of dirty memory at which the background kernel Contains the amount of dirty memory at which the background kernel
......
...@@ -34,6 +34,7 @@ extern int sysctl_compaction_handler(struct ctl_table *table, int write, ...@@ -34,6 +34,7 @@ extern int sysctl_compaction_handler(struct ctl_table *table, int write,
extern int sysctl_extfrag_threshold; extern int sysctl_extfrag_threshold;
extern int sysctl_extfrag_handler(struct ctl_table *table, int write, extern int sysctl_extfrag_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *length, loff_t *ppos); void __user *buffer, size_t *length, loff_t *ppos);
extern int sysctl_compact_unevictable_allowed;
extern int fragmentation_index(struct zone *zone, unsigned int order); extern int fragmentation_index(struct zone *zone, unsigned int order);
extern unsigned long try_to_compact_pages(gfp_t gfp_mask, unsigned int order, extern unsigned long try_to_compact_pages(gfp_t gfp_mask, unsigned int order,
......
...@@ -1335,6 +1335,15 @@ static struct ctl_table vm_table[] = { ...@@ -1335,6 +1335,15 @@ static struct ctl_table vm_table[] = {
.extra1 = &min_extfrag_threshold, .extra1 = &min_extfrag_threshold,
.extra2 = &max_extfrag_threshold, .extra2 = &max_extfrag_threshold,
}, },
{
.procname = "compact_unevictable_allowed",
.data = &sysctl_compact_unevictable_allowed,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec,
.extra1 = &zero,
.extra2 = &one,
},
#endif /* CONFIG_COMPACTION */ #endif /* CONFIG_COMPACTION */
{ {
......
...@@ -1046,6 +1046,12 @@ typedef enum { ...@@ -1046,6 +1046,12 @@ typedef enum {
ISOLATE_SUCCESS, /* Pages isolated, migrate */ ISOLATE_SUCCESS, /* Pages isolated, migrate */
} isolate_migrate_t; } isolate_migrate_t;
/*
* Allow userspace to control policy on scanning the unevictable LRU for
* compactable pages.
*/
int sysctl_compact_unevictable_allowed __read_mostly = 1;
/* /*
* Isolate all pages that can be migrated from the first suitable block, * Isolate all pages that can be migrated from the first suitable block,
* starting at the block pointed to by the migrate scanner pfn within * starting at the block pointed to by the migrate scanner pfn within
...@@ -1057,6 +1063,7 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone, ...@@ -1057,6 +1063,7 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
unsigned long low_pfn, end_pfn; unsigned long low_pfn, end_pfn;
struct page *page; struct page *page;
const isolate_mode_t isolate_mode = const isolate_mode_t isolate_mode =
(sysctl_compact_unevictable_allowed ? ISOLATE_UNEVICTABLE : 0) |
(cc->mode == MIGRATE_ASYNC ? ISOLATE_ASYNC_MIGRATE : 0); (cc->mode == MIGRATE_ASYNC ? ISOLATE_ASYNC_MIGRATE : 0);
/* /*
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment