Commit a63d83f4 authored by David Rientjes's avatar David Rientjes Committed by Linus Torvalds

oom: badness heuristic rewrite

This a complete rewrite of the oom killer's badness() heuristic which is
used to determine which task to kill in oom conditions.  The goal is to
make it as simple and predictable as possible so the results are better
understood and we end up killing the task which will lead to the most
memory freeing while still respecting the fine-tuning from userspace.

Instead of basing the heuristic on mm->total_vm for each task, the task's
rss and swap space is used instead.  This is a better indication of the
amount of memory that will be freeable if the oom killed task is chosen
and subsequently exits.  This helps specifically in cases where KDE or
GNOME is chosen for oom kill on desktop systems instead of a memory
hogging task.

The baseline for the heuristic is a proportion of memory that each task is
currently using in memory plus swap compared to the amount of "allowable"
memory.  "Allowable," in this sense, means the system-wide resources for
unconstrained oom conditions, the set of mempolicy nodes, the mems
attached to current's cpuset, or a memory controller's limit.  The
proportion is given on a scale of 0 (never kill) to 1000 (always kill),
roughly meaning that if a task has a badness() score of 500 that the task
consumes approximately 50% of allowable memory resident in RAM or in swap
space.

The proportion is always relative to the amount of "allowable" memory and
not the total amount of RAM systemwide so that mempolicies and cpusets may
operate in isolation; they shall not need to know the true size of the
machine on which they are running if they are bound to a specific set of
nodes or mems, respectively.

Root tasks are given 3% extra memory just like __vm_enough_memory()
provides in LSMs.  In the event of two tasks consuming similar amounts of
memory, it is generally better to save root's task.

Because of the change in the badness() heuristic's baseline, it is also
necessary to introduce a new user interface to tune it.  It's not possible
to redefine the meaning of /proc/pid/oom_adj with a new scale since the
ABI cannot be changed for backward compatability.  Instead, a new tunable,
/proc/pid/oom_score_adj, is added that ranges from -1000 to +1000.  It may
be used to polarize the heuristic such that certain tasks are never
considered for oom kill while others may always be considered.  The value
is added directly into the badness() score so a value of -500, for
example, means to discount 50% of its memory consumption in comparison to
other tasks either on the system, bound to the mempolicy, in the cpuset,
or sharing the same memory controller.

/proc/pid/oom_adj is changed so that its meaning is rescaled into the
units used by /proc/pid/oom_score_adj, and vice versa.  Changing one of
these per-task tunables will rescale the value of the other to an
equivalent meaning.  Although /proc/pid/oom_adj was originally defined as
a bitshift on the badness score, it now shares the same linear growth as
/proc/pid/oom_score_adj but with different granularity.  This is required
so the ABI is not broken with userspace applications and allows oom_adj to
be deprecated for future removal.
Signed-off-by: default avatarDavid Rientjes <rientjes@google.com>
Cc: Nick Piggin <npiggin@suse.de>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Balbir Singh <balbir@in.ibm.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent 74bcbf40
...@@ -33,7 +33,8 @@ Table of Contents ...@@ -33,7 +33,8 @@ Table of Contents
2 Modifying System Parameters 2 Modifying System Parameters
3 Per-Process Parameters 3 Per-Process Parameters
3.1 /proc/<pid>/oom_adj - Adjust the oom-killer score 3.1 /proc/<pid>/oom_adj & /proc/<pid>/oom_score_adj - Adjust the oom-killer
score
3.2 /proc/<pid>/oom_score - Display current oom-killer score 3.2 /proc/<pid>/oom_score - Display current oom-killer score
3.3 /proc/<pid>/io - Display the IO accounting fields 3.3 /proc/<pid>/io - Display the IO accounting fields
3.4 /proc/<pid>/coredump_filter - Core dump filtering settings 3.4 /proc/<pid>/coredump_filter - Core dump filtering settings
...@@ -1234,42 +1235,61 @@ of the kernel. ...@@ -1234,42 +1235,61 @@ of the kernel.
CHAPTER 3: PER-PROCESS PARAMETERS CHAPTER 3: PER-PROCESS PARAMETERS
------------------------------------------------------------------------------ ------------------------------------------------------------------------------
3.1 /proc/<pid>/oom_adj - Adjust the oom-killer score 3.1 /proc/<pid>/oom_adj & /proc/<pid>/oom_score_adj- Adjust the oom-killer score
------------------------------------------------------ --------------------------------------------------------------------------------
This file can be used to adjust the score used to select which processes These file can be used to adjust the badness heuristic used to select which
should be killed in an out-of-memory situation. Giving it a high score will process gets killed in out of memory conditions.
increase the likelihood of this process being killed by the oom-killer. Valid
values are in the range -16 to +15, plus the special value -17, which disables The badness heuristic assigns a value to each candidate task ranging from 0
oom-killing altogether for this process. (never kill) to 1000 (always kill) to determine which process is targeted. The
units are roughly a proportion along that range of allowed memory the process
The process to be killed in an out-of-memory situation is selected among all others may allocate from based on an estimation of its current memory and swap use.
based on its badness score. This value equals the original memory size of the process For example, if a task is using all allowed memory, its badness score will be
and is then updated according to its CPU time (utime + stime) and the 1000. If it is using half of its allowed memory, its score will be 500.
run time (uptime - start time). The longer it runs the smaller is the score.
Badness score is divided by the square root of the CPU time and then by There is an additional factor included in the badness score: root
the double square root of the run time. processes are given 3% extra memory over other tasks.
Swapped out tasks are killed first. Half of each child's memory size is added to The amount of "allowed" memory depends on the context in which the oom killer
the parent's score if they do not share the same memory. Thus forking servers was called. If it is due to the memory assigned to the allocating task's cpuset
are the prime candidates to be killed. Having only one 'hungry' child will make being exhausted, the allowed memory represents the set of mems assigned to that
parent less preferable than the child. cpuset. If it is due to a mempolicy's node(s) being exhausted, the allowed
memory represents the set of mempolicy nodes. If it is due to a memory
/proc/<pid>/oom_score shows process' current badness score. limit (or swap limit) being reached, the allowed memory is that configured
limit. Finally, if it is due to the entire system being out of memory, the
The following heuristics are then applied: allowed memory represents all allocatable resources.
* if the task was reniced, its score doubles
* superuser or direct hardware access tasks (CAP_SYS_ADMIN, CAP_SYS_RESOURCE The value of /proc/<pid>/oom_score_adj is added to the badness score before it
or CAP_SYS_RAWIO) have their score divided by 4 is used to determine which task to kill. Acceptable values range from -1000
* if oom condition happened in one cpuset and checked process does not belong (OOM_SCORE_ADJ_MIN) to +1000 (OOM_SCORE_ADJ_MAX). This allows userspace to
to it, its score is divided by 8 polarize the preference for oom killing either by always preferring a certain
* the resulting score is multiplied by two to the power of oom_adj, i.e. task or completely disabling it. The lowest possible value, -1000, is
points <<= oom_adj when it is positive and equivalent to disabling oom killing entirely for that task since it will always
points >>= -(oom_adj) otherwise report a badness score of 0.
The task with the highest badness score is then selected and its children Consequently, it is very simple for userspace to define the amount of memory to
are killed, process itself will be killed in an OOM situation when it does consider for each task. Setting a /proc/<pid>/oom_score_adj value of +500, for
not have children or some of them disabled oom like described above. example, is roughly equivalent to allowing the remainder of tasks sharing the
same system, cpuset, mempolicy, or memory controller resources to use at least
50% more memory. A value of -500, on the other hand, would be roughly
equivalent to discounting 50% of the task's allowed memory from being considered
as scoring against the task.
For backwards compatibility with previous kernels, /proc/<pid>/oom_adj may also
be used to tune the badness score. Its acceptable values range from -16
(OOM_ADJUST_MIN) to +15 (OOM_ADJUST_MAX) and a special value of -17
(OOM_DISABLE) to disable oom killing entirely for that task. Its value is
scaled linearly with /proc/<pid>/oom_score_adj.
Writing to /proc/<pid>/oom_score_adj or /proc/<pid>/oom_adj will change the
other with its scaled value.
Caveat: when a parent task is selected, the oom killer will sacrifice any first
generation children with seperate address spaces instead, if possible. This
avoids servers and important system daemons from being killed and loses the
minimal amount of work.
3.2 /proc/<pid>/oom_score - Display current oom-killer score 3.2 /proc/<pid>/oom_score - Display current oom-killer score
------------------------------------------------------------- -------------------------------------------------------------
......
...@@ -63,6 +63,7 @@ ...@@ -63,6 +63,7 @@
#include <linux/namei.h> #include <linux/namei.h>
#include <linux/mnt_namespace.h> #include <linux/mnt_namespace.h>
#include <linux/mm.h> #include <linux/mm.h>
#include <linux/swap.h>
#include <linux/rcupdate.h> #include <linux/rcupdate.h>
#include <linux/kallsyms.h> #include <linux/kallsyms.h>
#include <linux/stacktrace.h> #include <linux/stacktrace.h>
...@@ -430,12 +431,11 @@ static const struct file_operations proc_lstats_operations = { ...@@ -430,12 +431,11 @@ static const struct file_operations proc_lstats_operations = {
static int proc_oom_score(struct task_struct *task, char *buffer) static int proc_oom_score(struct task_struct *task, char *buffer)
{ {
unsigned long points = 0; unsigned long points = 0;
struct timespec uptime;
do_posix_clock_monotonic_gettime(&uptime);
read_lock(&tasklist_lock); read_lock(&tasklist_lock);
if (pid_alive(task)) if (pid_alive(task))
points = badness(task, NULL, NULL, uptime.tv_sec); points = oom_badness(task, NULL, NULL,
totalram_pages + total_swap_pages);
read_unlock(&tasklist_lock); read_unlock(&tasklist_lock);
return sprintf(buffer, "%lu\n", points); return sprintf(buffer, "%lu\n", points);
} }
...@@ -1038,7 +1038,15 @@ static ssize_t oom_adjust_write(struct file *file, const char __user *buf, ...@@ -1038,7 +1038,15 @@ static ssize_t oom_adjust_write(struct file *file, const char __user *buf,
} }
task->signal->oom_adj = oom_adjust; task->signal->oom_adj = oom_adjust;
/*
* Scale /proc/pid/oom_score_adj appropriately ensuring that a maximum
* value is always attainable.
*/
if (task->signal->oom_adj == OOM_ADJUST_MAX)
task->signal->oom_score_adj = OOM_SCORE_ADJ_MAX;
else
task->signal->oom_score_adj = (oom_adjust * OOM_SCORE_ADJ_MAX) /
-OOM_DISABLE;
unlock_task_sighand(task, &flags); unlock_task_sighand(task, &flags);
put_task_struct(task); put_task_struct(task);
...@@ -1051,6 +1059,82 @@ static const struct file_operations proc_oom_adjust_operations = { ...@@ -1051,6 +1059,82 @@ static const struct file_operations proc_oom_adjust_operations = {
.llseek = generic_file_llseek, .llseek = generic_file_llseek,
}; };
static ssize_t oom_score_adj_read(struct file *file, char __user *buf,
size_t count, loff_t *ppos)
{
struct task_struct *task = get_proc_task(file->f_path.dentry->d_inode);
char buffer[PROC_NUMBUF];
int oom_score_adj = OOM_SCORE_ADJ_MIN;
unsigned long flags;
size_t len;
if (!task)
return -ESRCH;
if (lock_task_sighand(task, &flags)) {
oom_score_adj = task->signal->oom_score_adj;
unlock_task_sighand(task, &flags);
}
put_task_struct(task);
len = snprintf(buffer, sizeof(buffer), "%d\n", oom_score_adj);
return simple_read_from_buffer(buf, count, ppos, buffer, len);
}
static ssize_t oom_score_adj_write(struct file *file, const char __user *buf,
size_t count, loff_t *ppos)
{
struct task_struct *task;
char buffer[PROC_NUMBUF];
unsigned long flags;
long oom_score_adj;
int err;
memset(buffer, 0, sizeof(buffer));
if (count > sizeof(buffer) - 1)
count = sizeof(buffer) - 1;
if (copy_from_user(buffer, buf, count))
return -EFAULT;
err = strict_strtol(strstrip(buffer), 0, &oom_score_adj);
if (err)
return -EINVAL;
if (oom_score_adj < OOM_SCORE_ADJ_MIN ||
oom_score_adj > OOM_SCORE_ADJ_MAX)
return -EINVAL;
task = get_proc_task(file->f_path.dentry->d_inode);
if (!task)
return -ESRCH;
if (!lock_task_sighand(task, &flags)) {
put_task_struct(task);
return -ESRCH;
}
if (oom_score_adj < task->signal->oom_score_adj &&
!capable(CAP_SYS_RESOURCE)) {
unlock_task_sighand(task, &flags);
put_task_struct(task);
return -EACCES;
}
task->signal->oom_score_adj = oom_score_adj;
/*
* Scale /proc/pid/oom_adj appropriately ensuring that OOM_DISABLE is
* always attainable.
*/
if (task->signal->oom_score_adj == OOM_SCORE_ADJ_MIN)
task->signal->oom_adj = OOM_DISABLE;
else
task->signal->oom_adj = (oom_score_adj * OOM_ADJUST_MAX) /
OOM_SCORE_ADJ_MAX;
unlock_task_sighand(task, &flags);
put_task_struct(task);
return count;
}
static const struct file_operations proc_oom_score_adj_operations = {
.read = oom_score_adj_read,
.write = oom_score_adj_write,
};
#ifdef CONFIG_AUDITSYSCALL #ifdef CONFIG_AUDITSYSCALL
#define TMPBUFLEN 21 #define TMPBUFLEN 21
static ssize_t proc_loginuid_read(struct file * file, char __user * buf, static ssize_t proc_loginuid_read(struct file * file, char __user * buf,
...@@ -2623,6 +2707,7 @@ static const struct pid_entry tgid_base_stuff[] = { ...@@ -2623,6 +2707,7 @@ static const struct pid_entry tgid_base_stuff[] = {
#endif #endif
INF("oom_score", S_IRUGO, proc_oom_score), INF("oom_score", S_IRUGO, proc_oom_score),
REG("oom_adj", S_IRUGO|S_IWUSR, proc_oom_adjust_operations), REG("oom_adj", S_IRUGO|S_IWUSR, proc_oom_adjust_operations),
REG("oom_score_adj", S_IRUGO|S_IWUSR, proc_oom_score_adj_operations),
#ifdef CONFIG_AUDITSYSCALL #ifdef CONFIG_AUDITSYSCALL
REG("loginuid", S_IWUSR|S_IRUGO, proc_loginuid_operations), REG("loginuid", S_IWUSR|S_IRUGO, proc_loginuid_operations),
REG("sessionid", S_IRUGO, proc_sessionid_operations), REG("sessionid", S_IRUGO, proc_sessionid_operations),
...@@ -2957,6 +3042,7 @@ static const struct pid_entry tid_base_stuff[] = { ...@@ -2957,6 +3042,7 @@ static const struct pid_entry tid_base_stuff[] = {
#endif #endif
INF("oom_score", S_IRUGO, proc_oom_score), INF("oom_score", S_IRUGO, proc_oom_score),
REG("oom_adj", S_IRUGO|S_IWUSR, proc_oom_adjust_operations), REG("oom_adj", S_IRUGO|S_IWUSR, proc_oom_adjust_operations),
REG("oom_score_adj", S_IRUGO|S_IWUSR, proc_oom_score_adj_operations),
#ifdef CONFIG_AUDITSYSCALL #ifdef CONFIG_AUDITSYSCALL
REG("loginuid", S_IWUSR|S_IRUGO, proc_loginuid_operations), REG("loginuid", S_IWUSR|S_IRUGO, proc_loginuid_operations),
REG("sessionid", S_IRUSR, proc_sessionid_operations), REG("sessionid", S_IRUSR, proc_sessionid_operations),
......
...@@ -125,6 +125,8 @@ void mem_cgroup_update_file_mapped(struct page *page, int val); ...@@ -125,6 +125,8 @@ void mem_cgroup_update_file_mapped(struct page *page, int val);
unsigned long mem_cgroup_soft_limit_reclaim(struct zone *zone, int order, unsigned long mem_cgroup_soft_limit_reclaim(struct zone *zone, int order,
gfp_t gfp_mask, int nid, gfp_t gfp_mask, int nid,
int zid); int zid);
u64 mem_cgroup_get_limit(struct mem_cgroup *mem);
#else /* CONFIG_CGROUP_MEM_RES_CTLR */ #else /* CONFIG_CGROUP_MEM_RES_CTLR */
struct mem_cgroup; struct mem_cgroup;
...@@ -304,6 +306,12 @@ unsigned long mem_cgroup_soft_limit_reclaim(struct zone *zone, int order, ...@@ -304,6 +306,12 @@ unsigned long mem_cgroup_soft_limit_reclaim(struct zone *zone, int order,
return 0; return 0;
} }
static inline
u64 mem_cgroup_get_limit(struct mem_cgroup *mem)
{
return 0;
}
#endif /* CONFIG_CGROUP_MEM_CONT */ #endif /* CONFIG_CGROUP_MEM_CONT */
#endif /* _LINUX_MEMCONTROL_H */ #endif /* _LINUX_MEMCONTROL_H */
......
#ifndef __INCLUDE_LINUX_OOM_H #ifndef __INCLUDE_LINUX_OOM_H
#define __INCLUDE_LINUX_OOM_H #define __INCLUDE_LINUX_OOM_H
/* /proc/<pid>/oom_adj set to -17 protects from the oom-killer */ /*
* /proc/<pid>/oom_adj set to -17 protects from the oom-killer
*/
#define OOM_DISABLE (-17) #define OOM_DISABLE (-17)
/* inclusive */ /* inclusive */
#define OOM_ADJUST_MIN (-16) #define OOM_ADJUST_MIN (-16)
#define OOM_ADJUST_MAX 15 #define OOM_ADJUST_MAX 15
/*
* /proc/<pid>/oom_score_adj set to OOM_SCORE_ADJ_MIN disables oom killing for
* pid.
*/
#define OOM_SCORE_ADJ_MIN (-1000)
#define OOM_SCORE_ADJ_MAX 1000
#ifdef __KERNEL__ #ifdef __KERNEL__
#include <linux/sched.h>
#include <linux/types.h> #include <linux/types.h>
#include <linux/nodemask.h> #include <linux/nodemask.h>
...@@ -27,6 +37,8 @@ enum oom_constraint { ...@@ -27,6 +37,8 @@ enum oom_constraint {
CONSTRAINT_MEMCG, CONSTRAINT_MEMCG,
}; };
extern unsigned int oom_badness(struct task_struct *p, struct mem_cgroup *mem,
const nodemask_t *nodemask, unsigned long totalpages);
extern int try_set_zonelist_oom(struct zonelist *zonelist, gfp_t gfp_flags); extern int try_set_zonelist_oom(struct zonelist *zonelist, gfp_t gfp_flags);
extern void clear_zonelist_oom(struct zonelist *zonelist, gfp_t gfp_flags); extern void clear_zonelist_oom(struct zonelist *zonelist, gfp_t gfp_flags);
......
...@@ -621,7 +621,8 @@ struct signal_struct { ...@@ -621,7 +621,8 @@ struct signal_struct {
struct tty_audit_buf *tty_audit_buf; struct tty_audit_buf *tty_audit_buf;
#endif #endif
int oom_adj; /* OOM kill score adjustment (bit shift) */ int oom_adj; /* OOM kill score adjustment (bit shift) */
int oom_score_adj; /* OOM kill score adjustment */
}; };
/* Context switch must be unlocked if interrupts are to be enabled */ /* Context switch must be unlocked if interrupts are to be enabled */
......
...@@ -899,6 +899,7 @@ static int copy_signal(unsigned long clone_flags, struct task_struct *tsk) ...@@ -899,6 +899,7 @@ static int copy_signal(unsigned long clone_flags, struct task_struct *tsk)
tty_audit_fork(sig); tty_audit_fork(sig);
sig->oom_adj = current->signal->oom_adj; sig->oom_adj = current->signal->oom_adj;
sig->oom_score_adj = current->signal->oom_score_adj;
return 0; return 0;
} }
......
...@@ -1126,6 +1126,24 @@ static int mem_cgroup_count_children(struct mem_cgroup *mem) ...@@ -1126,6 +1126,24 @@ static int mem_cgroup_count_children(struct mem_cgroup *mem)
return num; return num;
} }
/*
* Return the memory (and swap, if configured) limit for a memcg.
*/
u64 mem_cgroup_get_limit(struct mem_cgroup *memcg)
{
u64 limit;
u64 memsw;
limit = res_counter_read_u64(&memcg->res, RES_LIMIT) +
total_swap_pages;
memsw = res_counter_read_u64(&memcg->memsw, RES_LIMIT);
/*
* If memsw is finite and limits the amount of swap space available
* to this memcg, return that limit.
*/
return min(limit, memsw);
}
/* /*
* Visit the first child (need not be the first child as per the ordering * Visit the first child (need not be the first child as per the ordering
* of the cgroup list, since we track last_scanned_child) of @mem and use * of the cgroup list, since we track last_scanned_child) of @mem and use
......
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment