Commit 5187b28f authored by Pekka Riikonen's avatar Pekka Riikonen Committed by H. Peter Anvin

x86: Allow FPU to be used at interrupt time even with eagerfpu

With the addition of eagerfpu the irq_fpu_usable() now returns false
negatives especially in the case of ksoftirqd and interrupted idle task,
two common cases for FPU use for example in networking/crypto.  With
eagerfpu=off FPU use is possible in those contexts.  This is because of
the eagerfpu check in interrupted_kernel_fpu_idle():

...
  * For now, with eagerfpu we will return interrupted kernel FPU
  * state as not-idle. TBD: Ideally we can change the return value
  * to something like __thread_has_fpu(current). But we need to
  * be careful of doing __thread_clear_has_fpu() before saving
  * the FPU etc for supporting nested uses etc. For now, take
  * the simple route!
...
 	if (use_eager_fpu())
 		return 0;

As eagerfpu is automatically "on" on those CPUs that also have the
features like AES-NI this patch changes the eagerfpu check to return 1 in
case the kernel_fpu_begin() has not been said yet.  Once it has been the
__thread_has_fpu() will start returning 0.

Notice that with eagerfpu the __thread_has_fpu is always true initially.
FPU use is thus always possible no matter what task is under us, unless
the state has already been saved with kernel_fpu_begin().

[ hpa: this is a performance regression, not a correctness regression,
  but since it can be quite serious on CPUs which need encryption at
  interrupt time I am marking this for urgent/stable. ]
Signed-off-by: default avatarPekka Riikonen <priikone@iki.fi>
Link: http://lkml.kernel.org/r/alpine.GSO.2.00.1305131356320.18@git.silcnet.org
Cc: <stable@vger.kernel.org> v3.7+
Signed-off-by: default avatarH. Peter Anvin <hpa@linux.intel.com>
parent 2baad612
...@@ -22,23 +22,19 @@ ...@@ -22,23 +22,19 @@
/* /*
* Were we in an interrupt that interrupted kernel mode? * Were we in an interrupt that interrupted kernel mode?
* *
* For now, with eagerfpu we will return interrupted kernel FPU
* state as not-idle. TBD: Ideally we can change the return value
* to something like __thread_has_fpu(current). But we need to
* be careful of doing __thread_clear_has_fpu() before saving
* the FPU etc for supporting nested uses etc. For now, take
* the simple route!
*
* On others, we can do a kernel_fpu_begin/end() pair *ONLY* if that * On others, we can do a kernel_fpu_begin/end() pair *ONLY* if that
* pair does nothing at all: the thread must not have fpu (so * pair does nothing at all: the thread must not have fpu (so
* that we don't try to save the FPU state), and TS must * that we don't try to save the FPU state), and TS must
* be set (so that the clts/stts pair does nothing that is * be set (so that the clts/stts pair does nothing that is
* visible in the interrupted kernel thread). * visible in the interrupted kernel thread).
*
* Except for the eagerfpu case when we return 1 unless we've already
* been eager and saved the state in kernel_fpu_begin().
*/ */
static inline bool interrupted_kernel_fpu_idle(void) static inline bool interrupted_kernel_fpu_idle(void)
{ {
if (use_eager_fpu()) if (use_eager_fpu())
return 0; return __thread_has_fpu(current);
return !__thread_has_fpu(current) && return !__thread_has_fpu(current) &&
(read_cr0() & X86_CR0_TS); (read_cr0() & X86_CR0_TS);
...@@ -78,8 +74,8 @@ void __kernel_fpu_begin(void) ...@@ -78,8 +74,8 @@ void __kernel_fpu_begin(void)
struct task_struct *me = current; struct task_struct *me = current;
if (__thread_has_fpu(me)) { if (__thread_has_fpu(me)) {
__save_init_fpu(me);
__thread_clear_has_fpu(me); __thread_clear_has_fpu(me);
__save_init_fpu(me);
/* We do 'stts()' in __kernel_fpu_end() */ /* We do 'stts()' in __kernel_fpu_end() */
} else if (!use_eager_fpu()) { } else if (!use_eager_fpu()) {
this_cpu_write(fpu_owner_task, NULL); this_cpu_write(fpu_owner_task, NULL);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment