• Greg Thelen's avatar
    percpu: fix this_cpu_sub() subtrahend casting for unsigneds · bd09d9a3
    Greg Thelen authored
    this_cpu_sub() is implemented as negation and addition.
    
    This patch casts the adjustment to the counter type before negation to
    sign extend the adjustment.  This helps in cases where the counter type
    is wider than an unsigned adjustment.  An alternative to this patch is
    to declare such operations unsupported, but it seemed useful to avoid
    surprises.
    
    This patch specifically helps the following example:
      unsigned int delta = 1
      preempt_disable()
      this_cpu_write(long_counter, 0)
      this_cpu_sub(long_counter, delta)
      preempt_enable()
    
    Before this change long_counter on a 64 bit machine ends with value
    0xffffffff, rather than 0xffffffffffffffff.  This is because
    this_cpu_sub(pcp, delta) boils down to this_cpu_add(pcp, -delta),
    which is basically:
      long_counter = 0 + 0xffffffff
    
    Also apply the same cast to:
      __this_cpu_sub()
      __this_cpu_sub_return()
      this_cpu_sub_return()
    
    All percpu_test.ko passes, especially the following cases which
    previously failed:
    
      l -= ui_one;
      __this_cpu_sub(long_counter, ui_one);
      CHECK(l, long_counter, -1);
    
      l -= ui_one;
      this_cpu_sub(long_counter, ui_one);
      CHECK(l, long_counter, -1);
      CHECK(l, long_counter, 0xffffffffffffffff);
    
      ul -= ui_one;
      __this_cpu_sub(ulong_counter, ui_one);
      CHECK(ul, ulong_counter, -1);
      CHECK(ul, ulong_counter, 0xffffffffffffffff);
    
      ul = this_cpu_sub_return(ulong_counter, ui_one);
      CHECK(ul, ulong_counter, 2);
    
      ul = __this_cpu_sub_return(ulong_counter, ui_one);
      CHECK(ul, ulong_counter, 1);
    Signed-off-by: default avatarGreg Thelen <gthelen@google.com>
    Acked-by: default avatarTejun Heo <tj@kernel.org>
    Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    bd09d9a3
percpu.h 19.3 KB