1. 15 Oct, 2013 4 commits
  2. 25 Sep, 2013 9 commits
  3. 23 Sep, 2013 18 commits
    • Kirill Tkhai's avatar
      rcu: Fix CONFIG_RCU_NOCB_CPU_ALL panic on machines with sparse CPU mask · 5d5a0800
      Kirill Tkhai authored
      Some architectures have sparse cpu mask. UltraSparc's cpuinfo for example:
      
      CPU0: online
      CPU2: online
      
      So, set only possible CPUs when CONFIG_RCU_NOCB_CPU_ALL is enabled.
      
      Also, check that user passes right 'rcu_nocbs=' option.
      Signed-off-by: default avatarKirill Tkhai <tkhai@yandex.ru>
      CC: Dipankar Sarma <dipankar@in.ibm.com>
      [ paulmck: Fix pr_info() issue noted by scripts/checkpatch.pl. ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      5d5a0800
    • Paul E. McKenney's avatar
      rcu: Avoid sparse warnings in rcu_nocb_wake trace event · 15f5191b
      Paul E. McKenney authored
      The event-tracing macros do not like bool tracing arguments, so this
      commit makes them be of type char.  This change has the knock-on effect
      of making it illegal to pass a pointer into one of these arguments, so
      also change rcutiny's first call to trace_rcu_batch_end() to convert
      from pointer to boolean, prefixing with "!!".
      Reported-by: default avatarkbuild test robot <fengguang.wu@intel.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      15f5191b
    • Paul E. McKenney's avatar
      rcu: Track rcu_nocb_kthread()'s sleeping and awakening · 69a79bb1
      Paul E. McKenney authored
      This commit adds event traces to track all of rcu_nocb_kthread()'s
      blocking and awakening.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      69a79bb1
    • Paul E. McKenney's avatar
      rcu: Distinguish between NOCB and non-NOCB rcu_callback trace events · 756cbf6b
      Paul E. McKenney authored
      One way to distinguish between NOCB and non-NOCB rcu_callback trace
      events is that the former always print zero for the lazy and non-lazy
      queue lengths.  Unfortunately, this also means that we cannot see the NOCB
      queue lengths.  This commit therefore accesses the NOCB queue lengths,
      but negates them.  NOCB rcu_callback trace events should therefore have
      negative queue lengths.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      [ paulmck: Match operand size per kbuild test robot's advice. ]
      756cbf6b
    • Paul E. McKenney's avatar
      rcu: Add tracing for rcuo no-CBs CPU wakeup handshake · 9261dd0d
      Paul E. McKenney authored
      Lost wakeups from call_rcu() to the rcuo kthreads can result in hangs
      that are difficult to diagnose.  This commit therefore adds tracing to
      help pin down the cause of these hangs.
      Reported-by: default avatarClark Williams <williams@redhat.com>
      Reported-by: default avatarCarsten Emde <C.Emde@osadl.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      [ paulmck: Add const per kbuild test robot's advice. ]
      9261dd0d
    • Paul E. McKenney's avatar
      rcu: Add tracing of normal (non-NOCB) grace-period requests · bb311ecc
      Paul E. McKenney authored
      This commit adds tracing to the normal grace-period request points.
      These are rcu_gp_cleanup(), which checks for the need for another
      grace period at the end of the previous grace period, and
      rcu_start_gp_advanced(), which restarts RCU's state machine after
      an idle period.  These trace events are intended to help track down
      bugs where RCU remains idle despite there being work for it to do.
      Reported-by: default avatarClark Williams <williams@redhat.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      bb311ecc
    • Paul E. McKenney's avatar
      rcu: Add tracing to rcu_gp_kthread() · 63c4db78
      Paul E. McKenney authored
      This commit adds tracing to the rcu_gp_kthread() function in order to
      help trace down hangs potentially involving this kthread.
      Reported-by: default avatarClark Williams <williams@redhat.com>
      Reported-by: default avatarCarsten Emde <C.Emde@osadl.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      63c4db78
    • Paul E. McKenney's avatar
      rcu: Flag lockless access to ->gp_flags with ACCESS_ONCE() · 591c6d17
      Paul E. McKenney authored
      This commit applies ACCESS_ONCE() to an outside-of-lock access to
      ->gp_flags.  Although it is hard to imagine any sane compiler messing
      this particular case up, the documentation benefits are substantial.
      Plus the definition of "sane compiler" grows ever looser.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      591c6d17
    • Paul E. McKenney's avatar
      rcu: Prevent spurious-wakeup DoS attack on rcu_gp_kthread() · 88d6df61
      Paul E. McKenney authored
      Spurious wakeups in the force-quiescent-state loop in rcu_gp_kthread()
      cause the timeout to be recalculated, which would prevent rcu_gp_fqs()
      from ever being called.  This would in turn would prevent the grace period
      from ever ending for as long as there was at least one CPU in an extended
      quiescent state that had not yet passed through a quiescent state.
      
      This commit therefore avoids recalculating the timeout unless the
      previous pass's call to wait_event_interruptible_timeout() actually
      did time out, thus preventing the above scenario.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      88d6df61
    • Paul E. McKenney's avatar
      rcu: Improve grace-period start logic · f7be8209
      Paul E. McKenney authored
      This commit improves grace-period start logic by checking ->gp_flags
      under the lock and by issuing a warning if a grace period is already
      in progress.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      f7be8209
    • Paul E. McKenney's avatar
      rcu: Have rcutiny tracepoints use tracepoint_string() · 0d752924
      Paul E. McKenney authored
      This commit extends the work done in f7f7bac9 (rcu: Have the RCU
      tracepoints use the tracepoint_string infrastructure) to cover rcutiny.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      0d752924
    • Paul E. McKenney's avatar
      rcu: Reject memory-order-induced stall-warning false positives · 26cdfedf
      Paul E. McKenney authored
      If a system is idle from an RCU perspective for longer than specified
      by CONFIG_RCU_CPU_STALL_TIMEOUT, and if one CPU starts a grace period
      just as a second checks for CPU stalls, and if this second CPU happens
      to see the old value of rsp->jiffies_stall, it will incorrectly report a
      CPU stall.  This is quite rare, but apparently occurs deterministically
      on systems with about 6TB of memory.
      
      This commit therefore orders accesses to the data used to determine
      whether or not a CPU stall is in progress.  Grace-period initialization
      and cleanup first increments rsp->completed to mark the end of the
      previous grace period, then records the current jiffies in rsp->gp_start,
      then records the jiffies at which a stall can be expected to occur in
      rsp->jiffies_stall, and finally increments rsp->gpnum to mark the start
      of the new grace period.  Now, this ordering by itself does not prevent
      false positives.  For example, if grace-period initialization was delayed
      between recording rsp->gp_start and rsp->jiffies_stall, the CPU stall
      warning code might still see an old value of rsp->jiffies_stall.
      
      Therefore, this commit also orders the CPU stall warning accesses as
      well, loading rsp->gpnum and jiffies, then rsp->jiffies_stall, then
      rsp->gp_start, and finally rsp->completed.  This ordering means that
      the false-positive scenario in the previous paragraph would result
      in rsp->completed being greater than or equal to rsp->gpnum, which is
      never valid for a CPU stall, allowing the false positive to be rejected.
      Furthermore, any fetch that gets an old value of rsp->jiffies_stall
      must also get an old value of rsp->gpnum, which will again be rejected
      by the comparison of rsp->gpnum and rsp->completed.  Situations where
      rsp->gp_start is later than rsp->jiffies_stall are also rejected, as
      are situations where jiffies is less than rsp->jiffies_stall.
      
      Although use of unsynchronized accesses means that there are likely
      still some false-positive scenarios (synchronization has proven to be
      a very bad idea on large systems), this should get rid of a large class
      of these scenarios.
      Reported-by: default avatarFabian Herschel <fabian.herschel@suse.com>
      Reported-by: default avatarMichal Hocko <mhocko@suse.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Reviewed-by: default avatarMichal Hocko <mhocko@suse.cz>
      Tested-by: default avatarJochen Striepe <jochen@tolot.escape.de>
      26cdfedf
    • Paul E. McKenney's avatar
      rcu: Micro-optimize rcu_cpu_has_callbacks() · 69c8d28c
      Paul E. McKenney authored
      The for_each_rcu_flavor() loop unconditionally scans all flavors, even
      when the first flavor might have some non-lazy callbacks.  Once the
      loop has seen a non-lazy callback, further passes through the loop
      cannot change the state.  This is not a huge problem, given that there
      can be at most three RCU flavors (RCU-bh, RCU-preempt, and RCU-sched),
      but this code is on the path to idle, so speeding it up even a small
      amount would have some benefit.
      
      This commit therefore does two things:
      
      1.	Rearranges the order of the list of RCU flavors in order to
      	place the most active flavor first in the list.  The most active
      	RCU flavor is RCU-preempt, or, if there is no RCU-preempt,
      	RCU-sched.
      
      2.	Reworks the for_each_rcu_flavor() to exit early when the first
      	non-lazy callback is seen, or, in the case where the caller
      	does not care about non-lazy callbacks (RCU_FAST_NO_HZ=n),
      	when the first callback is seen.
      Reported-by: default avatarChen Gang <gang.chen@asianux.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      69c8d28c
    • Paul E. McKenney's avatar
      rcu: Silence unused-variable warnings · 289828e6
      Paul E. McKenney authored
      The "idle" variable in both rcu_eqs_enter_common() and
      rcu_eqs_exit_common() is only used in a WARN_ON_ONCE().  If the kernel
      is built disabling WARN_ON_ONCE(), the compiler will complain (rightly)
      that "idle" is unused.  This commit therefore adds a __maybe_unused to
      the declaration of both variables.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      289828e6
    • Christoph Lameter's avatar
      rcu: Replace __get_cpu_var() uses · c9d4b0af
      Christoph Lameter authored
      __get_cpu_var() is used for multiple purposes in the kernel source. One
      of them is address calculation via the form &__get_cpu_var(x). This
      calculates the address for the instance of the percpu variable of the
      current processor based on an offset.
      
      Other use cases are for storing and retrieving data from the current
      processors percpu area.  __get_cpu_var() can be used as an lvalue when
      writing data or on the right side of an assignment.
      
      __get_cpu_var() is defined as :
      
      __get_cpu_var() always only does an address determination. However,
      store and retrieve operations could use a segment prefix (or global
      register on other platforms) to avoid the address calculation.
      
      this_cpu_write() and this_cpu_read() can directly take an offset into
      a percpu area and use optimized assembly code to read and write per
      cpu variables.
      
      This patch converts __get_cpu_var into either an explicit address
      calculation using this_cpu_ptr() or into a use of this_cpu operations
      that use the offset. Thereby address calcualtions are avoided and less
      registers are used when code is generated.
      
      At the end of the patchset all uses of __get_cpu_var have been removed
      so the macro is removed too.
      
      The patchset includes passes over all arches as well. Once these
      operations are used throughout then specialized macros can be defined in
      non -x86 arches as well in order to optimize per cpu access by f.e. using
      a global register that may be set to the per cpu base.
      
      Transformations done to __get_cpu_var()
      
      1. Determine the address of the percpu instance of the current processor.
      
      	DEFINE_PER_CPU(int, y);
      	int *x = &__get_cpu_var(y);
      
          Converts to
      
      	int *x = this_cpu_ptr(&y);
      
      2. Same as #1 but this time an array structure is involved.
      
      	DEFINE_PER_CPU(int, y[20]);
      	int *x = __get_cpu_var(y);
      
          Converts to
      
      	int *x = this_cpu_ptr(y);
      
      3. Retrieve the content of the current processors instance of a per cpu
         variable.
      
      	DEFINE_PER_CPU(int, u);
      	int x = __get_cpu_var(y)
      
         Converts to
      
      	int x = __this_cpu_read(y);
      
      4. Retrieve the content of a percpu struct
      
      	DEFINE_PER_CPU(struct mystruct, y);
      	struct mystruct x = __get_cpu_var(y);
      
         Converts to
      
      	memcpy(this_cpu_ptr(&x), y, sizeof(x));
      
      5. Assignment to a per cpu variable
      
      	DEFINE_PER_CPU(int, y)
      	__get_cpu_var(y) = x;
      
         Converts to
      
      	this_cpu_write(y, x);
      
      6. Increment/Decrement etc of a per cpu variable
      
      	DEFINE_PER_CPU(int, y);
      	__get_cpu_var(y)++
      
         Converts to
      
      	this_cpu_inc(y)
      Signed-off-by: default avatarChristoph Lameter <cl@linux.com>
      [ paulmck: Address conflicts. ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      c9d4b0af
    • Paul E. McKenney's avatar
      rcu: Make list_splice_init_rcu() account for RCU readers · 2a855b64
      Paul E. McKenney authored
      The list_splice_init_rcu() function allows a list visible to RCU readers
      to be spliced into another list visible to RCU readers.  This is OK,
      except for the use of INIT_LIST_HEAD(), which does pointer updates
      without doing anything to make those updates safe for concurrent readers.
      
      Of course, most of the time INIT_LIST_HEAD() is being used in reader-free
      contexts, such as initialization or cleanup, so it is OK for it to update
      pointers in an unsafe-for-RCU-readers manner.  This commit therefore
      creates an INIT_LIST_HEAD_RCU() that uses ACCESS_ONCE() to make the updates
      reader-safe.  The reason that we can use ACCESS_ONCE() instead of the more
      typical rcu_assign_pointer() is that list_splice_init_rcu() is updating the
      pointers to reference something that is already visible to readers, so
      that there is no problem with pre-initialized values.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      2a855b64
    • Paul E. McKenney's avatar
      rcu: Fix dubious "if" condition in __call_rcu_nocb_enqueue() · 829511d8
      Paul E. McKenney authored
      This commit replaces an incorrect (but fortunately functional)
      bitwise OR ("|") operator with the correct logical OR ("||").
      Reported-by: default avatarkbuild test robot <fengguang.wu@intel.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      829511d8
    • Paul E. McKenney's avatar
      rcu: Convert local functions to static · 01896f7e
      Paul E. McKenney authored
      The rcu_cpu_stall_timeout kernel parameter, the rcu_dynticks per-CPU
      variable, and the rcu_gp_fqs() function are used only locally.  This
      commit therefore marks them as static.
      Reported-by: default avatarkbuild test robot <fengguang.wu@intel.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      01896f7e
  4. 20 Sep, 2013 2 commits
    • Paul E. McKenney's avatar
      rcu: Use proper cpp macro for ->gp_flags · b3f2d025
      Paul E. McKenney authored
      One of the ->gp_flags assignments used a raw number rather than the
      cpp macro that was intended for this purpose, which this commit fixes.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      b3f2d025
    • Paul E. McKenney's avatar
      mm: Place preemption point in do_mlockall() loop · 5c889690
      Paul E. McKenney authored
      There is a loop in do_mlockall() that lacks a preemption point, which
      means that the following can happen on non-preemptible builds of the
      kernel:
      
      > My fuzz tester keeps hitting this. Every instance shows the non-irq stack
      > came in from mlockall.  I'm only seeing this on one box, but that has more
      > ram (8gb) than my other machines, which might explain it.
      >
      > 	Dave
      >
      > INFO: rcu_preempt self-detected stall on CPU { 3}  (t=6500 jiffies g=470344 c=470343 q=0)
      > sending NMI to all CPUs:
      > NMI backtrace for cpu 3
      > CPU: 3 PID: 29664 Comm: trinity-child2 Not tainted 3.11.0-rc1+ #32
      > task: ffff88023e743fc0 ti: ffff88022f6f2000 task.ti: ffff88022f6f2000
      > RIP: 0010:[<ffffffff810bf7d1>]  [<ffffffff810bf7d1>] trace_hardirqs_off_caller+0x21/0xb0
      > RSP: 0018:ffff880244e03c30  EFLAGS: 00000046
      > RAX: ffff88023e743fc0 RBX: 0000000000000001 RCX: 000000000000003c
      > RDX: 000000000000000f RSI: 0000000000000004 RDI: ffffffff81033cab
      > RBP: ffff880244e03c38 R08: ffff880243288a80 R09: 0000000000000001
      > R10: 0000000000000000 R11: 0000000000000001 R12: ffff880243288a80
      > R13: ffff8802437eda40 R14: 0000000000080000 R15: 000000000000d010
      > FS:  00007f50ae33b740(0000) GS:ffff880244e00000(0000) knlGS:0000000000000000
      > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      > CR2: 000000000097f000 CR3: 0000000240fa0000 CR4: 00000000001407e0
      > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
      > Stack:
      >  ffffffff810bf86d ffff880244e03c98 ffffffff81033cab 0000000000000096
      >  000000000000d008 0000000300000002 0000000000000004 0000000000000003
      >  0000000000002710 ffffffff81c50d00 ffffffff81c50d00 ffff880244fcde00
      > Call Trace:
      >  <IRQ>
      >  [<ffffffff810bf86d>] ? trace_hardirqs_off+0xd/0x10
      >  [<ffffffff81033cab>] __x2apic_send_IPI_mask+0x1ab/0x1c0
      >  [<ffffffff81033cdc>] x2apic_send_IPI_all+0x1c/0x20
      >  [<ffffffff81030115>] arch_trigger_all_cpu_backtrace+0x65/0xa0
      >  [<ffffffff811144b1>] rcu_check_callbacks+0x331/0x8e0
      >  [<ffffffff8108bfa0>] ? hrtimer_run_queues+0x20/0x180
      >  [<ffffffff8109e905>] ? sched_clock_cpu+0xb5/0x100
      >  [<ffffffff81069557>] update_process_times+0x47/0x80
      >  [<ffffffff810bd115>] tick_sched_handle.isra.16+0x25/0x60
      >  [<ffffffff810bd231>] tick_sched_timer+0x41/0x60
      >  [<ffffffff8108ace1>] __run_hrtimer+0x81/0x4e0
      >  [<ffffffff810bd1f0>] ? tick_sched_do_timer+0x60/0x60
      >  [<ffffffff8108b93f>] hrtimer_interrupt+0xff/0x240
      >  [<ffffffff8102de84>] local_apic_timer_interrupt+0x34/0x60
      >  [<ffffffff81718c5f>] smp_apic_timer_interrupt+0x3f/0x60
      >  [<ffffffff817178ef>] apic_timer_interrupt+0x6f/0x80
      >  [<ffffffff8170e8e0>] ? retint_restore_args+0xe/0xe
      >  [<ffffffff8105f101>] ? __do_softirq+0xb1/0x440
      >  [<ffffffff8105f64d>] irq_exit+0xcd/0xe0
      >  [<ffffffff81718c65>] smp_apic_timer_interrupt+0x45/0x60
      >  [<ffffffff817178ef>] apic_timer_interrupt+0x6f/0x80
      >  <EOI>
      >  [<ffffffff8170e8e0>] ? retint_restore_args+0xe/0xe
      >  [<ffffffff8170b830>] ? wait_for_completion_killable+0x170/0x170
      >  [<ffffffff8170c853>] ? preempt_schedule_irq+0x53/0x90
      >  [<ffffffff8170e9f6>] retint_kernel+0x26/0x30
      >  [<ffffffff8107a523>] ? queue_work_on+0x43/0x90
      >  [<ffffffff8107c369>] schedule_on_each_cpu+0xc9/0x1a0
      >  [<ffffffff81167770>] ? lru_add_drain+0x50/0x50
      >  [<ffffffff811677c5>] lru_add_drain_all+0x15/0x20
      >  [<ffffffff81186965>] SyS_mlockall+0xa5/0x1a0
      >  [<ffffffff81716e94>] tracesys+0xdd/0xe2
      
      This commit addresses this problem by inserting the required preemption
      point.
      Reported-by: default avatarDave Jones <davej@redhat.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
      Cc: Michel Lespinasse <walken@google.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      5c889690
  5. 16 Sep, 2013 5 commits
  6. 15 Sep, 2013 2 commits
    • Linus Torvalds's avatar
      Merge branch 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus · d8efd82e
      Linus Torvalds authored
      Pull MIPS fixes from Ralf Baechle:
       "These are four patches for three construction sites:
      
         - Fix register decoding for the combination of multi-core processors
           and multi-threading.
      
         - Two more fixes that are part of the ongoing DECstation resurrection
           work.  One of these touches a DECstation-only network driver.
      
         - Finally Markos' trivial build fix for the AP/SP support.
      
        (With this applied now all MIPS defconfigs are building again)"
      
      * 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus:
        MIPS: kernel: vpe: Make vpe_attrs an array of pointers.
        MIPS: Fix SMP core calculations when using MT support.
        MIPS: DECstation I/O ASIC DMA interrupt handling fix
        MIPS: DECstation HRT initialization rearrangement
      d8efd82e
    • Linus Torvalds's avatar
      Merge branch 'for_linus' of git://cavan.codon.org.uk/platform-drivers-x86 · cd619e21
      Linus Torvalds authored
      Pull x86 platform updates from Matthew Garrett:
       "Nothing amazing here, almost entirely cleanups and minor bugfixes and
        one bit of hardware enablement in the amilo-rfkill driver"
      
      * 'for_linus' of git://cavan.codon.org.uk/platform-drivers-x86:
        platform/x86: panasonic-laptop: reuse module_acpi_driver
        samsung-laptop: fix config build error
        platform: x86: remove unnecessary platform_set_drvdata()
        amilo-rfkill: Enable using amilo-rfkill with the FSC Amilo L1310.
        wmi: parse_wdg() should return kernel error codes
        hp_wmi: Fix unregister order in hp_wmi_rfkill_setup()
        platform: replace strict_strto*() with kstrto*()
        x86: irst: use module_acpi_driver to simplify the code
        x86: smartconnect: use module_acpi_driver to simplify the code
        platform samsung-q10: use ACPI instead of direct EC calls
        thinkpad_acpi: add the ability setting TPACPI_LED_NONE by quirk
        thinkpad_acpi: return -NODEV while operating uninitialized LEDs
      cd619e21