1. 10 Aug, 2021 3 commits
    • Paul E. McKenney's avatar
      Merge branches 'doc.2021.07.20c', 'fixes.2021.08.06a', 'nocb.2021.07.20c',... · b770efc4
      Paul E. McKenney authored
      Merge branches 'doc.2021.07.20c', 'fixes.2021.08.06a', 'nocb.2021.07.20c', 'nolibc.2021.07.20c', 'tasks.2021.07.20c', 'torture.2021.07.27a' and 'torturescript.2021.07.27a' into HEAD
      
      doc.2021.07.20c: Documentation updates.
      fixes.2021.08.06a: Miscellaneous fixes.
      nocb.2021.07.20c: Callback-offloading (NOCB CPU) updates.
      nolibc.2021.07.20c: Tiny userspace library updates.
      tasks.2021.07.20c: Tasks RCU updates.
      torture.2021.07.27a: In-kernel torture-test updates.
      torturescript.2021.07.27a: Torture-test scripting updates.
      b770efc4
    • Sebastian Andrzej Siewior's avatar
      torture: Replace deprecated CPU-hotplug functions. · ed4fa244
      Sebastian Andrzej Siewior authored
      The functions get_online_cpus() and put_online_cpus() have been
      deprecated during the CPU hotplug rework. They map directly to
      cpus_read_lock() and cpus_read_unlock().
      
      Replace deprecated CPU-hotplug functions with the official version.
      The behavior remains unchanged.
      
      Cc: Davidlohr Bueso <dave@stgolabs.net>
      Cc: "Paul E. McKenney" <paulmck@kernel.org>
      Cc: Josh Triplett <josh@joshtriplett.org>
      Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      ed4fa244
    • Sebastian Andrzej Siewior's avatar
      rcu: Replace deprecated CPU-hotplug functions · d3dd95a8
      Sebastian Andrzej Siewior authored
      The functions get_online_cpus() and put_online_cpus() have been
      deprecated during the CPU hotplug rework. They map directly to
      cpus_read_lock() and cpus_read_unlock().
      
      Replace deprecated CPU-hotplug functions with the official version.
      The behavior remains unchanged.
      
      Cc: "Paul E. McKenney" <paulmck@kernel.org>
      Cc: Josh Triplett <josh@joshtriplett.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Cc: Lai Jiangshan <jiangshanlai@gmail.com>
      Cc: Joel Fernandes <joel@joelfernandes.org>
      Cc: rcu@vger.kernel.org
      Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      d3dd95a8
  2. 06 Aug, 2021 17 commits
    • Paul E. McKenney's avatar
      rcu: Print human-readable message for schedule() in RCU reader · 521c89b3
      Paul E. McKenney authored
      The WARN_ON_ONCE() invocation within the CONFIG_PREEMPT=y version of
      rcu_note_context_switch() triggers when there is a voluntary context
      switch in an RCU read-side critical section, but there is quite a gap
      between the output of that WARN_ON_ONCE() and this RCU-usage error.
      This commit therefore converts the WARN_ON_ONCE() to a WARN_ONCE()
      that explicitly describes the problem in its message.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      521c89b3
    • Frederic Weisbecker's avatar
      rcu: Explain why rcu_all_qs() is a stub in preemptible TREE RCU · 50895825
      Frederic Weisbecker authored
      The cond_resched() function reports an RCU quiescent state only in
      non-preemptible TREE RCU implementation.  This commit therefore adds a
      comment explaining why cond_resched() does nothing in preemptible kernels.
      Signed-off-by: default avatarFrederic Weisbecker <frederic@kernel.org>
      Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
      Cc: Joel Fernandes <joel@joelfernandes.org>
      Cc: Uladzislau Rezki <urezki@gmail.com>
      Cc: Boqun Feng <boqun.feng@gmail.com>
      Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Ingo Molnar <mingo@kernel.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      50895825
    • Liu Song's avatar
      rcu: Use per_cpu_ptr to get the pointer of per_cpu variable · 8211e922
      Liu Song authored
      There are a few remaining locations in kernel/rcu that still use
      "&per_cpu()".  This commit replaces them with "per_cpu_ptr(&)", and does
      not introduce any functional change.
      Reviewed-by: default avatarUladzislau Rezki (Sony) <urezki@gmail.com>
      Reviewed-by: default avatarNeeraj Upadhyay <neeraju@codeaurora.org>
      Reviewed-by: default avatarJoel Fernandes (Google) <joel@joelfernandes.org>
      Signed-off-by: default avatarLiu Song <liu.song11@zte.com.cn>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      8211e922
    • Liu Song's avatar
      rcu: Remove useless "ret" update in rcu_gp_fqs_loop() · eb880949
      Liu Song authored
      Within rcu_gp_fqs_loop(), the "ret" local variable is set to the
      return value from swait_event_idle_timeout_exclusive(), but "ret" is
      unconditionally overwritten later in the code.  This commit therefore
      removes this useless assignment.
      Signed-off-by: default avatarLiu Song <liu.song11@zte.com.cn>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      eb880949
    • Paul E. McKenney's avatar
      rcu: Mark accesses in tree_stall.h · d283aa1b
      Paul E. McKenney authored
      This commit marks the accesses in tree_stall.h so as to both avoid
      undesirable compiler optimizations and to keep KCSAN focused on the
      accesses of the core algorithm.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      d283aa1b
    • Paul E. McKenney's avatar
      rcu: Make rcu_gp_init() and rcu_gp_fqs_loop noinline to conserve stack · f74126dc
      Paul E. McKenney authored
      The kbuild test project found an oversized stack frame in rcu_gp_kthread()
      for some kernel configurations.  This oversizing was due to a very large
      amount of inlining, which is unnecessary due to the fact that this code
      executes infrequently.  This commit therefore marks rcu_gp_init() and
      rcu_gp_fqs_loop noinline_for_stack to conserve stack space.
      Reported-by: default avatarkernel test robot <lkp@intel.com>
      Tested-by: default avatarRong Chen <rong.a.chen@intel.com>
      [ paulmck: noinline_for_stack per Nathan Chancellor. ]
      Reviewed-by: default avatarNathan Chancellor <nathan@kernel.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      f74126dc
    • Paul E. McKenney's avatar
      rcu: Mark lockless ->qsmask read in rcu_check_boost_fail() · d9ee962f
      Paul E. McKenney authored
      Accesses to ->qsmask are normally protected by ->lock, but there is an
      exception in the diagnostic code in rcu_check_boost_fail().  This commit
      therefore applies data_race() to this access to avoid KCSAN complaining
      about the C-language writes protected by ->lock.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      d9ee962f
    • Paul E. McKenney's avatar
      srcutiny: Mark read-side data races · 65bfdd36
      Paul E. McKenney authored
      This commit marks some interrupt-induced read-side data races in
      __srcu_read_lock(), __srcu_read_unlock(), and srcu_torture_stats_print().
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      65bfdd36
    • Paul E. McKenney's avatar
      rcu: Start timing stall repetitions after warning complete · b169246f
      Paul E. McKenney authored
      Systems with low-bandwidth consoles can have very large printk()
      latencies, and on such systems it makes no sense to have the next RCU CPU
      stall warning message start output before the prior message completed.
      This commit therefore sets the time of the next stall only after the
      prints have completed.  While printing, the time of the next stall
      message is set to ULONG_MAX/2 jiffies into the future.
      Reviewed-by: default avatarSergey Senozhatsky <senozhatsky@chromium.org>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      b169246f
    • Sergey Senozhatsky's avatar
      rcu: Do not disable GP stall detection in rcu_cpu_stall_reset() · a80be428
      Sergey Senozhatsky authored
      rcu_cpu_stall_reset() is one of the functions virtual CPUs
      execute during VM resume in order to handle jiffies skew
      that can trigger false positive stall warnings. Paul has
      pointed out that this approach is problematic because
      rcu_cpu_stall_reset() disables RCU grace period stall-detection
      virtually forever, while in fact it can just restart the
      stall-detection timeout.
      Suggested-by: default avatar"Paul E. McKenney" <paulmck@kernel.org>
      Signed-off-by: default avatarSergey Senozhatsky <senozhatsky@chromium.org>
      Signed-off-by: default avatarSigned-off-by: Paul E. McKenney <paulmck@kernel.org>
      a80be428
    • Sergey Senozhatsky's avatar
      rcu/tree: Handle VM stoppage in stall detection · ccfc9dd6
      Sergey Senozhatsky authored
      The soft watchdog timer function checks if a virtual machine
      was suspended and hence what looks like a lockup in fact
      is a false positive.
      
      This is what kvm_check_and_clear_guest_paused() does: it
      tests guest PVCLOCK_GUEST_STOPPED (which is set by the host)
      and if it's set then we need to touch all watchdogs and bail
      out.
      
      Watchdog timer function runs from IRQ, so PVCLOCK_GUEST_STOPPED
      check works fine.
      
      There is, however, one more watchdog that runs from IRQ, so
      watchdog timer fn races with it, and that watchdog is not aware
      of PVCLOCK_GUEST_STOPPED - RCU stall detector.
      
      apic_timer_interrupt()
       smp_apic_timer_interrupt()
        hrtimer_interrupt()
         __hrtimer_run_queues()
          tick_sched_timer()
           tick_sched_handle()
            update_process_times()
             rcu_sched_clock_irq()
      
      This triggers RCU stalls on our devices during VM resume.
      
      If tick_sched_handle()->rcu_sched_clock_irq() runs on a VCPU
      before watchdog_timer_fn()->kvm_check_and_clear_guest_paused()
      then there is nothing on this VCPU that touches watchdogs and
      RCU reads stale gp stall timestamp and new jiffies value, which
      makes it think that RCU has stalled.
      
      Make RCU stall watchdog aware of PVCLOCK_GUEST_STOPPED and
      don't report RCU stalls when we resume the VM.
      Signed-off-by: default avatarSergey Senozhatsky <senozhatsky@chromium.org>
      Signed-off-by: default avatarSigned-off-by: Paul E. McKenney <paulmck@kernel.org>
      ccfc9dd6
    • Julian Wiedmann's avatar
      rculist: Unify documentation about missing list_empty_rcu() · 751b1710
      Julian Wiedmann authored
      We have two separate sections that talk about why list_empty_rcu()
      is not needed, so this commit consolidates them.
      Signed-off-by: default avatarJulian Wiedmann <jwi@linux.ibm.com>
      [ paulmck: The usual wordsmithing. ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      751b1710
    • Paul E. McKenney's avatar
      rcu: Mark accesses to ->rcu_read_lock_nesting · 5fcb3a5f
      Paul E. McKenney authored
      KCSAN flags accesses to ->rcu_read_lock_nesting as data races, but
      in the past, the overhead of marked accesses was excessive.  However,
      that was long ago, and much has changed since then, both in terms of
      hardware and of compilers.  Here is data taken on an eight-core laptop
      using Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz with a kernel built
      using gcc version 9.3.0, with all data in nanoseconds.
      
      Unmarked accesses (status quo), measured by three refscale runs:
      
      	Minimum reader duration:  3.286  2.851  3.395
      	Median reader duration:   3.698  3.531  3.4695
      	Maximum reader duration:  4.481  5.215  5.157
      
      Marked accesses, also measured by three refscale runs:
      
      	Minimum reader duration:  3.501  3.677  3.580
      	Median reader duration:   4.053  3.723  3.895
      	Maximum reader duration:  7.307  4.999  5.511
      
      This focused microbenhmark shows only sub-nanosecond differences which
      are unlikely to be visible at the system level.  This commit therefore
      marks data-racing accesses to ->rcu_read_lock_nesting.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      5fcb3a5f
    • Paul E. McKenney's avatar
      rcu: Weaken ->dynticks accesses and updates · 2be57f73
      Paul E. McKenney authored
      Accesses to the rcu_data structure's ->dynticks field have always been
      fully ordered because it was not possible to prove that weaker ordering
      was safe.  However, with the removal of the rcu_eqs_special_set() function
      and the advent of the Linux-kernel memory model, it is now easy to show
      that two of the four original full memory barriers can be weakened to
      acquire and release operations.  The remaining pair must remain full
      memory barriers.  This change makes the memory ordering requirements
      more evident, and it might well also speed up the to-idle and from-idle
      fastpaths on some architectures.
      
      The following litmus test, adapted from one supplied off-list by Frederic
      Weisbecker, models the RCU grace-period kthread detecting an idle CPU
      that is concurrently transitioning to non-idle:
      
      	C dynticks-from-idle
      
      	{
      		DYNTICKS=0; (* Initially idle. *)
      	}
      
      	P0(int *X, int *DYNTICKS)
      	{
      		int dynticks;
      		int x;
      
      		// Idle.
      		dynticks = READ_ONCE(*DYNTICKS);
      		smp_store_release(DYNTICKS, dynticks + 1);
      		smp_mb();
      		// Now non-idle
      		x = READ_ONCE(*X);
      	}
      
      	P1(int *X, int *DYNTICKS)
      	{
      		int dynticks;
      
      		WRITE_ONCE(*X, 1);
      		smp_mb();
      		dynticks = smp_load_acquire(DYNTICKS);
      	}
      
      	exists (1:dynticks=0 /\ 0:x=1)
      
      Running "herd7 -conf linux-kernel.cfg dynticks-from-idle.litmus" verifies
      this transition, namely, showing that if the RCU grace-period kthread (P1)
      sees another CPU as idle (P0), then any memory access prior to the start
      of the grace period (P1's write to X) will be seen by any RCU read-side
      critical section following the to-non-idle transition (P0's read from X).
      This is a straightforward use of full memory barriers to force ordering
      in a store-buffering (SB) litmus test.
      
      The following litmus test, also adapted from the one supplied off-list
      by Frederic Weisbecker, models the RCU grace-period kthread detecting
      a non-idle CPU that is concurrently transitioning to idle:
      
      	C dynticks-into-idle
      
      	{
      		DYNTICKS=1; (* Initially non-idle. *)
      	}
      
      	P0(int *X, int *DYNTICKS)
      	{
      		int dynticks;
      
      		// Non-idle.
      		WRITE_ONCE(*X, 1);
      		dynticks = READ_ONCE(*DYNTICKS);
      		smp_store_release(DYNTICKS, dynticks + 1);
      		smp_mb();
      		// Now idle.
      	}
      
      	P1(int *X, int *DYNTICKS)
      	{
      		int x;
      		int dynticks;
      
      		smp_mb();
      		dynticks = smp_load_acquire(DYNTICKS);
      		x = READ_ONCE(*X);
      	}
      
      	exists (1:dynticks=2 /\ 1:x=0)
      
      Running "herd7 -conf linux-kernel.cfg dynticks-into-idle.litmus" verifies
      this transition, namely, showing that if the RCU grace-period kthread
      (P1) sees another CPU as newly idle (P0), then any pre-idle memory access
      (P0's write to X) will be seen by any code following the grace period
      (P1's read from X).  This is a simple release-acquire pair forcing
      ordering in a message-passing (MP) litmus test.
      
      Of course, if the grace-period kthread detects the CPU as non-idle,
      it will refrain from reporting a quiescent state on behalf of that CPU,
      so there are no ordering requirements from the grace-period kthread in
      that case.  However, other subsystems call rcu_is_idle_cpu() to check
      for CPUs being non-idle from an RCU perspective.  That case is also
      verified by the above litmus tests with the proviso that the sense of
      the low-order bit of the DYNTICKS counter be inverted.
      
      Unfortunately, on x86 smp_mb() is as expensive as a cache-local atomic
      increment.  This commit therefore weakens only the read from ->dynticks.
      However, the updates are abstracted into a rcu_dynticks_inc() function
      to ease any future changes that might be needed.
      
      [ paulmck: Apply Linus Torvalds feedback. ]
      
      Link: https://lore.kernel.org/lkml/20210721202127.2129660-4-paulmck@kernel.org/Suggested-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Acked-by: default avatarMathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      2be57f73
    • Joel Fernandes (Google)'s avatar
      rcu: Remove special bit at the bottom of the ->dynticks counter · a86baa69
      Joel Fernandes (Google) authored
      Commit b8c17e66 ("rcu: Maintain special bits at bottom of ->dynticks
      counter") reserved a bit at the bottom of the ->dynticks counter to defer
      flushing of TLBs, but this facility never has been used.  This commit
      therefore removes this capability along with the rcu_eqs_special_set()
      function used to trigger it.
      
      Link: https://lore.kernel.org/linux-doc/CALCETrWNPOOdTrFabTDd=H7+wc6xJ9rJceg6OL1S0rTV5pfSsA@mail.gmail.com/Suggested-by: default avatarAndy Lutomirski <luto@kernel.org>
      Signed-off-by: default avatar"Joel Fernandes (Google)" <joel@joelfernandes.org>
      [ paulmck: Forward-port to v5.13-rc1. ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      a86baa69
    • Yanfei Xu's avatar
      rcu: Fix stall-warning deadlock due to non-release of rcu_node ->lock · dc87740c
      Yanfei Xu authored
      If rcu_print_task_stall() is invoked on an rcu_node structure that does
      not contain any tasks blocking the current grace period, it takes an
      early exit that fails to release that rcu_node structure's lock.  This
      results in a self-deadlock, which is detected by lockdep.
      
      To reproduce this bug:
      
      tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 3 --trust-make --configs "TREE03" --kconfig "CONFIG_PROVE_LOCKING=y" --bootargs "rcutorture.stall_cpu=30 rcutorture.stall_cpu_block=1 rcutorture.fwd_progress=0 rcutorture.test_boost=0"
      
      This will also result in other complaints, including RCU's scheduler
      hook complaining about blocking rather than preemption and an rcutorture
      writer stall.
      
      Only a partial RCU CPU stall warning message will be printed because of
      the self-deadlock.
      
      This commit therefore releases the lock on the rcu_print_task_stall()
      function's early exit path.
      
      Fixes: c583bcb8 ("rcu: Don't invoke try_invoke_on_locked_down_task() with irqs disabled")
      Tested-by: default avatarQais Yousef <qais.yousef@arm.com>
      Signed-off-by: default avatarYanfei Xu <yanfei.xu@windriver.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      dc87740c
    • Yanfei Xu's avatar
      rcu: Fix to include first blocked task in stall warning · e6a901a4
      Yanfei Xu authored
      The for loop in rcu_print_task_stall() always omits ts[0], which points
      to the first task blocking the stalled grace period.  This in turn fails
      to count this first task, which means that ndetected will be equal to
      zero when all CPUs have passed through their quiescent states and only
      one task is blocking the stalled grace period.  This zero value for
      ndetected will in turn result in an incorrect "All QSes seen" message:
      
      rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
      rcu:    Tasks blocked on level-1 rcu_node (CPUs 12-23):
              (detected by 15, t=6504 jiffies, g=164777, q=9011209)
      rcu: All QSes seen, last rcu_preempt kthread activity 1 (4295252379-4295252378), jiffies_till_next_fqs=1, root ->qsmask 0x2
      BUG: sleeping function called from invalid context at include/linux/uaccess.h:156
      in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 70613, name: msgstress04
      INFO: lockdep is turned off.
      Preemption disabled at:
      [<ffff8000104031a4>] create_object.isra.0+0x204/0x4b0
      CPU: 15 PID: 70613 Comm: msgstress04 Kdump: loaded Not tainted
      5.12.2-yoctodev-standard #1
      Hardware name: Marvell OcteonTX CN96XX board (DT)
      Call trace:
       dump_backtrace+0x0/0x2cc
       show_stack+0x24/0x30
       dump_stack+0x110/0x188
       ___might_sleep+0x214/0x2d0
       __might_sleep+0x7c/0xe0
      
      This commit therefore fixes the loop to include ts[0].
      
      Fixes: c583bcb8 ("rcu: Don't invoke try_invoke_on_locked_down_task() with irqs disabled")
      Tested-by: default avatarQais Yousef <qais.yousef@arm.com>
      Signed-off-by: default avatarYanfei Xu <yanfei.xu@windriver.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      e6a901a4
  3. 27 Jul, 2021 17 commits
    • Paul E. McKenney's avatar
      torture: Make kvm-test-1-run-qemu.sh check for reboot loops · 06ca9144
      Paul E. McKenney authored
      It turns out that certain types of early boot bugs can result in reboot
      loops, even within a guest OS running under qemu/KVM.  This commit
      therefore upgrades the kvm-test-1-run-qemu.sh script's hang-detection
      heuristics to detect such situations and to terminate the run when
      they occur.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      06ca9144
    • Paul E. McKenney's avatar
      torture: Add timestamps to kvm-test-1-run-qemu.sh output · 5f33809e
      Paul E. McKenney authored
      The kvm-test-1-run-qemu.sh script logs the torture-test start time and
      also when it starts getting impatient for the test to finish.  However, it
      does not timestamp these log messages, which can make debugging needlessly
      challenging.  This commit therefore adds timestamps to these messages.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      5f33809e
    • Paul E. McKenney's avatar
      torture: Don't use "test" command's "-a" argument · b3bf9632
      Paul E. McKenney authored
      There was a time long ago when the "test" command's documentation
      claimed that the "-a" and "-o" arguments did something useful.
      But this documentation now suggests letting the shell execute
      these boolean operators, so this commit applies that suggestion to
      kvm-test-1-run-qemu.sh.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      b3bf9632
    • Paul E. McKenney's avatar
      torture: Make kvm-test-1-run-batch.sh select per-scenario affinity masks · a5202e17
      Paul E. McKenney authored
      This commit causes kvm-test-1-run-batch.sh to use the new
      kvm-assign-cpus.sh and kvm-get-cpus-script.sh scripts to create a
      TORTURE_AFFINITY environment variable containing either an empty string
      (for no affinity) or a list of CPUs to pin the scenario's vCPUs to.
      The additional change to kvm-test-1-run.sh places the per-scenario
      number-of-CPUs information where it can easily be found.
      
      If there is some reason why affinity cannot be supplied, this commit
      prints and logs the reason via changes to kvm-again.sh.
      
      Finally, this commit updates the kvm-remote.sh script to copy the
      qemu-affinity output files back to the host system.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      a5202e17
    • Paul E. McKenney's avatar
      torture: Consistently name "qemu*" test output files · 9e528a84
      Paul E. McKenney authored
      There is "qemu-affinity", "qemu-cmd", "qemu-retval", but also "qemu_pid".
      This is hard to remember, not so good for bash tab completion, and just
      plain inconsistent.  This commit therefore renames the "qemu_pid" file to
      "qemu-pid".  A couple of the scripts must deal with old runs, and thus
      must handle both "qemu_pid" and "qemu-pid", but new runs will produce
      "qemu-pid".
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      9e528a84
    • Paul E. McKenney's avatar
      torture: Use numeric taskset argument in jitter.sh · 4567c76a
      Paul E. McKenney authored
      The jitter.sh script has some entertaining awk code to generate a
      hex mask from a randomly selected CPU number, which is handed to the
      "taskset" command.  Except that this command has a "-c" parameter to
      take a comma/dash-separated list of CPU numbers.  This commit therefore
      saves a few lines of awk by switching to a single-number CPU list.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      4567c76a
    • Paul E. McKenney's avatar
      rcutorture: Upgrade two-CPU scenarios to four CPUs · de290946
      Paul E. McKenney authored
      There is no way to place the vCPUs in a two-CPU rcutorture scenario to
      get variable memory latency.  This commit therefore upgrades the current
      two-CPU rcutorture scenarios to four CPUs.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      de290946
    • Paul E. McKenney's avatar
      torture: Make kvm-test-1-run-qemu.sh apply affinity · bdf5ca12
      Paul E. McKenney authored
      This commit causes the kvm-test-1-run-qemu.sh script to check the
      TORTURE_AFFINITY environment variable and to add "taskset" commands to
      the qemu-cmd file.  The first "taskset" command is applied only if the
      TORTURE_AFFINITY environment variable is a non-empty string, and this
      command pins the current scenario's guest OS to the specified CPUs.
      The second "taskset" command reports the guest OS's affinity in a new
      "qemu-affinity" file.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      bdf5ca12
    • Paul E. McKenney's avatar
      torture: Don't redirect qemu-cmd comment lines · 8220a118
      Paul E. McKenney authored
      Currently, kvm-test-1-run-qemu.sh applies redirection to each and every
      line of each qemu-cmd script.  Only the first line (the only one that
      is not a bash comment) needs to be redirected.  Although redirecting
      the comments is currently harmless, just adding to the comment, it is
      an accident waiting to happen.  This commit therefore adjusts the "sed"
      command to redirect only the qemu-system* command itself.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      8220a118
    • Paul E. McKenney's avatar
      torture: Make kvm.sh select per-scenario affinity masks · cdeef67d
      Paul E. McKenney authored
      This commit causes kvm.sh to use the new kvm-assign-cpus.sh and
      kvm-get-cpus-script.sh scripts to create a TORTURE_AFFINITY environment
      variable containing either an empty string (for no affinity) or a list
      of CPUs to pin the scenario's vCPUs to.  A later commit will make
      use of this information to actually pin the vCPUs.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      cdeef67d
    • Paul E. McKenney's avatar
      scftorture: Avoid NULL pointer exception on early exit · 586e4d41
      Paul E. McKenney authored
      When scftorture finds an error in the module parameters controlling
      the relative frequencies of smp_call_function*() variants, it takes an
      early exit.  So early that it has not allocated memory to track the
      kthreads running the test, which results in a segfault.  This commit
      therefore checks for the existence of the memory before attempting
      to stop the kthreads that would otherwise have been recorded in that
      non-existent memory.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      586e4d41
    • Paul E. McKenney's avatar
      scftorture: Add RPC-like IPI tests · 9b9a8067
      Paul E. McKenney authored
      This commit adds the single_weight_rpc module parameter, which causes the
      IPI handler to awaken the IPI sender.  In many scheduler configurations,
      this will result in an IPI back to the sender that is likely to be
      received at a time when the sender CPU is idle.  The intent is to stress
      IPI reception during CPU busy-to-idle transitions.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      9b9a8067
    • Paul E. McKenney's avatar
      locktorture: Count lock readers · af5f6e27
      Paul E. McKenney authored
      Currently, the lock_is_read_held variable is bool, so that a reader sets
      it to true just after lock acquisition and then to false just before
      lock release.  This works in a rough statistical sense, but can result
      in false negatives just after one of a pair of concurrent readers has
      released the lock.  This approach does have low overhead, but at the
      expense of the setting to true potentially never leaving the reader's
      store buffer, thus resulting in an unconditional false negative.
      
      This commit therefore converts this variable to atomic_t and makes
      the reader use atomic_inc() just after acquisition and atomic_dec()
      just before release.  This does increase overhead, but this increase is
      negligible compared to the 10-microsecond lock hold time.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      af5f6e27
    • Paul E. McKenney's avatar
      locktorture: Mark statistics data races · 5b237d65
      Paul E. McKenney authored
      The lock_stress_stats structure's ->n_lock_fail and ->n_lock_acquired
      fields are incremented and sampled locklessly using plain C-language
      statements, which KCSAN objects to.  This commit therefore marks the
      statistics gathering with data_race() to flag the intent.  While in
      the area, this commit also reduces the number of accesses to the
      ->n_lock_acquired field, thus eliminating some possible check/use
      confusion.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      5b237d65
    • Jiangong.Han's avatar
      rcuscale: Console output claims too few grace periods · 811192c5
      Jiangong.Han authored
      The rcuscale console output claims N grace periods, numbered from zero
      to N, which means that there were really N+1 grace periods.  The root
      cause of this bug is that rcu_scale_writer() stores the number of the
      last grace period (numbered from zero) into writer_n_durations[me]
      instead of the number of grace periods.  This commit therefore assigns
      the actual number of grace periods to writer_n_durations[me], and also
      makes the corresponding adjustment to the loop outputting per-grace-period
      measurements.
      
      Sample of old console output:
          rcu-scale: writer 0 gps: 133
          ......
          rcu-scale:    0 writer-duration:     0 44003961
          rcu-scale:    0 writer-duration:     1 32003582
          ......
          rcu-scale:    0 writer-duration:   132 28004391
          rcu-scale:    0 writer-duration:   133 27996410
      
      Sample of new console output:
          rcu-scale: writer 0 gps: 134
          ......
          rcu-scale:    0 writer-duration:     0 44003961
          rcu-scale:    0 writer-duration:     1 32003582
          ......
          rcu-scale:    0 writer-duration:   132 28004391
          rcu-scale:    0 writer-duration:   133 27996410
      Signed-off-by: default avatarJiangong.Han <jiangong.han@windriver.com>
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      811192c5
    • Paul E. McKenney's avatar
      rcutorture: Preempt rather than block when testing task stalls · 59e83666
      Paul E. McKenney authored
      Currently, rcu_torture_stall() does a one-jiffy timed wait when
      stall_cpu_block is set.  This works, but emits a pointless splat in
      CONFIG_PREEMPT=y kernels.  This commit avoids this splat by instead
      invoking preempt_schedule() in CONFIG_PREEMPT=y kernels.
      
      This uses an admittedly ugly #ifdef, but abstracted approaches just
      looked worse.  A prettier approach would provide a preempt_schedule()
      definition with a WARN_ON() for CONFIG_PREEMPT=n kernels, but this seems
      quite silly.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      59e83666
    • Paul E. McKenney's avatar
      refscale: Add measurement of clock readout · 25f6fa53
      Paul E. McKenney authored
      This commit adds a "clock" type to refscale, which checks the performance
      of ktime_get_real_fast_ns().  Use the "clocksource=" kernel boot parameter
      to select the underlying clock source.
      
      [ paulmck: Work around compiler false positive per kernel test robot. ]
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      25f6fa53
  4. 20 Jul, 2021 3 commits
    • Paul E. McKenney's avatar
      torture: Put kvm.sh batch-creation awk script into a temp file · a3d79412
      Paul E. McKenney authored
      This commit is a first step towards pinning guest-OS vCPUs so as
      to force latency differences, especially on multi-socket systems.
      The kvm.sh script puts its batch-creation awk script into a temporary
      file so that later commits can add the awk code needed to dole out CPUs
      so as to maximize latency differences.  This awk code will be used by
      multiple scripts.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      a3d79412
    • Paul E. McKenney's avatar
      torture: Move parse-console.sh call to PATH-aware scripts · d4b54d2d
      Paul E. McKenney authored
      The last line of kvm-test-1-run-qemu.sh invokes parse-console.sh, but
      kvm-test-1-run-qemu.sh is unaware of the PATH containing this script
      and does not have the job title handy.  This commit therefore moves
      the invocation of parse-console.sh to kvm-test-1-run.sh, which has
      PATH and title at hand.  This commit does not add an invocation of
      parse-console.sh to kvm-test-1-run-batch.sh because this latter script
      is run in the background, and the information will be gathered at the
      end of the full run.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      d4b54d2d
    • Paul E. McKenney's avatar
      torture: Make kvm-recheck.sh skip kcsan.sum for build-only runs · 2ab5bbc3
      Paul E. McKenney authored
      Currently, kvm-recheck.sh attempts to create a kcsan.sum file even for
      build-only runs.  This results in false-positive bash errors due to
      there being no console.log files in that case.  This commit therefore
      makes kvm-recheck.sh skip creating the kcsan.sum file for build-only runs.
      Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
      2ab5bbc3